Skip to contentSkip to conversation

perf: Detached events

RFC PATCH In review Alexander Shishkin 4 participants 34 messages Sep 5, 2017 – Jan 24, 2018

Subsystems (3)
lore
Share & tools
lore

Alexander Shishkin

author reviewer
#0 lore raw

cover perf: Detached events

Hi,

I'm going to keep this short.

Objective: include perf data (specifically, AUX/Intel PT) in process core dumps.

Obstacles and how this patchset deals with them:
(1) Need to be able to have perf events running without consumer (perf record) running in the background.
Detached events: a new flag to the perf syscall makes a 'detached' event, which exists after its file descriptor is released. Not all detached events are per-thread AUX events: this tries to take into account the need for system-wide persistent events too.

(2) Need to be able to kill those events, so they need to be accessible after they are created.
Event files: detached events exist as files in tracefs (at the moment), can be opened/mmaped/read/removed.

(3) Ring buffer contents from these events needs to end up in the core dump file.
Injecting perf ring buffer into the target task's address space.

(4) Inheritance will have to allocate ring buffers for such events for this feature to be useful.
A parentless detached event is created (with a ring buffer) upon inheritance, no output redirection, each event has its own ring buffer.

(5) Sideeffect of (4) is that we can't use GFP_KERNEL pages for such ring buffers or else we'll have to fail inherit_event() (and, therefore, user's fork()) when they exhaust their mlock limit.
Using shmemfs-backed pages for such a ring buffer and only pinning them while the corresponding target task is running. Other times these pages can be swapped out.

(6) Ring buffer memory accounting needs to take this new arrangement into
account: one user can use up at most NR_CPUS * buffer_size memory at any given point in time.
Only account the first such event and undo the accounting when the last event is gone.

(7) We'll also need to supply all the things that the [PT] decoder normally finds out via sysfs attributes, like clock ratios, capabilities, etc so that it also finds its way into the core dump file.
"PMU info" structure is appended to the user page.

I've also hack the perf tool to support all this, all these things can be found at [1]. I'm not posting the tooling patches though, them being thoroughly ugly and proof-of-concept. In short, perf record will create detached events with '--detached' and afterwards will open detached events via their path in tracefs.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/ash/linux.git/log/?h=perf-detached-shmem-wip

Alexander Shishkin (17):
perf: Allow mmapping only user page
perf: Factor out mlock accounting
tracefs: De-globalize instances' callbacks
tracefs: Add ->unlink callback to tracefs_dir_ops
perf: Introduce detached events
perf: Add buffers to the detached events
perf: Add pmu_info to user page
perf: Allow inheritance for detached events
perf: Use shmemfs pages for userspace-only per-thread detached events
perf: Implement pinning and scheduling for SHMEM events
perf: Implement mlock accounting for shmem ring buffers
perf: Track pinned events per user
perf: Re-inject shmem buffers after exec
perf: Add ioctl(REATTACH) for detached events
perf: Allow controlled non-root access to detached events perf/x86/intel/pt: Add PMU info
perf/x86/intel/bts: Add PMU info

 arch/x86/events/intel/bts.c     |  20 +-
 arch/x86/events/intel/pt.c      |  23 +-
 arch/x86/events/intel/pt.h      |  11 +
 fs/tracefs/inode.c              |  71 +++-
 include/linux/perf_event.h      |  33 ++
 include/linux/sched/user.h      |   6 +
 include/linux/tracefs.h         |   3 +-
 include/uapi/linux/perf_event.h |  15 +
 kernel/events/core.c            | 526 +++++++++++++++++++++++------
 kernel/events/internal.h        |  27 +-
 kernel/events/ring_buffer.c     | 730 ++++++++++++++++++++++++++++++++++++--
 kernel/trace/trace.c            |   8 +-
 kernel/user.c                   |   1 +
 13 files changed, 1315 insertions(+), 159 deletions(-)
Signature · 1 line
2.14.1
author reviewer
#1 lore raw

RFC PATCH 1/17 perf: Allow mmapping only user page

The 'user page' contains offsets and sizes of data and aux areas of the ring buffer. If a user wants to mmap a pre-existing buffer, they need to know these in order to issue mmap()s with correct offsets and sizes.

This enables mmapping of the user page if the ring buffer already exists.

---
 kernel/events/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/events/core.c +1 -1
index cb7eaf0f91..9389e27cb0 100644
@@ -5366,7 +5366,7 @@ static int perf_mmap(struct file *file, struct vm_area_struct *vma)
 again:
 	mutex_lock(&event->mmap_mutex);
 	if (event->rb) {
-		if (event->rb->nr_pages != nr_pages) {
+		if (nr_pages && event->rb->nr_pages != nr_pages) {
 			ret = -EINVAL;
 			goto unlock;
 		}
Signature · 1 line
2.14.1
maintainer
#2 lore raw

Re: [RFC PATCH 01/17] perf: Allow mmapping only user page

Quoted 3 lines · Alexander Shishkin

The 'user page' contains offsets and sizes of data and aux areas of the ring buffer. If a user wants to mmap a pre-existing buffer, they need to know these in order to issue mmap()s with correct offsets and sizes.

Ok, stupid question: shouldn't this be a properly defined interface instead of allowing userspace to poke inside the user page? Or are we prepared to handle any changes in the layout of that user page and there won't be any userspace crying because of it?

Thx.

Signature · 4 lines
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.
author reviewer
#3 lore raw

Re: [RFC PATCH 01/17] perf: Allow mmapping only user page

Quoted 9 lines · Borislav Petkov
Quoted 3 lines · Alexander Shishkin

The 'user page' contains offsets and sizes of data and aux areas of the ring buffer. If a user wants to mmap a pre-existing buffer, they need to know these in order to issue mmap()s with correct offsets and sizes.

Ok, stupid question: shouldn't this be a properly defined interface instead of allowing userspace to poke inside the user page? Or are we prepared to handle any changes in the layout of that user page and there won't be any userspace crying because of it?

Well, it is a 'defined' interface: there's the 'struct perf_event_mmap_page' with versioning and whatnot, which is used for all the ring buffer metainformation. Or am I misunderstanding your question?

Thanks,
--
Alex

maintainer
#4 lore raw

Re: [RFC PATCH 01/17] perf: Allow mmapping only user page

Quoted 3 lines · Alexander Shishkin

Well, it is a 'defined' interface: there's the 'struct perf_event_mmap_page' with versioning and whatnot, which is used for all the ring buffer metainformation. Or am I misunderstanding your question?

No, you're not. Looking at perf_event_mmap_page, sounds like my concerns are addressed. :)

Thx.

Signature · 4 lines
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.
author reviewer
#5 lore raw

RFC PATCH 2/17 perf: Factor out mlock accounting

In reply to Alexander Shishkin

This patch moves ring buffer memory accounting down the rb_alloc() path so that its callers won't have to worry about it. This also serves the additional purpose of slightly cleaning up perf_mmap().

---
 kernel/events/core.c        |  67 +++-----------------
 kernel/events/internal.h    |   5 +-
 kernel/events/ring_buffer.c | 145 ++++++++++++++++++++++++++++++++++++++------
 3 files changed, 136 insertions(+), 81 deletions(-)
kernel/events/core.c +7 -60
index 9389e27cb0..24099ed9e5 100644
@@ -5122,6 +5122,8 @@ void ring_buffer_put(struct ring_buffer *rb)
 	if (!atomic_dec_and_test(&rb->refcount))
 		return;
 
+	ring_buffer_unaccount(rb, false);
+
 	WARN_ON_ONCE(!list_empty(&rb->event_list));
 
 	call_rcu(&rb->rcu_head, rb_free_rcu);
@@ -5156,9 +5158,6 @@ static void perf_mmap_close(struct vm_area_struct *vma)
 	struct perf_event *event = vma->vm_file->private_data;
 
 	struct ring_buffer *rb = ring_buffer_get(event);
-	struct user_struct *mmap_user = rb->mmap_user;
-	int mmap_locked = rb->mmap_locked;
-	unsigned long size = perf_data_size(rb);
 
 	if (event->pmu->event_unmapped)
 		event->pmu->event_unmapped(event, vma->vm_mm);
@@ -5178,11 +5177,7 @@ static void perf_mmap_close(struct vm_area_struct *vma)
 		 */
 		perf_pmu_output_stop(event);
 
-		/* now it's safe to free the pages */
-		atomic_long_sub(rb->aux_nr_pages, &mmap_user->locked_vm);
-		vma->vm_mm->pinned_vm -= rb->aux_mmap_locked;
-
-		/* this has to be the last one */
+		/* now it's safe to free the pages; ought to be the last one */
 		rb_free_aux(rb);
 		WARN_ON_ONCE(atomic_read(&rb->aux_refcount));
 
@@ -5243,19 +5238,6 @@ static void perf_mmap_close(struct vm_area_struct *vma)
 	}
 	rcu_read_unlock();
 
-	/*
-	 * It could be there's still a few 0-ref events on the list; they'll
-	 * get cleaned up by free_event() -- they'll also still have their
-	 * ref on the rb and will free it whenever they are done with it.
-	 *
-	 * Aside from that, this buffer is 'fully' detached and unmapped,
-	 * undo the VM accounting.
-	 */
-
-	atomic_long_sub((size >> PAGE_SHIFT) + 1, &mmap_user->locked_vm);
-	vma->vm_mm->pinned_vm -= mmap_locked;
-	free_uid(mmap_user);
-
 out_put:
 	ring_buffer_put(rb); /* could be last */
 }
@@ -5270,13 +5252,9 @@ static const struct vm_operations_struct perf_mmap_vmops = {
 static int perf_mmap(struct file *file, struct vm_area_struct *vma)
 {
 	struct perf_event *event = file->private_data;
-	unsigned long user_locked, user_lock_limit;
-	struct user_struct *user = current_user();
-	unsigned long locked, lock_limit;
 	struct ring_buffer *rb = NULL;
 	unsigned long vma_size;
 	unsigned long nr_pages;
-	long user_extra = 0, extra = 0;
 	int ret = 0, flags = 0;
 
 	/*
@@ -5347,7 +5325,6 @@ static int perf_mmap(struct file *file, struct vm_area_struct *vma)
 		}
 
 		atomic_set(&rb->aux_mmap_count, 1);
-		user_extra = nr_pages;
 
 		goto accounting;
 	}
@@ -5384,49 +5361,24 @@ static int perf_mmap(struct file *file, struct vm_area_struct *vma)
 		goto unlock;
 	}
 
-	user_extra = nr_pages + 1;
-
 accounting:
-	user_lock_limit = sysctl_perf_event_mlock >> (PAGE_SHIFT - 10);
-
-	/*
-	 * Increase the limit linearly with more CPUs:
-	 */
-	user_lock_limit *= num_online_cpus();
-
-	user_locked = atomic_long_read(&user->locked_vm) + user_extra;
-
-	if (user_locked > user_lock_limit)
-		extra = user_locked - user_lock_limit;
-
-	lock_limit = rlimit(RLIMIT_MEMLOCK);
-	lock_limit >>= PAGE_SHIFT;
-	locked = vma->vm_mm->pinned_vm + extra;
-
-	if ((locked > lock_limit) && perf_paranoid_tracepoint_raw() &&
-		!capable(CAP_IPC_LOCK)) {
-		ret = -EPERM;
-		goto unlock;
-	}
-
 	WARN_ON(!rb && event->rb);
 
 	if (vma->vm_flags & VM_WRITE)
 		flags |= RING_BUFFER_WRITABLE;
 
 	if (!rb) {
-		rb = rb_alloc(nr_pages,
+		rb = rb_alloc(vma->vm_mm, nr_pages,
 			      event->attr.watermark ? event->attr.wakeup_watermark : 0,
 			      event->cpu, flags);
 
-		if (!rb) {
-			ret = -ENOMEM;
+		if (IS_ERR_OR_NULL(rb)) {
+			ret = PTR_ERR(rb);
+			rb = NULL;
 			goto unlock;
 		}
 
 		atomic_set(&rb->mmap_count, 1);
-		rb->mmap_user = get_current_user();
-		rb->mmap_locked = extra;
 
 		ring_buffer_attach(event, rb);
 
@@ -5435,15 +5387,10 @@ static int perf_mmap(struct file *file, struct vm_area_struct *vma)
 	} else {
 		ret = rb_alloc_aux(rb, event, vma->vm_pgoff, nr_pages,
 				   event->attr.aux_watermark, flags);
-		if (!ret)
-			rb->aux_mmap_locked = extra;
 	}
 
 unlock:
 	if (!ret) {
-		atomic_long_add(user_extra, &user->locked_vm);
-		vma->vm_mm->pinned_vm += extra;
-
 		atomic_inc(&event->mmap_count);
 	} else if (rb) {
 		atomic_dec(&rb->mmap_count);
kernel/events/internal.h +4 -1
index 843e970473..3e603c45eb 100644
@@ -36,6 +36,7 @@ struct ring_buffer {
 	atomic_t			mmap_count;
 	unsigned long			mmap_locked;
 	struct user_struct		*mmap_user;
+	struct mm_struct		*mmap_mapping;
 
 	/* AUX area */
 	long				aux_head;
@@ -56,6 +57,7 @@ struct ring_buffer {
 };
 
 extern void rb_free(struct ring_buffer *rb);
+extern void ring_buffer_unaccount(struct ring_buffer *rb, bool aux);
 
 static inline void rb_free_rcu(struct rcu_head *rcu_head)
 {
@@ -74,7 +76,8 @@ static inline void rb_toggle_paused(struct ring_buffer *rb, bool pause)
 }
 
 extern struct ring_buffer *
-rb_alloc(int nr_pages, long watermark, int cpu, int flags);
+rb_alloc(struct mm_struct *mm, int nr_pages, long watermark, int cpu,
+	 int flags);
 extern void perf_event_wakeup(struct perf_event *event);
 extern int rb_alloc_aux(struct ring_buffer *rb, struct perf_event *event,
 			pgoff_t pgoff, int nr_pages, long watermark, int flags);
kernel/events/ring_buffer.c +125 -20

229 lines not on this page · raw message

Signature · 1 line
2.14.1
author reviewer
#6 lore raw

RFC PATCH 3/17 tracefs: De-globalize instances' callbacks

In reply to Alexander Shishkin

Currently, tracefs has exactly one special 'instances' subdirectory, where the caller can have their own .mkdir/.rmdir callbacks, which allow the caller to handle user's mkdir/rmdir inside that directory. Tracefs allows one set of these callbacks (tracefs_dir_ops).

This patch de-globalizes tracefs_dir_ops so that it's possible to have multiple such subdirectories.

---
 fs/tracefs/inode.c | 35 +++++++++++++++++++++++++----------
 1 file changed, 25 insertions(+), 10 deletions(-)
fs/tracefs/inode.c +25 -10
index bea8ad876b..b14f03a655 100644
@@ -50,10 +50,10 @@ static const struct file_operations tracefs_file_operations = {
 	.llseek =	noop_llseek,
 };
 
-static struct tracefs_dir_ops {
+struct tracefs_dir_ops {
 	int (*mkdir)(const char *name);
 	int (*rmdir)(const char *name);
-} tracefs_ops;
+};
 
 static char *get_dname(struct dentry *dentry)
 {
@@ -72,6 +72,7 @@ static char *get_dname(struct dentry *dentry)
 
 static int tracefs_syscall_mkdir(struct inode *inode, struct dentry *dentry, umode_t mode)
 {
+	struct tracefs_dir_ops *tracefs_ops = dentry->d_parent->d_fsdata;
 	char *name;
 	int ret;
 
@@ -85,7 +86,7 @@ static int tracefs_syscall_mkdir(struct inode *inode, struct dentry *dentry, umo
 	 * mkdir routine to handle races.
 	 */
 	inode_unlock(inode);
-	ret = tracefs_ops.mkdir(name);
+	ret = tracefs_ops->mkdir(name);
 	inode_lock(inode);
 
 	kfree(name);
@@ -95,6 +96,7 @@ static int tracefs_syscall_mkdir(struct inode *inode, struct dentry *dentry, umo
 
 static int tracefs_syscall_rmdir(struct inode *inode, struct dentry *dentry)
 {
+	struct tracefs_dir_ops *tracefs_ops = dentry->d_fsdata;
 	char *name;
 	int ret;
 
@@ -112,7 +114,7 @@ static int tracefs_syscall_rmdir(struct inode *inode, struct dentry *dentry)
 	inode_unlock(inode);
 	inode_unlock(dentry->d_inode);
 
-	ret = tracefs_ops.rmdir(name);
+	ret = tracefs_ops->rmdir(name);
 
 	inode_lock_nested(inode, I_MUTEX_PARENT);
 	inode_lock(dentry->d_inode);
@@ -342,6 +344,9 @@ static struct dentry *start_creating(const char *name, struct dentry *parent)
 	if (IS_ERR(dentry)) {
 		inode_unlock(parent->d_inode);
 		simple_release_fs(&tracefs_mount, &tracefs_mount_count);
+	} else {
+		/* propagate dir ops */
+		dentry->d_fsdata = parent->d_fsdata;
 	}
 
 	return dentry;
@@ -482,18 +487,25 @@ struct dentry *tracefs_create_instance_dir(const char *name, struct dentry *pare
 					  int (*mkdir)(const char *name),
 					  int (*rmdir)(const char *name))
 {
+	struct tracefs_dir_ops *tracefs_ops = parent ? parent->d_fsdata : NULL;
 	struct dentry *dentry;
 
-	/* Only allow one instance of the instances directory. */
-	if (WARN_ON(tracefs_ops.mkdir || tracefs_ops.rmdir))
+	if (WARN_ON(tracefs_ops))
+		return NULL;
+
+	tracefs_ops = kzalloc(sizeof(*tracefs_ops), GFP_KERNEL);
+	if (!tracefs_ops)
 		return NULL;
 
 	dentry = __create_dir(name, parent, &tracefs_dir_inode_operations);
-	if (!dentry)
+	if (!dentry) {
+		kfree(tracefs_ops);
 		return NULL;
+	}
 
-	tracefs_ops.mkdir = mkdir;
-	tracefs_ops.rmdir = rmdir;
+	tracefs_ops->mkdir = mkdir;
+	tracefs_ops->rmdir = rmdir;
+	dentry->d_fsdata = tracefs_ops;
 
 	return dentry;
 }
@@ -513,8 +525,11 @@ static int __tracefs_remove(struct dentry *dentry, struct dentry *parent)
 				simple_unlink(parent->d_inode, dentry);
 				break;
 			}
-			if (!ret)
+			if (!ret) {
 				d_delete(dentry);
+				if (dentry->d_fsdata != parent->d_fsdata)
+					kfree(dentry->d_fsdata);
+			}
 			dput(dentry);
 		}
 	}
Signature · 1 line
2.14.1
maintainer
#7 lore raw

Re: [RFC PATCH 03/17] tracefs: De-globalize instances' callbacks

I just stumbled across this patch (and the following one). What purpose would this have? The "instances" directory is used to make multiple buffers. Why would we have more than one?

-- Steve

Quoted 119 lines · On Tue, 5 Sep 2017 16:30:12 +0300 Alexander Shishkin…

Currently, tracefs has exactly one special 'instances' subdirectory, where the caller can have their own .mkdir/.rmdir callbacks, which allow the caller to handle user's mkdir/rmdir inside that directory. Tracefs allows one set of these callbacks (tracefs_dir_ops).

This patch de-globalizes tracefs_dir_ops so that it's possible to have multiple such subdirectories.

---
 fs/tracefs/inode.c | 35 +++++++++++++++++++++++++----------
 1 file changed, 25 insertions(+), 10 deletions(-)
fs/tracefs/inode.c +25 -10
index bea8ad876b..b14f03a655 100644
@@ -50,10 +50,10 @@ static const struct file_operations tracefs_file_operations = {
 	.llseek =	noop_llseek,
 };
 
-static struct tracefs_dir_ops {
+struct tracefs_dir_ops {
 	int (*mkdir)(const char *name);
 	int (*rmdir)(const char *name);
-} tracefs_ops;
+};
 
 static char *get_dname(struct dentry *dentry)
 {
@@ -72,6 +72,7 @@ static char *get_dname(struct dentry *dentry)
 
 static int tracefs_syscall_mkdir(struct inode *inode, struct dentry *dentry, umode_t mode)
 {
+	struct tracefs_dir_ops *tracefs_ops = dentry->d_parent->d_fsdata;
 	char *name;
 	int ret;
 
@@ -85,7 +86,7 @@ static int tracefs_syscall_mkdir(struct inode *inode, struct dentry *dentry, umo
 	 * mkdir routine to handle races.
 	 */
 	inode_unlock(inode);
-	ret = tracefs_ops.mkdir(name);
+	ret = tracefs_ops->mkdir(name);
 	inode_lock(inode);
 
 	kfree(name);
@@ -95,6 +96,7 @@ static int tracefs_syscall_mkdir(struct inode *inode, struct dentry *dentry, umo
 
 static int tracefs_syscall_rmdir(struct inode *inode, struct dentry *dentry)
 {
+	struct tracefs_dir_ops *tracefs_ops = dentry->d_fsdata;
 	char *name;
 	int ret;
 
@@ -112,7 +114,7 @@ static int tracefs_syscall_rmdir(struct inode *inode, struct dentry *dentry)
 	inode_unlock(inode);
 	inode_unlock(dentry->d_inode);
 
-	ret = tracefs_ops.rmdir(name);
+	ret = tracefs_ops->rmdir(name);
 
 	inode_lock_nested(inode, I_MUTEX_PARENT);
 	inode_lock(dentry->d_inode);
@@ -342,6 +344,9 @@ static struct dentry *start_creating(const char *name, struct dentry *parent)
 	if (IS_ERR(dentry)) {
 		inode_unlock(parent->d_inode);
 		simple_release_fs(&tracefs_mount, &tracefs_mount_count);
+	} else {
+		/* propagate dir ops */
+		dentry->d_fsdata = parent->d_fsdata;
 	}
 
 	return dentry;
@@ -482,18 +487,25 @@ struct dentry *tracefs_create_instance_dir(const char *name, struct dentry *pare
 					  int (*mkdir)(const char *name),
 					  int (*rmdir)(const char *name))
 {
+	struct tracefs_dir_ops *tracefs_ops = parent ? parent->d_fsdata : NULL;
 	struct dentry *dentry;
 
-	/* Only allow one instance of the instances directory. */
-	if (WARN_ON(tracefs_ops.mkdir || tracefs_ops.rmdir))
+	if (WARN_ON(tracefs_ops))
+		return NULL;
+
+	tracefs_ops = kzalloc(sizeof(*tracefs_ops), GFP_KERNEL);
+	if (!tracefs_ops)
 		return NULL;
 
 	dentry = __create_dir(name, parent, &tracefs_dir_inode_operations);
-	if (!dentry)
+	if (!dentry) {
+		kfree(tracefs_ops);
 		return NULL;
+	}
 
-	tracefs_ops.mkdir = mkdir;
-	tracefs_ops.rmdir = rmdir;
+	tracefs_ops->mkdir = mkdir;
+	tracefs_ops->rmdir = rmdir;
+	dentry->d_fsdata = tracefs_ops;
 
 	return dentry;
 }
@@ -513,8 +525,11 @@ static int __tracefs_remove(struct dentry *dentry, struct dentry *parent)
 				simple_unlink(parent->d_inode, dentry);
 				break;
 			}
-			if (!ret)
+			if (!ret) {
 				d_delete(dentry);
+				if (dentry->d_fsdata != parent->d_fsdata)
+					kfree(dentry->d_fsdata);
+			}
 			dput(dentry);
 		}
 	}
author reviewer
#8 lore raw

RFC PATCH 4/17 tracefs: Add ->unlink callback to tracefs_dir_ops

In reply to Alexander Shishkin

In addition to mkdir and rmdir, also allow unlink operation within the 'instances' directory if such callback is defined.

---
 fs/tracefs/inode.c      | 36 +++++++++++++++++++++++++++++++++++-
 include/linux/tracefs.h |  3 ++-
 kernel/trace/trace.c    |  8 +++++++-
 3 files changed, 44 insertions(+), 3 deletions(-)
fs/tracefs/inode.c +35 -1
index b14f03a655..fba5a0ce07 100644
@@ -53,6 +53,7 @@ static const struct file_operations tracefs_file_operations = {
 struct tracefs_dir_ops {
 	int (*mkdir)(const char *name);
 	int (*rmdir)(const char *name);
+	int (*unlink)(const char *name);
 };
 
 static char *get_dname(struct dentry *dentry)
@@ -124,10 +125,41 @@ static int tracefs_syscall_rmdir(struct inode *inode, struct dentry *dentry)
 	return ret;
 }
 
+static int tracefs_syscall_unlink(struct inode *inode, struct dentry *dentry)
+{
+	struct tracefs_dir_ops *tracefs_ops = dentry->d_fsdata;
+	char *name;
+	int ret;
+
+	name = get_dname(dentry);
+	if (!name)
+		return -ENOMEM;
+
+	/*
+	 * The unlink call can call the generic functions that create
+	 * the files within the tracefs system. It is up to the individual
+	 * unlink routine to handle races.
+	 * This time we need to unlock not only the parent (inode) but
+	 * also the file that is being deleted.
+	 */
+	inode_unlock(inode);
+	inode_unlock(dentry->d_inode);
+
+	ret = tracefs_ops->unlink(name);
+
+	inode_lock_nested(inode, I_MUTEX_PARENT);
+	inode_lock(dentry->d_inode);
+
+	kfree(name);
+
+	return ret;
+}
+
 static const struct inode_operations tracefs_dir_inode_operations = {
 	.lookup		= simple_lookup,
 	.mkdir		= tracefs_syscall_mkdir,
 	.rmdir		= tracefs_syscall_rmdir,
+	.unlink		= tracefs_syscall_unlink,
 };
 
 static struct inode *tracefs_get_inode(struct super_block *sb)
@@ -485,7 +517,8 @@ struct dentry *tracefs_create_dir(const char *name, struct dentry *parent)
  */
 struct dentry *tracefs_create_instance_dir(const char *name, struct dentry *parent,
 					  int (*mkdir)(const char *name),
-					  int (*rmdir)(const char *name))
+					  int (*rmdir)(const char *name),
+					  int (*unlink)(const char *name))
 {
 	struct tracefs_dir_ops *tracefs_ops = parent ? parent->d_fsdata : NULL;
 	struct dentry *dentry;
@@ -505,6 +538,7 @@ struct dentry *tracefs_create_instance_dir(const char *name, struct dentry *pare
 
 	tracefs_ops->mkdir = mkdir;
 	tracefs_ops->rmdir = rmdir;
+	tracefs_ops->unlink = unlink;
 	dentry->d_fsdata = tracefs_ops;
 
 	return dentry;
include/linux/tracefs.h +2 -1
index 5b727a17be..e5bd1f01b6 100644
@@ -36,7 +36,8 @@ void tracefs_remove_recursive(struct dentry *dentry);
 
 struct dentry *tracefs_create_instance_dir(const char *name, struct dentry *parent,
 					   int (*mkdir)(const char *name),
-					   int (*rmdir)(const char *name));
+					   int (*rmdir)(const char *name),
+					   int (*unlink)(const char *name));
 
 bool tracefs_initialized(void);
 
kernel/trace/trace.c +7 -1
index 44004d8aa3..b9abd2029e 100644
@@ -7792,11 +7792,17 @@ static int instance_rmdir(const char *name)
 	return ret;
 }
 
+static int instance_unlink(const char *name)
+{
+	return -EACCES;
+}
+
 static __init void create_trace_instances(struct dentry *d_tracer)
 {
 	trace_instance_dir = tracefs_create_instance_dir("instances", d_tracer,
 							 instance_mkdir,
-							 instance_rmdir);
+							 instance_rmdir,
+							 instance_unlink);
 	if (WARN_ON(!trace_instance_dir))
 		return;
 }
Signature · 1 line
2.14.1
author reviewer
#9 lore raw

RFC PATCH 5/17 perf: Introduce detached events

In reply to Alexander Shishkin

There are usecases where it is desired to have perf events without the userspace tool running in the background to keep them alive, but instead only collect the data when it is needed, for example when an MCE event is triggered.

This patch adds a new flag to the perf_event_open() syscall that allows creating such events. Once created, the file descriptor can be closed and the event continues to exist on its own. To allow access to this event, a file is created in the tracefs, which the user can open.

Finally, when it is no longer needed, it can be destroyed by unlinking the file.

---
 include/linux/perf_event.h      |   4 ++
 include/uapi/linux/perf_event.h |   1 +
 kernel/events/core.c            | 138 ++++++++++++++++++++++++++++++++++++++--
 kernel/events/internal.h        |   6 ++
 4 files changed, 142 insertions(+), 7 deletions(-)
include/linux/perf_event.h +4 -0
index 82b2e3fef9..a07982f48d 100644
@@ -537,6 +537,7 @@ typedef void (*perf_overflow_handler_t)(struct perf_event *,
 #define PERF_EV_CAP_SOFTWARE		BIT(0)
 #define PERF_EV_CAP_READ_ACTIVE_PKG	BIT(1)
 
+#define PERF_TRACEFS_HASH_BITS		32
 #define SWEVENT_HLIST_BITS		8
 #define SWEVENT_HLIST_SIZE		(1 << SWEVENT_HLIST_BITS)
 
@@ -550,6 +551,7 @@ struct swevent_hlist {
 #define PERF_ATTACH_TASK	0x04
 #define PERF_ATTACH_TASK_DATA	0x08
 #define PERF_ATTACH_ITRACE	0x10
+#define PERF_ATTACH_DETACHED	0x20
 
 struct perf_cgroup;
 struct ring_buffer;
@@ -672,6 +674,8 @@ struct perf_event {
 	struct list_head		owner_entry;
 	struct task_struct		*owner;
 
+	struct dentry			*dent;
+
 	/* mmap bits */
 	struct mutex			mmap_mutex;
 	atomic_t			mmap_count;
include/uapi/linux/perf_event.h +1 -0
index 140ae638cf..89355584fa 100644
@@ -946,6 +946,7 @@ enum perf_callchain_context {
 #define PERF_FLAG_FD_OUTPUT		(1UL << 1)
 #define PERF_FLAG_PID_CGROUP		(1UL << 2) /* pid=cgroup id, per-cpu mode only */
 #define PERF_FLAG_FD_CLOEXEC		(1UL << 3) /* O_CLOEXEC */
+#define PERF_FLAG_DETACHED		(1UL << 4) /* event w/o owner */
 
 #if defined(__LITTLE_ENDIAN_BITFIELD)
 union perf_mem_data_src {
kernel/events/core.c +131 -7

223 lines not on this page · raw message

kernel/events/internal.h +6 -0
index 3e603c45eb..59136a0e98 100644
@@ -126,6 +126,12 @@ static inline unsigned long perf_aux_size(struct ring_buffer *rb)
 	return rb->aux_nr_pages << PAGE_SHIFT;
 }
 
+static inline bool is_detached_event(struct perf_event *event)
+{
+	lockdep_assert_held(&event->ctx->mutex);
+	return !!(event->attach_state & PERF_ATTACH_DETACHED);
+}
+
 #define __DEFINE_OUTPUT_COPY_BODY(advance_buf, memcpy_func, ...)	\
 {									\
 	unsigned long size, written;					\
Signature · 1 line
2.14.1
maintainer
#10 lore raw

Re: [RFC PATCH 05/17] perf: Introduce detached events

Quoted 29 lines · Alexander Shishkin

There are usecases where it is desired to have perf events without the userspace tool running in the background to keep them alive, but instead only collect the data when it is needed, for example when an MCE event is triggered.

This patch adds a new flag to the perf_event_open() syscall that allows creating such events. Once created, the file descriptor can be closed and the event continues to exist on its own. To allow access to this event, a file is created in the tracefs, which the user can open.

Finally, when it is no longer needed, it can be destroyed by unlinking the file.

kernel/events/core.c +11 -0
@@ -9387,6 +9416,27 @@ static void account_event(struct perf_event *event)
 	account_pmu_sb_event(event);
 }
 
+static int perf_event_detach(struct perf_event *event, struct task_struct *task,
+			     struct mm_struct *mm)
+{
+	char *filename;
+
+	filename = kasprintf(GFP_KERNEL, "%s:%x.event",
+			     task ? "task" : "cpu",
+			     hash_64((u64)event, PERF_TRACEFS_HASH_BITS));
+	if (!filename)
+		return -ENOMEM;
+

kernel/events/core.c static void account_event(struct perf_event *event) · line 9439

Quoted 10 lines
kernel/events/core.c +10 -0
+	event->dent = tracefs_create_file(filename, 0600,
+					  perf_tracefs_dir,
+					  event, &perf_fops);
+	kfree(filename);
+
+	if (!event->dent)
+		return -ENOMEM;
+
+	return 0;
+}

So I'm not opposed to the idea of creating events that live independent from of file descriptors. And stuffing them in a filesystem makes sense. However I'm not entire convinced on the details.

The above has a number of problems:

- there's a filesystem race; two concurrent syscalls can try and create the same file. In that case the error most certainly is not -ENOMEM.

- there's a hash collision, similar issue.

- there's some asymmetry in the create/destroy; that is you create the file with sys_perf_event_open() and remove it with unlink().

- the actual name is very opaque and hard to use; how would a tool find the right event to open?

Would it instead make sense to allow the user to creat() their own files in this filesystem (with whatever descriptive name they need) and then pass that fd like:

  sys_perf_event_open(.group_fd=fd, .flags=PERF_FLAG_FD_DETACH);

or something to associate the file with the event. Of course, that makes it very hard to create detached cgroup events :/

author reviewer
#11 lore raw

Re: [RFC PATCH 05/17] perf: Introduce detached events

Quoted 8 lines · Peter Zijlstra

So I'm not opposed to the idea of creating events that live independent from of file descriptors. And stuffing them in a filesystem makes sense. However I'm not entire convinced on the details.

The above has a number of problems:

- there's a filesystem race; two concurrent syscalls can try and create the same file. In that case the error most certainly is not -ENOMEM.

Indeed.

Quoted 4 lines

- there's a hash collision, similar issue.

- there's some asymmetry in the create/destroy; that is you create the file with sys_perf_event_open() and remove it with unlink().

There is also an ioctl() to turn it into a normal event fd that can then be closed.

Quoted 2 lines

- the actual name is very opaque and hard to use; how would a tool find the right event to open?

They can readlink("/proc/self/fd/$fd"), something that I hacked into the perf tool as well, although, truth be told I didn't actually need it for anything, partly because it's not a useful name. One use case that I could think of would be a task that's inherited a detached event wanting to get rid of it. They can scan their /proc/$pid/maps, find the vma by name and use that to locate the file.

Quoted 8 lines

Would it instead make sense to allow the user to creat() their own files in this filesystem (with whatever descriptive name they need) and then pass that fd like:

  sys_perf_event_open(.group_fd=fd, .flags=PERF_FLAG_FD_DETACH);

or something to associate the file with the event. Of course, that makes it very hard to create detached cgroup events :/

Yes, I like the idea of moving the burden of naming to the userspace, but then we have a problem with inheritance, which would still produce new events w/o user's input.

Maybe use a directory for the 'parent' event? Then the above would still work.

Regards,
--
Alex

author reviewer
#12 lore raw

RFC PATCH 6/17 perf: Add buffers to the detached events

In reply to Alexander Shishkin

Detached events make much more sense with ring buffers, which the user can mmap and read a snapshot of. Unlike the normal perf events, these ring buffers are allocated by the perf syscall, the sizes of data and aux areas are specified in the event attribute.

These ring buffers can be mmapped read-only.

---
 include/uapi/linux/perf_event.h |  3 +++
 kernel/events/core.c            | 19 ++++++++++++++++
 kernel/events/internal.h        |  2 ++
 kernel/events/ring_buffer.c     | 50 +++++++++++++++++++++++++++++++++++++++++
 4 files changed, 74 insertions(+)
include/uapi/linux/perf_event.h +3 -0
index 89355584fa..3d64d9ea80 100644
@@ -297,6 +297,7 @@ enum perf_event_read_format {
 					/* add: sample_stack_user */
 #define PERF_ATTR_SIZE_VER4	104	/* add: sample_regs_intr */
 #define PERF_ATTR_SIZE_VER5	112	/* add: aux_watermark */
+#define PERF_ATTR_SIZE_VER6	120	/* add: detached_* */
 
 /*
  * Hardware event_id to monitor via a performance monitoring event:
@@ -415,6 +416,8 @@ struct perf_event_attr {
 	__u32	aux_watermark;
 	__u16	sample_max_stack;
 	__u16	__reserved_2;	/* align to __u64 */
+	__u32	detached_nr_pages;
+	__u32	detached_aux_nr_pages;
 };
 
 #define perf_flags(attr)	(*(&(attr)->read_format + 1))
kernel/events/core.c +19 -0
index 320070410d..fef1f97974 100644
@@ -4185,6 +4185,9 @@ static void _free_event(struct perf_event *event)
 		tracefs_remove(event->dent);
 
 		event->attach_state &= ~PERF_ATTACH_DETACHED;
+
+		ring_buffer_unaccount(event->rb, false);
+		rb_free_detached(event->rb, event);
 	}
 
 	if (event->rb) {
@@ -5012,6 +5015,10 @@ static int perf_mmap_fault(struct vm_fault *vmf)
 	int ret = VM_FAULT_SIGBUS;
 
 	if (vmf->flags & FAULT_FLAG_MKWRITE) {
+		/* detached events R/O only */
+		if (event->dent)
+			return ret;
+
 		if (vmf->pgoff == 0)
 			ret = 0;
 		return ret;
@@ -9420,6 +9427,7 @@ static int perf_event_detach(struct perf_event *event, struct task_struct *task,
 			     struct mm_struct *mm)
 {
 	char *filename;
+	int err;
 
 	filename = kasprintf(GFP_KERNEL, "%s:%x.event",
 			     task ? "task" : "cpu",
@@ -9435,6 +9443,13 @@ static int perf_event_detach(struct perf_event *event, struct task_struct *task,
 	if (!event->dent)
 		return -ENOMEM;
 
+	err = rb_alloc_detached(event);
+	if (err) {
+		tracefs_remove(event->dent);
+		event->dent = NULL;
+		return err;
+	}
+
 	return 0;
 }
 /*
@@ -10017,6 +10032,9 @@ SYSCALL_DEFINE5(perf_event_open,
 		if (output_event || (group_fd != -1))
 			goto err_task;
 
+		if (!attr.detached_nr_pages)
+			goto err_task;
+
 		detached = 1;
 	}
 
@@ -10174,6 +10192,7 @@ SYSCALL_DEFINE5(perf_event_open,
 			goto err_context;
 
 		atomic_long_inc(&event->refcount);
+		atomic_inc(&event->mmap_count);
 
 		event_file->private_data = event;
 	}
kernel/events/internal.h +2 -0
index 59136a0e98..8e267d8faa 100644
@@ -82,6 +82,8 @@ extern void perf_event_wakeup(struct perf_event *event);
 extern int rb_alloc_aux(struct ring_buffer *rb, struct perf_event *event,
 			pgoff_t pgoff, int nr_pages, long watermark, int flags);
 extern void rb_free_aux(struct ring_buffer *rb);
+extern int rb_alloc_detached(struct perf_event *event);
+extern void rb_free_detached(struct ring_buffer *rb, struct perf_event *event);
 extern struct ring_buffer *ring_buffer_get(struct perf_event *event);
 extern void ring_buffer_put(struct ring_buffer *rb);
 
kernel/events/ring_buffer.c +50 -0

56 lines not on this page · raw message

Signature · 1 line
2.14.1
maintainer
#13 lore raw

Re: [RFC PATCH 06/17] perf: Add buffers to the detached events

include/uapi/linux/perf_event.h struct perf_event_attr { · line 421

Quoted 7 lines · Alexander Shishkin
include/uapi/linux/perf_event.h +2 -0
@@ -415,6 +416,8 @@ struct perf_event_attr {
 	__u32	aux_watermark;
 	__u16	sample_max_stack;
 	__u16	__reserved_2;	/* align to __u64 */
+	__u32	detached_nr_pages;
+	__u32	detached_aux_nr_pages;
 };

Not sure the naming makes sense; I don't see why this would be limited to detached events. That is, what would stop someone from pre-allocating buffers on 'regular' events if they wanted to?

author reviewer
#14 lore raw

RFC PATCH 7/17 perf: Add pmu_info to user page

In reply to Alexander Shishkin

Allow PMUs to supply additional static information that may be required by their decoders. Most of what Intel PT driver exports as capability attributes (timing packet freqencies, frequency ratios etc), its decoder needs to be able to correctly decode its binary stream. However, when decoding Intel PT stream from a core dump, we can't rely on the sysfs attributes, so we need to pack this information into the perf buffer, so that the resulting core dump is self-contained.

In order to do this, we append a PMU-specific structure to the user page. Such structures will include size, for versioning.

---
 include/linux/perf_event.h      | 17 ++++++++++
 include/uapi/linux/perf_event.h | 10 ++++++
 kernel/events/core.c            | 27 +--------------
 kernel/events/internal.h        |  2 +-
 kernel/events/ring_buffer.c     | 75 ++++++++++++++++++++++++++++++++++-------
 5 files changed, 92 insertions(+), 39 deletions(-)
include/linux/perf_event.h +17 -0
index a07982f48d..b7939e8811 100644
@@ -235,6 +235,8 @@ struct hw_perf_event {
 
 struct perf_event;
 
+struct pmu_info;
+
 /*
  * Common implementation detail of pmu::{start,commit,cancel}_txn
  */
@@ -285,6 +287,9 @@ struct pmu {
 	/* number of address filters this PMU can do */
 	unsigned int			nr_addr_filters;
 
+	/* PMU-specific data to append to the user page */
+	const struct pmu_info		*pmu_info;
+
 	/*
 	 * Fully disable/enable this PMU, can be used to protect from the PMI
 	 * as well as for lazy/batch writing of the MSRs.
@@ -508,6 +513,18 @@ struct perf_addr_filters_head {
 	unsigned int		nr_file_filters;
 };
 
+struct pmu_info {
+	/*
+	 * Size of this structure, for versioning.
+	 */
+	u32	note_size;
+
+	/*
+	 * Size of the container structure, not including this one
+	 */
+	u32	pmu_descsz;
+};
+
 /**
  * enum perf_event_active_state - the states of a event
  */
include/uapi/linux/perf_event.h +10 -0
index 3d64d9ea80..4cdd4fab9d 100644
@@ -599,6 +599,16 @@ struct perf_event_mmap_page {
 	__u64	aux_tail;
 	__u64	aux_offset;
 	__u64	aux_size;
+
+	/*
+	 * PMU data: static info that (AUX) decoder wants to know in order to
+	 * decode correctly:
+	 *
+	 *   pmu_offset >= sizeof(struct perf_event_mmap_page)
+	 *   pmu_offset + pmu_size <= PAGE_SIZE
+	 */
+	__u64	pmu_offset;
+	__u64	pmu_size;
 };
 
 #define PERF_RECORD_MISC_CPUMODE_MASK		(7 << 0)
kernel/events/core.c +1 -26
index fef1f97974..d62ab2d1de 100644
@@ -4926,28 +4926,6 @@ static void calc_timer_values(struct perf_event *event,
 	*running = ctx_time - event->tstamp_running;
 }
 
-static void perf_event_init_userpage(struct perf_event *event)
-{
-	struct perf_event_mmap_page *userpg;
-	struct ring_buffer *rb;
-
-	rcu_read_lock();
-	rb = rcu_dereference(event->rb);
-	if (!rb)
-		goto unlock;
-
-	userpg = rb->user_page;
-
-	/* Allow new userspace to detect that bit 0 is deprecated */
-	userpg->cap_bit0_is_deprecated = 1;
-	userpg->size = offsetof(struct perf_event_mmap_page, __reserved);
-	userpg->data_offset = PAGE_SIZE;
-	userpg->data_size = perf_data_size(rb);
-
-unlock:
-	rcu_read_unlock();
-}
-
 void __weak arch_perf_update_userpage(
 	struct perf_event *event, struct perf_event_mmap_page *userpg, u64 now)
 {
@@ -5385,9 +5363,7 @@ static int perf_mmap(struct file *file, struct vm_area_struct *vma)
 		flags |= RING_BUFFER_WRITABLE;
 
 	if (!rb) {
-		rb = rb_alloc(vma->vm_mm, nr_pages,
-			      event->attr.watermark ? event->attr.wakeup_watermark : 0,
-			      event->cpu, flags);
+		rb = rb_alloc(event, vma->vm_mm, nr_pages, flags);
 
 		if (IS_ERR_OR_NULL(rb)) {
 			ret = PTR_ERR(rb);
@@ -5399,7 +5375,6 @@ static int perf_mmap(struct file *file, struct vm_area_struct *vma)
 
 		ring_buffer_attach(event, rb);
 
-		perf_event_init_userpage(event);
 		perf_event_update_userpage(event);
 	} else {
 		ret = rb_alloc_aux(rb, event, vma->vm_pgoff, nr_pages,
kernel/events/internal.h +1 -1
index 8e267d8faa..4b345ee0d4 100644
@@ -76,7 +76,7 @@ static inline void rb_toggle_paused(struct ring_buffer *rb, bool pause)
 }
 
 extern struct ring_buffer *
-rb_alloc(struct mm_struct *mm, int nr_pages, long watermark, int cpu,
+rb_alloc(struct perf_event *event, struct mm_struct *mm, int nr_pages,
 	 int flags);
 extern void perf_event_wakeup(struct perf_event *event);
 extern int rb_alloc_aux(struct ring_buffer *rb, struct perf_event *event,
kernel/events/ring_buffer.c +63 -12

142 lines not on this page · raw message

Signature · 1 line
2.14.1
maintainer
#15 lore raw

Re: [RFC PATCH 07/17] perf: Add pmu_info to user page

Quoted 40 lines · Alexander Shishkin

Allow PMUs to supply additional static information that may be required by their decoders. Most of what Intel PT driver exports as capability attributes (timing packet freqencies, frequency ratios etc), its decoder needs to be able to correctly decode its binary stream. However, when decoding Intel PT stream from a core dump, we can't rely on the sysfs attributes, so we need to pack this information into the perf buffer, so that the resulting core dump is self-contained.

In order to do this, we append a PMU-specific structure to the user page. Such structures will include size, for versioning.

include/linux/perf_event.h +12 -0
@@ -508,6 +513,18 @@ struct perf_addr_filters_head {
 	unsigned int		nr_file_filters;
 };
 
+struct pmu_info {
+	/*
+	 * Size of this structure, for versioning.
+	 */
+	u32	note_size;
+
+	/*
+	 * Size of the container structure, not including this one
+	 */
+	u32	pmu_descsz;
+};
+
 /**
  * enum perf_event_active_state - the states of a event
  */
include/uapi/linux/perf_event.h +1 -0
index 3d64d9ea80..4cdd4fab9d 100644
@@ -599,6 +599,16 @@ struct perf_event_mmap_page {
 	__u64	aux_tail;
 	__u64	aux_offset;
 	__u64	aux_size;
+

include/uapi/linux/perf_event.h struct perf_event_mmap_page { · line 612

Quoted 10 lines
include/uapi/linux/perf_event.h +9 -0
+	/*
+	 * PMU data: static info that (AUX) decoder wants to know in order to
+	 * decode correctly:
+	 *
+	 *   pmu_offset >= sizeof(struct perf_event_mmap_page)
+	 *   pmu_offset + pmu_size <= PAGE_SIZE
+	 */
+	__u64	pmu_offset;
+	__u64	pmu_size;
 };

Why like this? Why not dump the data as part of
PERF_RECORD_ITRACE_START/PERF_RECORD_AUX ?

author reviewer
#16 lore raw

RFC PATCH 8/17 perf: Allow inheritance for detached events

In reply to Alexander Shishkin

This enables inheritance for detached events. Unlike traditional events, these do not have parents: inheritance produces a new independent event with the same attribute. If the 'parent' event has a ring buffer, so will the new event. Considering the mlock accounting, this buffer allocation may fail, which in turn will fail the parent's fork, something to be aware of.

This also effectively disables context cloning, because unlike the traditional events, these will each have its own ring buffer and context switch optimization can't work.

---
 include/linux/perf_event.h |  1 +
 kernel/events/core.c       | 64 ++++++++++++++++++++++++++++++++++++++--------
 2 files changed, 54 insertions(+), 11 deletions(-)
include/linux/perf_event.h +1 -0
index b7939e8811..0b45abad12 100644
@@ -780,6 +780,7 @@ struct perf_event_context {
 	int				nr_stat;
 	int				nr_freq;
 	int				rotate_disable;
+	int				clone_disable;
 	atomic_t			refcount;
 	struct task_struct		*task;
 
kernel/events/core.c +53 -11

124 lines not on this page · raw message

Signature · 1 line
2.14.1
maintainer
#17 lore raw

Re: [RFC PATCH 08/17] perf: Allow inheritance for detached events

Quoted 10 lines · Alexander Shishkin

This enables inheritance for detached events. Unlike traditional events, these do not have parents: inheritance produces a new independent event with the same attribute. If the 'parent' event has a ring buffer, so will the new event. Considering the mlock accounting, this buffer allocation may fail, which in turn will fail the parent's fork, something to be aware of.

This also effectively disables context cloning, because unlike the traditional events, these will each have its own ring buffer and context switch optimization can't work.

Right, so this thing is icky... as you know. More naming issues though, what will you go and call those files.

author reviewer
#18 lore raw

Re: [RFC PATCH 08/17] perf: Allow inheritance for detached events

Quoted 8 lines · Peter Zijlstra
Quoted 7 lines · Alexander Shishkin

This enables inheritance for detached events. Unlike traditional events, these do not have parents: inheritance produces a new independent event with the same attribute. If the 'parent' event has a ring buffer, so will the new event. Considering the mlock accounting, this buffer allocation may fail, which in turn will fail the parent's fork, something to be aware of.

Quoted 6 lines
Quoted 3 lines

This also effectively disables context cloning, because unlike the traditional events, these will each have its own ring buffer and context switch optimization can't work.

Right, so this thing is icky... as you know. More naming issues though, what will you go and call those files.

Yes. The failing-the-fork ickiness is dealt with later on in 11/17. But true about the naming.

Regards,
--
Alex

author reviewer
#19 lore raw

RFC PATCH 9/17 perf: Use shmemfs pages for userspace-only per-thread detached events

In reply to Alexander Shishkin

In order to work around the problem of using up mlocked memory for the detached events, we can pin the ring buffer pages only while they are in use (that is, the event is ACTIVE), and unpin them for the rest of the time. When not pinned in, these pages can be swapped out. This way, one user can have at most mlock_limit*nr_cpus kB of memory pinned at any given moment, however many events they actually have.

This enforces a constraint: pinning and unpinning may sleep and thus can't be done in the event scheduling path. Instead, we use a task work to do this, which this pattern us to userspace-only events. Also, since one userspace thread only needs one buffer (for whatever CPU it's running on at any given moment), we only do this for per-thread events.

The source for such swappable pages is shmemfs. This patch allows allocating perf ring buffer pages from an shmemfs file if the above constraints are met.

---
 include/linux/perf_event.h  |   1 +
 kernel/events/core.c        |   2 +-
 kernel/events/internal.h    |   8 +-
 kernel/events/ring_buffer.c | 177 +++++++++++++++++++++++++++++++++++++-------
 4 files changed, 160 insertions(+), 28 deletions(-)
include/linux/perf_event.h +1 -0
index 0b45abad12..341e9960bc 100644
@@ -569,6 +569,7 @@ struct swevent_hlist {
 #define PERF_ATTACH_TASK_DATA	0x08
 #define PERF_ATTACH_ITRACE	0x10
 #define PERF_ATTACH_DETACHED	0x20
+#define PERF_ATTACH_SHMEM	0x40
 
 struct perf_cgroup;
 struct ring_buffer;
kernel/events/core.c +1 -1
index 89c14644df..feff812e30 100644
@@ -9419,7 +9419,7 @@ static int perf_event_detach(struct perf_event *event, struct task_struct *task,
 	if (!event->dent)
 		return -ENOMEM;
 
-	err = rb_alloc_detached(event);
+	err = rb_alloc_detached(event, task, mm);
 	if (err) {
 		tracefs_remove(event->dent);
 		event->dent = NULL;
kernel/events/internal.h +7 -1
index 4b345ee0d4..8de9e9cb6a 100644
@@ -7,6 +7,7 @@
 /* Buffer handling */
 
 #define RING_BUFFER_WRITABLE		0x01
+#define RING_BUFFER_SHMEM		0x02
 
 struct ring_buffer {
 	atomic_t			refcount;
@@ -52,6 +53,9 @@ struct ring_buffer {
 	void				**aux_pages;
 	void				*aux_priv;
 
+	/* tmpfs file for kernel-owned ring buffers */
+	struct file			*shmem_file;
+
 	struct perf_event_mmap_page	*user_page;
 	void				*data_pages[0];
 };
@@ -82,7 +86,9 @@ extern void perf_event_wakeup(struct perf_event *event);
 extern int rb_alloc_aux(struct ring_buffer *rb, struct perf_event *event,
 			pgoff_t pgoff, int nr_pages, long watermark, int flags);
 extern void rb_free_aux(struct ring_buffer *rb);
-extern int rb_alloc_detached(struct perf_event *event);
+extern int rb_alloc_detached(struct perf_event *event,
+			     struct task_struct *task,
+			     struct mm_struct *mm);
 extern void rb_free_detached(struct ring_buffer *rb, struct perf_event *event);
 extern struct ring_buffer *ring_buffer_get(struct perf_event *event);
 extern void ring_buffer_put(struct ring_buffer *rb);
kernel/events/ring_buffer.c +151 -26

332 lines not on this page · raw message

Signature · 1 line
2.14.1
maintainer
#20 lore raw

Re: [RFC PATCH 09/17] perf: Use shmemfs pages for userspace-only per-thread detached events

Quoted 14 lines · Alexander Shishkin

In order to work around the problem of using up mlocked memory for the detached events, we can pin the ring buffer pages only while they are in use (that is, the event is ACTIVE), and unpin them for the rest of the time. When not pinned in, these pages can be swapped out. This way, one user can have at most mlock_limit*nr_cpus kB of memory pinned at any given moment, however many events they actually have.

This enforces a constraint: pinning and unpinning may sleep and thus can't be done in the event scheduling path. Instead, we use a task work to do this, which this pattern us to userspace-only events. Also, since one userspace thread only needs one buffer (for whatever CPU it's running on at any given moment), we only do this for per-thread events.

Quoted 3 lines

The source for such swappable pages is shmemfs. This patch allows allocating perf ring buffer pages from an shmemfs file if the above constraints are met.

Right, so why still allow that previous icky thing? What cases do we need that for?

author reviewer
#21 lore raw

Re: [RFC PATCH 09/17] perf: Use shmemfs pages for userspace-only per-thread detached events

Quoted 15 lines · Peter Zijlstra
Quoted 14 lines · Alexander Shishkin

In order to work around the problem of using up mlocked memory for the detached events, we can pin the ring buffer pages only while they are in use (that is, the event is ACTIVE), and unpin them for the rest of the time. When not pinned in, these pages can be swapped out. This way, one user can have at most mlock_limit*nr_cpus kB of memory pinned at any given moment, however many events they actually have.

This enforces a constraint: pinning and unpinning may sleep and thus can't be done in the event scheduling path. Instead, we use a task work to do this, which this pattern us to userspace-only events. Also, since one userspace thread only needs one buffer (for whatever CPU it's running on at any given moment), we only do this for per-thread events.

Quoted 6 lines
Quoted 3 lines

The source for such swappable pages is shmemfs. This patch allows allocating perf ring buffer pages from an shmemfs file if the above constraints are met.

Right, so why still allow that previous icky thing? What cases do we need that for?

8/17..12/17 are really one patch split into smaller chunks. The first one does the icky thing and then we get to what we actually want.

The idea is that you won't be able to enable inheritance for detached events unless they are shmem-backed.

Regards,
--
Alex

author reviewer
#22 lore raw

RFC PATCH 10/17 perf: Implement pinning and scheduling for SHMEM events

In reply to Alexander Shishkin

A SHMEM buffer is only pinned in while its task is scheduled in and the pinning is done in a task work, which also implies that the corresponding event can only be started from that task work.

Pinning is done on a per-cpu basis: if an event has previously been pinned on the local cpu, its unpinned and its pin count is dropped and a new event is pinned on this cpu. When the pin count goes to zero, we unpin the pages, when it goes to one, we pin them.

---
 include/linux/perf_event.h  |  10 +++
 kernel/events/core.c        | 134 ++++++++++++++++++++++++++++-
 kernel/events/internal.h    |   5 ++
 kernel/events/ring_buffer.c | 202 +++++++++++++++++++++++++++++++++++++++++++-
 4 files changed, 347 insertions(+), 4 deletions(-)
include/linux/perf_event.h +10 -0
index 341e9960bc..4b966dd0d8 100644
@@ -703,6 +703,13 @@ struct perf_event {
 	unsigned long			rcu_batches;
 	int				rcu_pending;
 
+	/*
+	 * Number of times (CPUs) this event's been pinned (on):
+	 *  xpinned -> 0: unpin the pages,
+	 *  xpinned -> 1: pin the pages. See get_pages_work().
+	 */
+	atomic_t			xpinned;
+
 	/* poll related */
 	wait_queue_head_t		waitq;
 	struct fasync_struct		*fasync;
@@ -735,6 +742,9 @@ struct perf_event {
 	struct bpf_prog			*prog;
 #endif
 
+	/* Task work to pin event's rb pages if needed */
+	struct callback_head		get_pages_work;
+
 #ifdef CONFIG_EVENT_TRACING
 	struct trace_event_call		*tp_event;
 	struct event_filter		*filter;
kernel/events/core.c +132 -2

194 lines not on this page · raw message

kernel/events/internal.h +5 -0
index 8de9e9cb6a..80d36a7277 100644
@@ -55,11 +55,16 @@ struct ring_buffer {
 
 	/* tmpfs file for kernel-owned ring buffers */
 	struct file			*shmem_file;
+	unsigned long			shmem_file_addr;
+	int				shmem_pages_in;
 
 	struct perf_event_mmap_page	*user_page;
 	void				*data_pages[0];
 };
 
+extern int rb_inject(struct perf_event *event);
+extern int rb_get_kernel_pages(struct perf_event *event);
+extern void rb_put_kernel_pages(struct ring_buffer *rb, bool final);
 extern void rb_free(struct ring_buffer *rb);
 extern void ring_buffer_unaccount(struct ring_buffer *rb, bool aux);
 
kernel/events/ring_buffer.c +200 -2

234 lines not on this page · raw message

Signature · 1 line
2.14.1
author reviewer
#23 lore raw

RFC PATCH 11/17 perf: Implement mlock accounting for shmem ring buffers

In reply to Alexander Shishkin

With shmem ring buffers, one can mlock at most nr_pages * nr_cpus, we only need to do the accounting once the event is created (by means of sys_perf_event_open()). This implements such accounting, by adding a shared reference counter: when it goes 0 -> 1, we account the pages, when it goes to 0, we undo the accounting.

---
 kernel/events/core.c        |  12 +++--
 kernel/events/internal.h    |   5 +-
 kernel/events/ring_buffer.c | 124 +++++++++++++++++++++++++++++++++++++-------
 3 files changed, 116 insertions(+), 25 deletions(-)
kernel/events/core.c +7 -5
index c80ffcdb5c..1fed69d4ba 100644
@@ -4309,7 +4309,6 @@ static void _free_event(struct perf_event *event)
 
 		event->attach_state &= ~PERF_ATTACH_DETACHED;
 
-		ring_buffer_unaccount(event->rb, false);
 		rb_free_detached(event->rb, event);
 	}
 
@@ -9525,9 +9524,11 @@ static void account_event(struct perf_event *event)
 	account_pmu_sb_event(event);
 }
 
-static int perf_event_detach(struct perf_event *event, struct task_struct *task,
-			     struct mm_struct *mm)
+static int
+perf_event_detach(struct perf_event *event, struct perf_event *parent_event,
+		  struct task_struct *task, struct mm_struct *mm)
 {
+	struct ring_buffer *parent_rb = parent_event ? parent_event->rb : NULL;
 	char *filename;
 	int err;
 
@@ -9545,7 +9546,7 @@ static int perf_event_detach(struct perf_event *event, struct task_struct *task,
 	if (!event->dent)
 		return -ENOMEM;
 
-	err = rb_alloc_detached(event, task, mm);
+	err = rb_alloc_detached(event, task, mm, parent_rb);
 	if (err) {
 		tracefs_remove(event->dent);
 		event->dent = NULL;
@@ -11015,7 +11016,8 @@ inherit_event(struct perf_event *parent_event,
 	if (detached) {
 		int err;
 
-		err = perf_event_detach(child_event, child, NULL);
+		err = perf_event_detach(child_event, parent_event, child,
+					NULL);
 		if (err) {
 			perf_free_event(child_event, child_ctx);
 			mutex_unlock(&parent_event->child_mutex);
kernel/events/internal.h +4 -1
index 80d36a7277..3dc66961d9 100644
@@ -53,6 +53,8 @@ struct ring_buffer {
 	void				**aux_pages;
 	void				*aux_priv;
 
+	atomic_t			*acct_refcount;
+
 	/* tmpfs file for kernel-owned ring buffers */
 	struct file			*shmem_file;
 	unsigned long			shmem_file_addr;
@@ -93,7 +95,8 @@ extern int rb_alloc_aux(struct ring_buffer *rb, struct perf_event *event,
 extern void rb_free_aux(struct ring_buffer *rb);
 extern int rb_alloc_detached(struct perf_event *event,
 			     struct task_struct *task,
-			     struct mm_struct *mm);
+			     struct mm_struct *mm,
+			     struct ring_buffer *parent_rb);
 extern void rb_free_detached(struct ring_buffer *rb, struct perf_event *event);
 extern struct ring_buffer *ring_buffer_get(struct perf_event *event);
 extern void ring_buffer_put(struct ring_buffer *rb);
kernel/events/ring_buffer.c +105 -19

204 lines not on this page · raw message

Signature · 1 line
2.14.1
author reviewer
#24 lore raw

RFC PATCH 12/17 perf: Track pinned events per user

In reply to Alexander Shishkin

Maintain a per-user cpu-indexed array of shmemfs-backed events, same way as mlock accounting.

---
 include/linux/sched/user.h  |  6 ++++
 kernel/events/core.c        | 14 ++++-----
 kernel/events/ring_buffer.c | 69 +++++++++++++++++++++++++++++++++++++--------
 kernel/user.c               |  1 +
 4 files changed, 71 insertions(+), 19 deletions(-)
include/linux/sched/user.h +6 -0
index 5d5415e129..bf10f95250 100644
@@ -5,6 +5,7 @@
 #include <linux/atomic.h>
 
 struct key;
+struct perf_event;
 
 /*
  * Some day this will be a full-fledged user tracking system..
@@ -39,6 +40,11 @@ struct user_struct {
 #if defined(CONFIG_PERF_EVENTS) || defined(CONFIG_BPF_SYSCALL)
 	atomic_long_t locked_vm;
 #endif
+#ifdef CONFIG_PERF_EVENTS
+	atomic_long_t nr_pinnable_events;
+	struct mutex pinned_mutex;
+	struct perf_event ** __percpu pinned_events;
+#endif
 };
 
 extern int uids_sysfs_init(void);
kernel/events/core.c +7 -7
index 1fed69d4ba..e00f1f6aaf 100644
@@ -384,7 +384,6 @@ static atomic_t perf_sched_count;
 static DEFINE_PER_CPU(atomic_t, perf_cgroup_events);
 static DEFINE_PER_CPU(int, perf_sched_cb_usages);
 static DEFINE_PER_CPU(struct pmu_event_list, pmu_sb_events);
-static DEFINE_PER_CPU(struct perf_event *, shmem_events);
 
 static atomic_t nr_mmap_events __read_mostly;
 static atomic_t nr_comm_events __read_mostly;
@@ -2086,7 +2085,8 @@ enum pin_event_t {
 
 static enum pin_event_t pin_event_pages(struct perf_event *event)
 {
-	struct perf_event **pinned_event = this_cpu_ptr(&shmem_events);
+	struct user_struct *user = event->rb->mmap_user;
+	struct perf_event **pinned_event = this_cpu_ptr(user->pinned_events);
 	struct perf_event *old_event = *pinned_event;
 
 	if (old_event == event)
@@ -4281,13 +4281,14 @@ static void _free_event(struct perf_event *event)
 	unaccount_event(event);
 
 	if (event->attach_state & PERF_ATTACH_SHMEM) {
+		struct user_struct *user = event->rb->mmap_user;
 		struct perf_event_context *ctx = event->ctx;
 		int cpu;
 
 		atomic_set(&event->xpinned, 0);
 		for_each_possible_cpu(cpu) {
 			struct perf_event **pinned_event =
-				per_cpu_ptr(&shmem_events, cpu);
+				per_cpu_ptr(user->pinned_events, cpu);
 
 			cmpxchg(pinned_event, event, NULL);
 		}
@@ -9530,7 +9531,7 @@ perf_event_detach(struct perf_event *event, struct perf_event *parent_event,
 {
 	struct ring_buffer *parent_rb = parent_event ? parent_event->rb : NULL;
 	char *filename;
-	int err;
+	int err = -ENOMEM;
 
 	filename = kasprintf(GFP_KERNEL, "%s:%x.event",
 			     task ? "task" : "cpu",
@@ -9550,10 +9551,9 @@ perf_event_detach(struct perf_event *event, struct perf_event *parent_event,
 	if (err) {
 		tracefs_remove(event->dent);
 		event->dent = NULL;
-		return err;
 	}
 
-	return 0;
+	return err;
 }
 /*
  * Allocate and initialize a event structure
@@ -10290,7 +10290,7 @@ SYSCALL_DEFINE5(perf_event_open,
 	}
 
 	if (detached) {
-		err = perf_event_detach(event, task, NULL);
+		err = perf_event_detach(event, NULL, task, NULL);
 		if (err)
 			goto err_context;
 
kernel/events/ring_buffer.c +57 -12

129 lines not on this page · raw message

kernel/user.c +1 -0
index 00281add65..e95a82d31d 100644
@@ -185,6 +185,7 @@ struct user_struct *alloc_uid(kuid_t uid)
 
 		new->uid = uid;
 		atomic_set(&new->__count, 1);
+		mutex_init(&new->pinned_mutex);
 
 		/*
 		 * Before adding this, check whether we raced
Signature · 1 line
2.14.1
author reviewer
#25 lore raw

RFC PATCH 13/17 perf: Re-inject shmem buffers after exec

In reply to Alexander Shishkin

A exec will unmap everything, but we want our shmem buffers to persist. This tells the page-pinning task work to mmap event's ring buffer after the task exec'ed.

---
 kernel/events/core.c | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
kernel/events/core.c +24 -0
index e00f1f6aaf..f0b77b33b4 100644
@@ -6482,6 +6482,29 @@ static void perf_event_addr_filters_exec(struct perf_event *event, void *data)
 		perf_event_stop(event, 1);
 }
 
+static void perf_shmem_ctx_exec(struct perf_event_context *ctx)
+{
+	struct perf_event *event;
+	unsigned long flags;
+
+	raw_spin_lock_irqsave(&ctx->lock, flags);
+
+	list_for_each_entry(event, &ctx->event_list, event_entry) {
+		if (event->attach_state & PERF_ATTACH_SHMEM) {
+			struct ring_buffer *rb;
+
+			/* called inside rcu read section */
+			rb = rcu_dereference(event->rb);
+			if (!rb)
+				continue;
+
+			rb->shmem_file_addr = 0;
+		}
+	}
+
+	raw_spin_unlock_irqrestore(&ctx->lock, flags);
+}
+
 void perf_event_exec(void)
 {
 	struct perf_event_context *ctx;
@@ -6497,6 +6520,7 @@ void perf_event_exec(void)
 
 		perf_iterate_ctx(ctx, perf_event_addr_filters_exec, NULL,
 				   true);
+		perf_shmem_ctx_exec(ctx);
 	}
 	rcu_read_unlock();
 }
Signature · 1 line
2.14.1
author reviewer
#26 lore raw

RFC PATCH 14/17 perf: Add ioctl(REATTACH) for detached events

In reply to Alexander Shishkin

This adds an ioctl command to demote a detached event to a 'normal' one that gets destroyed when its file descriptor is closed. It can still be used to mmap the buffers, but not very useful otherwise.

---
 include/uapi/linux/perf_event.h |  1 +
 kernel/events/core.c            | 32 +++++++++++++++++++++++++++++++-
 2 files changed, 32 insertions(+), 1 deletion(-)
include/uapi/linux/perf_event.h +1 -0
index 4cdd4fab9d..ae54bd496d 100644
@@ -435,6 +435,7 @@ struct perf_event_attr {
 #define PERF_EVENT_IOC_ID		_IOR('$', 7, __u64 *)
 #define PERF_EVENT_IOC_SET_BPF		_IOW('$', 8, __u32)
 #define PERF_EVENT_IOC_PAUSE_OUTPUT	_IOW('$', 9, __u32)
+#define PERF_EVENT_IOC_REATTACH		_IO ('$', 10)
 
 enum perf_event_ioc_flags {
 	PERF_IOC_FLAG_GROUP		= 1U << 0,
kernel/events/core.c +31 -1
index f0b77b33b4..fbee221d19 100644
@@ -4532,7 +4532,19 @@ EXPORT_SYMBOL_GPL(perf_event_release_kernel);
  */
 static int perf_release(struct inode *inode, struct file *file)
 {
-	perf_event_release_kernel(file->private_data);
+	struct perf_event *event = file->private_data;
+
+	/*
+	 * For a DETACHED event, perf_release() can't have the last reference,
+	 * because we grabbed one extra in the sys_perf_event_open, IOW it is
+	 * always put_event(). In order for it to be the last reference, we'd
+	 * first need to ioctl(REATTACH) on this event, which would drop the
+	 * PERF_ATTACH_DETACHED attach state.
+	 */
+	if (event->attach_state & PERF_ATTACH_DETACHED)
+		put_event(event);
+	else
+		perf_event_release_kernel(file->private_data);
 	return 0;
 }
 
@@ -4885,6 +4897,11 @@ static long _perf_ioctl(struct perf_event *event, unsigned int cmd, unsigned lon
 	void (*func)(struct perf_event *);
 	u32 flags = arg;
 
+	if (event->attach_state & PERF_ATTACH_DETACHED &&
+	    cmd != PERF_EVENT_IOC_REATTACH &&
+	    cmd != PERF_EVENT_IOC_ID)
+		return -EINVAL;
+
 	switch (cmd) {
 	case PERF_EVENT_IOC_ENABLE:
 		func = _perf_event_enable;
@@ -4948,6 +4965,19 @@ static long _perf_ioctl(struct perf_event *event, unsigned int cmd, unsigned lon
 		rcu_read_unlock();
 		return 0;
 	}
+	case PERF_EVENT_IOC_REATTACH:
+		/*
+		 * DETACHED state is serialized on ctx::mutex
+		 */
+		if (!is_detached_event(event))
+			return -EINVAL;
+
+		event->attach_state &= ~PERF_ATTACH_DETACHED;
+		tracefs_remove(event->dent);
+		event->dent = NULL;
+		put_event(event); /* can't be last */
+
+		return 0;
 	default:
 		return -ENOTTY;
 	}
Signature · 1 line
2.14.1
maintainer
#27 lore raw

Re: [RFC PATCH 14/17] perf: Add ioctl(REATTACH) for detached events

Quoted 3 lines · Alexander Shishkin

This adds an ioctl command to demote a detached event to a 'normal' one that gets destroyed when its file descriptor is closed. It can still be used to mmap the buffers, but not very useful otherwise.

why not simply use the fd obtained from open() on our special filesystem?

If you open and then unlink, you loose the 'detached' state and the filedesc is the only life-line.

author reviewer
#28 lore raw

RFC PATCH 15/17 perf: Allow controlled non-root access to detached events

In reply to Alexander Shishkin

The user who created the event should also be able to open its corresponding file in tracefs and/or remove it.

---
 kernel/events/core.c | 25 ++++++++++++++++++++++---
 1 file changed, 22 insertions(+), 3 deletions(-)
kernel/events/core.c +22 -3
index fbee221d19..802c0862a9 100644
@@ -5579,7 +5579,7 @@ static int perf_fasync(int fd, struct file *filp, int on)
 static int perf_open(struct inode *inode, struct file *file)
 {
 	struct perf_event *event = inode->i_private;
-	int ret;
+	int ret = 0;
 
 	if (WARN_ON_ONCE(!event))
 		return -EINVAL;
@@ -5587,7 +5587,13 @@ static int perf_open(struct inode *inode, struct file *file)
 	if (!atomic_long_inc_not_zero(&event->refcount))
 		return -ENOENT;
 
-	ret = simple_open(inode, file);
+	/* event's user is stable while we're holding the reference */
+	if (event->rb->mmap_user != current_user() &&
+	    !capable(CAP_SYS_ADMIN))
+		ret = -EACCES;
+
+	if (!ret)
+		ret = simple_open(inode, file);
 	if (ret)
 		put_event(event);
 
@@ -9593,7 +9599,7 @@ perf_event_detach(struct perf_event *event, struct perf_event *parent_event,
 	if (!filename)
 		return -ENOMEM;
 
-	event->dent = tracefs_create_file(filename, 0600,
+	event->dent = tracefs_create_file(filename, 0666,
 					  perf_tracefs_dir,
 					  event, &perf_fops);
 	kfree(filename);
@@ -11521,6 +11527,7 @@ static int perf_instance_unlink(const char *name)
 {
 	struct perf_event *event;
 	struct dentry *dent;
+	int ret = 0;
 
 	dent = lookup_one_len_unlocked(name, perf_tracefs_dir, strlen(name));
 	if (!dent)
@@ -11530,6 +11537,18 @@ static int perf_instance_unlink(const char *name)
 	if (!event)
 		return -EINVAL;
 
+	if (!atomic_long_inc_not_zero(&event->refcount))
+		return 0;
+
+	/* event's user is stable while we're holding the reference */
+	if (event->rb->mmap_user != current_user() &&
+	    !capable(CAP_SYS_ADMIN))
+		ret = -EACCES;
+	put_event(event);
+
+	if (ret)
+		return ret;
+
 	if (!(event->attach_state & PERF_ATTACH_CONTEXT))
 		return -EBUSY;
 
Signature · 1 line
2.14.1
maintainer
#29 lore raw

Re: [RFC PATCH 15/17] perf: Allow controlled non-root access to detached events

Quoted 25 lines · Alexander Shishkin
kernel/events/core.c +12 -1
@@ -5587,7 +5587,13 @@ static int perf_open(struct inode *inode, struct file *file)
 	if (!atomic_long_inc_not_zero(&event->refcount))
 		return -ENOENT;
 
-	ret = simple_open(inode, file);
+	/* event's user is stable while we're holding the reference */
+	if (event->rb->mmap_user != current_user() &&
+	    !capable(CAP_SYS_ADMIN))
+		ret = -EACCES;
+
+	if (!ret)
+		ret = simple_open(inode, file);
 	if (ret)
 		put_event(event);
 
@@ -11530,6 +11537,18 @@ static int perf_instance_unlink(const char *name)
 	if (!event)
 		return -EINVAL;
 
+	if (!atomic_long_inc_not_zero(&event->refcount))
+		return 0;
+
+	/* event's user is stable while we're holding the reference */
+	if (event->rb->mmap_user != current_user() &&

kernel/events/core.c static int perf_instance_unlink(const char *name) · line 11554

Quoted 10 lines
kernel/events/core.c +7 -0
+	    !capable(CAP_SYS_ADMIN))
+		ret = -EACCES;
+	put_event(event);
+
+	if (ret)
+		return ret;
+
 	if (!(event->attach_state & PERF_ATTACH_CONTEXT))
 		return -EBUSY;
 

Why aren't we using regular file permissions for this?

author reviewer
#30 lore raw

RFC PATCH 16/17 perf/x86/intel/pt: Add PMU info

In reply to Alexander Shishkin

Add PMU-specific data structure with family/model/stepping and clock information required by the decoder.

---
 arch/x86/events/intel/pt.c | 23 ++++++++++++++++++++++-
 arch/x86/events/intel/pt.h | 11 +++++++++++
 2 files changed, 33 insertions(+), 1 deletion(-)
arch/x86/events/intel/pt.c +22 -1
index 3b993942a0..053b96f491 100644
@@ -35,6 +35,8 @@
 
 static DEFINE_PER_CPU(struct pt, pt_ctx);
 
+#define PMU_NAME "intel_pt"
+
 static struct pt_pmu pt_pmu;
 
 /*
@@ -271,6 +273,22 @@ static int __init pt_pmu_hw_init(void)
 	return ret;
 }
 
+static struct intel_pt_pmu_info pt_pmu_info;
+
+static void pt_pmu_info_setup(void)
+{
+	BUILD_BUG_ON(sizeof(pt_pmu_info) +
+	             sizeof(struct perf_event_mmap_page) > PAGE_SIZE);
+	pt_pmu_info.pi.note_size  = sizeof(pt_pmu_info.pi);
+	pt_pmu_info.pi.pmu_descsz = sizeof(pt_pmu_info) - pt_pmu_info.pi.note_size;
+	pt_pmu_info.x86_family    = boot_cpu_data.x86;
+	pt_pmu_info.x86_model     = boot_cpu_data.x86_model;
+	pt_pmu_info.x86_step      = boot_cpu_data.x86_mask;
+	pt_pmu_info.x86_tsc_max_nonturbo_ratio = pt_pmu.max_nonturbo_ratio;
+	pt_pmu_info.x86_tsc_to_art_numerator   = pt_pmu.tsc_art_num;
+	pt_pmu_info.x86_tsc_to_art_denominator = pt_pmu.tsc_art_den;
+}
+
 #define RTIT_CTL_CYC_PSB (RTIT_CTL_CYCLEACC	| \
 			  RTIT_CTL_CYC_THRESH	| \
 			  RTIT_CTL_PSB_FREQ)
@@ -1512,6 +1530,8 @@ static __init int pt_init(void)
 		return -ENODEV;
 	}
 
+	pt_pmu_info_setup();
+
 	if (!pt_cap_get(PT_CAP_topa_multiple_entries))
 		pt_pmu.pmu.capabilities =
 			PERF_PMU_CAP_AUX_NO_SG | PERF_PMU_CAP_AUX_SW_DOUBLEBUF;
@@ -1531,8 +1551,9 @@ static __init int pt_init(void)
 	pt_pmu.pmu.addr_filters_validate = pt_event_addr_filters_validate;
 	pt_pmu.pmu.nr_addr_filters       =
 		pt_cap_get(PT_CAP_num_address_ranges);
+	pt_pmu.pmu.pmu_info		 = &pt_pmu_info.pi;
 
-	ret = perf_pmu_register(&pt_pmu.pmu, "intel_pt", -1);
+	ret = perf_pmu_register(&pt_pmu.pmu, PMU_NAME, -1);
 
 	return ret;
 }
arch/x86/events/intel/pt.h +11 -0
index 25fa9710f4..fc19080ca3 100644
@@ -189,4 +189,15 @@ struct pt {
 	int			vmx_on;
 };
 
+struct intel_pt_pmu_info {
+	struct pmu_info		pi;
+	u8			x86_family;
+	u8			x86_model;
+	u8			x86_step;
+	u8			x86_tsc_max_nonturbo_ratio;
+	u32			x86_tsc_to_art_numerator;
+	u32			x86_tsc_to_art_denominator;
+	u32			__reserved_0;
+};
+
 #endif /* __INTEL_PT_H__ */
Signature · 1 line
2.14.1
author reviewer
#31 lore raw

RFC PATCH 17/17 perf/x86/intel/bts: Add PMU info

In reply to Alexander Shishkin
---
 arch/x86/events/intel/bts.c | 20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)
arch/x86/events/intel/bts.c +18 -2
index 16076eb346..ce1dac7115 100644
@@ -21,13 +21,22 @@
 #include <linux/slab.h>
 #include <linux/debugfs.h>
 #include <linux/device.h>
-#include <linux/coredump.h>
 
 #include <asm-generic/sizes.h>
 #include <asm/perf_event.h>
 
 #include "../perf_event.h"
 
+#define PMU_NAME "intel_bts"
+
+static struct intel_bts_pmu_info {
+	struct pmu_info		pi;
+	u8			x86_family;
+	u8			x86_model;
+	u8			x86_step;
+	u8			__reserved_0[5];
+} bts_pmu_info;
+
 struct bts_ctx {
 	struct perf_output_handle	handle;
 	struct debug_store		ds_back;
@@ -582,6 +591,12 @@ static __init int bts_init(void)
 	if (!boot_cpu_has(X86_FEATURE_DTES64) || !x86_pmu.bts)
 		return -ENODEV;
 
+	bts_pmu_info.pi.note_size  = sizeof(bts_pmu_info.pi);
+	bts_pmu_info.pi.pmu_descsz = sizeof(bts_pmu_info) - bts_pmu_info.pi.note_size;
+	bts_pmu_info.x86_family    = boot_cpu_data.x86;
+	bts_pmu_info.x86_model	    = boot_cpu_data.x86_model;
+	bts_pmu_info.x86_step	    = boot_cpu_data.x86_mask;
+
 	bts_pmu.capabilities	= PERF_PMU_CAP_AUX_NO_SG | PERF_PMU_CAP_ITRACE |
 				  PERF_PMU_CAP_EXCLUSIVE;
 	bts_pmu.task_ctx_nr	= perf_sw_context;
@@ -593,7 +608,8 @@ static __init int bts_init(void)
 	bts_pmu.read		= bts_event_read;
 	bts_pmu.setup_aux	= bts_buffer_setup_aux;
 	bts_pmu.free_aux	= bts_buffer_free_aux;
+	bts_pmu.pmu_info	= &bts_pmu_info.pi;
 
-	return perf_pmu_register(&bts_pmu, "intel_bts", -1);
+	return perf_pmu_register(&bts_pmu, PMU_NAME, -1);
 }
 arch_initcall(bts_init);
Signature · 1 line
2.14.1
maintainer
#32 lore raw
In reply to Alexander Shishkin
Quoted 4 lines · Alexander Shishkin

Detached events: a new flag to the perf syscall makes a 'detached' event, which exists after its file descriptor is released. Not all detached events are per-thread AUX events: this tries to take into account the need for system-wide persistent events too.

Nice, thanks!

Quoted 4 lines

(2) Need to be able to kill those events, so they need to be accessible after they are created.
Event files: detached events exist as files in tracefs (at the moment), can be opened/mmaped/read/removed.

I guess I'll see when I continue reading but I remember us doing ioctls on the event fd.

Quoted 5 lines

(6) Ring buffer memory accounting needs to take this new arrangement into
account: one user can use up at most NR_CPUS * buffer_size memory at any given point in time.
Only account the first such event and undo the accounting when the last event is gone.

... and I guess we probably shouldn't allow the user to create too many events and shoot herself in the OOM-foot.

Quoted 10 lines

(7) We'll also need to supply all the things that the [PT] decoder normally finds out via sysfs attributes, like clock ratios, capabilities, etc so that it also finds its way into the core dump file.
"PMU info" structure is appended to the user page.

I've also hack the perf tool to support all this, all these things can be found at [1]. I'm not posting the tooling patches though, them being thoroughly ugly and proof-of-concept. In short, perf record will create detached events with '--detached' and afterwards will open detached events via their path in tracefs.

Sounds nice. I'd need to test all that just so I can be able to create detached RAS events (which are tracepoints) with it.

Thanks.

Signature · 4 lines
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.
author reviewer
#33 lore raw
Quoted 7 lines · Borislav Petkov
Quoted 4 lines · Alexander Shishkin

Detached events: a new flag to the perf syscall makes a 'detached' event, which exists after its file descriptor is released. Not all detached events are per-thread AUX events: this tries to take into account the need for system-wide persistent events too.

Nice, thanks!

Forgot to mention that I did hack the tracepoint support into the tooling as well to make sure it's a workable idea.

Quoted 7 lines
Quoted 4 lines

(2) Need to be able to kill those events, so they need to be accessible after they are created.
Event files: detached events exist as files in tracefs (at the moment), can be opened/mmaped/read/removed.

I guess I'll see when I continue reading but I remember us doing ioctls on the event fd.

Iirc that was for re-attaching to the event to make it 'normal' before closing.

Quoted 8 lines
Quoted 5 lines

(6) Ring buffer memory accounting needs to take this new arrangement into
account: one user can use up at most NR_CPUS * buffer_size memory at any given point in time.
Only account the first such event and undo the accounting when the last event is gone.

... and I guess we probably shouldn't allow the user to create too many events and shoot herself in the OOM-foot.

Well, they are still limited by the RLIMIT_MEMLOCK and perf_event_mlock sysctl for the total amount of memory that can be pinned for the ring buffers at any given time, so that should be fine.

Quoted 5 lines
Quoted 5 lines

(7) We'll also need to supply all the things that the [PT] decoder normally finds out via sysfs attributes, like clock ratios, capabilities, etc so that it also finds its way into the core dump file.
"PMU info" structure is appended to the user page.

Quoted 8 lines
Quoted 5 lines

I've also hack the perf tool to support all this, all these things can be found at [1]. I'm not posting the tooling patches though, them being thoroughly ugly and proof-of-concept. In short, perf record will create detached events with '--detached' and afterwards will open detached events via their path in tracefs.

Sounds nice. I'd need to test all that just so I can be able to create detached RAS events (which are tracepoints) with it.

Thanks, let me know how it goes.

Regards,
--
Alex