Skip to contentSkip to conversation

perf/hw_breakpoint: Fix breakpoint overcommit issue

PATCH In review Frederic Weisbecker 2 participants 9 messages Jul 9, 2019 – Jul 16, 2019

Subsystems (23)
lore
Share & tools
lore
#0 lore raw

cover perf/hw_breakpoint: Fix breakpoint overcommit issue

Syzbot has found a breakpoint overcommit issue:

https://lore.kernel.org/lkml/[email protected]/

It took me a long time to find out what the actual root problem was. Also its reproducer only worked on a few month old kernel but it didn't feel like the issue was actually solved.

I eventually cooked a reproducer that works with latest upstream, see in the end of this message.

The fix is just a few liner but implies to shut down the context swapping optimization for contexts containing breakpoints.

Also I feel like uprobes may be concerned as well as it seems to make use of event.hw->target after pmu::init().

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
perf/pin

HEAD: 35b749650cc72402ae47beb5e0048c36636a4002

Thanks,
Frederic
---

Frederic Weisbecker (2):
perf: Allow a pmu to pin context perf/hw_breakpoints: Pin perf contexts of breakpoints

 include/linux/perf_event.h    | 2 ++
 kernel/events/core.c          | 6 ++++++
 kernel/events/hw_breakpoint.c | 1 +
 3 files changed, 9 insertions(+)

---

#define _GNU_SOURCE
#include <linux/perf_event.h>
#include <linux/hw_breakpoint.h>
#include <unistd.h>
#include <sys/syscall.h>
#include <stdio.h>
#include <sys/types.h>
#include <fcntl.h>
#include <sys/wait.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>

static struct perf_event_attr attr = {
	.type = PERF_TYPE_BREAKPOINT,
	.size = sizeof(attr),
	.config = 0,
	.sample_period = 1,
	.sample_type = PERF_SAMPLE_IP,
	.read_format = PERF_FORMAT_ID,
	.inherit = 1,
	.pinned = 1,
	.wakeup_events = 1,
	.bp_type = HW_BREAKPOINT_W,
	.bp_addr = 0,
	.bp_len = 8,
};

static void loop(int secs)

{
	struct timespec tp;
	int start;

	clock_gettime(CLOCK_MONOTONIC, &tp);
	start = tp.tv_sec;

	for (;;) {
		clock_gettime(CLOCK_MONOTONIC, &tp);
		if (tp.tv_sec - start >= secs)
			return;
	}
}

int main(int argc, char **argv)

{
	int fd, i, status;
	pid_t child1, child2;

	for (i = 0; i < 4; i++) {
		fd = syscall(__NR_perf_event_open, &attr, 0, -1, -1, 0);
		if (fd < 0)
			perror("perf_event_open");
	}

	child1 = fork();
	if (child1 == 0) {
		loop(1);
		return 0;
	}

	child2 = fork();
	if (child2 == 0) {
		loop(2);
		fd = syscall(__NR_perf_event_open, &attr, 0, -1, -1, 0);
		if (fd < 0)
			perror("perf_event_open");

		return 0;
	}

	return 0;
}
#1 lore raw

PATCH 1/2 perf: Allow a pmu to pin context

Some PMUs rely on the fact that struct perf_event::hw::target always point to the actual event target during the whole event lifecycle, ie:
from pmu::init() to event::destroy().

Now perf event ctx swapping on task sched switch breaks that guarantee.

Solve that with providing a way for a PMU to pin the context of an event.

Reported-by [email protected]
Cc Borislav Petkov <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Masami Hiramatsu <[email protected]>
---
 include/linux/perf_event.h | 2 ++
 kernel/events/core.c       | 6 ++++++
 2 files changed, 8 insertions(+)
include/linux/perf_event.h +2 -0
index 2ddae518dce6..ea700a193865 100644
@@ -273,6 +273,8 @@ struct pmu {
 	/* number of address filters this PMU can do */
 	unsigned int			nr_addr_filters;
 
+	int				pin_ctx;
+
 	/*
 	 * Fully disable/enable this PMU, can be used to protect from the PMI
 	 * as well as for lazy/batch writing of the MSRs.
kernel/events/core.c +6 -0
index 23efe6792abc..6b4bc6fc47aa 100644
@@ -1689,6 +1689,9 @@ list_add_event(struct perf_event *event, struct perf_event_context *ctx)
 		ctx->nr_stat++;
 
 	ctx->generation++;
+
+	if (event->pmu->pin_ctx)
+		ctx->pin_count++;
 }
 
 /*
@@ -1885,6 +1888,9 @@ list_del_event(struct perf_event *event, struct perf_event_context *ctx)
 		perf_event_set_state(event, PERF_EVENT_STATE_OFF);
 
 	ctx->generation++;
+
+	if (event->pmu->pin_ctx)
+		ctx->pin_count--;
 }
 
 static void perf_group_detach(struct perf_event *event)
Signature · 1 line
2.21.0
#2 lore raw

PATCH 2/2 perf/hw_breakpoints: Pin perf contexts of breakpoints

In reply to Frederic Weisbecker

The breakpoints core deals with limited architecture resources that need to be shared among all the breakpoint perf events. For example x86 has 4 breakpoint registers per-CPU. This means that a task can't open more than 4 breakpoints in this architecture.

To control that we must track the number of breakpoints held for each task. This is simply implemented with a global list of all breakpoints that we walk and count on top of their target task.

But this wrongly assume that the target task of a perf event remains stable after the event creation. Context tasks can get swapped on context switches and it's fairly common for a perf event's initial target (event->hw.target) to become irrelevantly out of date at struct perf_event::destroy() time.

Here is an interesting scenario where it matters: TASK A and B are clones that share 4 equivalent breakpoints. A and B contexts are swapped on context switch. So task A now carries the breakpoints that were initialized in B. Task A quits and release its breakpoints whose target (event->hw.target) still point to B. Therefore the breakpoints slots are spuriously released on B. Afterward, if task B were to create a 5th breakpoint then it would be accepted. But on architecture breakpoint installation time, that 5th breakpoint will be eventually rejected:

	Can't find any breakpoint slot
	 WARNING: CPU: 6 PID: 2433 at arch/x86/kernel/hw_breakpoint.c:109 arch_install_hw_breakpoint+0x176/0x180
	 Modules linked in:
	 CPU: 6 PID: 2433 Comm: bp5 Not tainted 5.2.0-rc6+ #20
	 Hardware name: MSI MS-7850/Z87-G41 PC Mate(MS-7850), BIOS V1.3 08/18/2013
	 RIP: 0010:arch_install_hw_breakpoint+0x176/0x180
	 Code: 57 ff ff ff 0f 23 c8 e9 4f ff ff ff 0f 23 d0 e9 47 ff ff ff 48 c7 c7 d8 cd 2b 82 89 45 d0 c6 05 d4 bc 4f 01 01 e8 0a b8 09 00 <0f> 0b 8b 45 d0 e9 ed fe ff ff 0f 1f 44 00 00 55 48 89 e5 41 57 41
	 RSP: 0018:ffffc900006dfa48 EFLAGS: 00010082
	 RAX: 0000000000000000 RBX: ffff88821fb17f78 RCX: 0000000000000000
	 RDX: ffffffff81148b59 RSI: 0000000000000001 RDI: ffffffff81148bb5
	 RBP: ffffc900006dfa78 R08: 0000000000000000 R09: 0000000000000000
	 R10: ffffc900006dfaa0 R11: 0000000000000981 R12: 0000000000000003
	 R13: ffff88821206b800 R14: 0000000000017f80 R15: 0000000000000004
	 FS:  00007f46d1a54440(0000) GS:ffff88821fb00000(0000) knlGS:0000000000000000
	 CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
	 CR2: 00007f46d1590450 CR3: 000000020f1c8004 CR4: 00000000001706e0
	 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
	 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 00000000999906aa
	 Call Trace:
	  hw_breakpoint_add+0x44/0x50
	  event_sched_in.isra.119+0xab/0x240
	  group_sched_in+0x60/0x150
	  pinned_sched_in+0x85/0x170
	  ? group_sched_in+0x150/0x150
	  visit_groups_merge+0x10a/0x180
	  ctx_sched_in.isra.89+0x12b/0x150
	  perf_event_sched_in.isra.91+0x2f/0x70
	  ctx_resched+0x54/0x90
	  __perf_install_in_context+0x144/0x170
	  ? perf_duration_warn+0x40/0x40
	  remote_function+0x4a/0x60
	  generic_exec_single+0xad/0x110
	  ? perf_duration_warn+0x40/0x40
	  smp_call_function_single+0x9c/0x150
	  ? perf_duration_warn+0x40/0x40
	  task_function_call+0x4b/0x80
	  ? task_function_call+0x4b/0x80
	  ? __perf_event_enable+0x140/0x140
	  perf_install_in_context+0x99/0x160
	  __do_sys_perf_event_open+0xaf5/0xd80
	  __x64_sys_perf_event_open+0x20/0x30
	  do_syscall_64+0x4f/0x1b0
	  entry_SYSCALL_64_after_hwframe+0x49/0xbe
	 RIP: 0033:0x7f46d1590469
	 Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d ff 49 2b 00 f7 d8 64 89 01 48
	 RSP: 002b:00007ffc86f0f488 EFLAGS: 00000246 ORIG_RAX: 000000000000012a
	 RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f46d1590469
	 RDX: 00000000ffffffff RSI: 0000000000000000 RDI: 0000564f0c3a0080
	 RBP: 00007ffc86f0f4b0 R08: 0000000000000000 R09: 00007ffc86f0f598
	 R10: 00000000ffffffff R11: 0000000000000246 R12: 0000564f0c19f7e0
	 R13: 00007ffc86f0f590 R14: 0000000000000000 R15: 0000000000000000
	 irq event stamp: 12252
	 hardirqs last  enabled at (12251): [<ffffffff81282909>] kmem_cache_alloc+0xb9/0x6e0
	 hardirqs last disabled at (12252): [<ffffffff81184637>] generic_exec_single+0xa7/0x110
	 softirqs last  enabled at (12234): [<ffffffff81e003e7>] __do_softirq+0x3e7/0x497
	 softirqs last disabled at (12227): [<ffffffff810d4558>] irq_exit+0xc8/0xd0

We can't easily support dynamic event context swap when breakpoints are involved. The easiest solution to fix the issue is to pin a context as long as it contains a breakpoint PMU.

Reported-by [email protected]
Cc Borislav Petkov <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Masami Hiramatsu <[email protected]>
---
 kernel/events/hw_breakpoint.c | 1 +
 1 file changed, 1 insertion(+)
kernel/events/hw_breakpoint.c +1 -0
index c5cd852fe86b..514d704d60a2 100644
@@ -656,6 +656,7 @@ static struct pmu perf_breakpoint = {
 	.start		= hw_breakpoint_start,
 	.stop		= hw_breakpoint_stop,
 	.read		= hw_breakpoint_pmu_read,
+	.pin_ctx	= 1,
 };
 
 int __init init_hw_breakpoint(void)
Signature · 1 line
2.21.0
maintainer
#3 lore raw
In reply to Frederic Weisbecker
Quoted 9 lines · Frederic Weisbecker

Syzbot has found a breakpoint overcommit issue:

https://lore.kernel.org/lkml/[email protected]/

It took me a long time to find out what the actual root problem was. Also its reproducer only worked on a few month old kernel but it didn't feel like the issue was actually solved.

Quoted 8 lines

I eventually cooked a reproducer that works with latest upstream, see in the end of this message.

The fix is just a few liner but implies to shut down the context swapping optimization for contexts containing breakpoints.

Also I feel like uprobes may be concerned as well as it seems to make use of event.hw->target after pmu::init().

Can't we simply swizzle event.hw->target along too?

#4 lore raw
Quoted 13 lines · Peter Zijlstra
Quoted 12 lines · Frederic Weisbecker

Syzbot has found a breakpoint overcommit issue:

https://lore.kernel.org/lkml/[email protected]/

It took me a long time to find out what the actual root problem was. Also its reproducer only worked on a few month old kernel but it didn't feel like the issue was actually solved.

I eventually cooked a reproducer that works with latest upstream, see in the end of this message.

Quoted 7 lines
Quoted 5 lines

The fix is just a few liner but implies to shut down the context swapping optimization for contexts containing breakpoints.

Also I feel like uprobes may be concerned as well as it seems to make use of event.hw->target after pmu::init().

Can't we simply swizzle event.hw->target along too?

You mean remove it? But it's still needed by breakpoint code during all the event lifecycle (init, destroy and anytime in-between).

I wish we could use event->ctx->task instead but on pmu::init() there is no ctx yet (we could pass the task in parameter though) and on event->destroy() it's TASK_TOMBSTONE and retrieving the task at that time would be non trivial.

maintainer
#5 lore raw
Quoted 17 lines · Frederic Weisbecker
Quoted 16 lines · Peter Zijlstra
Quoted 15 lines · Frederic Weisbecker

Syzbot has found a breakpoint overcommit issue:

https://lore.kernel.org/lkml/[email protected]/

It took me a long time to find out what the actual root problem was. Also its reproducer only worked on a few month old kernel but it didn't feel like the issue was actually solved.

I eventually cooked a reproducer that works with latest upstream, see in the end of this message.

The fix is just a few liner but implies to shut down the context swapping optimization for contexts containing breakpoints.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

Also I feel like uprobes may be concerned as well as it seems to make use of event.hw->target after pmu::init().

Can't we simply swizzle event.hw->target along too?

You mean remove it? But it's still needed by breakpoint code during all the event lifecycle (init, destroy and anytime in-between).

No, I meant flip hw->target when we flip the context. It would mean iterating the events, which I suppose would suck.

Quoted 2 lines

I wish we could use event->ctx->task instead but on pmu::init() there is no ctx yet (we could pass the task in parameter though)

Right, that should be fairly easy.

Quoted 2 lines

and on event->destroy() it's TASK_TOMBSTONE and retrieving the task at that time would be non trivial.

Well, right, we can maybe make TOMBSTONE be the LSB instead of the whole word, then we can recover the task pointer... *yuck* though.

maintainer
#6 lore raw
Quoted 10 lines · Peter Zijlstra
Quoted 2 lines

I wish we could use event->ctx->task instead but on pmu::init() there is no ctx yet (we could pass the task in parameter though)

Right, that should be fairly easy.

Quoted 2 lines

and on event->destroy() it's TASK_TOMBSTONE and retrieving the task at that time would be non trivial.

Well, right, we can maybe make TOMBSTONE be the LSB instead of the whole word, then we can recover the task pointer... *yuck* though.

Something like the attached, completely untested patches.

I didn't do the hw_breakpoint bit, because I got lost in that, but this basically provides what you asked for I think.

Subject: perf: Clarify perf_event_exit_task_context()
From: Peter Zijlstra <[email protected]>
Date: Mon Jul 15 13:39:03 CEST 2019

The variables in perf_event_exit_task_context() are all called 'child' even though it must be current, this is confusing, apply:

s/child/task/

---
 kernel/events/core.c |   38 +++++++++++++++++++-------------------
 1 file changed, 19 insertions(+), 19 deletions(-)
kernel/events/core.c +19 -19
@@ -11434,15 +11434,15 @@ perf_event_exit_event(struct perf_event
 	put_event(parent_event);
 }
 
-static void perf_event_exit_task_context(struct task_struct *child, int ctxn)
+static void perf_event_exit_task_context(struct task_struct *task, int ctxn)
 {
-	struct perf_event_context *child_ctx, *clone_ctx = NULL;
-	struct perf_event *child_event, *next;
+	struct perf_event_context *task_ctx, *clone_ctx = NULL;
+	struct perf_event *task_event, *next;
 
-	WARN_ON_ONCE(child != current);
+	WARN_ON_ONCE(task != current);
 
-	child_ctx = perf_pin_task_context(child, ctxn);
-	if (!child_ctx)
+	task_ctx = perf_pin_task_context(task, ctxn);
+	if (!task_ctx)
 		return;
 
 	/*
@@ -11455,27 +11455,27 @@ static void perf_event_exit_task_context
 	 * without ctx::mutex (it cannot because of the move_group double mutex
 	 * lock thing). See the comments in perf_install_in_context().
 	 */
-	mutex_lock(&child_ctx->mutex);
+	mutex_lock(&task_ctx->mutex);
 
 	/*
 	 * In a single ctx::lock section, de-schedule the events and detach the
 	 * context from the task such that we cannot ever get it scheduled back
 	 * in.
 	 */
-	raw_spin_lock_irq(&child_ctx->lock);
-	task_ctx_sched_out(__get_cpu_context(child_ctx), child_ctx, EVENT_ALL);
+	raw_spin_lock_irq(&task_ctx->lock);
+	task_ctx_sched_out(__get_cpu_context(task_ctx), task_ctx, EVENT_ALL);
 
 	/*
 	 * Now that the context is inactive, destroy the task <-> ctx relation
 	 * and mark the context dead.
 	 */
-	RCU_INIT_POINTER(child->perf_event_ctxp[ctxn], NULL);
-	put_ctx(child_ctx); /* cannot be last */
-	WRITE_ONCE(child_ctx->task, TASK_TOMBSTONE);
+	RCU_INIT_POINTER(task->perf_event_ctxp[ctxn], NULL);
+	put_ctx(task_ctx); /* cannot be last */
+	WRITE_ONCE(task_ctx->task, TASK_TOMBSTONE);
 	put_task_struct(current); /* cannot be last */
 
-	clone_ctx = unclone_ctx(child_ctx);
-	raw_spin_unlock_irq(&child_ctx->lock);
+	clone_ctx = unclone_ctx(task_ctx);
+	raw_spin_unlock_irq(&task_ctx->lock);
 
 	if (clone_ctx)
 		put_ctx(clone_ctx);
@@ -11485,14 +11485,14 @@ static void perf_event_exit_task_context
 	 * won't get any samples after PERF_RECORD_EXIT. We can however still
 	 * get a few PERF_RECORD_READ events.
 	 */
-	perf_event_task(child, child_ctx, 0);
+	perf_event_task(task, task_ctx, 0);
 
-	list_for_each_entry_safe(child_event, next, &child_ctx->event_list, event_entry)
-		perf_event_exit_event(child_event, child_ctx, child);
+	list_for_each_entry_safe(task_event, next, &task_ctx->event_list, event_entry)
+		perf_event_exit_event(task_event, task_ctx, task);
 
-	mutex_unlock(&child_ctx->mutex);
+	mutex_unlock(&task_ctx->mutex);
 
-	put_ctx(child_ctx);
+	put_ctx(task_ctx);
 }
 
 /*

Subject: perf: Remove overload of TASK_TOMBSTONE
From: Peter Zijlstra <[email protected]>
Date: Mon Jul 15 13:40:02 CEST 2019

Currently TASK_TOMBSTONE is used for 2 different things.

---
 kernel/events/core.c |    5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
kernel/events/core.c +3 -2
@@ -165,10 +165,11 @@ static void perf_ctx_unlock(struct perf_
 }
 
 #define TASK_TOMBSTONE ((void *)-1L)
+#define TASK_KERNEL ((void *)-1L)
 
 static bool is_kernel_event(struct perf_event *event)
 {
-	return READ_ONCE(event->owner) == TASK_TOMBSTONE;
+	return READ_ONCE(event->owner) == TASK_KERNEL;
 }
 
 /*
@@ -11239,7 +11240,7 @@ perf_event_create_kernel_counter(struct
 	}
 
 	/* Mark owner so we could distinguish it from user events. */
-	event->owner = TASK_TOMBSTONE;
+	event->owner = TASK_KERNEL;
 
 	ctx = find_get_context(event->pmu, task, event);
 	if (IS_ERR(ctx)) {

Subject: perf: Remove TASK_TOMBSTONG
From: Peter Zijlstra <[email protected]>
Date: Mon Jul 15 13:42:35 CEST 2019

Instead of overwriting the entirely of ctx->task, only set the LSB to mark the ctx dead. This allows recovering the task pointer for fun and games later on.

---
 kernel/events/core.c |   51 ++++++++++++++++++++++++++++++++-------------------
 1 file changed, 32 insertions(+), 19 deletions(-)
kernel/events/core.c +32 -19

154 lines not on this page · raw message

Subject: perf: Delay put_task_struct() for 'dead' contexts
From: Peter Zijlstra <[email protected]>
Date: Mon Jul 15 13:50:05 CEST 2019

Currently we immediately do put_task_struct() when we mark a context dead, instead delay it until after the final put_ctx() by recovering the task pointer.

---
 kernel/events/core.c |   18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)
kernel/events/core.c +12 -6
@@ -180,10 +180,17 @@ static inline void mark_dead_task_ctx(st
 {
 	unsigned long task = (unsigned long)READ_ONCE(ctx->task);
 	WARN_ON_ONCE(!task);
-	task |= 1;
+	task |= 1;;
 	WRITE_ONCE(ctx->task, (struct task_struct *)task);
 }
 
+static inline struct task_struct *__dead_ctx_task(struct perf_event_context *ctx)
+{
+	unsigned long task = (unsigned long)READ_ONCE(ctx->task);
+	task &= ~1L;
+	return (struct task_struct *)task;
+}
+
 /*
  * On task ctx scheduling...
  *
@@ -1200,10 +1207,11 @@ static void free_ctx(struct rcu_head *he
 static void put_ctx(struct perf_event_context *ctx)
 {
 	if (refcount_dec_and_test(&ctx->refcount)) {
+		struct task_struct *task = __dead_ctx_task(ctx);
 		if (ctx->parent_ctx)
 			put_ctx(ctx->parent_ctx);
-		if (ctx->task && !is_dead_task(ctx->task))
-			put_task_struct(ctx->task);
+		if (task)
+			put_task_struct(task);
 		call_rcu(&ctx->rcu_head, free_ctx);
 	}
 }
@@ -11484,9 +11492,8 @@ static void perf_event_exit_task_context
 	 * and mark the context dead.
 	 */
 	RCU_INIT_POINTER(task->perf_event_ctxp[ctxn], NULL);
-	put_ctx(task_ctx); /* cannot be last */
+	put_ctx(task_ctx); /* matches perf_pin_task_context(), cannot be last */
 	mark_dead_task_ctx(task_ctx);
-	put_task_struct(current); /* cannot be last */
 
 	clone_ctx = unclone_ctx(task_ctx);
 	raw_spin_unlock_irq(&task_ctx->lock);
@@ -11595,7 +11602,6 @@ void perf_event_free_task(struct task_st
 		 */
 		RCU_INIT_POINTER(task->perf_event_ctxp[ctxn], NULL);
 		mark_dead_task_ctx(ctx);
-		put_task_struct(task); /* cannot be last */
 		raw_spin_unlock_irq(&ctx->lock);
 
 		list_for_each_entry_safe(event, tmp, &ctx->event_list, event_entry)

Subject: perf: Pass @task to pmu->event_init() and event->destroy()
From: Peter Zijlstra <[email protected]>
Date: Mon Jul 15 14:16:58 CEST 2019

Change the function signature of pmu->event_init() and event->destroy() to take an additional task pointer.

On pmu->event_init() we can trivially provide it, on event->destroy() we need to play games to recover it from ctx->task.

This should allow removal of event->hw.target.

---
 arch/alpha/kernel/perf_event.c                   |    4 -
 arch/arc/kernel/perf_event.c                     |    2 
 arch/arm/mach-imx/mmdc.c                         |    2 
 arch/arm/mm/cache-l2x0-pmu.c                     |    2 
 arch/csky/kernel/perf_event.c                    |    2 
 arch/mips/kernel/perf_event_mipsxx.c             |    4 -
 arch/nds32/kernel/perf_event_cpu.c               |    2 
 arch/powerpc/perf/8xx-pmu.c                      |    2 
 arch/powerpc/perf/core-book3s.c                  |    4 -
 arch/powerpc/perf/core-fsl-emb.c                 |    4 -
 arch/powerpc/perf/hv-24x7.c                      |    4 -
 arch/powerpc/perf/hv-gpci.c                      |    2 
 arch/powerpc/perf/imc-pmu.c                      |   12 ++---
 arch/riscv/kernel/perf_event.c                   |    6 +-
 arch/s390/kernel/perf_cpum_cf.c                  |    6 +-
 arch/s390/kernel/perf_cpum_cf_diag.c             |    6 +-
 arch/s390/kernel/perf_cpum_sf.c                  |    6 +-
 arch/sh/kernel/perf_event.c                      |    6 +-
 arch/sparc/kernel/perf_event.c                   |    4 -
 arch/x86/events/amd/iommu.c                      |    2 
 arch/x86/events/amd/power.c                      |    2 
 arch/x86/events/amd/uncore.c                     |    2 
 arch/x86/events/core.c                           |   10 ++--
 arch/x86/events/intel/bts.c                      |    4 -
 arch/x86/events/intel/cstate.c                   |    2 
 arch/x86/events/intel/uncore_snb.c               |    2 
 arch/x86/events/msr.c                            |    2 
 arch/x86/events/perf_event.h                     |    2 
 arch/xtensa/kernel/perf_event.c                  |    2 
 drivers/hwtracing/coresight/coresight-etm-perf.c |    4 -
 drivers/perf/arm-cci.c                           |    6 +-
 drivers/perf/arm-ccn.c                           |    2 
 drivers/perf/arm_dsu_pmu.c                       |    2 
 drivers/perf/arm_pmu.c                           |    4 -
 drivers/perf/arm_smmuv3_pmu.c                    |    2 
 drivers/perf/fsl_imx8_ddr_perf.c                 |    2 
 drivers/perf/hisilicon/hisi_uncore_pmu.c         |    2 
 drivers/perf/hisilicon/hisi_uncore_pmu.h         |    2 
 drivers/perf/qcom_l2_pmu.c                       |    2 
 drivers/perf/qcom_l3_pmu.c                       |    2 
 drivers/perf/thunderx2_pmu.c                     |    2 
 drivers/perf/xgene_pmu.c                         |    2 
 include/linux/perf_event.h                       |    4 -
 include/linux/trace_events.h                     |    4 -
 kernel/events/core.c                             |   48 ++++++++++++-----------
 kernel/trace/trace_event_perf.c                  |    4 -
 46 files changed, 104 insertions(+), 100 deletions(-)
arch/alpha/kernel/perf_event.c +2 -2
@@ -591,7 +591,7 @@ static int supported_cpu(void)
 
 
 
-static void hw_perf_event_destroy(struct perf_event *event)
+static void hw_perf_event_destroy(struct perf_event *event, struct task_struct *task)
 {
 	/* Nothing to be done! */
 	return;
@@ -687,7 +687,7 @@ static int __hw_perf_event_init(struct p
 /*
  * Main entry point to initialise a HW performance event.
  */
-static int alpha_pmu_event_init(struct perf_event *event)
+static int alpha_pmu_event_init(struct perf_event *event, struct task_struct *task)
 {
 	int err;
 
arch/arc/kernel/perf_event.c +1 -1
@@ -164,7 +164,7 @@ static int arc_pmu_cache_event(u64 confi
 }
 
 /* initializes hw_perf_event structure if event is supported */
-static int arc_pmu_event_init(struct perf_event *event)
+static int arc_pmu_event_init(struct perf_event *event, struct task_struct *task)
 {
 	struct hw_perf_event *hwc = &event->hw;
 	int ret;
arch/arm/mach-imx/mmdc.c +1 -1
@@ -272,7 +272,7 @@ static bool mmdc_pmu_group_is_valid(stru
 	return true;
 }
 
-static int mmdc_pmu_event_init(struct perf_event *event)
+static int mmdc_pmu_event_init(struct perf_event *event, struct task_struct *task)
 {
 	struct mmdc_pmu *pmu_mmdc = to_mmdc_pmu(event->pmu);
 	int cfg = event->attr.config;
arch/arm/mm/cache-l2x0-pmu.c +1 -1
@@ -291,7 +291,7 @@ static bool l2x0_pmu_group_is_valid(stru
 	return num_hw <= PMU_NR_COUNTERS;
 }
 
-static int l2x0_pmu_event_init(struct perf_event *event)
+static int l2x0_pmu_event_init(struct perf_event *event, struct task_struct *task)
 {
 	struct hw_perf_event *hw = &event->hw;
 
arch/csky/kernel/perf_event.c +1 -1
@@ -839,7 +839,7 @@ static int csky_pmu_cache_event(u64 conf
 	return csky_pmu_cache_map[cache_type][cache_op][cache_result];
 }
 
-static int csky_pmu_event_init(struct perf_event *event)
+static int csky_pmu_event_init(struct perf_event *event, struct task_struct *task)
 {
 	struct hw_perf_event *hwc = &event->hw;
 	int ret;
arch/mips/kernel/perf_event_mipsxx.c +2 -2
@@ -591,7 +591,7 @@ static void mipspmu_free_irq(void)
 static void reset_counters(void *arg);
 static int __hw_perf_event_init(struct perf_event *event);
 
-static void hw_perf_event_destroy(struct perf_event *event)
+static void hw_perf_event_destroy(struct perf_event *event, struct task_struct *task)
 {
 	if (atomic_dec_and_mutex_lock(&active_events,
 				&pmu_reserve_mutex)) {
@@ -606,7 +606,7 @@ static void hw_perf_event_destroy(struct
 	}
 }
 
-static int mipspmu_event_init(struct perf_event *event)
+static int mipspmu_event_init(struct perf_event *event, struct task_Struct *task)
 {
 	int err = 0;
 
arch/nds32/kernel/perf_event_cpu.c +1 -1
@@ -854,7 +854,7 @@ static int __hw_perf_event_init(struct p
 	return 0;
 }
 
-static int nds32_pmu_event_init(struct perf_event *event)
+static int nds32_pmu_event_init(struct perf_event *event, struct task_struct *task)
 {
 	struct nds32_pmu *nds32_pmu = to_nds32_pmu(event->pmu);
 	int err = 0;
arch/powerpc/perf/8xx-pmu.c +1 -1
@@ -68,7 +68,7 @@ static int event_type(struct perf_event
 	return -EOPNOTSUPP;
 }
 
-static int mpc8xx_pmu_event_init(struct perf_event *event)
+static int mpc8xx_pmu_event_init(struct perf_event *event, struct task_struct *task)
 {
 	int type = event_type(event);
 
arch/powerpc/perf/core-book3s.c +2 -2
@@ -1784,7 +1784,7 @@ static DEFINE_MUTEX(pmc_reserve_mutex);
 /*
  * Release the PMU if this is the last perf_event.
  */
-static void hw_perf_event_destroy(struct perf_event *event)
+static void hw_perf_event_destroy(struct perf_event *event, struct task_struct *task)
 {
 	if (!atomic_add_unless(&num_events, -1, 1)) {
 		mutex_lock(&pmc_reserve_mutex);
@@ -1836,7 +1836,7 @@ static bool is_event_blacklisted(u64 ev)
 	return false;
 }
 
-static int power_pmu_event_init(struct perf_event *event)
+static int power_pmu_event_init(struct perf_event *event, struct task_struct *task)
 {
 	u64 ev;
 	unsigned long flags;
arch/powerpc/perf/core-fsl-emb.c +2 -2
@@ -439,7 +439,7 @@ static void fsl_emb_pmu_stop(struct perf
 /*
  * Release the PMU if this is the last perf_event.
  */
-static void hw_perf_event_destroy(struct perf_event *event)
+static void hw_perf_event_destroy(struct perf_event *event, struct task_struct *task)
 {
 	if (!atomic_add_unless(&num_events, -1, 1)) {
 		mutex_lock(&pmc_reserve_mutex);
@@ -479,7 +479,7 @@ static int hw_perf_cache_event(u64 confi
 	return 0;
 }
 
-static int fsl_emb_pmu_event_init(struct perf_event *event)
+static int fsl_emb_pmu_event_init(struct perf_event *event, struct task_struct *task)
 {
 	u64 ev;
 	struct perf_event *events[MAX_HWEVENTS];
arch/powerpc/perf/hv-24x7.c +2 -2
@@ -603,7 +603,7 @@ static int event_uniq_add(struct rb_root
 	return 0;
 }
 
-static void event_uniq_destroy(struct rb_root *root)
+static void event_uniq_destroy(struct rb_root *root, struct task_struct *task)
 {
 	/*
 	 * the strings we point to are in the giant block of memory filled by
@@ -1277,7 +1277,7 @@ static int single_24x7_request(struct pe
 }
 
 
-static int h_24x7_event_init(struct perf_event *event)
+static int h_24x7_event_init(struct perf_event *event, struct task_struct *task)
 {
 	struct hv_perf_caps caps;
 	unsigned domain;
arch/powerpc/perf/hv-gpci.c +1 -1
@@ -213,7 +213,7 @@ static int h_gpci_event_add(struct perf_
 	return 0;
 }
 
-static int h_gpci_event_init(struct perf_event *event)
+static int h_gpci_event_init(struct perf_event *event, struct task_struct *task)
 {
 	u64 count;
 	u8 length;
arch/powerpc/perf/imc-pmu.c +6 -6
@@ -432,7 +432,7 @@ static int nest_pmu_cpumask_init(void)
 				 ppc_nest_imc_cpu_offline);
 }
 
-static void nest_imc_counters_release(struct perf_event *event)
+static void nest_imc_counters_release(struct perf_event *event, struct task_struct *task)
 {
 	int rc, node_id;
 	struct imc_pmu_ref *ref;
@@ -484,7 +484,7 @@ static void nest_imc_counters_release(st
 	mutex_unlock(&ref->lock);
 }
 
-static int nest_imc_event_init(struct perf_event *event)
+static int nest_imc_event_init(struct perf_event *event, struct task_struct *task)
 {
 	int chip_id, rc, node_id;
 	u32 l_config, config = event->attr.config;
@@ -708,7 +708,7 @@ static int core_imc_pmu_cpumask_init(voi
 				 ppc_core_imc_cpu_offline);
 }
 
-static void core_imc_counters_release(struct perf_event *event)
+static void core_imc_counters_release(struct perf_event *event, struct task_struct *task)
 {
 	int rc, core_id;
 	struct imc_pmu_ref *ref;
@@ -759,7 +759,7 @@ static void core_imc_counters_release(st
 	mutex_unlock(&ref->lock);
 }
 
-static int core_imc_event_init(struct perf_event *event)
+static int core_imc_event_init(struct perf_event *event, struct task_struct *task)
 {
 	int core_id, rc;
 	u64 config = event->attr.config;
@@ -885,7 +885,7 @@ static int thread_imc_cpu_init(void)
 			  ppc_thread_imc_cpu_offline);
 }
 
-static int thread_imc_event_init(struct perf_event *event)
+static int thread_imc_event_init(struct perf_event *event, struct task_struct *task)
 {
 	u32 config = event->attr.config;
 	struct task_struct *target;
@@ -1293,7 +1293,7 @@ static void trace_imc_event_del(struct p
 	trace_imc_event_stop(event, flags);
 }
 
-static int trace_imc_event_init(struct perf_event *event)
+static int trace_imc_event_init(struct perf_event *event, struct task_struct *task)
 {
 	struct task_struct *target;
 
arch/riscv/kernel/perf_event.c +3 -3
@@ -375,13 +375,13 @@ void release_pmc_hardware(void)
 
 static atomic_t riscv_active_events = ATOMIC_INIT(0);
 
-static void riscv_event_destroy(struct perf_event *event)
+static void riscv_event_destroy(struct perf_event *event, struct task_struct *task)
 {
 	if (atomic_dec_return(&riscv_active_events) == 0)
 		release_pmc_hardware();
 }
 
-static int riscv_event_init(struct perf_event *event)
+static int riscv_event_init(struct perf_event *event, struct task_struct *task)
 {
 	struct perf_event_attr *attr = &event->attr;
 	struct hw_perf_event *hwc = &event->hw;
@@ -413,7 +413,7 @@ static int riscv_event_init(struct perf_
 
 	event->destroy = riscv_event_destroy;
 	if (code < 0) {
-		event->destroy(event);
+		event->destroy(event, task);
 		return code;
 	}
 
arch/s390/kernel/perf_cpum_cf.c +3 -3
@@ -168,7 +168,7 @@ static atomic_t num_events = ATOMIC_INIT
 static DEFINE_MUTEX(pmc_reserve_mutex);
 
 /* Release the PMU if event is the last perf event */
-static void hw_perf_event_destroy(struct perf_event *event)
+static void hw_perf_event_destroy(struct perf_event *event, struct task_struct *task)
 {
 	if (!atomic_add_unless(&num_events, -1, 1)) {
 		mutex_lock(&pmc_reserve_mutex);
@@ -292,7 +292,7 @@ static int __hw_perf_event_init(struct p
 	return err;
 }
 
-static int cpumf_pmu_event_init(struct perf_event *event)
+static int cpumf_pmu_event_init(struct perf_event *event, struct task_struct *task)
 {
 	int err;
 
@@ -307,7 +307,7 @@ static int cpumf_pmu_event_init(struct p
 	}
 
 	if (unlikely(err) && event->destroy)
-		event->destroy(event);
+		event->destroy(event, task);
 
 	return err;
 }
arch/s390/kernel/perf_cpum_cf_diag.c +3 -3
@@ -180,7 +180,7 @@ static void cf_diag_disable(struct pmu *
 static atomic_t cf_diag_events = ATOMIC_INIT(0);
 
 /* Release the PMU if event is the last perf event */
-static void cf_diag_perf_event_destroy(struct perf_event *event)
+static void cf_diag_perf_event_destroy(struct perf_event *event, struct task_struct *task)
 {
 	debug_sprintf_event(cf_diag_dbg, 5,
 			    "%s event %p cpu %d cf_diag_events %d\n",
@@ -237,7 +237,7 @@ static int __hw_perf_event_init(struct p
 	return err;
 }
 
-static int cf_diag_event_init(struct perf_event *event)
+static int cf_diag_event_init(struct perf_event *event, struct task_struct *task)
 {
 	struct perf_event_attr *attr = &event->attr;
 	int err = -ENOENT;
@@ -275,7 +275,7 @@ static int cf_diag_event_init(struct per
 
 	err = __hw_perf_event_init(event);
 	if (unlikely(err))
-		event->destroy(event);
+		event->destroy(event, task);
 out:
 	debug_sprintf_event(cf_diag_dbg, 5, "%s err %d\n", __func__, err);
 	return err;
arch/s390/kernel/perf_cpum_sf.c +3 -3
@@ -581,7 +581,7 @@ static int reserve_pmc_hardware(void)
 	return 0;
 }
 
-static void hw_perf_event_destroy(struct perf_event *event)
+static void hw_perf_event_destroy(struct perf_event *event, struct task_struct *task)
 {
 	/* Release PMC if this is the last perf event */
 	if (!atomic_add_unless(&num_events, -1, 1)) {
@@ -823,7 +823,7 @@ static int __hw_perf_event_init(struct p
 	return err;
 }
 
-static int cpumsf_pmu_event_init(struct perf_event *event)
+static int cpumsf_pmu_event_init(struct perf_event *event, struct task_struct *task)
 {
 	int err;
 
@@ -867,7 +867,7 @@ static int cpumsf_pmu_event_init(struct
 	err = __hw_perf_event_init(event);
 	if (unlikely(err))
 		if (event->destroy)
-			event->destroy(event);
+			event->destroy(event, task);
 	return err;
 }
 
arch/sh/kernel/perf_event.c +3 -3
@@ -78,7 +78,7 @@ EXPORT_SYMBOL_GPL(perf_num_counters);
 /*
  * Release the PMU if this is the last perf_event.
  */
-static void hw_perf_event_destroy(struct perf_event *event)
+static void hw_perf_event_destroy(struct perf_event *event, struct task_struct *task)
 {
 	if (!atomic_add_unless(&num_events, -1, 1)) {
 		mutex_lock(&pmc_reserve_mutex);
@@ -295,7 +295,7 @@ static void sh_pmu_read(struct perf_even
 	sh_perf_event_update(event, &event->hw, event->hw.idx);
 }
 
-static int sh_pmu_event_init(struct perf_event *event)
+static int sh_pmu_event_init(struct perf_event *event, struct task_struct *task)
 {
 	int err;
 
@@ -316,7 +316,7 @@ static int sh_pmu_event_init(struct perf
 
 	if (unlikely(err)) {
 		if (event->destroy)
-			event->destroy(event);
+			event->destroy(event, task);
 	}
 
 	return err;
arch/sparc/kernel/perf_event.c +2 -2
@@ -1224,7 +1224,7 @@ static const struct perf_event_map *spar
 	return pmap;
 }
 
-static void hw_perf_event_destroy(struct perf_event *event)
+static void hw_perf_event_destroy(struct perf_event *event, struct task_struct *task)
 {
 	perf_event_release_pmc();
 }
@@ -1412,7 +1412,7 @@ static int sparc_pmu_add(struct perf_eve
 	return ret;
 }
 
-static int sparc_pmu_event_init(struct perf_event *event)
+static int sparc_pmu_event_init(struct perf_event *event, struct task_struct *task)
 {
 	struct perf_event_attr *attr = &event->attr;
 	struct perf_event *evts[MAX_HWEVENTS];
arch/x86/events/amd/iommu.c +1 -1
@@ -204,7 +204,7 @@ static int clear_avail_iommu_bnk_cntr(st
 	return 0;
 }
 
-static int perf_iommu_event_init(struct perf_event *event)
+static int perf_iommu_event_init(struct perf_event *event, struct task_struct *task)
 {
 	struct hw_perf_event *hwc = &event->hw;
 
arch/x86/events/amd/power.c +1 -1
@@ -124,7 +124,7 @@ static void pmu_event_del(struct perf_ev
 	pmu_event_stop(event, PERF_EF_UPDATE);
 }
 
-static int pmu_event_init(struct perf_event *event)
+static int pmu_event_init(struct perf_event *event, struct task_struct *task)
 {
 	u64 cfg = event->attr.config & AMD_POWER_EVENT_MASK;
 
arch/x86/events/amd/uncore.c +1 -1
@@ -180,7 +180,7 @@ static void amd_uncore_del(struct perf_e
 	hwc->idx = -1;
 }
 
-static int amd_uncore_event_init(struct perf_event *event)
+static int amd_uncore_event_init(struct perf_event *event, struct task_struct *task)
 {
 	struct amd_uncore *uncore;
 	struct hw_perf_event *hwc = &event->hw;
arch/x86/events/core.c +5 -5
@@ -277,15 +277,15 @@ static bool check_hw_exists(void)
 	return false;
 }
 
-static void hw_perf_event_destroy(struct perf_event *event)
+static void hw_perf_event_destroy(struct perf_event *event, struct task_struct *task)
 {
 	x86_release_hardware();
 	atomic_dec(&active_events);
 }
 
-void hw_perf_lbr_event_destroy(struct perf_event *event)
+void hw_perf_lbr_event_destroy(struct perf_event *event, struct task_struct *task)
 {
-	hw_perf_event_destroy(event);
+	hw_perf_event_destroy(event, task);
 
 	/* undo the lbr/bts event accounting */
 	x86_del_exclusive(x86_lbr_exclusive_lbr);
@@ -2041,7 +2041,7 @@ static int validate_group(struct perf_ev
 	return ret;
 }
 
-static int x86_pmu_event_init(struct perf_event *event)
+static int x86_pmu_event_init(struct perf_event *event, struct task_struct *task)
 {
 	struct pmu *tmp;
 	int err;
@@ -2075,7 +2075,7 @@ static int x86_pmu_event_init(struct per
 	}
 	if (err) {
 		if (event->destroy)
-			event->destroy(event);
+			event->destroy(event, task);
 	}
 
 	if (READ_ONCE(x86_pmu.attr_rdpmc) &&
arch/x86/events/intel/bts.c +2 -2
@@ -527,13 +527,13 @@ static int bts_event_add(struct perf_eve
 	return 0;
 }
 
-static void bts_event_destroy(struct perf_event *event)
+static void bts_event_destroy(struct perf_event *event, struct task_struct *task)
 {
 	x86_release_hardware();
 	x86_del_exclusive(x86_lbr_exclusive_bts);
 }
 
-static int bts_event_init(struct perf_event *event)
+static int bts_event_init(struct perf_event *event, struct task_struct *task)
 {
 	int ret;
 
arch/x86/events/intel/cstate.c +1 -1
@@ -295,7 +295,7 @@ static ssize_t cstate_get_attr_cpumask(s
 		return 0;
 }
 
-static int cstate_pmu_event_init(struct perf_event *event)
+static int cstate_pmu_event_init(struct perf_event *event, struct task_struct *task)
 {
 	u64 cfg = event->attr.config;
 	int cpu;
arch/x86/events/intel/uncore_snb.c +1 -1
@@ -444,7 +444,7 @@ static void snb_uncore_imc_disable_event
  * Keep the custom event_init() function compatible with old event
  * encoding for free running counters.
  */
-static int snb_uncore_imc_event_init(struct perf_event *event)
+static int snb_uncore_imc_event_init(struct perf_event *event, struct task_struct *task)
 {
 	struct intel_uncore_pmu *pmu;
 	struct intel_uncore_box *box;
arch/x86/events/msr.c +1 -1
@@ -178,7 +178,7 @@ const struct attribute_group *attr_updat
 	NULL,
 };
 
-static int msr_event_init(struct perf_event *event)
+static int msr_event_init(struct perf_event *event, struct task_struct *task)
 {
 	u64 cfg = event->attr.config;
 
arch/x86/events/perf_event.h +1 -1
@@ -810,7 +810,7 @@ void x86_release_hardware(void);
 
 int x86_pmu_max_precise(void);
 
-void hw_perf_lbr_event_destroy(struct perf_event *event);
+void hw_perf_lbr_event_destroy(struct perf_event *event, struct task_struct *task);
 
 int x86_setup_perfctr(struct perf_event *event);
 
arch/xtensa/kernel/perf_event.c +1 -1
@@ -195,7 +195,7 @@ static void xtensa_pmu_disable(struct pm
 	set_er(get_er(XTENSA_PMU_PMG) & ~XTENSA_PMU_PMG_PMEN, XTENSA_PMU_PMG);
 }
 
-static int xtensa_pmu_event_init(struct perf_event *event)
+static int xtensa_pmu_event_init(struct perf_event *event, struct task_struct *task)
 {
 	int ret;
 
drivers/hwtracing/coresight/coresight-etm-perf.c +2 -2
@@ -96,13 +96,13 @@ static int etm_addr_filters_alloc(struct
 	return 0;
 }
 
-static void etm_event_destroy(struct perf_event *event)
+static void etm_event_destroy(struct perf_event *event, struct task_struct *task)
 {
 	kfree(event->hw.addr_filters);
 	event->hw.addr_filters = NULL;
 }
 
-static int etm_event_init(struct perf_event *event)
+static int etm_event_init(struct perf_event *event, struct task_struct *task)
 {
 	int ret = 0;
 
drivers/perf/arm-cci.c +3 -3
@@ -1081,7 +1081,7 @@ static void cci_pmu_put_hw(struct cci_pm
 	pmu_free_irq(cci_pmu);
 }
 
-static void hw_perf_event_destroy(struct perf_event *event)
+static void hw_perf_event_destroy(struct perf_event *event, struct task_struct *task)
 {
 	struct cci_pmu *cci_pmu = to_cci_pmu(event->pmu);
 	atomic_t *active_events = &cci_pmu->active_events;
@@ -1314,7 +1314,7 @@ static int __hw_perf_event_init(struct p
 	return 0;
 }
 
-static int cci_pmu_event_init(struct perf_event *event)
+static int cci_pmu_event_init(struct perf_event *event, struct task_struct *task)
 {
 	struct cci_pmu *cci_pmu = to_cci_pmu(event->pmu);
 	atomic_t *active_events = &cci_pmu->active_events;
@@ -1354,7 +1354,7 @@ static int cci_pmu_event_init(struct per
 
 	err = __hw_perf_event_init(event);
 	if (err)
-		hw_perf_event_destroy(event);
+		hw_perf_event_destroy(event, task);
 
 	return err;
 }
drivers/perf/arm-ccn.c +1 -1
@@ -715,7 +715,7 @@ static void arm_ccn_pmu_event_release(st
 	ccn->dt.pmu_counters[hw->idx].event = NULL;
 }
 
-static int arm_ccn_pmu_event_init(struct perf_event *event)
+static int arm_ccn_pmu_event_init(struct perf_event *event, struct task_struct *task)
 {
 	struct arm_ccn *ccn;
 	struct hw_perf_event *hw = &event->hw;
drivers/perf/arm_dsu_pmu.c +1 -1
@@ -540,7 +540,7 @@ static bool dsu_pmu_validate_group(struc
 	return dsu_pmu_validate_event(event->pmu, &fake_hw, event);
 }
 
-static int dsu_pmu_event_init(struct perf_event *event)
+static int dsu_pmu_event_init(struct perf_event *event, struct task_struct *task)
 {
 	struct dsu_pmu *dsu_pmu = to_dsu_pmu(event->pmu);
 
drivers/perf/arm_pmu.c +2 -2
@@ -419,7 +419,7 @@ __hw_perf_event_init(struct perf_event *
 	return 0;
 }
 
-static int armpmu_event_init(struct perf_event *event)
+static int armpmu_event_init(struct perf_event *event, struct task_struct *task)
 {
 	struct arm_pmu *armpmu = to_arm_pmu(event->pmu);
 
@@ -771,7 +771,7 @@ static int cpu_pmu_init(struct arm_pmu *
 	return err;
 }
 
-static void cpu_pmu_destroy(struct arm_pmu *cpu_pmu)
+static void cpu_pmu_destroy(struct arm_pmu *cpu_pmu, struct task_struct *task)
 {
 	cpu_pm_pmu_unregister(cpu_pmu);
 	cpuhp_state_remove_instance_nocalls(CPUHP_AP_PERF_ARM_STARTING,
drivers/perf/arm_smmuv3_pmu.c +1 -1
@@ -317,7 +317,7 @@ static int smmu_pmu_get_event_idx(struct
  * the core perf events code.
  */
 
-static int smmu_pmu_event_init(struct perf_event *event)
+static int smmu_pmu_event_init(struct perf_event *event, struct task_struct *task)
 {
 	struct hw_perf_event *hwc = &event->hw;
 	struct smmu_pmu *smmu_pmu = to_smmu_pmu(event->pmu);
drivers/perf/fsl_imx8_ddr_perf.c +1 -1
@@ -189,7 +189,7 @@ static u32 ddr_perf_read_counter(struct
 	return readl_relaxed(pmu->base + COUNTER_READ + counter * 4);
 }
 
-static int ddr_perf_event_init(struct perf_event *event)
+static int ddr_perf_event_init(struct perf_event *event, struct task_struct *task)
 {
 	struct ddr_pmu *pmu = to_ddr_pmu(event->pmu);
 	struct hw_perf_event *hwc = &event->hw;
drivers/perf/hisilicon/hisi_uncore_pmu.c +1 -1
@@ -123,7 +123,7 @@ static void hisi_uncore_pmu_clear_event_
 	clear_bit(idx, hisi_pmu->pmu_events.used_mask);
 }
 
-int hisi_uncore_pmu_event_init(struct perf_event *event)
+int hisi_uncore_pmu_event_init(struct perf_event *event, struct task_struct *task)
 {
 	struct hw_perf_event *hwc = &event->hw;
 	struct hisi_pmu *hisi_pmu;
drivers/perf/hisilicon/hisi_uncore_pmu.h +1 -1
@@ -85,7 +85,7 @@ void hisi_uncore_pmu_start(struct perf_e
 void hisi_uncore_pmu_stop(struct perf_event *event, int flags);
 void hisi_uncore_pmu_set_event_period(struct perf_event *event);
 void hisi_uncore_pmu_event_update(struct perf_event *event);
-int hisi_uncore_pmu_event_init(struct perf_event *event);
+int hisi_uncore_pmu_event_init(struct perf_event *event, struct task_struct *task);
 void hisi_uncore_pmu_enable(struct pmu *pmu);
 void hisi_uncore_pmu_disable(struct pmu *pmu);
 ssize_t hisi_event_sysfs_show(struct device *dev,
drivers/perf/qcom_l2_pmu.c +1 -1
@@ -477,7 +477,7 @@ static void l2_cache_pmu_disable(struct
 	cluster_pmu_disable();
 }
 
-static int l2_cache_event_init(struct perf_event *event)
+static int l2_cache_event_init(struct perf_event *event, struct task_struct *task)
 {
 	struct hw_perf_event *hwc = &event->hw;
 	struct cluster_pmu *cluster;
drivers/perf/qcom_l3_pmu.c +1 -1
@@ -475,7 +475,7 @@ static bool qcom_l3_cache__validate_even
 	return counters <= L3_NUM_COUNTERS;
 }
 
-static int qcom_l3_cache__event_init(struct perf_event *event)
+static int qcom_l3_cache__event_init(struct perf_event *event, struct task_struct *task)
 {
 	struct l3cache_pmu *l3pmu = to_l3cache_pmu(event->pmu);
 	struct hw_perf_event *hwc = &event->hw;
drivers/perf/thunderx2_pmu.c +1 -1
@@ -407,7 +407,7 @@ static bool tx2_uncore_validate_event_gr
 }
 
 
-static int tx2_uncore_event_init(struct perf_event *event)
+static int tx2_uncore_event_init(struct perf_event *event, struct task_struct *task)
 {
 	struct hw_perf_event *hwc = &event->hw;
 	struct tx2_uncore_pmu *tx2_pmu;
drivers/perf/xgene_pmu.c +1 -1
@@ -887,7 +887,7 @@ static void xgene_perf_pmu_disable(struc
 	xgene_pmu->ops->stop_counters(pmu_dev);
 }
 
-static int xgene_perf_event_init(struct perf_event *event)
+static int xgene_perf_event_init(struct perf_event *event, struct task_struct *task)
 {
 	struct xgene_pmu_dev *pmu_dev = to_pmu_dev(event->pmu);
 	struct hw_perf_event *hw = &event->hw;
include/linux/perf_event.h +2 -2
@@ -297,7 +297,7 @@ struct pmu {
 	 *
 	 * Other error return values are allowed.
 	 */
-	int (*event_init)		(struct perf_event *event);
+	int (*event_init)		(struct perf_event *event, struct task_struct *task);
 
 	/*
 	 * Notification that the event was mapped or unmapped.  Called
@@ -681,7 +681,7 @@ struct perf_event {
 	struct perf_addr_filter_range	*addr_filter_ranges;
 	unsigned long			addr_filters_gen;
 
-	void (*destroy)(struct perf_event *);
+	void (*destroy)(struct perf_event *, struct task_struct *);
 	struct rcu_head			rcu_head;
 
 	struct pid_namespace		*ns;
include/linux/trace_events.h +2 -2
@@ -572,7 +572,7 @@ extern int  perf_trace_add(struct perf_e
 extern void perf_trace_del(struct perf_event *event, int flags);
 #ifdef CONFIG_KPROBE_EVENTS
 extern int  perf_kprobe_init(struct perf_event *event, bool is_retprobe);
-extern void perf_kprobe_destroy(struct perf_event *event);
+extern void perf_kprobe_destroy(struct perf_event *event, struct task_struct *task);
 extern int bpf_get_kprobe_info(const struct perf_event *event,
 			       u32 *fd_type, const char **symbol,
 			       u64 *probe_offset, u64 *probe_addr,
@@ -581,7 +581,7 @@ extern int bpf_get_kprobe_info(const str
 #ifdef CONFIG_UPROBE_EVENTS
 extern int  perf_uprobe_init(struct perf_event *event,
 			     unsigned long ref_ctr_offset, bool is_retprobe);
-extern void perf_uprobe_destroy(struct perf_event *event);
+extern void perf_uprobe_destroy(struct perf_event *event, struct task_struct *task);
 extern int bpf_get_uprobe_info(const struct perf_event *event,
 			       u32 *fd_type, const char **filename,
 			       u64 *probe_offset, bool perf_type_tracepoint);
kernel/events/core.c +26 -22

172 lines not on this page · raw message

kernel/trace/trace_event_perf.c +2 -2
@@ -280,7 +280,7 @@ int perf_kprobe_init(struct perf_event *
 	return ret;
 }
 
-void perf_kprobe_destroy(struct perf_event *p_event)
+void perf_kprobe_destroy(struct perf_event *p_event, struct task_struct *task)
 {
 	perf_trace_event_close(p_event);
 	perf_trace_event_unreg(p_event);
@@ -333,7 +333,7 @@ int perf_uprobe_init(struct perf_event *
 	return ret;
 }
 
-void perf_uprobe_destroy(struct perf_event *p_event)
+void perf_uprobe_destroy(struct perf_event *p_event, struct task_struct *task)
 {
 	mutex_lock(&event_mutex);
 	perf_trace_event_close(p_event);

Subject: perf/power: Remove some simple hw.target usage
From: Peter Zijlstra <[email protected]>
Date: Mon Jul 15 14:34:17 CEST 2019

---
 arch/powerpc/perf/imc-pmu.c |    7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
arch/powerpc/perf/imc-pmu.c +1 -6
@@ -888,7 +888,6 @@ static int thread_imc_cpu_init(void)
 static int thread_imc_event_init(struct perf_event *event, struct task_struct *task)
 {
 	u32 config = event->attr.config;
-	struct task_struct *target;
 	struct imc_pmu *pmu;
 
 	if (event->attr.type != event->pmu->type)
@@ -908,8 +907,7 @@ static int thread_imc_event_init(struct
 	if (((config & IMC_EVENT_OFFSET_MASK) > pmu->counter_mem_size))
 		return -EINVAL;
 
-	target = event->hw.target;
-	if (!target)
+	if (!task)
 		return -EINVAL;
 
 	event->pmu->task_ctx_nr = perf_sw_context;
@@ -1295,8 +1293,6 @@ static void trace_imc_event_del(struct p
 
 static int trace_imc_event_init(struct perf_event *event, struct task_struct *task)
 {
-	struct task_struct *target;
-
 	if (event->attr.type != event->pmu->type)
 		return -ENOENT;
 
@@ -1308,7 +1304,6 @@ static int trace_imc_event_init(struct p
 		return -ENOENT;
 
 	event->hw.idx = -1;
-	target = event->hw.target;
 
 	event->pmu->task_ctx_nr = perf_hw_context;
 	return 0;
#7 lore raw
Quoted 9 lines · Peter Zijlstra
Quoted 8 lines · Peter Zijlstra
Quoted 2 lines

I wish we could use event->ctx->task instead but on pmu::init() there is no ctx yet (we could pass the task in parameter though)

Right, that should be fairly easy.

Quoted 2 lines

and on event->destroy() it's TASK_TOMBSTONE and retrieving the task at that time would be non trivial.

Quoted 8 lines
Quoted 2 lines

Well, right, we can maybe make TOMBSTONE be the LSB instead of the whole word, then we can recover the task pointer... *yuck* though.

Something like the attached, completely untested patches.

I didn't do the hw_breakpoint bit, because I got lost in that, but this basically provides what you asked for I think.

Thanks they look good! I can take them and work on top if you like.

maintainer
#8 lore raw
Quoted 13 lines · Frederic Weisbecker
Quoted 12 lines · Peter Zijlstra
Quoted 10 lines · Peter Zijlstra
Quoted 2 lines

I wish we could use event->ctx->task instead but on pmu::init() there is no ctx yet (we could pass the task in parameter though)

Right, that should be fairly easy.

Quoted 2 lines

and on event->destroy() it's TASK_TOMBSTONE and retrieving the task at that time would be non trivial.

Well, right, we can maybe make TOMBSTONE be the LSB instead of the whole word, then we can recover the task pointer... *yuck* though.

Quoted 7 lines
Quoted 5 lines

Something like the attached, completely untested patches.

I didn't do the hw_breakpoint bit, because I got lost in that, but this basically provides what you asked for I think.

Thanks they look good! I can take them and work on top if you like.

please do.