perf/hw_breakpoint: Fix breakpoint overcommit issue
Subsystems (23)
cover perf/hw_breakpoint: Fix breakpoint overcommit issue
Syzbot has found a breakpoint overcommit issue:
https://lore.kernel.org/lkml/[email protected]/
It took me a long time to find out what the actual root problem was. Also its reproducer only worked on a few month old kernel but it didn't feel like the issue was actually solved.
I eventually cooked a reproducer that works with latest upstream, see in the end of this message.
The fix is just a few liner but implies to shut down the context swapping optimization for contexts containing breakpoints.
Also I feel like uprobes may be concerned as well as it seems to make use of event.hw->target after pmu::init().
git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
perf/pin
HEAD: 35b749650cc72402ae47beb5e0048c36636a4002
Thanks,
Frederic
---
Frederic Weisbecker (2):
perf: Allow a pmu to pin context perf/hw_breakpoints: Pin perf contexts of breakpoints
include/linux/perf_event.h | 2 ++ kernel/events/core.c | 6 ++++++ kernel/events/hw_breakpoint.c | 1 + 3 files changed, 9 insertions(+)
---
#define _GNU_SOURCE
#include <linux/perf_event.h>
#include <linux/hw_breakpoint.h>
#include <unistd.h>
#include <sys/syscall.h>
#include <stdio.h>
#include <sys/types.h>
#include <fcntl.h>
#include <sys/wait.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
static struct perf_event_attr attr = {
.type = PERF_TYPE_BREAKPOINT,
.size = sizeof(attr),
.config = 0,
.sample_period = 1,
.sample_type = PERF_SAMPLE_IP,
.read_format = PERF_FORMAT_ID,
.inherit = 1,
.pinned = 1,
.wakeup_events = 1,
.bp_type = HW_BREAKPOINT_W,
.bp_addr = 0,
.bp_len = 8,
};static void loop(int secs)
{
struct timespec tp;
int start;
clock_gettime(CLOCK_MONOTONIC, &tp);
start = tp.tv_sec;
for (;;) {
clock_gettime(CLOCK_MONOTONIC, &tp);
if (tp.tv_sec - start >= secs)
return;
}
}int main(int argc, char **argv)
{
int fd, i, status;
pid_t child1, child2;
for (i = 0; i < 4; i++) {
fd = syscall(__NR_perf_event_open, &attr, 0, -1, -1, 0);
if (fd < 0)
perror("perf_event_open");
}
child1 = fork();
if (child1 == 0) {
loop(1);
return 0;
}
child2 = fork();
if (child2 == 0) {
loop(2);
fd = syscall(__NR_perf_event_open, &attr, 0, -1, -1, 0);
if (fd < 0)
perror("perf_event_open");
return 0;
}
return 0;
}PATCH 1/2 perf: Allow a pmu to pin context
Some PMUs rely on the fact that struct perf_event::hw::target always point to the actual event target during the whole event lifecycle, ie:
from pmu::init() to event::destroy().
Now perf event ctx swapping on task sched switch breaks that guarantee.
Solve that with providing a way for a PMU to pin the context of an event.
--- include/linux/perf_event.h | 2 ++ kernel/events/core.c | 6 ++++++ 2 files changed, 8 insertions(+)
include/linux/perf_event.h +2 -0
@@ -273,6 +273,8 @@ struct pmu {/* number of address filters this PMU can do */ unsigned int nr_addr_filters; +int pin_ctx; +/* * Fully disable/enable this PMU, can be used to protect from the PMI * as well as for lazy/batch writing of the MSRs.
kernel/events/core.c +6 -0
@@ -1689,6 +1689,9 @@ list_add_event(struct perf_event *event, struct perf_event_context *ctx) ctx->nr_stat++; ctx->generation++; + +if (event->pmu->pin_ctx) + ctx->pin_count++; }/*
@@ -1885,6 +1888,9 @@ list_del_event(struct perf_event *event, struct perf_event_context *ctx)perf_event_set_state (event,PERF_EVENT_STATE_OFF ); ctx->generation++; + +if (event->pmu->pin_ctx) + ctx->pin_count--; }static void perf_group_detach (struct perf_event *event)
Signature · 1 line
2.21.0
PATCH 2/2 perf/hw_breakpoints: Pin perf contexts of breakpoints
In reply to Frederic WeisbeckerThe breakpoints core deals with limited architecture resources that need to be shared among all the breakpoint perf events. For example x86 has 4 breakpoint registers per-CPU. This means that a task can't open more than 4 breakpoints in this architecture.
To control that we must track the number of breakpoints held for each task. This is simply implemented with a global list of all breakpoints that we walk and count on top of their target task.
But this wrongly assume that the target task of a perf event remains stable after the event creation. Context tasks can get swapped on context switches and it's fairly common for a perf event's initial target (event->hw.target) to become irrelevantly out of date at struct perf_event::destroy() time.
Here is an interesting scenario where it matters: TASK A and B are clones that share 4 equivalent breakpoints. A and B contexts are swapped on context switch. So task A now carries the breakpoints that were initialized in B. Task A quits and release its breakpoints whose target (event->hw.target) still point to B. Therefore the breakpoints slots are spuriously released on B. Afterward, if task B were to create a 5th breakpoint then it would be accepted. But on architecture breakpoint installation time, that 5th breakpoint will be eventually rejected:
Can't find any breakpoint slot WARNING: CPU: 6 PID: 2433 at arch/x86/kernel/hw_breakpoint.c:109 arch_install_hw_breakpoint+0x176/0x180 Modules linked in: CPU: 6 PID: 2433 Comm: bp5 Not tainted 5.2.0-rc6+ #20 Hardware name: MSI MS-7850/Z87-G41 PC Mate(MS-7850), BIOS V1.3 08/18/2013 RIP: 0010:arch_install_hw_breakpoint+0x176/0x180 Code: 57 ff ff ff 0f 23 c8 e9 4f ff ff ff 0f 23 d0 e9 47 ff ff ff 48 c7 c7 d8 cd 2b 82 89 45 d0 c6 05 d4 bc 4f 01 01 e8 0a b8 09 00 <0f> 0b 8b 45 d0 e9 ed fe ff ff 0f 1f 44 00 00 55 48 89 e5 41 57 41 RSP: 0018:ffffc900006dfa48 EFLAGS: 00010082 RAX: 0000000000000000 RBX: ffff88821fb17f78 RCX: 0000000000000000 RDX: ffffffff81148b59 RSI: 0000000000000001 RDI: ffffffff81148bb5 RBP: ffffc900006dfa78 R08: 0000000000000000 R09: 0000000000000000 R10: ffffc900006dfaa0 R11: 0000000000000981 R12: 0000000000000003 R13: ffff88821206b800 R14: 0000000000017f80 R15: 0000000000000004 FS: 00007f46d1a54440(0000) GS:ffff88821fb00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f46d1590450 CR3: 000000020f1c8004 CR4: 00000000001706e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 00000000999906aa Call Trace: hw_breakpoint_add+0x44/0x50 event_sched_in.isra.119+0xab/0x240 group_sched_in+0x60/0x150 pinned_sched_in+0x85/0x170 ? group_sched_in+0x150/0x150 visit_groups_merge+0x10a/0x180 ctx_sched_in.isra.89+0x12b/0x150 perf_event_sched_in.isra.91+0x2f/0x70 ctx_resched+0x54/0x90 __perf_install_in_context+0x144/0x170 ? perf_duration_warn+0x40/0x40 remote_function+0x4a/0x60 generic_exec_single+0xad/0x110 ? perf_duration_warn+0x40/0x40 smp_call_function_single+0x9c/0x150 ? perf_duration_warn+0x40/0x40 task_function_call+0x4b/0x80 ? task_function_call+0x4b/0x80 ? __perf_event_enable+0x140/0x140 perf_install_in_context+0x99/0x160 __do_sys_perf_event_open+0xaf5/0xd80 __x64_sys_perf_event_open+0x20/0x30 do_syscall_64+0x4f/0x1b0 entry_SYSCALL_64_after_hwframe+0x49/0xbe RIP: 0033:0x7f46d1590469 Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d ff 49 2b 00 f7 d8 64 89 01 48 RSP: 002b:00007ffc86f0f488 EFLAGS: 00000246 ORIG_RAX: 000000000000012a RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f46d1590469 RDX: 00000000ffffffff RSI: 0000000000000000 RDI: 0000564f0c3a0080 RBP: 00007ffc86f0f4b0 R08: 0000000000000000 R09: 00007ffc86f0f598 R10: 00000000ffffffff R11: 0000000000000246 R12: 0000564f0c19f7e0 R13: 00007ffc86f0f590 R14: 0000000000000000 R15: 0000000000000000 irq event stamp: 12252 hardirqs last enabled at (12251): [<ffffffff81282909>] kmem_cache_alloc+0xb9/0x6e0 hardirqs last disabled at (12252): [<ffffffff81184637>] generic_exec_single+0xa7/0x110 softirqs last enabled at (12234): [<ffffffff81e003e7>] __do_softirq+0x3e7/0x497 softirqs last disabled at (12227): [<ffffffff810d4558>] irq_exit+0xc8/0xd0
We can't easily support dynamic event context swap when breakpoints are involved. The easiest solution to fix the issue is to pin a context as long as it contains a breakpoint PMU.
--- kernel/events/hw_breakpoint.c | 1 + 1 file changed, 1 insertion(+)
kernel/events/hw_breakpoint.c +1 -0
@@ -656,6 +656,7 @@ static struct pmu perf_breakpoint = { .start = hw_breakpoint_start, .stop = hw_breakpoint_stop, .read = hw_breakpoint_pmu_read, + .pin_ctx = 1, };int __init init_hw_breakpoint (void )
Signature · 1 line
2.21.0
Peter Zijlstra
maintainerQuoted 9 lines · Frederic Weisbecker
Syzbot has found a breakpoint overcommit issue:
https://lore.kernel.org/lkml/[email protected]/
It took me a long time to find out what the actual root problem was. Also its reproducer only worked on a few month old kernel but it didn't feel like the issue was actually solved.
Quoted 8 lines
I eventually cooked a reproducer that works with latest upstream, see in the end of this message.
The fix is just a few liner but implies to shut down the context swapping optimization for contexts containing breakpoints.
Also I feel like uprobes may be concerned as well as it seems to make use of
event.hw->targetafterpmu::init().
Can't we simply swizzle event.hw->target along too?
Quoted 13 lines · Peter Zijlstra
Quoted 12 lines · Frederic Weisbecker
Syzbot has found a breakpoint overcommit issue:
https://lore.kernel.org/lkml/[email protected]/
It took me a long time to find out what the actual root problem was. Also its reproducer only worked on a few month old kernel but it didn't feel like the issue was actually solved.
I eventually cooked a reproducer that works with latest upstream, see in the end of this message.
Quoted 7 lines
Quoted 5 lines
The fix is just a few liner but implies to shut down the context swapping optimization for contexts containing breakpoints.
Also I feel like uprobes may be concerned as well as it seems to make use of
event.hw->targetafterpmu::init().Can't we simply swizzle
event.hw->targetalong too?
You mean remove it? But it's still needed by breakpoint code during all the event lifecycle (init, destroy and anytime in-between).
I wish we could use event->ctx->task instead but on pmu::init() there is no ctx yet (we could pass the task in parameter though) and on event->destroy() it's TASK_TOMBSTONE and retrieving the task at that time would be non trivial.
Peter Zijlstra
maintainerQuoted 17 lines · Frederic Weisbecker
Quoted 16 lines · Peter Zijlstra
Quoted 15 lines · Frederic Weisbecker
Syzbot has found a breakpoint overcommit issue:
https://lore.kernel.org/lkml/[email protected]/
It took me a long time to find out what the actual root problem was. Also its reproducer only worked on a few month old kernel but it didn't feel like the issue was actually solved.
I eventually cooked a reproducer that works with latest upstream, see in the end of this message.
The fix is just a few liner but implies to shut down the context swapping optimization for contexts containing breakpoints.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Also I feel like uprobes may be concerned as well as it seems to make use of
event.hw->targetafterpmu::init().Can't we simply swizzle
event.hw->targetalong too?You mean remove it? But it's still needed by breakpoint code during all the event lifecycle (init, destroy and anytime in-between).
No, I meant flip hw->target when we flip the context. It would mean iterating the events, which I suppose would suck.
Quoted 2 lines
I wish we could use
event->ctx->taskinstead but onpmu::init()there is no ctx yet (we could pass the task in parameter though)
Right, that should be fairly easy.
Quoted 2 lines
and on
event->destroy()it'sTASK_TOMBSTONEand retrieving the task at that time would be non trivial.
Well, right, we can maybe make TOMBSTONE be the LSB instead of the whole word, then we can recover the task pointer... *yuck* though.
Peter Zijlstra
maintainerQuoted 10 lines · Peter Zijlstra
Quoted 2 lines
I wish we could use
event->ctx->taskinstead but onpmu::init()there is no ctx yet (we could pass the task in parameter though)Right, that should be fairly easy.
Quoted 2 lines
and on
event->destroy()it'sTASK_TOMBSTONEand retrieving the task at that time would be non trivial.Well, right, we can maybe make TOMBSTONE be the LSB instead of the whole word, then we can recover the task pointer... *yuck* though.
Something like the attached, completely untested patches.
I didn't do the hw_breakpoint bit, because I got lost in that, but this basically provides what you asked for I think.
Subject: perf: Clarify perf_event_exit_task_context()
From: Peter Zijlstra <[email protected]>
Date: Mon Jul 15 13:39:03 CEST 2019
The variables in perf_event_exit_task_context() are all called 'child' even though it must be current, this is confusing, apply:
s/child/task/
--- kernel/events/core.c | 38 +++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 19 deletions(-)
kernel/events/core.c +19 -19
@@ -11434,15 +11434,15 @@ perf_event_exit_event(struct perf_eventput_event (parent_event); } -static void perf_event_exit_task_context (struct task_struct *child,int ctxn) +static void perf_event_exit_task_context (struct task_struct *task,int ctxn) { -struct perf_event_context *child_ctx, *clone_ctx =NULL ; -struct perf_event *child_event, *next; +struct perf_event_context *task_ctx, *clone_ctx =NULL ; +struct perf_event *task_event, *next; -WARN_ON_ONCE (child != current); +WARN_ON_ONCE (task != current); - child_ctx =perf_pin_task_context (child, ctxn); -if (!child_ctx) + task_ctx =perf_pin_task_context (task, ctxn); +if (!task_ctx)return ;/*
@@ -11455,27 +11455,27 @@ static void perf_event_exit_task_context* without ctx::mutex (it cannot because of the move_group double mutex * lock thing). See the comments in perf_install_in_context(). */ -mutex_lock (&child_ctx->mutex); +mutex_lock (&task_ctx->mutex);/* * In a single ctx::lock section, de-schedule the events and detach the * context from the task such that we cannot ever get it scheduled back * in. */ -raw_spin_lock_irq (&child_ctx->lock); -task_ctx_sched_out (__get_cpu_context (child_ctx), child_ctx,EVENT_ALL ); +raw_spin_lock_irq (&task_ctx->lock); +task_ctx_sched_out (__get_cpu_context (task_ctx), task_ctx,EVENT_ALL );/* * Now that the context is inactive, destroy the task <-> ctx relation * and mark the context dead. */ -RCU_INIT_POINTER (child->perf_event_ctxp[ctxn],NULL ); -put_ctx (child_ctx);/* cannot be last */ -WRITE_ONCE (child_ctx->task,TASK_TOMBSTONE ); +RCU_INIT_POINTER (task->perf_event_ctxp[ctxn],NULL ); +put_ctx (task_ctx);/* cannot be last */ +WRITE_ONCE (task_ctx->task,TASK_TOMBSTONE );put_task_struct (current);/* cannot be last */ - clone_ctx =unclone_ctx (child_ctx); -raw_spin_unlock_irq (&child_ctx->lock); + clone_ctx =unclone_ctx (task_ctx); +raw_spin_unlock_irq (&task_ctx->lock);if (clone_ctx)put_ctx (clone_ctx);
@@ -11485,14 +11485,14 @@ static void perf_event_exit_task_context* won't get any samples after PERF_RECORD_EXIT. We can however still * get a few PERF_RECORD_READ events. */ -perf_event_task (child, child_ctx,0 ); +perf_event_task (task, task_ctx,0 ); -list_for_each_entry_safe (child_event, next, &child_ctx->event_list, event_entry) -perf_event_exit_event (child_event, child_ctx, child); +list_for_each_entry_safe (task_event, next, &task_ctx->event_list, event_entry) +perf_event_exit_event (task_event, task_ctx, task); -mutex_unlock (&child_ctx->mutex); +mutex_unlock (&task_ctx->mutex); -put_ctx (child_ctx); +put_ctx (task_ctx); }/*
Subject: perf: Remove overload of TASK_TOMBSTONE
From: Peter Zijlstra <[email protected]>
Date: Mon Jul 15 13:40:02 CEST 2019
Currently TASK_TOMBSTONE is used for 2 different things.
--- kernel/events/core.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
kernel/events/core.c +3 -2
@@ -165,10 +165,11 @@ static void perf_ctx_unlock(struct perf_ }#define TASK_TOMBSTONE ((void *)-1L) +#define TASK_KERNEL ((void *)-1L)static bool is_kernel_event (struct perf_event *event) { -return READ_ONCE (event->owner) ==TASK_TOMBSTONE ; +return READ_ONCE (event->owner) ==TASK_KERNEL ; }/*
@@ -11239,7 +11240,7 @@ perf_event_create_kernel_counter(struct }/* Mark owner so we could distinguish it from user events. */ - event->owner =TASK_TOMBSTONE ; + event->owner =TASK_KERNEL ; ctx =find_get_context (event->pmu, task, event);if (IS_ERR (ctx)) {
Subject: perf: Remove TASK_TOMBSTONG
From: Peter Zijlstra <[email protected]>
Date: Mon Jul 15 13:42:35 CEST 2019
Instead of overwriting the entirely of ctx->task, only set the LSB to mark the ctx dead. This allows recovering the task pointer for fun and games later on.
--- kernel/events/core.c | 51 ++++++++++++++++++++++++++++++++------------------- 1 file changed, 32 insertions(+), 19 deletions(-)
kernel/events/core.c +32 -19
154 lines not on this page · raw message
Subject: perf: Delay put_task_struct() for 'dead' contexts
From: Peter Zijlstra <[email protected]>
Date: Mon Jul 15 13:50:05 CEST 2019
Currently we immediately do put_task_struct() when we mark a context dead, instead delay it until after the final put_ctx() by recovering the task pointer.
--- kernel/events/core.c | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-)
kernel/events/core.c +12 -6
@@ -180,10 +180,17 @@ static inline void mark_dead_task_ctx(st {unsigned long task = (unsigned long )READ_ONCE (ctx->task);WARN_ON_ONCE (!task); - task |=1 ; + task |=1 ;;WRITE_ONCE (ctx->task, (struct task_struct *)task); } +static inline struct task_struct *__dead_ctx_task (struct perf_event_context *ctx) +{ +unsigned long task = (unsigned long )READ_ONCE (ctx->task); + task &= ~1L ; +return (struct task_struct *)task; +} +/* * On task ctx scheduling... *
@@ -1200,10 +1207,11 @@ static void free_ctx(struct rcu_head *hestatic void put_ctx (struct perf_event_context *ctx) {if (refcount_dec_and_test (&ctx->refcount)) { +struct task_struct *task =__dead_ctx_task (ctx);if (ctx->parent_ctx)put_ctx (ctx->parent_ctx); -if (ctx->task && !is_dead_task (ctx->task)) -put_task_struct (ctx->task); +if (task) +put_task_struct (task);call_rcu (&ctx->rcu_head, free_ctx); } }
@@ -11484,9 +11492,8 @@ static void perf_event_exit_task_context* and mark the context dead. */ RCU_INIT_POINTER (task->perf_event_ctxp[ctxn],NULL ); -put_ctx (task_ctx);/* cannot be last */ +put_ctx (task_ctx);/* matches perf_pin_task_context(), cannot be last */ mark_dead_task_ctx (task_ctx); -put_task_struct (current);/* cannot be last */ clone_ctx =unclone_ctx (task_ctx);raw_spin_unlock_irq (&task_ctx->lock);
@@ -11595,7 +11602,6 @@ void perf_event_free_task(struct task_st*/ RCU_INIT_POINTER (task->perf_event_ctxp[ctxn],NULL );mark_dead_task_ctx (ctx); -put_task_struct (task);/* cannot be last */ raw_spin_unlock_irq (&ctx->lock);list_for_each_entry_safe (event, tmp, &ctx->event_list, event_entry)
Subject: perf: Pass @task to pmu->event_init() and event->destroy()
From: Peter Zijlstra <[email protected]>
Date: Mon Jul 15 14:16:58 CEST 2019
Change the function signature of pmu->event_init() and event->destroy() to take an additional task pointer.
On pmu->event_init() we can trivially provide it, on event->destroy() we need to play games to recover it from ctx->task.
This should allow removal of event->hw.target.
--- arch/alpha/kernel/perf_event.c | 4 - arch/arc/kernel/perf_event.c | 2 arch/arm/mach-imx/mmdc.c | 2 arch/arm/mm/cache-l2x0-pmu.c | 2 arch/csky/kernel/perf_event.c | 2 arch/mips/kernel/perf_event_mipsxx.c | 4 - arch/nds32/kernel/perf_event_cpu.c | 2 arch/powerpc/perf/8xx-pmu.c | 2 arch/powerpc/perf/core-book3s.c | 4 - arch/powerpc/perf/core-fsl-emb.c | 4 - arch/powerpc/perf/hv-24x7.c | 4 - arch/powerpc/perf/hv-gpci.c | 2 arch/powerpc/perf/imc-pmu.c | 12 ++--- arch/riscv/kernel/perf_event.c | 6 +- arch/s390/kernel/perf_cpum_cf.c | 6 +- arch/s390/kernel/perf_cpum_cf_diag.c | 6 +- arch/s390/kernel/perf_cpum_sf.c | 6 +- arch/sh/kernel/perf_event.c | 6 +- arch/sparc/kernel/perf_event.c | 4 - arch/x86/events/amd/iommu.c | 2 arch/x86/events/amd/power.c | 2 arch/x86/events/amd/uncore.c | 2 arch/x86/events/core.c | 10 ++-- arch/x86/events/intel/bts.c | 4 - arch/x86/events/intel/cstate.c | 2 arch/x86/events/intel/uncore_snb.c | 2 arch/x86/events/msr.c | 2 arch/x86/events/perf_event.h | 2 arch/xtensa/kernel/perf_event.c | 2 drivers/hwtracing/coresight/coresight-etm-perf.c | 4 - drivers/perf/arm-cci.c | 6 +- drivers/perf/arm-ccn.c | 2 drivers/perf/arm_dsu_pmu.c | 2 drivers/perf/arm_pmu.c | 4 - drivers/perf/arm_smmuv3_pmu.c | 2 drivers/perf/fsl_imx8_ddr_perf.c | 2 drivers/perf/hisilicon/hisi_uncore_pmu.c | 2 drivers/perf/hisilicon/hisi_uncore_pmu.h | 2 drivers/perf/qcom_l2_pmu.c | 2 drivers/perf/qcom_l3_pmu.c | 2 drivers/perf/thunderx2_pmu.c | 2 drivers/perf/xgene_pmu.c | 2 include/linux/perf_event.h | 4 - include/linux/trace_events.h | 4 - kernel/events/core.c | 48 ++++++++++++----------- kernel/trace/trace_event_perf.c | 4 - 46 files changed, 104 insertions(+), 100 deletions(-)
arch/alpha/kernel/perf_event.c +2 -2
@@ -591,7 +591,7 @@ static int supported_cpu(void) -static void hw_perf_event_destroy (struct perf_event *event) +static void hw_perf_event_destroy (struct perf_event *event,struct task_struct *task) {/* Nothing to be done! */ return ;
@@ -687,7 +687,7 @@ static int __hw_perf_event_init(struct p/* * Main entry point to initialise a HW performance event. */ -static int alpha_pmu_event_init (struct perf_event *event) +static int alpha_pmu_event_init (struct perf_event *event,struct task_struct *task) {int err;
arch/arc/kernel/perf_event.c +1 -1
@@ -164,7 +164,7 @@ static int arc_pmu_cache_event(u64 confi }/* initializes hw_perf_event structure if event is supported */ -static int arc_pmu_event_init (struct perf_event *event) +static int arc_pmu_event_init (struct perf_event *event,struct task_struct *task) {struct hw_perf_event *hwc = &event->hw;int ret;
arch/arm/mach-imx/mmdc.c +1 -1
@@ -272,7 +272,7 @@ static bool mmdc_pmu_group_is_valid(strureturn true; } -static int mmdc_pmu_event_init (struct perf_event *event) +static int mmdc_pmu_event_init (struct perf_event *event,struct task_struct *task) {struct mmdc_pmu *pmu_mmdc =to_mmdc_pmu (event->pmu);int cfg = event->attr.config;
arch/arm/mm/cache-l2x0-pmu.c +1 -1
@@ -291,7 +291,7 @@ static bool l2x0_pmu_group_is_valid(strureturn num_hw <=PMU_NR_COUNTERS ; } -static int l2x0_pmu_event_init (struct perf_event *event) +static int l2x0_pmu_event_init (struct perf_event *event,struct task_struct *task) {struct hw_perf_event *hw = &event->hw;
arch/csky/kernel/perf_event.c +1 -1
@@ -839,7 +839,7 @@ static int csky_pmu_cache_event(u64 confreturn csky_pmu_cache_map[cache_type][cache_op][cache_result]; } -static int csky_pmu_event_init (struct perf_event *event) +static int csky_pmu_event_init (struct perf_event *event,struct task_struct *task) {struct hw_perf_event *hwc = &event->hw;int ret;
arch/mips/kernel/perf_event_mipsxx.c +2 -2
@@ -591,7 +591,7 @@ static void mipspmu_free_irq(void)static void reset_counters (void *arg);static int __hw_perf_event_init (struct perf_event *event); -static void hw_perf_event_destroy (struct perf_event *event) +static void hw_perf_event_destroy (struct perf_event *event,struct task_struct *task) {if (atomic_dec_and_mutex_lock (&active_events, &pmu_reserve_mutex)) {
@@ -606,7 +606,7 @@ static void hw_perf_event_destroy(struct } } -static int mipspmu_event_init (struct perf_event *event) +static int mipspmu_event_init (struct perf_event *event,struct task_Struct *task) {int err =0 ;
arch/nds32/kernel/perf_event_cpu.c +1 -1
@@ -854,7 +854,7 @@ static int __hw_perf_event_init(struct preturn 0 ; } -static int nds32_pmu_event_init (struct perf_event *event) +static int nds32_pmu_event_init (struct perf_event *event,struct task_struct *task) {struct nds32_pmu *nds32_pmu =to_nds32_pmu (event->pmu);int err =0 ;
arch/powerpc/perf/8xx-pmu.c +1 -1
@@ -68,7 +68,7 @@ static int event_type(struct perf_eventreturn -EOPNOTSUPP ; } -static int mpc8xx_pmu_event_init (struct perf_event *event) +static int mpc8xx_pmu_event_init (struct perf_event *event,struct task_struct *task) {int type =event_type (event);
arch/powerpc/perf/core-book3s.c +2 -2
@@ -1784,7 +1784,7 @@ static DEFINE_MUTEX(pmc_reserve_mutex);/* * Release the PMU if this is the last perf_event. */ -static void hw_perf_event_destroy (struct perf_event *event) +static void hw_perf_event_destroy (struct perf_event *event,struct task_struct *task) {if (!atomic_add_unless (&num_events,-1 ,1 )) {mutex_lock (&pmc_reserve_mutex);
@@ -1836,7 +1836,7 @@ static bool is_event_blacklisted(u64 ev)return false; } -static int power_pmu_event_init (struct perf_event *event) +static int power_pmu_event_init (struct perf_event *event,struct task_struct *task) {u64 ev;unsigned long flags;
arch/powerpc/perf/core-fsl-emb.c +2 -2
@@ -439,7 +439,7 @@ static void fsl_emb_pmu_stop(struct perf/* * Release the PMU if this is the last perf_event. */ -static void hw_perf_event_destroy (struct perf_event *event) +static void hw_perf_event_destroy (struct perf_event *event,struct task_struct *task) {if (!atomic_add_unless (&num_events,-1 ,1 )) {mutex_lock (&pmc_reserve_mutex);
@@ -479,7 +479,7 @@ static int hw_perf_cache_event(u64 confireturn 0 ; } -static int fsl_emb_pmu_event_init (struct perf_event *event) +static int fsl_emb_pmu_event_init (struct perf_event *event,struct task_struct *task) {u64 ev;struct perf_event *events[MAX_HWEVENTS ];
arch/powerpc/perf/hv-24x7.c +2 -2
@@ -603,7 +603,7 @@ static int event_uniq_add(struct rb_rootreturn 0 ; } -static void event_uniq_destroy (struct rb_root *root) +static void event_uniq_destroy (struct rb_root *root,struct task_struct *task) {/* * the strings we point to are in the giant block of memory filled by
@@ -1277,7 +1277,7 @@ static int single_24x7_request(struct pe } -static int h_24x7_event_init (struct perf_event *event) +static int h_24x7_event_init (struct perf_event *event,struct task_struct *task) {struct hv_perf_caps caps;unsigned domain;
arch/powerpc/perf/hv-gpci.c +1 -1
@@ -213,7 +213,7 @@ static int h_gpci_event_add(struct perf_return 0 ; } -static int h_gpci_event_init (struct perf_event *event) +static int h_gpci_event_init (struct perf_event *event,struct task_struct *task) {u64 count;u8 length;
arch/powerpc/perf/imc-pmu.c +6 -6
@@ -432,7 +432,7 @@ static int nest_pmu_cpumask_init(void) ppc_nest_imc_cpu_offline); } -static void nest_imc_counters_release (struct perf_event *event) +static void nest_imc_counters_release (struct perf_event *event,struct task_struct *task) {int rc, node_id;struct imc_pmu_ref *ref;
@@ -484,7 +484,7 @@ static void nest_imc_counters_release(stmutex_unlock (&ref->lock); } -static int nest_imc_event_init (struct perf_event *event) +static int nest_imc_event_init (struct perf_event *event,struct task_struct *task) {int chip_id, rc, node_id;u32 l_config, config = event->attr.config;
@@ -708,7 +708,7 @@ static int core_imc_pmu_cpumask_init(voi ppc_core_imc_cpu_offline); } -static void core_imc_counters_release (struct perf_event *event) +static void core_imc_counters_release (struct perf_event *event,struct task_struct *task) {int rc, core_id;struct imc_pmu_ref *ref;
@@ -759,7 +759,7 @@ static void core_imc_counters_release(stmutex_unlock (&ref->lock); } -static int core_imc_event_init (struct perf_event *event) +static int core_imc_event_init (struct perf_event *event,struct task_struct *task) {int core_id, rc;u64 config = event->attr.config;
@@ -885,7 +885,7 @@ static int thread_imc_cpu_init(void) ppc_thread_imc_cpu_offline); } -static int thread_imc_event_init (struct perf_event *event) +static int thread_imc_event_init (struct perf_event *event,struct task_struct *task) {u32 config = event->attr.config;struct task_struct *target;
@@ -1293,7 +1293,7 @@ static void trace_imc_event_del(struct ptrace_imc_event_stop (event, flags); } -static int trace_imc_event_init (struct perf_event *event) +static int trace_imc_event_init (struct perf_event *event,struct task_struct *task) {struct task_struct *target;
arch/riscv/kernel/perf_event.c +3 -3
@@ -375,13 +375,13 @@ void release_pmc_hardware(void)static atomic_t riscv_active_events =ATOMIC_INIT (0 ); -static void riscv_event_destroy (struct perf_event *event) +static void riscv_event_destroy (struct perf_event *event,struct task_struct *task) {if (atomic_dec_return (&riscv_active_events) ==0 )release_pmc_hardware (); } -static int riscv_event_init (struct perf_event *event) +static int riscv_event_init (struct perf_event *event,struct task_struct *task) {struct perf_event_attr *attr = &event->attr;struct hw_perf_event *hwc = &event->hw;
@@ -413,7 +413,7 @@ static int riscv_event_init(struct perf_ event->destroy = riscv_event_destroy;if (code <0 ) { - event->destroy (event); + event->destroy (event, task);return code; }
arch/s390/kernel/perf_cpum_cf.c +3 -3
@@ -168,7 +168,7 @@ static atomic_t num_events = ATOMIC_INITstatic DEFINE_MUTEX (pmc_reserve_mutex);/* Release the PMU if event is the last perf event */ -static void hw_perf_event_destroy (struct perf_event *event) +static void hw_perf_event_destroy (struct perf_event *event,struct task_struct *task) {if (!atomic_add_unless (&num_events,-1 ,1 )) {mutex_lock (&pmc_reserve_mutex);
@@ -292,7 +292,7 @@ static int __hw_perf_event_init(struct preturn err; } -static int cpumf_pmu_event_init (struct perf_event *event) +static int cpumf_pmu_event_init (struct perf_event *event,struct task_struct *task) {int err;
@@ -307,7 +307,7 @@ static int cpumf_pmu_event_init(struct p }if (unlikely (err) && event->destroy) - event->destroy (event); + event->destroy (event, task);return err; }
arch/s390/kernel/perf_cpum_cf_diag.c +3 -3
@@ -180,7 +180,7 @@ static void cf_diag_disable(struct pmu *static atomic_t cf_diag_events =ATOMIC_INIT (0 );/* Release the PMU if event is the last perf event */ -static void cf_diag_perf_event_destroy (struct perf_event *event) +static void cf_diag_perf_event_destroy (struct perf_event *event,struct task_struct *task) { debug_sprintf_event(cf_diag_dbg, 5, "%s event %p cpu %d cf_diag_events %d\n",
@@ -237,7 +237,7 @@ static int __hw_perf_event_init(struct preturn err; } -static int cf_diag_event_init (struct perf_event *event) +static int cf_diag_event_init (struct perf_event *event,struct task_struct *task) {struct perf_event_attr *attr = &event->attr;int err = -ENOENT ;
@@ -275,7 +275,7 @@ static int cf_diag_event_init(struct per err =__hw_perf_event_init (event);if (unlikely (err)) - event->destroy (event); + event->destroy (event, task); out:debug_sprintf_event (cf_diag_dbg,5 ,"%s err %d\n" , __func__, err);return err;
arch/s390/kernel/perf_cpum_sf.c +3 -3
@@ -581,7 +581,7 @@ static int reserve_pmc_hardware(void)return 0 ; } -static void hw_perf_event_destroy (struct perf_event *event) +static void hw_perf_event_destroy (struct perf_event *event,struct task_struct *task) {/* Release PMC if this is the last perf event */ if (!atomic_add_unless (&num_events,-1 ,1 )) {
@@ -823,7 +823,7 @@ static int __hw_perf_event_init(struct preturn err; } -static int cpumsf_pmu_event_init (struct perf_event *event) +static int cpumsf_pmu_event_init (struct perf_event *event,struct task_struct *task) {int err;
@@ -867,7 +867,7 @@ static int cpumsf_pmu_event_init(struct err =__hw_perf_event_init (event);if (unlikely (err))if (event->destroy) - event->destroy (event); + event->destroy (event, task);return err; }
arch/sh/kernel/perf_event.c +3 -3
@@ -78,7 +78,7 @@ EXPORT_SYMBOL_GPL(perf_num_counters);/* * Release the PMU if this is the last perf_event. */ -static void hw_perf_event_destroy (struct perf_event *event) +static void hw_perf_event_destroy (struct perf_event *event,struct task_struct *task) {if (!atomic_add_unless (&num_events,-1 ,1 )) {mutex_lock (&pmc_reserve_mutex);
@@ -295,7 +295,7 @@ static void sh_pmu_read(struct perf_evensh_perf_event_update (event, &event->hw, event->hw.idx); } -static int sh_pmu_event_init (struct perf_event *event) +static int sh_pmu_event_init (struct perf_event *event,struct task_struct *task) {int err;
@@ -316,7 +316,7 @@ static int sh_pmu_event_init(struct perfif (unlikely (err)) {if (event->destroy) - event->destroy (event); + event->destroy (event, task); }return err;
arch/sparc/kernel/perf_event.c +2 -2
@@ -1224,7 +1224,7 @@ static const struct perf_event_map *sparreturn pmap; } -static void hw_perf_event_destroy (struct perf_event *event) +static void hw_perf_event_destroy (struct perf_event *event,struct task_struct *task) {perf_event_release_pmc (); }
@@ -1412,7 +1412,7 @@ static int sparc_pmu_add(struct perf_evereturn ret; } -static int sparc_pmu_event_init (struct perf_event *event) +static int sparc_pmu_event_init (struct perf_event *event,struct task_struct *task) {struct perf_event_attr *attr = &event->attr;struct perf_event *evts[MAX_HWEVENTS ];
arch/x86/events/amd/iommu.c +1 -1
@@ -204,7 +204,7 @@ static int clear_avail_iommu_bnk_cntr(streturn 0 ; } -static int perf_iommu_event_init (struct perf_event *event) +static int perf_iommu_event_init (struct perf_event *event,struct task_struct *task) {struct hw_perf_event *hwc = &event->hw;
arch/x86/events/amd/power.c +1 -1
@@ -124,7 +124,7 @@ static void pmu_event_del(struct perf_evpmu_event_stop (event,PERF_EF_UPDATE ); } -static int pmu_event_init (struct perf_event *event) +static int pmu_event_init (struct perf_event *event,struct task_struct *task) {u64 cfg = event->attr.config &AMD_POWER_EVENT_MASK ;
arch/x86/events/amd/uncore.c +1 -1
@@ -180,7 +180,7 @@ static void amd_uncore_del(struct perf_e hwc->idx =-1 ; } -static int amd_uncore_event_init (struct perf_event *event) +static int amd_uncore_event_init (struct perf_event *event,struct task_struct *task) {struct amd_uncore *uncore;struct hw_perf_event *hwc = &event->hw;
arch/x86/events/core.c +5 -5
@@ -277,15 +277,15 @@ static bool check_hw_exists(void)return false; } -static void hw_perf_event_destroy (struct perf_event *event) +static void hw_perf_event_destroy (struct perf_event *event,struct task_struct *task) {x86_release_hardware ();atomic_dec (&active_events); } -void hw_perf_lbr_event_destroy (struct perf_event *event) +void hw_perf_lbr_event_destroy (struct perf_event *event,struct task_struct *task) { -hw_perf_event_destroy (event); +hw_perf_event_destroy (event, task);/* undo the lbr/bts event accounting */ x86_del_exclusive (x86_lbr_exclusive_lbr);
@@ -2041,7 +2041,7 @@ static int validate_group(struct perf_evreturn ret; } -static int x86_pmu_event_init (struct perf_event *event) +static int x86_pmu_event_init (struct perf_event *event,struct task_struct *task) {struct pmu *tmp;int err;
@@ -2075,7 +2075,7 @@ static int x86_pmu_event_init(struct per }if (err) {if (event->destroy) - event->destroy (event); + event->destroy (event, task); } if (READ_ONCE(x86_pmu.attr_rdpmc) &&
arch/x86/events/intel/bts.c +2 -2
@@ -527,13 +527,13 @@ static int bts_event_add(struct perf_evereturn 0 ; } -static void bts_event_destroy (struct perf_event *event) +static void bts_event_destroy (struct perf_event *event,struct task_struct *task) {x86_release_hardware ();x86_del_exclusive (x86_lbr_exclusive_bts); } -static int bts_event_init (struct perf_event *event) +static int bts_event_init (struct perf_event *event,struct task_struct *task) {int ret;
arch/x86/events/intel/cstate.c +1 -1
@@ -295,7 +295,7 @@ static ssize_t cstate_get_attr_cpumask(sreturn 0 ; } -static int cstate_pmu_event_init (struct perf_event *event) +static int cstate_pmu_event_init (struct perf_event *event,struct task_struct *task) {u64 cfg = event->attr.config;int cpu;
arch/x86/events/intel/uncore_snb.c +1 -1
@@ -444,7 +444,7 @@ static void snb_uncore_imc_disable_event* Keep the custom event_init() function compatible with old event * encoding for free running counters. */ -static int snb_uncore_imc_event_init (struct perf_event *event) +static int snb_uncore_imc_event_init (struct perf_event *event,struct task_struct *task) {struct intel_uncore_pmu *pmu;struct intel_uncore_box *box;
arch/x86/events/msr.c +1 -1
@@ -178,7 +178,7 @@ const struct attribute_group *attr_updat NULL, }; -static int msr_event_init (struct perf_event *event) +static int msr_event_init (struct perf_event *event,struct task_struct *task) {u64 cfg = event->attr.config;
arch/x86/events/perf_event.h +1 -1
@@ -810,7 +810,7 @@ void x86_release_hardware(void);int x86_pmu_max_precise (void ); -void hw_perf_lbr_event_destroy (struct perf_event *event); +void hw_perf_lbr_event_destroy (struct perf_event *event,struct task_struct *task);int x86_setup_perfctr (struct perf_event *event);
arch/xtensa/kernel/perf_event.c +1 -1
@@ -195,7 +195,7 @@ static void xtensa_pmu_disable(struct pmset_er (get_er (XTENSA_PMU_PMG ) & ~XTENSA_PMU_PMG_PMEN ,XTENSA_PMU_PMG ); } -static int xtensa_pmu_event_init (struct perf_event *event) +static int xtensa_pmu_event_init (struct perf_event *event,struct task_struct *task) {int ret;
drivers/hwtracing/coresight/coresight-etm-perf.c +2 -2
@@ -96,13 +96,13 @@ static int etm_addr_filters_alloc(structreturn 0 ; } -static void etm_event_destroy (struct perf_event *event) +static void etm_event_destroy (struct perf_event *event,struct task_struct *task) {kfree (event->hw.addr_filters); event->hw.addr_filters =NULL ; } -static int etm_event_init (struct perf_event *event) +static int etm_event_init (struct perf_event *event,struct task_struct *task) {int ret =0 ;
drivers/perf/arm-cci.c +3 -3
@@ -1081,7 +1081,7 @@ static void cci_pmu_put_hw(struct cci_pmpmu_free_irq (cci_pmu); } -static void hw_perf_event_destroy (struct perf_event *event) +static void hw_perf_event_destroy (struct perf_event *event,struct task_struct *task) {struct cci_pmu *cci_pmu =to_cci_pmu (event->pmu);atomic_t *active_events = &cci_pmu->active_events;
@@ -1314,7 +1314,7 @@ static int __hw_perf_event_init(struct preturn 0 ; } -static int cci_pmu_event_init (struct perf_event *event) +static int cci_pmu_event_init (struct perf_event *event,struct task_struct *task) {struct cci_pmu *cci_pmu =to_cci_pmu (event->pmu);atomic_t *active_events = &cci_pmu->active_events;
@@ -1354,7 +1354,7 @@ static int cci_pmu_event_init(struct per err =__hw_perf_event_init (event);if (err) -hw_perf_event_destroy (event); +hw_perf_event_destroy (event, task);return err; }
drivers/perf/arm-ccn.c +1 -1
@@ -715,7 +715,7 @@ static void arm_ccn_pmu_event_release(st ccn->dt.pmu_counters[hw->idx].event =NULL ; } -static int arm_ccn_pmu_event_init (struct perf_event *event) +static int arm_ccn_pmu_event_init (struct perf_event *event,struct task_struct *task) {struct arm_ccn *ccn;struct hw_perf_event *hw = &event->hw;
drivers/perf/arm_dsu_pmu.c +1 -1
@@ -540,7 +540,7 @@ static bool dsu_pmu_validate_group(strucreturn dsu_pmu_validate_event (event->pmu, &fake_hw, event); } -static int dsu_pmu_event_init (struct perf_event *event) +static int dsu_pmu_event_init (struct perf_event *event,struct task_struct *task) {struct dsu_pmu *dsu_pmu =to_dsu_pmu (event->pmu);
drivers/perf/arm_pmu.c +2 -2
@@ -419,7 +419,7 @@ __hw_perf_event_init(struct perf_event *return 0 ; } -static int armpmu_event_init (struct perf_event *event) +static int armpmu_event_init (struct perf_event *event,struct task_struct *task) {struct arm_pmu *armpmu =to_arm_pmu (event->pmu);
@@ -771,7 +771,7 @@ static int cpu_pmu_init(struct arm_pmu *return err; } -static void cpu_pmu_destroy (struct arm_pmu *cpu_pmu) +static void cpu_pmu_destroy (struct arm_pmu *cpu_pmu,struct task_struct *task) {cpu_pm_pmu_unregister (cpu_pmu); cpuhp_state_remove_instance_nocalls(CPUHP_AP_PERF_ARM_STARTING,
drivers/perf/arm_smmuv3_pmu.c +1 -1
@@ -317,7 +317,7 @@ static int smmu_pmu_get_event_idx(struct* the core perf events code. */ -static int smmu_pmu_event_init (struct perf_event *event) +static int smmu_pmu_event_init (struct perf_event *event,struct task_struct *task) {struct hw_perf_event *hwc = &event->hw;struct smmu_pmu *smmu_pmu =to_smmu_pmu (event->pmu);
drivers/perf/fsl_imx8_ddr_perf.c +1 -1
@@ -189,7 +189,7 @@ static u32 ddr_perf_read_counter(structreturn readl_relaxed (pmu->base +COUNTER_READ + counter *4 ); } -static int ddr_perf_event_init (struct perf_event *event) +static int ddr_perf_event_init (struct perf_event *event,struct task_struct *task) {struct ddr_pmu *pmu =to_ddr_pmu (event->pmu);struct hw_perf_event *hwc = &event->hw;
drivers/perf/hisilicon/hisi_uncore_pmu.c +1 -1
@@ -123,7 +123,7 @@ static void hisi_uncore_pmu_clear_event_clear_bit (idx, hisi_pmu->pmu_events.used_mask); } -int hisi_uncore_pmu_event_init (struct perf_event *event) +int hisi_uncore_pmu_event_init (struct perf_event *event,struct task_struct *task) {struct hw_perf_event *hwc = &event->hw;struct hisi_pmu *hisi_pmu;
drivers/perf/hisilicon/hisi_uncore_pmu.h +1 -1
@@ -85,7 +85,7 @@ void hisi_uncore_pmu_start(struct perf_evoid hisi_uncore_pmu_stop (struct perf_event *event,int flags);void hisi_uncore_pmu_set_event_period (struct perf_event *event);void hisi_uncore_pmu_event_update (struct perf_event *event); -int hisi_uncore_pmu_event_init (struct perf_event *event); +int hisi_uncore_pmu_event_init (struct perf_event *event,struct task_struct *task);void hisi_uncore_pmu_enable (struct pmu *pmu);void hisi_uncore_pmu_disable (struct pmu *pmu); ssize_t hisi_event_sysfs_show(struct device *dev,
drivers/perf/qcom_l2_pmu.c +1 -1
@@ -477,7 +477,7 @@ static void l2_cache_pmu_disable(structcluster_pmu_disable (); } -static int l2_cache_event_init (struct perf_event *event) +static int l2_cache_event_init (struct perf_event *event,struct task_struct *task) {struct hw_perf_event *hwc = &event->hw;struct cluster_pmu *cluster;
drivers/perf/qcom_l3_pmu.c +1 -1
@@ -475,7 +475,7 @@ static bool qcom_l3_cache__validate_evenreturn counters <=L3_NUM_COUNTERS ; } -static int qcom_l3_cache__event_init (struct perf_event *event) +static int qcom_l3_cache__event_init (struct perf_event *event,struct task_struct *task) {struct l3cache_pmu *l3pmu =to_l3cache_pmu (event->pmu);struct hw_perf_event *hwc = &event->hw;
drivers/perf/thunderx2_pmu.c +1 -1
@@ -407,7 +407,7 @@ static bool tx2_uncore_validate_event_gr } -static int tx2_uncore_event_init (struct perf_event *event) +static int tx2_uncore_event_init (struct perf_event *event,struct task_struct *task) {struct hw_perf_event *hwc = &event->hw;struct tx2_uncore_pmu *tx2_pmu;
drivers/perf/xgene_pmu.c +1 -1
@@ -887,7 +887,7 @@ static void xgene_perf_pmu_disable(struc xgene_pmu->ops->stop_counters (pmu_dev); } -static int xgene_perf_event_init (struct perf_event *event) +static int xgene_perf_event_init (struct perf_event *event,struct task_struct *task) {struct xgene_pmu_dev *pmu_dev =to_pmu_dev (event->pmu);struct hw_perf_event *hw = &event->hw;
include/linux/perf_event.h +2 -2
@@ -297,7 +297,7 @@ struct pmu {* * Other error return values are allowed. */ -int (*event_init) (struct perf_event *event); +int (*event_init) (struct perf_event *event,struct task_struct *task);/* * Notification that the event was mapped or unmapped. Called
@@ -681,7 +681,7 @@ struct perf_event {struct perf_addr_filter_range *addr_filter_ranges;unsigned long addr_filters_gen; -void (*destroy)(struct perf_event *); +void (*destroy)(struct perf_event *,struct task_struct *);struct rcu_head rcu_head;struct pid_namespace *ns;
include/linux/trace_events.h +2 -2
@@ -572,7 +572,7 @@ extern int perf_trace_add(struct perf_eextern void perf_trace_del (struct perf_event *event,int flags);#ifdef CONFIG_KPROBE_EVENTS extern int perf_kprobe_init (struct perf_event *event,bool is_retprobe); -extern void perf_kprobe_destroy (struct perf_event *event); +extern void perf_kprobe_destroy (struct perf_event *event,struct task_struct *task); extern int bpf_get_kprobe_info(const struct perf_event *event, u32 *fd_type, const char **symbol, u64 *probe_offset, u64 *probe_addr,
@@ -581,7 +581,7 @@ extern int bpf_get_kprobe_info(const str#ifdef CONFIG_UPROBE_EVENTS extern int perf_uprobe_init (struct perf_event *event,unsigned long ref_ctr_offset,bool is_retprobe); -extern void perf_uprobe_destroy (struct perf_event *event); +extern void perf_uprobe_destroy (struct perf_event *event,struct task_struct *task);extern int bpf_get_uprobe_info (const struct perf_event *event,u32 *fd_type,const char **filename,u64 *probe_offset,bool perf_type_tracepoint);
kernel/events/core.c +26 -22
172 lines not on this page · raw message
kernel/trace/trace_event_perf.c +2 -2
@@ -280,7 +280,7 @@ int perf_kprobe_init(struct perf_event *return ret; } -void perf_kprobe_destroy (struct perf_event *p_event) +void perf_kprobe_destroy (struct perf_event *p_event,struct task_struct *task) {perf_trace_event_close (p_event);perf_trace_event_unreg (p_event);
@@ -333,7 +333,7 @@ int perf_uprobe_init(struct perf_event *return ret; } -void perf_uprobe_destroy (struct perf_event *p_event) +void perf_uprobe_destroy (struct perf_event *p_event,struct task_struct *task) {mutex_lock (&event_mutex);perf_trace_event_close (p_event);
Subject: perf/power: Remove some simple hw.target usage
From: Peter Zijlstra <[email protected]>
Date: Mon Jul 15 14:34:17 CEST 2019
--- arch/powerpc/perf/imc-pmu.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-)
arch/powerpc/perf/imc-pmu.c +1 -6
@@ -888,7 +888,6 @@ static int thread_imc_cpu_init(void)static int thread_imc_event_init (struct perf_event *event,struct task_struct *task) {u32 config = event->attr.config; -struct task_struct *target;struct imc_pmu *pmu;if (event->attr.type != event->pmu->type)
@@ -908,8 +907,7 @@ static int thread_imc_event_init(structif (((config &IMC_EVENT_OFFSET_MASK ) > pmu->counter_mem_size))return -EINVAL ; - target = event->hw.target; -if (!target) +if (!task)return -EINVAL ; event->pmu->task_ctx_nr = perf_sw_context;
@@ -1295,8 +1293,6 @@ static void trace_imc_event_del(struct pstatic int trace_imc_event_init (struct perf_event *event,struct task_struct *task) { -struct task_struct *target; -if (event->attr.type != event->pmu->type)return -ENOENT ;
@@ -1308,7 +1304,6 @@ static int trace_imc_event_init(struct preturn -ENOENT ; event->hw.idx =-1 ; - target = event->hw.target; event->pmu->task_ctx_nr = perf_hw_context;return 0 ;
Quoted 9 lines · Peter Zijlstra
Quoted 8 lines · Peter Zijlstra
Quoted 2 lines
I wish we could use
event->ctx->taskinstead but onpmu::init()there is no ctx yet (we could pass the task in parameter though)Right, that should be fairly easy.
Quoted 2 lines
and on
event->destroy()it'sTASK_TOMBSTONEand retrieving the task at that time would be non trivial.
Quoted 8 lines
Quoted 2 lines
Well, right, we can maybe make TOMBSTONE be the LSB instead of the whole word, then we can recover the task pointer... *yuck* though.
Something like the attached, completely untested patches.
I didn't do the
hw_breakpointbit, because I got lost in that, but this basically provides what you asked for I think.
Thanks they look good! I can take them and work on top if you like.
Peter Zijlstra
maintainerQuoted 13 lines · Frederic Weisbecker
Quoted 12 lines · Peter Zijlstra
Quoted 10 lines · Peter Zijlstra
Quoted 2 lines
I wish we could use
event->ctx->taskinstead but onpmu::init()there is no ctx yet (we could pass the task in parameter though)Right, that should be fairly easy.
Quoted 2 lines
and on
event->destroy()it'sTASK_TOMBSTONEand retrieving the task at that time would be non trivial.Well, right, we can maybe make TOMBSTONE be the LSB instead of the whole word, then we can recover the task pointer... *yuck* though.
Quoted 7 lines
Quoted 5 lines
Something like the attached, completely untested patches.
I didn't do the
hw_breakpointbit, because I got lost in that, but this basically provides what you asked for I think.Thanks they look good! I can take them and work on top if you like.
please do.