Skip to contentSkip to conversation

perf: Sharing PMU counters across compatible events

PATCH Superseded Song Liu 1 participant 3 messages Oct 3, 2018 perf events

v4 of 13 Superseded Latest is v13 →

lore
Share & tools
lore

Song Liu

author
#0 lore raw

v4 cover perf: Sharing PMU counters across compatible events

Changes v3 -> v4:
1. In __perf_install_in_context() always reprogram ctx for cpu and cgroup events to make sure event duplications set up properly.
2. Add a prototype (2/2) to show potential solution of sharing PMU among CPU events and task events.

Changes v2 -> v3:
1. Simplified event_pmu_add() as Jiri Olsa suggested

Changes v1 -> v2:
1. Fixed a bug reported by 0day kernel testing robot

This is to follow up earlier discussion on sharing hardware PMU counters across compatible events:

https://marc.info/?t=151213803600016
https://marc.info/?t=152547569200001

This version limits PMU sharing to events within same ctx. As a result, compatible events are not evaluated during task schedule. This vesion also limits PMU sharing to hardware counting events. From our discussion with internal users, this is sufficient for most cases.

This version introduces virtual master event. The virtual master event does require careful handling. But it is makes the logic of event add/del cleaner.

Cc Tejun Heo <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Jiri Olsa <[email protected]>

Song Liu (2):
perf: Sharing PMU counters across compatible events perf tools: monitoring per task counter with per cgroup event

 include/linux/perf_event.h |  61 ++++++++
 kernel/events/core.c       | 294 ++++++++++++++++++++++++++++++++++---
 tools/perf/builtin-stat.c  |  26 ++++
 tools/perf/util/cgroup.c   |  76 ++++++++++
 tools/perf/util/cgroup.h   |   5 +
 tools/perf/util/target.h   |   1 +
 6 files changed, 440 insertions(+), 23 deletions(-)

--
2.17.1

author
#1 lore raw

PATCH v4 1/2 perf: Sharing PMU counters across compatible events

This patch tries to enable PMU sharing. To make perf event scheduling fast, we use special data structures.

An array of "struct perf_event_dup" is added to the perf_event_context, to remember all the duplicated events under this ctx. All the events under this ctx has a "dup_id" pointing to its perf_event_dup. Compatible events under the same ctx share the same perf_event_dup. The following figure shows a simplified version of the data structure.

      ctx ->  perf_event_dup -> master
                     ^
                     |
         perf_event /|
                     |
         perf_event /

Connection among perf_event and perf_event_dup are built when events are added or removed from the ctx. So these are not on the critical path of schedule or perf_rotate_context().

On the critical paths (add, del read), sharing PMU counters doesn't increase the complexity. Helper functions event_pmu_[add|del|read]() are introduced to cover these cases. All these functions have O(1) time complexity.

We allocate a separate perf_event for perf_event_dup->master. This needs extra attention, because perf_event_alloc() may sleep. To allocate the master event properly, a new pointer, tmp_master, is added to perf_event. tmp_master carries a separate perf_event into list_[add|del]_event(). The master event has valid ->ctx and holds ctx->refcount.

Details about the handling of the master event is added to include/linux/perf_event.h, before struct perf_event_dup.

Cc Tejun Heo <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Jiri Olsa <[email protected]>
---
 include/linux/perf_event.h |  61 ++++++++
 kernel/events/core.c       | 294 ++++++++++++++++++++++++++++++++++---
 2 files changed, 332 insertions(+), 23 deletions(-)
include/linux/perf_event.h +61 -0
index 53c500f0ca79..d092378a3eca 100644
@@ -695,6 +695,12 @@ struct perf_event {
 #endif
 
 	struct list_head		sb_list;
+
+	/* for PMU sharing */
+	struct perf_event		*tmp_master;
+	u64				dup_id;
+	u64				dup_base_count;
+	u64				dup_base_child_count;
 #endif /* CONFIG_PERF_EVENTS */
 };
 
@@ -704,6 +710,58 @@ struct perf_event_groups {
 	u64		index;
 };
 
+/*
+ * Sharing PMU across compatible events
+ *
+ * If two perf_events in the same perf_event_context are counting same
+ * hardware events (instructions, cycles, etc.), they could share the
+ * hardware PMU counter.
+ *
+ * When a perf_event is added to the ctx (list_add_event), it is compared
+ * against other events in the ctx. If they can share the PMU counter,
+ * a perf_event_dup is allocated to represent the sharing.
+ *
+ * Each perf_event_dup has a virtual master event, which is called by
+ * pmu->add() and pmu->del(). We cannot call perf_event_alloc() in
+ * list_add_event(), so it is allocated and carried by event->tmp_master
+ * into list_add_event().
+ *
+ * Virtual master in different cases/paths:
+ *
+ * < I > perf_event_open() -> close() path:
+ *
+ * 1. Allocated by perf_event_alloc() in sys_perf_event_open();
+ * 2. event->tmp_master->ctx assigned in perf_install_in_context();
+ * 3.a. if used by ctx->dup_events, freed in perf_event_release_kernel();
+ * 3.b. if not used by ctx->dup_events, freed in perf_event_open().
+ *
+ * < II > inherit_event() path:
+ *
+ * 1. Allocated by perf_event_alloc() in inherit_event();
+ * 2. tmp_master->ctx assigned in inherit_event();
+ * 3.a. if used by ctx->dup_events, freed in perf_event_release_kernel();
+ * 3.b. if not used by ctx->dup_events, freed in inherit_event().
+ *
+ * < III > perf_pmu_migrate_context() path:
+ * all dup_events removed during migration (no sharing after the move).
+ *
+ * < IV > perf_event_create_kernel_counter() path:
+ * not supported yet.
+ */
+struct perf_event_dup {
+	/*
+	 * master event being called by pmu->add() and pmu->del().
+	 * This event is allocated with perf_event_alloc(). When
+	 * attached to a ctx, this event should hold ctx->refcount.
+	 */
+	struct perf_event       *master;
+	/* number of events in the ctx that shares the master */
+	int			total_event_count;
+	/* number of active events of the master */
+	int			active_event_count;
+};
+
+#define MAX_PERF_EVENT_DUP_PER_CTX 4
 /**
  * struct perf_event_context - event context structure
  *
@@ -759,6 +817,9 @@ struct perf_event_context {
 #endif
 	void				*task_ctx_data; /* pmu specific data */
 	struct rcu_head			rcu_head;
+
+	/* for PMU sharing. array is needed for O(1) access */
+	struct perf_event_dup		dup_events[MAX_PERF_EVENT_DUP_PER_CTX];
 };
 
 /*
kernel/events/core.c +271 -23

466 lines not on this page · raw message

--
2.17.1

author
#2 lore raw

PATCH v4 2/2 perf tools: monitoring per task counter with per cgroup event

In reply to Song Liu

This is just a prototype.

Previous patch enables sharing hardware PMU among perf_events within same perf_event_context. This sharing comes with limitation that per CPU event cannot share hardware PMU with per task event. This limitation becomes a blocker when certain events could only use a specific PMU, for example, ref-cycles in some Intel CPUs. The following two commands will not share the PMU (when run in parallel):

   perf stat -e ref-cycles -I 1000
   perf stat -e ref-cycles -I 1000 --pid <pid>

This patch shows a prototype that solves this problem with cgroup events. With this patch, the following two commands can share the PMU:

   perf stat -e ref-cycles -I 1000
   perf stat -e ref-cycles -I 1000 --pid <pid> --create-cgroup

The second command creates a cgroup for the pid, and move the pid to that cgroup. Then, a cgroup event (instead of task event) is created to monitor the process.

Alternatively, we can also create a mechanism in the kernel that is very similar to cgroup perf events. I am also open to other suggestions.

Cc Tejun Heo <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Jiri Olsa <[email protected]>
---
 tools/perf/builtin-stat.c | 26 ++++++++++++++
 tools/perf/util/cgroup.c  | 76 +++++++++++++++++++++++++++++++++++++++
 tools/perf/util/cgroup.h  |  5 +++
 tools/perf/util/target.h  |  1 +
 4 files changed, 108 insertions(+)
tools/perf/builtin-stat.c +26 -0
index b86aba1c8028..66a4da2d506e 100644
@@ -646,6 +646,17 @@ static void print_counters(struct timespec *ts, int argc, const char **argv)
 				    ts, argc, argv);
 }
 
+static void cleanup(void)
+{
+	/* clean up cgroups */
+	if (target.create_cgroup) {
+		char name[32];
+
+		scnprintf(name, 31, "perf.%u", getpid());
+		cgroup__cleanup(name);
+	}
+}
+
 static volatile int signr = -1;
 
 static void skip_signal(int signo)
@@ -661,6 +672,7 @@ static void skip_signal(int signo)
 	 * and fast PID recycling
 	 */
 	child_pid = -1;
+	cleanup();
 }
 
 static void sig_atexit(void)
@@ -725,6 +737,8 @@ static const struct option stat_options[] = {
 		   "stat events on existing process id"),
 	OPT_STRING('t', "tid", &target.tid, "tid",
 		   "stat events on existing thread id"),
+	OPT_BOOLEAN(0, "create-cgroup", &target.create_cgroup,
+		    "create a cgroup for the pid/tid"),
 	OPT_BOOLEAN('a', "all-cpus", &target.system_wide,
 		    "system-wide collection from all CPUs"),
 	OPT_BOOLEAN('g', "group", &group,
@@ -1607,6 +1621,17 @@ int cmd_stat(int argc, const char **argv)
 	perf_stat__collect_metric_expr(evsel_list);
 	perf_stat__init_shadow_stats();
 
+	if (target.create_cgroup) {
+		char name[32];
+
+		scnprintf(name, 31, "perf.%u", getpid());
+		cgroup__create(name);
+		cgroup__add_pid(name, strtoul(target.pid, NULL, 0));
+
+		cgroup__add_evlist(name, evsel_list);
+		target.pid = NULL;
+	}
+
 	if (stat_config.csv_sep) {
 		stat_config.csv_output = true;
 		if (!strcmp(stat_config.csv_sep, "\\t"))
@@ -1906,5 +1931,6 @@ int cmd_stat(int argc, const char **argv)
 
 	runtime_stat_delete(&stat_config);
 
+	cleanup();
 	return status;
 }
tools/perf/util/cgroup.c +76 -0

79 lines not on this page · raw message

tools/perf/util/cgroup.h +5 -0
index f033a80c1b14..7bdd8d99d130 100644
@@ -26,4 +26,9 @@ void evlist__set_default_cgroup(struct perf_evlist *evlist, struct cgroup *cgrou
 
 int parse_cgroups(const struct option *opt, const char *str, int unset);
 
+int cgroup__create(const char *name);
+int cgroup__cleanup(const char *name);
+int cgroup__add_pid(const char *name, pid_t pid);
+int cgroup__add_evlist(const char *name, struct perf_evlist *evlist);
+
 #endif /* __CGROUP_H__ */
tools/perf/util/target.h +1 -0
index 6ef01a83b24e..03c9ac06660a 100644
@@ -15,6 +15,7 @@ struct target {
 	bool	     uses_mmap;
 	bool	     default_per_cpu;
 	bool	     per_thread;
+	bool	     create_cgroup;
 };
 
 enum target_errno {

--
2.17.1