Skip to contentSkip to conversation

perf metricgroup: Fix uncore metric expressions

PATCH Superseded Ian Rogers 3 participants 9 messages Sep 10, 2020 – Sep 23, 2020

Subsystems (4)
lore
Share & tools
lore

Ian Rogers

author reviewer
#0 lore raw

PATCH v3 1/1 perf metricgroup: Fix uncore metric expressions

A metric like DRAM_BW_Use has on SkylakeX events uncore_imc/cas_count_read/ and uncore_imc/case_count_write/. These events open 6 events per socket with pmu names of uncore_imc_[0-5]. The current metric setup code in find_evsel_group assumes one ID will map to 1 event to be recorded in metric_events. For events with multiple matches, the first event is recorded in metric_events (avoiding matching >1 event with the same name) and the evlist_used updated so that duplicate events aren't removed when the evlist has unused events removed.

Before this change:

$ /tmp/perf/perf stat -M DRAM_BW_Use -a -- sleep 1

Performance counter stats for 'system wide':

             41.14 MiB  uncore_imc/cas_count_read/
     1,002,614,251 ns   duration_time

       1.002614251 seconds time elapsed

After this change:

$ /tmp/perf/perf stat -M DRAM_BW_Use -a -- sleep 1

Performance counter stats for 'system wide':

            157.47 MiB  uncore_imc/cas_count_read/ #     0.00 DRAM_BW_Use
            126.97 MiB  uncore_imc/cas_count_write/
     1,003,019,728 ns   duration_time

v3. cleans up searching for the same event within metric_events to use a
helper and avoids a redundant search. It uses a continue loop to make the search for similarly named events shorter.
v2. avoids iterating over the whole evlist as suggested by
[email protected]. It also fixes the metric_leader computation that was broken in the same commits.

Erroneous duplication introduced in:
commit 2440689d62e9 ("perf metricgroup: Remove duped metric group events").

Fixes ded80bda8bc9 ("perf expr: Migrate expr ids table to a hashmap").
---
 tools/perf/util/metricgroup.c | 75 ++++++++++++++++++++++++++---------
 1 file changed, 56 insertions(+), 19 deletions(-)
tools/perf/util/metricgroup.c +56 -19
index 662f4e8777d5..7b66452cb9f9 100644
@@ -149,6 +149,18 @@ static void expr_ids__exit(struct expr_ids *ids)
 		free(ids->id[i].id);
 }
 
+static bool contains_event(struct evsel **metric_events, int num_events,
+			const char *event_name)
+{
+	int i;
+
+	for (i = 0; i < num_events; i++) {
+		if (!strcmp(metric_events[i]->name, event_name))
+			return true;
+	}
+	return false;
+}
+
 /**
  * Find a group of events in perf_evlist that correspond to those from a parsed
  * metric expression. Note, as find_evsel_group is called in the same order as
@@ -179,7 +191,11 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
 	int i = 0, matched_events = 0, events_to_match;
 	const int idnum = (int)hashmap__size(&pctx->ids);
 
-	/* duration_time is grouped separately. */
+	/*
+	 * duration_time is always grouped separately, when events are grouped
+	 * (ie has_constraint is false) then ignore it in the matching loop and
+	 * add it to metric_events at the end.
+	 */
 	if (!has_constraint &&
 	    hashmap__find(&pctx->ids, "duration_time", (void **)&val_ptr))
 		events_to_match = idnum - 1;
@@ -206,23 +222,20 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
 				sizeof(struct evsel *) * idnum);
 			current_leader = ev->leader;
 		}
-		if (hashmap__find(&pctx->ids, ev->name, (void **)&val_ptr)) {
-			if (has_constraint) {
-				/*
-				 * Events aren't grouped, ensure the same event
-				 * isn't matched from two groups.
-				 */
-				for (i = 0; i < matched_events; i++) {
-					if (!strcmp(ev->name,
-						    metric_events[i]->name)) {
-						break;
-					}
-				}
-				if (i != matched_events)
-					continue;
-			}
+		/*
+		 * Check for duplicate events with the same name. For example,
+		 * uncore_imc/cas_count_read/ will turn into 6 events per socket
+		 * on skylakex. Only the first such event is placed in
+		 * metric_events. If events aren't grouped then this also
+		 * ensures that the same event in different sibling groups
+		 * aren't both added to metric_events.
+		 */
+		if (contains_event(metric_events, matched_events, ev->name))
+			continue;
+		/* Does this event belong to the parse context? */
+		if (hashmap__find(&pctx->ids, ev->name, (void **)&val_ptr))
 			metric_events[matched_events++] = ev;
-		}
+
 		if (matched_events == events_to_match)
 			break;
 	}
@@ -238,7 +251,7 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
 	}
 
 	if (matched_events != idnum) {
-		/* Not whole match */
+		/* Not a whole match */
 		return NULL;
 	}
 
@@ -246,8 +259,32 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
 
 	for (i = 0; i < idnum; i++) {
 		ev = metric_events[i];
-		ev->metric_leader = ev;
+		/* Don't free the used events. */
 		set_bit(ev->idx, evlist_used);
+		/*
+		 * The metric leader points to the identically named event in
+		 * metric_events.
+		 */
+		ev->metric_leader = ev;
+		/*
+		 * Mark two events with identical names in the same group (or
+		 * globally) as being in use as uncore events may be duplicated
+		 * for each pmu. Set the metric leader of such events to be the
+		 * event that appears in metric_events.
+		 */
+		evlist__for_each_entry_continue(perf_evlist, ev) {
+			/*
+			 * If events are grouped then the search can terminate
+			 * when then group is left.
+			 */
+			if (!has_constraint &&
+			    ev->leader != metric_events[i]->leader)
+				break;
+			if (!strcmp(metric_events[i]->name, ev->name)) {
+				set_bit(ev->idx, evlist_used);
+				ev->metric_leader = metric_events[i];
+			}
+		}
 	}
 
 	return metric_events[0];
Signature · 1 line
2.28.0.526.ge36021eeef-goog
maintainer Acked-by on v3 1/1
#1 lore raw
Quoted 36 lines · Ian Rogers

A metric like DRAM_BW_Use has on SkylakeX events uncore_imc/cas_count_read/ and uncore_imc/case_count_write/. These events open 6 events per socket with pmu names of uncore_imc_[0-5]. The current metric setup code in find_evsel_group assumes one ID will map to 1 event to be recorded in metric_events. For events with multiple matches, the first event is recorded in metric_events (avoiding matching >1 event with the same name) and the evlist_used updated so that duplicate events aren't removed when the evlist has unused events removed.

Before this change:

$ /tmp/perf/perf stat -M DRAM_BW_Use -a -- sleep 1

Performance counter stats for 'system wide':

             41.14 MiB  uncore_imc/cas_count_read/
     1,002,614,251 ns   duration_time

       1.002614251 seconds time elapsed

After this change:

$ /tmp/perf/perf stat -M DRAM_BW_Use -a -- sleep 1

Performance counter stats for 'system wide':

            157.47 MiB  uncore_imc/cas_count_read/ #     0.00 DRAM_BW_Use
            126.97 MiB  uncore_imc/cas_count_write/
     1,003,019,728 ns   duration_time

v3. cleans up searching for the same event within metric_events to use a
helper and avoids a redundant search. It uses a continue loop to make the search for similarly named events shorter.
v2. avoids iterating over the whole evlist as suggested by
[email protected]. It also fixes the metric_leader computation that was broken in the same commits.

Quoted 6 lines

Erroneous duplication introduced in:
commit 2440689d62e9 ("perf metricgroup: Remove duped metric group events").

Fixes ded80bda8bc9 ("perf expr: Migrate expr ids table to a hashmap").

Thanks
Namhyung

Quoted 123 lines
---
 tools/perf/util/metricgroup.c | 75 ++++++++++++++++++++++++++---------
 1 file changed, 56 insertions(+), 19 deletions(-)
tools/perf/util/metricgroup.c +56 -20
index 662f4e8777d5..7b66452cb9f9 100644
@@ -149,6 +149,18 @@ static void expr_ids__exit(struct expr_ids *ids)
                free(ids->id[i].id);
 }
 
+static bool contains_event(struct evsel **metric_events, int num_events,
+                       const char *event_name)
+{
+       int i;
+
+       for (i = 0; i < num_events; i++) {
+               if (!strcmp(metric_events[i]->name, event_name))
+                       return true;
+       }
+       return false;
+}
+
 /**
  * Find a group of events in perf_evlist that correspond to those from a parsed
  * metric expression. Note, as find_evsel_group is called in the same order as
@@ -179,7 +191,11 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
        int i = 0, matched_events = 0, events_to_match;
        const int idnum = (int)hashmap__size(&pctx->ids);
 
-       /* duration_time is grouped separately. */
+       /*
+        * duration_time is always grouped separately, when events are grouped
+        * (ie has_constraint is false) then ignore it in the matching loop and
+        * add it to metric_events at the end.
+        */
        if (!has_constraint &&
            hashmap__find(&pctx->ids, "duration_time", (void **)&val_ptr))
                events_to_match = idnum - 1;
@@ -206,23 +222,20 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
                                sizeof(struct evsel *) * idnum);
                        current_leader = ev->leader;
                }
-               if (hashmap__find(&pctx->ids, ev->name, (void **)&val_ptr)) {
-                       if (has_constraint) {
-                               /*
-                                * Events aren't grouped, ensure the same event
-                                * isn't matched from two groups.
-                                */
-                               for (i = 0; i < matched_events; i++) {
-                                       if (!strcmp(ev->name,
-                                                   metric_events[i]->name)) {
-                                               break;
-                                       }
-                               }
-                               if (i != matched_events)
-                                       continue;
-                       }
+               /*
+                * Check for duplicate events with the same name. For example,
+                * uncore_imc/cas_count_read/ will turn into 6 events per socket
+                * on skylakex. Only the first such event is placed in
+                * metric_events. If events aren't grouped then this also
+                * ensures that the same event in different sibling groups
+                * aren't both added to metric_events.
+                */
+               if (contains_event(metric_events, matched_events, ev->name))
+                       continue;
+               /* Does this event belong to the parse context? */
+               if (hashmap__find(&pctx->ids, ev->name, (void **)&val_ptr))
                        metric_events[matched_events++] = ev;
-               }
+
                if (matched_events == events_to_match)
                        break;
        }
@@ -238,7 +251,7 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
        }
 
        if (matched_events != idnum) {
-               /* Not whole match */
+               /* Not a whole match */
                return NULL;
        }
 
@@ -246,8 +259,32 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
 
        for (i = 0; i < idnum; i++) {
                ev = metric_events[i];
-               ev->metric_leader = ev;
+               /* Don't free the used events. */
                set_bit(ev->idx, evlist_used);
+               /*
+                * The metric leader points to the identically named event in
+                * metric_events.
+                */
+               ev->metric_leader = ev;
+               /*
+                * Mark two events with identical names in the same group (or
+                * globally) as being in use as uncore events may be duplicated
+                * for each pmu. Set the metric leader of such events to be the
+                * event that appears in metric_events.
+                */
+               evlist__for_each_entry_continue(perf_evlist, ev) {
+                       /*
+                        * If events are grouped then the search can terminate
+                        * when then group is left.
+                        */
+                       if (!has_constraint &&
+                           ev->leader != metric_events[i]->leader)
+                               break;
+                       if (!strcmp(metric_events[i]->name, ev->name)) {
+                               set_bit(ev->idx, evlist_used);
+                               ev->metric_leader = metric_events[i];
+                       }
+               }
        }
 
        return metric_events[0];
--

2.28.0.526.ge36021eeef-goog

#2 lore raw

Em Fri, Sep 11, 2020 at 12:07:35PM +0900, Namhyung Kim escreveu:

Quoted 8 lines · Ian Rogers
Quoted 7 lines

v3. cleans up searching for the same event within metric_events to use a
helper and avoids a redundant search. It uses a continue loop to make the search for similarly named events shorter.
v2. avoids iterating over the whole evlist as suggested by
[email protected]. It also fixes the metric_leader computation that was broken in the same commits.

Quoted 8 lines
Quoted 6 lines

Erroneous duplication introduced in:
commit 2440689d62e9 ("perf metricgroup: Remove duped metric group events").

Fixes ded80bda8bc9 ("perf expr: Migrate expr ids table to a hashmap").

On my perf/urgent branch (upstream should be the same):

[acme@five perf]$ patch -p1 < /wb/1.patch patching file tools/perf/util/metricgroup.c Hunk #1 succeeded at 150 with fuzz 2 (offset 1 line). Hunk #2 succeeded at 192 (offset 1 line). Hunk #3 succeeded at 223 (offset 1 line). Hunk #4 succeeded at 252 (offset 1 line). Hunk #5 succeeded at 260 (offset 1 line). [acme@five perf]$

I'm fixing it up, please check that doing this is safe on your side.

- Arnaldo

author reviewer
#3 lore raw

PATCH v4 1/1 perf metricgroup: Fix uncore metric expressions

A metric like DRAM_BW_Use has on SkylakeX events uncore_imc/cas_count_read/ and uncore_imc/case_count_write/. These events open 6 events per socket with pmu names of uncore_imc_[0-5]. The current metric setup code in find_evsel_group assumes one ID will map to 1 event to be recorded in metric_events. For events with multiple matches, the first event is recorded in metric_events (avoiding matching >1 event with the same name) and the evlist_used updated so that duplicate events aren't removed when the evlist has unused events removed.

Before this change:

$ /tmp/perf/perf stat -M DRAM_BW_Use -a -- sleep 1

Performance counter stats for 'system wide':

             41.14 MiB  uncore_imc/cas_count_read/
     1,002,614,251 ns   duration_time

       1.002614251 seconds time elapsed

After this change:

$ /tmp/perf/perf stat -M DRAM_BW_Use -a -- sleep 1

Performance counter stats for 'system wide':

            157.47 MiB  uncore_imc/cas_count_read/ #     0.00 DRAM_BW_Use
            126.97 MiB  uncore_imc/cas_count_write/
     1,003,019,728 ns   duration_time

Erroneous duplication introduced in:
commit 2440689d62e9 ("perf metricgroup: Remove duped metric group events").

Fixes ded80bda8bc9 ("perf expr: Migrate expr ids table to a hashmap").
---
 tools/perf/util/metricgroup.c | 75 ++++++++++++++++++++++++++---------
 1 file changed, 56 insertions(+), 19 deletions(-)
tools/perf/util/metricgroup.c +56 -19
index ab5030fcfed4..d948a7f910cf 100644
@@ -150,6 +150,18 @@ static void expr_ids__exit(struct expr_ids *ids)
 		free(ids->id[i].id);
 }
 
+static bool contains_event(struct evsel **metric_events, int num_events,
+			const char *event_name)
+{
+	int i;
+
+	for (i = 0; i < num_events; i++) {
+		if (!strcmp(metric_events[i]->name, event_name))
+			return true;
+	}
+	return false;
+}
+
 /**
  * Find a group of events in perf_evlist that correpond to those from a parsed
  * metric expression. Note, as find_evsel_group is called in the same order as
@@ -180,7 +192,11 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
 	int i = 0, matched_events = 0, events_to_match;
 	const int idnum = (int)hashmap__size(&pctx->ids);
 
-	/* duration_time is grouped separately. */
+	/*
+	 * duration_time is always grouped separately, when events are grouped
+	 * (ie has_constraint is false) then ignore it in the matching loop and
+	 * add it to metric_events at the end.
+	 */
 	if (!has_constraint &&
 	    hashmap__find(&pctx->ids, "duration_time", (void **)&val_ptr))
 		events_to_match = idnum - 1;
@@ -207,23 +223,20 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
 				sizeof(struct evsel *) * idnum);
 			current_leader = ev->leader;
 		}
-		if (hashmap__find(&pctx->ids, ev->name, (void **)&val_ptr)) {
-			if (has_constraint) {
-				/*
-				 * Events aren't grouped, ensure the same event
-				 * isn't matched from two groups.
-				 */
-				for (i = 0; i < matched_events; i++) {
-					if (!strcmp(ev->name,
-						    metric_events[i]->name)) {
-						break;
-					}
-				}
-				if (i != matched_events)
-					continue;
-			}
+		/*
+		 * Check for duplicate events with the same name. For example,
+		 * uncore_imc/cas_count_read/ will turn into 6 events per socket
+		 * on skylakex. Only the first such event is placed in
+		 * metric_events. If events aren't grouped then this also
+		 * ensures that the same event in different sibling groups
+		 * aren't both added to metric_events.
+		 */
+		if (contains_event(metric_events, matched_events, ev->name))
+			continue;
+		/* Does this event belong to the parse context? */
+		if (hashmap__find(&pctx->ids, ev->name, (void **)&val_ptr))
 			metric_events[matched_events++] = ev;
-		}
+
 		if (matched_events == events_to_match)
 			break;
 	}
@@ -239,7 +252,7 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
 	}
 
 	if (matched_events != idnum) {
-		/* Not whole match */
+		/* Not a whole match */
 		return NULL;
 	}
 
@@ -247,8 +260,32 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
 
 	for (i = 0; i < idnum; i++) {
 		ev = metric_events[i];
-		ev->metric_leader = ev;
+		/* Don't free the used events. */
 		set_bit(ev->idx, evlist_used);
+		/*
+		 * The metric leader points to the identically named event in
+		 * metric_events.
+		 */
+		ev->metric_leader = ev;
+		/*
+		 * Mark two events with identical names in the same group (or
+		 * globally) as being in use as uncore events may be duplicated
+		 * for each pmu. Set the metric leader of such events to be the
+		 * event that appears in metric_events.
+		 */
+		evlist__for_each_entry_continue(perf_evlist, ev) {
+			/*
+			 * If events are grouped then the search can terminate
+			 * when then group is left.
+			 */
+			if (!has_constraint &&
+			    ev->leader != metric_events[i]->leader)
+				break;
+			if (!strcmp(metric_events[i]->name, ev->name)) {
+				set_bit(ev->idx, evlist_used);
+				ev->metric_leader = metric_events[i];
+			}
+		}
 	}
 
 	return metric_events[0];
Signature · 1 line
2.28.0.618.gf4bc123cb7-goog
#4 lore raw

Em Thu, Sep 17, 2020 at 01:18:07PM -0700, Ian Rogers escreveu:

Quoted 8 lines

A metric like DRAM_BW_Use has on SkylakeX events uncore_imc/cas_count_read/ and uncore_imc/case_count_write/. These events open 6 events per socket with pmu names of uncore_imc_[0-5]. The current metric setup code in find_evsel_group assumes one ID will map to 1 event to be recorded in metric_events. For events with multiple matches, the first event is recorded in metric_events (avoiding matching >1 event with the same name) and the evlist_used updated so that duplicate events aren't removed when the evlist has unused events removed.

Namhyung, please check if you still Acks this as you provided it for v3.

Thanks,

- Arnaldo

Quoted 148 lines

Before this change:

$ /tmp/perf/perf stat -M DRAM_BW_Use -a -- sleep 1

Performance counter stats for 'system wide':

             41.14 MiB  uncore_imc/cas_count_read/
     1,002,614,251 ns   duration_time

       1.002614251 seconds time elapsed

After this change:

$ /tmp/perf/perf stat -M DRAM_BW_Use -a -- sleep 1

Performance counter stats for 'system wide':

            157.47 MiB  uncore_imc/cas_count_read/ #     0.00 DRAM_BW_Use
            126.97 MiB  uncore_imc/cas_count_write/
     1,003,019,728 ns   duration_time

Erroneous duplication introduced in:
commit 2440689d62e9 ("perf metricgroup: Remove duped metric group events").

Fixes ded80bda8bc9 ("perf expr: Migrate expr ids table to a hashmap").
---
 tools/perf/util/metricgroup.c | 75 ++++++++++++++++++++++++++---------
 1 file changed, 56 insertions(+), 19 deletions(-)
tools/perf/util/metricgroup.c +56 -19
index ab5030fcfed4..d948a7f910cf 100644
@@ -150,6 +150,18 @@ static void expr_ids__exit(struct expr_ids *ids)
 		free(ids->id[i].id);
 }
 
+static bool contains_event(struct evsel **metric_events, int num_events,
+			const char *event_name)
+{
+	int i;
+
+	for (i = 0; i < num_events; i++) {
+		if (!strcmp(metric_events[i]->name, event_name))
+			return true;
+	}
+	return false;
+}
+
 /**
  * Find a group of events in perf_evlist that correpond to those from a parsed
  * metric expression. Note, as find_evsel_group is called in the same order as
@@ -180,7 +192,11 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
 	int i = 0, matched_events = 0, events_to_match;
 	const int idnum = (int)hashmap__size(&pctx->ids);
 
-	/* duration_time is grouped separately. */
+	/*
+	 * duration_time is always grouped separately, when events are grouped
+	 * (ie has_constraint is false) then ignore it in the matching loop and
+	 * add it to metric_events at the end.
+	 */
 	if (!has_constraint &&
 	    hashmap__find(&pctx->ids, "duration_time", (void **)&val_ptr))
 		events_to_match = idnum - 1;
@@ -207,23 +223,20 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
 				sizeof(struct evsel *) * idnum);
 			current_leader = ev->leader;
 		}
-		if (hashmap__find(&pctx->ids, ev->name, (void **)&val_ptr)) {
-			if (has_constraint) {
-				/*
-				 * Events aren't grouped, ensure the same event
-				 * isn't matched from two groups.
-				 */
-				for (i = 0; i < matched_events; i++) {
-					if (!strcmp(ev->name,
-						    metric_events[i]->name)) {
-						break;
-					}
-				}
-				if (i != matched_events)
-					continue;
-			}
+		/*
+		 * Check for duplicate events with the same name. For example,
+		 * uncore_imc/cas_count_read/ will turn into 6 events per socket
+		 * on skylakex. Only the first such event is placed in
+		 * metric_events. If events aren't grouped then this also
+		 * ensures that the same event in different sibling groups
+		 * aren't both added to metric_events.
+		 */
+		if (contains_event(metric_events, matched_events, ev->name))
+			continue;
+		/* Does this event belong to the parse context? */
+		if (hashmap__find(&pctx->ids, ev->name, (void **)&val_ptr))
 			metric_events[matched_events++] = ev;
-		}
+
 		if (matched_events == events_to_match)
 			break;
 	}
@@ -239,7 +252,7 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
 	}
 
 	if (matched_events != idnum) {
-		/* Not whole match */
+		/* Not a whole match */
 		return NULL;
 	}
 
@@ -247,8 +260,32 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
 
 	for (i = 0; i < idnum; i++) {
 		ev = metric_events[i];
-		ev->metric_leader = ev;
+		/* Don't free the used events. */
 		set_bit(ev->idx, evlist_used);
+		/*
+		 * The metric leader points to the identically named event in
+		 * metric_events.
+		 */
+		ev->metric_leader = ev;
+		/*
+		 * Mark two events with identical names in the same group (or
+		 * globally) as being in use as uncore events may be duplicated
+		 * for each pmu. Set the metric leader of such events to be the
+		 * event that appears in metric_events.
+		 */
+		evlist__for_each_entry_continue(perf_evlist, ev) {
+			/*
+			 * If events are grouped then the search can terminate
+			 * when then group is left.
+			 */
+			if (!has_constraint &&
+			    ev->leader != metric_events[i]->leader)
+				break;
+			if (!strcmp(metric_events[i]->name, ev->name)) {
+				set_bit(ev->idx, evlist_used);
+				ev->metric_leader = metric_events[i];
+			}
+		}
 	}
 
 	return metric_events[0];
Signature · 1 line
2.28.0.618.gf4bc123cb7-goog
Signature · 1 line
- Arnaldo
maintainer Acked-by
#5 lore raw

Hi Arnaldo,

Quoted 11 lines · Arnaldo Carvalho de Melo

Em Thu, Sep 17, 2020 at 01:18:07PM -0700, Ian Rogers escreveu:

Quoted 8 lines

A metric like DRAM_BW_Use has on SkylakeX events uncore_imc/cas_count_read/ and uncore_imc/case_count_write/. These events open 6 events per socket with pmu names of uncore_imc_[0-5]. The current metric setup code in find_evsel_group assumes one ID will map to 1 event to be recorded in metric_events. For events with multiple matches, the first event is recorded in metric_events (avoiding matching >1 event with the same name) and the evlist_used updated so that duplicate events aren't removed when the evlist has unused events removed.

Quoted 1 line

Namhyung, please check if you still Acks this as you provided it for v3.

Sure,

Thanks
Namhyung

Quoted 153 lines
Quoted 147 lines

Before this change:

$ /tmp/perf/perf stat -M DRAM_BW_Use -a -- sleep 1

Performance counter stats for 'system wide':

             41.14 MiB  uncore_imc/cas_count_read/
     1,002,614,251 ns   duration_time

       1.002614251 seconds time elapsed

After this change:

$ /tmp/perf/perf stat -M DRAM_BW_Use -a -- sleep 1

Performance counter stats for 'system wide':

            157.47 MiB  uncore_imc/cas_count_read/ #     0.00 DRAM_BW_Use
            126.97 MiB  uncore_imc/cas_count_write/
     1,003,019,728 ns   duration_time

Erroneous duplication introduced in:
commit 2440689d62e9 ("perf metricgroup: Remove duped metric group events").

Fixes ded80bda8bc9 ("perf expr: Migrate expr ids table to a hashmap").
---
 tools/perf/util/metricgroup.c | 75 ++++++++++++++++++++++++++---------
 1 file changed, 56 insertions(+), 19 deletions(-)
tools/perf/util/metricgroup.c +56 -20
index ab5030fcfed4..d948a7f910cf 100644
@@ -150,6 +150,18 @@ static void expr_ids__exit(struct expr_ids *ids)
              free(ids->id[i].id);
 }
 
+static bool contains_event(struct evsel **metric_events, int num_events,
+                     const char *event_name)
+{
+     int i;
+
+     for (i = 0; i < num_events; i++) {
+             if (!strcmp(metric_events[i]->name, event_name))
+                     return true;
+     }
+     return false;
+}
+
 /**
  * Find a group of events in perf_evlist that correpond to those from a parsed
  * metric expression. Note, as find_evsel_group is called in the same order as
@@ -180,7 +192,11 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
      int i = 0, matched_events = 0, events_to_match;
      const int idnum = (int)hashmap__size(&pctx->ids);
 
-     /* duration_time is grouped separately. */
+     /*
+      * duration_time is always grouped separately, when events are grouped
+      * (ie has_constraint is false) then ignore it in the matching loop and
+      * add it to metric_events at the end.
+      */
      if (!has_constraint &&
          hashmap__find(&pctx->ids, "duration_time", (void **)&val_ptr))
              events_to_match = idnum - 1;
@@ -207,23 +223,20 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
                              sizeof(struct evsel *) * idnum);
                      current_leader = ev->leader;
              }
-             if (hashmap__find(&pctx->ids, ev->name, (void **)&val_ptr)) {
-                     if (has_constraint) {
-                             /*
-                              * Events aren't grouped, ensure the same event
-                              * isn't matched from two groups.
-                              */
-                             for (i = 0; i < matched_events; i++) {
-                                     if (!strcmp(ev->name,
-                                                 metric_events[i]->name)) {
-                                             break;
-                                     }
-                             }
-                             if (i != matched_events)
-                                     continue;
-                     }
+             /*
+              * Check for duplicate events with the same name. For example,
+              * uncore_imc/cas_count_read/ will turn into 6 events per socket
+              * on skylakex. Only the first such event is placed in
+              * metric_events. If events aren't grouped then this also
+              * ensures that the same event in different sibling groups
+              * aren't both added to metric_events.
+              */
+             if (contains_event(metric_events, matched_events, ev->name))
+                     continue;
+             /* Does this event belong to the parse context? */
+             if (hashmap__find(&pctx->ids, ev->name, (void **)&val_ptr))
                      metric_events[matched_events++] = ev;
-             }
+
              if (matched_events == events_to_match)
                      break;
      }
@@ -239,7 +252,7 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
      }
 
      if (matched_events != idnum) {
-             /* Not whole match */
+             /* Not a whole match */
              return NULL;
      }
 
@@ -247,8 +260,32 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
 
      for (i = 0; i < idnum; i++) {
              ev = metric_events[i];
-             ev->metric_leader = ev;
+             /* Don't free the used events. */
              set_bit(ev->idx, evlist_used);
+             /*
+              * The metric leader points to the identically named event in
+              * metric_events.
+              */
+             ev->metric_leader = ev;
+             /*
+              * Mark two events with identical names in the same group (or
+              * globally) as being in use as uncore events may be duplicated
+              * for each pmu. Set the metric leader of such events to be the
+              * event that appears in metric_events.
+              */
+             evlist__for_each_entry_continue(perf_evlist, ev) {
+                     /*
+                      * If events are grouped then the search can terminate
+                      * when then group is left.
+                      */
+                     if (!has_constraint &&
+                         ev->leader != metric_events[i]->leader)
+                             break;
+                     if (!strcmp(metric_events[i]->name, ev->name)) {
+                             set_bit(ev->idx, evlist_used);
+                             ev->metric_leader = metric_events[i];
+                     }
+             }
      }
 
      return metric_events[0];
--
Quoted 2 lines

2.28.0.618.gf4bc123cb7-goog

--

- Arnaldo

author reviewer
#6 lore raw
Quoted 16 lines · Namhyung Kim

Hi Arnaldo,

Quoted 11 lines · Arnaldo Carvalho de Melo

Em Thu, Sep 17, 2020 at 01:18:07PM -0700, Ian Rogers escreveu:

Quoted 8 lines

A metric like DRAM_BW_Use has on SkylakeX events uncore_imc/cas_count_read/ and uncore_imc/case_count_write/. These events open 6 events per socket with pmu names of uncore_imc_[0-5]. The current metric setup code in find_evsel_group assumes one ID will map to 1 event to be recorded in metric_events. For events with multiple matches, the first event is recorded in metric_events (avoiding matching >1 event with the same name) and the evlist_used updated so that duplicate events aren't removed when the evlist has unused events removed.

Quoted 5 lines
Quoted 1 line

Namhyung, please check if you still Acks this as you provided it for v3.

Sure,

Thanks Namhyung and Arnaldo, could we merge this?

Ian

Quoted 156 lines

Thanks
Namhyung

Quoted 153 lines
Quoted 147 lines

Before this change:

$ /tmp/perf/perf stat -M DRAM_BW_Use -a -- sleep 1

Performance counter stats for 'system wide':

             41.14 MiB  uncore_imc/cas_count_read/
     1,002,614,251 ns   duration_time

       1.002614251 seconds time elapsed

After this change:

$ /tmp/perf/perf stat -M DRAM_BW_Use -a -- sleep 1

Performance counter stats for 'system wide':

            157.47 MiB  uncore_imc/cas_count_read/ #     0.00 DRAM_BW_Use
            126.97 MiB  uncore_imc/cas_count_write/
     1,003,019,728 ns   duration_time

Erroneous duplication introduced in:
commit 2440689d62e9 ("perf metricgroup: Remove duped metric group events").

Fixes ded80bda8bc9 ("perf expr: Migrate expr ids table to a hashmap").
---
 tools/perf/util/metricgroup.c | 75 ++++++++++++++++++++++++++---------
 1 file changed, 56 insertions(+), 19 deletions(-)
tools/perf/util/metricgroup.c +56 -20
index ab5030fcfed4..d948a7f910cf 100644
@@ -150,6 +150,18 @@ static void expr_ids__exit(struct expr_ids *ids)
              free(ids->id[i].id);
 }
 
+static bool contains_event(struct evsel **metric_events, int num_events,
+                     const char *event_name)
+{
+     int i;
+
+     for (i = 0; i < num_events; i++) {
+             if (!strcmp(metric_events[i]->name, event_name))
+                     return true;
+     }
+     return false;
+}
+
 /**
  * Find a group of events in perf_evlist that correpond to those from a parsed
  * metric expression. Note, as find_evsel_group is called in the same order as
@@ -180,7 +192,11 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
      int i = 0, matched_events = 0, events_to_match;
      const int idnum = (int)hashmap__size(&pctx->ids);
 
-     /* duration_time is grouped separately. */
+     /*
+      * duration_time is always grouped separately, when events are grouped
+      * (ie has_constraint is false) then ignore it in the matching loop and
+      * add it to metric_events at the end.
+      */
      if (!has_constraint &&
          hashmap__find(&pctx->ids, "duration_time", (void **)&val_ptr))
              events_to_match = idnum - 1;
@@ -207,23 +223,20 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
                              sizeof(struct evsel *) * idnum);
                      current_leader = ev->leader;
              }
-             if (hashmap__find(&pctx->ids, ev->name, (void **)&val_ptr)) {
-                     if (has_constraint) {
-                             /*
-                              * Events aren't grouped, ensure the same event
-                              * isn't matched from two groups.
-                              */
-                             for (i = 0; i < matched_events; i++) {
-                                     if (!strcmp(ev->name,
-                                                 metric_events[i]->name)) {
-                                             break;
-                                     }
-                             }
-                             if (i != matched_events)
-                                     continue;
-                     }
+             /*
+              * Check for duplicate events with the same name. For example,
+              * uncore_imc/cas_count_read/ will turn into 6 events per socket
+              * on skylakex. Only the first such event is placed in
+              * metric_events. If events aren't grouped then this also
+              * ensures that the same event in different sibling groups
+              * aren't both added to metric_events.
+              */
+             if (contains_event(metric_events, matched_events, ev->name))
+                     continue;
+             /* Does this event belong to the parse context? */
+             if (hashmap__find(&pctx->ids, ev->name, (void **)&val_ptr))
                      metric_events[matched_events++] = ev;
-             }
+
              if (matched_events == events_to_match)
                      break;
      }
@@ -239,7 +252,7 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
      }
 
      if (matched_events != idnum) {
-             /* Not whole match */
+             /* Not a whole match */
              return NULL;
      }
 
@@ -247,8 +260,32 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
 
      for (i = 0; i < idnum; i++) {
              ev = metric_events[i];
-             ev->metric_leader = ev;
+             /* Don't free the used events. */
              set_bit(ev->idx, evlist_used);
+             /*
+              * The metric leader points to the identically named event in
+              * metric_events.
+              */
+             ev->metric_leader = ev;
+             /*
+              * Mark two events with identical names in the same group (or
+              * globally) as being in use as uncore events may be duplicated
+              * for each pmu. Set the metric leader of such events to be the
+              * event that appears in metric_events.
+              */
+             evlist__for_each_entry_continue(perf_evlist, ev) {
+                     /*
+                      * If events are grouped then the search can terminate
+                      * when then group is left.
+                      */
+                     if (!has_constraint &&
+                         ev->leader != metric_events[i]->leader)
+                             break;
+                     if (!strcmp(metric_events[i]->name, ev->name)) {
+                             set_bit(ev->idx, evlist_used);
+                             ev->metric_leader = metric_events[i];
+                     }
+             }
      }
 
      return metric_events[0];
--
Quoted 2 lines

2.28.0.618.gf4bc123cb7-goog

--

- Arnaldo

#7 lore raw

Em Wed, Sep 23, 2020 at 12:59:28PM -0700, Ian Rogers escreveu:

Quoted 17 lines · Namhyung Kim
Quoted 16 lines

Hi Arnaldo,

Quoted 11 lines · Arnaldo Carvalho de Melo

Em Thu, Sep 17, 2020 at 01:18:07PM -0700, Ian Rogers escreveu:

Quoted 8 lines

A metric like DRAM_BW_Use has on SkylakeX events uncore_imc/cas_count_read/ and uncore_imc/case_count_write/. These events open 6 events per socket with pmu names of uncore_imc_[0-5]. The current metric setup code in find_evsel_group assumes one ID will map to 1 event to be recorded in metric_events. For events with multiple matches, the first event is recorded in metric_events (avoiding matching >1 event with the same name) and the evlist_used updated so that duplicate events aren't removed when the evlist has unused events removed.

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

Namhyung, please check if you still Acks this as you provided it for v3.

Sure,

Thanks Namhyung and Arnaldo, could we merge this?

[acme@five perf]$ git log --oneline perf/urgent
dcc81be0fc4e6694 (quaco/perf/urgent, perf/urgent) perf metricgroup: Fix uncore metric expressions
0f1b550e29c1f952 perf parse-event: Release cpu_map refcount if evsel alloc failed
5d680be3b014d583 perf parse-event: Fix cpu map refcounting
5925fa68fe824465 (torvalds/master) Merge tag 'perf-tools-fixes-for-v5.9-2020-09-16' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux

I have it already locally, need to finish a round of testing so that I can push it publicly,

- Arnaldo

Quoted 158 lines

Ian

Quoted 156 lines

Thanks
Namhyung

Quoted 153 lines
Quoted 147 lines

Before this change:

$ /tmp/perf/perf stat -M DRAM_BW_Use -a -- sleep 1

Performance counter stats for 'system wide':

             41.14 MiB  uncore_imc/cas_count_read/
     1,002,614,251 ns   duration_time

       1.002614251 seconds time elapsed

After this change:

$ /tmp/perf/perf stat -M DRAM_BW_Use -a -- sleep 1

Performance counter stats for 'system wide':

            157.47 MiB  uncore_imc/cas_count_read/ #     0.00 DRAM_BW_Use
            126.97 MiB  uncore_imc/cas_count_write/
     1,003,019,728 ns   duration_time

Erroneous duplication introduced in:
commit 2440689d62e9 ("perf metricgroup: Remove duped metric group events").

Fixes ded80bda8bc9 ("perf expr: Migrate expr ids table to a hashmap").
---
 tools/perf/util/metricgroup.c | 75 ++++++++++++++++++++++++++---------
 1 file changed, 56 insertions(+), 19 deletions(-)
tools/perf/util/metricgroup.c +56 -20
index ab5030fcfed4..d948a7f910cf 100644
@@ -150,6 +150,18 @@ static void expr_ids__exit(struct expr_ids *ids)
              free(ids->id[i].id);
 }
 
+static bool contains_event(struct evsel **metric_events, int num_events,
+                     const char *event_name)
+{
+     int i;
+
+     for (i = 0; i < num_events; i++) {
+             if (!strcmp(metric_events[i]->name, event_name))
+                     return true;
+     }
+     return false;
+}
+
 /**
  * Find a group of events in perf_evlist that correpond to those from a parsed
  * metric expression. Note, as find_evsel_group is called in the same order as
@@ -180,7 +192,11 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
      int i = 0, matched_events = 0, events_to_match;
      const int idnum = (int)hashmap__size(&pctx->ids);
 
-     /* duration_time is grouped separately. */
+     /*
+      * duration_time is always grouped separately, when events are grouped
+      * (ie has_constraint is false) then ignore it in the matching loop and
+      * add it to metric_events at the end.
+      */
      if (!has_constraint &&
          hashmap__find(&pctx->ids, "duration_time", (void **)&val_ptr))
              events_to_match = idnum - 1;
@@ -207,23 +223,20 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
                              sizeof(struct evsel *) * idnum);
                      current_leader = ev->leader;
              }
-             if (hashmap__find(&pctx->ids, ev->name, (void **)&val_ptr)) {
-                     if (has_constraint) {
-                             /*
-                              * Events aren't grouped, ensure the same event
-                              * isn't matched from two groups.
-                              */
-                             for (i = 0; i < matched_events; i++) {
-                                     if (!strcmp(ev->name,
-                                                 metric_events[i]->name)) {
-                                             break;
-                                     }
-                             }
-                             if (i != matched_events)
-                                     continue;
-                     }
+             /*
+              * Check for duplicate events with the same name. For example,
+              * uncore_imc/cas_count_read/ will turn into 6 events per socket
+              * on skylakex. Only the first such event is placed in
+              * metric_events. If events aren't grouped then this also
+              * ensures that the same event in different sibling groups
+              * aren't both added to metric_events.
+              */
+             if (contains_event(metric_events, matched_events, ev->name))
+                     continue;
+             /* Does this event belong to the parse context? */
+             if (hashmap__find(&pctx->ids, ev->name, (void **)&val_ptr))
                      metric_events[matched_events++] = ev;
-             }
+
              if (matched_events == events_to_match)
                      break;
      }
@@ -239,7 +252,7 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
      }
 
      if (matched_events != idnum) {
-             /* Not whole match */
+             /* Not a whole match */
              return NULL;
      }
 
@@ -247,8 +260,32 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
 
      for (i = 0; i < idnum; i++) {
              ev = metric_events[i];
-             ev->metric_leader = ev;
+             /* Don't free the used events. */
              set_bit(ev->idx, evlist_used);
+             /*
+              * The metric leader points to the identically named event in
+              * metric_events.
+              */
+             ev->metric_leader = ev;
+             /*
+              * Mark two events with identical names in the same group (or
+              * globally) as being in use as uncore events may be duplicated
+              * for each pmu. Set the metric leader of such events to be the
+              * event that appears in metric_events.
+              */
+             evlist__for_each_entry_continue(perf_evlist, ev) {
+                     /*
+                      * If events are grouped then the search can terminate
+                      * when then group is left.
+                      */
+                     if (!has_constraint &&
+                         ev->leader != metric_events[i]->leader)
+                             break;
+                     if (!strcmp(metric_events[i]->name, ev->name)) {
+                             set_bit(ev->idx, evlist_used);
+                             ev->metric_leader = metric_events[i];
+                     }
+             }
      }
 
      return metric_events[0];
--
Quoted 2 lines

2.28.0.618.gf4bc123cb7-goog

--

- Arnaldo

Signature · 1 line
- Arnaldo
author reviewer
#8 lore raw
In reply to Arnaldo Carvalho de Melo

Should be ok. I cherry-picked the patch on to a local checkout of perf/urgent and sent as an in-reply-to here:
https://lore.kernel.org/lkml/[email protected]/T/#m6c12d6a224540ed5f222e58fff9807f1cdc4238f

Thanks,
Ian

Quoted 33 lines · Arnaldo Carvalho de Melo

Em Fri, Sep 11, 2020 at 12:07:35PM +0900, Namhyung Kim escreveu:

Quoted 8 lines · Ian Rogers
Quoted 7 lines

v3. cleans up searching for the same event within metric_events to use a
helper and avoids a redundant search. It uses a continue loop to make the search for similarly named events shorter.
v2. avoids iterating over the whole evlist as suggested by
[email protected]. It also fixes the metric_leader computation that was broken in the same commits.

Quoted 8 lines
Quoted 6 lines

Erroneous duplication introduced in:
commit 2440689d62e9 ("perf metricgroup: Remove duped metric group events").

Fixes ded80bda8bc9 ("perf expr: Migrate expr ids table to a hashmap").

On my perf/urgent branch (upstream should be the same):

[acme@five perf]$ patch -p1 < /wb/1.patch patching file tools/perf/util/metricgroup.c Hunk #1 succeeded at 150 with fuzz 2 (offset 1 line). Hunk #2 succeeded at 192 (offset 1 line). Hunk #3 succeeded at 223 (offset 1 line). Hunk #4 succeeded at 252 (offset 1 line). Hunk #5 succeeded at 260 (offset 1 line). [acme@five perf]$

I'm fixing it up, please check that doing this is safe on your side.

- Arnaldo