Skip to contentSkip to conversation

perf test: Make metric testing more robust.

PATCH In review Ian Rogers 3 participants 3 messages Dec 23, 2021 – Feb 15, 2022 perf events

lore
Share & tools
lore

Ian Rogers

author reviewer
#0 lore raw

PATCH v3 1/1 perf test: Make metric testing more robust.

When testing metric expressions we fake counter values from 1 going upward. For some metrics this can yield negative values that are clipped to zero, and then cause divide by zero failures. Such clipping is questionable but may be a result of tools automatically generating metrics. A workaround for this case is to try a second time with counter values going in the opposite direction.

This case was seen in a metric like:
event1 / max(event2 - event3, 0)
But it may also happen in more sensible metrics like:
event1 / (event2 + event3 - 1 - event4)

v2. Rebase and more detail in commit message.
v3. Is a rebase.

---
 tools/perf/tests/pmu-events.c | 32 ++++++++++++++++++++++++++------
 1 file changed, 26 insertions(+), 6 deletions(-)
tools/perf/tests/pmu-events.c +26 -6
index df1c9a3cc05b..b2ddf928d32a 100644
@@ -962,8 +962,18 @@ static int test__parsing(struct test_suite *test __maybe_unused,
 			}
 
 			if (expr__parse(&result, ctx, pe->metric_expr)) {
-				expr_failure("Parse failed", map, pe);
-				ret++;
+				/*
+				 * Parsing failed, make numbers go from large to
+				 * small which can resolve divide by zero
+				 * issues.
+				 */
+				k = 1024;
+				hashmap__for_each_entry(ctx->ids, cur, bkt)
+					expr__add_id_val(ctx, strdup(cur->key), k--);
+				if (expr__parse(&result, ctx, pe->metric_expr)) {
+					expr_failure("Parse failed", map, pe);
+					ret++;
+				}
 			}
 		}
 	}
@@ -1022,10 +1032,20 @@ static int metric_parse_fake(const char *str)
 		}
 	}
 
-	if (expr__parse(&result, ctx, str))
-		pr_err("expr__parse failed\n");
-	else
-		ret = 0;
+	ret = 0;
+	if (expr__parse(&result, ctx, str)) {
+		/*
+		 * Parsing failed, make numbers go from large to small which can
+		 * resolve divide by zero issues.
+		 */
+		i = 1024;
+		hashmap__for_each_entry(ctx->ids, cur, bkt)
+			expr__add_id_val(ctx, strdup(cur->key), i--);
+		if (expr__parse(&result, ctx, str)) {
+			pr_err("expr__parse failed\n");
+			ret = -1;
+		}
+	}
 
 out:
 	expr__ctx_free(ctx);
Signature · 1 line
2.34.1.307.g9b7440fafd-goog
Reviewed-by on v3 1/1
#1 lore raw
Quoted 7 lines · Ian Rogers

When testing metric expressions we fake counter values from 1 going upward. For some metrics this can yield negative values that are clipped to zero, and then cause divide by zero failures. Such clipping is questionable but may be a result of tools automatically generating metrics. A workaround for this case is to try a second time with counter values going in the opposite direction.

Quoted 7 lines

This case was seen in a metric like:

   event1 / max(event2 - event3, 0)
But it may also happen in more sensible metrics like:
   event1 / (event2 + event3 - 1 - event4)

v2. Rebase and more detail in commit message.
v3. Is a rebase.

Incorrect location for this info

Quoted 2 lines
Quoted 9 lines
---
  tools/perf/tests/pmu-events.c | 32 ++++++++++++++++++++++++++------
  1 file changed, 26 insertions(+), 6 deletions(-)
tools/perf/tests/pmu-events.c +0 -0
index df1c9a3cc05b..b2ddf928d32a 100644

tools/perf/tests/pmu-events.c static int test__parsing(struct test_suite *test __maybe_unused, · line 967

Quoted 8 lines
tools/perf/tests/pmu-events.c +3 -2
@@ -962,8 +962,18 @@ static int test__parsing(struct test_suite *test __maybe_unused,
  			}
 
  			if (expr__parse(&result, ctx, pe->metric_expr)) {
-				expr_failure("Parse failed", map, pe);
-				ret++;
+				/*
+				 * Parsing failed, make numbers go from large to
+				 * small which can resolve divide by zero

can or may?

Quoted 38 lines
tools/perf/tests/pmu-events.c +23 -4
+				 * issues.
+				 */
+				k = 1024;
+				hashmap__for_each_entry(ctx->ids, cur, bkt)
+					expr__add_id_val(ctx, strdup(cur->key), k--);
+				if (expr__parse(&result, ctx, pe->metric_expr)) {
+					expr_failure("Parse failed", map, pe);
+					ret++;
+				}
  			}
  		}
  	}
@@ -1022,10 +1032,20 @@ static int metric_parse_fake(const char *str)
  		}
  	}
 
-	if (expr__parse(&result, ctx, str))
-		pr_err("expr__parse failed\n");
-	else
-		ret = 0;
+	ret = 0;
+	if (expr__parse(&result, ctx, str)) {
+		/*
+		 * Parsing failed, make numbers go from large to small which can
+		 * resolve divide by zero issues.
+		 */
+		i = 1024;
+		hashmap__for_each_entry(ctx->ids, cur, bkt)
+			expr__add_id_val(ctx, strdup(cur->key), i--);
+		if (expr__parse(&result, ctx, str)) {
+			pr_err("expr__parse failed\n");
+			ret = -1;
+		}
+	}
 
  out:
  	expr__ctx_free(ctx);
#2 lore raw

Em Fri, Dec 24, 2021 at 08:49:35AM +0000, John Garry escreveu:

Quoted 16 lines · Ian Rogers
Quoted 7 lines

When testing metric expressions we fake counter values from 1 going upward. For some metrics this can yield negative values that are clipped to zero, and then cause divide by zero failures. Such clipping is questionable but may be a result of tools automatically generating metrics. A workaround for this case is to try a second time with counter values going in the opposite direction.

Quoted 7 lines

This case was seen in a metric like:

   event1 / max(event2 - event3, 0)
But it may also happen in more sensible metrics like:
   event1 / (event2 + event3 - 1 - event4)

v2. Rebase and more detail in commit message.
v3. Is a rebase.

Quoted 6 lines

Incorrect location for this info

Quoted 2 lines

Found thru the cracks, applying.

- Arnaldo

Quoted 55 lines
Quoted 9 lines
---
  tools/perf/tests/pmu-events.c | 32 ++++++++++++++++++++++++++------
  1 file changed, 26 insertions(+), 6 deletions(-)
tools/perf/tests/pmu-events.c +0 -0
index df1c9a3cc05b..b2ddf928d32a 100644

tools/perf/tests/pmu-events.c static int test__parsing(struct test_suite *test __maybe_unused, · line 966

Quoted 7 lines
tools/perf/tests/pmu-events.c +3 -2
@@ -962,8 +962,18 @@ static int test__parsing(struct test_suite *test __maybe_unused,
  			}
  			if (expr__parse(&result, ctx, pe->metric_expr)) {
-				expr_failure("Parse failed", map, pe);
-				ret++;
+				/*
+				 * Parsing failed, make numbers go from large to
+				 * small which can resolve divide by zero

can or may?

Quoted 36 lines
tools/perf/tests/pmu-events.c +23 -4
+				 * issues.
+				 */
+				k = 1024;
+				hashmap__for_each_entry(ctx->ids, cur, bkt)
+					expr__add_id_val(ctx, strdup(cur->key), k--);
+				if (expr__parse(&result, ctx, pe->metric_expr)) {
+					expr_failure("Parse failed", map, pe);
+					ret++;
+				}
  			}
  		}
  	}
@@ -1022,10 +1032,20 @@ static int metric_parse_fake(const char *str)
  		}
  	}
-	if (expr__parse(&result, ctx, str))
-		pr_err("expr__parse failed\n");
-	else
-		ret = 0;
+	ret = 0;
+	if (expr__parse(&result, ctx, str)) {
+		/*
+		 * Parsing failed, make numbers go from large to small which can
+		 * resolve divide by zero issues.
+		 */
+		i = 1024;
+		hashmap__for_each_entry(ctx->ids, cur, bkt)
+			expr__add_id_val(ctx, strdup(cur->key), i--);
+		if (expr__parse(&result, ctx, str)) {
+			pr_err("expr__parse failed\n");
+			ret = -1;
+		}
+	}
  out:
  	expr__ctx_free(ctx);
 
Signature · 1 line
- Arnaldo