perf metric: Add support to reuse metric
Jiri Olsa
reviewerv4 cover perf metric: Add support to reuse metric
hi,
this patchset is adding the support to reused metric in another metric.
For example, to define IPC by using CPI with change like:
{
"BriefDescription": "Instructions Per Cycle (per Logical Processor)",
- "MetricExpr": "INST_RETIRED.ANY / CPU_CLK_UNHALTED.THREAD",
+ "MetricExpr": "1/CPI",
"MetricGroup": "TopDownL1",
"MetricName": "IPC"
},I won't be able to find all the possible places we could use this at, so I wonder you guys (who was asking for this) would try it and come up with comments if there's something missing or we could already use it at some places.
It's based on Arnaldo's tmp.perf/core.
v4 changes:
- removed acks from patch because it changed a bit
with the last fixes:
perf metric: Collect referenced metrics in struct metric_ref_node
- fixed runtime metrics [Kajol Jain]
- increased recursion depth [Paul A. Clarke]
- changed patches due to dependencies:
perf metric: Collect referenced metrics in struct metric_ref_node perf metric: Add recursion check when processing nested metrics perf metric: Rename struct egroup to metric
perf metric: Rename group_list to metric_list
Also available in here:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/metric
thanks,
jirka
---
Jiri Olsa (19):
perf metric: Fix memory leak in expr__add_id function perf metric: Add expr__add_id function
perf metric: Change expr__get_id to return struct expr_id_data perf metric: Add expr__del_id function
perf metric: Add macros for iterating map events
perf metric: Add add_metric function
perf metric: Rename __metricgroup__add_metric to __add_metric perf metric: Collect referenced metrics in struct metric_ref_node perf metric: Collect referenced metrics in struct metric_expr perf metric: Add referenced metrics to hash data
perf metric: Compute referenced metrics
perf metric: Add events for the current list
perf metric: Add cache_miss_cycles to metric parse test perf metric: Add DCache_L2 to metric parse test
perf metric: Add recursion check when processing nested metrics perf metric: Make compute_single function more precise perf metric: Add metric group test
perf metric: Rename struct egroup to metric
perf metric: Rename group_list to metric_list
tools/perf/tests/parse-metric.c | 206 +++++++++++++++++++++++++++++++++++++++++++++++++---- tools/perf/util/expr.c | 143 ++++++++++++++++++++++++++++++++----- tools/perf/util/expr.h | 30 +++++++- tools/perf/util/expr.y | 16 +++-- tools/perf/util/metricgroup.c | 466 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------------------------ tools/perf/util/metricgroup.h | 6 ++ tools/perf/util/stat-shadow.c | 20 ++++-- 7 files changed, 751 insertions(+), 136 deletions(-)
Jiri Olsa
reviewerPATCH v4 1/19 perf metric: Fix memory leak in expr__add_id function
Arnaldo found that we don't release value data in case the hashmap__set fails. Releasing it in case of an error.
--- tools/perf/util/expr.c | 2 ++ 1 file changed, 2 insertions(+)
tools/perf/util/expr.c +2 -0
@@ -47,6 +47,8 @@ int expr__add_id_val(struct expr_parse_ctx *ctx, const char *id, double val) } ret =hashmap__set (&ctx->ids, id, data_ptr, (const void **)&old_key, (void **)&old_data); +if (ret) +free (data_ptr);free (old_key);free (old_data);return ret;
Signature · 1 line
2.25.4
Jiri Olsa
reviewerPATCH v4 2/19 perf metric: Add expr__add_id function
In reply to Jiri OlsaAdd the expr__add_id() function to data for ID with zero value, which is used when scanning the expression for IDs.
--- tools/perf/util/expr.c | 31 +++++++++++++++++++++++++------ tools/perf/util/expr.h | 1 + tools/perf/util/expr.y | 2 +- 3 files changed, 27 insertions(+), 7 deletions(-)
tools/perf/util/expr.c +25 -6
@@ -32,6 +32,26 @@ static bool key_equal(const void *key1, const void *key2,return !strcmp ((const char *)key1, (const char *)key2); } +/* Caller must make sure id is allocated */ +int expr__add_id (struct expr_parse_ctx *ctx,const char *id) +{ +struct expr_id_data *data_ptr =NULL , *old_data =NULL ; +char *old_key =NULL ; +int ret; + + data_ptr =malloc (sizeof (*data_ptr)); +if (!data_ptr) +return -ENOMEM ; + + ret =hashmap__set (&ctx->ids, id, data_ptr, + (const void **)&old_key, (void **)&old_data); +if (ret) +free (data_ptr); +free (old_key); +free (old_data); +return ret; +} +/* Caller must make sure id is allocated */ int expr__add_id_val (struct expr_parse_ctx *ctx,const char *id,double val) {
@@ -39,12 +59,11 @@ int expr__add_id_val(struct expr_parse_ctx *ctx, const char *id, double val)char *old_key =NULL ;int ret; -if (val !=0.0 ) { - data_ptr =malloc (sizeof (*data_ptr)); -if (!data_ptr) -return -ENOMEM ; - data_ptr->val = val; - } + data_ptr =malloc (sizeof (*data_ptr)); +if (!data_ptr) +return -ENOMEM ; + data_ptr->val = val; + ret =hashmap__set (&ctx->ids, id, data_ptr, (const void **)&old_key, (void **)&old_data);if (ret)
tools/perf/util/expr.h +1 -0
@@ -26,6 +26,7 @@ struct expr_scanner_ctx {void expr__ctx_init (struct expr_parse_ctx *ctx);void expr__ctx_clear (struct expr_parse_ctx *ctx); +int expr__add_id (struct expr_parse_ctx *ctx,const char *id);int expr__add_id_val (struct expr_parse_ctx *ctx,const char *id,double val);int expr__get_id (struct expr_parse_ctx *ctx,const char *id,double *val_ptr); int expr__parse(double *final_val, struct expr_parse_ctx *ctx,
tools/perf/util/expr.y +1 -1
@@ -69,7 +69,7 @@ all_other: all_other other other: ID { - expr__add_id_val(ctx, $1, 0.0); + expr__add_id(ctx, $1); } | MIN | MAX | IF | ELSE | SMT_ON | NUMBER | '|' | '^' | '&' | '-' | '+' | '*' | '/' | '%' | '(' | ')' | ','
Signature · 1 line
2.25.4
Jiri Olsa
reviewerPATCH v4 3/19 perf metric: Change expr__get_id to return struct expr_id_data
In reply to Jiri OlsaChanging expr__get_id to use and return struct expr_id_data pointer as value for the ID. This way we can access data other than value for given ID in following changes.
--- tools/perf/util/expr.c | 10 +++------- tools/perf/util/expr.h | 3 ++- tools/perf/util/expr.y | 14 +++++++++----- 3 files changed, 14 insertions(+), 13 deletions(-)
tools/perf/util/expr.c +3 -7
@@ -73,14 +73,10 @@ int expr__add_id_val(struct expr_parse_ctx *ctx, const char *id, double val)return ret; } -int expr__get_id (struct expr_parse_ctx *ctx,const char *id,double *val_ptr) +int expr__get_id (struct expr_parse_ctx *ctx,const char *id, +struct expr_id_data **data) { -struct expr_id_data *data; - -if (!hashmap__find (&ctx->ids, id, (void **)&data)) -return -1 ; - *val_ptr = (data ==NULL ) ?0.0 : data->val; -return 0 ; +return hashmap__find (&ctx->ids, id, (void **)data) ?0 :-1 ; }void expr__ctx_init (struct expr_parse_ctx *ctx)
tools/perf/util/expr.h +2 -1
@@ -28,7 +28,8 @@ void expr__ctx_init(struct expr_parse_ctx *ctx);void expr__ctx_clear (struct expr_parse_ctx *ctx);int expr__add_id (struct expr_parse_ctx *ctx,const char *id);int expr__add_id_val (struct expr_parse_ctx *ctx,const char *id,double val); -int expr__get_id (struct expr_parse_ctx *ctx,const char *id,double *val_ptr); +int expr__get_id (struct expr_parse_ctx *ctx,const char *id, +struct expr_id_data **data);int expr__parse (double *final_val,struct expr_parse_ctx *ctx,const char *expr,int runtime); int expr__find_other(const char *expr, const char *one,
tools/perf/util/expr.y +9 -5
@@ -85,12 +85,16 @@ if_expr: ; expr: NUMBER - | ID { if (expr__get_id(ctx, $1, &$$)) { - pr_debug("%s not found\n", $1); + | ID { + struct expr_id_data *data; + + if (expr__get_id(ctx, $1, &data) || !data) { + pr_debug("%s not found\n", $1); + free($1); + YYABORT; + } + $$ = data->val; free($1); - YYABORT; - } - free($1); } | expr '|' expr { $$ = (long)$1 | (long)$3; } | expr '&' expr { $$ = (long)$1 & (long)$3; }
Signature · 1 line
2.25.4
Jiri Olsa
reviewerPATCH v4 4/19 perf metric: Add expr__del_id function
In reply to Jiri OlsaAdding expr__del_id function to remove ID from hashmap. It will save us few lines in following changes.
--- tools/perf/util/expr.c | 21 +++++++++++++-------- tools/perf/util/expr.h | 1 + 2 files changed, 14 insertions(+), 8 deletions(-)
tools/perf/util/expr.c +13 -8
@@ -79,6 +79,17 @@ int expr__get_id(struct expr_parse_ctx *ctx, const char *id,return hashmap__find (&ctx->ids, id, (void **)data) ?0 :-1 ; } +void expr__del_id (struct expr_parse_ctx *ctx,const char *id) +{ +struct expr_id_data *old_val =NULL ; +char *old_key =NULL ; + +hashmap__delete (&ctx->ids, id, + (const void **)&old_key, (void **)&old_val); +free (old_key); +free (old_val); +} +void expr__ctx_init (struct expr_parse_ctx *ctx) {hashmap__init (&ctx->ids, key_hash, key_equal,NULL );
@@ -136,16 +147,10 @@ int expr__parse(double *final_val, struct expr_parse_ctx *ctx,int expr__find_other (const char *expr,const char *one,struct expr_parse_ctx *ctx,int runtime) { -struct expr_id_data *old_val =NULL ; -char *old_key =NULL ;int ret =__expr__parse (NULL , ctx, expr,EXPR_OTHER , runtime); -if (one) { -hashmap__delete (&ctx->ids, one, - (const void **)&old_key, (void **)&old_val); -free (old_key); -free (old_val); - } +if (one) +expr__del_id (ctx, one);return ret; }
tools/perf/util/expr.h +1 -0
@@ -26,6 +26,7 @@ struct expr_scanner_ctx {void expr__ctx_init (struct expr_parse_ctx *ctx);void expr__ctx_clear (struct expr_parse_ctx *ctx); +void expr__del_id (struct expr_parse_ctx *ctx,const char *id);int expr__add_id (struct expr_parse_ctx *ctx,const char *id);int expr__add_id_val (struct expr_parse_ctx *ctx,const char *id,double val); int expr__get_id(struct expr_parse_ctx *ctx, const char *id,
Signature · 1 line
2.25.4
Jiri Olsa
reviewerPATCH v4 5/19 perf metric: Add macros for iterating map events
In reply to Jiri OlsaAdding following macros to iterate events and metric:
map_for_each_event(__pe, __idx, __map)
- iterates over all pmu_events_map events
map_for_each_metric(__pe, __idx, __map, __metric)
- iterates over all metrics that match __metric argument
and use it in metricgroup__add_metric function. Macros will be be used from other places in following changes.
--- tools/perf/util/metricgroup.c | 77 ++++++++++++++++++----------------- 1 file changed, 40 insertions(+), 37 deletions(-)
tools/perf/util/metricgroup.c +40 -37
@@ -614,6 +614,17 @@ static int __metricgroup__add_metric(struct list_head *group_list,return 0 ; } +#define map_for_each_event (__pe, __idx, __map) \ + for (__idx = 0, __pe = &__map->table[__idx]; \ + __pe->name || __pe->metric_group || __pe->metric_name; \ + __pe = &__map->table[++__idx]) + +#define map_for_each_metric (__pe, __idx, __map, __metric) \ + map_for_each_event(__pe, __idx, __map) \ + if (__pe->metric_expr && \ + (match_metric(__pe->metric_group, __metric) || \ + match_metric(__pe->metric_name, __metric))) + static int metricgroup__add_metric(const char *metric, bool metric_no_group, struct strbuf *events, struct list_head *group_list,
@@ -624,49 +635,41 @@ static int metricgroup__add_metric(const char *metric, bool metric_no_group,int i, ret;bool has_match = false; -for (i =0 ; ; i++) { - pe = &map->table[i]; - -if (!pe->name && !pe->metric_group && !pe->metric_name) { -/* End of pmu events. */ -if (!has_match) -return -EINVAL ; -break ; - } -if (!pe->metric_expr) -continue ; -if (match_metric (pe->metric_group, metric) || -match_metric (pe->metric_name, metric)) { - has_match = true; -pr_debug ("metric expr %s for %s\n" , pe->metric_expr, pe->metric_name); - -if (!strstr (pe->metric_expr,"?" )) { - ret =__metricgroup__add_metric (group_list, - pe, - metric_no_group, -1 ); -if (ret) -return ret; - }else { -int j, count; +map_for_each_metric (pe, i, map, metric) { +pr_debug ("metric expr %s for %s\n" , pe->metric_expr, pe->metric_name); + has_match = true; + +if (!strstr (pe->metric_expr,"?" )) { + ret =__metricgroup__add_metric (group_list, + pe, + metric_no_group, +1 ); +if (ret) +return ret; + }else { +int j, count; - count =arch_get_runtimeparam (); + count =arch_get_runtimeparam (); -/* This loop is added to create multiple -* events depend on count value and add -* those events to group_list. -*/ +/* This loop is added to create multiple +* events depend on count value and add +* those events to group_list. +*/ -for (j =0 ; j < count; j++) { - ret =__metricgroup__add_metric ( - group_list, pe, - metric_no_group, j); -if (ret) -return ret; - } +for (j =0 ; j < count; j++) { + ret =__metricgroup__add_metric ( + group_list, pe, + metric_no_group, j); +if (ret) +return ret; } } } + +/* End of pmu events. */ +if (!has_match) +return -EINVAL ; +list_for_each_entry (eg, group_list, nd) {if (events->len >0 )strbuf_addf (events,"," );
Signature · 1 line
2.25.4
Jiri Olsa
reviewerPATCH v4 6/19 perf metric: Add add_metric function
In reply to Jiri OlsaDecouple metric adding logging into add_metric function, so it can be used from other places in following changes.
--- tools/perf/util/metricgroup.c | 62 ++++++++++++++++++++--------------- 1 file changed, 36 insertions(+), 26 deletions(-)
tools/perf/util/metricgroup.c +36 -26
@@ -625,6 +625,39 @@ static int __metricgroup__add_metric(struct list_head *group_list, (match_metric(__pe->metric_group, __metric) || \ match_metric(__pe->metric_name, __metric))) +static int add_metric (struct list_head *group_list, +struct pmu_event *pe, +bool metric_no_group) +{ +int ret =0 ; + +pr_debug ("metric expr %s for %s\n" , pe->metric_expr, pe->metric_name); + +if (!strstr (pe->metric_expr,"?" )) { + ret =__metricgroup__add_metric (group_list, + pe, + metric_no_group, +1 ); + }else { +int j, count; + + count =arch_get_runtimeparam (); + +/* This loop is added to create multiple +* events depend on count value and add +* those events to group_list. +*/ + +for (j =0 ; j < count && !ret; j++) { + ret =__metricgroup__add_metric ( + group_list, pe, + metric_no_group, j); + } + } + +return ret; +} + static int metricgroup__add_metric(const char *metric, bool metric_no_group, struct strbuf *events, struct list_head *group_list,
@@ -636,34 +669,11 @@ static int metricgroup__add_metric(const char *metric, bool metric_no_group,bool has_match = false;map_for_each_metric (pe, i, map, metric) { -pr_debug ("metric expr %s for %s\n" , pe->metric_expr, pe->metric_name); has_match = true; -if (!strstr (pe->metric_expr,"?" )) { - ret =__metricgroup__add_metric (group_list, - pe, - metric_no_group, -1 ); -if (ret) -return ret; - }else { -int j, count; - - count =arch_get_runtimeparam (); - -/* This loop is added to create multiple -* events depend on count value and add -* those events to group_list. -*/ - -for (j =0 ; j < count; j++) { - ret =__metricgroup__add_metric ( - group_list, pe, - metric_no_group, j); -if (ret) -return ret; - } - } + ret =add_metric (group_list, pe, metric_no_group); +if (ret) +return ret; }/* End of pmu events. */
Signature · 1 line
2.25.4
Jiri Olsa
reviewerPATCH v4 7/19 perf metric: Rename __metricgroup__add_metric to __add_metric
In reply to Jiri OlsaRenaming __metricgroup__add_metric to __add_metric to fit in the current function names.
--- tools/perf/util/metricgroup.c | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-)
tools/perf/util/metricgroup.c +6 -11
@@ -571,10 +571,10 @@ int __weak arch_get_runtimeparam(void)return 1 ; } -static int __metricgroup__add_metric (struct list_head *group_list, -struct pmu_event *pe, -bool metric_no_group, -int runtime) +static int __add_metric (struct list_head *group_list, +struct pmu_event *pe, +bool metric_no_group, +int runtime) {struct egroup *eg;
@@ -634,10 +634,7 @@ static int add_metric(struct list_head *group_list,pr_debug ("metric expr %s for %s\n" , pe->metric_expr, pe->metric_name);if (!strstr (pe->metric_expr,"?" )) { - ret =__metricgroup__add_metric (group_list, - pe, - metric_no_group, -1 ); + ret =__add_metric (group_list, pe, metric_no_group,1 ); }else {int j, count;
@@ -649,9 +646,7 @@ static int add_metric(struct list_head *group_list,*/ for (j =0 ; j < count && !ret; j++) { - ret =__metricgroup__add_metric ( - group_list, pe, - metric_no_group, j); + ret =__add_metric (group_list, pe, metric_no_group, j); } }
Signature · 1 line
2.25.4
Jiri Olsa
reviewerPATCH v4 8/19 perf metric: Collect referenced metrics in struct metric_ref_node
In reply to Jiri OlsaCollecting referenced metrics in struct metric_ref_node object, so we can process them later on.
The change will parse nested metric names out of expression and 'resolve' them.
All referenced metrics are dissolved into one context, meaning all nested metrics events and added to the parent context.
--- tools/perf/util/metricgroup.c | 170 +++++++++++++++++++++++++++++++--- 1 file changed, 156 insertions(+), 14 deletions(-)
tools/perf/util/metricgroup.c +156 -14
@@ -102,12 +102,25 @@ void metricgroup__rblist_exit(struct rblist *metric_events)rblist__exit (metric_events); } +/* +* A node in the list of referenced metrics. metric_expr +* is held as a convenience to avoid a search through the +* metric list. +*/ +struct metric_ref_node { +const char *metric_name; +const char *metric_expr; +struct list_head list; +}; +struct egroup {struct list_head nd;struct expr_parse_ctx pctx;const char *metric_name;const char *metric_expr;const char *metric_unit; +struct list_head metric_refs; +int metric_refs_cnt;int runtime;bool has_constraint; };
@@ -574,27 +587,72 @@ int __weak arch_get_runtimeparam(void)static int __add_metric (struct list_head *group_list,struct pmu_event *pe,bool metric_no_group, -int runtime) +int runtime, +struct egroup **egp) { +struct metric_ref_node *ref;struct egroup *eg; - eg =malloc (sizeof (*eg)); -if (!eg) -return -ENOMEM ; +if (*egp ==NULL ) { +/* +* We got in here for the parent group, +* allocate it and put it on the list. +*/ + eg =malloc (sizeof (*eg)); +if (!eg) +return -ENOMEM ; + +expr__ctx_init (&eg->pctx); + eg->metric_name = pe->metric_name; + eg->metric_expr = pe->metric_expr; + eg->metric_unit = pe->unit; + eg->runtime = runtime; + eg->has_constraint = metric_no_group ||metricgroup__has_constraint (pe); +INIT_LIST_HEAD (&eg->metric_refs); + eg->metric_refs_cnt =0 ; + *egp = eg; + }else { +/* +* We got here for the referenced metric, via the +* recursive metricgroup__add_metric call, add +* it to the parent group. +*/ + eg = *egp; + + ref =malloc (sizeof (*ref)); +if (!ref) +return -ENOMEM ; + +/* +* Intentionally passing just const char pointers, +* from 'pe' object, so they never go away. We don't +* need to change them, so there's no need to create +* our own copy. +*/ + ref->metric_name = pe->metric_name; + ref->metric_expr = pe->metric_expr; -expr__ctx_init (&eg->pctx); - eg->metric_name = pe->metric_name; - eg->metric_expr = pe->metric_expr; - eg->metric_unit = pe->unit; - eg->runtime = runtime; - eg->has_constraint = metric_no_group ||metricgroup__has_constraint (pe); +list_add (&ref->list, &eg->metric_refs); + eg->metric_refs_cnt++; + } +/* +* For both the parent and referenced metrics, we parse +* all the metric's IDs and add it to the parent context. +*/ if (expr__find_other (pe->metric_expr,NULL , &eg->pctx, runtime) <0 ) {expr__ctx_clear (&eg->pctx);free (eg);return -EINVAL ; } +/* +* We add new group only in the 'parent' call, +* so bail out for referenced metric case. +*/ +if (eg->metric_refs_cnt) +return 0 ; +if (list_empty (group_list))list_add (&eg->nd, group_list);else {
@@ -625,16 +683,78 @@ static int __add_metric(struct list_head *group_list, (match_metric(__pe->metric_group, __metric) || \ match_metric(__pe->metric_name, __metric))) +static struct pmu_event *find_metric (const char *metric,struct pmu_events_map *map) +{ +struct pmu_event *pe; +int i; + +map_for_each_event (pe, i, map) { +if (match_metric (pe->metric_name, metric)) +return pe; + } + +return NULL ; +} + +static int add_metric (struct list_head *group_list, +struct pmu_event *pe, +bool metric_no_group, +struct egroup **egp); + +static int resolve_metric (struct egroup *eg, +bool metric_no_group, +struct list_head *group_list, +struct pmu_events_map *map) +{ +struct hashmap_entry *cur; +size_t bkt; +bool all; +int ret; + +/* +* Iterate all the parsed IDs and if there's metric, +* add it to the context. +*/ +do { + all = true; +hashmap__for_each_entry ((&eg->pctx.ids), cur, bkt) { +struct pmu_event *pe; + + pe =find_metric (cur->key, map); +if (!pe) +continue ; + + all = false; +/* The metric key itself needs to go out.. */ +expr__del_id (&eg->pctx, cur->key); + +/* ... and it gets resolved to the parent context. */ + ret =add_metric (group_list, pe, metric_no_group, &eg); +if (ret) +return ret; + +/* +* We added new metric to hashmap, so we need +* to break the iteration and start over. +*/ +break ; + } + }while (!all); + +return 0 ; +} +static int add_metric (struct list_head *group_list,struct pmu_event *pe, -bool metric_no_group) +bool metric_no_group, +struct egroup **egp) {int ret =0 ;pr_debug ("metric expr %s for %s\n" , pe->metric_expr, pe->metric_name);if (!strstr (pe->metric_expr,"?" )) { - ret =__add_metric (group_list, pe, metric_no_group,1 ); + ret =__add_metric (group_list, pe, metric_no_group,1 , egp); }else {int j, count;
@@ -646,7 +766,7 @@ static int add_metric(struct list_head *group_list,*/ for (j =0 ; j < count && !ret; j++) { - ret =__add_metric (group_list, pe, metric_no_group, j); + ret =__add_metric (group_list, pe, metric_no_group, j, egp); } }
@@ -657,6 +777,7 @@ static int metricgroup__add_metric(const char *metric, bool metric_no_group,struct strbuf *events, struct list_head *group_list, struct pmu_events_map *map) + { struct pmu_event *pe;struct egroup *eg;
@@ -665,8 +786,18 @@ static int metricgroup__add_metric(const char *metric, bool metric_no_group,map_for_each_metric (pe, i, map, metric) { has_match = true; + eg =NULL ; - ret =add_metric (group_list, pe, metric_no_group); + ret =add_metric (group_list, pe, metric_no_group, &eg); +if (ret) +return ret; + +/* +* Process any possible referenced metrics +* included in the expression. +*/ + ret =resolve_metric (eg, metric_no_group, + group_list, map);if (ret)return ret; }
@@ -723,11 +854,22 @@ static int metricgroup__add_metric_list(const char *list, bool metric_no_group,return ret; } +static void egroup__free_refs (struct egroup *egroup) +{ +struct metric_ref_node *ref, *tmp; + +list_for_each_entry_safe (ref, tmp, &egroup->metric_refs, list) { +list_del (&ref->list); +free (ref); + } +} +static void metricgroup__free_egroups (struct list_head *group_list) {struct egroup *eg, *egtmp; list_for_each_entry_safe (eg, egtmp, group_list, nd) { +egroup__free_refs (eg);expr__ctx_clear (&eg->pctx);list_del_init (&eg->nd);free (eg);
Signature · 1 line
2.25.4
Jiri Olsa
reviewerPATCH v4 9/19 perf metric: Collect referenced metrics in struct metric_expr
In reply to Jiri OlsaAdd referenced metrics into struct metric_expr object, so they are accessible when computing the metric.
Storing just name and expression itself, so the metric can be resolved and computed.
--- tools/perf/util/metricgroup.c | 32 ++++++++++++++++++++++++++++++++ tools/perf/util/metricgroup.h | 6 ++++++ 2 files changed, 38 insertions(+)
tools/perf/util/metricgroup.c +32 -0
@@ -83,6 +83,7 @@ static void metric_event_delete(struct rblist *rblist __maybe_unused,struct metric_expr *expr, *tmp;list_for_each_entry_safe (expr, tmp, &me->head, nd) { +free (expr->metric_refs);free (expr); }
@@ -248,6 +249,7 @@ static int metricgroup__setup_events(struct list_head *groups, list_for_each_entry (eg, groups, nd) {struct evsel **metric_events; +struct metric_ref *metric_refs =NULL ; metric_events =calloc (sizeof (void *),hashmap__size (&eg->pctx.ids) +1 );
@@ -279,6 +281,36 @@ static int metricgroup__setup_events(struct list_head *groups,free (metric_events);break ; } + +/* +* Collect and store collected nested expressions +* for metric processing. +*/ +if (eg->metric_refs_cnt) { +struct metric_ref_node *ref; + + metric_refs =zalloc (sizeof (struct metric_ref ) * (eg->metric_refs_cnt +1 )); +if (!metric_refs) { + ret = -ENOMEM ; +free (metric_events); +break ; + } + + i =0 ; +list_for_each_entry (ref, &eg->metric_refs, list) { +/* +* Intentionally passing just const char pointers, +* originally from 'struct pmu_event' object. +* We don't need to change them, so there's no +* need to create our own copy. +*/ + metric_refs[i].metric_name = ref->metric_name; + metric_refs[i].metric_expr = ref->metric_expr; + i++; + } + }; + + expr->metric_refs = metric_refs; expr->metric_expr = eg->metric_expr; expr->metric_name = eg->metric_name; expr->metric_unit = eg->metric_unit;
tools/perf/util/metricgroup.h +6 -0
@@ -18,12 +18,18 @@ struct metric_event {struct list_head head;/* list of metric_expr */ }; +struct metric_ref { +const char *metric_name; +const char *metric_expr; +}; +struct metric_expr {struct list_head nd;const char *metric_expr;const char *metric_name;const char *metric_unit;struct evsel **metric_events; +struct metric_ref *metric_refs;int runtime; };
Signature · 1 line
2.25.4
Jiri Olsa
reviewerPATCH v4 10/19 perf metric: Add referenced metrics to hash data
In reply to Jiri OlsaAdding referenced metrics to the parsing context so they can be resolved during the metric processing.
Adding expr__add_ref function to store referenced metrics into parse context.
--- tools/perf/util/expr.c | 54 +++++++++++++++++++++++++++++++++++ tools/perf/util/expr.h | 13 ++++++++- tools/perf/util/stat-shadow.c | 20 +++++++++---- 3 files changed, 80 insertions(+), 7 deletions(-)
tools/perf/util/expr.c +54 -0
@@ -4,10 +4,14 @@#include <errno.h> #include <stdlib.h> #include <string.h> +#include "metricgroup.h" +#include "debug.h" #include "expr.h" #include "expr-bison.h" #include "expr-flex.h" #include <linux/kernel.h> +#include <linux/zalloc.h> +#include <ctype.h> #ifdef PARSER_DEBUG extern int expr_debug;
@@ -63,6 +67,7 @@ int expr__add_id_val(struct expr_parse_ctx *ctx, const char *id, double val)if (!data_ptr)return -ENOMEM ; data_ptr->val = val; + data_ptr->is_ref = false; ret =hashmap__set (&ctx->ids, id, data_ptr, (const void **)&old_key, (void **)&old_data);
@@ -73,6 +78,55 @@ int expr__add_id_val(struct expr_parse_ctx *ctx, const char *id, double val)return ret; } +int expr__add_ref (struct expr_parse_ctx *ctx,struct metric_ref *ref) +{ +struct expr_id_data *data_ptr =NULL , *old_data =NULL ; +char *old_key =NULL ; +char *name, *p; +int ret; + + data_ptr =zalloc (sizeof (*data_ptr)); +if (!data_ptr) +return -ENOMEM ; + + name =strdup (ref->metric_name); +if (!name) { +free (data_ptr); +return -ENOMEM ; + } + +/* +* The jevents tool converts all metric expressions +* to lowercase, including metric references, hence +* we need to add lowercase name for metric, so it's +* properly found. +*/ +for (p = name; *p; p++) + *p =tolower (*p); + +/* +* Intentionally passing just const char pointers, +* originally from 'struct pmu_event' object. +* We don't need to change them, so there's no +* need to create our own copy. +*/ + data_ptr->ref.metric_name = ref->metric_name; + data_ptr->ref.metric_expr = ref->metric_expr; + data_ptr->is_ref = true; + + ret =hashmap__set (&ctx->ids, name, data_ptr, + (const void **)&old_key, (void **)&old_data); +if (ret) +free (data_ptr); + +pr_debug2 ("adding ref metric %s: %s\n" , + ref->metric_name, ref->metric_expr); + +free (old_key); +free (old_data); +return ret; +} +int expr__get_id (struct expr_parse_ctx *ctx,const char *id,struct expr_id_data **data) {
tools/perf/util/expr.h +12 -1
@@ -11,12 +11,22 @@#include "util/hashmap.h" //#endif +struct metric_ref ; +struct expr_parse_ctx {struct hashmap ids; };struct expr_id_data { -double val; +union { +double val; +struct { +const char *metric_name; +const char *metric_expr; + } ref; + }; + +bool is_ref; };struct expr_scanner_ctx {
@@ -29,6 +39,7 @@ void expr__ctx_clear(struct expr_parse_ctx *ctx);void expr__del_id (struct expr_parse_ctx *ctx,const char *id);int expr__add_id (struct expr_parse_ctx *ctx,const char *id);int expr__add_id_val (struct expr_parse_ctx *ctx,const char *id,double val); +int expr__add_ref (struct expr_parse_ctx *ctx,struct metric_ref *ref);int expr__get_id (struct expr_parse_ctx *ctx,const char *id,struct expr_id_data **data); int expr__parse(double *final_val, struct expr_parse_ctx *ctx,
tools/perf/util/stat-shadow.c +14 -6
@@ -731,13 +731,14 @@ static void print_smi_cost(struct perf_stat_config *config, }static int prepare_metric (struct evsel **metric_events, +struct metric_ref *metric_refs,struct expr_parse_ctx *pctx,int cpu,struct runtime_stat *st) {double scale;char *n, *pn; -int i; +int i, j, ret;expr__ctx_init (pctx);for (i =0 ; metric_events[i]; i++) {
@@ -778,12 +779,19 @@ static int prepare_metric(struct evsel **metric_events,expr__add_id_val (pctx, n,avg_stats (stats)*scale); } +for (j =0 ; metric_refs && metric_refs[j].metric_name; j++) { + ret =expr__add_ref (pctx, &metric_refs[j]); +if (ret) +return ret; + } +return i; } static void generic_metric(struct perf_stat_config *config, const char *metric_expr, struct evsel **metric_events, + struct metric_ref *metric_refs, char *name, const char *metric_name, const char *metric_unit,
@@ -798,7 +806,7 @@ static void generic_metric(struct perf_stat_config *config,int i;void *ctxp = out->ctx; - i =prepare_metric (metric_events, &pctx, cpu, st); + i =prepare_metric (metric_events, metric_refs, &pctx, cpu, st);if (i <0 )return ;
@@ -847,7 +855,7 @@ double test_generic_metric(struct metric_expr *mexp, int cpu, struct runtime_stastruct expr_parse_ctx pctx;double ratio; -if (prepare_metric (mexp->metric_events, &pctx, cpu, st) <0 ) +if (prepare_metric (mexp->metric_events, mexp->metric_refs, &pctx, cpu, st) <0 )return 0. ;if (expr__parse (&ratio, &pctx, mexp->metric_expr,1 ))
@@ -1064,8 +1072,8 @@ void perf_stat__print_shadow_stats(struct perf_stat_config *config,else print_metric (config, ctxp,NULL ,NULL , name,0 ); }else if (evsel->metric_expr) { -generic_metric (config, evsel->metric_expr, evsel->metric_events, evsel->name, - evsel->metric_name,NULL ,1 , cpu, out, st); +generic_metric (config, evsel->metric_expr, evsel->metric_events,NULL , + evsel->name, evsel->metric_name,NULL ,1 , cpu, out, st); }else if (runtime_stat_n (st,STAT_NSECS ,0 , cpu) !=0 ) {char unit ='M' ;char unit_buf[10 ];
@@ -1093,7 +1101,7 @@ void perf_stat__print_shadow_stats(struct perf_stat_config *config,if (num++ >0 ) out->new_line (config, ctxp);generic_metric (config, mexp->metric_expr, mexp->metric_events, - evsel->name, mexp->metric_name, + mexp->metric_refs, evsel->name, mexp->metric_name, mexp->metric_unit, mexp->runtime, cpu, out, st); } }
Signature · 1 line
2.25.4
Jiri Olsa
reviewerPATCH v4 11/19 perf metric: Compute referenced metrics
In reply to Jiri OlsaAdding computation (expr__parse call) of referenced metric at the point when it needs to be resolved during the parent metric computation.
Once the inner metric is computed, the result is stored and used if there's another usage of that metric.
--- tools/perf/util/expr.c | 31 +++++++++++++++++++++++++++++++ tools/perf/util/expr.h | 3 +++ tools/perf/util/expr.y | 4 ++-- 3 files changed, 36 insertions(+), 2 deletions(-)
tools/perf/util/expr.c +31 -0
@@ -112,6 +112,7 @@ int expr__add_ref(struct expr_parse_ctx *ctx, struct metric_ref *ref)*/ data_ptr->ref.metric_name = ref->metric_name; data_ptr->ref.metric_expr = ref->metric_expr; + data_ptr->ref.counted = false; data_ptr->is_ref = true; ret = hashmap__set(&ctx->ids, name, data_ptr,
@@ -133,6 +134,34 @@ int expr__get_id(struct expr_parse_ctx *ctx, const char *id,return hashmap__find (&ctx->ids, id, (void **)data) ?0 :-1 ; } +int expr__resolve_id (struct expr_parse_ctx *ctx,const char *id, +struct expr_id_data **datap) +{ +struct expr_id_data *data; + +if (expr__get_id (ctx, id, datap) || !*datap) { +pr_debug ("%s not found\n" , id); +return -1 ; + } + + data = *datap; + +pr_debug2 ("lookup: is_ref %d, counted %d, val %f: %s\n" , + data->is_ref, data->ref.counted, data->val, id); + +if (data->is_ref && !data->ref.counted) { + data->ref.counted = true; +pr_debug ("processing metric: %s ENTRY\n" , id); +if (expr__parse (&data->val, ctx, data->ref.metric_expr,1 )) { +pr_debug ("%s failed to count\n" , id); +return -1 ; + } +pr_debug ("processing metric: %s EXIT: %f\n" , id, data->val); + } + +return 0 ; +} +void expr__del_id (struct expr_parse_ctx *ctx,const char *id) {struct expr_id_data *old_val =NULL ;
@@ -173,6 +202,8 @@ __expr__parse(double *val, struct expr_parse_ctx *ctx, const char *expr,void *scanner;int ret; +pr_debug2 ("parsing metric: %s\n" , expr); + ret =expr_lex_init_extra (&scanner_ctx, &scanner);if (ret)return ret;
tools/perf/util/expr.h +3 -0
@@ -23,6 +23,7 @@ struct expr_id_data {struct {const char *metric_name;const char *metric_expr; +bool counted; } ref; };
@@ -42,6 +43,8 @@ int expr__add_id_val(struct expr_parse_ctx *ctx, const char *id, double val);int expr__add_ref (struct expr_parse_ctx *ctx,struct metric_ref *ref);int expr__get_id (struct expr_parse_ctx *ctx,const char *id,struct expr_id_data **data); +int expr__resolve_id (struct expr_parse_ctx *ctx,const char *id, +struct expr_id_data **datap);int expr__parse (double *final_val,struct expr_parse_ctx *ctx,const char *expr,int runtime); int expr__find_other(const char *expr, const char *one,
tools/perf/util/expr.y +2 -2
@@ -88,11 +88,11 @@ expr: NUMBER | ID { struct expr_id_data *data; - if (expr__get_id(ctx, $1, &data) || !data) { - pr_debug("%s not found\n", $1); + if (expr__resolve_id(ctx, $1, &data)) { free($1); YYABORT; } + $$ = data->val; free($1); }
Signature · 1 line
2.25.4
Jiri Olsa
reviewerPATCH v4 12/19 perf metric: Add events for the current list
In reply to Jiri OlsaThere's no need to iterate the whole list of groups, when adding new events. The currently created groups are the ones we want to add.
--- tools/perf/util/metricgroup.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
tools/perf/util/metricgroup.c +6 -3
@@ -813,6 +813,7 @@ static int metricgroup__add_metric(const char *metric, bool metric_no_group, {struct pmu_event *pe;struct egroup *eg; +LIST_HEAD (list);int i, ret;bool has_match = false;
@@ -820,7 +821,7 @@ static int metricgroup__add_metric(const char *metric, bool metric_no_group, has_match = true; eg =NULL ; - ret =add_metric (group_list, pe, metric_no_group, &eg); + ret =add_metric (&list, pe, metric_no_group, &eg);if (ret)return ret;
@@ -829,7 +830,7 @@ static int metricgroup__add_metric(const char *metric, bool metric_no_group,* included in the expression. */ ret =resolve_metric (eg, metric_no_group, - group_list, map); + &list, map);if (ret)return ret; }
@@ -838,7 +839,7 @@ static int metricgroup__add_metric(const char *metric, bool metric_no_group,if (!has_match)return -EINVAL ; -list_for_each_entry (eg, group_list, nd) { +list_for_each_entry (eg, &list, nd) {if (events->len >0 )strbuf_addf (events,"," );
@@ -850,6 +851,8 @@ static int metricgroup__add_metric(const char *metric, bool metric_no_group, &eg->pctx); } } + +list_splice (&list, group_list);return 0 ; }
Signature · 1 line
2.25.4
Jiri Olsa
reviewerPATCH v4 13/19 perf metric: Add cache_miss_cycles to metric parse test
In reply to Jiri OlsaAdding test that compute metric with other metrics in it.
cache_miss_cycles = metric:dcache_miss_cpi + metric:icache_miss_cycles
--- tools/perf/tests/parse-metric.c | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+)
tools/perf/tests/parse-metric.c +33 -0
@@ -11,6 +11,8 @@#include "debug.h" #include "expr.h" #include "stat.h" +#include <perf/cpumap.h> +#include <perf/evlist.h> static struct pmu_event pme_test[] = { {
@@ -22,6 +24,18 @@ static struct pmu_event pme_test[] = { "( 1 + cpu_clk_unhalted.one_thread_active / cpu_clk_unhalted.ref_xclk ) )))", .metric_name = "Frontend_Bound_SMT", }, +{ + .metric_expr = "l1d\\-loads\\-misses / inst_retired.any", + .metric_name = "dcache_miss_cpi", +}, +{ + .metric_expr = "l1i\\-loads\\-misses / inst_retired.any", + .metric_name = "icache_miss_cycles", +}, +{ + .metric_expr = "(dcache_miss_cpi + icache_miss_cycles)", + .metric_name = "cache_miss_cycles", +}, };static struct pmu_events_map map = {
@@ -162,9 +176,28 @@ static int test_frontend(void)return 0 ; } +static int test_cache_miss_cycles (void ) +{ +double ratio; +struct value vals[] = { + { .event ="l1d-loads-misses" , .val =300 }, + { .event ="l1i-loads-misses" , .val =200 }, + { .event ="inst_retired.any" , .val =400 }, + {0 }, + }; + +TEST_ASSERT_VAL ("failed to compute metric" , +compute_metric ("cache_miss_cycles" , vals, &ratio) ==0 ); + +TEST_ASSERT_VAL ("cache_miss_cycles failed, wrong ratio" , + ratio ==1.25 ); +return 0 ; +} +int test__parse_metric (struct test *test__maybe_unused ,int subtest__maybe_unused ) {TEST_ASSERT_VAL ("IPC failed" ,test_ipc () ==0 );TEST_ASSERT_VAL ("frontend failed" ,test_frontend () ==0 ); +TEST_ASSERT_VAL ("cache_miss_cycles failed" ,test_cache_miss_cycles () ==0 );return 0 ; }
Signature · 1 line
2.25.4
Jiri Olsa
reviewerPATCH v4 14/19 perf metric: Add DCache_L2 to metric parse test
In reply to Jiri OlsaAdding test that compute DCache_L2 metrics with other related metrics in it.
--- tools/perf/tests/parse-metric.c | 71 +++++++++++++++++++++++++++++++++ 1 file changed, 71 insertions(+)
tools/perf/tests/parse-metric.c +71 -0
@@ -36,6 +36,27 @@ static struct pmu_event pme_test[] = { .metric_expr = "(dcache_miss_cpi + icache_miss_cycles)", .metric_name = "cache_miss_cycles", }, +{ + .metric_expr = "l2_rqsts.demand_data_rd_hit + l2_rqsts.pf_hit + l2_rqsts.rfo_hit", + .metric_name = "DCache_L2_All_Hits", +}, +{ + .metric_expr = "max(l2_rqsts.all_demand_data_rd - l2_rqsts.demand_data_rd_hit, 0) + " + "l2_rqsts.pf_miss + l2_rqsts.rfo_miss", + .metric_name = "DCache_L2_All_Miss", +}, +{ + .metric_expr = "dcache_l2_all_hits + dcache_l2_all_miss", + .metric_name = "DCache_L2_All", +}, +{ + .metric_expr = "d_ratio(dcache_l2_all_hits, dcache_l2_all)", + .metric_name = "DCache_L2_Hits", +}, +{ + .metric_expr = "d_ratio(dcache_l2_all_miss, dcache_l2_all)", + .metric_name = "DCache_L2_Misses", +}, }; static struct pmu_events_map map = {
@@ -194,10 +215,60 @@ static int test_cache_miss_cycles(void)return 0 ; } + +/* +* DCache_L2_All_Hits = l2_rqsts.demand_data_rd_hit + l2_rqsts.pf_hit + l2_rqsts.rfo_hi +* DCache_L2_All_Miss = max(l2_rqsts.all_demand_data_rd - l2_rqsts.demand_data_rd_hit, 0) + +* l2_rqsts.pf_miss + l2_rqsts.rfo_miss +* DCache_L2_All = dcache_l2_all_hits + dcache_l2_all_miss +* DCache_L2_Hits = d_ratio(dcache_l2_all_hits, dcache_l2_all) +* DCache_L2_Misses = d_ratio(dcache_l2_all_miss, dcache_l2_all) +* +* l2_rqsts.demand_data_rd_hit = 100 +* l2_rqsts.pf_hit = 200 +* l2_rqsts.rfo_hi = 300 +* l2_rqsts.all_demand_data_rd = 400 +* l2_rqsts.pf_miss = 500 +* l2_rqsts.rfo_miss = 600 +* +* DCache_L2_All_Hits = 600 +* DCache_L2_All_Miss = MAX(400 - 100, 0) + 500 + 600 = 1400 +* DCache_L2_All = 600 + 1400 = 2000 +* DCache_L2_Hits = 600 / 2000 = 0.3 +* DCache_L2_Misses = 1400 / 2000 = 0.7 +*/ +static int test_dcache_l2 (void ) +{ +double ratio; +struct value vals[] = { + { .event ="l2_rqsts.demand_data_rd_hit" , .val =100 }, + { .event ="l2_rqsts.pf_hit" , .val =200 }, + { .event ="l2_rqsts.rfo_hit" , .val =300 }, + { .event ="l2_rqsts.all_demand_data_rd" , .val =400 }, + { .event ="l2_rqsts.pf_miss" , .val =500 }, + { .event ="l2_rqsts.rfo_miss" , .val =600 }, + {0 }, + }; + +TEST_ASSERT_VAL ("failed to compute metric" , +compute_metric ("DCache_L2_Hits" , vals, &ratio) ==0 ); + +TEST_ASSERT_VAL ("DCache_L2_Hits failed, wrong ratio" , + ratio ==0.3 ); + +TEST_ASSERT_VAL ("failed to compute metric" , +compute_metric ("DCache_L2_Misses" , vals, &ratio) ==0 ); + +TEST_ASSERT_VAL ("DCache_L2_Misses failed, wrong ratio" , + ratio ==0.7 ); +return 0 ; +} +int test__parse_metric (struct test *test__maybe_unused ,int subtest__maybe_unused ) {TEST_ASSERT_VAL ("IPC failed" ,test_ipc () ==0 );TEST_ASSERT_VAL ("frontend failed" ,test_frontend () ==0 );TEST_ASSERT_VAL ("cache_miss_cycles failed" ,test_cache_miss_cycles () ==0 ); +TEST_ASSERT_VAL ("DCache_L2 failed" ,test_dcache_l2 () ==0 );return 0 ; }
Signature · 1 line
2.25.4
Jiri Olsa
reviewerPATCH v4 16/19 perf metric: Make compute_single function more precise
In reply to Jiri OlsaSo far compute_single function relies on the fact, that there's only single metric defined within evlist in all tests. In following patch we will add test for metric group, so we need to be able to compute metric by given name.
Adding the name argument to compute_single and iterating evlist and evsel's expression to find the given metric.
--- tools/perf/tests/parse-metric.c | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-)
tools/perf/tests/parse-metric.c +13 -9
@@ -108,17 +108,21 @@ static void load_runtime_stat(struct runtime_stat *st, struct evlist *evlist, }static double compute_single (struct rblist *metric_events,struct evlist *evlist, -struct runtime_stat *st) +struct runtime_stat *st,const char *name) { -struct evsel *evsel =evlist__first (evlist); +struct metric_expr *mexp;struct metric_event *me; +struct evsel *evsel; - me =metricgroup__lookup (metric_events, evsel, false); -if (me !=NULL ) { -struct metric_expr *mexp; - - mexp = list_first_entry(&me->head, struct metric_expr, nd); -return test_generic_metric (mexp,0 , st); +evlist__for_each_entry (evlist, evsel) { + me =metricgroup__lookup (metric_events, evsel, false); +if (me !=NULL ) { + list_for_each_entry (mexp, &me->head, nd) { +if (strcmp (mexp->metric_name, name)) +continue ; +return test_generic_metric (mexp,0 , st); + } + } }return 0. ; }
@@ -162,7 +166,7 @@ static int compute_metric(const char *name, struct value *vals, double *ratio)load_runtime_stat (&st, evlist, vals);/* And execute the metric */ - *ratio =compute_single (&metric_events, evlist, &st); + *ratio =compute_single (&metric_events, evlist, &st, name);/* ... clenup. */ metricgroup__rblist_exit (&metric_events);
Signature · 1 line
2.25.4
Jiri Olsa
reviewerPATCH v4 15/19 perf metric: Add recursion check when processing nested metrics
In reply to Jiri OlsaKeeping the stack of nested metrics via 'struct expr_id' objects and checking if we are in recursion via already processed metric.
The stack is implemented as static array within the struct egroup with 100 entries, which should be enough nesting depth for any metric we have or plan to have at the moment.
Adding test that simulates the recursion and checks we can detect it.
Committer notes:
Bumped RECURSION_ID_MAX to 1000 as per Jiri's reply to Paul Clark on the patch series e-mail discussion.
--- tools/perf/tests/parse-metric.c | 34 +++++++++- tools/perf/util/expr.c | 2 + tools/perf/util/expr.h | 9 ++- tools/perf/util/metricgroup.c | 117 +++++++++++++++++++++++++++++--- 4 files changed, 149 insertions(+), 13 deletions(-)
tools/perf/tests/parse-metric.c +32 -2
@@ -57,6 +57,18 @@ static struct pmu_event pme_test[] = { .metric_expr = "d_ratio(dcache_l2_all_miss, dcache_l2_all)", .metric_name = "DCache_L2_Misses", }, +{ + .metric_expr = "ipc + m2", + .metric_name = "M1", +}, +{ + .metric_expr = "ipc + m1", + .metric_name = "M2", +}, +{ + .metric_expr = "1/m3", + .metric_name = "M3", +} }; static struct pmu_events_map map = {
@@ -139,8 +151,8 @@ static int compute_metric(const char *name, struct value *vals, double *ratio) err =metricgroup__parse_groups_test (evlist, &map, name, false, false, &metric_events); - -TEST_ASSERT_VAL ("failed to parse metric" , err ==0 ); +if (err) +return err;if (perf_evlist__alloc_stats (evlist, false))return -1 ;
@@ -264,11 +276,29 @@ static int test_dcache_l2(void)return 0 ; } +static int test_recursion_fail (void ) +{ +double ratio; +struct value vals[] = { + { .event ="inst_retired.any" , .val =300 }, + { .event ="cpu_clk_unhalted.thread" , .val =200 }, + {0 }, + }; + +TEST_ASSERT_VAL ("failed to find recursion" , +compute_metric ("M1" , vals, &ratio) ==-1 ); + +TEST_ASSERT_VAL ("failed to find recursion" , +compute_metric ("M3" , vals, &ratio) ==-1 ); +return 0 ; +} +int test__parse_metric (struct test *test__maybe_unused ,int subtest__maybe_unused ) {TEST_ASSERT_VAL ("IPC failed" ,test_ipc () ==0 );TEST_ASSERT_VAL ("frontend failed" ,test_frontend () ==0 );TEST_ASSERT_VAL ("cache_miss_cycles failed" ,test_cache_miss_cycles () ==0 );TEST_ASSERT_VAL ("DCache_L2 failed" ,test_dcache_l2 () ==0 ); +TEST_ASSERT_VAL ("recursion fail failed" ,test_recursion_fail () ==0 );return 0 ; }
tools/perf/util/expr.c +2 -0
@@ -47,6 +47,8 @@ int expr__add_id(struct expr_parse_ctx *ctx, const char *id)if (!data_ptr)return -ENOMEM ; + data_ptr->parent = ctx->parent; + ret =hashmap__set (&ctx->ids, id, data_ptr, (const void **)&old_key, (void **)&old_data);if (ret)
tools/perf/util/expr.h +8 -1
@@ -13,8 +13,14 @@struct metric_ref ; +struct expr_id { +char *id; +struct expr_id *parent; +}; +struct expr_parse_ctx { -struct hashmap ids; +struct hashmap ids; +struct expr_id *parent; };struct expr_id_data {
@@ -25,6 +31,7 @@ struct expr_id_data {const char *metric_expr;bool counted; } ref; +struct expr_id *parent; };bool is_ref;
tools/perf/util/metricgroup.c +107 -10
225 lines not on this page · raw message
Signature · 1 line
2.25.4
Jiri Olsa
reviewerPATCH v4 17/19 perf metric: Add metric group test
In reply to Jiri OlsaAdding test for metric group plus compute_metric_group function to get metrics values within the group.
--- tools/perf/tests/parse-metric.c | 48 +++++++++++++++++++++++++++++++-- 1 file changed, 46 insertions(+), 2 deletions(-)
tools/perf/tests/parse-metric.c +46 -2
@@ -18,6 +18,7 @@ static struct pmu_event pme_test[] = { { .metric_expr = "inst_retired.any / cpu_clk_unhalted.thread", .metric_name = "IPC", + .metric_group = "group1", }, { .metric_expr = "idq_uops_not_delivered.core / (4 * (( ( cpu_clk_unhalted.thread / 2 ) * "
@@ -35,6 +36,7 @@ static struct pmu_event pme_test[] = { { .metric_expr = "(dcache_miss_cpi + icache_miss_cycles)", .metric_name = "cache_miss_cycles", + .metric_group = "group1", }, { .metric_expr = "l2_rqsts.demand_data_rd_hit + l2_rqsts.pf_hit + l2_rqsts.rfo_hit",
@@ -127,7 +129,9 @@ static double compute_single(struct rblist *metric_events, struct evlist *evlistreturn 0. ; } -static int compute_metric (const char *name,struct value *vals,double *ratio) +static int __compute_metric (const char *name,struct value *vals, +const char *name1,double *ratio1, +const char *name2,double *ratio2) {struct rblist metric_events = { .nr_entries =0 ,
@@ -166,7 +170,10 @@ static int compute_metric(const char *name, struct value *vals, double *ratio)load_runtime_stat (&st, evlist, vals);/* And execute the metric */ - *ratio =compute_single (&metric_events, evlist, &st, name); +if (name1 && ratio1) + *ratio1 =compute_single (&metric_events, evlist, &st, name1); +if (name2 && ratio2) + *ratio2 =compute_single (&metric_events, evlist, &st, name2);/* ... clenup. */ metricgroup__rblist_exit (&metric_events);
@@ -177,6 +184,18 @@ static int compute_metric(const char *name, struct value *vals, double *ratio)return 0 ; } +static int compute_metric (const char *name,struct value *vals,double *ratio) +{ +return __compute_metric (name, vals, name, ratio,NULL ,NULL ); +} + +static int compute_metric_group (const char *name,struct value *vals, +const char *name1,double *ratio1, +const char *name2,double *ratio2) +{ +return __compute_metric (name, vals, name1, ratio1, name2, ratio2); +} +static int test_ipc (void ) {double ratio;
@@ -297,6 +316,30 @@ static int test_recursion_fail(void)return 0 ; } +static int test_metric_group (void ) +{ +double ratio1, ratio2; +struct value vals[] = { + { .event ="cpu_clk_unhalted.thread" , .val =200 }, + { .event ="l1d-loads-misses" , .val =300 }, + { .event ="l1i-loads-misses" , .val =200 }, + { .event ="inst_retired.any" , .val =400 }, + {0 }, + }; + +TEST_ASSERT_VAL ("failed to find recursion" , +compute_metric_group ("group1" , vals, +"IPC" , &ratio1, +"cache_miss_cycles" , &ratio2) ==0 ); + +TEST_ASSERT_VAL ("group IPC failed, wrong ratio" , + ratio1 ==2.0 ); + +TEST_ASSERT_VAL ("group cache_miss_cycles failed, wrong ratio" , + ratio2 ==1.25 ); +return 0 ; +} +int test__parse_metric (struct test *test__maybe_unused ,int subtest__maybe_unused ) {TEST_ASSERT_VAL ("IPC failed" ,test_ipc () ==0 );
@@ -304,5 +347,6 @@ int test__parse_metric(struct test *test __maybe_unused, int subtest __maybe_unuTEST_ASSERT_VAL ("cache_miss_cycles failed" ,test_cache_miss_cycles () ==0 );TEST_ASSERT_VAL ("DCache_L2 failed" ,test_dcache_l2 () ==0 );TEST_ASSERT_VAL ("recursion fail failed" ,test_recursion_fail () ==0 ); +TEST_ASSERT_VAL ("test metric group" ,test_metric_group () ==0 );return 0 ; }
Signature · 1 line
2.25.4
Jiri Olsa
reviewerPATCH v4 18/19 perf metric: Rename struct egroup to metric
In reply to Jiri OlsaRenaming struct egroup to metric, because it seems to make more sense. Plus renaming all the variables that hold egroup to appropriate names.
--- tools/perf/util/metricgroup.c | 142 +++++++++++++++++----------------- 1 file changed, 71 insertions(+), 71 deletions(-)
tools/perf/util/metricgroup.c +71 -71
@@ -115,7 +115,7 @@ struct metric_ref_node {struct list_head list; }; -struct egroup { +struct metric {struct list_head nd;struct expr_parse_ctx pctx;const char *metric_name;
@@ -262,7 +262,7 @@ static int metricgroup__setup_events(struct list_head *groups,struct metric_expr *expr;int i =0 ;int ret =0 ; -struct egroup *eg; +struct metric *m;struct evsel *evsel, *tmp;unsigned long *evlist_used;
@@ -270,23 +270,23 @@ static int metricgroup__setup_events(struct list_head *groups,if (!evlist_used)return -ENOMEM ; - list_for_each_entry (eg, groups, nd) { + list_for_each_entry (m, groups, nd) {struct evsel **metric_events;struct metric_ref *metric_refs =NULL ; metric_events =calloc (sizeof (void *), -hashmap__size (&eg->pctx.ids) +1 ); +hashmap__size (&m->pctx.ids) +1 );if (!metric_events) { ret = -ENOMEM ;break ; } - evsel =find_evsel_group (perf_evlist, &eg->pctx, + evsel =find_evsel_group (perf_evlist, &m->pctx, metric_no_merge, - eg->has_constraint, metric_events, + m->has_constraint, metric_events, evlist_used);if (!evsel) {pr_debug ("Cannot resolve %s: %s\n" , - eg->metric_name, eg->metric_expr); + m->metric_name, m->metric_expr);free (metric_events);continue ; }
@@ -309,10 +309,10 @@ static int metricgroup__setup_events(struct list_head *groups,* Collect and store collected nested expressions * for metric processing. */ -if (eg->metric_refs_cnt) { +if (m->metric_refs_cnt) {struct metric_ref_node *ref; - metric_refs =zalloc (sizeof (struct metric_ref ) * (eg->metric_refs_cnt +1 )); + metric_refs =zalloc (sizeof (struct metric_ref ) * (m->metric_refs_cnt +1 ));if (!metric_refs) { ret = -ENOMEM ;free (metric_events);
@@ -320,7 +320,7 @@ static int metricgroup__setup_events(struct list_head *groups, } i =0 ; -list_for_each_entry (ref, &eg->metric_refs, list) { +list_for_each_entry (ref, &m->metric_refs, list) {/* * Intentionally passing just const char pointers, * originally from 'struct pmu_event' object.
@@ -334,11 +334,11 @@ static int metricgroup__setup_events(struct list_head *groups, }; expr->metric_refs = metric_refs; - expr->metric_expr = eg->metric_expr; - expr->metric_name = eg->metric_name; - expr->metric_unit = eg->metric_unit; + expr->metric_expr = m->metric_expr; + expr->metric_name = m->metric_name; + expr->metric_unit = m->metric_unit; expr->metric_events = metric_events; - expr->runtime = eg->runtime; + expr->runtime = m->runtime;list_add (&expr->nd, &me->head); }
@@ -643,41 +643,41 @@ static int __add_metric(struct list_head *group_list,struct pmu_event *pe, bool metric_no_group, int runtime, - struct egroup **egp, + struct metric **mp, struct expr_id *parent, struct expr_ids *ids) { struct metric_ref_node *ref; -struct egroup *eg; +struct metric *m; -if (*egp ==NULL ) { +if (*mp ==NULL ) {/* * We got in here for the parent group, * allocate it and put it on the list. */ - eg =zalloc (sizeof (*eg)); -if (!eg) + m =zalloc (sizeof (*m)); +if (!m)return -ENOMEM ; -expr__ctx_init (&eg->pctx); - eg->metric_name = pe->metric_name; - eg->metric_expr = pe->metric_expr; - eg->metric_unit = pe->unit; - eg->runtime = runtime; - eg->has_constraint = metric_no_group ||metricgroup__has_constraint (pe); -INIT_LIST_HEAD (&eg->metric_refs); - eg->metric_refs_cnt =0 ; - *egp = eg; +expr__ctx_init (&m->pctx); + m->metric_name = pe->metric_name; + m->metric_expr = pe->metric_expr; + m->metric_unit = pe->unit; + m->runtime = runtime; + m->has_constraint = metric_no_group ||metricgroup__has_constraint (pe); +INIT_LIST_HEAD (&m->metric_refs); + m->metric_refs_cnt =0 ; + *mp = m; parent =expr_ids__alloc (ids);if (!parent) { -free (eg); +free (m);return -EINVAL ; } parent->id =strdup (pe->metric_name);if (!parent->id) { -free (eg); +free (m);return -ENOMEM ; } }else {
@@ -686,7 +686,7 @@ static int __add_metric(struct list_head *group_list,* recursive metricgroup__add_metric call, add * it to the parent group. */ - eg = *egp; + m = *mp; ref =malloc (sizeof (*ref));if (!ref)
@@ -701,21 +701,21 @@ static int __add_metric(struct list_head *group_list, ref->metric_name = pe->metric_name; ref->metric_expr = pe->metric_expr; -list_add (&ref->list, &eg->metric_refs); - eg->metric_refs_cnt++; +list_add (&ref->list, &m->metric_refs); + m->metric_refs_cnt++; }/* Force all found IDs in metric to have us as parent ID. */ WARN_ON_ONCE (!parent); - eg->pctx.parent = parent; + m->pctx.parent = parent;/* * For both the parent and referenced metrics, we parse * all the metric's IDs and add it to the parent context. */ -if (expr__find_other (pe->metric_expr,NULL , &eg->pctx, runtime) <0 ) { -expr__ctx_clear (&eg->pctx); -free (eg); +if (expr__find_other (pe->metric_expr,NULL , &m->pctx, runtime) <0 ) { +expr__ctx_clear (&m->pctx); +free (m);return -EINVAL ; }
@@ -723,23 +723,23 @@ static int __add_metric(struct list_head *group_list,* We add new group only in the 'parent' call, * so bail out for referenced metric case. */ -if (eg->metric_refs_cnt) +if (m->metric_refs_cnt)return 0 ;if (list_empty (group_list)) -list_add (&eg->nd, group_list); +list_add (&m->nd, group_list);else {struct list_head *pos;/* Place the largest groups at the front. */ list_for_each_prev (pos, group_list) { - struct egroup *old = list_entry(pos, struct egroup, nd); + struct metric *old = list_entry(pos, struct metric, nd); -if (hashmap__size (&eg->pctx.ids) <= +if (hashmap__size (&m->pctx.ids) <=hashmap__size (&old->pctx.ids))break ; } -list_add (&eg->nd, pos); +list_add (&m->nd, pos); }return 0 ;
@@ -769,7 +769,7 @@ static struct pmu_event *find_metric(const char *metric, struct pmu_events_map *return NULL ; } -static int recursion_check (struct egroup *eg,const char *id,struct expr_id **parent, +static int recursion_check (struct metric *m,const char *id,struct expr_id **parent,struct expr_ids *ids) {struct expr_id_data *data;
@@ -782,7 +782,7 @@ static int recursion_check(struct egroup *eg, const char *id, struct expr_id **p* if we already processed 'id', if we did, it's recursion * and we fail. */ - ret =expr__get_id (&eg->pctx, id, &data); + ret =expr__get_id (&m->pctx, id, &data);if (ret)return ret;
@@ -816,11 +816,11 @@ static int recursion_check(struct egroup *eg, const char *id, struct expr_id **pstatic int add_metric (struct list_head *group_list,struct pmu_event *pe,bool metric_no_group, -struct egroup **egp, +struct metric **mp,struct expr_id *parent,struct expr_ids *ids); -static int resolve_metric(struct egroup *eg, +static int resolve_metric(struct metric *m, bool metric_no_group, struct list_head *group_list, struct pmu_events_map *map,
@@ -837,7 +837,7 @@ static int resolve_metric(struct egroup *eg,*/ do { all = true; - hashmap__for_each_entry((&eg->pctx.ids), cur, bkt) { + hashmap__for_each_entry((&m->pctx.ids), cur, bkt) { struct expr_id *parent; struct pmu_event *pe;
@@ -845,16 +845,16 @@ static int resolve_metric(struct egroup *eg,if (!pe)continue ; - ret =recursion_check (eg, cur->key, &parent, ids); + ret =recursion_check (m, cur->key, &parent, ids);if (ret)return ret; all = false;/* The metric key itself needs to go out.. */ -expr__del_id (&eg->pctx, cur->key); +expr__del_id (&m->pctx, cur->key);/* ... and it gets resolved to the parent context. */ - ret =add_metric (group_list, pe, metric_no_group, &eg, parent, ids); + ret =add_metric (group_list, pe, metric_no_group, &m, parent, ids);if (ret)return ret;
@@ -872,7 +872,7 @@ static int resolve_metric(struct egroup *eg,static int add_metric (struct list_head *group_list,struct pmu_event *pe,bool metric_no_group, -struct egroup **egp, +struct metric **m,struct expr_id *parent,struct expr_ids *ids) {
@@ -881,7 +881,7 @@ static int add_metric(struct list_head *group_list,pr_debug ("metric expr %s for %s\n" , pe->metric_expr, pe->metric_name);if (!strstr (pe->metric_expr,"?" )) { - ret =__add_metric (group_list, pe, metric_no_group,1 , egp, parent, ids); + ret =__add_metric (group_list, pe, metric_no_group,1 , m, parent, ids); }else {int j, count;
@@ -893,7 +893,7 @@ static int add_metric(struct list_head *group_list,*/ for (j =0 ; j < count && !ret; j++) { - ret =__add_metric (group_list, pe, metric_no_group, j, egp, parent, ids); + ret =__add_metric (group_list, pe, metric_no_group, j, m, parent, ids); } }
@@ -908,16 +908,16 @@ static int metricgroup__add_metric(const char *metric, bool metric_no_group, {struct expr_ids ids = {0 };struct pmu_event *pe; -struct egroup *eg; +struct metric *m;LIST_HEAD (list);int i, ret;bool has_match = false;map_for_each_metric (pe, i, map, metric) { has_match = true; - eg =NULL ; + m =NULL ; - ret =add_metric (&list, pe, metric_no_group, &eg,NULL , &ids); + ret =add_metric (&list, pe, metric_no_group, &m,NULL , &ids);if (ret)return ret;
@@ -925,7 +925,7 @@ static int metricgroup__add_metric(const char *metric, bool metric_no_group,* Process any possible referenced metrics * included in the expression. */ - ret =resolve_metric (eg, metric_no_group, + ret =resolve_metric (m, metric_no_group, &list, map, &ids);if (ret)return ret;
@@ -935,16 +935,16 @@ static int metricgroup__add_metric(const char *metric, bool metric_no_group,if (!has_match)return -EINVAL ; -list_for_each_entry (eg, &list, nd) { +list_for_each_entry (m, &list, nd) {if (events->len >0 )strbuf_addf (events,"," ); -if (eg->has_constraint) { +if (m->has_constraint) {metricgroup__add_metric_non_group (events, - &eg->pctx); + &m->pctx); }else {metricgroup__add_metric_weak_group (events, - &eg->pctx); + &m->pctx); } }
@@ -986,25 +986,25 @@ static int metricgroup__add_metric_list(const char *list, bool metric_no_group,return ret; } -static void egroup__free_refs (struct egroup *egroup) +static void metric__free_refs (struct metric *metric) {struct metric_ref_node *ref, *tmp; -list_for_each_entry_safe (ref, tmp, &egroup->metric_refs, list) { +list_for_each_entry_safe (ref, tmp, &metric->metric_refs, list) {list_del (&ref->list);free (ref); } } -static void metricgroup__free_egroups (struct list_head *group_list) +static void metricgroup__free_metrics (struct list_head *group_list) { -struct egroup *eg, *egtmp; +struct metric *m, *tmp; - list_for_each_entry_safe (eg, egtmp, group_list, nd) { -egroup__free_refs (eg); -expr__ctx_clear (&eg->pctx); -list_del_init (&eg->nd); -free (eg); + list_for_each_entry_safe (m, tmp, group_list, nd) { +metric__free_refs (m); +expr__ctx_clear (&m->pctx); +list_del_init (&m->nd); +free (m); } }
@@ -1037,7 +1037,7 @@ static int parse_groups(struct evlist *perf_evlist, const char *str, ret =metricgroup__setup_events (&group_list, metric_no_merge, perf_evlist, metric_events); out: -metricgroup__free_egroups (&group_list); +metricgroup__free_metrics (&group_list);return ret; }
Signature · 1 line
2.25.4
Jiri Olsa
reviewerPATCH v4 19/19 perf metric: Rename group_list to metric_list
In reply to Jiri OlsaFollowing the previous change that rename egroup to metric, there's no reason to call the list 'group_list' anymore, renaming it to metric_list.
--- tools/perf/util/metricgroup.c | 42 +++++++++++++++++------------------ 1 file changed, 21 insertions(+), 21 deletions(-)
tools/perf/util/metricgroup.c +21 -21
@@ -639,7 +639,7 @@ int __weak arch_get_runtimeparam(void)return 1 ; } -static int __add_metric(struct list_head *group_list, +static int __add_metric(struct list_head *metric_list, struct pmu_event *pe, bool metric_no_group, int runtime,
@@ -726,13 +726,13 @@ static int __add_metric(struct list_head *group_list,if (m->metric_refs_cnt)return 0 ; -if (list_empty (group_list)) -list_add (&m->nd, group_list); +if (list_empty (metric_list)) +list_add (&m->nd, metric_list);else {struct list_head *pos;/* Place the largest groups at the front. */ -list_for_each_prev (pos, group_list) { +list_for_each_prev (pos, metric_list) { struct metric *old = list_entry(pos, struct metric, nd); if (hashmap__size(&m->pctx.ids) <=
@@ -813,7 +813,7 @@ static int recursion_check(struct metric *m, const char *id, struct expr_id **pareturn p->id ?0 : -ENOMEM ; } -static int add_metric(struct list_head *group_list, +static int add_metric(struct list_head *metric_list, struct pmu_event *pe, bool metric_no_group, struct metric **mp,
@@ -822,7 +822,7 @@ static int add_metric(struct list_head *group_list,static int resolve_metric (struct metric *m,bool metric_no_group, -struct list_head *group_list, +struct list_head *metric_list,struct pmu_events_map *map,struct expr_ids *ids) {
@@ -854,7 +854,7 @@ static int resolve_metric(struct metric *m,expr__del_id (&m->pctx, cur->key);/* ... and it gets resolved to the parent context. */ - ret =add_metric (group_list, pe, metric_no_group, &m, parent, ids); + ret =add_metric (metric_list, pe, metric_no_group, &m, parent, ids);if (ret)return ret;
@@ -869,7 +869,7 @@ static int resolve_metric(struct metric *m,return 0 ; } -static int add_metric(struct list_head *group_list, +static int add_metric(struct list_head *metric_list, struct pmu_event *pe, bool metric_no_group, struct metric **m,
@@ -881,7 +881,7 @@ static int add_metric(struct list_head *group_list,pr_debug ("metric expr %s for %s\n" , pe->metric_expr, pe->metric_name);if (!strstr (pe->metric_expr,"?" )) { - ret =__add_metric (group_list, pe, metric_no_group,1 , m, parent, ids); + ret =__add_metric (metric_list, pe, metric_no_group,1 , m, parent, ids); }else {int j, count;
@@ -889,11 +889,11 @@ static int add_metric(struct list_head *group_list,/* This loop is added to create multiple * events depend on count value and add -* those events to group_list. +* those events to metric_list. */ for (j =0 ; j < count && !ret; j++) { - ret =__add_metric (group_list, pe, metric_no_group, j, m, parent, ids); + ret =__add_metric (metric_list, pe, metric_no_group, j, m, parent, ids); } }
@@ -902,7 +902,7 @@ static int add_metric(struct list_head *group_list,static int metricgroup__add_metric (const char *metric,bool metric_no_group,struct strbuf *events, -struct list_head *group_list, +struct list_head *metric_list,struct pmu_events_map *map) {
@@ -948,14 +948,14 @@ static int metricgroup__add_metric(const char *metric, bool metric_no_group, } } -list_splice (&list, group_list); +list_splice (&list, metric_list);expr_ids__exit (&ids);return 0 ; }static int metricgroup__add_metric_list (const char *list,bool metric_no_group,struct strbuf *events, -struct list_head *group_list, +struct list_head *metric_list,struct pmu_events_map *map) {char *llist, *nlist, *p;
@@ -971,7 +971,7 @@ static int metricgroup__add_metric_list(const char *list, bool metric_no_group,while ((p =strsep (&llist,"," )) !=NULL ) { ret =metricgroup__add_metric (p, metric_no_group, events, - group_list, map); + metric_list, map);if (ret == -EINVAL ) {fprintf (stderr,"Cannot find metric or group `%s'\n" , p);
@@ -996,11 +996,11 @@ static void metric__free_refs(struct metric *metric) } } -static void metricgroup__free_metrics (struct list_head *group_list) +static void metricgroup__free_metrics (struct list_head *metric_list) {struct metric *m, *tmp; - list_for_each_entry_safe (m, tmp, group_list, nd) { + list_for_each_entry_safe (m, tmp, metric_list, nd) {metric__free_refs (m);expr__ctx_clear (&m->pctx);list_del_init (&m->nd);
@@ -1017,13 +1017,13 @@ static int parse_groups(struct evlist *perf_evlist, const char *str, {struct parse_events_error parse_error;struct strbuf extra_events; -LIST_HEAD (group_list); +LIST_HEAD (metric_list);int ret;if (metric_events->nr_entries ==0 )metricgroup__rblist_init (metric_events); ret =metricgroup__add_metric_list (str, metric_no_group, - &extra_events, &group_list, map); + &extra_events, &metric_list, map);if (ret)return ret;pr_debug ("adding %s\n" , extra_events.buf);
@@ -1034,10 +1034,10 @@ static int parse_groups(struct evlist *perf_evlist, const char *str, goto out; }strbuf_release (&extra_events); - ret =metricgroup__setup_events (&group_list, metric_no_merge, + ret =metricgroup__setup_events (&metric_list, metric_no_merge, perf_evlist, metric_events); out: -metricgroup__free_metrics (&group_list); +metricgroup__free_metrics (&metric_list);return ret; }
Signature · 1 line
2.25.4
Quoted 32 lines · Jiri Olsa
this patchset is adding the support to reused metric in another metric.
For example, to define IPC by using CPI with change like:
{
"BriefDescription": "Instructions Per Cycle (per Logical Processor)",(quoted diff) +1
-1- "MetricExpr": "INST_RETIRED.ANY / CPU_CLK_UNHALTED.THREAD", + "MetricExpr": "1/CPI", "MetricGroup": "TopDownL1", "MetricName": "IPC" },I won't be able to find all the possible places we could use this at, so I wonder you guys (who was asking for this) would try it and come up with comments if there's something missing or we could already use it at some places.
It's based on Arnaldo's tmp.perf/core.
v4 changes:
- removed acks from patch because it changed a bit
with the last fixes:
perf metric: Collect referenced metrics in structmetric_ref_node
- fixed runtime metrics [Kajol Jain]
- increased recursion depth [Paul A. Clarke]
- changed patches due to dependencies:
perf metric: Collect referenced metrics in structmetric_ref_nodeperf metric: Add recursion check when processing nested metrics perf metric: Rename struct egroup to metric
perf metric: Renamegroup_listtometric_list
Quoted 3 lines
Also available in here:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/metric
I built and ran from the above git branch, and things seem to work. Indeed, I was able to apply my changes to exploit the new capabilities via modifications to tools/perf/pmu-events/arch/powerpc/power9/metrics.json, as I posted earlier (and will submit once this set gets merged).
One thing I noted, but which also occurs without these patches, is that the perf metrics are not computed unless run as root:
--
$ perf stat --metrics br_misprediction_percent command
Performance counter stats for 'command':
1,823,530,051 pm_br_pred:u
2,662,705 pm_br_mpred_cmpl:u $ /usr/bin/sudo perf stat --metrics br_misprediction_percent command
Performance counter stats for 'command':
1,824,655,269 pm_br_pred # 0.09 br_misprediction_percent
1,654,466 pm_br_mpred_cmpl--
Is that expected? I don't think it's always been that way.
PC
Ian Rogers
reviewerQuoted 57 lines · Paul A. Clarke
Quoted 32 lines · Jiri Olsa
this patchset is adding the support to reused metric in another metric.
For example, to define IPC by using CPI with change like:
{
"BriefDescription": "Instructions Per Cycle (per Logical Processor)",(quoted diff) +1
-1- "MetricExpr": "INST_RETIRED.ANY / CPU_CLK_UNHALTED.THREAD", + "MetricExpr": "1/CPI", "MetricGroup": "TopDownL1", "MetricName": "IPC" },I won't be able to find all the possible places we could use this at, so I wonder you guys (who was asking for this) would try it and come up with comments if there's something missing or we could already use it at some places.
It's based on Arnaldo's tmp.perf/core.
v4 changes:
- removed acks from patch because it changed a bit
with the last fixes:
perf metric: Collect referenced metrics in structmetric_ref_node
- fixed runtime metrics [Kajol Jain]
- increased recursion depth [Paul A. Clarke]
- changed patches due to dependencies:
perf metric: Collect referenced metrics in structmetric_ref_nodeperf metric: Add recursion check when processing nested metrics perf metric: Rename struct egroup to metric
perf metric: Renamegroup_listtometric_listQuoted 3 lines
Also available in here:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/metricI built and ran from the above git branch, and things seem to work. Indeed, I was able to apply my changes to exploit the new capabilities via modifications to
tools/perf/pmu-events/arch/powerpc/power9/metrics.json, as I posted earlier (and will submit once this set gets merged).Tested-by Paul A. Clarke <[email protected]>One thing I noted, but which also occurs without these patches, is that the perf metrics are not computed unless run as root:
--$ perf stat --metrics br_misprediction_percent commandPerformance counter stats for 'command':
1,823,530,051 pm_br_pred:u 2,662,705 pm_br_mpred_cmpl:u$ /usr/bin/sudo perf stat --metrics br_misprediction_percent command
Quoted 7 lines
Performance counter stats for 'command':
1,824,655,269 pm_br_pred # 0.09 br_misprediction_percent 1,654,466 pm_br_mpred_cmpl--
Is that expected? I don't think it's always been that way.
I agree Paul, this seems broken. I've noticed a bunch of issues with printing CSV, per-socket output and so on. Jiri may have a better idea but I plan to look at problems in this area later, and hopefully stick a few tests on it :-)
Thanks,
Ian
Quoted 1 line
PC
Quoted 51 lines · Ian Rogers
Quoted 50 lines · Paul A. Clarke
Quoted 32 lines · Jiri Olsa
this patchset is adding the support to reused metric in another metric.
For example, to define IPC by using CPI with change like:
{
"BriefDescription": "Instructions Per Cycle (per Logical Processor)",(quoted diff) +1
-1- "MetricExpr": "INST_RETIRED.ANY / CPU_CLK_UNHALTED.THREAD", + "MetricExpr": "1/CPI", "MetricGroup": "TopDownL1", "MetricName": "IPC" },I won't be able to find all the possible places we could use this at, so I wonder you guys (who was asking for this) would try it and come up with comments if there's something missing or we could already use it at some places.
It's based on Arnaldo's tmp.perf/core.
v4 changes:
- removed acks from patch because it changed a bit
with the last fixes:
perf metric: Collect referenced metrics in structmetric_ref_node
- fixed runtime metrics [Kajol Jain]
- increased recursion depth [Paul A. Clarke]
- changed patches due to dependencies:
perf metric: Collect referenced metrics in structmetric_ref_nodeperf metric: Add recursion check when processing nested metrics perf metric: Rename struct egroup to metric
perf metric: Renamegroup_listtometric_listQuoted 3 lines
Also available in here:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/metricI built and ran from the above git branch, and things seem to work. Indeed, I was able to apply my changes to exploit the new capabilities via modifications to
tools/perf/pmu-events/arch/powerpc/power9/metrics.json, as I posted earlier (and will submit once this set gets merged).Tested-by Paul A. Clarke <[email protected]>One thing I noted, but which also occurs without these patches, is that the perf metrics are not computed unless run as root:
--$ perf stat --metrics br_misprediction_percent command
Quoted 4 lines
Quoted 4 lines
Performance counter stats for 'command':
1,823,530,051 pm_br_pred:u 2,662,705 pm_br_mpred_cmpl:u
it's because when you don't run as root, events will get the :u suffix, so they will not match the events in metric.. we need to decide if we want the metric to match events without modifiers or be strict about it
Quoted 9 lines
Quoted 9 lines
$ /usr/bin/sudo perf stat --metrics br_misprediction_percent commandPerformance counter stats for 'command':
1,824,655,269 pm_br_pred # 0.09 br_misprediction_percent 1,654,466 pm_br_mpred_cmpl--
Quoted 6 lines
Quoted 1 line
Is that expected? I don't think it's always been that way.
I agree Paul, this seems broken. I've noticed a bunch of issues with printing CSV, per-socket output and so on. Jiri may have a better idea but I plan to look at problems in this area later, and hopefully stick a few tests on it :-)
ok, that'd be great, thanks
jirka
Arnaldo Carvalho de Melo
maintainerEm Mon, Aug 03, 2020 at 06:26:36PM +0200, Jiri Olsa escreveu:
Quoted 52 lines · Ian Rogers
Quoted 51 lines · Paul A. Clarke
Quoted 50 lines · Jiri Olsa
Quoted 32 lines
this patchset is adding the support to reused metric in another metric.
For example, to define IPC by using CPI with change like:
{
"BriefDescription": "Instructions Per Cycle (per Logical Processor)",(quoted diff) +1
-1- "MetricExpr": "INST_RETIRED.ANY / CPU_CLK_UNHALTED.THREAD", + "MetricExpr": "1/CPI", "MetricGroup": "TopDownL1", "MetricName": "IPC" },I won't be able to find all the possible places we could use this at, so I wonder you guys (who was asking for this) would try it and come up with comments if there's something missing or we could already use it at some places.
It's based on Arnaldo's tmp.perf/core.
v4 changes:
- removed acks from patch because it changed a bit
with the last fixes:
perf metric: Collect referenced metrics in structmetric_ref_node
- fixed runtime metrics [Kajol Jain]
- increased recursion depth [Paul A. Clarke]
- changed patches due to dependencies:
perf metric: Collect referenced metrics in structmetric_ref_nodeperf metric: Add recursion check when processing nested metrics perf metric: Rename struct egroup to metric
perf metric: Renamegroup_listtometric_listQuoted 3 lines
Also available in here:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/metricI built and ran from the above git branch, and things seem to work. Indeed, I was able to apply my changes to exploit the new capabilities via modifications to
tools/perf/pmu-events/arch/powerpc/power9/metrics.json, as I posted earlier (and will submit once this set gets merged).Tested-by Paul A. Clarke <[email protected]>One thing I noted, but which also occurs without these patches, is that the perf metrics are not computed unless run as root:
--$ perf stat --metrics br_misprediction_percent command
Quoted 6 lines
Quoted 4 lines
Quoted 4 lines
Performance counter stats for 'command':
1,823,530,051 pm_br_pred:u 2,662,705 pm_br_mpred_cmpl:uit's because when you don't run as root, events will get the :u suffix,
s/as root/with the required permissions/
I.e. using CAP_PERFMON, etc as per the explanation at Documentation/admin-guide/perf-security.rst.
Just to help more people be aware of a more secure way of using perf.
Quoted 12 lines
so they will not match the events in metric.. we need to decide if we want the metric to match events without modifiers or be strict about it
Quoted 9 lines
Quoted 9 lines
$ /usr/bin/sudo perf stat --metrics br_misprediction_percent commandPerformance counter stats for 'command':
1,824,655,269 pm_br_pred # 0.09 br_misprediction_percent 1,654,466 pm_br_mpred_cmpl--
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Is that expected? I don't think it's always been that way.
I agree Paul, this seems broken. I've noticed a bunch of issues with printing CSV, per-socket output and so on. Jiri may have a better idea but I plan to look at problems in this area later, and hopefully stick a few tests on it :-)
ok, that'd be great, thanks
Yeah, tests are good, keep them coming :-)
- Arnaldo
Quoted 12 lines · Paul A. Clarke
Quoted 12 lines
v4 changes:
- removed acks from patch because it changed a bit
with the last fixes:
perf metric: Collect referenced metrics in structmetric_ref_node
- fixed runtime metrics [Kajol Jain]
- increased recursion depth [Paul A. Clarke]
- changed patches due to dependencies:
perf metric: Collect referenced metrics in structmetric_ref_nodeperf metric: Add recursion check when processing nested metrics perf metric: Rename struct egroup to metric
perf metric: Renamegroup_listtometric_list
Quoted 8 lines
Quoted 3 lines
Also available in here:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/metricI built and ran from the above git branch, and things seem to work. Indeed, I was able to apply my changes to exploit the new capabilities via modifications to
tools/perf/pmu-events/arch/powerpc/power9/metrics.json, as I posted earlier (and will submit once this set gets merged).
Hi Paul,
I was just wondering: Does perf subtest 10.3 work ok for you with the metric reuse?
That's "Parsing of PMU event table metrics" subtest.
Hi Jirka,
If I add something like this:
{
"BriefDescription": "dummy test1", "MetricExpr": "Bad_Speculation + Frontend_Bound", "MetricGroup": "TopdownL1", "MetricName": "dummytest", "PublicDescription": "dummy test2"
},
I get "Parse event failed metric 'dummytest' id 'bad_speculation' expr 'bad_speculation + frontend_bound'"
Thanks,
john
Quoted 19 lines
Tested-by Paul A. Clarke<[email protected]>One thing I noted, but which also occurs without these patches, is that the perf metrics are not computed unless run as root:
--$ perf stat --metrics br_misprediction_percent commandPerformance counter stats for 'command':
1,823,530,051 pm_br_pred:u 2,662,705 pm_br_mpred_cmpl:u$ /usr/bin/sudo perf stat --metrics br_misprediction_percent commandPerformance counter stats for 'command':
1,824,655,269 pm_br_pred # 0.09 br_misprediction_percent 1,654,466 pm_br_mpred_cmpl--
Quoted 21 lines · John Garry
Quoted 12 lines · Paul A. Clarke
Quoted 12 lines
v4 changes:
- removed acks from patch because it changed a bit
with the last fixes:
perf metric: Collect referenced metrics in structmetric_ref_node
- fixed runtime metrics [Kajol Jain]
- increased recursion depth [Paul A. Clarke]
- changed patches due to dependencies:
perf metric: Collect referenced metrics in structmetric_ref_nodeperf metric: Add recursion check when processing nested metrics perf metric: Rename struct egroup to metric
perf metric: Renamegroup_listtometric_listQuoted 7 lines
Quoted 3 lines
Also available in here:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/metricI built and ran from the above git branch, and things seem to work. Indeed, I was able to apply my changes to exploit the new capabilities via modifications to
tools/perf/pmu-events/arch/powerpc/power9/metrics.json, as I posted earlier (and will submit once this set gets merged).
Quoted 4 lines
I was just wondering: Does perf subtest 10.3 work ok for you with the metric reuse?
That's "Parsing of PMU event table metrics" subtest.
I confess I'm not sure what you are asking. Using the latest mainline (84196390620ac0e5070ae36af84c137c6216a7dc), perf subtest 10.3 does pass for me:
--
$ ./perf test 10 10: PMU events : 10.1: PMU event table sanity : Ok 10.2: PMU event map aliases : Ok 10.3: Parsing of PMU event table metrics : Ok 10.4: Parsing of PMU event table metrics with fake PMUs : Ok --
PC
Quoted 29 lines · Paul A. Clarke
Quoted 16 lines · John Garry
Quoted 15 lines · Paul A. Clarke
Quoted 15 lines
v4 changes:
- removed acks from patch because it changed a bit
with the last fixes:
perf metric: Collect referenced metrics in structmetric_ref_node
- fixed runtime metrics [Kajol Jain]
- increased recursion depth [Paul A. Clarke]
- changed patches due to dependencies:
perf metric: Collect referenced metrics in structmetric_ref_nodeperf metric: Add recursion check when processing nested metrics perf metric: Rename struct egroup to metric
perf metric: Renamegroup_listtometric_listAlso available in here:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/metricQuoted 8 lines
Quoted 4 lines
I built and ran from the above git branch, and things seem to work. Indeed, I was able to apply my changes to exploit the new capabilities via modifications to
tools/perf/pmu-events/arch/powerpc/power9/metrics.json, as I posted earlier (and will submit once this set gets merged).I was just wondering: Does perf subtest 10.3 work ok for you with the metric reuse?
That's "Parsing of PMU event table metrics" subtest.
I confess I'm not sure what you are asking. Using the latest mainline (
84196390620ac0e5070ae36af84c137c6216a7dc), perf subtest 10.3 does pass for me:
--
Quoted 7 lines
$ ./perf test 10 10: PMU events : 10.1: PMU event table sanity : Ok 10.2: PMU event map aliases : Ok 10.3: Parsing of PMU event table metrics : Ok 10.4: Parsing of PMU event table metrics with fake PMUs : Ok --
Since commit 8989f5f07605 ("perf stat: Update POWER9 metrics to utilize other metrics"), power9 has reused metrics.
And I am finding that subtest 10.3 caused problems when I tried to introduce metric reuse on arm64, so I was just asking you to check.
Now I am a bit confused...
Thanks for checking,
john
Quoted 37 lines · John Garry
Quoted 29 lines · Paul A. Clarke
Quoted 16 lines · John Garry
Quoted 15 lines · Paul A. Clarke
Quoted 15 lines
v4 changes:
- removed acks from patch because it changed a bit
with the last fixes:
perf metric: Collect referenced metrics in structmetric_ref_node
- fixed runtime metrics [Kajol Jain]
- increased recursion depth [Paul A. Clarke]
- changed patches due to dependencies:
perf metric: Collect referenced metrics in structmetric_ref_nodeperf metric: Add recursion check when processing nested metrics perf metric: Rename struct egroup to metric
perf metric: Renamegroup_listtometric_listAlso available in here:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/metricQuoted 8 lines
Quoted 4 lines
I built and ran from the above git branch, and things seem to work. Indeed, I was able to apply my changes to exploit the new capabilities via modifications to
tools/perf/pmu-events/arch/powerpc/power9/metrics.json, as I posted earlier (and will submit once this set gets merged).I was just wondering: Does perf subtest 10.3 work ok for you with the metric reuse?
That's "Parsing of PMU event table metrics" subtest.
I confess I'm not sure what you are asking. Using the latest mainline (
84196390620ac0e5070ae36af84c137c6216a7dc), perf subtest 10.3 does pass for me:
--Quoted 7 lines
$ ./perf test 10 10: PMU events : 10.1: PMU event table sanity : Ok 10.2: PMU event map aliases : Ok 10.3: Parsing of PMU event table metrics : Ok 10.4: Parsing of PMU event table metrics with fake PMUs : Ok --
Quoted 7 lines
Since commit
8989f5f07605("perf stat: Update POWER9 metrics to utilize other metrics"), power9 has reused metrics.And I am finding that subtest 10.3 caused problems when I tried to introduce metric reuse on arm64, so I was just asking you to check.
Now I am a bit confused...
I now understand your original request! :-)
The above test was run on a POWER8 system.
I do see failures on a POWER9 system:
--
$ ./perf test 10 10: PMU events : 10.1: PMU event table sanity : Ok 10.2: PMU event map aliases : Ok 10.3: Parsing of PMU event table metrics : Skip (some metrics failed) 10.4: Parsing of PMU event table metrics with fake PMUs : Ok $ ./perf test --verbose 10 2>&1 | grep -i '^Parse event failed' | wc -l 112 --
PC
Quoted 14 lines · Paul A. Clarke
Quoted 8 lines
Quoted 1 line
--
Since commit
8989f5f07605("perf stat: Update POWER9 metrics to utilize other metrics"), power9 has reused metrics.And I am finding that subtest 10.3 caused problems when I tried to introduce metric reuse on arm64, so I was just asking you to check.
Now I am a bit confused...
I now understand your original request!:-)
The above test was run on a POWER8 system.
I do see failures on a POWER9 system:
--
Quoted 8 lines
$ ./perf test 10 10: PMU events : 10.1: PMU event table sanity : Ok 10.2: PMU event map aliases : Ok 10.3: Parsing of PMU event table metrics : Skip (some metrics failed) 10.4: Parsing of PMU event table metrics with fake PMUs : Ok $ ./perf test --verbose 10 2>&1 | grep -i '^Parse event failed' | wc -l 112
ok, thanks for confirming.
I'm looking at fixing it, and the solution doesn't look simple :(
John