perf metric fixes and test
Ian Rogers
reviewerv3 cover perf metric fixes and test
Add a test that all perf metrics (for your architecture) are parsable with the simple expression parser. Attempt to parse all events in metrics but only fail if the metric is for the current CPU. Fix bugs in the expr parser, x86 and powerpc metrics. Improve debug messages around add PMU config term failures.
v3 adds parse event testing of ids and improves debug messages for add
PMU. These messages are paticular visible with 'perf test 10 -vvv'. It moves the testing logic from tests/expr.c to tests/pmu-events.c as suggested by John Garry <[email protected]>.
v2 adds Fixes tags to commit messages for when broken metrics were
first added. Adds a debug warning for division by zero in expr, and adds a workaround for id values in the expr test necessary for powerpc. It also fixes broken power8 and power9 metrics.
Ian Rogers (12):
perf expr: unlimited escaped characters in a symbol perf metrics: fix parse errors in cascade lake metrics perf metrics: fix parse errors in skylake metrics perf expr: allow ',' to be an other token
perf expr: increase max other
perf expr: parse numbers as doubles
perf expr: debug lex if debugging yacc
perf metrics: fix parse errors in power8 metrics perf metrics: fix parse errors in power9 metrics perf expr: print a debug message for division by zero perf parse-events: expand add PMU error/verbose messages perf test: improve pmu event metric testing
tools/perf/arch/x86/util/intel-pt.c | 32 ++-- .../arch/powerpc/power8/metrics.json | 2 +- .../arch/powerpc/power9/metrics.json | 2 +- .../arch/x86/cascadelakex/clx-metrics.json | 10 +- .../arch/x86/skylakex/skx-metrics.json | 4 +- tools/perf/tests/builtin-test.c | 5 + tools/perf/tests/expr.c | 1 + tools/perf/tests/pmu-events.c | 156 +++++++++++++++++- tools/perf/tests/pmu.c | 4 +- tools/perf/tests/tests.h | 2 + tools/perf/util/expr.c | 1 + tools/perf/util/expr.h | 2 +- tools/perf/util/expr.l | 16 +- tools/perf/util/expr.y | 16 +- tools/perf/util/parse-events.c | 29 +++- tools/perf/util/pmu.c | 32 ++-- tools/perf/util/pmu.h | 2 +- 17 files changed, 261 insertions(+), 55 deletions(-)
Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Ian Rogers
reviewerPATCH v3 1/12 perf expr: unlimited escaped characters in a symbol
Current expression allows 2 escaped '-,=' characters. However, some metrics require more, for example Haswell DRAM_BW_Use.
--- tools/perf/util/expr.l | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/util/expr.l +1 -1
@@ -73,7 +73,7 @@ number [0-9]+ sch [-,=] spec \\{sch} sym [0-9a-zA-Z_\.:@]+ -symbol {spec}*{sym}*{spec}*{sym}* +symbol ({spec}|{sym})+ %% struct expr_scanner_ctx *sctx = expr_get_extra(yyscanner);
Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Ian Rogers
reviewerPATCH v3 2/12 perf metrics: fix parse errors in cascade lake metrics
In reply to Ian RogersRemove over escaping with \\.
Remove extraneous if 1 if 0 == 1 else 0 else 0.
--- .../pmu-events/arch/x86/cascadelakex/clx-metrics.json | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
tools/perf/pmu-events/arch/x86/cascadelakex/clx-metrics.json +5 -5
@@ -328,31 +328,31 @@ }, { "BriefDescription": "Average latency of data read request to external memory (in nanoseconds). Accounts for demand loads and L1/L2 prefetches", - "MetricExpr": "1000000000 * ( cha@event\\=0x36\\\\\\,umask\\=0x21@ / cha@event\\=0x35\\\\\\,umask\\=0x21@ ) / ( cha_0@event\\=0x0@ / duration_time )", + "MetricExpr": "1000000000 * ( cha@event\\=0x36\\,umask\\=0x21@ / cha@event\\=0x35\\,umask\\=0x21@ ) / ( cha_0@event\\=0x0@ / duration_time )", "MetricGroup": "Memory_Lat", "MetricName": "DRAM_Read_Latency" }, { "BriefDescription": "Average number of parallel data read requests to external memory. Accounts for demand loads and L1/L2 prefetches", - "MetricExpr": "cha@event\\=0x36\\\\\\,umask\\=0x21@ / cha@event\\=0x36\\\\\\,umask\\=0x21\\\\\\,thresh\\=1@", + "MetricExpr": "cha@event\\=0x36\\,umask\\=0x21@ / cha@event\\=0x36\\,umask\\=0x21\\,thresh\\=1@", "MetricGroup": "Memory_BW", "MetricName": "DRAM_Parallel_Reads" }, { "BriefDescription": "Average latency of data read request to external 3D X-Point memory [in nanoseconds]. Accounts for demand loads and L1/L2 data-read prefetches", - "MetricExpr": "( 1000000000 * ( imc@event\\=0xe0\\\\\\,umask\\=0x1@ / imc@event\\=0xe3@ ) / imc_0@event\\=0x0@ ) if 1 if 0 == 1 else 0 else 0", + "MetricExpr": "( 1000000000 * ( imc@event\\=0xe0\\,umask\\=0x1@ / imc@event\\=0xe3@ ) / imc_0@event\\=0x0@ )", "MetricGroup": "Memory_Lat", "MetricName": "MEM_PMM_Read_Latency" }, { "BriefDescription": "Average 3DXP Memory Bandwidth Use for reads [GB / sec]", - "MetricExpr": "( ( 64 * imc@event\\=0xe3@ / 1000000000 ) / duration_time ) if 1 if 0 == 1 else 0 else 0", + "MetricExpr": "( ( 64 * imc@event\\=0xe3@ / 1000000000 ) / duration_time )", "MetricGroup": "Memory_BW", "MetricName": "PMM_Read_BW" }, { "BriefDescription": "Average 3DXP Memory Bandwidth Use for Writes [GB / sec]", - "MetricExpr": "( ( 64 * imc@event\\=0xe7@ / 1000000000 ) / duration_time ) if 1 if 0 == 1 else 0 else 0", + "MetricExpr": "( ( 64 * imc@event\\=0xe7@ / 1000000000 ) / duration_time )", "MetricGroup": "Memory_BW", "MetricName": "PMM_Write_BW" },
Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Ian Rogers
reviewerPATCH v3 3/12 perf metrics: fix parse errors in skylake metrics
In reply to Ian RogersRemove over escaping with \\.
--- tools/perf/pmu-events/arch/x86/skylakex/skx-metrics.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/pmu-events/arch/x86/skylakex/skx-metrics.json +2 -2
@@ -328,13 +328,13 @@ }, { "BriefDescription": "Average latency of data read request to external memory (in nanoseconds). Accounts for demand loads and L1/L2 prefetches", - "MetricExpr": "1000000000 * ( cha@event\\=0x36\\\\\\,umask\\=0x21@ / cha@event\\=0x35\\\\\\,umask\\=0x21@ ) / ( cha_0@event\\=0x0@ / duration_time )", + "MetricExpr": "1000000000 * ( cha@event\\=0x36\\,umask\\=0x21@ / cha@event\\=0x35\\,umask\\=0x21@ ) / ( cha_0@event\\=0x0@ / duration_time )", "MetricGroup": "Memory_Lat", "MetricName": "DRAM_Read_Latency" }, { "BriefDescription": "Average number of parallel data read requests to external memory. Accounts for demand loads and L1/L2 prefetches", - "MetricExpr": "cha@event\\=0x36\\\\\\,umask\\=0x21@ / cha@event\\=0x36\\\\\\,umask\\=0x21\\\\\\,thresh\\=1@", + "MetricExpr": "cha@event\\=0x36\\,umask\\=0x21@ / cha@event\\=0x36\\,umask\\=0x21\\,thresh\\=1@", "MetricGroup": "Memory_BW", "MetricName": "DRAM_Parallel_Reads" },
Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Ian Rogers
reviewerPATCH v3 4/12 perf expr: allow ',' to be an other token
In reply to Ian RogersCorrects parse errors in expr__find_other of expressions with min.
--- tools/perf/util/expr.y | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/util/expr.y +1 -1
@@ -80,7 +80,7 @@ other: ID ctx->ids[ctx->num_ids++].name = $1; } | -MIN | MAX | IF | ELSE | SMT_ON | NUMBER | '|' | '^' | '&' | '-' | '+' | '*' | '/' | '%' | '(' | ')' +MIN | MAX | IF | ELSE | SMT_ON | NUMBER | '|' | '^' | '&' | '-' | '+' | '*' | '/' | '%' | '(' | ')' | ',' all_expr: if_expr { *final_val = $1; }
Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Ian Rogers
reviewerPATCH v3 5/12 perf expr: increase max other
In reply to Ian RogersLarge metrics such as Branch_Misprediction_Cost_SMT on x86 broadwell need more space.
--- tools/perf/util/expr.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/util/expr.h +1 -1
@@ -2,7 +2,7 @@#ifndef PARSE_CTX_H #define PARSE_CTX_H 1 -#define EXPR_MAX_OTHER 20 +#define EXPR_MAX_OTHER 64#define MAX_PARSE_ID EXPR_MAX_OTHER struct expr_parse_id {
Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Ian Rogers
reviewerPATCH v3 6/12 perf expr: parse numbers as doubles
In reply to Ian RogersThis is expected in expr.y and metrics use floating point values such as x86 broadwell IFetch_Line_Utilization.
--- tools/perf/util/expr.l | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)
tools/perf/util/expr.l +7 -7
@@ -10,12 +10,12 @@ char *expr_get_text(yyscan_t yyscanner); YYSTYPE *expr_get_lval(yyscan_t yyscanner); -static int __value(YYSTYPE *yylval, char *str, int base, int token) +static double __value(YYSTYPE *yylval, char *str, int token) { - u64 num; + double num; errno = 0; - num = strtoull(str, NULL, base); + num = strtod(str, NULL); if (errno) return EXPR_ERROR;
@@ -23,12 +23,12 @@ static int __value(YYSTYPE *yylval, char *str, int base, int token) return token; } -static int value(yyscan_t scanner, int base) +static int value(yyscan_t scanner) { YYSTYPE *yylval = expr_get_lval(scanner); char *text = expr_get_text(scanner); - return __value(yylval, text, base, NUMBER); + return __value(yylval, text, NUMBER); } /*
@@ -68,7 +68,7 @@ static int str(yyscan_t scanner, int token) } %} -number [0-9]+ +number [0-9]*\.?[0-9]+ sch [-,=] spec \\{sch}
@@ -92,7 +92,7 @@ min { return MIN; } if { return IF; } else { return ELSE; } #smt_on { return SMT_ON; } -{number} { return value(yyscanner, 10); } +{number} { return value(yyscanner); } {symbol} { return str(yyscanner, ID); } "|" { return '|'; } "^" { return '^'; }
Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Ian Rogers
reviewerPATCH v3 7/12 perf expr: debug lex if debugging yacc
In reply to Ian RogersOnly effects parser debugging (disabled by default). Enables displaying '--accepting rule at line .. ("...").
--- tools/perf/util/expr.c | 1 + 1 file changed, 1 insertion(+)
tools/perf/util/expr.c +1 -0
@@ -44,6 +44,7 @@ __expr__parse(double *val, struct expr_parse_ctx *ctx, const char *expr,#ifdef PARSER_DEBUG expr_debug =1 ; +expr_set_debug (1 , scanner);#endif ret =expr_parse (val, ctx, scanner);
Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Ian Rogers
reviewerPATCH v3 8/12 perf metrics: fix parse errors in power8 metrics
In reply to Ian RogersMismatched parentheses.
--- tools/perf/pmu-events/arch/powerpc/power8/metrics.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/pmu-events/arch/powerpc/power8/metrics.json +1 -1
@@ -169,7 +169,7 @@ }, { "BriefDescription": "Cycles GCT empty where dispatch was held", - "MetricExpr": "(PM_GCT_NOSLOT_DISP_HELD_MAP + PM_GCT_NOSLOT_DISP_HELD_SRQ + PM_GCT_NOSLOT_DISP_HELD_ISSQ + PM_GCT_NOSLOT_DISP_HELD_OTHER) / PM_RUN_INST_CMPL)", + "MetricExpr": "(PM_GCT_NOSLOT_DISP_HELD_MAP + PM_GCT_NOSLOT_DISP_HELD_SRQ + PM_GCT_NOSLOT_DISP_HELD_ISSQ + PM_GCT_NOSLOT_DISP_HELD_OTHER) / PM_RUN_INST_CMPL", "MetricGroup": "cpi_breakdown", "MetricName": "gct_empty_disp_held_cpi" },
Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Ian Rogers
reviewerPATCH v3 9/12 perf metrics: fix parse errors in power9 metrics
In reply to Ian RogersMismatched parentheses.
--- tools/perf/pmu-events/arch/powerpc/power9/metrics.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/pmu-events/arch/powerpc/power9/metrics.json +1 -1
@@ -362,7 +362,7 @@ }, { "BriefDescription": "Completion stall for other reasons", - "MetricExpr": "PM_CMPLU_STALL - PM_CMPLU_STALL_NTC_DISP_FIN - PM_CMPLU_STALL_NTC_FLUSH - PM_CMPLU_STALL_LSU - PM_CMPLU_STALL_EXEC_UNIT - PM_CMPLU_STALL_BRU)/PM_RUN_INST_CMPL", + "MetricExpr": "(PM_CMPLU_STALL - PM_CMPLU_STALL_NTC_DISP_FIN - PM_CMPLU_STALL_NTC_FLUSH - PM_CMPLU_STALL_LSU - PM_CMPLU_STALL_EXEC_UNIT - PM_CMPLU_STALL_BRU)/PM_RUN_INST_CMPL", "MetricGroup": "cpi_breakdown", "MetricName": "other_stall_cpi" },
Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Ian Rogers
reviewerPATCH v3 10/12 perf expr: print a debug message for division by zero
In reply to Ian RogersIf an expression yields 0 and is then divided-by/modulus-by then the parsing aborts. Add a debug error message to better enable debugging when this happens.
--- tools/perf/util/expr.y | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)
tools/perf/util/expr.y +12 -2
@@ -103,8 +103,18 @@ expr: NUMBER | expr '+' expr { $$ = $1 + $3; } | expr '-' expr { $$ = $1 - $3; } | expr '*' expr { $$ = $1 * $3; } - | expr '/' expr { if ($3 == 0) YYABORT; $$ = $1 / $3; } - | expr '%' expr { if ((long)$3 == 0) YYABORT; $$ = (long)$1 % (long)$3; } + | expr '/' expr { if ($3 == 0) { + pr_debug("division by zero\n"); + YYABORT; + } + $$ = $1 / $3; + } + | expr '%' expr { if ((long)$3 == 0) { + pr_debug("division by zero\n"); + YYABORT; + } + $$ = (long)$1 % (long)$3; + } | '-' expr %prec NEG { $$ = -$2; } | '(' if_expr ')' { $$ = $2; } | MIN '(' expr ',' expr ')' { $$ = $3 < $5 ? $3 : $5; }
Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Ian Rogers
reviewerPATCH v3 11/12 perf parse-events: expand add PMU error/verbose messages
In reply to Ian RogersOn a CPU like skylakex an uncore_iio_0 PMU may alias with uncore_iio_free_running_0. The latter PMU doesn't support fc_mask as a parameter and so pmu_config_term fails. Typically parse_events_add_pmu is called in a loop where if one alias succeeds errors are ignored, however, if multiple errors occur parse_events__handle_error will currently give a WARN_ONCE.
This change removes the WARN_ONCE in parse_events__handle_error and makes it a pr_debug. It adds verbose messages to parse_events_add_pmu warning that non-fatal errors may occur, while giving details on the pmu and config terms for useful context. pmu_config_term is altered so the failing term and pmu are present in the case of the 'unknown term' error which makes spotting the free_running case more straightforward.
Before:
$ perf --debug verbose=3 stat -M llc_misses.pcie_read sleep 1
Using CPUID GenuineIntel-6-55-4
metric expr unc_iio_data_req_of_cpu.mem_read.part0 + unc_iio_data_req_of_cpu.mem_read.part1 + unc_iio_data_req_of_cpu.mem_read.part2 + unc_iio_data_req_of_cpu.mem_read.part3 for LLC_MISSES.PCIE_READ
found event unc_iio_data_req_of_cpu.mem_read.part0
found event unc_iio_data_req_of_cpu.mem_read.part1
found event unc_iio_data_req_of_cpu.mem_read.part2
found event unc_iio_data_req_of_cpu.mem_read.part3
metric expr unc_iio_data_req_of_cpu.mem_read.part0 + unc_iio_data_req_of_cpu.mem_read.part1 + unc_iio_data_req_of_cpu.mem_read.part2 + unc_iio_data_req_of_cpu.mem_read.part3 for LLC_MISSES.PCIE_READ
found event unc_iio_data_req_of_cpu.mem_read.part0
found event unc_iio_data_req_of_cpu.mem_read.part1
found event unc_iio_data_req_of_cpu.mem_read.part2
found event unc_iio_data_req_of_cpu.mem_read.part3
adding {unc_iio_data_req_of_cpu.mem_read.part0,unc_iio_data_req_of_cpu.mem_read.part1,unc_iio_data_req_of_cpu.mem_read.part2,unc_iio_data_req_of_cpu.mem_read.part3}:W,{unc_iio_data_req_of_cpu.mem_read.part0,unc_iio_data_req_of_cpu.mem_read.part1,unc_iio_data_req_of_cpu.mem_read.part2,unc_iio_data_req_of_cpu.mem_read.part3}:W
intel_pt default config: tsc,mtc,mtc_period=3,psb_period=3,pt,branch
WARNING: multiple event parsing errors
...
Invalid event/parameter 'fc_mask'
...After:
$ perf --debug verbose=3 stat -M llc_misses.pcie_read sleep 1
Using CPUID GenuineIntel-6-55-4
metric expr unc_iio_data_req_of_cpu.mem_read.part0 + unc_iio_data_req_of_cpu.mem_read.part1 + unc_iio_data_req_of_cpu.mem_read.part2 + unc_iio_data_req_of_cpu.mem_read.part3 for LLC_MISSES.PCIE_READ
found event unc_iio_data_req_of_cpu.mem_read.part0
found event unc_iio_data_req_of_cpu.mem_read.part1
found event unc_iio_data_req_of_cpu.mem_read.part2
found event unc_iio_data_req_of_cpu.mem_read.part3
metric expr unc_iio_data_req_of_cpu.mem_read.part0 + unc_iio_data_req_of_cpu.mem_read.part1 + unc_iio_data_req_of_cpu.mem_read.part2 + unc_iio_data_req_of_cpu.mem_read.part3 for LLC_MISSES.PCIE_READ
found event unc_iio_data_req_of_cpu.mem_read.part0
found event unc_iio_data_req_of_cpu.mem_read.part1
found event unc_iio_data_req_of_cpu.mem_read.part2
found event unc_iio_data_req_of_cpu.mem_read.part3
adding {unc_iio_data_req_of_cpu.mem_read.part0,unc_iio_data_req_of_cpu.mem_read.part1,unc_iio_data_req_of_cpu.mem_read.part2,unc_iio_data_req_of_cpu.mem_read.part3}:W,{unc_iio_data_req_of_cpu.mem_read.part0,unc_iio_data_req_of_cpu.mem_read.part1,unc_iio_data_req_of_cpu.mem_read.part2,unc_iio_data_req_of_cpu.mem_read.part3}:W
intel_pt default config: tsc,mtc,mtc_period=3,psb_period=3,pt,branch
Attempting to add event pmu 'uncore_iio_free_running_5' with 'unc_iio_data_req_of_cpu.mem_read.part0,' that may result in non-fatal errors
After aliases, add event pmu 'uncore_iio_free_running_5' with 'fc_mask,ch_mask,umask,event,' that may result in non-fatal errors
Attempting to add event pmu 'uncore_iio_free_running_3' with 'unc_iio_data_req_of_cpu.mem_read.part0,' that may result in non-fatal errors
After aliases, add event pmu 'uncore_iio_free_running_3' with 'fc_mask,ch_mask,umask,event,' that may result in non-fatal errors
Attempting to add event pmu 'uncore_iio_free_running_1' with 'unc_iio_data_req_of_cpu.mem_read.part0,' that may result in non-fatal errors
After aliases, add event pmu 'uncore_iio_free_running_1' with 'fc_mask,ch_mask,umask,event,' that may result in non-fatal errors
Multiple errors dropping message: unknown term 'fc_mask' for pmu 'uncore_iio_free_running_3' (valid terms: event,umask,config,config1,config2,name,period,percore)
...--- tools/perf/arch/x86/util/intel-pt.c | 32 ++++++++++++++++++----------- tools/perf/tests/pmu.c | 4 ++-- tools/perf/util/parse-events.c | 29 +++++++++++++++++++++++++- tools/perf/util/pmu.c | 32 ++++++++++++++++++----------- tools/perf/util/pmu.h | 2 +- 5 files changed, 71 insertions(+), 28 deletions(-)
tools/perf/arch/x86/util/intel-pt.c +20 -12
@@ -58,7 +58,8 @@ struct intel_pt_recording {size_t priv_size; }; -static int intel_pt_parse_terms_with_default (struct list_head *formats, +static int intel_pt_parse_terms_with_default (const char *pmu_name, +struct list_head *formats,const char *str,u64 *config) {
@@ -77,7 +78,8 @@ static int intel_pt_parse_terms_with_default(struct list_head *formats, goto out_free; attr.config = *config; - err =perf_pmu__config_terms (formats, &attr, terms, true,NULL ); + err =perf_pmu__config_terms (pmu_name, formats, &attr, terms, true, +NULL );if (err) goto out_free;
@@ -87,11 +89,12 @@ static int intel_pt_parse_terms_with_default(struct list_head *formats,return err; } -static int intel_pt_parse_terms (struct list_head *formats,const char *str, -u64 *config) +static int intel_pt_parse_terms (const char *pmu_name,struct list_head *formats, +const char *str,u64 *config) { *config =0 ; -return intel_pt_parse_terms_with_default (formats, str, config); +return intel_pt_parse_terms_with_default (pmu_name, formats, str, + config); }static u64 intel_pt_masked_bits (u64 mask,u64 bits)
@@ -228,7 +231,8 @@ static u64 intel_pt_default_config(struct perf_pmu *intel_pt_pmu)pr_debug2 ("%s default config: %s\n" , intel_pt_pmu->name, buf); -intel_pt_parse_terms (&intel_pt_pmu->format, buf, &config); +intel_pt_parse_terms (intel_pt_pmu->name, &intel_pt_pmu->format, buf, + &config);return config; }
@@ -336,13 +340,16 @@ static int intel_pt_info_fill(struct auxtrace_record *itr,if (priv_size != ptr->priv_size)return -EINVAL ; -intel_pt_parse_terms (&intel_pt_pmu->format,"tsc" , &tsc_bit); -intel_pt_parse_terms (&intel_pt_pmu->format,"noretcomp" , - &noretcomp_bit); -intel_pt_parse_terms (&intel_pt_pmu->format,"mtc" , &mtc_bit); +intel_pt_parse_terms (intel_pt_pmu->name, &intel_pt_pmu->format, +"tsc" , &tsc_bit); +intel_pt_parse_terms (intel_pt_pmu->name, &intel_pt_pmu->format, +"noretcomp" , &noretcomp_bit); +intel_pt_parse_terms (intel_pt_pmu->name, &intel_pt_pmu->format, +"mtc" , &mtc_bit); mtc_freq_bits =perf_pmu__format_bits (&intel_pt_pmu->format,"mtc_period" ); -intel_pt_parse_terms (&intel_pt_pmu->format,"cyc" , &cyc_bit); +intel_pt_parse_terms (intel_pt_pmu->name, &intel_pt_pmu->format, +"cyc" , &cyc_bit);intel_pt_tsc_ctc_ratio (&tsc_ctc_ratio_n, &tsc_ctc_ratio_d);
@@ -768,7 +775,8 @@ static int intel_pt_recording_options(struct auxtrace_record *itr, } } -intel_pt_parse_terms (&intel_pt_pmu->format,"tsc" , &tsc_bit); +intel_pt_parse_terms (intel_pt_pmu->name, &intel_pt_pmu->format, +"tsc" , &tsc_bit);if (opts->full_auxtrace && (intel_pt_evsel->core.attr.config & tsc_bit)) have_timing_info = true;
tools/perf/tests/pmu.c +2 -2
@@ -156,8 +156,8 @@ int test__pmu(struct test *test __maybe_unused, int subtest __maybe_unused)if (ret)break ; - ret =perf_pmu__config_terms (&formats, &attr, terms, - false,NULL ); + ret =perf_pmu__config_terms ("perf-pmu-test" , &formats, &attr, + terms, false,NULL );if (ret)break ;
tools/perf/util/parse-events.c +28 -1
@@ -204,7 +204,8 @@ void parse_events__handle_error(struct parse_events_error *err, int idx, err->help = help;break ;default : -WARN_ONCE (1 ,"WARNING: multiple event parsing errors\n" ); +pr_debug ("Multiple errors dropping message: %s (%s)\n" , + err->str, err->help);free (err->str); err->str = str;free (err->help);
@@ -1424,6 +1425,19 @@ int parse_events_add_pmu(struct parse_events_state *parse_state,bool use_uncore_alias;LIST_HEAD (config_terms); +if (verbose >1 ) { +fprintf (stderr,"Attempting to add event pmu '%s' with '" , + name); +if (head_config) { +struct parse_events_term *term; + +list_for_each_entry (term, head_config, list) { +fprintf (stderr,"%s," , term->config); + } + } +fprintf (stderr,"' that may result in non-fatal errors\n" ); + } + pmu =perf_pmu__find (name);if (!pmu) {char *err_str;
@@ -1460,6 +1474,19 @@ int parse_events_add_pmu(struct parse_events_state *parse_state,if (perf_pmu__check_alias (pmu, head_config, &info))return -EINVAL ; +if (verbose >1 ) { +fprintf (stderr,"After aliases, add event pmu '%s' with '" , + name); +if (head_config) { +struct parse_events_term *term; + +list_for_each_entry (term, head_config, list) { +fprintf (stderr,"%s," , term->config); + } + } +fprintf (stderr,"' that may result in non-fatal errors\n" ); + } +/* * Configure hardcoded terms first, no need to check * return value when called with fail == 0 ;)
tools/perf/util/pmu.c +20 -12
@@ -1045,7 +1045,8 @@ static char *pmu_formats_string(struct list_head *formats)* Setup one of config[12] attr members based on the * user input data - term parameter. */ -static int pmu_config_term(struct list_head *formats, +static int pmu_config_term(const char *pmu_name, + struct list_head *formats, struct perf_event_attr *attr, struct parse_events_term *term, struct list_head *head_terms,
@@ -1071,16 +1072,23 @@ static int pmu_config_term(struct list_head *formats, format =pmu_find_format (formats, term->config);if (!format) { -if (verbose >0 ) -printf ("Invalid event/parameter '%s'\n" , term->config); +char *pmu_term =pmu_formats_string (formats); +char *unknown_term; +char *help_msg; + +asprintf (&unknown_term, +"unknown term '%s' for pmu '%s'" , + term->config, pmu_name); + help_msg =parse_events_formats_error_string (pmu_term);if (err) { -char *pmu_term =pmu_formats_string (formats); -parse_events__handle_error (err, term->err_term, -strdup ("unknown term" ), -parse_events_formats_error_string (pmu_term)); -free (pmu_term); + unknown_term, + help_msg); + }else { +pr_debug ("%s (%s)\n" , unknown_term, help_msg); +free (unknown_term); } +free (pmu_term);return -EINVAL ; }
@@ -1157,7 +1165,7 @@ static int pmu_config_term(struct list_head *formats,return 0 ; } -int perf_pmu__config_terms (struct list_head *formats, +int perf_pmu__config_terms (const char *pmu_name,struct list_head *formats,struct perf_event_attr *attr,struct list_head *head_terms,bool zero,struct parse_events_error *err)
@@ -1165,7 +1173,7 @@ int perf_pmu__config_terms(struct list_head *formats,struct parse_events_term *term;list_for_each_entry (term, head_terms, list) { -if (pmu_config_term (formats, attr, term, head_terms, +if (pmu_config_term (pmu_name, formats, attr, term, head_terms, zero, err))return -EINVAL ; }
@@ -1185,8 +1193,8 @@ int perf_pmu__config(struct perf_pmu *pmu, struct perf_event_attr *attr,bool zero = !!pmu->default_config; attr->type = pmu->type; -return perf_pmu__config_terms (&pmu->format, attr, head_terms, - zero, err); +return perf_pmu__config_terms (pmu->name, &pmu->format, attr, + head_terms, zero, err); } static struct perf_pmu_alias *pmu_find_alias(struct perf_pmu *pmu,
tools/perf/util/pmu.h +1 -1
@@ -75,7 +75,7 @@ struct perf_pmu *perf_pmu__find(const char *name);int perf_pmu__config (struct perf_pmu *pmu,struct perf_event_attr *attr,struct list_head *head_terms,struct parse_events_error *error); -int perf_pmu__config_terms (struct list_head *formats, +int perf_pmu__config_terms (const char *pmu_name,struct list_head *formats,struct perf_event_attr *attr,struct list_head *head_terms,bool zero,struct parse_events_error *error);
Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Ian Rogers
reviewerPATCH v3 12/12 perf test: improve pmu event metric testing
In reply to Ian RogersAdd a basic floating point number test to expr.
Break pmu-events test into 2 and add a test to verify that all pmu metric expressions simply parse. Try to parse all metric ids/events, failing if metrics for the current architecture fail to parse.
Tested on skylakex with the patch set in place. May fail on other architectures if metrics are invalid.
--- tools/perf/tests/builtin-test.c | 5 + tools/perf/tests/expr.c | 1 + tools/perf/tests/pmu-events.c | 156 ++++++++++++++++++++++++++++++-- tools/perf/tests/tests.h | 2 + 4 files changed, 158 insertions(+), 6 deletions(-)
tools/perf/tests/builtin-test.c +5 -0
@@ -75,6 +75,11 @@ static struct test generic_tests[] = { { .desc = "PMU events", .func = test__pmu_events, + .subtest = { + .get_nr = test__pmu_events_subtest_get_nr, + .get_desc = test__pmu_events_subtest_get_desc, + }, + }, { .desc = "DSO data read",
tools/perf/tests/expr.c +1 -0
@@ -39,6 +39,7 @@ int test__expr(struct test *t __maybe_unused, int subtest __maybe_unused) ret |=test (&ctx,"min(1,2) + 1" ,2 ); ret |=test (&ctx,"max(1,2) + 1" ,3 ); ret |=test (&ctx,"1+1 if 3*4 else 0" ,2 ); + ret |=test (&ctx,"1.1 + 2.1" ,3.2 );if (ret)return ret;
tools/perf/tests/pmu-events.c +150 -6
180 lines not on this page · raw message
tools/perf/tests/tests.h +2 -0
@@ -50,6 +50,8 @@ int test__perf_evsel__tp_sched_test(struct test *test, int subtest);int test__syscall_openat_tp_fields (struct test *test,int subtest);int test__pmu (struct test *test,int subtest);int test__pmu_events (struct test *test,int subtest); +const char *test__pmu_events_subtest_get_desc (int subtest); +int test__pmu_events_subtest_get_nr (void );int test__attr (struct test *test,int subtest);int test__dso_data (struct test *test,int subtest);int test__dso_data_cache (struct test *test,int subtest);
Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Re: [PATCH v3 12/12] perf test: improve pmu event metric testing
Quoted 3 lines · Ian Rogers
Add a basic floating point number test to expr.
Break pmu-events test into 2 and add a test to verify that all pmu metric expressions simply parse.
Could we add also add something in jevents to ensure this?
Thanks,
John
Try to parse all metric ids/events, failing if
Quoted 2 lines
metrics for the current architecture fail to parse.
Ian Rogers
reviewerRe: [PATCH v3 12/12] perf test: improve pmu event metric testing
Quoted 7 lines · John Garry
Quoted 3 lines · Ian Rogers
Add a basic floating point number test to expr.
Break pmu-events test into 2 and add a test to verify that all pmu metric expressions simply parse.Could we add also add something in jevents to ensure this?
I think it is an interesting possibility. Instead of strings we could also parse the metrics into C functions, that could cause build time errors at least for the simple expressions. An issue I've faced is that if jevents fails, such as a json parse error, it has an exit code of 0 and creates an empty map file. This allows the build to proceed but with the pmu-events functionality broken. I'd prefer a build to fail as early as possible.
Thanks,
Ian
Quoted 6 lines
Thanks,
JohnTry to parse all metric ids/events, failing if
Quoted 2 lines
metrics for the current architecture fail to parse.
Re: [PATCH v3 12/12] perf test: improve pmu event metric testing
Quoted 9 lines · Ian Rogers
Quoted 7 lines · John Garry
Quoted 3 lines · Ian Rogers
Add a basic floating point number test to expr.
Break pmu-events test into 2 and add a test to verify that all pmu metric expressions simply parse.Could we add also add something in jevents to ensure this?
Quoted 7 lines
I think it is an interesting possibility. Instead of strings we could also parse the metrics into C functions, that could cause build time errors at least for the simple expressions. An issue I've faced is that if jevents fails, such as a json parse error, it has an exit code of 0 and creates an empty map file. This allows the build to proceed but with the pmu-events functionality broken. I'd prefer a build to fail as early as possible.
Yeah, the idea is to allow perf to continue to build even when we have broken JSONs, but without aliases. It's been that way since day one, so maybe that can be turned off now.
Thanks,
John
Quoted 16 lines · Ian Rogers
Add a test that all perf metrics (for your architecture) are parsable with the simple expression parser. Attempt to parse all events in metrics but only fail if the metric is for the current CPU. Fix bugs in the expr parser, x86 and powerpc metrics. Improve debug messages around add PMU config term failures.
v3 adds parse event testing of ids and improves debug messages for add
PMU. These messages are paticular visible with 'perf test 10 -vvv'. It moves the testing logic fromtests/expr.ctotests/pmu-events.cas suggested by John Garry <[email protected]>.
v2 adds Fixes tags to commit messages for when broken metrics were
first added. Adds a debug warning for division by zero in expr, and adds a workaround for id values in the expr test necessary for powerpc. It also fixes broken power8 and power9 metrics.
Quoted 13 lines
Ian Rogers (12):
perf expr: unlimited escaped characters in a symbol perf metrics: fix parse errors in cascade lake metrics perf metrics: fix parse errors in skylake metrics perf expr: allow ',' to be an other token
perf expr: increase max other
perf expr: parse numbers as doubles
perf expr: debug lex if debugging yacc
perf metrics: fix parse errors in power8 metrics perf metrics: fix parse errors in power9 metrics perf expr: print a debug message for division by zero perf parse-events: expand add PMU error/verbose messages perf test: improve pmu event metric testing
this does not apply on top of changes from Kajol Jain which are now in Arnaldo's perf/core.. could you please rebase?
thanks,
jirka
Ian Rogers
reviewerQuoted 32 lines · Jiri Olsa
Quoted 16 lines · Ian Rogers
Add a test that all perf metrics (for your architecture) are parsable with the simple expression parser. Attempt to parse all events in metrics but only fail if the metric is for the current CPU. Fix bugs in the expr parser, x86 and powerpc metrics. Improve debug messages around add PMU config term failures.
v3 adds parse event testing of ids and improves debug messages for add
PMU. These messages are paticular visible with 'perf test 10 -vvv'. It moves the testing logic fromtests/expr.ctotests/pmu-events.cas suggested by John Garry <[email protected]>.
v2 adds Fixes tags to commit messages for when broken metrics were
first added. Adds a debug warning for division by zero in expr, and adds a workaround for id values in the expr test necessary for powerpc. It also fixes broken power8 and power9 metrics.Quoted 13 lines
Ian Rogers (12):
perf expr: unlimited escaped characters in a symbol perf metrics: fix parse errors in cascade lake metrics perf metrics: fix parse errors in skylake metrics perf expr: allow ',' to be an other token
perf expr: increase max other
perf expr: parse numbers as doubles
perf expr: debug lex if debugging yacc
perf metrics: fix parse errors in power8 metrics perf metrics: fix parse errors in power9 metrics perf expr: print a debug message for division by zero perf parse-events: expand add PMU error/verbose messages perf test: improve pmu event metric testing
Quoted 3 lines
this does not apply on top of changes from Kajol Jain which are now in Arnaldo's perf/core.. could you please rebase?
Thanks! Done, v4 is here:
https://lore.kernel.org/lkml/[email protected]/T/#m0637a37d54b694c508904790d4c6f9bc24332d0b
The power8/power9 fixes were acked-by IBM, would it be useful for Intel to do the same for skylake/cascade lake? Should I drop those patches to wait for updated ones from Jin Yao? The problem is that without them the test will fail on Intel. It'd be really nice to get the test landed.
Thanks,
Ian
Quoted 3 lines
thanks,
jirka