perf metric fixes and test
Ian Rogers
reviewerv2 cover perf metric fixes and test
Add a test that all perf metrics (for your architecture) are parsable. Fix bugs in the expr parser and in x86 metrics. Untested on architectures other than x86.
v2 adds Fixes tags to commit messages for when broken metrics were
first added. Adds a debug warning for division by zero in expr, and adds a workaround for id values in the expr test necessary for powerpc. It also fixes broken power8 and power9 metrics.
Ian Rogers (11):
perf expr: unlimited escaped characters in a symbol perf metrics: fix parse errors in cascade lake metrics perf metrics: fix parse errors in skylake metrics perf expr: allow ',' to be an other token perf expr: increase max other
perf expr: parse numbers as doubles
perf expr: debug lex if debugging yacc
perf metrics: fix parse errors in power8 metrics perf metrics: fix parse errors in power9 metrics perf expr: print a debug message for division by zero perf test: add expr test for pmu metrics
.../arch/powerpc/power8/metrics.json | 4 +- .../arch/powerpc/power9/metrics.json | 2 +- .../arch/x86/cascadelakex/clx-metrics.json | 10 +- .../arch/x86/skylakex/skx-metrics.json | 4 +- tools/perf/tests/builtin-test.c | 5 + tools/perf/tests/expr.c | 96 ++++++++++++++++++- tools/perf/tests/tests.h | 2 + tools/perf/util/expr.c | 1 + tools/perf/util/expr.h | 2 +- tools/perf/util/expr.l | 16 ++-- tools/perf/util/expr.y | 16 +++- 11 files changed, 135 insertions(+), 23 deletions(-)
Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Ian Rogers
reviewerPATCH v2 1/11 perf expr: unlimited escaped characters in a symbol
Current expression allows 2 escaped '-,=' characters. However, some metrics require more, for example Haswell DRAM_BW_Use.
--- tools/perf/util/expr.l | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/util/expr.l +1 -1
@@ -73,7 +73,7 @@ number [0-9]+ sch [-,=] spec \\{sch} sym [0-9a-zA-Z_\.:@]+ -symbol {spec}*{sym}*{spec}*{sym}* +symbol ({spec}|{sym})+ %% struct expr_scanner_ctx *sctx = expr_get_extra(yyscanner);
Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Re: [PATCH v2 01/11] perf expr: unlimited escaped characters in a symbol
Quoted 14 lines · Ian Rogers
Current expression allows 2 escaped '-,=' characters. However, some metrics require more, for example Haswell
DRAM_BW_Use.Fixes 26226a97724d (perf expr: Move expr lexer to flex)Signed-off-by Ian Rogers <[email protected]>--- tools/perf/util/expr.l | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)tools/perf/util/expr.l +0
-0
tools/perf/util/expr.l number [0-9]+ · line 76
Quoted 5 lines
tools/perf/util/expr.l +1
-1@@ -73,7 +73,7 @@ number [0-9]+ sch [-,=] spec \\{sch} sym [0-9a-zA-Z_\.:@]+ -symbol {spec}*{sym}*{spec}*{sym}* +symbol ({spec}|{sym})+
yep, much better ;-)
thanks,
jirka
Quoted 6 lines
tools/perf/util/expr.l +0
-0%% struct expr_scanner_ctx *sctx = expr_get_extra(yyscanner);Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Re: [PATCH v2 01/11] perf expr: unlimited escaped characters in a symbol
Quoted 10 lines · Jiri Olsa
Quoted 9 lines · Ian Rogers
Current expression allows 2 escaped '-,=' characters. However, some metrics require more, for example Haswell
DRAM_BW_Use.Fixes 26226a97724d (perf expr: Move expr lexer to flex)Signed-off-by Ian Rogers <[email protected]>--- tools/perf/util/expr.l | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 10 lines
tools/perf/util/expr.l number [0-9]+ · line 76
Quoted 10 lines
tools/perf/util/expr.l +1
-1@@ -73,7 +73,7 @@ number [0-9]+ sch [-,=] spec \\{sch} sym [0-9a-zA-Z_\.:@]+ -symbol {spec}*{sym}*{spec}*{sym}* +symbol ({spec}|{sym})+
Tested by: [email protected]
Thanks,
Kajol Jain
Quoted 13 lines
yep, much better ;-)
thanks,
jirkaQuoted 6 lines
tools/perf/util/expr.l +0
-0%% struct expr_scanner_ctx *sctx = expr_get_extra(yyscanner);Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Ian Rogers
reviewerPATCH v2 2/11 perf metrics: fix parse errors in cascade lake metrics
In reply to Ian RogersRemove over escaping with \\.
Remove extraneous if 1 if 0 == 1 else 0 else 0.
--- .../pmu-events/arch/x86/cascadelakex/clx-metrics.json | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
tools/perf/pmu-events/arch/x86/cascadelakex/clx-metrics.json +5 -5
@@ -328,31 +328,31 @@ }, { "BriefDescription": "Average latency of data read request to external memory (in nanoseconds). Accounts for demand loads and L1/L2 prefetches", - "MetricExpr": "1000000000 * ( cha@event\\=0x36\\\\\\,umask\\=0x21@ / cha@event\\=0x35\\\\\\,umask\\=0x21@ ) / ( cha_0@event\\=0x0@ / duration_time )", + "MetricExpr": "1000000000 * ( cha@event\\=0x36\\,umask\\=0x21@ / cha@event\\=0x35\\,umask\\=0x21@ ) / ( cha_0@event\\=0x0@ / duration_time )", "MetricGroup": "Memory_Lat", "MetricName": "DRAM_Read_Latency" }, { "BriefDescription": "Average number of parallel data read requests to external memory. Accounts for demand loads and L1/L2 prefetches", - "MetricExpr": "cha@event\\=0x36\\\\\\,umask\\=0x21@ / cha@event\\=0x36\\\\\\,umask\\=0x21\\\\\\,thresh\\=1@", + "MetricExpr": "cha@event\\=0x36\\,umask\\=0x21@ / cha@event\\=0x36\\,umask\\=0x21\\,thresh\\=1@", "MetricGroup": "Memory_BW", "MetricName": "DRAM_Parallel_Reads" }, { "BriefDescription": "Average latency of data read request to external 3D X-Point memory [in nanoseconds]. Accounts for demand loads and L1/L2 data-read prefetches", - "MetricExpr": "( 1000000000 * ( imc@event\\=0xe0\\\\\\,umask\\=0x1@ / imc@event\\=0xe3@ ) / imc_0@event\\=0x0@ ) if 1 if 0 == 1 else 0 else 0", + "MetricExpr": "( 1000000000 * ( imc@event\\=0xe0\\,umask\\=0x1@ / imc@event\\=0xe3@ ) / imc_0@event\\=0x0@ )", "MetricGroup": "Memory_Lat", "MetricName": "MEM_PMM_Read_Latency" }, { "BriefDescription": "Average 3DXP Memory Bandwidth Use for reads [GB / sec]", - "MetricExpr": "( ( 64 * imc@event\\=0xe3@ / 1000000000 ) / duration_time ) if 1 if 0 == 1 else 0 else 0", + "MetricExpr": "( ( 64 * imc@event\\=0xe3@ / 1000000000 ) / duration_time )", "MetricGroup": "Memory_BW", "MetricName": "PMM_Read_BW" }, { "BriefDescription": "Average 3DXP Memory Bandwidth Use for Writes [GB / sec]", - "MetricExpr": "( ( 64 * imc@event\\=0xe7@ / 1000000000 ) / duration_time ) if 1 if 0 == 1 else 0 else 0", + "MetricExpr": "( ( 64 * imc@event\\=0xe7@ / 1000000000 ) / duration_time )", "MetricGroup": "Memory_BW", "MetricName": "PMM_Write_BW" },
Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Ian Rogers
reviewerPATCH v2 3/11 perf metrics: fix parse errors in skylake metrics
In reply to Ian RogersRemove over escaping with \\.
--- tools/perf/pmu-events/arch/x86/skylakex/skx-metrics.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/pmu-events/arch/x86/skylakex/skx-metrics.json +2 -2
@@ -328,13 +328,13 @@ }, { "BriefDescription": "Average latency of data read request to external memory (in nanoseconds). Accounts for demand loads and L1/L2 prefetches", - "MetricExpr": "1000000000 * ( cha@event\\=0x36\\\\\\,umask\\=0x21@ / cha@event\\=0x35\\\\\\,umask\\=0x21@ ) / ( cha_0@event\\=0x0@ / duration_time )", + "MetricExpr": "1000000000 * ( cha@event\\=0x36\\,umask\\=0x21@ / cha@event\\=0x35\\,umask\\=0x21@ ) / ( cha_0@event\\=0x0@ / duration_time )", "MetricGroup": "Memory_Lat", "MetricName": "DRAM_Read_Latency" }, { "BriefDescription": "Average number of parallel data read requests to external memory. Accounts for demand loads and L1/L2 prefetches", - "MetricExpr": "cha@event\\=0x36\\\\\\,umask\\=0x21@ / cha@event\\=0x36\\\\\\,umask\\=0x21\\\\\\,thresh\\=1@", + "MetricExpr": "cha@event\\=0x36\\,umask\\=0x21@ / cha@event\\=0x36\\,umask\\=0x21\\,thresh\\=1@", "MetricGroup": "Memory_BW", "MetricName": "DRAM_Parallel_Reads" },
Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Ian Rogers
reviewerPATCH v2 4/11 perf expr: allow ',' to be an other token
In reply to Ian RogersCorrects parse errors in expr__find_other of expressions with min.
--- tools/perf/util/expr.y | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/util/expr.y +1 -1
@@ -80,7 +80,7 @@ other: ID ctx->ids[ctx->num_ids++].name = $1; } | -MIN | MAX | IF | ELSE | SMT_ON | NUMBER | '|' | '^' | '&' | '-' | '+' | '*' | '/' | '%' | '(' | ')' +MIN | MAX | IF | ELSE | SMT_ON | NUMBER | '|' | '^' | '&' | '-' | '+' | '*' | '/' | '%' | '(' | ')' | ',' all_expr: if_expr { *final_val = $1; }
Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Re: [PATCH v2 04/11] perf expr: allow ',' to be an other token
Quoted 12 lines · Ian Rogers
Corrects parse errors in
expr__find_otherof expressions with min.Signed-off-by Ian Rogers <[email protected]>--- tools/perf/util/expr.y | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)tools/perf/util/expr.y +0
-0
tools/perf/util/expr.y other: ID · line 83
Quoted 5 lines
tools/perf/util/expr.y +1
-1@@ -80,7 +80,7 @@ other: ID ctx->ids[ctx->num_ids++].name = $1; } | -MIN | MAX | IF | ELSE | SMT_ON | NUMBER | '|' | '^' | '&' | '-' | '+' | '*' | '/' | '%' | '(' | ')' +MIN | MAX | IF | ELSE | SMT_ON | NUMBER | '|' | '^' | '&' | '-' | '+' | '*' | '/' | '%' | '(' | ')' | ','
ugh, thanks
jirka
Quoted 6 lines
tools/perf/util/expr.y +0
-0all_expr: if_expr { *final_val = $1; }Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Ian Rogers
reviewerPATCH v2 5/11 perf expr: increase max other
In reply to Ian RogersLarge metrics such as Branch_Misprediction_Cost_SMT on x86 broadwell need more space.
--- tools/perf/util/expr.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/util/expr.h +1 -1
@@ -2,7 +2,7 @@#ifndef PARSE_CTX_H #define PARSE_CTX_H 1 -#define EXPR_MAX_OTHER 20 +#define EXPR_MAX_OTHER 64#define MAX_PARSE_ID EXPR_MAX_OTHER struct expr_parse_id {
Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Re: [PATCH v2 05/11] perf expr: increase max other
Quoted 24 lines · Ian Rogers
Large metrics such as
Branch_Misprediction_Cost_SMTon x86 broadwell need more space.Signed-off-by Ian Rogers <[email protected]>--- tools/perf/util/expr.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)tools/perf/util/expr.h +1
-1@@ -2,7 +2,7 @@#ifndef PARSE_CTX_H #define PARSE_CTX_H 1 -#define EXPR_MAX_OTHER 20 +#define EXPR_MAX_OTHER 64#define MAX_PARSE_ID EXPR_MAX_OTHER struct expr_parse_id {Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
ok, and we should probably start to think about what Andi suggested
in here: https://lore.kernel.org/lkml/[email protected]/
jirka
Ian Rogers
reviewerRe: [PATCH v2 05/11] perf expr: increase max other
Quoted 22 lines · Jiri Olsa
Quoted 20 lines · Ian Rogers
Large metrics such as
Branch_Misprediction_Cost_SMTon x86 broadwell need more space.Signed-off-by Ian Rogers <[email protected]>--- tools/perf/util/expr.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)tools/perf/util/expr.h +1
-1@@ -2,7 +2,7 @@#ifndef PARSE_CTX_H #define PARSE_CTX_H 1 -#define EXPR_MAX_OTHER 20 +#define EXPR_MAX_OTHER 64#define MAX_PARSE_ID EXPR_MAX_OTHER
Quoted 7 lines
Quoted 4 lines
tools/perf/util/expr.h +0
-1struct expr_parse_id { --2.26.2.303.gf8c07b1a785-goog
ok, and we should probably start to think about what Andi suggested
in here: https://lore.kernel.org/lkml/[email protected]/
Agreed, a hash table would make sense. This was the smallest value that would let the test on x86 pass.
Thanks,
Ian
Quoted 2 lines
jirka
Ian Rogers
reviewerRe: [PATCH v2 05/11] perf expr: increase max other
Quoted 32 lines · Ian Rogers
Quoted 22 lines · Jiri Olsa
Quoted 20 lines · Ian Rogers
Large metrics such as
Branch_Misprediction_Cost_SMTon x86 broadwell need more space.Signed-off-by Ian Rogers <[email protected]>--- tools/perf/util/expr.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)tools/perf/util/expr.h +1
-1@@ -2,7 +2,7 @@#ifndef PARSE_CTX_H #define PARSE_CTX_H 1 -#define EXPR_MAX_OTHER 20 +#define EXPR_MAX_OTHER 64#define MAX_PARSE_ID EXPR_MAX_OTHERQuoted 7 lines
Quoted 4 lines
tools/perf/util/expr.h +0
-1struct expr_parse_id { --2.26.2.303.gf8c07b1a785-goog
ok, and we should probably start to think about what Andi suggested
in here: https://lore.kernel.org/lkml/[email protected]/
Quoted 2 lines
Agreed, a hash table would make sense. This was the smallest value that would let the test on x86 pass.
Fwiw, I have done this based on tools/lib/bpf/hashmap.h in CLs following on from this patch set. I'm holding off sending so I can rebase on acme's perf/next when the CLs acked by Jirka land:
https://lore.kernel.org/lkml/20200503221650.GA1916255@krava/
The libbpf dependency for a hashmap is counter intuitive, so maybe there's something better to do there.
Thanks,
Ian
Quoted 5 lines
Thanks,
IanQuoted 2 lines
jirka
Ian Rogers
reviewerPATCH v2 6/11 perf expr: parse numbers as doubles
In reply to Ian RogersThis is expected in expr.y and metrics use floating point values such as x86 broadwell IFetch_Line_Utilization.
--- tools/perf/util/expr.l | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)
tools/perf/util/expr.l +7 -7
@@ -10,12 +10,12 @@ char *expr_get_text(yyscan_t yyscanner); YYSTYPE *expr_get_lval(yyscan_t yyscanner); -static int __value(YYSTYPE *yylval, char *str, int base, int token) +static double __value(YYSTYPE *yylval, char *str, int token) { - u64 num; + double num; errno = 0; - num = strtoull(str, NULL, base); + num = strtod(str, NULL); if (errno) return EXPR_ERROR;
@@ -23,12 +23,12 @@ static int __value(YYSTYPE *yylval, char *str, int base, int token) return token; } -static int value(yyscan_t scanner, int base) +static int value(yyscan_t scanner) { YYSTYPE *yylval = expr_get_lval(scanner); char *text = expr_get_text(scanner); - return __value(yylval, text, base, NUMBER); + return __value(yylval, text, NUMBER); } /*
@@ -68,7 +68,7 @@ static int str(yyscan_t scanner, int token) } %} -number [0-9]+ +number [0-9]*\.?[0-9]+ sch [-,=] spec \\{sch}
@@ -92,7 +92,7 @@ min { return MIN; } if { return IF; } else { return ELSE; } #smt_on { return SMT_ON; } -{number} { return value(yyscanner, 10); } +{number} { return value(yyscanner); } {symbol} { return str(yyscanner, ID); } "|" { return '|'; } "^" { return '^'; }
Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Re: [PATCH v2 06/11] perf expr: parse numbers as doubles
Quoted 2 lines · Ian Rogers
This is expected in expr.y and metrics use floating point values such as x86 broadwell
IFetch_Line_Utilization.
ugh, I wonder why we did not get any compiler warning when expr.y expects double.. good catch
jirka
Quoted 63 lines
Fixes 26226a97724d (perf expr: Move expr lexer to flex)Signed-off-by Ian Rogers <[email protected]>--- tools/perf/util/expr.l | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)tools/perf/util/expr.l +7
-7@@ -10,12 +10,12 @@ char *expr_get_text(yyscan_t yyscanner); YYSTYPE *expr_get_lval(yyscan_t yyscanner); -static int __value(YYSTYPE *yylval, char *str, int base, int token) +static double __value(YYSTYPE *yylval, char *str, int token) { - u64 num; + double num; errno = 0; - num = strtoull(str, NULL, base); + num = strtod(str, NULL); if (errno) return EXPR_ERROR;@@ -23,12 +23,12 @@ static int __value(YYSTYPE *yylval, char *str, int base, int token) return token; } -static int value(yyscan_t scanner, int base) +static int value(yyscan_t scanner) { YYSTYPE *yylval = expr_get_lval(scanner); char *text = expr_get_text(scanner); - return __value(yylval, text, base, NUMBER); + return __value(yylval, text, NUMBER); } /*@@ -68,7 +68,7 @@ static int str(yyscan_t scanner, int token) } %} -number [0-9]+ +number [0-9]*\.?[0-9]+ sch [-,=] spec \\{sch}@@ -92,7 +92,7 @@ min { return MIN; } if { return IF; } else { return ELSE; } #smt_on { return SMT_ON; } -{number} { return value(yyscanner, 10); } +{number} { return value(yyscanner); } {symbol} { return str(yyscanner, ID); } "|" { return '|'; } "^" { return '^'; }Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Re: [PATCH v2 06/11] perf expr: parse numbers as doubles
In reply to Ian RogersQuoted 45 lines · Ian Rogers
This is expected in expr.y and metrics use floating point values such as x86 broadwell
IFetch_Line_Utilization.Fixes 26226a97724d (perf expr: Move expr lexer to flex)Signed-off-by Ian Rogers <[email protected]>--- tools/perf/util/expr.l | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)tools/perf/util/expr.l +5
-5@@ -10,12 +10,12 @@ char *expr_get_text(yyscan_t yyscanner); YYSTYPE *expr_get_lval(yyscan_t yyscanner); -static int __value(YYSTYPE *yylval, char *str, int base, int token) +static double __value(YYSTYPE *yylval, char *str, int token) { - u64 num; + double num; errno = 0; - num = strtoull(str, NULL, base); + num = strtod(str, NULL); if (errno) return EXPR_ERROR;@@ -23,12 +23,12 @@ static int __value(YYSTYPE *yylval, char *str, int base, int token) return token; } -static int value(yyscan_t scanner, int base) +static int value(yyscan_t scanner) { YYSTYPE *yylval = expr_get_lval(scanner); char *text = expr_get_text(scanner); - return __value(yylval, text, base, NUMBER); + return __value(yylval, text, NUMBER); } /*
tools/perf/util/expr.l static int str(yyscan_t scanner, int token) · line 72
Quoted 6 lines
tools/perf/util/expr.l +1
-1@@ -68,7 +68,7 @@ static int str(yyscan_t scanner, int token) } %} -number [0-9]+ +number [0-9]*\.?[0-9]+
Hi Ian,
In this patch I saw the parsing of expression with '+number [0-9]*\.?[0-9]+'
Could you please explain why '?' is introduced here, so that I can be sure that this is not conflicting with my change to add '?'
In this patch : https://lkml.org/lkml/2020/4/1/1427
I have used '?' symbol as part of metric expression in order to replace '?' with runtime parameter.
Thanks,
Kajol Jain
Quoted 12 lines
tools/perf/util/expr.l +1
-1sch [-,=] spec \\{sch}@@ -92,7 +92,7 @@ min { return MIN; } if { return IF; } else { return ELSE; } #smt_on { return SMT_ON; } -{number} { return value(yyscanner, 10); } +{number} { return value(yyscanner); } {symbol} { return str(yyscanner, ID); } "|" { return '|'; } "^" { return '^'; }
Ian Rogers
reviewerRe: [PATCH v2 06/11] perf expr: parse numbers as doubles
Quoted 50 lines · kajoljain
Quoted 48 lines · Ian Rogers
This is expected in expr.y and metrics use floating point values such as x86 broadwell
IFetch_Line_Utilization.Fixes 26226a97724d (perf expr: Move expr lexer to flex)Signed-off-by Ian Rogers <[email protected]>--- tools/perf/util/expr.l | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)tools/perf/util/expr.l +5
-5@@ -10,12 +10,12 @@ char *expr_get_text(yyscan_t yyscanner); YYSTYPE *expr_get_lval(yyscan_t yyscanner); -static int __value(YYSTYPE *yylval, char *str, int base, int token) +static double __value(YYSTYPE *yylval, char *str, int token) { - u64 num; + double num; errno = 0; - num = strtoull(str, NULL, base); + num = strtod(str, NULL); if (errno) return EXPR_ERROR;@@ -23,12 +23,12 @@ static int __value(YYSTYPE *yylval, char *str, int base, int token) return token; } -static int value(yyscan_t scanner, int base) +static int value(yyscan_t scanner) { YYSTYPE *yylval = expr_get_lval(scanner); char *text = expr_get_text(scanner); - return __value(yylval, text, base, NUMBER); + return __value(yylval, text, NUMBER); } /*@@ -68,7 +68,7 @@ static int str(yyscan_t scanner, int token) } %}
Quoted 8 lines
tools/perf/util/expr.l static int str(yyscan_t scanner, int token) · line 72
Quoted 3 lines
tools/perf/util/expr.l +1
-1-number [0-9]+ +number [0-9]*\.?[0-9]+Hi Ian,
In this patch I saw the parsing of expression with '+number [0-9]*\.?[0-9]+'
Could you please explain why '?' is introduced here, so that I can be sure that this is not conflicting with my change to add '?'
Hi Kajol,
the '?' here is part of the regular expression. Basically it is saying that a number is a possible set of integers possibly followed by a '.' and then 1 or more integers. The expression comes from having seen Intel's metrics are of the form '.1234' in some of their topdown spreadsheets, so we need to be able to handle cases like '1.2', '123' and '.1234'. Having looked at your patch I don't believe it interferes.
Quoted 3 lines
In this patch : https://lkml.org/lkml/2020/4/1/1427
I have used '?' symbol as part of metric expression in order to replace '?' with runtime parameter.
Interesting, I'll follow up with comments on that e-mail.
Thanks!
Ian
Quoted 16 lines
Thanks,
Kajol JainQuoted 12 lines
tools/perf/util/expr.l +1
-1sch [-,=] spec \\{sch}@@ -92,7 +92,7 @@ min { return MIN; } if { return IF; } else { return ELSE; } #smt_on { return SMT_ON; } -{number} { return value(yyscanner, 10); } +{number} { return value(yyscanner); } {symbol} { return str(yyscanner, ID); } "|" { return '|'; } "^" { return '^'; }
Re: [PATCH v2 06/11] perf expr: parse numbers as doubles
In reply to Ian RogersQuoted 45 lines · Ian Rogers
This is expected in expr.y and metrics use floating point values such as x86 broadwell
IFetch_Line_Utilization.Fixes 26226a97724d (perf expr: Move expr lexer to flex)Signed-off-by Ian Rogers <[email protected]>--- tools/perf/util/expr.l | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)tools/perf/util/expr.l +5
-5@@ -10,12 +10,12 @@ char *expr_get_text(yyscan_t yyscanner); YYSTYPE *expr_get_lval(yyscan_t yyscanner); -static int __value(YYSTYPE *yylval, char *str, int base, int token) +static double __value(YYSTYPE *yylval, char *str, int token) { - u64 num; + double num; errno = 0; - num = strtoull(str, NULL, base); + num = strtod(str, NULL); if (errno) return EXPR_ERROR;@@ -23,12 +23,12 @@ static int __value(YYSTYPE *yylval, char *str, int base, int token) return token; } -static int value(yyscan_t scanner, int base) +static int value(yyscan_t scanner) { YYSTYPE *yylval = expr_get_lval(scanner); char *text = expr_get_text(scanner); - return __value(yylval, text, base, NUMBER); + return __value(yylval, text, NUMBER); } /*
tools/perf/util/expr.l static int str(yyscan_t scanner, int token) · line 72
Quoted 6 lines
tools/perf/util/expr.l +1
-1@@ -68,7 +68,7 @@ static int str(yyscan_t scanner, int token) } %} -number [0-9]+ +number [0-9]*\.?[0-9]+
Acked By: Kajol Jain <[email protected]>
Thanks,
Kajol Jain
Quoted 12 lines
tools/perf/util/expr.l +1
-1sch [-,=] spec \\{sch}@@ -92,7 +92,7 @@ min { return MIN; } if { return IF; } else { return ELSE; } #smt_on { return SMT_ON; } -{number} { return value(yyscanner, 10); } +{number} { return value(yyscanner); } {symbol} { return str(yyscanner, ID); } "|" { return '|'; } "^" { return '^'; }
Ian Rogers
reviewerPATCH v2 7/11 perf expr: debug lex if debugging yacc
In reply to Ian RogersOnly effects parser debugging (disabled by default). Enables displaying '--accepting rule at line .. ("...").
--- tools/perf/util/expr.c | 1 + 1 file changed, 1 insertion(+)
tools/perf/util/expr.c +1 -0
@@ -44,6 +44,7 @@ __expr__parse(double *val, struct expr_parse_ctx *ctx, const char *expr,#ifdef PARSER_DEBUG expr_debug =1 ; +expr_set_debug (1 , scanner);#endif ret =expr_parse (val, ctx, scanner);
Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Ian Rogers
reviewerPATCH v2 8/11 perf metrics: fix parse errors in power8 metrics
In reply to Ian RogersMismatched parentheses.
--- tools/perf/pmu-events/arch/powerpc/power8/metrics.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/pmu-events/arch/powerpc/power8/metrics.json +2 -2
@@ -169,7 +169,7 @@ }, { "BriefDescription": "Cycles GCT empty where dispatch was held", - "MetricExpr": "(PM_GCT_NOSLOT_DISP_HELD_MAP + PM_GCT_NOSLOT_DISP_HELD_SRQ + PM_GCT_NOSLOT_DISP_HELD_ISSQ + PM_GCT_NOSLOT_DISP_HELD_OTHER) / PM_RUN_INST_CMPL)", + "MetricExpr": "(PM_GCT_NOSLOT_DISP_HELD_MAP + PM_GCT_NOSLOT_DISP_HELD_SRQ + PM_GCT_NOSLOT_DISP_HELD_ISSQ + PM_GCT_NOSLOT_DISP_HELD_OTHER) / PM_RUN_INST_CMPL", "MetricGroup": "cpi_breakdown", "MetricName": "gct_empty_disp_held_cpi" },
@@ -886,7 +886,7 @@ }, { "BriefDescription": "GCT slot utilization (11 to 14) as a % of cycles this thread had atleast 1 slot valid", - "MetricExpr": "PM_GCT_UTIL_11_14_ENTRIES / ( PM_RUN_CYC - PM_GCT_NOSLOT_CYC) * 100", + "MetricExpr": "PM_GCT_UTIL_11_14_ENTRIES / ( PM_RUN_CYC - PM_GCT_NOSLOT_CYC ) * 100", "MetricGroup": "general", "MetricName": "gct_util_11to14_slots_percent" },
Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Re: [PATCH v2 08/11] perf metrics: fix parse errors in power8 metrics
Quoted 13 lines · Ian Rogers
Mismatched parentheses.
Fixes dd81eafacc52 (perf vendor events power8: Cpi_breakdown & estimated_dcache_miss_cpi metrics)Signed-off-by Ian Rogers <[email protected]>--- tools/perf/pmu-events/arch/powerpc/power8/metrics.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)tools/perf/pmu-events/arch/powerpc/power8/metrics.json +0
-0
tools/perf/pmu-events/arch/powerpc/power8/metrics.json line 172
Quoted 5 lines
tools/perf/pmu-events/arch/powerpc/power8/metrics.json +1
-1@@ -169,7 +169,7 @@ }, { "BriefDescription": "Cycles GCT empty where dispatch was held", - "MetricExpr": "(PM_GCT_NOSLOT_DISP_HELD_MAP + PM_GCT_NOSLOT_DISP_HELD_SRQ + PM_GCT_NOSLOT_DISP_HELD_ISSQ + PM_GCT_NOSLOT_DISP_HELD_OTHER) / PM_RUN_INST_CMPL)", + "MetricExpr": "(PM_GCT_NOSLOT_DISP_HELD_MAP + PM_GCT_NOSLOT_DISP_HELD_SRQ + PM_GCT_NOSLOT_DISP_HELD_ISSQ + PM_GCT_NOSLOT_DISP_HELD_OTHER) / PM_RUN_INST_CMPL",
OK. (Thank you!)
tools/perf/pmu-events/arch/powerpc/power8/metrics.json line 889
Quoted 9 lines
tools/perf/pmu-events/arch/powerpc/power8/metrics.json +1
-1"MetricGroup": "cpi_breakdown", "MetricName": "gct_empty_disp_held_cpi" },@@ -886,7 +886,7 @@ }, { "BriefDescription": "GCT slot utilization (11 to 14) as a % of cycles this thread had atleast 1 slot valid", - "MetricExpr": "PM_GCT_UTIL_11_14_ENTRIES / ( PM_RUN_CYC - PM_GCT_NOSLOT_CYC) * 100", + "MetricExpr": "PM_GCT_UTIL_11_14_ENTRIES / ( PM_RUN_CYC - PM_GCT_NOSLOT_CYC ) * 100",
I think this is just a whitespace change? Is it necessary? Curiosity, more than complaint.
tools/perf/pmu-events/arch/powerpc/power8/metrics.json line 892
Quoted 3 lines
tools/perf/pmu-events/arch/powerpc/power8/metrics.json +0
-0"MetricGroup": "general", "MetricName": "gct_util_11to14_slots_percent" },
PC
Ian Rogers
reviewerRe: [PATCH v2 08/11] perf metrics: fix parse errors in power8 metrics
Quoted 23 lines · Paul Clarke
Quoted 13 lines · Ian Rogers
Mismatched parentheses.
Fixes dd81eafacc52 (perf vendor events power8: Cpi_breakdown & estimated_dcache_miss_cpi metrics)Signed-off-by Ian Rogers <[email protected]>--- tools/perf/pmu-events/arch/powerpc/power8/metrics.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)tools/perf/pmu-events/arch/powerpc/power8/metrics.json +0
-0tools/perf/pmu-events/arch/powerpc/power8/metrics.json line 172
Quoted 5 lines
tools/perf/pmu-events/arch/powerpc/power8/metrics.json +1
-1@@ -169,7 +169,7 @@ }, { "BriefDescription": "Cycles GCT empty where dispatch was held", - "MetricExpr": "(PM_GCT_NOSLOT_DISP_HELD_MAP + PM_GCT_NOSLOT_DISP_HELD_SRQ + PM_GCT_NOSLOT_DISP_HELD_ISSQ + PM_GCT_NOSLOT_DISP_HELD_OTHER) / PM_RUN_INST_CMPL)", + "MetricExpr": "(PM_GCT_NOSLOT_DISP_HELD_MAP + PM_GCT_NOSLOT_DISP_HELD_SRQ + PM_GCT_NOSLOT_DISP_HELD_ISSQ + PM_GCT_NOSLOT_DISP_HELD_OTHER) / PM_RUN_INST_CMPL",OK. (Thank you!)
Quoted 12 lines
tools/perf/pmu-events/arch/powerpc/power8/metrics.json line 889
Quoted 9 lines
tools/perf/pmu-events/arch/powerpc/power8/metrics.json +1
-1"MetricGroup": "cpi_breakdown", "MetricName": "gct_empty_disp_held_cpi" },@@ -886,7 +886,7 @@ }, { "BriefDescription": "GCT slot utilization (11 to 14) as a % of cycles this thread had atleast 1 slot valid", - "MetricExpr": "PM_GCT_UTIL_11_14_ENTRIES / ( PM_RUN_CYC - PM_GCT_NOSLOT_CYC) * 100", + "MetricExpr": "PM_GCT_UTIL_11_14_ENTRIES / ( PM_RUN_CYC - PM_GCT_NOSLOT_CYC ) * 100",I think this is just a whitespace change? Is it necessary? Curiosity, more than complaint.
Sorry about that, the space isn't necessary and this doesn't need to change. For the curious, originally the parse test would make all metrics equal to 1.0 and this metric would trigger a divide by zero because of this. This motivated adding a debug print for this case.
Thanks,
Ian
Quoted 7 lines
tools/perf/pmu-events/arch/powerpc/power8/metrics.json line 892
Quoted 3 lines
tools/perf/pmu-events/arch/powerpc/power8/metrics.json +0
-0"MetricGroup": "general", "MetricName": "gct_util_11to14_slots_percent" },Reviewed-by Paul A. Clarke <[email protected]>PC
Ian Rogers
reviewerPATCH v2 9/11 perf metrics: fix parse errors in power9 metrics
In reply to Ian RogersMismatched parentheses.
--- tools/perf/pmu-events/arch/powerpc/power9/metrics.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/pmu-events/arch/powerpc/power9/metrics.json +1 -1
@@ -362,7 +362,7 @@ }, { "BriefDescription": "Completion stall for other reasons", - "MetricExpr": "PM_CMPLU_STALL - PM_CMPLU_STALL_NTC_DISP_FIN - PM_CMPLU_STALL_NTC_FLUSH - PM_CMPLU_STALL_LSU - PM_CMPLU_STALL_EXEC_UNIT - PM_CMPLU_STALL_BRU)/PM_RUN_INST_CMPL", + "MetricExpr": "(PM_CMPLU_STALL - PM_CMPLU_STALL_NTC_DISP_FIN - PM_CMPLU_STALL_NTC_FLUSH - PM_CMPLU_STALL_LSU - PM_CMPLU_STALL_EXEC_UNIT - PM_CMPLU_STALL_BRU)/PM_RUN_INST_CMPL", "MetricGroup": "cpi_breakdown", "MetricName": "other_stall_cpi" },
Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Re: [PATCH v2 09/11] perf metrics: fix parse errors in power9 metrics
Quoted 13 lines · Ian Rogers
Mismatched parentheses.
Fixes 7f3cf5ac7743 (perf vendor events power9: Cpi_breakdown & estimated_dcache_miss_cpi metrics)Signed-off-by Ian Rogers <[email protected]>--- tools/perf/pmu-events/arch/powerpc/power9/metrics.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)tools/perf/pmu-events/arch/powerpc/power9/metrics.json +0
-0
tools/perf/pmu-events/arch/powerpc/power9/metrics.json line 365
Quoted 5 lines
tools/perf/pmu-events/arch/powerpc/power9/metrics.json +1
-1@@ -362,7 +362,7 @@ }, { "BriefDescription": "Completion stall for other reasons", - "MetricExpr": "PM_CMPLU_STALL - PM_CMPLU_STALL_NTC_DISP_FIN - PM_CMPLU_STALL_NTC_FLUSH - PM_CMPLU_STALL_LSU - PM_CMPLU_STALL_EXEC_UNIT - PM_CMPLU_STALL_BRU)/PM_RUN_INST_CMPL", + "MetricExpr": "(PM_CMPLU_STALL - PM_CMPLU_STALL_NTC_DISP_FIN - PM_CMPLU_STALL_NTC_FLUSH - PM_CMPLU_STALL_LSU - PM_CMPLU_STALL_EXEC_UNIT - PM_CMPLU_STALL_BRU)/PM_RUN_INST_CMPL",
OK. (Thank you!!)
tools/perf/pmu-events/arch/powerpc/power9/metrics.json line 368
Quoted 3 lines
tools/perf/pmu-events/arch/powerpc/power9/metrics.json +0
-0"MetricGroup": "cpi_breakdown", "MetricName": "other_stall_cpi" },
(I like how you got the "power8" changes to be 8th in the series and the "power9" changes to be 9th ;-)
PC
Ian Rogers
reviewerPATCH v2 10/11 perf expr: print a debug message for division by zero
In reply to Ian RogersIf an expression yields 0 and is then divided-by/modulus-by then the parsing aborts. Add a debug error message to better enable debugging when this happens.
--- tools/perf/util/expr.y | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)
tools/perf/util/expr.y +12 -2
@@ -103,8 +103,18 @@ expr: NUMBER | expr '+' expr { $$ = $1 + $3; } | expr '-' expr { $$ = $1 - $3; } | expr '*' expr { $$ = $1 * $3; } - | expr '/' expr { if ($3 == 0) YYABORT; $$ = $1 / $3; } - | expr '%' expr { if ((long)$3 == 0) YYABORT; $$ = (long)$1 % (long)$3; } + | expr '/' expr { if ($3 == 0) { + pr_debug("division by zero\n"); + YYABORT; + } + $$ = $1 / $3; + } + | expr '%' expr { if ((long)$3 == 0) { + pr_debug("division by zero\n"); + YYABORT; + } + $$ = (long)$1 % (long)$3; + } | '-' expr %prec NEG { $$ = -$2; } | '(' if_expr ')' { $$ = $2; } | MIN '(' expr ',' expr ')' { $$ = $3 < $5 ? $3 : $5; }
Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Re: [PATCH v2 10/11] perf expr: print a debug message for division by zero
Quoted 16 lines · Ian Rogers
If an expression yields 0 and is then divided-by/modulus-by then the parsing aborts. Add a debug error message to better enable debugging when this happens.
Signed-off-by Ian Rogers <[email protected]>--- tools/perf/util/expr.y | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)tools/perf/util/expr.y +0
-0@@ -103,8 +103,18 @@ expr: NUMBER | expr '+' expr { $$ = $1 + $3; } | expr '-' expr { $$ = $1 - $3; }
tools/perf/util/expr.y expr: NUMBER · line 112
Quoted 10 lines
tools/perf/util/expr.y +7
-2| expr '*' expr { $$ = $1 * $3; } - | expr '/' expr { if ($3 == 0) YYABORT; $$ = $1 / $3; } - | expr '%' expr { if ((long)$3 == 0) YYABORT; $$ = (long)$1 % (long)$3; } + | expr '/' expr { if ($3 == 0) { + pr_debug("division by zero\n"); + YYABORT; + } + $$ = $1 / $3; + } + | expr '%' expr { if ((long)$3 == 0) {
is that (long) cast necessary? it's missing for the '/' case
jirka
Quoted 11 lines
tools/perf/util/expr.y +5
-0+ pr_debug("division by zero\n"); + YYABORT; + } + $$ = (long)$1 % (long)$3; + } | '-' expr %prec NEG { $$ = -$2; } | '(' if_expr ')' { $$ = $2; } | MIN '(' expr ',' expr ')' { $$ = $3 < $5 ? $3 : $5; }Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Ian Rogers
reviewerRe: [PATCH v2 10/11] perf expr: print a debug message for division by zero
Quoted 22 lines · Jiri Olsa
Quoted 20 lines · Ian Rogers
If an expression yields 0 and is then divided-by/modulus-by then the parsing aborts. Add a debug error message to better enable debugging when this happens.
Signed-off-by Ian Rogers <[email protected]>--- tools/perf/util/expr.y | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)tools/perf/util/expr.y +1
-2@@ -103,8 +103,18 @@ expr: NUMBER | expr '+' expr { $$ = $1 + $3; } | expr '-' expr { $$ = $1 - $3; } | expr '*' expr { $$ = $1 * $3; } - | expr '/' expr { if ($3 == 0) YYABORT; $$ = $1 / $3; } - | expr '%' expr { if ((long)$3 == 0) YYABORT; $$ = (long)$1 % (long)$3; } + | expr '/' expr { if ($3 == 0) {
Quoted 8 lines
tools/perf/util/expr.y expr: NUMBER · line 112
Quoted 6 lines
tools/perf/util/expr.y +6
-0+ pr_debug("division by zero\n"); + YYABORT; + } + $$ = $1 / $3; + } + | expr '%' expr { if ((long)$3 == 0) {is that (long) cast necessary? it's missing for the '/' case
Andi Kleen could say for sure :-) From my observation, the values are typically doubles and definitely need to be in the divide case. There is also code handling them as longs such as |, &, ^:
https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git/tree/tools/perf/util/expr.y#n100
There's an experiment to see if all of the long handling code could be removed. This change isn't modifying the existing behavior.
Thanks,
Ian
Quoted 14 lines
jirka
Quoted 11 lines
tools/perf/util/expr.y +5
-1+ pr_debug("division by zero\n"); + YYABORT; + } + $$ = (long)$1 % (long)$3; + } | '-' expr %prec NEG { $$ = -$2; } | '(' if_expr ')' { $$ = $2; } | MIN '(' expr ',' expr ')' { $$ = $3 < $5 ? $3 : $5; } --2.26.2.303.gf8c07b1a785-goog
Ian Rogers
reviewerPATCH v2 11/11 perf test: add expr test for pmu metrics
In reply to Ian RogersAdd basic floating point number test.
Verify that all pmu metrics, for the current architecture, parse.
--- tools/perf/tests/builtin-test.c | 5 ++ tools/perf/tests/expr.c | 96 ++++++++++++++++++++++++++++++++- tools/perf/tests/tests.h | 2 + 3 files changed, 102 insertions(+), 1 deletion(-)
tools/perf/tests/builtin-test.c +5 -0
@@ -63,6 +63,11 @@ static struct test generic_tests[] = { { .desc = "Simple expression parser", .func = test__expr, + .subtest = { + .get_nr = test__expr_subtest_get_nr, + .get_desc = test__expr_subtest_get_desc, + }, + }, { .desc = "PERF_RECORD_* events & perf_sample fields",
tools/perf/tests/expr.c +95 -1
120 lines not on this page · raw message
tools/perf/tests/tests.h +2 -0
@@ -72,6 +72,8 @@ int test__keep_tracking(struct test *test, int subtest);int test__parse_no_sample_id_all (struct test *test,int subtest);int test__dwarf_unwind (struct test *test,int subtest);int test__expr (struct test *test,int subtest); +const char *test__expr_subtest_get_desc (int subtest); +int test__expr_subtest_get_nr (void );int test__hists_filter (struct test *test,int subtest);int test__mmap_thread_lookup (struct test *test,int subtest);int test__thread_maps_share (struct test *test,int subtest);
Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Re: [PATCH v2 11/11] perf test: add expr test for pmu metrics
On Wed, Apr 22, 2020 at 03:04:30PM -0700, Ian Rogers wrote:
SNIP
Quoted 16 lines
tools/perf/tests/expr.c +16
-0+ +static int pmu_tests (void ) +{ +struct pmu_events_map *map; +struct pmu_event *pe; +int i, j, k; +const char **ids; +int idnum; +int ret =0 ; +struct expr_parse_ctx ctx; +double result; + + i =0 ; +for (;;) { + map = &pmu_events_map[i++]; +if (!map->table) {
tools/perf/tests/expr.c int test__expr(struct test *t __maybe_unused, int subtest __maybe_unused) · line 96
Quoted 10 lines
tools/perf/tests/expr.c +10
-0+ map =NULL ; +break ; + } + j =0 ; +for (;;) { + pe = &map->table[j++]; +if (!pe->name && !pe->metric_group && !pe->metric_name) +break ; +if (!pe->metric_expr) +continue ;
so we go throught all the metrics for the current cpu and test the parsing on them.. great!
thanks,
jirka
Ian Rogers
reviewerRe: [PATCH v2 11/11] perf test: add expr test for pmu metrics
Quoted 26 lines · Jiri Olsa
On Wed, Apr 22, 2020 at 03:04:30PM -0700, Ian Rogers wrote:
SNIP
Quoted 21 lines
tools/perf/tests/expr.c +21
-0+ +static int pmu_tests (void ) +{ +struct pmu_events_map *map; +struct pmu_event *pe; +int i, j, k; +const char **ids; +int idnum; +int ret =0 ; +struct expr_parse_ctx ctx; +double result; + + i =0 ; +for (;;) { + map = &pmu_events_map[i++]; +if (!map->table) { + map =NULL ; +break ; + } + j =0 ; +for (;;) {
Quoted 8 lines
tools/perf/tests/expr.c int test__expr(struct test *t __maybe_unused, int subtest __maybe_unused) · line 96
Quoted 5 lines
tools/perf/tests/expr.c +5
-0+ pe = &map->table[j++]; +if (!pe->name && !pe->metric_group && !pe->metric_name) +break ; +if (!pe->metric_expr) +continue ;so we go throught all the metrics for the current cpu and test the parsing on them.. great!
It's not just the current CPU (such as skylake) it is every map (skylake, cascade lake, etc), but this only works for the architecture that jevents built.
If jevents built all architectures then this could check them as well. Perhaps there should be a jevents test suite, but I think even then this test has value.
A worthy addition to this is checking that the events within the expression parse, but this is good progress and worth landing.
Thanks,
Ian
Quoted 3 lines
thanks,
jirka
Re: [PATCH v2 11/11] perf test: add expr test for pmu metrics
Quoted 36 lines · Ian Rogers
Quoted 26 lines · Jiri Olsa
On Wed, Apr 22, 2020 at 03:04:30PM -0700, Ian Rogers wrote:
SNIP
Quoted 21 lines
tools/perf/tests/expr.c +21
-0+ +static int pmu_tests (void ) +{ +struct pmu_events_map *map; +struct pmu_event *pe; +int i, j, k; +const char **ids; +int idnum; +int ret =0 ; +struct expr_parse_ctx ctx; +double result; + + i =0 ; +for (;;) { + map = &pmu_events_map[i++]; +if (!map->table) { + map =NULL ; +break ; + } + j =0 ; +for (;;) {Quoted 8 lines
tools/perf/tests/expr.c int test__expr(struct test *t __maybe_unused, int subtest __maybe_unused) · line 96
Quoted 5 lines
tools/perf/tests/expr.c +5
-0+ pe = &map->table[j++]; +if (!pe->name && !pe->metric_group && !pe->metric_name) +break ; +if (!pe->metric_expr) +continue ;so we go throught all the metrics for the current cpu and test the parsing on them.. great!
Quoted 6 lines
It's not just the current CPU (such as skylake) it is every map (skylake, cascade lake, etc), but this only works for the architecture that jevents built.
If jevents built all architectures then this could check them as well. Perhaps there should be a jevents test suite, but I think even then this test has value.
note: there is test__pmu_events(), which verifies that some test events generated in pmu-events.c are as expected, and also verifies that we create PMU events aliases as expected (for those test events). Nothing is done for metrics, ATM.
Thanks,
John
Ian Rogers
reviewerRe: [PATCH v2 11/11] perf test: add expr test for pmu metrics
Quoted 45 lines · John Garry
Quoted 36 lines · Ian Rogers
Quoted 26 lines · Jiri Olsa
On Wed, Apr 22, 2020 at 03:04:30PM -0700, Ian Rogers wrote:
SNIP
Quoted 21 lines
tools/perf/tests/expr.c +21
-0+ +static int pmu_tests (void ) +{ +struct pmu_events_map *map; +struct pmu_event *pe; +int i, j, k; +const char **ids; +int idnum; +int ret =0 ; +struct expr_parse_ctx ctx; +double result; + + i =0 ; +for (;;) { + map = &pmu_events_map[i++]; +if (!map->table) { + map =NULL ; +break ; + } + j =0 ; +for (;;) {Quoted 8 lines
tools/perf/tests/expr.c int test__expr(struct test *t __maybe_unused, int subtest __maybe_unused) · line 96
Quoted 5 lines
tools/perf/tests/expr.c +5
-0+ pe = &map->table[j++]; +if (!pe->name && !pe->metric_group && !pe->metric_name) +break ; +if (!pe->metric_expr) +continue ;so we go throught all the metrics for the current cpu and test the parsing on them.. great!
Quoted 6 lines
It's not just the current CPU (such as skylake) it is every map (skylake, cascade lake, etc), but this only works for the architecture that jevents built.
If jevents built all architectures then this could check them as well. Perhaps there should be a jevents test suite, but I think even then this test has value.
Quoted 4 lines
note: there is
test__pmu_events(), which verifies that some test events generated inpmu-events.care as expected, and also verifies that we create PMU events aliases as expected (for those test events). Nothing is done for metrics, ATM.
Thanks John, that sounds like the right place to start.
Ian
Quoted 2 lines
Thanks,
John
Quoted 8 lines · Ian Rogers
Add a test that all perf metrics (for your architecture) are parsable. Fix bugs in the expr parser and in x86 metrics. Untested on architectures other than x86.
v2 adds Fixes tags to commit messages for when broken metrics were
first added. Adds a debug warning for division by zero in expr, and adds a workaround for id values in the expr test necessary for powerpc. It also fixes broken power8 and power9 metrics.
looks good to me
Jin Yao, is there a metric that's not working for you with this patchset applied?
thanks,
jirka
Quoted 30 lines
Ian Rogers (11):
perf expr: unlimited escaped characters in a symbol perf metrics: fix parse errors in cascade lake metrics perf metrics: fix parse errors in skylake metrics perf expr: allow ',' to be an other token perf expr: increase max other
perf expr: parse numbers as doubles
perf expr: debug lex if debugging yacc
perf metrics: fix parse errors in power8 metrics perf metrics: fix parse errors in power9 metrics perf expr: print a debug message for division by zero perf test: add expr test for pmu metrics.../arch/powerpc/power8/metrics.json | 4 +- .../arch/powerpc/power9/metrics.json | 2 +- .../arch/x86/cascadelakex/clx-metrics.json | 10 +- .../arch/x86/skylakex/skx-metrics.json | 4 +- tools/perf/tests/builtin-test.c | 5 + tools/perf/tests/expr.c | 96 ++++++++++++++++++- tools/perf/tests/tests.h | 2 + tools/perf/util/expr.c | 1 + tools/perf/util/expr.h | 2 +- tools/perf/util/expr.l | 16 ++-- tools/perf/util/expr.y | 16 +++- 11 files changed, 135 insertions(+), 23 deletions(-)Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Hi Jiri,
Quoted 10 lines · Jiri Olsa
Quoted 8 lines · Ian Rogers
Add a test that all perf metrics (for your architecture) are parsable. Fix bugs in the expr parser and in x86 metrics. Untested on architectures other than x86.
v2 adds Fixes tags to commit messages for when broken metrics were
first added. Adds a debug warning for division by zero in expr, and adds a workaround for id values in the expr test necessary for powerpc. It also fixes broken power8 and power9 metrics.
Quoted 8 lines
looks good to me
Jin Yao, is there a metric that's not working for you with this patchset applied?
thanks,
jirka
Let me look for a CLX for testing, but maybe need some time.
BTW, suppose this patchset can work well, does it mean we will change the json file format in future?
For example,
before:
cha@event\\=0x36\\\\\\
after:
cha@event\\=0x36\\
"\\\\" are removed.
If so, we need to change our event generation script.
Thanks
Jin Yao
Quoted 31 lines
Quoted 30 lines
Ian Rogers (11):
perf expr: unlimited escaped characters in a symbol perf metrics: fix parse errors in cascade lake metrics perf metrics: fix parse errors in skylake metrics perf expr: allow ',' to be an other token perf expr: increase max other
perf expr: parse numbers as doubles
perf expr: debug lex if debugging yacc
perf metrics: fix parse errors in power8 metrics perf metrics: fix parse errors in power9 metrics perf expr: print a debug message for division by zero perf test: add expr test for pmu metrics.../arch/powerpc/power8/metrics.json | 4 +- .../arch/powerpc/power9/metrics.json | 2 +- .../arch/x86/cascadelakex/clx-metrics.json | 10 +- .../arch/x86/skylakex/skx-metrics.json | 4 +- tools/perf/tests/builtin-test.c | 5 + tools/perf/tests/expr.c | 96 ++++++++++++++++++- tools/perf/tests/tests.h | 2 + tools/perf/util/expr.c | 1 + tools/perf/util/expr.h | 2 +- tools/perf/util/expr.l | 16 ++-- tools/perf/util/expr.y | 16 +++- 11 files changed, 135 insertions(+), 23 deletions(-)Signature · 1 line
2.26.2.303.gf8c07b1a785-goog
Quoted 32 lines · Yao
Hi Jiri,
Quoted 10 lines · Jiri Olsa
Quoted 8 lines · Ian Rogers
Add a test that all perf metrics (for your architecture) are parsable. Fix bugs in the expr parser and in x86 metrics. Untested on architectures other than x86.
v2 adds Fixes tags to commit messages for when broken metrics were
first added. Adds a debug warning for division by zero in expr, and adds a workaround for id values in the expr test necessary for powerpc. It also fixes broken power8 and power9 metrics.Quoted 8 lines
looks good to me
Jin Yao, is there a metric that's not working for you with this patchset applied?
thanks,
jirkaLet me look for a CLX for testing, but maybe need some time.
BTW, suppose this patchset can work well, does it mean we will change the json file format in future?
For example,
before:
cha@event\\=0x36\\\\\\
Quoted 6 lines
after:
cha@event\\=0x36\\"\\\\" are removed.
If so, we need to change our event generation script.
ok, maybe I got the wrong idea that the extra \\\\ were just superfluous, what was the actual error there? and what's the reason for that many '\' in there?
jirka
Ian Rogers
reviewerQuoted 37 lines · Jiri Olsa
Quoted 35 lines · Yao
Hi Jiri,
Quoted 10 lines · Jiri Olsa
Quoted 8 lines · Ian Rogers
Add a test that all perf metrics (for your architecture) are parsable. Fix bugs in the expr parser and in x86 metrics. Untested on architectures other than x86.
v2 adds Fixes tags to commit messages for when broken metrics were
first added. Adds a debug warning for division by zero in expr, and adds a workaround for id values in the expr test necessary for powerpc. It also fixes broken power8 and power9 metrics.Quoted 8 lines
looks good to me
Jin Yao, is there a metric that's not working for you with this patchset applied?
thanks,
jirkaLet me look for a CLX for testing, but maybe need some time.
BTW, suppose this patchset can work well, does it mean we will change the json file format in future?
For example,
before:
cha@event\\=0x36\\\\\\after:
cha@event\\=0x36\\
Quoted 7 lines
Quoted 3 lines
"\\\\" are removed.
If so, we need to change our event generation script.
ok, maybe I got the wrong idea that the extra \\\\ were just superfluous, what was the actual error there? and what's the reason for that many '\' in there?
I believe they are superfluous and break even before the flex change. I commented on it here with a reproduction of a parse events error on skylake:
https://lore.kernel.org/lkml/CAP-5=fUnWAycQehCJ9=btquV2c3DVDX+tTEc85H8py9Kfehq4w@mail.gmail.com/
Fixing the script that generates this would be great! With the test landed it should be much harder for this to be broken in the future.
Thanks,
Ian
Quoted 2 lines
jirka
Quoted 51 lines · Ian Rogers
Quoted 37 lines · Jiri Olsa
Quoted 35 lines · Yao
Hi Jiri,
Quoted 10 lines · Jiri Olsa
Quoted 8 lines · Ian Rogers
Add a test that all perf metrics (for your architecture) are parsable. Fix bugs in the expr parser and in x86 metrics. Untested on architectures other than x86.
v2 adds Fixes tags to commit messages for when broken metrics were
first added. Adds a debug warning for division by zero in expr, and adds a workaround for id values in the expr test necessary for powerpc. It also fixes broken power8 and power9 metrics.Quoted 8 lines
looks good to me
Jin Yao, is there a metric that's not working for you with this patchset applied?
thanks,
jirkaLet me look for a CLX for testing, but maybe need some time.
BTW, suppose this patchset can work well, does it mean we will change the json file format in future?
For example,
before:
cha@event\\=0x36\\\\\\after:
cha@event\\=0x36\\Quoted 7 lines
Quoted 3 lines
"\\\\" are removed.
If so, we need to change our event generation script.
ok, maybe I got the wrong idea that the extra \\\\ were just superfluous, what was the actual error there? and what's the reason for that many '\' in there?
I believe they are superfluous and break even before the flex change. I commented on it here with a reproduction of a parse events error on skylake:
https://lore.kernel.org/lkml/CAP-5=fUnWAycQehCJ9=btquV2c3DVDX+tTEc85H8py9Kfehq4w@mail.gmail.com/
Quoted 6 lines
Fixing the script that generates this would be great! With the test landed it should be much harder for this to be broken in the future.
Thanks,
Ian
Tested on CLX and the error disappeared.
BTW, I will post the latest version of event list once the testing is completed.
Thanks
Jin Yao
Quoted 3 lines
Quoted 3 lines
jirka