Copy hashmap to tools/perf/util, use in perf expr
Subsystems (5)
Ian Rogers
reviewerv2 cover Copy hashmap to tools/perf/util, use in perf expr
Perf's expr code currently builds an array of strings then removes duplicates. The array is larger than necessary and has recently been increased in size. When this was done it was commented that a hashmap would be preferable.
libbpf has a hashmap but libbpf isn't currently required to build perf. To satisfy various concerns this change copies libbpf's hashmap into tools/perf/util, it then adds a check in perf that the two are in sync.
Andrii's patch to hashmap from bpf-next is brought into this set to fix issues with hashmap__clear.
Two minor changes to libbpf's hashmap are made that remove an unused dependency and fix a compiler warning.
Two perf test changes are also brought in as they need refactoring to account for the expr API change and it is expected they will land ahead of this.
https://lore.kernel.org/lkml/[email protected]/
Tested with 'perf test' and 'make -C tools/perf build-test'.
The hashmap change was originally part of an RFC:
https://lore.kernel.org/lkml/[email protected]/
v2. moves hashmap into tools/perf/util rather than libapi, to allow hashmap's libbpf symbols to be visible when built statically for testing.
Andrii Nakryiko (1):
libbpf: Fix memory leak and possible double-free in hashmap__clear
Ian Rogers (6):
libbpf hashmap: Remove unused #include
libbpf hashmap: Fix signedness warnings
tools lib/api: Copy libbpf hashmap to tools/perf/util perf test: Provide a subtest callback to ask for the reason for
skipping a subtest
perf test: Improve pmu event metric testing
perf expr: Migrate expr ids table to a hashmap
tools/lib/bpf/hashmap.c | 10 +- tools/lib/bpf/hashmap.h | 1 - tools/perf/check-headers.sh | 4 + tools/perf/tests/builtin-test.c | 18 ++- tools/perf/tests/expr.c | 40 +++--- tools/perf/tests/pmu-events.c | 169 ++++++++++++++++++++++- tools/perf/tests/tests.h | 4 + tools/perf/util/Build | 4 + tools/perf/util/expr.c | 129 +++++++++-------- tools/perf/util/expr.h | 26 ++-- tools/perf/util/expr.y | 22 +-- tools/perf/util/hashmap.c | 238 ++++++++++++++++++++++++++++++++ tools/perf/util/hashmap.h | 177 ++++++++++++++++++++++++ tools/perf/util/metricgroup.c | 87 ++++++------ tools/perf/util/stat-shadow.c | 49 ++++--- 15 files changed, 798 insertions(+), 180 deletions(-) create mode 100644 tools/perf/util/hashmap.c create mode 100644 tools/perf/util/hashmap.h
Signature · 1 line
2.26.2.761.g0e0b3e54be-goog
Ian Rogers
reviewerPATCH v2 1/7 libbpf: Fix memory leak and possible double-free in hashmap__clear
From: Andrii Nakryiko <[email protected]>
Fix memory leak in hashmap_clear() not freeing hashmap_entry structs for each of the remaining entries. Also NULL-out bucket list to prevent possible double-free between hashmap__clear() and hashmap__free().
Running test_progs-asan flavor clearly showed this problem.
--- tools/lib/bpf/hashmap.c | 7 +++++++ 1 file changed, 7 insertions(+)
tools/lib/bpf/hashmap.c +7 -0
@@ -59,7 +59,14 @@ struct hashmap *hashmap__new(hashmap_hash_fn hash_fn,void hashmap__clear (struct hashmap *map) { +struct hashmap_entry *cur, *tmp; +int bkt; + +hashmap__for_each_entry_safe (map, cur, tmp, bkt) { +free (cur); + }free (map->buckets); + map->buckets =NULL ; map->cap = map->cap_bits = map->sz =0 ; }
Signature · 1 line
2.26.2.761.g0e0b3e54be-goog
Ian Rogers
reviewerPATCH v2 2/7 libbpf hashmap: Remove unused #include
In reply to Ian RogersRemove #include of libbpf_internal.h that is unused. Discussed in this thread:
https://lore.kernel.org/lkml/CAEf4BzZRmiEds_8R8g4vaAeWvJzPb4xYLnpF0X2VNY8oTzkphQ@mail.gmail.com/
--- tools/lib/bpf/hashmap.h | 1 - 1 file changed, 1 deletion(-)
tools/lib/bpf/hashmap.h +0 -1
@@ -15,7 +15,6 @@#else #include <bits/reg.h> #endif -#include "libbpf_internal.h" static inline size_t hash_bits (size_t h,int bits) {
Signature · 1 line
2.26.2.761.g0e0b3e54be-goog
Re: [PATCH v2 2/7] libbpf hashmap: Remove unused #include
Quoted 7 lines · Ian Rogers
Remove #include of
libbpf_internal.hthat is unused. Discussed in this thread:
https://lore.kernel.org/lkml/CAEf4BzZRmiEds_8R8g4vaAeWvJzPb4xYLnpF0X2VNY8oTzkphQ@mail.gmail.com/Signed-off-by Ian Rogers <[email protected]>---
Quoted 18 lines
tools/lib/bpf/hashmap.h | 1 - 1 file changed, 1 deletion(-)tools/lib/bpf/hashmap.h +0
-2@@ -15,7 +15,6 @@#else #include <bits/reg.h> #endif -#include "libbpf_internal.h" static inline size_t hash_bits (size_t h,int bits) { --2.26.2.761.g0e0b3e54be-goog
Ian Rogers
reviewerPATCH v2 3/7 libbpf hashmap: Fix signedness warnings
In reply to Ian RogersFixes the following warnings:
hashmap.c: In function ‘hashmap__clear’:
hashmap.h:150:20: error: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘long unsigned int’} [-Werror=sign-compare]
150 | for (bkt = 0; bkt < map->cap; bkt++) \
hashmap.c: In function ‘hashmap_grow’:
hashmap.h:150:20: error: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘long unsigned int’} [-Werror=sign-compare]
150 | for (bkt = 0; bkt < map->cap; bkt++) \
--- tools/lib/bpf/hashmap.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
tools/lib/bpf/hashmap.c +2 -3
@@ -60,7 +60,7 @@ struct hashmap *hashmap__new(hashmap_hash_fn hash_fn,void hashmap__clear (struct hashmap *map) {struct hashmap_entry *cur, *tmp; -int bkt; +size_t bkt;hashmap__for_each_entry_safe (map, cur, tmp, bkt) {free (cur);
@@ -100,8 +100,7 @@ static int hashmap_grow(struct hashmap *map)struct hashmap_entry **new_buckets;struct hashmap_entry *cur, *tmp;size_t new_cap_bits, new_cap; -size_t h; -int bkt; +size_t h, bkt; new_cap_bits = map->cap_bits +1 ;if (new_cap_bits <HASHMAP_MIN_CAP_BITS )
Signature · 1 line
2.26.2.761.g0e0b3e54be-goog
Re: [PATCH v2 3/7] libbpf hashmap: Fix signedness warnings
Quoted 7 lines · Ian Rogers
Fixes the following warnings:
hashmap.c: In function ‘hashmap__clear’:
hashmap.h:150:20: error: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘long unsigned int’} [-Werror=sign-compare]
150 | for (bkt = 0; bkt <map->cap; bkt++) \
Quoted 6 lines
hashmap.c: In function ‘hashmap_grow’:
hashmap.h:150:20: error: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘long unsigned int’} [-Werror=sign-compare]
150 | for (bkt = 0; bkt <map->cap; bkt++) \Signed-off-by Ian Rogers <[email protected]>---
Quoted 29 lines
tools/lib/bpf/hashmap.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)tools/lib/bpf/hashmap.c +2
-4@@ -60,7 +60,7 @@ struct hashmap *hashmap__new(hashmap_hash_fn hash_fn,void hashmap__clear (struct hashmap *map) {struct hashmap_entry *cur, *tmp; -int bkt; +size_t bkt;hashmap__for_each_entry_safe (map, cur, tmp, bkt) {free (cur);@@ -100,8 +100,7 @@ static int hashmap_grow(struct hashmap *map)struct hashmap_entry **new_buckets;struct hashmap_entry *cur, *tmp;size_t new_cap_bits, new_cap; -size_t h; -int bkt; +size_t h, bkt; new_cap_bits = map->cap_bits +1 ;if (new_cap_bits <HASHMAP_MIN_CAP_BITS ) --2.26.2.761.g0e0b3e54be-goog
Ian Rogers
reviewerPATCH v2 4/7 tools lib/api: Copy libbpf hashmap to tools/perf/util
In reply to Ian RogersAllow use of hashmap in perf. Modify perf's check-headers.sh script to check that the files are kept in sync, in the same way kernel headers are checked. This will warn if they are out of sync at the start of a perf build.
--- tools/perf/check-headers.sh | 4 + tools/perf/util/Build | 4 + tools/perf/util/hashmap.c | 238 ++++++++++++++++++++++++++++++++++++ tools/perf/util/hashmap.h | 177 +++++++++++++++++++++++++++ 4 files changed, 423 insertions(+) create mode 100644 tools/perf/util/hashmap.c create mode 100644 tools/perf/util/hashmap.h
tools/perf/check-headers.sh +4 -0
@@ -128,4 +128,8 @@ check arch/x86/lib/insn.c '-I "^#include [\"<]\(../include/\)*asm/in # diff non-symmetric files check_2 tools/perf/arch/x86/entry/syscalls/syscall_64.tbl arch/x86/entry/syscalls/syscall_64.tbl +# check duplicated library files +check_2 tools/perf/util/hashmap.h tools/lib/bpf/hashmap.h +check_2 tools/perf/util/hashmap.c tools/lib/bpf/hashmap.c + cd tools/perf
tools/perf/util/Build +4 -0
@@ -136,6 +136,10 @@ perf-$(CONFIG_LIBELF) += symbol-elf.o perf-$(CONFIG_LIBELF) += probe-file.o perf-$(CONFIG_LIBELF) += probe-event.o +ifndef CONFIG_LIBBPF +perf-y += hashmap.o +endif + ifndef CONFIG_LIBELF perf-y += symbol-minimal.o endif
tools/perf/util/hashmap.c new file +238 -0
238 lines not on this page · raw message
tools/perf/util/hashmap.h new file +177 -0
177 lines not on this page · raw message
Signature · 1 line
2.26.2.761.g0e0b3e54be-goog
Re: [PATCH v2 4/7] tools lib/api: Copy libbpf hashmap to tools/perf/util
Quoted 8 lines · Ian Rogers
Allow use of hashmap in perf. Modify perf's
check-headers.shscript to check that the files are kept in sync, in the same way kernel headers are checked. This will warn if they are out of sync at the start of a perf build.Signed-off-by Ian Rogers <[email protected]>---
Given you want to make sure they stay 1 to 1, would just creating a symlink work instead of copying the code?
Either way, I think hashmap is stable and not going to change frequently, so whichever way is fine with me.
Quoted 8 lines
tools/perf/check-headers.sh | 4 + tools/perf/util/Build | 4 + tools/perf/util/hashmap.c | 238 ++++++++++++++++++++++++++++++++++++ tools/perf/util/hashmap.h | 177 +++++++++++++++++++++++++++ 4 files changed, 423 insertions(+) create mode 100644 tools/perf/util/hashmap.c create mode 100644 tools/perf/util/hashmap.h
[...]
Ian Rogers
reviewerPATCH v2 5/7 perf test: Provide a subtest callback to ask for the reason for skipping a subtest
In reply to Ian RogersNow subtests can inform why a test was skipped. The upcoming patch improvint PMU event metric testing will use it.
--- tools/perf/tests/builtin-test.c | 11 +++++++++-- tools/perf/tests/tests.h | 1 + 2 files changed, 10 insertions(+), 2 deletions(-)
tools/perf/tests/builtin-test.c +9 -2
@@ -429,8 +429,15 @@ static int test_and_print(struct test *t, bool force_skip, int subtest)case TEST_OK :pr_info (" Ok\n" );break ; -case TEST_SKIP : -color_fprintf (stderr,PERF_COLOR_YELLOW ," Skip\n" ); +case TEST_SKIP : { +const char *skip_reason =NULL ; +if (t->subtest.skip_reason) + skip_reason = t->subtest.skip_reason (subtest); +if (skip_reason) +color_fprintf (stderr,PERF_COLOR_YELLOW ," Skip (%s)\n" , skip_reason); +else +color_fprintf (stderr,PERF_COLOR_YELLOW ," Skip\n" ); + }break ;case TEST_FAIL :default :
tools/perf/tests/tests.h +1 -0
@@ -34,6 +34,7 @@ struct test {bool skip_if_fail;int (*get_nr)(void );const char *(*get_desc)(int subtest); +const char *(*skip_reason)(int subtest); } subtest;bool (*is_supported)(void );void *priv;
Signature · 1 line
2.26.2.761.g0e0b3e54be-goog
Ian Rogers
reviewerPATCH v2 6/7 perf test: Improve pmu event metric testing
In reply to Ian RogersBreak pmu-events test into 2 and add a test to verify that all pmu metric expressions simply parse. Try to parse all metric ids/events, skip/warn if metrics for the current architecture fail to parse. To support warning for a skip, and an ability for a subtest to describe why it skips.
Tested on power9, skylakex, haswell, broadwell, westmere, sandybridge and ivybridge.
May skip/warn on other architectures if metrics are invalid. In particular s390 is untested, but its expressions are trivial. The untested architectures with expressions are power8, cascadelakex, tremontx, skylake, jaketown, ivytown and variants of haswell and broadwell.
v3. addresses review comments from John Garry <[email protected]>, Jiri Olsa <[email protected]> and Arnaldo Carvalho de Melo <[email protected]>.
v2. changes the commit message as event parsing errors no longer cause the test to fail.
--- tools/perf/tests/builtin-test.c | 7 ++ tools/perf/tests/pmu-events.c | 168 ++++++++++++++++++++++++++++++-- tools/perf/tests/tests.h | 3 + 3 files changed, 172 insertions(+), 6 deletions(-)
tools/perf/tests/builtin-test.c +7 -0
@@ -75,6 +75,13 @@ static struct test generic_tests[] = { { .desc = "PMU events", .func = test__pmu_events, + .subtest = { + .skip_if_fail = false, + .get_nr = test__pmu_events_subtest_get_nr, + .get_desc = test__pmu_events_subtest_get_desc, + .skip_reason = test__pmu_events_subtest_skip_reason, + }, + }, { .desc = "DSO data read",
tools/perf/tests/pmu-events.c +162 -6
192 lines not on this page · raw message
tools/perf/tests/tests.h +3 -0
@@ -51,6 +51,9 @@ int test__perf_evsel__tp_sched_test(struct test *test, int subtest);int test__syscall_openat_tp_fields (struct test *test,int subtest);int test__pmu (struct test *test,int subtest);int test__pmu_events (struct test *test,int subtest); +const char *test__pmu_events_subtest_get_desc (int subtest); +const char *test__pmu_events_subtest_skip_reason (int subtest); +int test__pmu_events_subtest_get_nr (void );int test__attr (struct test *test,int subtest);int test__dso_data (struct test *test,int subtest);int test__dso_data_cache (struct test *test,int subtest);
Signature · 1 line
2.26.2.761.g0e0b3e54be-goog
Ian Rogers
reviewerPATCH v2 7/7 perf expr: Migrate expr ids table to a hashmap
In reply to Ian RogersUse a hashmap between a char* string and a double* value. While bpf's hashmap entries are size_t in size, we can't guarantee sizeof(size_t) >= sizeof(double). Avoid a memory allocation when gathering ids by making 0.0 a special value encoded as NULL.
Original map suggestion by Andi Kleen:
https://lore.kernel.org/lkml/[email protected]/
and seconded by Jiri Olsa:
https://lore.kernel.org/lkml/20200423112915.GH1136647@krava/
--- tools/perf/tests/expr.c | 40 ++++++----- tools/perf/tests/pmu-events.c | 25 +++---- tools/perf/util/expr.c | 129 +++++++++++++++++++--------------- tools/perf/util/expr.h | 26 +++---- tools/perf/util/expr.y | 22 +----- tools/perf/util/metricgroup.c | 87 +++++++++++------------ tools/perf/util/stat-shadow.c | 49 ++++++++----- 7 files changed, 197 insertions(+), 181 deletions(-)
tools/perf/tests/expr.c +21 -19
@@ -19,11 +19,9 @@ static int test(struct expr_parse_ctx *ctx, const char *e, double val2)int test__expr (struct test *t__maybe_unused ,int subtest__maybe_unused ) {const char *p; -const char **other; -double val; -int i, ret; +double val, *val_ptr; +int ret;struct expr_parse_ctx ctx; -int num_other;expr__ctx_init (&ctx);expr__add_id (&ctx,"FOO" ,1 );
@@ -52,25 +50,29 @@ int test__expr(struct test *t __maybe_unused, int subtest __maybe_unused) ret =expr__parse (&val, &ctx, p,1 );TEST_ASSERT_VAL ("missing operand" , ret ==-1 ); +hashmap__clear (&ctx.ids);TEST_ASSERT_VAL ("find other" , -expr__find_other ("FOO + BAR + BAZ + BOZO" ,"FOO" , &other, &num_other,1 ) ==0 ); -TEST_ASSERT_VAL ("find other" , num_other ==3 ); -TEST_ASSERT_VAL ("find other" , !strcmp (other[0 ],"BAR" )); -TEST_ASSERT_VAL ("find other" , !strcmp (other[1 ],"BAZ" )); -TEST_ASSERT_VAL ("find other" , !strcmp (other[2 ],"BOZO" )); -TEST_ASSERT_VAL ("find other" , other[3 ] ==NULL ); +expr__find_other ("FOO + BAR + BAZ + BOZO" ,"FOO" , + &ctx,1 ) ==0 ); +TEST_ASSERT_VAL ("find other" ,hashmap__size (&ctx.ids) ==3 ); +TEST_ASSERT_VAL ("find other" ,hashmap__find (&ctx.ids,"BAR" , + (void **)&val_ptr)); +TEST_ASSERT_VAL ("find other" ,hashmap__find (&ctx.ids,"BAZ" , + (void **)&val_ptr)); +TEST_ASSERT_VAL ("find other" ,hashmap__find (&ctx.ids,"BOZO" , + (void **)&val_ptr)); +hashmap__clear (&ctx.ids);TEST_ASSERT_VAL ("find other" , -expr__find_other ("EVENT1\\,param\\=?@ + EVENT2\\,param\\=?@" ,NULL , - &other, &num_other,3 ) ==0 ); -TEST_ASSERT_VAL ("find other" , num_other ==2 ); -TEST_ASSERT_VAL ("find other" , !strcmp (other[0 ],"EVENT1,param=3/" )); -TEST_ASSERT_VAL ("find other" , !strcmp (other[1 ],"EVENT2,param=3/" )); -TEST_ASSERT_VAL ("find other" , other[2 ] ==NULL ); +expr__find_other ("EVENT1\\,param\\=?@ + EVENT2\\,param\\=?@" , +NULL , &ctx,3 ) ==0 ); +TEST_ASSERT_VAL ("find other" ,hashmap__size (&ctx.ids) ==2 ); +TEST_ASSERT_VAL ("find other" ,hashmap__find (&ctx.ids,"EVENT1,param=3/" , + (void **)&val_ptr)); +TEST_ASSERT_VAL ("find other" ,hashmap__find (&ctx.ids,"EVENT2,param=3/" , + (void **)&val_ptr)); -for (i =0 ; i < num_other; i++) -zfree (&other[i]); -free ((void *)other); +expr__ctx_clear (&ctx);return 0 ; }
tools/perf/tests/pmu-events.c +13 -12
@@ -433,8 +433,6 @@ static int test_parsing(void)struct pmu_events_map *map;struct pmu_event *pe;int i, j, k; -const char **ids; -int idnum;int ret =0 ;struct expr_parse_ctx ctx;double result;
@@ -446,29 +444,34 @@ static int test_parsing(void)break ; j =0 ;for (;;) { +struct hashmap_entry *cur; +size_t bkt; + pe = &map->table[j++];if (!pe->name && !pe->metric_group && !pe->metric_name)break ;if (!pe->metric_expr)continue ; -if (expr__find_other (pe->metric_expr,NULL , - &ids, &idnum,0 ) <0 ) { +expr__ctx_init (&ctx); +if (expr__find_other (pe->metric_expr,NULL , &ctx,0 ) + <0 ) {expr_failure ("Parse other failed" , map, pe); ret++;continue ; } -expr__ctx_init (&ctx);/* * Add all ids with a made up value. The value may * trigger divide by zero when subtracted and so try to * make them unique. */ -for (k =0 ; k < idnum; k++) -expr__add_id (&ctx, ids[k], k +1 ); + k =1 ; +hashmap__for_each_entry ((&ctx.ids), cur, bkt) +expr__add_id (&ctx,strdup (cur->key), k++); -for (k =0 ; k < idnum; k++) { -if (check_parse_id (ids[k], map == cpus_map, pe)) +hashmap__for_each_entry ((&ctx.ids), cur, bkt) { +if (check_parse_id (cur->key, map == cpus_map, + pe)) ret++; }
@@ -476,9 +479,7 @@ static int test_parsing(void)expr_failure ("Parse failed" , map, pe); ret++; } -for (k =0 ; k < idnum; k++) -zfree (&ids[k]); -free (ids); +expr__ctx_clear (&ctx); } }/* TODO: fail when not ok */
tools/perf/util/expr.c +72 -57
158 lines not on this page · raw message
tools/perf/util/expr.h +13 -13
@@ -2,17 +2,14 @@#ifndef PARSE_CTX_H #define PARSE_CTX_H 1 -#define EXPR_MAX_OTHER 64 -#define MAX_PARSE_ID EXPR_MAX_OTHER - -struct expr_parse_id { -const char *name; -double val; -}; +#ifdef HAVE_LIBBPF_SUPPORT +#include <bpf/hashmap.h> +#else +#include "hashmap.h" +#endif struct expr_parse_ctx { -int num_ids; -struct expr_parse_id ids[MAX_PARSE_ID ]; +struct hashmap ids; }; struct expr_scanner_ctx {
@@ -21,9 +18,12 @@ struct expr_scanner_ctx { };void expr__ctx_init (struct expr_parse_ctx *ctx); -void expr__add_id (struct expr_parse_ctx *ctx,const char *id,double val); -int expr__parse (double *final_val,struct expr_parse_ctx *ctx,const char *expr,int runtime); -int expr__find_other (const char *expr,const char *one,const char ***other, -int *num_other,int runtime); +void expr__ctx_clear (struct expr_parse_ctx *ctx); +int expr__add_id (struct expr_parse_ctx *ctx,const char *id,double val); +int expr__get_id (struct expr_parse_ctx *ctx,const char *id,double *val_ptr); +int expr__parse (double *final_val,struct expr_parse_ctx *ctx, +const char *expr,int runtime); +int expr__find_other (const char *expr,const char *one, +struct expr_parse_ctx *ids,int runtime);#endif
tools/perf/util/expr.y +2 -20
@@ -46,19 +46,6 @@ static void expr_error(double *final_val __maybe_unused, pr_debug("%s\n", s); } -static int lookup_id(struct expr_parse_ctx *ctx, char *id, double *val) -{ - int i; - - for (i = 0; i < ctx->num_ids; i++) { - if (!strcasecmp(ctx->ids[i].name, id)) { - *val = ctx->ids[i].val; - return 0; - } - } - return -1; -} - %} %%
@@ -72,12 +59,7 @@ all_other: all_other other other: ID { - if (ctx->num_ids + 1 >= EXPR_MAX_OTHER) { - pr_err("failed: way too many variables"); - YYABORT; - } - - ctx->ids[ctx->num_ids++].name = $1; + expr__add_id(ctx, $1, 0.0); } | MIN | MAX | IF | ELSE | SMT_ON | NUMBER | '|' | '^' | '&' | '-' | '+' | '*' | '/' | '%' | '(' | ')' | ','
@@ -92,7 +74,7 @@ if_expr: ; expr: NUMBER - | ID { if (lookup_id(ctx, $1, &$$) < 0) { + | ID { if (expr__get_id(ctx, $1, &$$)) { pr_debug("%s not found\n", $1); YYABORT; }
tools/perf/util/metricgroup.c +44 -43
182 lines not on this page · raw message
tools/perf/util/stat-shadow.c +32 -17
116 lines not on this page · raw message
Signature · 1 line
2.26.2.761.g0e0b3e54be-goog
Re: [PATCH v2 7/7] perf expr: Migrate expr ids table to a hashmap
Quoted 41 lines · Ian Rogers
Use a hashmap between a char* string and a double* value. While bpf's hashmap entries are
size_tin size, we can't guaranteesizeof(size_t)>= sizeof(double). Avoid a memory allocation when gathering ids by making 0.0 a special value encoded as NULL.Original map suggestion by Andi Kleen:
https://lore.kernel.org/lkml/[email protected]/
and seconded by Jiri Olsa:
https://lore.kernel.org/lkml/20200423112915.GH1136647@krava/Signed-off-by Ian Rogers <[email protected]>--- tools/perf/tests/expr.c | 40 ++++++----- tools/perf/tests/pmu-events.c | 25 +++---- tools/perf/util/expr.c | 129 +++++++++++++++++++--------------- tools/perf/util/expr.h | 26 +++---- tools/perf/util/expr.y | 22 +----- tools/perf/util/metricgroup.c | 87 +++++++++++------------ tools/perf/util/stat-shadow.c | 49 ++++++++----- 7 files changed, 197 insertions(+), 181 deletions(-)tools/perf/tests/expr.c +2
-4@@ -19,11 +19,9 @@ static int test(struct expr_parse_ctx *ctx, const char *e, double val2)int test__expr (struct test *t__maybe_unused ,int subtest__maybe_unused ) {const char *p; -const char **other; -double val; -int i, ret; +double val, *val_ptr; +int ret;struct expr_parse_ctx ctx; -int num_other;expr__ctx_init (&ctx);expr__add_id (&ctx,"FOO" ,1 );
tools/perf/tests/expr.c int test__expr(struct test *t __maybe_unused, int subtest __maybe_unused) · line 53
Quoted 4 lines
tools/perf/tests/expr.c +1
-0@@ -52,25 +50,29 @@ int test__expr(struct test *t __maybe_unused, int subtest __maybe_unused) ret =expr__parse (&val, &ctx, p,1 );TEST_ASSERT_VAL ("missing operand" , ret ==-1 ); +hashmap__clear (&ctx.ids);
hashmap__clear() will free up memory allocated for hashmap itself and hash entries, but not keys/values. Unless it's happening somewhere else, you'll need to do something similar to expr__ctx_clear() below?
Same below for another "lone" hashmap_clear() call.
Quoted 2 lines
tools/perf/tests/expr.c +0
-1TEST_ASSERT_VAL("find other", -expr__find_other ("FOO + BAR + BAZ + BOZO" ,"FOO" , &other, &num_other,1 ) ==0 );
[...]
Ian Rogers
reviewerRe: [PATCH v2 7/7] perf expr: Migrate expr ids table to a hashmap
Quoted 46 lines · Andrii Nakryiko
Quoted 44 lines · Ian Rogers
Use a hashmap between a char* string and a double* value. While bpf's hashmap entries are
size_tin size, we can't guaranteesizeof(size_t)>= sizeof(double). Avoid a memory allocation when gathering ids by making 0.0 a special value encoded as NULL.Original map suggestion by Andi Kleen:
https://lore.kernel.org/lkml/[email protected]/
and seconded by Jiri Olsa:
https://lore.kernel.org/lkml/20200423112915.GH1136647@krava/Signed-off-by Ian Rogers <[email protected]>--- tools/perf/tests/expr.c | 40 ++++++----- tools/perf/tests/pmu-events.c | 25 +++---- tools/perf/util/expr.c | 129 +++++++++++++++++++--------------- tools/perf/util/expr.h | 26 +++---- tools/perf/util/expr.y | 22 +----- tools/perf/util/metricgroup.c | 87 +++++++++++------------ tools/perf/util/stat-shadow.c | 49 ++++++++----- 7 files changed, 197 insertions(+), 181 deletions(-)tools/perf/tests/expr.c +2
-4@@ -19,11 +19,9 @@ static int test(struct expr_parse_ctx *ctx, const char *e, double val2)int test__expr (struct test *t__maybe_unused ,int subtest__maybe_unused ) {const char *p; -const char **other; -double val; -int i, ret; +double val, *val_ptr; +int ret;struct expr_parse_ctx ctx; -int num_other;expr__ctx_init (&ctx);expr__add_id (&ctx,"FOO" ,1 );@@ -52,25 +50,29 @@ int test__expr(struct test *t __maybe_unused, int subtest __maybe_unused) ret =expr__parse (&val, &ctx, p,1 );TEST_ASSERT_VAL ("missing operand" , ret ==-1 );
Quoted 5 lines
tools/perf/tests/expr.c int test__expr(struct test *t __maybe_unused, int subtest __maybe_unused) · line 53
Quoted 1 line
tools/perf/tests/expr.c +1
-0+hashmap__clear (&ctx.ids);
hashmap__clear()will free up memory allocated for hashmap itself and hash entries, but not keys/values. Unless it's happening somewhere else, you'll need to do something similar toexpr__ctx_clear()below?
In this case the const char* keys come from the literals added on lines 27 and 28 and so didn't need free-ing - which is what expr__ctx_clear() does. I've made these literals strdups and switched to expr__ctx_clear() as you suggest, as this is more reflective of the real use.
Quoted 1 line
Same below for another "lone"
hashmap_clear()call.
This was a memory leak, thanks!
Ian
Quoted 4 lines
Quoted 2 lines
tools/perf/tests/expr.c +0
-1TEST_ASSERT_VAL("find other", -expr__find_other ("FOO + BAR + BAZ + BOZO" ,"FOO" , &other, &num_other,1 ) ==0 );[...]
Re: [PATCH v2 7/7] perf expr: Migrate expr ids table to a hashmap
In reply to Ian RogersOn Fri, May 15, 2020 at 09:50:07AM -0700, Ian Rogers wrote:
SNIP
Quoted 27 lines
tools/perf/util/metricgroup.c +2
-5@@ -85,8 +85,7 @@ static void metricgroup__rblist_init(struct rblist *metric_events)struct egroup {struct list_head nd; -int idnum; -const char **ids; +struct expr_parse_ctx pctx;const char *metric_name;const char *metric_expr;const char *metric_unit;@@ -94,19 +93,21 @@ struct egroup { };static struct evsel *find_evsel_group (struct evlist *perf_evlist, -const char **ids, -int idnum, +struct expr_parse_ctx *pctx,struct evsel **metric_events,bool *evlist_used) {struct evsel *ev; -int i =0 , j =0 ;bool leader_found;
tools/perf/util/metricgroup.c struct egroup { · line 110
Quoted 10 lines
tools/perf/util/metricgroup.c +5
-1+const size_t idnum =hashmap__size (&pctx->ids); +size_t i =0 ; +int j =0 ; +double *val_ptr; evlist__for_each_entry (perf_evlist, ev) {if (evlist_used[j++])continue ; -if (!strcmp (ev->name, ids[i])) { +if (hashmap__find (&pctx->ids, ev->name, (void **)&val_ptr)) {
hum, you sure it's doing the same thing as before?
hashmap__find will succede all the time in here, while the previous code was looking for the start of the group ... the logic in here is little convoluted, so maybe I'm missing some point in here ;-)
jirka
Quoted 12 lines
tools/perf/util/metricgroup.c +2
-1if (!metric_events[i]) metric_events[i] = ev; i++;@@ -118,7 +119,8 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,memset (metric_events,0 ,sizeof (struct evsel *) * idnum); -if (!strcmp (ev->name, ids[i])) { +if (hashmap__find (&pctx->ids, ev->name, + (void **)&val_ptr)) {if (!metric_events[i]) metric_events[i] = ev;
SNIP
Ian Rogers
reviewerRe: [PATCH v2 7/7] perf expr: Migrate expr ids table to a hashmap
Quoted 45 lines · Jiri Olsa
On Fri, May 15, 2020 at 09:50:07AM -0700, Ian Rogers wrote:
SNIP
Quoted 27 lines
tools/perf/util/metricgroup.c +2
-5@@ -85,8 +85,7 @@ static void metricgroup__rblist_init(struct rblist *metric_events)struct egroup {struct list_head nd; -int idnum; -const char **ids; +struct expr_parse_ctx pctx;const char *metric_name;const char *metric_expr;const char *metric_unit;@@ -94,19 +93,21 @@ struct egroup { };static struct evsel *find_evsel_group (struct evlist *perf_evlist, -const char **ids, -int idnum, +struct expr_parse_ctx *pctx,struct evsel **metric_events,bool *evlist_used) {struct evsel *ev; -int i =0 , j =0 ;bool leader_found;tools/perf/util/metricgroup.c struct egroup { · line 110
Quoted 10 lines
tools/perf/util/metricgroup.c +5
-1+const size_t idnum =hashmap__size (&pctx->ids); +size_t i =0 ; +int j =0 ; +double *val_ptr; evlist__for_each_entry (perf_evlist, ev) {if (evlist_used[j++])continue ; -if (!strcmp (ev->name, ids[i])) { +if (hashmap__find (&pctx->ids, ev->name, (void **)&val_ptr)) {hum, you sure it's doing the same thing as before?
Quoted 4 lines
hashmap__findwill succede all the time in here, while the previous code was looking for the start of the group ... the logic in here is little convoluted, so maybe I'm missing some point in here ;-)
If we have a metric like "A + B" and another like "C / D" then by we'll generate a string (the extra_events strbuf in the code) like "{A,B}:W,{C,D}:W" from __metricgroup__add_metric. This will turn into an evlist in metricgroup__parse_groups of A,B,C,D. The code is trying to associate the events A,B with the first metric and C,D with the second. The code doesn't support sharing of events and events are marked as used and can't be part of other metrics. The evlist order is also reflective of the order of metrics, so if there were metrics "A + B + C" and "A + B", as the first metric is first in the evlist we don't run the risk of C being placed with A and B in a different group.
The old code used the order of events to match within a metric and say for metric "A+B+C" we want to match A then B, and so on. The new code acts more like a set, so "A + B + C" becomes a set containing A, B and C, we check A is in the set then B and then C. For both pieces of code they are only working because of the evlist_used "bitmap" and that the order in the evlists and metrics matches.
The current code could just use ordering to match first n1 events with the first metric, the next n2 events with the second and so on. So both the find now, and the strcmp before always return true in this branch.
In the RFC patch set I want to share events and so I do checks related to the group leader so that I know when moving from one group to another in the evlist. The find/strcmp becomes load bearing as I will re-use events as long as they match.
https://lore.kernel.org/lkml/[email protected]/
Quoted 8 lines
jirka
tools/perf/util/metricgroup.c static struct evsel *find_evsel_group(struct evlist *perf_evlist, · line 120
Quoted 6 lines
tools/perf/util/metricgroup.c +0
-0if (!metric_events[i]) metric_events[i] = ev; i++;@@ -118,7 +119,8 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,memset (metric_events,0 ,sizeof (struct evsel *) * idnum);
This re-check was unnecessary in the old code and unnecessary even more so now as the hashmap_find is given exactly the same arguments. I'll remove it in v3 while addressing Andrii's memory leak fixes.
Thanks,
Ian
Quoted 8 lines
Quoted 5 lines
tools/perf/util/metricgroup.c +2
-1-if (!strcmp (ev->name, ids[i])) { +if (hashmap__find (&pctx->ids, ev->name, + (void **)&val_ptr)) {if (!metric_events[i]) metric_events[i] = ev;SNIP
Re: [PATCH v2 7/7] perf expr: Migrate expr ids table to a hashmap
Quoted 83 lines · Ian Rogers
Quoted 45 lines · Jiri Olsa
On Fri, May 15, 2020 at 09:50:07AM -0700, Ian Rogers wrote:
SNIP
Quoted 27 lines
tools/perf/util/metricgroup.c +2
-5@@ -85,8 +85,7 @@ static void metricgroup__rblist_init(struct rblist *metric_events)struct egroup {struct list_head nd; -int idnum; -const char **ids; +struct expr_parse_ctx pctx;const char *metric_name;const char *metric_expr;const char *metric_unit;@@ -94,19 +93,21 @@ struct egroup { };static struct evsel *find_evsel_group (struct evlist *perf_evlist, -const char **ids, -int idnum, +struct expr_parse_ctx *pctx,struct evsel **metric_events,bool *evlist_used) {struct evsel *ev; -int i =0 , j =0 ;bool leader_found;tools/perf/util/metricgroup.c struct egroup { · line 110
Quoted 10 lines
tools/perf/util/metricgroup.c +5
-1+const size_t idnum =hashmap__size (&pctx->ids); +size_t i =0 ; +int j =0 ; +double *val_ptr; evlist__for_each_entry (perf_evlist, ev) {if (evlist_used[j++])continue ; -if (!strcmp (ev->name, ids[i])) { +if (hashmap__find (&pctx->ids, ev->name, (void **)&val_ptr)) {hum, you sure it's doing the same thing as before?
Quoted 4 lines
hashmap__findwill succede all the time in here, while the previous code was looking for the start of the group ... the logic in here is little convoluted, so maybe I'm missing some point in here ;-)If we have a metric like "A + B" and another like "C / D" then by we'll generate a string (the
extra_eventsstrbuf in the code) like "{A,B}:W,{C,D}:W" from__metricgroup__add_metric. This will turn into an evlist inmetricgroup__parse_groupsof A,B,C,D. The code is trying to associate the events A,B with the first metric and C,D with the second. The code doesn't support sharing of events and events are marked as used and can't be part of other metrics. The evlist order is also reflective of the order of metrics, so if there were metrics "A + B + C" and "A + B", as the first metric is first in the evlist we don't run the risk of C being placed with A and B in a different group.The old code used the order of events to match within a metric and say for metric "A+B+C" we want to match A then B, and so on. The new code acts more like a set, so "A + B + C" becomes a set containing A, B and C, we check A is in the set then B and then C. For both pieces of code they are only working because of the
evlist_used"bitmap" and that the order in the evlists and metrics matches.The current code could just use ordering to match first n1 events with the first metric, the next n2 events with the second and so on. So both the find now, and the strcmp before always return true in this branch.
In the RFC patch set I want to share events and so I do checks related to the group leader so that I know when moving from one group to another in the evlist. The find/strcmp becomes load bearing as I will re-use events as long as they match.
https://lore.kernel.org/lkml/[email protected]/Quoted 2 lines
jirka
Quoted 10 lines
Quoted 6 lines
tools/perf/util/metricgroup.c static struct evsel *find_evsel_group(struct evlist *perf_evlist, · line 120
Quoted 6 lines
tools/perf/util/metricgroup.c +0
-0if (!metric_events[i]) metric_events[i] = ev; i++;@@ -118,7 +119,8 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,memset (metric_events,0 ,sizeof (struct evsel *) * idnum);This re-check was unnecessary in the old code and unnecessary even more so now as the
hashmap_findis given exactly the same arguments. I'll remove it in v3 while addressing Andrii's memory leak fixes.
ok, that was my other confusion, because it's useless ;-)
thanks for the explanation,
jirka
Quoted 13 lines
Thanks,
IanQuoted 8 lines
Quoted 5 lines
tools/perf/util/metricgroup.c +2
-1-if (!strcmp (ev->name, ids[i])) { +if (hashmap__find (&pctx->ids, ev->name, + (void **)&val_ptr)) {if (!metric_events[i]) metric_events[i] = ev;SNIP
Re: [PATCH v2 7/7] perf expr: Migrate expr ids table to a hashmap
In reply to Ian RogersOn Fri, May 15, 2020 at 09:50:07AM -0700, Ian Rogers wrote:
SNIP
Quoted 21 lines
tools/perf/util/expr.c +8
-0@@ -4,25 +4,76 @@#include "expr.h" #include "expr-bison.h" #include "expr-flex.h" +#include <linux/kernel.h> #ifdef PARSER_DEBUG extern int expr_debug;#endif +static size_t key_hash (const void *key,void *ctx__maybe_unused ) +{ +const char *str = (const char *)key; +size_t hash =0 ; + +while (*str !='\0' ) { + hash *=31 ;
tools/perf/util/expr.c line 29
Quoted 10 lines
tools/perf/util/expr.c +10
-0+ hash += *str; + str++; + } +return hash; +} + +static bool key_equal (const void *key1,const void *key2, +void *ctx__maybe_unused ) +{ +return !strcmp ((const char *)key1, (const char *)key2);
should that be strcasecmp ? would it affect the key_hash as well?
jirka
Ian Rogers
reviewerRe: [PATCH v2 7/7] perf expr: Migrate expr ids table to a hashmap
Quoted 30 lines · Jiri Olsa
On Fri, May 15, 2020 at 09:50:07AM -0700, Ian Rogers wrote:
SNIP
Quoted 25 lines
tools/perf/util/expr.c +12
-0@@ -4,25 +4,76 @@#include "expr.h" #include "expr-bison.h" #include "expr-flex.h" +#include <linux/kernel.h> #ifdef PARSER_DEBUG extern int expr_debug;#endif +static size_t key_hash (const void *key,void *ctx__maybe_unused ) +{ +const char *str = (const char *)key; +size_t hash =0 ; + +while (*str !='\0' ) { + hash *=31 ; + hash += *str; + str++; + } +return hash;
Quoted 8 lines
tools/perf/util/expr.c line 29
Quoted 6 lines
tools/perf/util/expr.c +6
-0+} + +static bool key_equal (const void *key1,const void *key2, +void *ctx__maybe_unused ) +{ +return !strcmp ((const char *)key1, (const char *)key2);should that be strcasecmp ? would it affect the
key_hashas well?
The original code does make use of strcasecmp in one place, but in the group matching (the main useless use for this code) it doesn't. I don't think it is a regression to keep it as this, and would like a test case for when it does matter. Is that ok?
Thanks,
Ian
Quoted 2 lines
jirka
Em Fri, May 15, 2020 at 09:50:00AM -0700, Ian Rogers escreveu:
Quoted 10 lines
Perf's expr code currently builds an array of strings then removes duplicates. The array is larger than necessary and has recently been increased in size. When this was done it was commented that a hashmap would be preferable.
libbpf has a hashmap but libbpf isn't currently required to build perf. To satisfy various concerns this change copies libbpf's hashmap into
tools/perf/util, it then adds a check in perf that the two are in sync.
Quoted 5 lines
Andrii's patch to hashmap from bpf-next is brought into this set to fix issues with
hashmap__clear.Two minor changes to libbpf's hashmap are made that remove an unused dependency and fix a compiler warning.
Andrii/Alexei/Daniel, what do you think about me merging these fixes in my perf-tools-next branch?
- Arnaldo
Quoted 48 lines
Two perf test changes are also brought in as they need refactoring to account for the expr API change and it is expected they will land ahead of this.
https://lore.kernel.org/lkml/[email protected]/Tested with 'perf test' and 'make -C
tools/perfbuild-test'.The hashmap change was originally part of an RFC:
https://lore.kernel.org/lkml/[email protected]/v2. moves hashmap into
tools/perf/utilrather than libapi, to allow hashmap's libbpf symbols to be visible when built statically for testing.Andrii Nakryiko (1):
libbpf: Fix memory leak and possible double-free inhashmap__clearIan Rogers (6):
libbpf hashmap: Remove unused #include
libbpf hashmap: Fix signedness warnings
tools lib/api: Copy libbpf hashmap totools/perf/utilperf test: Provide a subtest callback to ask for the reason for
skipping a subtest
perf test: Improve pmu event metric testing
perf expr: Migrate expr ids table to a hashmaptools/lib/bpf/hashmap.c | 10 +- tools/lib/bpf/hashmap.h | 1 - tools/perf/check-headers.sh | 4 + tools/perf/tests/builtin-test.c | 18 ++- tools/perf/tests/expr.c | 40 +++--- tools/perf/tests/pmu-events.c | 169 ++++++++++++++++++++++- tools/perf/tests/tests.h | 4 + tools/perf/util/Build | 4 + tools/perf/util/expr.c | 129 +++++++++-------- tools/perf/util/expr.h | 26 ++-- tools/perf/util/expr.y | 22 +-- tools/perf/util/hashmap.c | 238 ++++++++++++++++++++++++++++++++ tools/perf/util/hashmap.h | 177 ++++++++++++++++++++++++ tools/perf/util/metricgroup.c | 87 ++++++------ tools/perf/util/stat-shadow.c | 49 ++++--- 15 files changed, 798 insertions(+), 180 deletions(-) create mode 100644 tools/perf/util/hashmap.c create mode 100644 tools/perf/util/hashmap.hSignature · 1 line
2.26.2.761.g0e0b3e54be-goog
Signature · 1 line
- Arnaldo
Quoted 12 lines · Arnaldo Carvalho de Melo
Em Fri, May 15, 2020 at 09:50:00AM -0700, Ian Rogers escreveu:
Quoted 10 lines
Perf's expr code currently builds an array of strings then removes duplicates. The array is larger than necessary and has recently been increased in size. When this was done it was commented that a hashmap would be preferable.
libbpf has a hashmap but libbpf isn't currently required to build perf. To satisfy various concerns this change copies libbpf's hashmap into
tools/perf/util, it then adds a check in perf that the two are in sync.
Quoted 8 lines
Quoted 5 lines
Andrii's patch to hashmap from bpf-next is brought into this set to fix issues with
hashmap__clear.Two minor changes to libbpf's hashmap are made that remove an unused dependency and fix a compiler warning.
Andrii/Alexei/Daniel, what do you think about me merging these fixes in my perf-tools-next branch?
I'm ok with the idea, but it's up to maintainers to coordinate this :)
Quoted 55 lines
- Arnaldo
Quoted 45 lines
Two perf test changes are also brought in as they need refactoring to account for the expr API change and it is expected they will land ahead of this.
https://lore.kernel.org/lkml/[email protected]/Tested with 'perf test' and 'make -C
tools/perfbuild-test'.The hashmap change was originally part of an RFC:
https://lore.kernel.org/lkml/[email protected]/v2. moves hashmap into
tools/perf/utilrather than libapi, to allow hashmap's libbpf symbols to be visible when built statically for testing.Andrii Nakryiko (1):
libbpf: Fix memory leak and possible double-free inhashmap__clearIan Rogers (6):
libbpf hashmap: Remove unused #include
libbpf hashmap: Fix signedness warnings
tools lib/api: Copy libbpf hashmap totools/perf/utilperf test: Provide a subtest callback to ask for the reason for
skipping a subtest
perf test: Improve pmu event metric testing
perf expr: Migrate expr ids table to a hashmaptools/lib/bpf/hashmap.c | 10 +- tools/lib/bpf/hashmap.h | 1 - tools/perf/check-headers.sh | 4 + tools/perf/tests/builtin-test.c | 18 ++- tools/perf/tests/expr.c | 40 +++--- tools/perf/tests/pmu-events.c | 169 ++++++++++++++++++++++- tools/perf/tests/tests.h | 4 + tools/perf/util/Build | 4 + tools/perf/util/expr.c | 129 +++++++++-------- tools/perf/util/expr.h | 26 ++-- tools/perf/util/expr.y | 22 +-- tools/perf/util/hashmap.c | 238 ++++++++++++++++++++++++++++++++ tools/perf/util/hashmap.h | 177 ++++++++++++++++++++++++ tools/perf/util/metricgroup.c | 87 ++++++------ tools/perf/util/stat-shadow.c | 49 ++++--- 15 files changed, 798 insertions(+), 180 deletions(-) create mode 100644 tools/perf/util/hashmap.c create mode 100644 tools/perf/util/hashmap.hQuoted 3 lines
--
2.26.2.761.g0e0b3e54be-goog--
- Arnaldo
<[email protected]>,Stephane Eranian <[email protected]>
From: Arnaldo Carvalho de Melo <[email protected]>
Message-ID: <[email protected]>
Quoted 22 lines · Andrii Nakryiko
Quoted 4 lines · Arnaldo Carvalho de Melo
Em Fri, May 15, 2020 at 09:50:00AM -0700, Ian Rogers escreveu:
Quoted 2 lines
Perf's expr code currently builds an array of strings then removes duplicates. The array is larger than necessary and has recently
been
Quoted 1 line
Quoted 1 line
increased in size. When this was done it was commented that a
hashmap
Quoted 4 lines
Quoted 4 lines
would be preferable.
libbpf has a hashmap but libbpf isn't currently required to build perf. To satisfy various concerns this change copies libbpf's
hashmap
Quoted 1 line
Quoted 1 line
into
tools/perf/util, it then adds a check in perf that the two arein
Quoted 6 lines
Quoted 6 lines
sync.
Andrii's patch to hashmap from bpf-next is brought into this set to fix issues with
hashmap__clear.Two minor changes to libbpf's hashmap are made that remove an
Quoted 8 lines
unused
Quoted 3 lines
Quoted 1 line
dependency and fix a compiler warning.
Andrii/Alexei/Daniel, what do you think about me merging these fixes
in my
Quoted 1 line
perf-tools-next branch?
I'm ok with the idea, but it's up to maintainers to coordinate this :)
Good to know, do I'll take all patches except the ones touching libppf, will just make sure the copy is done with the patches applied.
At some point they'll land in libbpf and the warning from check_headers.sh will be resolved.
Thanks,
- Arnaldo
Signature · 1 line
Sent from my Android device with K-9 Mail. Please excuse my brevity.
Ian Rogers
reviewerQuoted 34 lines · On Fri, May 15, 2020 at 2:19 PM <[email protected]> wrote:
<[email protected]>,Stephane Eranian <[email protected]>
From: Arnaldo Carvalho de Melo <[email protected]>
Message-ID: <[email protected]>Quoted 26 lines · Andrii Nakryiko
Quoted 4 lines · Arnaldo Carvalho de Melo
Em Fri, May 15, 2020 at 09:50:00AM -0700, Ian Rogers escreveu:
Quoted 2 lines
Perf's expr code currently builds an array of strings then removes duplicates. The array is larger than necessary and has recently
been
Quoted 1 line
Quoted 1 line
increased in size. When this was done it was commented that a
hashmap
Quoted 4 lines
Quoted 4 lines
would be preferable.
libbpf has a hashmap but libbpf isn't currently required to build perf. To satisfy various concerns this change copies libbpf's
hashmap
Quoted 1 line
Quoted 1 line
into
tools/perf/util, it then adds a check in perf that the two arein
Quoted 6 lines
Quoted 6 lines
sync.
Andrii's patch to hashmap from bpf-next is brought into this set to fix issues with
hashmap__clear.Two minor changes to libbpf's hashmap are made that remove an
unused
Quoted 3 lines
Quoted 1 line
dependency and fix a compiler warning.
Andrii/Alexei/Daniel, what do you think about me merging these fixes
Quoted 8 lines
Quoted 4 lines
in my
Quoted 1 line
perf-tools-next branch?
I'm ok with the idea, but it's up to maintainers to coordinate this :)
Good to know, do I'll take all patches except the ones touching libppf, will just make sure the copy is done with the patches applied.
At some point they'll land in libbpf and the warning from
check_headers.shwill be resolved.
So tools/perf/util's hashmap will be ahead of libbpf's, as without the fixes the perf build is broken by Werror. This will cause check_headers to warn in perf builds, which would usually mean our header was older than the source one, but in this case it means the opposite, we're waiting for the libbpf patches to merge. Aside from some interim warnings everything will resolve itself and Arnaldo avoids landing patches in libbpf that can interfere with bpf-next.
It takes some getting your head around but sounds good to me :-) I think the only workable alternatives would be to explore having a single version of the code in some kind of shared libhashmap or to implement another hashmap in libapi. I'd like to get the rest of this work unblocked and so it'd be nice to land this and we can always refactor later - I like Arnaldo's plan. Can a bpf maintainer make sure the hashmap changes get pulled into bpf-next?
Thanks!
Ian
Quoted 6 lines
Thanks,
- Arnaldo
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
Daniel Borkmann
maintainerOn 5/15/20 11:18 PM, [email protected] wrote:
[...]
Quoted 9 lines
Quoted 5 lines
Quoted 1 line
Andrii/Alexei/Daniel, what do you think about me merging these fixes
in my
Quoted 1 line
perf-tools-next branch?
I'm ok with the idea, but it's up to maintainers to coordinate this :)
Good to know, do I'll take all patches except the ones touching libppf, will just make sure the copy is done with the patches applied.
At some point they'll land in libbpf and the warning from
check_headers.shwill be resolved.
Works for me, I've just taken in Ian's two libbpf ones into bpf-next.
Thanks everyone,
Daniel