Compress the pmu_event tables
Subsystems (2)
Ian Rogers
reviewerv5 cover Compress the pmu_event tables
jevents.py creates a number of large arrays from the json events. The arrays contain pointers to strings that need relocating. The relocations have file size, run time and memory costs. These changes refactor the pmu_events API so that the storage of the pmu_event struct isn't exposed. The format is then changed to an offset within a combined big string, with adjacent pmu_event struct variables being next to each other in the string separated by \0 - meaning only the first variable of the struct needs its offset recording.
Some related fixes are contained with the patches. The architecture jevents.py creates tables for can now be set by the JEVENTS_ARCH make variable, with a new 'all' that generates the events and metrics for all architectures.
An example of the improvement to the file size on x86 is:
no jevents - the same 19,788,464bytes
x86 jevents - ~16.7% file size saving 23,744,288bytes vs 28,502,632bytes all jevents - ~19.5% file size saving 24,469,056bytes vs 30,379,920bytes default build options plus NO_LIBBFD=1.
I originally suggested fixing this problem in:
https://lore.kernel.org/linux-perf-users/CAP-5=fVB8G4bdb9T=FncRTh9oBVKCS=+=eowAO+YSgAhab+Dtg@mail.gmail.com/
v5. Renamed two functions to be more inline with the code and added
extra commit message detail on the event sorting order as suggested by John Garry <[email protected]>.
v4. Fixed an issue with the empty-pmu-events.c spotted by John Garry
<[email protected]>.
v3. Fix an ARM build issue with a missed weak symbol. Perform some
pytype clean up.
v2. Split the substring folding optimization to its own patch and
comment tweaks as suggested by Namhyung Kim <[email protected]>. Recompute the file size savings with the latest json events and metrics.
Ian Rogers (14):
perf jevent: Add an 'all' architecture argument perf jevents: Remove the type/version variables perf jevents: Provide path to json file on error perf jevents: Sort json files entries
perf pmu-events: Hide pmu_sys_event_tables perf pmu-events: Avoid passing pmu_events_map perf pmu-events: Hide pmu_events_map
perf test: Use full metric resolution
perf pmu-events: Move test events/metrics to json perf pmu-events: Don't assume pmu_event is an array perf pmu-events: Hide the pmu_events
perf metrics: Copy entire pmu_event in find metric perf jevents: Compress the pmu_events_table perf jevents: Fold strings optimization
tools/perf/arch/arm64/util/pmu.c | 4 +- .../arch/test/test_soc/cpu/metrics.json | 64 +++ tools/perf/pmu-events/empty-pmu-events.c | 204 +++++++- tools/perf/pmu-events/jevents.py | 478 +++++++++++++++--- tools/perf/pmu-events/pmu-events.h | 40 +- tools/perf/tests/expand-cgroup.c | 25 +- tools/perf/tests/parse-metric.c | 77 +-- tools/perf/tests/pmu-events.c | 466 +++++++---------- tools/perf/util/metricgroup.c | 275 ++++++---- tools/perf/util/metricgroup.h | 5 +- tools/perf/util/pmu.c | 139 ++--- tools/perf/util/pmu.h | 10 +- tools/perf/util/s390-sample-raw.c | 50 +- 13 files changed, 1131 insertions(+), 706 deletions(-) create mode 100644 tools/perf/pmu-events/arch/test/test_soc/cpu/metrics.json
Signature · 1 line
2.37.1.595.g718a3a8f04-goog
Ian Rogers
reviewerPATCH v5 1/14 perf jevent: Add an 'all' architecture argument
When 'all' is passed as the architecture generate a mapping table for all architectures. This simplifies testing. To identify the table for an architecture add an arch variable to the pmu_events_map.
--- tools/perf/pmu-events/empty-pmu-events.c | 2 + tools/perf/pmu-events/jevents.py | 70 +++++++++++++++--------- tools/perf/pmu-events/pmu-events.h | 1 + tools/perf/tests/pmu-events.c | 3 +- 4 files changed, 47 insertions(+), 29 deletions(-)
tools/perf/pmu-events/empty-pmu-events.c +2 -0
@@ -110,12 +110,14 @@ static const struct pmu_event pme_test_soc_cpu[] = {const struct pmu_events_map pmu_events_map[] = { { + .arch ="testarch" , .cpuid ="testcpu" , .version ="v1" , .type ="core" , .table = pme_test_soc_cpu, }, { + .arch =0 , .cpuid =0 , .version =0 , .type =0 ,
tools/perf/pmu-events/jevents.py +43 -27
96 lines not on this page · raw message
tools/perf/pmu-events/pmu-events.h +1 -0
@@ -38,6 +38,7 @@ struct pmu_event {* The cpuid can contain any character other than the comma. */ struct pmu_events_map { +const char *arch;const char *cpuid;const char *version;const char *type;/* core, uncore etc */
tools/perf/tests/pmu-events.c +1 -2
@@ -864,8 +864,7 @@ static void expr_failure(const char *msg, const struct pmu_events_map *map, const struct pmu_event *pe) { - pr_debug("%s for map %s %s %s\n", - msg, map->cpuid, map->version, map->type); + pr_debug("%s for map %s %s\n", msg, map->arch, map->cpuid);pr_debug ("On metric %s\n" , pe->metric_name);pr_debug ("On expression %s\n" , pe->metric_expr); }
Signature · 1 line
2.37.1.595.g718a3a8f04-goog
Ian Rogers
reviewerPATCH v5 2/14 perf jevents: Remove the type/version variables
In reply to Ian Rogerspmu_events_map has a type variable that is always initialized to "core" and a version variable that is never read. Remove these from the API as it is straightforward to add them back when necessary.
--- tools/perf/pmu-events/empty-pmu-events.c | 4 ---- tools/perf/pmu-events/jevents.py | 6 ------ tools/perf/pmu-events/pmu-events.h | 2 -- tools/perf/tests/expand-cgroup.c | 2 -- tools/perf/tests/parse-metric.c | 2 -- 5 files changed, 16 deletions(-)
tools/perf/pmu-events/empty-pmu-events.c +0 -4
@@ -112,15 +112,11 @@ const struct pmu_events_map pmu_events_map[] = { { .arch = "testarch", .cpuid = "testcpu", - .version = "v1", - .type = "core", .table = pme_test_soc_cpu, }, { .arch = 0, .cpuid = 0, - .version = 0, - .type = 0, .table = 0, }, };
tools/perf/pmu-events/jevents.py +0 -6
@@ -313,8 +313,6 @@ def print_mapping_table(archs: Sequence[str]) -> None: _args.output_file.write("""{ \t.arch = "testarch", \t.cpuid = "testcpu", -\t.version = "v1", -\t.type = "core", \t.table = pme_test_soc_cpu, }, """)
@@ -330,8 +328,6 @@ def print_mapping_table(archs: Sequence[str]) -> None: _args.output_file.write(f"""{{ \t.arch = "{arch}", \t.cpuid = "{cpuid}", -\t.version = "{row[1]}", -\t.type = "{row[3]}", \t.table = {tblname} }}, """)
@@ -340,8 +336,6 @@ def print_mapping_table(archs: Sequence[str]) -> None: _args.output_file.write("""{ \t.arch = 0, \t.cpuid = 0, -\t.version = 0, -\t.type = 0, \t.table = 0, } };
tools/perf/pmu-events/pmu-events.h +0 -2
@@ -40,8 +40,6 @@ struct pmu_event {struct pmu_events_map {const char *arch;const char *cpuid; -const char *version; -const char *type;/* core, uncore etc */ const struct pmu_event *table; };
tools/perf/tests/expand-cgroup.c +0 -2
@@ -197,8 +197,6 @@ static int expand_metric_events(void) };const struct pmu_events_map ev_map = { .cpuid ="test" , - .version ="1" , - .type ="core" , .table = pme_test, };
tools/perf/tests/parse-metric.c +0 -2
@@ -81,8 +81,6 @@ static struct pmu_event pme_test[] = {static const struct pmu_events_map map = { .cpuid ="test" , - .version ="1" , - .type ="core" , .table = pme_test, };
Signature · 1 line
2.37.1.595.g718a3a8f04-goog
Ian Rogers
reviewerPATCH v5 3/14 perf jevents: Provide path to json file on error
In reply to Ian RogersIf a JSONDecoderError or similar is raised then it is useful to know the path. Print this and then raise the exception agan.
--- tools/perf/pmu-events/jevents.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
tools/perf/pmu-events/jevents.py +6 -1
@@ -225,7 +225,12 @@ class JsonEvent: def read_json_events(path: str) -> Sequence[JsonEvent]: """Read json events from the specified file.""" - return json.load(open(path), object_hook=lambda d: JsonEvent(d)) + + try: + return json.load(open(path), object_hook=lambda d: JsonEvent(d)) + except BaseException as err: + print(f"Exception processing {path}") + raise def preprocess_arch_std_files(archpath: str) -> None:
Signature · 1 line
2.37.1.595.g718a3a8f04-goog
Ian Rogers
reviewerPATCH v5 4/14 perf jevents: Sort json files entries
In reply to Ian RogersSort the json files entries on conversion to C. The sort order tries to replicated cmp_sevent from pmu.c so that the input there is already sorted except for sysfs events. Specifically, the sort order is given by the tuple:
(not j.desc is None, fix_none(j.topic), fix_none(j.name), fix_none(j.pmu), fix_none(j.metric_name))
which is putting events with descriptions and topics before those without, then sorting by name, then pmu and finally metric_name
Add the topic to JsonEvent on reading to simplify. Remove an unnecessary lambda in the json reading.
--- tools/perf/pmu-events/jevents.py | 48 +++++++++++++++++++++++--------- 1 file changed, 35 insertions(+), 13 deletions(-)
tools/perf/pmu-events/jevents.py +35 -13
@@ -18,6 +18,8 @@ _sys_event_tables = [] _arch_std_events = {} # Track whether an events table is currently being defined and needs closing. _close_table = False +# Events to write out when the table is closed +_pending_events = [] def removesuffix(s: str, suffix: str) -> str:
@@ -128,6 +130,7 @@ class JsonEvent: eventcode |= int(jd['ExtSel']) << 8 configcode = int(jd['ConfigCode'], 0) if 'ConfigCode' in jd else None self.name = jd['EventName'].lower() if 'EventName' in jd else None + self.topic = '' self.compat = jd.get('Compat') self.desc = fixdesc(jd.get('BriefDescription')) self.long_desc = fixdesc(jd.get('PublicDescription'))
@@ -200,7 +203,7 @@ class JsonEvent: s += f'\t{attr} = {value},\n' return s + '}' - def to_c_string(self, topic_local: str) -> str: + def to_c_string(self) -> str: """Representation of the event as a C struct initializer.""" def attr_string(attr: str, value: str) -> str:
@@ -212,25 +215,27 @@ class JsonEvent: return attr_string(attr, getattr(self, attr)) s = '{\n' - s += f'\t.topic = "{topic_local}",\n' for attr in [ 'aggr_mode', 'compat', 'deprecated', 'desc', 'event', 'long_desc', 'metric_constraint', 'metric_expr', 'metric_group', 'metric_name', - 'name', 'perpkg', 'pmu', 'unit' + 'name', 'perpkg', 'pmu', 'topic', 'unit' ]: s += str_if_present(self, attr) s += '},\n' return s -def read_json_events(path: str) -> Sequence[JsonEvent]: +def read_json_events(path: str, topic: str) -> Sequence[JsonEvent]: """Read json events from the specified file.""" try: - return json.load(open(path), object_hook=lambda d: JsonEvent(d)) + result = json.load(open(path), object_hook=JsonEvent) except BaseException as err: print(f"Exception processing {path}") raise + for event in result: + event.topic = topic + return result def preprocess_arch_std_files(archpath: str) -> None:
@@ -238,7 +243,7 @@ def preprocess_arch_std_files(archpath: str) -> None: global _arch_std_events for item in os.scandir(archpath): if item.is_file() and item.name.endswith('.json'): - for event in read_json_events(item.path): + for event in read_json_events(item.path, topic=''): if event.name: _arch_std_events[event.name.lower()] = event
@@ -252,19 +257,36 @@ def print_events_table_prefix(tblname: str) -> None: _close_table = True -def print_events_table_entries(item: os.DirEntry, topic: str) -> None: - """Create contents of an events table.""" +def add_events_table_entries(item: os.DirEntry, topic: str) -> None: + """Add contents of file to _pending_events table.""" if not _close_table: raise IOError('Table entries missing prefix') - for event in read_json_events(item.path): - _args.output_file.write(event.to_c_string(topic)) + for e in read_json_events(item.path, topic): + _pending_events.append(e) def print_events_table_suffix() -> None: """Optionally close events table.""" + + def event_cmp_key(j: JsonEvent): + def fix_none(s: str): + if s is None: + return '' + return s + + return (not j.desc is None, fix_none(j.topic), fix_none(j.name), fix_none(j.pmu), + fix_none(j.metric_name)) + global _close_table - if _close_table: - _args.output_file.write("""{ + if not _close_table: + return + + global _pending_events + for event in sorted(_pending_events, key=event_cmp_key): + _args.output_file.write(event.to_c_string()) + _pending_events = [] + + _args.output_file.write("""{ \t.name = 0, \t.event = 0, \t.desc = 0,
@@ -307,7 +329,7 @@ def process_one_file(parents: Sequence[str], item: os.DirEntry) -> None: if not item.is_file() or not item.name.endswith('.json'): return - print_events_table_entries(item, get_topic(item.name)) + add_events_table_entries(item, get_topic(item.name)) def print_mapping_table(archs: Sequence[str]) -> None:
Signature · 1 line
2.37.1.595.g718a3a8f04-goog
Ian Rogers
reviewerPATCH v5 5/14 perf pmu-events: Hide pmu_sys_event_tables
In reply to Ian RogersMove usage of the table to pmu-events.c so it may be hidden. By abstracting the table the implementation can later be changed.
--- tools/perf/pmu-events/empty-pmu-events.c | 37 ++++++++++++++++++- tools/perf/pmu-events/jevents.py | 45 ++++++++++++++++++++++-- tools/perf/pmu-events/pmu-events.h | 11 +++--- tools/perf/tests/pmu-events.c | 14 +------- tools/perf/util/pmu.c | 27 -------------- tools/perf/util/pmu.h | 2 -- 6 files changed, 84 insertions(+), 52 deletions(-)
tools/perf/pmu-events/empty-pmu-events.c +36 -1
@@ -6,6 +6,8 @@* The test cpu/soc is provided for testing. */ #include "pmu-events/pmu-events.h" +#include <string.h> +#include <stddef.h> static const struct pmu_event pme_test_soc_cpu[] = { {
@@ -145,7 +147,12 @@ static const struct pmu_event pme_test_soc_sys[] = { }, }; -const struct pmu_sys_events pmu_sys_event_tables[] = { +struct pmu_sys_events { +const char *name; +const struct pmu_event *table; +}; + +static const struct pmu_sys_events pmu_sys_event_tables[] = { { .table = pme_test_soc_sys, .name ="pme_test_soc_sys" ,
@@ -154,3 +161,31 @@ const struct pmu_sys_events pmu_sys_event_tables[] = { .table = 0 }, }; + +const struct pmu_event *find_sys_events_table (const char *name) +{ +for (const struct pmu_sys_events *tables = &pmu_sys_event_tables[0 ]; + tables->name; + tables++) { +if (!strcmp (tables->name, name)) +return tables->table; + } +return NULL ; +} + +int pmu_for_each_sys_event (pmu_event_iter_fn fn,void *data) +{ +for (const struct pmu_sys_events *tables = &pmu_sys_event_tables[0 ]; + tables->name; + tables++) { +for (const struct pmu_event *pe = &tables->table[0 ]; + pe->name || pe->metric_group || pe->metric_name; + pe++) { +int ret =fn (pe, data); + +if (ret) +return ret; + } + } +return 0 ; +}
tools/perf/pmu-events/jevents.py +42 -3
63 lines not on this page · raw message
tools/perf/pmu-events/pmu-events.h +5 -6
@@ -43,16 +43,15 @@ struct pmu_events_map { const struct pmu_event *table; }; -struct pmu_sys_events { -const char *name; -const struct pmu_event *table; -}; -/* * Global table mapping each known CPU for the architecture to its * table of PMU events. */ extern const struct pmu_events_map pmu_events_map[]; -extern const struct pmu_sys_events pmu_sys_event_tables[]; + +const struct pmu_event *find_sys_events_table (const char *name); + +typedef int (*pmu_event_iter_fn )(const struct pmu_event *pe,void *data); +int pmu_for_each_sys_event (pmu_event_iter_fn fn,void *data);#endif
tools/perf/tests/pmu-events.c +1 -13
@@ -286,18 +286,6 @@ static const struct pmu_events_map *__test_pmu_get_events_map(void)return NULL ; } -static const struct pmu_event *__test_pmu_get_sys_events_table (void ) -{ -const struct pmu_sys_events *tables = &pmu_sys_event_tables[0 ]; - -for ( ; tables->name; tables++) { -if (!strcmp ("pme_test_soc_sys" , tables->name)) -return tables->table; - } - -return NULL ; -} -static int compare_pmu_events (const struct pmu_event *e1,const struct pmu_event *e2) {if (!is_same (e1->name, e2->name)) {
@@ -451,7 +439,7 @@ static int compare_alias_to_test_event(struct perf_pmu_alias *alias,static int test__pmu_event_table (struct test_suite *test__maybe_unused ,int subtest__maybe_unused ) { -const struct pmu_event *sys_event_tables =__test_pmu_get_sys_events_table (); +const struct pmu_event *sys_event_tables =find_sys_events_table ("pme_test_soc_sys" );const struct pmu_events_map *map =__test_pmu_get_events_map ();const struct pmu_event *table;int map_events =0 , expected_events;
tools/perf/util/pmu.c +0 -27
@@ -868,33 +868,6 @@ static void pmu_add_cpu_aliases(struct list_head *head, struct perf_pmu *pmu)pmu_add_cpu_aliases_map (head, pmu, map); } -void pmu_for_each_sys_event (pmu_sys_event_iter_fn fn,void *data) -{ -int i =0 ; - -while (1 ) { -const struct pmu_sys_events *event_table; -int j =0 ; - - event_table = &pmu_sys_event_tables[i++]; - -if (!event_table->table) -break ; - -while (1 ) { -const struct pmu_event *pe = &event_table->table[j++]; -int ret; - -if (!pe->name && !pe->metric_group && !pe->metric_name) -break ; - - ret =fn (pe, data); -if (ret) -break ; - } - } -} -struct pmu_sys_event_iter_data {struct list_head *head;struct perf_pmu *pmu;
tools/perf/util/pmu.h +0 -2
@@ -133,8 +133,6 @@ const struct pmu_events_map *pmu_events_map__find(void);bool pmu_uncore_alias_match (const char *pmu_name,const char *name);void perf_pmu_free_alias (struct perf_pmu_alias *alias); -typedef int (*pmu_sys_event_iter_fn )(const struct pmu_event *pe,void *data); -void pmu_for_each_sys_event (pmu_sys_event_iter_fn fn,void *data);int perf_pmu__convert_scale (const char *scale,char **end,double *sval);int perf_pmu__caps_parse (struct perf_pmu *pmu);
Signature · 1 line
2.37.1.595.g718a3a8f04-goog
Ian Rogers
reviewerPATCH v5 6/14 perf pmu-events: Avoid passing pmu_events_map
In reply to Ian RogersPreparation for hiding pmu_events_map as an implementation detail. While the map is passed, the table of events is all that is normally wanted.
While modifying the function's types, rename pmu_events_map__find to pmu_events_table__find to match later encapsulation. Similarly rename pmu_add_cpu_aliases_map to pmu_add_cpu_aliases_table.
--- tools/perf/arch/arm64/util/pmu.c | 4 +- tools/perf/tests/expand-cgroup.c | 6 +-- tools/perf/tests/parse-metric.c | 7 +-- tools/perf/tests/pmu-events.c | 63 +++++++++++------------- tools/perf/util/metricgroup.c | 82 +++++++++++++++---------------- tools/perf/util/metricgroup.h | 4 +- tools/perf/util/pmu.c | 35 ++++++------- tools/perf/util/pmu.h | 8 +-- tools/perf/util/s390-sample-raw.c | 12 ++--- 9 files changed, 101 insertions(+), 120 deletions(-)
tools/perf/arch/arm64/util/pmu.c +2 -2
@@ -3,7 +3,7 @@#include "../../../util/cpumap.h" #include "../../../util/pmu.h" -const struct pmu_events_map *pmu_events_map__find (void ) +const struct pmu_event *pmu_events_table__find (void ) {struct perf_pmu *pmu =NULL ;
@@ -18,7 +18,7 @@ const struct pmu_events_map *pmu_events_map__find(void)if (pmu->cpus->nr !=cpu__max_cpu ().cpu)return NULL ; -return perf_pmu__find_map (pmu); +return perf_pmu__find_table (pmu); }return NULL ;
tools/perf/tests/expand-cgroup.c +1 -5
@@ -195,16 +195,12 @@ static int expand_metric_events(void) .metric_name = NULL, }, }; -const struct pmu_events_map ev_map = { - .cpuid ="test" , - .table = pme_test, - }; evlist =evlist__new ();TEST_ASSERT_VAL ("failed to get evlist" , evlist);rblist__init (&metric_events); - ret =metricgroup__parse_groups_test (evlist, &ev_map, metric_str, + ret =metricgroup__parse_groups_test (evlist, pme_test, metric_str, false, false, &metric_events);if (ret <0 ) {pr_debug ("failed to parse '%s' metric\n" , metric_str);
tools/perf/tests/parse-metric.c +1 -6
@@ -79,11 +79,6 @@ static struct pmu_event pme_test[] = { } }; -static const struct pmu_events_map map = { - .cpuid ="test" , - .table = pme_test, -}; -struct value {const char *event;u64 val;
@@ -166,7 +161,7 @@ static int __compute_metric(const char *name, struct value *vals,runtime_stat__init (&st);/* Parse the metric into metric_events list. */ - err =metricgroup__parse_groups_test (evlist, &map, name, + err =metricgroup__parse_groups_test (evlist, pme_test, name, false, false, &metric_events);if (err)
tools/perf/tests/pmu-events.c +28 -35
173 lines not on this page · raw message
tools/perf/util/metricgroup.c +39 -43
270 lines not on this page · raw message
tools/perf/util/metricgroup.h +2 -2
@@ -71,9 +71,9 @@ int metricgroup__parse_groups(const struct option *opt,bool metric_no_merge, struct rblist *metric_events);const struct pmu_event *metricgroup__find_metric (const char *metric, -const struct pmu_events_map *map); +const struct pmu_event *table); int metricgroup__parse_groups_test(struct evlist *evlist, - const struct pmu_events_map *map, + const struct pmu_event *table, const char *str, bool metric_no_group, bool metric_no_merge,
tools/perf/util/pmu.c +18 -17
@@ -710,9 +710,9 @@ static char *perf_pmu__getcpuid(struct perf_pmu *pmu)return cpuid; } -const struct pmu_events_map *perf_pmu__find_map (struct perf_pmu *pmu) +const struct pmu_event *perf_pmu__find_table (struct perf_pmu *pmu) { -const struct pmu_events_map *map; +const struct pmu_event *table =NULL ;char *cpuid =perf_pmu__getcpuid (pmu);int i;
@@ -724,22 +724,23 @@ const struct pmu_events_map *perf_pmu__find_map(struct perf_pmu *pmu) i =0 ;for (;;) { - map = &pmu_events_map[i++]; -if (!map->table) { - map =NULL ; +const struct pmu_events_map *map = &pmu_events_map[i++]; + +if (!map->table)break ; - } -if (!strcmp_cpuid_str (map->cpuid, cpuid)) +if (!strcmp_cpuid_str (map->cpuid, cpuid)) { + table = map->table;break ; + } }free (cpuid); -return map; +return table; } -const struct pmu_events_map *__weak pmu_events_map__find (void ) +__weak const struct pmu_event *pmu_events_table__find (void ) { -return perf_pmu__find_map (NULL ); +return perf_pmu__find_table (NULL ); }/*
@@ -823,8 +824,8 @@ bool pmu_uncore_alias_match(const char *pmu_name, const char *name)* to the current running CPU. Then, add all PMU events from that table * as aliases. */ -void pmu_add_cpu_aliases_map (struct list_head *head,struct perf_pmu *pmu, -const struct pmu_events_map *map) +void pmu_add_cpu_aliases_table (struct list_head *head,struct perf_pmu *pmu, +const struct pmu_event *table) {int i;const char *name = pmu->name;
@@ -834,7 +835,7 @@ void pmu_add_cpu_aliases_map(struct list_head *head, struct perf_pmu *pmu, i =0 ;while (1 ) {const char *cpu_name =is_arm_pmu_core (name) ? name :"cpu" ; -const struct pmu_event *pe = &map->table[i++]; +const struct pmu_event *pe = &table[i++];const char *pname = pe->pmu ? pe->pmu : cpu_name;if (!pe->name) {
@@ -859,13 +860,13 @@ void pmu_add_cpu_aliases_map(struct list_head *head, struct perf_pmu *pmu,static void pmu_add_cpu_aliases (struct list_head *head,struct perf_pmu *pmu) { -const struct pmu_events_map *map; +const struct pmu_event *table; - map =perf_pmu__find_map (pmu); -if (!map) + table =perf_pmu__find_table (pmu); +if (!table)return ; -pmu_add_cpu_aliases_map (head, pmu, map); +pmu_add_cpu_aliases_table (head, pmu, table); }struct pmu_sys_event_iter_data {
tools/perf/util/pmu.h +4 -4
@@ -125,11 +125,11 @@ int perf_pmu__scan_file(struct perf_pmu *pmu, const char *name, const char *fmt,int perf_pmu__test (void );struct perf_event_attr *perf_pmu__get_default_config (struct perf_pmu *pmu); -void pmu_add_cpu_aliases_map (struct list_head *head,struct perf_pmu *pmu, -const struct pmu_events_map *map); +void pmu_add_cpu_aliases_table (struct list_head *head,struct perf_pmu *pmu, +const struct pmu_event *table); -const struct pmu_events_map *perf_pmu__find_map (struct perf_pmu *pmu); -const struct pmu_events_map *pmu_events_map__find (void ); +const struct pmu_event *perf_pmu__find_table (struct perf_pmu *pmu); +const struct pmu_event *pmu_events_table__find (void );bool pmu_uncore_alias_match (const char *pmu_name,const char *name);void perf_pmu_free_alias (struct perf_pmu_alias *alias);
tools/perf/util/s390-sample-raw.c +6 -6
@@ -135,12 +135,12 @@ static int get_counterset_start(int setnr)* the name of this counter. * If no match is found a NULL pointer is returned. */ -static const char *get_counter_name (int set,int nr,const struct pmu_events_map *map) +static const char *get_counter_name (int set,int nr,const struct pmu_event *table) {int rc, event_nr, wanted =get_counterset_start (set) + nr; -if (map) { -const struct pmu_event *evp = map->table; +if (table) { +const struct pmu_event *evp = table;for (; evp->name || evp->event || evp->desc; ++evp) {if (evp->name ==NULL || evp->event ==NULL )
@@ -159,10 +159,10 @@ static void s390_cpumcfdg_dump(struct perf_sample *sample)unsigned char *buf = sample->raw_data;const char *color =PERF_COLOR_BLUE ;struct cf_ctrset_entry *cep, ce; -const struct pmu_events_map *map; +const struct pmu_event *table;u64 *p; - map =pmu_events_map__find (); + table =pmu_events_table__find ();while (offset < len) { cep = (struct cf_ctrset_entry *)(buf + offset);
@@ -180,7 +180,7 @@ static void s390_cpumcfdg_dump(struct perf_sample *sample)color_fprintf (stdout, color," [%#08zx] Counterset:%d" " Counters:%d\n" , offset, ce.set, ce.ctr);for (i =0 , p = (u64 *)(cep +1 ); i < ce.ctr; ++i, ++p) { -const char *ev_name =get_counter_name (ce.set, i, map); +const char *ev_name =get_counter_name (ce.set, i, table); color_fprintf(stdout, color, "\tCounter:%03d %s Value:%#018lx\n", i,
Signature · 1 line
2.37.1.595.g718a3a8f04-goog
Ian Rogers
reviewerPATCH v5 7/14 perf pmu-events: Hide pmu_events_map
In reply to Ian RogersMove usage of the table to pmu-events.c so it may be hidden. By abstracting the table the implementation can later be changed.
--- tools/perf/pmu-events/empty-pmu-events.c | 81 ++++++++- tools/perf/pmu-events/jevents.py | 81 ++++++++- tools/perf/pmu-events/pmu-events.h | 29 +-- tools/perf/tests/pmu-events.c | 218 +++++++++++------------ tools/perf/util/metricgroup.c | 15 +- tools/perf/util/pmu.c | 34 +--- tools/perf/util/pmu.h | 2 +- 7 files changed, 280 insertions(+), 180 deletions(-)
tools/perf/pmu-events/empty-pmu-events.c +79 -2
@@ -6,6 +6,8 @@* The test cpu/soc is provided for testing. */ #include "pmu-events/pmu-events.h" +#include "util/header.h" +#include "util/pmu.h" #include <string.h> #include <stddef.h>
@@ -110,7 +112,26 @@ static const struct pmu_event pme_test_soc_cpu[] = { }, }; -const struct pmu_events_map pmu_events_map[] = { + +/* +* Map a CPU to its table of PMU events. The CPU is identified by the +* cpuid field, which is an arch-specific identifier for the CPU. +* The identifier specified in tools/perf/pmu-events/arch/xxx/mapfile +* must match the get_cpuid_str() in tools/perf/arch/xxx/util/header.c) +* +* The cpuid can contain any character other than the comma. +*/ +struct pmu_events_map { +const char *arch; +const char *cpuid; +const struct pmu_event *table; +}; + +/* +* Global table mapping each known CPU for the architecture to its +* table of PMU events. +*/ +static const struct pmu_events_map pmu_events_map[] = { { .arch ="testarch" , .cpuid ="testcpu" ,
@@ -162,6 +183,62 @@ static const struct pmu_sys_events pmu_sys_event_tables[] = { }, }; +const struct pmu_event *perf_pmu__find_table (struct perf_pmu *pmu) +{ +const struct pmu_event *table =NULL ; +char *cpuid =perf_pmu__getcpuid (pmu); +int i; + +/* on some platforms which uses cpus map, cpuid can be NULL for +* PMUs other than CORE PMUs. +*/ +if (!cpuid) +return NULL ; + + i =0 ; +for (;;) { +const struct pmu_events_map *map = &pmu_events_map[i++]; + +if (!map->table) +break ; + +if (!strcmp_cpuid_str (map->cpuid, cpuid)) { + table = map->table; +break ; + } + } +free (cpuid); +return table; +} + +const struct pmu_event *find_core_events_table (const char *arch,const char *cpuid) +{ +for (const struct pmu_events_map *tables = &pmu_events_map[0 ]; + tables->table; + tables++) { +if (!strcmp (tables->arch, arch) && !strcmp_cpuid_str (tables->cpuid, cpuid)) +return tables->table; + } +return NULL ; +} + +int pmu_for_each_core_event (pmu_event_iter_fn fn,void *data) +{ +for (const struct pmu_events_map *tables = &pmu_events_map[0 ]; + tables->table; + tables++) { +for (const struct pmu_event *pe = &tables->table[0 ]; + pe->name || pe->metric_group || pe->metric_name; + pe++) { +int ret =fn (pe, &tables->table[0 ], data); + +if (ret) +return ret; + } + } +return 0 ; +} +const struct pmu_event *find_sys_events_table (const char *name) { for (const struct pmu_sys_events *tables = &pmu_sys_event_tables[0];
@@ -181,7 +258,7 @@ int pmu_for_each_sys_event(pmu_event_iter_fn fn, void *data)for (const struct pmu_event *pe = &tables->table[0 ]; pe->name || pe->metric_group || pe->metric_name; pe++) { -int ret =fn (pe, data); +int ret =fn (pe, &tables->table[0 ], data);if (ret)return ret;
tools/perf/pmu-events/jevents.py +79 -2
105 lines not on this page · raw message
tools/perf/pmu-events/pmu-events.h +8 -21
@@ -2,6 +2,8 @@#ifndef PMU_EVENTS_H #define PMU_EVENTS_H +struct perf_pmu ; +enum aggr_mode_class { PerChip =1 , PerCore
@@ -28,30 +30,15 @@ struct pmu_event { const char *metric_constraint; }; -/* -* -* Map a CPU to its table of PMU events. The CPU is identified by the -* cpuid field, which is an arch-specific identifier for the CPU. -* The identifier specified in tools/perf/pmu-events/arch/xxx/mapfile -* must match the get_cpuid_str() in tools/perf/arch/xxx/util/header.c) -* -* The cpuid can contain any character other than the comma. -*/ -struct pmu_events_map { -const char *arch; -const char *cpuid; -const struct pmu_event *table; -}; +typedef int (*pmu_event_iter_fn )(const struct pmu_event *pe, +const struct pmu_event *table, +void *data); -/* -* Global table mapping each known CPU for the architecture to its -* table of PMU events. -*/ -extern const struct pmu_events_map pmu_events_map[]; +const struct pmu_event *perf_pmu__find_table (struct perf_pmu *pmu); +const struct pmu_event *find_core_events_table (const char *arch,const char *cpuid); +int pmu_for_each_core_event (pmu_event_iter_fn fn,void *data);const struct pmu_event *find_sys_events_table (const char *name); - -typedef int (*pmu_event_iter_fn )(const struct pmu_event *pe,void *data);int pmu_for_each_sys_event (pmu_event_iter_fn fn,void *data);#endif
tools/perf/tests/pmu-events.c +100 -118
286 lines not on this page · raw message
tools/perf/util/metricgroup.c +9 -6
@@ -502,14 +502,14 @@ struct metricgroup_print_sys_idata {bool details; }; -typedef int (*metricgroup_sys_event_iter_fn )(const struct pmu_event *pe,void *); -struct metricgroup_iter_data { -metricgroup_sys_event_iter_fn fn; +pmu_event_iter_fn fn;void *data; }; -static int metricgroup__sys_event_iter (const struct pmu_event *pe,void *data) +static int metricgroup__sys_event_iter (const struct pmu_event *pe, +const struct pmu_event *table__maybe_unused , +void *data) {struct metricgroup_iter_data *d = data;struct perf_pmu *pmu =NULL ;
@@ -522,13 +522,15 @@ static int metricgroup__sys_event_iter(const struct pmu_event *pe, void *data)if (!pmu->id ||strcmp (pmu->id, pe->compat))continue ; -return d->fn (pe, d->data); +return d->fn (pe, table, d->data); }return 0 ; } -static int metricgroup__print_sys_event_iter (const struct pmu_event *pe,void *data) +static int metricgroup__print_sys_event_iter (const struct pmu_event *pe, +const struct pmu_event *table__maybe_unused , +void *data) {struct metricgroup_print_sys_idata *d = data;
@@ -1101,6 +1103,7 @@ static int add_metric(struct list_head *metric_list, }static int metricgroup__add_metric_sys_event_iter (const struct pmu_event *pe, +const struct pmu_event *table__maybe_unused ,void *data) {struct metricgroup_add_iter_data *d = data;
tools/perf/util/pmu.c +4 -30
@@ -690,7 +690,7 @@ static int is_arm_pmu_core(const char *name)return file_available (path); } -static char *perf_pmu__getcpuid (struct perf_pmu *pmu) +char *perf_pmu__getcpuid (struct perf_pmu *pmu) {char *cpuid;static bool printed;
@@ -710,34 +710,6 @@ static char *perf_pmu__getcpuid(struct perf_pmu *pmu)return cpuid; } -const struct pmu_event *perf_pmu__find_table (struct perf_pmu *pmu) -{ -const struct pmu_event *table =NULL ; -char *cpuid =perf_pmu__getcpuid (pmu); -int i; - -/* on some platforms which uses cpus map, cpuid can be NULL for -* PMUs other than CORE PMUs. -*/ -if (!cpuid) -return NULL ; - - i =0 ; -for (;;) { -const struct pmu_events_map *map = &pmu_events_map[i++]; - -if (!map->table) -break ; - -if (!strcmp_cpuid_str (map->cpuid, cpuid)) { - table = map->table; -break ; - } - } -free (cpuid); -return table; -} -__weak const struct pmu_event *pmu_events_table__find (void ) {return perf_pmu__find_table (NULL );
@@ -874,7 +846,9 @@ struct pmu_sys_event_iter_data {struct perf_pmu *pmu; }; -static int pmu_add_sys_aliases_iter_fn (const struct pmu_event *pe,void *data) +static int pmu_add_sys_aliases_iter_fn (const struct pmu_event *pe, +const struct pmu_event *table__maybe_unused , +void *data) {struct pmu_sys_event_iter_data *idata = data;struct perf_pmu *pmu = idata->pmu;
tools/perf/util/pmu.h +1 -1
@@ -128,7 +128,7 @@ struct perf_event_attr *perf_pmu__get_default_config(struct perf_pmu *pmu);void pmu_add_cpu_aliases_table (struct list_head *head,struct perf_pmu *pmu,const struct pmu_event *table); -const struct pmu_event *perf_pmu__find_table (struct perf_pmu *pmu); +char *perf_pmu__getcpuid (struct perf_pmu *pmu);const struct pmu_event *pmu_events_table__find (void );bool pmu_uncore_alias_match (const char *pmu_name,const char *name);void perf_pmu_free_alias (struct perf_pmu_alias *alias);
Signature · 1 line
2.37.1.595.g718a3a8f04-goog
Ian Rogers
reviewerPATCH v5 8/14 perf test: Use full metric resolution
In reply to Ian RogersThe simple metric resolution doesn't handle recursion properly, switch to use the full resolution as with the parse-metric tests which also increases coverage. Don't set the values for the metric backward as failures to generate a result are ignored.
--- tools/perf/tests/pmu-events.c | 222 ++++++++++++---------------------- 1 file changed, 77 insertions(+), 145 deletions(-)
tools/perf/tests/pmu-events.c +77 -145
@@ -9,10 +9,12 @@#include <linux/zalloc.h> #include "debug.h" #include "../pmu-events/pmu-events.h" +#include <perf/evlist.h> #include "util/evlist.h" #include "util/expr.h" #include "util/parse-events.h" #include "metricgroup.h" +#include "stat.h" struct perf_pmu_test_event {/* used for matching against events from generated pmu-events.c */
@@ -801,27 +803,6 @@ static int check_parse_id(const char *id, struct parse_events_error *error,return ret; } -static int check_parse_cpu (const char *id,bool same_cpu,const struct pmu_event *pe) -{ -struct parse_events_error error; -int ret; - -parse_events_error__init (&error); - ret =check_parse_id (id, &error,NULL ); -if (ret && same_cpu) { -pr_warning ("Parse event failed metric '%s' id '%s' expr '%s'\n" , - pe->metric_name, id, pe->metric_expr); -pr_warning ("Error string '%s' help '%s'\n" , error.str, - error.help); - }else if (ret) { -pr_debug3 ("Parse event failed, but for an event that may not be supported by this CPU.\nid '%s' metric '%s' expr '%s'\n" , - id, pe->metric_name, pe->metric_expr); - ret =0 ; - } -parse_events_error__exit (&error); -return ret; -} -static int check_parse_fake (const char *id) {struct parse_events_error error;
@@ -838,160 +819,111 @@ struct metric {struct metric_ref metric_ref; }; -static int resolve_metric_simple (struct expr_parse_ctx *pctx, -struct list_head *compound_list, -const struct pmu_event *map, -const char *metric_name) -{ -struct hashmap_entry *cur, *cur_tmp; -struct metric *metric, *tmp; -size_t bkt; -bool all; -int rc; - -do { - all = true; -hashmap__for_each_entry_safe (pctx->ids, cur, cur_tmp, bkt) { -struct metric_ref *ref; -const struct pmu_event *pe; - - pe =metricgroup__find_metric (cur->key, map); -if (!pe) -continue ; - -if (!strcmp (metric_name, (char *)cur->key)) { -pr_warning ("Recursion detected for metric %s\n" , metric_name); - rc =-1 ; - goto out_err; - } - - all = false; - -/* The metric key itself needs to go out.. */ -expr__del_id (pctx, cur->key); - - metric =malloc (sizeof (*metric)); -if (!metric) { - rc = -ENOMEM ; - goto out_err; - } - - ref = &metric->metric_ref; - ref->metric_name = pe->metric_name; - ref->metric_expr = pe->metric_expr; -list_add_tail (&metric->list, compound_list); - - rc =expr__find_ids (pe->metric_expr,NULL , pctx); -if (rc) - goto out_err; -break ;/* The hashmap has been modified, so restart */ - } - }while (!all); - -return 0 ; - -out_err: -list_for_each_entry_safe (metric, tmp, compound_list, list) -free (metric); - -return rc; - -} - -static void expr_failure (const char *msg,const struct pmu_event *pe) -{ -pr_debug ("%s\nOn metric %s\nOn expression %s\n" , msg, pe->metric_name, pe->metric_expr); -} - - -struct test__parsing_data { -const struct pmu_event *cpus_table; -struct expr_parse_ctx *ctx; -int failures; -}; -static int test__parsing_callback (const struct pmu_event *pe,const struct pmu_event *table, -void *vdata) +void *data) { -struct test__parsing_data *data = vdata; -struct metric *metric, *tmp; -struct hashmap_entry *cur; -LIST_HEAD (compound_list); -size_t bkt; +int *failures = data;int k; -double result; +struct evlist *evlist; +struct perf_cpu_map *cpus; +struct runtime_stat st; +struct evsel *evsel; +struct rblist metric_events = { + .nr_entries =0 , + }; +int err =0 ;if (!pe->metric_expr)return 0 ;pr_debug ("Found metric '%s'\n" , pe->metric_name); + (*failures)++; -expr__ctx_clear (data->ctx); -if (expr__find_ids (pe->metric_expr,NULL , data->ctx) <0 ) { -expr_failure ("Parse find ids failed" , pe); - data->failures++; -return 0 ; +/* +* We need to prepare evlist for stat mode running on CPU 0 +* because that's where all the stats are going to be created. +*/ + evlist =evlist__new (); +if (!evlist) +return -ENOMEM ; + + cpus =perf_cpu_map__new ("0" ); +if (!cpus) { +evlist__delete (evlist); +return -ENOMEM ; } -if (resolve_metric_simple (data->ctx, &compound_list, table, - pe->metric_name)) { -expr_failure ("Could not resolve metrics" , pe); - data->failures++; -return TEST_FAIL ;/* Don't tolerate errors due to severity */ +perf_evlist__set_maps (&evlist->core, cpus,NULL ); +runtime_stat__init (&st); + + err =metricgroup__parse_groups_test (evlist, table, pe->metric_name, + false, false, + &metric_events); +if (err) { +if (!strcmp (pe->metric_name,"M1" ) || !strcmp (pe->metric_name,"M2" ) || + !strcmp (pe->metric_name,"M3" )) { + (*failures)--; +pr_debug ("Expected broken metric %s skipping\n" , pe->metric_name); + err =0 ; + } + goto out_err; } + err =evlist__alloc_stats (evlist, false); +if (err) + goto out_err;/* * Add all ids with a made up value. The value may trigger divide by * zero when subtracted and so try to make them unique. */ k =1 ; -hashmap__for_each_entry (data->ctx->ids, cur, bkt) -expr__add_id_val (data->ctx,strdup (cur->key), k++); - -hashmap__for_each_entry (data->ctx->ids, cur, bkt) { -if (check_parse_cpu (cur->key, table == data->cpus_table, pe)) - data->failures++; +perf_stat__reset_shadow_stats (); +evlist__for_each_entry (evlist, evsel) { +perf_stat__update_shadow_stats (evsel, k,0 , &st); +if (!strcmp (evsel->name,"duration_time" )) +update_stats (&walltime_nsecs_stats, k); + k++; } +evlist__for_each_entry (evlist, evsel) { +struct metric_event *me =metricgroup__lookup (&metric_events, evsel, false); -list_for_each_entry_safe (metric, tmp, &compound_list, list) { -expr__add_ref (data->ctx, &metric->metric_ref); -free (metric); - } +if (me !=NULL ) { +struct metric_expr *mexp; -if (expr__parse (&result, data->ctx, pe->metric_expr)) { -/* -* Parsing failed, make numbers go from large to small which can -* resolve divide by zero issues. -*/ - k =1024 ; -hashmap__for_each_entry (data->ctx->ids, cur, bkt) -expr__add_id_val (data->ctx,strdup (cur->key), k--); -if (expr__parse (&result, data->ctx, pe->metric_expr)) { -expr_failure ("Parse failed" , pe); - data->failures++; + list_for_each_entry (mexp, &me->head, nd) { +if (strcmp (mexp->metric_name, pe->metric_name)) +continue ; +pr_debug ("Result %f\n" ,test_generic_metric (mexp,0 , &st)); + err =0 ; + (*failures)--; + goto out_err; + } } } -return 0 ; +pr_debug ("Didn't find parsed metric %s" , pe->metric_name); + err =1 ; +out_err: +if (err) +pr_debug ("Broken metric %s\n" , pe->metric_name); + +/* ... cleanup. */ +metricgroup__rblist_exit (&metric_events); +runtime_stat__exit (&st); +evlist__free_stats (evlist); +perf_cpu_map__put (cpus); +evlist__delete (evlist); +return err; }static int test__parsing (struct test_suite *test__maybe_unused ,int subtest__maybe_unused ) { -struct test__parsing_data data = { - .cpus_table =pmu_events_table__find (), - .ctx =expr__ctx_new (), - .failures =0 , - }; +int failures =0 ; -if (!data.ctx) { -pr_debug ("expr__ctx_new failed" ); -return TEST_FAIL ; - } -pmu_for_each_core_event (test__parsing_callback, &data); -pmu_for_each_sys_event (test__parsing_callback, &data); +pmu_for_each_core_event (test__parsing_callback, &failures); +pmu_for_each_sys_event (test__parsing_callback, &failures); -expr__ctx_free (data.ctx); -return data.failures ==0 ?TEST_OK :TEST_FAIL ; +return failures ==0 ?TEST_OK :TEST_FAIL ; }struct test_metric {
Signature · 1 line
2.37.1.595.g718a3a8f04-goog
Ian Rogers
reviewerPATCH v5 9/14 perf pmu-events: Move test events/metrics to json
In reply to Ian RogersMove arrays of pmu_events into the json code so that it may be regenerated and modified by the jevents.py script.
--- .../arch/test/test_soc/cpu/metrics.json | 64 +++++++++++++++++ tools/perf/pmu-events/empty-pmu-events.c | 64 +++++++++++++++++ tools/perf/tests/expand-cgroup.c | 17 +---- tools/perf/tests/parse-metric.c | 68 +------------------ 4 files changed, 132 insertions(+), 81 deletions(-) create mode 100644 tools/perf/pmu-events/arch/test/test_soc/cpu/metrics.json
tools/perf/pmu-events/arch/test/test_soc/cpu/metrics.json new file +64 -0
@@ -0,0 +1,64 @@ +[ + { + "MetricExpr": "1 / IPC", + "MetricName": "CPI" + }, + { + "MetricExpr": "inst_retired.any / cpu_clk_unhalted.thread", + "MetricName": "IPC", + "MetricGroup": "group1" + }, + { + "MetricExpr": "idq_uops_not_delivered.core / (4 * (( ( cpu_clk_unhalted.thread / 2 ) * ( 1 + cpu_clk_unhalted.one_thread_active / cpu_clk_unhalted.ref_xclk ) )))", + "MetricName": "Frontend_Bound_SMT" + }, + { + "MetricExpr": "l1d\\-loads\\-misses / inst_retired.any", + "MetricName": "dcache_miss_cpi" + }, + { + "MetricExpr": "l1i\\-loads\\-misses / inst_retired.any", + "MetricName": "icache_miss_cycles" + }, + { + "MetricExpr": "(dcache_miss_cpi + icache_miss_cycles)", + "MetricName": "cache_miss_cycles", + "MetricGroup": "group1" + }, + { + "MetricExpr": "l2_rqsts.demand_data_rd_hit + l2_rqsts.pf_hit + l2_rqsts.rfo_hit", + "MetricName": "DCache_L2_All_Hits" + }, + { + "MetricExpr": "max(l2_rqsts.all_demand_data_rd - l2_rqsts.demand_data_rd_hit, 0) + l2_rqsts.pf_miss + l2_rqsts.rfo_miss", + "MetricName": "DCache_L2_All_Miss" + }, + { + "MetricExpr": "dcache_l2_all_hits + dcache_l2_all_miss", + "MetricName": "DCache_L2_All" + }, + { + "MetricExpr": "d_ratio(dcache_l2_all_hits, dcache_l2_all)", + "MetricName": "DCache_L2_Hits" + }, + { + "MetricExpr": "d_ratio(dcache_l2_all_miss, dcache_l2_all)", + "MetricName": "DCache_L2_Misses" + }, + { + "MetricExpr": "ipc + M2", + "MetricName": "M1" + }, + { + "MetricExpr": "ipc + M1", + "MetricName": "M2" + }, + { + "MetricExpr": "1/M3", + "MetricName": "M3" + }, + { + "MetricExpr": "64 * l1d.replacement / 1000000000 / duration_time", + "MetricName": "L1D_Cache_Fill_BW" + } +]
tools/perf/pmu-events/empty-pmu-events.c +64 -0
70 lines not on this page · raw message
tools/perf/tests/expand-cgroup.c +2 -15
@@ -180,26 +180,13 @@ static int expand_metric_events(void)struct evlist *evlist;struct rblist metric_events;const char metric_str[] ="CPI" ; - -struct pmu_event pme_test[] = { - { - .metric_expr ="instructions / cycles" , - .metric_name ="IPC" , - }, - { - .metric_expr ="1 / IPC" , - .metric_name ="CPI" , - }, - { - .metric_expr =NULL , - .metric_name =NULL , - }, - }; +const struct pmu_event *pme_test; evlist =evlist__new ();TEST_ASSERT_VAL ("failed to get evlist" , evlist);rblist__init (&metric_events); + pme_test =find_core_events_table ("testarch" ,"testcpu" ); ret =metricgroup__parse_groups_test (evlist, pme_test, metric_str, false, false, &metric_events);if (ret <0 ) {
tools/perf/tests/parse-metric.c +2 -66
86 lines not on this page · raw message
Signature · 1 line
2.37.1.595.g718a3a8f04-goog
Ian Rogers
reviewerPATCH v5 10/14 perf pmu-events: Don't assume pmu_event is an array
In reply to Ian RogersCurrent code assumes that a struct pmu_event can be iterated over forward until a NULL pmu_event is encountered. This makes it difficult to refactor pmu_event. Add a loop function taking a callback function that's passed the struct pmu_event. This way the pmu_event is only needed for one element and not an entire array.
Switch existing code iterating over the pmu_event arrays to use the new loop function pmu_events_table_for_each_event.
--- tools/perf/pmu-events/empty-pmu-events.c | 34 +++-- tools/perf/pmu-events/jevents.py | 34 +++-- tools/perf/pmu-events/pmu-events.h | 3 + tools/perf/tests/pmu-events.c | 136 +++++++++-------- tools/perf/util/metricgroup.c | 181 ++++++++++++++++------- tools/perf/util/pmu.c | 65 ++++---- tools/perf/util/s390-sample-raw.c | 42 ++++-- 7 files changed, 313 insertions(+), 182 deletions(-)
tools/perf/pmu-events/empty-pmu-events.c +20 -14
@@ -247,6 +247,20 @@ static const struct pmu_sys_events pmu_sys_event_tables[] = { }, }; +int pmu_events_table_for_each_event (const struct pmu_event *table,pmu_event_iter_fn fn, +void *data) +{ +for (const struct pmu_event *pe = &table[0 ]; + pe->name || pe->metric_group || pe->metric_name; + pe++) { +int ret =fn (pe, table, data); + +if (ret) +return ret; + } +return 0 ; +} +const struct pmu_event *perf_pmu__find_table (struct perf_pmu *pmu) {const struct pmu_event *table =NULL ;
@@ -291,14 +305,10 @@ int pmu_for_each_core_event(pmu_event_iter_fn fn, void *data)for (const struct pmu_events_map *tables = &pmu_events_map[0 ]; tables->table; tables++) { -for (const struct pmu_event *pe = &tables->table[0 ]; - pe->name || pe->metric_group || pe->metric_name; - pe++) { -int ret =fn (pe, &tables->table[0 ], data); +int ret =pmu_events_table_for_each_event (tables->table, fn, data); -if (ret) -return ret; - } +if (ret) +return ret; }return 0 ; }
@@ -319,14 +329,10 @@ int pmu_for_each_sys_event(pmu_event_iter_fn fn, void *data)for (const struct pmu_sys_events *tables = &pmu_sys_event_tables[0 ]; tables->name; tables++) { -for (const struct pmu_event *pe = &tables->table[0 ]; - pe->name || pe->metric_group || pe->metric_name; - pe++) { -int ret =fn (pe, &tables->table[0 ], data); +int ret =pmu_events_table_for_each_event (tables->table, fn, data); -if (ret) -return ret; - } +if (ret) +return ret; }return 0 ; }
tools/perf/pmu-events/jevents.py +20 -14
@@ -410,6 +410,20 @@ static const struct pmu_sys_events pmu_sys_event_tables[] = { \t}, }; +int pmu_events_table_for_each_event(const struct pmu_event *table, pmu_event_iter_fn fn, + void *data) +{ + for (const struct pmu_event *pe = &table[0]; + pe->name || pe->metric_group || pe->metric_name; + pe++) { + int ret = fn(pe, table, data); + + if (ret) + return ret; + } + return 0; +} + const struct pmu_event *perf_pmu__find_table(struct perf_pmu *pmu) { const struct pmu_event *table = NULL;
@@ -453,14 +467,10 @@ int pmu_for_each_core_event(pmu_event_iter_fn fn, void *data) for (const struct pmu_events_map *tables = &pmu_events_map[0]; tables->table; tables++) { - for (const struct pmu_event *pe = &tables->table[0]; - pe->name || pe->metric_group || pe->metric_name; - pe++) { - int ret = fn(pe, &tables->table[0], data); + int ret = pmu_events_table_for_each_event(tables->table, fn, data); - if (ret) - return ret; - } + if (ret) + return ret; } return 0; }
@@ -481,14 +491,10 @@ int pmu_for_each_sys_event(pmu_event_iter_fn fn, void *data) for (const struct pmu_sys_events *tables = &pmu_sys_event_tables[0]; tables->name; tables++) { - for (const struct pmu_event *pe = &tables->table[0]; - pe->name || pe->metric_group || pe->metric_name; - pe++) { - int ret = fn(pe, &tables->table[0], data); + int ret = pmu_events_table_for_each_event(tables->table, fn, data); - if (ret) - return ret; - } + if (ret) + return ret; } return 0; }
tools/perf/pmu-events/pmu-events.h +3 -0
@@ -34,6 +34,9 @@ typedef int (*pmu_event_iter_fn)(const struct pmu_event *pe,const struct pmu_event *table, void *data); +int pmu_events_table_for_each_event (const struct pmu_event *table,pmu_event_iter_fn fn, +void *data); +const struct pmu_event *perf_pmu__find_table (struct perf_pmu *pmu);const struct pmu_event *find_core_events_table (const char *arch,const char *cpuid);int pmu_for_each_core_event (pmu_event_iter_fn fn,void *data);
tools/perf/tests/pmu-events.c +78 -58
162 lines not on this page · raw message
tools/perf/util/metricgroup.c +125 -56
245 lines not on this page · raw message
tools/perf/util/pmu.c +37 -28
78 lines not on this page · raw message
tools/perf/util/s390-sample-raw.c +30 -12
56 lines not on this page · raw message
Signature · 1 line
2.37.1.595.g718a3a8f04-goog
Ian Rogers
reviewerPATCH v5 11/14 perf pmu-events: Hide the pmu_events
In reply to Ian RogersHide that the pmu_event structs are an array with a new wrapper struct.
--- tools/perf/arch/arm64/util/pmu.c | 2 +- tools/perf/pmu-events/empty-pmu-events.c | 44 ++++++++++++---------- tools/perf/pmu-events/jevents.py | 47 +++++++++++++----------- tools/perf/pmu-events/pmu-events.h | 12 +++--- tools/perf/tests/expand-cgroup.c | 2 +- tools/perf/tests/parse-metric.c | 2 +- tools/perf/tests/pmu-events.c | 16 ++++---- tools/perf/util/metricgroup.c | 42 ++++++++++----------- tools/perf/util/metricgroup.h | 5 +-- tools/perf/util/pmu.c | 10 ++--- tools/perf/util/pmu.h | 4 +- tools/perf/util/s390-sample-raw.c | 6 +-- 12 files changed, 101 insertions(+), 91 deletions(-)
tools/perf/arch/arm64/util/pmu.c +1 -1
@@ -3,7 +3,7 @@#include "../../../util/cpumap.h" #include "../../../util/pmu.h" -const struct pmu_event *pmu_events_table__find (void ) +const struct pmu_events_table *pmu_events_table__find (void ) {struct perf_pmu *pmu =NULL ;
tools/perf/pmu-events/empty-pmu-events.c +24 -20
134 lines not on this page · raw message
tools/perf/pmu-events/jevents.py +26 -21
148 lines not on this page · raw message
tools/perf/pmu-events/pmu-events.h +7 -5
@@ -30,18 +30,20 @@ struct pmu_event { const char *metric_constraint; }; +struct pmu_events_table ; +typedef int (*pmu_event_iter_fn )(const struct pmu_event *pe, -const struct pmu_event *table, +const struct pmu_events_table *table,void *data); -int pmu_events_table_for_each_event (const struct pmu_event *table,pmu_event_iter_fn fn, +int pmu_events_table_for_each_event (const struct pmu_events_table *table,pmu_event_iter_fn fn,void *data); -const struct pmu_event *perf_pmu__find_table (struct perf_pmu *pmu); -const struct pmu_event *find_core_events_table (const char *arch,const char *cpuid); +const struct pmu_events_table *perf_pmu__find_table (struct perf_pmu *pmu); +const struct pmu_events_table *find_core_events_table (const char *arch,const char *cpuid);int pmu_for_each_core_event (pmu_event_iter_fn fn,void *data); -const struct pmu_event *find_sys_events_table (const char *name); +const struct pmu_events_table *find_sys_events_table (const char *name);int pmu_for_each_sys_event (pmu_event_iter_fn fn,void *data);#endif
tools/perf/tests/expand-cgroup.c +1 -1
@@ -180,7 +180,7 @@ static int expand_metric_events(void)struct evlist *evlist;struct rblist metric_events;const char metric_str[] ="CPI" ; -const struct pmu_event *pme_test; +const struct pmu_events_table *pme_test; evlist =evlist__new ();TEST_ASSERT_VAL ("failed to get evlist" , evlist);
tools/perf/tests/parse-metric.c +1 -1
@@ -72,7 +72,7 @@ static int __compute_metric(const char *name, struct value *vals,struct rblist metric_events = { .nr_entries =0 , }; -const struct pmu_event *pme_test; +const struct pmu_events_table *pme_test;struct perf_cpu_map *cpus;struct runtime_stat st;struct evlist *evlist;
tools/perf/tests/pmu-events.c +8 -8
@@ -424,7 +424,7 @@ static int compare_alias_to_test_event(struct perf_pmu_alias *alias, }static int test__pmu_event_table_core_callback (const struct pmu_event *pe, -const struct pmu_event *table__maybe_unused , +const struct pmu_events_table *table__maybe_unused ,void *data) {int *map_events = data;
@@ -461,7 +461,7 @@ static int test__pmu_event_table_core_callback(const struct pmu_event *pe, }static int test__pmu_event_table_sys_callback (const struct pmu_event *pe, -const struct pmu_event *table__maybe_unused , +const struct pmu_events_table *table__maybe_unused ,void *data) {int *map_events = data;
@@ -495,8 +495,8 @@ static int test__pmu_event_table_sys_callback(const struct pmu_event *pe,static int test__pmu_event_table (struct test_suite *test__maybe_unused ,int subtest__maybe_unused ) { -const struct pmu_event *sys_event_table =find_sys_events_table ("pme_test_soc_sys" ); -const struct pmu_event *table =find_core_events_table ("testarch" ,"testcpu" ); +const struct pmu_events_table *sys_event_table =find_sys_events_table ("pme_test_soc_sys" ); +const struct pmu_events_table *table =find_core_events_table ("testarch" ,"testcpu" );int map_events =0 , expected_events, err;/* ignore 3x sentinels */
@@ -544,7 +544,7 @@ static int __test_core_pmu_event_aliases(char *pmu_name, int *count)struct perf_pmu *pmu;LIST_HEAD (aliases);int res =0 ; -const struct pmu_event *table =find_core_events_table ("testarch" ,"testcpu" ); +const struct pmu_events_table *table =find_core_events_table ("testarch" ,"testcpu" );struct perf_pmu_alias *a, *tmp;if (!table)
@@ -597,7 +597,7 @@ static int __test_uncore_pmu_event_aliases(struct perf_pmu_test_pmu *test_pmu)struct perf_pmu *pmu = &test_pmu->pmu;const char *pmu_name = pmu->name;struct perf_pmu_alias *a, *tmp, *alias; -const struct pmu_event *events_table; +const struct pmu_events_table *events_table;LIST_HEAD (aliases);int res =0 ;
@@ -839,7 +839,7 @@ struct metric {struct metric_ref metric_ref; }; -static int test__parsing_callback (const struct pmu_event *pe,const struct pmu_event *table, +static int test__parsing_callback (const struct pmu_event *pe,const struct pmu_events_table *table,void *data) {int *failures = data;
@@ -1016,7 +1016,7 @@ static int metric_parse_fake(const char *str) }static int test__parsing_fake_callback (const struct pmu_event *pe, -const struct pmu_event *table__maybe_unused , +const struct pmu_events_table *table__maybe_unused ,void *data__maybe_unused ) {if (!pe->metric_expr)
tools/perf/util/metricgroup.c +21 -21
162 lines not on this page · raw message
tools/perf/util/metricgroup.h +2 -3
@@ -11,7 +11,6 @@ struct evlist;struct evsel ;struct option ;struct rblist ; -struct pmu_events_map ;struct cgroup ;/**
@@ -71,9 +70,9 @@ int metricgroup__parse_groups(const struct option *opt,bool metric_no_merge, struct rblist *metric_events);const struct pmu_event *metricgroup__find_metric (const char *metric, -const struct pmu_event *table); +const struct pmu_events_table *table); int metricgroup__parse_groups_test(struct evlist *evlist, - const struct pmu_event *table, + const struct pmu_events_table *table, const char *str, bool metric_no_group, bool metric_no_merge,
tools/perf/util/pmu.c +5 -5
@@ -710,7 +710,7 @@ char *perf_pmu__getcpuid(struct perf_pmu *pmu)return cpuid; } -__weak const struct pmu_event *pmu_events_table__find (void ) +__weak const struct pmu_events_table *pmu_events_table__find (void ) {return perf_pmu__find_table (NULL ); }
@@ -799,7 +799,7 @@ struct pmu_add_cpu_aliases_map_data { };static int pmu_add_cpu_aliases_map_callback (const struct pmu_event *pe, -const struct pmu_event *table__maybe_unused , +const struct pmu_events_table *table__maybe_unused ,void *vdata) {struct pmu_add_cpu_aliases_map_data *data = vdata;
@@ -827,7 +827,7 @@ static int pmu_add_cpu_aliases_map_callback(const struct pmu_event *pe,* as aliases. */ void pmu_add_cpu_aliases_table (struct list_head *head,struct perf_pmu *pmu, -const struct pmu_event *table) +const struct pmu_events_table *table) {struct pmu_add_cpu_aliases_map_data data = { .head = head,
@@ -841,7 +841,7 @@ void pmu_add_cpu_aliases_table(struct list_head *head, struct perf_pmu *pmu,static void pmu_add_cpu_aliases (struct list_head *head,struct perf_pmu *pmu) { -const struct pmu_event *table; +const struct pmu_events_table *table; table =perf_pmu__find_table (pmu);if (!table)
@@ -856,7 +856,7 @@ struct pmu_sys_event_iter_data { };static int pmu_add_sys_aliases_iter_fn (const struct pmu_event *pe, -const struct pmu_event *table__maybe_unused , +const struct pmu_events_table *table__maybe_unused ,void *data) {struct pmu_sys_event_iter_data *idata = data;
tools/perf/util/pmu.h +2 -2
@@ -126,10 +126,10 @@ int perf_pmu__test(void);struct perf_event_attr *perf_pmu__get_default_config (struct perf_pmu *pmu);void pmu_add_cpu_aliases_table (struct list_head *head,struct perf_pmu *pmu, -const struct pmu_event *table); +const struct pmu_events_table *table);char *perf_pmu__getcpuid (struct perf_pmu *pmu); -const struct pmu_event *pmu_events_table__find (void ); +const struct pmu_events_table *pmu_events_table__find (void );bool pmu_uncore_alias_match (const char *pmu_name,const char *name);void perf_pmu_free_alias (struct perf_pmu_alias *alias);
tools/perf/util/s390-sample-raw.c +3 -3
@@ -135,7 +135,7 @@ struct get_counter_name_data { };static int get_counter_name_callback (const struct pmu_event *evp, -const struct pmu_event *table__maybe_unused , +const struct pmu_events_table *table__maybe_unused ,void *vdata) {struct get_counter_name_data *data = vdata;
@@ -157,7 +157,7 @@ static int get_counter_name_callback(const struct pmu_event *evp,* the name of this counter. * If no match is found a NULL pointer is returned. */ -static const char *get_counter_name (int set,int nr,const struct pmu_event *table) +static const char *get_counter_name (int set,int nr,const struct pmu_events_table *table) {struct get_counter_name_data data = { .wanted =get_counterset_start (set) + nr,
@@ -177,7 +177,7 @@ static void s390_cpumcfdg_dump(struct perf_sample *sample)unsigned char *buf = sample->raw_data;const char *color =PERF_COLOR_BLUE ;struct cf_ctrset_entry *cep, ce; -const struct pmu_event *table; +const struct pmu_events_table *table;u64 *p; table =pmu_events_table__find ();
Signature · 1 line
2.37.1.595.g718a3a8f04-goog
Ian Rogers
reviewerPATCH v5 12/14 perf metrics: Copy entire pmu_event in find metric
In reply to Ian RogersThe pmu_event passed to the pmu_events_table_for_each_event is invalid after the loop. Copy the entire struct in metricgroup__find_metric. Reduce the scope of this function to static.
--- tools/perf/util/metricgroup.c | 33 ++++++++++++++++++--------------- tools/perf/util/metricgroup.h | 2 -- 2 files changed, 18 insertions(+), 17 deletions(-)
tools/perf/util/metricgroup.c +18 -15
@@ -879,6 +879,10 @@ struct metricgroup_add_iter_data {const struct pmu_events_table *table; }; +static bool metricgroup__find_metric (const char *metric, +const struct pmu_events_table *table, +struct pmu_event *pe); + static int add_metric(struct list_head *metric_list, const struct pmu_event *pe, const char *modifier,
@@ -914,7 +918,7 @@ static int resolve_metric(struct list_head *metric_list,size_t bkt;struct to_resolve {/* The metric to resolve. */ -const struct pmu_event *pe; +struct pmu_event pe;/* * The key in the IDs map, this may differ from in case, * etc. from pe->metric_name.
@@ -928,16 +932,15 @@ static int resolve_metric(struct list_head *metric_list,* the pending array. */ hashmap__for_each_entry (root_metric->pctx->ids, cur, bkt) { -const struct pmu_event *pe; +struct pmu_event pe; - pe =metricgroup__find_metric (cur->key, table); -if (pe) { +if (metricgroup__find_metric (cur->key, table, &pe)) { pending =realloc (pending, (pending_cnt +1 ) *sizeof (struct to_resolve ));if (!pending)return -ENOMEM ; - pending[pending_cnt].pe = pe; +memcpy (&pending[pending_cnt].pe, &pe,sizeof (pe)); pending[pending_cnt].key = cur->key; pending_cnt++; }
@@ -952,7 +955,7 @@ static int resolve_metric(struct list_head *metric_list,* context. */ for (i =0 ; i < pending_cnt; i++) { - ret =add_metric (metric_list, pending[i].pe, modifier, metric_no_group, + ret =add_metric (metric_list, &pending[i].pe, modifier, metric_no_group, root_metric, visited, table);if (ret)break ;
@@ -1073,7 +1076,7 @@ static int __add_metric(struct list_head *metric_list,struct metricgroup__find_metric_data {const char *metric; -const struct pmu_event *pe; +struct pmu_event *pe; }; static int metricgroup__find_metric_callback(const struct pmu_event *pe,
@@ -1085,21 +1088,21 @@ static int metricgroup__find_metric_callback(const struct pmu_event *pe,if (!match_metric (pe->metric_name, data->metric))return 0 ; - data->pe = pe; -return -1 ; +memcpy (data->pe, pe,sizeof (*pe)); +return 1 ; } -const struct pmu_event *metricgroup__find_metric (const char *metric, -const struct pmu_events_table *table) +static bool metricgroup__find_metric (const char *metric, +const struct pmu_events_table *table, +struct pmu_event *pe) {struct metricgroup__find_metric_data data = { .metric = metric, - .pe =NULL , + .pe = pe, }; -pmu_events_table_for_each_event (table, metricgroup__find_metric_callback, &data); - -return data.pe; +return pmu_events_table_for_each_event (table, metricgroup__find_metric_callback, &data) + ? true : false; } static int add_metric(struct list_head *metric_list,
tools/perf/util/metricgroup.h +0 -2
@@ -69,8 +69,6 @@ int metricgroup__parse_groups(const struct option *opt,bool metric_no_group, bool metric_no_merge, struct rblist *metric_events); -const struct pmu_event *metricgroup__find_metric (const char *metric, -const struct pmu_events_table *table); int metricgroup__parse_groups_test(struct evlist *evlist, const struct pmu_events_table *table, const char *str,
Signature · 1 line
2.37.1.595.g718a3a8f04-goog
Ian Rogers
reviewerPATCH v5 13/14 perf jevents: Compress the pmu_events_table
In reply to Ian RogersThe pmu_events array requires 15 pointers per entry which in position independent code need relocating. Change the array to be an array of offsets within a big C string. Only the offset of the first variable is required, subsequent variables are stored in order after the \0 terminator (requiring a byte per variable rather than 4 bytes per offset).
The file size savings are:
no jevents - the same 19,788,464bytes
x86 jevents - ~16.7% file size saving 23,744,288bytes vs 28,502,632bytes all jevents - ~19.5% file size saving 24,469,056bytes vs 30,379,920bytes default build options plus NO_LIBBFD=1.
For example, the x86 build savings come from .rela.dyn and .data.rel.ro becoming smaller by 3,157,032bytes and 3,042,016bytes respectively. .rodata increases by 1,432,448bytes, giving an overall 4,766,600bytes saving.
To make metric strings more shareable, the topic is changed from say 'skx metrics' to just 'metrics'.
To try to help with the memory layout the pmu_events are ordered as used by perf qsort comparator functions.
--- tools/perf/pmu-events/jevents.py | 207 ++++++++++++++++++++++++------- 1 file changed, 162 insertions(+), 45 deletions(-)
tools/perf/pmu-events/jevents.py +162 -45
@@ -6,7 +6,8 @@ import csv import json import os import sys -from typing import (Callable, Optional, Sequence) +from typing import (Callable, Dict, Optional, Sequence, Set, Tuple) +import collections # Global command line arguments. _args = None
@@ -20,6 +21,19 @@ _arch_std_events = {} _close_table = False # Events to write out when the table is closed _pending_events = [] +# Global BigCString shared by all structures. +_bcs = None +# Order specific JsonEvent attributes will be visited. +_json_event_attributes = [ + # cmp_sevent related attributes. + 'name', 'pmu', 'topic', 'desc', 'metric_name', 'metric_group', + # Seems useful, put it early. + 'event', + # Short things in alphabetical order. + 'aggr_mode', 'compat', 'deprecated', 'perpkg', 'unit', + # Longer things (the last won't be iterated over during decompress). + 'metric_constraint', 'metric_expr', 'long_desc' +] def removesuffix(s: str, suffix: str) -> str:
@@ -39,6 +53,66 @@ def file_name_to_table_name(parents: Sequence[str], dirname: str) -> str: tblname += '_' + dirname return tblname.replace('-', '_') +def c_len(s: str) -> int: + """Return the length of s a C string + + This doesn't handle all escape characters properly. It first assumes + all \ are for escaping, it then adjusts as it will have over counted + \\. The code uses \000 rather than \0 as a terminator as an adjacent + number would be folded into a string of \0 (ie. "\0" + "5" doesn't + equal a terminator followed by the number 5 but the escape of + \05). The code adjusts for \000 but not properly for all octal, hex + or unicode values. + """ + try: + utf = s.encode(encoding='utf-8',errors='strict') + except: + print(f'broken string {s}') + raise + return len(utf) - utf.count(b'\\') + utf.count(b'\\\\') - (utf.count(b'\\000') * 2) + +class BigCString: + """A class to hold many strings concatenated together. + + Generating a large number of stand-alone C strings creates a large + number of relocations in position independent code. The BigCString + is a helper for this case. It builds a single string which within it + are all the other C strings (to avoid memory issues the string + itself is held as a list of strings). The offsets within the big + string are recorded and when stored to disk these don't need + relocation. + """ + strings: Set[str] + big_string: Sequence[str] + offsets: Dict[str, int] + + def __init__(self): + self.strings = set() + + def add(self, s: str) -> None: + """Called to add to the big string.""" + self.strings.add(s) + + def compute(self) -> None: + """Called once all strings are added to compute the string and offsets.""" + + # big_string_offset is the current location within the C string + # being appended to - comments, etc. don't count. big_string is + # the string contents represented as a list. Strings are immutable + # in Python and so appending to one causes memory issues, while + # lists are mutable. + big_string_offset = 0 + self.big_string = [] + self.offsets = {} + # Emit all strings in a sorted manner. + for s in sorted(self.strings): + self.offsets[s] = big_string_offset + self.big_string.append(f'/* offset={big_string_offset} */ "') + self.big_string.append(s) + self.big_string.append('"\n') + big_string_offset += c_len(s) + +_bcs = BigCString() class JsonEvent: """Representation of an event loaded from a json file dictionary."""
@@ -203,26 +277,18 @@ class JsonEvent: s += f'\t{attr} = {value},\n' return s + '}' + def build_c_string(self) -> str: + s = '' + for attr in _json_event_attributes: + x = getattr(self, attr) + s += f'{x}\\000' if x else '\\000' + return s + def to_c_string(self) -> str: """Representation of the event as a C struct initializer.""" - def attr_string(attr: str, value: str) -> str: - return f'\t.{attr} = \"{value}\",\n' - - def str_if_present(self, attr: str) -> str: - if not getattr(self, attr): - return '' - return attr_string(attr, getattr(self, attr)) - - s = '{\n' - for attr in [ - 'aggr_mode', 'compat', 'deprecated', 'desc', 'event', 'long_desc', - 'metric_constraint', 'metric_expr', 'metric_group', 'metric_name', - 'name', 'perpkg', 'pmu', 'topic', 'unit' - ]: - s += str_if_present(self, attr) - s += '},\n' - return s + s = self.build_c_string() + return f'{{ { _bcs.offsets[s] } }}, /* {s} */\n' def read_json_events(path: str, topic: str) -> Sequence[JsonEvent]:
@@ -237,7 +303,6 @@ def read_json_events(path: str, topic: str) -> Sequence[JsonEvent]: event.topic = topic return result - def preprocess_arch_std_files(archpath: str) -> None: """Read in all architecture standard events.""" global _arch_std_events
@@ -253,7 +318,7 @@ def print_events_table_prefix(tblname: str) -> None: global _close_table if _close_table: raise IOError('Printing table prefix but last table has no suffix') - _args.output_file.write(f'static const struct pmu_event {tblname}[] = {{\n') + _args.output_file.write(f'static const struct compact_pmu_event {tblname}[] = {{\n') _close_table = True
@@ -268,13 +333,13 @@ def add_events_table_entries(item: os.DirEntry, topic: str) -> None: def print_events_table_suffix() -> None: """Optionally close events table.""" - def event_cmp_key(j: JsonEvent): - def fix_none(s: str): + def event_cmp_key(j: JsonEvent) -> Tuple[bool, str, str, str, str]: + def fix_none(s: Optional[str]) -> str: if s is None: return '' return s - return (not j.desc is None, fix_none(j.topic), fix_none(j.name), fix_none(j.pmu), + return (j.desc is not None, fix_none(j.topic), fix_none(j.name), fix_none(j.pmu), fix_none(j.metric_name)) global _close_table
@@ -286,23 +351,37 @@ def print_events_table_suffix() -> None: _args.output_file.write(event.to_c_string()) _pending_events = [] - _args.output_file.write("""{ -\t.name = 0, -\t.event = 0, -\t.desc = 0, -}, -}; -""") + _args.output_file.write('};\n\n') _close_table = False +def get_topic(topic: str) -> str: + if topic.endswith('metrics.json'): + return 'metrics' + return removesuffix(topic, '.json').replace('-', ' ') + +def preprocess_one_file(parents: Sequence[str], item: os.DirEntry) -> None: + + if item.is_dir(): + return + + # base dir or too deep + level = len(parents) + if level == 0 or level > 4: + return + + # Ignore other directories. If the file name does not have a .json + # extension, ignore it. It could be a readme.txt for instance. + if not item.is_file() or not item.name.endswith('.json'): + return + + topic = get_topic(item.name) + for event in read_json_events(item.path, topic): + _bcs.add(event.build_c_string()) def process_one_file(parents: Sequence[str], item: os.DirEntry) -> None: """Process a JSON file during the main walk.""" global _sys_event_tables - def get_topic(topic: str) -> str: - return removesuffix(topic, '.json').replace('-', ' ') - def is_leaf_dir(path: str) -> bool: for item in os.scandir(path): if item.is_dir():
@@ -337,7 +416,8 @@ def print_mapping_table(archs: Sequence[str]) -> None: _args.output_file.write(""" /* Struct used to make the PMU event table implementation opaque to callers. */ struct pmu_events_table { - const struct pmu_event *entries; + const struct compact_pmu_event *entries; + size_t length; }; /*
@@ -365,7 +445,10 @@ const struct pmu_events_map pmu_events_map[] = { _args.output_file.write("""{ \t.arch = "testarch", \t.cpuid = "testcpu", -\t.table = { pme_test_soc_cpu }, +\t.table = { +\t.entries = pme_test_soc_cpu, +\t.length = ARRAY_SIZE(pme_test_soc_cpu), +\t} }, """) else:
@@ -380,7 +463,10 @@ const struct pmu_events_map pmu_events_map[] = { _args.output_file.write(f"""{{ \t.arch = "{arch}", \t.cpuid = "{cpuid}", -\t.table = {{ {tblname} }} +\t.table = {{ +\t\t.entries = {tblname}, +\t\t.length = ARRAY_SIZE({tblname}) +\t}} }}, """) first = False
@@ -388,7 +474,7 @@ const struct pmu_events_map pmu_events_map[] = { _args.output_file.write("""{ \t.arch = 0, \t.cpuid = 0, -\t.table = { 0 }, +\t.table = { 0, 0 }, } }; """)
@@ -406,23 +492,41 @@ static const struct pmu_sys_events pmu_sys_event_tables[] = { """) for tblname in _sys_event_tables: _args.output_file.write(f"""\t{{ -\t\t.table = {{ {tblname} }}, +\t\t.table = {{ +\t\t\t.entries = {tblname}, +\t\t\t.length = ARRAY_SIZE({tblname}) +\t\t}}, \t\t.name = \"{tblname}\", \t}}, """) _args.output_file.write("""\t{ -\t\t.table = { 0 } +\t\t.table = { 0, 0 } \t}, }; -int pmu_events_table_for_each_event(const struct pmu_events_table *table, pmu_event_iter_fn fn, +static void decompress(int offset, struct pmu_event *pe) +{ +\tconst char *p = &big_c_string[offset]; +""") + for attr in _json_event_attributes: + _args.output_file.write(f""" +\tpe->{attr} = (*p == '\\0' ? NULL : p); +""") + if attr == _json_event_attributes[-1]: + continue + _args.output_file.write('\twhile (*p++);') + _args.output_file.write("""} + +int pmu_events_table_for_each_event(const struct pmu_events_table *table, + pmu_event_iter_fn fn, void *data) { - for (const struct pmu_event *pe = &table->entries[0]; - pe->name || pe->metric_group || pe->metric_name; - pe++) { - int ret = fn(pe, table, data); + for (size_t i = 0; i < table->length; i++) { + struct pmu_event pe; + int ret; + decompress(table->entries[i].offset, &pe); + ret = fn(&pe, table, data); if (ret) return ret; }
@@ -531,7 +635,7 @@ def main() -> None: help='Root of tree containing architecture directories containing json files' ) ap.add_argument( - 'output_file', type=argparse.FileType('w'), nargs='?', default=sys.stdout) + 'output_file', type=argparse.FileType('w', encoding='utf-8'), nargs='?', default=sys.stdout) _args = ap.parse_args() _args.output_file.write("""
@@ -541,6 +645,10 @@ def main() -> None: #include <string.h> #include <stddef.h> +struct compact_pmu_event { + int offset; +}; + """) archs = [] for item in os.scandir(_args.starting_dir):
@@ -556,6 +664,15 @@ def main() -> None: for arch in archs: arch_path = f'{_args.starting_dir}/{arch}' preprocess_arch_std_files(arch_path) + ftw(arch_path, [], preprocess_one_file) + + _bcs.compute() + _args.output_file.write('static const char *const big_c_string =\n') + for s in _bcs.big_string: + _args.output_file.write(s) + _args.output_file.write(';\n\n') + for arch in archs: + arch_path = f'{_args.starting_dir}/{arch}' ftw(arch_path, [], process_one_file) print_events_table_suffix()
Signature · 1 line
2.37.1.595.g718a3a8f04-goog
Ian Rogers
reviewerPATCH v5 14/14 perf jevents: Fold strings optimization
In reply to Ian RogersIf a shorter string ends a longer string then the shorter string may reuse the longer string at an offset. For example, on x86 the event arith.cycles_div_busy and cycles_div_busy can be folded, even though they have difference names the strings are identical after 6 characters. cycles_div_busy can reuse the arith.cycles_div_busy string at an offset of 6.
In pmu-events.c this looks like the following where the 'also:' lists
folded strings:
/* offset=177541 */ "arith.cycles_div_busy\000\000pipeline\000Cycles the divider is busy\000\000\000event=0x14,period=2000000,umask=0x1\000\000\000\000\000\000\000\000\000" /* also: cycles_div_busy\000\000pipeline\000Cycles the divider is busy\000\000\000event=0x14,period=2000000,umask=0x1\000\000\000\000\000\000\000\000\000 */
As jevents.py combines multiple strings for an event into a larger string, the amount of folding is minimal as all parts of the event must align. Other organizations can benefit more from folding, but lose space by say recording more offsets.
--- tools/perf/pmu-events/jevents.py | 55 ++++++++++++++++++++++++++++---- 1 file changed, 48 insertions(+), 7 deletions(-)
tools/perf/pmu-events/jevents.py +48 -7
@@ -80,7 +80,9 @@ class BigCString: are all the other C strings (to avoid memory issues the string itself is held as a list of strings). The offsets within the big string are recorded and when stored to disk these don't need - relocation. + relocation. To reduce the size of the string further, identical + strings are merged. If a longer string ends-with the same value as a + shorter string, these entries are also merged. """ strings: Set[str] big_string: Sequence[str]
@@ -96,6 +98,33 @@ class BigCString: def compute(self) -> None: """Called once all strings are added to compute the string and offsets.""" + folded_strings = {} + # Determine if two strings can be folded, ie. let 1 string use the + # end of another. First reverse all strings and sort them. + sorted_reversed_strings = sorted([x[::-1] for x in self.strings]) + + # Strings 'xyz' and 'yz' will now be [ 'zy', 'zyx' ]. Scan forward + # for each string to see if there is a better candidate to fold it + # into, in the example rather than using 'yz' we can use'xyz' at + # an offset of 1. We record which string can be folded into which + # in folded_strings, we don't need to record the offset as it is + # trivially computed from the string lengths. + for pos,s in enumerate(sorted_reversed_strings): + best_pos = pos + for check_pos in range(pos + 1, len(sorted_reversed_strings)): + if sorted_reversed_strings[check_pos].startswith(s): + best_pos = check_pos + else: + break + if pos != best_pos: + folded_strings[s[::-1]] = sorted_reversed_strings[best_pos][::-1] + + # Compute reverse mappings for debugging. + fold_into_strings = collections.defaultdict(set) + for key, val in folded_strings.items(): + if key != val: + fold_into_strings[val].add(key) + # big_string_offset is the current location within the C string # being appended to - comments, etc. don't count. big_string is # the string contents represented as a list. Strings are immutable
@@ -104,13 +133,25 @@ class BigCString: big_string_offset = 0 self.big_string = [] self.offsets = {} - # Emit all strings in a sorted manner. + + # Emit all strings that aren't folded in a sorted manner. for s in sorted(self.strings): - self.offsets[s] = big_string_offset - self.big_string.append(f'/* offset={big_string_offset} */ "') - self.big_string.append(s) - self.big_string.append('"\n') - big_string_offset += c_len(s) + if s not in folded_strings: + self.offsets[s] = big_string_offset + self.big_string.append(f'/* offset={big_string_offset} */ "') + self.big_string.append(s) + self.big_string.append('"') + if s in fold_into_strings: + self.big_string.append(' /* also: ' + ', '.join(fold_into_strings[s]) + ' */') + self.big_string.append('\n') + big_string_offset += c_len(s) + continue + + # Compute the offsets of the folded strings. + for s in folded_strings.keys(): + assert s not in self.offsets + folded_s = folded_strings[s] + self.offsets[s] = self.offsets[folded_s] + c_len(folded_s) - c_len(s) _bcs = BigCString()
Signature · 1 line
2.37.1.595.g718a3a8f04-goog
Arnaldo Carvalho de Melo
maintainerEm Fri, Aug 12, 2022 at 04:09:35PM -0700, Ian Rogers escreveu:
Quoted 14 lines
jevents.pycreates a number of large arrays from the json events. The arrays contain pointers to strings that need relocating. The relocations have file size, run time and memory costs. These changes refactor thepmu_eventsAPI so that the storage of thepmu_eventstruct isn't exposed. The format is then changed to an offset within a combined big string, with adjacentpmu_eventstruct variables being next to each other in the string separated by \0 - meaning only the first variable of the struct needs its offset recording.Some related fixes are contained with the patches. The architecture
jevents.pycreates tables for can now be set by theJEVENTS_ARCHmake variable, with a new 'all' that generates the events and metrics for all architectures.
Quoted 5 lines
An example of the improvement to the file size on x86 is:
no jevents - the same 19,788,464bytes
x86 jevents - ~16.7% file size saving 23,744,288bytes vs 28,502,632bytes all jevents - ~19.5% file size saving 24,469,056bytes vs 30,379,920bytes default build options plus NO_LIBBFD=1.
Applied locally, going thru further tests.
- Arnaldo
Quoted 49 lines
I originally suggested fixing this problem in:
https://lore.kernel.org/linux-perf-users/CAP-5=fVB8G4bdb9T=FncRTh9oBVKCS=+=eowAO+YSgAhab+Dtg@mail.gmail.com/v5. Renamed two functions to be more inline with the code and added
extra commit message detail on the event sorting order as suggested by John Garry <[email protected]>.
v4. Fixed an issue with theempty-pmu-events.cspotted by John Garry
<[email protected]>.
v3. Fix an ARM build issue with a missed weak symbol. Perform some
pytype clean up.
v2. Split the substring folding optimization to its own patch and
comment tweaks as suggested by Namhyung Kim <[email protected]>. Recompute the file size savings with the latest json events and metrics.Ian Rogers (14):
perf jevent: Add an 'all' architecture argument perf jevents: Remove the type/version variables perf jevents: Provide path to json file on error perf jevents: Sort json files entries
perf pmu-events: Hidepmu_sys_event_tablesperf pmu-events: Avoid passingpmu_events_mapperf pmu-events: Hidepmu_events_map
perf test: Use full metric resolution
perf pmu-events: Move test events/metrics to json perf pmu-events: Don't assumepmu_eventis an array perf pmu-events: Hide thepmu_events
perf metrics: Copy entirepmu_eventin find metric perf jevents: Compress thepmu_events_tableperf jevents: Fold strings optimizationtools/perf/arch/arm64/util/pmu.c | 4 +- .../arch/test/test_soc/cpu/metrics.json | 64 +++ tools/perf/pmu-events/empty-pmu-events.c | 204 +++++++- tools/perf/pmu-events/jevents.py | 478 +++++++++++++++--- tools/perf/pmu-events/pmu-events.h | 40 +- tools/perf/tests/expand-cgroup.c | 25 +- tools/perf/tests/parse-metric.c | 77 +-- tools/perf/tests/pmu-events.c | 466 +++++++---------- tools/perf/util/metricgroup.c | 275 ++++++---- tools/perf/util/metricgroup.h | 5 +- tools/perf/util/pmu.c | 139 ++--- tools/perf/util/pmu.h | 10 +- tools/perf/util/s390-sample-raw.c | 50 +- 13 files changed, 1131 insertions(+), 706 deletions(-) create mode 100644 tools/perf/pmu-events/arch/test/test_soc/cpu/metrics.jsonSignature · 1 line
2.37.1.595.g718a3a8f04-goog
Signature · 1 line
- Arnaldo