add support for self monitoring
Subsystems (2)
cover add support for self monitoring
This series adds self monitoring support i.e allows to configure and read performance measurement unit (PMU) counters in runtime without using perf utility. This has certain adventages when application runs on isolated cores with nohz_full kernel parameter.
Events can be read directly using rte_pmu_read() or using dedicated tracepoint rte_eal_trace_pmu_read(). The latter will cause events to be stored inside CTF file.
By design, all enabled events are grouped together and the same group is attached to lcores that use self monitoring funtionality.
Events are enabled by names, which need to be read from standard location under sysfs i.e
/sys/bus/event_source/devices/PMU/events
where PMU is a core pmu i.e one measuring cpu events. As of today raw events are not supported.
Tomasz Duszynski (4):
eal: add generic support for reading PMU events eal/arm: support reading ARM PMU events in runtime eal/x86: support reading Intel PMU events in runtime
eal: add PMU support to tracing library
app/test/meson.build | 1 + app/test/test_pmu.c | 47 ++ app/test/test_trace_perf.c | 4 + doc/guides/prog_guide/profile_app.rst | 13 + doc/guides/prog_guide/trace_lib.rst | 32 ++ lib/eal/arm/include/meson.build | 1 + lib/eal/arm/include/rte_pmu_pmc.h | 37 ++ lib/eal/arm/meson.build | 4 + lib/eal/arm/rte_pmu.c | 103 +++++ lib/eal/common/eal_common_trace_points.c | 3 + lib/eal/common/meson.build | 3 + lib/eal/common/pmu_private.h | 41 ++ lib/eal/common/rte_pmu.c | 518 +++++++++++++++++++++++ lib/eal/include/meson.build | 1 + lib/eal/include/rte_eal_trace.h | 11 + lib/eal/include/rte_pmu.h | 207 +++++++++ lib/eal/linux/eal.c | 4 + lib/eal/version.map | 4 + lib/eal/x86/include/meson.build | 1 + lib/eal/x86/include/rte_pmu_pmc.h | 32 ++ 20 files changed, 1067 insertions(+) create mode 100644 app/test/test_pmu.c create mode 100644 lib/eal/arm/include/rte_pmu_pmc.h create mode 100644 lib/eal/arm/rte_pmu.c create mode 100644 lib/eal/common/pmu_private.h create mode 100644 lib/eal/common/rte_pmu.c create mode 100644 lib/eal/include/rte_pmu.h create mode 100644 lib/eal/x86/include/rte_pmu_pmc.h
--
2.25.1
PATCH 1/4 eal: add generic support for reading PMU events
Add support for programming PMU counters and reading their values in runtime bypassing kernel completely.
This is especially useful in cases where CPU cores are isolated (nohz_full) i.e run dedicated tasks. In such cases one cannot use standard perf utility without sacrificing latency and performance.
--- app/test/meson.build | 1 + app/test/test_pmu.c | 41 +++ doc/guides/prog_guide/profile_app.rst | 8 + lib/eal/common/meson.build | 3 + lib/eal/common/pmu_private.h | 41 +++ lib/eal/common/rte_pmu.c | 455 ++++++++++++++++++++++++++ lib/eal/include/meson.build | 1 + lib/eal/include/rte_pmu.h | 204 ++++++++++++ lib/eal/linux/eal.c | 4 + lib/eal/version.map | 3 + 10 files changed, 761 insertions(+) create mode 100644 app/test/test_pmu.c create mode 100644 lib/eal/common/pmu_private.h create mode 100644 lib/eal/common/rte_pmu.c create mode 100644 lib/eal/include/rte_pmu.h
app/test/meson.build +1 -0
@@ -143,6 +143,7 @@ test_sources = files( 'test_timer_racecond.c', 'test_timer_secondary.c', 'test_ticketlock.c', + 'test_pmu.c', 'test_trace.c', 'test_trace_register.c', 'test_trace_perf.c',
app/test/test_pmu.c new file +41 -0
41 lines not on this page · raw message
doc/guides/prog_guide/profile_app.rst +8 -0
@@ -7,6 +7,14 @@ Profile Your Application The following sections describe methods of profiling DPDK applications on different architectures. +Performance counter based profiling +----------------------------------- + +Majority of architectures support some sort hardware measurement unit which provides a set of +programmable counters that monitor specific events. There are different tools which can gather +that information, perf being an example here. Though in some scenarios, eg. when CPU cores are +isolated (nohz_full) and run dedicated tasks, using perf is less than ideal. In such cases one can +read specific events directly from application via ``rte_pmu_read()``. Profiling on x86 ----------------
lib/eal/common/meson.build +3 -0
@@ -38,6 +38,9 @@ sources += files( 'rte_service.c', 'rte_version.c', ) +if is_linux + sources += files('rte_pmu.c') +endif if is_linux or is_windows sources += files('eal_common_dynmem.c') endif
lib/eal/common/pmu_private.h new file +41 -0
41 lines not on this page · raw message
lib/eal/common/rte_pmu.c new file +455 -0
455 lines not on this page · raw message
lib/eal/include/meson.build +1 -0
@@ -36,6 +36,7 @@ headers += files( 'rte_pci_dev_features.h', 'rte_per_lcore.h', 'rte_pflock.h', + 'rte_pmu.h', 'rte_random.h', 'rte_reciprocal.h', 'rte_seqcount.h',
lib/eal/include/rte_pmu.h new file +204 -0
204 lines not on this page · raw message
lib/eal/linux/eal.c +4 -0
@@ -53,6 +53,7 @@#include "eal_options.h" #include "eal_vfio.h" #include "hotplug_mp.h" +#include "pmu_private.h" #define MEMSIZE_IF_NO_HUGE_PAGE (64ULL * 1024ULL * 1024ULL)
@@ -1206,6 +1207,8 @@ rte_eal_init(int argc, char **argv)return -1 ; } +eal_pmu_init (); +if (rte_eal_tailqs_init () <0 ) {rte_eal_init_alert ("Cannot init tail queues for objects" ); rte_errno =EFAULT ;
@@ -1372,6 +1375,7 @@ rte_eal_cleanup(void)eal_bus_cleanup ();rte_trace_save ();eal_trace_fini (); +eal_pmu_fini ();/* after this point, any DPDK pointers will become dangling */ rte_eal_memory_detach ();eal_mp_dev_hotplug_cleanup ();
lib/eal/version.map +3 -0
@@ -432,6 +432,8 @@ EXPERIMENTAL { rte_thread_set_priority; # added in 22.11 + rte_pmu_add_event; # WINDOWS_NO_EXPORT + rte_pmu_read; # WINDOWS_NO_EXPORT rte_thread_attr_get_affinity; rte_thread_attr_init; rte_thread_attr_set_affinity;
@@ -483,4 +485,5 @@ INTERNAL { rte_mem_map; rte_mem_page_size; rte_mem_unmap; + rte_pmu_enable_group; };
Signature · 1 line
2.25.1
Re: [PATCH 1/4] eal: add generic support for reading PMU events
Quoted 722 lines · Tomasz Duszynski
Add support for programming PMU counters and reading their values in runtime bypassing kernel completely.
This is especially useful in cases where CPU cores are isolated (
nohz_full) i.e run dedicated tasks. In such cases one cannot use standard perf utility without sacrificing latency and performance.Signed-off-by Tomasz Duszynski <[email protected]>--- app/test/meson.build | 1 + app/test/test_pmu.c | 41 +++ doc/guides/prog_guide/profile_app.rst | 8 + lib/eal/common/meson.build | 3 + lib/eal/common/pmu_private.h | 41 +++ lib/eal/common/rte_pmu.c | 455 ++++++++++++++++++++++++++ lib/eal/include/meson.build | 1 + lib/eal/include/rte_pmu.h | 204 ++++++++++++ lib/eal/linux/eal.c | 4 + lib/eal/version.map | 3 + 10 files changed, 761 insertions(+)create mode 100644
app/test/test_pmu.c
create mode 100644lib/eal/common/pmu_private.h
create mode 100644lib/eal/common/rte_pmu.c
create mode 100644lib/eal/include/rte_pmu.happ/test/meson.build +1
-0@@ -143,6 +143,7 @@ test_sources = files( 'test_timer_racecond.c', 'test_timer_secondary.c', 'test_ticketlock.c', + 'test_pmu.c', 'test_trace.c', 'test_trace_register.c', 'test_trace_perf.c',app/test/test_pmu.c new file +41
-0@@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(C) 2022 Marvell International Ltd. +*/ + +#include <rte_pmu.h> + +#include "test.h" + +static int +test_pmu_read (void ) +{ +uint64_t val =0 ; +int tries =10 ; +int event =-1 ; + +while (tries--) + val +=rte_pmu_read (event); + +if (val ==0 ) +return TEST_FAILED ; + +return TEST_SUCCESS ; +} + +static struct unit_test_suite pmu_tests = { + .suite_name ="pmu autotest" , + .setup =NULL , + .teardown =NULL , + .unit_test_cases = { +TEST_CASE (test_pmu_read), +TEST_CASES_END () + } +}; + +static int +test_pmu (void ) +{ +return unit_test_suite_runner (&pmu_tests); +} + +REGISTER_TEST_COMMAND (pmu_autotest, test_pmu);doc/guides/prog_guide/profile_app.rst +8
-0@@ -7,6 +7,14 @@ Profile Your Application The following sections describe methods of profiling DPDK applications on different architectures. +Performance counter based profiling +----------------------------------- + +Majority of architectures support some sort hardware measurement unit which provides a set of +programmable counters that monitor specific events. There are different tools which can gather +that information, perf being an example here. Though in some scenarios, eg. when CPU cores are +isolated (nohz_full) and run dedicated tasks, using perf is less than ideal. In such cases one can +read specific events directly from application via ``rte_pmu_read()``. Profiling on x86 ----------------lib/eal/common/meson.build +3
-0@@ -38,6 +38,9 @@ sources += files( 'rte_service.c', 'rte_version.c', ) +if is_linux + sources += files('rte_pmu.c') +endif if is_linux or is_windows sources += files('eal_common_dynmem.c') endiflib/eal/common/pmu_private.h new file +41
-0@@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2022 Marvell +*/ + +#ifndef _PMU_PRIVATE_H_ +#define _PMU_PRIVATE_H_ + +/** +* Architecture specific PMU init callback. +* +* @return +* 0 in case of success, negative value otherwise. +*/ +int +pmu_arch_init (void ); + +/** +* Architecture specific PMU cleanup callback. +*/ +void +pmu_arch_fini (void ); + +/** +* Apply architecture specific settings to config before passing it to syscall. +*/ +void +pmu_arch_fixup_config (uint64_t config[3 ]); + +/** +* Initialize PMU tracing internals. +*/ +void +eal_pmu_init (void ); + +/** +* Cleanup PMU internals. +*/ +void +eal_pmu_fini (void ); + +#endif /* _PMU_PRIVATE_H_ */ lib/eal/common/rte_pmu.c new file +455
-0@@ -0,0 +1,455 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(C) 2022 Marvell International Ltd. +*/ + +#include <ctype.h> +#include <dirent.h> +#include <errno.h> +#include <regex.h> +#include <sys/ioctl.h> +#include <sys/mman.h> +#include <sys/queue.h> +#include <sys/syscall.h> +#include <unistd.h> + +#include <rte_eal_paging.h> +#include <rte_malloc.h> +#include <rte_pmu.h> +#include <rte_tailq.h> + +#include "pmu_private.h" + +#define EVENT_SOURCE_DEVICES_PATH "/sys/bus/event_source/devices" + +#ifndef GENMASK_ULL +#define GENMASK_ULL (h, l) ((~0ULL - (1ULL << (l)) + 1) & (~0ULL >> ((64 - 1 - (h))))) +#endif + +#ifndef FIELD_PREP +#define FIELD_PREP (m, v) (((uint64_t)(v) << (__builtin_ffsll(m) - 1)) & (m)) +#endif + +struct rte_pmu *pmu; + +/* +* Following __rte_weak functions provide default no-op. Architectures should override them if +* necessary. +*/ + +int +__rte_weak pmu_arch_init(void) +{ +return 0 ; +} + +void +__rte_weak pmu_arch_fini(void) +{ +} + +void +__rte_weak pmu_arch_fixup_config(uint64_t config[3]) +{ +RTE_SET_USED (config); +} + +static int +get_term_format (const char *name,int *num,uint64_t *mask) +{ +char *config =NULL ; +char path[PATH_MAX ]; +int high, low, ret; +FILE *fp; + +/* quiesce -Wmaybe-uninitialized warning */ + *num =0 ; + *mask =0 ; + +snprintf (path,sizeof (path),EVENT_SOURCE_DEVICES_PATH "/%s/format/%s" , pmu->name, name); + fp =fopen (path,"r" ); +if (!fp) +return -errno; + + errno =0 ; + ret =fscanf (fp,"%m[^:]:%d-%d" , &config, &low, &high); +if (ret <2 ) { + ret = -ENODATA ; + goto out; + } +if (errno) { + ret = -errno; + goto out; + } + +if (ret ==2 ) + high = low; + + *mask =GENMASK_ULL (high, low); +/* Last digit should be [012]. If last digit is missing 0 is implied. */ + *num = config[strlen (config) -1 ]; + *num =isdigit (*num) ? *num -'0' :0 ; + + ret =0 ; +out: +free (config); +fclose (fp); + +return ret; +} + +static int +parse_event (char *buf,uint64_t config[3 ]) +{ +char *token, *term; +int num, ret, val; +uint64_t mask; + + config[0 ] = config[1 ] = config[2 ] =0 ; + + token =strtok (buf,"," ); +while (token) { + errno =0 ; +/* <term>=<value> */ + ret =sscanf (token,"%m[^=]=%i" , &term, &val); +if (ret <1 ) +return -ENODATA ; +if (errno) +return -errno; +if (ret ==1 ) + val =1 ; + + ret =get_term_format (term, &num, &mask); +free (term); +if (ret) +return ret; + + config[num] |=FIELD_PREP (mask, val); + token =strtok (NULL ,"," ); + } + +return 0 ; +} + +static int +get_event_config (const char *name,uint64_t config[3 ]) +{ +char path[PATH_MAX ], buf[BUFSIZ ]; +FILE *fp; +int ret; + +snprintf (path,sizeof (path),EVENT_SOURCE_DEVICES_PATH "/%s/events/%s" , pmu->name, name); + fp =fopen (path,"r" ); +if (!fp) +return -errno; + + ret =fread (buf,1 ,sizeof (buf), fp); +if (ret ==0 ) { +fclose (fp); + +return -EINVAL ; + } +fclose (fp); + buf[ret] ='\0' ; + +return parse_event (buf, config); +} + +static int +do_perf_event_open (uint64_t config[3 ],int lcore_id,int group_fd) +{ +struct perf_event_attr attr = { + .size =sizeof (struct perf_event_attr ), + .type =PERF_TYPE_RAW , + .exclude_kernel =1 , + .exclude_hv =1 , + .disabled =1 , + }; + +pmu_arch_fixup_config (config); + + attr.config = config[0 ]; + attr.config1 = config[1 ]; + attr.config2 = config[2 ]; + +return syscall (SYS_perf_event_open, &attr,rte_gettid (),rte_lcore_to_cpu_id (lcore_id), + group_fd,0 ); +} + +static int +open_events (int lcore_id) +{ +struct rte_pmu_event_group *group = &pmu->group[lcore_id]; +struct rte_pmu_event *event; +uint64_t config[3 ]; +int num =0 , ret; + +/* group leader gets created first, with fd = -1 */ + group->fds[0 ] =-1 ; + + TAILQ_FOREACH(event, &pmu->event_list, next) { + ret =get_event_config (event->name, config); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to get %s event config\n" , event->name); +continue ; + } + + ret =do_perf_event_open (config, lcore_id, group->fds[0 ]); +if (ret ==-1 ) { +if (errno ==EOPNOTSUPP ) +RTE_LOG (ERR ,EAL ,"64 bit counters not supported\n" ); + + ret = -errno; + goto out; + } + + group->fds[event->index] = ret; + num++; + } + +return 0 ; +out: +for (--num; num >=0 ; num--) { +close (group->fds[num]); + group->fds[num] =-1 ; + } + + +return ret; +} + +static int +mmap_events (int lcore_id) +{ +struct rte_pmu_event_group *group = &pmu->group[lcore_id]; +void *addr; +int ret, i; + +for (i =0 ; i < pmu->num_group_events; i++) { + addr =mmap (0 ,rte_mem_page_size (),PROT_READ ,MAP_SHARED , group->fds[i],0 ); +if (addr ==MAP_FAILED ) { + ret = -errno; + goto out; + } + + group->mmap_pages[i] = addr; + } + +return 0 ; +out: +for (; i; i--) { +munmap (group->mmap_pages[i -1 ],rte_mem_page_size ()); + group->mmap_pages[i -1 ] =NULL ; + } + +return ret; +} + +static void +cleanup_events (int lcore_id) +{ +struct rte_pmu_event_group *group = &pmu->group[lcore_id]; +int i; + +if (!group->fds) +return ; + +if (group->fds[0 ] !=-1 ) +ioctl (group->fds[0 ],PERF_EVENT_IOC_DISABLE ,PERF_IOC_FLAG_GROUP ); + +for (i =0 ; i < pmu->num_group_events; i++) { +if (group->mmap_pages[i]) { +munmap (group->mmap_pages[i],rte_mem_page_size ()); + group->mmap_pages[i] =NULL ; + } + +if (group->fds[i] !=-1 ) { +close (group->fds[i]); + group->fds[i] =-1 ; + } + } + +rte_free (group->mmap_pages); +rte_free (group->fds); + + group->mmap_pages =NULL ; + group->fds =NULL ; + group->enabled = false; +} + +int __rte_noinline +rte_pmu_enable_group (int lcore_id) +{ +struct rte_pmu_event_group *group = &pmu->group[lcore_id]; +int ret; + +if (pmu->num_group_events ==0 ) { +RTE_LOG (DEBUG ,EAL ,"no matching PMU events\n" ); + +return 0 ; + } + + group->fds =rte_zmalloc (NULL , pmu->num_group_events,sizeof (*group->fds)); +if (!group->fds) { +RTE_LOG (ERR ,EAL ,"failed to alloc descriptor memory\n" ); + +return -ENOMEM ; + } + + group->mmap_pages =rte_zmalloc (NULL , pmu->num_group_events,sizeof (*group->mmap_pages)); +if (!group->mmap_pages) { +RTE_LOG (ERR ,EAL ,"failed to alloc userpage memory\n" ); + + ret = -ENOMEM ; + goto out; + } + + ret =open_events (lcore_id); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to open events on lcore-worker-%d\n" , lcore_id); + goto out; + } + + ret =mmap_events (lcore_id); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to map events on lcore-worker-%d\n" , lcore_id); + goto out; + } + +if (ioctl (group->fds[0 ],PERF_EVENT_IOC_ENABLE ,PERF_IOC_FLAG_GROUP ) ==-1 ) { +RTE_LOG (ERR ,EAL ,"failed to enable events on lcore-worker-%d\n" , lcore_id); + + ret = -errno; + goto out; + } + +return 0 ; + +out: +cleanup_events (lcore_id); + +return ret; +} + +static int +scan_pmus (void ) +{ +char path[PATH_MAX ]; +struct dirent *dent; +const char *name; +DIR *dirp; + + dirp =opendir (EVENT_SOURCE_DEVICES_PATH ); +if (!dirp) +return -errno; + +while ((dent =readdir (dirp))) { + name = dent->d_name; +if (name[0 ] =='.' ) +continue ; + +/* sysfs entry should either contain cpus or be a cpu */ +if (!strcmp (name,"cpu" )) +break ; + +snprintf (path,sizeof (path),EVENT_SOURCE_DEVICES_PATH "/%s/cpus" , name); +if (access (path,F_OK ) ==0 ) +break ; + } + +closedir (dirp); + +if (dent) { + pmu->name =strdup (name); +if (!pmu->name) +return -ENOMEM ; + } + +return pmu->name ?0 : -ENODEV ; +} + +int +rte_pmu_add_event (const char *name) +{ +struct rte_pmu_event *event; +char path[PATH_MAX ]; + +snprintf (path,sizeof (path),EVENT_SOURCE_DEVICES_PATH "/%s/events/%s" , pmu->name, name); +if (access (path,R_OK )) +return -ENODEV ; + + TAILQ_FOREACH(event, &pmu->event_list, next) { +if (!strcmp (event->name, name)) +return event->index; +continue ; + } + + event =rte_zmalloc (NULL ,1 ,sizeof (*event)); +if (!event) +return -ENOMEM ; + + event->name =strdup (name); +if (!event->name) { +rte_free (event); + +return -ENOMEM ; + } + + event->index = pmu->num_group_events++; +TAILQ_INSERT_TAIL (&pmu->event_list, event, next); + +RTE_LOG (DEBUG ,EAL ,"%s even added at index %d\n" , name, event->index); + +return event->index; +} + +void +eal_pmu_init (void ) +{ +int ret; + + pmu =rte_calloc (NULL ,1 ,sizeof (*pmu),RTE_CACHE_LINE_SIZE ); +if (!pmu) { +RTE_LOG (ERR ,EAL ,"failed to alloc PMU\n" ); + +return ; + } + +TAILQ_INIT (&pmu->event_list); + + ret =scan_pmus (); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to find core pmu\n" ); + goto out; + } + + ret =pmu_arch_init (); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to setup arch for PMU\n" ); + goto out; + } + +return ; +out: +free (pmu->name); +rte_free (pmu); +} + +void +eal_pmu_fini (void ) +{ +struct rte_pmu_event *event, *tmp; +int lcore_id; + + RTE_TAILQ_FOREACH_SAFE(event, &pmu->event_list, next, tmp) { +TAILQ_REMOVE (&pmu->event_list, event, next); +free (event->name); +rte_free (event); + } + +RTE_LCORE_FOREACH_WORKER (lcore_id) +cleanup_events (lcore_id); + +pmu_arch_fini (); +free (pmu->name); +rte_free (pmu); +}lib/eal/include/meson.build +1
-0@@ -36,6 +36,7 @@ headers += files( 'rte_pci_dev_features.h', 'rte_per_lcore.h', 'rte_pflock.h', + 'rte_pmu.h', 'rte_random.h', 'rte_reciprocal.h', 'rte_seqcount.h',lib/eal/include/rte_pmu.h new file +79
-0@@ -0,0 +1,204 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2022 Marvell +*/ + +#ifndef _RTE_PMU_H_ +#define _RTE_PMU_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include <rte_common.h> +#include <rte_compat.h> + +#ifdef RTE_EXEC_ENV_LINUX + +#include <linux/perf_event.h> + +#include <rte_atomic.h> +#include <rte_branch_prediction.h> +#include <rte_lcore.h> +#include <rte_log.h> + +/** + * @file + * + * PMU event tracing operations + * + * This file defines generic API and types necessary to setup PMU and + * read selected counters in runtime. + */ + +/** + * A structure describing a group of events. + */ +struct rte_pmu_event_group { + int *fds; /**< array of event descriptors */ + void **mmap_pages; /**< array of pointers to mmapped perf_event_attr structures */ + bool enabled; /**< true if group was enabled on particular lcore */ +}; + +/** + * A structure describing an event. + */ +struct rte_pmu_event { + char *name; /** name of an event */ + int index; /** event index into fds/mmap_pages */ + TAILQ_ENTRY(rte_pmu_event) next; /** list entry */ +}; + +/** + * A PMU state container. + */ +struct rte_pmu { + char *name; /** name of core PMU listed under /sys/bus/event_source/devices */ + struct rte_pmu_event_group group[RTE_MAX_LCORE]; /**< per lcore event group data */ + int num_group_events; /**< number of events in a group */ + TAILQ_HEAD(, rte_pmu_event) event_list; /**< list of matching events */ +}; + +/** Pointer to the PMU state container */ +extern struct rte_pmu *pmu; + +/** Each architecture supporting PMU needs to provide its own version */ +#ifndef rte_pmu_pmc_read +#define rte_pmu_pmc_read(index) ({ 0; }) +#endif + +/** + * @internal + * + * Read PMU counter. + * + * @param pc + * Pointer to the mmapped user page. + * @return + * Counter value read from hardware. + */ +__rte_internal
lib/eal/include/rte_pmu.h line 89
Quoted 10 lines
lib/eal/include/rte_pmu.h new file +10
-0+static __rte_always_inline uint64_t +rte_pmu_read_userpage (struct perf_event_mmap_page *pc) +{ +uint64_t offset, width, pmc =0 ; +uint32_t seq, index; +int tries =100 ; + +for (;;) { + seq = pc->lock; +rte_compiler_barrier ();
I'm guessing this should be a load-acquire instead. Less heavy-handed than a compiler barrier on TSO CPUs, and works on weakly ordered systems as well (unlike the compiler barrier).
This looks like an open-coded sequence lock, so take a look in rte_seqcount.h for inspiration.
Quoted 163 lines
lib/eal/include/rte_pmu.h new file +115
-0+ index = pc->index; + offset = pc->offset; + width = pc->pmc_width; + +if (likely (pc->cap_user_rdpmc && index)) { + pmc =rte_pmu_pmc_read (index -1 ); + pmc <<=64 - width; + pmc >>=64 - width; + } + +rte_compiler_barrier (); + +if (likely (pc->lock == seq)) +return pmc + offset; + +if (--tries ==0 ) { +RTE_LOG (DEBUG ,EAL ,"failed to get perf_event_mmap_page lock\n" ); +break ; + } + } + +return 0 ; +} + +/** +* @internal +* +* Enable group of events for a given lcore. +* +* @param lcore_id +* The identifier of the lcore. +* @return +* 0 in case of success, negative value otherwise. +*/ +__rte_internal +int +rte_pmu_enable_group (int lcore_id); + +/** +* @warning +* @b EXPERIMENTAL: this API may change without prior notice +* +* Add event to the group of enabled events. +* +* @param name +* Name of an event listed under /sys/bus/event_source/devices/pmu/events. +* @return +* Event index in case of success, negative value otherwise. +*/ +__rte_experimental +int +rte_pmu_add_event (const char *name); + +/** +* @warning +* @b EXPERIMENTAL: this API may change without prior notice +* +* Read hardware counter configured to count occurrences of an event. +* +* @param index +* Index of an event to be read. +* @return +* Event value read from register. In case of errors or lack of support +* 0 is returned. In other words, stream of zeros in a trace file +* indicates problem with reading particular PMU event register. +*/ +__rte_experimental +static __rte_always_inline uint64_t +rte_pmu_read (int index) +{ +int lcore_id =rte_lcore_id (); +struct rte_pmu_event_group *group; +int ret; + +if (!pmu) +return 0 ; + + group = &pmu->group[lcore_id]; +if (!group->enabled) { + ret =rte_pmu_enable_group (lcore_id); +if (ret) +return 0 ; + + group->enabled = true; + } + +if (index <0 || index >= pmu->num_group_events) +return 0 ; + +return rte_pmu_read_userpage (group->mmap_pages[index]); +} + +#else /* !RTE_EXEC_ENV_LINUX */ + +__rte_experimental +static int __rte_unused +rte_pmu_add_event(__rte_unused const char *name) +{ +return -1 ; +} + +__rte_experimental +static __rte_always_inline uint64_t +rte_pmu_read(__rte_unused int index) +{ +return 0 ; +} + +#endif /* RTE_EXEC_ENV_LINUX */ + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_PMU_H_ */ lib/eal/linux/eal.c +4
-0@@ -53,6 +53,7 @@#include "eal_options.h" #include "eal_vfio.h" #include "hotplug_mp.h" +#include "pmu_private.h" #define MEMSIZE_IF_NO_HUGE_PAGE (64ULL * 1024ULL * 1024ULL)@@ -1206,6 +1207,8 @@ rte_eal_init(int argc, char **argv)return -1 ; } +eal_pmu_init (); +if (rte_eal_tailqs_init () <0 ) {rte_eal_init_alert ("Cannot init tail queues for objects" ); rte_errno =EFAULT ;@@ -1372,6 +1375,7 @@ rte_eal_cleanup(void)eal_bus_cleanup ();rte_trace_save ();eal_trace_fini (); +eal_pmu_fini ();/* after this point, any DPDK pointers will become dangling */ rte_eal_memory_detach ();eal_mp_dev_hotplug_cleanup ();lib/eal/version.map +3
-0@@ -432,6 +432,8 @@ EXPERIMENTAL { rte_thread_set_priority; # added in 22.11 + rte_pmu_add_event; # WINDOWS_NO_EXPORT + rte_pmu_read; # WINDOWS_NO_EXPORT rte_thread_attr_get_affinity; rte_thread_attr_init; rte_thread_attr_set_affinity;@@ -483,4 +485,5 @@ INTERNAL { rte_mem_map; rte_mem_page_size; rte_mem_unmap; + rte_pmu_enable_group; };
PATCH 2/4 eal/arm: support reading ARM PMU events in runtime
In reply to Tomasz DuszynskiAdd support for reading ARM PMU events in runtime.
--- app/test/test_pmu.c | 4 ++ lib/eal/arm/include/meson.build | 1 + lib/eal/arm/include/rte_pmu_pmc.h | 37 +++++++++++ lib/eal/arm/meson.build | 4 ++ lib/eal/arm/rte_pmu.c | 103 ++++++++++++++++++++++++++++++ lib/eal/include/rte_pmu.h | 3 + 6 files changed, 152 insertions(+) create mode 100644 lib/eal/arm/include/rte_pmu_pmc.h create mode 100644 lib/eal/arm/rte_pmu.c
app/test/test_pmu.c +4 -0
@@ -13,6 +13,10 @@ test_pmu_read(void)int tries =10 ;int event =-1 ; +#if defined(RTE_ARCH_ARM64 ) + event =rte_pmu_add_event ("cpu_cycles" ); +#endif +while (tries--) val +=rte_pmu_read (event);
lib/eal/arm/include/meson.build +1 -0
@@ -20,6 +20,7 @@ arch_headers = files( 'rte_pause_32.h', 'rte_pause_64.h', 'rte_pause.h', + 'rte_pmu_pmc.h', 'rte_power_intrinsics.h', 'rte_prefetch_32.h', 'rte_prefetch_64.h',
lib/eal/arm/include/rte_pmu_pmc.h new file +37 -0
@@ -0,0 +1,37 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2022 Marvell. +*/ + +#ifndef _RTE_PMU_PMC_ARM_H_ +#define _RTE_PMU_PMC_ARM_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +static __rte_always_inline uint64_t +rte_pmu_pmc_read (int index) +{ +uint64_t val; + +if (index ==31 ) { +/* CPU Cycles (0x11) must be read via pmccntr_el0 */ + asmvolatile ("mrs %0, pmccntr_el0" :"=r" (val)); + }else { + asmvolatile ( +"msr pmselr_el0, %x0\n" +"mrs %0, pmxevcntr_el0\n" + :"=r" (val) + :"rZ" (index) + ); + } + +return val; +} +#define rte_pmu_pmc_read rte_pmu_pmc_read + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_PMU_PMC_ARM_H_ */
lib/eal/arm/meson.build +4 -0
@@ -9,3 +9,7 @@ sources += files( 'rte_hypervisor.c', 'rte_power_intrinsics.c', ) + +if is_linux + sources += files('rte_pmu.c') +endif
lib/eal/arm/rte_pmu.c new file +103 -0
103 lines not on this page · raw message
lib/eal/include/rte_pmu.h +3 -0
@@ -20,6 +20,9 @@ extern "C" {#include <rte_branch_prediction.h> #include <rte_lcore.h> #include <rte_log.h> +#if defined(RTE_ARCH_ARM64 ) +#include <rte_pmu_pmc.h> +#endif /** * @file
Signature · 1 line
2.25.1
PATCH 3/4 eal/x86: support reading Intel PMU events in runtime
In reply to Tomasz DuszynskiAdd support for reading Intel PMU events in runtime.
--- app/test/test_pmu.c | 2 ++ lib/eal/include/rte_pmu.h | 2 +- lib/eal/x86/include/meson.build | 1 + lib/eal/x86/include/rte_pmu_pmc.h | 32 +++++++++++++++++++++++++++++++ 4 files changed, 36 insertions(+), 1 deletion(-) create mode 100644 lib/eal/x86/include/rte_pmu_pmc.h
app/test/test_pmu.c +2 -0
@@ -15,6 +15,8 @@ test_pmu_read(void)#if defined(RTE_ARCH_ARM64 ) event =rte_pmu_add_event ("cpu_cycles" ); +#elif defined(RTE_ARCH_X86_64 ) + event =rte_pmu_add_event ("cpu-cycles" );#endif while (tries--)
lib/eal/include/rte_pmu.h +1 -1
@@ -20,7 +20,7 @@ extern "C" {#include <rte_branch_prediction.h> #include <rte_lcore.h> #include <rte_log.h> -#if defined(RTE_ARCH_ARM64 ) +#if defined(RTE_ARCH_ARM64 ) || defined(RTE_ARCH_X86_64 )#include <rte_pmu_pmc.h> #endif
lib/eal/x86/include/meson.build +1 -0
@@ -9,6 +9,7 @@ arch_headers = files( 'rte_io.h', 'rte_memcpy.h', 'rte_pause.h', + 'rte_pmu_pmc.h', 'rte_power_intrinsics.h', 'rte_prefetch.h', 'rte_rtm.h',
lib/eal/x86/include/rte_pmu_pmc.h new file +32 -0
@@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2022 Marvell. +*/ + +#ifndef _RTE_PMU_PMC_X86_H_ +#define _RTE_PMU_PMC_X86_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +__rte_internal +static __rte_always_inline uint64_t +rte_pmu_pmc_read (int index) +{ +uint32_t high, low; + + asmvolatile ( +"rdpmc\n" + :"=a" (low),"=d" (high) + :"c" (index) + ); + +return ((uint64_t )high <<32 ) | (uint64_t )low; +} +#define rte_pmu_pmc_read rte_pmu_pmc_read + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_PMU_PMC_X86_H_ */
Signature · 1 line
2.25.1
PATCH 4/4 eal: add PMU support to tracing library
In reply to Tomasz DuszynskiIn order to profile app one needs to store significant amount of samples somewhere for an analysis latern on. Since trace library supports storing data in a CTF format lets take adventage of that and add a dedicated PMU tracepoint.
--- app/test/test_trace_perf.c | 4 ++ doc/guides/prog_guide/profile_app.rst | 5 ++ doc/guides/prog_guide/trace_lib.rst | 32 ++++++++++++ lib/eal/common/eal_common_trace_points.c | 3 ++ lib/eal/common/rte_pmu.c | 63 ++++++++++++++++++++++++ lib/eal/include/rte_eal_trace.h | 11 +++++ lib/eal/version.map | 1 + 7 files changed, 119 insertions(+)
app/test/test_trace_perf.c +4 -0
@@ -114,6 +114,8 @@ worker_fn_##func(void *arg) \#define GENERIC_DOUBLE rte_eal_trace_generic_double(3.66666)#define GENERIC_STR rte_eal_trace_generic_str("hello world")#define VOID_FP app_dpdk_test_fp() +/* 0 corresponds first event passed via --trace= */ +#define READ_PMU rte_eal_trace_pmu_read(0)WORKER_DEFINE (GENERIC_VOID)WORKER_DEFINE (GENERIC_U64)
@@ -122,6 +124,7 @@ WORKER_DEFINE(GENERIC_FLOAT) WORKER_DEFINE(GENERIC_DOUBLE) WORKER_DEFINE(GENERIC_STR) WORKER_DEFINE(VOID_FP) +WORKER_DEFINE(READ_PMU) static void run_test(const char *str, lcore_function_t f, struct test_data *data, size_t sz)
@@ -174,6 +177,7 @@ test_trace_perf(void)run_test ("double" , worker_fn_GENERIC_DOUBLE, data, sz);run_test ("string" , worker_fn_GENERIC_STR, data, sz);run_test ("void_fp" , worker_fn_VOID_FP, data, sz); +run_test ("read_pmu" , worker_fn_READ_PMU, data, sz);rte_free (data);return TEST_SUCCESS ;
doc/guides/prog_guide/profile_app.rst +5 -0
@@ -16,6 +16,11 @@ that information, perf being an example here. Though in some scenarios, eg. when isolated (nohz_full) and run dedicated tasks, using perf is less than ideal. In such cases one can read specific events directly from application via ``rte_pmu_read()``. +Alternatively tracing library can be used which offers dedicated tracepoint +``rte_eal_trace_pmu_event()``. + +Refer to :doc:`../prog_guide/trace_lib` for more details. + Profiling on x86 ----------------
doc/guides/prog_guide/trace_lib.rst +32 -0
@@ -46,6 +46,7 @@ DPDK tracing library features trace format and is compatible with ``LTTng``. For detailed information, refer to `Common Trace Format <https://diamon.org/ctf/>`_. +- Support reading PMU events on ARM64 and x86 (Intel) How to add a tracepoint? ------------------------
@@ -137,6 +138,37 @@ the user must use ``RTE_TRACE_POINT_FP`` instead of ``RTE_TRACE_POINT``. ``RTE_TRACE_POINT_FP`` is compiled out by default and it can be enabled using the ``enable_trace_fp`` option for meson build. +PMU tracepoint +-------------- + +Performance measurement unit (PMU) event values can be read from hardware +registers using predefined ``rte_pmu_read`` tracepoint. + +Tracing is enabled via ``--trace`` EAL option by passing both expression +matching PMU tracepoint name i.e ``lib.eal.pmu.read`` and expression +``e=ev1[,ev2,...]`` matching particular events:: + + --trace='*pmu.read\|e=cpu_cycles,l1d_cache' + +Event names are available under ``/sys/bus/event_source/devices/PMU/events`` +directory, where ``PMU`` is a placeholder for either a ``cpu`` or a directory +containing ``cpus``. + +In contrary to other tracepoints this does not need any extra variables +added to source files. Instead, caller passes index which follows the order of +events specified via ``--trace`` parameter. In the following example index ``0`` +corresponds to ``cpu_cyclces`` while index ``1`` corresponds to ``l1d_cache``. + +.. code-block:: c + + ... + rte_eal_trace_pmu_read(0); + rte_eal_trace_pmu_read(1); + ... + +PMU tracing support must be explicitly enabled using the ``enable_trace_fp`` +option for meson build. + Event record mode -----------------
lib/eal/common/eal_common_trace_points.c +3 -0
@@ -75,3 +75,6 @@ RTE_TRACE_POINT_REGISTER(rte_eal_trace_intr_enable, lib.eal.intr.enable) RTE_TRACE_POINT_REGISTER(rte_eal_trace_intr_disable, lib.eal.intr.disable) + +RTE_TRACE_POINT_REGISTER (rte_eal_trace_pmu_read, + lib.eal.pmu.read)
lib/eal/common/rte_pmu.c +63 -0
86 lines not on this page · raw message
lib/eal/include/rte_eal_trace.h +11 -0
@@ -17,6 +17,7 @@ extern "C" {#include <rte_alarm.h> #include <rte_interrupts.h> +#include <rte_pmu.h> #include <rte_trace_point.h> #include "eal_interrupts.h"
@@ -279,6 +280,16 @@ RTE_TRACE_POINT(rte_trace_point_emit_string (cpuset); ) +/* PMU */ +RTE_TRACE_POINT_FP ( + rte_eal_trace_pmu_read, + RTE_TRACE_POINT_ARGS(int index), + uint64_t val; + rte_trace_point_emit_int(index); + val = rte_pmu_read(index); + rte_trace_point_emit_u64(val); +) + #ifdef __cplusplus }#endif
lib/eal/version.map +1 -0
@@ -432,6 +432,7 @@ EXPERIMENTAL { rte_thread_set_priority; # added in 22.11 + __rte_eal_trace_pmu_read; # WINDOWS_NO_EXPORT rte_pmu_add_event; # WINDOWS_NO_EXPORT rte_pmu_read; # WINDOWS_NO_EXPORT rte_thread_attr_get_affinity;
Signature · 1 line
2.25.1
v2 cover add support for self monitoring
In reply to Tomasz DuszynskiThis series adds self monitoring support i.e allows to configure and read performance measurement unit (PMU) counters in runtime without using perf utility. This has certain adventages when application runs on isolated cores with nohz_full kernel parameter.
Events can be read directly using rte_pmu_read() or using dedicated tracepoint rte_eal_trace_pmu_read(). The latter will cause events to be stored inside CTF file.
By design, all enabled events are grouped together and the same group is attached to lcores that use self monitoring funtionality.
Events are enabled by names, which need to be read from standard location under sysfs i.e
/sys/bus/event_source/devices/PMU/events
where PMU is a core pmu i.e one measuring cpu events. As of today raw events are not supported.
v2:
- fix problems reported by test build infra
Tomasz Duszynski (4):
eal: add generic support for reading PMU events eal/arm: support reading ARM PMU events in runtime eal/x86: support reading Intel PMU events in runtime
eal: add PMU support to tracing library
app/test/meson.build | 1 + app/test/test_pmu.c | 47 ++ app/test/test_trace_perf.c | 4 + doc/guides/prog_guide/profile_app.rst | 13 + doc/guides/prog_guide/trace_lib.rst | 32 ++ lib/eal/arm/include/meson.build | 1 + lib/eal/arm/include/rte_pmu_pmc.h | 39 ++ lib/eal/arm/meson.build | 4 + lib/eal/arm/rte_pmu.c | 103 +++++ lib/eal/common/eal_common_trace_points.c | 3 + lib/eal/common/meson.build | 3 + lib/eal/common/pmu_private.h | 41 ++ lib/eal/common/rte_pmu.c | 519 +++++++++++++++++++++++ lib/eal/include/meson.build | 1 + lib/eal/include/rte_eal_trace.h | 11 + lib/eal/include/rte_pmu.h | 207 +++++++++ lib/eal/linux/eal.c | 4 + lib/eal/version.map | 6 + lib/eal/x86/include/meson.build | 1 + lib/eal/x86/include/rte_pmu_pmc.h | 33 ++ 20 files changed, 1073 insertions(+) create mode 100644 app/test/test_pmu.c create mode 100644 lib/eal/arm/include/rte_pmu_pmc.h create mode 100644 lib/eal/arm/rte_pmu.c create mode 100644 lib/eal/common/pmu_private.h create mode 100644 lib/eal/common/rte_pmu.c create mode 100644 lib/eal/include/rte_pmu.h create mode 100644 lib/eal/x86/include/rte_pmu_pmc.h
--
2.25.1
PATCH v2 1/4 eal: add generic support for reading PMU events
Add support for programming PMU counters and reading their values in runtime bypassing kernel completely.
This is especially useful in cases where CPU cores are isolated (nohz_full) i.e run dedicated tasks. In such cases one cannot use standard perf utility without sacrificing latency and performance.
--- app/test/meson.build | 1 + app/test/test_pmu.c | 41 +++ doc/guides/prog_guide/profile_app.rst | 8 + lib/eal/common/meson.build | 3 + lib/eal/common/pmu_private.h | 41 +++ lib/eal/common/rte_pmu.c | 456 ++++++++++++++++++++++++++ lib/eal/include/meson.build | 1 + lib/eal/include/rte_pmu.h | 204 ++++++++++++ lib/eal/linux/eal.c | 4 + lib/eal/version.map | 5 + 10 files changed, 764 insertions(+) create mode 100644 app/test/test_pmu.c create mode 100644 lib/eal/common/pmu_private.h create mode 100644 lib/eal/common/rte_pmu.c create mode 100644 lib/eal/include/rte_pmu.h
app/test/meson.build +1 -0
@@ -143,6 +143,7 @@ test_sources = files( 'test_timer_racecond.c', 'test_timer_secondary.c', 'test_ticketlock.c', + 'test_pmu.c', 'test_trace.c', 'test_trace_register.c', 'test_trace_perf.c',
app/test/test_pmu.c new file +41 -0
41 lines not on this page · raw message
doc/guides/prog_guide/profile_app.rst +8 -0
@@ -7,6 +7,14 @@ Profile Your Application The following sections describe methods of profiling DPDK applications on different architectures. +Performance counter based profiling +----------------------------------- + +Majority of architectures support some sort hardware measurement unit which provides a set of +programmable counters that monitor specific events. There are different tools which can gather +that information, perf being an example here. Though in some scenarios, eg. when CPU cores are +isolated (nohz_full) and run dedicated tasks, using perf is less than ideal. In such cases one can +read specific events directly from application via ``rte_pmu_read()``. Profiling on x86 ----------------
lib/eal/common/meson.build +3 -0
@@ -38,6 +38,9 @@ sources += files( 'rte_service.c', 'rte_version.c', ) +if is_linux + sources += files('rte_pmu.c') +endif if is_linux or is_windows sources += files('eal_common_dynmem.c') endif
lib/eal/common/pmu_private.h new file +41 -0
41 lines not on this page · raw message
lib/eal/common/rte_pmu.c new file +456 -0
456 lines not on this page · raw message
lib/eal/include/meson.build +1 -0
@@ -36,6 +36,7 @@ headers += files( 'rte_pci_dev_features.h', 'rte_per_lcore.h', 'rte_pflock.h', + 'rte_pmu.h', 'rte_random.h', 'rte_reciprocal.h', 'rte_seqcount.h',
lib/eal/include/rte_pmu.h new file +204 -0
204 lines not on this page · raw message
lib/eal/linux/eal.c +4 -0
@@ -53,6 +53,7 @@#include "eal_options.h" #include "eal_vfio.h" #include "hotplug_mp.h" +#include "pmu_private.h" #define MEMSIZE_IF_NO_HUGE_PAGE (64ULL * 1024ULL * 1024ULL)
@@ -1206,6 +1207,8 @@ rte_eal_init(int argc, char **argv)return -1 ; } +eal_pmu_init (); +if (rte_eal_tailqs_init () <0 ) {rte_eal_init_alert ("Cannot init tail queues for objects" ); rte_errno =EFAULT ;
@@ -1372,6 +1375,7 @@ rte_eal_cleanup(void)eal_bus_cleanup ();rte_trace_save ();eal_trace_fini (); +eal_pmu_fini ();/* after this point, any DPDK pointers will become dangling */ rte_eal_memory_detach ();eal_mp_dev_hotplug_cleanup ();
lib/eal/version.map +5 -0
@@ -440,6 +440,10 @@ EXPERIMENTAL { rte_thread_detach; rte_thread_equal; rte_thread_join; + + # added in 23.03 + rte_pmu_add_event; # WINDOWS_NO_EXPORT + rte_pmu_read; # WINDOWS_NO_EXPORT }; INTERNAL {
@@ -483,4 +487,5 @@ INTERNAL { rte_mem_map; rte_mem_page_size; rte_mem_unmap; + rte_pmu_enable_group; };
Signature · 1 line
2.25.1
PATCH v2 2/4 eal/arm: support reading ARM PMU events in runtime
In reply to Tomasz DuszynskiAdd support for reading ARM PMU events in runtime.
--- app/test/test_pmu.c | 4 ++ lib/eal/arm/include/meson.build | 1 + lib/eal/arm/include/rte_pmu_pmc.h | 39 +++++++++++ lib/eal/arm/meson.build | 4 ++ lib/eal/arm/rte_pmu.c | 103 ++++++++++++++++++++++++++++++ lib/eal/include/rte_pmu.h | 3 + 6 files changed, 154 insertions(+) create mode 100644 lib/eal/arm/include/rte_pmu_pmc.h create mode 100644 lib/eal/arm/rte_pmu.c
app/test/test_pmu.c +4 -0
@@ -13,6 +13,10 @@ test_pmu_read(void)int tries =10 ;int event =-1 ; +#if defined(RTE_ARCH_ARM64 ) + event =rte_pmu_add_event ("cpu_cycles" ); +#endif +while (tries--) val +=rte_pmu_read (event);
lib/eal/arm/include/meson.build +1 -0
@@ -20,6 +20,7 @@ arch_headers = files( 'rte_pause_32.h', 'rte_pause_64.h', 'rte_pause.h', + 'rte_pmu_pmc.h', 'rte_power_intrinsics.h', 'rte_prefetch_32.h', 'rte_prefetch_64.h',
lib/eal/arm/include/rte_pmu_pmc.h new file +39 -0
@@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2022 Marvell. +*/ + +#ifndef _RTE_PMU_PMC_ARM_H_ +#define _RTE_PMU_PMC_ARM_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include <rte_common.h> + +static __rte_always_inline uint64_t +rte_pmu_pmc_read (int index) +{ +uint64_t val; + +if (index ==31 ) { +/* CPU Cycles (0x11) must be read via pmccntr_el0 */ + asmvolatile ("mrs %0, pmccntr_el0" :"=r" (val)); + }else { + asmvolatile ( +"msr pmselr_el0, %x0\n" +"mrs %0, pmxevcntr_el0\n" + :"=r" (val) + :"rZ" (index) + ); + } + +return val; +} +#define rte_pmu_pmc_read rte_pmu_pmc_read + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_PMU_PMC_ARM_H_ */
lib/eal/arm/meson.build +4 -0
@@ -9,3 +9,7 @@ sources += files( 'rte_hypervisor.c', 'rte_power_intrinsics.c', ) + +if is_linux + sources += files('rte_pmu.c') +endif
lib/eal/arm/rte_pmu.c new file +103 -0
103 lines not on this page · raw message
lib/eal/include/rte_pmu.h +3 -0
@@ -20,6 +20,9 @@ extern "C" {#include <rte_branch_prediction.h> #include <rte_lcore.h> #include <rte_log.h> +#if defined(RTE_ARCH_ARM64 ) +#include <rte_pmu_pmc.h> +#endif /** * @file
Signature · 1 line
2.25.1
PATCH v2 3/4 eal/x86: support reading Intel PMU events in runtime
In reply to Tomasz DuszynskiAdd support for reading Intel PMU events in runtime.
--- app/test/test_pmu.c | 2 ++ lib/eal/include/rte_pmu.h | 2 +- lib/eal/x86/include/meson.build | 1 + lib/eal/x86/include/rte_pmu_pmc.h | 33 +++++++++++++++++++++++++++++++ 4 files changed, 37 insertions(+), 1 deletion(-) create mode 100644 lib/eal/x86/include/rte_pmu_pmc.h
app/test/test_pmu.c +2 -0
@@ -15,6 +15,8 @@ test_pmu_read(void)#if defined(RTE_ARCH_ARM64 ) event =rte_pmu_add_event ("cpu_cycles" ); +#elif defined(RTE_ARCH_X86_64 ) + event =rte_pmu_add_event ("cpu-cycles" );#endif while (tries--)
lib/eal/include/rte_pmu.h +1 -1
@@ -20,7 +20,7 @@ extern "C" {#include <rte_branch_prediction.h> #include <rte_lcore.h> #include <rte_log.h> -#if defined(RTE_ARCH_ARM64 ) +#if defined(RTE_ARCH_ARM64 ) || defined(RTE_ARCH_X86_64 )#include <rte_pmu_pmc.h> #endif
lib/eal/x86/include/meson.build +1 -0
@@ -9,6 +9,7 @@ arch_headers = files( 'rte_io.h', 'rte_memcpy.h', 'rte_pause.h', + 'rte_pmu_pmc.h', 'rte_power_intrinsics.h', 'rte_prefetch.h', 'rte_rtm.h',
lib/eal/x86/include/rte_pmu_pmc.h new file +33 -0
@@ -0,0 +1,33 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2022 Marvell. +*/ + +#ifndef _RTE_PMU_PMC_X86_H_ +#define _RTE_PMU_PMC_X86_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include <rte_common.h> + +static __rte_always_inline uint64_t +rte_pmu_pmc_read (int index) +{ +uint32_t high, low; + + asmvolatile ( +"rdpmc\n" + :"=a" (low),"=d" (high) + :"c" (index) + ); + +return ((uint64_t )high <<32 ) | (uint64_t )low; +} +#define rte_pmu_pmc_read rte_pmu_pmc_read + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_PMU_PMC_X86_H_ */
Signature · 1 line
2.25.1
PATCH v2 4/4 eal: add PMU support to tracing library
In reply to Tomasz DuszynskiIn order to profile app one needs to store significant amount of samples somewhere for an analysis latern on. Since trace library supports storing data in a CTF format lets take adventage of that and add a dedicated PMU tracepoint.
--- app/test/test_trace_perf.c | 4 ++ doc/guides/prog_guide/profile_app.rst | 5 ++ doc/guides/prog_guide/trace_lib.rst | 32 ++++++++++++ lib/eal/common/eal_common_trace_points.c | 3 ++ lib/eal/common/rte_pmu.c | 63 ++++++++++++++++++++++++ lib/eal/include/rte_eal_trace.h | 11 +++++ lib/eal/version.map | 1 + 7 files changed, 119 insertions(+)
app/test/test_trace_perf.c +4 -0
@@ -114,6 +114,8 @@ worker_fn_##func(void *arg) \#define GENERIC_DOUBLE rte_eal_trace_generic_double(3.66666)#define GENERIC_STR rte_eal_trace_generic_str("hello world")#define VOID_FP app_dpdk_test_fp() +/* 0 corresponds first event passed via --trace= */ +#define READ_PMU rte_eal_trace_pmu_read(0)WORKER_DEFINE (GENERIC_VOID)WORKER_DEFINE (GENERIC_U64)
@@ -122,6 +124,7 @@ WORKER_DEFINE(GENERIC_FLOAT) WORKER_DEFINE(GENERIC_DOUBLE) WORKER_DEFINE(GENERIC_STR) WORKER_DEFINE(VOID_FP) +WORKER_DEFINE(READ_PMU) static void run_test(const char *str, lcore_function_t f, struct test_data *data, size_t sz)
@@ -174,6 +177,7 @@ test_trace_perf(void)run_test ("double" , worker_fn_GENERIC_DOUBLE, data, sz);run_test ("string" , worker_fn_GENERIC_STR, data, sz);run_test ("void_fp" , worker_fn_VOID_FP, data, sz); +run_test ("read_pmu" , worker_fn_READ_PMU, data, sz);rte_free (data);return TEST_SUCCESS ;
doc/guides/prog_guide/profile_app.rst +5 -0
@@ -16,6 +16,11 @@ that information, perf being an example here. Though in some scenarios, eg. when isolated (nohz_full) and run dedicated tasks, using perf is less than ideal. In such cases one can read specific events directly from application via ``rte_pmu_read()``. +Alternatively tracing library can be used which offers dedicated tracepoint +``rte_eal_trace_pmu_event()``. + +Refer to :doc:`../prog_guide/trace_lib` for more details. + Profiling on x86 ----------------
doc/guides/prog_guide/trace_lib.rst +32 -0
@@ -46,6 +46,7 @@ DPDK tracing library features trace format and is compatible with ``LTTng``. For detailed information, refer to `Common Trace Format <https://diamon.org/ctf/>`_. +- Support reading PMU events on ARM64 and x86 (Intel) How to add a tracepoint? ------------------------
@@ -137,6 +138,37 @@ the user must use ``RTE_TRACE_POINT_FP`` instead of ``RTE_TRACE_POINT``. ``RTE_TRACE_POINT_FP`` is compiled out by default and it can be enabled using the ``enable_trace_fp`` option for meson build. +PMU tracepoint +-------------- + +Performance measurement unit (PMU) event values can be read from hardware +registers using predefined ``rte_pmu_read`` tracepoint. + +Tracing is enabled via ``--trace`` EAL option by passing both expression +matching PMU tracepoint name i.e ``lib.eal.pmu.read`` and expression +``e=ev1[,ev2,...]`` matching particular events:: + + --trace='*pmu.read\|e=cpu_cycles,l1d_cache' + +Event names are available under ``/sys/bus/event_source/devices/PMU/events`` +directory, where ``PMU`` is a placeholder for either a ``cpu`` or a directory +containing ``cpus``. + +In contrary to other tracepoints this does not need any extra variables +added to source files. Instead, caller passes index which follows the order of +events specified via ``--trace`` parameter. In the following example index ``0`` +corresponds to ``cpu_cyclces`` while index ``1`` corresponds to ``l1d_cache``. + +.. code-block:: c + + ... + rte_eal_trace_pmu_read(0); + rte_eal_trace_pmu_read(1); + ... + +PMU tracing support must be explicitly enabled using the ``enable_trace_fp`` +option for meson build. + Event record mode -----------------
lib/eal/common/eal_common_trace_points.c +3 -0
@@ -75,3 +75,6 @@ RTE_TRACE_POINT_REGISTER(rte_eal_trace_intr_enable, lib.eal.intr.enable) RTE_TRACE_POINT_REGISTER(rte_eal_trace_intr_disable, lib.eal.intr.disable) + +RTE_TRACE_POINT_REGISTER (rte_eal_trace_pmu_read, + lib.eal.pmu.read)
lib/eal/common/rte_pmu.c +63 -0
86 lines not on this page · raw message
lib/eal/include/rte_eal_trace.h +11 -0
@@ -17,6 +17,7 @@ extern "C" {#include <rte_alarm.h> #include <rte_interrupts.h> +#include <rte_pmu.h> #include <rte_trace_point.h> #include "eal_interrupts.h"
@@ -279,6 +280,16 @@ RTE_TRACE_POINT(rte_trace_point_emit_string (cpuset); ) +/* PMU */ +RTE_TRACE_POINT_FP ( + rte_eal_trace_pmu_read, + RTE_TRACE_POINT_ARGS(int index), + uint64_t val; + rte_trace_point_emit_int(index); + val = rte_pmu_read(index); + rte_trace_point_emit_u64(val); +) + #ifdef __cplusplus }#endif
lib/eal/version.map +1 -0
@@ -442,6 +442,7 @@ EXPERIMENTAL { rte_thread_join; # added in 23.03 + __rte_eal_trace_pmu_read; # WINDOWS_NO_EXPORT rte_pmu_add_event; # WINDOWS_NO_EXPORT rte_pmu_read; # WINDOWS_NO_EXPORT };
Signature · 1 line
2.25.1
v3 cover add support for self monitoring
In reply to Tomasz DuszynskiThis series adds self monitoring support i.e allows to configure and read performance measurement unit (PMU) counters in runtime without using perf utility. This has certain adventages when application runs on isolated cores with nohz_full kernel parameter.
Events can be read directly using rte_pmu_read() or using dedicated tracepoint rte_eal_trace_pmu_read(). The latter will cause events to be stored inside CTF file.
By design, all enabled events are grouped together and the same group is attached to lcores that use self monitoring funtionality.
Events are enabled by names, which need to be read from standard location under sysfs i.e
/sys/bus/event_source/devices/PMU/events
where PMU is a core pmu i.e one measuring cpu events. As of today raw events are not supported.
v3:
- fix shared build
v2:
- fix problems reported by test build infra
Tomasz Duszynski (4):
eal: add generic support for reading PMU events eal/arm: support reading ARM PMU events in runtime eal/x86: support reading Intel PMU events in runtime
eal: add PMU support to tracing library
app/test/meson.build | 1 + app/test/test_pmu.c | 47 ++ app/test/test_trace_perf.c | 4 + doc/guides/prog_guide/profile_app.rst | 13 + doc/guides/prog_guide/trace_lib.rst | 32 ++ lib/eal/arm/include/meson.build | 1 + lib/eal/arm/include/rte_pmu_pmc.h | 39 ++ lib/eal/arm/meson.build | 4 + lib/eal/arm/rte_pmu.c | 104 +++++ lib/eal/common/eal_common_trace_points.c | 3 + lib/eal/common/meson.build | 3 + lib/eal/common/pmu_private.h | 41 ++ lib/eal/common/rte_pmu.c | 520 +++++++++++++++++++++++ lib/eal/include/meson.build | 1 + lib/eal/include/rte_eal_trace.h | 11 + lib/eal/include/rte_pmu.h | 207 +++++++++ lib/eal/linux/eal.c | 4 + lib/eal/version.map | 7 + lib/eal/x86/include/meson.build | 1 + lib/eal/x86/include/rte_pmu_pmc.h | 33 ++ 20 files changed, 1076 insertions(+) create mode 100644 app/test/test_pmu.c create mode 100644 lib/eal/arm/include/rte_pmu_pmc.h create mode 100644 lib/eal/arm/rte_pmu.c create mode 100644 lib/eal/common/pmu_private.h create mode 100644 lib/eal/common/rte_pmu.c create mode 100644 lib/eal/include/rte_pmu.h create mode 100644 lib/eal/x86/include/rte_pmu_pmc.h
--
2.25.1
PATCH v3 1/4 eal: add generic support for reading PMU events
Add support for programming PMU counters and reading their values in runtime bypassing kernel completely.
This is especially useful in cases where CPU cores are isolated (nohz_full) i.e run dedicated tasks. In such cases one cannot use standard perf utility without sacrificing latency and performance.
--- app/test/meson.build | 1 + app/test/test_pmu.c | 41 +++ doc/guides/prog_guide/profile_app.rst | 8 + lib/eal/common/meson.build | 3 + lib/eal/common/pmu_private.h | 41 +++ lib/eal/common/rte_pmu.c | 457 ++++++++++++++++++++++++++ lib/eal/include/meson.build | 1 + lib/eal/include/rte_pmu.h | 204 ++++++++++++ lib/eal/linux/eal.c | 4 + lib/eal/version.map | 6 + 10 files changed, 766 insertions(+) create mode 100644 app/test/test_pmu.c create mode 100644 lib/eal/common/pmu_private.h create mode 100644 lib/eal/common/rte_pmu.c create mode 100644 lib/eal/include/rte_pmu.h
app/test/meson.build +1 -0
@@ -143,6 +143,7 @@ test_sources = files( 'test_timer_racecond.c', 'test_timer_secondary.c', 'test_ticketlock.c', + 'test_pmu.c', 'test_trace.c', 'test_trace_register.c', 'test_trace_perf.c',
app/test/test_pmu.c new file +41 -0
41 lines not on this page · raw message
doc/guides/prog_guide/profile_app.rst +8 -0
@@ -7,6 +7,14 @@ Profile Your Application The following sections describe methods of profiling DPDK applications on different architectures. +Performance counter based profiling +----------------------------------- + +Majority of architectures support some sort hardware measurement unit which provides a set of +programmable counters that monitor specific events. There are different tools which can gather +that information, perf being an example here. Though in some scenarios, eg. when CPU cores are +isolated (nohz_full) and run dedicated tasks, using perf is less than ideal. In such cases one can +read specific events directly from application via ``rte_pmu_read()``. Profiling on x86 ----------------
lib/eal/common/meson.build +3 -0
@@ -38,6 +38,9 @@ sources += files( 'rte_service.c', 'rte_version.c', ) +if is_linux + sources += files('rte_pmu.c') +endif if is_linux or is_windows sources += files('eal_common_dynmem.c') endif
lib/eal/common/pmu_private.h new file +41 -0
41 lines not on this page · raw message
lib/eal/common/rte_pmu.c new file +457 -0
457 lines not on this page · raw message
lib/eal/include/meson.build +1 -0
@@ -36,6 +36,7 @@ headers += files( 'rte_pci_dev_features.h', 'rte_per_lcore.h', 'rte_pflock.h', + 'rte_pmu.h', 'rte_random.h', 'rte_reciprocal.h', 'rte_seqcount.h',
lib/eal/include/rte_pmu.h new file +204 -0
204 lines not on this page · raw message
lib/eal/linux/eal.c +4 -0
@@ -53,6 +53,7 @@#include "eal_options.h" #include "eal_vfio.h" #include "hotplug_mp.h" +#include "pmu_private.h" #define MEMSIZE_IF_NO_HUGE_PAGE (64ULL * 1024ULL * 1024ULL)
@@ -1206,6 +1207,8 @@ rte_eal_init(int argc, char **argv)return -1 ; } +eal_pmu_init (); +if (rte_eal_tailqs_init () <0 ) {rte_eal_init_alert ("Cannot init tail queues for objects" ); rte_errno =EFAULT ;
@@ -1372,6 +1375,7 @@ rte_eal_cleanup(void)eal_bus_cleanup ();rte_trace_save ();eal_trace_fini (); +eal_pmu_fini ();/* after this point, any DPDK pointers will become dangling */ rte_eal_memory_detach ();eal_mp_dev_hotplug_cleanup ();
lib/eal/version.map +6 -0
@@ -440,6 +440,11 @@ EXPERIMENTAL { rte_thread_detach; rte_thread_equal; rte_thread_join; + + # added in 23.03 + rte_pmu; # WINDOWS_NO_EXPORT + rte_pmu_add_event; # WINDOWS_NO_EXPORT + rte_pmu_read; # WINDOWS_NO_EXPORT }; INTERNAL {
@@ -483,4 +488,5 @@ INTERNAL { rte_mem_map; rte_mem_page_size; rte_mem_unmap; + rte_pmu_enable_group; };
Signature · 1 line
2.25.1
Re: [PATCH v3 1/4] eal: add generic support for reading PMU events
Hi Tomasz,
Quoted 617 lines · Tomasz Duszynski
Add support for programming PMU counters and reading their values in runtime bypassing kernel completely.
This is especially useful in cases where CPU cores are isolated (
nohz_full) i.e run dedicated tasks. In such cases one cannot use standard perf utility without sacrificing latency and performance.Signed-off-by Tomasz Duszynski <[email protected]>--- app/test/meson.build | 1 + app/test/test_pmu.c | 41 +++ doc/guides/prog_guide/profile_app.rst | 8 + lib/eal/common/meson.build | 3 + lib/eal/common/pmu_private.h | 41 +++ lib/eal/common/rte_pmu.c | 457 ++++++++++++++++++++++++++ lib/eal/include/meson.build | 1 + lib/eal/include/rte_pmu.h | 204 ++++++++++++ lib/eal/linux/eal.c | 4 + lib/eal/version.map | 6 + 10 files changed, 766 insertions(+)create mode 100644
app/test/test_pmu.c
create mode 100644lib/eal/common/pmu_private.h
create mode 100644lib/eal/common/rte_pmu.c
create mode 100644lib/eal/include/rte_pmu.happ/test/meson.build +1
-0@@ -143,6 +143,7 @@ test_sources = files( 'test_timer_racecond.c', 'test_timer_secondary.c', 'test_ticketlock.c', + 'test_pmu.c', 'test_trace.c', 'test_trace_register.c', 'test_trace_perf.c',app/test/test_pmu.c new file +41
-0@@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(C) 2022 Marvell International Ltd. +*/ + +#include <rte_pmu.h> + +#include "test.h" + +static int +test_pmu_read (void ) +{ +uint64_t val =0 ; +int tries =10 ; +int event =-1 ; + +while (tries--) + val +=rte_pmu_read (event); + +if (val ==0 ) +return TEST_FAILED ; + +return TEST_SUCCESS ; +} + +static struct unit_test_suite pmu_tests = { + .suite_name ="pmu autotest" , + .setup =NULL , + .teardown =NULL , + .unit_test_cases = { +TEST_CASE (test_pmu_read), +TEST_CASES_END () + } +}; + +static int +test_pmu (void ) +{ +return unit_test_suite_runner (&pmu_tests); +} + +REGISTER_TEST_COMMAND (pmu_autotest, test_pmu);doc/guides/prog_guide/profile_app.rst +8
-0@@ -7,6 +7,14 @@ Profile Your Application The following sections describe methods of profiling DPDK applications on different architectures. +Performance counter based profiling +----------------------------------- + +Majority of architectures support some sort hardware measurement unit which provides a set of +programmable counters that monitor specific events. There are different tools which can gather +that information, perf being an example here. Though in some scenarios, eg. when CPU cores are +isolated (nohz_full) and run dedicated tasks, using perf is less than ideal. In such cases one can +read specific events directly from application via ``rte_pmu_read()``. Profiling on x86 ----------------lib/eal/common/meson.build +3
-0@@ -38,6 +38,9 @@ sources += files( 'rte_service.c', 'rte_version.c', ) +if is_linux + sources += files('rte_pmu.c') +endif if is_linux or is_windows sources += files('eal_common_dynmem.c') endiflib/eal/common/pmu_private.h new file +41
-0@@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2022 Marvell +*/ + +#ifndef _PMU_PRIVATE_H_ +#define _PMU_PRIVATE_H_ + +/** +* Architecture specific PMU init callback. +* +* @return +* 0 in case of success, negative value otherwise. +*/ +int +pmu_arch_init (void ); + +/** +* Architecture specific PMU cleanup callback. +*/ +void +pmu_arch_fini (void ); + +/** +* Apply architecture specific settings to config before passing it to syscall. +*/ +void +pmu_arch_fixup_config (uint64_t config[3 ]); + +/** +* Initialize PMU tracing internals. +*/ +void +eal_pmu_init (void ); + +/** +* Cleanup PMU internals. +*/ +void +eal_pmu_fini (void ); + +#endif /* _PMU_PRIVATE_H_ */ lib/eal/common/rte_pmu.c new file +447
-0@@ -0,0 +1,457 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(C) 2022 Marvell International Ltd. +*/ + +#include <ctype.h> +#include <dirent.h> +#include <errno.h> +#include <regex.h> +#include <stdlib.h> +#include <string.h> +#include <sys/ioctl.h> +#include <sys/mman.h> +#include <sys/queue.h> +#include <sys/syscall.h> +#include <unistd.h> + +#include <rte_eal_paging.h> +#include <rte_malloc.h> +#include <rte_pmu.h> +#include <rte_tailq.h> + +#include "pmu_private.h" + +#define EVENT_SOURCE_DEVICES_PATH "/sys/bus/event_source/devices" + +#ifndef GENMASK_ULL +#define GENMASK_ULL (h, l) ((~0ULL - (1ULL << (l)) + 1) & (~0ULL >> ((64 - 1 - (h))))) +#endif + +#ifndef FIELD_PREP +#define FIELD_PREP (m, v) (((uint64_t)(v) << (__builtin_ffsll(m) - 1)) & (m)) +#endif + +struct rte_pmu *rte_pmu; + +/* +* Following __rte_weak functions provide default no-op. Architectures should override them if +* necessary. +*/ + +int +__rte_weak pmu_arch_init(void) +{ +return 0 ; +} + +void +__rte_weak pmu_arch_fini(void) +{ +} + +void +__rte_weak pmu_arch_fixup_config(uint64_t config[3]) +{ +RTE_SET_USED (config); +} + +static int +get_term_format (const char *name,int *num,uint64_t *mask) +{ +char *config =NULL ; +char path[PATH_MAX ]; +int high, low, ret; +FILE *fp; + +/* quiesce -Wmaybe-uninitialized warning */ + *num =0 ; + *mask =0 ; + +snprintf (path,sizeof (path),EVENT_SOURCE_DEVICES_PATH "/%s/format/%s" , rte_pmu->name, name); + fp =fopen (path,"r" ); +if (!fp) +return -errno; + + errno =0 ; + ret =fscanf (fp,"%m[^:]:%d-%d" , &config, &low, &high); +if (ret <2 ) { + ret = -ENODATA ; + goto out; + } +if (errno) { + ret = -errno; + goto out; + } + +if (ret ==2 ) + high = low; + + *mask =GENMASK_ULL (high, low); +/* Last digit should be [012]. If last digit is missing 0 is implied. */ + *num = config[strlen (config) -1 ]; + *num =isdigit (*num) ? *num -'0' :0 ; + + ret =0 ; +out: +free (config); +fclose (fp); + +return ret; +} + +static int +parse_event (char *buf,uint64_t config[3 ]) +{ +char *token, *term; +int num, ret, val; +uint64_t mask; + + config[0 ] = config[1 ] = config[2 ] =0 ; + + token =strtok (buf,"," ); +while (token) { + errno =0 ; +/* <term>=<value> */ + ret =sscanf (token,"%m[^=]=%i" , &term, &val); +if (ret <1 ) +return -ENODATA ; +if (errno) +return -errno; +if (ret ==1 ) + val =1 ; + + ret =get_term_format (term, &num, &mask); +free (term); +if (ret) +return ret; + + config[num] |=FIELD_PREP (mask, val); + token =strtok (NULL ,"," ); + } + +return 0 ; +} + +static int +get_event_config (const char *name,uint64_t config[3 ]) +{ +char path[PATH_MAX ], buf[BUFSIZ ]; +FILE *fp; +int ret; + +snprintf (path,sizeof (path),EVENT_SOURCE_DEVICES_PATH "/%s/events/%s" , rte_pmu->name, name); + fp =fopen (path,"r" ); +if (!fp) +return -errno; + + ret =fread (buf,1 ,sizeof (buf), fp); +if (ret ==0 ) { +fclose (fp); + +return -EINVAL ; + } +fclose (fp); + buf[ret] ='\0' ; + +return parse_event (buf, config); +} + +static int +do_perf_event_open (uint64_t config[3 ],int lcore_id,int group_fd) +{ +struct perf_event_attr attr = { + .size =sizeof (struct perf_event_attr ), + .type =PERF_TYPE_RAW , + .exclude_kernel =1 , + .exclude_hv =1 , + .disabled =1 , + }; + +pmu_arch_fixup_config (config); + + attr.config = config[0 ]; + attr.config1 = config[1 ]; + attr.config2 = config[2 ]; + +return syscall (SYS_perf_event_open, &attr,rte_gettid (),rte_lcore_to_cpu_id (lcore_id), + group_fd,0 ); +} + +static int +open_events (int lcore_id) +{ +struct rte_pmu_event_group *group = &rte_pmu->group[lcore_id]; +struct rte_pmu_event *event; +uint64_t config[3 ]; +int num =0 , ret; + +/* group leader gets created first, with fd = -1 */ + group->fds[0 ] =-1 ; + + TAILQ_FOREACH(event, &rte_pmu->event_list, next) { + ret =get_event_config (event->name, config); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to get %s event config\n" , event->name); +continue ; + } + + ret =do_perf_event_open (config, lcore_id, group->fds[0 ]); +if (ret ==-1 ) { +if (errno ==EOPNOTSUPP ) +RTE_LOG (ERR ,EAL ,"64 bit counters not supported\n" ); + + ret = -errno; + goto out; + } + + group->fds[event->index] = ret; + num++; + } + +return 0 ; +out: +for (--num; num >=0 ; num--) { +close (group->fds[num]); + group->fds[num] =-1 ; + } + + +return ret; +} + +static int +mmap_events (int lcore_id) +{ +struct rte_pmu_event_group *group = &rte_pmu->group[lcore_id]; +void *addr; +int ret, i; + +for (i =0 ; i < rte_pmu->num_group_events; i++) { + addr =mmap (0 ,rte_mem_page_size (),PROT_READ ,MAP_SHARED , group->fds[i],0 ); +if (addr ==MAP_FAILED ) { + ret = -errno; + goto out; + } + + group->mmap_pages[i] = addr; + } + +return 0 ; +out: +for (; i; i--) { +munmap (group->mmap_pages[i -1 ],rte_mem_page_size ()); + group->mmap_pages[i -1 ] =NULL ; + } + +return ret; +} + +static void +cleanup_events (int lcore_id) +{ +struct rte_pmu_event_group *group = &rte_pmu->group[lcore_id]; +int i; + +if (!group->fds) +return ; + +if (group->fds[0 ] !=-1 ) +ioctl (group->fds[0 ],PERF_EVENT_IOC_DISABLE ,PERF_IOC_FLAG_GROUP ); + +for (i =0 ; i < rte_pmu->num_group_events; i++) { +if (group->mmap_pages[i]) { +munmap (group->mmap_pages[i],rte_mem_page_size ()); + group->mmap_pages[i] =NULL ; + } + +if (group->fds[i] !=-1 ) { +close (group->fds[i]); + group->fds[i] =-1 ; + } + } + +rte_free (group->mmap_pages); +rte_free (group->fds); + + group->mmap_pages =NULL ; + group->fds =NULL ; + group->enabled = false; +} + +int __rte_noinline +rte_pmu_enable_group (int lcore_id) +{ +struct rte_pmu_event_group *group = &rte_pmu->group[lcore_id]; +int ret; + +if (rte_pmu->num_group_events ==0 ) { +RTE_LOG (DEBUG ,EAL ,"no matching PMU events\n" ); + +return 0 ; + } + + group->fds =rte_calloc (NULL , rte_pmu->num_group_events,sizeof (*group->fds),0 ); +if (!group->fds) { +RTE_LOG (ERR ,EAL ,"failed to alloc descriptor memory\n" ); + +return -ENOMEM ; + } + + group->mmap_pages =rte_calloc (NULL , rte_pmu->num_group_events,sizeof (*group->mmap_pages),0 ); +if (!group->mmap_pages) { +RTE_LOG (ERR ,EAL ,"failed to alloc userpage memory\n" ); + + ret = -ENOMEM ; + goto out; + } + + ret =open_events (lcore_id); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to open events on lcore-worker-%d\n" , lcore_id); + goto out; + } + + ret =mmap_events (lcore_id); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to map events on lcore-worker-%d\n" , lcore_id); + goto out; + } + +if (ioctl (group->fds[0 ],PERF_EVENT_IOC_ENABLE ,PERF_IOC_FLAG_GROUP ) ==-1 ) { +RTE_LOG (ERR ,EAL ,"failed to enable events on lcore-worker-%d\n" , lcore_id); + + ret = -errno; + goto out; + } + +return 0 ; + +out: +cleanup_events (lcore_id); + +return ret; +} + +static int +scan_pmus (void ) +{ +char path[PATH_MAX ]; +struct dirent *dent; +const char *name; +DIR *dirp; + + dirp =opendir (EVENT_SOURCE_DEVICES_PATH ); +if (!dirp) +return -errno; + +while ((dent =readdir (dirp))) { + name = dent->d_name; +if (name[0 ] =='.' ) +continue ; + +/* sysfs entry should either contain cpus or be a cpu */ +if (!strcmp (name,"cpu" )) +break ; + +snprintf (path,sizeof (path),EVENT_SOURCE_DEVICES_PATH "/%s/cpus" , name); +if (access (path,F_OK ) ==0 ) +break ; + } + +closedir (dirp); + +if (dent) { + rte_pmu->name =strdup (name); +if (!rte_pmu->name) +return -ENOMEM ; + } + +return rte_pmu->name ?0 : -ENODEV ; +} + +int +rte_pmu_add_event (const char *name) +{ +struct rte_pmu_event *event; +char path[PATH_MAX ]; + +snprintf (path,sizeof (path),EVENT_SOURCE_DEVICES_PATH "/%s/events/%s" , rte_pmu->name, name); +if (access (path,R_OK )) +return -ENODEV ; + + TAILQ_FOREACH(event, &rte_pmu->event_list, next) { +if (!strcmp (event->name, name)) +return event->index; +continue ; + } + + event =rte_calloc (NULL ,1 ,sizeof (*event),0 ); +if (!event) +return -ENOMEM ; + + event->name =strdup (name); +if (!event->name) { +rte_free (event); + +return -ENOMEM ; + } + + event->index = rte_pmu->num_group_events++; +TAILQ_INSERT_TAIL (&rte_pmu->event_list, event, next); + +RTE_LOG (DEBUG ,EAL ,"%s even added at index %d\n" , name, event->index); + +return event->index; +} + +void +eal_pmu_init (void ) +{ +int ret; + + rte_pmu =rte_calloc (NULL ,1 ,sizeof (*rte_pmu),RTE_CACHE_LINE_SIZE ); +if (!rte_pmu) { +RTE_LOG (ERR ,EAL ,"failed to alloc PMU\n" ); + +return ; + } + +TAILQ_INIT (&rte_pmu->event_list); + + ret =scan_pmus (); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to find core pmu\n" ); + goto out; + } + + ret =pmu_arch_init (); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to setup arch for PMU\n" ); + goto out; + } + +return ; +out: +free (rte_pmu->name); +rte_free (rte_pmu); +} + +void +eal_pmu_fini (void ) +{ +struct rte_pmu_event *event, *tmp; +int lcore_id; + + RTE_TAILQ_FOREACH_SAFE(event, &rte_pmu->event_list, next, tmp) { +TAILQ_REMOVE (&rte_pmu->event_list, event, next); +free (event->name);
lib/eal/common/rte_pmu.c line 457
Quoted 10 lines
lib/eal/common/rte_pmu.c new file +10
-0+rte_free (event); + } + +RTE_LCORE_FOREACH_WORKER (lcore_id) +cleanup_events (lcore_id); + +pmu_arch_fini (); +free (rte_pmu->name); +rte_free (rte_pmu); +}
There may be some problems with the implementation of eal_pmu_fini(), but I'm not sure.
I checked some test reports for this series. It seems that the test case of debug_autotest in
the DPDK unit test has an issue when the child process in this test case calls the function of rte_exit().
The call chain is as follows:
test_debug() -> test_exit() -> test_exit_val() -> rte_exit() -> rte_eal_cleanup() -> eal_pmu_fini().
The issue may be related to memory free from the error message as follows:
test_exit_valEAL: Error: Invalid memory
EAL: Error - exiting with code: 1
Cause: test_exit_valEAL: Error: Invalid memory
EAL: Error - exiting with code: 2
Cause: test_exit_valEAL: Error: Invalid memory
EAL: Error - exiting with code: 255
Cause: test_exit_valEAL: Error: Invalid memory
EAL: Error - exiting with code: -1
Cause: test_exit_valEAL: Error: Invalid memory
The above error message will disappear when I comment out the calling to the eal_pmu_fini() in
the rte_eal_cleanup().
Quoted 268 lines
lib/eal/include/meson.build +1
-0@@ -36,6 +36,7 @@ headers += files( 'rte_pci_dev_features.h', 'rte_per_lcore.h', 'rte_pflock.h', + 'rte_pmu.h', 'rte_random.h', 'rte_reciprocal.h', 'rte_seqcount.h',lib/eal/include/rte_pmu.h new file +204
-0@@ -0,0 +1,204 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2022 Marvell +*/ + +#ifndef _RTE_PMU_H_ +#define _RTE_PMU_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include <rte_common.h> +#include <rte_compat.h> + +#ifdef RTE_EXEC_ENV_LINUX + +#include <linux/perf_event.h> + +#include <rte_atomic.h> +#include <rte_branch_prediction.h> +#include <rte_lcore.h> +#include <rte_log.h> + +/** +* @file +* +* PMU event tracing operations +* +* This file defines generic API and types necessary to setup PMU and +* read selected counters in runtime. +*/ + +/** +* A structure describing a group of events. +*/ +struct rte_pmu_event_group { +int *fds;/**< array of event descriptors */ +void **mmap_pages;/**< array of pointers to mmapped perf_event_attr structures */ +bool enabled;/**< true if group was enabled on particular lcore */ +}; + +/** +* A structure describing an event. +*/ +struct rte_pmu_event { +char *name;/** name of an event */ +int index;/** event index into fds/mmap_pages */ +TAILQ_ENTRY (rte_pmu_event) next;/** list entry */ +}; + +/** +* A PMU state container. +*/ +struct rte_pmu { +char *name;/** name of core PMU listed under /sys/bus/event_source/devices */ +struct rte_pmu_event_group group[RTE_MAX_LCORE ];/**< per lcore event group data */ +int num_group_events;/**< number of events in a group */ + TAILQ_HEAD(, rte_pmu_event) event_list; /**< list of matching events */ +}; + +/** Pointer to the PMU state container */ +extern struct rte_pmu *rte_pmu; + +/** Each architecture supporting PMU needs to provide its own version */ +#ifndef rte_pmu_pmc_read +#define rte_pmu_pmc_read (index) ({ 0; }) +#endif + +/** +* @internal +* +* Read PMU counter. +* +* @param pc +* Pointer to the mmapped user page. +* @return +* Counter value read from hardware. +*/ +__rte_internal +static __rte_always_inline uint64_t +rte_pmu_read_userpage (struct perf_event_mmap_page *pc) +{ +uint64_t offset, width, pmc =0 ; +uint32_t seq, index; +int tries =100 ; + +for (;;) { + seq = pc->lock; +rte_compiler_barrier (); + index = pc->index; + offset = pc->offset; + width = pc->pmc_width; + +if (likely (pc->cap_user_rdpmc && index)) { + pmc =rte_pmu_pmc_read (index -1 ); + pmc <<=64 - width; + pmc >>=64 - width; + } + +rte_compiler_barrier (); + +if (likely (pc->lock == seq)) +return pmc + offset; + +if (--tries ==0 ) { +RTE_LOG (DEBUG ,EAL ,"failed to get perf_event_mmap_page lock\n" ); +break ; + } + } + +return 0 ; +} + +/** +* @internal +* +* Enable group of events for a given lcore. +* +* @param lcore_id +* The identifier of the lcore. +* @return +* 0 in case of success, negative value otherwise. +*/ +__rte_internal +int +rte_pmu_enable_group (int lcore_id); + +/** +* @warning +* @b EXPERIMENTAL: this API may change without prior notice +* +* Add event to the group of enabled events. +* +* @param name +* Name of an event listed under /sys/bus/event_source/devices/pmu/events. +* @return +* Event index in case of success, negative value otherwise. +*/ +__rte_experimental +int +rte_pmu_add_event (const char *name); + +/** +* @warning +* @b EXPERIMENTAL: this API may change without prior notice +* +* Read hardware counter configured to count occurrences of an event. +* +* @param index +* Index of an event to be read. +* @return +* Event value read from register. In case of errors or lack of support +* 0 is returned. In other words, stream of zeros in a trace file +* indicates problem with reading particular PMU event register. +*/ +__rte_experimental +static __rte_always_inline uint64_t +rte_pmu_read (int index) +{ +int lcore_id =rte_lcore_id (); +struct rte_pmu_event_group *group; +int ret; + +if (!rte_pmu) +return 0 ; + + group = &rte_pmu->group[lcore_id]; +if (!group->enabled) { + ret =rte_pmu_enable_group (lcore_id); +if (ret) +return 0 ; + + group->enabled = true; + } + +if (index <0 || index >= rte_pmu->num_group_events) +return 0 ; + +return rte_pmu_read_userpage ((struct perf_event_mmap_page *)group->mmap_pages[index]); +} + +#else /* !RTE_EXEC_ENV_LINUX */ + +__rte_experimental +static int __rte_unused +rte_pmu_add_event(__rte_unused const char *name) +{ +return -1 ; +} + +__rte_experimental +static __rte_always_inline uint64_t +rte_pmu_read(__rte_unused int index) +{ +return 0 ; +} + +#endif /* RTE_EXEC_ENV_LINUX */ + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_PMU_H_ */ lib/eal/linux/eal.c +4
-0@@ -53,6 +53,7 @@#include "eal_options.h" #include "eal_vfio.h" #include "hotplug_mp.h" +#include "pmu_private.h" #define MEMSIZE_IF_NO_HUGE_PAGE (64ULL * 1024ULL * 1024ULL)@@ -1206,6 +1207,8 @@ rte_eal_init(int argc, char **argv)return -1 ; } +eal_pmu_init (); +if (rte_eal_tailqs_init () <0 ) {rte_eal_init_alert ("Cannot init tail queues for objects" ); rte_errno =EFAULT ;@@ -1372,6 +1375,7 @@ rte_eal_cleanup(void)eal_bus_cleanup ();rte_trace_save ();eal_trace_fini (); +eal_pmu_fini ();/* after this point, any DPDK pointers will become dangling */ rte_eal_memory_detach ();eal_mp_dev_hotplug_cleanup ();lib/eal/version.map +5
-0@@ -440,6 +440,11 @@ EXPERIMENTAL { rte_thread_detach; rte_thread_equal; rte_thread_join; + + # added in 23.03 + rte_pmu; # WINDOWS_NO_EXPORT + rte_pmu_add_event; # WINDOWS_NO_EXPORT + rte_pmu_read; # WINDOWS_NO_EXPORT }; INTERNAL {
lib/eal/version.map INTERNAL { · line 492
Quoted 5 lines
lib/eal/version.map +1
-0@@ -483,4 +488,5 @@ INTERNAL { rte_mem_map; rte_mem_page_size; rte_mem_unmap; + rte_pmu_enable_group; };
Best regards,
Min Zhou
RE: [EXT] Re: [PATCH v3 1/4] eal: add generic support for reading PMU events
Hello Min,
Quoted 12 lines
-----Original Message-----
From: zhoumin <[email protected]>
Sent: Wednesday, November 30, 2022 9:33 AM
To: Tomasz Duszynski <[email protected]>; [email protected]
Cc: [email protected]; Jerin Jacob Kollanukkaran <[email protected]>
Subject: [EXT] Re: [PATCH v3 1/4] eal: add generic support for reading PMU eventsExternal Email
----------------------------------------------------------------------
Hi Tomasz,
[...]
Quoted 44 lines
Quoted 9 lines
lib/eal/common/rte_pmu.c new file +9
-0+void +eal_pmu_fini (void ) +{ +struct rte_pmu_event *event, *tmp; +int lcore_id; + + RTE_TAILQ_FOREACH_SAFE(event, &rte_pmu->event_list, next, tmp) { +TAILQ_REMOVE (&rte_pmu->event_list, event, next); +free (event->name);lib/eal/common/rte_pmu.c line 457
Quoted 10 lines
lib/eal/common/rte_pmu.c new file +10
-0+rte_free (event); + } + +RTE_LCORE_FOREACH_WORKER (lcore_id) +cleanup_events (lcore_id); + +pmu_arch_fini (); +free (rte_pmu->name); +rte_free (rte_pmu); +}There may be some problems with the implementation of
eal_pmu_fini(), but I'm not sure.I checked some test reports for this series. It seems that the test case of
debug_autotestinthe DPDK unit test has an issue when the child process in this test case calls the function of
rte_exit().The call chain is as follows:
test_debug()->test_exit()->test_exit_val()->rte_exit()->rte_eal_cleanup()->eal_pmu_fini().The issue may be related to memory free from the error message as follows:
test_exit_valEAL: Error: Invalid memory
EAL: Error - exiting with code: 1
Cause:test_exit_valEAL: Error: Invalid memory
EAL: Error - exiting with code: 2
Cause:test_exit_valEAL: Error: Invalid memory
EAL: Error - exiting with code: 255
Cause:test_exit_valEAL: Error: Invalid memory
EAL: Error - exiting with code: -1
Cause:test_exit_valEAL: Error: Invalid memory
Quoted 4 lines
The above error message will disappear when I comment out the calling to the
eal_pmu_fini()inthe
rte_eal_cleanup().
Thanks for pointing this out. This was apparently happening due to freeing same hugepage memory in forked process multiple times.
PATCH v3 2/4 eal/arm: support reading ARM PMU events in runtime
In reply to Tomasz DuszynskiAdd support for reading ARM PMU events in runtime.
--- app/test/test_pmu.c | 4 ++ lib/eal/arm/include/meson.build | 1 + lib/eal/arm/include/rte_pmu_pmc.h | 39 +++++++++++ lib/eal/arm/meson.build | 4 ++ lib/eal/arm/rte_pmu.c | 104 ++++++++++++++++++++++++++++++ lib/eal/include/rte_pmu.h | 3 + 6 files changed, 155 insertions(+) create mode 100644 lib/eal/arm/include/rte_pmu_pmc.h create mode 100644 lib/eal/arm/rte_pmu.c
app/test/test_pmu.c +4 -0
@@ -13,6 +13,10 @@ test_pmu_read(void)int tries =10 ;int event =-1 ; +#if defined(RTE_ARCH_ARM64 ) + event =rte_pmu_add_event ("cpu_cycles" ); +#endif +while (tries--) val +=rte_pmu_read (event);
lib/eal/arm/include/meson.build +1 -0
@@ -20,6 +20,7 @@ arch_headers = files( 'rte_pause_32.h', 'rte_pause_64.h', 'rte_pause.h', + 'rte_pmu_pmc.h', 'rte_power_intrinsics.h', 'rte_prefetch_32.h', 'rte_prefetch_64.h',
lib/eal/arm/include/rte_pmu_pmc.h new file +39 -0
@@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2022 Marvell. +*/ + +#ifndef _RTE_PMU_PMC_ARM_H_ +#define _RTE_PMU_PMC_ARM_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include <rte_common.h> + +static __rte_always_inline uint64_t +rte_pmu_pmc_read (int index) +{ +uint64_t val; + +if (index ==31 ) { +/* CPU Cycles (0x11) must be read via pmccntr_el0 */ + asmvolatile ("mrs %0, pmccntr_el0" :"=r" (val)); + }else { + asmvolatile ( +"msr pmselr_el0, %x0\n" +"mrs %0, pmxevcntr_el0\n" + :"=r" (val) + :"rZ" (index) + ); + } + +return val; +} +#define rte_pmu_pmc_read rte_pmu_pmc_read + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_PMU_PMC_ARM_H_ */
lib/eal/arm/meson.build +4 -0
@@ -9,3 +9,7 @@ sources += files( 'rte_hypervisor.c', 'rte_power_intrinsics.c', ) + +if is_linux + sources += files('rte_pmu.c') +endif
lib/eal/arm/rte_pmu.c new file +104 -0
104 lines not on this page · raw message
lib/eal/include/rte_pmu.h +3 -0
@@ -20,6 +20,9 @@ extern "C" {#include <rte_branch_prediction.h> #include <rte_lcore.h> #include <rte_log.h> +#if defined(RTE_ARCH_ARM64 ) +#include <rte_pmu_pmc.h> +#endif /** * @file
Signature · 1 line
2.25.1
PATCH v3 3/4 eal/x86: support reading Intel PMU events in runtime
In reply to Tomasz DuszynskiAdd support for reading Intel PMU events in runtime.
--- app/test/test_pmu.c | 2 ++ lib/eal/include/rte_pmu.h | 2 +- lib/eal/x86/include/meson.build | 1 + lib/eal/x86/include/rte_pmu_pmc.h | 33 +++++++++++++++++++++++++++++++ 4 files changed, 37 insertions(+), 1 deletion(-) create mode 100644 lib/eal/x86/include/rte_pmu_pmc.h
app/test/test_pmu.c +2 -0
@@ -15,6 +15,8 @@ test_pmu_read(void)#if defined(RTE_ARCH_ARM64 ) event =rte_pmu_add_event ("cpu_cycles" ); +#elif defined(RTE_ARCH_X86_64 ) + event =rte_pmu_add_event ("cpu-cycles" );#endif while (tries--)
lib/eal/include/rte_pmu.h +1 -1
@@ -20,7 +20,7 @@ extern "C" {#include <rte_branch_prediction.h> #include <rte_lcore.h> #include <rte_log.h> -#if defined(RTE_ARCH_ARM64 ) +#if defined(RTE_ARCH_ARM64 ) || defined(RTE_ARCH_X86_64 )#include <rte_pmu_pmc.h> #endif
lib/eal/x86/include/meson.build +1 -0
@@ -9,6 +9,7 @@ arch_headers = files( 'rte_io.h', 'rte_memcpy.h', 'rte_pause.h', + 'rte_pmu_pmc.h', 'rte_power_intrinsics.h', 'rte_prefetch.h', 'rte_rtm.h',
lib/eal/x86/include/rte_pmu_pmc.h new file +33 -0
@@ -0,0 +1,33 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2022 Marvell. +*/ + +#ifndef _RTE_PMU_PMC_X86_H_ +#define _RTE_PMU_PMC_X86_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include <rte_common.h> + +static __rte_always_inline uint64_t +rte_pmu_pmc_read (int index) +{ +uint32_t high, low; + + asmvolatile ( +"rdpmc\n" + :"=a" (low),"=d" (high) + :"c" (index) + ); + +return ((uint64_t )high <<32 ) | (uint64_t )low; +} +#define rte_pmu_pmc_read rte_pmu_pmc_read + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_PMU_PMC_X86_H_ */
Signature · 1 line
2.25.1
PATCH v3 4/4 eal: add PMU support to tracing library
In reply to Tomasz DuszynskiIn order to profile app one needs to store significant amount of samples somewhere for an analysis latern on. Since trace library supports storing data in a CTF format lets take adventage of that and add a dedicated PMU tracepoint.
--- app/test/test_trace_perf.c | 4 ++ doc/guides/prog_guide/profile_app.rst | 5 ++ doc/guides/prog_guide/trace_lib.rst | 32 ++++++++++++ lib/eal/common/eal_common_trace_points.c | 3 ++ lib/eal/common/rte_pmu.c | 63 ++++++++++++++++++++++++ lib/eal/include/rte_eal_trace.h | 11 +++++ lib/eal/version.map | 1 + 7 files changed, 119 insertions(+)
app/test/test_trace_perf.c +4 -0
@@ -114,6 +114,8 @@ worker_fn_##func(void *arg) \#define GENERIC_DOUBLE rte_eal_trace_generic_double(3.66666)#define GENERIC_STR rte_eal_trace_generic_str("hello world")#define VOID_FP app_dpdk_test_fp() +/* 0 corresponds first event passed via --trace= */ +#define READ_PMU rte_eal_trace_pmu_read(0)WORKER_DEFINE (GENERIC_VOID)WORKER_DEFINE (GENERIC_U64)
@@ -122,6 +124,7 @@ WORKER_DEFINE(GENERIC_FLOAT) WORKER_DEFINE(GENERIC_DOUBLE) WORKER_DEFINE(GENERIC_STR) WORKER_DEFINE(VOID_FP) +WORKER_DEFINE(READ_PMU) static void run_test(const char *str, lcore_function_t f, struct test_data *data, size_t sz)
@@ -174,6 +177,7 @@ test_trace_perf(void)run_test ("double" , worker_fn_GENERIC_DOUBLE, data, sz);run_test ("string" , worker_fn_GENERIC_STR, data, sz);run_test ("void_fp" , worker_fn_VOID_FP, data, sz); +run_test ("read_pmu" , worker_fn_READ_PMU, data, sz);rte_free (data);return TEST_SUCCESS ;
doc/guides/prog_guide/profile_app.rst +5 -0
@@ -16,6 +16,11 @@ that information, perf being an example here. Though in some scenarios, eg. when isolated (nohz_full) and run dedicated tasks, using perf is less than ideal. In such cases one can read specific events directly from application via ``rte_pmu_read()``. +Alternatively tracing library can be used which offers dedicated tracepoint +``rte_eal_trace_pmu_event()``. + +Refer to :doc:`../prog_guide/trace_lib` for more details. + Profiling on x86 ----------------
doc/guides/prog_guide/trace_lib.rst +32 -0
@@ -46,6 +46,7 @@ DPDK tracing library features trace format and is compatible with ``LTTng``. For detailed information, refer to `Common Trace Format <https://diamon.org/ctf/>`_. +- Support reading PMU events on ARM64 and x86 (Intel) How to add a tracepoint? ------------------------
@@ -137,6 +138,37 @@ the user must use ``RTE_TRACE_POINT_FP`` instead of ``RTE_TRACE_POINT``. ``RTE_TRACE_POINT_FP`` is compiled out by default and it can be enabled using the ``enable_trace_fp`` option for meson build. +PMU tracepoint +-------------- + +Performance measurement unit (PMU) event values can be read from hardware +registers using predefined ``rte_pmu_read`` tracepoint. + +Tracing is enabled via ``--trace`` EAL option by passing both expression +matching PMU tracepoint name i.e ``lib.eal.pmu.read`` and expression +``e=ev1[,ev2,...]`` matching particular events:: + + --trace='*pmu.read\|e=cpu_cycles,l1d_cache' + +Event names are available under ``/sys/bus/event_source/devices/PMU/events`` +directory, where ``PMU`` is a placeholder for either a ``cpu`` or a directory +containing ``cpus``. + +In contrary to other tracepoints this does not need any extra variables +added to source files. Instead, caller passes index which follows the order of +events specified via ``--trace`` parameter. In the following example index ``0`` +corresponds to ``cpu_cyclces`` while index ``1`` corresponds to ``l1d_cache``. + +.. code-block:: c + + ... + rte_eal_trace_pmu_read(0); + rte_eal_trace_pmu_read(1); + ... + +PMU tracing support must be explicitly enabled using the ``enable_trace_fp`` +option for meson build. + Event record mode -----------------
lib/eal/common/eal_common_trace_points.c +3 -0
@@ -75,3 +75,6 @@ RTE_TRACE_POINT_REGISTER(rte_eal_trace_intr_enable, lib.eal.intr.enable) RTE_TRACE_POINT_REGISTER(rte_eal_trace_intr_disable, lib.eal.intr.disable) + +RTE_TRACE_POINT_REGISTER (rte_eal_trace_pmu_read, + lib.eal.pmu.read)
lib/eal/common/rte_pmu.c +63 -0
86 lines not on this page · raw message
lib/eal/include/rte_eal_trace.h +11 -0
@@ -17,6 +17,7 @@ extern "C" {#include <rte_alarm.h> #include <rte_interrupts.h> +#include <rte_pmu.h> #include <rte_trace_point.h> #include "eal_interrupts.h"
@@ -279,6 +280,16 @@ RTE_TRACE_POINT(rte_trace_point_emit_string (cpuset); ) +/* PMU */ +RTE_TRACE_POINT_FP ( + rte_eal_trace_pmu_read, + RTE_TRACE_POINT_ARGS(int index), + uint64_t val; + rte_trace_point_emit_int(index); + val = rte_pmu_read(index); + rte_trace_point_emit_u64(val); +) + #ifdef __cplusplus }#endif
lib/eal/version.map +1 -0
@@ -442,6 +442,7 @@ EXPERIMENTAL { rte_thread_join; # added in 23.03 + __rte_eal_trace_pmu_read; # WINDOWS_NO_EXPORT rte_pmu; # WINDOWS_NO_EXPORT rte_pmu_add_event; # WINDOWS_NO_EXPORT rte_pmu_read; # WINDOWS_NO_EXPORT
Signature · 1 line
2.25.1
Quoted 16 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Tuesday, 29 November 2022 10.28This series adds self monitoring support i.e allows to configure and read performance measurement unit (PMU) counters in runtime without using perf utility. This has certain adventages when application runs on
isolated cores withnohz_fullkernel parameter.Events can be read directly using
rte_pmu_read()or using dedicated tracepointrte_eal_trace_pmu_read(). The latter will cause events to be stored inside CTF file.By design, all enabled events are grouped together and the same group is attached to lcores that use self monitoring funtionality.
Quoted 7 lines
Events are enabled by names, which need to be read from standard location under sysfs i.e
/sys/bus/event_source/devices/PMU/eventswhere PMU is a core pmu i.e one measuring cpu events. As of today raw events are not supported.
Hi Thomasz,
I am very interested in this patch series for fast path profiling purposes. (Not using EAL trace, but our proprietary profiler.)
However, it seems that rte_pmu_read() is quite longwinded, compared to rte_pmu_pmc_read().
But perhaps I am just worrying too much, so I will ask: What is the performance cost of using rte_pmu_read() - compared to rte_pmu_pmc_read() - in the fast path?
If there is a non-negligible difference, could you please provide an example of how to configure PMU events and use rte_pmu_pmc_read() in an application?
I would primarily be interested in data cache misses and branch mispredictions. But feel free to make your own choices for the example.
Hi Morten,
Quoted 33 lines
-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: Tuesday, November 29, 2022 11:43 AM
To: Tomasz Duszynski <[email protected]>; [email protected]
Cc: [email protected]; Jerin Jacob Kollanukkaran <[email protected]>
Subject: [EXT] RE: [PATCH v3 0/4] add support for self monitoringExternal Email
----------------------------------------------------------------------
Quoted 15 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Tuesday, 29 November 2022 10.28This series adds self monitoring support i.e allows to configure and read performance measurement unit (PMU) counters in runtime without using perf utility. This has certain adventages when application runs on isolated cores with
nohz_fullkernel parameter.Events can be read directly using
rte_pmu_read()or using dedicated tracepointrte_eal_trace_pmu_read(). The latter will cause events to be stored inside CTF file.By design, all enabled events are grouped together and the same group is attached to lcores that use self monitoring funtionality.
Quoted 7 lines
Events are enabled by names, which need to be read from standard location under sysfs i.e
/sys/bus/event_source/devices/PMU/eventswhere PMU is a core pmu i.e one measuring cpu events. As of today raw events are not supported.
Quoted 7 lines
Hi Thomasz,
I am very interested in this patch series for fast path profiling purposes. (Not using EAL trace, but our proprietary profiler.)
However, it seems that
rte_pmu_read()is quite longwinded, compared torte_pmu_pmc_read().
We need some bit of extra logic to set thigs up before performing reading actual counter but in reality cycles are mostly consumed by rte_pmu_pmc_read(). This obviously differs among platforms so if you want precise measurements you need to get your hands dirty.
That said, below are results coming from dpdk-test after running trace_perf_autotest - just to give you some idea.
X86-64
RTE>>trace_perf_autotest
Timer running at 3000.00MHz
void: cycles=17.739375 ns=5.913125
u64: cycles=17.348296 ns=5.782765
int: cycles=17.098724 ns=5.699575
float: cycles=17.099946 ns=5.699982
double: cycles=17.229702 ns=5.743234
string: cycles=31.159907 ns=10.386636
void_fp: cycles=0.679842 ns=0.226614
read_pmu: cycles=49.325117 ns=16.441706ARM64 with RTE_ARM_EAL_RDTSC_USE_PMU
RTE>>trace_perf_autotest
Timer running at 2480.00MHz
void: cycles=9.413568 ns=3.795793
u64: cycles=9.386003 ns=3.784678
int: cycles=9.438701 ns=3.805928
float: cycles=9.359377 ns=3.773942
double: cycles=9.372279 ns=3.779145
string: cycles=24.474899 ns=9.868911
void_fp: cycles=0.505513 ns=0.203836
read_pmu: cycles=17.442853 ns=7.033409Quoted 6 lines
But perhaps I am just worrying too much, so I will ask: What is the performance cost of using
rte_pmu_read()- compared torte_pmu_pmc_read()- in the fast path?If there is a non-negligible difference, could you please provide an example of how to configure PMU events and use
rte_pmu_pmc_read()in an application?
Series come with some docs so you can check there how to run it.
Quoted 2 lines
I would primarily be interested in data cache misses and branch mispredictions. But feel free to make your own choices for the example.
Raw events are not supported right now which means you don't have fine control over all events. You can use only events from CPU PMU (/sys/bus/event_source/devices/<PMU>/events).
v4 cover add support for self monitoring
In reply to Tomasz DuszynskiThis series adds self monitoring support i.e allows to configure and read performance measurement unit (PMU) counters in runtime without using perf utility. This has certain adventages when application runs on isolated cores with nohz_full kernel parameter.
Events can be read directly using rte_pmu_read() or using dedicated tracepoint rte_eal_trace_pmu_read(). The latter will cause events to be stored inside CTF file.
By design, all enabled events are grouped together and the same group is attached to lcores that use self monitoring funtionality.
Events are enabled by names, which need to be read from standard location under sysfs i.e
/sys/bus/event_source/devices/PMU/events
where PMU is a core pmu i.e one measuring cpu events. As of today raw events are not supported.
v4:
- fix freeing mem detected by debug_autotest
v3:
- fix shared build
v2:
- fix problems reported by test build infra
Tomasz Duszynski (4):
eal: add generic support for reading PMU events eal/arm: support reading ARM PMU events in runtime eal/x86: support reading Intel PMU events in runtime
eal: add PMU support to tracing library
app/test/meson.build | 1 + app/test/test_pmu.c | 47 ++ app/test/test_trace_perf.c | 4 + doc/guides/prog_guide/profile_app.rst | 13 + doc/guides/prog_guide/trace_lib.rst | 32 ++ lib/eal/arm/include/meson.build | 1 + lib/eal/arm/include/rte_pmu_pmc.h | 39 ++ lib/eal/arm/meson.build | 4 + lib/eal/arm/rte_pmu.c | 104 +++++ lib/eal/common/eal_common_trace_points.c | 3 + lib/eal/common/meson.build | 3 + lib/eal/common/pmu_private.h | 41 ++ lib/eal/common/rte_pmu.c | 519 +++++++++++++++++++++++ lib/eal/include/meson.build | 1 + lib/eal/include/rte_eal_trace.h | 11 + lib/eal/include/rte_pmu.h | 207 +++++++++ lib/eal/linux/eal.c | 4 + lib/eal/version.map | 7 + lib/eal/x86/include/meson.build | 1 + lib/eal/x86/include/rte_pmu_pmc.h | 33 ++ 20 files changed, 1075 insertions(+) create mode 100644 app/test/test_pmu.c create mode 100644 lib/eal/arm/include/rte_pmu_pmc.h create mode 100644 lib/eal/arm/rte_pmu.c create mode 100644 lib/eal/common/pmu_private.h create mode 100644 lib/eal/common/rte_pmu.c create mode 100644 lib/eal/include/rte_pmu.h create mode 100644 lib/eal/x86/include/rte_pmu_pmc.h
--
2.25.1
PATCH v4 1/4 eal: add generic support for reading PMU events
Add support for programming PMU counters and reading their values in runtime bypassing kernel completely.
This is especially useful in cases where CPU cores are isolated (nohz_full) i.e run dedicated tasks. In such cases one cannot use standard perf utility without sacrificing latency and performance.
--- app/test/meson.build | 1 + app/test/test_pmu.c | 41 +++ doc/guides/prog_guide/profile_app.rst | 8 + lib/eal/common/meson.build | 3 + lib/eal/common/pmu_private.h | 41 +++ lib/eal/common/rte_pmu.c | 456 ++++++++++++++++++++++++++ lib/eal/include/meson.build | 1 + lib/eal/include/rte_pmu.h | 204 ++++++++++++ lib/eal/linux/eal.c | 4 + lib/eal/version.map | 6 + 10 files changed, 765 insertions(+) create mode 100644 app/test/test_pmu.c create mode 100644 lib/eal/common/pmu_private.h create mode 100644 lib/eal/common/rte_pmu.c create mode 100644 lib/eal/include/rte_pmu.h
app/test/meson.build +1 -0
@@ -143,6 +143,7 @@ test_sources = files( 'test_timer_racecond.c', 'test_timer_secondary.c', 'test_ticketlock.c', + 'test_pmu.c', 'test_trace.c', 'test_trace_register.c', 'test_trace_perf.c',
app/test/test_pmu.c new file +41 -0
41 lines not on this page · raw message
doc/guides/prog_guide/profile_app.rst +8 -0
@@ -7,6 +7,14 @@ Profile Your Application The following sections describe methods of profiling DPDK applications on different architectures. +Performance counter based profiling +----------------------------------- + +Majority of architectures support some sort hardware measurement unit which provides a set of +programmable counters that monitor specific events. There are different tools which can gather +that information, perf being an example here. Though in some scenarios, eg. when CPU cores are +isolated (nohz_full) and run dedicated tasks, using perf is less than ideal. In such cases one can +read specific events directly from application via ``rte_pmu_read()``. Profiling on x86 ----------------
lib/eal/common/meson.build +3 -0
@@ -38,6 +38,9 @@ sources += files( 'rte_service.c', 'rte_version.c', ) +if is_linux + sources += files('rte_pmu.c') +endif if is_linux or is_windows sources += files('eal_common_dynmem.c') endif
lib/eal/common/pmu_private.h new file +41 -0
41 lines not on this page · raw message
lib/eal/common/rte_pmu.c new file +456 -0
456 lines not on this page · raw message
lib/eal/include/meson.build +1 -0
@@ -36,6 +36,7 @@ headers += files( 'rte_pci_dev_features.h', 'rte_per_lcore.h', 'rte_pflock.h', + 'rte_pmu.h', 'rte_random.h', 'rte_reciprocal.h', 'rte_seqcount.h',
lib/eal/include/rte_pmu.h new file +204 -0
204 lines not on this page · raw message
lib/eal/linux/eal.c +4 -0
@@ -53,6 +53,7 @@#include "eal_options.h" #include "eal_vfio.h" #include "hotplug_mp.h" +#include "pmu_private.h" #define MEMSIZE_IF_NO_HUGE_PAGE (64ULL * 1024ULL * 1024ULL)
@@ -1206,6 +1207,8 @@ rte_eal_init(int argc, char **argv)return -1 ; } +eal_pmu_init (); +if (rte_eal_tailqs_init () <0 ) {rte_eal_init_alert ("Cannot init tail queues for objects" ); rte_errno =EFAULT ;
@@ -1372,6 +1375,7 @@ rte_eal_cleanup(void)eal_bus_cleanup ();rte_trace_save ();eal_trace_fini (); +eal_pmu_fini ();/* after this point, any DPDK pointers will become dangling */ rte_eal_memory_detach ();eal_mp_dev_hotplug_cleanup ();
lib/eal/version.map +6 -0
@@ -440,6 +440,11 @@ EXPERIMENTAL { rte_thread_detach; rte_thread_equal; rte_thread_join; + + # added in 23.03 + rte_pmu; # WINDOWS_NO_EXPORT + rte_pmu_add_event; # WINDOWS_NO_EXPORT + rte_pmu_read; # WINDOWS_NO_EXPORT }; INTERNAL {
@@ -483,4 +488,5 @@ INTERNAL { rte_mem_map; rte_mem_page_size; rte_mem_unmap; + rte_pmu_enable_group; };
Signature · 1 line
2.25.1
RE: [PATCH v4 1/4] eal: add generic support for reading PMU events
Quoted 548 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Tuesday, 13 December 2022 11.44Add support for programming PMU counters and reading their values in runtime bypassing kernel completely.
This is especially useful in cases where CPU cores are isolated (
nohz_full) i.e run dedicated tasks. In such cases one cannot use standard perf utility without sacrificing latency and performance.Signed-off-by Tomasz Duszynski <[email protected]>---
+++
b/lib/eal/common/rte_pmu.clib/eal/common/rte_pmu.c new file +26
-0@@ -0,0 +1,456 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(C) 2022 Marvell International Ltd. +*/ + +#include <ctype.h> +#include <dirent.h> +#include <errno.h> +#include <regex.h> +#include <stdlib.h> +#include <string.h> +#include <sys/ioctl.h> +#include <sys/mman.h> +#include <sys/queue.h> +#include <sys/syscall.h> +#include <unistd.h> + +#include <rte_eal_paging.h> +#include <rte_pmu.h> +#include <rte_tailq.h> + +#include "pmu_private.h" + +#define EVENT_SOURCE_DEVICES_PATH "/sys/bus/event_source/devices" + +#ifndef GENMASK_ULL +#define GENMASK_ULL (h, l) ((~0ULL - (1ULL << (l)) + 1) & (~0ULL >>((64 - 1 - (h)))))
lib/eal/common/rte_pmu.c new file +4
-0+#endif + +#ifndef FIELD_PREP +#define FIELD_PREP (m, v) (((uint64_t)(v) << (__builtin_ffsll(m) - 1))& (m))
lib/eal/common/rte_pmu.c new file +6
-0+#endif + +struct rte_pmu *rte_pmu; + +/* +* Following __rte_weak functions provide default no-op. Architectures should override them if
+ * necessary.
+ */lib/eal/common/rte_pmu.c new file +31
-0+ +int +__rte_weak pmu_arch_init(void) +{ +return 0 ; +} + +void +__rte_weak pmu_arch_fini(void) +{ +} + +void +__rte_weak pmu_arch_fixup_config(uint64_t config[3]) +{ +RTE_SET_USED (config); +} + +static int +get_term_format (const char *name,int *num,uint64_t *mask) +{ +char *config =NULL ; +char path[PATH_MAX ]; +int high, low, ret; +FILE *fp; + +/* quiesce -Wmaybe-uninitialized warning */ + *num =0 ; + *mask =0 ; + + snprintf(path, sizeof(path), EVENT_SOURCE_DEVICES_PATH"/%s/format/%s",
rte_pmu->name, name);lib/eal/common/rte_pmu.c new file +20
-0+ fp =fopen (path,"r" ); +if (!fp) +return -errno; + + errno =0 ; + ret =fscanf (fp,"%m[^:]:%d-%d" , &config, &low, &high); +if (ret <2 ) { + ret = -ENODATA ; + goto out; + } +if (errno) { + ret = -errno; + goto out; + } + +if (ret ==2 ) + high = low; + + *mask =GENMASK_ULL (high, low); +/* Last digit should be [012]. If last digit is missing 0 is implied. */
lib/eal/common/rte_pmu.c new file +52
-0+ *num = config[strlen (config) -1 ]; + *num =isdigit (*num) ? *num -'0' :0 ; + + ret =0 ; +out: +free (config); +fclose (fp); + +return ret; +} + +static int +parse_event (char *buf,uint64_t config[3 ]) +{ +char *token, *term; +int num, ret, val; +uint64_t mask; + + config[0 ] = config[1 ] = config[2 ] =0 ; + + token =strtok (buf,"," ); +while (token) { + errno =0 ; +/* <term>=<value> */ + ret =sscanf (token,"%m[^=]=%i" , &term, &val); +if (ret <1 ) +return -ENODATA ; +if (errno) +return -errno; +if (ret ==1 ) + val =1 ; + + ret =get_term_format (term, &num, &mask); +free (term); +if (ret) +return ret; + + config[num] |=FIELD_PREP (mask, val); + token =strtok (NULL ,"," ); + } + +return 0 ; +} + +static int +get_event_config (const char *name,uint64_t config[3 ]) +{ +char path[PATH_MAX ], buf[BUFSIZ ]; +FILE *fp; +int ret; + + snprintf(path, sizeof(path), EVENT_SOURCE_DEVICES_PATH"/%s/events/%s",
rte_pmu->name, name);lib/eal/common/rte_pmu.c new file +34
-0+ fp =fopen (path,"r" ); +if (!fp) +return -errno; + + ret =fread (buf,1 ,sizeof (buf), fp); +if (ret ==0 ) { +fclose (fp); + +return -EINVAL ; + } +fclose (fp); + buf[ret] ='\0' ; + +return parse_event (buf, config); +} + +static int +do_perf_event_open (uint64_t config[3 ],int lcore_id,int group_fd) +{ +struct perf_event_attr attr = { + .size =sizeof (struct perf_event_attr ), + .type =PERF_TYPE_RAW , + .exclude_kernel =1 , + .exclude_hv =1 , + .disabled =1 , + }; + +pmu_arch_fixup_config (config); + + attr.config = config[0 ]; + attr.config1 = config[1 ]; + attr.config2 = config[2 ]; + + return syscall(SYS_perf_event_open, &attr, rte_gettid(),
rte_lcore_to_cpu_id(lcore_id),lib/eal/common/rte_pmu.c new file +18
-0+ group_fd, 0); +} + +static int +open_events (int lcore_id) +{ +struct rte_pmu_event_group *group = &rte_pmu->group[lcore_id]; +struct rte_pmu_event *event; +uint64_t config[3 ]; +int num =0 , ret; + +/* group leader gets created first, with fd = -1 */ + group->fds[0 ] =-1 ; + + TAILQ_FOREACH(event, &rte_pmu->event_list, next) { + ret =get_event_config (event->name, config); +if (ret) { + RTE_LOG(ERR, EAL, "failed to get %s event config\n",
event->name);lib/eal/common/rte_pmu.c new file +7
-0+continue ; + } + + ret =do_perf_event_open (config, lcore_id, group->fds[0 ]); +if (ret ==-1 ) { +if (errno ==EOPNOTSUPP ) + RTE_LOG(ERR, EAL, "64 bit counters notsupported\n");
lib/eal/common/rte_pmu.c new file +29
-0+ + ret = -errno; + goto out; + } + + group->fds[event->index] = ret; + num++; + } + +return 0 ; +out: +for (--num; num >=0 ; num--) { +close (group->fds[num]); + group->fds[num] =-1 ; + } + + +return ret; +} + +static int +mmap_events (int lcore_id) +{ +struct rte_pmu_event_group *group = &rte_pmu->group[lcore_id]; +void *addr; +int ret, i; + +for (i =0 ; i < rte_pmu->num_group_events; i++) { + addr = mmap(0, rte_mem_page_size(), PROT_READ, MAP_SHARED,group->fds[i], 0);
lib/eal/common/rte_pmu.c new file +29
-0+if (addr ==MAP_FAILED ) { + ret = -errno; + goto out; + } + + group->mmap_pages[i] = addr; + } + +return 0 ; +out: +for (; i; i--) { +munmap (group->mmap_pages[i -1 ],rte_mem_page_size ()); + group->mmap_pages[i -1 ] =NULL ; + } + +return ret; +} + +static void +cleanup_events (int lcore_id) +{ +struct rte_pmu_event_group *group = &rte_pmu->group[lcore_id]; +int i; + +if (!group->fds) +return ; + +if (group->fds[0 ] !=-1 ) + ioctl(group->fds[0], PERF_EVENT_IOC_DISABLE,
PERF_IOC_FLAG_GROUP);lib/eal/common/rte_pmu.c new file +34
-0+ +for (i =0 ; i < rte_pmu->num_group_events; i++) { +if (group->mmap_pages[i]) { +munmap (group->mmap_pages[i],rte_mem_page_size ()); + group->mmap_pages[i] =NULL ; + } + +if (group->fds[i] !=-1 ) { +close (group->fds[i]); + group->fds[i] =-1 ; + } + } + +free (group->mmap_pages); +free (group->fds); + + group->mmap_pages =NULL ; + group->fds =NULL ; + group->enabled = false; +} + +int __rte_noinline +rte_pmu_enable_group (int lcore_id) +{ +struct rte_pmu_event_group *group = &rte_pmu->group[lcore_id]; +int ret; + +if (rte_pmu->num_group_events ==0 ) { +RTE_LOG (DEBUG ,EAL ,"no matching PMU events\n" ); + +return 0 ; + } + + group->fds = calloc(rte_pmu->num_group_events, sizeof(*group-Quoted 1 line
fds));
lib/eal/common/rte_pmu.c new file +7
-0+if (!group->fds) { +RTE_LOG (ERR ,EAL ,"failed to alloc descriptor memory\n" ); + +return -ENOMEM ; + } + + group->mmap_pages = calloc(rte_pmu->num_group_events,
sizeof(*group->mmap_pages));lib/eal/common/rte_pmu.c new file +10
-0+if (!group->mmap_pages) { +RTE_LOG (ERR ,EAL ,"failed to alloc userpage memory\n" ); + + ret = -ENOMEM ; + goto out; + } + + ret =open_events (lcore_id); +if (ret) { + RTE_LOG(ERR, EAL, "failed to open events on lcore-worker-%d\n",
lcore_id);lib/eal/common/rte_pmu.c new file +6
-0+ goto out; + } + + ret =mmap_events (lcore_id); +if (ret) { + RTE_LOG(ERR, EAL, "failed to map events on lcore-worker-%d\n",
lcore_id);lib/eal/common/rte_pmu.c new file +4
-0+ goto out; + } + + if (ioctl(group->fds[0], PERF_EVENT_IOC_ENABLE,
PERF_IOC_FLAG_GROUP) == -1) {(quoted diff) +1
-0+ RTE_LOG(ERR, EAL, "failed to enable events on lcore-worker-%d\n",
lcore_id);lib/eal/common/rte_pmu.c new file +35
-0+ + ret = -errno; + goto out; + } + +return 0 ; + +out: +cleanup_events (lcore_id); + +return ret; +} + +static int +scan_pmus (void ) +{ +char path[PATH_MAX ]; +struct dirent *dent; +const char *name; +DIR *dirp; + + dirp =opendir (EVENT_SOURCE_DEVICES_PATH ); +if (!dirp) +return -errno; + +while ((dent =readdir (dirp))) { + name = dent->d_name; +if (name[0 ] =='.' ) +continue ; + +/* sysfs entry should either contain cpus or be a cpu */ +if (!strcmp (name,"cpu" )) +break ; + + snprintf(path, sizeof(path), EVENT_SOURCE_DEVICES_PATH"/%s/cpus", name);
lib/eal/common/rte_pmu.c new file +22
-0+if (access (path,F_OK ) ==0 ) +break ; + } + +closedir (dirp); + +if (dent) { + rte_pmu->name =strdup (name); +if (!rte_pmu->name) +return -ENOMEM ; + } + +return rte_pmu->name ?0 : -ENODEV ; +} + +int +rte_pmu_add_event (const char *name) +{ +struct rte_pmu_event *event; +char path[PATH_MAX ]; + + snprintf(path, sizeof(path), EVENT_SOURCE_DEVICES_PATH"/%s/events/%s",
rte_pmu->name, name);lib/eal/common/rte_pmu.c new file +24
-0+if (access (path,R_OK )) +return -ENODEV ; + + TAILQ_FOREACH(event, &rte_pmu->event_list, next) { +if (!strcmp (event->name, name)) +return event->index; +continue ; + } + + event =calloc (1 ,sizeof (*event)); +if (!event) +return -ENOMEM ; + + event->name =strdup (name); +if (!event->name) { +free (event); + +return -ENOMEM ; + } + + event->index = rte_pmu->num_group_events++; +TAILQ_INSERT_TAIL (&rte_pmu->event_list, event, next); + + RTE_LOG(DEBUG, EAL, "%s even added at index %d\n", name, event-Quoted 1 line
index);
lib/eal/common/rte_pmu.c new file +55
-0+ +return event->index; +} + +void +eal_pmu_init (void ) +{ +int ret; + + rte_pmu =calloc (1 ,sizeof (*rte_pmu)); +if (!rte_pmu) { +RTE_LOG (ERR ,EAL ,"failed to alloc PMU\n" ); + +return ; + } + +TAILQ_INIT (&rte_pmu->event_list); + + ret =scan_pmus (); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to find core pmu\n" ); + goto out; + } + + ret =pmu_arch_init (); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to setup arch for PMU\n" ); + goto out; + } + +return ; +out: +free (rte_pmu->name); +free (rte_pmu); +} + +void +eal_pmu_fini (void ) +{ +struct rte_pmu_event *event, *tmp; +int lcore_id; + + RTE_TAILQ_FOREACH_SAFE(event, &rte_pmu->event_list, next, tmp) { +TAILQ_REMOVE (&rte_pmu->event_list, event, next); +free (event->name); +free (event); + } + +RTE_LCORE_FOREACH_WORKER (lcore_id) +cleanup_events (lcore_id); + +pmu_arch_fini (); +free (rte_pmu->name); +free (rte_pmu); +}lib/eal/include/meson.build +1
-0@@ -36,6 +36,7 @@ headers += files( 'rte_pci_dev_features.h', 'rte_per_lcore.h', 'rte_pflock.h', + 'rte_pmu.h', 'rte_random.h', 'rte_reciprocal.h', 'rte_seqcount.h',lib/eal/include/rte_pmu.h new file +38
-0@@ -0,0 +1,204 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2022 Marvell +*/ + +#ifndef _RTE_PMU_H_ +#define _RTE_PMU_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include <rte_common.h> +#include <rte_compat.h> + +#ifdef RTE_EXEC_ENV_LINUX + +#include <linux/perf_event.h> + +#include <rte_atomic.h> +#include <rte_branch_prediction.h> +#include <rte_lcore.h> +#include <rte_log.h> + +/** + * @file + * + * PMU event tracing operations + * + * This file defines generic API and types necessary to setup PMU and + * read selected counters in runtime. + */ + +/** + * A structure describing a group of events. + */ +struct rte_pmu_event_group { + int *fds; /**< array of event descriptors */ + void **mmap_pages; /**< array of pointers to mmapped
Quoted 1 line
perf_event_attrstructures */
There seems to be a lot of indirection involved here. Why are these arrays not statically sized, instead of dynamically allocated?
Also, what is the reason for hiding the type struct perf_event_mmap_page **mmap_pages opaque by using void **mmap_pages instead?
Quoted 24 lines
lib/eal/include/rte_pmu.h new file +1
-0+bool enabled;/**< true if group was enabled on particular lcore */
lib/eal/include/rte_pmu.h new file +16
-0+}; + +/** +* A structure describing an event. +*/ +struct rte_pmu_event { +char *name;/** name of an event */ +int index;/** event index into fds/mmap_pages */ +TAILQ_ENTRY (rte_pmu_event) next;/** list entry */ +}; + +/** +* A PMU state container. +*/ +struct rte_pmu { +char *name;/** name of core PMU listed under
/sys/bus/event_source/devices*/lib/eal/include/rte_pmu.h new file +1
-0+struct rte_pmu_event_group group[RTE_MAX_LCORE ];/**< per lcore event group data */
lib/eal/include/rte_pmu.h new file +2
-0+int num_group_events;/**< number of events in a group */ + TAILQ_HEAD(, rte_pmu_event) event_list; /**< list of matchingevents */
lib/eal/include/rte_pmu.h line 62
Quoted 4 lines
lib/eal/include/rte_pmu.h new file +4
-0+}; + +/** Pointer to the PMU state container */ +extern struct rte_pmu *rte_pmu;
Again, why not just extern struct rte_pmu, instead of dynamic allocation?
Quoted 106 lines
lib/eal/include/rte_pmu.h new file +2
-0+ +/** Each architecture supporting PMU needs to provide its own version */
lib/eal/include/rte_pmu.h new file +42
-0+#ifndef rte_pmu_pmc_read +#define rte_pmu_pmc_read (index) ({ 0; }) +#endif + +/** +* @internal +* +* Read PMU counter. +* +* @param pc +* Pointer to the mmapped user page. +* @return +* Counter value read from hardware. +*/ +__rte_internal +static __rte_always_inline uint64_t +rte_pmu_read_userpage (struct perf_event_mmap_page *pc) +{ +uint64_t offset, width, pmc =0 ; +uint32_t seq, index; +int tries =100 ; + +for (;;) { + seq = pc->lock; +rte_compiler_barrier (); + index = pc->index; + offset = pc->offset; + width = pc->pmc_width; + +if (likely (pc->cap_user_rdpmc && index)) { + pmc =rte_pmu_pmc_read (index -1 ); + pmc <<=64 - width; + pmc >>=64 - width; + } + +rte_compiler_barrier (); + +if (likely (pc->lock == seq)) +return pmc + offset; + +if (--tries ==0 ) { + RTE_LOG(DEBUG, EAL, "failed to get
perf_event_mmap_pagelock\n");lib/eal/include/rte_pmu.h new file +29
-0+break ; + } + } + +return 0 ; +} + +/** +* @internal +* +* Enable group of events for a given lcore. +* +* @param lcore_id +* The identifier of the lcore. +* @return +* 0 in case of success, negative value otherwise. +*/ +__rte_internal +int +rte_pmu_enable_group (int lcore_id); + +/** +* @warning +* @b EXPERIMENTAL: this API may change without prior notice +* +* Add event to the group of enabled events. +* +* @param name +* Name of an event listed under
/sys/bus/event_source/devices/pmu/events.
+ * @return
+ * Event index in case of success, negative value otherwise.
+ */lib/eal/include/rte_pmu.h new file +14
-0+__rte_experimental +int +rte_pmu_add_event (const char *name); + +/** +* @warning +* @b EXPERIMENTAL: this API may change without prior notice +* +* Read hardware counter configured to count occurrences of an event. +* +* @param index +* Index of an event to be read. +* @return +* Event value read from register. In case of errors or lack of support
+ * 0 is returned. In other words, stream of zeros in a trace file
+ * indicates problem with reading particular PMU event register.
+ */lib/eal/include/rte_pmu.h new file +9
-0+__rte_experimental +static __rte_always_inline uint64_t +rte_pmu_read (int index) +{ +int lcore_id =rte_lcore_id (); +struct rte_pmu_event_group *group; +int ret; + +if (!rte_pmu)
lib/eal/include/rte_pmu.h line 174
Quoted 10 lines
lib/eal/include/rte_pmu.h new file +10
-0+return 0 ; + + group = &rte_pmu->group[lcore_id]; +if (!group->enabled) { + ret =rte_pmu_enable_group (lcore_id); +if (ret) +return 0 ; + + group->enabled = true; + }
Why is the group not enabled in the setup function, rte_pmu_add_event(), instead of here, in the hot path?
Quoted 6 lines
lib/eal/include/rte_pmu.h new file +5
-0+ +if (index <0 || index >= rte_pmu->num_group_events) +return 0 ; + + return rte_pmu_read_userpage((struct perf_event_mmap_page*)group->mmap_pages[index]);
Using fixed size arrays instead of multiple indirections via pointers is faster. It could be:
return rte_pmu_read_userpage((struct perf_event_mmap_page *)rte_pmu.group[lcore_id].mmap_pages[index]);
With our without suggested performance improvements...
RE: [PATCH v4 1/4] eal: add generic support for reading PMU events
Hello Morten,
Thanks for review. Answers inline.
[...]
Quoted 11 lines
Quoted 11 lines
lib/eal/include/rte_pmu.h new file +11
-0+/** +* @file +* +* PMU event tracing operations +* +* This file defines generic API and types necessary to setup PMU and +* read selected counters in runtime. +*/ + +/** +* A structure describing a group of events.
Quoted 9 lines
Quoted 5 lines
+ */
lib/eal/include/rte_pmu.h new file +3
-0+struct rte_pmu_event_group { +int *fds;/**< array of event descriptors */ +void **mmap_pages;/**< array of pointers to mmapped
perf_event_attrstructures */There seems to be a lot of indirection involved here. Why are these arrays not statically sized, instead of dynamically allocated?
Different architectures/pmus impose limits on number of simultaneously enabled counters. So in order relief the pain of thinking about it and adding macros for each and every arch I decided to allocate the number user wants dynamically. Also assumption holds that user knows about tradeoffs of using too many counters hence will not enable too many events at once.
Quoted 2 lines
Also, what is the reason for hiding the type struct
perf_event_mmap_page**mmap_pages opaque by using void **mmap_pages instead?
I think, that part doing mmap/munmap was written first hence void ** was chosen in the first place.
Quoted 22 lines
Quoted 22 lines
(quoted diff) +1
-0+ bool enabled; /**< true if group was enabled on particular lcore*/
lib/eal/include/rte_pmu.h new file +15
-0+}; + +/** +* A structure describing an event. +*/ +struct rte_pmu_event { +char *name;/** name of an event */ +int index;/** event index into fds/mmap_pages */ +TAILQ_ENTRY (rte_pmu_event) next;/** list entry */ }; + +/** +* A PMU state container. +*/ +struct rte_pmu { +char *name;/** name of core PMU listed under
/sys/bus/event_source/devices*/(quoted diff) +1
-0+ struct rte_pmu_event_group group[RTE_MAX_LCORE]; /**< per lcoreevent group data */
lib/eal/include/rte_pmu.h new file +1
-0+int num_group_events;/**< number of events in a group */
Quoted 9 lines
app/test/test_pmu.c
Quoted 6 lines
(quoted diff) +1
-0+ TAILQ_HEAD(, rte_pmu_event) event_list; /**< list of matchingevents */
app/test/test_pmu.c new file +4
-0+}; + +/** Pointer to the PMU state container */ extern struct rte_pmu +*rte_pmu;Again, why not just extern struct
rte_pmu, instead of dynamic allocation?
No strong opinions here since this is a matter of personal preference. Can be removed in the next version.
Quoted 109 lines
Quoted 109 lines
(quoted diff) +2
-0+ +/** Each architecture supporting PMU needs to provide its own version*/
lib/eal/include/rte_pmu.h new file +40
-0+#ifndef rte_pmu_pmc_read +#define rte_pmu_pmc_read (index) ({ 0; }) #endif + +/** +* @internal +* +* Read PMU counter. +* +* @param pc +* Pointer to the mmapped user page. +* @return +* Counter value read from hardware. +*/ +__rte_internal +static __rte_always_inline uint64_t +rte_pmu_read_userpage (struct perf_event_mmap_page *pc) { +uint64_t offset, width, pmc =0 ; +uint32_t seq, index; +int tries =100 ; + +for (;;) { + seq = pc->lock; +rte_compiler_barrier (); + index = pc->index; + offset = pc->offset; + width = pc->pmc_width; + +if (likely (pc->cap_user_rdpmc && index)) { + pmc =rte_pmu_pmc_read (index -1 ); + pmc <<=64 - width; + pmc >>=64 - width; + } + +rte_compiler_barrier (); + +if (likely (pc->lock == seq)) +return pmc + offset; + +if (--tries ==0 ) { + RTE_LOG(DEBUG, EAL, "failed to get
perf_event_mmap_pagelock\n");lib/eal/include/rte_pmu.h new file +29
-0+break ; + } + } + +return 0 ; +} + +/** +* @internal +* +* Enable group of events for a given lcore. +* +* @param lcore_id +* The identifier of the lcore. +* @return +* 0 in case of success, negative value otherwise. +*/ +__rte_internal +int +rte_pmu_enable_group (int lcore_id); + +/** +* @warning +* @b EXPERIMENTAL: this API may change without prior notice +* +* Add event to the group of enabled events. +* +* @param name +* Name of an event listed under
/sys/bus/event_source/devices/pmu/events.
+ * @return
+ * Event index in case of success, negative value otherwise.
+ */lib/eal/include/rte_pmu.h new file +14
-0+__rte_experimental +int +rte_pmu_add_event (const char *name); + +/** +* @warning +* @b EXPERIMENTAL: this API may change without prior notice +* +* Read hardware counter configured to count occurrences of an event. +* +* @param index +* Index of an event to be read. +* @return +* Event value read from register. In case of errors or lack of support
+ * 0 is returned. In other words, stream of zeros in a trace file
+ * indicates problem with reading particular PMU event register.
+ */lib/eal/include/rte_pmu.h new file +14
-0+__rte_experimental +static __rte_always_inline uint64_t +rte_pmu_read (int index) +{ +int lcore_id =rte_lcore_id (); +struct rte_pmu_event_group *group; +int ret; + +if (!rte_pmu) +return 0 ; + + group = &rte_pmu->group[lcore_id]; +if (!group->enabled) { + ret =rte_pmu_enable_group (lcore_id);
Quoted 9 lines
lib/eal/include/rte_pmu.h line 174
Quoted 5 lines
lib/eal/include/rte_pmu.h new file +5
-0+if (ret) +return 0 ; + + group->enabled = true; + }Why is the group not enabled in the setup function,
rte_pmu_add_event(), instead of here, in the hot path?
When this is executed for the very first time then cpu will have obviously more work to do but afterwards setup path is not taken hence much less cpu cycles are required.
Setup is executed by main lcore solely, before lcores are executed hence some info passed to SYS_perf_event_open ioctl() is missing, pid (via rte_gettid()) being an example here.
Quoted 15 lines
Quoted 6 lines
lib/eal/include/rte_pmu.h new file +5
-0+ +if (index <0 || index >= rte_pmu->num_group_events) +return 0 ; + + return rte_pmu_read_userpage((struct perf_event_mmap_page*)group->mmap_pages[index]);
Using fixed size arrays instead of multiple indirections via pointers is faster. It could be:
return
rte_pmu_read_userpage((struct perf_event_mmap_page *)rte_pmu.group[lcore_id].mmap_pages[index]);With our without suggested performance improvements...
Series-acked-by Morten Brørup <[email protected]>
RE: [PATCH v4 1/4] eal: add generic support for reading PMU events
+CC: Mattias, see my comment below about per-thread constructor for this
Quoted 32 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Wednesday, 14 December 2022 10.39Hello Morten,
Thanks for review. Answers inline.
[...]
Quoted 6 lines
Quoted 6 lines
+/**
+ * @file
+ *
+ * PMU event tracing operations
+ *
+ * This file defines generic API and types necessary to setup PMUand
Quoted 4 lines
Quoted 4 lines
+ * read selected counters in runtime.
+ */(quoted diff) +2
-0+ +/**Quoted 8 lines
Quoted 6 lines
+ * A structure describing a group of events.
+ */lib/eal/include/rte_pmu.h new file +3
-0+struct rte_pmu_event_group { +int *fds;/**< array of event descriptors */ +void **mmap_pages;/**< array of pointers to mmapped
perf_event_attrstructures */There seems to be a lot of indirection involved here. Why are these
arrays not statically sized,
Quoted 2 lines
instead of dynamically allocated?
Quoted 7 lines
Different architectures/pmus impose limits on number of simultaneously enabled counters. So in order
relief the pain of thinking about it and adding macros for each and every arch I decided to allocate
the number user wants dynamically. Also assumption holds that user knows about tradeoffs of using
too many counters hence will not enable too many events at once.
The DPDK convention is to use fixed size arrays (with a maximum size, e.g. RTE_MAX_ETHPORTS) in the fast path, for performance reasons.
Please use fixed size arrays instead of dynamically allocated arrays.
Quoted 7 lines
Quoted 2 lines
Also, what is the reason for hiding the type struct
perf_event_mmap_page**mmap_pages opaque byQuoted 1 line
using void **mmap_pages instead?
I think, that part doing mmap/munmap was written first hence void ** was chosen in the first place.
Please update it, so the actual type is reflected here.
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
(quoted diff) +1
-0+ bool enabled; /**< true if group was enabled on particular lcore*/
lib/eal/include/rte_pmu.h new file +13
-0+}; + +/** +* A structure describing an event. +*/ +struct rte_pmu_event { +char *name;/** name of an event */ +int index;/** event index into fds/mmap_pages */ +TAILQ_ENTRY (rte_pmu_event) next;/** list entry */ }; + +/** +* A PMU state container. +*/
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
(quoted diff) +2
-0+struct rte_pmu { + char *name; /** name of core PMU listed under
/sys/bus/event_source/devices*/(quoted diff) +1
-0+ struct rte_pmu_event_group group[RTE_MAX_LCORE]; /**< per lcoreevent group data */
(quoted diff) +2
-0+ int num_group_events; /**< number of events in a group */ + TAILQ_HEAD(, rte_pmu_event) event_list; /**< list of matchingevents */
The event_list is used in slow path only, so it can remain a list - i.e. no change requested here. :-)
Quoted 6 lines
Quoted 6 lines
app/test/test_pmu.c
Quoted 4 lines
app/test/test_pmu.c new file +4
-0+}; + +/** Pointer to the PMU state container */ extern struct rte_pmu +*rte_pmu;Again, why not just extern struct
rte_pmu, instead of dynamic
Quoted 6 lines
allocation?
Quoted 1 line
No strong opinions here since this is a matter of personal preference. Can be removed
in the next version.
Yes, please.
Quoted 18 lines
Quoted 3 lines
Review
Quoted 3 lines
(quoted diff) +2
-0+ +/** Each architecture supporting PMU needs to provide its ownversion
Quoted 14 lines
Quoted 14 lines
*/
lib/eal/include/rte_pmu.h new file +13
-0+#ifndef rte_pmu_pmc_read +#define rte_pmu_pmc_read (index) ({ 0; }) #endif + +/** +* @internal +* +* Read PMU counter. +* +* @param pc +* Pointer to the mmapped user page. +* @return +* Counter value read from hardware. +*/
Quoted 8 lines
Quoted 8 lines
lib/eal/include/rte_pmu.h line 87
Quoted 8 lines
lib/eal/include/rte_pmu.h new file +8
-0+__rte_internal +static __rte_always_inline uint64_t +rte_pmu_read_userpage (struct perf_event_mmap_page *pc) { +uint64_t offset, width, pmc =0 ; +uint32_t seq, index; +int tries =100 ; + +for (;;) {
As a matter of personal preference, I would write this loop differently:
+ for (tries = 100; tries != 0; tries--) {
Quoted 7 lines
Quoted 7 lines
lib/eal/include/rte_pmu.h line 94
Quoted 7 lines
lib/eal/include/rte_pmu.h new file +7
-0+ seq = pc->lock; +rte_compiler_barrier (); + index = pc->index; + offset = pc->offset; + width = pc->pmc_width; + +if (likely (pc->cap_user_rdpmc && index)) {
Why "&& index"? The way I read [man perf_event_open], index 0 is perfectly valid.
[man perf_event_open]: https://man7.org/linux/man-pages/man2/perf_event_open.2.html
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
lib/eal/include/rte_pmu.h new file +7
-0+ pmc =rte_pmu_pmc_read (index -1 ); + pmc <<=64 - width; + pmc >>=64 - width; + } + +rte_compiler_barrier (); +
Quoted 8 lines
Quoted 8 lines
Review
Quoted 8 lines
lib/eal/include/rte_pmu.h new file +5
-0+if (likely (pc->lock == seq)) +return pmc + offset; + +if (--tries ==0 ) { + RTE_LOG(DEBUG, EAL, "failed to get
perf_event_mmap_pagelock\n");(quoted diff) +2
-0+ break; + }
- Remove the 4 above lines of code, and move the debug log message to the end of the function instead.
Quoted 1 line
Quoted 1 line
Review
Quoted 1 line
(quoted diff) +1
-0+ }
+ RTE_LOG(DEBUG, EAL, "failed to get perf_event_mmap_page lock\n");
Quoted 47 lines
Quoted 31 lines
Quoted 31 lines
lib/eal/include/rte_pmu.h new file +26
-0+ +return 0 ; +} + +/** +* @internal +* +* Enable group of events for a given lcore. +* +* @param lcore_id +* The identifier of the lcore. +* @return +* 0 in case of success, negative value otherwise. +*/ +__rte_internal +int +rte_pmu_enable_group (int lcore_id); + +/** +* @warning +* @b EXPERIMENTAL: this API may change without prior notice +* +* Add event to the group of enabled events. +* +* @param name +* Name of an event listed under
/sys/bus/event_source/devices/pmu/events.
+ * @return
+ * Event index in case of success, negative value otherwise.
+ */
+__rte_experimentalQuoted 8 lines
lib/eal/include/rte_pmu.h
Quoted 8 lines
lib/eal/include/rte_pmu.h new file +8
-0+int +rte_pmu_add_event (const char *name); + +/** +* @warning +* @b EXPERIMENTAL: this API may change without prior notice +* +* Read hardware counter configured to count occurrences of an event.
Quoted 7 lines
Quoted 7 lines
+ *
+ * @param index
+ * Index of an event to be read.
+ * @return
+ * Event value read from register. In case of errors or lack of
support
+ * 0 is returned. In other words, stream of zeros in a trace
Quoted 6 lines
file
Quoted 5 lines
lib/eal/include/rte_pmu.h line 158
Quoted 5 lines
+ * indicates problem with reading particular PMU event register.
+ */lib/eal/include/rte_pmu.h new file +3
-0+__rte_experimental +static __rte_always_inline uint64_t +rte_pmu_read (int index)
The index type can be changed from int to uint32_t. This also eliminates the "(index < 0" part of the comparison further below in this function.
Quoted 10 lines
Quoted 10 lines
lib/eal/include/rte_pmu.h line 168
Quoted 10 lines
lib/eal/include/rte_pmu.h new file +10
-0+{ +int lcore_id =rte_lcore_id (); +struct rte_pmu_event_group *group; +int ret; + +if (!rte_pmu) +return 0 ; + + group = &rte_pmu->group[lcore_id]; +if (!group->enabled) {
Optimized: if (unlikely(!group->enabled)) {
Quoted 17 lines
Quoted 8 lines
lib/eal/include/rte_pmu.h line 174
Quoted 6 lines
lib/eal/include/rte_pmu.h new file +6
-0+ ret =rte_pmu_enable_group (lcore_id); +if (ret) +return 0 ; + + group->enabled = true; + }Why is the group not enabled in the setup function,
rte_pmu_add_event(), instead of here, in theQuoted 2 lines
hot path?
When this is executed for the very first time then cpu will have obviously more work to do
but afterwards setup path is not taken hence much less cpu cycles are required.
Quoted 4 lines
Setup is executed by main lcore solely, before lcores are executed hence some info passed to
SYS_perf_event_openioctl()is missing, pid (viarte_gettid()) being an example here.
OK. Thank you for the explanation. Since impossible at setup, it has to be done at runtime.
@Mattias: Another good example of something that would belong in per-thread constructors, as my suggested feature creep in [1].
[1]: http://inbox.dpdk.org/dev/[email protected]/
Quoted 3 lines
Quoted 3 lines
lib/eal/include/rte_pmu.h line 176
Quoted 3 lines
lib/eal/include/rte_pmu.h new file +2
-0+ +if (index <0 || index >= rte_pmu->num_group_events)
Optimized: if (unlikely(index >= rte_pmu.num_group_events))
Quoted 15 lines
Quoted 6 lines
Quoted 4 lines
lib/eal/common/rte_pmu.c new file +3
-0+return 0 ; + + return rte_pmu_read_userpage((struct perf_event_mmap_page*)group->mmap_pages[index]);
Using fixed size arrays instead of multiple indirections via pointers
is faster. It could be:
Quoted 7 lines
return
rte_pmu_read_userpage((struct perf_event_mmap_page *)rte_pmu.group[lcore_id].mmap_pages[index]);With our without suggested performance improvements...
Series-acked-by Morten Brørup <[email protected]>
RE: [PATCH v4 1/4] eal: add generic support for reading PMU events
Quoted 62 lines
From: Morten Brørup [mailto:[email protected]]
Sent: Wednesday, 14 December 2022 11.41+CC: Mattias, see my comment below about per-thread constructor for this
Quoted 5 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Wednesday, 14 December 2022 10.39Hello Morten,
Quoted 7 lines
Thanks for review. Answers inline.
[...]
Quoted 3 lines
lib/eal/include/rte_pmu.h line 158
Quoted 3 lines
lib/eal/include/rte_pmu.h new file +3
-0+__rte_experimental +static __rte_always_inline uint64_t +rte_pmu_read (int index)The index type can be changed from int to
uint32_t. This also eliminates the "(index < 0" part of the comparison further below in this function.Quoted 10 lines
Quoted 10 lines
lib/eal/include/rte_pmu.h line 168
Quoted 10 lines
lib/eal/include/rte_pmu.h new file +10
-0+{ +int lcore_id =rte_lcore_id (); +struct rte_pmu_event_group *group; +int ret; + +if (!rte_pmu) +return 0 ; + + group = &rte_pmu->group[lcore_id]; +if (!group->enabled) {Optimized: if (
unlikely(!group->enabled)) {Quoted 12 lines
Quoted 8 lines
lib/eal/include/rte_pmu.h line 174
Quoted 6 lines
lib/eal/include/rte_pmu.h new file +6
-0+ ret =rte_pmu_enable_group (lcore_id); +if (ret) +return 0 ; + + group->enabled = true; + }Why is the group not enabled in the setup function,
rte_pmu_add_event(), instead of here, in theQuoted 2 lines
hot path?
Quoted 8 lines
When this is executed for the very first time then cpu will have obviously more work to do
but afterwards setup path is not taken hence much less cpu cycles are required.Setup is executed by main lcore solely, before lcores are executed hence some info passed to
SYS_perf_event_openioctl()is missing, pid (viarte_gettid()) beingan
Quoted 1 line
example here.
OK. Thank you for the explanation. Since impossible at setup, it has to be done at runtime.
Quoted 6 lines
@Mattias: Another good example of something that would belong in per- thread constructors, as my suggested feature creep in [1].
[1]:
http://inbox.dpdk.org/dev/98CBD80474FA8B44BF855DF32C47DC35D87553@smarts
erver.smartshare.dk/
I just realized that this initialization is per-lcore (not per thread), so you can use rte_lcore_callback_register() to register a per-lcore initialization function, and move rte_pmu_enable_group(lcore_id) there.
-Morten
RE: [PATCH v4 1/4] eal: add generic support for reading PMU events
Quoted 71 lines
From: Morten Brørup [mailto:[email protected]]
Sent: Thursday, 15 December 2022 09.22Quoted 23 lines
From: Morten Brørup [mailto:[email protected]]
Sent: Wednesday, 14 December 2022 11.41+CC: Mattias, see my comment below about per-thread constructor for this
Quoted 5 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Wednesday, 14 December 2022 10.39Hello Morten,
Quoted 7 lines
Thanks for review. Answers inline.
[...]
Quoted 3 lines
lib/eal/include/rte_pmu.h line 158
Quoted 3 lines
lib/eal/include/rte_pmu.h new file +3
-0+__rte_experimental +static __rte_always_inline uint64_t +rte_pmu_read (int index)The index type can be changed from int to
uint32_t. This also eliminates the "(index < 0" part of the comparison further below in this function.Quoted 28 lines
Quoted 10 lines
Quoted 10 lines
lib/eal/include/rte_pmu.h line 168
Quoted 10 lines
lib/eal/include/rte_pmu.h new file +10
-0+{ +int lcore_id =rte_lcore_id (); +struct rte_pmu_event_group *group; +int ret; + +if (!rte_pmu) +return 0 ; + + group = &rte_pmu->group[lcore_id]; +if (!group->enabled) {Optimized: if (
unlikely(!group->enabled)) {Quoted 15 lines
Quoted 8 lines
lib/eal/include/rte_pmu.h line 174
Quoted 6 lines
lib/eal/include/rte_pmu.h new file +6
-0+ ret =rte_pmu_enable_group (lcore_id); +if (ret) +return 0 ; + + group->enabled = true; + }Why is the group not enabled in the setup function,
rte_pmu_add_event(), instead of here, in theQuoted 2 lines
hot path?
When this is executed for the very first time then cpu will have obviously more work to do
but afterwards setup path is not taken hence much less cpu cyclesare
Quoted 5 lines
Quoted 5 lines
required.
Setup is executed by main lcore solely, before lcores are executed hence some info passed to
SYS_perf_event_openioctl()is missing, pid (viarte_gettid())being
Quoted 4 lines
an
Quoted 1 line
example here.
OK. Thank you for the explanation. Since impossible at setup, it has
to
Quoted 5 lines
be done at runtime.
@Mattias: Another good example of something that would belong in per- thread constructors, as my suggested feature creep in [1].
Quoted 8 lines
Quoted 2 lines
[1]:
http://inbox.dpdk.org/dev/98CBD80474FA8B44BF855DF32C47DC35D87553@smarts
Quoted 1 line
erver.smartshare.dk/
I just realized that this initialization is per-lcore (not per thread), so you can use
rte_lcore_callback_register()to register a per-lcore initialization function, and moverte_pmu_enable_group(lcore_id)there.
Sorry, Thomasz!
You can't use rte_lcore_callback_register()... it doesn't provide per-lcore thread constructors/destructors the way I thought. :-(
RE: [PATCH v4 1/4] eal: add generic support for reading PMU events
In reply to Morten BrørupHi Morten,
A few comments inline.
Quoted 52 lines
-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: Wednesday, December 14, 2022 11:41 AM
To: Tomasz Duszynski <[email protected]>; [email protected]
Cc: [email protected]; Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; [email protected]
Subject: [EXT] RE: [PATCH v4 1/4] eal: add generic support for reading PMU eventsExternal Email
----------------------------------------------------------------------
+CC: Mattias, see my comment below about per-thread constructor for thisQuoted 32 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Wednesday, 14 December 2022 10.39Hello Morten,
Thanks for review. Answers inline.
[...]
Quoted 6 lines
Quoted 6 lines
+/**
+ * @file
+ *
+ * PMU event tracing operations
+ *
+ * This file defines generic API and types necessary to setup PMUand
Quoted 4 lines
Quoted 4 lines
+ * read selected counters in runtime.
+ */(quoted diff) +2
-0+ +/**Quoted 8 lines
Quoted 6 lines
+ * A structure describing a group of events.
+ */lib/eal/include/rte_pmu.h new file +3
-0+struct rte_pmu_event_group { +int *fds;/**< array of event descriptors */ +void **mmap_pages;/**< array of pointers to mmapped
perf_event_attrstructures */There seems to be a lot of indirection involved here. Why are these
arrays not statically sized,
Quoted 2 lines
instead of dynamically allocated?
Quoted 6 lines
Different architectures/pmus impose limits on number of simultaneously enabled counters. So in order relief the pain of thinking about it and adding macros for each and every arch I decided to allocate the number user wants dynamically. Also assumption holds that user knows about tradeoffs of using too many counters hence will not enable too many events at once.
Quoted 5 lines
The DPDK convention is to use fixed size arrays (with a maximum size, e.g.
RTE_MAX_ETHPORTS) in the fast path, for performance reasons.Please use fixed size arrays instead of dynamically allocated arrays.
I do agree that from maintenance angle fixed arrays are much more convenient but when optimization kicks in then that statement does not necessarily hold true anymore.
For example, in this case performance dropped by ~0.3% which is insignificant imo. So given simpler code, next patchset will use fixed arrays.
Quoted 85 lines
Quoted 7 lines
Quoted 2 lines
Also, what is the reason for hiding the type struct
perf_event_mmap_page**mmap_pages opaque byQuoted 1 line
using void **mmap_pages instead?
I think, that part doing mmap/munmap was written first hence void ** was chosen in the first place.
Please update it, so the actual type is reflected here.
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
(quoted diff) +1
-0+ bool enabled; /**< true if group was enabled on particular lcore*/
lib/eal/include/rte_pmu.h new file +13
-0+}; + +/** +* A structure describing an event. +*/ +struct rte_pmu_event { +char *name;/** name of an event */ +int index;/** event index into fds/mmap_pages */ +TAILQ_ENTRY (rte_pmu_event) next;/** list entry */ }; + +/** +* A PMU state container. +*/ Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
(quoted diff) +2
-0+struct rte_pmu { + char *name; /** name of core PMU listed under
/sys/bus/event_source/devices*/(quoted diff) +1
-0+ struct rte_pmu_event_group group[RTE_MAX_LCORE]; /**< per lcoreevent group data */
(quoted diff) +2
-0+ int num_group_events; /**< number of events in a group */ + TAILQ_HEAD(, rte_pmu_event) event_list; /**< list of matchingevents */
The
event_listis used in slow path only, so it can remain a list - i.e. no change requested here. :-)Quoted 6 lines
Quoted 6 lines
app/test/test_pmu.c
Quoted 4 lines
app/test/test_pmu.c new file +4
-0+}; + +/** Pointer to the PMU state container */ extern struct rte_pmu +*rte_pmu;Again, why not just extern struct
rte_pmu, instead of dynamicQuoted 6 lines
allocation?
Quoted 1 line
No strong opinions here since this is a matter of personal preference. Can be removed
in the next version.Yes, please.
Quoted 18 lines
Quoted 3 lines
Review
Quoted 3 lines
(quoted diff) +2
-0+ +/** Each architecture supporting PMU needs to provide its ownversion
Quoted 14 lines
Quoted 14 lines
*/
lib/eal/include/rte_pmu.h new file +13
-0+#ifndef rte_pmu_pmc_read +#define rte_pmu_pmc_read (index) ({ 0; }) #endif + +/** +* @internal +* +* Read PMU counter. +* +* @param pc +* Pointer to the mmapped user page. +* @return +* Counter value read from hardware. +*/ Quoted 8 lines
Quoted 8 lines
lib/eal/include/rte_pmu.h line 87
Quoted 8 lines
lib/eal/include/rte_pmu.h new file +8
-0+__rte_internal +static __rte_always_inline uint64_t rte_pmu_read_userpage(struct +perf_event_mmap_page *pc) { +uint64_t offset, width, pmc =0 ; +uint32_t seq, index; +int tries =100 ; + +for (;;) {As a matter of personal preference, I would write this loop differently:
+ for (tries = 100; tries != 0; tries--) {
Quoted 10 lines
Quoted 7 lines
Quoted 7 lines
lib/eal/include/rte_pmu.h line 94
Quoted 7 lines
lib/eal/include/rte_pmu.h new file +7
-0+ seq = pc->lock; +rte_compiler_barrier (); + index = pc->index; + offset = pc->offset; + width = pc->pmc_width; + +if (likely (pc->cap_user_rdpmc && index)) {Why "&& index"? The way I read [man
perf_event_open], index 0 is perfectly valid.
Valid index starts at 1. 0 means that either hw counter is stopped or isn't active. Maybe this is not initially clear from man but there's example later on how to get actual number.
Quoted 76 lines
[man
perf_event_open]: https://urldefense.proofpoint.com/v2/url?u=https-3A__man7.org_linux_man-
2Dpages_man2_perf-5Fevent- 5Fopen.2.html&d=DwIFAw&c=nKjWec2b6R0mOyPaz7xtfQ&r=PZNXgrbjdlXxVEEGYkxIxRndyEUwWU_ad5ce22YI6Is&m=tny gBVwOnoZDV7hItku1HtmsI8R3F6vPJdr7ON3hE5iAds96T2C9JTNcnt6ptN4Q&s=s10yJogwRRXHqAuIay47H- aWl9SL5wpQ9tCjfiQUgrY&e=Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
lib/eal/include/rte_pmu.h new file +7
-0+ pmc =rte_pmu_pmc_read (index -1 ); + pmc <<=64 - width; + pmc >>=64 - width; + } + +rte_compiler_barrier (); +Quoted 8 lines
Quoted 8 lines
Review
Quoted 8 lines
lib/eal/include/rte_pmu.h new file +5
-0+if (likely (pc->lock == seq)) +return pmc + offset; + +if (--tries ==0 ) { + RTE_LOG(DEBUG, EAL, "failed to get
perf_event_mmap_pagelock\n");(quoted diff) +2
-0+ break; + }- Remove the 4 above lines of code, and move the debug log message to the end of the function instead.
Quoted 1 line
Quoted 1 line
Review
Quoted 1 line
(quoted diff) +1
-0+ }+
RTE_LOG(DEBUG, EAL, "failed to get perf_event_mmap_page lock\n");Quoted 47 lines
Quoted 31 lines
Quoted 31 lines
lib/eal/include/rte_pmu.h new file +26
-0+ +return 0 ; +} + +/** +* @internal +* +* Enable group of events for a given lcore. +* +* @param lcore_id +* The identifier of the lcore. +* @return +* 0 in case of success, negative value otherwise. +*/ +__rte_internal +int +rte_pmu_enable_group (int lcore_id); + +/** +* @warning +* @b EXPERIMENTAL: this API may change without prior notice +* +* Add event to the group of enabled events. +* +* @param name +* Name of an event listed under
/sys/bus/event_source/devices/pmu/events.
+ * @return
+ * Event index in case of success, negative value otherwise.
+ */
+__rte_experimentalQuoted 8 lines
lib/eal/include/rte_pmu.h
Quoted 8 lines
lib/eal/include/rte_pmu.h new file +8
-0+int +rte_pmu_add_event (const char *name); + +/** +* @warning +* @b EXPERIMENTAL: this API may change without prior notice +* +* Read hardware counter configured to count occurrences of an event.
Quoted 7 lines
Quoted 7 lines
+ *
+ * @param index
+ * Index of an event to be read.
+ * @return
+ * Event value read from register. In case of errors or lack of
support
+ * 0 is returned. In other words, stream of zeros in a trace
Quoted 9 lines
Quoted 5 lines
file
Quoted 4 lines
Review
Quoted 4 lines
+ * indicates problem with reading particular PMU event register.
+ */(quoted diff) +2
-0+__rte_experimental +static __rte_always_inline uint64_t rte_pmu_read(int index)The index type can be changed from int to
uint32_t. This also eliminates the "(index < 0" part of the comparison further below in this function.
That's true.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
lib/eal/include/rte_pmu.h new file +4
-0+{ +int lcore_id =rte_lcore_id (); +struct rte_pmu_event_group *group; +int ret;
Quoted 9 lines
Quoted 6 lines
Quoted 6 lines
lib/eal/include/rte_pmu.h line 168
Quoted 6 lines
lib/eal/include/rte_pmu.h new file +6
-0+ +if (!rte_pmu) +return 0 ; + + group = &rte_pmu->group[lcore_id]; +if (!group->enabled) {Optimized: if (
unlikely(!group->enabled)) {
Compiler will optimize the branch itself correctly. Extra hint is not required.
Quoted 53 lines
Quoted 12 lines
Quoted 8 lines
lib/eal/include/rte_pmu.h line 174
Quoted 6 lines
lib/eal/include/rte_pmu.h new file +6
-0+ ret =rte_pmu_enable_group (lcore_id); +if (ret) +return 0 ; + + group->enabled = true; + }Why is the group not enabled in the setup function,
rte_pmu_add_event(), instead of here, in theQuoted 2 lines
hot path?
Quoted 7 lines
When this is executed for the very first time then cpu will have obviously more work to do but afterwards setup path is not taken hence much less cpu cycles are required.
Setup is executed by main lcore solely, before lcores are executed hence some info passed to
SYS_perf_event_openioctl()is missing, pid (viarte_gettid()) being an example here.OK. Thank you for the explanation. Since impossible at setup, it has to be done at runtime.
@Mattias: Another good example of something that would belong in per-thread constructors, as my suggested feature creep in [1].
[1]: https://urldefense.proofpoint.com/v2/url?u=http-
3A__inbox.dpdk.org_dev_98CBD80474FA8B44BF855DF32C47DC35D87553- 40smartserver.smartshare.dk_&d=DwIFAw&c=nKjWec2b6R0mOyPaz7xtfQ&r=PZNXgrbjdlXxVEEGYkxIxRndyEUwWU_ad5 ce22YI6Is&m=tnygBVwOnoZDV7hItku1HtmsI8R3F6vPJdr7ON3hE5iAds96T2C9JTNcnt6ptN4Q&s=aSAnYqgVnrgDp6yyMtGC uWgJjDlgqj1wHf1nGWyHCNo&e=Quoted 3 lines
Quoted 3 lines
lib/eal/include/rte_pmu.h line 176
Quoted 3 lines
lib/eal/include/rte_pmu.h new file +2
-0+ +if (index <0 || index >= rte_pmu->num_group_events)Optimized: if (
unlikely(index >= rte_pmu.num_group_events))Quoted 16 lines
Quoted 7 lines
Quoted 4 lines
lib/eal/common/rte_pmu.c new file +3
-0+return 0 ; + + return rte_pmu_read_userpage((struct perf_event_mmap_page*)group->mmap_pages[index]);
Using fixed size arrays instead of multiple indirections via pointers
is faster. It could be:
Quoted 7 lines
return
rte_pmu_read_userpage((struct perf_event_mmap_page *)rte_pmu.group[lcore_id].mmap_pages[index]);With our without suggested performance improvements...
Series-acked-by Morten Brørup <[email protected]>
RE: [PATCH v4 1/4] eal: add generic support for reading PMU events
Quoted 67 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Thursday, 5 January 2023 22.14Hi Morten,
A few comments inline.
Quoted 7 lines
From: Morten Brørup <[email protected]>
Sent: Wednesday, December 14, 2022 11:41 AMExternal Email
----------------------------------------------------------------------
+CC: Mattias, see my comment below about per-thread constructor forthis
Quoted 8 lines
Quoted 8 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Wednesday, 14 December 2022 10.39Hello Morten,
Thanks for review. Answers inline.
Quoted 8 lines
Quoted 8 lines
[...]
Quoted 6 lines
Quoted 6 lines
+/**
+ * @file
+ *
+ * PMU event tracing operations
+ *
+ * This file defines generic API and types necessary to setupPMU
Quoted 13 lines
Quoted 13 lines
and
Quoted 12 lines
Quoted 10 lines
+ * read selected counters in runtime.
+ */lib/eal/include/rte_pmu.h new file +7
-0+ +/** +* A structure describing a group of events. +*/ +struct rte_pmu_event_group { +int *fds;/**< array of event descriptors */ +void **mmap_pages;/**< array of pointers to mmapped
perf_event_attrstructures */There seems to be a lot of indirection involved here. Why are
these
Quoted 5 lines
Quoted 5 lines
arrays not statically sized,
Quoted 2 lines
instead of dynamically allocated?
Different architectures/pmus impose limits on number of
simultaneously
Quoted 1 line
Quoted 1 line
enabled counters. So in order relief the pain of thinking about it
and
Quoted 1 line
Quoted 1 line
adding macros for each and every arch I decided to allocate the
number
Quoted 5 lines
Quoted 3 lines
user wants dynamically. Also assumption holds that user knows about tradeoffs of using too many counters hence will not enable too many events at once.
The DPDK convention is to use fixed size arrays (with a maximum size,
e.g.
RTE_MAX_ETHPORTS) in theQuoted 4 lines
fast path, for performance reasons.
Please use fixed size arrays instead of dynamically allocated arrays.
Quoted 8 lines
I do agree that from maintenance angle fixed arrays are much more convenient
but when optimization kicks in then that statement does not necessarily hold true anymore.For example, in this case performance dropped by ~0.3% which is insignificant imo. So
given simpler code, next patchset will use fixed arrays.
I fail to understand how pointer chasing can perform better than obtaining an address by multiplying by a constant. Modern CPUs work in mysterious ways, and you obviously tested this, so I believe your test results. But in theory, pointer chasing touches more cache lines, and should perform worse in a loaded system where pointers in the chain have been evicted from the cache.
Anyway, you agreed to use fixed arrays, so I am happy. :-)
Quoted 102 lines
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
Also, what is the reason for hiding the type struct
perf_event_mmap_page**mmap_pages opaque byQuoted 1 line
using void **mmap_pages instead?
Quoted 8 lines
Quoted 2 lines
I think, that part doing mmap/munmap was written first hence void ** was chosen in the first place.
Please update it, so the actual type is reflected here.
Quoted 3 lines
Quoted 3 lines
Review
Quoted 3 lines
(quoted diff) +1
-0+ bool enabled; /**< true if group was enabled on particularlcore
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
*/
lib/eal/include/rte_pmu.h new file +9
-0+}; + +/** +* A structure describing an event. +*/ +struct rte_pmu_event { +char *name;/** name of an event */ +int index;/** event index into fds/mmap_pages */ +TAILQ_ENTRY (rte_pmu_event) next;/** list entry */ };Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Review
Quoted 8 lines
lib/eal/include/rte_pmu.h new file +6
-0+ +/** +* A PMU state container. +*/ +struct rte_pmu { +char *name;/** name of core PMU listed under
/sys/bus/event_source/devices*/(quoted diff) +1
-0+ struct rte_pmu_event_group group[RTE_MAX_LCORE]; /**< perlcore
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Review
Quoted 3 lines
event group data */
(quoted diff) +2
-0+ int num_group_events; /**< number of events in a group */ + TAILQ_HEAD(, rte_pmu_event) event_list; /**< list ofmatching
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
events */
The
event_listis used in slow path only, so it can remain a list -i.e. no change requested here.
Quoted 12 lines
:-)
Quoted 10 lines
Quoted 6 lines
app/test/test_pmu.c
Quoted 4 lines
app/test/test_pmu.c new file +4
-0+}; + +/** Pointer to the PMU state container */ extern struct rte_pmu +*rte_pmu;Again, why not just extern struct
rte_pmu, instead of dynamicallocation?
Quoted 1 line
No strong opinions here since this is a matter of personal
preference.
Quoted 9 lines
Quoted 2 lines
Can be removed
in the next version.Yes, please.
Quoted 4 lines
Quoted 3 lines
Review
Quoted 3 lines
(quoted diff) +2
-0+ +/** Each architecture supporting PMU needs to provide its ownversion
Quoted 16 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
*/
lib/eal/include/rte_pmu.h new file +7
-0+#ifndef rte_pmu_pmc_read +#define rte_pmu_pmc_read (index) ({ 0; }) #endif + +/** +* @internal +* +* Read PMU counter. Quoted 8 lines
Quoted 8 lines
Review
Quoted 8 lines
+ *
+ * @param pc
+ * Pointer to the mmapped user page.
+ * @return
+ * Counter value read from hardware.
+ */(quoted diff) +2
-0+__rte_internal +static __rte_always_inline uint64_trte_pmu_read_userpage(struct
Quoted 8 lines
Quoted 6 lines
Quoted 6 lines
lib/eal/include/rte_pmu.h line 87
Quoted 6 lines
lib/eal/include/rte_pmu.h new file +6
-0+perf_event_mmap_page *pc) { +uint64_t offset, width, pmc =0 ; +uint32_t seq, index; +int tries =100 ; + +for (;;) {As a matter of personal preference, I would write this loop
differently:
Quoted 12 lines
+ for (tries = 100; tries != 0; tries--) {
Quoted 7 lines
Quoted 7 lines
lib/eal/include/rte_pmu.h line 94
Quoted 7 lines
lib/eal/include/rte_pmu.h new file +7
-0+ seq = pc->lock; +rte_compiler_barrier (); + index = pc->index; + offset = pc->offset; + width = pc->pmc_width; + +if (likely (pc->cap_user_rdpmc && index)) {Why "&& index"? The way I read [man
perf_event_open], index 0 is
Quoted 7 lines
perfectly valid.
Quoted 1 line
Valid index starts at 1. 0 means that either hw counter is stopped or isn't active. Maybe this is not
initially clear from man but there's example later on how to get actual number.
OK. Thanks for the reference.
Please add a comment about the special meaning of index 0 in the code.
Quoted 111 lines
Quoted 2 lines
[man
perf_event_open]:https://urldefense.proofpoint.com/v2/url?u=https-
3A__man7.org_linux_man-Quoted 2 lines
2Dpages_man2_perf-5Fevent- 5Fopen.2.html&d=DwIFAw&c=nKjWec2b6R0mOyPaz7xtfQ&r=PZNXgrbjdlXxVEEGYkxI
xRndyEUwWU_ad5ce22YI6Is&m=tny
Quoted 1 line
gBVwOnoZDV7hItku1HtmsI8R3F6vPJdr7ON3hE5iAds96T2C9JTNcnt6ptN4Q&s=s10yJo
gwRRXHqAuIay47H-
Quoted 11 lines
aWl9SL5wpQ9tCjfiQUgrY&e=
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
lib/eal/include/rte_pmu.h new file +9
-0+ pmc =rte_pmu_pmc_read (index -1 ); + pmc <<=64 - width; + pmc >>=64 - width; + } + +rte_compiler_barrier (); + +if (likely (pc->lock == seq)) +return pmc + offset;Quoted 8 lines
Quoted 6 lines
Quoted 6 lines
Review
Quoted 6 lines
lib/eal/include/rte_pmu.h new file +3
-0+ +if (--tries ==0 ) { + RTE_LOG(DEBUG, EAL, "failed to get
perf_event_mmap_pagelock\n");(quoted diff) +2
-0+ break; + }- Remove the 4 above lines of code, and move the debug log message to
the end of the function
Quoted 51 lines
instead.
Quoted 1 line
Quoted 1 line
Review
Quoted 1 line
(quoted diff) +1
-0+ }+
RTE_LOG(DEBUG, EAL, "failed to get perf_event_mmap_page lock\n");Quoted 45 lines
Quoted 31 lines
Quoted 31 lines
lib/eal/include/rte_pmu.h new file +26
-0+ +return 0 ; +} + +/** +* @internal +* +* Enable group of events for a given lcore. +* +* @param lcore_id +* The identifier of the lcore. +* @return +* 0 in case of success, negative value otherwise. +*/ +__rte_internal +int +rte_pmu_enable_group (int lcore_id); + +/** +* @warning +* @b EXPERIMENTAL: this API may change without prior notice +* +* Add event to the group of enabled events. +* +* @param name +* Name of an event listed under
/sys/bus/event_source/devices/pmu/events.
+ * @return
+ * Event index in case of success, negative value otherwise.
+ */
+__rte_experimentalQuoted 8 lines
lib/eal/include/rte_pmu.h
Quoted 8 lines
lib/eal/include/rte_pmu.h new file +8
-0+int +rte_pmu_add_event (const char *name); + +/** +* @warning +* @b EXPERIMENTAL: this API may change without prior notice +* +* Read hardware counter configured to count occurrences of an event.
Quoted 5 lines
Quoted 5 lines
+ *
+ * @param index
+ * Index of an event to be read.
+ * @return
+ * Event value read from register. In case of errors or lackof
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines
support
+ * 0 is returned. In other words, stream of zeros in a tracefile
Quoted 1 line
Quoted 1 line
+ * indicates problem with reading particular PMU event
register.
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
Review
Quoted 3 lines
+ */
(quoted diff) +2
-0+__rte_experimental +static __rte_always_inline uint64_t rte_pmu_read(int index)The index type can be changed from int to
uint32_t. This alsoeliminates the "(index < 0" part of
Quoted 2 lines
the comparison further below in this function.
That's true.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
lib/eal/include/rte_pmu.h new file +4
-0+{ +int lcore_id =rte_lcore_id (); +struct rte_pmu_event_group *group; +int ret;Quoted 9 lines
Quoted 6 lines
Quoted 6 lines
lib/eal/include/rte_pmu.h line 168
Quoted 6 lines
lib/eal/include/rte_pmu.h new file +6
-0+ +if (!rte_pmu) +return 0 ; + + group = &rte_pmu->group[lcore_id]; +if (!group->enabled) {Optimized: if (
unlikely(!group->enabled)) {
Quoted 2 lines
Compiler will optimize the branch itself correctly. Extra hint is not required.
I haven't reviewed the output from this, so I'll take your word for it. I suggested the unlikely() because I previously tested some very simple code, and it optimized for taking the "if":
void testb(bool b)
{
if (!b)
exit(1);
exit(99);
}I guess I should experiment with more realistic code, and update my optimization notes!
You could add the unlikely() for readability purposes. ;-)
Quoted 61 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
lib/eal/include/rte_pmu.h line 174
Quoted 7 lines
lib/eal/include/rte_pmu.h new file +6
-0+ ret =rte_pmu_enable_group (lcore_id); +if (ret) +return 0 ; + + group->enabled = true; + }Why is the group not enabled in the setup function,
Quoted 6 lines
Quoted 6 lines
rte_pmu_add_event(), instead of here, in theQuoted 2 lines
hot path?
When this is executed for the very first time then cpu will have obviously more work to do but afterwards setup path is not taken
hence
Quoted 4 lines
Quoted 4 lines
much less cpu cycles are required.
Setup is executed by main lcore solely, before lcores are executed hence some info passed to
SYS_perf_event_openioctl()is missing,pid
Quoted 3 lines
Quoted 1 line
(via
rte_gettid()) being an example here.OK. Thank you for the explanation. Since impossible at setup, it has
to be done at runtime.
Quoted 2 lines
@Mattias: Another good example of something that would belong in per-
thread constructors, as my
Quoted 5 lines
suggested feature creep in [1].
[1]: https://urldefense.proofpoint.com/v2/url?u=http-
3A__inbox.dpdk.org_dev_98CBD80474FA8B44BF855DF32C47DC35D87553- 40smartserver.smartshare.dk_&d=DwIFAw&c=nKjWec2b6R0mOyPaz7xtfQ&r=PZNXg
rbjdlXxVEEGYkxIxRndyEUwWU_ad5Quoted 1 line
ce22YI6Is&m=tnygBVwOnoZDV7hItku1HtmsI8R3F6vPJdr7ON3hE5iAds96T2C9JTNcnt
6ptN4Q&s=aSAnYqgVnrgDp6yyMtGC
Quoted 24 lines
uWgJjDlgqj1wHf1nGWyHCNo&e=
Quoted 3 lines
Quoted 3 lines
lib/eal/include/rte_pmu.h line 176
Quoted 3 lines
lib/eal/include/rte_pmu.h new file +2
-0+ +if (index <0 || index >= rte_pmu->num_group_events)Optimized: if (
unlikely(index >= rte_pmu.num_group_events))Quoted 16 lines
Quoted 7 lines
Quoted 4 lines
lib/eal/common/rte_pmu.c new file +3
-0+return 0 ; + + return rte_pmu_read_userpage((struct perf_event_mmap_page*)group->mmap_pages[index]);
Using fixed size arrays instead of multiple indirections via pointers
is faster. It could be:
Quoted 7 lines
return
rte_pmu_read_userpage((struct perf_event_mmap_page *)rte_pmu.group[lcore_id].mmap_pages[index]);With our without suggested performance improvements...
Series-acked-by Morten Brørup <[email protected]>
RE: [PATCH v4 1/4] eal: add generic support for reading PMU events
Quoted 327 lines
-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: Thursday, January 5, 2023 11:08 PM
To: Tomasz Duszynski <[email protected]>; [email protected]
Cc: [email protected]; Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; [email protected]
Subject: [EXT] RE: [PATCH v4 1/4] eal: add generic support for reading PMU eventsExternal Email
----------------------------------------------------------------------
Quoted 68 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Thursday, 5 January 2023 22.14Hi Morten,
A few comments inline.
Review
Quoted 8 lines
From: Morten Brørup <[email protected]>
Sent: Wednesday, December 14, 2022 11:41 AMExternal Email
---------------------------------------------------------------------
(quoted diff) +1
-1- +CC: Mattias, see my comment below about per-thread constructor forthis
Quoted 8 lines
Quoted 8 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Wednesday, 14 December 2022 10.39Hello Morten,
Thanks for review. Answers inline.
Quoted 8 lines
Quoted 8 lines
[...]
Quoted 6 lines
Quoted 6 lines
+/**
+ * @file
+ *
+ * PMU event tracing operations
+ *
+ * This file defines generic API and types necessary to setupPMU
Quoted 13 lines
Quoted 13 lines
and
Quoted 12 lines
Quoted 10 lines
+ * read selected counters in runtime.
+ */lib/eal/include/rte_pmu.h new file +7
-0+ +/** +* A structure describing a group of events. +*/ +struct rte_pmu_event_group { +int *fds;/**< array of event descriptors */ +void **mmap_pages;/**< array of pointers to mmapped
perf_event_attrstructures */There seems to be a lot of indirection involved here. Why are
these
Quoted 5 lines
Quoted 5 lines
arrays not statically sized,
Quoted 2 lines
instead of dynamically allocated?
Different architectures/pmus impose limits on number of
simultaneously
Quoted 1 line
Quoted 1 line
enabled counters. So in order relief the pain of thinking about it
and
Quoted 1 line
Quoted 1 line
adding macros for each and every arch I decided to allocate the
number
Quoted 5 lines
Quoted 3 lines
user wants dynamically. Also assumption holds that user knows about tradeoffs of using too many counters hence will not enable too many events at once.
The DPDK convention is to use fixed size arrays (with a maximum size,
e.g.
RTE_MAX_ETHPORTS) in theQuoted 4 lines
fast path, for performance reasons.
Please use fixed size arrays instead of dynamically allocated arrays.
Quoted 7 lines
I do agree that from maintenance angle fixed arrays are much more convenient but when optimization kicks in then that statement does not necessarily hold true anymore.
For example, in this case performance dropped by ~0.3% which is insignificant imo. So given simpler code, next patchset will use fixed arrays.
I fail to understand how pointer chasing can perform better than obtaining an address by multiplying by a constant. Modern CPUs work in mysterious ways, and you obviously tested this, so I believe your test results. But in theory, pointer chasing touches more cache lines, and should perform worse in a loaded system where pointers in the chain have been evicted from the cache.
Anyway, you agreed to use fixed arrays, so I am happy. :-)
Quoted 102 lines
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
Also, what is the reason for hiding the type struct
perf_event_mmap_page**mmap_pages opaque byQuoted 1 line
using void **mmap_pages instead?
Quoted 8 lines
Quoted 2 lines
I think, that part doing mmap/munmap was written first hence void ** was chosen in the first place.
Please update it, so the actual type is reflected here.
Quoted 3 lines
Quoted 3 lines
Review
Quoted 3 lines
(quoted diff) +1
-0+ bool enabled; /**< true if group was enabled on particularlcore
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
*/
lib/eal/include/rte_pmu.h new file +9
-0+}; + +/** +* A structure describing an event. +*/ +struct rte_pmu_event { +char *name;/** name of an event */ +int index;/** event index into fds/mmap_pages */ +TAILQ_ENTRY (rte_pmu_event) next;/** list entry */ };Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Review
Quoted 8 lines
lib/eal/include/rte_pmu.h new file +6
-0+ +/** +* A PMU state container. +*/ +struct rte_pmu { +char *name;/** name of core PMU listed under
/sys/bus/event_source/devices*/(quoted diff) +1
-0+ struct rte_pmu_event_group group[RTE_MAX_LCORE]; /**< perlcore
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Review
Quoted 3 lines
event group data */
(quoted diff) +2
-0+ int num_group_events; /**< number of events in a group */ + TAILQ_HEAD(, rte_pmu_event) event_list; /**< list ofmatching
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
events */
The
event_listis used in slow path only, so it can remain a list -i.e. no change requested here.
Quoted 12 lines
:-)
Quoted 10 lines
Quoted 6 lines
app/test/test_pmu.c
Quoted 4 lines
app/test/test_pmu.c new file +4
-0+}; + +/** Pointer to the PMU state container */ extern struct +rte_pmu *rte_pmu;Again, why not just extern struct
rte_pmu, instead of dynamicallocation?
Quoted 1 line
No strong opinions here since this is a matter of personal
preference.
Quoted 9 lines
Quoted 2 lines
Can be removed
in the next version.Yes, please.
Quoted 4 lines
Quoted 3 lines
Review
Quoted 3 lines
(quoted diff) +2
-0+ +/** Each architecture supporting PMU needs to provide its ownversion
Quoted 16 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
*/
lib/eal/include/rte_pmu.h new file +7
-0+#ifndef rte_pmu_pmc_read +#define rte_pmu_pmc_read (index) ({ 0; }) #endif + +/** +* @internal +* +* Read PMU counter. Quoted 8 lines
Quoted 8 lines
Review
Quoted 8 lines
+ *
+ * @param pc
+ * Pointer to the mmapped user page.
+ * @return
+ * Counter value read from hardware.
+ */(quoted diff) +2
-0+__rte_internal +static __rte_always_inline uint64_trte_pmu_read_userpage(struct
Quoted 8 lines
Quoted 6 lines
Quoted 6 lines
lib/eal/include/rte_pmu.h line 87
Quoted 6 lines
lib/eal/include/rte_pmu.h new file +6
-0+perf_event_mmap_page *pc) { +uint64_t offset, width, pmc =0 ; +uint32_t seq, index; +int tries =100 ; + +for (;;) {As a matter of personal preference, I would write this loop
differently:
Quoted 12 lines
+ for (tries = 100; tries != 0; tries--) {
Quoted 7 lines
Quoted 7 lines
lib/eal/include/rte_pmu.h line 94
Quoted 7 lines
lib/eal/include/rte_pmu.h new file +7
-0+ seq = pc->lock; +rte_compiler_barrier (); + index = pc->index; + offset = pc->offset; + width = pc->pmc_width; + +if (likely (pc->cap_user_rdpmc && index)) {Why "&& index"? The way I read [man
perf_event_open], index 0 isQuoted 6 lines
perfectly valid.
Quoted 1 line
Valid index starts at 1. 0 means that either hw counter is stopped or isn't active. Maybe this is not initially clear from man but there's example later on how to get actual number.
OK. Thanks for the reference.
Please add a comment about the special meaning of index 0 in the code.
Quoted 113 lines
Quoted 2 lines
[man
perf_event_open]:https://urldefense.proofpoint.com/v2/url?u=https-
3A__man7.org_linux_man-Quoted 3 lines
2Dpages_man2_perf-5Fevent- 5Fopen.2.html&d=DwIFAw&c=nKjWec2b6R0mOyPaz7xtfQ&r=PZNXgrbjdlXxVEEGYkx I
xRndyEUwWU_ad5ce22YI6Is&m=tny
Quoted 2 lines
gBVwOnoZDV7hItku1HtmsI8R3F6vPJdr7ON3hE5iAds96T2C9JTNcnt6ptN4Q&s=s10yJ o
gwRRXHqAuIay47H-
Quoted 11 lines
aWl9SL5wpQ9tCjfiQUgrY&e=
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
lib/eal/include/rte_pmu.h new file +9
-0+ pmc =rte_pmu_pmc_read (index -1 ); + pmc <<=64 - width; + pmc >>=64 - width; + } + +rte_compiler_barrier (); + +if (likely (pc->lock == seq)) +return pmc + offset;Quoted 8 lines
Quoted 6 lines
Quoted 6 lines
Review
Quoted 6 lines
lib/eal/include/rte_pmu.h new file +3
-0+ +if (--tries ==0 ) { + RTE_LOG(DEBUG, EAL, "failed to get
perf_event_mmap_pagelock\n");(quoted diff) +2
-0+ break; + }- Remove the 4 above lines of code, and move the debug log message to
the end of the function
Quoted 51 lines
instead.
Quoted 1 line
Quoted 1 line
Review
Quoted 1 line
(quoted diff) +1
-0+ }+
RTE_LOG(DEBUG, EAL, "failed to get perf_event_mmap_page lock\n");Quoted 45 lines
Quoted 31 lines
Quoted 31 lines
lib/eal/include/rte_pmu.h new file +26
-0+ +return 0 ; +} + +/** +* @internal +* +* Enable group of events for a given lcore. +* +* @param lcore_id +* The identifier of the lcore. +* @return +* 0 in case of success, negative value otherwise. +*/ +__rte_internal +int +rte_pmu_enable_group (int lcore_id); + +/** +* @warning +* @b EXPERIMENTAL: this API may change without prior notice +* +* Add event to the group of enabled events. +* +* @param name +* Name of an event listed under
/sys/bus/event_source/devices/pmu/events.
+ * @return
+ * Event index in case of success, negative value otherwise.
+ */
+__rte_experimentalQuoted 8 lines
lib/eal/include/rte_pmu.h
Quoted 8 lines
lib/eal/include/rte_pmu.h new file +8
-0+int +rte_pmu_add_event (const char *name); + +/** +* @warning +* @b EXPERIMENTAL: this API may change without prior notice +* +* Read hardware counter configured to count occurrences of an event.
Quoted 5 lines
Quoted 5 lines
+ *
+ * @param index
+ * Index of an event to be read.
+ * @return
+ * Event value read from register. In case of errors or lackof
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines
support
+ * 0 is returned. In other words, stream of zeros in a tracefile
Quoted 1 line
Quoted 1 line
+ * indicates problem with reading particular PMU event
register.
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
Review
Quoted 3 lines
+ */
(quoted diff) +2
-0+__rte_experimental +static __rte_always_inline uint64_t rte_pmu_read(int index)The index type can be changed from int to
uint32_t. This alsoeliminates the "(index < 0" part of
Quoted 2 lines
the comparison further below in this function.
That's true.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
lib/eal/include/rte_pmu.h new file +4
-0+{ +int lcore_id =rte_lcore_id (); +struct rte_pmu_event_group *group; +int ret;Quoted 9 lines
Quoted 6 lines
Quoted 6 lines
lib/eal/include/rte_pmu.h line 168
Quoted 6 lines
lib/eal/include/rte_pmu.h new file +6
-0+ +if (!rte_pmu) +return 0 ; + + group = &rte_pmu->group[lcore_id]; +if (!group->enabled) {Optimized: if (
unlikely(!group->enabled)) {Quoted 2 lines
Compiler will optimize the branch itself correctly. Extra hint is not required.
I haven't reviewed the output from this, so I'll take your word for it. I suggested the
unlikely()because I previously tested some very simple code, and it optimized for taking the "if":void testb(bool b)
Quoted 9 lines
{ if (!b) exit(1); exit(99); }I guess I should experiment with more realistic code, and update my optimization notes!
I think this may be too simple to draw far-reaching conclusions from it. Compiler will make the fall-through path more likely. If I recall Intel Optimization Reference Manual has some more info on this.
Lets take a different example.
int main(int argc, char *argv[])
{
int *p;
p = malloc(sizeof(*p));
if (!p)
return 1;
*p = atoi(argv[1]);
if (*p < 0)
return 2;
free(p);
return 0;
}If compiled with -O3 and disassembled I got below.
00000000000010a0 <main>:
10a0: f3 0f 1e fa endbr64
10a4: 55 push %rbp
10a5: bf 04 00 00 00 mov $0x4,%edi
10aa: 53 push %rbx
10ab: 48 89 f3 mov %rsi,%rbx
10ae: 48 83 ec 08 sub $0x8,%rsp
10b2: e8 d9 ff ff ff call 1090 <malloc@plt>
10b7: 48 85 c0 test %rax,%rax
10ba: 74 31 je 10ed <main+0x4d>
10bc: 48 8b 7b 08 mov 0x8(%rbx),%rdi
10c0: ba 0a 00 00 00 mov $0xa,%edx
10c5: 31 f6 xor %esi,%esi
10c7: 48 89 c5 mov %rax,%rbp
10ca: e8 b1 ff ff ff call 1080 <strtol@plt>
10cf: 49 89 c0 mov %rax,%r8
10d2: b8 02 00 00 00 mov $0x2,%eax
10d7: 45 85 c0 test %r8d,%r8d
10da: 78 0a js 10e6 <main+0x46>
10dc: 48 89 ef mov %rbp,%rdi
10df: e8 8c ff ff ff call 1070 <free@plt>
10e4: 31 c0 xor %eax,%eax
10e6: 48 83 c4 08 add $0x8,%rsp
10ea: 5b pop %rbx
10eb: 5d pop %rbp
10ec: c3 ret
10ed: b8 01 00 00 00 mov $0x1,%eax
10f2: eb f2 jmp 10e6 <main+0x46>Looking at both 10ba and 10da suggests that code was laid out in a way that jumping is frowned upon. Also potentially lest frequently executed code (at 10ed) is pushed further down the memory hence optimizing cache line usage.
That said, each and every scenario needs analysis on its own.
Quoted 2 lines
You could add the
unlikely()for readability purposes. ;-)
Sure. That won't hurt performance.
Quoted 63 lines
Quoted 63 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
lib/eal/include/rte_pmu.h line 174
Quoted 7 lines
lib/eal/include/rte_pmu.h new file +6
-0+ ret =rte_pmu_enable_group (lcore_id); +if (ret) +return 0 ; + + group->enabled = true; + }Why is the group not enabled in the setup function,
Quoted 6 lines
Quoted 6 lines
rte_pmu_add_event(), instead of here, in theQuoted 2 lines
hot path?
When this is executed for the very first time then cpu will have obviously more work to do but afterwards setup path is not taken
hence
Quoted 4 lines
Quoted 4 lines
much less cpu cycles are required.
Setup is executed by main lcore solely, before lcores are executed hence some info passed to
SYS_perf_event_openioctl()is missing,pid
Quoted 3 lines
Quoted 1 line
(via
rte_gettid()) being an example here.OK. Thank you for the explanation. Since impossible at setup, it has
to be done at runtime.
Quoted 2 lines
@Mattias: Another good example of something that would belong in per-
thread constructors, as my
Quoted 6 lines
suggested feature creep in [1].
[1]: https://urldefense.proofpoint.com/v2/url?u=http-
3A__inbox.dpdk.org_dev_98CBD80474FA8B44BF855DF32C47DC35D87553- 40smartserver.smartshare.dk_&d=DwIFAw&c=nKjWec2b6R0mOyPaz7xtfQ&r=PZNX g
rbjdlXxVEEGYkxIxRndyEUwWU_ad5Quoted 2 lines
ce22YI6Is&m=tnygBVwOnoZDV7hItku1HtmsI8R3F6vPJdr7ON3hE5iAds96T2C9JTNcn t
6ptN4Q&s=aSAnYqgVnrgDp6yyMtGC
Quoted 24 lines
uWgJjDlgqj1wHf1nGWyHCNo&e=
Quoted 3 lines
Quoted 3 lines
lib/eal/include/rte_pmu.h line 176
Quoted 3 lines
lib/eal/include/rte_pmu.h new file +2
-0+ +if (index <0 || index >= rte_pmu->num_group_events)Optimized: if (
unlikely(index >= rte_pmu.num_group_events))Quoted 16 lines
Quoted 7 lines
Quoted 4 lines
lib/eal/common/rte_pmu.c new file +3
-0+return 0 ; + + return rte_pmu_read_userpage((struct perf_event_mmap_page*)group->mmap_pages[index]);
Using fixed size arrays instead of multiple indirections via pointers
is faster. It could be:
Quoted 7 lines
return
rte_pmu_read_userpage((struct perf_event_mmap_page *)rte_pmu.group[lcore_id].mmap_pages[index]);With our without suggested performance improvements...
Series-acked-by Morten Brørup <[email protected]>
RE: [PATCH v4 1/4] eal: add generic support for reading PMU events
Quoted 31 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Sunday, 8 January 2023 16.41Quoted 16 lines
From: Morten Brørup <[email protected]>
Sent: Thursday, January 5, 2023 11:08 PMQuoted 13 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Thursday, 5 January 2023 22.14Hi Morten,
A few comments inline.
Quoted 6 lines
From: Morten Brørup <[email protected]>
Sent: Wednesday, December 14, 2022 11:41 AMExternal Email
-------------------------------------------------------------------
--
Quoted 11 lines
Quoted 11 lines
Review
Quoted 2 lines
(quoted diff) +1
-1- +CC: Mattias, see my comment below about per-thread constructor forthis
Quoted 8 lines
Quoted 8 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Wednesday, 14 December 2022 10.39Hello Morten,
Thanks for review. Answers inline.
[...]
Quoted 32 lines
Quoted 14 lines
Quoted 14 lines
Quoted 13 lines
Quoted 10 lines
Quoted 10 lines
Review
Quoted 10 lines
(quoted diff) +10
-0+{ + int lcore_id = rte_lcore_id(); + struct rte_pmu_event_group *group; + int ret; + + if (!rte_pmu) + return 0; + + group = &rte_pmu->group[lcore_id]; + if (!group->enabled) {Optimized: if (
unlikely(!group->enabled)) {Quoted 1 line
Quoted 1 line
Compiler will optimize the branch itself correctly. Extra hint is
not
Quoted 3 lines
Quoted 1 line
required.
I haven't reviewed the output from this, so I'll take your word for
it. I suggested the
unlikely()Quoted 1 line
because I previously tested some very simple code, and it optimized
for taking the "if":
Quoted 10 lines
void testb(bool b)
{ if (!b) exit(1); exit(99); }I guess I should experiment with more realistic code, and update my
Quoted 8 lines
optimization notes!
Quoted 1 line
I think this may be too simple to draw far-reaching conclusions from it. Compiler will make the
fall-through path more likely. If I recall Intel Optimization Reference Manual has some more
info on this.
IIRC, the Intel Optimization Reference Manual discusses branch optimization for assembler, not C.
Quoted 49 lines
Lets take a different example.
int
main(int argc, char *argv[]){ int *p; p = malloc(sizeof(*p)); if (!p) return 1; *p = atoi(argv[1]); if (*p < 0) return 2; free(p); return 0; }If compiled with -O3 and disassembled I got below.
00000000000010a0<main>:10a0: f3 0f 1e fa endbr64 10a4: 55 push %rbp 10a5: bf 04 00 00 00 mov $0x4,%edi 10aa: 53 push %rbx 10ab: 48 89 f3 mov %rsi,%rbx 10ae: 48 83 ec 08 sub $0x8,%rsp 10b2: e8 d9 ff ff ff call 1090 <malloc@plt> 10b7: 48 85 c0 test %rax,%rax 10ba: 74 31 je 10ed <main+0x4d> 10bc: 48 8b 7b 08 mov 0x8(%rbx),%rdi 10c0: ba 0a 00 00 00 mov $0xa,%edx 10c5: 31 f6 xor %esi,%esi 10c7: 48 89 c5 mov %rax,%rbp 10ca: e8 b1 ff ff ff call 1080 <strtol@plt> 10cf: 49 89 c0 mov %rax,%r8 10d2: b8 02 00 00 00 mov $0x2,%eax 10d7: 45 85 c0 test %r8d,%r8d 10da: 78 0a js 10e6 <main+0x46> 10dc: 48 89 ef mov %rbp,%rdi 10df: e8 8c ff ff ff call 1070 <free@plt> 10e4: 31 c0 xor %eax,%eax 10e6: 48 83 c4 08 add $0x8,%rsp 10ea: 5b pop %rbx 10eb: 5d pop %rbp 10ec: c3 ret 10ed: b8 01 00 00 00 mov $0x1,%eax 10f2: eb f2 jmp 10e6 <main+0x46>
Quoted 4 lines
Looking at both 10ba and 10da suggests that code was laid out in a way that jumping is frowned upon. Also
potentially lest frequently executed code (at 10ed) is pushed further down the memory hence optimizing cache line usage.
In my notes, I have (ptr == NULL) marked as considered unlikely, but (int == 0) marked as considered likely. Since group->enabled is bool, I guessed the compiler would treat it like int and consider (!group->enabled) as likely.
Like in your example here, I also have (int < 0) marked as considered unlikely.
Quoted 2 lines
That said, each and every scenario needs analysis on its own.
Agree. Theory is good, validation is better. ;-)
Quoted 5 lines
Quoted 3 lines
You could add the
unlikely()for readability purposes. ;-)Sure. That won't hurt performance.
I think we are both in agreement about the intentions here, so I won't hold you back with further academic discussions at this point. I might resume the discussion with your next patch version, though. ;-)
Re: [PATCH v4 1/4] eal: add generic support for reading PMU events
In reply to Tomasz DuszynskiQuoted 229 lines · Tomasz Duszynski
Add support for programming PMU counters and reading their values in runtime bypassing kernel completely.
This is especially useful in cases where CPU cores are isolated (
nohz_full) i.e run dedicated tasks. In such cases one cannot use standard perf utility without sacrificing latency and performance.Signed-off-by Tomasz Duszynski <[email protected]>--- app/test/meson.build | 1 + app/test/test_pmu.c | 41 +++ doc/guides/prog_guide/profile_app.rst | 8 + lib/eal/common/meson.build | 3 + lib/eal/common/pmu_private.h | 41 +++ lib/eal/common/rte_pmu.c | 456 ++++++++++++++++++++++++++ lib/eal/include/meson.build | 1 + lib/eal/include/rte_pmu.h | 204 ++++++++++++ lib/eal/linux/eal.c | 4 + lib/eal/version.map | 6 + 10 files changed, 765 insertions(+)create mode 100644
app/test/test_pmu.c
create mode 100644lib/eal/common/pmu_private.h
create mode 100644lib/eal/common/rte_pmu.c
create mode 100644lib/eal/include/rte_pmu.happ/test/meson.build +1
-0@@ -143,6 +143,7 @@ test_sources = files( 'test_timer_racecond.c', 'test_timer_secondary.c', 'test_ticketlock.c', + 'test_pmu.c', 'test_trace.c', 'test_trace_register.c', 'test_trace_perf.c',app/test/test_pmu.c new file +41
-0@@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(C) 2022 Marvell International Ltd. +*/ + +#include <rte_pmu.h> + +#include "test.h" + +static int +test_pmu_read (void ) +{ +uint64_t val =0 ; +int tries =10 ; +int event =-1 ; + +while (tries--) + val +=rte_pmu_read (event); + +if (val ==0 ) +return TEST_FAILED ; + +return TEST_SUCCESS ; +} + +static struct unit_test_suite pmu_tests = { + .suite_name ="pmu autotest" , + .setup =NULL , + .teardown =NULL , + .unit_test_cases = { +TEST_CASE (test_pmu_read), +TEST_CASES_END () + } +}; + +static int +test_pmu (void ) +{ +return unit_test_suite_runner (&pmu_tests); +} + +REGISTER_TEST_COMMAND (pmu_autotest, test_pmu);doc/guides/prog_guide/profile_app.rst +8
-0@@ -7,6 +7,14 @@ Profile Your Application The following sections describe methods of profiling DPDK applications on different architectures. +Performance counter based profiling +----------------------------------- + +Majority of architectures support some sort hardware measurement unit which provides a set of +programmable counters that monitor specific events. There are different tools which can gather +that information, perf being an example here. Though in some scenarios, eg. when CPU cores are +isolated (nohz_full) and run dedicated tasks, using perf is less than ideal. In such cases one can +read specific events directly from application via ``rte_pmu_read()``. Profiling on x86 ----------------lib/eal/common/meson.build +3
-0@@ -38,6 +38,9 @@ sources += files( 'rte_service.c', 'rte_version.c', ) +if is_linux + sources += files('rte_pmu.c') +endif if is_linux or is_windows sources += files('eal_common_dynmem.c') endiflib/eal/common/pmu_private.h new file +41
-0@@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2022 Marvell +*/ + +#ifndef _PMU_PRIVATE_H_ +#define _PMU_PRIVATE_H_ + +/** +* Architecture specific PMU init callback. +* +* @return +* 0 in case of success, negative value otherwise. +*/ +int +pmu_arch_init (void ); + +/** +* Architecture specific PMU cleanup callback. +*/ +void +pmu_arch_fini (void ); + +/** +* Apply architecture specific settings to config before passing it to syscall. +*/ +void +pmu_arch_fixup_config (uint64_t config[3 ]); + +/** +* Initialize PMU tracing internals. +*/ +void +eal_pmu_init (void ); + +/** +* Cleanup PMU internals. +*/ +void +eal_pmu_fini (void ); + +#endif /* _PMU_PRIVATE_H_ */ lib/eal/common/rte_pmu.c new file +59
-0@@ -0,0 +1,456 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(C) 2022 Marvell International Ltd. +*/ + +#include <ctype.h> +#include <dirent.h> +#include <errno.h> +#include <regex.h> +#include <stdlib.h> +#include <string.h> +#include <sys/ioctl.h> +#include <sys/mman.h> +#include <sys/queue.h> +#include <sys/syscall.h> +#include <unistd.h> + +#include <rte_eal_paging.h> +#include <rte_pmu.h> +#include <rte_tailq.h> + +#include "pmu_private.h" + +#define EVENT_SOURCE_DEVICES_PATH "/sys/bus/event_source/devices" + +#ifndef GENMASK_ULL +#define GENMASK_ULL (h, l) ((~0ULL - (1ULL << (l)) + 1) & (~0ULL >> ((64 - 1 - (h))))) +#endif + +#ifndef FIELD_PREP +#define FIELD_PREP (m, v) (((uint64_t)(v) << (__builtin_ffsll(m) - 1)) & (m)) +#endif + +struct rte_pmu *rte_pmu; + +/* +* Following __rte_weak functions provide default no-op. Architectures should override them if +* necessary. +*/ + +int +__rte_weak pmu_arch_init(void) +{ +return 0 ; +} + +void +__rte_weak pmu_arch_fini(void) +{ +} + +void +__rte_weak pmu_arch_fixup_config(uint64_t config[3]) +{ +RTE_SET_USED (config); +} + +static int +get_term_format (const char *name,int *num,uint64_t *mask) +{
lib/eal/common/rte_pmu.c line 69
Quoted 10 lines
lib/eal/common/rte_pmu.c new file +10
-0+char *config =NULL ; +char path[PATH_MAX ]; +int high, low, ret; +FILE *fp; + +/* quiesce -Wmaybe-uninitialized warning */ + *num =0 ; + *mask =0 ; + +snprintf (path,sizeof (path),EVENT_SOURCE_DEVICES_PATH "/%s/format/%s" , rte_pmu->name, name)
This code might crash in case a long name is supplied, which is maybe not want you want. A trunacte and a "file not found" is probably better. I believe there is a snprintf lookalike with these properties in DPDK.
Quoted 170 lines
lib/eal/common/rte_pmu.c new file +170
-0+ fp =fopen (path,"r" ); +if (!fp) +return -errno; + + errno =0 ; + ret =fscanf (fp,"%m[^:]:%d-%d" , &config, &low, &high); +if (ret <2 ) { + ret = -ENODATA ; + goto out; + } +if (errno) { + ret = -errno; + goto out; + } + +if (ret ==2 ) + high = low; + + *mask =GENMASK_ULL (high, low); +/* Last digit should be [012]. If last digit is missing 0 is implied. */ + *num = config[strlen (config) -1 ]; + *num =isdigit (*num) ? *num -'0' :0 ; + + ret =0 ; +out: +free (config); +fclose (fp); + +return ret; +} + +static int +parse_event (char *buf,uint64_t config[3 ]) +{ +char *token, *term; +int num, ret, val; +uint64_t mask; + + config[0 ] = config[1 ] = config[2 ] =0 ; + + token =strtok (buf,"," ); +while (token) { + errno =0 ; +/* <term>=<value> */ + ret =sscanf (token,"%m[^=]=%i" , &term, &val); +if (ret <1 ) +return -ENODATA ; +if (errno) +return -errno; +if (ret ==1 ) + val =1 ; + + ret =get_term_format (term, &num, &mask); +free (term); +if (ret) +return ret; + + config[num] |=FIELD_PREP (mask, val); + token =strtok (NULL ,"," ); + } + +return 0 ; +} + +static int +get_event_config (const char *name,uint64_t config[3 ]) +{ +char path[PATH_MAX ], buf[BUFSIZ ]; +FILE *fp; +int ret; + +snprintf (path,sizeof (path),EVENT_SOURCE_DEVICES_PATH "/%s/events/%s" , rte_pmu->name, name); + fp =fopen (path,"r" ); +if (!fp) +return -errno; + + ret =fread (buf,1 ,sizeof (buf), fp); +if (ret ==0 ) { +fclose (fp); + +return -EINVAL ; + } +fclose (fp); + buf[ret] ='\0' ; + +return parse_event (buf, config); +} + +static int +do_perf_event_open (uint64_t config[3 ],int lcore_id,int group_fd) +{ +struct perf_event_attr attr = { + .size =sizeof (struct perf_event_attr ), + .type =PERF_TYPE_RAW , + .exclude_kernel =1 , + .exclude_hv =1 , + .disabled =1 , + }; + +pmu_arch_fixup_config (config); + + attr.config = config[0 ]; + attr.config1 = config[1 ]; + attr.config2 = config[2 ]; + +return syscall (SYS_perf_event_open, &attr,rte_gettid (),rte_lcore_to_cpu_id (lcore_id), + group_fd,0 ); +} + +static int +open_events (int lcore_id) +{ +struct rte_pmu_event_group *group = &rte_pmu->group[lcore_id]; +struct rte_pmu_event *event; +uint64_t config[3 ]; +int num =0 , ret; + +/* group leader gets created first, with fd = -1 */ + group->fds[0 ] =-1 ; + + TAILQ_FOREACH(event, &rte_pmu->event_list, next) { + ret =get_event_config (event->name, config); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to get %s event config\n" , event->name); +continue ; + } + + ret =do_perf_event_open (config, lcore_id, group->fds[0 ]); +if (ret ==-1 ) { +if (errno ==EOPNOTSUPP ) +RTE_LOG (ERR ,EAL ,"64 bit counters not supported\n" ); + + ret = -errno; + goto out; + } + + group->fds[event->index] = ret; + num++; + } + +return 0 ; +out: +for (--num; num >=0 ; num--) { +close (group->fds[num]); + group->fds[num] =-1 ; + } + + +return ret; +} + +static int +mmap_events (int lcore_id) +{ +struct rte_pmu_event_group *group = &rte_pmu->group[lcore_id]; +void *addr; +int ret, i; + +for (i =0 ; i < rte_pmu->num_group_events; i++) { + addr =mmap (0 ,rte_mem_page_size (),PROT_READ ,MAP_SHARED , group->fds[i],0 ); +if (addr ==MAP_FAILED ) { + ret = -errno; + goto out; + } + + group->mmap_pages[i] = addr; + } + +return 0 ; +out:
lib/eal/common/rte_pmu.c line 249
Quoted 10 lines
lib/eal/common/rte_pmu.c new file +10
-0+for (; i; i--) { +munmap (group->mmap_pages[i -1 ],rte_mem_page_size ()); + group->mmap_pages[i -1 ] =NULL ; + } + +return ret; +} + +static void +cleanup_events (int lcore_id)
lcore_id is an unsigned int.
lib/eal/common/rte_pmu.c line 255
Quoted 6 lines
lib/eal/common/rte_pmu.c new file +6
-0+{ +struct rte_pmu_event_group *group = &rte_pmu->group[lcore_id]; +int i; + +if (!group->fds) +return ;
group->fds == NULL
This coding style violating appears throughput the patch set.
Quoted 16 lines
lib/eal/common/rte_pmu.c new file +16
-0+ +if (group->fds[0 ] !=-1 ) +ioctl (group->fds[0 ],PERF_EVENT_IOC_DISABLE ,PERF_IOC_FLAG_GROUP ); + +for (i =0 ; i < rte_pmu->num_group_events; i++) { +if (group->mmap_pages[i]) { +munmap (group->mmap_pages[i],rte_mem_page_size ()); + group->mmap_pages[i] =NULL ; + } + +if (group->fds[i] !=-1 ) { +close (group->fds[i]); + group->fds[i] =-1 ; + } + } +
lib/eal/common/rte_pmu.c line 281
Quoted 10 lines
lib/eal/common/rte_pmu.c new file +10
-0+free (group->mmap_pages); +free (group->fds); + + group->mmap_pages =NULL ; + group->fds =NULL ; + group->enabled = false; +} + +int __rte_noinline +rte_pmu_enable_group (int lcore_id)
unsigned
Quoted 160 lines
lib/eal/common/rte_pmu.c new file +160
-0+{ +struct rte_pmu_event_group *group = &rte_pmu->group[lcore_id]; +int ret; + +if (rte_pmu->num_group_events ==0 ) { +RTE_LOG (DEBUG ,EAL ,"no matching PMU events\n" ); + +return 0 ; + } + + group->fds =calloc (rte_pmu->num_group_events,sizeof (*group->fds)); +if (!group->fds) { +RTE_LOG (ERR ,EAL ,"failed to alloc descriptor memory\n" ); + +return -ENOMEM ; + } + + group->mmap_pages =calloc (rte_pmu->num_group_events,sizeof (*group->mmap_pages)); +if (!group->mmap_pages) { +RTE_LOG (ERR ,EAL ,"failed to alloc userpage memory\n" ); + + ret = -ENOMEM ; + goto out; + } + + ret =open_events (lcore_id); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to open events on lcore-worker-%d\n" , lcore_id); + goto out; + } + + ret =mmap_events (lcore_id); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to map events on lcore-worker-%d\n" , lcore_id); + goto out; + } + +if (ioctl (group->fds[0 ],PERF_EVENT_IOC_ENABLE ,PERF_IOC_FLAG_GROUP ) ==-1 ) { +RTE_LOG (ERR ,EAL ,"failed to enable events on lcore-worker-%d\n" , lcore_id); + + ret = -errno; + goto out; + } + +return 0 ; + +out: +cleanup_events (lcore_id); + +return ret; +} + +static int +scan_pmus (void ) +{ +char path[PATH_MAX ]; +struct dirent *dent; +const char *name; +DIR *dirp; + + dirp =opendir (EVENT_SOURCE_DEVICES_PATH ); +if (!dirp) +return -errno; + +while ((dent =readdir (dirp))) { + name = dent->d_name; +if (name[0 ] =='.' ) +continue ; + +/* sysfs entry should either contain cpus or be a cpu */ +if (!strcmp (name,"cpu" )) +break ; + +snprintf (path,sizeof (path),EVENT_SOURCE_DEVICES_PATH "/%s/cpus" , name); +if (access (path,F_OK ) ==0 ) +break ; + } + +closedir (dirp); + +if (dent) { + rte_pmu->name =strdup (name); +if (!rte_pmu->name) +return -ENOMEM ; + } + +return rte_pmu->name ?0 : -ENODEV ; +} + +int +rte_pmu_add_event (const char *name) +{ +struct rte_pmu_event *event; +char path[PATH_MAX ]; + +snprintf (path,sizeof (path),EVENT_SOURCE_DEVICES_PATH "/%s/events/%s" , rte_pmu->name, name); +if (access (path,R_OK )) +return -ENODEV ; + + TAILQ_FOREACH(event, &rte_pmu->event_list, next) { +if (!strcmp (event->name, name)) +return event->index; +continue ; + } + + event =calloc (1 ,sizeof (*event)); +if (!event) +return -ENOMEM ; + + event->name =strdup (name); +if (!event->name) { +free (event); + +return -ENOMEM ; + } + + event->index = rte_pmu->num_group_events++; +TAILQ_INSERT_TAIL (&rte_pmu->event_list, event, next); + +RTE_LOG (DEBUG ,EAL ,"%s even added at index %d\n" , name, event->index); + +return event->index; +} + +void +eal_pmu_init (void ) +{ +int ret; + + rte_pmu =calloc (1 ,sizeof (*rte_pmu)); +if (!rte_pmu) { +RTE_LOG (ERR ,EAL ,"failed to alloc PMU\n" ); + +return ; + } + +TAILQ_INIT (&rte_pmu->event_list); + + ret =scan_pmus (); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to find core pmu\n" ); + goto out; + } + + ret =pmu_arch_init (); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to setup arch for PMU\n" ); + goto out; + } + +return ; +out: +free (rte_pmu->name); +free (rte_pmu); +} + +void +eal_pmu_fini (void ) +{ +struct rte_pmu_event *event, *tmp;
lib/eal/common/rte_pmu.c line 451
Quoted 10 lines
lib/eal/common/rte_pmu.c new file +10
-0+int lcore_id; + + RTE_TAILQ_FOREACH_SAFE(event, &rte_pmu->event_list, next, tmp) { +TAILQ_REMOVE (&rte_pmu->event_list, event, next); +free (event->name); +free (event); + } + +RTE_LCORE_FOREACH_WORKER (lcore_id) +cleanup_events (lcore_id);
Why is the main lcore left out?
Quoted 278 lines
lib/eal/common/rte_pmu.c new file +5
-0+ +pmu_arch_fini (); +free (rte_pmu->name); +free (rte_pmu); +}lib/eal/include/meson.build +1
-0@@ -36,6 +36,7 @@ headers += files( 'rte_pci_dev_features.h', 'rte_per_lcore.h', 'rte_pflock.h', + 'rte_pmu.h', 'rte_random.h', 'rte_reciprocal.h', 'rte_seqcount.h',lib/eal/include/rte_pmu.h new file +204
-0@@ -0,0 +1,204 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2022 Marvell +*/ + +#ifndef _RTE_PMU_H_ +#define _RTE_PMU_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include <rte_common.h> +#include <rte_compat.h> + +#ifdef RTE_EXEC_ENV_LINUX + +#include <linux/perf_event.h> + +#include <rte_atomic.h> +#include <rte_branch_prediction.h> +#include <rte_lcore.h> +#include <rte_log.h> + +/** +* @file +* +* PMU event tracing operations +* +* This file defines generic API and types necessary to setup PMU and +* read selected counters in runtime. +*/ + +/** +* A structure describing a group of events. +*/ +struct rte_pmu_event_group { +int *fds;/**< array of event descriptors */ +void **mmap_pages;/**< array of pointers to mmapped perf_event_attr structures */ +bool enabled;/**< true if group was enabled on particular lcore */ +}; + +/** +* A structure describing an event. +*/ +struct rte_pmu_event { +char *name;/** name of an event */ +int index;/** event index into fds/mmap_pages */ +TAILQ_ENTRY (rte_pmu_event) next;/** list entry */ +}; + +/** +* A PMU state container. +*/ +struct rte_pmu { +char *name;/** name of core PMU listed under /sys/bus/event_source/devices */ +struct rte_pmu_event_group group[RTE_MAX_LCORE ];/**< per lcore event group data */ +int num_group_events;/**< number of events in a group */ + TAILQ_HEAD(, rte_pmu_event) event_list; /**< list of matching events */ +}; + +/** Pointer to the PMU state container */ +extern struct rte_pmu *rte_pmu; + +/** Each architecture supporting PMU needs to provide its own version */ +#ifndef rte_pmu_pmc_read +#define rte_pmu_pmc_read (index) ({ 0; }) +#endif + +/** +* @internal +* +* Read PMU counter. +* +* @param pc +* Pointer to the mmapped user page. +* @return +* Counter value read from hardware. +*/ +__rte_internal +static __rte_always_inline uint64_t +rte_pmu_read_userpage (struct perf_event_mmap_page *pc) +{ +uint64_t offset, width, pmc =0 ; +uint32_t seq, index; +int tries =100 ; + +for (;;) { + seq = pc->lock; +rte_compiler_barrier (); + index = pc->index; + offset = pc->offset; + width = pc->pmc_width; + +if (likely (pc->cap_user_rdpmc && index)) { + pmc =rte_pmu_pmc_read (index -1 ); + pmc <<=64 - width; + pmc >>=64 - width; + } + +rte_compiler_barrier (); + +if (likely (pc->lock == seq)) +return pmc + offset; + +if (--tries ==0 ) { +RTE_LOG (DEBUG ,EAL ,"failed to get perf_event_mmap_page lock\n" ); +break ; + } + } + +return 0 ; +} + +/** +* @internal +* +* Enable group of events for a given lcore. +* +* @param lcore_id +* The identifier of the lcore. +* @return +* 0 in case of success, negative value otherwise. +*/ +__rte_internal +int +rte_pmu_enable_group (int lcore_id); + +/** +* @warning +* @b EXPERIMENTAL: this API may change without prior notice +* +* Add event to the group of enabled events. +* +* @param name +* Name of an event listed under /sys/bus/event_source/devices/pmu/events. +* @return +* Event index in case of success, negative value otherwise. +*/ +__rte_experimental +int +rte_pmu_add_event (const char *name); + +/** +* @warning +* @b EXPERIMENTAL: this API may change without prior notice +* +* Read hardware counter configured to count occurrences of an event. +* +* @param index +* Index of an event to be read. +* @return +* Event value read from register. In case of errors or lack of support +* 0 is returned. In other words, stream of zeros in a trace file +* indicates problem with reading particular PMU event register. +*/ +__rte_experimental +static __rte_always_inline uint64_t +rte_pmu_read (int index) +{ +int lcore_id =rte_lcore_id (); +struct rte_pmu_event_group *group; +int ret; + +if (!rte_pmu) +return 0 ; + + group = &rte_pmu->group[lcore_id]; +if (!group->enabled) { + ret =rte_pmu_enable_group (lcore_id); +if (ret) +return 0 ; + + group->enabled = true; + } + +if (index <0 || index >= rte_pmu->num_group_events) +return 0 ; + +return rte_pmu_read_userpage ((struct perf_event_mmap_page *)group->mmap_pages[index]); +} + +#else /* !RTE_EXEC_ENV_LINUX */ + +__rte_experimental +static int __rte_unused +rte_pmu_add_event(__rte_unused const char *name) +{ +return -1 ; +} + +__rte_experimental +static __rte_always_inline uint64_t +rte_pmu_read(__rte_unused int index) +{ +return 0 ; +} + +#endif /* RTE_EXEC_ENV_LINUX */ + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_PMU_H_ */ lib/eal/linux/eal.c +4
-0@@ -53,6 +53,7 @@#include "eal_options.h" #include "eal_vfio.h" #include "hotplug_mp.h" +#include "pmu_private.h" #define MEMSIZE_IF_NO_HUGE_PAGE (64ULL * 1024ULL * 1024ULL)@@ -1206,6 +1207,8 @@ rte_eal_init(int argc, char **argv)return -1 ; } +eal_pmu_init (); +if (rte_eal_tailqs_init () <0 ) {rte_eal_init_alert ("Cannot init tail queues for objects" ); rte_errno =EFAULT ;@@ -1372,6 +1375,7 @@ rte_eal_cleanup(void)eal_bus_cleanup ();rte_trace_save ();eal_trace_fini (); +eal_pmu_fini ();/* after this point, any DPDK pointers will become dangling */ rte_eal_memory_detach ();eal_mp_dev_hotplug_cleanup ();lib/eal/version.map +6
-0@@ -440,6 +440,11 @@ EXPERIMENTAL { rte_thread_detach; rte_thread_equal; rte_thread_join; + + # added in 23.03 + rte_pmu; # WINDOWS_NO_EXPORT + rte_pmu_add_event; # WINDOWS_NO_EXPORT + rte_pmu_read; # WINDOWS_NO_EXPORT }; INTERNAL {@@ -483,4 +488,5 @@ INTERNAL { rte_mem_map; rte_mem_page_size; rte_mem_unmap; + rte_pmu_enable_group; };
RE: [PATCH v4 1/4] eal: add generic support for reading PMU events
Quoted 247 lines
-----Original Message-----
From: Mattias Rönnblom <[email protected]>
Sent: Thursday, December 15, 2022 9:46 AM
To: Tomasz Duszynski <[email protected]>; [email protected]
Cc: [email protected]; Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; [email protected]
Subject: [EXT] Re: [PATCH v4 1/4] eal: add generic support for reading PMU eventsExternal Email
----------------------------------------------------------------------
Quoted 90 lines · Tomasz Duszynski
Add support for programming PMU counters and reading their values in runtime bypassing kernel completely.
This is especially useful in cases where CPU cores are isolated (
nohz_full) i.e run dedicated tasks. In such cases one cannot use standard perf utility without sacrificing latency and performance.Signed-off-by Tomasz Duszynski <[email protected]>--- app/test/meson.build | 1 + app/test/test_pmu.c | 41 +++ doc/guides/prog_guide/profile_app.rst | 8 + lib/eal/common/meson.build | 3 + lib/eal/common/pmu_private.h | 41 +++ lib/eal/common/rte_pmu.c | 456 ++++++++++++++++++++++++++ lib/eal/include/meson.build | 1 + lib/eal/include/rte_pmu.h | 204 ++++++++++++ lib/eal/linux/eal.c | 4 + lib/eal/version.map | 6 + 10 files changed, 765 insertions(+)create mode 100644
app/test/test_pmu.c
create mode 100644lib/eal/common/pmu_private.h
create mode 100644lib/eal/common/rte_pmu.c
create mode 100644lib/eal/include/rte_pmu.happ/test/meson.build index renamed from app/test/meson.build +0
-0
f34d19e3c3..93b3300309100644app/test/meson.build +1
-0@@ -143,6 +143,7 @@ test_sources = files( 'test_timer_racecond.c', 'test_timer_secondary.c', 'test_ticketlock.c', + 'test_pmu.c', 'test_trace.c', 'test_trace_register.c', 'test_trace_perf.c',app/test/test_pmu.c new file mode renamed from app/test/test_pmu.c +0
-0100644 index 0000000000..fd331af9ee
app/test/test_pmu.c new file +41
-0@@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(C) 2022 Marvell International Ltd. +*/ + +#include <rte_pmu.h> + +#include "test.h" + +static int +test_pmu_read (void ) +{ +uint64_t val =0 ; +int tries =10 ; +int event =-1 ; + +while (tries--) + val +=rte_pmu_read (event); + +if (val ==0 ) +return TEST_FAILED ; + +return TEST_SUCCESS ; +} + +static struct unit_test_suite pmu_tests = { + .suite_name ="pmu autotest" , + .setup =NULL , + .teardown =NULL , + .unit_test_cases = { +TEST_CASE (test_pmu_read), +TEST_CASES_END () + } +}; + +static int +test_pmu (void ) +{ +return unit_test_suite_runner (&pmu_tests); +} + +REGISTER_TEST_COMMAND (pmu_autotest, test_pmu);doc/guides/prog_guide/profile_app.rst +0
-0@@ -7,6 +7,14 @@ Profile Your Application The following sections describe methods of profiling DPDK applications ondoc/guides/prog_guide/profile_app.rst Profile Your Application · line 17
Quoted 10 lines
doc/guides/prog_guide/profile_app.rst +8
-0different architectures. +Performance counter based profiling +----------------------------------- + +Majority of architectures support some sort hardware measurement unit +which provides a set of programmable counters that monitor specific +events. There are different tools which can gather that information, +perf being an example here. Though in some scenarios, eg. when CPU +cores are isolated (nohz_full) and run dedicated tasks, using perf is less than ideal. In suchcases one can read specific events directly from application via ``rte_pmu_read()``.
Quoted 134 lines
doc/guides/prog_guide/profile_app.rst +0
-0Profiling on x86 ----------------lib/eal/common/meson.build +3
-0@@ -38,6 +38,9 @@ sources += files( 'rte_service.c', 'rte_version.c', ) +if is_linux + sources += files('rte_pmu.c') +endif if is_linux or is_windows sources += files('eal_common_dynmem.c') endif(quoted diff) +0
-0
b/lib/eal/common/pmu_private.hnew file mode 100644 index 0000000000..cade4245e6lib/eal/common/pmu_private.h new file +41
-0@@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2022 Marvell +*/ + +#ifndef _PMU_PRIVATE_H_ +#define _PMU_PRIVATE_H_ + +/** +* Architecture specific PMU init callback. +* +* @return +* 0 in case of success, negative value otherwise. +*/ +int +pmu_arch_init (void ); + +/** +* Architecture specific PMU cleanup callback. +*/ +void +pmu_arch_fini (void ); + +/** +* Apply architecture specific settings to config before passing it to syscall. +*/ +void +pmu_arch_fixup_config (uint64_t config[3 ]); + +/** +* Initialize PMU tracing internals. +*/ +void +eal_pmu_init (void ); + +/** +* Cleanup PMU internals. +*/ +void +eal_pmu_fini (void ); + +#endif /* _PMU_PRIVATE_H_ */ lib/eal/common/rte_pmu.c new renamed from lib/eal/common/rte_pmu.c +0
-0file mode 100644 index 0000000000..049fe19fe3
lib/eal/common/rte_pmu.c new file +65
-0@@ -0,0 +1,456 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(C) 2022 Marvell International Ltd. +*/ + +#include <ctype.h> +#include <dirent.h> +#include <errno.h> +#include <regex.h> +#include <stdlib.h> +#include <string.h> +#include <sys/ioctl.h> +#include <sys/mman.h> +#include <sys/queue.h> +#include <sys/syscall.h> +#include <unistd.h> + +#include <rte_eal_paging.h> +#include <rte_pmu.h> +#include <rte_tailq.h> + +#include "pmu_private.h" + +#define EVENT_SOURCE_DEVICES_PATH "/sys/bus/event_source/devices" + +#ifndef GENMASK_ULL +#define GENMASK_ULL (h, l) ((~0ULL - (1ULL << (l)) + 1) & (~0ULL >> +((64 - 1 - (h))))) #endif + +#ifndef FIELD_PREP +#define FIELD_PREP (m, v) (((uint64_t)(v) << (__builtin_ffsll(m) - 1)) +& (m)) #endif + +struct rte_pmu *rte_pmu; + +/* +* Following __rte_weak functions provide default no-op. +Architectures should override them if +* necessary. +*/ + +int +__rte_weak pmu_arch_init(void) +{ +return 0 ; +} + +void +__rte_weak pmu_arch_fini(void) +{ +} + +void +__rte_weak pmu_arch_fixup_config(uint64_t config[3]) { +RTE_SET_USED (config); +} + +static int +get_term_format (const char *name,int *num,uint64_t *mask) { +char *config =NULL ; +char path[PATH_MAX ]; +int high, low, ret; +FILE *fp; + +/* quiesce -Wmaybe-uninitialized warning */ + *num =0 ;
Quoted 9 lines
lib/eal/common/rte_pmu.c
Quoted 4 lines
lib/eal/common/rte_pmu.c new file +4
-0+ *mask =0 ; + +snprintf (path,sizeof (path),EVENT_SOURCE_DEVICES_PATH +"/%s/format/%s" , rte_pmu->name, name)This code might crash in case a long name is supplied, which is maybe not want you want. A trunacte and a "file not found" is probably better.
I believe there is a snprintf lookalike with these properties in DPDK.
In scenario, which is pretty unlikely especially because sysfs files have sane names, where 'path' cannot accommodate the whole string there will be NUL implicitly appended by snprintf. Hence fopen will fail. Not sure how this may go wrong.
Quoted 172 lines
Quoted 60 lines
lib/eal/common/rte_pmu.c new file +60
-0+ fp =fopen (path,"r" ); +if (!fp) +return -errno; + + errno =0 ; + ret =fscanf (fp,"%m[^:]:%d-%d" , &config, &low, &high); +if (ret <2 ) { + ret = -ENODATA ; + goto out; + } +if (errno) { + ret = -errno; + goto out; + } + +if (ret ==2 ) + high = low; + + *mask =GENMASK_ULL (high, low); +/* Last digit should be [012]. If last digit is missing 0 is implied. */ + *num = config[strlen (config) -1 ]; + *num =isdigit (*num) ? *num -'0' :0 ; + + ret =0 ; +out: +free (config); +fclose (fp); + +return ret; +} + +static int +parse_event (char *buf,uint64_t config[3 ]) { +char *token, *term; +int num, ret, val; +uint64_t mask; + + config[0 ] = config[1 ] = config[2 ] =0 ; + + token =strtok (buf,"," ); +while (token) { + errno =0 ; +/* <term>=<value> */ + ret =sscanf (token,"%m[^=]=%i" , &term, &val); +if (ret <1 ) +return -ENODATA ; +if (errno) +return -errno; +if (ret ==1 ) + val =1 ; + + ret =get_term_format (term, &num, &mask); +free (term); +if (ret) +return ret; + + config[num] |=FIELD_PREP (mask, val); + token =strtok (NULL ,"," ); + } +lib/eal/common/rte_pmu.c
Quoted 10 lines
lib/eal/common/rte_pmu.c new file +10
-0+return 0 ; +} + +static int +get_event_config (const char *name,uint64_t config[3 ]) { +char path[PATH_MAX ], buf[BUFSIZ ]; +FILE *fp; +int ret; + + snprintf(path, sizeof(path), EVENT_SOURCE_DEVICES_PATH "/%s/events/%s", rte_pmu->name,name);
Quoted 101 lines
lib/eal/common/rte_pmu.c new file +101
-0+ fp =fopen (path,"r" ); +if (!fp) +return -errno; + + ret =fread (buf,1 ,sizeof (buf), fp); +if (ret ==0 ) { +fclose (fp); + +return -EINVAL ; + } +fclose (fp); + buf[ret] ='\0' ; + +return parse_event (buf, config); +} + +static int +do_perf_event_open (uint64_t config[3 ],int lcore_id,int group_fd) { +struct perf_event_attr attr = { + .size =sizeof (struct perf_event_attr ), + .type =PERF_TYPE_RAW , + .exclude_kernel =1 , + .exclude_hv =1 , + .disabled =1 , + }; + +pmu_arch_fixup_config (config); + + attr.config = config[0 ]; + attr.config1 = config[1 ]; + attr.config2 = config[2 ]; + +return syscall (SYS_perf_event_open, &attr,rte_gettid (),rte_lcore_to_cpu_id (lcore_id), + group_fd,0 ); +} + +static int +open_events (int lcore_id) +{ +struct rte_pmu_event_group *group = &rte_pmu->group[lcore_id]; +struct rte_pmu_event *event; +uint64_t config[3 ]; +int num =0 , ret; + +/* group leader gets created first, with fd = -1 */ + group->fds[0 ] =-1 ; + + TAILQ_FOREACH(event, &rte_pmu->event_list, next) { + ret =get_event_config (event->name, config); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to get %s event config\n" , event->name); +continue ; + } + + ret =do_perf_event_open (config, lcore_id, group->fds[0 ]); +if (ret ==-1 ) { +if (errno ==EOPNOTSUPP ) +RTE_LOG (ERR ,EAL ,"64 bit counters not supported\n" ); + + ret = -errno; + goto out; + } + + group->fds[event->index] = ret; + num++; + } + +return 0 ; +out: +for (--num; num >=0 ; num--) { +close (group->fds[num]); + group->fds[num] =-1 ; + } + + +return ret; +} + +static int +mmap_events (int lcore_id) +{ +struct rte_pmu_event_group *group = &rte_pmu->group[lcore_id]; +void *addr; +int ret, i; + +for (i =0 ; i < rte_pmu->num_group_events; i++) { + addr =mmap (0 ,rte_mem_page_size (),PROT_READ ,MAP_SHARED , group->fds[i],0 ); +if (addr ==MAP_FAILED ) { + ret = -errno; + goto out; + } + + group->mmap_pages[i] = addr; + } + +return 0 ; +out: +for (; i; i--) { +munmap (group->mmap_pages[i -1 ],rte_mem_page_size ()); + group->mmap_pages[i -1 ] =NULL ; + }
Quoted 9 lines
lib/eal/common/rte_pmu.c line 249
Quoted 6 lines
lib/eal/common/rte_pmu.c new file +6
-0+ +return ret; +} + +static void +cleanup_events (int lcore_id)
lcore_idis an unsigned int.
True, unsigned seems to be more common.
Quoted 11 lines
lib/eal/common/rte_pmu.c line 255
Quoted 6 lines
lib/eal/common/rte_pmu.c new file +6
-0+{ +struct rte_pmu_event_group *group = &rte_pmu->group[lcore_id]; +int i; + +if (!group->fds) +return ;
group->fds== NULLThis coding style violating appears throughput the patch set.
Good point.
Quoted 196 lines
Quoted 16 lines
lib/eal/common/rte_pmu.c new file +16
-0+ +if (group->fds[0 ] !=-1 ) +ioctl (group->fds[0 ],PERF_EVENT_IOC_DISABLE ,PERF_IOC_FLAG_GROUP ); + +for (i =0 ; i < rte_pmu->num_group_events; i++) { +if (group->mmap_pages[i]) { +munmap (group->mmap_pages[i],rte_mem_page_size ()); + group->mmap_pages[i] =NULL ; + } + +if (group->fds[i] !=-1 ) { +close (group->fds[i]); + group->fds[i] =-1 ; + } + } +lib/eal/common/rte_pmu.c line 281
Quoted 10 lines
lib/eal/common/rte_pmu.c new file +10
-0+free (group->mmap_pages); +free (group->fds); + + group->mmap_pages =NULL ; + group->fds =NULL ; + group->enabled = false; +} + +int __rte_noinline +rte_pmu_enable_group (int lcore_id)unsigned
Quoted 87 lines
lib/eal/common/rte_pmu.c new file +87
-0+{ +struct rte_pmu_event_group *group = &rte_pmu->group[lcore_id]; +int ret; + +if (rte_pmu->num_group_events ==0 ) { +RTE_LOG (DEBUG ,EAL ,"no matching PMU events\n" ); + +return 0 ; + } + + group->fds =calloc (rte_pmu->num_group_events,sizeof (*group->fds)); +if (!group->fds) { +RTE_LOG (ERR ,EAL ,"failed to alloc descriptor memory\n" ); + +return -ENOMEM ; + } + + group->mmap_pages =calloc (rte_pmu->num_group_events,sizeof (*group->mmap_pages)); +if (!group->mmap_pages) { +RTE_LOG (ERR ,EAL ,"failed to alloc userpage memory\n" ); + + ret = -ENOMEM ; + goto out; + } + + ret =open_events (lcore_id); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to open events on lcore-worker-%d\n" , lcore_id); + goto out; + } + + ret =mmap_events (lcore_id); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to map events on lcore-worker-%d\n" , lcore_id); + goto out; + } + +if (ioctl (group->fds[0 ],PERF_EVENT_IOC_ENABLE ,PERF_IOC_FLAG_GROUP ) ==-1 ) { +RTE_LOG (ERR ,EAL ,"failed to enable events on lcore-worker-%d\n" , +lcore_id); + + ret = -errno; + goto out; + } + +return 0 ; + +out: +cleanup_events (lcore_id); + +return ret; +} + +static int +scan_pmus (void ) +{ +char path[PATH_MAX ]; +struct dirent *dent; +const char *name; +DIR *dirp; + + dirp =opendir (EVENT_SOURCE_DEVICES_PATH ); +if (!dirp) +return -errno; + +while ((dent =readdir (dirp))) { + name = dent->d_name; +if (name[0 ] =='.' ) +continue ; + +/* sysfs entry should either contain cpus or be a cpu */ +if (!strcmp (name,"cpu" )) +break ; + +snprintf (path,sizeof (path),EVENT_SOURCE_DEVICES_PATH "/%s/cpus" , name); +if (access (path,F_OK ) ==0 ) +break ; + } + +closedir (dirp); + +if (dent) { + rte_pmu->name =strdup (name); +if (!rte_pmu->name) +return -ENOMEM ; + } +lib/eal/common/rte_pmu.c
Quoted 10 lines
lib/eal/common/rte_pmu.c new file +10
-0+return rte_pmu->name ?0 : -ENODEV ; +} + +int +rte_pmu_add_event (const char *name) +{ +struct rte_pmu_event *event; +char path[PATH_MAX ]; + + snprintf(path, sizeof(path), EVENT_SOURCE_DEVICES_PATH "/%s/events/%s", rte_pmu->name,name);
Quoted 69 lines
lib/eal/common/rte_pmu.c new file +69
-0+if (access (path,R_OK )) +return -ENODEV ; + + TAILQ_FOREACH(event, &rte_pmu->event_list, next) { +if (!strcmp (event->name, name)) +return event->index; +continue ; + } + + event =calloc (1 ,sizeof (*event)); +if (!event) +return -ENOMEM ; + + event->name =strdup (name); +if (!event->name) { +free (event); + +return -ENOMEM ; + } + + event->index = rte_pmu->num_group_events++; +TAILQ_INSERT_TAIL (&rte_pmu->event_list, event, next); + +RTE_LOG (DEBUG ,EAL ,"%s even added at index %d\n" , name, +event->index); + +return event->index; +} + +void +eal_pmu_init (void ) +{ +int ret; + + rte_pmu =calloc (1 ,sizeof (*rte_pmu)); +if (!rte_pmu) { +RTE_LOG (ERR ,EAL ,"failed to alloc PMU\n" ); + +return ; + } + +TAILQ_INIT (&rte_pmu->event_list); + + ret =scan_pmus (); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to find core pmu\n" ); + goto out; + } + + ret =pmu_arch_init (); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to setup arch for PMU\n" ); + goto out; + } + +return ; +out: +free (rte_pmu->name); +free (rte_pmu); +} + +void +eal_pmu_fini (void ) +{ +struct rte_pmu_event *event, *tmp; +int lcore_id; + + RTE_TAILQ_FOREACH_SAFE(event, &rte_pmu->event_list, next, tmp) { +TAILQ_REMOVE (&rte_pmu->event_list, event, next);
Quoted 9 lines
lib/eal/common/rte_pmu.c line 451
Quoted 6 lines
lib/eal/common/rte_pmu.c new file +6
-0+free (event->name); +free (event); + } + +RTE_LCORE_FOREACH_WORKER (lcore_id) +cleanup_events (lcore_id);Why is the main lcore left out?
Main lcore was omitted because it's pretty uncommon for it to do a heavy-lifting so usefulness of reading counters is questionable. It can be added for completeness though.
Do you have any specific use case on your mind?
Quoted 273 lines
Quoted 273 lines
lib/eal/common/rte_pmu.c new file +5
-0+ +pmu_arch_fini (); +free (rte_pmu->name); +free (rte_pmu); +}lib/eal/include/meson.build +1
-0@@ -36,6 +36,7 @@ headers += files( 'rte_pci_dev_features.h', 'rte_per_lcore.h', 'rte_pflock.h', + 'rte_pmu.h', 'rte_random.h', 'rte_reciprocal.h', 'rte_seqcount.h',lib/eal/include/rte_pmu.h new renamed from lib/eal/include/rte_pmu.h +0
-0file mode 100644 index 0000000000..e4b4f6b052
lib/eal/include/rte_pmu.h new file +200
-0@@ -0,0 +1,204 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2022 Marvell + */ + +#ifndef _RTE_PMU_H_ +#define _RTE_PMU_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include <rte_common.h> +#include <rte_compat.h> + +#ifdef RTE_EXEC_ENV_LINUX + +#include <linux/perf_event.h> + +#include <rte_atomic.h> +#include <rte_branch_prediction.h> +#include <rte_lcore.h> +#include <rte_log.h> + +/** + * @file + * + * PMU event tracing operations + * + * This file defines generic API and types necessary to setup PMU and + * read selected counters in runtime. + */ + +/** + * A structure describing a group of events. + */ +struct rte_pmu_event_group { + int *fds; /**< array of event descriptors */ + void **mmap_pages; /**< array of pointers to mmapped perf_event_attr structures */ + bool enabled; /**< true if group was enabled on particular lcore */ +}; + +/** + * A structure describing an event. + */ +struct rte_pmu_event { + char *name; /** name of an event */ + int index; /** event index into fds/mmap_pages */ + TAILQ_ENTRY(rte_pmu_event) next; /** list entry */ }; + +/** + * A PMU state container. + */ +struct rte_pmu { + char *name; /** name of core PMU listed under /sys/bus/event_source/devices */ + struct rte_pmu_event_group group[RTE_MAX_LCORE]; /**< per lcore event group data */ + int num_group_events; /**< number of events in a group */ + TAILQ_HEAD(, rte_pmu_event) event_list; /**< list of matching events +*/ }; + +/** Pointer to the PMU state container */ extern struct rte_pmu +*rte_pmu; + +/** Each architecture supporting PMU needs to provide its own version +*/ #ifndef rte_pmu_pmc_read #define rte_pmu_pmc_read(index) ({ 0; }) +#endif + +/** + * @internal + * + * Read PMU counter. + * + * @param pc + * Pointer to the mmapped user page. + * @return + * Counter value read from hardware. + */ +__rte_internal +static __rte_always_inline uint64_t +rte_pmu_read_userpage(struct perf_event_mmap_page *pc) { + uint64_t offset, width, pmc = 0; + uint32_t seq, index; + int tries = 100; + + for (;;) { + seq = pc->lock; + rte_compiler_barrier(); + index = pc->index; + offset = pc->offset; + width = pc->pmc_width; + + if (likely(pc->cap_user_rdpmc && index)) { + pmc = rte_pmu_pmc_read(index - 1); + pmc <<= 64 - width; + pmc >>= 64 - width; + } + + rte_compiler_barrier(); + + if (likely(pc->lock == seq)) + return pmc + offset; + + if (--tries == 0) { + RTE_LOG(DEBUG, EAL, "failed to get perf_event_mmap_page lock\n"); + break; + } + } + + return 0; +} + +/** + * @internal + * + * Enable group of events for a given lcore. + * + * @param lcore_id + * The identifier of the lcore. + * @return + * 0 in case of success, negative value otherwise. + */ +__rte_internal +int +rte_pmu_enable_group(int lcore_id); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Add event to the group of enabled events. + * + * @param name + * Name of an event listed under /sys/bus/event_source/devices/pmu/events. + * @return + * Event index in case of success, negative value otherwise. + */ +__rte_experimental +int +rte_pmu_add_event(const char *name); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Read hardware counter configured to count occurrences of an event. + * + * @param index + * Index of an event to be read. + * @return + * Event value read from register. In case of errors or lack of support + * 0 is returned. In other words, stream of zeros in a trace file + * indicates problem with reading particular PMU event register. + */ +__rte_experimental +static __rte_always_inline uint64_t +rte_pmu_read(int index) +{ + int lcore_id = rte_lcore_id(); + struct rte_pmu_event_group *group; + int ret; + + if (!rte_pmu) + return 0; + + group = &rte_pmu->group[lcore_id]; + if (!group->enabled) { + ret = rte_pmu_enable_group(lcore_id); + if (ret) + return 0; + + group->enabled = true; + } + + if (index < 0 || index >= rte_pmu->num_group_events) + return 0; + + return rte_pmu_read_userpage((struct perf_event_mmap_page +*)group->mmap_pages[index]); } + +#else /* !RTE_EXEC_ENV_LINUX */ + +__rte_experimental +static int __rte_unused +rte_pmu_add_event(__rte_unused const char *name) { + return -1; +} + +__rte_experimental +static __rte_always_inline uint64_t +rte_pmu_read(__rte_unused int index) +{ + return 0; +} + +#endif /* RTE_EXEC_ENV_LINUX */ + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_PMU_H_ */lib/eal/linux/eal.c index renamed from lib/eal/linux/eal.c +0
-0
8c118d0d9f..751a13b597100644lib/eal/linux/eal.c +4
-0@@ -53,6 +53,7 @@#include "eal_options.h" #include "eal_vfio.h" #include "hotplug_mp.h" +#include "pmu_private.h" #define MEMSIZE_IF_NO_HUGE_PAGE (64ULL * 1024ULL * 1024ULL)@@ -1206,6 +1207,8 @@ rte_eal_init(int argc, char **argv)return -1 ; } +eal_pmu_init (); +if (rte_eal_tailqs_init () <0 ) {rte_eal_init_alert ("Cannot init tail queues for objects" ); rte_errno =EFAULT ;@@ -1372,6 +1375,7 @@ rte_eal_cleanup(void)eal_bus_cleanup ();rte_trace_save ();eal_trace_fini (); +eal_pmu_fini ();/* after this point, any DPDK pointers will become dangling */ rte_eal_memory_detach ();eal_mp_dev_hotplug_cleanup ();lib/eal/version.map index renamed from lib/eal/version.map +0
-0
7ad12a7dc9..9225f46f67100644lib/eal/version.map +6
-0@@ -440,6 +440,11 @@ EXPERIMENTAL { rte_thread_detach; rte_thread_equal; rte_thread_join; + + # added in 23.03 + rte_pmu; # WINDOWS_NO_EXPORT + rte_pmu_add_event; # WINDOWS_NO_EXPORT + rte_pmu_read; # WINDOWS_NO_EXPORT }; INTERNAL {@@ -483,4 +488,5 @@ INTERNAL { rte_mem_map; rte_mem_page_size; rte_mem_unmap; + rte_pmu_enable_group; };
RE: [PATCH v4 1/4] eal: add generic support for reading PMU events
In reply to Tomasz DuszynskiQuoted 32 lines
-----Original Message-----
From: Tomasz Duszynski <[email protected]>
Sent: Tuesday, December 13, 2022 6:44 PM
To: [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Tomasz Duszynski <[email protected]>
Subject: [PATCH v4 1/4] eal: add generic support for reading PMU eventsAdd support for programming PMU counters and reading their values in runtime bypassing kernel completely.
This is especially useful in cases where CPU cores are isolated (
nohz_full) i.e run dedicated tasks. In such cases one cannot use standard perf utility without sacrificing latency and performance.Signed-off-by Tomasz Duszynski <[email protected]>--- app/test/meson.build | 1 + app/test/test_pmu.c | 41 +++ doc/guides/prog_guide/profile_app.rst | 8 + lib/eal/common/meson.build | 3 + lib/eal/common/pmu_private.h | 41 +++ lib/eal/common/rte_pmu.c | 456 ++++++++++++++++++++++++++ lib/eal/include/meson.build | 1 + lib/eal/include/rte_pmu.h | 204 ++++++++++++ lib/eal/linux/eal.c | 4 + lib/eal/version.map | 6 + 10 files changed, 765 insertions(+) create mode 100644 app/test/test_pmu.c create mode 100644 lib/eal/common/pmu_private.h create mode 100644
lib/eal/common/rte_pmu.ccreate mode 100644lib/eal/include/rte_pmu.h
<snip>
Quoted 168 lines
lib/eal/common/rte_pmu.c new file +68
-0@@ -0,0 +1,456 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(C) 2022 Marvell International Ltd. +*/ + +#include <ctype.h> +#include <dirent.h> +#include <errno.h> +#include <regex.h> +#include <stdlib.h> +#include <string.h> +#include <sys/ioctl.h> +#include <sys/mman.h> +#include <sys/queue.h> +#include <sys/syscall.h> +#include <unistd.h> + +#include <rte_eal_paging.h> +#include <rte_pmu.h> +#include <rte_tailq.h> + +#include "pmu_private.h" + +#define EVENT_SOURCE_DEVICES_PATH "/sys/bus/event_source/devices" + +#ifndef GENMASK_ULL +#define GENMASK_ULL (h, l) ((~0ULL - (1ULL << (l)) + 1) & (~0ULL >> ((64 +- 1 - (h))))) #endif + +#ifndef FIELD_PREP +#define FIELD_PREP (m, v) (((uint64_t)(v) << (__builtin_ffsll(m) - 1)) & +(m)) #endif + +struct rte_pmu *rte_pmu; + +/* +* Following __rte_weak functions provide default no-op. Architectures +should override them if +* necessary. +*/ + +int +__rte_weak pmu_arch_init(void) +{ +return 0 ; +} + +void +__rte_weak pmu_arch_fini(void) +{ +} + +void +__rte_weak pmu_arch_fixup_config(uint64_t config[3]) { +RTE_SET_USED (config); +} + +static int +get_term_format (const char *name,int *num,uint64_t *mask) { +char *config =NULL ; +char path[PATH_MAX ]; +int high, low, ret; +FILE *fp; + +/* quiesce -Wmaybe-uninitialized warning */ + *num =0 ; + *mask =0 ; + + snprintf(path, sizeof(path), EVENT_SOURCE_DEVICES_PATH "/%s/format/%s", rte_pmu->name,name);
lib/eal/common/rte_pmu.c new file +70
-0+ fp =fopen (path,"r" ); +if (!fp) +return -errno; + + errno =0 ; + ret =fscanf (fp,"%m[^:]:%d-%d" , &config, &low, &high); +if (ret <2 ) { + ret = -ENODATA ; + goto out; + } +if (errno) { + ret = -errno; + goto out; + } + +if (ret ==2 ) + high = low; + + *mask =GENMASK_ULL (high, low); +/* Last digit should be [012]. If last digit is missing 0 is implied. */ + *num = config[strlen (config) -1 ]; + *num =isdigit (*num) ? *num -'0' :0 ; + + ret =0 ; +out: +free (config); +fclose (fp); + +return ret; +} + +static int +parse_event (char *buf,uint64_t config[3 ]) { +char *token, *term; +int num, ret, val; +uint64_t mask; + + config[0 ] = config[1 ] = config[2 ] =0 ; + + token =strtok (buf,"," ); +while (token) { + errno =0 ; +/* <term>=<value> */ + ret =sscanf (token,"%m[^=]=%i" , &term, &val); +if (ret <1 ) +return -ENODATA ; +if (errno) +return -errno; +if (ret ==1 ) + val =1 ; + + ret =get_term_format (term, &num, &mask); +free (term); +if (ret) +return ret; + + config[num] |=FIELD_PREP (mask, val); + token =strtok (NULL ,"," ); + } + +return 0 ; +} + +static int +get_event_config (const char *name,uint64_t config[3 ]) { +char path[PATH_MAX ], buf[BUFSIZ ]; +FILE *fp; +int ret; + + snprintf(path, sizeof(path), EVENT_SOURCE_DEVICES_PATH "/%s/events/%s", rte_pmu->name,name);
lib/eal/common/rte_pmu.c new file +23
-0+ fp =fopen (path,"r" ); +if (!fp) +return -errno; + + ret =fread (buf,1 ,sizeof (buf), fp); +if (ret ==0 ) { +fclose (fp); + +return -EINVAL ; + } +fclose (fp); + buf[ret] ='\0' ; + +return parse_event (buf, config); +} + +static int +do_perf_event_open (uint64_t config[3 ],int lcore_id,int group_fd) { +struct perf_event_attr attr = { + .size =sizeof (struct perf_event_attr ), + .type =PERF_TYPE_RAW , + .exclude_kernel =1 , + .exclude_hv =1 ,
lib/eal/common/rte_pmu.c
Quoted 10 lines
lib/eal/common/rte_pmu.c new file +10
-0+ .disabled = 1, + }; + +pmu_arch_fixup_config (config); + + attr.config = config[0 ]; + attr.config1 = config[1 ]; + attr.config2 = config[2 ]; + + return syscall(SYS_perf_event_open, &attr, rte_gettid(),
Looks like using '0' instead of rte_gettid() takes the same effect. A small optimization.
Quoted 204 lines
rte_lcore_to_cpu_id(lcore_id),lib/eal/common/rte_pmu.c new file +203
-0+ group_fd, 0); +} + +static int +open_events (int lcore_id) +{ +struct rte_pmu_event_group *group = &rte_pmu->group[lcore_id]; +struct rte_pmu_event *event; +uint64_t config[3 ]; +int num =0 , ret; + +/* group leader gets created first, with fd = -1 */ + group->fds[0 ] =-1 ; + + TAILQ_FOREACH(event, &rte_pmu->event_list, next) { + ret =get_event_config (event->name, config); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to get %s event config\n" , event->name); +continue ; + } + + ret =do_perf_event_open (config, lcore_id, group->fds[0 ]); +if (ret ==-1 ) { +if (errno ==EOPNOTSUPP ) +RTE_LOG (ERR ,EAL ,"64 bit counters not supported\n" ); + + ret = -errno; + goto out; + } + + group->fds[event->index] = ret; + num++; + } + +return 0 ; +out: +for (--num; num >=0 ; num--) { +close (group->fds[num]); + group->fds[num] =-1 ; + } + + +return ret; +} + +static int +mmap_events (int lcore_id) +{ +struct rte_pmu_event_group *group = &rte_pmu->group[lcore_id]; +void *addr; +int ret, i; + +for (i =0 ; i < rte_pmu->num_group_events; i++) { + addr =mmap (0 ,rte_mem_page_size (),PROT_READ ,MAP_SHARED , group->fds[i],0 ); +if (addr ==MAP_FAILED ) { + ret = -errno; + goto out; + } + + group->mmap_pages[i] = addr; + } + +return 0 ; +out: +for (; i; i--) { +munmap (group->mmap_pages[i -1 ],rte_mem_page_size ()); + group->mmap_pages[i -1 ] =NULL ; + } + +return ret; +} + +static void +cleanup_events (int lcore_id) +{ +struct rte_pmu_event_group *group = &rte_pmu->group[lcore_id]; +int i; + +if (!group->fds) +return ; + +if (group->fds[0 ] !=-1 ) +ioctl (group->fds[0 ],PERF_EVENT_IOC_DISABLE ,PERF_IOC_FLAG_GROUP ); + +for (i =0 ; i < rte_pmu->num_group_events; i++) { +if (group->mmap_pages[i]) { +munmap (group->mmap_pages[i],rte_mem_page_size ()); + group->mmap_pages[i] =NULL ; + } + +if (group->fds[i] !=-1 ) { +close (group->fds[i]); + group->fds[i] =-1 ; + } + } + +free (group->mmap_pages); +free (group->fds); + + group->mmap_pages =NULL ; + group->fds =NULL ; + group->enabled = false; +} + +int __rte_noinline +rte_pmu_enable_group (int lcore_id) +{ +struct rte_pmu_event_group *group = &rte_pmu->group[lcore_id]; +int ret; + +if (rte_pmu->num_group_events ==0 ) { +RTE_LOG (DEBUG ,EAL ,"no matching PMU events\n" ); + +return 0 ; + } + + group->fds =calloc (rte_pmu->num_group_events,sizeof (*group->fds)); +if (!group->fds) { +RTE_LOG (ERR ,EAL ,"failed to alloc descriptor memory\n" ); + +return -ENOMEM ; + } + + group->mmap_pages =calloc (rte_pmu->num_group_events,sizeof (*group->mmap_pages)); +if (!group->mmap_pages) { +RTE_LOG (ERR ,EAL ,"failed to alloc userpage memory\n" ); + + ret = -ENOMEM ; + goto out; + } + + ret =open_events (lcore_id); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to open events on lcore-worker-%d\n" , lcore_id); + goto out; + } + + ret =mmap_events (lcore_id); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to map events on lcore-worker-%d\n" , lcore_id); + goto out; + } + +if (ioctl (group->fds[0 ],PERF_EVENT_IOC_ENABLE ,PERF_IOC_FLAG_GROUP ) ==-1 ) { +RTE_LOG (ERR ,EAL ,"failed to enable events on lcore-worker-%d\n" , +lcore_id); + + ret = -errno; + goto out; + } + +return 0 ; + +out: +cleanup_events (lcore_id); + +return ret; +} + +static int +scan_pmus (void ) +{ +char path[PATH_MAX ]; +struct dirent *dent; +const char *name; +DIR *dirp; + + dirp =opendir (EVENT_SOURCE_DEVICES_PATH ); +if (!dirp) +return -errno; + +while ((dent =readdir (dirp))) { + name = dent->d_name; +if (name[0 ] =='.' ) +continue ; + +/* sysfs entry should either contain cpus or be a cpu */ +if (!strcmp (name,"cpu" )) +break ; + +snprintf (path,sizeof (path),EVENT_SOURCE_DEVICES_PATH "/%s/cpus" , name); +if (access (path,F_OK ) ==0 ) +break ; + } + +closedir (dirp); + +if (dent) { + rte_pmu->name =strdup (name); +if (!rte_pmu->name) +return -ENOMEM ; + } + +return rte_pmu->name ?0 : -ENODEV ; +} + +int +rte_pmu_add_event (const char *name) +{ +struct rte_pmu_event *event; +char path[PATH_MAX ]; + + snprintf(path, sizeof(path), EVENT_SOURCE_DEVICES_PATH "/%s/events/%s", rte_pmu->name,
Quoted 1 line
name);
Better to check if rte_pmu is available.
See below.
Quoted 49 lines
lib/eal/common/rte_pmu.c new file +49
-0+if (access (path,R_OK )) +return -ENODEV ; + + TAILQ_FOREACH(event, &rte_pmu->event_list, next) { +if (!strcmp (event->name, name)) +return event->index; +continue ; + } + + event =calloc (1 ,sizeof (*event)); +if (!event) +return -ENOMEM ; + + event->name =strdup (name); +if (!event->name) { +free (event); + +return -ENOMEM ; + } + + event->index = rte_pmu->num_group_events++; +TAILQ_INSERT_TAIL (&rte_pmu->event_list, event, next); + +RTE_LOG (DEBUG ,EAL ,"%s even added at index %d\n" , name, +event->index); + +return event->index; +} + +void +eal_pmu_init (void ) +{ +int ret; + + rte_pmu =calloc (1 ,sizeof (*rte_pmu)); +if (!rte_pmu) { +RTE_LOG (ERR ,EAL ,"failed to alloc PMU\n" ); + +return ; + } + +TAILQ_INIT (&rte_pmu->event_list); + + ret =scan_pmus (); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to find core pmu\n" ); + goto out; + } +
lib/eal/common/rte_pmu.c line 435
Quoted 10 lines
lib/eal/common/rte_pmu.c new file +10
-0+ ret =pmu_arch_init (); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to setup arch for PMU\n" ); + goto out; + } + +return ; +out: +free (rte_pmu->name); +free (rte_pmu);
Set rte_pmu to NULL to prevent unintentional use?
lib/eal/common/rte_pmu.c line 444
Quoted 9 lines
lib/eal/common/rte_pmu.c new file +9
-0+} + +void +eal_pmu_fini (void ) +{ +struct rte_pmu_event *event, *tmp; +int lcore_id; + + RTE_TAILQ_FOREACH_SAFE(event, &rte_pmu->event_list, next, tmp) {
rte_pmu can be unavailable if init fails. Better to check before accessing.
Quoted 12 lines
lib/eal/common/rte_pmu.c new file +12
-0+TAILQ_REMOVE (&rte_pmu->event_list, event, next); +free (event->name); +free (event); + } + +RTE_LCORE_FOREACH_WORKER (lcore_id) +cleanup_events (lcore_id); + +pmu_arch_fini (); +free (rte_pmu->name); +free (rte_pmu); +}
<snip>
RE: [PATCH v4 1/4] eal: add generic support for reading PMU events
Hi Ruifeng,
Quoted 493 lines
-----Original Message-----
From: Ruifeng Wang <[email protected]>
Sent: Monday, January 9, 2023 8:37 AM
To: Tomasz Duszynski <[email protected]>; [email protected]
Cc: [email protected]; Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; [email protected]; nd <[email protected]>
Subject: [EXT] RE: [PATCH v4 1/4] eal: add generic support for reading PMU eventsExternal Email
----------------------------------------------------------------------
Quoted 33 lines
-----Original Message-----
From: Tomasz Duszynski <[email protected]>
Sent: Tuesday, December 13, 2022 6:44 PM
To: [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Tomasz Duszynski <[email protected]>
Subject: [PATCH v4 1/4] eal: add generic support for reading PMU eventsAdd support for programming PMU counters and reading their values in runtime bypassing kernel completely.
This is especially useful in cases where CPU cores are isolated (
nohz_full) i.e run dedicated tasks. In such cases one cannot use standard perf utility without sacrificing latency and performance.Signed-off-by Tomasz Duszynski <[email protected]>--- app/test/meson.build | 1 + app/test/test_pmu.c | 41 +++ doc/guides/prog_guide/profile_app.rst | 8 + lib/eal/common/meson.build | 3 + lib/eal/common/pmu_private.h | 41 +++ lib/eal/common/rte_pmu.c | 456 ++++++++++++++++++++++++++ lib/eal/include/meson.build | 1 + lib/eal/include/rte_pmu.h | 204 ++++++++++++ lib/eal/linux/eal.c | 4 + lib/eal/version.map | 6 + 10 files changed, 765 insertions(+) create mode 100644 app/test/test_pmu.c create mode 100644
lib/eal/common/pmu_private.hcreate mode 100644lib/eal/common/rte_pmu.ccreate mode 100644lib/eal/include/rte_pmu.h<snip>
Quoted 172 lines
lib/eal/common/rte_pmu.c new renamed from lib/eal/common/rte_pmu.c +0
-0file mode 100644 index 0000000000..049fe19fe3
lib/eal/common/rte_pmu.c new file +71
-0@@ -0,0 +1,456 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(C) 2022 Marvell International Ltd. +*/ + +#include <ctype.h> +#include <dirent.h> +#include <errno.h> +#include <regex.h> +#include <stdlib.h> +#include <string.h> +#include <sys/ioctl.h> +#include <sys/mman.h> +#include <sys/queue.h> +#include <sys/syscall.h> +#include <unistd.h> + +#include <rte_eal_paging.h> +#include <rte_pmu.h> +#include <rte_tailq.h> + +#include "pmu_private.h" + +#define EVENT_SOURCE_DEVICES_PATH "/sys/bus/event_source/devices" + +#ifndef GENMASK_ULL +#define GENMASK_ULL (h, l) ((~0ULL - (1ULL << (l)) + 1) & (~0ULL >> +((64 +- 1 - (h))))) #endif + +#ifndef FIELD_PREP +#define FIELD_PREP (m, v) (((uint64_t)(v) << (__builtin_ffsll(m) - 1)) +& +(m)) #endif + +struct rte_pmu *rte_pmu; + +/* +* Following __rte_weak functions provide default no-op. +Architectures should override them if +* necessary. +*/ + +int +__rte_weak pmu_arch_init(void) +{ +return 0 ; +} + +void +__rte_weak pmu_arch_fini(void) +{ +} + +void +__rte_weak pmu_arch_fixup_config(uint64_t config[3]) { +RTE_SET_USED (config); +} + +static int +get_term_format (const char *name,int *num,uint64_t *mask) { +char *config =NULL ; +char path[PATH_MAX ]; +int high, low, ret; +FILE *fp; + +/* quiesce -Wmaybe-uninitialized warning */ + *num =0 ; + *mask =0 ; + + snprintf(path, sizeof(path), EVENT_SOURCE_DEVICES_PATH +"/%s/format/%s", rte_pmu->name,name);
lib/eal/common/rte_pmu.c new file +71
-0+ fp =fopen (path,"r" ); +if (!fp) +return -errno; + + errno =0 ; + ret =fscanf (fp,"%m[^:]:%d-%d" , &config, &low, &high); +if (ret <2 ) { + ret = -ENODATA ; + goto out; + } +if (errno) { + ret = -errno; + goto out; + } + +if (ret ==2 ) + high = low; + + *mask =GENMASK_ULL (high, low); +/* Last digit should be [012]. If last digit is missing 0 is implied. */ + *num = config[strlen (config) -1 ]; + *num =isdigit (*num) ? *num -'0' :0 ; + + ret =0 ; +out: +free (config); +fclose (fp); + +return ret; +} + +static int +parse_event (char *buf,uint64_t config[3 ]) { +char *token, *term; +int num, ret, val; +uint64_t mask; + + config[0 ] = config[1 ] = config[2 ] =0 ; + + token =strtok (buf,"," ); +while (token) { + errno =0 ; +/* <term>=<value> */ + ret =sscanf (token,"%m[^=]=%i" , &term, &val); +if (ret <1 ) +return -ENODATA ; +if (errno) +return -errno; +if (ret ==1 ) + val =1 ; + + ret =get_term_format (term, &num, &mask); +free (term); +if (ret) +return ret; + + config[num] |=FIELD_PREP (mask, val); + token =strtok (NULL ,"," ); + } + +return 0 ; +} + +static int +get_event_config (const char *name,uint64_t config[3 ]) { +char path[PATH_MAX ], buf[BUFSIZ ]; +FILE *fp; +int ret; + + snprintf(path, sizeof(path), EVENT_SOURCE_DEVICES_PATH +"/%s/events/%s", rte_pmu->name,name);
lib/eal/common/rte_pmu.c new file +23
-0+ fp =fopen (path,"r" ); +if (!fp) +return -errno; + + ret =fread (buf,1 ,sizeof (buf), fp); +if (ret ==0 ) { +fclose (fp); + +return -EINVAL ; + } +fclose (fp); + buf[ret] ='\0' ; + +return parse_event (buf, config); +} + +static int +do_perf_event_open (uint64_t config[3 ],int lcore_id,int group_fd) { +struct perf_event_attr attr = { + .size =sizeof (struct perf_event_attr ), + .type =PERF_TYPE_RAW , + .exclude_kernel =1 , + .exclude_hv =1 ,lib/eal/common/rte_pmu.c
Quoted 10 lines
lib/eal/common/rte_pmu.c new file +10
-0+ .disabled = 1, + }; + +pmu_arch_fixup_config (config); + + attr.config = config[0 ]; + attr.config1 = config[1 ]; + attr.config2 = config[2 ]; + + return syscall(SYS_perf_event_open, &attr, rte_gettid(),Looks like using '0' instead of
rte_gettid()takes the same effect. A small optimization.Quoted 205 lines
rte_lcore_to_cpu_id(lcore_id),lib/eal/common/rte_pmu.c new file +204
-0+ group_fd, 0); +} + +static int +open_events (int lcore_id) +{ +struct rte_pmu_event_group *group = &rte_pmu->group[lcore_id]; +struct rte_pmu_event *event; +uint64_t config[3 ]; +int num =0 , ret; + +/* group leader gets created first, with fd = -1 */ + group->fds[0 ] =-1 ; + + TAILQ_FOREACH(event, &rte_pmu->event_list, next) { + ret =get_event_config (event->name, config); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to get %s event config\n" , event->name); +continue ; + } + + ret =do_perf_event_open (config, lcore_id, group->fds[0 ]); +if (ret ==-1 ) { +if (errno ==EOPNOTSUPP ) +RTE_LOG (ERR ,EAL ,"64 bit counters not supported\n" ); + + ret = -errno; + goto out; + } + + group->fds[event->index] = ret; + num++; + } + +return 0 ; +out: +for (--num; num >=0 ; num--) { +close (group->fds[num]); + group->fds[num] =-1 ; + } + + +return ret; +} + +static int +mmap_events (int lcore_id) +{ +struct rte_pmu_event_group *group = &rte_pmu->group[lcore_id]; +void *addr; +int ret, i; + +for (i =0 ; i < rte_pmu->num_group_events; i++) { + addr =mmap (0 ,rte_mem_page_size (),PROT_READ ,MAP_SHARED , group->fds[i],0 ); +if (addr ==MAP_FAILED ) { + ret = -errno; + goto out; + } + + group->mmap_pages[i] = addr; + } + +return 0 ; +out: +for (; i; i--) { +munmap (group->mmap_pages[i -1 ],rte_mem_page_size ()); + group->mmap_pages[i -1 ] =NULL ; + } + +return ret; +} + +static void +cleanup_events (int lcore_id) +{ +struct rte_pmu_event_group *group = &rte_pmu->group[lcore_id]; +int i; + +if (!group->fds) +return ; + +if (group->fds[0 ] !=-1 ) +ioctl (group->fds[0 ],PERF_EVENT_IOC_DISABLE ,PERF_IOC_FLAG_GROUP ); + +for (i =0 ; i < rte_pmu->num_group_events; i++) { +if (group->mmap_pages[i]) { +munmap (group->mmap_pages[i],rte_mem_page_size ()); + group->mmap_pages[i] =NULL ; + } + +if (group->fds[i] !=-1 ) { +close (group->fds[i]); + group->fds[i] =-1 ; + } + } + +free (group->mmap_pages); +free (group->fds); + + group->mmap_pages =NULL ; + group->fds =NULL ; + group->enabled = false; +} + +int __rte_noinline +rte_pmu_enable_group (int lcore_id) +{ +struct rte_pmu_event_group *group = &rte_pmu->group[lcore_id]; +int ret; + +if (rte_pmu->num_group_events ==0 ) { +RTE_LOG (DEBUG ,EAL ,"no matching PMU events\n" ); + +return 0 ; + } + + group->fds =calloc (rte_pmu->num_group_events,sizeof (*group->fds)); +if (!group->fds) { +RTE_LOG (ERR ,EAL ,"failed to alloc descriptor memory\n" ); + +return -ENOMEM ; + } + + group->mmap_pages =calloc (rte_pmu->num_group_events,sizeof (*group->mmap_pages)); +if (!group->mmap_pages) { +RTE_LOG (ERR ,EAL ,"failed to alloc userpage memory\n" ); + + ret = -ENOMEM ; + goto out; + } + + ret =open_events (lcore_id); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to open events on lcore-worker-%d\n" , lcore_id); + goto out; + } + + ret =mmap_events (lcore_id); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to map events on lcore-worker-%d\n" , lcore_id); + goto out; + } + +if (ioctl (group->fds[0 ],PERF_EVENT_IOC_ENABLE ,PERF_IOC_FLAG_GROUP ) ==-1 ) { +RTE_LOG (ERR ,EAL ,"failed to enable events on lcore-worker-%d\n" , +lcore_id); + + ret = -errno; + goto out; + } + +return 0 ; + +out: +cleanup_events (lcore_id); + +return ret; +} + +static int +scan_pmus (void ) +{ +char path[PATH_MAX ]; +struct dirent *dent; +const char *name; +DIR *dirp; + + dirp =opendir (EVENT_SOURCE_DEVICES_PATH ); +if (!dirp) +return -errno; + +while ((dent =readdir (dirp))) { + name = dent->d_name; +if (name[0 ] =='.' ) +continue ; + +/* sysfs entry should either contain cpus or be a cpu */ +if (!strcmp (name,"cpu" )) +break ; + +snprintf (path,sizeof (path),EVENT_SOURCE_DEVICES_PATH "/%s/cpus" , name); +if (access (path,F_OK ) ==0 ) +break ; + } + +closedir (dirp); + +if (dent) { + rte_pmu->name =strdup (name); +if (!rte_pmu->name) +return -ENOMEM ; + } + +return rte_pmu->name ?0 : -ENODEV ; +} + +int +rte_pmu_add_event (const char *name) +{ +struct rte_pmu_event *event; +char path[PATH_MAX ]; + + snprintf(path, sizeof(path), EVENT_SOURCE_DEVICES_PATH +"/%s/events/%s", rte_pmu->name,Quoted 1 line
name);
Better to check if
rte_pmuis available.
See below.Quoted 53 lines
lib/eal/common/rte_pmu.c new file +53
-0+if (access (path,R_OK )) +return -ENODEV ; + + TAILQ_FOREACH(event, &rte_pmu->event_list, next) { +if (!strcmp (event->name, name)) +return event->index; +continue ; + } + + event =calloc (1 ,sizeof (*event)); +if (!event) +return -ENOMEM ; + + event->name =strdup (name); +if (!event->name) { +free (event); + +return -ENOMEM ; + } + + event->index = rte_pmu->num_group_events++; +TAILQ_INSERT_TAIL (&rte_pmu->event_list, event, next); + +RTE_LOG (DEBUG ,EAL ,"%s even added at index %d\n" , name, +event->index); + +return event->index; +} + +void +eal_pmu_init (void ) +{ +int ret; + + rte_pmu =calloc (1 ,sizeof (*rte_pmu)); +if (!rte_pmu) { +RTE_LOG (ERR ,EAL ,"failed to alloc PMU\n" ); + +return ; + } + +TAILQ_INIT (&rte_pmu->event_list); + + ret =scan_pmus (); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to find core pmu\n" ); + goto out; + } + + ret =pmu_arch_init (); +if (ret) { +RTE_LOG (ERR ,EAL ,"failed to setup arch for PMU\n" ); + goto out;
Quoted 9 lines
lib/eal/common/rte_pmu.c line 435
Quoted 6 lines
lib/eal/common/rte_pmu.c new file +6
-0+ } + +return ; +out: +free (rte_pmu->name); +free (rte_pmu);Set
rte_pmuto NULL to prevent unintentional use?
Next series will take use of global pmu instance so this will no longer be required though your suggestions may be applied to other pointers around.
Quoted 12 lines
lib/eal/common/rte_pmu.c line 444
Quoted 9 lines
lib/eal/common/rte_pmu.c new file +9
-0+} + +void +eal_pmu_fini (void ) +{ +struct rte_pmu_event *event, *tmp; +int lcore_id; + + RTE_TAILQ_FOREACH_SAFE(event, &rte_pmu->event_list, next, tmp) {
rte_pmucan be unavailable if init fails. Better to check before accessing.
Yep.
Quoted 13 lines
Quoted 12 lines
lib/eal/common/rte_pmu.c new file +12
-0+TAILQ_REMOVE (&rte_pmu->event_list, event, next); +free (event->name); +free (event); + } + +RTE_LCORE_FOREACH_WORKER (lcore_id) +cleanup_events (lcore_id); + +pmu_arch_fini (); +free (rte_pmu->name); +free (rte_pmu); +}<snip>
PATCH v4 2/4 eal/arm: support reading ARM PMU events in runtime
In reply to Tomasz DuszynskiAdd support for reading ARM PMU events in runtime.
--- app/test/test_pmu.c | 4 ++ lib/eal/arm/include/meson.build | 1 + lib/eal/arm/include/rte_pmu_pmc.h | 39 +++++++++++ lib/eal/arm/meson.build | 4 ++ lib/eal/arm/rte_pmu.c | 104 ++++++++++++++++++++++++++++++ lib/eal/include/rte_pmu.h | 3 + 6 files changed, 155 insertions(+) create mode 100644 lib/eal/arm/include/rte_pmu_pmc.h create mode 100644 lib/eal/arm/rte_pmu.c
app/test/test_pmu.c +4 -0
@@ -13,6 +13,10 @@ test_pmu_read(void)int tries =10 ;int event =-1 ; +#if defined(RTE_ARCH_ARM64 ) + event =rte_pmu_add_event ("cpu_cycles" ); +#endif +while (tries--) val +=rte_pmu_read (event);
lib/eal/arm/include/meson.build +1 -0
@@ -20,6 +20,7 @@ arch_headers = files( 'rte_pause_32.h', 'rte_pause_64.h', 'rte_pause.h', + 'rte_pmu_pmc.h', 'rte_power_intrinsics.h', 'rte_prefetch_32.h', 'rte_prefetch_64.h',
lib/eal/arm/include/rte_pmu_pmc.h new file +39 -0
@@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2022 Marvell. +*/ + +#ifndef _RTE_PMU_PMC_ARM_H_ +#define _RTE_PMU_PMC_ARM_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include <rte_common.h> + +static __rte_always_inline uint64_t +rte_pmu_pmc_read (int index) +{ +uint64_t val; + +if (index ==31 ) { +/* CPU Cycles (0x11) must be read via pmccntr_el0 */ + asmvolatile ("mrs %0, pmccntr_el0" :"=r" (val)); + }else { + asmvolatile ( +"msr pmselr_el0, %x0\n" +"mrs %0, pmxevcntr_el0\n" + :"=r" (val) + :"rZ" (index) + ); + } + +return val; +} +#define rte_pmu_pmc_read rte_pmu_pmc_read + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_PMU_PMC_ARM_H_ */
lib/eal/arm/meson.build +4 -0
@@ -9,3 +9,7 @@ sources += files( 'rte_hypervisor.c', 'rte_power_intrinsics.c', ) + +if is_linux + sources += files('rte_pmu.c') +endif
lib/eal/arm/rte_pmu.c new file +104 -0
104 lines not on this page · raw message
lib/eal/include/rte_pmu.h +3 -0
@@ -20,6 +20,9 @@ extern "C" {#include <rte_branch_prediction.h> #include <rte_lcore.h> #include <rte_log.h> +#if defined(RTE_ARCH_ARM64 ) +#include <rte_pmu_pmc.h> +#endif /** * @file
Signature · 1 line
2.25.1
PATCH v4 3/4 eal/x86: support reading Intel PMU events in runtime
In reply to Tomasz DuszynskiAdd support for reading Intel PMU events in runtime.
--- app/test/test_pmu.c | 2 ++ lib/eal/include/rte_pmu.h | 2 +- lib/eal/x86/include/meson.build | 1 + lib/eal/x86/include/rte_pmu_pmc.h | 33 +++++++++++++++++++++++++++++++ 4 files changed, 37 insertions(+), 1 deletion(-) create mode 100644 lib/eal/x86/include/rte_pmu_pmc.h
app/test/test_pmu.c +2 -0
@@ -15,6 +15,8 @@ test_pmu_read(void)#if defined(RTE_ARCH_ARM64 ) event =rte_pmu_add_event ("cpu_cycles" ); +#elif defined(RTE_ARCH_X86_64 ) + event =rte_pmu_add_event ("cpu-cycles" );#endif while (tries--)
lib/eal/include/rte_pmu.h +1 -1
@@ -20,7 +20,7 @@ extern "C" {#include <rte_branch_prediction.h> #include <rte_lcore.h> #include <rte_log.h> -#if defined(RTE_ARCH_ARM64 ) +#if defined(RTE_ARCH_ARM64 ) || defined(RTE_ARCH_X86_64 )#include <rte_pmu_pmc.h> #endif
lib/eal/x86/include/meson.build +1 -0
@@ -9,6 +9,7 @@ arch_headers = files( 'rte_io.h', 'rte_memcpy.h', 'rte_pause.h', + 'rte_pmu_pmc.h', 'rte_power_intrinsics.h', 'rte_prefetch.h', 'rte_rtm.h',
lib/eal/x86/include/rte_pmu_pmc.h new file +33 -0
@@ -0,0 +1,33 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2022 Marvell. +*/ + +#ifndef _RTE_PMU_PMC_X86_H_ +#define _RTE_PMU_PMC_X86_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include <rte_common.h> + +static __rte_always_inline uint64_t +rte_pmu_pmc_read (int index) +{ +uint32_t high, low; + + asmvolatile ( +"rdpmc\n" + :"=a" (low),"=d" (high) + :"c" (index) + ); + +return ((uint64_t )high <<32 ) | (uint64_t )low; +} +#define rte_pmu_pmc_read rte_pmu_pmc_read + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_PMU_PMC_X86_H_ */
Signature · 1 line
2.25.1
PATCH v4 4/4 eal: add PMU support to tracing library
In reply to Tomasz DuszynskiIn order to profile app one needs to store significant amount of samples somewhere for an analysis latern on. Since trace library supports storing data in a CTF format lets take adventage of that and add a dedicated PMU tracepoint.
--- app/test/test_trace_perf.c | 4 ++ doc/guides/prog_guide/profile_app.rst | 5 ++ doc/guides/prog_guide/trace_lib.rst | 32 ++++++++++++ lib/eal/common/eal_common_trace_points.c | 3 ++ lib/eal/common/rte_pmu.c | 63 ++++++++++++++++++++++++ lib/eal/include/rte_eal_trace.h | 11 +++++ lib/eal/version.map | 1 + 7 files changed, 119 insertions(+)
app/test/test_trace_perf.c +4 -0
@@ -114,6 +114,8 @@ worker_fn_##func(void *arg) \#define GENERIC_DOUBLE rte_eal_trace_generic_double(3.66666)#define GENERIC_STR rte_eal_trace_generic_str("hello world")#define VOID_FP app_dpdk_test_fp() +/* 0 corresponds first event passed via --trace= */ +#define READ_PMU rte_eal_trace_pmu_read(0)WORKER_DEFINE (GENERIC_VOID)WORKER_DEFINE (GENERIC_U64)
@@ -122,6 +124,7 @@ WORKER_DEFINE(GENERIC_FLOAT) WORKER_DEFINE(GENERIC_DOUBLE) WORKER_DEFINE(GENERIC_STR) WORKER_DEFINE(VOID_FP) +WORKER_DEFINE(READ_PMU) static void run_test(const char *str, lcore_function_t f, struct test_data *data, size_t sz)
@@ -174,6 +177,7 @@ test_trace_perf(void)run_test ("double" , worker_fn_GENERIC_DOUBLE, data, sz);run_test ("string" , worker_fn_GENERIC_STR, data, sz);run_test ("void_fp" , worker_fn_VOID_FP, data, sz); +run_test ("read_pmu" , worker_fn_READ_PMU, data, sz);rte_free (data);return TEST_SUCCESS ;
doc/guides/prog_guide/profile_app.rst +5 -0
@@ -16,6 +16,11 @@ that information, perf being an example here. Though in some scenarios, eg. when isolated (nohz_full) and run dedicated tasks, using perf is less than ideal. In such cases one can read specific events directly from application via ``rte_pmu_read()``. +Alternatively tracing library can be used which offers dedicated tracepoint +``rte_eal_trace_pmu_event()``. + +Refer to :doc:`../prog_guide/trace_lib` for more details. + Profiling on x86 ----------------
doc/guides/prog_guide/trace_lib.rst +32 -0
@@ -46,6 +46,7 @@ DPDK tracing library features trace format and is compatible with ``LTTng``. For detailed information, refer to `Common Trace Format <https://diamon.org/ctf/>`_. +- Support reading PMU events on ARM64 and x86 (Intel) How to add a tracepoint? ------------------------
@@ -137,6 +138,37 @@ the user must use ``RTE_TRACE_POINT_FP`` instead of ``RTE_TRACE_POINT``. ``RTE_TRACE_POINT_FP`` is compiled out by default and it can be enabled using the ``enable_trace_fp`` option for meson build. +PMU tracepoint +-------------- + +Performance measurement unit (PMU) event values can be read from hardware +registers using predefined ``rte_pmu_read`` tracepoint. + +Tracing is enabled via ``--trace`` EAL option by passing both expression +matching PMU tracepoint name i.e ``lib.eal.pmu.read`` and expression +``e=ev1[,ev2,...]`` matching particular events:: + + --trace='*pmu.read\|e=cpu_cycles,l1d_cache' + +Event names are available under ``/sys/bus/event_source/devices/PMU/events`` +directory, where ``PMU`` is a placeholder for either a ``cpu`` or a directory +containing ``cpus``. + +In contrary to other tracepoints this does not need any extra variables +added to source files. Instead, caller passes index which follows the order of +events specified via ``--trace`` parameter. In the following example index ``0`` +corresponds to ``cpu_cyclces`` while index ``1`` corresponds to ``l1d_cache``. + +.. code-block:: c + + ... + rte_eal_trace_pmu_read(0); + rte_eal_trace_pmu_read(1); + ... + +PMU tracing support must be explicitly enabled using the ``enable_trace_fp`` +option for meson build. + Event record mode -----------------
lib/eal/common/eal_common_trace_points.c +3 -0
@@ -75,3 +75,6 @@ RTE_TRACE_POINT_REGISTER(rte_eal_trace_intr_enable, lib.eal.intr.enable) RTE_TRACE_POINT_REGISTER(rte_eal_trace_intr_disable, lib.eal.intr.disable) + +RTE_TRACE_POINT_REGISTER (rte_eal_trace_pmu_read, + lib.eal.pmu.read)
lib/eal/common/rte_pmu.c +63 -0
86 lines not on this page · raw message
lib/eal/include/rte_eal_trace.h +11 -0
@@ -17,6 +17,7 @@ extern "C" {#include <rte_alarm.h> #include <rte_interrupts.h> +#include <rte_pmu.h> #include <rte_trace_point.h> #include "eal_interrupts.h"
@@ -279,6 +280,16 @@ RTE_TRACE_POINT(rte_trace_point_emit_string (cpuset); ) +/* PMU */ +RTE_TRACE_POINT_FP ( + rte_eal_trace_pmu_read, + RTE_TRACE_POINT_ARGS(int index), + uint64_t val; + rte_trace_point_emit_int(index); + val = rte_pmu_read(index); + rte_trace_point_emit_u64(val); +) + #ifdef __cplusplus }#endif
lib/eal/version.map +1 -0
@@ -442,6 +442,7 @@ EXPERIMENTAL { rte_thread_join; # added in 23.03 + __rte_eal_trace_pmu_read; # WINDOWS_NO_EXPORT rte_pmu; # WINDOWS_NO_EXPORT rte_pmu_add_event; # WINDOWS_NO_EXPORT rte_pmu_read; # WINDOWS_NO_EXPORT
Signature · 1 line
2.25.1
v5 cover add support for self monitoring
In reply to Tomasz DuszynskiThis series adds self monitoring support i.e allows to configure and read performance measurement unit (PMU) counters in runtime without using perf utility. This has certain adventages when application runs on isolated cores with nohz_full kernel parameter.
Events can be read directly using rte_pmu_read() or using dedicated tracepoint rte_eal_trace_pmu_read(). The latter will cause events to be stored inside CTF file.
By design, all enabled events are grouped together and the same group is attached to lcores that use self monitoring funtionality.
Events are enabled by names, which need to be read from standard location under sysfs i.e
/sys/bus/event_source/devices/PMU/events
where PMU is a core pmu i.e one measuring cpu events. As of today raw events are not supported.
v5:
- address review comments
- fix sign extension while reading pmu on x86
- fix regex mentioned in doc
- various minor changes/improvements here and there v4:
- fix freeing mem detected by debug_autotest v3:
- fix shared build
v2:
- fix problems reported by test build infra
Tomasz Duszynski (4):
eal: add generic support for reading PMU events eal/arm: support reading ARM PMU events in runtime eal/x86: support reading Intel PMU events in runtime
eal: add PMU support to tracing library
app/test/meson.build | 1 + app/test/test_pmu.c | 47 +++ app/test/test_trace_perf.c | 4 + doc/guides/prog_guide/profile_app.rst | 13 + doc/guides/prog_guide/trace_lib.rst | 32 ++ lib/eal/arm/include/meson.build | 1 + lib/eal/arm/include/rte_pmu_pmc.h | 39 ++ lib/eal/arm/meson.build | 4 + lib/eal/arm/rte_pmu.c | 104 +++++ lib/eal/common/eal_common_trace_points.c | 3 + lib/eal/common/meson.build | 3 + lib/eal/common/pmu_private.h | 41 ++ lib/eal/common/rte_pmu.c | 504 +++++++++++++++++++++++ lib/eal/include/meson.build | 1 + lib/eal/include/rte_eal_trace.h | 10 + lib/eal/include/rte_pmu.h | 202 +++++++++ lib/eal/linux/eal.c | 4 + lib/eal/version.map | 7 + lib/eal/x86/include/meson.build | 1 + lib/eal/x86/include/rte_pmu_pmc.h | 33 ++ 20 files changed, 1054 insertions(+) create mode 100644 app/test/test_pmu.c create mode 100644 lib/eal/arm/include/rte_pmu_pmc.h create mode 100644 lib/eal/arm/rte_pmu.c create mode 100644 lib/eal/common/pmu_private.h create mode 100644 lib/eal/common/rte_pmu.c create mode 100644 lib/eal/include/rte_pmu.h create mode 100644 lib/eal/x86/include/rte_pmu_pmc.h
--
2.34.1
PATCH v5 1/4 eal: add generic support for reading PMU events
Add support for programming PMU counters and reading their values in runtime bypassing kernel completely.
This is especially useful in cases where CPU cores are isolated (nohz_full) i.e run dedicated tasks. In such cases one cannot use standard perf utility without sacrificing latency and performance.
--- app/test/meson.build | 1 + app/test/test_pmu.c | 41 +++ doc/guides/prog_guide/profile_app.rst | 8 + lib/eal/common/meson.build | 3 + lib/eal/common/pmu_private.h | 41 +++ lib/eal/common/rte_pmu.c | 435 ++++++++++++++++++++++++++ lib/eal/include/meson.build | 1 + lib/eal/include/rte_pmu.h | 199 ++++++++++++ lib/eal/linux/eal.c | 4 + lib/eal/version.map | 6 + 10 files changed, 739 insertions(+) create mode 100644 app/test/test_pmu.c create mode 100644 lib/eal/common/pmu_private.h create mode 100644 lib/eal/common/rte_pmu.c create mode 100644 lib/eal/include/rte_pmu.h
app/test/meson.build +1 -0
@@ -143,6 +143,7 @@ test_sources = files( 'test_timer_racecond.c', 'test_timer_secondary.c', 'test_ticketlock.c', + 'test_pmu.c', 'test_trace.c', 'test_trace_register.c', 'test_trace_perf.c',
app/test/test_pmu.c new file +41 -0
41 lines not on this page · raw message
doc/guides/prog_guide/profile_app.rst +8 -0
@@ -7,6 +7,14 @@ Profile Your Application The following sections describe methods of profiling DPDK applications on different architectures. +Performance counter based profiling +----------------------------------- + +Majority of architectures support some sort hardware measurement unit which provides a set of +programmable counters that monitor specific events. There are different tools which can gather +that information, perf being an example here. Though in some scenarios, eg. when CPU cores are +isolated (nohz_full) and run dedicated tasks, using perf is less than ideal. In such cases one can +read specific events directly from application via ``rte_pmu_read()``. Profiling on x86 ----------------
lib/eal/common/meson.build +3 -0
@@ -38,6 +38,9 @@ sources += files( 'rte_service.c', 'rte_version.c', ) +if is_linux + sources += files('rte_pmu.c') +endif if is_linux or is_windows sources += files('eal_common_dynmem.c') endif
lib/eal/common/pmu_private.h new file +41 -0
41 lines not on this page · raw message
lib/eal/common/rte_pmu.c new file +435 -0
435 lines not on this page · raw message
lib/eal/include/meson.build +1 -0
@@ -36,6 +36,7 @@ headers += files( 'rte_pci_dev_features.h', 'rte_per_lcore.h', 'rte_pflock.h', + 'rte_pmu.h', 'rte_random.h', 'rte_reciprocal.h', 'rte_seqcount.h',
lib/eal/include/rte_pmu.h new file +199 -0
199 lines not on this page · raw message
lib/eal/linux/eal.c +4 -0
@@ -53,6 +53,7 @@#include "eal_options.h" #include "eal_vfio.h" #include "hotplug_mp.h" +#include "pmu_private.h" #define MEMSIZE_IF_NO_HUGE_PAGE (64ULL * 1024ULL * 1024ULL)
@@ -1206,6 +1207,8 @@ rte_eal_init(int argc, char **argv)return -1 ; } +eal_pmu_init (); +if (rte_eal_tailqs_init () <0 ) {rte_eal_init_alert ("Cannot init tail queues for objects" ); rte_errno =EFAULT ;
@@ -1372,6 +1375,7 @@ rte_eal_cleanup(void)eal_bus_cleanup ();rte_trace_save ();eal_trace_fini (); +eal_pmu_fini ();/* after this point, any DPDK pointers will become dangling */ rte_eal_memory_detach ();eal_mp_dev_hotplug_cleanup ();
lib/eal/version.map +6 -0
@@ -440,6 +440,11 @@ EXPERIMENTAL { rte_thread_detach; rte_thread_equal; rte_thread_join; + + # added in 23.03 + rte_pmu; # WINDOWS_NO_EXPORT + rte_pmu_add_event; # WINDOWS_NO_EXPORT + rte_pmu_read; # WINDOWS_NO_EXPORT }; INTERNAL {
@@ -483,4 +488,5 @@ INTERNAL { rte_mem_map; rte_mem_page_size; rte_mem_unmap; + rte_pmu_enable_group; # WINDOWS_NO_EXPORT };
Signature · 1 line
2.34.1
RE: [PATCH v5 1/4] eal: add generic support for reading PMU events
Quoted 12 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Wednesday, 11 January 2023 00.47Add support for programming PMU counters and reading their values in runtime bypassing kernel completely.
This is especially useful in cases where CPU cores are isolated (
nohz_full) i.e run dedicated tasks. In such cases one cannot use standard perf utility without sacrificing latency and performance.Signed-off-by Tomasz Duszynski <[email protected]>---
[...]
Quoted 19 lines
(quoted diff) +2
-0+static int +do_perf_event_open(uint64_t config[3], unsigned int lcore_id, int
group_fd)lib/eal/common/rte_pmu.c new file +16
-0+{ +struct perf_event_attr attr = { + .size =sizeof (struct perf_event_attr ), + .type =PERF_TYPE_RAW , + .exclude_kernel =1 , + .exclude_hv =1 , + .disabled =1 , + }; + +pmu_arch_fixup_config (config); + + attr.config = config[0 ]; + attr.config1 = config[1 ]; + attr.config2 = config[2 ]; + + return syscall(SYS_perf_event_open, &attr, 0,
Quoted 2 lines
rte_lcore_to_cpu_id(lcore_id),group_fd, 0);
+}
If SYS_perf_event_open() must be called from the worker thread itself, then lcore_id must not be passed as a parameter to do_perf_event_open(). Otherwise, I would expect to be able to call do_perf_event_open() from the main thread and pass any lcore_id of a worker thread. This comment applies to all functions that must be called from the worker thread itself. It also applies to the functions that call such functions.
[...]
Quoted 9 lines
lib/eal/include/rte_pmu.h new file +6
-0+/** +* A structure describing a group of events. +*/ +struct rte_pmu_event_group { +int fds[MAX_NUM_GROUP_EVENTS ];/**< array of event descriptors */ +struct perf_event_mmap_page *mmap_pages[MAX_NUM_GROUP_EVENTS ];/**< array of user pages */
(quoted diff) +1
-0+ bool enabled; /**< true if group was enabled on particular lcore*/
lib/eal/include/rte_pmu.h line 52
Quoted 10 lines
lib/eal/include/rte_pmu.h new file +10
-0+}; + +/** +* A structure describing an event. +*/ +struct rte_pmu_event { +char *name;/** name of an event */ +unsigned int index;/** event index into fds/mmap_pages */ +TAILQ_ENTRY (rte_pmu_event) next;/** list entry */ +};
Move the "enabled" field up, making it the first field in this structure. This might reduce the number of instructions required to check (!group->enabled) in rte_pmu_read().
Also, each instance of the structure is used individually per lcore, so the structure should be cache line aligned to avoid unnecessarily crossing cache lines.
I.e.:
struct rte_pmu_event_group {
bool enabled; /**< true if group was enabled on particular lcore */
int fds[MAX_NUM_GROUP_EVENTS]; /**< array of event descriptors */
struct perf_event_mmap_page *mmap_pages[MAX_NUM_GROUP_EVENTS]; /**< array of user pages */
} __rte_cache_aligned;
Quoted 13 lines
lib/eal/include/rte_pmu.h new file +6
-0+ +/** +* A PMU state container. +*/ +struct rte_pmu { +char *name;/** name of core PMU listed under
/sys/bus/event_source/devices*/(quoted diff) +1
-0+ struct rte_pmu_event_group group[RTE_MAX_LCORE]; /**< per lcoreevent group data */
(quoted diff) +1
-0+ unsigned int num_group_events; /**< number of events in a group*/
(quoted diff) +1
-0+ TAILQ_HEAD(, rte_pmu_event) event_list; /**< list of matchingevents */
lib/eal/include/rte_pmu.h line 65
Quoted 4 lines
lib/eal/include/rte_pmu.h new file +4
-0+}; + +/** Pointer to the PMU state container */ +extern struct rte_pmu rte_pmu;
Just "The PMU state container". It is not a pointer anymore. :-)
[...]
Quoted 15 lines
lib/eal/include/rte_pmu.h new file +15
-0+/** +* @internal +* +* Read PMU counter. +* +* @param pc +* Pointer to the mmapped user page. +* @return +* Counter value read from hardware. +*/ +__rte_internal +static __rte_always_inline uint64_t +rte_pmu_read_userpage (struct perf_event_mmap_page *pc) +{ +uint64_t width, offset;
lib/eal/include/rte_pmu.h line 96
Quoted 10 lines
lib/eal/include/rte_pmu.h new file +10
-0+uint32_t seq, index; +int64_t pmc; + +for (;;) { + seq = pc->lock; +rte_compiler_barrier (); + index = pc->index; + offset = pc->offset; + width = pc->pmc_width; +
Please add a comment here about the special meaning of index == 0.
Quoted 15 lines
lib/eal/include/rte_pmu.h new file +15
-0+if (likely (pc->cap_user_rdpmc && index)) { + pmc =rte_pmu_pmc_read (index -1 ); + pmc <<=64 - width; + pmc >>=64 - width; + offset += pmc; + } + +rte_compiler_barrier (); + +if (likely (pc->lock == seq)) +return offset; + } + +return 0 ; +}
[...]
Quoted 18 lines
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Read hardware counter configured to count occurrences of an event.
+ *
+ * @param index
+ * Index of an event to be read.
+ * @return
+ * Event value read from register. In case of errors or lack of
support
+ * 0 is returned. In other words, stream of zeros in a trace file
+ * indicates problem with reading particular PMU event register.
+ */lib/eal/include/rte_pmu.h new file +4
-0+__rte_experimental +static __rte_always_inline uint64_t +rte_pmu_read (unsigned int index) +{
lib/eal/include/rte_pmu.h line 168
Quoted 10 lines
lib/eal/include/rte_pmu.h new file +10
-0+struct rte_pmu_event_group *group; +int ret, lcore_id =rte_lcore_id (); + + group = &rte_pmu.group[lcore_id]; +if (unlikely (!group->enabled)) { + ret =rte_pmu_enable_group (lcore_id); +if (ret) +return 0 ; + + group->enabled = true;
Group->enabled should be set inside rte_pmu_enable_group(), not here.
Quoted 7 lines
lib/eal/include/rte_pmu.h new file +7
-0+ } + +if (unlikely (index >= rte_pmu.num_group_events)) +return 0 ; + +return rte_pmu_read_userpage (group->mmap_pages[index]); +}
RE: [PATCH v5 1/4] eal: add generic support for reading PMU events
Quoted 47 lines
-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: Wednesday, January 11, 2023 10:06 AM
To: Tomasz Duszynski <[email protected]>; [email protected]
Cc: [email protected]; Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; [email protected]; [email protected]
Subject: [EXT] RE: [PATCH v5 1/4] eal: add generic support for reading PMU eventsExternal Email
----------------------------------------------------------------------
Quoted 12 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Wednesday, 11 January 2023 00.47Add support for programming PMU counters and reading their values in runtime bypassing kernel completely.
This is especially useful in cases where CPU cores are isolated (
nohz_full) i.e run dedicated tasks. In such cases one cannot use standard perf utility without sacrificing latency and performance.Signed-off-by Tomasz Duszynski <[email protected]>---
[...]
Quoted 19 lines
(quoted diff) +2
-0+static int +do_perf_event_open(uint64_t config[3], unsigned int lcore_id, int
group_fd)lib/eal/common/rte_pmu.c new file +16
-0+{ +struct perf_event_attr attr = { + .size =sizeof (struct perf_event_attr ), + .type =PERF_TYPE_RAW , + .exclude_kernel =1 , + .exclude_hv =1 , + .disabled =1 , + }; + +pmu_arch_fixup_config (config); + + attr.config = config[0 ]; + attr.config1 = config[1 ]; + attr.config2 = config[2 ]; + + return syscall(SYS_perf_event_open, &attr, 0,
Quoted 9 lines
Quoted 2 lines
rte_lcore_to_cpu_id(lcore_id),group_fd, 0);
+}If
SYS_perf_event_open()must be called from the worker thread itself, thenlcore_idmust not be passed as a parameter todo_perf_event_open(). Otherwise, I would expect to be able to calldo_perf_event_open()from the main thread and pass anylcore_idof a worker thread. This comment applies to all functions that must be called from the worker thread itself. It also applies to the functions that call such functions.
Lcore_id is being passed around so that we don't need to call rte_lcore_id() each and every time.
Quoted 15 lines
[...]
Quoted 13 lines
lib/eal/include/rte_pmu.h new file +6
-0+/** +* A structure describing a group of events. +*/ +struct rte_pmu_event_group { +int fds[MAX_NUM_GROUP_EVENTS ];/**< array of event descriptors */ +struct perf_event_mmap_page *mmap_pages[MAX_NUM_GROUP_EVENTS ];/**< array of user pages */
(quoted diff) +1
-0+ bool enabled; /**< true if group was enabled on particular lcore*/
lib/eal/include/rte_pmu.h new file +4
-0+}; + +/** +* A structure describing an event.
Quoted 9 lines
lib/eal/include/rte_pmu.h
Quoted 5 lines
+ */
lib/eal/include/rte_pmu.h new file +4
-0+struct rte_pmu_event { +char *name;/** name of an event */ +unsigned int index;/** event index into fds/mmap_pages */ +TAILQ_ENTRY (rte_pmu_event) next;/** list entry */ };Move the "enabled" field up, making it the first field in this structure. This might reduce the number of instructions required to check (!group->enabled) in
rte_pmu_read().
This will be called once and no this will not produce more instructions. Why should it? In both cases compiler will need to load data at some offset and archs do have instructions for that.
Quoted 10 lines
Also, each instance of the structure is used individually per lcore, so the structure should be cache line aligned to avoid unnecessarily crossing cache lines.
I.e.:
struct
rte_pmu_event_group{
bool enabled; /**< true if group was enabled on particular lcore */
int fds[MAX_NUM_GROUP_EVENTS]; /**< array of event descriptors */
structperf_event_mmap_page*mmap_pages[MAX_NUM_GROUP_EVENTS]; /**< array of user pages */ }__rte_cache_aligned;
Yes, this can be aligned. While at it, I'd be more inclined to move mmap_pages up instead of enable.
Quoted 12 lines
Quoted 12 lines
lib/eal/include/rte_pmu.h new file +6
-0+ +/** +* A PMU state container. +*/ +struct rte_pmu { +char *name;/** name of core PMU listed under
/sys/bus/event_source/devices*/(quoted diff) +1
-0+ struct rte_pmu_event_group group[RTE_MAX_LCORE]; /**< per lcoreevent group data */
(quoted diff) +1
-0+ unsigned int num_group_events; /**< number of events in a group*/
Quoted 9 lines
app/test/test_pmu.c
Quoted 6 lines
(quoted diff) +1
-0+ TAILQ_HEAD(, rte_pmu_event) event_list; /**< list of matchingevents */
app/test/test_pmu.c new file +4
-0+}; + +/** Pointer to the PMU state container */ extern struct rte_pmu +rte_pmu;Just "The PMU state container". It is not a pointer anymore. :-)
Good catch.
Quoted 20 lines
[...]
Quoted 18 lines
lib/eal/include/rte_pmu.h new file +18
-0+/** +* @internal +* +* Read PMU counter. +* +* @param pc +* Pointer to the mmapped user page. +* @return +* Counter value read from hardware. +*/ +__rte_internal +static __rte_always_inline uint64_t +rte_pmu_read_userpage (struct perf_event_mmap_page *pc) { +uint64_t width, offset; +uint32_t seq, index; +int64_t pmc; + +for (;;) {
Quoted 8 lines
lib/eal/include/rte_pmu.h line 96
Quoted 6 lines
lib/eal/include/rte_pmu.h new file +6
-0+ seq = pc->lock; +rte_compiler_barrier (); + index = pc->index; + offset = pc->offset; + width = pc->pmc_width; +Please add a comment here about the special meaning of index == 0.
Okay.
Quoted 41 lines
Quoted 16 lines
lib/eal/include/rte_pmu.h new file +15
-0+if (likely (pc->cap_user_rdpmc && index)) { + pmc =rte_pmu_pmc_read (index -1 ); + pmc <<=64 - width; + pmc >>=64 - width; + offset += pmc; + } + +rte_compiler_barrier (); + +if (likely (pc->lock == seq)) +return offset; + } + +return 0 ; +}[...]
Quoted 22 lines
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Read hardware counter configured to count occurrences of an event.
+ *
+ * @param index
+ * Index of an event to be read.
+ * @return
+ * Event value read from register. In case of errors or lack of
support
+ * 0 is returned. In other words, stream of zeros in a trace file
+ * indicates problem with reading particular PMU event register.
+ */lib/eal/include/rte_pmu.h new file +8
-0+__rte_experimental +static __rte_always_inline uint64_t +rte_pmu_read (unsigned int index) +{ +struct rte_pmu_event_group *group; +int ret, lcore_id =rte_lcore_id (); + + group = &rte_pmu.group[lcore_id];
Quoted 9 lines
lib/eal/include/rte_pmu.h line 168
Quoted 6 lines
lib/eal/include/rte_pmu.h new file +6
-0+if (unlikely (!group->enabled)) { + ret =rte_pmu_enable_group (lcore_id); +if (ret) +return 0 ; + + group->enabled = true;
Group->enabledshould be set insiderte_pmu_enable_group(), not here.
This is easier to follow imo and not against coding guidelines so I prefer to leave it as is.
Quoted 8 lines
Quoted 7 lines
lib/eal/include/rte_pmu.h new file +7
-0+ } + +if (unlikely (index >= rte_pmu.num_group_events)) +return 0 ; + +return rte_pmu_read_userpage (group->mmap_pages[index]); +}
RE: [PATCH v5 1/4] eal: add generic support for reading PMU events
Quoted 48 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Wednesday, 11 January 2023 17.21Quoted 33 lines
From: Morten Brørup <[email protected]>
Sent: Wednesday, January 11, 2023 10:06 AMQuoted 12 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Wednesday, 11 January 2023 00.47Add support for programming PMU counters and reading their values in runtime bypassing kernel completely.
This is especially useful in cases where CPU cores are isolated (
nohz_full) i.e run dedicated tasks. In such cases one cannot use standard perf utility without sacrificing latency and performance.Signed-off-by Tomasz Duszynski <[email protected]>---
[...]
Quoted 15 lines
(quoted diff) +2
-0+static int +do_perf_event_open(uint64_t config[3], unsigned int lcore_id, int
group_fd)lib/eal/common/rte_pmu.c new file +12
-0+{ +struct perf_event_attr attr = { + .size =sizeof (struct perf_event_attr ), + .type =PERF_TYPE_RAW , + .exclude_kernel =1 , + .exclude_hv =1 , + .disabled =1 , + }; + +pmu_arch_fixup_config (config); + + attr.config = config[0 ];Quoted 8 lines
Quoted 6 lines
lib/eal/common/rte_pmu.c new file +4
-0+ attr.config1 = config[1 ]; + attr.config2 = config[2 ]; + + return syscall(SYS_perf_event_open, &attr, 0,
rte_lcore_to_cpu_id(lcore_id),group_fd, 0);
+}If
SYS_perf_event_open()must be called from the worker thread itself,then
lcore_idmust not beQuoted 1 line
passed as a parameter to
do_perf_event_open(). Otherwise, I wouldexpect to be able to call
Quoted 1 line
do_perf_event_open()from the main thread and pass anylcore_idof a
Quoted 8 lines
worker thread.
Quoted 1 line
This comment applies to all functions that must be called from the
worker thread itself. It also
Quoted 2 lines
applies to the functions that call such functions.
Lcore_idis being passed around so that we don't need to callrte_lcore_id()each and every time.
Please take a look at the rte_lcore_id() implementation. :-)
Regardless, my argument still stands: If a function cannot be called with the lcore_id parameter set to any valid lcore id, it should not be a parameter to the function.
Quoted 23 lines
Quoted 15 lines
[...]
Quoted 12 lines
lib/eal/include/rte_pmu.h new file +6
-0+/** +* A structure describing a group of events. +*/ +struct rte_pmu_event_group { +int fds[MAX_NUM_GROUP_EVENTS ];/**< array of event descriptors */ +struct perf_event_mmap_page *mmap_pages[MAX_NUM_GROUP_EVENTS ];/**< array of user pages */
(quoted diff) +1
-0+ bool enabled; /**< true if group was enabled on particular lcore*/
lib/eal/include/rte_pmu.h new file +3
-0+}; + +/** Quoted 8 lines
lib/eal/include/rte_pmu.h
Quoted 6 lines
+ * A structure describing an event.
+ */lib/eal/include/rte_pmu.h new file +4
-0+struct rte_pmu_event { +char *name;/** name of an event */ +unsigned int index;/** event index into fds/mmap_pages */ +TAILQ_ENTRY (rte_pmu_event) next;/** list entry */ };Move the "enabled" field up, making it the first field in this
Quoted 7 lines
structure. This might reduce the
Quoted 1 line
number of instructions required to check (!group->enabled) in
rte_pmu_read().Quoted 1 line
This will be called once and no this will not produce more instructions. Why should it?
It seems I was not clearly describing my intention here here. rte_pmu_read() a hot function, where the comparison "if (!group->enabled)" itself will be executed many times.
Quoted 2 lines
In both cases compiler will need to load data at some offset and archs do have instructions for that.
Yes, the instructions are: address = BASE + sizeof(struct rte_pmu_event_group) * lcore_id + offsetof(struct rte_pmu_event, enabled).
I meant you could avoid the extra instructions stemming from the addition: "+ offsetof()". But you are right... Both BASE and offsetof(struct rte_pmu_event, enabled) are known in advance, and can be merged at compile time to avoid the addition.
Quoted 9 lines
Quoted 2 lines
Also, each instance of the structure is used individually per lcore,
so the structure should be
Quoted 6 lines
cache line aligned to avoid unnecessarily crossing cache lines.
I.e.:
struct
rte_pmu_event_group{
bool enabled; /**< true if group was enabled on particular lcore
Quoted 8 lines
*/
Quoted 2 lines
int fds[MAX_NUM_GROUP_EVENTS]; /**< array of event descriptors */ struct perf_event_mmap_page *mmap_pages[MAX_NUM_GROUP_EVENTS];/**< array of user pages */ }
Quoted 1 line
__rte_cache_aligned;Yes, this can be aligned. While at it, I'd be more inclined to move
mmap_pagesup instead of enable.
Yes, moving up mmap_pages is better.
Quoted 108 lines
Quoted 13 lines
Quoted 13 lines
lib/eal/include/rte_pmu.h new file +6
-0+ +/** +* A PMU state container. +*/ +struct rte_pmu { +char *name;/** name of core PMU listed under
/sys/bus/event_source/devices*/(quoted diff) +1
-0+ struct rte_pmu_event_group group[RTE_MAX_LCORE]; /**< per lcoreevent group data */
(quoted diff) +1
-0+ unsigned int num_group_events; /**< number of events in a group*/
Quoted 9 lines
app/test/test_pmu.c
Quoted 6 lines
(quoted diff) +1
-0+ TAILQ_HEAD(, rte_pmu_event) event_list; /**< list of matchingevents */
app/test/test_pmu.c new file +4
-0+}; + +/** Pointer to the PMU state container */ extern struct rte_pmu +rte_pmu;Just "The PMU state container". It is not a pointer anymore. :-)
Good catch.
Quoted 20 lines
[...]
Quoted 18 lines
lib/eal/include/rte_pmu.h new file +18
-0+/** +* @internal +* +* Read PMU counter. +* +* @param pc +* Pointer to the mmapped user page. +* @return +* Counter value read from hardware. +*/ +__rte_internal +static __rte_always_inline uint64_t +rte_pmu_read_userpage (struct perf_event_mmap_page *pc) { +uint64_t width, offset; +uint32_t seq, index; +int64_t pmc; + +for (;;) {Quoted 8 lines
lib/eal/include/rte_pmu.h line 96
Quoted 6 lines
lib/eal/include/rte_pmu.h new file +6
-0+ seq = pc->lock; +rte_compiler_barrier (); + index = pc->index; + offset = pc->offset; + width = pc->pmc_width; +Please add a comment here about the special meaning of index == 0.
Okay.
Quoted 19 lines
Quoted 16 lines
lib/eal/include/rte_pmu.h new file +15
-0+if (likely (pc->cap_user_rdpmc && index)) { + pmc =rte_pmu_pmc_read (index -1 ); + pmc <<=64 - width; + pmc >>=64 - width; + offset += pmc; + } + +rte_compiler_barrier (); + +if (likely (pc->lock == seq)) +return offset; + } + +return 0 ; +}[...]
Quoted 5 lines
Quoted 5 lines
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Read hardware counter configured to count occurrences of anevent.
Quoted 17 lines
Quoted 17 lines
+ *
+ * @param index
+ * Index of an event to be read.
+ * @return
+ * Event value read from register. In case of errors or lack of
support
+ * 0 is returned. In other words, stream of zeros in a trace file
+ * indicates problem with reading particular PMU event register.
+ */lib/eal/include/rte_pmu.h new file +8
-0+__rte_experimental +static __rte_always_inline uint64_t +rte_pmu_read (unsigned int index) +{ +struct rte_pmu_event_group *group; +int ret, lcore_id =rte_lcore_id (); + + group = &rte_pmu.group[lcore_id];Quoted 9 lines
lib/eal/include/rte_pmu.h line 168
Quoted 6 lines
lib/eal/include/rte_pmu.h new file +6
-0+if (unlikely (!group->enabled)) { + ret =rte_pmu_enable_group (lcore_id); +if (ret) +return 0 ; + + group->enabled = true;
Group->enabledshould be set insiderte_pmu_enable_group(), not here.
Quoted 2 lines
This is easier to follow imo and not against coding guidelines so I prefer to leave it as is.
OK. It makes the rte_pmu_read() source code slightly shorter, but probably has zero effect on the generated code. No strong preference - feel free to follow your personal preference on this.
Quoted 10 lines
Quoted 9 lines
Quoted 8 lines
lib/eal/include/rte_pmu.h new file +7
-0+ } + +if (unlikely (index >= rte_pmu.num_group_events)) +return 0 ; + +return rte_pmu_read_userpage (group->mmap_pages[index]); +}
PATCH v5 2/4 eal/arm: support reading ARM PMU events in runtime
In reply to Tomasz DuszynskiAdd support for reading ARM PMU events in runtime.
--- app/test/test_pmu.c | 4 ++ lib/eal/arm/include/meson.build | 1 + lib/eal/arm/include/rte_pmu_pmc.h | 39 +++++++++++ lib/eal/arm/meson.build | 4 ++ lib/eal/arm/rte_pmu.c | 104 ++++++++++++++++++++++++++++++ lib/eal/include/rte_pmu.h | 3 + 6 files changed, 155 insertions(+) create mode 100644 lib/eal/arm/include/rte_pmu_pmc.h create mode 100644 lib/eal/arm/rte_pmu.c
app/test/test_pmu.c +4 -0
@@ -13,6 +13,10 @@ test_pmu_read(void)int tries =10 ;int event =-1 ; +#if defined(RTE_ARCH_ARM64 ) + event =rte_pmu_add_event ("cpu_cycles" ); +#endif +while (tries--) val +=rte_pmu_read (event);
lib/eal/arm/include/meson.build +1 -0
@@ -20,6 +20,7 @@ arch_headers = files( 'rte_pause_32.h', 'rte_pause_64.h', 'rte_pause.h', + 'rte_pmu_pmc.h', 'rte_power_intrinsics.h', 'rte_prefetch_32.h', 'rte_prefetch_64.h',
lib/eal/arm/include/rte_pmu_pmc.h new file +39 -0
@@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2023 Marvell. +*/ + +#ifndef _RTE_PMU_PMC_ARM_H_ +#define _RTE_PMU_PMC_ARM_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include <rte_common.h> + +static __rte_always_inline uint64_t +rte_pmu_pmc_read (int index) +{ +uint64_t val; + +if (index ==31 ) { +/* CPU Cycles (0x11) must be read via pmccntr_el0 */ + asmvolatile ("mrs %0, pmccntr_el0" :"=r" (val)); + }else { + asmvolatile ( +"msr pmselr_el0, %x0\n" +"mrs %0, pmxevcntr_el0\n" + :"=r" (val) + :"rZ" (index) + ); + } + +return val; +} +#define rte_pmu_pmc_read rte_pmu_pmc_read + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_PMU_PMC_ARM_H_ */
lib/eal/arm/meson.build +4 -0
@@ -9,3 +9,7 @@ sources += files( 'rte_hypervisor.c', 'rte_power_intrinsics.c', ) + +if is_linux + sources += files('rte_pmu.c') +endif
lib/eal/arm/rte_pmu.c new file +104 -0
104 lines not on this page · raw message
lib/eal/include/rte_pmu.h +3 -0
@@ -20,6 +20,9 @@ extern "C" {#include <rte_branch_prediction.h> #include <rte_lcore.h> #include <rte_log.h> +#if defined(RTE_ARCH_ARM64 ) +#include <rte_pmu_pmc.h> +#endif /** * @file
Signature · 1 line
2.34.1
PATCH v5 3/4 eal/x86: support reading Intel PMU events in runtime
In reply to Tomasz DuszynskiAdd support for reading Intel PMU events in runtime.
--- app/test/test_pmu.c | 2 ++ lib/eal/include/rte_pmu.h | 2 +- lib/eal/x86/include/meson.build | 1 + lib/eal/x86/include/rte_pmu_pmc.h | 33 +++++++++++++++++++++++++++++++ 4 files changed, 37 insertions(+), 1 deletion(-) create mode 100644 lib/eal/x86/include/rte_pmu_pmc.h
app/test/test_pmu.c +2 -0
@@ -15,6 +15,8 @@ test_pmu_read(void)#if defined(RTE_ARCH_ARM64 ) event =rte_pmu_add_event ("cpu_cycles" ); +#elif defined(RTE_ARCH_X86_64 ) + event =rte_pmu_add_event ("cpu-cycles" );#endif while (tries--)
lib/eal/include/rte_pmu.h +1 -1
@@ -20,7 +20,7 @@ extern "C" {#include <rte_branch_prediction.h> #include <rte_lcore.h> #include <rte_log.h> -#if defined(RTE_ARCH_ARM64 ) +#if defined(RTE_ARCH_ARM64 ) || defined(RTE_ARCH_X86_64 )#include <rte_pmu_pmc.h> #endif
lib/eal/x86/include/meson.build +1 -0
@@ -9,6 +9,7 @@ arch_headers = files( 'rte_io.h', 'rte_memcpy.h', 'rte_pause.h', + 'rte_pmu_pmc.h', 'rte_power_intrinsics.h', 'rte_prefetch.h', 'rte_rtm.h',
lib/eal/x86/include/rte_pmu_pmc.h new file +33 -0
@@ -0,0 +1,33 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2023 Marvell. +*/ + +#ifndef _RTE_PMU_PMC_X86_H_ +#define _RTE_PMU_PMC_X86_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include <rte_common.h> + +static __rte_always_inline uint64_t +rte_pmu_pmc_read (int index) +{ +uint64_t low, high; + + asmvolatile ( +"rdpmc\n" + :"=a" (low),"=d" (high) + :"c" (index) + ); + +return low | (high <<32 ); +} +#define rte_pmu_pmc_read rte_pmu_pmc_read + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_PMU_PMC_X86_H_ */
Signature · 1 line
2.34.1
PATCH v5 4/4 eal: add PMU support to tracing library
In reply to Tomasz DuszynskiIn order to profile app one needs to store significant amount of samples somewhere for an analysis latern on. Since trace library supports storing data in a CTF format lets take adventage of that and add a dedicated PMU tracepoint.
--- app/test/test_trace_perf.c | 4 ++ doc/guides/prog_guide/profile_app.rst | 5 ++ doc/guides/prog_guide/trace_lib.rst | 32 +++++++++++ lib/eal/common/eal_common_trace_points.c | 3 + lib/eal/common/rte_pmu.c | 73 +++++++++++++++++++++++- lib/eal/include/rte_eal_trace.h | 10 ++++ lib/eal/version.map | 1 + 7 files changed, 126 insertions(+), 2 deletions(-)
app/test/test_trace_perf.c +4 -0
@@ -114,6 +114,8 @@ worker_fn_##func(void *arg) \#define GENERIC_DOUBLE rte_eal_trace_generic_double(3.66666)#define GENERIC_STR rte_eal_trace_generic_str("hello world")#define VOID_FP app_dpdk_test_fp() +/* 0 corresponds first event passed via --trace= */ +#define READ_PMU rte_eal_trace_pmu_read(0)WORKER_DEFINE (GENERIC_VOID)WORKER_DEFINE (GENERIC_U64)
@@ -122,6 +124,7 @@ WORKER_DEFINE(GENERIC_FLOAT) WORKER_DEFINE(GENERIC_DOUBLE) WORKER_DEFINE(GENERIC_STR) WORKER_DEFINE(VOID_FP) +WORKER_DEFINE(READ_PMU) static void run_test(const char *str, lcore_function_t f, struct test_data *data, size_t sz)
@@ -174,6 +177,7 @@ test_trace_perf(void)run_test ("double" , worker_fn_GENERIC_DOUBLE, data, sz);run_test ("string" , worker_fn_GENERIC_STR, data, sz);run_test ("void_fp" , worker_fn_VOID_FP, data, sz); +run_test ("read_pmu" , worker_fn_READ_PMU, data, sz);rte_free (data);return TEST_SUCCESS ;
doc/guides/prog_guide/profile_app.rst +5 -0
@@ -16,6 +16,11 @@ that information, perf being an example here. Though in some scenarios, eg. when isolated (nohz_full) and run dedicated tasks, using perf is less than ideal. In such cases one can read specific events directly from application via ``rte_pmu_read()``. +Alternatively tracing library can be used which offers dedicated tracepoint +``rte_eal_trace_pmu_event()``. + +Refer to :doc:`../prog_guide/trace_lib` for more details. + Profiling on x86 ----------------
doc/guides/prog_guide/trace_lib.rst +32 -0
@@ -46,6 +46,7 @@ DPDK tracing library features trace format and is compatible with ``LTTng``. For detailed information, refer to `Common Trace Format <https://diamon.org/ctf/>`_. +- Support reading PMU events on ARM64 and x86 (Intel) How to add a tracepoint? ------------------------
@@ -137,6 +138,37 @@ the user must use ``RTE_TRACE_POINT_FP`` instead of ``RTE_TRACE_POINT``. ``RTE_TRACE_POINT_FP`` is compiled out by default and it can be enabled using the ``enable_trace_fp`` option for meson build. +PMU tracepoint +-------------- + +Performance measurement unit (PMU) event values can be read from hardware +registers using predefined ``rte_pmu_read`` tracepoint. + +Tracing is enabled via ``--trace`` EAL option by passing both expression +matching PMU tracepoint name i.e ``lib.eal.pmu.read`` and expression +``e=ev1[,ev2,...]`` matching particular events:: + + --trace='.*pmu.read\|e=cpu_cycles,l1d_cache' + +Event names are available under ``/sys/bus/event_source/devices/PMU/events`` +directory, where ``PMU`` is a placeholder for either a ``cpu`` or a directory +containing ``cpus``. + +In contrary to other tracepoints this does not need any extra variables +added to source files. Instead, caller passes index which follows the order of +events specified via ``--trace`` parameter. In the following example index ``0`` +corresponds to ``cpu_cyclces`` while index ``1`` corresponds to ``l1d_cache``. + +.. code-block:: c + + ... + rte_eal_trace_pmu_read(0); + rte_eal_trace_pmu_read(1); + ... + +PMU tracing support must be explicitly enabled using the ``enable_trace_fp`` +option for meson build. + Event record mode -----------------
lib/eal/common/eal_common_trace_points.c +3 -0
@@ -75,3 +75,6 @@ RTE_TRACE_POINT_REGISTER(rte_eal_trace_intr_enable, lib.eal.intr.enable) RTE_TRACE_POINT_REGISTER(rte_eal_trace_intr_disable, lib.eal.intr.disable) + +RTE_TRACE_POINT_REGISTER (rte_eal_trace_pmu_read, + lib.eal.pmu.read)
lib/eal/common/rte_pmu.c +71 -2
111 lines not on this page · raw message
lib/eal/include/rte_eal_trace.h +10 -0
@@ -17,6 +17,7 @@ extern "C" {#include <rte_alarm.h> #include <rte_interrupts.h> +#include <rte_pmu.h> #include <rte_trace_point.h> #include "eal_interrupts.h"
@@ -279,6 +280,15 @@ RTE_TRACE_POINT(rte_trace_point_emit_string (cpuset); ) +/* PMU */ +RTE_TRACE_POINT_FP ( + rte_eal_trace_pmu_read, + RTE_TRACE_POINT_ARGS(unsigned int index), + uint64_t val; + val = rte_pmu_read(index); + rte_trace_point_emit_u64(val); +) + #ifdef __cplusplus }#endif
lib/eal/version.map +1 -0
@@ -442,6 +442,7 @@ EXPERIMENTAL { rte_thread_join; # added in 23.03 + __rte_eal_trace_pmu_read; # WINDOWS_NO_EXPORT rte_pmu; # WINDOWS_NO_EXPORT rte_pmu_add_event; # WINDOWS_NO_EXPORT rte_pmu_read; # WINDOWS_NO_EXPORT
Signature · 1 line
2.34.1
hi,
don't interpret this as an objection to the functionality but this looks like a clear example of something that doesn't belong in the EAL. has there been a discussion as to whether or not this should be in a separate library?
a basic test is whether or not an implementation exists or can be reasonably provided for all platforms and that isn't strictly evident here. red flag is to see yet more code being added conditionally compiled for a single platform.
Morten, Bruce comments?
thanks
Quoted 69 lines · Tomasz Duszynski
This series adds self monitoring support i.e allows to configure and read performance measurement unit (PMU) counters in runtime without using perf utility. This has certain adventages when application runs on isolated cores with
nohz_fullkernel parameter.Events can be read directly using
rte_pmu_read()or using dedicated tracepointrte_eal_trace_pmu_read(). The latter will cause events to be stored inside CTF file.By design, all enabled events are grouped together and the same group is attached to lcores that use self monitoring funtionality.
Events are enabled by names, which need to be read from standard location under sysfs i.e
/sys/bus/event_source/devices/PMU/eventswhere PMU is a core pmu i.e one measuring cpu events. As of today raw events are not supported.
v5:
- address review comments
- fix sign extension while reading pmu on x86
- fix regex mentioned in doc
- various minor changes/improvements here and there v4:
- fix freeing mem detected bydebug_autotestv3:
- fix shared build
v2:
- fix problems reported by test build infraTomasz Duszynski (4):
eal: add generic support for reading PMU events eal/arm: support reading ARM PMU events in runtime eal/x86: support reading Intel PMU events in runtime
eal: add PMU support to tracing libraryapp/test/meson.build | 1 + app/test/test_pmu.c | 47 +++ app/test/test_trace_perf.c | 4 + doc/guides/prog_guide/profile_app.rst | 13 + doc/guides/prog_guide/trace_lib.rst | 32 ++ lib/eal/arm/include/meson.build | 1 + lib/eal/arm/include/rte_pmu_pmc.h | 39 ++ lib/eal/arm/meson.build | 4 + lib/eal/arm/rte_pmu.c | 104 +++++ lib/eal/common/eal_common_trace_points.c | 3 + lib/eal/common/meson.build | 3 + lib/eal/common/pmu_private.h | 41 ++ lib/eal/common/rte_pmu.c | 504 +++++++++++++++++++++++ lib/eal/include/meson.build | 1 + lib/eal/include/rte_eal_trace.h | 10 + lib/eal/include/rte_pmu.h | 202 +++++++++ lib/eal/linux/eal.c | 4 + lib/eal/version.map | 7 + lib/eal/x86/include/meson.build | 1 + lib/eal/x86/include/rte_pmu_pmc.h | 33 ++ 20 files changed, 1054 insertions(+) create mode 100644 app/test/test_pmu.c create mode 100644 lib/eal/arm/include/rte_pmu_pmc.h create mode 100644 lib/eal/arm/rte_pmu.c create mode 100644 lib/eal/common/pmu_private.h create mode 100644 lib/eal/common/rte_pmu.c create mode 100644 lib/eal/include/rte_pmu.h create mode 100644 lib/eal/x86/include/rte_pmu_pmc.h--
2.34.1
Quoted 57 lines
From: Tyler Retzlaff [mailto:[email protected]]
Sent: Wednesday, 11 January 2023 01.32Quoted 3 lines · Tomasz Duszynski
This series adds self monitoring support i.e allows to configure and read performance measurement unit (PMU) counters in runtime without using perf utility. This has certain adventages when application runs
on
Quoted 4 lines
isolated cores with
nohz_fullkernel parameter.Events can be read directly using
rte_pmu_read()or using dedicated tracepointrte_eal_trace_pmu_read(). The latter will cause events tobe
Quoted 31 lines
stored inside CTF file.
By design, all enabled events are grouped together and the same group is attached to lcores that use self monitoring funtionality.
Events are enabled by names, which need to be read from standard location under sysfs i.e
/sys/bus/event_source/devices/PMU/eventswhere PMU is a core pmu i.e one measuring cpu events. As of today raw events are not supported.
v5:
- address review comments
- fix sign extension while reading pmu on x86
- fix regex mentioned in doc
- various minor changes/improvements here and there v4:
- fix freeing mem detected bydebug_autotestv3:
- fix shared build
v2:
- fix problems reported by test build infraTomasz Duszynski (4):
eal: add generic support for reading PMU events eal/arm: support reading ARM PMU events in runtime eal/x86: support reading Intel PMU events in runtime
eal: add PMU support to tracing libraryQuoted 13 lines
app/test/meson.build | 1 + app/test/test_pmu.c | 47 +++ app/test/test_trace_perf.c | 4 + doc/guides/prog_guide/profile_app.rst | 13 + doc/guides/prog_guide/trace_lib.rst | 32 ++ lib/eal/arm/include/meson.build | 1 + lib/eal/arm/include/rte_pmu_pmc.h | 39 ++ lib/eal/arm/meson.build | 4 + lib/eal/arm/rte_pmu.c | 104 +++++ lib/eal/common/eal_common_trace_points.c | 3 + lib/eal/common/meson.build | 3 + lib/eal/common/pmu_private.h | 41 ++ lib/eal/common/rte_pmu.c | 504
Quoted 19 lines
+++++++++++++++++++++++
Quoted 18 lines
lib/eal/include/meson.build | 1 + lib/eal/include/rte_eal_trace.h | 10 + lib/eal/include/rte_pmu.h | 202 +++++++++ lib/eal/linux/eal.c | 4 + lib/eal/version.map | 7 + lib/eal/x86/include/meson.build | 1 + lib/eal/x86/include/rte_pmu_pmc.h | 33 ++ 20 files changed, 1054 insertions(+) create mode 100644 app/test/test_pmu.c create mode 100644 lib/eal/arm/include/rte_pmu_pmc.h create mode 100644 lib/eal/arm/rte_pmu.c create mode 100644 lib/eal/common/pmu_private.h create mode 100644 lib/eal/common/rte_pmu.c create mode 100644 lib/eal/include/rte_pmu.h create mode 100644 lib/eal/x86/include/rte_pmu_pmc.h--
2.34.1
[Moved Tyler's post down here.]
Quoted 8 lines
hi,
don't interpret this as an objection to the functionality but this looks
like a clear example of something that doesn't belong in the EAL. has there been a discussion as to whether or not this should be in a separate library?
IIRC, there has been no such discussion.
Although I agree that this doesn't belong in EAL, I would point to the trace library as a reference for allowing it into the EAL.
For the records, I also oppose to the trace library being part of the EAL.
On the other hand, it would be interesting to determine if it is *impossible* adding this functionality as any other normal DPDK library, i.e. outside of the EAL, or if there is an unavoidable tie-in to the EAL.
@Tomasz, if this is impossible, please describe the unavoidable tie-in to the EAL. No need for a long report, just a few words. You (and this functionality) shouldn't suffer from our long term ambition to move stuff out of the EAL.
Quoted 5 lines
a basic test is whether or not an implementation exists or can be reasonably provided for all platforms and that isn't strictly evident here. red flag is to see yet more code being added conditionally compiled for a single platform.
Another basic test: Can DPDK applications run without it? If they can, an Environment Abstraction Layer does not need to have it, and thus it does not need to be part of the EAL.
Quoted 4 lines
Morten, Bruce comments?
thanks
Quoted 109 lines
-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: Wednesday, January 11, 2023 10:31 AM
To: Tyler Retzlaff <[email protected]>; Tomasz Duszynski <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; [email protected]; [email protected]
Subject: [EXT] RE: [PATCH v5 0/4] add support for self monitoringExternal Email
----------------------------------------------------------------------
Quoted 58 lines
From: Tyler Retzlaff [mailto:[email protected]]
Sent: Wednesday, 11 January 2023 01.32Quoted 4 lines · Tomasz Duszynski
This series adds self monitoring support i.e allows to configure and read performance measurement unit (PMU) counters in runtime without using perf utility. This has certain adventages when application runs
on
Quoted 4 lines
isolated cores with
nohz_fullkernel parameter.Events can be read directly using
rte_pmu_read()or using dedicated tracepointrte_eal_trace_pmu_read(). The latter will cause events tobe
Quoted 31 lines
stored inside CTF file.
By design, all enabled events are grouped together and the same group is attached to lcores that use self monitoring funtionality.
Events are enabled by names, which need to be read from standard location under sysfs i.e
/sys/bus/event_source/devices/PMU/eventswhere PMU is a core pmu i.e one measuring cpu events. As of today raw events are not supported.
v5:
- address review comments
- fix sign extension while reading pmu on x86
- fix regex mentioned in doc
- various minor changes/improvements here and there v4:
- fix freeing mem detected bydebug_autotestv3:
- fix shared build
v2:
- fix problems reported by test build infraTomasz Duszynski (4):
eal: add generic support for reading PMU events eal/arm: support reading ARM PMU events in runtime eal/x86: support reading Intel PMU events in runtime
eal: add PMU support to tracing libraryQuoted 13 lines
app/test/meson.build | 1 + app/test/test_pmu.c | 47 +++ app/test/test_trace_perf.c | 4 + doc/guides/prog_guide/profile_app.rst | 13 + doc/guides/prog_guide/trace_lib.rst | 32 ++ lib/eal/arm/include/meson.build | 1 + lib/eal/arm/include/rte_pmu_pmc.h | 39 ++ lib/eal/arm/meson.build | 4 + lib/eal/arm/rte_pmu.c | 104 +++++ lib/eal/common/eal_common_trace_points.c | 3 + lib/eal/common/meson.build | 3 + lib/eal/common/pmu_private.h | 41 ++ lib/eal/common/rte_pmu.c | 504Quoted 19 lines
+++++++++++++++++++++++
Quoted 18 lines
lib/eal/include/meson.build | 1 + lib/eal/include/rte_eal_trace.h | 10 + lib/eal/include/rte_pmu.h | 202 +++++++++ lib/eal/linux/eal.c | 4 + lib/eal/version.map | 7 + lib/eal/x86/include/meson.build | 1 + lib/eal/x86/include/rte_pmu_pmc.h | 33 ++ 20 files changed, 1054 insertions(+) create mode 100644
app/test/test_pmu.ccreate mode 100644lib/eal/arm/include/rte_pmu_pmc.hcreate mode 100644lib/eal/arm/rte_pmu.ccreate mode 100644lib/eal/common/pmu_private.hcreate mode 100644lib/eal/common/rte_pmu.ccreate mode 100644lib/eal/include/rte_pmu.hcreate mode 100644lib/eal/x86/include/rte_pmu_pmc.h--
2.34.1[Moved Tyler's post down here.]
Quoted 7 lines
hi,
don't interpret this as an objection to the functionality but this looks like a clear example of something that doesn't belong in the EAL. has there been a discussion as to whether or not this should be in a separate library?
IIRC, there has been no such discussion.
Although I agree that this doesn't belong in EAL, I would point to the trace library as a reference for allowing it into the EAL.
For the records, I also oppose to the trace library being part of the EAL.
Quoted 8 lines
On the other hand, it would be interesting to determine if it is *impossible* adding this functionality as any other normal DPDK library, i.e. outside of the EAL, or if there is an unavoidable tie-in to the EAL.
@Tomasz, if this is impossible, please describe the unavoidable tie-in to the EAL. No need for a long report, just a few words. You (and this functionality) shouldn't suffer from our long term ambition to move stuff out of the EAL.
You can read about rationale here https://lore.kernel.org/dpdk-dev/DM4PR18MB436872EBC5922084C5DAFC1DD2FC9@DM4PR18MB4368.namprd18.prod.outlook.com/#t
As for the NO-NO there isn't any in fact. There are some tradeoffs though.
For example, seems eal cannot depend on other libs so if someone needs to finetune some part of EAL for whatever reason, then relevant part needs to modified each and every time. I.e specific includes and trcepoints need to be added each time.
On the other hand, if this is coupled with eal then adding tracepoints to some parts will be easier. Or they can just be added to specific points and live there.
No strong opinions besides that. I'd like to know what others think.
Quoted 13 lines
Quoted 5 lines
a basic test is whether or not an implementation exists or can be reasonably provided for all platforms and that isn't strictly evident here. red flag is to see yet more code being added conditionally compiled for a single platform.
Another basic test: Can DPDK applications run without it? If they can, an Environment Abstraction Layer does not need to have it, and thus it does not need to be part of the EAL.
Quoted 4 lines
Morten, Bruce comments?
thanks
Quoted 118 lines · Tomasz Duszynski
Quoted 109 lines
-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: Wednesday, January 11, 2023 10:31 AM
To: Tyler Retzlaff <[email protected]>; Tomasz Duszynski <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; [email protected]; [email protected]
Subject: [EXT] RE: [PATCH v5 0/4] add support for self monitoringExternal Email
----------------------------------------------------------------------
Quoted 58 lines
From: Tyler Retzlaff [mailto:[email protected]]
Sent: Wednesday, 11 January 2023 01.32Quoted 4 lines · Tomasz Duszynski
This series adds self monitoring support i.e allows to configure and read performance measurement unit (PMU) counters in runtime without using perf utility. This has certain adventages when application runs
on
Quoted 4 lines
isolated cores with
nohz_fullkernel parameter.Events can be read directly using
rte_pmu_read()or using dedicated tracepointrte_eal_trace_pmu_read(). The latter will cause events tobe
Quoted 31 lines
stored inside CTF file.
By design, all enabled events are grouped together and the same group is attached to lcores that use self monitoring funtionality.
Events are enabled by names, which need to be read from standard location under sysfs i.e
/sys/bus/event_source/devices/PMU/eventswhere PMU is a core pmu i.e one measuring cpu events. As of today raw events are not supported.
v5:
- address review comments
- fix sign extension while reading pmu on x86
- fix regex mentioned in doc
- various minor changes/improvements here and there v4:
- fix freeing mem detected bydebug_autotestv3:
- fix shared build
v2:
- fix problems reported by test build infraTomasz Duszynski (4):
eal: add generic support for reading PMU events eal/arm: support reading ARM PMU events in runtime eal/x86: support reading Intel PMU events in runtime
eal: add PMU support to tracing libraryQuoted 13 lines
app/test/meson.build | 1 + app/test/test_pmu.c | 47 +++ app/test/test_trace_perf.c | 4 + doc/guides/prog_guide/profile_app.rst | 13 + doc/guides/prog_guide/trace_lib.rst | 32 ++ lib/eal/arm/include/meson.build | 1 + lib/eal/arm/include/rte_pmu_pmc.h | 39 ++ lib/eal/arm/meson.build | 4 + lib/eal/arm/rte_pmu.c | 104 +++++ lib/eal/common/eal_common_trace_points.c | 3 + lib/eal/common/meson.build | 3 + lib/eal/common/pmu_private.h | 41 ++ lib/eal/common/rte_pmu.c | 504Quoted 19 lines
+++++++++++++++++++++++
Quoted 18 lines
lib/eal/include/meson.build | 1 + lib/eal/include/rte_eal_trace.h | 10 + lib/eal/include/rte_pmu.h | 202 +++++++++ lib/eal/linux/eal.c | 4 + lib/eal/version.map | 7 + lib/eal/x86/include/meson.build | 1 + lib/eal/x86/include/rte_pmu_pmc.h | 33 ++ 20 files changed, 1054 insertions(+) create mode 100644
app/test/test_pmu.ccreate mode 100644lib/eal/arm/include/rte_pmu_pmc.hcreate mode 100644lib/eal/arm/rte_pmu.ccreate mode 100644lib/eal/common/pmu_private.hcreate mode 100644lib/eal/common/rte_pmu.ccreate mode 100644lib/eal/include/rte_pmu.hcreate mode 100644lib/eal/x86/include/rte_pmu_pmc.h--
2.34.1[Moved Tyler's post down here.]
Quoted 7 lines
hi,
don't interpret this as an objection to the functionality but this looks like a clear example of something that doesn't belong in the EAL. has there been a discussion as to whether or not this should be in a separate library?
IIRC, there has been no such discussion.
Although I agree that this doesn't belong in EAL, I would point to the trace library as a reference for allowing it into the EAL.
For the records, I also oppose to the trace library being part of the EAL.
Quoted 8 lines
On the other hand, it would be interesting to determine if it is *impossible* adding this functionality as any other normal DPDK library, i.e. outside of the EAL, or if there is an unavoidable tie-in to the EAL.
@Tomasz, if this is impossible, please describe the unavoidable tie-in to the EAL. No need for a long report, just a few words. You (and this functionality) shouldn't suffer from our long term ambition to move stuff out of the EAL.
Quoted 8 lines
You can read about rationale here https://lore.kernel.org/dpdk-dev/DM4PR18MB436872EBC5922084C5DAFC1DD2FC9@DM4PR18MB4368.namprd18.prod.outlook.com/#t
As for the NO-NO there isn't any in fact. There are some tradeoffs though.
For example, seems eal cannot depend on other libs so if someone needs to finetune some part of EAL for whatever reason, then relevant part needs to modified each and every time. I.e specific includes and trcepoints need to be added each time.
Well, EAL can depend on other libs, but then those libs cannot in turn directly depend upon DPDK. This is where breaking out first some of the smaller widely used parts of DPDK e.g. logging, would be good, as it would then in turn allow other, potentially bigger parts of EAL to be taken out.
See [1] for a rough first attempt at this, which allows simlification of telemetry as it no longer needs a "dependency injection" style to have logging. Moving out logging would also allow logging from kvargs library too - another lib which is used by EAL rather than depending on it. Similarly for tracing functionality - if that were pulled out of EAL, it could be used by telemetry, kvargs and any other parts removed from EAL.
/Bruce
Hi Tyler,
Quoted 8 lines
-----Original Message-----
From: Tyler Retzlaff <[email protected]>
Sent: Wednesday, January 11, 2023 1:32 AM
To: Tomasz Duszynski <[email protected]>; [email protected]; [email protected]
Cc: [email protected]; [email protected]; Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [EXT] Re: [PATCH v5 0/4] add support for self monitoring
Quoted 8 lines
External Email
----------------------------------------------------------------------
hi,don't interpret this as an objection to the functionality but this looks like a clear example of something that doesn't belong in the EAL. has there been a discussion as to whether or not this should be in a separate library?
No, I don't recall anybody having any concerns about the code placement. Rationale behind making this part of eal was based on the fact that tracing itself is a part of eal and since this was meant to be extension to tracing, code placement decision came out naturally.
During development phase idea evolved a bit and what initially was supposed to be solely yet another tracepoint become generic API to read pmu and tracepoint based on that. Which means both can be used independently.
That said, since this code has both platform agnostic and platform specific parts this can either be split into:
1. library + eal platform code
2. all under eal
Either approach seems legit. Thoughts?
Quoted 4 lines
a basic test is whether or not an implementation exists or can be reasonably provided for all platforms and that isn't strictly evident here. red flag is to see yet more code being added conditionally compiled for a single platform.
Even libs are not entirely pristine and have platform specific ifdefs lurking so not sure where this red flag is coming from.
Quoted 74 lines
Morten, Bruce comments?
thanks
Quoted 68 lines · Tomasz Duszynski
This series adds self monitoring support i.e allows to configure and read performance measurement unit (PMU) counters in runtime without using perf utility. This has certain adventages when application runs on isolated cores with
nohz_fullkernel parameter.Events can be read directly using
rte_pmu_read()or using dedicated tracepointrte_eal_trace_pmu_read(). The latter will cause events to be stored inside CTF file.By design, all enabled events are grouped together and the same group is attached to lcores that use self monitoring funtionality.
Events are enabled by names, which need to be read from standard location under sysfs i.e
/sys/bus/event_source/devices/PMU/eventswhere PMU is a core pmu i.e one measuring cpu events. As of today raw events are not supported.
v5:
- address review comments
- fix sign extension while reading pmu on x86
- fix regex mentioned in doc
- various minor changes/improvements here and there v4:
- fix freeing mem detected bydebug_autotestv3:
- fix shared build
v2:
- fix problems reported by test build infraTomasz Duszynski (4):
eal: add generic support for reading PMU events eal/arm: support reading ARM PMU events in runtime eal/x86: support reading Intel PMU events in runtime
eal: add PMU support to tracing libraryapp/test/meson.build | 1 + app/test/test_pmu.c | 47 +++ app/test/test_trace_perf.c | 4 + doc/guides/prog_guide/profile_app.rst | 13 + doc/guides/prog_guide/trace_lib.rst | 32 ++ lib/eal/arm/include/meson.build | 1 + lib/eal/arm/include/rte_pmu_pmc.h | 39 ++ lib/eal/arm/meson.build | 4 + lib/eal/arm/rte_pmu.c | 104 +++++ lib/eal/common/eal_common_trace_points.c | 3 + lib/eal/common/meson.build | 3 + lib/eal/common/pmu_private.h | 41 ++ lib/eal/common/rte_pmu.c | 504 +++++++++++++++++++++++ lib/eal/include/meson.build | 1 + lib/eal/include/rte_eal_trace.h | 10 + lib/eal/include/rte_pmu.h | 202 +++++++++ lib/eal/linux/eal.c | 4 + lib/eal/version.map | 7 + lib/eal/x86/include/meson.build | 1 + lib/eal/x86/include/rte_pmu_pmc.h | 33 ++ 20 files changed, 1054 insertions(+) create mode 100644 app/test/test_pmu.c create mode 100644
lib/eal/arm/include/rte_pmu_pmc.hcreate mode 100644lib/eal/arm/rte_pmu.ccreate mode 100644lib/eal/common/pmu_private.hcreate mode 100644lib/eal/common/rte_pmu.ccreate mode 100644lib/eal/include/rte_pmu.hcreate mode 100644lib/eal/x86/include/rte_pmu_pmc.h--
2.34.1
Quoted 34 lines · Tomasz Duszynski
Hi Tyler,
Quoted 8 lines
-----Original Message-----
From: Tyler Retzlaff <[email protected]>
Sent: Wednesday, January 11, 2023 1:32 AM
To: Tomasz Duszynski <[email protected]>; [email protected]; [email protected]
Cc: [email protected]; [email protected]; Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [EXT] Re: [PATCH v5 0/4] add support for self monitoringQuoted 8 lines
External Email
----------------------------------------------------------------------
hi,don't interpret this as an objection to the functionality but this looks like a clear example of something that doesn't belong in the EAL. has there been a discussion as to whether or not this should be in a separate library?
No, I don't recall anybody having any concerns about the code placement. Rationale behind making this part of eal was based on the fact that tracing itself is a part of eal and since this was meant to be extension to tracing, code placement decision came out naturally.
During development phase idea evolved a bit and what initially was supposed to be solely yet another tracepoint become generic API to read pmu and tracepoint based on that. Which means both can be used independently.
That said, since this code has both platform agnostic and platform specific parts this can either be split into:
1. library + eal platform code
2. all under ealEither approach seems legit. Thoughts?
Quoted 1 line
Quoted 6 lines
Quoted 3 lines
a basic test is whether or not an implementation exists or can be reasonably provided for all platforms and that isn't strictly evident here. red flag is to see yet more code being added conditionally compiled for a single platform.
Even libs are not entirely pristine and have platform specific ifdefs lurking so not sure where this red flag is coming from.
i think red flag was probably the wrong term to use sorry for that. rather i should say it is an indicator that the api probably doesn't belong in the eal.
fundamentally the purpose of the abstraction library is to relieve the application from having to do conditional compilation and/or execution for the subject apis coming from eal. including and exporting apis that work for only one platform is in direct contradiction.
please explore adding this as a separate library, it is understood that there are tradeoffs involved.
thanks!
Quoted 61 lines
-----Original Message-----
From: Tyler Retzlaff <[email protected]>
Sent: Wednesday, January 11, 2023 10:06 PM
To: Tomasz Duszynski <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; [email protected]; [email protected]
Subject: Re: [EXT] Re: [PATCH v5 0/4] add support for self monitoringQuoted 18 lines · Tomasz Duszynski
Hi Tyler,
Quoted 16 lines
-----Original Message-----
From: Tyler Retzlaff <[email protected]>
Sent: Wednesday, January 11, 2023 1:32 AM
To: Tomasz Duszynski <[email protected]>; [email protected]; [email protected]
Cc: [email protected]; [email protected]; Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [EXT] Re: [PATCH v5 0/4] add support for self monitoringExternal Email
---------------------------------------------------------------------
-
hi,Quoted 7 lines
Quoted 3 lines
don't interpret this as an objection to the functionality but this
looks like a clear example of something that doesn't belong in the
EAL. has there been a discussion as to whether or not this should be in a separate library?No, I don't recall anybody having any concerns about the code
placement. Rationale behind making this part of eal was based on the
fact that tracing itself is a part of eal and since this was meant to be extension to tracing,code placement decision came out naturally.
Quoted 6 lines
During development phase idea evolved a bit and what initially was supposed to be solely yet another tracepoint become generic API to read pmu and tracepoint based on that. Which means both can be used independently.
That said, since this code has both platform agnostic and platform specific parts this can either
be split into:
Quoted 6 lines
1. library + eal platform code
2. all under ealEither approach seems legit. Thoughts?
Quoted 1 line
Quoted 6 lines
Quoted 3 lines
a basic test is whether or not an implementation exists or can be
reasonably provided for all platforms and that isn't strictly evident
here. red flag is to see yet more code being added conditionally compiled for a single platform.Even libs are not entirely pristine and have platform specific ifdefs lurking so not sure where this red flag is coming from.
i think red flag was probably the wrong term to use sorry for that. rather i should say it is an indicator that the api probably doesn't belong in the eal.
Quoted 8 lines
fundamentally the purpose of the abstraction library is to relieve the application from having to do conditional compilation and/or execution for the subject apis coming from eal. including and exporting apis that work for only one platform is in direct contradiction.
please explore adding this as a separate library, it is understood that there are tradeoffs involved.
thanks!
Any ideas how to name the library?
Quoted 65 lines · Tomasz Duszynski
Quoted 65 lines
-----Original Message-----
From: Tyler Retzlaff <[email protected]>
Sent: Wednesday, January 11, 2023 10:06 PM
To: Tomasz Duszynski <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; [email protected]; [email protected]
Subject: Re: [EXT] Re: [PATCH v5 0/4] add support for self monitoringQuoted 18 lines · Tomasz Duszynski
Hi Tyler,
Quoted 16 lines
-----Original Message-----
From: Tyler Retzlaff <[email protected]>
Sent: Wednesday, January 11, 2023 1:32 AM
To: Tomasz Duszynski <[email protected]>; [email protected]; [email protected]
Cc: [email protected]; [email protected]; Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [EXT] Re: [PATCH v5 0/4] add support for self monitoringExternal Email
---------------------------------------------------------------------
-
hi,Quoted 7 lines
Quoted 3 lines
don't interpret this as an objection to the functionality but this
looks like a clear example of something that doesn't belong in the
EAL. has there been a discussion as to whether or not this should be in a separate library?No, I don't recall anybody having any concerns about the code
placement. Rationale behind making this part of eal was based on the
fact that tracing itself is a part of eal and since this was meant to be extension to tracing,code placement decision came out naturally.
Quoted 6 lines
During development phase idea evolved a bit and what initially was supposed to be solely yet another tracepoint become generic API to read pmu and tracepoint based on that. Which means both can be used independently.
That said, since this code has both platform agnostic and platform specific parts this can either
be split into:
Quoted 6 lines
1. library + eal platform code
2. all under ealEither approach seems legit. Thoughts?
Quoted 1 line
Quoted 6 lines
Quoted 3 lines
a basic test is whether or not an implementation exists or can be
reasonably provided for all platforms and that isn't strictly evident
here. red flag is to see yet more code being added conditionally compiled for a single platform.Even libs are not entirely pristine and have platform specific ifdefs lurking so not sure where this red flag is coming from.
i think red flag was probably the wrong term to use sorry for that. rather i should say it is an indicator that the api probably doesn't belong in the eal.
fundamentally the purpose of the abstraction library is to relieve the application from having to do conditional compilation and/or execution for the subject apis coming from eal. including and exporting apis that work for only one platform is in direct contradiction.
Quoted 6 lines
Quoted 4 lines
please explore adding this as a separate library, it is understood that there are tradeoffs involved.
thanks!
Any ideas how to name the library?
naming is always so hard and i'm definitely not authoritative.
it seems like lib/pmu would be the least churn against your existing patch series, here are some other suggestions that might work.
lib/pmu (measuring unit)
lib/pmc (measuring counters)
lib/pcq (counter query)
lib/pmq (measuring query)
Quoted 70 lines
From: Tyler Retzlaff [mailto:[email protected]]
Sent: Friday, 13 January 2023 20.22Quoted 5 lines · Tomasz Duszynski
Quoted 5 lines
From: Tyler Retzlaff <[email protected]>
Sent: Wednesday, January 11, 2023 10:06 PMOn Wed, Jan 11, 2023 at 09:39:35AM +0000, Tomasz Duszynski wrote:
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Hi Tyler,
Quoted 6 lines
From: Tyler Retzlaff <[email protected]>
Sent: Wednesday, January 11, 2023 1:32 AMhi,
don't interpret this as an objection to the functionality but
this
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
looks like a clear example of something that doesn't belong in
the
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
EAL. has there been a discussion as to whether or not this should
be in a separate library?
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
No, I don't recall anybody having any concerns about the code placement. Rationale behind making this part of eal was based on
the
Quoted 1 line
Quoted 1 line
Quoted 1 line
fact that tracing itself is a part of eal and since this was meant
to be extension to tracing,
Quoted 5 lines
Quoted 5 lines
code placement decision came out naturally.
Quoted 4 lines
During development phase idea evolved a bit and what initially was supposed to be solely yet another tracepoint become generic API to read pmu and tracepoint based on that. Which means both can be
used independently.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
That said, since this code has both platform agnostic and platform
specific parts this can either
Quoted 9 lines
Quoted 9 lines
be split into:
Quoted 8 lines
1. library + eal platform code
2. all under ealEither approach seems legit. Thoughts?
Quoted 3 lines
a basic test is whether or not an implementation exists or can be reasonably provided for all platforms and that isn't strictly
evident
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
here. red flag is to see yet more code being added conditionally
compiled for a single platform.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Even libs are not entirely pristine and have platform specific
ifdefs
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
lurking so not sure where this red flag is coming from.
i think red flag was probably the wrong term to use sorry for that. rather i should say it is an indicator that the api probably doesn't
belong in the eal.
Quoted 2 lines
Quoted 2 lines
fundamentally the purpose of the abstraction library is to relieve
the application from having to
Quoted 1 line
Quoted 1 line
do conditional compilation and/or execution for the subject apis
coming from eal. including and
Quoted 1 line
Quoted 1 line
exporting apis that work for only one platform is in direct
contradiction.
Quoted 2 lines
Quoted 2 lines
please explore adding this as a separate library, it is understood
that there are tradeoffs
Quoted 2 lines
Quoted 2 lines
involved.
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
thanks!
Any ideas how to name the library?
naming is always so hard and i'm definitely not authoritative.
it seems like lib/pmu would be the least churn against your existing patch series, here are some other suggestions that might work.
+1 to lib/pmu
Less work, as Tyler already mentioned. Furthermore:
Both Intel and ARM use the term Performance Monitoring Unit (abbreviated PMU).
Microsoft does too [1].
[1]: https://learn.microsoft.com/en-us/windows-hardware/test/wpt/recording-pmu-events
RISC-V uses the term Hardware Performance Monitor (abbreviated HPM). I haven't checked other CPU vendors.
v6 cover add support for self monitoring
In reply to Tomasz DuszynskiThis series adds self monitoring support i.e allows to configure and read performance measurement unit (PMU) counters in runtime without using perf utility. This has certain adventages when application runs on isolated cores with nohz_full kernel parameter.
Events can be read directly using rte_pmu_read() or using dedicated tracepoint rte_eal_trace_pmu_read(). The latter will cause events to be stored inside CTF file.
By design, all enabled events are grouped together and the same group is attached to lcores that use self monitoring funtionality.
Events are enabled by names, which need to be read from standard location under sysfs i.e
/sys/bus/event_source/devices/PMU/events
where PMU is a core pmu i.e one measuring cpu events. As of today raw events are not supported.
v6:
- move codebase to the separate library
- address review comments
v5:
- address review comments
- fix sign extension while reading pmu on x86
- fix regex mentioned in doc
- various minor changes/improvements here and there v4:
- fix freeing mem detected by debug_autotest v3:
- fix shared build
v2:
- fix problems reported by test build infra
Tomasz Duszynski (4):
lib: add generic support for reading PMU events
pmu: support reading ARM PMU events in runtime
pmu: support reading Intel x86_64 PMU events in runtime
eal: add PMU support to tracing library
MAINTAINERS | 5 + app/test/meson.build | 4 + app/test/test_pmu.c | 48 +++ app/test/test_trace_perf.c | 10 + doc/api/doxy-api-index.md | 3 +- doc/api/doxy-api.conf.in | 1 + doc/guides/prog_guide/profile_app.rst | 13 + doc/guides/prog_guide/trace_lib.rst | 32 ++ doc/guides/rel_notes/release_23_03.rst | 7 + lib/eal/common/eal_common_trace.c | 13 +- lib/eal/common/eal_common_trace_points.c | 5 + lib/eal/include/rte_eal_trace.h | 13 + lib/eal/meson.build | 3 + lib/eal/version.map | 3 + lib/meson.build | 1 + lib/pmu/meson.build | 21 + lib/pmu/pmu_arm64.c | 94 +++++ lib/pmu/pmu_private.h | 29 ++ lib/pmu/rte_pmu.c | 497 +++++++++++++++++++++++ lib/pmu/rte_pmu.h | 226 +++++++++++ lib/pmu/rte_pmu_pmc_arm64.h | 30 ++ lib/pmu/rte_pmu_pmc_x86_64.h | 24 ++ lib/pmu/version.map | 20 + 23 files changed, 1100 insertions(+), 2 deletions(-) create mode 100644 app/test/test_pmu.c create mode 100644 lib/pmu/meson.build create mode 100644 lib/pmu/pmu_arm64.c create mode 100644 lib/pmu/pmu_private.h create mode 100644 lib/pmu/rte_pmu.c create mode 100644 lib/pmu/rte_pmu.h create mode 100644 lib/pmu/rte_pmu_pmc_arm64.h create mode 100644 lib/pmu/rte_pmu_pmc_x86_64.h create mode 100644 lib/pmu/version.map
--
2.34.1
PATCH v6 1/4 lib: add generic support for reading PMU events
Add support for programming PMU counters and reading their values in runtime bypassing kernel completely.
This is especially useful in cases where CPU cores are isolated (nohz_full) i.e run dedicated tasks. In such cases one cannot use standard perf utility without sacrificing latency and performance.
--- MAINTAINERS | 5 + app/test/meson.build | 4 + app/test/test_pmu.c | 42 +++ doc/api/doxy-api-index.md | 3 +- doc/api/doxy-api.conf.in | 1 + doc/guides/prog_guide/profile_app.rst | 8 + doc/guides/rel_notes/release_23_03.rst | 7 + lib/meson.build | 1 + lib/pmu/meson.build | 13 + lib/pmu/pmu_private.h | 29 ++ lib/pmu/rte_pmu.c | 436 +++++++++++++++++++++++++ lib/pmu/rte_pmu.h | 206 ++++++++++++ lib/pmu/version.map | 19 ++ 13 files changed, 773 insertions(+), 1 deletion(-) create mode 100644 app/test/test_pmu.c create mode 100644 lib/pmu/meson.build create mode 100644 lib/pmu/pmu_private.h create mode 100644 lib/pmu/rte_pmu.c create mode 100644 lib/pmu/rte_pmu.h create mode 100644 lib/pmu/version.map
MAINTAINERS +5 -0
@@ -1697,6 +1697,11 @@ M: Nithin Dabilpuram <[email protected]> M: Pavan Nikhilesh <[email protected]> F: lib/node/ +PMU - EXPERIMENTAL +M: Tomasz Duszynski <[email protected]> +F: lib/pmu/ +F: app/test/test_pmu* + Test Applications -----------------
app/test/meson.build +4 -0
@@ -360,6 +360,10 @@ if dpdk_conf.has('RTE_LIB_METRICS') test_sources += ['test_metrics.c'] fast_tests += [['metrics_autotest', true, true]] endif +if is_linux + test_sources += ['test_pmu.c'] + fast_tests += [['pmu_autotest', true, true]] +endif if not is_windows and dpdk_conf.has('RTE_LIB_TELEMETRY') test_sources += ['test_telemetry_json.c', 'test_telemetry_data.c'] fast_tests += [['telemetry_json_autotest', true, true]]
app/test/test_pmu.c new file +42 -0
42 lines not on this page · raw message
doc/api/doxy-api-index.md +2 -1
@@ -222,7 +222,8 @@ The public API headers are grouped by topics: [log](@ref rte_log.h), [errno](@ref rte_errno.h), [trace](@ref rte_trace.h), - [trace_point](@ref rte_trace_point.h) + [trace_point](@ref rte_trace_point.h), + [pmu](@ref rte_pmu.h) - **misc**: [EAL config](@ref rte_eal.h),
doc/api/doxy-api.conf.in +1 -0
@@ -63,6 +63,7 @@ INPUT = @TOPDIR@/doc/api/doxy-api-index.md \ @TOPDIR@/lib/pci \ @TOPDIR@/lib/pdump \ @TOPDIR@/lib/pipeline \ + @TOPDIR@/lib/pmu \ @TOPDIR@/lib/port \ @TOPDIR@/lib/power \ @TOPDIR@/lib/rawdev \
doc/guides/prog_guide/profile_app.rst +8 -0
@@ -7,6 +7,14 @@ Profile Your Application The following sections describe methods of profiling DPDK applications on different architectures. +Performance counter based profiling +----------------------------------- + +Majority of architectures support some sort hardware measurement unit which provides a set of +programmable counters that monitor specific events. There are different tools which can gather +that information, perf being an example here. Though in some scenarios, eg. when CPU cores are +isolated (nohz_full) and run dedicated tasks, using perf is less than ideal. In such cases one can +read specific events directly from application via ``rte_pmu_read()``. Profiling on x86 ----------------
doc/guides/rel_notes/release_23_03.rst +7 -0
@@ -57,6 +57,13 @@ New Features * **Added multi-process support for axgbe PMD.** +* **Added PMU library.** + + Added a new PMU (performance measurement unit) library which allows applications + to perform self monitoring activities without depending on external utilities like perf. + After integration with :doc:`../prog_guide/trace_lib` data gathered from hardware counters + can be stored in CTF format for further analysis. + Removed Items -------------
lib/meson.build +1 -0
@@ -11,6 +11,7 @@ libraries = [ 'kvargs', # eal depends on kvargs 'telemetry', # basic info querying + 'pmu', 'eal', # everything depends on eal 'ring', 'rcu', # rcu depends on ring
lib/pmu/meson.build new file +13 -0
@@ -0,0 +1,13 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(C) 2023 Marvell International Ltd. + +if not is_linux + build = false + reason = 'only supported on Linux' + subdir_done() +endif + +includes = [global_inc] + +sources = files('rte_pmu.c') +headers = files('rte_pmu.h')
lib/pmu/pmu_private.h new file +29 -0
@@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2023 Marvell +*/ + +#ifndef _PMU_PRIVATE_H_ +#define _PMU_PRIVATE_H_ + +/** +* Architecture specific PMU init callback. +* +* @return +* 0 in case of success, negative value otherwise. +*/ +int +pmu_arch_init (void ); + +/** +* Architecture specific PMU cleanup callback. +*/ +void +pmu_arch_fini (void ); + +/** +* Apply architecture specific settings to config before passing it to syscall. +*/ +void +pmu_arch_fixup_config (uint64_t config[3 ]); + +#endif /* _PMU_PRIVATE_H_ */
lib/pmu/rte_pmu.c new file +436 -0
436 lines not on this page · raw message
lib/pmu/rte_pmu.h new file +206 -0
206 lines not on this page · raw message
lib/pmu/version.map new file +19 -0
@@ -0,0 +1,19 @@ +DPDK_23 { + local: *; +}; + +EXPERIMENTAL { + global: + + rte_pmu; + rte_pmu_add_event; + rte_pmu_fini; + rte_pmu_init; + rte_pmu_read; +}; + +INTERNAL { + global: + + rte_pmu_enable_group; +};
Signature · 1 line
2.34.1
RE: [PATCH v6 1/4] lib: add generic support for reading PMU events
Quoted 6 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Friday, 20 January 2023 00.39Add support for programming PMU counters and reading their values in runtime bypassing kernel completely.
Quoted 6 lines
This is especially useful in cases where CPU cores are isolated (
nohz_full) i.e run dedicated tasks. In such cases one cannot use standard perf utility without sacrificing latency and performance.Signed-off-by Tomasz Duszynski <[email protected]>---
If you insist on passing lcore_id around as a function parameter, the function description must mention that the lcore_id parameter must be set to rte_lcore_id() for the functions where this is a requirement, including all functions that use those functions.
Alternatively, follow my previous suggestion: Omit the lcore_id function parameter, and use rte_lcore_id() instead.
RE: [PATCH v6 1/4] lib: add generic support for reading PMU events
Quoted 22 lines
-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: Friday, January 20, 2023 10:47 AM
To: Tomasz Duszynski <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Cc: Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [EXT] RE: [PATCH v6 1/4] lib: add generic support for reading PMU eventsExternal Email
----------------------------------------------------------------------
Quoted 10 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Friday, 20 January 2023 00.39Add support for programming PMU counters and reading their values in runtime bypassing kernel completely.
This is especially useful in cases where CPU cores are isolated (
nohz_full) i.e run dedicated tasks. In such cases one cannot use standard perf utility without sacrificing latency and performance.
Quoted 7 lines
Quoted 2 lines
Signed-off-by Tomasz Duszynski <[email protected]>---
If you insist on passing
lcore_idaround as a function parameter, the function description must mention that thelcore_idparameter must be set torte_lcore_id()for the functions where this is a requirement, including all functions that use those functions.
Not sure why are you insisting so much on removing that rte_lcore_id(). Yes that macro evaluates to integer but if you don't think about internals this resembles a function call.
Then natural pattern is to call it once and reuse results if possible. Passing lcore_id around implies that calls are per l-core, why would that confuse anyone reading that code?
Besides, all functions taking it are internal stuff hence you cannot call it elsewhere.
Quoted 3 lines
Alternatively, follow my previous suggestion: Omit the
lcore_idfunction parameter, and userte_lcore_id()instead.
RE: [PATCH v6 1/4] lib: add generic support for reading PMU events
Quoted 16 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Thursday, 26 January 2023 10.40Quoted 13 lines
From: Morten Brørup <[email protected]>
Sent: Friday, January 20, 2023 10:47 AMQuoted 10 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Friday, 20 January 2023 00.39Add support for programming PMU counters and reading their values in runtime bypassing kernel completely.
This is especially useful in cases where CPU cores are isolated (
nohz_full) i.e run dedicated tasks. In such cases one cannot use standard perf utility without sacrificing latency and performance.
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
Signed-off-by Tomasz Duszynski <[email protected]>---
If you insist on passing
lcore_idaround as a function parameter, thefunction description must
Quoted 1 line
mention that the
lcore_idparameter must be set torte_lcore_id()forthe functions where this is a
Quoted 1 line
requirement, including all functions that use those functions.
Perhaps I'm stating this wrong, so let me try to rephrase:
As I understand it, some of the setup functions must be called from the EAL thread that executes that function - due to some syscall (SYS_perf_event_open) needing to be called from the thread itself.
Those functions should not take an lcore_id parameter. Otherwise, I would expect to be able to call those functions from e.g. the main thread and pass the lcore_id of any EAL thread as a parameter, which you at the bottom of this email [1] explained is not possible.
Quoted 6 lines
Quoted 1 line
Not sure why are you insisting so much on removing that
rte_lcore_id(). Yes that macro evaluates
to integer but if you don't think about internals this resembles a function call.
I agree with this argument. And for that reason, passing lcore_id around could be relevant.
I only wanted to bring your attention to the low cost of fetching it inside the functions, as an alternative to passing it as an argument.
Quoted 2 lines
Then natural pattern is to call it once and reuse results if possible.
Yes, and I would usually agree to using this pattern.
Quoted 3 lines
Passing
lcore_idaround
implies that calls are per l-core, why would that confuse anyone reading that code?
This is where I disagree: Passing lcore_id as a parameter to a function does NOT imply that the function is running on that lcore!
E.g rte_mempool_default_cache(struct rte_mempool *mp, unsigned lcore_id) [2] takes lcore_id as a parameter, and does not assume that lcore_id==rte_lcore_id().
[2]: https://elixir.bootlin.com/dpdk/latest/source/lib/mempool/rte_mempool.h#L1315
Quoted 3 lines
Besides, all functions taking it are internal stuff hence you cannot call it elsewhere.
OK. I agree that this reduces the risk of incorrect use.
Generally, I think that internal functions should be documented too. Not to the full extent, like public functions, but some documentation is nice.
And if there are special requirements to a function parameter, it should be documented with that function. Requiring that the lcore_id parameter must be == rte_lcore_id() is certainly a special requirement.
It might just be me worrying too much, so... If nobody else complains about this, I can live with it as is. Assuming that none of the public functions have this special requirement (either directly or indirectly, by calling functions with the special requirement).
Quoted 6 lines
Quoted 2 lines
Alternatively, follow my previous suggestion: Omit the
lcore_idfunction parameter, and use
Quoted 2 lines
rte_lcore_id()instead.
Re: [PATCH v6 1/4] lib: add generic support for reading PMU events
Quoted 63 lines · Morten Brørup
Quoted 16 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Thursday, 26 January 2023 10.40Quoted 13 lines
From: Morten Brørup <[email protected]>
Sent: Friday, January 20, 2023 10:47 AMQuoted 10 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Friday, 20 January 2023 00.39Add support for programming PMU counters and reading their values in runtime bypassing kernel completely.
This is especially useful in cases where CPU cores are isolated (
nohz_full) i.e run dedicated tasks. In such cases one cannot use standard perf utility without sacrificing latency and performance.Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
Signed-off-by Tomasz Duszynski <[email protected]>---
If you insist on passing
lcore_idaround as a function parameter, thefunction description must
Quoted 1 line
mention that the
lcore_idparameter must be set torte_lcore_id()forthe functions where this is a
Quoted 1 line
requirement, including all functions that use those functions.
Perhaps I'm stating this wrong, so let me try to rephrase:
As I understand it, some of the setup functions must be called from the EAL thread that executes that function - due to some syscall (
SYS_perf_event_open) needing to be called from the thread itself.Those functions should not take an
lcore_idparameter. Otherwise, I would expect to be able to call those functions from e.g. the main thread and pass thelcore_idof any EAL thread as a parameter, which you at the bottom of this email [1] explained is not possible.Quoted 6 lines
Quoted 1 line
Not sure why are you insisting so much on removing that
rte_lcore_id(). Yes that macro evaluates
to integer but if you don't think about internals this resembles a function call.I agree with this argument. And for that reason, passing
lcore_idaround could be relevant.I only wanted to bring your attention to the low cost of fetching it inside the functions, as an alternative to passing it as an argument.
Quoted 2 lines
Then natural pattern is to call it once and reuse results if possible.
Yes, and I would usually agree to using this pattern.
Quoted 3 lines
Passing
lcore_idaround
implies that calls are per l-core, why would that confuse anyone reading that code?This is where I disagree: Passing
lcore_idas a parameter to a function does NOT imply that the function is running on that lcore!E.g
rte_mempool_default_cache(struct rte_mempool *mp, unsigned lcore_id)[2] takeslcore_idas a parameter, and does not assume that lcore_id==rte_lcore_id().[2]: https://elixir.bootlin.com/dpdk/latest/source/lib/mempool/rte_mempool.h#L1315
Quoted 3 lines
Besides, all functions taking it are internal stuff hence you cannot call it elsewhere.
Quoted 8 lines
OK. I agree that this reduces the risk of incorrect use.
Generally, I think that internal functions should be documented too. Not to the full extent, like public functions, but some documentation is nice.
And if there are special requirements to a function parameter, it should be documented with that function. Requiring that the
lcore_idparameter must be ==rte_lcore_id()is certainly a special requirement.It might just be me worrying too much, so... If nobody else complains about this, I can live with it as is. Assuming that none of the public functions have this special requirement (either directly or indirectly, by calling functions with the special requirement).
I would tend to agree with you Morten. If the lcore_id parameter to the function must be rte_lcore_id(), then I think it's error prone to have that as an explicit parameter, and that the function should always get the core id itself.
Other possible complication is - how does this work with threads that are not pinned to a particular physical core? Do things work as expected in that case?
/Bruce
RE: [EXT] Re: [PATCH v6 1/4] lib: add generic support for reading PMU events
Quoted 110 lines
-----Original Message-----
From: Bruce Richardson <[email protected]>
Sent: Thursday, January 26, 2023 1:59 PM
To: Morten Brørup <[email protected]>
Cc: Tomasz Duszynski <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [EXT] Re: [PATCH v6 1/4] lib: add generic support for reading PMU eventsExternal Email
----------------------------------------------------------------------
Quoted 22 lines · Morten Brørup
Quoted 22 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Thursday, 26 January 2023 10.40Quoted 13 lines
From: Morten Brørup <[email protected]>
Sent: Friday, January 20, 2023 10:47 AMQuoted 10 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Friday, 20 January 2023 00.39Add support for programming PMU counters and reading their values in runtime bypassing kernel completely.
This is especially useful in cases where CPU cores are isolated (
nohz_full) i.e run dedicated tasks. In such cases one cannot use standard perf utility without sacrificing latency and performance.Quoted 5 lines
Quoted 2 lines
Signed-off-by Tomasz Duszynski <[email protected]>---
If you insist on passing
lcore_idaround as a function parameter, thefunction description must
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
mention that the
lcore_idparameter must be set torte_lcore_id()forthe functions where this is a
Quoted 1 line
requirement, including all functions that use those functions.
Perhaps I'm stating this wrong, so let me try to rephrase:
As I understand it, some of the setup functions must be called from the EAL thread that executes
that function - due to some syscall (
SYS_perf_event_open) needing to be called from the thread itself.Quoted 2 lines
Those functions should not take an
lcore_idparameter. Otherwise, I would expect to be able tocall those functions from e.g. the main thread and pass the
lcore_idof any EAL thread as a parameter, which you at the bottom of this email [1] explained is not possible.Quoted 10 lines
[1]:
https://urldefense.proofpoint.com/v2/url?u=http-3A__inbox.dpdk.org_dev
_DM4PR18MB4368461EC42603F77A7DC1BCD2E09-40DM4PR18MB4368.namprd18.prod. outlook.com_&d=DwIDAw&c=nKjWec2b6R0mOyPaz7xtfQ&r=PZNXgrbjdlXxVEEGYkxIx RndyEUwWU_ad5ce22YI6Is&m=wEvFmuH_S_EhAgRZQTC7z3pQ1Sr_cEsbFAXxgE2Fi2ESd 4sMgg-tgVOVDepp-JYO&s=wU4g1LLV4EHyRYpj2inWOK8MDcUKq7txrZ7RXZhUM2I&e=Quoted 2 lines
Quoted 1 line
Quoted 8 lines
Quoted 4 lines
Not sure why are you insisting so much on removing that
rte_lcore_id(). Yes that macro evaluates
to integer but if you don't think about internals this resembles a function call.I agree with this argument. And for that reason, passing
lcore_idaround could be relevant.I only wanted to bring your attention to the low cost of fetching it inside the functions, as an
alternative to passing it as an argument.
Quoted 11 lines
Quoted 3 lines
Then natural pattern is to call it once and reuse results if possible.
Yes, and I would usually agree to using this pattern.
Quoted 3 lines
Passing
lcore_idaround
implies that calls are per l-core, why would that confuse anyone reading that code?This is where I disagree: Passing
lcore_idas a parameter to a function does NOT imply that thefunction is running on that lcore!
Quoted 2 lines
E.g
rte_mempool_default_cache(struct rte_mempool *mp, unsigned lcore_id)[2] takeslcore_idas aparameter, and does not assume that lcore_id==rte_lcore_id().
Quoted 9 lines
[2]:
https://urldefense.proofpoint.com/v2/url?u=https-3A__elixir.bootlin.co
m_dpdk_latest_source_lib_mempool_rte-5Fmempool.h-23L1315&d=DwIDAw&c=nK jWec2b6R0mOyPaz7xtfQ&r=PZNXgrbjdlXxVEEGYkxIxRndyEUwWU_ad5ce22YI6Is&m=w EvFmuH_S_EhAgRZQTC7z3pQ1Sr_cEsbFAXxgE2Fi2ESd4sMgg-tgVOVDepp-JYO&s=Ayyj pEtATWUHfWnGMn5j2XDLMjgxxJTh5gQV0m77z5Q&e=Quoted 1 line
Quoted 6 lines
Quoted 2 lines
Besides, all functions taking it are internal stuff hence you cannot call it elsewhere.
OK. I agree that this reduces the risk of incorrect use.
Generally, I think that internal functions should be documented too. Not to the full extent, like
public functions, but some documentation is nice.
Quoted 2 lines
And if there are special requirements to a function parameter, it should be documented with that
function. Requiring that the
lcore_idparameter must be ==rte_lcore_id()is certainly a special requirement.Quoted 2 lines
It might just be me worrying too much, so... If nobody else complains about this, I can live with
it as is. Assuming that none of the public functions have this special requirement (either directly or indirectly, by calling functions with the special requirement).
Quoted 1 line
Quoted 7 lines
I would tend to agree with you Morten. If the
lcore_idparameter to the function must berte_lcore_id(), then I think it's error prone to have that as an explicit parameter, and that the function should always get the core id itself.Other possible complication is - how does this work with threads that are not pinned to a particular physical core? Do things work as expected in that case?
It's assumed that once set of counters is enabled on particular l-core then this thread shouldn't be migrating back and for the obvious reasons.
But, once scheduled elsewhere all should still work as expected.
Quoted 1 line
/Bruce
RE: [EXT] Re: [PATCH v6 1/4] lib: add generic support for reading PMU events
Quoted 8 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Thursday, 26 January 2023 16.28Quoted 4 lines
From: Bruce Richardson <[email protected]>
Sent: Thursday, January 26, 2023 1:59 PMOther possible complication is - how does this work with threads that
are not pinned to a
Quoted 8 lines
Quoted 2 lines
particular physical core? Do things work as expected in that case?
It's assumed that once set of counters is enabled on particular l-core then this thread shouldn't be migrating
back and for the obvious reasons.But, once scheduled elsewhere all should still work as expected.
Just to elaborate what Tomasz stated here...
The patch contains this line (code comments are mine):
return syscall(SYS_perf_event_open, &attr, /*pid*/ 0, /*cpu*/ -1, group_fd, 0);
And man 2 perf_event_open [1] says:
pid == 0 and cpu == -1:
This measures the calling process/thread on any CPU.
So it should work just fine, even when a thread is not pinned to a particular physical core.
[1]: https://man7.org/linux/man-pages/man2/perf_event_open.2.html
RE: [PATCH v6 1/4] lib: add generic support for reading PMU events
In reply to Morten BrørupQuoted 74 lines
-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: Thursday, January 26, 2023 1:30 PM
To: Tomasz Duszynski <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Cc: Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [EXT] RE: [PATCH v6 1/4] lib: add generic support for reading PMU eventsExternal Email
----------------------------------------------------------------------
Quoted 16 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Thursday, 26 January 2023 10.40Quoted 13 lines
From: Morten Brørup <[email protected]>
Sent: Friday, January 20, 2023 10:47 AMQuoted 10 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Friday, 20 January 2023 00.39Add support for programming PMU counters and reading their values in runtime bypassing kernel completely.
This is especially useful in cases where CPU cores are isolated (
nohz_full) i.e run dedicated tasks. In such cases one cannot use standard perf utility without sacrificing latency and performance.Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
Signed-off-by Tomasz Duszynski <[email protected]>---
If you insist on passing
lcore_idaround as a function parameter, thefunction description must
Quoted 1 line
mention that the
lcore_idparameter must be set torte_lcore_id()forthe functions where this is a
Quoted 1 line
requirement, including all functions that use those functions.
Perhaps I'm stating this wrong, so let me try to rephrase:
As I understand it, some of the setup functions must be called from the EAL thread that executes that function - due to some syscall (
SYS_perf_event_open) needing to be called from the thread itself.Those functions should not take an
lcore_idparameter. Otherwise, I would expect to be able to call those functions from e.g. the main thread and pass thelcore_idof any EAL thread as a parameter, which you at the bottom of this email [1] explained is not possible.[1]: https://urldefense.proofpoint.com/v2/url?u=http-
3A__inbox.dpdk.org_dev_DM4PR18MB4368461EC42603F77A7DC1BCD2E09- 40DM4PR18MB4368.namprd18.prod.outlook.com_&d=DwIFAw&c=nKjWec2b6R0mOyPaz7xtfQ&r=PZNXgrbjdlXxVEEGYkxI xRndyEUwWU_ad5ce22YI6Is&m=QkMcmM2epUOCdRd6xI5o3d2nQaqruy0GvOQUgbn75cLlzobEVMwLBUiGXADiuvVz&s=5K9oM8 e7u52C_0_5xtWIKl31aXRHhJDKoQTDQp5EHWY&e=Quoted 6 lines
Quoted 1 line
Not sure why are you insisting so much on removing that
rte_lcore_id(). Yes that macro evaluates
to integer but if you don't think about internals this resembles a function call.I agree with this argument. And for that reason, passing
lcore_idaround could be relevant.I only wanted to bring your attention to the low cost of fetching it inside the functions, as an alternative to passing it as an argument.
Quoted 2 lines
Then natural pattern is to call it once and reuse results if possible.
Yes, and I would usually agree to using this pattern.
Quoted 3 lines
Passing
lcore_idaround
implies that calls are per l-core, why would that confuse anyone reading that code?
Quoted 6 lines
This is where I disagree: Passing
lcore_idas a parameter to a function does NOT imply that the function is running on that lcore!E.g
rte_mempool_default_cache(struct rte_mempool *mp, unsigned lcore_id)[2] takeslcore_idas a parameter, and does not assume that lcore_id==rte_lcore_id().
Oh, now I got your point!
Okay then, if this is going to cause confusion because of misleading self-documenting code I'll change that.
Quoted 29 lines
[2]: https://urldefense.proofpoint.com/v2/url?u=https-
3A__elixir.bootlin.com_dpdk_latest_source_lib_mempool_rte-5Fmempool.h- 23L1315&d=DwIFAw&c=nKjWec2b6R0mOyPaz7xtfQ&r=PZNXgrbjdlXxVEEGYkxIxRndyEUwWU_ad5ce22YI6Is&m=QkMcmM2ep UOCdRd6xI5o3d2nQaqruy0GvOQUgbn75cLlzobEVMwLBUiGXADiuvVz&s=4pnL_TZcVhj476u19ybcn2Rbad6OTb3k2U- nhFvhZ0k&e=Quoted 3 lines
Besides, all functions taking it are internal stuff hence you cannot call it elsewhere.
OK. I agree that this reduces the risk of incorrect use.
Generally, I think that internal functions should be documented too. Not to the full extent, like public functions, but some documentation is nice.
And if there are special requirements to a function parameter, it should be documented with that function. Requiring that the
lcore_idparameter must be ==rte_lcore_id()is certainly a special requirement.It might just be me worrying too much, so... If nobody else complains about this, I can live with it as is. Assuming that none of the public functions have this special requirement (either directly or indirectly, by calling functions with the special requirement).
Quoted 6 lines
Quoted 2 lines
Alternatively, follow my previous suggestion: Omit the
lcore_idfunction parameter, and use
Quoted 2 lines
rte_lcore_id()instead.
Re: [PATCH v6 1/4] lib: add generic support for reading PMU events
In reply to Tomasz DuszynskiQuoted 51 lines · Tomasz Duszynski
Add support for programming PMU counters and reading their values in runtime bypassing kernel completely.
This is especially useful in cases where CPU cores are isolated (
nohz_full) i.e run dedicated tasks. In such cases one cannot use standard perf utility without sacrificing latency and performance.Signed-off-by Tomasz Duszynski <[email protected]>--- MAINTAINERS | 5 + app/test/meson.build | 4 + app/test/test_pmu.c | 42 +++ doc/api/doxy-api-index.md | 3 +- doc/api/doxy-api.conf.in | 1 + doc/guides/prog_guide/profile_app.rst | 8 + doc/guides/rel_notes/release_23_03.rst | 7 + lib/meson.build | 1 + lib/pmu/meson.build | 13 + lib/pmu/pmu_private.h | 29 ++ lib/pmu/rte_pmu.c | 436 +++++++++++++++++++++++++ lib/pmu/rte_pmu.h | 206 ++++++++++++ lib/pmu/version.map | 19 ++ 13 files changed, 773 insertions(+), 1 deletion(-) create mode 100644 app/test/test_pmu.c create mode 100644 lib/pmu/meson.build create mode 100644 lib/pmu/pmu_private.h create mode 100644 lib/pmu/rte_pmu.c create mode 100644 lib/pmu/rte_pmu.h create mode 100644 lib/pmu/version.mapMAINTAINERS +5
-0@@ -1697,6 +1697,11 @@ M: Nithin Dabilpuram <[email protected]> M: Pavan Nikhilesh <[email protected]> F: lib/node/ +PMU - EXPERIMENTAL +M: Tomasz Duszynski <[email protected]> +F: lib/pmu/ +F: app/test/test_pmu* + Test Applications -----------------app/test/meson.build +0
-0
app/test/meson.build if dpdk_conf.has('RTE_LIB_METRICS') · line 366
Quoted 7 lines
app/test/meson.build +4
-0@@ -360,6 +360,10 @@ if dpdk_conf.has('RTE_LIB_METRICS') test_sources += ['test_metrics.c'] fast_tests += [['metrics_autotest', true, true]] endif +if is_linux + test_sources += ['test_pmu.c'] + fast_tests += [['pmu_autotest', true, true]] +endif
traditionally we don't conditionally include tests at the meson.build level, instead we run all tests and have them skip when executed for unsupported exec environments.
you can take a look at test_eventdev.c as an example for a test that is skipped on windows, i'm sure it could be adapted to skip on freebsd if you aren't supporting it.
RE: [EXT] Re: [PATCH v6 1/4] lib: add generic support for reading PMU events
Quoted 73 lines
-----Original Message-----
From: Tyler Retzlaff <[email protected]>
Sent: Friday, January 20, 2023 7:30 PM
To: Tomasz Duszynski <[email protected]>
Cc: [email protected]; Thomas Monjalon <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [EXT] Re: [PATCH v6 1/4] lib: add generic support for reading PMU eventsExternal Email
----------------------------------------------------------------------
Quoted 49 lines · Tomasz Duszynski
Add support for programming PMU counters and reading their values in runtime bypassing kernel completely.
This is especially useful in cases where CPU cores are isolated (
nohz_full) i.e run dedicated tasks. In such cases one cannot use standard perf utility without sacrificing latency and performance.Signed-off-by Tomasz Duszynski <[email protected]>--- MAINTAINERS | 5 + app/test/meson.build | 4 + app/test/test_pmu.c | 42 +++ doc/api/doxy-api-index.md | 3 +- doc/api/doxy-api.conf.in | 1 + doc/guides/prog_guide/profile_app.rst | 8 + doc/guides/rel_notes/release_23_03.rst | 7 + lib/meson.build | 1 + lib/pmu/meson.build | 13 + lib/pmu/pmu_private.h | 29 ++ lib/pmu/rte_pmu.c | 436 +++++++++++++++++++++++++ lib/pmu/rte_pmu.h | 206 ++++++++++++ lib/pmu/version.map | 19 ++ 13 files changed, 773 insertions(+), 1 deletion(-) create mode100644
app/test/test_pmu.ccreate mode 100644lib/pmu/meson.buildcreate mode 100644lib/pmu/pmu_private.hcreate mode 100644lib/pmu/rte_pmu.ccreate mode 100644lib/pmu/rte_pmu.hcreate mode 100644lib/pmu/version.mapMAINTAINERS index 9a0f416d2e..9f13eafd95 renamed from MAINTAINERS +0
-0100644
MAINTAINERS +5
-0@@ -1697,6 +1697,11 @@ M: Nithin Dabilpuram <[email protected]> M: Pavan Nikhilesh <[email protected]> F: lib/node/ +PMU - EXPERIMENTAL +M: Tomasz Duszynski <[email protected]> +F: lib/pmu/ +F: app/test/test_pmu* + Test Applications -----------------app/test/meson.build index renamed from app/test/meson.build +0
-0
f34d19e3c3..b2c2a618b1100644app/test/meson.build if dpdk_conf.has('RTE_LIB_METRICS') · line 364
Quoted 5 lines
app/test/meson.build +3
-0@@ -360,6 +360,10 @@ if dpdk_conf.has('RTE_LIB_METRICS') test_sources += ['test_metrics.c'] fast_tests += [['metrics_autotest', true, true]] endif +if is_linux + test_sources += ['test_pmu.c'] + fast_tests += [['pmu_autotest', true, true]] endiftraditionally we don't conditionally include tests at the meson.build level, instead we run all tests and have them skip when executed for unsupported exec environments.
Quoted 2 lines
you can take a look at
test_eventdev.cas an example for a test that is skipped on windows, i'm sure it could be adapted to skip on freebsd if you aren't supporting it.
Right, this looks better. Thanks for pointing this out.
PATCH v6 2/4 pmu: support reading ARM PMU events in runtime
In reply to Tomasz DuszynskiAdd support for reading ARM PMU events in runtime.
--- app/test/test_pmu.c | 4 ++ lib/pmu/meson.build | 7 +++ lib/pmu/pmu_arm64.c | 94 +++++++++++++++++++++++++++++++++++++ lib/pmu/rte_pmu.h | 4 ++ lib/pmu/rte_pmu_pmc_arm64.h | 30 ++++++++++++ 5 files changed, 139 insertions(+) create mode 100644 lib/pmu/pmu_arm64.c create mode 100644 lib/pmu/rte_pmu_pmc_arm64.h
app/test/test_pmu.c +4 -0
@@ -15,6 +15,10 @@ test_pmu_read(void)if (rte_pmu_init () <0 )return TEST_FAILED ; +#if defined(RTE_ARCH_ARM64 ) + event =rte_pmu_add_event ("cpu_cycles" ); +#endif +while (tries--) val +=rte_pmu_read (event);
lib/pmu/meson.build +7 -0
@@ -11,3 +11,10 @@ includes = [global_inc] sources = files('rte_pmu.c') headers = files('rte_pmu.h') +indirect_headers += files( + 'rte_pmu_pmc_arm64.h', +) + +if dpdk_conf.has('RTE_ARCH_ARM64') + sources += files('pmu_arm64.c') +endif
lib/pmu/pmu_arm64.c new file +94 -0
94 lines not on this page · raw message
lib/pmu/rte_pmu.h +4 -0
@@ -26,6 +26,10 @@ extern "C" {#include <rte_compat.h> #include <rte_lcore.h> +#if defined(RTE_ARCH_ARM64 ) +#include "rte_pmu_pmc_arm64.h" +#endif +/** Maximum number of events in a group */ #define MAX_NUM_GROUP_EVENTS 8
lib/pmu/rte_pmu_pmc_arm64.h new file +30 -0
@@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2023 Marvell. +*/ +#ifndef _RTE_PMU_PMC_ARM64_H_ +#define _RTE_PMU_PMC_ARM64_H_ + +#include <rte_common.h> + +static __rte_always_inline uint64_t +rte_pmu_pmc_read (int index) +{ +uint64_t val; + +if (index ==31 ) { +/* CPU Cycles (0x11) must be read via pmccntr_el0 */ + asmvolatile ("mrs %0, pmccntr_el0" :"=r" (val)); + }else { + asmvolatile ( +"msr pmselr_el0, %x0\n" +"mrs %0, pmxevcntr_el0\n" + :"=r" (val) + :"rZ" (index) + ); + } + +return val; +} +#define rte_pmu_pmc_read rte_pmu_pmc_read + +#endif /* _RTE_PMU_PMC_ARM64_H_ */
Signature · 1 line
2.34.1
PATCH v6 3/4 pmu: support reading Intel x86_64 PMU events in runtime
In reply to Tomasz DuszynskiAdd support for reading Intel x86_64 PMU events in runtime.
--- app/test/test_pmu.c | 2 ++ lib/pmu/meson.build | 1 + lib/pmu/rte_pmu.h | 2 ++ lib/pmu/rte_pmu_pmc_x86_64.h | 24 ++++++++++++++++++++++++ 4 files changed, 29 insertions(+) create mode 100644 lib/pmu/rte_pmu_pmc_x86_64.h
app/test/test_pmu.c +2 -0
@@ -17,6 +17,8 @@ test_pmu_read(void)#if defined(RTE_ARCH_ARM64 ) event =rte_pmu_add_event ("cpu_cycles" ); +#elif defined(RTE_ARCH_X86_64 ) + event =rte_pmu_add_event ("cpu-cycles" );#endif while (tries--)
lib/pmu/meson.build +1 -0
@@ -13,6 +13,7 @@ sources = files('rte_pmu.c') headers = files('rte_pmu.h') indirect_headers += files( 'rte_pmu_pmc_arm64.h', + 'rte_pmu_pmc_x86_64.h', ) if dpdk_conf.has('RTE_ARCH_ARM64')
lib/pmu/rte_pmu.h +2 -0
@@ -28,6 +28,8 @@ extern "C" {#if defined(RTE_ARCH_ARM64 )#include "rte_pmu_pmc_arm64.h" +#elif defined(RTE_ARCH_X86_64 ) +#include "rte_pmu_pmc_x86_64.h" #endif /** Maximum number of events in a group */
lib/pmu/rte_pmu_pmc_x86_64.h new file +24 -0
@@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2023 Marvell. +*/ +#ifndef _RTE_PMU_PMC_X86_64_H_ +#define _RTE_PMU_PMC_X86_64_H_ + +#include <rte_common.h> + +static __rte_always_inline uint64_t +rte_pmu_pmc_read (int index) +{ +uint64_t low, high; + + asmvolatile ( +"rdpmc\n" + :"=a" (low),"=d" (high) + :"c" (index) + ); + +return low | (high <<32 ); +} +#define rte_pmu_pmc_read rte_pmu_pmc_read + +#endif /* _RTE_PMU_PMC_X86_64_H_ */
Signature · 1 line
2.34.1
PATCH v6 4/4 eal: add PMU support to tracing library
In reply to Tomasz DuszynskiIn order to profile app one needs to store significant amount of samples somewhere for an analysis latern on. Since trace library supports storing data in a CTF format lets take adventage of that and add a dedicated PMU tracepoint.
--- app/test/test_trace_perf.c | 10 ++++ doc/guides/prog_guide/profile_app.rst | 5 ++ doc/guides/prog_guide/trace_lib.rst | 32 +++++++++++++ lib/eal/common/eal_common_trace.c | 13 ++++- lib/eal/common/eal_common_trace_points.c | 5 ++ lib/eal/include/rte_eal_trace.h | 13 +++++ lib/eal/meson.build | 3 ++ lib/eal/version.map | 3 ++ lib/pmu/rte_pmu.c | 61 ++++++++++++++++++++++++ lib/pmu/rte_pmu.h | 14 ++++++ lib/pmu/version.map | 1 + 11 files changed, 159 insertions(+), 1 deletion(-)
app/test/test_trace_perf.c +10 -0
@@ -114,6 +114,10 @@ worker_fn_##func(void *arg) \#define GENERIC_DOUBLE rte_eal_trace_generic_double(3.66666)#define GENERIC_STR rte_eal_trace_generic_str("hello world")#define VOID_FP app_dpdk_test_fp() +#ifdef RTE_EXEC_ENV_LINUX +/* 0 corresponds first event passed via --trace= */ +#define READ_PMU rte_eal_trace_pmu_read(0) +#endif WORKER_DEFINE (GENERIC_VOID)WORKER_DEFINE (GENERIC_U64)
@@ -122,6 +126,9 @@ WORKER_DEFINE(GENERIC_FLOAT) WORKER_DEFINE(GENERIC_DOUBLE) WORKER_DEFINE(GENERIC_STR) WORKER_DEFINE(VOID_FP) +#ifdef RTE_EXEC_ENV_LINUX +WORKER_DEFINE(READ_PMU) +#endif static void run_test (const char *str,lcore_function_t f,struct test_data *data,size_t sz)
@@ -174,6 +181,9 @@ test_trace_perf(void)run_test ("double" , worker_fn_GENERIC_DOUBLE, data, sz);run_test ("string" , worker_fn_GENERIC_STR, data, sz);run_test ("void_fp" , worker_fn_VOID_FP, data, sz); +#ifdef RTE_EXEC_ENV_LINUX +run_test ("read_pmu" , worker_fn_READ_PMU, data, sz); +#endif rte_free (data);return TEST_SUCCESS ;
doc/guides/prog_guide/profile_app.rst +5 -0
@@ -16,6 +16,11 @@ that information, perf being an example here. Though in some scenarios, eg. when isolated (nohz_full) and run dedicated tasks, using perf is less than ideal. In such cases one can read specific events directly from application via ``rte_pmu_read()``. +Alternatively tracing library can be used which offers dedicated tracepoint +``rte_eal_trace_pmu_event()``. + +Refer to :doc:`../prog_guide/trace_lib` for more details. + Profiling on x86 ----------------
doc/guides/prog_guide/trace_lib.rst +32 -0
@@ -46,6 +46,7 @@ DPDK tracing library features trace format and is compatible with ``LTTng``. For detailed information, refer to `Common Trace Format <https://diamon.org/ctf/>`_. +- Support reading PMU events on ARM64 and x86-64 (Intel) How to add a tracepoint? ------------------------
@@ -137,6 +138,37 @@ the user must use ``RTE_TRACE_POINT_FP`` instead of ``RTE_TRACE_POINT``. ``RTE_TRACE_POINT_FP`` is compiled out by default and it can be enabled using the ``enable_trace_fp`` option for meson build. +PMU tracepoint +-------------- + +Performance measurement unit (PMU) event values can be read from hardware +registers using predefined ``rte_pmu_read`` tracepoint. + +Tracing is enabled via ``--trace`` EAL option by passing both expression +matching PMU tracepoint name i.e ``lib.eal.pmu.read`` and expression +``e=ev1[,ev2,...]`` matching particular events:: + + --trace='.*pmu.read\|e=cpu_cycles,l1d_cache' + +Event names are available under ``/sys/bus/event_source/devices/PMU/events`` +directory, where ``PMU`` is a placeholder for either a ``cpu`` or a directory +containing ``cpus``. + +In contrary to other tracepoints this does not need any extra variables +added to source files. Instead, caller passes index which follows the order of +events specified via ``--trace`` parameter. In the following example index ``0`` +corresponds to ``cpu_cyclces`` while index ``1`` corresponds to ``l1d_cache``. + +.. code-block:: c + + ... + rte_eal_trace_pmu_read(0); + rte_eal_trace_pmu_read(1); + ... + +PMU tracing support must be explicitly enabled using the ``enable_trace_fp`` +option for meson build. + Event record mode -----------------
lib/eal/common/eal_common_trace.c +12 -1
@@ -11,6 +11,9 @@#include <rte_errno.h> #include <rte_lcore.h> #include <rte_per_lcore.h> +#ifdef RTE_EXEC_ENV_LINUX +#include <rte_pmu.h> +#endif #include <rte_string_fns.h> #include "eal_trace.h"
@@ -71,8 +74,13 @@ eal_trace_init(void) goto free_meta;/* Apply global configurations */ - STAILQ_FOREACH(arg, &trace.args, next) + STAILQ_FOREACH(arg, &trace.args, next) {trace_args_apply (arg->val); +#ifdef RTE_EXEC_ENV_LINUX +if (rte_pmu_init () ==0 ) +rte_pmu_add_events_by_pattern (arg->val); +#endif + }rte_trace_mode_set (trace.mode);
@@ -88,6 +96,9 @@ eal_trace_init(void)void eal_trace_fini (void ) { +#ifdef RTE_EXEC_ENV_LINUX +rte_pmu_fini (); +#endif trace_mem_free ();trace_metadata_destroy ();eal_trace_args_free ();
lib/eal/common/eal_common_trace_points.c +5 -0
@@ -75,3 +75,8 @@ RTE_TRACE_POINT_REGISTER(rte_eal_trace_intr_enable, lib.eal.intr.enable) RTE_TRACE_POINT_REGISTER(rte_eal_trace_intr_disable, lib.eal.intr.disable) + +#ifdef RTE_EXEC_ENV_LINUX +RTE_TRACE_POINT_REGISTER (rte_eal_trace_pmu_read, + lib.eal.pmu.read) +#endif
lib/eal/include/rte_eal_trace.h +13 -0
@@ -17,6 +17,9 @@ extern "C" {#include <rte_alarm.h> #include <rte_interrupts.h> +#ifdef RTE_EXEC_ENV_LINUX +#include <rte_pmu.h> +#endif #include <rte_trace_point.h> #include "eal_interrupts.h"
@@ -279,6 +282,16 @@ RTE_TRACE_POINT(rte_trace_point_emit_string (cpuset); ) +#ifdef RTE_EXEC_ENV_LINUX +RTE_TRACE_POINT_FP ( + rte_eal_trace_pmu_read, + RTE_TRACE_POINT_ARGS(unsigned int index), + uint64_t val; + val = rte_pmu_read(index); + rte_trace_point_emit_u64(val); +) +#endif + #ifdef __cplusplus }#endif
lib/eal/meson.build +3 -0
@@ -26,6 +26,9 @@ deps += ['kvargs'] if not is_windows deps += ['telemetry'] endif +if is_linux + deps += ['pmu'] +endif if dpdk_conf.has('RTE_USE_LIBBSD') ext_deps += libbsd endif
lib/eal/version.map +3 -0
@@ -440,6 +440,9 @@ EXPERIMENTAL { rte_thread_detach; rte_thread_equal; rte_thread_join; + + # added in 23.03 + __rte_eal_trace_pmu_read; # WINDOWS_NO_EXPORT }; INTERNAL {
lib/pmu/rte_pmu.c +61 -0
67 lines not on this page · raw message
lib/pmu/rte_pmu.h +14 -0
@@ -166,6 +166,20 @@ __rte_experimentalint rte_pmu_add_event (const char *name); +/** +* @warning +* @b EXPERIMENTAL: this API may change without prior notice +* +* Add events matching pattern to the group of enabled events. +* +* @param pattern +* Pattern e=ev1[,ev2,...] matching events, where evX is a placeholder for an event listed under +* /sys/bus/event_source/devices/pmu/events. +*/ +__rte_experimental +int +rte_pmu_add_events_by_pattern (const char *pattern); +/** * @warning * @b EXPERIMENTAL: this API may change without prior notice
lib/pmu/version.map +1 -0
@@ -7,6 +7,7 @@ EXPERIMENTAL { rte_pmu; rte_pmu_add_event; + rte_pmu_add_events_by_pattern; rte_pmu_fini; rte_pmu_init; rte_pmu_read;
Signature · 1 line
2.34.1
v7 cover add support for self monitoring
In reply to Tomasz DuszynskiThis series adds self monitoring support i.e allows to configure and read performance measurement unit (PMU) counters in runtime without using perf utility. This has certain adventages when application runs on isolated cores with nohz_full kernel parameter.
Events can be read directly using rte_pmu_read() or using dedicated tracepoint rte_eal_trace_pmu_read(). The latter will cause events to be stored inside CTF file.
By design, all enabled events are grouped together and the same group is attached to lcores that use self monitoring funtionality.
Events are enabled by names, which need to be read from standard location under sysfs i.e
/sys/bus/event_source/devices/PMU/events
where PMU is a core pmu i.e one measuring cpu events. As of today raw events are not supported.
v7:
- use per-lcore event group instead of global table index by lcore-id
- don't add pmu_autotest to fast tests because due to lack of suported on every arch
v6:
- move codebase to the separate library
- address review comments
v5:
- address review comments
- fix sign extension while reading pmu on x86
- fix regex mentioned in doc
- various minor changes/improvements here and there
v4:
- fix freeing mem detected by debug_autotest
v3:
- fix shared build
v2:
- fix problems reported by test build infra
Tomasz Duszynski (4):
lib: add generic support for reading PMU events
pmu: support reading ARM PMU events in runtime
pmu: support reading Intel x86_64 PMU events in runtime
eal: add PMU support to tracing library
MAINTAINERS | 5 + app/test/meson.build | 1 + app/test/test_pmu.c | 61 +++ app/test/test_trace_perf.c | 10 + doc/api/doxy-api-index.md | 3 +- doc/api/doxy-api.conf.in | 1 + doc/guides/prog_guide/profile_app.rst | 13 + doc/guides/prog_guide/trace_lib.rst | 32 ++ doc/guides/rel_notes/release_23_03.rst | 7 + lib/eal/common/eal_common_trace.c | 13 +- lib/eal/common/eal_common_trace_points.c | 5 + lib/eal/include/rte_eal_trace.h | 13 + lib/eal/meson.build | 3 + lib/eal/version.map | 3 + lib/meson.build | 1 + lib/pmu/meson.build | 21 + lib/pmu/pmu_arm64.c | 94 ++++ lib/pmu/pmu_private.h | 29 ++ lib/pmu/rte_pmu.c | 525 +++++++++++++++++++++++ lib/pmu/rte_pmu.h | 225 ++++++++++ lib/pmu/rte_pmu_pmc_arm64.h | 30 ++ lib/pmu/rte_pmu_pmc_x86_64.h | 24 ++ lib/pmu/version.map | 21 + 23 files changed, 1138 insertions(+), 2 deletions(-) create mode 100644 app/test/test_pmu.c create mode 100644 lib/pmu/meson.build create mode 100644 lib/pmu/pmu_arm64.c create mode 100644 lib/pmu/pmu_private.h create mode 100644 lib/pmu/rte_pmu.c create mode 100644 lib/pmu/rte_pmu.h create mode 100644 lib/pmu/rte_pmu_pmc_arm64.h create mode 100644 lib/pmu/rte_pmu_pmc_x86_64.h create mode 100644 lib/pmu/version.map
--
2.34.1
PATCH v7 1/4 lib: add generic support for reading PMU events
Add support for programming PMU counters and reading their values in runtime bypassing kernel completely.
This is especially useful in cases where CPU cores are isolated (nohz_full) i.e run dedicated tasks. In such cases one cannot use standard perf utility without sacrificing latency and performance.
--- MAINTAINERS | 5 + app/test/meson.build | 1 + app/test/test_pmu.c | 55 +++ doc/api/doxy-api-index.md | 3 +- doc/api/doxy-api.conf.in | 1 + doc/guides/prog_guide/profile_app.rst | 8 + doc/guides/rel_notes/release_23_03.rst | 7 + lib/meson.build | 1 + lib/pmu/meson.build | 13 + lib/pmu/pmu_private.h | 29 ++ lib/pmu/rte_pmu.c | 464 +++++++++++++++++++++++++ lib/pmu/rte_pmu.h | 205 +++++++++++ lib/pmu/version.map | 20 ++ 13 files changed, 811 insertions(+), 1 deletion(-) create mode 100644 app/test/test_pmu.c create mode 100644 lib/pmu/meson.build create mode 100644 lib/pmu/pmu_private.h create mode 100644 lib/pmu/rte_pmu.c create mode 100644 lib/pmu/rte_pmu.h create mode 100644 lib/pmu/version.map
MAINTAINERS +5 -0
@@ -1697,6 +1697,11 @@ M: Nithin Dabilpuram <[email protected]> M: Pavan Nikhilesh <[email protected]> F: lib/node/ +PMU - EXPERIMENTAL +M: Tomasz Duszynski <[email protected]> +F: lib/pmu/ +F: app/test/test_pmu* + Test Applications -----------------
app/test/meson.build +1 -0
@@ -111,6 +111,7 @@ test_sources = files( 'test_reciprocal_division_perf.c', 'test_red.c', 'test_pie.c', + 'test_pmu.c', 'test_reorder.c', 'test_rib.c', 'test_rib6.c',
app/test/test_pmu.c new file +55 -0
55 lines not on this page · raw message
doc/api/doxy-api-index.md +2 -1
@@ -222,7 +222,8 @@ The public API headers are grouped by topics: [log](@ref rte_log.h), [errno](@ref rte_errno.h), [trace](@ref rte_trace.h), - [trace_point](@ref rte_trace_point.h) + [trace_point](@ref rte_trace_point.h), + [pmu](@ref rte_pmu.h) - **misc**: [EAL config](@ref rte_eal.h),
doc/api/doxy-api.conf.in +1 -0
@@ -63,6 +63,7 @@ INPUT = @TOPDIR@/doc/api/doxy-api-index.md \ @TOPDIR@/lib/pci \ @TOPDIR@/lib/pdump \ @TOPDIR@/lib/pipeline \ + @TOPDIR@/lib/pmu \ @TOPDIR@/lib/port \ @TOPDIR@/lib/power \ @TOPDIR@/lib/rawdev \
doc/guides/prog_guide/profile_app.rst +8 -0
@@ -7,6 +7,14 @@ Profile Your Application The following sections describe methods of profiling DPDK applications on different architectures. +Performance counter based profiling +----------------------------------- + +Majority of architectures support some sort hardware measurement unit which provides a set of +programmable counters that monitor specific events. There are different tools which can gather +that information, perf being an example here. Though in some scenarios, eg. when CPU cores are +isolated (nohz_full) and run dedicated tasks, using perf is less than ideal. In such cases one can +read specific events directly from application via ``rte_pmu_read()``. Profiling on x86 ----------------
doc/guides/rel_notes/release_23_03.rst +7 -0
@@ -55,6 +55,13 @@ New Features Also, make sure to start the actual text at the margin. ======================================================= +* **Added PMU library.** + + Added a new PMU (performance measurement unit) library which allows applications + to perform self monitoring activities without depending on external utilities like perf. + After integration with :doc:`../prog_guide/trace_lib` data gathered from hardware counters + can be stored in CTF format for further analysis. + * **Added multi-process support for axgbe PMD.** * **Updated Corigine nfp driver.**
lib/meson.build +1 -0
@@ -11,6 +11,7 @@ libraries = [ 'kvargs', # eal depends on kvargs 'telemetry', # basic info querying + 'pmu', 'eal', # everything depends on eal 'ring', 'rcu', # rcu depends on ring
lib/pmu/meson.build new file +13 -0
@@ -0,0 +1,13 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(C) 2023 Marvell International Ltd. + +if not is_linux + build = false + reason = 'only supported on Linux' + subdir_done() +endif + +includes = [global_inc] + +sources = files('rte_pmu.c') +headers = files('rte_pmu.h')
lib/pmu/pmu_private.h new file +29 -0
@@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2023 Marvell +*/ + +#ifndef _PMU_PRIVATE_H_ +#define _PMU_PRIVATE_H_ + +/** +* Architecture specific PMU init callback. +* +* @return +* 0 in case of success, negative value otherwise. +*/ +int +pmu_arch_init (void ); + +/** +* Architecture specific PMU cleanup callback. +*/ +void +pmu_arch_fini (void ); + +/** +* Apply architecture specific settings to config before passing it to syscall. +*/ +void +pmu_arch_fixup_config (uint64_t config[3 ]); + +#endif /* _PMU_PRIVATE_H_ */
lib/pmu/rte_pmu.c new file +464 -0
464 lines not on this page · raw message
lib/pmu/rte_pmu.h new file +205 -0
205 lines not on this page · raw message
lib/pmu/version.map new file +20 -0
@@ -0,0 +1,20 @@ +DPDK_23 { + local: *; +}; + +EXPERIMENTAL { + global: + + per_lcore__event_group; + rte_pmu; + rte_pmu_add_event; + rte_pmu_fini; + rte_pmu_init; + rte_pmu_read; +}; + +INTERNAL { + global: + + rte_pmu_enable_group; +};
Signature · 1 line
2.34.1
PATCH v7 2/4 pmu: support reading ARM PMU events in runtime
In reply to Tomasz DuszynskiAdd support for reading ARM PMU events in runtime.
--- app/test/test_pmu.c | 4 ++ lib/pmu/meson.build | 7 +++ lib/pmu/pmu_arm64.c | 94 +++++++++++++++++++++++++++++++++++++ lib/pmu/rte_pmu.h | 4 ++ lib/pmu/rte_pmu_pmc_arm64.h | 30 ++++++++++++ 5 files changed, 139 insertions(+) create mode 100644 lib/pmu/pmu_arm64.c create mode 100644 lib/pmu/rte_pmu_pmc_arm64.h
app/test/test_pmu.c +4 -0
@@ -26,6 +26,10 @@ test_pmu_read(void)if (rte_pmu_init () <0 )return TEST_FAILED ; +#if defined(RTE_ARCH_ARM64 ) + event =rte_pmu_add_event ("cpu_cycles" ); +#endif +while (tries--) val +=rte_pmu_read (event);
lib/pmu/meson.build +7 -0
@@ -11,3 +11,10 @@ includes = [global_inc] sources = files('rte_pmu.c') headers = files('rte_pmu.h') +indirect_headers += files( + 'rte_pmu_pmc_arm64.h', +) + +if dpdk_conf.has('RTE_ARCH_ARM64') + sources += files('pmu_arm64.c') +endif
lib/pmu/pmu_arm64.c new file +94 -0
94 lines not on this page · raw message
lib/pmu/rte_pmu.h +4 -0
@@ -26,6 +26,10 @@ extern "C" {#include <rte_compat.h> #include <rte_spinlock.h> +#if defined(RTE_ARCH_ARM64 ) +#include "rte_pmu_pmc_arm64.h" +#endif +/** Maximum number of events in a group */ #define MAX_NUM_GROUP_EVENTS 8
lib/pmu/rte_pmu_pmc_arm64.h new file +30 -0
@@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2023 Marvell. +*/ +#ifndef _RTE_PMU_PMC_ARM64_H_ +#define _RTE_PMU_PMC_ARM64_H_ + +#include <rte_common.h> + +static __rte_always_inline uint64_t +rte_pmu_pmc_read (int index) +{ +uint64_t val; + +if (index ==31 ) { +/* CPU Cycles (0x11) must be read via pmccntr_el0 */ + asmvolatile ("mrs %0, pmccntr_el0" :"=r" (val)); + }else { + asmvolatile ( +"msr pmselr_el0, %x0\n" +"mrs %0, pmxevcntr_el0\n" + :"=r" (val) + :"rZ" (index) + ); + } + +return val; +} +#define rte_pmu_pmc_read rte_pmu_pmc_read + +#endif /* _RTE_PMU_PMC_ARM64_H_ */
Signature · 1 line
2.34.1
PATCH v7 3/4 pmu: support reading Intel x86_64 PMU events in runtime
In reply to Tomasz DuszynskiAdd support for reading Intel x86_64 PMU events in runtime.
--- app/test/test_pmu.c | 2 ++ lib/pmu/meson.build | 1 + lib/pmu/rte_pmu.h | 2 ++ lib/pmu/rte_pmu_pmc_x86_64.h | 24 ++++++++++++++++++++++++ 4 files changed, 29 insertions(+) create mode 100644 lib/pmu/rte_pmu_pmc_x86_64.h
app/test/test_pmu.c +2 -0
@@ -28,6 +28,8 @@ test_pmu_read(void)#if defined(RTE_ARCH_ARM64 ) event =rte_pmu_add_event ("cpu_cycles" ); +#elif defined(RTE_ARCH_X86_64 ) + event =rte_pmu_add_event ("cpu-cycles" );#endif while (tries--)
lib/pmu/meson.build +1 -0
@@ -13,6 +13,7 @@ sources = files('rte_pmu.c') headers = files('rte_pmu.h') indirect_headers += files( 'rte_pmu_pmc_arm64.h', + 'rte_pmu_pmc_x86_64.h', ) if dpdk_conf.has('RTE_ARCH_ARM64')
lib/pmu/rte_pmu.h +2 -0
@@ -28,6 +28,8 @@ extern "C" {#if defined(RTE_ARCH_ARM64 )#include "rte_pmu_pmc_arm64.h" +#elif defined(RTE_ARCH_X86_64 ) +#include "rte_pmu_pmc_x86_64.h" #endif /** Maximum number of events in a group */
lib/pmu/rte_pmu_pmc_x86_64.h new file +24 -0
@@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2023 Marvell. +*/ +#ifndef _RTE_PMU_PMC_X86_64_H_ +#define _RTE_PMU_PMC_X86_64_H_ + +#include <rte_common.h> + +static __rte_always_inline uint64_t +rte_pmu_pmc_read (int index) +{ +uint64_t low, high; + + asmvolatile ( +"rdpmc\n" + :"=a" (low),"=d" (high) + :"c" (index) + ); + +return low | (high <<32 ); +} +#define rte_pmu_pmc_read rte_pmu_pmc_read + +#endif /* _RTE_PMU_PMC_X86_64_H_ */
Signature · 1 line
2.34.1
PATCH v7 4/4 eal: add PMU support to tracing library
In reply to Tomasz DuszynskiIn order to profile app one needs to store significant amount of samples somewhere for an analysis latern on. Since trace library supports storing data in a CTF format lets take adventage of that and add a dedicated PMU tracepoint.
--- app/test/test_trace_perf.c | 10 ++++ doc/guides/prog_guide/profile_app.rst | 5 ++ doc/guides/prog_guide/trace_lib.rst | 32 +++++++++++++ lib/eal/common/eal_common_trace.c | 13 ++++- lib/eal/common/eal_common_trace_points.c | 5 ++ lib/eal/include/rte_eal_trace.h | 13 +++++ lib/eal/meson.build | 3 ++ lib/eal/version.map | 3 ++ lib/pmu/rte_pmu.c | 61 ++++++++++++++++++++++++ lib/pmu/rte_pmu.h | 14 ++++++ lib/pmu/version.map | 1 + 11 files changed, 159 insertions(+), 1 deletion(-)
app/test/test_trace_perf.c +10 -0
@@ -114,6 +114,10 @@ worker_fn_##func(void *arg) \#define GENERIC_DOUBLE rte_eal_trace_generic_double(3.66666)#define GENERIC_STR rte_eal_trace_generic_str("hello world")#define VOID_FP app_dpdk_test_fp() +#ifdef RTE_EXEC_ENV_LINUX +/* 0 corresponds first event passed via --trace= */ +#define READ_PMU rte_eal_trace_pmu_read(0) +#endif WORKER_DEFINE (GENERIC_VOID)WORKER_DEFINE (GENERIC_U64)
@@ -122,6 +126,9 @@ WORKER_DEFINE(GENERIC_FLOAT) WORKER_DEFINE(GENERIC_DOUBLE) WORKER_DEFINE(GENERIC_STR) WORKER_DEFINE(VOID_FP) +#ifdef RTE_EXEC_ENV_LINUX +WORKER_DEFINE(READ_PMU) +#endif static void run_test (const char *str,lcore_function_t f,struct test_data *data,size_t sz)
@@ -174,6 +181,9 @@ test_trace_perf(void)run_test ("double" , worker_fn_GENERIC_DOUBLE, data, sz);run_test ("string" , worker_fn_GENERIC_STR, data, sz);run_test ("void_fp" , worker_fn_VOID_FP, data, sz); +#ifdef RTE_EXEC_ENV_LINUX +run_test ("read_pmu" , worker_fn_READ_PMU, data, sz); +#endif rte_free (data);return TEST_SUCCESS ;
doc/guides/prog_guide/profile_app.rst +5 -0
@@ -16,6 +16,11 @@ that information, perf being an example here. Though in some scenarios, eg. when isolated (nohz_full) and run dedicated tasks, using perf is less than ideal. In such cases one can read specific events directly from application via ``rte_pmu_read()``. +Alternatively tracing library can be used which offers dedicated tracepoint +``rte_eal_trace_pmu_event()``. + +Refer to :doc:`../prog_guide/trace_lib` for more details. + Profiling on x86 ----------------
doc/guides/prog_guide/trace_lib.rst +32 -0
@@ -46,6 +46,7 @@ DPDK tracing library features trace format and is compatible with ``LTTng``. For detailed information, refer to `Common Trace Format <https://diamon.org/ctf/>`_. +- Support reading PMU events on ARM64 and x86-64 (Intel) How to add a tracepoint? ------------------------
@@ -137,6 +138,37 @@ the user must use ``RTE_TRACE_POINT_FP`` instead of ``RTE_TRACE_POINT``. ``RTE_TRACE_POINT_FP`` is compiled out by default and it can be enabled using the ``enable_trace_fp`` option for meson build. +PMU tracepoint +-------------- + +Performance measurement unit (PMU) event values can be read from hardware +registers using predefined ``rte_pmu_read`` tracepoint. + +Tracing is enabled via ``--trace`` EAL option by passing both expression +matching PMU tracepoint name i.e ``lib.eal.pmu.read`` and expression +``e=ev1[,ev2,...]`` matching particular events:: + + --trace='.*pmu.read\|e=cpu_cycles,l1d_cache' + +Event names are available under ``/sys/bus/event_source/devices/PMU/events`` +directory, where ``PMU`` is a placeholder for either a ``cpu`` or a directory +containing ``cpus``. + +In contrary to other tracepoints this does not need any extra variables +added to source files. Instead, caller passes index which follows the order of +events specified via ``--trace`` parameter. In the following example index ``0`` +corresponds to ``cpu_cyclces`` while index ``1`` corresponds to ``l1d_cache``. + +.. code-block:: c + + ... + rte_eal_trace_pmu_read(0); + rte_eal_trace_pmu_read(1); + ... + +PMU tracing support must be explicitly enabled using the ``enable_trace_fp`` +option for meson build. + Event record mode -----------------
lib/eal/common/eal_common_trace.c +12 -1
@@ -11,6 +11,9 @@#include <rte_errno.h> #include <rte_lcore.h> #include <rte_per_lcore.h> +#ifdef RTE_EXEC_ENV_LINUX +#include <rte_pmu.h> +#endif #include <rte_string_fns.h> #include "eal_trace.h"
@@ -71,8 +74,13 @@ eal_trace_init(void) goto free_meta;/* Apply global configurations */ - STAILQ_FOREACH(arg, &trace.args, next) + STAILQ_FOREACH(arg, &trace.args, next) {trace_args_apply (arg->val); +#ifdef RTE_EXEC_ENV_LINUX +if (rte_pmu_init () ==0 ) +rte_pmu_add_events_by_pattern (arg->val); +#endif + }rte_trace_mode_set (trace.mode);
@@ -88,6 +96,9 @@ eal_trace_init(void)void eal_trace_fini (void ) { +#ifdef RTE_EXEC_ENV_LINUX +rte_pmu_fini (); +#endif trace_mem_free ();trace_metadata_destroy ();eal_trace_args_free ();
lib/eal/common/eal_common_trace_points.c +5 -0
@@ -75,3 +75,8 @@ RTE_TRACE_POINT_REGISTER(rte_eal_trace_intr_enable, lib.eal.intr.enable) RTE_TRACE_POINT_REGISTER(rte_eal_trace_intr_disable, lib.eal.intr.disable) + +#ifdef RTE_EXEC_ENV_LINUX +RTE_TRACE_POINT_REGISTER (rte_eal_trace_pmu_read, + lib.eal.pmu.read) +#endif
lib/eal/include/rte_eal_trace.h +13 -0
@@ -17,6 +17,9 @@ extern "C" {#include <rte_alarm.h> #include <rte_interrupts.h> +#ifdef RTE_EXEC_ENV_LINUX +#include <rte_pmu.h> +#endif #include <rte_trace_point.h> #include "eal_interrupts.h"
@@ -279,6 +282,16 @@ RTE_TRACE_POINT(rte_trace_point_emit_string (cpuset); ) +#ifdef RTE_EXEC_ENV_LINUX +RTE_TRACE_POINT_FP ( + rte_eal_trace_pmu_read, + RTE_TRACE_POINT_ARGS(unsigned int index), + uint64_t val; + val = rte_pmu_read(index); + rte_trace_point_emit_u64(val); +) +#endif + #ifdef __cplusplus }#endif
lib/eal/meson.build +3 -0
@@ -26,6 +26,9 @@ deps += ['kvargs'] if not is_windows deps += ['telemetry'] endif +if is_linux + deps += ['pmu'] +endif if dpdk_conf.has('RTE_USE_LIBBSD') ext_deps += libbsd endif
lib/eal/version.map +3 -0
@@ -440,6 +440,9 @@ EXPERIMENTAL { rte_thread_detach; rte_thread_equal; rte_thread_join; + + # added in 23.03 + __rte_eal_trace_pmu_read; # WINDOWS_NO_EXPORT }; INTERNAL {
lib/pmu/rte_pmu.c +61 -0
67 lines not on this page · raw message
lib/pmu/rte_pmu.h +14 -0
@@ -169,6 +169,20 @@ __rte_experimentalint rte_pmu_add_event (const char *name); +/** +* @warning +* @b EXPERIMENTAL: this API may change without prior notice +* +* Add events matching pattern to the group of enabled events. +* +* @param pattern +* Pattern e=ev1[,ev2,...] matching events, where evX is a placeholder for an event listed under +* /sys/bus/event_source/devices/pmu/events. +*/ +__rte_experimental +int +rte_pmu_add_events_by_pattern (const char *pattern); +/** * @warning * @b EXPERIMENTAL: this API may change without prior notice
lib/pmu/version.map +1 -0
@@ -8,6 +8,7 @@ EXPERIMENTAL { per_lcore__event_group; rte_pmu; rte_pmu_add_event; + rte_pmu_add_events_by_pattern; rte_pmu_fini; rte_pmu_init; rte_pmu_read;
Signature · 1 line
2.34.1
Quoted 16 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Wednesday, 1 February 2023 14.18This series adds self monitoring support i.e allows to configure and read performance measurement unit (PMU) counters in runtime without using perf utility. This has certain adventages when application runs on
isolated cores withnohz_fullkernel parameter.Events can be read directly using
rte_pmu_read()or using dedicated tracepointrte_eal_trace_pmu_read(). The latter will cause events to be stored inside CTF file.By design, all enabled events are grouped together and the same group is attached to lcores that use self monitoring funtionality.
Quoted 7 lines
Events are enabled by names, which need to be read from standard location under sysfs i.e
/sys/bus/event_source/devices/PMU/eventswhere PMU is a core pmu i.e one measuring cpu events. As of today raw events are not supported.
I like the modifications in v7.
Hi Morten,
Quoted 30 lines
-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: Wednesday, February 1, 2023 2:51 PM
To: Tomasz Duszynski <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; [email protected]; [email protected]
Subject: [EXT] RE: [PATCH v7 0/4] add support for self monitoringExternal Email
----------------------------------------------------------------------
Quoted 18 lines
From: Tomasz Duszynski [mailto:[email protected]]
Sent: Wednesday, 1 February 2023 14.18This series adds self monitoring support i.e allows to configure and read performance measurement unit (PMU) counters in runtime without using perf utility. This has certain adventages when application runs on isolated cores with
nohz_fullkernel parameter.Events can be read directly using
rte_pmu_read()or using dedicated tracepointrte_eal_trace_pmu_read(). The latter will cause events to be stored inside CTF file.By design, all enabled events are grouped together and the same group is attached to lcores that use self monitoring funtionality.
Events are enabled by names, which need to be read from standard location under sysfs i.e
Quoted 8 lines
Quoted 4 lines
/sys/bus/event_source/devices/PMU/eventswhere PMU is a core pmu i.e one measuring cpu events. As of today raw events are not supported.
I like the modifications in v7.
Series-acked-by Morten Brørup <[email protected]>
Thanks.
v8 cover add support for self monitoring
In reply to Tomasz DuszynskiThis series adds self monitoring support i.e allows to configure and read performance measurement unit (PMU) counters in runtime without using perf utility. This has certain adventages when application runs on isolated cores with nohz_full kernel parameter.
Events can be read directly using rte_pmu_read() or using dedicated tracepoint rte_eal_trace_pmu_read(). The latter will cause events to be stored inside CTF file.
By design, all enabled events are grouped together and the same group is attached to lcores that use self monitoring funtionality.
Events are enabled by names, which need to be read from standard location under sysfs i.e
/sys/bus/event_source/devices/PMU/events
where PMU is a core pmu i.e one measuring cpu events. As of today raw events are not supported.
v8:
- just rebase series
v7:
- use per-lcore event group instead of global table index by lcore-id
- don't add pmu_autotest to fast tests because due to lack of suported on every arch
v6:
- move codebase to the separate library
- address review comments
v5:
- address review comments
- fix sign extension while reading pmu on x86
- fix regex mentioned in doc
- various minor changes/improvements here and there
v4:
- fix freeing mem detected by debug_autotest
v3:
- fix shared build
v2:
- fix problems reported by test build infra
Tomasz Duszynski (4):
lib: add generic support for reading PMU events
pmu: support reading ARM PMU events in runtime
pmu: support reading Intel x86_64 PMU events in runtime
eal: add PMU support to tracing library
MAINTAINERS | 5 + app/test/meson.build | 1 + app/test/test_pmu.c | 61 +++ app/test/test_trace_perf.c | 10 + doc/api/doxy-api-index.md | 3 +- doc/api/doxy-api.conf.in | 1 + doc/guides/prog_guide/profile_app.rst | 13 + doc/guides/prog_guide/trace_lib.rst | 32 ++ doc/guides/rel_notes/release_23_03.rst | 9 + lib/eal/common/eal_common_trace.c | 13 +- lib/eal/common/eal_common_trace_points.c | 5 + lib/eal/include/rte_eal_trace.h | 13 + lib/eal/meson.build | 3 + lib/eal/version.map | 1 + lib/meson.build | 1 + lib/pmu/meson.build | 21 + lib/pmu/pmu_arm64.c | 94 ++++ lib/pmu/pmu_private.h | 29 ++ lib/pmu/rte_pmu.c | 525 +++++++++++++++++++++++ lib/pmu/rte_pmu.h | 225 ++++++++++ lib/pmu/rte_pmu_pmc_arm64.h | 30 ++ lib/pmu/rte_pmu_pmc_x86_64.h | 24 ++ lib/pmu/version.map | 21 + 23 files changed, 1138 insertions(+), 2 deletions(-) create mode 100644 app/test/test_pmu.c create mode 100644 lib/pmu/meson.build create mode 100644 lib/pmu/pmu_arm64.c create mode 100644 lib/pmu/pmu_private.h create mode 100644 lib/pmu/rte_pmu.c create mode 100644 lib/pmu/rte_pmu.h create mode 100644 lib/pmu/rte_pmu_pmc_arm64.h create mode 100644 lib/pmu/rte_pmu_pmc_x86_64.h create mode 100644 lib/pmu/version.map
--
2.34.1