metrics: make number of metrics names configurable
PATCH 1/1 metrics: make number of metrics names configurable
The maximum number of metrics is hardcoded at 256. This severely limits the usefulness of the library. It should be configurable like other limits in DPDK.
--- config/common_base | 1 + config/meson.build | 2 +- lib/librte_metrics/rte_metrics.h | 1 - 3 files changed, 2 insertions(+), 2 deletions(-)
config/common_base +1 -0
@@ -917,6 +917,7 @@ CONFIG_RTE_LIBRTE_JOBSTATS=y # Compile the device metrics library # CONFIG_RTE_LIBRTE_METRICS=y +CONFIG_RTE_METRICS_MAX_METRICS=256 # # Compile the bitrate statistics library
config/meson.build +1 -1
@@ -238,7 +238,7 @@ dpdk_conf.set('RTE_ENABLE_TRACE_FP', get_option('enable_trace_fp')) dpdk_conf.set('RTE_MAX_VFIO_GROUPS', 64) dpdk_conf.set('RTE_DRIVER_MEMPOOL_BUCKET_SIZE_KB', 64) dpdk_conf.set('RTE_LIBRTE_DPAA2_USE_PHYS_IOVA', true) - +dpdk_conf.set('RTE_METRICS_MAX_METRICS', 256) compile_time_cpuflags = [] subdir(arch_subdir)
lib/librte_metrics/rte_metrics.h +0 -1
@@ -34,7 +34,6 @@ extern int metrics_initialized;/** Maximum length of metric name (including null-terminator) */ #define RTE_METRICS_MAX_NAME_LEN 64 -#define RTE_METRICS_MAX_METRICS 256/** * Global metric special id.
Signature · 1 line
2.26.2
Quoted 8 lines · On Thu, 2 Jul 2020 10:28:52 -0700 Stephen Hemminger…
The maximum number of metrics is hardcoded at 256. This severely limits the usefulness of the library. It should be configurable like other limits in DPDK.
Fixes 349950ddb9c5 ("metrics: add information metrics library")Signed-off-by Stephen Hemminger <[email protected]>
Ping, this should be addressed, and doesn't require much effort to review.
Quoted 31 lines · Stephen Hemminger
The maximum number of metrics is hardcoded at 256. This severely limits the usefulness of the library. It should be configurable like other limits in DPDK.
Fixes 349950ddb9c5 ("metrics: add information metrics library")Signed-off-by Stephen Hemminger <[email protected]>--- config/common_base | 1 + config/meson.build | 2 +- lib/librte_metrics/rte_metrics.h | 1 - 3 files changed, 2 insertions(+), 2 deletions(-)config/common_base +1
-0@@ -917,6 +917,7 @@ CONFIG_RTE_LIBRTE_JOBSTATS=y # Compile the device metrics library # CONFIG_RTE_LIBRTE_METRICS=y +CONFIG_RTE_METRICS_MAX_METRICS=256 # # Compile the bitrate statistics libraryconfig/meson.build +0
-0
config/meson.build dpdk_conf.set('RTE_ENABLE_TRACE_FP', get_option('enable_trace_fp')) · line 242
Quoted 6 lines
config/meson.build +1
-1@@ -238,7 +238,7 @@ dpdk_conf.set('RTE_ENABLE_TRACE_FP', get_option('enable_trace_fp')) dpdk_conf.set('RTE_MAX_VFIO_GROUPS', 64) dpdk_conf.set('RTE_DRIVER_MEMPOOL_BUCKET_SIZE_KB', 64) dpdk_conf.set('RTE_LIBRTE_DPAA2_USE_PHYS_IOVA', true) - +dpdk_conf.set('RTE_METRICS_MAX_METRICS', 256)
The meson.build file should really just be used for computed values, I think. For build-time constants like this it's probably better put in config/rte_config.h file.
Regards,
/Bruce
Quoted 34 lines · On Tue, 28 Jul 2020 11:24:58 +0100 Bruce Richardson…
Quoted 33 lines · Stephen Hemminger
The maximum number of metrics is hardcoded at 256. This severely limits the usefulness of the library. It should be configurable like other limits in DPDK.
Fixes 349950ddb9c5 ("metrics: add information metrics library")Signed-off-by Stephen Hemminger <[email protected]>--- config/common_base | 1 + config/meson.build | 2 +- lib/librte_metrics/rte_metrics.h | 1 - 3 files changed, 2 insertions(+), 2 deletions(-)config/common_base +1
-0@@ -917,6 +917,7 @@ CONFIG_RTE_LIBRTE_JOBSTATS=y # Compile the device metrics library # CONFIG_RTE_LIBRTE_METRICS=y +CONFIG_RTE_METRICS_MAX_METRICS=256 # # Compile the bitrate statistics libraryconfig/meson.build +0
-0@@ -238,7 +238,7 @@ dpdk_conf.set('RTE_ENABLE_TRACE_FP', get_option('enable_trace_fp')) dpdk_conf.set('RTE_MAX_VFIO_GROUPS', 64) dpdk_conf.set('RTE_DRIVER_MEMPOOL_BUCKET_SIZE_KB', 64)
Quoted 8 lines
config/meson.build dpdk_conf.set('RTE_ENABLE_TRACE_FP', get_option('enable_trace_fp')) · line 242
Quoted 4 lines
config/meson.build +1
-1dpdk_conf.set('RTE_LIBRTE_DPAA2_USE_PHYS_IOVA', true) - +dpdk_conf.set('RTE_METRICS_MAX_METRICS', 256)The meson.build file should really just be used for computed values, I think. For build-time constants like this it's probably better put in
config/rte_config.hfile.
config/rte_config.h is generated isn't it?
Quoted 46 lines · Stephen Hemminger
Quoted 34 lines · On Tue, 28 Jul 2020 11:24:58 +0100 Bruce Richardson…
Quoted 33 lines · Stephen Hemminger
The maximum number of metrics is hardcoded at 256. This severely limits the usefulness of the library. It should be configurable like other limits in DPDK.
Fixes 349950ddb9c5 ("metrics: add information metrics library")Signed-off-by Stephen Hemminger <[email protected]>--- config/common_base | 1 + config/meson.build | 2 +- lib/librte_metrics/rte_metrics.h | 1 - 3 files changed, 2 insertions(+), 2 deletions(-)config/common_base +1
-0@@ -917,6 +917,7 @@ CONFIG_RTE_LIBRTE_JOBSTATS=y # Compile the device metrics library # CONFIG_RTE_LIBRTE_METRICS=y +CONFIG_RTE_METRICS_MAX_METRICS=256 # # Compile the bitrate statistics libraryconfig/meson.build +0
-0@@ -238,7 +238,7 @@ dpdk_conf.set('RTE_ENABLE_TRACE_FP', get_option('enable_trace_fp')) dpdk_conf.set('RTE_MAX_VFIO_GROUPS', 64) dpdk_conf.set('RTE_DRIVER_MEMPOOL_BUCKET_SIZE_KB', 64)Quoted 8 lines
config/meson.build dpdk_conf.set('RTE_ENABLE_TRACE_FP', get_option('enable_trace_fp')) · line 242
Quoted 4 lines
config/meson.build +1
-1dpdk_conf.set('RTE_LIBRTE_DPAA2_USE_PHYS_IOVA', true) - +dpdk_conf.set('RTE_METRICS_MAX_METRICS', 256)The meson.build file should really just be used for computed values, I think. For build-time constants like this it's probably better put in
config/rte_config.hfile.
Quoted 2 lines
config/rte_config.his generated isn't it?
For make builds it is, for meson there is a static rte_config.h which includes the dynamically-generated rte_build_config.h.
While nothing mandates this design, and we can change it if people find it confusing, I though it worthwhile to have a config header file for stuff like this that is not normally configurable, but some folks might want to tweak in custom builds. For 20.11 I'll try and remember to update the docs to cover it a bit, so that people are more aware its there.
/Bruce
PATCH 1/1 rte_metrics: move maximum number of metrics into rte_config
In reply to Stephen HemmingerIf using lots of queues and ports, and having per port or per queue metrics it is easy to exceed the upper bound of the metric library. Move the limit into rte_config where user can change it.
Ideally, there would be no upper bound and a dynamic structure such as red-black tree or hash table would be used for these.
--- config/rte_config.h | 3 +++ lib/librte_metrics/rte_metrics.h | 1 - 2 files changed, 3 insertions(+), 1 deletion(-)
config/rte_config.h +3 -0
@@ -104,6 +104,9 @@#define RTE_GRAPH_BURST_SIZE 256#define RTE_LIBRTE_GRAPH_STATS 1 +/* rte_metrics defines */ +#define RTE_METRICS_MAX_METRICS 256 +/****** driver defines ********/ /* QuickAssist device */
lib/librte_metrics/rte_metrics.h +0 -1
@@ -34,7 +34,6 @@ extern int metrics_initialized;/** Maximum length of metric name (including null-terminator) */ #define RTE_METRICS_MAX_NAME_LEN 64 -#define RTE_METRICS_MAX_METRICS 256/** * Global metric special id.
Signature · 1 line
2.27.0
Re: [dpdk-dev] [PATCH] rte_metrics: move maximum number of metrics into rte_config
Quoted 8 lines · On Fri, 4 Sep 2020 15:31:19 -0700 Stephen Hemminger…
If using lots of queues and ports, and having per port or per queue metrics it is easy to exceed the upper bound of the metric library. Move the limit into
rte_configwhere user can change it.Ideally, there would be no upper bound and a dynamic structure such as red-black tree or hash table would be used for these.
Signed-off-by Stephen Hemminger <[email protected]>
Note: this version of the patch assumes meson build, it won't work with make based build.
Ignore the CI failures.
Re: [dpdk-dev] [PATCH] rte_metrics: move maximum number of metrics into rte_config
In reply to Stephen Hemminger05/09/2020 00:31, Stephen Hemminger:
config/rte_config.h line 108
Quoted 13 lines
If using lots of queues and ports, and having per port or per queue metrics it is easy to exceed the upper bound of the metric library. Move the limit into
rte_configwhere user can change it.Ideally, there would be no upper bound and a dynamic structure such as red-black tree or hash table would be used for these.
Signed-off-by Stephen Hemminger <[email protected]>---
config/rte_config.h +2
-0+/* rte_metrics defines */ +#define RTE_METRICS_MAX_METRICS 256
Not sure we want to go in the direction of adding such tuning in rte_config.h.
Re: [dpdk-dev] [PATCH] rte_metrics: move maximum number of metrics into rte_config
Quoted 8 lines · On Tue, 20 Oct 2020 13:50:55 +0200 Thomas Monjalon <[email protected]> wrote:
05/09/2020 00:31, Stephen Hemminger:
Quoted 7 lines
If using lots of queues and ports, and having per port or per queue metrics it is easy to exceed the upper bound of the metric library. Move the limit into
rte_configwhere user can change it.Ideally, there would be no upper bound and a dynamic structure such as red-black tree or hash table would be used for these.
Quoted 11 lines
config/rte_config.h line 108
Quoted 6 lines
Signed-off-by Stephen Hemminger <[email protected]>---
config/rte_config.h +2
-0+/* rte_metrics defines */ +#define RTE_METRICS_MAX_METRICS 256Not sure we want to go in the direction of adding such tuning in
rte_config.h.
The only other option is to rewrite rte_metrics to support a dynamic data structure!
Re: [dpdk-dev] [PATCH] rte_metrics: move maximum number of metrics into rte_config
20/10/2020 17:02, Stephen Hemminger:
Quoted 19 lines
Quoted 8 lines · Thomas Monjalon
05/09/2020 00:31, Stephen Hemminger:
Quoted 7 lines
If using lots of queues and ports, and having per port or per queue metrics it is easy to exceed the upper bound of the metric library. Move the limit into
rte_configwhere user can change it.Ideally, there would be no upper bound and a dynamic structure such as red-black tree or hash table would be used for these.
Quoted 9 lines
config/rte_config.h line 108
Quoted 6 lines
Signed-off-by Stephen Hemminger <[email protected]>---
config/rte_config.h +2
-0+/* rte_metrics defines */ +#define RTE_METRICS_MAX_METRICS 256Not sure we want to go in the direction of adding such tuning in
rte_config.h.
Quoted 2 lines
The only other option is to rewrite
rte_metricsto support a dynamic data structure!
Yes it seems a better solution.
How rte_metrics will fit in the picture of the new telemetry? Maybe we will deprecate rte_metrics in future?