Skip to contentSkip to conversation

metrics: make number of metrics names configurable

PATCH In review Stephen Hemminger 3 participants 10 messages Jul 2, 2020 – Oct 20, 2020 Arm/Amlogic Meson SoC support

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 metrics: make number of metrics names configurable

The maximum number of metrics is hardcoded at 256. This severely limits the usefulness of the library. It should be configurable like other limits in DPDK.

Fixes 349950ddb9c5 ("metrics: add information metrics library")
---
 config/common_base               | 1 +
 config/meson.build               | 2 +-
 lib/librte_metrics/rte_metrics.h | 1 -
 3 files changed, 2 insertions(+), 2 deletions(-)
config/common_base +1 -0
index fe30c515e5a3..f0212faec80c 100644
@@ -917,6 +917,7 @@ CONFIG_RTE_LIBRTE_JOBSTATS=y
 # Compile the device metrics library
 #
 CONFIG_RTE_LIBRTE_METRICS=y
+CONFIG_RTE_METRICS_MAX_METRICS=256
 
 #
 # Compile the bitrate statistics library
config/meson.build +1 -1
index 351e268c1f5b..cc8cb8fbf2f0 100644
@@ -238,7 +238,7 @@ dpdk_conf.set('RTE_ENABLE_TRACE_FP', get_option('enable_trace_fp'))
 dpdk_conf.set('RTE_MAX_VFIO_GROUPS', 64)
 dpdk_conf.set('RTE_DRIVER_MEMPOOL_BUCKET_SIZE_KB', 64)
 dpdk_conf.set('RTE_LIBRTE_DPAA2_USE_PHYS_IOVA', true)
-
+dpdk_conf.set('RTE_METRICS_MAX_METRICS', 256)
 
 compile_time_cpuflags = []
 subdir(arch_subdir)
lib/librte_metrics/rte_metrics.h +0 -1
index fbe64ddf2b47..40f015b8bb93 100644
@@ -34,7 +34,6 @@ extern int metrics_initialized;
 
 /** Maximum length of metric name (including null-terminator) */
 #define RTE_METRICS_MAX_NAME_LEN 64
-#define RTE_METRICS_MAX_METRICS 256
 
 /**
  * Global metric special id.
Signature · 1 line
2.26.2
#1 lore raw
Quoted 8 lines · On Thu, 2 Jul 2020 10:28:52 -0700 Stephen Hemminger…

The maximum number of metrics is hardcoded at 256. This severely limits the usefulness of the library. It should be configurable like other limits in DPDK.

Fixes 349950ddb9c5 ("metrics: add information metrics library")

Ping, this should be addressed, and doesn't require much effort to review.

#2 lore raw
In reply to Stephen Hemminger
Quoted 31 lines · Stephen Hemminger

The maximum number of metrics is hardcoded at 256. This severely limits the usefulness of the library. It should be configurable like other limits in DPDK.

Fixes 349950ddb9c5 ("metrics: add information metrics library")
---
 config/common_base               | 1 +
 config/meson.build               | 2 +-
 lib/librte_metrics/rte_metrics.h | 1 -
 3 files changed, 2 insertions(+), 2 deletions(-)
config/common_base +1 -0
index fe30c515e5a3..f0212faec80c 100644
@@ -917,6 +917,7 @@ CONFIG_RTE_LIBRTE_JOBSTATS=y
 # Compile the device metrics library
 #
 CONFIG_RTE_LIBRTE_METRICS=y
+CONFIG_RTE_METRICS_MAX_METRICS=256
 
 #
 # Compile the bitrate statistics library
config/meson.build +0 -0
index 351e268c1f5b..cc8cb8fbf2f0 100644

config/meson.build dpdk_conf.set('RTE_ENABLE_TRACE_FP', get_option('enable_trace_fp')) · line 242

Quoted 6 lines
config/meson.build +1 -1
@@ -238,7 +238,7 @@ dpdk_conf.set('RTE_ENABLE_TRACE_FP', get_option('enable_trace_fp'))
 dpdk_conf.set('RTE_MAX_VFIO_GROUPS', 64)
 dpdk_conf.set('RTE_DRIVER_MEMPOOL_BUCKET_SIZE_KB', 64)
 dpdk_conf.set('RTE_LIBRTE_DPAA2_USE_PHYS_IOVA', true)
-
+dpdk_conf.set('RTE_METRICS_MAX_METRICS', 256)
 

The meson.build file should really just be used for computed values, I think. For build-time constants like this it's probably better put in config/rte_config.h file.

Regards,
/Bruce

#3 lore raw
Quoted 34 lines · On Tue, 28 Jul 2020 11:24:58 +0100 Bruce Richardson…
Quoted 33 lines · Stephen Hemminger

The maximum number of metrics is hardcoded at 256. This severely limits the usefulness of the library. It should be configurable like other limits in DPDK.

Fixes 349950ddb9c5 ("metrics: add information metrics library")
---
 config/common_base               | 1 +
 config/meson.build               | 2 +-
 lib/librte_metrics/rte_metrics.h | 1 -
 3 files changed, 2 insertions(+), 2 deletions(-)
config/common_base +1 -0
index fe30c515e5a3..f0212faec80c 100644
@@ -917,6 +917,7 @@ CONFIG_RTE_LIBRTE_JOBSTATS=y
 # Compile the device metrics library
 #
 CONFIG_RTE_LIBRTE_METRICS=y
+CONFIG_RTE_METRICS_MAX_METRICS=256
 
 #
 # Compile the bitrate statistics library
config/meson.build +0 -0
index 351e268c1f5b..cc8cb8fbf2f0 100644
@@ -238,7 +238,7 @@ dpdk_conf.set('RTE_ENABLE_TRACE_FP', get_option('enable_trace_fp'))
 dpdk_conf.set('RTE_MAX_VFIO_GROUPS', 64)
 dpdk_conf.set('RTE_DRIVER_MEMPOOL_BUCKET_SIZE_KB', 64)
Quoted 8 lines

config/meson.build dpdk_conf.set('RTE_ENABLE_TRACE_FP', get_option('enable_trace_fp')) · line 242

Quoted 4 lines
config/meson.build +1 -1
 dpdk_conf.set('RTE_LIBRTE_DPAA2_USE_PHYS_IOVA', true)
-
+dpdk_conf.set('RTE_METRICS_MAX_METRICS', 256)
 

The meson.build file should really just be used for computed values, I think. For build-time constants like this it's probably better put in config/rte_config.h file.

config/rte_config.h is generated isn't it?

#4 lore raw
Quoted 46 lines · Stephen Hemminger
Quoted 34 lines · On Tue, 28 Jul 2020 11:24:58 +0100 Bruce Richardson…
Quoted 33 lines · Stephen Hemminger

The maximum number of metrics is hardcoded at 256. This severely limits the usefulness of the library. It should be configurable like other limits in DPDK.

Fixes 349950ddb9c5 ("metrics: add information metrics library")
---
 config/common_base               | 1 +
 config/meson.build               | 2 +-
 lib/librte_metrics/rte_metrics.h | 1 -
 3 files changed, 2 insertions(+), 2 deletions(-)
config/common_base +1 -0
index fe30c515e5a3..f0212faec80c 100644
@@ -917,6 +917,7 @@ CONFIG_RTE_LIBRTE_JOBSTATS=y
 # Compile the device metrics library
 #
 CONFIG_RTE_LIBRTE_METRICS=y
+CONFIG_RTE_METRICS_MAX_METRICS=256
 
 #
 # Compile the bitrate statistics library
config/meson.build +0 -0
index 351e268c1f5b..cc8cb8fbf2f0 100644
@@ -238,7 +238,7 @@ dpdk_conf.set('RTE_ENABLE_TRACE_FP', get_option('enable_trace_fp'))
 dpdk_conf.set('RTE_MAX_VFIO_GROUPS', 64)
 dpdk_conf.set('RTE_DRIVER_MEMPOOL_BUCKET_SIZE_KB', 64)
Quoted 8 lines

config/meson.build dpdk_conf.set('RTE_ENABLE_TRACE_FP', get_option('enable_trace_fp')) · line 242

Quoted 4 lines
config/meson.build +1 -1
 dpdk_conf.set('RTE_LIBRTE_DPAA2_USE_PHYS_IOVA', true)
-
+dpdk_conf.set('RTE_METRICS_MAX_METRICS', 256)
 

The meson.build file should really just be used for computed values, I think. For build-time constants like this it's probably better put in config/rte_config.h file.

Quoted 2 lines

config/rte_config.h is generated isn't it?

For make builds it is, for meson there is a static rte_config.h which includes the dynamically-generated rte_build_config.h.

While nothing mandates this design, and we can change it if people find it confusing, I though it worthwhile to have a config header file for stuff like this that is not normally configurable, but some folks might want to tweak in custom builds. For 20.11 I'll try and remember to update the docs to cover it a bit, so that people are more aware its there.

/Bruce

#5 lore raw

PATCH 1/1 rte_metrics: move maximum number of metrics into rte_config

In reply to Stephen Hemminger

If using lots of queues and ports, and having per port or per queue metrics it is easy to exceed the upper bound of the metric library. Move the limit into rte_config where user can change it.

Ideally, there would be no upper bound and a dynamic structure such as red-black tree or hash table would be used for these.

---
 config/rte_config.h              | 3 +++
 lib/librte_metrics/rte_metrics.h | 1 -
 2 files changed, 3 insertions(+), 1 deletion(-)
config/rte_config.h +3 -0
index 9bb915347cb6..b880974f5787 100644
@@ -104,6 +104,9 @@
 #define RTE_GRAPH_BURST_SIZE 256
 #define RTE_LIBRTE_GRAPH_STATS 1
 
+/* rte_metrics defines */
+#define RTE_METRICS_MAX_METRICS 256
+
 /****** driver defines ********/
 
 /* QuickAssist device */
lib/librte_metrics/rte_metrics.h +0 -1
index fbe64ddf2b47..40f015b8bb93 100644
@@ -34,7 +34,6 @@ extern int metrics_initialized;
 
 /** Maximum length of metric name (including null-terminator) */
 #define RTE_METRICS_MAX_NAME_LEN 64
-#define RTE_METRICS_MAX_METRICS 256
 
 /**
  * Global metric special id.
Signature · 1 line
2.27.0
#6 lore raw

Re: [dpdk-dev] [PATCH] rte_metrics: move maximum number of metrics into rte_config

Quoted 8 lines · On Fri, 4 Sep 2020 15:31:19 -0700 Stephen Hemminger…

If using lots of queues and ports, and having per port or per queue metrics it is easy to exceed the upper bound of the metric library. Move the limit into rte_config where user can change it.

Ideally, there would be no upper bound and a dynamic structure such as red-black tree or hash table would be used for these.

Note: this version of the patch assumes meson build, it won't work with make based build.

Ignore the CI failures.

#7 lore raw

Re: [dpdk-dev] [PATCH] rte_metrics: move maximum number of metrics into rte_config

In reply to Stephen Hemminger

05/09/2020 00:31, Stephen Hemminger:

config/rte_config.h line 108

Quoted 13 lines

If using lots of queues and ports, and having per port or per queue metrics it is easy to exceed the upper bound of the metric library. Move the limit into rte_config where user can change it.

Ideally, there would be no upper bound and a dynamic structure such as red-black tree or hash table would be used for these.

---

config/rte_config.h +2 -0
+/* rte_metrics defines */
+#define RTE_METRICS_MAX_METRICS 256

Not sure we want to go in the direction of adding such tuning in rte_config.h.

#8 lore raw

Re: [dpdk-dev] [PATCH] rte_metrics: move maximum number of metrics into rte_config

Quoted 8 lines · On Tue, 20 Oct 2020 13:50:55 +0200 Thomas Monjalon <[email protected]> wrote:

05/09/2020 00:31, Stephen Hemminger:

Quoted 7 lines

If using lots of queues and ports, and having per port or per queue metrics it is easy to exceed the upper bound of the metric library. Move the limit into rte_config where user can change it.

Ideally, there would be no upper bound and a dynamic structure such as red-black tree or hash table would be used for these.

Quoted 11 lines

config/rte_config.h line 108

Quoted 6 lines

---

config/rte_config.h +2 -0
+/* rte_metrics defines */
+#define RTE_METRICS_MAX_METRICS 256  

Not sure we want to go in the direction of adding such tuning in rte_config.h.

The only other option is to rewrite rte_metrics to support a dynamic data structure!

#9 lore raw

Re: [dpdk-dev] [PATCH] rte_metrics: move maximum number of metrics into rte_config

20/10/2020 17:02, Stephen Hemminger:

Quoted 19 lines
Quoted 8 lines · Thomas Monjalon

05/09/2020 00:31, Stephen Hemminger:

Quoted 7 lines

If using lots of queues and ports, and having per port or per queue metrics it is easy to exceed the upper bound of the metric library. Move the limit into rte_config where user can change it.

Ideally, there would be no upper bound and a dynamic structure such as red-black tree or hash table would be used for these.

Quoted 9 lines

config/rte_config.h line 108

Quoted 6 lines

---

config/rte_config.h +2 -0
+/* rte_metrics defines */
+#define RTE_METRICS_MAX_METRICS 256  

Not sure we want to go in the direction of adding such tuning in rte_config.h.

Quoted 2 lines

The only other option is to rewrite rte_metrics to support a dynamic data structure!

Yes it seems a better solution.

How rte_metrics will fit in the picture of the new telemetry? Maybe we will deprecate rte_metrics in future?