Skip to contentSkip to conversation

metrics/windows: build rte_metrics library

PATCH Superseded Tyler Retzlaff 5 participants 13 messages Jan 11, 2021 – Jan 20, 2021

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 metrics/windows: build rte_metrics library

From: Tyler Retzlaff <[email protected]>

declare reduced set of exports for rte_metrics on windows and include it in the set of libraries to build and install.

rte_metrics_tel* exports depend on conditionally available component. this change minimally allows enabling the building of rte_metrics on windows.

---
 lib/librte_metrics/rte_metrics_exports.def | 8 ++++++++
 lib/meson.build                            | 1 +
 2 files changed, 9 insertions(+)
 create mode 100644 lib/librte_metrics/rte_metrics_exports.def
lib/librte_metrics/rte_metrics_exports.def new file +8 -0
new file mode 100644
index 000000000..40ec342a0
@@ -0,0 +1,8 @@
+EXPORTS
+	rte_metrics_get_names
+	rte_metrics_get_values
+	rte_metrics_init
+	rte_metrics_reg_name
+	rte_metrics_reg_names
+	rte_metrics_update_value
+	rte_metrics_update_values
lib/meson.build +1 -0
index ed00f8914..845397fba 100644
@@ -45,6 +45,7 @@ if is_windows
 		'mempool', 'mbuf', 'net', 'meter', 'ethdev', 'pci',
 		'cmdline',
 		'hash',
+		'metrics',
 	] # only supported libraries for windows
 endif
 
Signature · 1 line
2.29.0.vfs.0.0
#1 lore raw

+ Pallavi, Khoa

Quoted 8 lines · Tyler Retzlaff

From: Tyler Retzlaff <[email protected]>

declare reduced set of exports for rte_metrics on windows and include it in the set of libraries to build and install.

rte_metrics_tel* exports depend on conditionally available component. this change minimally allows enabling the building of rte_metrics on windows.

What's the future plan? Once we talked about a script that would inspect objects and filter missing functions from the list in .map to build .def. Is MSFT working on it? Or do we choose another way?

lib/librte_metrics/rte_metrics_exports.def line 4

Quoted 7 lines
lib/librte_metrics/rte_metrics_exports.def new file +4 -0
@@ -0,0 +1,8 @@
+EXPORTS
+	rte_metrics_get_names
+	rte_metrics_get_values
+	rte_metrics_init

+ rte_metrics_deinit

Quoted 4 lines
lib/librte_metrics/rte_metrics_exports.def new file +4 -0
+	rte_metrics_reg_name
+	rte_metrics_reg_names
+	rte_metrics_update_value
+	rte_metrics_update_values
#4 lore raw

PATCH v2 1/1 metrics/windows: build rte_metrics library

In reply to Tyler Retzlaff

From: Tyler Retzlaff <[email protected]>

declare reduced set of exports for rte_metrics on windows and include it in the set of libraries to build and install.

rte_metrics_tel* exports depend on conditionally available component. this change minimally allows enabling the building of rte_metrics on windows.

---
 lib/librte_metrics/rte_metrics_exports.def | 9 +++++++++
 lib/meson.build                            | 1 +
 2 files changed, 10 insertions(+)
 create mode 100644 lib/librte_metrics/rte_metrics_exports.def
lib/librte_metrics/rte_metrics_exports.def new file +9 -0
new file mode 100644
index 000000000..4cb7352ae
@@ -0,0 +1,9 @@
+EXPORTS
+	rte_metrics_get_names
+	rte_metrics_get_values
+	rte_metrics_init
+	rte_metrics_deinit
+	rte_metrics_reg_name
+	rte_metrics_reg_names
+	rte_metrics_update_value
+	rte_metrics_update_values
lib/meson.build +1 -0
index ed00f8914..845397fba 100644
@@ -45,6 +45,7 @@ if is_windows
 		'mempool', 'mbuf', 'net', 'meter', 'ethdev', 'pci',
 		'cmdline',
 		'hash',
+		'metrics',
 	] # only supported libraries for windows
 endif
 
Signature · 1 line
2.29.0.vfs.0.0
#2 lore raw
In reply to Dmitry Kozlyuk
Quoted 5 lines · Dmitry Kozlyuk

+ Pallavi, Khoa

What's the future plan? Once we talked about a script that would inspect objects and filter missing functions from the list in .map to build .def. Is MSFT working on it? Or do we choose another way?

MSFT has a task tracking work to resolve this. We agree it is cumbersome to try and keep the .map and .def files in sync by hand.

#3 lore raw
Quoted 11 lines

Subject: Re: [dpdk-dev] [PATCH] metrics/windows: build rte_metrics library

Quoted 5 lines · Dmitry Kozlyuk

+ Pallavi, Khoa

What's the future plan? Once we talked about a script that would inspect objects and filter missing functions from the list in .map to build .def. Is MSFT working on it? Or do we choose another way?

MSFT has a task tracking work to resolve this. We agree it is cumbersome to try and keep the .map and .def files in sync by hand.

Is there a DD to complete this task? Dmitry already sent an email with the details on how to resolve it using DUMPBIN.

#5 lore raw
In reply to Tyler Retzlaff

12/01/2021 02:30, Tyler Retzlaff:

Quoted 10 lines

From: Tyler Retzlaff <[email protected]>

declare reduced set of exports for rte_metrics on windows and include it in the set of libraries to build and install.

rte_metrics_tel* exports depend on conditionally available component. this change minimally allows enabling the building of rte_metrics on windows.

Not sure it makes sense without the new telemetry feature. Please focus on telemetry lib instead of half-enabling the old metrics lib.

#6 lore raw
Quoted 5 lines · Thomas Monjalon

Not sure it makes sense without the new telemetry feature. Please focus on telemetry lib instead of half-enabling the old metrics lib.

can you elaborate? (or reference a mailing list discussion) that gives some guidance?

is the telemetry lib a replacement for metrics? the component we have now relies on the non-telemetry functions exported from metrics but does not use the telemetry functions.

also, i notice that the meson.build for telemetry lib has an include path that references rte_metrics but does not appear to actually include any of the headers from rte_metrics (vestigial? missed in previous cleanup perhaps?)

thanks!

#7 lore raw

19/01/2021 22:31, Tyler Retzlaff:

Quoted 10 lines · Thomas Monjalon
Quoted 5 lines

Not sure it makes sense without the new telemetry feature. Please focus on telemetry lib instead of half-enabling the old metrics lib.

can you elaborate? (or reference a mailing list discussion) that gives some guidance?

Quoted 7 lines

is the telemetry lib a replacement for metrics? the component we have now relies on the non-telemetry functions exported from metrics but does not use the telemetry functions.

also, i notice that the meson.build for telemetry lib has an include path that references rte_metrics but does not appear to actually include any of the headers from rte_metrics (vestigial? missed in previous cleanup perhaps?)

I think Bruce and Ciara will better explain than me
the intent of the telemetry lib and the compatibility path with the metrics lib.

#8 lore raw
Quoted 15 lines · Thomas Monjalon

19/01/2021 22:31, Tyler Retzlaff:

Quoted 14 lines · Thomas Monjalon
Quoted 5 lines

Not sure it makes sense without the new telemetry feature. Please focus on telemetry lib instead of half-enabling the old metrics lib.

can you elaborate? (or reference a mailing list discussion) that gives some guidance?

is the telemetry lib a replacement for metrics? the component we have now relies on the non-telemetry functions exported from metrics but does not use the telemetry functions.

Quoted 7 lines
Quoted 3 lines

also, i notice that the meson.build for telemetry lib has an include path that references rte_metrics but does not appear to actually include any of the headers from rte_metrics (vestigial? missed in previous cleanup perhaps?)

I think Bruce and Ciara will better explain than me
the intent of the telemetry lib and the compatibility path with the metrics lib.

The include path addition for metrics to the telemetry library does indeed look like it was just missed being removed, since I can compile things up successfully with it removed.

With regards to interaction between telemetry library and metrics library, they are complementary but not replacements for each other. The metrics library provides support for tracking metrics, mostly on a per-port basis. The original telemetry library implementation was based on top of the metrics library and supported reporting out data from that library. More recent versions of the telemetry library have reworked that support to allow the reporting of arbitrary telemetry data, not just from the metrics library, but the old interface is still supported.

/Bruce

#9 lore raw

20/01/2021 11:37, Bruce Richardson:

Quoted 28 lines · Thomas Monjalon
Quoted 15 lines

19/01/2021 22:31, Tyler Retzlaff:

Quoted 14 lines · Thomas Monjalon
Quoted 5 lines

Not sure it makes sense without the new telemetry feature. Please focus on telemetry lib instead of half-enabling the old metrics lib.

can you elaborate? (or reference a mailing list discussion) that gives some guidance?

is the telemetry lib a replacement for metrics? the component we have now relies on the non-telemetry functions exported from metrics but does not use the telemetry functions.

Quoted 7 lines
Quoted 3 lines

also, i notice that the meson.build for telemetry lib has an include path that references rte_metrics but does not appear to actually include any of the headers from rte_metrics (vestigial? missed in previous cleanup perhaps?)

I think Bruce and Ciara will better explain than me
the intent of the telemetry lib and the compatibility path with the metrics lib.

The include path addition for metrics to the telemetry library does indeed look like it was just missed being removed, since I can compile things up successfully with it removed.

Quoted 8 lines

With regards to interaction between telemetry library and metrics library, they are complementary but not replacements for each other. The metrics library provides support for tracking metrics, mostly on a per-port basis. The original telemetry library implementation was based on top of the metrics library and supported reporting out data from that library. More recent versions of the telemetry library have reworked that support to allow the reporting of arbitrary telemetry data, not just from the metrics library, but the old interface is still supported.

The question is what is available in rte_metrics which is not already available with rte_telemetry only?
I think rte_metrics is used only for rte_bitratestats and rte_latencystats. Can we use rte_telemetry for this and forget rte_metrics?

#10 lore raw
Quoted 38 lines · Thomas Monjalon

20/01/2021 11:37, Bruce Richardson:

Quoted 28 lines · Thomas Monjalon
Quoted 15 lines

19/01/2021 22:31, Tyler Retzlaff:

Quoted 14 lines · Thomas Monjalon
Quoted 5 lines

Not sure it makes sense without the new telemetry feature. Please focus on telemetry lib instead of half-enabling the old metrics lib.

can you elaborate? (or reference a mailing list discussion) that gives some guidance?

is the telemetry lib a replacement for metrics? the component we have now relies on the non-telemetry functions exported from metrics but does not use the telemetry functions.

Quoted 7 lines
Quoted 3 lines

also, i notice that the meson.build for telemetry lib has an include path that references rte_metrics but does not appear to actually include any of the headers from rte_metrics (vestigial? missed in previous cleanup perhaps?)

I think Bruce and Ciara will better explain than me
the intent of the telemetry lib and the compatibility path with the metrics lib.

The include path addition for metrics to the telemetry library does indeed look like it was just missed being removed, since I can compile things up successfully with it removed.

Quoted 8 lines

With regards to interaction between telemetry library and metrics library, they are complementary but not replacements for each other. The metrics library provides support for tracking metrics, mostly on a per-port basis. The original telemetry library implementation was based on top of the metrics library and supported reporting out data from that library. More recent versions of the telemetry library have reworked that support to allow the reporting of arbitrary telemetry data, not just from the metrics library, but the old interface is still supported.

Quoted 5 lines

The question is what is available in rte_metrics which is not already available with rte_telemetry only?
I think rte_metrics is used only for rte_bitratestats and rte_latencystats. Can we use rte_telemetry for this and forget rte_metrics?

No, we can't.
In short, metrics is a data store. Telemetry is a data output mechanism.

Telemetry cannot directly replace metrics because it does not store any data, it simply matches incoming requests against callbacks to retrieve and export data. The actual management of the underlying data sources is the responsibility of the library/app which has registered the callback.

Metrics, on the other hand, is a data storage library, and allows the registration of groups of statictics which can be tracked and updated. It does also provide some callbacks to allow the data to be output by telemetry, but this functionality is not the primary goal of the lib. Both bitratestats and latencystats use metrics to manage their stats data - again any output provided is incidental to the primary purpose.

Regards,
/Bruce

#11 lore raw

20/01/2021 12:54, Bruce Richardson:

Quoted 52 lines · Thomas Monjalon
Quoted 38 lines

20/01/2021 11:37, Bruce Richardson:

Quoted 28 lines · Thomas Monjalon
Quoted 15 lines

19/01/2021 22:31, Tyler Retzlaff:

Quoted 14 lines · Thomas Monjalon
Quoted 5 lines

Not sure it makes sense without the new telemetry feature. Please focus on telemetry lib instead of half-enabling the old metrics lib.

can you elaborate? (or reference a mailing list discussion) that gives some guidance?

is the telemetry lib a replacement for metrics? the component we have now relies on the non-telemetry functions exported from metrics but does not use the telemetry functions.

Quoted 7 lines
Quoted 3 lines

also, i notice that the meson.build for telemetry lib has an include path that references rte_metrics but does not appear to actually include any of the headers from rte_metrics (vestigial? missed in previous cleanup perhaps?)

I think Bruce and Ciara will better explain than me
the intent of the telemetry lib and the compatibility path with the metrics lib.

The include path addition for metrics to the telemetry library does indeed look like it was just missed being removed, since I can compile things up successfully with it removed.

Quoted 8 lines

With regards to interaction between telemetry library and metrics library, they are complementary but not replacements for each other. The metrics library provides support for tracking metrics, mostly on a per-port basis. The original telemetry library implementation was based on top of the metrics library and supported reporting out data from that library. More recent versions of the telemetry library have reworked that support to allow the reporting of arbitrary telemetry data, not just from the metrics library, but the old interface is still supported.

Quoted 5 lines

The question is what is available in rte_metrics which is not already available with rte_telemetry only?
I think rte_metrics is used only for rte_bitratestats and rte_latencystats. Can we use rte_telemetry for this and forget rte_metrics?

No, we can't.
In short, metrics is a data store. Telemetry is a data output mechanism.

Telemetry cannot directly replace metrics because it does not store any data, it simply matches incoming requests against callbacks to retrieve and export data. The actual management of the underlying data sources is the responsibility of the library/app which has registered the callback.

Quoted 6 lines

Metrics, on the other hand, is a data storage library, and allows the registration of groups of statictics which can be tracked and updated. It does also provide some callbacks to allow the data to be output by telemetry, but this functionality is not the primary goal of the lib. Both bitratestats and latencystats use metrics to manage their stats data - again any output provided is incidental to the primary purpose.

When you say "we can't", you mean it is not a desirable addition? I am fine with keeping the design as is, split in libs.

#12 lore raw
Quoted 60 lines · Thomas Monjalon

20/01/2021 12:54, Bruce Richardson:

Quoted 52 lines · Thomas Monjalon
Quoted 38 lines

20/01/2021 11:37, Bruce Richardson:

Quoted 28 lines · Thomas Monjalon
Quoted 15 lines

19/01/2021 22:31, Tyler Retzlaff:

Quoted 14 lines · Thomas Monjalon
Quoted 5 lines

Not sure it makes sense without the new telemetry feature. Please focus on telemetry lib instead of half-enabling the old metrics lib.

can you elaborate? (or reference a mailing list discussion) that gives some guidance?

is the telemetry lib a replacement for metrics? the component we have now relies on the non-telemetry functions exported from metrics but does not use the telemetry functions.

Quoted 7 lines
Quoted 3 lines

also, i notice that the meson.build for telemetry lib has an include path that references rte_metrics but does not appear to actually include any of the headers from rte_metrics (vestigial? missed in previous cleanup perhaps?)

I think Bruce and Ciara will better explain than me
the intent of the telemetry lib and the compatibility path with the metrics lib.

The include path addition for metrics to the telemetry library does indeed look like it was just missed being removed, since I can compile things up successfully with it removed.

Quoted 8 lines

With regards to interaction between telemetry library and metrics library, they are complementary but not replacements for each other. The metrics library provides support for tracking metrics, mostly on a per-port basis. The original telemetry library implementation was based on top of the metrics library and supported reporting out data from that library. More recent versions of the telemetry library have reworked that support to allow the reporting of arbitrary telemetry data, not just from the metrics library, but the old interface is still supported.

Quoted 5 lines

The question is what is available in rte_metrics which is not already available with rte_telemetry only?
I think rte_metrics is used only for rte_bitratestats and rte_latencystats. Can we use rte_telemetry for this and forget rte_metrics?

No, we can't.
In short, metrics is a data store. Telemetry is a data output mechanism.

Telemetry cannot directly replace metrics because it does not store any data, it simply matches incoming requests against callbacks to retrieve and export data. The actual management of the underlying data sources is the responsibility of the library/app which has registered the callback.

Quoted 6 lines

Metrics, on the other hand, is a data storage library, and allows the registration of groups of statictics which can be tracked and updated. It does also provide some callbacks to allow the data to be output by telemetry, but this functionality is not the primary goal of the lib. Both bitratestats and latencystats use metrics to manage their stats data - again any output provided is incidental to the primary purpose.

Quoted 3 lines

When you say "we can't", you mean it is not a desirable addition? I am fine with keeping the design as is, split in libs.

Well, I primarily meant "we can't" as in the library is not designed to do that right now, and so cannot do what is asked. For the second interpretation of it being a desirable addition or not, I also agree that expanding the scope of telemetry to also cover data management is not something we want. Any rework of metrics and associated library should be kept separate from telemetry, and allow each lib to try and focus on just doing one thing. So +1 for keeping things split.

/Bruce