lib/telemetry: add support to fetch global metrics
PATCH 1/1 lib/telemetry: add support to fetch global metrics
telemetry has support for fetching port based stats from metrics library.
Metrics library also has global stats which are
not fetched by telemetry, so extend telemetry to
fetch the global metrics.
--- doc/guides/howto/telemetry.rst | 9 +- doc/guides/rel_notes/release_19_08.rst | 5 + lib/librte_telemetry/rte_telemetry.c | 119 ++++++++++++---- lib/librte_telemetry/rte_telemetry_internal.h | 31 ++++- lib/librte_telemetry/rte_telemetry_parser.c | 130 +++++++++++++++--- usertools/dpdk-telemetry-client.py | 13 +- 6 files changed, 253 insertions(+), 54 deletions(-)
doc/guides/howto/telemetry.rst +5 -4
@@ -11,9 +11,10 @@ Introduction ------------ The ``librte_telemetry`` provides the functionality so that users may query -metrics from incoming port traffic. The application which initializes packet -forwarding will act as the server, sending metrics to the requesting application -which acts as the client. +metrics from incoming port traffic and global stats(application stats). +The application which initializes packet forwarding will act as the server, +sending metrics to the requesting application which acts as the client. + In DPDK, applications are used to initialize the ``telemetry``. To view incoming traffic on featured ports, the application should be run first (ie. after ports
@@ -79,7 +80,7 @@ any DPDK application is applicable. the menu. #. Send traffic to any or all available ports from a traffic generator. - Select a query option(recursive or singular polling). + Select a query option(recursive or singular polling or global stats). The metrics will then be displayed on the client terminal in JSON format. #. Once finished, unregister the client using the menu command.
doc/guides/rel_notes/release_19_08.rst +5 -0
@@ -54,6 +54,11 @@ New Features Also, make sure to start the actual text at the margin. ========================================================= +* **Updated telemetry library for global metrics support.** + + Updated ``librte_telemetry`` to fetch the global metrics from the + ``librte_metrics`` library. + Removed Items -------------
lib/librte_telemetry/rte_telemetry.c +89 -30
176 lines not on this page · raw message
lib/librte_telemetry/rte_telemetry_internal.h +29 -2
@@ -24,6 +24,8 @@ extern int telemetry_log_level; #define TELEMETRY_LOG_INFO(fmt, args...) \ TELEMETRY_LOG(INFO, fmt, ## args) +#define MAX_METRICS 256 + typedef struct telemetry_client { char *file_path; int fd;
@@ -48,6 +50,28 @@ enum rte_telemetry_parser_actions { ACTION_DELETE = 2 }; +enum rte_telemetry_stats_type { +PORT_STATS =0 , +GLOBAL_STATS =1 +}; + +/* @internal */ +struct telemetry_encode_param { +enum rte_telemetry_stats_type type; +union { +struct port_param { +uint32_t num_metric_ids; +uint32_t metric_ids[MAX_METRICS ]; +uint32_t num_port_ids; +uint32_t port_ids[RTE_MAX_ETHPORTS ]; + } pp; +struct global_param { +uint32_t num_metric_ids; +uint32_t metric_ids[MAX_METRICS ]; + } gp; + }; +}; +int32_t rte_telemetry_parse_client_message (struct telemetry_impl *telemetry,char *buf);
@@ -72,10 +96,13 @@ int32_t rte_telemetry_is_port_active(int port_id);int32_t -rte_telemetry_send_ports_stats_values (uint32_t *metric_ids,int num_metric_ids, -uint32_t *port_ids,int num_port_ids,struct telemetry_impl *telemetry); +rte_telemetry_send_ports_stats_values (struct telemetry_encode_param *ep, +struct telemetry_impl *telemetry);int32_t rte_telemetry_socket_messaging_testing (int index,int socket); +int32_t +rte_telemetry_send_global_stats_values (struct telemetry_encode_param *ep, +struct telemetry_impl *telemetry);#endif
lib/librte_telemetry/rte_telemetry_parser.c +114 -16
196 lines not on this page · raw message
usertools/dpdk-telemetry-client.py +11 -2
@@ -12,6 +12,7 @@ METRICS_REQ = "{\"action\":0,\"command\":\"ports_all_stat_values\",\"data\":null}" API_REG = "{\"action\":1,\"command\":\"clients\",\"data\":{\"client_path\":\"" API_UNREG = "{\"action\":2,\"command\":\"clients\",\"data\":{\"client_path\":\"" +GLOBAL_METRICS_REQ = "{\"action\":0,\"command\":\"global_stat_values\",\"data\":null}" DEFAULT_FP = "/var/run/dpdk/default_client" class Socket:
@@ -79,12 +80,18 @@ def repeatedlyRequestMetrics(self, sleep_time): # Recursively requests metrics f self.requestMetrics() time.sleep(sleep_time) + def requestGlobalMetrics(self): #Requests global metrics for given client + self.socket.client_fd.send(GLOBAL_METRICS_REQ) + data = self.socket.client_fd.recv(BUFFER_SIZE) + print "\nResponse: \n", str(data) + def interactiveMenu(self, sleep_time): # Creates Interactive menu within the script - while self.choice != 3: + while self.choice != 4: print("\nOptions Menu") print("[1] Send for Metrics for all ports") print("[2] Send for Metrics for all ports recursively") - print("[3] Unregister client") + print("[3] Send for global Metrics") + print("[4] Unregister client") try: self.choice = int(input("\n:"))
@@ -95,6 +102,8 @@ def interactiveMenu(self, sleep_time): # Creates Interactive menu within the scr elif self.choice == 2: self.repeatedlyRequestMetrics(sleep_time) elif self.choice == 3: + self.requestGlobalMetrics() + elif self.choice == 4: self.unregister() self.unregistered = 1 else:
Signature · 1 line
2.21.0
Quoted 8 lines · Reshma Pattan
telemetry has support for fetching port based stats from metrics library.
Metrics library also has global stats which are
not fetched by telemetry, so extend telemetry to
fetch the global metrics.Signed-off-by Reshma Pattan <[email protected]>
Quoted 8 lines
--- doc/guides/howto/telemetry.rst | 9 +- doc/guides/rel_notes/release_19_08.rst | 5 + lib/librte_telemetry/rte_telemetry.c | 119 ++++++++++++---- lib/librte_telemetry/rte_telemetry_internal.h | 31 ++++- lib/librte_telemetry/rte_telemetry_parser.c | 130 +++++++++++++++--- usertools/dpdk-telemetry-client.py | 13 +- 6 files changed, 253 insertions(+), 54 deletions(-)
Looks good to me, thanks!
17/05/2019 19:07, Reshma Pattan:
Quoted 9 lines
telemetry has support for fetching port based stats from metrics library.
Metrics library also has global stats which are
not fetched by telemetry, so extend telemetry to
fetch the global metrics.Signed-off-by Reshma Pattan <[email protected]>---
I see some errors with GCC:
rte_telemetry_parser.c:362:27: error: unused variable ‘p’
rte_telemetry.c:550:11: error: unused variable ‘i’
rte_telemetry.c:613:16: error: comparison of integer expressions of different signedness: ‘int’ and ‘uint32_t’
Quoted 4 lines
-----Original Message-----
From: Thomas Monjalon [mailto:[email protected]]
<snip>
Quoted 9 lines
I see some errors with GCC:
rte_telemetry_parser.c:362:27: error: unused variable ‘p’
rte_telemetry.c:550:11: error: unused variable ‘i’
rte_telemetry.c:613:16: error: comparison of integer expressions of different
signedness: ‘int’ and ‘uint32_t’
Hmm, yes these have to be fixed, strange they were not caught on my board with gcc (GCC) 9.0.1 20190312 (Red Hat 9.0.1-0.10)
Will fix and send v2.
Thanks,
Reshma
Quoted 4 lines
-----Original Message-----
From: Thomas Monjalon [mailto:[email protected]]
<snip>
Quoted 2 lines
I see some errors with GCC:
On which GCC version the errors are shown?
Quoted 7 lines
rte_telemetry_parser.c:362:27: error: unused variable ‘p’
rte_telemetry.c:550:11: error: unused variable ‘i’
rte_telemetry.c:613:16: error: comparison of integer expressions of different
signedness: ‘int’ and ‘uint32_t’
18/06/2019 18:55, Pattan, Reshma:
Quoted 7 lines
Quoted 5 lines
-----Original Message-----
From: Thomas Monjalon [mailto:[email protected]]I see some errors with GCC:
On which GCC version the errors are shown?
GCC 8.3
Given the type of the errors I think it should be visible with older versions.
Quoted 6 lines
Quoted 6 lines
rte_telemetry_parser.c:362:27: error: unused variable ‘p’
rte_telemetry.c:550:11: error: unused variable ‘i’
rte_telemetry.c:613:16: error: comparison of integer expressions of different
signedness: ‘int’ and ‘uint32_t’
PATCH v2 1/1 lib/telemetry: add support to fetch global metrics
In reply to Reshma Pattantelemetry has support for fetching port based stats from metrics library.
Metrics library also has global stats which are not fetched by telemetry, so extend telemetry to fetch the global metrics.
---
v2: fix GCC compilation issues.
rebase with latest release notes.
--- doc/guides/howto/telemetry.rst | 9 +- doc/guides/rel_notes/release_19_08.rst | 5 + lib/librte_telemetry/rte_telemetry.c | 122 +++++++++++----- lib/librte_telemetry/rte_telemetry_internal.h | 31 ++++- lib/librte_telemetry/rte_telemetry_parser.c | 130 +++++++++++++++--- usertools/dpdk-telemetry-client.py | 13 +- 6 files changed, 255 insertions(+), 55 deletions(-)
doc/guides/howto/telemetry.rst +5 -4
@@ -11,9 +11,10 @@ Introduction ------------ The ``librte_telemetry`` provides the functionality so that users may query -metrics from incoming port traffic. The application which initializes packet -forwarding will act as the server, sending metrics to the requesting application -which acts as the client. +metrics from incoming port traffic and global stats(application stats). +The application which initializes packet forwarding will act as the server, +sending metrics to the requesting application which acts as the client. + In DPDK, applications are used to initialize the ``telemetry``. To view incoming traffic on featured ports, the application should be run first (ie. after ports
@@ -79,7 +80,7 @@ any DPDK application is applicable. the menu. #. Send traffic to any or all available ports from a traffic generator. - Select a query option(recursive or singular polling). + Select a query option(recursive or singular polling or global stats). The metrics will then be displayed on the client terminal in JSON format. #. Once finished, unregister the client using the menu command.
doc/guides/rel_notes/release_19_08.rst +5 -0
@@ -88,6 +88,11 @@ New Features * Added multi-queue support to allow one af_xdp vdev with multiple netdev queues +* **Updated telemetry library for global metrics support.** + + Updated ``librte_telemetry`` to fetch the global metrics from the + ``librte_metrics`` library. + Removed Items -------------
lib/librte_telemetry/rte_telemetry.c +91 -31
180 lines not on this page · raw message
lib/librte_telemetry/rte_telemetry_internal.h +29 -2
@@ -24,6 +24,8 @@ extern int telemetry_log_level; #define TELEMETRY_LOG_INFO(fmt, args...) \ TELEMETRY_LOG(INFO, fmt, ## args) +#define MAX_METRICS 256 + typedef struct telemetry_client { char *file_path; int fd;
@@ -48,6 +50,28 @@ enum rte_telemetry_parser_actions { ACTION_DELETE = 2 }; +enum rte_telemetry_stats_type { +PORT_STATS =0 , +GLOBAL_STATS =1 +}; + +/* @internal */ +struct telemetry_encode_param { +enum rte_telemetry_stats_type type; +union { +struct port_param { +uint32_t num_metric_ids; +uint32_t metric_ids[MAX_METRICS ]; +uint32_t num_port_ids; +uint32_t port_ids[RTE_MAX_ETHPORTS ]; + } pp; +struct global_param { +uint32_t num_metric_ids; +uint32_t metric_ids[MAX_METRICS ]; + } gp; + }; +}; +int32_t rte_telemetry_parse_client_message (struct telemetry_impl *telemetry,char *buf);
@@ -72,10 +96,13 @@ int32_t rte_telemetry_is_port_active(int port_id);int32_t -rte_telemetry_send_ports_stats_values (uint32_t *metric_ids,int num_metric_ids, -uint32_t *port_ids,int num_port_ids,struct telemetry_impl *telemetry); +rte_telemetry_send_ports_stats_values (struct telemetry_encode_param *ep, +struct telemetry_impl *telemetry);int32_t rte_telemetry_socket_messaging_testing (int index,int socket); +int32_t +rte_telemetry_send_global_stats_values (struct telemetry_encode_param *ep, +struct telemetry_impl *telemetry);#endif
lib/librte_telemetry/rte_telemetry_parser.c +114 -16
196 lines not on this page · raw message
usertools/dpdk-telemetry-client.py +11 -2
@@ -12,6 +12,7 @@ METRICS_REQ = "{\"action\":0,\"command\":\"ports_all_stat_values\",\"data\":null}" API_REG = "{\"action\":1,\"command\":\"clients\",\"data\":{\"client_path\":\"" API_UNREG = "{\"action\":2,\"command\":\"clients\",\"data\":{\"client_path\":\"" +GLOBAL_METRICS_REQ = "{\"action\":0,\"command\":\"global_stat_values\",\"data\":null}" DEFAULT_FP = "/var/run/dpdk/default_client" class Socket:
@@ -79,12 +80,18 @@ def repeatedlyRequestMetrics(self, sleep_time): # Recursively requests metrics f self.requestMetrics() time.sleep(sleep_time) + def requestGlobalMetrics(self): #Requests global metrics for given client + self.socket.client_fd.send(GLOBAL_METRICS_REQ) + data = self.socket.client_fd.recv(BUFFER_SIZE) + print "\nResponse: \n", str(data) + def interactiveMenu(self, sleep_time): # Creates Interactive menu within the script - while self.choice != 3: + while self.choice != 4: print("\nOptions Menu") print("[1] Send for Metrics for all ports") print("[2] Send for Metrics for all ports recursively") - print("[3] Unregister client") + print("[3] Send for global Metrics") + print("[4] Unregister client") try: self.choice = int(input("\n:"))
@@ -95,6 +102,8 @@ def interactiveMenu(self, sleep_time): # Creates Interactive menu within the scr elif self.choice == 2: self.repeatedlyRequestMetrics(sleep_time) elif self.choice == 3: + self.requestGlobalMetrics() + elif self.choice == 4: self.unregister() self.unregistered = 1 else:
Signature · 1 line
2.21.0
Quoted 9 lines · Reshma Pattan
telemetry has support for fetching port based stats from metrics library.
Metrics library also has global stats which are not fetched by telemetry, so extend telemetry to fetch the global metrics.
Signed-off-by Reshma Pattan <[email protected]>Acked-by Kevin Laatz <[email protected]>
Hi Reshma, Kevin,
Not related to this patch, but when telemetry library enabled, it is causing some warnings [1] with gcc9
The [-Waddress-of-packed-member] warnings already disable for gcc, but you need following update to "lib/librte_telemetry/Makefile":
-CFLAGS += -I$(SRCDIR)
+CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)
Can you please take care of this?
[1]
In file included from .../dpdk/x86_64-native-linuxapp-gcc/include/rte_ethdev.h:160,
from .../dpdk/lib/librte_telemetry/rte_telemetry_parser.c:13:.../dpdk/x86_64-native-linuxapp-gcc/include/rte_ether.h: In function ‘rte_is_broadcast_ether_addr’:.../dpdk/x86_64-native-linuxapp-gcc/include/rte_ether.h:152:2: warning:
converting a packed ‘const struct rte_ether_addr’ pointer (alignment 1) to a ‘unaligned_uint16_t’ {aka ‘const short unsigned int’} pointer (alignment 2) may result in an unaligned pointer value [-Waddress-of-packed-member]
152 | const unaligned_uint16_t *ea_words = (const unaligned_uint16_t *)ea;
| ^~~~~Quoted 7 lines
Hi Reshma, Kevin,
Not related to this patch, but when telemetry library enabled, it is causing some warnings [1] with gcc9
Quoted 7 lines
The [-Waddress-of-packed-member] warnings already disable for gcc, but you need following update to "
lib/librte_telemetry/Makefile":
-CFLAGS += -I$(SRCDIR)
+CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)Can you please take care of this?
Sure, I'll look into it. Thanks.
Quoted 15 lines
[1]
In file included from .../dpdk/x86_64-native-linuxapp-gcc/include/rte_ethdev.h:160,
from.../dpdk/lib/librte_telemetry/rte_telemetry_parser.c:13:.../dpdk/x86_64-native-linuxapp-gcc/include/rte_ether.h: In function ‘rte_is_broadcast_ether_addr’:.../dpdk/x86_64-native-linuxapp-gcc/include/rte_ether.h:152:2: warning:
converting a packed ‘const struct rte_ether_addr’ pointer (alignment 1) to a ‘unaligned_uint16_t’ {aka ‘const short unsigned int’} pointer (alignment 2) may result in an unaligned pointer value [-Waddress-of-packed-member]
152 | constunaligned_uint16_t*ea_words= (constunaligned_uint16_t*)ea; | ^~~~~
Thomas Monjalon
Applied18/06/2019 15:49, Reshma Pattan:
Quoted 7 lines
telemetry has support for fetching port based stats from metrics library.
Metrics library also has global stats which are not fetched by telemetry, so extend telemetry to fetch the global metrics.
Quoted 5 lines
Signed-off-by Reshma Pattan <[email protected]>Acked-by Kevin Laatz <[email protected]>---
v2: fix GCC compilation issues.
rebase with latest release notes.
Applied, thanks
Chautru, Nicolas
AppliedHi Reshma,
I am still seeing build issue when rebasing to this : ie. checking uint32_t is negative in rte_telemetry.c:558 Thanks
Nic
-----Original Message-----
From: dev [mailto:[email protected]] On Behalf Of Thomas Monjalon
Sent: Monday, June 24, 2019 7:54 AM
To: Pattan, Reshma <[email protected]>
Cc: [email protected]; Laatz, Kevin <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v2] lib/telemetry: add support to fetch global metrics
18/06/2019 15:49, Reshma Pattan:
Quoted 11 lines
telemetry has support for fetching port based stats from metrics library.
Metrics library also has global stats which are not fetched by telemetry, so extend telemetry to fetch the global metrics.
Signed-off-by Reshma Pattan <[email protected]>Acked-by Kevin Laatz <[email protected]>---
v2: fix GCC compilation issues.
rebase with latest release notes.
Applied, thanks
Quoted 11 lines
-----Original Message-----
From: Chautru, Nicolas
Sent: Monday, June 24, 2019 6:37 PM
To: Thomas Monjalon <[email protected]>; Pattan, Reshma <[email protected]>
Cc: [email protected]; Laatz, Kevin <[email protected]>
Subject: RE: [dpdk-dev] [PATCH v2] lib/telemetry: add support to fetch global metrics
Quoted 4 lines
Hi Reshma,
I am still seeing build issue when rebasing to this : ie. checkinguint32_tis negative in rte_telemetry.c:558 Thanks Nic
Which compiler and version , can you give the more details. <snip>
Quoted 22 lines
-----Original Message-----
From: Pattan, Reshma
Sent: Tuesday, June 25, 2019 9:39 AM
To: Chautru, Nicolas <[email protected]>; Thomas Monjalon <[email protected]>
Cc: [email protected]; Laatz, Kevin <[email protected]>
Subject: RE: [dpdk-dev] [PATCH v2] lib/telemetry: add support to fetch global metricsQuoted 9 lines
-----Original Message-----
From: Chautru, Nicolas
Sent: Monday, June 24, 2019 6:37 PM
To: Thomas Monjalon <[email protected]>; Pattan, Reshma <[email protected]>
Cc: [email protected]; Laatz, Kevin <[email protected]>
Subject: RE: [dpdk-dev] [PATCH v2] lib/telemetry: add support to fetch global metrics
Quoted 7 lines
Quoted 4 lines
Hi Reshma,
I am still seeing build issue when rebasing to this : ie. checkinguint32_tis negative in rte_telemetry.c:558 Thanks NicWhich compiler and version , can you give the more details. <snip>
Hi,
WERROR_FLAGS is missing in the make file so we missed to see these errors, me and Kevin are looking into this. Kevin will send the patch.
Thanks,
Reshma