Skip to contentSkip to conversation

usertools: add telemetry python3 compatibility

PATCH Superseded Ciara Power 4 participants 6 messages Jan 16, 2020 – Feb 16, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 usertools: add telemetry python3 compatibility

The client script for use with the telemetry library did not support python3, as the data being sent over the socket was in string format. Python3 requires the data be explicitly converted to bytes before being sent. Similarily, the received bytes need to be decoded into string format.

---
 usertools/dpdk-telemetry-client.py | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)
usertools/dpdk-telemetry-client.py +9 -8
index 290345dcc..71a8a8852 100755
@@ -65,18 +65,19 @@ def register(self): # Connects a client to DPDK-instance
         self.socket.recv_fd.settimeout(2)
         self.socket.send_fd.connect("/var/run/dpdk/rte/telemetry")
         JSON = (API_REG + self.file_path + "\"}}")
-        self.socket.send_fd.sendall(JSON)
+        self.socket.send_fd.sendall(JSON.encode())
+
         self.socket.recv_fd.listen(1)
         self.socket.client_fd = self.socket.recv_fd.accept()[0]
 
     def unregister(self): # Unregister a given client
-        self.socket.client_fd.send(API_UNREG + self.file_path + "\"}}")
+        self.socket.client_fd.send((API_UNREG + self.file_path + "\"}}").encode())
         self.socket.client_fd.close()
 
     def requestMetrics(self): # Requests metrics for given client
-        self.socket.client_fd.send(METRICS_REQ)
-        data = self.socket.client_fd.recv(BUFFER_SIZE)
-        print("\nResponse: \n", str(data))
+        self.socket.client_fd.send(METRICS_REQ.encode())
+        data = self.socket.client_fd.recv(BUFFER_SIZE).decode()
+        print("\nResponse: \n", data)
 
     def repeatedlyRequestMetrics(self, sleep_time): # Recursively requests metrics for given client
         print("\nPlease enter the number of times you'd like to continuously request Metrics:")
@@ -88,9 +89,9 @@ def repeatedlyRequestMetrics(self, sleep_time): # Recursively requests metrics f
             time.sleep(sleep_time)
 
     def requestGlobalMetrics(self): #Requests global metrics for given client
-        self.socket.client_fd.send(GLOBAL_METRICS_REQ)
-        data = self.socket.client_fd.recv(BUFFER_SIZE)
-        print("\nResponse: \n", str(data))
+        self.socket.client_fd.send(GLOBAL_METRICS_REQ.encode())
+        data = self.socket.client_fd.recv(BUFFER_SIZE).decode()
+        print("\nResponse: \n", data)
 
     def interactiveMenu(self, sleep_time): # Creates Interactive menu within the script
         while self.choice != 4:
Signature · 1 line
2.17.1
#1 lore raw

+Cc Robin, known Python expert :)

16/01/2020 18:24, Ciara Power:

Quoted 56 lines

The client script for use with the telemetry library did not support python3, as the data being sent over the socket was in string format. Python3 requires the data be explicitly converted to bytes before being sent. Similarily, the received bytes need to be decoded into string format.

---
 usertools/dpdk-telemetry-client.py | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)
usertools/dpdk-telemetry-client.py +9 -8
index 290345dcc..71a8a8852 100755
@@ -65,18 +65,19 @@ def register(self): # Connects a client to DPDK-instance
         self.socket.recv_fd.settimeout(2)
         self.socket.send_fd.connect("/var/run/dpdk/rte/telemetry")
         JSON = (API_REG + self.file_path + "\"}}")
-        self.socket.send_fd.sendall(JSON)
+        self.socket.send_fd.sendall(JSON.encode())
+
         self.socket.recv_fd.listen(1)
         self.socket.client_fd = self.socket.recv_fd.accept()[0]
 
     def unregister(self): # Unregister a given client
-        self.socket.client_fd.send(API_UNREG + self.file_path + "\"}}")
+        self.socket.client_fd.send((API_UNREG + self.file_path + "\"}}").encode())
         self.socket.client_fd.close()
 
     def requestMetrics(self): # Requests metrics for given client
-        self.socket.client_fd.send(METRICS_REQ)
-        data = self.socket.client_fd.recv(BUFFER_SIZE)
-        print("\nResponse: \n", str(data))
+        self.socket.client_fd.send(METRICS_REQ.encode())
+        data = self.socket.client_fd.recv(BUFFER_SIZE).decode()
+        print("\nResponse: \n", data)
 
     def repeatedlyRequestMetrics(self, sleep_time): # Recursively requests metrics for given client
         print("\nPlease enter the number of times you'd like to continuously request Metrics:")
@@ -88,9 +89,9 @@ def repeatedlyRequestMetrics(self, sleep_time): # Recursively requests metrics f
             time.sleep(sleep_time)
 
     def requestGlobalMetrics(self): #Requests global metrics for given client
-        self.socket.client_fd.send(GLOBAL_METRICS_REQ)
-        data = self.socket.client_fd.recv(BUFFER_SIZE)
-        print("\nResponse: \n", str(data))
+        self.socket.client_fd.send(GLOBAL_METRICS_REQ.encode())
+        data = self.socket.client_fd.recv(BUFFER_SIZE).decode()
+        print("\nResponse: \n", data)
 
     def interactiveMenu(self, sleep_time): # Creates Interactive menu within the script
         while self.choice != 4:
Reviewed-by
#2 lore raw

16/01/2020 18:24, Ciara Power:

Quoted 9 lines

The client script for use with the telemetry library did not support python3, as the data being sent over the socket was in string format. Python3 requires the data be explicitly converted to bytes before being sent. Similarily, the received bytes need to be decoded into string format.

Overall, it looks good to me. One minor grudge:

Mind that when using this script with python2, the literal strings actually are bytes. This means that .encode() does not make any sense on them. As it turns out, the str objects of python2 do have an .encode() method that does not do anything (it returns the byte string object unchanged), so calling it does not cause any problem.

Long story short, for consistency you should consider adding a future import at the top:

from __future__ import unicode_literals

So that all literal strings are unicode with python2 as with python3. See related commit 4da069194ef4 ("usertools: fix pmdinfo with python 3 and pyelftools>=0.24").

Maybe the patch title should contain the word "fix" and some Fixes:
lines as there already were some attemps to make this script python3 compatible.

Signature · 1 line
Robin
#3 lore raw

PATCH v2 1/1 usertools: fix telemetry python3 compatibility

In reply to Ciara Power

The client script for use with the telemetry library did not support Python3, as the data being sent over the socket was in string format. Python3 requires the data be explicitly converted to bytes before being sent. Similarily, the received bytes need to be decoded into string format.

Fixes 53f293c9a783 ("usertools: replace unsafe input function")
Fixes fe35622659ed ("usertools: fix telemetry client with python 3")
Fixes d1b94da4a4e0 ("usertools: add client script for telemetry")
Fixes 4080e46c8078 ("telemetry: support global metrics")

---

V2: added future import for unicode literals to ensure consistency of data types for Python2 and Python3.

---
 usertools/dpdk-telemetry-client.py | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)
usertools/dpdk-telemetry-client.py +10 -8
index 290345dcc..35edb7cd2 100755
@@ -3,6 +3,7 @@
 # Copyright(c) 2018 Intel Corporation
 
 from __future__ import print_function
+from __future__ import unicode_literals
 
 import socket
 import os
@@ -65,18 +66,19 @@ def register(self): # Connects a client to DPDK-instance
         self.socket.recv_fd.settimeout(2)
         self.socket.send_fd.connect("/var/run/dpdk/rte/telemetry")
         JSON = (API_REG + self.file_path + "\"}}")
-        self.socket.send_fd.sendall(JSON)
+        self.socket.send_fd.sendall(JSON.encode())
+
         self.socket.recv_fd.listen(1)
         self.socket.client_fd = self.socket.recv_fd.accept()[0]
 
     def unregister(self): # Unregister a given client
-        self.socket.client_fd.send(API_UNREG + self.file_path + "\"}}")
+        self.socket.client_fd.send((API_UNREG + self.file_path + "\"}}").encode())
         self.socket.client_fd.close()
 
     def requestMetrics(self): # Requests metrics for given client
-        self.socket.client_fd.send(METRICS_REQ)
-        data = self.socket.client_fd.recv(BUFFER_SIZE)
-        print("\nResponse: \n", str(data))
+        self.socket.client_fd.send(METRICS_REQ.encode())
+        data = self.socket.client_fd.recv(BUFFER_SIZE).decode()
+        print("\nResponse: \n", data)
 
     def repeatedlyRequestMetrics(self, sleep_time): # Recursively requests metrics for given client
         print("\nPlease enter the number of times you'd like to continuously request Metrics:")
@@ -88,9 +90,9 @@ def repeatedlyRequestMetrics(self, sleep_time): # Recursively requests metrics f
             time.sleep(sleep_time)
 
     def requestGlobalMetrics(self): #Requests global metrics for given client
-        self.socket.client_fd.send(GLOBAL_METRICS_REQ)
-        data = self.socket.client_fd.recv(BUFFER_SIZE)
-        print("\nResponse: \n", str(data))
+        self.socket.client_fd.send(GLOBAL_METRICS_REQ.encode())
+        data = self.socket.client_fd.recv(BUFFER_SIZE).decode()
+        print("\nResponse: \n", data)
 
     def interactiveMenu(self, sleep_time): # Creates Interactive menu within the script
         while self.choice != 4:
Signature · 1 line
2.17.1
Acked-by on v2 1/1
#4 lore raw

Re: [dpdk-dev] [PATCH v2] usertools: fix telemetry python3 compatibility

Quoted 18 lines

The client script for use with the telemetry library did not support Python3, as the data being sent over the socket was in string format. Python3 requires the data be explicitly converted to bytes before being sent. Similarily, the received bytes need to be decoded into string format.

Fixes 53f293c9a783 ("usertools: replace unsafe input function")
Fixes fe35622659ed ("usertools: fix telemetry client with python 3")
Fixes d1b94da4a4e0 ("usertools: add client script for telemetry")
Fixes 4080e46c8078 ("telemetry: support global metrics")

<snip>

#5 lore raw

Re: [dpdk-dev] [PATCH v2] usertools: fix telemetry python3 compatibility

24/01/2020 11:11, Laatz, Kevin:

Quoted 15 lines
Quoted 15 lines

The client script for use with the telemetry library did not support Python3, as the data being sent over the socket was in string format. Python3 requires the data be explicitly converted to bytes before being sent. Similarily, the received bytes need to be decoded into string format.

Fixes 53f293c9a783 ("usertools: replace unsafe input function")
Fixes fe35622659ed ("usertools: fix telemetry client with python 3")
Fixes d1b94da4a4e0 ("usertools: add client script for telemetry")
Fixes 4080e46c8078 ("telemetry: support global metrics")
Quoted 6 lines
Quoted 3 lines

<snip>

Applied, thanks