Skip to contentSkip to conversation

adding support for python 3 only.

PATCH Superseded Louise Kilheeney 13 participants 175 messages May 22, 2020 – Oct 2, 2020

lore
Share & tools
lore
#0 lore raw

cover adding support for python 3 only.

This patch set converts all python scripts in the project to use python3 only. This is due to python2 being EOL in January 2020.

One script not tested: ./config/arm/armv8_machine.py

Louise Kilheeney (9):
app/test-bbdev: support python3 only usertools/dpdk-devbind: support python3 only usertools/dpdk-telemetry-client: support python3 only usertools/dpdk-pmdinfo: support python3 only usertools/cpu_layout: support python3 only app/test-cmdline: support python3 only app/test: support python3 only
devtools: support python3 only
config/arm: support python3 only

 app/test-bbdev/test-bbdev.py          |  7 +++----
 app/test-cmdline/cmdline_test.py      |  3 +--
 app/test-cmdline/cmdline_test_data.py |  1 +
 app/test/autotest.py                  |  3 +--
 app/test/autotest_data.py             |  1 +
 app/test/autotest_runner.py           | 14 +++++++-------
 app/test/autotest_test_funcs.py       |  1 +
 config/arm/armv8_machine.py           |  2 +-
 devtools/update_version_map_abi.py    |  3 +--
 mk/rte.sdktest.mk                     |  6 +++---
 usertools/cpu_layout.py               |  9 ++-------
 usertools/dpdk-devbind.py             |  3 +--
 usertools/dpdk-pmdinfo.py             |  4 +---
 usertools/dpdk-telemetry-client.py    | 16 ++++------------
 14 files changed, 28 insertions(+), 45 deletions(-)
Signature · 1 line
2.17.1
#1 lore raw

PATCH 1/9 app/test-bbdev: support python3 only

Changed script to explicitly use python3 only.

Cc Nicolas Chautru <[email protected]>
---
 app/test-bbdev/test-bbdev.py | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
app/test-bbdev/test-bbdev.py +3 -4
index 0194be046..50add4580 100755
@@ -1,5 +1,4 @@
-#!/usr/bin/env python
-
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Intel Corporation
 
@@ -12,7 +11,7 @@
 from threading import Timer
 
 def kill(process):
-    print "ERROR: Test app timed out"
+    print ("ERROR: Test app timed out")
     process.kill()
 
 if "RTE_SDK" in os.environ:
@@ -66,7 +65,7 @@ def kill(process):
 args = parser.parse_args()
 
 if not os.path.exists(args.testapp_path):
-    print "No such file: " + args.testapp_path
+    print ("No such file: " + args.testapp_path)
     sys.exit(1)
 
 params = [args.testapp_path]
Signature · 1 line
2.17.1
Acked-by Nic Chautru on v1 1/9
#2 lore raw

Re: [dpdk-dev] [PATCH 20.08 1/9] app/test-bbdev: support python3 only

Kilheeney, Louise <[email protected]> :

Quoted 45 lines

Sent: Friday, May 22, 2020 6:23 AM
To: [email protected]
Cc: Kilheeney, Louise <[email protected]>; Chautru, Nicolas <[email protected]>
Subject: [PATCH 20.08 1/9] app/test-bbdev: support python3 only

Changed script to explicitly use python3 only.

Cc Nicolas Chautru <[email protected]>
---
 app/test-bbdev/test-bbdev.py | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
app/test-bbdev/test-bbdev.py +1 -2
index 0194be046..50add4580 100755
@@ -1,5 +1,4 @@
-#!/usr/bin/env python
-
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause  # Copyright(c) 2017 Intel

Corporation

app/test-bbdev/test-bbdev.py +2 -3
@@ -12,7 +11,7 @@
 from threading import Timer
 
 def kill(process):
-    print "ERROR: Test app timed out"
+    print ("ERROR: Test app timed out")
     process.kill()
 
 if "RTE_SDK" in os.environ:
@@ -66,7 +65,7 @@ def kill(process):
 args = parser.parse_args()
 
 if not os.path.exists(args.testapp_path):
-    print "No such file: " + args.testapp_path
+    print ("No such file: " + args.testapp_path)
     sys.exit(1)
 
 params = [args.testapp_path]
--
Quoted 1 line

2.17.1

I double checked locally, fine with me. Thanks.

Acked-by Nic Chautru
#3 lore raw

PATCH 2/9 usertools/dpdk-devbind: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only.

---
 usertools/dpdk-devbind.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
usertools/dpdk-devbind.py +1 -2
index 40b4beea4..441301f2d 100755
@@ -1,9 +1,8 @@
-#! /usr/bin/env python
+#! /usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 #
 
-from __future__ import print_function
 import sys
 import os
 import getopt
Signature · 1 line
2.17.1
#4 lore raw

Re: [dpdk-dev] [PATCH 20.08 2/9] usertools/dpdk-devbind: support python3 only

Quoted 4 lines · Louise Kilheeney

Changed script to explicitly use python3 only.

---

There is a bunch of old cruft in a few scripts, such as python compatibility checks and reimplementation of check_output. it would be nice if those were fixed as well.

If you look for imports or the word "python" in code, chance are you'll find some comments saying something about "let's do this hack for python 2 compatibility" :)

Signature · 2 lines
Thanks,
Anatoly
#5 lore raw

Re: [dpdk-dev] [PATCH 20.08 2/9] usertools/dpdk-devbind: support python3 only

Quoted 10 lines · Anatoly
Quoted 4 lines · Louise Kilheeney

Changed script to explicitly use python3 only.

---

There is a bunch of old cruft in a few scripts, such as python compatibility checks and reimplementation of check_output. it would be nice if those were fixed as well.

If you look for imports or the word "python" in code, chance are you'll find some comments saying something about "let's do this hack for python 2 compatibility" :)

I would also suggest using /usr/bin/env python3 rather than explicit path to /usr/bin/python3.

Signature · 2 lines
Thanks,
Anatoly
#6 lore raw

PATCH 3/9 usertools/dpdk-telemetry-client: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only.

Cc Kevin Laatz <[email protected]>
---
 usertools/dpdk-telemetry-client.py | 16 ++++------------
 1 file changed, 4 insertions(+), 12 deletions(-)
usertools/dpdk-telemetry-client.py +4 -12
index 35edb7cd2..fa599046a 100755
@@ -1,10 +1,7 @@
-#! /usr/bin/env python
+#! /usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Intel Corporation
 
-from __future__ import print_function
-from __future__ import unicode_literals
-
 import socket
 import os
 import sys
@@ -18,11 +15,6 @@
 GLOBAL_METRICS_REQ = "{\"action\":0,\"command\":\"global_stat_values\",\"data\":null}"
 DEFAULT_FP = "/var/run/dpdk/default_client"
 
-try:
-    raw_input  # Python 2
-except NameError:
-    raw_input = input  # Python 3
-
 class Socket:
 
     def __init__(self):
@@ -49,7 +41,7 @@ def __init__(self): # Creates a client instance
     def __del__(self):
         try:
             if self.unregistered == 0:
-                self.unregister();
+                self.unregister()
         except:
             print("Error - Client could not be destroyed")
 
@@ -82,7 +74,7 @@ def requestMetrics(self): # Requests metrics for given client
 
     def repeatedlyRequestMetrics(self, sleep_time): # Recursively requests metrics for given client
         print("\nPlease enter the number of times you'd like to continuously request Metrics:")
-        n_requests = int(raw_input("\n:"))
+        n_requests = int(input("\n:"))
         print("\033[F") #Removes the user input from screen, cleans it up
         print("\033[K")
         for i in range(n_requests):
@@ -103,7 +95,7 @@ def interactiveMenu(self, sleep_time): # Creates Interactive menu within the scr
             print("[4] Unregister client")
 
             try:
-                self.choice = int(raw_input("\n:"))
+                self.choice = int(input("\n:"))
                 print("\033[F") #Removes the user input for screen, cleans it up
                 print("\033[K")
                 if self.choice == 1:
Signature · 1 line
2.17.1
#7 lore raw

PATCH 4/9 usertools/dpdk-pmdinfo: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only.

Cc Neil Horman <[email protected]>
---
 usertools/dpdk-pmdinfo.py | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
usertools/dpdk-pmdinfo.py +1 -3
index 12f20735e..f5cd077f5 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2016  Neil Horman <[email protected]>
 
@@ -7,8 +7,6 @@
 # Utility to dump PMD_INFO_STRING support from an object file
 #
 # -------------------------------------------------------------------------
-from __future__ import print_function
-from __future__ import unicode_literals
 import json
 import io
 import os
Signature · 1 line
2.17.1
#8 lore raw

PATCH 5/9 usertools/cpu_layout: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only.

---
 usertools/cpu_layout.py | 9 ++-------
 1 file changed, 2 insertions(+), 7 deletions(-)
usertools/cpu_layout.py +2 -7
index 6f129b1db..89a48cec4 100755
@@ -1,14 +1,9 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 # Copyright(c) 2017 Cavium, Inc. All rights reserved.
 
-from __future__ import print_function
 import sys
-try:
-    xrange # Python 2
-except NameError:
-    xrange = range # Python 3
 
 sockets = []
 cores = []
@@ -17,7 +12,7 @@
 fd = open("{}/kernel_max".format(base_path))
 max_cpus = int(fd.read())
 fd.close()
-for cpu in xrange(max_cpus + 1):
+for cpu in range(max_cpus + 1):
     try:
         fd = open("{}/cpu{}/topology/core_id".format(base_path, cpu))
     except IOError:
Signature · 1 line
2.17.1
#9 lore raw

PATCH 6/9 app/test-cmdline: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only.

Cc Olivier Matz <[email protected]>
---
 app/test-cmdline/cmdline_test.py      | 3 +--
 app/test-cmdline/cmdline_test_data.py | 1 +
 mk/rte.sdktest.mk                     | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)
app/test-cmdline/cmdline_test.py +1 -2
index 3a8fac426..0e7c72459 100755
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # Script that runs cmdline_test app and feeds keystrokes into it.
-from __future__ import print_function
 import cmdline_test_data
 import os
 import pexpect
app/test-cmdline/cmdline_test_data.py +1 -0
index 114d2cb6a..2d9b3262a 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
mk/rte.sdktest.mk +1 -1
index 803018ba3..ad7ef138f 100644
@@ -63,7 +63,7 @@ coverage:
 	@mkdir -p $(AUTOTEST_DIR) ; \
 	cd $(AUTOTEST_DIR) ; \
 	if [ -f $(RTE_OUTPUT)/app/test ]; then \
-		python $(RTE_SDK)/test/cmdline_test/cmdline_test.py \
+		python3 $(RTE_SDK)/test/cmdline_test/cmdline_test.py \
 			$(RTE_OUTPUT)/app/cmdline_test; \
 		ulimit -S -n 100 ; \
 		python $(RTE_SDK)/app/test/autotest.py \
Signature · 1 line
2.17.1
#10 lore raw

PATCH 7/9 app/test: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only.

---
 app/test/autotest.py            |  3 +--
 app/test/autotest_data.py       |  1 +
 app/test/autotest_runner.py     | 14 +++++++-------
 app/test/autotest_test_funcs.py |  1 +
 mk/rte.sdktest.mk               |  4 ++--
 5 files changed, 12 insertions(+), 11 deletions(-)
app/test/autotest.py +1 -2
index b42f48879..9eef1efbe 100644
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # Script that uses either test app or qemu controlled by python-pexpect
-from __future__ import print_function
 import autotest_data
 import autotest_runner
 import sys
app/test/autotest_data.py +1 -0
index 7b1d01389..e33258026 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
app/test/autotest_runner.py +7 -7
index 95e74c760..8c15cbdc2 100644
@@ -1,10 +1,10 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # The main logic behind running autotests in parallel
 
-from __future__ import print_function
-import StringIO
+import io
 import csv
 from multiprocessing import Pool, Queue
 import pexpect
@@ -78,7 +78,7 @@ def pool_init(queue, result_queue):
     cmdline = "%s %s" % (cmdline, prefix_cmdline)
 
     # prepare logging of init
-    startuplog = StringIO.StringIO()
+    startuplog = io.StringIO()
 
     # run test app
     try:
@@ -86,7 +86,7 @@ def pool_init(queue, result_queue):
         print("\n%s %s\n" % ("=" * 20, prefix), file=startuplog)
         print("\ncmdline=%s" % cmdline, file=startuplog)
 
-        pool_child = pexpect.spawn(cmdline, logfile=startuplog)
+        pool_child = pexpect.spawn(cmdline, logfile=startuplog, encoding='utf-8')
 
         # wait for target to boot
         if not wait_prompt(pool_child):
@@ -138,7 +138,7 @@ def run_test(target, test):
     # create log buffer for each test
     # in multiprocessing environment, the logging would be
     # interleaved and will create a mess, hence the buffering
-    logfile = StringIO.StringIO()
+    logfile = io.StringIO()
     pool_child.logfile = logfile
 
     # make a note when the test started
@@ -210,9 +210,9 @@ def __init__(self, cmdline, target, blacklist, whitelist, n_processes):
         # parse the binary for available test commands
         binary = cmdline.split()[0]
         stripped = 'not stripped' not in \
-                   subprocess.check_output(['file', binary])
+                   subprocess.check_output(['file', binary]).decode()
         if not stripped:
-            symbols = subprocess.check_output(['nm', binary]).decode('utf-8')
+            symbols = subprocess.check_output(['nm', binary]).decode()
             self.avail_cmds = re.findall('test_register_(\w+)', symbols)
         else:
             self.avail_cmds = None
app/test/autotest_test_funcs.py +1 -0
index 26688b713..775dfd1dc 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
mk/rte.sdktest.mk +2 -2
index ad7ef138f..81eff0c08 100644
@@ -49,7 +49,7 @@ test test-fast test-perf test-drivers test-dump:
 	@mkdir -p $(AUTOTEST_DIR) ; \
 	cd $(AUTOTEST_DIR) ; \
 	if [ -f $(RTE_OUTPUT)/app/test ]; then \
-		python $(RTE_SDK)/app/test/autotest.py \
+		python3 $(RTE_SDK)/app/test/autotest.py \
 			$(RTE_OUTPUT)/app/test \
 			$(RTE_TARGET) \
 			$(BLACKLIST) $(WHITELIST); \
@@ -66,7 +66,7 @@ coverage:
 		python3 $(RTE_SDK)/test/cmdline_test/cmdline_test.py \
 			$(RTE_OUTPUT)/app/cmdline_test; \
 		ulimit -S -n 100 ; \
-		python $(RTE_SDK)/app/test/autotest.py \
+		python3 $(RTE_SDK)/app/test/autotest.py \
 			$(RTE_OUTPUT)/app/test \
 			$(RTE_TARGET) \
 			$(BLACKLIST) $(WHITELIST) ; \
Signature · 1 line
2.17.1
#11 lore raw

PATCH 8/9 devtools: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only.

Cc Neil Horman <[email protected]>
Cc Ray Kinsella <[email protected]>
---
 devtools/update_version_map_abi.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
devtools/update_version_map_abi.py +1 -2
index 616412a1c..58aa368f9 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2019 Intel Corporation
 
@@ -9,7 +9,6 @@
 from the devtools/update-abi.sh utility.
 """
 
-from __future__ import print_function
 import argparse
 import sys
 import re
Signature · 1 line
2.17.1
Acked-by on v1 8/9
#12 lore raw

Re: [dpdk-dev] [PATCH 20.08 8/9] devtools: support python3 only

yes - I noticed the rpm packaging has become very pedantic about this of late.

Ray K

Quoted 22 lines · Louise Kilheeney

Changed script to explicitly use python3 only.

Cc Neil Horman <[email protected]>
Cc Ray Kinsella <[email protected]>
---
 devtools/update_version_map_abi.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
devtools/update_version_map_abi.py +1 -1
index 616412a1c..58aa368f9 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2019 Intel Corporation
 

devtools/update_version_map_abi.py line 14

Quoted 7 lines
devtools/update_version_map_abi.py +0 -1
@@ -9,7 +9,6 @@
 from the devtools/update-abi.sh utility.
 """
 
-from __future__ import print_function
 import argparse
 import sys
 import re
#13 lore raw

PATCH 9/9 config/arm: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only.

Cc Thomas Monjalon <[email protected]>
---
 config/arm/armv8_machine.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
config/arm/armv8_machine.py +1 -1
index 404866d2f..a0e6a8769 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Cavium, Inc
 
Signature · 1 line
2.17.1
#14 lore raw

Re: [dpdk-dev] [PATCH 20.08 9/9] config/arm: support python3 only

22/05/2020 15:23, Louise Kilheeney:

Quoted 1 line

Changed script to explicitly use python3 only.

What is the reason of this change?

Quoted 16 lines
Cc Thomas Monjalon <[email protected]>
---
 config/arm/armv8_machine.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
config/arm/armv8_machine.py +1 -1
index 404866d2f..a0e6a8769 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Cavium, Inc
#15 lore raw

Re: [dpdk-dev] [PATCH 20.08 9/9] config/arm: support python3 only

since python 2 is EOL, Making these scripts to use python3-only, it's part of a general update to have everything use python3. This is already using python 3 and this makes it explicit.

Best Regards
Louise Kilheeney
-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Friday 22 May 2020 14:46
To: Kilheeney, Louise <[email protected]>
Cc: [email protected]
Subject: Re: [PATCH 20.08 9/9] config/arm: support python3 only

22/05/2020 15:23, Louise Kilheeney:

Quoted 1 line

Changed script to explicitly use python3 only.

What is the reason of this change?

Quoted 16 lines
Cc Thomas Monjalon <[email protected]>
---
 config/arm/armv8_machine.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
config/arm/armv8_machine.py +1 -1
index 404866d2f..a0e6a8769 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/python3
 # SPDX-License-Identifier: BSD-3-Clause  # Copyright(c) 2017 Cavium, 

Inc

#16 lore raw

Re: [dpdk-dev] [PATCH 20.08 9/9] config/arm: support python3 only

22/05/2020 16:10, Kilheeney, Louise:

Quoted 9 lines

From: Thomas Monjalon <[email protected]>

Quoted 4 lines

22/05/2020 15:23, Louise Kilheeney:

Quoted 1 line

Changed script to explicitly use python3 only.

What is the reason of this change?

since python 2 is EOL, Making these scripts to use python3-only, it's part of a general update to have everything use python3. This is already using python 3 and this makes it explicit.

This is to avoid maintaining python 2 compatibility I guess. Please insert this real reason in the git commit log.

Quoted 5 lines

config/arm/armv8_machine.py line 1

Quoted 5 lines
config/arm/armv8_machine.py +1 -1
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/python3

Are you sure all OS have /usr/bin/python3?
Is it called "python" sometimes?
Is it always located in /usr/bin/?
Shouldn't we use "#! /usr/bin/env python3" ?

#17 lore raw

Re: [dpdk-dev] [PATCH 20.08 9/9] config/arm: support python3 only

Quoted 5 lines · Thomas Monjalon

22/05/2020 16:10, Kilheeney, Louise:

Quoted 4 lines

From: Thomas Monjalon <[email protected]>

Quoted 3 lines

22/05/2020 15:23, Louise Kilheeney:

Quoted 1 line

Changed script to explicitly use python3 only.

Quoted 9 lines
Quoted 5 lines
Quoted 1 line

What is the reason of this change?

since python 2 is EOL, Making these scripts to use python3-only, it's part of a general update to have everything use python3. This is already using python 3 and this makes it explicit.

This is to avoid maintaining python 2 compatibility I guess. Please insert this real reason in the git commit log.

Also, python 2 may not be installed (unless explicitly requested) on modern distro releases. My Ubuntu 20.04 still only had a "python3" binary, no "python" binary.

Quoted 12 lines
Quoted 6 lines

config/arm/armv8_machine.py line 1

Quoted 6 lines
config/arm/armv8_machine.py +1 -1
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/python3

Are you sure all OS have /usr/bin/python3?
Is it called "python" sometimes?
Is it always located in /usr/bin/?
Shouldn't we use "#! /usr/bin/env python3" ?

Good catch.
+1 for this. On BSD python is in /usr/local/bin

Tested-by
#18 lore raw

Re: [dpdk-dev] [PATCH 20.08 9/9] config/arm: support python3 only

Quoted 31 lines · Bruce Richardson
Quoted 5 lines · Thomas Monjalon

22/05/2020 16:10, Kilheeney, Louise:

Quoted 4 lines

From: Thomas Monjalon <[email protected]>

Quoted 3 lines

22/05/2020 15:23, Louise Kilheeney:

Quoted 1 line

Changed script to explicitly use python3 only.

Quoted 9 lines
Quoted 5 lines
Quoted 1 line

What is the reason of this change?

since python 2 is EOL, Making these scripts to use python3-only, it's part of a general update to have everything use python3. This is already using python 3 and this makes it explicit.

This is to avoid maintaining python 2 compatibility I guess. Please insert this real reason in the git commit log.

Also, python 2 may not be installed (unless explicitly requested) on modern distro releases. My Ubuntu 20.04 still only had a "python3" binary, no "python" binary.

Quoted 12 lines
Quoted 6 lines

config/arm/armv8_machine.py line 1

Quoted 6 lines
config/arm/armv8_machine.py +1 -1
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/python3

Are you sure all OS have /usr/bin/python3?
Is it called "python" sometimes?
Is it always located in /usr/bin/?
Shouldn't we use "#! /usr/bin/env python3" ?

Quoted 2 lines

Good catch.
+1 for this. On BSD python is in /usr/local/bin

Tested config/arm/armv8_machine.py on an arm64 platform, it works fine. +1 for the patch with changes recomended by Thomas.

#19 lore raw
In reply to Louise Kilheeney

Hi Louise,

2020-05-22, Louise Kilheeney:

Quoted 5 lines

This patch set converts all python scripts in the project to use python3 only. This is due to python2 being EOL in January 2020.

One script not tested: ./config/arm/armv8_machine.py

Quoted 10 lines

Louise Kilheeney (9):
app/test-bbdev: support python3 only usertools/dpdk-devbind: support python3 only usertools/dpdk-telemetry-client: support python3 only usertools/dpdk-pmdinfo: support python3 only usertools/cpu_layout: support python3 only app/test-cmdline: support python3 only app/test: support python3 only
devtools: support python3 only
config/arm: support python3 only

How did you test your changes?

Also, I had understood that python 2 support would not be dropped before 20.11: http://mails.dpdk.org/archives/dev/2020-February/157793.html

Did I miss something?

Signature · 1 line
Robin
#20 lore raw
Quoted 19 lines · Robin Jarry

Hi Louise,

2020-05-22, Louise Kilheeney:

Quoted 5 lines

This patch set converts all python scripts in the project to use python3 only. This is due to python2 being EOL in January 2020.

One script not tested: ./config/arm/armv8_machine.py

Quoted 10 lines

Louise Kilheeney (9):
app/test-bbdev: support python3 only usertools/dpdk-devbind: support python3 only usertools/dpdk-telemetry-client: support python3 only usertools/dpdk-pmdinfo: support python3 only usertools/cpu_layout: support python3 only app/test-cmdline: support python3 only app/test: support python3 only
devtools: support python3 only
config/arm: support python3 only

Quoted 7 lines

How did you test your changes?

Also, I had understood that python 2 support would not be dropped before 20.11: http://mails.dpdk.org/archives/dev/2020-February/157793.html

Did I miss something?

The 20.11 was just a suggestion from me, but I'm not sure it was ever definitively agreed upon. I'd be ok with removing python2 support in either 20.08 or deferring to 20.11. I don't think the patches would be different no matter what release it being targetted. :-)
Is there a particular concern around removing python2 support? I thought all linux distros now have python3, and many are stopping installing python2 by default also.

#21 lore raw

2020-05-25, Bruce Richardson:

Quoted 7 lines

The 20.11 was just a suggestion from me, but I'm not sure it was ever definitively agreed upon. I'd be ok with removing python2 support in either 20.08 or deferring to 20.11. I don't think the patches would be different no matter what release it being targetted. :-)
Is there a particular concern around removing python2 support? I thought all linux distros now have python3, and many are stopping installing python2 by default also.

Absolutely no concern around getting rid of Python 2 support :-)

Thanks for the clarification.

Signature · 1 line
Robin
#22 lore raw
In reply to Robin Jarry

I tested all scripts by running each of them manually, if you have any input into what tests you thinks would be good to run please do let me know.

Louise Kilheeney

Quoted 35 lines

-----Original Message-----
From: Robin Jarry <[email protected]>
Sent: Monday 25 May 2020 08:27
To: Kilheeney, Louise <[email protected]>
Cc: [email protected]
Subject: Re: [dpdk-dev] [PATCH 20.08 0/9] adding support for python 3 only.

Hi Louise,

2020-05-22, Louise Kilheeney:

Quoted 5 lines

This patch set converts all python scripts in the project to use python3 only. This is due to python2 being EOL in January 2020.

One script not tested: ./config/arm/armv8_machine.py

Quoted 10 lines

Louise Kilheeney (9):
app/test-bbdev: support python3 only usertools/dpdk-devbind: support python3 only usertools/dpdk-telemetry-client: support python3 only usertools/dpdk-pmdinfo: support python3 only usertools/cpu_layout: support python3 only app/test-cmdline: support python3 only app/test: support python3 only
devtools: support python3 only
config/arm: support python3 only

How did you test your changes?

Also, I had understood that python 2 support would not be dropped before 20.11: http://mails.dpdk.org/archives/dev/2020-February/157793.html

Did I miss something?

--
Robin

#23 lore raw

2020-05-25, Kilheeney, Louise:

Quoted 3 lines

I tested all scripts by running each of them manually, if you have any input into what tests you thinks would be good to run please do let me know.

This should be enough to debunk the most obvious problems. Thanks.

Signature · 1 line
Robin
#24 lore raw

v2 cover adding support for python 3 only.

In reply to Louise Kilheeney

v2: fix comments on v1.

This patch set converts all python scripts in the project to use python3 only. This is due to python2 being EOL in January 2020.

One script not tested: ./config/arm/armv8_machine.py

Louise Kilheeney (9):
app/test-bbdev: support python3 only usertools/dpdk-devbind: support python3 only usertools/dpdk-telemetry-client: support python3 only usertools/dpdk-pmdinfo: support python3 only usertools/cpu_layout: support python3 only app/test-cmdline: support python3 only app/test: support python3 only
devtools: support python3 only
config/arm: support python3 only

 app/test-bbdev/test-bbdev.py          |  7 +++----
 app/test-cmdline/cmdline_test.py      |  3 +--
 app/test-cmdline/cmdline_test_data.py |  1 +
 app/test/autotest.py                  |  3 +--
 app/test/autotest_data.py             |  1 +
 app/test/autotest_runner.py           | 19 ++++++++-----------
 app/test/autotest_test_funcs.py       |  1 +
 config/arm/armv8_machine.py           |  2 +-
 devtools/update_version_map_abi.py    |  3 +--
 mk/rte.sdktest.mk                     |  6 +++---
 usertools/cpu_layout.py               |  9 ++-------
 usertools/dpdk-devbind.py             | 17 ++++-------------
 usertools/dpdk-pmdinfo.py             |  4 +---
 usertools/dpdk-telemetry-client.py    | 16 ++++------------
 14 files changed, 32 insertions(+), 60 deletions(-)
Signature · 1 line
2.17.1
#25 lore raw

PATCH v2 1/9 app/test-bbdev: support python3 only

Changed script to explicitly use python3 only to avoid maintaining python 2.

Cc Nicolas Chautru <[email protected]>
---
 app/test-bbdev/test-bbdev.py | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
app/test-bbdev/test-bbdev.py +3 -4
index 0194be046..ff2208381 100755
@@ -1,5 +1,4 @@
-#!/usr/bin/env python
-
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Intel Corporation
 
@@ -12,7 +11,7 @@
 from threading import Timer
 
 def kill(process):
-    print "ERROR: Test app timed out"
+    print("ERROR: Test app timed out")
     process.kill()
 
 if "RTE_SDK" in os.environ:
@@ -66,7 +65,7 @@ def kill(process):
 args = parser.parse_args()
 
 if not os.path.exists(args.testapp_path):
-    print "No such file: " + args.testapp_path
+    print ("No such file: " + args.testapp_path)
     sys.exit(1)
 
 params = [args.testapp_path]
Signature · 1 line
2.17.1
Acked-by on v2 1/9
#26 lore raw

Re: [dpdk-dev] [PATCH v2 1/9] app/test-bbdev: support python3 only

Quoted 32 lines · Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2.

Cc Nicolas Chautru <[email protected]>
---
  app/test-bbdev/test-bbdev.py | 7 +++----
  1 file changed, 3 insertions(+), 4 deletions(-)
app/test-bbdev/test-bbdev.py +2 -3
index 0194be046..ff2208381 100755
@@ -1,5 +1,4 @@
-#!/usr/bin/env python
-
+#!/usr/bin/env python3
  # SPDX-License-Identifier: BSD-3-Clause
  # Copyright(c) 2017 Intel Corporation
 
@@ -12,7 +11,7 @@
  from threading import Timer
 
  def kill(process):
-    print "ERROR: Test app timed out"
+    print("ERROR: Test app timed out")
      process.kill()
 
  if "RTE_SDK" in os.environ:

app/test-bbdev/test-bbdev.py def kill(process): · line 68

Quoted 5 lines
app/test-bbdev/test-bbdev.py +1 -1
@@ -66,7 +65,7 @@ def kill(process):
  args = parser.parse_args()
 
  if not os.path.exists(args.testapp_path):
-    print "No such file: " + args.testapp_path
+    print ("No such file: " + args.testapp_path)

Nitpick: don't leave space before parens :)

Quoted 4 lines
app/test-bbdev/test-bbdev.py +0 -0
      sys.exit(1)
 
  params = [args.testapp_path]
Signature · 2 lines
Thanks,
Anatoly
#27 lore raw

PATCH v2 2/9 usertools/dpdk-devbind: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2.

---
v2: removed check_output function as not required for python3.

---
 usertools/dpdk-devbind.py | 17 ++++-------------
 1 file changed, 4 insertions(+), 13 deletions(-)
usertools/dpdk-devbind.py +4 -13
index dc008823f..f29dd7f64 100755
@@ -1,9 +1,8 @@
-#! /usr/bin/env python
+#! /usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 #
 
-from __future__ import print_function
 import sys
 import os
 import getopt
@@ -143,14 +142,6 @@ def usage():
 
     """ % locals())  # replace items from local variables
 
-
-# This is roughly compatible with check_output function in subprocess module
-# which is only available in python 2.7.
-def check_output(args, stderr=None):
-    '''Run a command and capture its output'''
-    return subprocess.Popen(args, stdout=subprocess.PIPE,
-                            stderr=stderr).communicate()[0]
-
 # check if a specific kernel module is loaded
 def module_is_loaded(module):
     global loaded_modules
@@ -207,7 +198,7 @@ def get_pci_device_details(dev_id, probe_lspci):
     device = {}
 
     if probe_lspci:
-        extra_info = check_output(["lspci", "-vmmks", dev_id]).splitlines()
+        extra_info = subprocess.check_output(["lspci", "-vmmks", dev_id]).splitlines()
 
         # parse lspci details
         for line in extra_info:
@@ -244,7 +235,7 @@ def get_device_details(devices_type):
     # first loop through and read details for all devices
     # request machine readable format, with numeric IDs and String
     dev = {}
-    dev_lines = check_output(["lspci", "-Dvmmnnk"]).splitlines()
+    dev_lines = subprocess.check_output(["lspci", "-Dvmmnnk"]).splitlines()
     for dev_line in dev_lines:
         if len(dev_line) == 0:
             if device_type_match(dev, devices_type):
@@ -272,7 +263,7 @@ def get_device_details(devices_type):
         # check what is the interface if any for an ssh connection if
         # any to this host, so we can mark it later.
         ssh_if = []
-        route = check_output(["ip", "-o", "route"])
+        route = subprocess.check_output(["ip", "-o", "route"])
         # filter out all lines for 169.254 routes
         route = "\n".join(filter(lambda ln: not ln.startswith("169.254"),
                              route.decode().splitlines()))
Signature · 1 line
2.17.1
Reviewed-by on v2 2/9
#28 lore raw

Re: [dpdk-dev] [PATCH v2 2/9] usertools/dpdk-devbind: support python3 only

Quoted 16 lines · Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2.

---
v2: removed check_output function as not required for python3.

---
  usertools/dpdk-devbind.py | 17 ++++-------------
  1 file changed, 4 insertions(+), 13 deletions(-)
usertools/dpdk-devbind.py +0 -0
index dc008823f..f29dd7f64 100755

usertools/dpdk-devbind.py line 1

Quoted 2 lines
usertools/dpdk-devbind.py +1 -1
@@ -1,9 +1,8 @@
-#! /usr/bin/env python
+#! /usr/bin/env python3

Maybe drop the space before '/' too here.

Signature · 2 lines
Thanks,
Anatoly
#29 lore raw

PATCH v2 3/9 usertools/dpdk-telemetry-client: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maaintaining python 2.

Cc Kevin Laatz <[email protected]>
---
 usertools/dpdk-telemetry-client.py | 16 ++++------------
 1 file changed, 4 insertions(+), 12 deletions(-)
usertools/dpdk-telemetry-client.py +4 -12
index 35edb7cd2..fa599046a 100755
@@ -1,10 +1,7 @@
-#! /usr/bin/env python
+#! /usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Intel Corporation
 
-from __future__ import print_function
-from __future__ import unicode_literals
-
 import socket
 import os
 import sys
@@ -18,11 +15,6 @@
 GLOBAL_METRICS_REQ = "{\"action\":0,\"command\":\"global_stat_values\",\"data\":null}"
 DEFAULT_FP = "/var/run/dpdk/default_client"
 
-try:
-    raw_input  # Python 2
-except NameError:
-    raw_input = input  # Python 3
-
 class Socket:
 
     def __init__(self):
@@ -49,7 +41,7 @@ def __init__(self): # Creates a client instance
     def __del__(self):
         try:
             if self.unregistered == 0:
-                self.unregister();
+                self.unregister()
         except:
             print("Error - Client could not be destroyed")
 
@@ -82,7 +74,7 @@ def requestMetrics(self): # Requests metrics for given client
 
     def repeatedlyRequestMetrics(self, sleep_time): # Recursively requests metrics for given client
         print("\nPlease enter the number of times you'd like to continuously request Metrics:")
-        n_requests = int(raw_input("\n:"))
+        n_requests = int(input("\n:"))
         print("\033[F") #Removes the user input from screen, cleans it up
         print("\033[K")
         for i in range(n_requests):
@@ -103,7 +95,7 @@ def interactiveMenu(self, sleep_time): # Creates Interactive menu within the scr
             print("[4] Unregister client")
 
             try:
-                self.choice = int(raw_input("\n:"))
+                self.choice = int(input("\n:"))
                 print("\033[F") #Removes the user input for screen, cleans it up
                 print("\033[K")
                 if self.choice == 1:
Signature · 1 line
2.17.1
#31 lore raw

PATCH v2 4/9 usertools/dpdk-pmdinfo: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2.

Cc Neil Horman <[email protected]>
---
 usertools/dpdk-pmdinfo.py | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
usertools/dpdk-pmdinfo.py +1 -3
index 12f20735e..f5cd077f5 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2016  Neil Horman <[email protected]>
 
@@ -7,8 +7,6 @@
 # Utility to dump PMD_INFO_STRING support from an object file
 #
 # -------------------------------------------------------------------------
-from __future__ import print_function
-from __future__ import unicode_literals
 import json
 import io
 import os
Signature · 1 line
2.17.1
Acked-by on v2 4/9
#32 lore raw

Re: [dpdk-dev] [PATCH v2 4/9] usertools/dpdk-pmdinfo: support python3 only

Quoted 6 lines · Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2.

Cc Neil Horman <[email protected]>
Quoted 27 lines
---
 usertools/dpdk-pmdinfo.py | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
usertools/dpdk-pmdinfo.py +1 -3
index 12f20735e..f5cd077f5 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2016  Neil Horman <[email protected]>
 
@@ -7,8 +7,6 @@
 # Utility to dump PMD_INFO_STRING support from an object file
 #
 # -------------------------------------------------------------------------
-from __future__ import print_function
-from __future__ import unicode_literals
 import json
 import io
 import os
Signature · 1 line
2.17.1
Acked-by on v2 4/9
#33 lore raw

Re: [dpdk-dev] [PATCH v2 4/9] usertools/dpdk-pmdinfo: support python3 only

In reply to Louise Kilheeney
Quoted 33 lines · Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2.

Cc Neil Horman <[email protected]>
---
 usertools/dpdk-pmdinfo.py | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
usertools/dpdk-pmdinfo.py +1 -3
index 12f20735e..f5cd077f5 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2016  Neil Horman <[email protected]>
 
@@ -7,8 +7,6 @@
 # Utility to dump PMD_INFO_STRING support from an object file
 #
 # -------------------------------------------------------------------------
-from __future__ import print_function
-from __future__ import unicode_literals
 import json
 import io
 import os
Signature · 1 line
2.17.1

Series

#34 lore raw

PATCH v2 5/9 usertools/cpu_layout: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2.

---
 usertools/cpu_layout.py | 9 ++-------
 1 file changed, 2 insertions(+), 7 deletions(-)
usertools/cpu_layout.py +2 -7
index 6f129b1db..89a48cec4 100755
@@ -1,14 +1,9 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 # Copyright(c) 2017 Cavium, Inc. All rights reserved.
 
-from __future__ import print_function
 import sys
-try:
-    xrange # Python 2
-except NameError:
-    xrange = range # Python 3
 
 sockets = []
 cores = []
@@ -17,7 +12,7 @@
 fd = open("{}/kernel_max".format(base_path))
 max_cpus = int(fd.read())
 fd.close()
-for cpu in xrange(max_cpus + 1):
+for cpu in range(max_cpus + 1):
     try:
         fd = open("{}/cpu{}/topology/core_id".format(base_path, cpu))
     except IOError:
Signature · 1 line
2.17.1
#36 lore raw

PATCH v2 6/9 app/test-cmdline: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2.

Cc Olivier Matz <[email protected]>
---
 app/test-cmdline/cmdline_test.py      | 3 +--
 app/test-cmdline/cmdline_test_data.py | 1 +
 mk/rte.sdktest.mk                     | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)
app/test-cmdline/cmdline_test.py +1 -2
index 3a8fac426..0e7c72459 100755
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # Script that runs cmdline_test app and feeds keystrokes into it.
-from __future__ import print_function
 import cmdline_test_data
 import os
 import pexpect
app/test-cmdline/cmdline_test_data.py +1 -0
index 114d2cb6a..2d9b3262a 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
mk/rte.sdktest.mk +1 -1
index 803018ba3..ad7ef138f 100644
@@ -63,7 +63,7 @@ coverage:
 	@mkdir -p $(AUTOTEST_DIR) ; \
 	cd $(AUTOTEST_DIR) ; \
 	if [ -f $(RTE_OUTPUT)/app/test ]; then \
-		python $(RTE_SDK)/test/cmdline_test/cmdline_test.py \
+		python3 $(RTE_SDK)/test/cmdline_test/cmdline_test.py \
 			$(RTE_OUTPUT)/app/cmdline_test; \
 		ulimit -S -n 100 ; \
 		python $(RTE_SDK)/app/test/autotest.py \
Signature · 1 line
2.17.1
#38 lore raw

PATCH v2 7/9 app/test: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2.

---
v2: removed use of iter function as not needed for python3.

---
 app/test/autotest.py            |  3 +--
 app/test/autotest_data.py       |  1 +
 app/test/autotest_runner.py     | 19 ++++++++-----------
 app/test/autotest_test_funcs.py |  1 +
 mk/rte.sdktest.mk               |  4 ++--
 5 files changed, 13 insertions(+), 15 deletions(-)
app/test/autotest.py +1 -2
index b42f48879..9eef1efbe 100644
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # Script that uses either test app or qemu controlled by python-pexpect
-from __future__ import print_function
 import autotest_data
 import autotest_runner
 import sys
app/test/autotest_data.py +1 -0
index 7b1d01389..e33258026 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
app/test/autotest_runner.py +8 -11
index 95e74c760..b50469dac 100644
@@ -1,10 +1,10 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # The main logic behind running autotests in parallel
 
-from __future__ import print_function
-import StringIO
+import io
 import csv
 from multiprocessing import Pool, Queue
 import pexpect
@@ -50,10 +50,7 @@ def first_cpu_on_node(node_nr):
                 map(os.path.basename, cpu_path)
             )
     )
-    # for compatibility between python 3 and 2 we need to make interable out
-    # of filter return as it returns list in python 2 and a generator in 3
-    m = next(iter(cpu_name))
-    return int(m.group(1))
+    return int(next(cpu_name).group(1))
 
 
 pool_child = None  # per-process child
@@ -78,7 +75,7 @@ def pool_init(queue, result_queue):
     cmdline = "%s %s" % (cmdline, prefix_cmdline)
 
     # prepare logging of init
-    startuplog = StringIO.StringIO()
+    startuplog = io.StringIO()
 
     # run test app
     try:
@@ -86,7 +83,7 @@ def pool_init(queue, result_queue):
         print("\n%s %s\n" % ("=" * 20, prefix), file=startuplog)
         print("\ncmdline=%s" % cmdline, file=startuplog)
 
-        pool_child = pexpect.spawn(cmdline, logfile=startuplog)
+        pool_child = pexpect.spawn(cmdline, logfile=startuplog, encoding='utf-8')
 
         # wait for target to boot
         if not wait_prompt(pool_child):
@@ -138,7 +135,7 @@ def run_test(target, test):
     # create log buffer for each test
     # in multiprocessing environment, the logging would be
     # interleaved and will create a mess, hence the buffering
-    logfile = StringIO.StringIO()
+    logfile = io.StringIO()
     pool_child.logfile = logfile
 
     # make a note when the test started
@@ -210,9 +207,9 @@ def __init__(self, cmdline, target, blacklist, whitelist, n_processes):
         # parse the binary for available test commands
         binary = cmdline.split()[0]
         stripped = 'not stripped' not in \
-                   subprocess.check_output(['file', binary])
+                   subprocess.check_output(['file', binary]).decode()
         if not stripped:
-            symbols = subprocess.check_output(['nm', binary]).decode('utf-8')
+            symbols = subprocess.check_output(['nm', binary]).decode()
             self.avail_cmds = re.findall('test_register_(\w+)', symbols)
         else:
             self.avail_cmds = None
app/test/autotest_test_funcs.py +1 -0
index 26688b713..775dfd1dc 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
mk/rte.sdktest.mk +2 -2
index ad7ef138f..81eff0c08 100644
@@ -49,7 +49,7 @@ test test-fast test-perf test-drivers test-dump:
 	@mkdir -p $(AUTOTEST_DIR) ; \
 	cd $(AUTOTEST_DIR) ; \
 	if [ -f $(RTE_OUTPUT)/app/test ]; then \
-		python $(RTE_SDK)/app/test/autotest.py \
+		python3 $(RTE_SDK)/app/test/autotest.py \
 			$(RTE_OUTPUT)/app/test \
 			$(RTE_TARGET) \
 			$(BLACKLIST) $(WHITELIST); \
@@ -66,7 +66,7 @@ coverage:
 		python3 $(RTE_SDK)/test/cmdline_test/cmdline_test.py \
 			$(RTE_OUTPUT)/app/cmdline_test; \
 		ulimit -S -n 100 ; \
-		python $(RTE_SDK)/app/test/autotest.py \
+		python3 $(RTE_SDK)/app/test/autotest.py \
 			$(RTE_OUTPUT)/app/test \
 			$(RTE_TARGET) \
 			$(BLACKLIST) $(WHITELIST) ; \
Signature · 1 line
2.17.1
#40 lore raw

PATCH v2 8/9 devtools: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2.

Cc Neil Horman <[email protected]>
Cc Ray Kinsella <[email protected]>
---
 devtools/update_version_map_abi.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
devtools/update_version_map_abi.py +1 -2
index e2104e61e..830e6c58c 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2019 Intel Corporation
 
@@ -9,7 +9,6 @@
 from the devtools/update-abi.sh utility.
 """
 
-from __future__ import print_function
 import argparse
 import sys
 import re
Signature · 1 line
2.17.1
#41 lore raw

PATCH v2 9/9 config/arm: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2.

Cc Thomas Monjalon <[email protected]>

---
v2: fixed python3 path.

---
 config/arm/armv8_machine.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
config/arm/armv8_machine.py +1 -1
index 404866d2f..1f689d9a8 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Cavium, Inc
 
Signature · 1 line
2.17.1
#43 lore raw

v3 cover dding support for python 3 only.

In reply to Louise Kilheeney

v3: fix comments on v2.

This patch set converts all python scripts in the project to use python3 only. This is due to python2 being EOL in January 2020.

One script not tested: ./config/arm/armv8_machine.py

Louise Kilheeney (9):
app/test-bbdev: support python3 only usertools/dpdk-devbind: support python3 only usertools/dpdk-telemetry-client: support python3 only usertools/dpdk-pmdinfo: support python3 only usertools/cpu_layout: support python3 only app/test-cmdline: support python3 only app/test: support python3 only
devtools: support python3 only
config/arm: support python3 only

 app/test-bbdev/test-bbdev.py          |  7 +++----
 app/test-cmdline/cmdline_test.py      |  3 +--
 app/test-cmdline/cmdline_test_data.py |  1 +
 app/test/autotest.py                  |  3 +--
 app/test/autotest_data.py             |  1 +
 app/test/autotest_runner.py           | 19 ++++++++-----------
 app/test/autotest_test_funcs.py       |  1 +
 config/arm/armv8_machine.py           |  2 +-
 devtools/update_version_map_abi.py    |  3 +--
 mk/rte.sdktest.mk                     |  6 +++---
 usertools/cpu_layout.py               |  9 ++-------
 usertools/dpdk-devbind.py             | 17 ++++-------------
 usertools/dpdk-pmdinfo.py             |  4 +---
 usertools/dpdk-telemetry-client.py    | 16 ++++------------
 14 files changed, 32 insertions(+), 60 deletions(-)
Signature · 1 line
2.17.1
#44 lore raw

PATCH v3 1/9 app/test-bbdev: support python3 only

Changed script to explicitly use python3 only to avoid maintaining python 2.

Cc Nicolas Chautru <[email protected]>

---
v3: removed extra white space

---
 app/test-bbdev/test-bbdev.py | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
app/test-bbdev/test-bbdev.py +3 -4
index 0194be046..0d41c743d 100755
@@ -1,5 +1,4 @@
-#!/usr/bin/env python
-
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Intel Corporation
 
@@ -12,7 +11,7 @@
 from threading import Timer
 
 def kill(process):
-    print "ERROR: Test app timed out"
+    print("ERROR: Test app timed out")
     process.kill()
 
 if "RTE_SDK" in os.environ:
@@ -66,7 +65,7 @@ def kill(process):
 args = parser.parse_args()
 
 if not os.path.exists(args.testapp_path):
-    print "No such file: " + args.testapp_path
+    print("No such file: " + args.testapp_path)
     sys.exit(1)
 
 params = [args.testapp_path]
Signature · 1 line
2.17.1
#45 lore raw

PATCH v3 2/9 usertools/dpdk-devbind: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2.

---
v3: removed extra white space
v2: removed check_output function as not required for python3.

---
 usertools/dpdk-devbind.py | 17 ++++-------------
 1 file changed, 4 insertions(+), 13 deletions(-)
usertools/dpdk-devbind.py +4 -13
index dc008823f..3e7d525e8 100755
@@ -1,9 +1,8 @@
-#! /usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 #
 
-from __future__ import print_function
 import sys
 import os
 import getopt
@@ -143,14 +142,6 @@ def usage():
 
     """ % locals())  # replace items from local variables
 
-
-# This is roughly compatible with check_output function in subprocess module
-# which is only available in python 2.7.
-def check_output(args, stderr=None):
-    '''Run a command and capture its output'''
-    return subprocess.Popen(args, stdout=subprocess.PIPE,
-                            stderr=stderr).communicate()[0]
-
 # check if a specific kernel module is loaded
 def module_is_loaded(module):
     global loaded_modules
@@ -207,7 +198,7 @@ def get_pci_device_details(dev_id, probe_lspci):
     device = {}
 
     if probe_lspci:
-        extra_info = check_output(["lspci", "-vmmks", dev_id]).splitlines()
+        extra_info = subprocess.check_output(["lspci", "-vmmks", dev_id]).splitlines()
 
         # parse lspci details
         for line in extra_info:
@@ -244,7 +235,7 @@ def get_device_details(devices_type):
     # first loop through and read details for all devices
     # request machine readable format, with numeric IDs and String
     dev = {}
-    dev_lines = check_output(["lspci", "-Dvmmnnk"]).splitlines()
+    dev_lines = subprocess.check_output(["lspci", "-Dvmmnnk"]).splitlines()
     for dev_line in dev_lines:
         if len(dev_line) == 0:
             if device_type_match(dev, devices_type):
@@ -272,7 +263,7 @@ def get_device_details(devices_type):
         # check what is the interface if any for an ssh connection if
         # any to this host, so we can mark it later.
         ssh_if = []
-        route = check_output(["ip", "-o", "route"])
+        route = subprocess.check_output(["ip", "-o", "route"])
         # filter out all lines for 169.254 routes
         route = "\n".join(filter(lambda ln: not ln.startswith("169.254"),
                              route.decode().splitlines()))
Signature · 1 line
2.17.1
#46 lore raw

PATCH v3 3/9 usertools/dpdk-telemetry-client: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maaintaining python 2.

Cc Kevin Laatz <[email protected]>
---
 usertools/dpdk-telemetry-client.py | 16 ++++------------
 1 file changed, 4 insertions(+), 12 deletions(-)
usertools/dpdk-telemetry-client.py +4 -12
index 35edb7cd2..fa599046a 100755
@@ -1,10 +1,7 @@
-#! /usr/bin/env python
+#! /usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Intel Corporation
 
-from __future__ import print_function
-from __future__ import unicode_literals
-
 import socket
 import os
 import sys
@@ -18,11 +15,6 @@
 GLOBAL_METRICS_REQ = "{\"action\":0,\"command\":\"global_stat_values\",\"data\":null}"
 DEFAULT_FP = "/var/run/dpdk/default_client"
 
-try:
-    raw_input  # Python 2
-except NameError:
-    raw_input = input  # Python 3
-
 class Socket:
 
     def __init__(self):
@@ -49,7 +41,7 @@ def __init__(self): # Creates a client instance
     def __del__(self):
         try:
             if self.unregistered == 0:
-                self.unregister();
+                self.unregister()
         except:
             print("Error - Client could not be destroyed")
 
@@ -82,7 +74,7 @@ def requestMetrics(self): # Requests metrics for given client
 
     def repeatedlyRequestMetrics(self, sleep_time): # Recursively requests metrics for given client
         print("\nPlease enter the number of times you'd like to continuously request Metrics:")
-        n_requests = int(raw_input("\n:"))
+        n_requests = int(input("\n:"))
         print("\033[F") #Removes the user input from screen, cleans it up
         print("\033[K")
         for i in range(n_requests):
@@ -103,7 +95,7 @@ def interactiveMenu(self, sleep_time): # Creates Interactive menu within the scr
             print("[4] Unregister client")
 
             try:
-                self.choice = int(raw_input("\n:"))
+                self.choice = int(input("\n:"))
                 print("\033[F") #Removes the user input for screen, cleans it up
                 print("\033[K")
                 if self.choice == 1:
Signature · 1 line
2.17.1
#47 lore raw

PATCH v3 4/9 usertools/dpdk-pmdinfo: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2.

Cc Neil Horman <[email protected]>
---
 usertools/dpdk-pmdinfo.py | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
usertools/dpdk-pmdinfo.py +1 -3
index 12f20735e..f5cd077f5 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2016  Neil Horman <[email protected]>
 
@@ -7,8 +7,6 @@
 # Utility to dump PMD_INFO_STRING support from an object file
 #
 # -------------------------------------------------------------------------
-from __future__ import print_function
-from __future__ import unicode_literals
 import json
 import io
 import os
Signature · 1 line
2.17.1
#48 lore raw

PATCH v3 5/9 usertools/cpu_layout: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2.

---
 usertools/cpu_layout.py | 9 ++-------
 1 file changed, 2 insertions(+), 7 deletions(-)
usertools/cpu_layout.py +2 -7
index 6f129b1db..89a48cec4 100755
@@ -1,14 +1,9 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 # Copyright(c) 2017 Cavium, Inc. All rights reserved.
 
-from __future__ import print_function
 import sys
-try:
-    xrange # Python 2
-except NameError:
-    xrange = range # Python 3
 
 sockets = []
 cores = []
@@ -17,7 +12,7 @@
 fd = open("{}/kernel_max".format(base_path))
 max_cpus = int(fd.read())
 fd.close()
-for cpu in xrange(max_cpus + 1):
+for cpu in range(max_cpus + 1):
     try:
         fd = open("{}/cpu{}/topology/core_id".format(base_path, cpu))
     except IOError:
Signature · 1 line
2.17.1
#49 lore raw

PATCH v3 6/9 app/test-cmdline: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2.

Cc Olivier Matz <[email protected]>
---
 app/test-cmdline/cmdline_test.py      | 3 +--
 app/test-cmdline/cmdline_test_data.py | 1 +
 mk/rte.sdktest.mk                     | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)
app/test-cmdline/cmdline_test.py +1 -2
index 3a8fac426..0e7c72459 100755
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # Script that runs cmdline_test app and feeds keystrokes into it.
-from __future__ import print_function
 import cmdline_test_data
 import os
 import pexpect
app/test-cmdline/cmdline_test_data.py +1 -0
index 114d2cb6a..2d9b3262a 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
mk/rte.sdktest.mk +1 -1
index 803018ba3..ad7ef138f 100644
@@ -63,7 +63,7 @@ coverage:
 	@mkdir -p $(AUTOTEST_DIR) ; \
 	cd $(AUTOTEST_DIR) ; \
 	if [ -f $(RTE_OUTPUT)/app/test ]; then \
-		python $(RTE_SDK)/test/cmdline_test/cmdline_test.py \
+		python3 $(RTE_SDK)/test/cmdline_test/cmdline_test.py \
 			$(RTE_OUTPUT)/app/cmdline_test; \
 		ulimit -S -n 100 ; \
 		python $(RTE_SDK)/app/test/autotest.py \
Signature · 1 line
2.17.1
#50 lore raw

PATCH v3 7/9 app/test: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2.

---
v2: removed use of iter function as not needed for python3.

---
 app/test/autotest.py            |  3 +--
 app/test/autotest_data.py       |  1 +
 app/test/autotest_runner.py     | 19 ++++++++-----------
 app/test/autotest_test_funcs.py |  1 +
 mk/rte.sdktest.mk               |  4 ++--
 5 files changed, 13 insertions(+), 15 deletions(-)
app/test/autotest.py +1 -2
index b42f48879..9eef1efbe 100644
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # Script that uses either test app or qemu controlled by python-pexpect
-from __future__ import print_function
 import autotest_data
 import autotest_runner
 import sys
app/test/autotest_data.py +1 -0
index 7b1d01389..e33258026 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
app/test/autotest_runner.py +8 -11
index 95e74c760..b50469dac 100644
@@ -1,10 +1,10 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # The main logic behind running autotests in parallel
 
-from __future__ import print_function
-import StringIO
+import io
 import csv
 from multiprocessing import Pool, Queue
 import pexpect
@@ -50,10 +50,7 @@ def first_cpu_on_node(node_nr):
                 map(os.path.basename, cpu_path)
             )
     )
-    # for compatibility between python 3 and 2 we need to make interable out
-    # of filter return as it returns list in python 2 and a generator in 3
-    m = next(iter(cpu_name))
-    return int(m.group(1))
+    return int(next(cpu_name).group(1))
 
 
 pool_child = None  # per-process child
@@ -78,7 +75,7 @@ def pool_init(queue, result_queue):
     cmdline = "%s %s" % (cmdline, prefix_cmdline)
 
     # prepare logging of init
-    startuplog = StringIO.StringIO()
+    startuplog = io.StringIO()
 
     # run test app
     try:
@@ -86,7 +83,7 @@ def pool_init(queue, result_queue):
         print("\n%s %s\n" % ("=" * 20, prefix), file=startuplog)
         print("\ncmdline=%s" % cmdline, file=startuplog)
 
-        pool_child = pexpect.spawn(cmdline, logfile=startuplog)
+        pool_child = pexpect.spawn(cmdline, logfile=startuplog, encoding='utf-8')
 
         # wait for target to boot
         if not wait_prompt(pool_child):
@@ -138,7 +135,7 @@ def run_test(target, test):
     # create log buffer for each test
     # in multiprocessing environment, the logging would be
     # interleaved and will create a mess, hence the buffering
-    logfile = StringIO.StringIO()
+    logfile = io.StringIO()
     pool_child.logfile = logfile
 
     # make a note when the test started
@@ -210,9 +207,9 @@ def __init__(self, cmdline, target, blacklist, whitelist, n_processes):
         # parse the binary for available test commands
         binary = cmdline.split()[0]
         stripped = 'not stripped' not in \
-                   subprocess.check_output(['file', binary])
+                   subprocess.check_output(['file', binary]).decode()
         if not stripped:
-            symbols = subprocess.check_output(['nm', binary]).decode('utf-8')
+            symbols = subprocess.check_output(['nm', binary]).decode()
             self.avail_cmds = re.findall('test_register_(\w+)', symbols)
         else:
             self.avail_cmds = None
app/test/autotest_test_funcs.py +1 -0
index 26688b713..775dfd1dc 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
mk/rte.sdktest.mk +2 -2
index ad7ef138f..81eff0c08 100644
@@ -49,7 +49,7 @@ test test-fast test-perf test-drivers test-dump:
 	@mkdir -p $(AUTOTEST_DIR) ; \
 	cd $(AUTOTEST_DIR) ; \
 	if [ -f $(RTE_OUTPUT)/app/test ]; then \
-		python $(RTE_SDK)/app/test/autotest.py \
+		python3 $(RTE_SDK)/app/test/autotest.py \
 			$(RTE_OUTPUT)/app/test \
 			$(RTE_TARGET) \
 			$(BLACKLIST) $(WHITELIST); \
@@ -66,7 +66,7 @@ coverage:
 		python3 $(RTE_SDK)/test/cmdline_test/cmdline_test.py \
 			$(RTE_OUTPUT)/app/cmdline_test; \
 		ulimit -S -n 100 ; \
-		python $(RTE_SDK)/app/test/autotest.py \
+		python3 $(RTE_SDK)/app/test/autotest.py \
 			$(RTE_OUTPUT)/app/test \
 			$(RTE_TARGET) \
 			$(BLACKLIST) $(WHITELIST) ; \
Signature · 1 line
2.17.1
#51 lore raw

PATCH v3 8/9 devtools: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2.

Cc Neil Horman <[email protected]>
Cc Ray Kinsella <[email protected]>
---
 devtools/update_version_map_abi.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
devtools/update_version_map_abi.py +1 -2
index e2104e61e..830e6c58c 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2019 Intel Corporation
 
@@ -9,7 +9,6 @@
 from the devtools/update-abi.sh utility.
 """
 
-from __future__ import print_function
 import argparse
 import sys
 import re
Signature · 1 line
2.17.1
#52 lore raw

PATCH v3 9/9 config/arm: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2.

Cc Thomas Monjalon <[email protected]>

---
v2: fixed python3 path.

---
 config/arm/armv8_machine.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
config/arm/armv8_machine.py +1 -1
index 404866d2f..1f689d9a8 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Cavium, Inc
 
Signature · 1 line
2.17.1
#53 lore raw

Re: [dpdk-dev] [PATCH v3 0/9] dding support for python 3 only.

In reply to Louise Kilheeney

Hello Louise,

Quoted 5 lines · Louise Kilheeney

v3: fix comments on v2.

This patch set converts all python scripts in the project to use python3 only. This is due to python2 being EOL in January 2020.

Thanks for working on this.

Looking at the deprecation notices,
https://doc.dpdk.org/guides/rel_notes/deprecation.html:
"python: Since the beginning of 2020, Python 2 has officially reached
end-of-support: https://www.python.org/doc/sunset-python-2/.
Python 2 support will be completely removed in 20.11.
In 20.08, explicit deprecation warnings will be displayed when running scripts with Python 2."

So I understand that this series targets 20.11 but I did not see the deprecation series for 20.08.

Can you handle this for 20.08?
Thanks.

Signature · 1 line
David Marchand
#54 lore raw

Re: [dpdk-dev] [PATCH v3 0/9] dding support for python 3 only.

Thanks David, I will get on to that now and add warnings hopefully have it up by the end of day.

Lousie

Quoted 39 lines

-----Original Message-----
From: David Marchand <[email protected]>
Sent: Wednesday 8 July 2020 09:59
To: Kilheeney, Louise <[email protected]>
Cc: dev <[email protected]>; Robin Jarry <[email protected]>; Burakov, Anatoly <[email protected]>; Richardson, Bruce <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v3 0/9] dding support for python 3 only.

Hello Louise,

Quoted 5 lines · Louise Kilheeney

v3: fix comments on v2.

This patch set converts all python scripts in the project to use python3 only. This is due to python2 being EOL in January 2020.

Thanks for working on this.

Looking at the deprecation notices,
https://doc.dpdk.org/guides/rel_notes/deprecation.html:
"python: Since the beginning of 2020, Python 2 has officially reached
end-of-support: https://www.python.org/doc/sunset-python-2/.
Python 2 support will be completely removed in 20.11.
In 20.08, explicit deprecation warnings will be displayed when running scripts with Python 2."

So I understand that this series targets 20.11 but I did not see the deprecation series for 20.08.

Can you handle this for 20.08?
Thanks.

--
David Marchand

#55 lore raw

v4 cover adding support for python 3 only.

In reply to Louise Kilheeney

This patch set converts all python scripts in the project to use python3 only and removes all deprecation notices associated with these changes. This is due to python2 being EOL in January 2020.

Louise Kilheeney (9):
usertools/dpdk-telemetry-client: support python3 only usertools/dpdk-devbind: support python3 only usertools/dpdk-pmdinfo: support python3 only usertools/cpu_layout: support python3 only app/test-cmdline: support python3 only app/test: support python3 only
devtools: support python3 only
config/arm: support python3 only app/test-bbdev: support python3 only

 app/test-bbdev/test-bbdev.py          |  7 +------
 app/test-cmdline/cmdline_test.py      |  7 +------
 app/test-cmdline/cmdline_test_data.py |  1 +
 app/test/autotest.py                  |  7 +------
 app/test/autotest_data.py             |  1 +
 app/test/autotest_runner.py           | 21 ++++++++-------------
 app/test/autotest_test_funcs.py       |  1 +
 config/arm/armv8_machine.py           |  2 +-
 devtools/update_version_map_abi.py    |  7 +------
 mk/rte.sdktest.mk                     |  6 +++---
 usertools/cpu_layout.py               | 13 ++-----------
 usertools/dpdk-devbind.py             | 22 ++++------------------
 usertools/dpdk-pmdinfo.py             |  7 +------
 usertools/dpdk-telemetry-client.py    | 20 ++++----------------
 14 files changed, 30 insertions(+), 92 deletions(-)
Signature · 1 line
2.17.1
#56 lore raw

PATCH v4 1/9 usertools/dpdk-telemetry-client: support python3 only

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Kevin Laatz <[email protected]>
---
 usertools/dpdk-telemetry-client.py | 20 ++++----------------
 1 file changed, 4 insertions(+), 16 deletions(-)
usertools/dpdk-telemetry-client.py +4 -16
index 98d28fa89b..fa599046a4 100755
@@ -1,10 +1,7 @@
-#! /usr/bin/env python
+#! /usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Intel Corporation
 
-from __future__ import print_function
-from __future__ import unicode_literals
-
 import socket
 import os
 import sys
@@ -18,15 +15,6 @@
 GLOBAL_METRICS_REQ = "{\"action\":0,\"command\":\"global_stat_values\",\"data\":null}"
 DEFAULT_FP = "/var/run/dpdk/default_client"
 
-try:
-    raw_input  # Python 2
-except NameError:
-    raw_input = input  # Python 3
-
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 class Socket:
 
     def __init__(self):
@@ -53,7 +41,7 @@ def __init__(self): # Creates a client instance
     def __del__(self):
         try:
             if self.unregistered == 0:
-                self.unregister();
+                self.unregister()
         except:
             print("Error - Client could not be destroyed")
 
@@ -86,7 +74,7 @@ def requestMetrics(self): # Requests metrics for given client
 
     def repeatedlyRequestMetrics(self, sleep_time): # Recursively requests metrics for given client
         print("\nPlease enter the number of times you'd like to continuously request Metrics:")
-        n_requests = int(raw_input("\n:"))
+        n_requests = int(input("\n:"))
         print("\033[F") #Removes the user input from screen, cleans it up
         print("\033[K")
         for i in range(n_requests):
@@ -107,7 +95,7 @@ def interactiveMenu(self, sleep_time): # Creates Interactive menu within the scr
             print("[4] Unregister client")
 
             try:
-                self.choice = int(raw_input("\n:"))
+                self.choice = int(input("\n:"))
                 print("\033[F") #Removes the user input for screen, cleans it up
                 print("\033[K")
                 if self.choice == 1:
Signature · 1 line
2.17.1
#58 lore raw

PATCH v4 2/9 usertools/dpdk-devbind: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

---
 usertools/dpdk-devbind.py | 22 ++++------------------
 1 file changed, 4 insertions(+), 18 deletions(-)
usertools/dpdk-devbind.py +4 -18
index 86b6b53c40..bcdc5da881 100755
@@ -1,19 +1,14 @@
-#! /usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 #
 
-from __future__ import print_function
 import sys
 import os
 import getopt
 import subprocess
 from os.path import exists, abspath, dirname, basename
 
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 # The PCI base class for all devices
 network_class = {'Class': '02', 'Vendor': None, 'Device': None,
                     'SVendor': None, 'SDevice': None}
@@ -147,14 +142,6 @@ def usage():
 
     """ % locals())  # replace items from local variables
 
-
-# This is roughly compatible with check_output function in subprocess module
-# which is only available in python 2.7.
-def check_output(args, stderr=None):
-    '''Run a command and capture its output'''
-    return subprocess.Popen(args, stdout=subprocess.PIPE,
-                            stderr=stderr).communicate()[0]
-
 # check if a specific kernel module is loaded
 def module_is_loaded(module):
     global loaded_modules
@@ -211,8 +198,7 @@ def get_pci_device_details(dev_id, probe_lspci):
     device = {}
 
     if probe_lspci:
-        extra_info = check_output(["lspci", "-vmmks", dev_id]).splitlines()
-
+        extra_info = subprocess.check_output(["lspci", "-vmmks", dev_id]).splitlines()
         # parse lspci details
         for line in extra_info:
             if len(line) == 0:
@@ -248,7 +234,7 @@ def get_device_details(devices_type):
     # first loop through and read details for all devices
     # request machine readable format, with numeric IDs and String
     dev = {}
-    dev_lines = check_output(["lspci", "-Dvmmnnk"]).splitlines()
+    dev_lines = subprocess.check_output(["lspci", "-Dvmmnnk"]).splitlines()
     for dev_line in dev_lines:
         if len(dev_line) == 0:
             if device_type_match(dev, devices_type):
@@ -276,7 +262,7 @@ def get_device_details(devices_type):
         # check what is the interface if any for an ssh connection if
         # any to this host, so we can mark it later.
         ssh_if = []
-        route = check_output(["ip", "-o", "route"])
+        route = subprocess.check_output(["ip", "-o", "route"])
         # filter out all lines for 169.254 routes
         route = "\n".join(filter(lambda ln: not ln.startswith("169.254"),
                              route.decode().splitlines()))
Signature · 1 line
2.17.1
#60 lore raw

PATCH v4 3/9 usertools/dpdk-pmdinfo: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Neil Horman <[email protected]>
---
 usertools/dpdk-pmdinfo.py | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
usertools/dpdk-pmdinfo.py +1 -6
index f9ed755176..1661982791 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2016  Neil Horman <[email protected]>
 
@@ -7,8 +7,6 @@
 # Utility to dump PMD_INFO_STRING support from an object file
 #
 # -------------------------------------------------------------------------
-from __future__ import print_function
-from __future__ import unicode_literals
 import json
 import io
 import os
@@ -28,9 +26,6 @@
 pcidb = None
 
 # ===========================================
-if sys.version_info.major < 3:
-        print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-        print("Please use Python 3 instead", file=sys.stderr)
 
 class Vendor:
     """
Signature · 1 line
2.17.1
Reviewed-by on v4 3/9
#61 lore raw

Re: [dpdk-dev] [PATCH v4 3/9] usertools/dpdk-pmdinfo: support python3 only

Quoted 10 lines · Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Neil Horman <[email protected]>
---
 usertools/dpdk-pmdinfo.py | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
#62 lore raw

PATCH v4 4/9 usertools/cpu_layout: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

---
 usertools/cpu_layout.py | 13 ++-----------
 1 file changed, 2 insertions(+), 11 deletions(-)
usertools/cpu_layout.py +2 -11
index 5423c7965f..89a48cec46 100755
@@ -1,18 +1,9 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 # Copyright(c) 2017 Cavium, Inc. All rights reserved.
 
-from __future__ import print_function
 import sys
-try:
-    xrange # Python 2
-except NameError:
-    xrange = range # Python 3
-
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
 
 sockets = []
 cores = []
@@ -21,7 +12,7 @@
 fd = open("{}/kernel_max".format(base_path))
 max_cpus = int(fd.read())
 fd.close()
-for cpu in xrange(max_cpus + 1):
+for cpu in range(max_cpus + 1):
     try:
         fd = open("{}/cpu{}/topology/core_id".format(base_path, cpu))
     except IOError:
Signature · 1 line
2.17.1
#64 lore raw

PATCH v4 5/9 app/test-cmdline: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Olivier Matz <[email protected]>
---
 app/test-cmdline/cmdline_test.py      | 7 +------
 app/test-cmdline/cmdline_test_data.py | 1 +
 mk/rte.sdktest.mk                     | 2 +-
 3 files changed, 3 insertions(+), 7 deletions(-)
app/test-cmdline/cmdline_test.py +1 -6
index 954428e2bf..ec55647f6e 100755
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # Script that runs cmdline_test app and feeds keystrokes into it.
-from __future__ import print_function
 import cmdline_test_data
 import os
 import pexpect
@@ -19,10 +18,6 @@ def runTest(child, test):
         return 0
     child.expect(test["Result"], 1)
 
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 #
 # history test is a special case
 #
app/test-cmdline/cmdline_test_data.py +1 -0
index 114d2cb6a0..2d9b3262a6 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
mk/rte.sdktest.mk +1 -1
index 803018ba3a..ad7ef138f0 100644
@@ -63,7 +63,7 @@ coverage:
 	@mkdir -p $(AUTOTEST_DIR) ; \
 	cd $(AUTOTEST_DIR) ; \
 	if [ -f $(RTE_OUTPUT)/app/test ]; then \
-		python $(RTE_SDK)/test/cmdline_test/cmdline_test.py \
+		python3 $(RTE_SDK)/test/cmdline_test/cmdline_test.py \
 			$(RTE_OUTPUT)/app/cmdline_test; \
 		ulimit -S -n 100 ; \
 		python $(RTE_SDK)/app/test/autotest.py \
Signature · 1 line
2.17.1
#65 lore raw

Re: [dpdk-dev] [PATCH v4 5/9] app/test-cmdline: support python3 only

Quoted 7 lines · Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Olivier Matz <[email protected]>

---

With this patch applied I'm seeing a failure with the final test the history fill test. It passes without this patch applied. Do you see the same thing?

/Bruce

#66 lore raw

PATCH v4 6/9 app/test: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

---
 app/test/autotest.py            |  7 +------
 app/test/autotest_data.py       |  1 +
 app/test/autotest_runner.py     | 21 ++++++++-------------
 app/test/autotest_test_funcs.py |  1 +
 mk/rte.sdktest.mk               |  4 ++--
 5 files changed, 13 insertions(+), 21 deletions(-)
app/test/autotest.py +1 -6
index cf7584ccd7..9eef1efbe5 100644
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # Script that uses either test app or qemu controlled by python-pexpect
-from __future__ import print_function
 import autotest_data
 import autotest_runner
 import sys
@@ -17,10 +16,6 @@ def usage():
     usage()
     sys.exit(1)
 
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 target = sys.argv[2]
 
 test_whitelist = None
app/test/autotest_data.py +1 -0
index 4b7da45e09..097638941f 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
app/test/autotest_runner.py +8 -13
index 95e74c760d..998fe57a55 100644
@@ -1,10 +1,10 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # The main logic behind running autotests in parallel
 
-from __future__ import print_function
-import StringIO
+import io
 import csv
 from multiprocessing import Pool, Queue
 import pexpect
@@ -50,11 +50,7 @@ def first_cpu_on_node(node_nr):
                 map(os.path.basename, cpu_path)
             )
     )
-    # for compatibility between python 3 and 2 we need to make interable out
-    # of filter return as it returns list in python 2 and a generator in 3
-    m = next(iter(cpu_name))
-    return int(m.group(1))
-
+    return int(next(cpu_name).group(1))
 
 pool_child = None  # per-process child
 
@@ -78,7 +74,7 @@ def pool_init(queue, result_queue):
     cmdline = "%s %s" % (cmdline, prefix_cmdline)
 
     # prepare logging of init
-    startuplog = StringIO.StringIO()
+    startuplog = io.StringIO()
 
     # run test app
     try:
@@ -86,8 +82,7 @@ def pool_init(queue, result_queue):
         print("\n%s %s\n" % ("=" * 20, prefix), file=startuplog)
         print("\ncmdline=%s" % cmdline, file=startuplog)
 
-        pool_child = pexpect.spawn(cmdline, logfile=startuplog)
-
+        pool_child = pexpect.spawn(cmdline, logfile=startuplog, encoding='utf-8')
         # wait for target to boot
         if not wait_prompt(pool_child):
             pool_child.close()
@@ -138,7 +133,7 @@ def run_test(target, test):
     # create log buffer for each test
     # in multiprocessing environment, the logging would be
     # interleaved and will create a mess, hence the buffering
-    logfile = StringIO.StringIO()
+    logfile = io.StringIO()
     pool_child.logfile = logfile
 
     # make a note when the test started
@@ -210,9 +205,9 @@ def __init__(self, cmdline, target, blacklist, whitelist, n_processes):
         # parse the binary for available test commands
         binary = cmdline.split()[0]
         stripped = 'not stripped' not in \
-                   subprocess.check_output(['file', binary])
+                   subprocess.check_output(['file', binary]).decode()
         if not stripped:
-            symbols = subprocess.check_output(['nm', binary]).decode('utf-8')
+            symbols = subprocess.check_output(['nm', binary]).decode()
             self.avail_cmds = re.findall('test_register_(\w+)', symbols)
         else:
             self.avail_cmds = None
app/test/autotest_test_funcs.py +1 -0
index 26688b7132..775dfd1dc5 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
mk/rte.sdktest.mk +2 -2
index ad7ef138f0..81eff0c08f 100644
@@ -49,7 +49,7 @@ test test-fast test-perf test-drivers test-dump:
 	@mkdir -p $(AUTOTEST_DIR) ; \
 	cd $(AUTOTEST_DIR) ; \
 	if [ -f $(RTE_OUTPUT)/app/test ]; then \
-		python $(RTE_SDK)/app/test/autotest.py \
+		python3 $(RTE_SDK)/app/test/autotest.py \
 			$(RTE_OUTPUT)/app/test \
 			$(RTE_TARGET) \
 			$(BLACKLIST) $(WHITELIST); \
@@ -66,7 +66,7 @@ coverage:
 		python3 $(RTE_SDK)/test/cmdline_test/cmdline_test.py \
 			$(RTE_OUTPUT)/app/cmdline_test; \
 		ulimit -S -n 100 ; \
-		python $(RTE_SDK)/app/test/autotest.py \
+		python3 $(RTE_SDK)/app/test/autotest.py \
 			$(RTE_OUTPUT)/app/test \
 			$(RTE_TARGET) \
 			$(BLACKLIST) $(WHITELIST) ; \
Signature · 1 line
2.17.1
#67 lore raw

PATCH v4 7/9 devtools: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Neil Horman <[email protected]>
Cc Ray Kinsella <[email protected]>
---
 devtools/update_version_map_abi.py | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
devtools/update_version_map_abi.py +1 -6
index 80a61641ed..830e6c58c8 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2019 Intel Corporation
 
@@ -9,7 +9,6 @@
 from the devtools/update-abi.sh utility.
 """
 
-from __future__ import print_function
 import argparse
 import sys
 import re
@@ -160,10 +159,6 @@ def __generate_internal_abi(f_out, lines):
     print("};", file=f_out)
 
 def __main():
-    if sys.version_info.major < 3:
-        print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-        print("Please use Python 3 instead", file=sys.stderr)
-
     arg_parser = argparse.ArgumentParser(
         description='Merge versions in linker version script.')
 
Signature · 1 line
2.17.1
Acked-by on v4 7/9
#68 lore raw

Re: [dpdk-dev] [PATCH v4 7/9] devtools: support python3 only

Quoted 31 lines · Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Neil Horman <[email protected]>
Cc Ray Kinsella <[email protected]>
---
 devtools/update_version_map_abi.py | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
devtools/update_version_map_abi.py +1 -2
index 80a61641ed..830e6c58c8 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2019 Intel Corporation
 
@@ -9,7 +9,6 @@
 from the devtools/update-abi.sh utility.
 """
 
-from __future__ import print_function
 import argparse
 import sys
 import re

devtools/update_version_map_abi.py def __generate_internal_abi(f_out, lines): · line 164

Quoted 10 lines
devtools/update_version_map_abi.py +0 -4
@@ -160,10 +159,6 @@ def __generate_internal_abi(f_out, lines):
     print("};", file=f_out)
 
 def __main():
-    if sys.version_info.major < 3:
-        print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-        print("Please use Python 3 instead", file=sys.stderr)
-
     arg_parser = argparse.ArgumentParser(
         description='Merge versions in linker version script.')
 
#69 lore raw

PATCH v4 8/9 config/arm: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2.

Cc Thomas Monjalon <[email protected]>
---
 config/arm/armv8_machine.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
config/arm/armv8_machine.py +1 -1
index 404866d2f8..1f689d9a83 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Cavium, Inc
 
Signature · 1 line
2.17.1
#70 lore raw

PATCH v4 9/9 app/test-bbdev: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Nicolas Chautru <[email protected]>
---
 app/test-bbdev/test-bbdev.py | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
app/test-bbdev/test-bbdev.py +1 -6
index 5ae2dc6c49..7d67dbe30a 100755
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Intel Corporation
 
-from __future__ import print_function
 import sys
 import os
 import argparse
@@ -16,10 +15,6 @@ def kill(process):
     print("ERROR: Test app timed out")
     process.kill()
 
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 if "RTE_SDK" in os.environ:
     dpdk_path = os.environ["RTE_SDK"]
 else:
Signature · 1 line
2.17.1
#71 lore raw

v5 cover adding support for python 3 only.

In reply to Louise Kilheeney

This patch set converts all python scripts in the project to use python3 only and removes all deprecation notices associated with these changes. This is due to python2 being EOL in January 2020.

Louise Kilheeney (9):
usertools/dpdk-telemetry-client: support python3 only usertools/dpdk-devbind: support python3 only usertools/dpdk-pmdinfo: support python3 only usertools/cpu_layout: support python3 only app/test-cmdline: support python3 only app/test: support python3 only
devtools: support python3 only
config/arm: support python3 only app/test-bbdev: support python3 only

 app/test-bbdev/test-bbdev.py          |  7 +------
 app/test-cmdline/cmdline_test.py      |  9 ++-------
 app/test-cmdline/cmdline_test_data.py |  1 +
 app/test/autotest.py                  |  7 +------
 app/test/autotest_data.py             |  1 +
 app/test/autotest_runner.py           | 21 ++++++++-------------
 app/test/autotest_test_funcs.py       |  1 +
 config/arm/armv8_machine.py           |  2 +-
 devtools/update_version_map_abi.py    |  7 +------
 mk/rte.sdktest.mk                     |  6 +++---
 usertools/cpu_layout.py               | 13 ++-----------
 usertools/dpdk-devbind.py             | 22 ++++------------------
 usertools/dpdk-pmdinfo.py             |  7 +------
 usertools/dpdk-telemetry-client.py    | 20 ++++----------------
 14 files changed, 31 insertions(+), 93 deletions(-)
Signature · 1 line
2.17.1
#72 lore raw

PATCH v5 1/9 usertools/dpdk-telemetry-client: support python3 only

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Kevin Laatz <[email protected]>
---
 usertools/dpdk-telemetry-client.py | 20 ++++----------------
 1 file changed, 4 insertions(+), 16 deletions(-)
usertools/dpdk-telemetry-client.py +4 -16
index 98d28fa89b..fa599046a4 100755
@@ -1,10 +1,7 @@
-#! /usr/bin/env python
+#! /usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Intel Corporation
 
-from __future__ import print_function
-from __future__ import unicode_literals
-
 import socket
 import os
 import sys
@@ -18,15 +15,6 @@
 GLOBAL_METRICS_REQ = "{\"action\":0,\"command\":\"global_stat_values\",\"data\":null}"
 DEFAULT_FP = "/var/run/dpdk/default_client"
 
-try:
-    raw_input  # Python 2
-except NameError:
-    raw_input = input  # Python 3
-
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 class Socket:
 
     def __init__(self):
@@ -53,7 +41,7 @@ def __init__(self): # Creates a client instance
     def __del__(self):
         try:
             if self.unregistered == 0:
-                self.unregister();
+                self.unregister()
         except:
             print("Error - Client could not be destroyed")
 
@@ -86,7 +74,7 @@ def requestMetrics(self): # Requests metrics for given client
 
     def repeatedlyRequestMetrics(self, sleep_time): # Recursively requests metrics for given client
         print("\nPlease enter the number of times you'd like to continuously request Metrics:")
-        n_requests = int(raw_input("\n:"))
+        n_requests = int(input("\n:"))
         print("\033[F") #Removes the user input from screen, cleans it up
         print("\033[K")
         for i in range(n_requests):
@@ -107,7 +95,7 @@ def interactiveMenu(self, sleep_time): # Creates Interactive menu within the scr
             print("[4] Unregister client")
 
             try:
-                self.choice = int(raw_input("\n:"))
+                self.choice = int(input("\n:"))
                 print("\033[F") #Removes the user input for screen, cleans it up
                 print("\033[K")
                 if self.choice == 1:
Signature · 1 line
2.17.1
#73 lore raw

PATCH v5 2/9 usertools/dpdk-devbind: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

---
 usertools/dpdk-devbind.py | 22 ++++------------------
 1 file changed, 4 insertions(+), 18 deletions(-)
usertools/dpdk-devbind.py +4 -18
index 86b6b53c40..bcdc5da881 100755
@@ -1,19 +1,14 @@
-#! /usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 #
 
-from __future__ import print_function
 import sys
 import os
 import getopt
 import subprocess
 from os.path import exists, abspath, dirname, basename
 
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 # The PCI base class for all devices
 network_class = {'Class': '02', 'Vendor': None, 'Device': None,
                     'SVendor': None, 'SDevice': None}
@@ -147,14 +142,6 @@ def usage():
 
     """ % locals())  # replace items from local variables
 
-
-# This is roughly compatible with check_output function in subprocess module
-# which is only available in python 2.7.
-def check_output(args, stderr=None):
-    '''Run a command and capture its output'''
-    return subprocess.Popen(args, stdout=subprocess.PIPE,
-                            stderr=stderr).communicate()[0]
-
 # check if a specific kernel module is loaded
 def module_is_loaded(module):
     global loaded_modules
@@ -211,8 +198,7 @@ def get_pci_device_details(dev_id, probe_lspci):
     device = {}
 
     if probe_lspci:
-        extra_info = check_output(["lspci", "-vmmks", dev_id]).splitlines()
-
+        extra_info = subprocess.check_output(["lspci", "-vmmks", dev_id]).splitlines()
         # parse lspci details
         for line in extra_info:
             if len(line) == 0:
@@ -248,7 +234,7 @@ def get_device_details(devices_type):
     # first loop through and read details for all devices
     # request machine readable format, with numeric IDs and String
     dev = {}
-    dev_lines = check_output(["lspci", "-Dvmmnnk"]).splitlines()
+    dev_lines = subprocess.check_output(["lspci", "-Dvmmnnk"]).splitlines()
     for dev_line in dev_lines:
         if len(dev_line) == 0:
             if device_type_match(dev, devices_type):
@@ -276,7 +262,7 @@ def get_device_details(devices_type):
         # check what is the interface if any for an ssh connection if
         # any to this host, so we can mark it later.
         ssh_if = []
-        route = check_output(["ip", "-o", "route"])
+        route = subprocess.check_output(["ip", "-o", "route"])
         # filter out all lines for 169.254 routes
         route = "\n".join(filter(lambda ln: not ln.startswith("169.254"),
                              route.decode().splitlines()))
Signature · 1 line
2.17.1
#74 lore raw

PATCH v5 3/9 usertools/dpdk-pmdinfo: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Neil Horman <[email protected]>
---
 usertools/dpdk-pmdinfo.py | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
usertools/dpdk-pmdinfo.py +1 -6
index f9ed755176..1661982791 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2016  Neil Horman <[email protected]>
 
@@ -7,8 +7,6 @@
 # Utility to dump PMD_INFO_STRING support from an object file
 #
 # -------------------------------------------------------------------------
-from __future__ import print_function
-from __future__ import unicode_literals
 import json
 import io
 import os
@@ -28,9 +26,6 @@
 pcidb = None
 
 # ===========================================
-if sys.version_info.major < 3:
-        print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-        print("Please use Python 3 instead", file=sys.stderr)
 
 class Vendor:
     """
Signature · 1 line
2.17.1
Acked-by on v5 3/9
#75 lore raw

Re: [dpdk-dev] [PATCH v5 3/9] usertools/dpdk-pmdinfo: support python3 only

Quoted 44 lines · Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Neil Horman <[email protected]>
---
 usertools/dpdk-pmdinfo.py | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
usertools/dpdk-pmdinfo.py +1 -6
index f9ed755176..1661982791 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2016  Neil Horman <[email protected]>
 
@@ -7,8 +7,6 @@
 # Utility to dump PMD_INFO_STRING support from an object file
 #
 # -------------------------------------------------------------------------
-from __future__ import print_function
-from __future__ import unicode_literals
 import json
 import io
 import os
@@ -28,9 +26,6 @@
 pcidb = None
 
 # ===========================================
-if sys.version_info.major < 3:
-        print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-        print("Please use Python 3 instead", file=sys.stderr)
 
 class Vendor:
     """
Signature · 1 line
2.17.1
#76 lore raw

PATCH v5 4/9 usertools/cpu_layout: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

---
 usertools/cpu_layout.py | 13 ++-----------
 1 file changed, 2 insertions(+), 11 deletions(-)
usertools/cpu_layout.py +2 -11
index 5423c7965f..89a48cec46 100755
@@ -1,18 +1,9 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 # Copyright(c) 2017 Cavium, Inc. All rights reserved.
 
-from __future__ import print_function
 import sys
-try:
-    xrange # Python 2
-except NameError:
-    xrange = range # Python 3
-
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
 
 sockets = []
 cores = []
@@ -21,7 +12,7 @@
 fd = open("{}/kernel_max".format(base_path))
 max_cpus = int(fd.read())
 fd.close()
-for cpu in xrange(max_cpus + 1):
+for cpu in range(max_cpus + 1):
     try:
         fd = open("{}/cpu{}/topology/core_id".format(base_path, cpu))
     except IOError:
Signature · 1 line
2.17.1
#77 lore raw

PATCH v5 5/9 app/test-cmdline: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Olivier Matz <[email protected]>

---
v5: fixed python3 issue casuing script to fail

---
 app/test-cmdline/cmdline_test.py      | 9 ++-------
 app/test-cmdline/cmdline_test_data.py | 1 +
 mk/rte.sdktest.mk                     | 2 +-
 3 files changed, 4 insertions(+), 8 deletions(-)
app/test-cmdline/cmdline_test.py +2 -7
index 954428e2bf..074fed7e7f 100755
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # Script that runs cmdline_test app and feeds keystrokes into it.
-from __future__ import print_function
 import cmdline_test_data
 import os
 import pexpect
@@ -19,10 +18,6 @@ def runTest(child, test):
         return 0
     child.expect(test["Result"], 1)
 
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 #
 # history test is a special case
 #
@@ -43,7 +38,7 @@ def runHistoryTest(child):
     i = 0
 
     # fill the history with numbers
-    while i < history_size / 10:
+    while i < int(history_size / 10):
         # add 1 to prevent from parsing as octals
         child.send("1" + str(i).zfill(8) + cmdline_test_data.ENTER)
         # the app will simply print out the number
app/test-cmdline/cmdline_test_data.py +1 -0
index 114d2cb6a0..2d9b3262a6 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
mk/rte.sdktest.mk +1 -1
index 803018ba3a..ad7ef138f0 100644
@@ -63,7 +63,7 @@ coverage:
 	@mkdir -p $(AUTOTEST_DIR) ; \
 	cd $(AUTOTEST_DIR) ; \
 	if [ -f $(RTE_OUTPUT)/app/test ]; then \
-		python $(RTE_SDK)/test/cmdline_test/cmdline_test.py \
+		python3 $(RTE_SDK)/test/cmdline_test/cmdline_test.py \
 			$(RTE_OUTPUT)/app/cmdline_test; \
 		ulimit -S -n 100 ; \
 		python $(RTE_SDK)/app/test/autotest.py \
Signature · 1 line
2.17.1
Reviewed-by on v5 5/9
#78 lore raw

Re: [dpdk-dev] [PATCH v5 5/9] app/test-cmdline: support python3 only

Quoted 9 lines · Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Olivier Matz <[email protected]>

---
v5: fixed python3 issue casuing script to fail

It would be good to add a little more detail of the failure and fix to help reviewers, I believe the fix is the typecast to int() around the division to avoid floating point numbers messing up the loop count.

Quoted 66 lines
---
 app/test-cmdline/cmdline_test.py      | 9 ++-------
 app/test-cmdline/cmdline_test_data.py | 1 +
 mk/rte.sdktest.mk                     | 2 +-
 3 files changed, 4 insertions(+), 8 deletions(-)
app/test-cmdline/cmdline_test.py +2 -7
index 954428e2bf..074fed7e7f 100755
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # Script that runs cmdline_test app and feeds keystrokes into it.
-from __future__ import print_function
 import cmdline_test_data
 import os
 import pexpect
@@ -19,10 +18,6 @@ def runTest(child, test):
         return 0
     child.expect(test["Result"], 1)
 
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 #
 # history test is a special case
 #
@@ -43,7 +38,7 @@ def runHistoryTest(child):
     i = 0
 
     # fill the history with numbers
-    while i < history_size / 10:
+    while i < int(history_size / 10):
         # add 1 to prevent from parsing as octals
         child.send("1" + str(i).zfill(8) + cmdline_test_data.ENTER)
         # the app will simply print out the number
app/test-cmdline/cmdline_test_data.py +1 -0
index 114d2cb6a0..2d9b3262a6 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
mk/rte.sdktest.mk +1 -1
index 803018ba3a..ad7ef138f0 100644
@@ -63,7 +63,7 @@ coverage:
 	@mkdir -p $(AUTOTEST_DIR) ; \
 	cd $(AUTOTEST_DIR) ; \
 	if [ -f $(RTE_OUTPUT)/app/test ]; then \
-		python $(RTE_SDK)/test/cmdline_test/cmdline_test.py \
+		python3 $(RTE_SDK)/test/cmdline_test/cmdline_test.py \
 			$(RTE_OUTPUT)/app/cmdline_test; \
 		ulimit -S -n 100 ; \
 		python $(RTE_SDK)/app/test/autotest.py \
Signature · 1 line
2.17.1
#79 lore raw

PATCH v5 6/9 app/test: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

---
 app/test/autotest.py            |  7 +------
 app/test/autotest_data.py       |  1 +
 app/test/autotest_runner.py     | 21 ++++++++-------------
 app/test/autotest_test_funcs.py |  1 +
 mk/rte.sdktest.mk               |  4 ++--
 5 files changed, 13 insertions(+), 21 deletions(-)
app/test/autotest.py +1 -6
index cf7584ccd7..9eef1efbe5 100644
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # Script that uses either test app or qemu controlled by python-pexpect
-from __future__ import print_function
 import autotest_data
 import autotest_runner
 import sys
@@ -17,10 +16,6 @@ def usage():
     usage()
     sys.exit(1)
 
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 target = sys.argv[2]
 
 test_whitelist = None
app/test/autotest_data.py +1 -0
index 4b7da45e09..097638941f 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
app/test/autotest_runner.py +8 -13
index 95e74c760d..998fe57a55 100644
@@ -1,10 +1,10 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # The main logic behind running autotests in parallel
 
-from __future__ import print_function
-import StringIO
+import io
 import csv
 from multiprocessing import Pool, Queue
 import pexpect
@@ -50,11 +50,7 @@ def first_cpu_on_node(node_nr):
                 map(os.path.basename, cpu_path)
             )
     )
-    # for compatibility between python 3 and 2 we need to make interable out
-    # of filter return as it returns list in python 2 and a generator in 3
-    m = next(iter(cpu_name))
-    return int(m.group(1))
-
+    return int(next(cpu_name).group(1))
 
 pool_child = None  # per-process child
 
@@ -78,7 +74,7 @@ def pool_init(queue, result_queue):
     cmdline = "%s %s" % (cmdline, prefix_cmdline)
 
     # prepare logging of init
-    startuplog = StringIO.StringIO()
+    startuplog = io.StringIO()
 
     # run test app
     try:
@@ -86,8 +82,7 @@ def pool_init(queue, result_queue):
         print("\n%s %s\n" % ("=" * 20, prefix), file=startuplog)
         print("\ncmdline=%s" % cmdline, file=startuplog)
 
-        pool_child = pexpect.spawn(cmdline, logfile=startuplog)
-
+        pool_child = pexpect.spawn(cmdline, logfile=startuplog, encoding='utf-8')
         # wait for target to boot
         if not wait_prompt(pool_child):
             pool_child.close()
@@ -138,7 +133,7 @@ def run_test(target, test):
     # create log buffer for each test
     # in multiprocessing environment, the logging would be
     # interleaved and will create a mess, hence the buffering
-    logfile = StringIO.StringIO()
+    logfile = io.StringIO()
     pool_child.logfile = logfile
 
     # make a note when the test started
@@ -210,9 +205,9 @@ def __init__(self, cmdline, target, blacklist, whitelist, n_processes):
         # parse the binary for available test commands
         binary = cmdline.split()[0]
         stripped = 'not stripped' not in \
-                   subprocess.check_output(['file', binary])
+                   subprocess.check_output(['file', binary]).decode()
         if not stripped:
-            symbols = subprocess.check_output(['nm', binary]).decode('utf-8')
+            symbols = subprocess.check_output(['nm', binary]).decode()
             self.avail_cmds = re.findall('test_register_(\w+)', symbols)
         else:
             self.avail_cmds = None
app/test/autotest_test_funcs.py +1 -0
index 26688b7132..775dfd1dc5 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
mk/rte.sdktest.mk +2 -2
index ad7ef138f0..81eff0c08f 100644
@@ -49,7 +49,7 @@ test test-fast test-perf test-drivers test-dump:
 	@mkdir -p $(AUTOTEST_DIR) ; \
 	cd $(AUTOTEST_DIR) ; \
 	if [ -f $(RTE_OUTPUT)/app/test ]; then \
-		python $(RTE_SDK)/app/test/autotest.py \
+		python3 $(RTE_SDK)/app/test/autotest.py \
 			$(RTE_OUTPUT)/app/test \
 			$(RTE_TARGET) \
 			$(BLACKLIST) $(WHITELIST); \
@@ -66,7 +66,7 @@ coverage:
 		python3 $(RTE_SDK)/test/cmdline_test/cmdline_test.py \
 			$(RTE_OUTPUT)/app/cmdline_test; \
 		ulimit -S -n 100 ; \
-		python $(RTE_SDK)/app/test/autotest.py \
+		python3 $(RTE_SDK)/app/test/autotest.py \
 			$(RTE_OUTPUT)/app/test \
 			$(RTE_TARGET) \
 			$(BLACKLIST) $(WHITELIST) ; \
Signature · 1 line
2.17.1
#80 lore raw

PATCH v5 7/9 devtools: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Neil Horman <[email protected]>
Cc Ray Kinsella <[email protected]>
---
 devtools/update_version_map_abi.py | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
devtools/update_version_map_abi.py +1 -6
index 80a61641ed..830e6c58c8 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2019 Intel Corporation
 
@@ -9,7 +9,6 @@
 from the devtools/update-abi.sh utility.
 """
 
-from __future__ import print_function
 import argparse
 import sys
 import re
@@ -160,10 +159,6 @@ def __generate_internal_abi(f_out, lines):
     print("};", file=f_out)
 
 def __main():
-    if sys.version_info.major < 3:
-        print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-        print("Please use Python 3 instead", file=sys.stderr)
-
     arg_parser = argparse.ArgumentParser(
         description='Merge versions in linker version script.')
 
Signature · 1 line
2.17.1
Acked-by on v5 7/9
#81 lore raw

Re: [dpdk-dev] [PATCH v5 7/9] devtools: support python3 only

Quoted 45 lines · Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Neil Horman <[email protected]>
Cc Ray Kinsella <[email protected]>
---
 devtools/update_version_map_abi.py | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
devtools/update_version_map_abi.py +1 -6
index 80a61641ed..830e6c58c8 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2019 Intel Corporation
 
@@ -9,7 +9,6 @@
 from the devtools/update-abi.sh utility.
 """
 
-from __future__ import print_function
 import argparse
 import sys
 import re
@@ -160,10 +159,6 @@ def __generate_internal_abi(f_out, lines):
     print("};", file=f_out)
 
 def __main():
-    if sys.version_info.major < 3:
-        print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-        print("Please use Python 3 instead", file=sys.stderr)
-
     arg_parser = argparse.ArgumentParser(
         description='Merge versions in linker version script.')
 
Signature · 1 line
2.17.1
#82 lore raw

PATCH v5 8/9 config/arm: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2.

Cc Thomas Monjalon <[email protected]>
---
 config/arm/armv8_machine.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
config/arm/armv8_machine.py +1 -1
index 404866d2f8..1f689d9a83 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Cavium, Inc
 
Signature · 1 line
2.17.1
#83 lore raw

PATCH v5 9/9 app/test-bbdev: support python3 only

In reply to Louise Kilheeney

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Nicolas Chautru <[email protected]>
---
 app/test-bbdev/test-bbdev.py | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
app/test-bbdev/test-bbdev.py +1 -6
index 5ae2dc6c49..7d67dbe30a 100755
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Intel Corporation
 
-from __future__ import print_function
 import sys
 import os
 import argparse
@@ -16,10 +15,6 @@ def kill(process):
     print("ERROR: Test app timed out")
     process.kill()
 
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 if "RTE_SDK" in os.environ:
     dpdk_path = os.environ["RTE_SDK"]
 else:
Signature · 1 line
2.17.1
#84 lore raw
In reply to Louise Kilheeney

Hello Louise,

There was a comment from Bruce on patch 5 commitlog. There is a conflict since make removal.
And I think buildtools/map_to_win.py has been missed.

Signature · 1 line
David Marchand
#85 lore raw

v6 cover adding support for python 3 only

In reply to Louise Kilheeney

This patch set converts all python scripts in the project to use python3 only and removes all deprecation notices associated with these changes. This is due to python2 being EOL in January 2020.

---
v6:
- rebased, removing conflict with make removal patchset.
- added changes to buildtools/map_to_win.py

Kevin Laatz (3):
app/test-cmdline: support python3 only
app/test: support python3 only
buildtools: support python3 only

Louise Kilheeney (7):
usertools/dpdk-telemetry-client: support python3 only usertools/dpdk-devbind: support python3 only usertools/dpdk-pmdinfo: support python3 only usertools/cpu_layout: support python3 only
devtools: support python3 only
config/arm: support python3 only app/test-bbdev: support python3 only

 app/test-bbdev/test-bbdev.py          |  7 +------
 app/test-cmdline/cmdline_test.py      |  9 ++-------
 app/test-cmdline/cmdline_test_data.py |  1 +
 app/test/autotest.py                  |  7 +------
 app/test/autotest_data.py             |  1 +
 app/test/autotest_runner.py           | 21 ++++++++-------------
 app/test/autotest_test_funcs.py       |  1 +
 buildtools/map_to_win.py              |  3 +--
 config/arm/armv8_machine.py           |  2 +-
 devtools/update_version_map_abi.py    |  7 +------
 usertools/cpu_layout.py               | 13 ++-----------
 usertools/dpdk-devbind.py             | 22 ++++------------------
 usertools/dpdk-pmdinfo.py             |  7 +------
 usertools/dpdk-telemetry-client.py    | 20 ++++----------------
 14 files changed, 29 insertions(+), 92 deletions(-)
Signature · 1 line
2.25.1
#86 lore raw

PATCH v6 1/10 usertools/dpdk-telemetry-client: support python3 only

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Kevin Laatz <[email protected]>
---
 usertools/dpdk-telemetry-client.py | 20 ++++----------------
 1 file changed, 4 insertions(+), 16 deletions(-)
usertools/dpdk-telemetry-client.py +4 -16
index 98d28fa89b..fa599046a4 100755
@@ -1,10 +1,7 @@
-#! /usr/bin/env python
+#! /usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Intel Corporation
 
-from __future__ import print_function
-from __future__ import unicode_literals
-
 import socket
 import os
 import sys
@@ -18,15 +15,6 @@
 GLOBAL_METRICS_REQ = "{\"action\":0,\"command\":\"global_stat_values\",\"data\":null}"
 DEFAULT_FP = "/var/run/dpdk/default_client"
 
-try:
-    raw_input  # Python 2
-except NameError:
-    raw_input = input  # Python 3
-
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 class Socket:
 
     def __init__(self):
@@ -53,7 +41,7 @@ def __init__(self): # Creates a client instance
     def __del__(self):
         try:
             if self.unregistered == 0:
-                self.unregister();
+                self.unregister()
         except:
             print("Error - Client could not be destroyed")
 
@@ -86,7 +74,7 @@ def requestMetrics(self): # Requests metrics for given client
 
     def repeatedlyRequestMetrics(self, sleep_time): # Recursively requests metrics for given client
         print("\nPlease enter the number of times you'd like to continuously request Metrics:")
-        n_requests = int(raw_input("\n:"))
+        n_requests = int(input("\n:"))
         print("\033[F") #Removes the user input from screen, cleans it up
         print("\033[K")
         for i in range(n_requests):
@@ -107,7 +95,7 @@ def interactiveMenu(self, sleep_time): # Creates Interactive menu within the scr
             print("[4] Unregister client")
 
             try:
-                self.choice = int(raw_input("\n:"))
+                self.choice = int(input("\n:"))
                 print("\033[F") #Removes the user input for screen, cleans it up
                 print("\033[K")
                 if self.choice == 1:
Signature · 1 line
2.25.1
#87 lore raw

PATCH v6 2/10 usertools/dpdk-devbind: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

---
 usertools/dpdk-devbind.py | 22 ++++------------------
 1 file changed, 4 insertions(+), 18 deletions(-)
usertools/dpdk-devbind.py +4 -18
index 094c2ffc8b..8278a748d4 100755
@@ -1,9 +1,8 @@
-#! /usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 #
 
-from __future__ import print_function
 import sys
 import os
 import getopt
@@ -12,10 +11,6 @@
 from os.path import exists, abspath, dirname, basename
 from os.path import join as path_join
 
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 # The PCI base class for all devices
 network_class = {'Class': '02', 'Vendor': None, 'Device': None,
                     'SVendor': None, 'SDevice': None}
@@ -154,14 +149,6 @@ def usage():
 
     """ % locals())  # replace items from local variables
 
-
-# This is roughly compatible with check_output function in subprocess module
-# which is only available in python 2.7.
-def check_output(args, stderr=None):
-    '''Run a command and capture its output'''
-    return subprocess.Popen(args, stdout=subprocess.PIPE,
-                            stderr=stderr).communicate()[0]
-
 # check if a specific kernel module is loaded
 def module_is_loaded(module):
     global loaded_modules
@@ -218,8 +205,7 @@ def get_pci_device_details(dev_id, probe_lspci):
     device = {}
 
     if probe_lspci:
-        extra_info = check_output(["lspci", "-vmmks", dev_id]).splitlines()
-
+        extra_info = subprocess.check_output(["lspci", "-vmmks", dev_id]).splitlines()
         # parse lspci details
         for line in extra_info:
             if len(line) == 0:
@@ -255,7 +241,7 @@ def get_device_details(devices_type):
     # first loop through and read details for all devices
     # request machine readable format, with numeric IDs and String
     dev = {}
-    dev_lines = check_output(["lspci", "-Dvmmnnk"]).splitlines()
+    dev_lines = subprocess.check_output(["lspci", "-Dvmmnnk"]).splitlines()
     for dev_line in dev_lines:
         if len(dev_line) == 0:
             if device_type_match(dev, devices_type):
@@ -283,7 +269,7 @@ def get_device_details(devices_type):
         # check what is the interface if any for an ssh connection if
         # any to this host, so we can mark it later.
         ssh_if = []
-        route = check_output(["ip", "-o", "route"])
+        route = subprocess.check_output(["ip", "-o", "route"])
         # filter out all lines for 169.254 routes
         route = "\n".join(filter(lambda ln: not ln.startswith("169.254"),
                              route.decode().splitlines()))
Signature · 1 line
2.25.1
#88 lore raw

PATCH v6 3/10 usertools/dpdk-pmdinfo: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Neil Horman <[email protected]>
---
 usertools/dpdk-pmdinfo.py | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
usertools/dpdk-pmdinfo.py +1 -6
index f9ed755176..1661982791 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2016  Neil Horman <[email protected]>
 
@@ -7,8 +7,6 @@
 # Utility to dump PMD_INFO_STRING support from an object file
 #
 # -------------------------------------------------------------------------
-from __future__ import print_function
-from __future__ import unicode_literals
 import json
 import io
 import os
@@ -28,9 +26,6 @@
 pcidb = None
 
 # ===========================================
-if sys.version_info.major < 3:
-        print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-        print("Please use Python 3 instead", file=sys.stderr)
 
 class Vendor:
     """
Signature · 1 line
2.25.1
#89 lore raw

PATCH v6 4/10 usertools/cpu_layout: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

---
 usertools/cpu_layout.py | 13 ++-----------
 1 file changed, 2 insertions(+), 11 deletions(-)
usertools/cpu_layout.py +2 -11
index 5423c7965f..89a48cec46 100755
@@ -1,18 +1,9 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 # Copyright(c) 2017 Cavium, Inc. All rights reserved.
 
-from __future__ import print_function
 import sys
-try:
-    xrange # Python 2
-except NameError:
-    xrange = range # Python 3
-
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
 
 sockets = []
 cores = []
@@ -21,7 +12,7 @@
 fd = open("{}/kernel_max".format(base_path))
 max_cpus = int(fd.read())
 fd.close()
-for cpu in xrange(max_cpus + 1):
+for cpu in range(max_cpus + 1):
     try:
         fd = open("{}/cpu{}/topology/core_id".format(base_path, cpu))
     except IOError:
Signature · 1 line
2.25.1
#90 lore raw

PATCH v6 5/10 app/test-cmdline: support python3 only

In reply to Kevin Laatz

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Olivier Matz <[email protected]>

---
v5:
- fixed python3 issue causing script to fail. Divisions without the typecast lead to a floating point result, which was messing up the loop.

v6:
- Removed changes to mk/rte.sdktest.mk since it no longer exists.

---
 app/test-cmdline/cmdline_test.py      | 9 ++-------
 app/test-cmdline/cmdline_test_data.py | 1 +
 2 files changed, 3 insertions(+), 7 deletions(-)
app/test-cmdline/cmdline_test.py +2 -7
index 954428e2bf..074fed7e7f 100755
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # Script that runs cmdline_test app and feeds keystrokes into it.
-from __future__ import print_function
 import cmdline_test_data
 import os
 import pexpect
@@ -19,10 +18,6 @@ def runTest(child, test):
         return 0
     child.expect(test["Result"], 1)
 
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 #
 # history test is a special case
 #
@@ -43,7 +38,7 @@ def runHistoryTest(child):
     i = 0
 
     # fill the history with numbers
-    while i < history_size / 10:
+    while i < int(history_size / 10):
         # add 1 to prevent from parsing as octals
         child.send("1" + str(i).zfill(8) + cmdline_test_data.ENTER)
         # the app will simply print out the number
app/test-cmdline/cmdline_test_data.py +1 -0
index 114d2cb6a0..2d9b3262a6 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
Signature · 1 line
2.25.1
#91 lore raw

PATCH v6 6/10 app/test: support python3 only

In reply to Kevin Laatz

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

---
v6:
- Removed changes to mk/rte.sdktest.mk since the file no longer exists

---
 app/test/autotest.py            |  7 +------
 app/test/autotest_data.py       |  1 +
 app/test/autotest_runner.py     | 21 ++++++++-------------
 app/test/autotest_test_funcs.py |  1 +
 4 files changed, 11 insertions(+), 19 deletions(-)
app/test/autotest.py +1 -6
index cf7584ccd7..9eef1efbe5 100644
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # Script that uses either test app or qemu controlled by python-pexpect
-from __future__ import print_function
 import autotest_data
 import autotest_runner
 import sys
@@ -17,10 +16,6 @@ def usage():
     usage()
     sys.exit(1)
 
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 target = sys.argv[2]
 
 test_whitelist = None
app/test/autotest_data.py +1 -0
index 4b7da45e09..097638941f 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
app/test/autotest_runner.py +8 -13
index 95e74c760d..998fe57a55 100644
@@ -1,10 +1,10 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # The main logic behind running autotests in parallel
 
-from __future__ import print_function
-import StringIO
+import io
 import csv
 from multiprocessing import Pool, Queue
 import pexpect
@@ -50,11 +50,7 @@ def first_cpu_on_node(node_nr):
                 map(os.path.basename, cpu_path)
             )
     )
-    # for compatibility between python 3 and 2 we need to make interable out
-    # of filter return as it returns list in python 2 and a generator in 3
-    m = next(iter(cpu_name))
-    return int(m.group(1))
-
+    return int(next(cpu_name).group(1))
 
 pool_child = None  # per-process child
 
@@ -78,7 +74,7 @@ def pool_init(queue, result_queue):
     cmdline = "%s %s" % (cmdline, prefix_cmdline)
 
     # prepare logging of init
-    startuplog = StringIO.StringIO()
+    startuplog = io.StringIO()
 
     # run test app
     try:
@@ -86,8 +82,7 @@ def pool_init(queue, result_queue):
         print("\n%s %s\n" % ("=" * 20, prefix), file=startuplog)
         print("\ncmdline=%s" % cmdline, file=startuplog)
 
-        pool_child = pexpect.spawn(cmdline, logfile=startuplog)
-
+        pool_child = pexpect.spawn(cmdline, logfile=startuplog, encoding='utf-8')
         # wait for target to boot
         if not wait_prompt(pool_child):
             pool_child.close()
@@ -138,7 +133,7 @@ def run_test(target, test):
     # create log buffer for each test
     # in multiprocessing environment, the logging would be
     # interleaved and will create a mess, hence the buffering
-    logfile = StringIO.StringIO()
+    logfile = io.StringIO()
     pool_child.logfile = logfile
 
     # make a note when the test started
@@ -210,9 +205,9 @@ def __init__(self, cmdline, target, blacklist, whitelist, n_processes):
         # parse the binary for available test commands
         binary = cmdline.split()[0]
         stripped = 'not stripped' not in \
-                   subprocess.check_output(['file', binary])
+                   subprocess.check_output(['file', binary]).decode()
         if not stripped:
-            symbols = subprocess.check_output(['nm', binary]).decode('utf-8')
+            symbols = subprocess.check_output(['nm', binary]).decode()
             self.avail_cmds = re.findall('test_register_(\w+)', symbols)
         else:
             self.avail_cmds = None
app/test/autotest_test_funcs.py +1 -0
index 26688b7132..775dfd1dc5 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
Signature · 1 line
2.25.1
#92 lore raw

PATCH v6 7/10 devtools: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Neil Horman <[email protected]>
Cc Ray Kinsella <[email protected]>
---
 devtools/update_version_map_abi.py | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
devtools/update_version_map_abi.py +1 -6
index 10c3bc8098..3536a54b44 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2019 Intel Corporation
 
@@ -9,7 +9,6 @@
 from the devtools/update-abi.sh utility.
 """
 
-from __future__ import print_function
 import argparse
 import sys
 import re
@@ -160,10 +159,6 @@ def __generate_internal_abi(f_out, lines):
     print("};", file=f_out)
 
 def __main():
-    if sys.version_info.major < 3:
-        print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-        print("Please use Python 3 instead", file=sys.stderr)
-
     arg_parser = argparse.ArgumentParser(
         description='Merge versions in linker version script.')
 
Signature · 1 line
2.25.1
#93 lore raw

PATCH v6 8/10 config/arm: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2.

Cc Thomas Monjalon <[email protected]>
---
 config/arm/armv8_machine.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
config/arm/armv8_machine.py +1 -1
index 404866d2f8..1f689d9a83 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Cavium, Inc
 
Signature · 1 line
2.25.1
#94 lore raw

PATCH v6 9/10 app/test-bbdev: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Nicolas Chautru <[email protected]>
---
 app/test-bbdev/test-bbdev.py | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
app/test-bbdev/test-bbdev.py +1 -6
index 5ae2dc6c49..7d67dbe30a 100755
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Intel Corporation
 
-from __future__ import print_function
 import sys
 import os
 import argparse
@@ -16,10 +15,6 @@ def kill(process):
     print("ERROR: Test app timed out")
     process.kill()
 
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 if "RTE_SDK" in os.environ:
     dpdk_path = os.environ["RTE_SDK"]
 else:
Signature · 1 line
2.25.1
#95 lore raw

PATCH v6 10/10 buildtools: support python3 only

In reply to Kevin Laatz

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

---
 buildtools/map_to_win.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
buildtools/map_to_win.py +1 -2
index 2990b58634..2a6cb88605 100644
@@ -1,8 +1,7 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2019 Intel Corporation
 
-from __future__ import print_function
 import sys
 from os.path import dirname, basename, join, exists
 
Signature · 1 line
2.25.1
#96 lore raw

v7 cover adding support for python 3 only

In reply to Kevin Laatz

This patch set converts all python scripts in the project to use python3 only and removes all deprecation notices associated with these changes. This is due to python2 being EOL in January 2020.

---
v6:
- rebased, removing conflict with make removal patchset.
- added changes to buildtools/map_to_win.py

v7:
- typo in email Cc'ing David Marchand
- added maintainers for buildtools patch

Kevin Laatz (3):
app/test-cmdline: support python3 only
app/test: support python3 only
buildtools: support python3 only

Louise Kilheeney (7):
usertools/dpdk-telemetry-client: support python3 only usertools/dpdk-devbind: support python3 only usertools/dpdk-pmdinfo: support python3 only usertools/cpu_layout: support python3 only
devtools: support python3 only
config/arm: support python3 only app/test-bbdev: support python3 only

 app/test-bbdev/test-bbdev.py          |  7 +------
 app/test-cmdline/cmdline_test.py      |  9 ++-------
 app/test-cmdline/cmdline_test_data.py |  1 +
 app/test/autotest.py                  |  7 +------
 app/test/autotest_data.py             |  1 +
 app/test/autotest_runner.py           | 21 ++++++++-------------
 app/test/autotest_test_funcs.py       |  1 +
 buildtools/map_to_win.py              |  3 +--
 config/arm/armv8_machine.py           |  2 +-
 devtools/update_version_map_abi.py    |  7 +------
 usertools/cpu_layout.py               | 13 ++-----------
 usertools/dpdk-devbind.py             | 22 ++++------------------
 usertools/dpdk-pmdinfo.py             |  7 +------
 usertools/dpdk-telemetry-client.py    | 20 ++++----------------
 14 files changed, 29 insertions(+), 92 deletions(-)
Signature · 1 line
2.25.1
#97 lore raw

PATCH v7 1/10 usertools/dpdk-telemetry-client: support python3 only

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Kevin Laatz <[email protected]>
---
 usertools/dpdk-telemetry-client.py | 20 ++++----------------
 1 file changed, 4 insertions(+), 16 deletions(-)
usertools/dpdk-telemetry-client.py +4 -16
index 98d28fa89b..fa599046a4 100755
@@ -1,10 +1,7 @@
-#! /usr/bin/env python
+#! /usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Intel Corporation
 
-from __future__ import print_function
-from __future__ import unicode_literals
-
 import socket
 import os
 import sys
@@ -18,15 +15,6 @@
 GLOBAL_METRICS_REQ = "{\"action\":0,\"command\":\"global_stat_values\",\"data\":null}"
 DEFAULT_FP = "/var/run/dpdk/default_client"
 
-try:
-    raw_input  # Python 2
-except NameError:
-    raw_input = input  # Python 3
-
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 class Socket:
 
     def __init__(self):
@@ -53,7 +41,7 @@ def __init__(self): # Creates a client instance
     def __del__(self):
         try:
             if self.unregistered == 0:
-                self.unregister();
+                self.unregister()
         except:
             print("Error - Client could not be destroyed")
 
@@ -86,7 +74,7 @@ def requestMetrics(self): # Requests metrics for given client
 
     def repeatedlyRequestMetrics(self, sleep_time): # Recursively requests metrics for given client
         print("\nPlease enter the number of times you'd like to continuously request Metrics:")
-        n_requests = int(raw_input("\n:"))
+        n_requests = int(input("\n:"))
         print("\033[F") #Removes the user input from screen, cleans it up
         print("\033[K")
         for i in range(n_requests):
@@ -107,7 +95,7 @@ def interactiveMenu(self, sleep_time): # Creates Interactive menu within the scr
             print("[4] Unregister client")
 
             try:
-                self.choice = int(raw_input("\n:"))
+                self.choice = int(input("\n:"))
                 print("\033[F") #Removes the user input for screen, cleans it up
                 print("\033[K")
                 if self.choice == 1:
Signature · 1 line
2.25.1
#98 lore raw

Re: [dpdk-dev] [PATCH v7 01/10] usertools/dpdk-telemetry-client: support python3 only

Hi Kevin, all,

2020-09-21, Kevin Laatz:

Quoted 9 lines

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Kevin Laatz <[email protected]>

[snip]

usertools/dpdk-telemetry-client.py def __init__(self): # Creates a client instance · line 44

Quoted 2 lines
usertools/dpdk-telemetry-client.py +1 -1
-                self.unregister();
+                self.unregister()

Is this related to Python 3? If not, it may be better to make code cleanups in a separate series.

Quoted 24 lines

except:
print("Error - Client could not be destroyed")

usertools/dpdk-telemetry-client.py +2 -2
@@ -86,7 +74,7 @@ def requestMetrics(self): # Requests metrics for given client
 
     def repeatedlyRequestMetrics(self, sleep_time): # Recursively requests metrics for given client
         print("\nPlease enter the number of times you'd like to continuously request Metrics:")
-        n_requests = int(raw_input("\n:"))
+        n_requests = int(input("\n:"))
         print("\033[F") #Removes the user input from screen, cleans it up
         print("\033[K")
         for i in range(n_requests):
@@ -107,7 +95,7 @@ def interactiveMenu(self, sleep_time): # Creates Interactive menu within the scr
             print("[4] Unregister client")
 
             try:
-                self.choice = int(raw_input("\n:"))
+                self.choice = int(input("\n:"))
                 print("\033[F") #Removes the user input for screen, cleans it up
                 print("\033[K")
                 if self.choice == 1:
Signature · 1 line
2.25.1
Signature · 1 line
Robin
#99 lore raw

PATCH v7 2/10 usertools/dpdk-devbind: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

---
 usertools/dpdk-devbind.py | 22 ++++------------------
 1 file changed, 4 insertions(+), 18 deletions(-)
usertools/dpdk-devbind.py +4 -18
index 094c2ffc8b..8278a748d4 100755
@@ -1,9 +1,8 @@
-#! /usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 #
 
-from __future__ import print_function
 import sys
 import os
 import getopt
@@ -12,10 +11,6 @@
 from os.path import exists, abspath, dirname, basename
 from os.path import join as path_join
 
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 # The PCI base class for all devices
 network_class = {'Class': '02', 'Vendor': None, 'Device': None,
                     'SVendor': None, 'SDevice': None}
@@ -154,14 +149,6 @@ def usage():
 
     """ % locals())  # replace items from local variables
 
-
-# This is roughly compatible with check_output function in subprocess module
-# which is only available in python 2.7.
-def check_output(args, stderr=None):
-    '''Run a command and capture its output'''
-    return subprocess.Popen(args, stdout=subprocess.PIPE,
-                            stderr=stderr).communicate()[0]
-
 # check if a specific kernel module is loaded
 def module_is_loaded(module):
     global loaded_modules
@@ -218,8 +205,7 @@ def get_pci_device_details(dev_id, probe_lspci):
     device = {}
 
     if probe_lspci:
-        extra_info = check_output(["lspci", "-vmmks", dev_id]).splitlines()
-
+        extra_info = subprocess.check_output(["lspci", "-vmmks", dev_id]).splitlines()
         # parse lspci details
         for line in extra_info:
             if len(line) == 0:
@@ -255,7 +241,7 @@ def get_device_details(devices_type):
     # first loop through and read details for all devices
     # request machine readable format, with numeric IDs and String
     dev = {}
-    dev_lines = check_output(["lspci", "-Dvmmnnk"]).splitlines()
+    dev_lines = subprocess.check_output(["lspci", "-Dvmmnnk"]).splitlines()
     for dev_line in dev_lines:
         if len(dev_line) == 0:
             if device_type_match(dev, devices_type):
@@ -283,7 +269,7 @@ def get_device_details(devices_type):
         # check what is the interface if any for an ssh connection if
         # any to this host, so we can mark it later.
         ssh_if = []
-        route = check_output(["ip", "-o", "route"])
+        route = subprocess.check_output(["ip", "-o", "route"])
         # filter out all lines for 169.254 routes
         route = "\n".join(filter(lambda ln: not ln.startswith("169.254"),
                              route.decode().splitlines()))
Signature · 1 line
2.25.1
#101 lore raw

PATCH v7 3/10 usertools/dpdk-pmdinfo: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Neil Horman <[email protected]>
---
 usertools/dpdk-pmdinfo.py | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
usertools/dpdk-pmdinfo.py +1 -6
index f9ed755176..1661982791 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2016  Neil Horman <[email protected]>
 
@@ -7,8 +7,6 @@
 # Utility to dump PMD_INFO_STRING support from an object file
 #
 # -------------------------------------------------------------------------
-from __future__ import print_function
-from __future__ import unicode_literals
 import json
 import io
 import os
@@ -28,9 +26,6 @@
 pcidb = None
 
 # ===========================================
-if sys.version_info.major < 3:
-        print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-        print("Please use Python 3 instead", file=sys.stderr)
 
 class Vendor:
     """
Signature · 1 line
2.25.1
Acked-by on v7 3/10
#102 lore raw

Re: [dpdk-dev] [PATCH v7 03/10] usertools/dpdk-pmdinfo: support python3 only

Hi Kevin, all,

2020-09-21, Kevin Laatz:

Quoted 10 lines

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Neil Horman <[email protected]>
#103 lore raw

PATCH v7 4/10 usertools/cpu_layout: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

---
 usertools/cpu_layout.py | 13 ++-----------
 1 file changed, 2 insertions(+), 11 deletions(-)
usertools/cpu_layout.py +2 -11
index 5423c7965f..89a48cec46 100755
@@ -1,18 +1,9 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 # Copyright(c) 2017 Cavium, Inc. All rights reserved.
 
-from __future__ import print_function
 import sys
-try:
-    xrange # Python 2
-except NameError:
-    xrange = range # Python 3
-
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
 
 sockets = []
 cores = []
@@ -21,7 +12,7 @@
 fd = open("{}/kernel_max".format(base_path))
 max_cpus = int(fd.read())
 fd.close()
-for cpu in xrange(max_cpus + 1):
+for cpu in range(max_cpus + 1):
     try:
         fd = open("{}/cpu{}/topology/core_id".format(base_path, cpu))
     except IOError:
Signature · 1 line
2.25.1
#105 lore raw

PATCH v7 5/10 app/test-cmdline: support python3 only

In reply to Kevin Laatz

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Olivier Matz <[email protected]>

---
v5:
- fixed python3 issue causing script to fail. Divisions without the typecast lead to a floating point result, which was messing up the loop.

v6:
- Removed changes to mk/rte.sdktest.mk since it no longer exists.

---
 app/test-cmdline/cmdline_test.py      | 9 ++-------
 app/test-cmdline/cmdline_test_data.py | 1 +
 2 files changed, 3 insertions(+), 7 deletions(-)
app/test-cmdline/cmdline_test.py +2 -7
index 954428e2bf..074fed7e7f 100755
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # Script that runs cmdline_test app and feeds keystrokes into it.
-from __future__ import print_function
 import cmdline_test_data
 import os
 import pexpect
@@ -19,10 +18,6 @@ def runTest(child, test):
         return 0
     child.expect(test["Result"], 1)
 
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 #
 # history test is a special case
 #
@@ -43,7 +38,7 @@ def runHistoryTest(child):
     i = 0
 
     # fill the history with numbers
-    while i < history_size / 10:
+    while i < int(history_size / 10):
         # add 1 to prevent from parsing as octals
         child.send("1" + str(i).zfill(8) + cmdline_test_data.ENTER)
         # the app will simply print out the number
app/test-cmdline/cmdline_test_data.py +1 -0
index 114d2cb6a0..2d9b3262a6 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
Signature · 1 line
2.25.1
#106 lore raw

Re: [dpdk-dev] [PATCH v7 05/10] app/test-cmdline: support python3 only

Hi Kevin, all,

2020-09-21, Kevin Laatz:

Quoted 8 lines

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Olivier Matz <[email protected]>

[snip]

app/test-cmdline/cmdline_test.py def runHistoryTest(child): · line 44

Quoted 9 lines
app/test-cmdline/cmdline_test.py +1 -1
@@ -43,7 +38,7 @@ def runHistoryTest(child):
     i = 0
 
     # fill the history with numbers
-    while i < history_size / 10:
+    while i < int(history_size / 10):
         # add 1 to prevent from parsing as octals
         child.send("1" + str(i).zfill(8) + cmdline_test_data.ENTER)
         # the app will simply print out the number

It would be better to use the integer division operator: //

Signature · 1 line
Robin
#107 lore raw

PATCH v7 6/10 app/test: support python3 only

In reply to Kevin Laatz

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

---
v6:
- Removed changes to mk/rte.sdktest.mk since the file no longer exists

---
 app/test/autotest.py            |  7 +------
 app/test/autotest_data.py       |  1 +
 app/test/autotest_runner.py     | 21 ++++++++-------------
 app/test/autotest_test_funcs.py |  1 +
 4 files changed, 11 insertions(+), 19 deletions(-)
app/test/autotest.py +1 -6
index cf7584ccd7..9eef1efbe5 100644
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # Script that uses either test app or qemu controlled by python-pexpect
-from __future__ import print_function
 import autotest_data
 import autotest_runner
 import sys
@@ -17,10 +16,6 @@ def usage():
     usage()
     sys.exit(1)
 
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 target = sys.argv[2]
 
 test_whitelist = None
app/test/autotest_data.py +1 -0
index 4b7da45e09..097638941f 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
app/test/autotest_runner.py +8 -13
index 95e74c760d..998fe57a55 100644
@@ -1,10 +1,10 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # The main logic behind running autotests in parallel
 
-from __future__ import print_function
-import StringIO
+import io
 import csv
 from multiprocessing import Pool, Queue
 import pexpect
@@ -50,11 +50,7 @@ def first_cpu_on_node(node_nr):
                 map(os.path.basename, cpu_path)
             )
     )
-    # for compatibility between python 3 and 2 we need to make interable out
-    # of filter return as it returns list in python 2 and a generator in 3
-    m = next(iter(cpu_name))
-    return int(m.group(1))
-
+    return int(next(cpu_name).group(1))
 
 pool_child = None  # per-process child
 
@@ -78,7 +74,7 @@ def pool_init(queue, result_queue):
     cmdline = "%s %s" % (cmdline, prefix_cmdline)
 
     # prepare logging of init
-    startuplog = StringIO.StringIO()
+    startuplog = io.StringIO()
 
     # run test app
     try:
@@ -86,8 +82,7 @@ def pool_init(queue, result_queue):
         print("\n%s %s\n" % ("=" * 20, prefix), file=startuplog)
         print("\ncmdline=%s" % cmdline, file=startuplog)
 
-        pool_child = pexpect.spawn(cmdline, logfile=startuplog)
-
+        pool_child = pexpect.spawn(cmdline, logfile=startuplog, encoding='utf-8')
         # wait for target to boot
         if not wait_prompt(pool_child):
             pool_child.close()
@@ -138,7 +133,7 @@ def run_test(target, test):
     # create log buffer for each test
     # in multiprocessing environment, the logging would be
     # interleaved and will create a mess, hence the buffering
-    logfile = StringIO.StringIO()
+    logfile = io.StringIO()
     pool_child.logfile = logfile
 
     # make a note when the test started
@@ -210,9 +205,9 @@ def __init__(self, cmdline, target, blacklist, whitelist, n_processes):
         # parse the binary for available test commands
         binary = cmdline.split()[0]
         stripped = 'not stripped' not in \
-                   subprocess.check_output(['file', binary])
+                   subprocess.check_output(['file', binary]).decode()
         if not stripped:
-            symbols = subprocess.check_output(['nm', binary]).decode('utf-8')
+            symbols = subprocess.check_output(['nm', binary]).decode()
             self.avail_cmds = re.findall('test_register_(\w+)', symbols)
         else:
             self.avail_cmds = None
app/test/autotest_test_funcs.py +1 -0
index 26688b7132..775dfd1dc5 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
Signature · 1 line
2.25.1
#109 lore raw

PATCH v7 7/10 devtools: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Neil Horman <[email protected]>
Cc Ray Kinsella <[email protected]>
---
 devtools/update_version_map_abi.py | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
devtools/update_version_map_abi.py +1 -6
index 10c3bc8098..3536a54b44 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2019 Intel Corporation
 
@@ -9,7 +9,6 @@
 from the devtools/update-abi.sh utility.
 """
 
-from __future__ import print_function
 import argparse
 import sys
 import re
@@ -160,10 +159,6 @@ def __generate_internal_abi(f_out, lines):
     print("};", file=f_out)
 
 def __main():
-    if sys.version_info.major < 3:
-        print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-        print("Please use Python 3 instead", file=sys.stderr)
-
     arg_parser = argparse.ArgumentParser(
         description='Merge versions in linker version script.')
 
Signature · 1 line
2.25.1
Acked-by on v7 7/10
#110 lore raw

Re: [dpdk-dev] [PATCH v7 07/10] devtools: support python3 only

Hi Kevin, all,

2020-09-21, Kevin Laatz:

Quoted 11 lines

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Neil Horman <[email protected]>
Cc Ray Kinsella <[email protected]>
#111 lore raw

PATCH v7 8/10 config/arm: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2.

Cc Thomas Monjalon <[email protected]>
---
 config/arm/armv8_machine.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
config/arm/armv8_machine.py +1 -1
index 404866d2f8..1f689d9a83 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Cavium, Inc
 
Signature · 1 line
2.25.1
#113 lore raw

PATCH v7 9/10 app/test-bbdev: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Nicolas Chautru <[email protected]>
---
 app/test-bbdev/test-bbdev.py | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
app/test-bbdev/test-bbdev.py +1 -6
index 5ae2dc6c49..7d67dbe30a 100755
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Intel Corporation
 
-from __future__ import print_function
 import sys
 import os
 import argparse
@@ -16,10 +15,6 @@ def kill(process):
     print("ERROR: Test app timed out")
     process.kill()
 
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 if "RTE_SDK" in os.environ:
     dpdk_path = os.environ["RTE_SDK"]
 else:
Signature · 1 line
2.25.1
#115 lore raw

PATCH v7 10/10 buildtools: support python3 only

In reply to Kevin Laatz

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Dmitry Kozlyuk <[email protected]>
Cc Narcisa Ana Maria Vasile <[email protected]>
Cc Dmitry Malloy <[email protected]>
Cc Pallavi Kadam <[email protected]>

---
v7:
- Cc Maintainers

---
 buildtools/map_to_win.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
buildtools/map_to_win.py +1 -2
index 2990b58634..2a6cb88605 100644
@@ -1,8 +1,7 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2019 Intel Corporation
 
-from __future__ import print_function
 import sys
 from os.path import dirname, basename, join, exists
 
Signature · 1 line
2.25.1
Acked-by on v7 10/10
#116 lore raw

Re: [dpdk-dev] [PATCH v7 10/10] buildtools: support python3 only

Hi Kevin, all,

2020-09-21, Kevin Laatz:

Quoted 9 lines

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Dmitry Kozlyuk <[email protected]>
Cc Narcisa Ana Maria Vasile <[email protected]>
Cc Dmitry Malloy <[email protected]>
Cc Pallavi Kadam <[email protected]>
#117 lore raw

v8 cover adding support for python 3 only

In reply to Kevin Laatz

This patch set converts all python scripts in the project to use python3 only and removes all deprecation notices associated with these changes. This is due to python2 being EOL in January 2020.

---
v6:
- rebased, removing conflict with make removal patchset.
- added changes to buildtools/map_to_win.py

v7:
- typo in email Cc'ing David Marchand
- added maintainers for buildtools patch

v8:
- removed unrelated cleanup
- replaced integer cast with integer division operator

Kevin Laatz (3):
app/test-cmdline: support python3 only
app/test: support python3 only
buildtools: support python3 only

Louise Kilheeney (7):
usertools/dpdk-telemetry-client: support python3 only usertools/dpdk-devbind: support python3 only usertools/dpdk-pmdinfo: support python3 only usertools/cpu_layout: support python3 only
devtools: support python3 only
config/arm: support python3 only app/test-bbdev: support python3 only

 app/test-bbdev/test-bbdev.py          |  7 +------
 app/test-cmdline/cmdline_test.py      |  9 ++-------
 app/test-cmdline/cmdline_test_data.py |  1 +
 app/test/autotest.py                  |  7 +------
 app/test/autotest_data.py             |  1 +
 app/test/autotest_runner.py           | 21 ++++++++-------------
 app/test/autotest_test_funcs.py       |  1 +
 buildtools/map_to_win.py              |  3 +--
 config/arm/armv8_machine.py           |  2 +-
 devtools/update_version_map_abi.py    |  7 +------
 usertools/cpu_layout.py               | 13 ++-----------
 usertools/dpdk-devbind.py             | 22 ++++------------------
 usertools/dpdk-pmdinfo.py             |  7 +------
 usertools/dpdk-telemetry-client.py    | 18 +++---------------
 14 files changed, 28 insertions(+), 91 deletions(-)
Signature · 1 line
2.25.1
#118 lore raw

PATCH v8 1/10 usertools/dpdk-telemetry-client: support python3 only

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Kevin Laatz <[email protected]>

---
v8: removed cleanup not related to python3 only support

---
 usertools/dpdk-telemetry-client.py | 18 +++---------------
 1 file changed, 3 insertions(+), 15 deletions(-)
usertools/dpdk-telemetry-client.py +3 -15
index 98d28fa89b..d8e439027c 100755
@@ -1,10 +1,7 @@
-#! /usr/bin/env python
+#! /usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Intel Corporation
 
-from __future__ import print_function
-from __future__ import unicode_literals
-
 import socket
 import os
 import sys
@@ -18,15 +15,6 @@
 GLOBAL_METRICS_REQ = "{\"action\":0,\"command\":\"global_stat_values\",\"data\":null}"
 DEFAULT_FP = "/var/run/dpdk/default_client"
 
-try:
-    raw_input  # Python 2
-except NameError:
-    raw_input = input  # Python 3
-
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 class Socket:
 
     def __init__(self):
@@ -86,7 +74,7 @@ def requestMetrics(self): # Requests metrics for given client
 
     def repeatedlyRequestMetrics(self, sleep_time): # Recursively requests metrics for given client
         print("\nPlease enter the number of times you'd like to continuously request Metrics:")
-        n_requests = int(raw_input("\n:"))
+        n_requests = int(input("\n:"))
         print("\033[F") #Removes the user input from screen, cleans it up
         print("\033[K")
         for i in range(n_requests):
@@ -107,7 +95,7 @@ def interactiveMenu(self, sleep_time): # Creates Interactive menu within the scr
             print("[4] Unregister client")
 
             try:
-                self.choice = int(raw_input("\n:"))
+                self.choice = int(input("\n:"))
                 print("\033[F") #Removes the user input for screen, cleans it up
                 print("\033[K")
                 if self.choice == 1:
Signature · 1 line
2.25.1
Acked-by on v8 1/10
#119 lore raw

Re: [dpdk-dev] [PATCH v8 01/10] usertools/dpdk-telemetry-client: support python3 only

Hi Kevin, all,

2020-09-28, Kevin Laatz:

Quoted 10 lines

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Kevin Laatz <[email protected]>
#120 lore raw

PATCH v8 2/10 usertools/dpdk-devbind: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

---
 usertools/dpdk-devbind.py | 22 ++++------------------
 1 file changed, 4 insertions(+), 18 deletions(-)
usertools/dpdk-devbind.py +4 -18
index 094c2ffc8b..8278a748d4 100755
@@ -1,9 +1,8 @@
-#! /usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 #
 
-from __future__ import print_function
 import sys
 import os
 import getopt
@@ -12,10 +11,6 @@
 from os.path import exists, abspath, dirname, basename
 from os.path import join as path_join
 
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 # The PCI base class for all devices
 network_class = {'Class': '02', 'Vendor': None, 'Device': None,
                     'SVendor': None, 'SDevice': None}
@@ -154,14 +149,6 @@ def usage():
 
     """ % locals())  # replace items from local variables
 
-
-# This is roughly compatible with check_output function in subprocess module
-# which is only available in python 2.7.
-def check_output(args, stderr=None):
-    '''Run a command and capture its output'''
-    return subprocess.Popen(args, stdout=subprocess.PIPE,
-                            stderr=stderr).communicate()[0]
-
 # check if a specific kernel module is loaded
 def module_is_loaded(module):
     global loaded_modules
@@ -218,8 +205,7 @@ def get_pci_device_details(dev_id, probe_lspci):
     device = {}
 
     if probe_lspci:
-        extra_info = check_output(["lspci", "-vmmks", dev_id]).splitlines()
-
+        extra_info = subprocess.check_output(["lspci", "-vmmks", dev_id]).splitlines()
         # parse lspci details
         for line in extra_info:
             if len(line) == 0:
@@ -255,7 +241,7 @@ def get_device_details(devices_type):
     # first loop through and read details for all devices
     # request machine readable format, with numeric IDs and String
     dev = {}
-    dev_lines = check_output(["lspci", "-Dvmmnnk"]).splitlines()
+    dev_lines = subprocess.check_output(["lspci", "-Dvmmnnk"]).splitlines()
     for dev_line in dev_lines:
         if len(dev_line) == 0:
             if device_type_match(dev, devices_type):
@@ -283,7 +269,7 @@ def get_device_details(devices_type):
         # check what is the interface if any for an ssh connection if
         # any to this host, so we can mark it later.
         ssh_if = []
-        route = check_output(["ip", "-o", "route"])
+        route = subprocess.check_output(["ip", "-o", "route"])
         # filter out all lines for 169.254 routes
         route = "\n".join(filter(lambda ln: not ln.startswith("169.254"),
                              route.decode().splitlines()))
Signature · 1 line
2.25.1
#122 lore raw

PATCH v8 3/10 usertools/dpdk-pmdinfo: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Neil Horman <[email protected]>
---
 usertools/dpdk-pmdinfo.py | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
usertools/dpdk-pmdinfo.py +1 -6
index f9ed755176..1661982791 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2016  Neil Horman <[email protected]>
 
@@ -7,8 +7,6 @@
 # Utility to dump PMD_INFO_STRING support from an object file
 #
 # -------------------------------------------------------------------------
-from __future__ import print_function
-from __future__ import unicode_literals
 import json
 import io
 import os
@@ -28,9 +26,6 @@
 pcidb = None
 
 # ===========================================
-if sys.version_info.major < 3:
-        print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-        print("Please use Python 3 instead", file=sys.stderr)
 
 class Vendor:
     """
Signature · 1 line
2.25.1
Acked-by on v8 3/10
#123 lore raw

Re: [dpdk-dev] [PATCH v8 03/10] usertools/dpdk-pmdinfo: support python3 only

Hi Kevin, all,

2020-09-28, Kevin Laatz:

Quoted 10 lines

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Neil Horman <[email protected]>
#124 lore raw

PATCH v8 4/10 usertools/cpu_layout: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

---
 usertools/cpu_layout.py | 13 ++-----------
 1 file changed, 2 insertions(+), 11 deletions(-)
usertools/cpu_layout.py +2 -11
index 5423c7965f..89a48cec46 100755
@@ -1,18 +1,9 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 # Copyright(c) 2017 Cavium, Inc. All rights reserved.
 
-from __future__ import print_function
 import sys
-try:
-    xrange # Python 2
-except NameError:
-    xrange = range # Python 3
-
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
 
 sockets = []
 cores = []
@@ -21,7 +12,7 @@
 fd = open("{}/kernel_max".format(base_path))
 max_cpus = int(fd.read())
 fd.close()
-for cpu in xrange(max_cpus + 1):
+for cpu in range(max_cpus + 1):
     try:
         fd = open("{}/cpu{}/topology/core_id".format(base_path, cpu))
     except IOError:
Signature · 1 line
2.25.1
#126 lore raw

PATCH v8 5/10 app/test-cmdline: support python3 only

In reply to Kevin Laatz

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Olivier Matz <[email protected]>

---
v5:
- fixed python3 issue causing script to fail. Divisions without the typecast lead to a floating point result, which was messing up the loop.

v6:
- Removed changes to mk/rte.sdktest.mk since it no longer exists.

v8:
- Replaced integer cast with integer division operator.

---
 app/test-cmdline/cmdline_test.py      | 9 ++-------
 app/test-cmdline/cmdline_test_data.py | 1 +
 2 files changed, 3 insertions(+), 7 deletions(-)
app/test-cmdline/cmdline_test.py +2 -7
index 954428e2bf..f3377313e9 100755
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # Script that runs cmdline_test app and feeds keystrokes into it.
-from __future__ import print_function
 import cmdline_test_data
 import os
 import pexpect
@@ -19,10 +18,6 @@ def runTest(child, test):
         return 0
     child.expect(test["Result"], 1)
 
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 #
 # history test is a special case
 #
@@ -43,7 +38,7 @@ def runHistoryTest(child):
     i = 0
 
     # fill the history with numbers
-    while i < history_size / 10:
+    while i < history_size // 10:
         # add 1 to prevent from parsing as octals
         child.send("1" + str(i).zfill(8) + cmdline_test_data.ENTER)
         # the app will simply print out the number
app/test-cmdline/cmdline_test_data.py +1 -0
index 114d2cb6a0..2d9b3262a6 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
Signature · 1 line
2.25.1
Acked-by on v8 5/10
#127 lore raw

Re: [dpdk-dev] [PATCH v8 05/10] app/test-cmdline: support python3 only

Hi Kevin, all,

2020-09-28, Kevin Laatz:

Quoted 15 lines

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Olivier Matz <[email protected]>

---
v5:
- fixed python3 issue causing script to fail. Divisions without the typecast lead to a floating point result, which was messing up the loop.

Quoted 5 lines

v6:
- Removed changes to mk/rte.sdktest.mk since it no longer exists.

v8:
- Replaced integer cast with integer division operator.

#128 lore raw

PATCH v8 6/10 app/test: support python3 only

In reply to Kevin Laatz

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

---
v6:
- Removed changes to mk/rte.sdktest.mk since the file no longer exists

---
 app/test/autotest.py            |  7 +------
 app/test/autotest_data.py       |  1 +
 app/test/autotest_runner.py     | 21 ++++++++-------------
 app/test/autotest_test_funcs.py |  1 +
 4 files changed, 11 insertions(+), 19 deletions(-)
app/test/autotest.py +1 -6
index cf7584ccd7..9eef1efbe5 100644
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # Script that uses either test app or qemu controlled by python-pexpect
-from __future__ import print_function
 import autotest_data
 import autotest_runner
 import sys
@@ -17,10 +16,6 @@ def usage():
     usage()
     sys.exit(1)
 
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 target = sys.argv[2]
 
 test_whitelist = None
app/test/autotest_data.py +1 -0
index 4b7da45e09..097638941f 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
app/test/autotest_runner.py +8 -13
index 95e74c760d..998fe57a55 100644
@@ -1,10 +1,10 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # The main logic behind running autotests in parallel
 
-from __future__ import print_function
-import StringIO
+import io
 import csv
 from multiprocessing import Pool, Queue
 import pexpect
@@ -50,11 +50,7 @@ def first_cpu_on_node(node_nr):
                 map(os.path.basename, cpu_path)
             )
     )
-    # for compatibility between python 3 and 2 we need to make interable out
-    # of filter return as it returns list in python 2 and a generator in 3
-    m = next(iter(cpu_name))
-    return int(m.group(1))
-
+    return int(next(cpu_name).group(1))
 
 pool_child = None  # per-process child
 
@@ -78,7 +74,7 @@ def pool_init(queue, result_queue):
     cmdline = "%s %s" % (cmdline, prefix_cmdline)
 
     # prepare logging of init
-    startuplog = StringIO.StringIO()
+    startuplog = io.StringIO()
 
     # run test app
     try:
@@ -86,8 +82,7 @@ def pool_init(queue, result_queue):
         print("\n%s %s\n" % ("=" * 20, prefix), file=startuplog)
         print("\ncmdline=%s" % cmdline, file=startuplog)
 
-        pool_child = pexpect.spawn(cmdline, logfile=startuplog)
-
+        pool_child = pexpect.spawn(cmdline, logfile=startuplog, encoding='utf-8')
         # wait for target to boot
         if not wait_prompt(pool_child):
             pool_child.close()
@@ -138,7 +133,7 @@ def run_test(target, test):
     # create log buffer for each test
     # in multiprocessing environment, the logging would be
     # interleaved and will create a mess, hence the buffering
-    logfile = StringIO.StringIO()
+    logfile = io.StringIO()
     pool_child.logfile = logfile
 
     # make a note when the test started
@@ -210,9 +205,9 @@ def __init__(self, cmdline, target, blacklist, whitelist, n_processes):
         # parse the binary for available test commands
         binary = cmdline.split()[0]
         stripped = 'not stripped' not in \
-                   subprocess.check_output(['file', binary])
+                   subprocess.check_output(['file', binary]).decode()
         if not stripped:
-            symbols = subprocess.check_output(['nm', binary]).decode('utf-8')
+            symbols = subprocess.check_output(['nm', binary]).decode()
             self.avail_cmds = re.findall('test_register_(\w+)', symbols)
         else:
             self.avail_cmds = None
app/test/autotest_test_funcs.py +1 -0
index 26688b7132..775dfd1dc5 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
Signature · 1 line
2.25.1
#129 lore raw

PATCH v8 7/10 devtools: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Neil Horman <[email protected]>
Cc Ray Kinsella <[email protected]>
---
 devtools/update_version_map_abi.py | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
devtools/update_version_map_abi.py +1 -6
index 10c3bc8098..3536a54b44 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2019 Intel Corporation
 
@@ -9,7 +9,6 @@
 from the devtools/update-abi.sh utility.
 """
 
-from __future__ import print_function
 import argparse
 import sys
 import re
@@ -160,10 +159,6 @@ def __generate_internal_abi(f_out, lines):
     print("};", file=f_out)
 
 def __main():
-    if sys.version_info.major < 3:
-        print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-        print("Please use Python 3 instead", file=sys.stderr)
-
     arg_parser = argparse.ArgumentParser(
         description='Merge versions in linker version script.')
 
Signature · 1 line
2.25.1
#130 lore raw

PATCH v8 8/10 config/arm: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2.

Cc Thomas Monjalon <[email protected]>
---
 config/arm/armv8_machine.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
config/arm/armv8_machine.py +1 -1
index 404866d2f8..1f689d9a83 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Cavium, Inc
 
Signature · 1 line
2.25.1
#131 lore raw

PATCH v8 9/10 app/test-bbdev: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Nicolas Chautru <[email protected]>
---
 app/test-bbdev/test-bbdev.py | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
app/test-bbdev/test-bbdev.py +1 -6
index 5ae2dc6c49..7d67dbe30a 100755
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Intel Corporation
 
-from __future__ import print_function
 import sys
 import os
 import argparse
@@ -16,10 +15,6 @@ def kill(process):
     print("ERROR: Test app timed out")
     process.kill()
 
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 if "RTE_SDK" in os.environ:
     dpdk_path = os.environ["RTE_SDK"]
 else:
Signature · 1 line
2.25.1
#132 lore raw

PATCH v8 10/10 buildtools: support python3 only

In reply to Kevin Laatz

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Dmitry Kozlyuk <[email protected]>
Cc Narcisa Ana Maria Vasile <[email protected]>
Cc Dmitry Malloy <[email protected]>
Cc Pallavi Kadam <[email protected]>

---
v7:
- Cc Maintainers

---
 buildtools/map_to_win.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
buildtools/map_to_win.py +1 -2
index 2990b58634..2a6cb88605 100644
@@ -1,8 +1,7 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2019 Intel Corporation
 
-from __future__ import print_function
 import sys
 from os.path import dirname, basename, join, exists
 
Signature · 1 line
2.25.1
Acked-by on v8 10/10
#133 lore raw

Re: [dpdk-dev] [PATCH v8 10/10] buildtools: support python3 only

Hi Kevin, all,

2020-09-28, Kevin Laatz:

Quoted 9 lines

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Dmitry Kozlyuk <[email protected]>
Cc Narcisa Ana Maria Vasile <[email protected]>
Cc Dmitry Malloy <[email protected]>
Cc Pallavi Kadam <[email protected]>
Acked-by on the v8 series
#134 lore raw

Re: [dpdk-dev] [PATCH v8 00/10] adding support for python 3 only

In reply to Kevin Laatz

Hi Kevin, all,

2020-09-28, Kevin Laatz:

Quoted 9 lines

This patch set converts all python scripts in the project to use python3 only and removes all deprecation notices associated with these changes. This is due to python2 being EOL in January 2020.

---
v6:
- rebased, removing conflict with make removal patchset.
- added changes to buildtools/map_to_win.py

Quoted 7 lines

v7:
- typo in email Cc'ing David Marchand
- added maintainers for buildtools patch

v8:
- removed unrelated cleanup
- replaced integer cast with integer division operator

For all patches in the series:

#135 lore raw

Re: [dpdk-dev] [PATCH v8 00/10] adding support for python 3 only

Quoted 16 lines · Robin Jarry

Hi Kevin, all,

2020-09-28, Kevin Laatz:

Quoted 13 lines

This patch set converts all python scripts in the project to use python3 only and removes all deprecation notices associated with these changes. This is due to python2 being EOL in January 2020.

---
v6:
- rebased, removing conflict with make removal patchset.
- added changes to buildtools/map_to_win.py

v7:
- typo in email Cc'ing David Marchand
- added maintainers for buildtools patch

Quoted 7 lines
Quoted 3 lines

v8:
- removed unrelated cleanup
- replaced integer cast with integer division operator

For all patches in the series:

Great. Looking forward to having this merged as I'm tired of seeing the warnings from our scripts when I forget to explicitly call them using python3!

I expect, once applied, the deprecation notice should also be removed.

#136 lore raw

Re: [dpdk-dev] [PATCH v8 00/10] adding support for python 3 only

In reply to Kevin Laatz

Hello Kevin, Louise,

Quoted 4 lines · Kevin Laatz

This patch set converts all python scripts in the project to use python3 only and removes all deprecation notices associated with these changes. This is due to python2 being EOL in January 2020.

Thanks for working on this.

I noticed some leftovers:
- doc/guides/contributing/coding_style.rst and doc/guides/linux_gsg/sys_reqs.rst still mention that scripts work with python2,
- the deprecation notice can be dropped and this will go with a corresponding update of the release note,

Signature · 1 line
David Marchand
#137 lore raw

Re: [dpdk-dev] [PATCH v8 00/10] adding support for python 3 only

Quoted 8 lines · David Marchand

Hello Kevin, Louise,

Quoted 4 lines · Kevin Laatz

This patch set converts all python scripts in the project to use python3 only and removes all deprecation notices associated with these changes. This is due to python2 being EOL in January 2020.

Quoted 6 lines

Thanks for working on this.

I noticed some leftovers:
- doc/guides/contributing/coding_style.rst and doc/guides/linux_gsg/sys_reqs.rst still mention that scripts work with python2,

For the GSG doc, there is an overlap with the make removal doc updates which already fix that (since meson itself requires py3):

http://patches.dpdk.org/patch/78191/

Good catch on the former.

Quoted 7 lines

- the deprecation notice can be dropped and this will go with a corresponding update of the release note,

Signature · 1 line
David Marchand
#138 lore raw

Re: [dpdk-dev] [PATCH v8 00/10] adding support for python 3 only

Quoted 13 lines · Bruce Richardson
Quoted 6 lines · David Marchand

Hello Kevin, Louise,

Quoted 3 lines · Kevin Laatz

This patch set converts all python scripts in the project to use python3 only and removes all deprecation notices associated with these changes. This is due to python2 being EOL in January 2020.

Quoted 6 lines

Thanks for working on this.

I noticed some leftovers:
- doc/guides/contributing/coding_style.rst and doc/guides/linux_gsg/sys_reqs.rst still mention that scripts work with python2,

Quoted 6 lines

For the GSG doc, there is an overlap with the make removal doc updates which already fix that (since meson itself requires py3):

http://patches.dpdk.org/patch/78191/

Good catch on the former.

Just noticed doc/guides/conf.py was also missed... Will work on this and the above for v9

Quoted 2 lines
Quoted 2 lines

- the deprecation notice can be dropped and this will go with a corresponding update of the release note,

Will take care of this in the v9 as well. Thanks.

Quoted 3 lines
Quoted 3 lines
Signature · 1 line
David Marchand
#139 lore raw

v9 cover adding support for python 3 only

In reply to Kevin Laatz

This patch set converts all python scripts in the project to use python3 only and removes all deprecation notices associated with these changes. This is due to python2 being EOL in January 2020.

---
v6:
- rebased, removing conflict with make removal patchset.
- added changes to buildtools/map_to_win.py

v7:
- typo in email Cc'ing David Marchand
- added maintainers for buildtools patch

v8:
- removed unrelated cleanup
- replaced integer cast with integer division operator

v9:
- include documentation updates

Kevin Laatz (4):
app/test-cmdline: support python3 only
app/test: support python3 only
buildtools: support python3 only
doc: support python3 only

Louise Kilheeney (7):
usertools/dpdk-telemetry-client: support python3 only usertools/dpdk-devbind: support python3 only usertools/dpdk-pmdinfo: support python3 only usertools/cpu_layout: support python3 only
devtools: support python3 only
config/arm: support python3 only app/test-bbdev: support python3 only

 app/test-bbdev/test-bbdev.py             |  7 +------
 app/test-cmdline/cmdline_test.py         |  9 ++-------
 app/test-cmdline/cmdline_test_data.py    |  1 +
 app/test/autotest.py                     |  7 +------
 app/test/autotest_data.py                |  1 +
 app/test/autotest_runner.py              | 21 ++++++++-------------
 app/test/autotest_test_funcs.py          |  1 +
 buildtools/map_to_win.py                 |  3 +--
 config/arm/armv8_machine.py              |  2 +-
 devtools/update_version_map_abi.py       |  7 +------
 doc/guides/contributing/coding_style.rst |  2 +-
 doc/guides/linux_gsg/sys_reqs.rst        |  2 +-
 doc/guides/rel_notes/deprecation.rst     |  6 ------
 doc/guides/rel_notes/release_20_11.rst   |  2 ++
 usertools/cpu_layout.py                  | 13 ++-----------
 usertools/dpdk-devbind.py                | 22 ++++------------------
 usertools/dpdk-pmdinfo.py                |  7 +------
 usertools/dpdk-telemetry-client.py       | 18 +++---------------
 18 files changed, 32 insertions(+), 99 deletions(-)
Signature · 1 line
2.25.1
#140 lore raw

PATCH v9 1/11 usertools/dpdk-telemetry-client: support python3 only

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Kevin Laatz <[email protected]>

---
v8: removed cleanup not related to python3 only support

---
 usertools/dpdk-telemetry-client.py | 18 +++---------------
 1 file changed, 3 insertions(+), 15 deletions(-)
usertools/dpdk-telemetry-client.py +3 -15
index 98d28fa89b..d8e439027c 100755
@@ -1,10 +1,7 @@
-#! /usr/bin/env python
+#! /usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Intel Corporation
 
-from __future__ import print_function
-from __future__ import unicode_literals
-
 import socket
 import os
 import sys
@@ -18,15 +15,6 @@
 GLOBAL_METRICS_REQ = "{\"action\":0,\"command\":\"global_stat_values\",\"data\":null}"
 DEFAULT_FP = "/var/run/dpdk/default_client"
 
-try:
-    raw_input  # Python 2
-except NameError:
-    raw_input = input  # Python 3
-
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 class Socket:
 
     def __init__(self):
@@ -86,7 +74,7 @@ def requestMetrics(self): # Requests metrics for given client
 
     def repeatedlyRequestMetrics(self, sleep_time): # Recursively requests metrics for given client
         print("\nPlease enter the number of times you'd like to continuously request Metrics:")
-        n_requests = int(raw_input("\n:"))
+        n_requests = int(input("\n:"))
         print("\033[F") #Removes the user input from screen, cleans it up
         print("\033[K")
         for i in range(n_requests):
@@ -107,7 +95,7 @@ def interactiveMenu(self, sleep_time): # Creates Interactive menu within the scr
             print("[4] Unregister client")
 
             try:
-                self.choice = int(raw_input("\n:"))
+                self.choice = int(input("\n:"))
                 print("\033[F") #Removes the user input for screen, cleans it up
                 print("\033[K")
                 if self.choice == 1:
Signature · 1 line
2.25.1
#141 lore raw

PATCH v9 2/11 usertools/dpdk-devbind: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

---
 usertools/dpdk-devbind.py | 22 ++++------------------
 1 file changed, 4 insertions(+), 18 deletions(-)
usertools/dpdk-devbind.py +4 -18
index 094c2ffc8b..8278a748d4 100755
@@ -1,9 +1,8 @@
-#! /usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 #
 
-from __future__ import print_function
 import sys
 import os
 import getopt
@@ -12,10 +11,6 @@
 from os.path import exists, abspath, dirname, basename
 from os.path import join as path_join
 
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 # The PCI base class for all devices
 network_class = {'Class': '02', 'Vendor': None, 'Device': None,
                     'SVendor': None, 'SDevice': None}
@@ -154,14 +149,6 @@ def usage():
 
     """ % locals())  # replace items from local variables
 
-
-# This is roughly compatible with check_output function in subprocess module
-# which is only available in python 2.7.
-def check_output(args, stderr=None):
-    '''Run a command and capture its output'''
-    return subprocess.Popen(args, stdout=subprocess.PIPE,
-                            stderr=stderr).communicate()[0]
-
 # check if a specific kernel module is loaded
 def module_is_loaded(module):
     global loaded_modules
@@ -218,8 +205,7 @@ def get_pci_device_details(dev_id, probe_lspci):
     device = {}
 
     if probe_lspci:
-        extra_info = check_output(["lspci", "-vmmks", dev_id]).splitlines()
-
+        extra_info = subprocess.check_output(["lspci", "-vmmks", dev_id]).splitlines()
         # parse lspci details
         for line in extra_info:
             if len(line) == 0:
@@ -255,7 +241,7 @@ def get_device_details(devices_type):
     # first loop through and read details for all devices
     # request machine readable format, with numeric IDs and String
     dev = {}
-    dev_lines = check_output(["lspci", "-Dvmmnnk"]).splitlines()
+    dev_lines = subprocess.check_output(["lspci", "-Dvmmnnk"]).splitlines()
     for dev_line in dev_lines:
         if len(dev_line) == 0:
             if device_type_match(dev, devices_type):
@@ -283,7 +269,7 @@ def get_device_details(devices_type):
         # check what is the interface if any for an ssh connection if
         # any to this host, so we can mark it later.
         ssh_if = []
-        route = check_output(["ip", "-o", "route"])
+        route = subprocess.check_output(["ip", "-o", "route"])
         # filter out all lines for 169.254 routes
         route = "\n".join(filter(lambda ln: not ln.startswith("169.254"),
                              route.decode().splitlines()))
Signature · 1 line
2.25.1
#142 lore raw

PATCH v9 3/11 usertools/dpdk-pmdinfo: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Neil Horman <[email protected]>
---
 usertools/dpdk-pmdinfo.py | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
usertools/dpdk-pmdinfo.py +1 -6
index f9ed755176..1661982791 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2016  Neil Horman <[email protected]>
 
@@ -7,8 +7,6 @@
 # Utility to dump PMD_INFO_STRING support from an object file
 #
 # -------------------------------------------------------------------------
-from __future__ import print_function
-from __future__ import unicode_literals
 import json
 import io
 import os
@@ -28,9 +26,6 @@
 pcidb = None
 
 # ===========================================
-if sys.version_info.major < 3:
-        print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-        print("Please use Python 3 instead", file=sys.stderr)
 
 class Vendor:
     """
Signature · 1 line
2.25.1
#143 lore raw

PATCH v9 4/11 usertools/cpu_layout: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

---
 usertools/cpu_layout.py | 13 ++-----------
 1 file changed, 2 insertions(+), 11 deletions(-)
usertools/cpu_layout.py +2 -11
index 5423c7965f..89a48cec46 100755
@@ -1,18 +1,9 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 # Copyright(c) 2017 Cavium, Inc. All rights reserved.
 
-from __future__ import print_function
 import sys
-try:
-    xrange # Python 2
-except NameError:
-    xrange = range # Python 3
-
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
 
 sockets = []
 cores = []
@@ -21,7 +12,7 @@
 fd = open("{}/kernel_max".format(base_path))
 max_cpus = int(fd.read())
 fd.close()
-for cpu in xrange(max_cpus + 1):
+for cpu in range(max_cpus + 1):
     try:
         fd = open("{}/cpu{}/topology/core_id".format(base_path, cpu))
     except IOError:
Signature · 1 line
2.25.1
#144 lore raw

PATCH v9 5/11 app/test-cmdline: support python3 only

In reply to Kevin Laatz

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Olivier Matz <[email protected]>

---
v5:
- fixed python3 issue causing script to fail. Divisions without the typecast lead to a floating point result, which was messing up the loop.

v6:
- Removed changes to mk/rte.sdktest.mk since it no longer exists.

v8:
- Replaced integer cast with integer division operator.

---
 app/test-cmdline/cmdline_test.py      | 9 ++-------
 app/test-cmdline/cmdline_test_data.py | 1 +
 2 files changed, 3 insertions(+), 7 deletions(-)
app/test-cmdline/cmdline_test.py +2 -7
index 954428e2bf..f3377313e9 100755
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # Script that runs cmdline_test app and feeds keystrokes into it.
-from __future__ import print_function
 import cmdline_test_data
 import os
 import pexpect
@@ -19,10 +18,6 @@ def runTest(child, test):
         return 0
     child.expect(test["Result"], 1)
 
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 #
 # history test is a special case
 #
@@ -43,7 +38,7 @@ def runHistoryTest(child):
     i = 0
 
     # fill the history with numbers
-    while i < history_size / 10:
+    while i < history_size // 10:
         # add 1 to prevent from parsing as octals
         child.send("1" + str(i).zfill(8) + cmdline_test_data.ENTER)
         # the app will simply print out the number
app/test-cmdline/cmdline_test_data.py +1 -0
index 114d2cb6a0..2d9b3262a6 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
Signature · 1 line
2.25.1
#145 lore raw

PATCH v9 6/11 app/test: support python3 only

In reply to Kevin Laatz

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

---
v6:
- Removed changes to mk/rte.sdktest.mk since the file no longer exists

---
 app/test/autotest.py            |  7 +------
 app/test/autotest_data.py       |  1 +
 app/test/autotest_runner.py     | 21 ++++++++-------------
 app/test/autotest_test_funcs.py |  1 +
 4 files changed, 11 insertions(+), 19 deletions(-)
app/test/autotest.py +1 -6
index cf7584ccd7..9eef1efbe5 100644
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # Script that uses either test app or qemu controlled by python-pexpect
-from __future__ import print_function
 import autotest_data
 import autotest_runner
 import sys
@@ -17,10 +16,6 @@ def usage():
     usage()
     sys.exit(1)
 
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 target = sys.argv[2]
 
 test_whitelist = None
app/test/autotest_data.py +1 -0
index 4b7da45e09..097638941f 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
app/test/autotest_runner.py +8 -13
index 95e74c760d..998fe57a55 100644
@@ -1,10 +1,10 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # The main logic behind running autotests in parallel
 
-from __future__ import print_function
-import StringIO
+import io
 import csv
 from multiprocessing import Pool, Queue
 import pexpect
@@ -50,11 +50,7 @@ def first_cpu_on_node(node_nr):
                 map(os.path.basename, cpu_path)
             )
     )
-    # for compatibility between python 3 and 2 we need to make interable out
-    # of filter return as it returns list in python 2 and a generator in 3
-    m = next(iter(cpu_name))
-    return int(m.group(1))
-
+    return int(next(cpu_name).group(1))
 
 pool_child = None  # per-process child
 
@@ -78,7 +74,7 @@ def pool_init(queue, result_queue):
     cmdline = "%s %s" % (cmdline, prefix_cmdline)
 
     # prepare logging of init
-    startuplog = StringIO.StringIO()
+    startuplog = io.StringIO()
 
     # run test app
     try:
@@ -86,8 +82,7 @@ def pool_init(queue, result_queue):
         print("\n%s %s\n" % ("=" * 20, prefix), file=startuplog)
         print("\ncmdline=%s" % cmdline, file=startuplog)
 
-        pool_child = pexpect.spawn(cmdline, logfile=startuplog)
-
+        pool_child = pexpect.spawn(cmdline, logfile=startuplog, encoding='utf-8')
         # wait for target to boot
         if not wait_prompt(pool_child):
             pool_child.close()
@@ -138,7 +133,7 @@ def run_test(target, test):
     # create log buffer for each test
     # in multiprocessing environment, the logging would be
     # interleaved and will create a mess, hence the buffering
-    logfile = StringIO.StringIO()
+    logfile = io.StringIO()
     pool_child.logfile = logfile
 
     # make a note when the test started
@@ -210,9 +205,9 @@ def __init__(self, cmdline, target, blacklist, whitelist, n_processes):
         # parse the binary for available test commands
         binary = cmdline.split()[0]
         stripped = 'not stripped' not in \
-                   subprocess.check_output(['file', binary])
+                   subprocess.check_output(['file', binary]).decode()
         if not stripped:
-            symbols = subprocess.check_output(['nm', binary]).decode('utf-8')
+            symbols = subprocess.check_output(['nm', binary]).decode()
             self.avail_cmds = re.findall('test_register_(\w+)', symbols)
         else:
             self.avail_cmds = None
app/test/autotest_test_funcs.py +1 -0
index 26688b7132..775dfd1dc5 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
Signature · 1 line
2.25.1
#146 lore raw

PATCH v9 7/11 devtools: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Neil Horman <[email protected]>
Cc Ray Kinsella <[email protected]>
---
 devtools/update_version_map_abi.py | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
devtools/update_version_map_abi.py +1 -6
index 10c3bc8098..3536a54b44 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2019 Intel Corporation
 
@@ -9,7 +9,6 @@
 from the devtools/update-abi.sh utility.
 """
 
-from __future__ import print_function
 import argparse
 import sys
 import re
@@ -160,10 +159,6 @@ def __generate_internal_abi(f_out, lines):
     print("};", file=f_out)
 
 def __main():
-    if sys.version_info.major < 3:
-        print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-        print("Please use Python 3 instead", file=sys.stderr)
-
     arg_parser = argparse.ArgumentParser(
         description='Merge versions in linker version script.')
 
Signature · 1 line
2.25.1
#147 lore raw

PATCH v9 8/11 config/arm: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2.

Cc Thomas Monjalon <[email protected]>
---
 config/arm/armv8_machine.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
config/arm/armv8_machine.py +1 -1
index 404866d2f8..1f689d9a83 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Cavium, Inc
 
Signature · 1 line
2.25.1
#148 lore raw

PATCH v9 9/11 app/test-bbdev: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Nicolas Chautru <[email protected]>
---
 app/test-bbdev/test-bbdev.py | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
app/test-bbdev/test-bbdev.py +1 -6
index 5ae2dc6c49..7d67dbe30a 100755
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Intel Corporation
 
-from __future__ import print_function
 import sys
 import os
 import argparse
@@ -16,10 +15,6 @@ def kill(process):
     print("ERROR: Test app timed out")
     process.kill()
 
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 if "RTE_SDK" in os.environ:
     dpdk_path = os.environ["RTE_SDK"]
 else:
Signature · 1 line
2.25.1
#149 lore raw

PATCH v9 10/11 buildtools: support python3 only

In reply to Kevin Laatz

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Dmitry Kozlyuk <[email protected]>
Cc Narcisa Ana Maria Vasile <[email protected]>
Cc Dmitry Malloy <[email protected]>
Cc Pallavi Kadam <[email protected]>

---
v7:
- Cc Maintainers

---
 buildtools/map_to_win.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
buildtools/map_to_win.py +1 -2
index 2990b58634..2a6cb88605 100644
@@ -1,8 +1,7 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2019 Intel Corporation
 
-from __future__ import print_function
 import sys
 from os.path import dirname, basename, join, exists
 
Signature · 1 line
2.25.1
#150 lore raw

PATCH v9 11/11 doc: support python3 only

In reply to Kevin Laatz

Remove references to python 2 from the documentation.

The deprecation notice for python 2 support has been removed. A section was added to the rel_notes for this patchset.

Cc Ray Kinsella <[email protected]>
Cc Neil Horman <[email protected]>
Cc John McNamara <[email protected]>
Cc Marko Kovacevic <[email protected]>
---
 doc/guides/contributing/coding_style.rst | 2 +-
 doc/guides/linux_gsg/sys_reqs.rst        | 2 +-
 doc/guides/rel_notes/deprecation.rst     | 6 ------
 doc/guides/rel_notes/release_20_11.rst   | 2 ++
 4 files changed, 4 insertions(+), 8 deletions(-)
doc/guides/contributing/coding_style.rst +1 -1
index 0be9546a6a..4fc010ca49 100644
@@ -765,7 +765,7 @@ specializations, run the ``app/test`` binary, and use the ``dump_log_types``
 Python Code
 -----------
 
-All Python code should work with Python 2.7+ and 3.2+ and be compliant with
+All Python code should work with 3.2+ and be compliant with
 `PEP8 (Style Guide for Python Code) <https://www.python.org/dev/peps/pep-0008/>`_.
 
 The ``pep8`` tool can be used for testing compliance with the guidelines.
doc/guides/linux_gsg/sys_reqs.rst +1 -1
index 6b9f573fce..66d9723dd6 100644
@@ -47,7 +47,7 @@ Compilation of the DPDK
 
     * Python v3.5+ is needed to build DPDK using meson and ninja
 
-    * Python 2.7+ or 3.2+, to use various helper scripts included in the DPDK package.
+    * Python 3.2+, to use various helper scripts included in the DPDK package.
 
 *   Meson (version 0.47.1+) and ninja
 
doc/guides/rel_notes/deprecation.rst +0 -6
index 1f888fa90e..0a7e874b2c 100644
@@ -308,12 +308,6 @@ Deprecation Notices
   In this case the function will return -1 unless the environment is unset first
   (using ``rte_power_unset_env``). Other function usage scenarios will not change.
 
-* python: Since the beginning of 2020, Python 2 has officially reached
-  end-of-support: https://www.python.org/doc/sunset-python-2/.
-  Python 2 support will be completely removed in 20.11.
-  In 20.08, explicit deprecation warnings will be displayed when running
-  scripts with Python 2.
-
 * dpdk-setup.sh: This old script relies on deprecated stuff, and especially
   ``make``. Given environments are too much variables for such a simple script,
   it will be removed in DPDK 20.11.
doc/guides/rel_notes/release_20_11.rst +2 -0
index cc726095a7..2eb83c8137 100644
@@ -68,6 +68,8 @@ Removed Items
    Also, make sure to start the actual text at the margin.
    =======================================================
 
+* Removed python 2 support since it was EOL'd in January 2020.
+
 
 API Changes
 -----------
Signature · 1 line
2.25.1
Acked-by on v9 11/11
#151 lore raw

Re: [dpdk-dev] [PATCH v9 11/11] doc: support python3 only

Quoted 10 lines · Kevin Laatz

Remove references to python 2 from the documentation.

The deprecation notice for python 2 support has been removed. A section was added to the rel_notes for this patchset.

Cc Ray Kinsella <[email protected]>
Cc Neil Horman <[email protected]>
Cc John McNamara <[email protected]>
Cc Marko Kovacevic <[email protected]>
Quoted 8 lines
---
 doc/guides/contributing/coding_style.rst | 2 +-
 doc/guides/linux_gsg/sys_reqs.rst        | 2 +-
 doc/guides/rel_notes/deprecation.rst     | 6 ------
 doc/guides/rel_notes/release_20_11.rst   | 2 ++
 4 files changed, 4 insertions(+), 8 deletions(-)

[SNIP]

#152 lore raw

Re: [dpdk-dev] [PATCH v9 00/11] adding support for python 3 only

In reply to Kevin Laatz

Hello Kevin,

Quoted 27 lines · Kevin Laatz

This patch set converts all python scripts in the project to use python3 only and removes all deprecation notices associated with these changes. This is due to python2 being EOL in January 2020.

---
v6:
- rebased, removing conflict with make removal patchset.
- added changes to buildtools/map_to_win.py

v7:
- typo in email Cc'ing David Marchand
- added maintainers for buildtools patch

v8:
- removed unrelated cleanup
- replaced integer cast with integer division operator

v9:
- include documentation updates

Kevin Laatz (4):
app/test-cmdline: support python3 only
app/test: support python3 only
buildtools: support python3 only
doc: support python3 only

Quoted 8 lines

Louise Kilheeney (7):
usertools/dpdk-telemetry-client: support python3 only usertools/dpdk-devbind: support python3 only usertools/dpdk-pmdinfo: support python3 only usertools/cpu_layout: support python3 only
devtools: support python3 only
config/arm: support python3 only app/test-bbdev: support python3 only

I can still see one issue in doc/guides/conf.py as you mentioned yesterday. There is also a script that directly calls /usr/bin/python3, worth fixing from my pov.

$ for file in $(find app/ buildtools/ config/ devtools/ doc/ drivers/
examples/ kernel/ lib/ license/ usertools/ -name "*.py"); do head -1
$file |grep -q env.*python3 || echo $file; done
doc/guides/conf.py
usertools/dpdk-telemetry.py
Signature · 1 line
David Marchand
#153 lore raw

Re: [dpdk-dev] [PATCH v9 00/11] adding support for python 3 only

Quoted 37 lines · David Marchand

Hello Kevin,

Quoted 34 lines · Kevin Laatz

This patch set converts all python scripts in the project to use python3 only and removes all deprecation notices associated with these changes. This is due to python2 being EOL in January 2020.

---
v6:
- rebased, removing conflict with make removal patchset.
- added changes to buildtools/map_to_win.py

v7:
- typo in email Cc'ing David Marchand
- added maintainers for buildtools patch

v8:
- removed unrelated cleanup
- replaced integer cast with integer division operator

v9:
- include documentation updates

Kevin Laatz (4):
app/test-cmdline: support python3 only
app/test: support python3 only
buildtools: support python3 only
doc: support python3 only

Louise Kilheeney (7):
usertools/dpdk-telemetry-client: support python3 only usertools/dpdk-devbind: support python3 only usertools/dpdk-pmdinfo: support python3 only usertools/cpu_layout: support python3 only
devtools: support python3 only
config/arm: support python3 only app/test-bbdev: support python3 only

Hi David,

Quoted 9 lines

I can still see one issue in doc/guides/conf.py as you mentioned yesterday. There is also a script that directly calls /usr/bin/python3, worth fixing from my pov.

$ for file in $(find app/ buildtools/ config/ devtools/ doc/ drivers/
examples/ kernel/ lib/ license/ usertools/ -name "*.py"); do head -1
$file |grep -q env.*python3 || echo $file; done
doc/guides/conf.py
usertools/dpdk-telemetry.py

doc/guides/conf.py is indirectly called from sphinx which potentially causes some compatibilty issues when we make conf.py only support python3 (if sphinx in using python2) - this is why I left this out of the patchset, otherwise the doc build will be broken :-(

I can change direct call to /usr/bin/python3.

-Kevin

#154 lore raw

Re: [dpdk-dev] [PATCH v9 00/11] adding support for python 3 only

Quoted 51 lines · Kevin Laatz
Quoted 37 lines · David Marchand

Hello Kevin,

Quoted 34 lines · Kevin Laatz

This patch set converts all python scripts in the project to use python3 only and removes all deprecation notices associated with these changes. This is due to python2 being EOL in January 2020.

---
v6:
- rebased, removing conflict with make removal patchset.
- added changes to buildtools/map_to_win.py

v7:
- typo in email Cc'ing David Marchand
- added maintainers for buildtools patch

v8:
- removed unrelated cleanup
- replaced integer cast with integer division operator

v9:
- include documentation updates

Kevin Laatz (4):
app/test-cmdline: support python3 only
app/test: support python3 only
buildtools: support python3 only
doc: support python3 only

Louise Kilheeney (7):
usertools/dpdk-telemetry-client: support python3 only usertools/dpdk-devbind: support python3 only usertools/dpdk-pmdinfo: support python3 only usertools/cpu_layout: support python3 only
devtools: support python3 only
config/arm: support python3 only app/test-bbdev: support python3 only

Hi David,

Quoted 9 lines

I can still see one issue in doc/guides/conf.py as you mentioned yesterday. There is also a script that directly calls /usr/bin/python3, worth fixing from my pov.

$ for file in $(find app/ buildtools/ config/ devtools/ doc/ drivers/
examples/ kernel/ lib/ license/ usertools/ -name "*.py"); do head -1
$file |grep -q env.*python3 || echo $file; done
doc/guides/conf.py
usertools/dpdk-telemetry.py
Quoted 4 lines

doc/guides/conf.py is indirectly called from sphinx which potentially causes some compatibilty issues when we make conf.py only support python3 (if sphinx in using python2) - this is why I left this out of the patchset, otherwise the doc build will be broken :-(

I can change direct call to /usr/bin/python3.

I think we should err on the side of caution for the doc one - it can always be changed by a later patch anyway. Cleaning up the dpdk-telemetry one is probably worth doing though, I agree.

/Bruce

#155 lore raw

Re: [dpdk-dev] [PATCH v9 00/11] adding support for python 3 only

2020-09-29, Bruce Richardson:

Quoted 3 lines

I think we should err on the side of caution for the doc one - it can always be changed by a later patch anyway. Cleaning up the dpdk-telemetry one is probably worth doing though, I agree.

I'm not sure I agree. Since python 2 support is officially dropped in 20.11. Why bother with preserving it only for the docs?

Signature · 1 line
Robin
#156 lore raw

Re: [dpdk-dev] [PATCH v9 00/11] adding support for python 3 only

Quoted 8 lines · Robin Jarry

2020-09-29, Bruce Richardson:

Quoted 3 lines

I think we should err on the side of caution for the doc one - it can always be changed by a later patch anyway. Cleaning up the dpdk-telemetry one is probably worth doing though, I agree.

I'm not sure I agree. Since python 2 support is officially dropped in 20.11. Why bother with preserving it only for the docs?

Because we are not building the docs directly, sphinx is, and some systems may still have a python2 version of sphinx installed. While I think it's ok to say that to use DPDK you need to have python3 available, I don't like telling users that their entire toolset needs to be updated to python3 too. From what I can see, Centos/Redhat 7, which we still support, appears to only have a python2 version of sphinx-build.

/Bruce

Acked-by on the v9 series
#157 lore raw

Re: [dpdk-dev] [PATCH v9 00/11] adding support for python 3 only

In reply to Kevin Laatz
Quoted 46 lines · On Tue, 29 Sep 2020 11:22:13 +0100 Kevin Laatz <[email protected]> wrote:

This patch set converts all python scripts in the project to use python3 only and removes all deprecation notices associated with these changes. This is due to python2 being EOL in January 2020.

---
v6:
- rebased, removing conflict with make removal patchset.
- added changes to buildtools/map_to_win.py

v7:
- typo in email Cc'ing David Marchand
- added maintainers for buildtools patch

v8:
- removed unrelated cleanup
- replaced integer cast with integer division operator

v9:
- include documentation updates

Kevin Laatz (4):
app/test-cmdline: support python3 only
app/test: support python3 only
buildtools: support python3 only
doc: support python3 only

Louise Kilheeney (7):
usertools/dpdk-telemetry-client: support python3 only usertools/dpdk-devbind: support python3 only usertools/dpdk-pmdinfo: support python3 only usertools/cpu_layout: support python3 only
devtools: support python3 only
config/arm: support python3 only app/test-bbdev: support python3 only

 app/test-bbdev/test-bbdev.py             |  7 +------
 app/test-cmdline/cmdline_test.py         |  9 ++-------
 app/test-cmdline/cmdline_test_data.py    |  1 +
 app/test/autotest.py                     |  7 +------
 app/test/autotest_data.py                |  1 +
 app/test/autotest_runner.py              | 21 ++++++++-------------
 app/test/autotest_test_funcs.py          |  1 +
 buildtools/map_to_win.py                 |  3 +--
 config/arm/armv8_machine.py              |  2 +-
 devtools/update_version_map_abi.py       |  7 +------
 doc/guides/contributing/coding_style.rst |  2 +-
Quoted 8 lines
 doc/guides/linux_gsg/sys_reqs.rst        |  2 +-
 doc/guides/rel_notes/deprecation.rst     |  6 ------
 doc/guides/rel_notes/release_20_11.rst   |  2 ++
 usertools/cpu_layout.py                  | 13 ++-----------
 usertools/dpdk-devbind.py                | 22 ++++------------------
 usertools/dpdk-pmdinfo.py                |  7 +------
 usertools/dpdk-telemetry-client.py       | 18 +++---------------
 18 files changed, 32 insertions(+), 99 deletions(-)

Looks good Ack for whole series

#158 lore raw

v10 cover adding support for python 3 only

In reply to Kevin Laatz

This patch set converts all python scripts in the project to use python3 only and removes all deprecation notices associated with these changes. This is due to python2 being EOL in January 2020.

---
v6:
- rebased, removing conflict with make removal patchset.
- added changes to buildtools/map_to_win.py

v7:
- typo in email Cc'ing David Marchand
- added maintainers for buildtools patch

v8:
- removed unrelated cleanup
- replaced integer cast with integer division operator

v9:
- include documentation updates

v10:
- made doc/guides/conf.py support python3 only

Kevin Laatz (4):
app/test-cmdline: support python3 only
app/test: support python3 only
buildtools: support python3 only
doc: support python3 only

Louise Kilheeney (7):
usertools/dpdk-telemetry: support python3 only
usertools/dpdk-devbind: support python3 only
usertools/dpdk-pmdinfo: support python3 only
usertools/cpu_layout: support python3 only
devtools: support python3 only
config/arm: support python3 only
app/test-bbdev: support python3 only

 app/test-bbdev/test-bbdev.py             |  7 +------
 app/test-cmdline/cmdline_test.py         |  9 ++-------
 app/test-cmdline/cmdline_test_data.py    |  1 +
 app/test/autotest.py                     |  7 +------
 app/test/autotest_data.py                |  1 +
 app/test/autotest_runner.py              | 21 ++++++++-------------
 app/test/autotest_test_funcs.py          |  1 +
 buildtools/map_to_win.py                 |  3 +--
 config/arm/armv8_machine.py              |  2 +-
 devtools/update_version_map_abi.py       |  7 +------
 doc/guides/conf.py                       |  9 ++-------
 doc/guides/contributing/coding_style.rst |  2 +-
 doc/guides/linux_gsg/sys_reqs.rst        |  2 +-
 doc/guides/rel_notes/deprecation.rst     |  6 ------
 doc/guides/rel_notes/release_20_11.rst   |  2 ++
 usertools/cpu_layout.py                  | 13 ++-----------
 usertools/dpdk-devbind.py                | 22 ++++------------------
 usertools/dpdk-pmdinfo.py                |  7 +------
 usertools/dpdk-telemetry-client.py       | 18 +++---------------
 usertools/dpdk-telemetry.py              |  2 +-
 20 files changed, 35 insertions(+), 107 deletions(-)
Signature · 1 line
2.25.1
#159 lore raw

PATCH v10 1/11 usertools/dpdk-telemetry: support python3 only

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Kevin Laatz <[email protected]>

---
v8: removed cleanup not related to python3 only support

v10: fix direct call to directly call to /usr/bin/python3 in
dpdk-telemetry.py

---
 usertools/dpdk-telemetry-client.py | 18 +++---------------
 usertools/dpdk-telemetry.py        |  2 +-
 2 files changed, 4 insertions(+), 16 deletions(-)
usertools/dpdk-telemetry-client.py +3 -15
index 98d28fa89b..d8e439027c 100755
@@ -1,10 +1,7 @@
-#! /usr/bin/env python
+#! /usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Intel Corporation
 
-from __future__ import print_function
-from __future__ import unicode_literals
-
 import socket
 import os
 import sys
@@ -18,15 +15,6 @@
 GLOBAL_METRICS_REQ = "{\"action\":0,\"command\":\"global_stat_values\",\"data\":null}"
 DEFAULT_FP = "/var/run/dpdk/default_client"
 
-try:
-    raw_input  # Python 2
-except NameError:
-    raw_input = input  # Python 3
-
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 class Socket:
 
     def __init__(self):
@@ -86,7 +74,7 @@ def requestMetrics(self): # Requests metrics for given client
 
     def repeatedlyRequestMetrics(self, sleep_time): # Recursively requests metrics for given client
         print("\nPlease enter the number of times you'd like to continuously request Metrics:")
-        n_requests = int(raw_input("\n:"))
+        n_requests = int(input("\n:"))
         print("\033[F") #Removes the user input from screen, cleans it up
         print("\033[K")
         for i in range(n_requests):
@@ -107,7 +95,7 @@ def interactiveMenu(self, sleep_time): # Creates Interactive menu within the scr
             print("[4] Unregister client")
 
             try:
-                self.choice = int(raw_input("\n:"))
+                self.choice = int(input("\n:"))
                 print("\033[F") #Removes the user input for screen, cleans it up
                 print("\033[K")
                 if self.choice == 1:
usertools/dpdk-telemetry.py +1 -1
index 8e4039d574..181859658f 100755
@@ -1,4 +1,4 @@
-#! /usr/bin/python3
+#! /usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2020 Intel Corporation
 
Signature · 1 line
2.25.1
#160 lore raw

PATCH v10 2/11 usertools/dpdk-devbind: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

---
 usertools/dpdk-devbind.py | 22 ++++------------------
 1 file changed, 4 insertions(+), 18 deletions(-)
usertools/dpdk-devbind.py +4 -18
index 094c2ffc8b..8278a748d4 100755
@@ -1,9 +1,8 @@
-#! /usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 #
 
-from __future__ import print_function
 import sys
 import os
 import getopt
@@ -12,10 +11,6 @@
 from os.path import exists, abspath, dirname, basename
 from os.path import join as path_join
 
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 # The PCI base class for all devices
 network_class = {'Class': '02', 'Vendor': None, 'Device': None,
                     'SVendor': None, 'SDevice': None}
@@ -154,14 +149,6 @@ def usage():
 
     """ % locals())  # replace items from local variables
 
-
-# This is roughly compatible with check_output function in subprocess module
-# which is only available in python 2.7.
-def check_output(args, stderr=None):
-    '''Run a command and capture its output'''
-    return subprocess.Popen(args, stdout=subprocess.PIPE,
-                            stderr=stderr).communicate()[0]
-
 # check if a specific kernel module is loaded
 def module_is_loaded(module):
     global loaded_modules
@@ -218,8 +205,7 @@ def get_pci_device_details(dev_id, probe_lspci):
     device = {}
 
     if probe_lspci:
-        extra_info = check_output(["lspci", "-vmmks", dev_id]).splitlines()
-
+        extra_info = subprocess.check_output(["lspci", "-vmmks", dev_id]).splitlines()
         # parse lspci details
         for line in extra_info:
             if len(line) == 0:
@@ -255,7 +241,7 @@ def get_device_details(devices_type):
     # first loop through and read details for all devices
     # request machine readable format, with numeric IDs and String
     dev = {}
-    dev_lines = check_output(["lspci", "-Dvmmnnk"]).splitlines()
+    dev_lines = subprocess.check_output(["lspci", "-Dvmmnnk"]).splitlines()
     for dev_line in dev_lines:
         if len(dev_line) == 0:
             if device_type_match(dev, devices_type):
@@ -283,7 +269,7 @@ def get_device_details(devices_type):
         # check what is the interface if any for an ssh connection if
         # any to this host, so we can mark it later.
         ssh_if = []
-        route = check_output(["ip", "-o", "route"])
+        route = subprocess.check_output(["ip", "-o", "route"])
         # filter out all lines for 169.254 routes
         route = "\n".join(filter(lambda ln: not ln.startswith("169.254"),
                              route.decode().splitlines()))
Signature · 1 line
2.25.1
#161 lore raw

PATCH v10 3/11 usertools/dpdk-pmdinfo: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Neil Horman <[email protected]>
---
 usertools/dpdk-pmdinfo.py | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
usertools/dpdk-pmdinfo.py +1 -6
index f9ed755176..1661982791 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2016  Neil Horman <[email protected]>
 
@@ -7,8 +7,6 @@
 # Utility to dump PMD_INFO_STRING support from an object file
 #
 # -------------------------------------------------------------------------
-from __future__ import print_function
-from __future__ import unicode_literals
 import json
 import io
 import os
@@ -28,9 +26,6 @@
 pcidb = None
 
 # ===========================================
-if sys.version_info.major < 3:
-        print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-        print("Please use Python 3 instead", file=sys.stderr)
 
 class Vendor:
     """
Signature · 1 line
2.25.1
#162 lore raw

PATCH v10 4/11 usertools/cpu_layout: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

---
 usertools/cpu_layout.py | 13 ++-----------
 1 file changed, 2 insertions(+), 11 deletions(-)
usertools/cpu_layout.py +2 -11
index 5423c7965f..89a48cec46 100755
@@ -1,18 +1,9 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 # Copyright(c) 2017 Cavium, Inc. All rights reserved.
 
-from __future__ import print_function
 import sys
-try:
-    xrange # Python 2
-except NameError:
-    xrange = range # Python 3
-
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
 
 sockets = []
 cores = []
@@ -21,7 +12,7 @@
 fd = open("{}/kernel_max".format(base_path))
 max_cpus = int(fd.read())
 fd.close()
-for cpu in xrange(max_cpus + 1):
+for cpu in range(max_cpus + 1):
     try:
         fd = open("{}/cpu{}/topology/core_id".format(base_path, cpu))
     except IOError:
Signature · 1 line
2.25.1
#163 lore raw

Re: [dpdk-dev] [PATCH v10 04/11] usertools/cpu_layout: support python3 only

Quoted 19 lines · Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

---
 usertools/cpu_layout.py | 13 ++-----------
 1 file changed, 2 insertions(+), 11 deletions(-)
usertools/cpu_layout.py +0 -0
index 5423c7965f..89a48cec46 100755

usertools/cpu_layout.py line 6

Quoted 8 lines
usertools/cpu_layout.py +1 -2
@@ -1,18 +1,9 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 # Copyright(c) 2017 Cavium, Inc. All rights reserved.
 
-from __future__ import print_function
 import sys

sys is not needed anymore.
Fixed while applying.

Quoted 11 lines
usertools/cpu_layout.py +0 -8
-try:
-    xrange # Python 2
-except NameError:
-    xrange = range # Python 3
-
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
 
 sockets = []
 cores = []
Signature · 1 line
David Marchand
#164 lore raw

PATCH v10 5/11 app/test-cmdline: support python3 only

In reply to Kevin Laatz

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Olivier Matz <[email protected]>

---
v5:
- fixed python3 issue causing script to fail. Divisions without the typecast lead to a floating point result, which was messing up the loop.

v6:
- Removed changes to mk/rte.sdktest.mk since it no longer exists.

v8:
- Replaced integer cast with integer division operator.

---
 app/test-cmdline/cmdline_test.py      | 9 ++-------
 app/test-cmdline/cmdline_test_data.py | 1 +
 2 files changed, 3 insertions(+), 7 deletions(-)
app/test-cmdline/cmdline_test.py +2 -7
index 954428e2bf..f3377313e9 100755
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # Script that runs cmdline_test app and feeds keystrokes into it.
-from __future__ import print_function
 import cmdline_test_data
 import os
 import pexpect
@@ -19,10 +18,6 @@ def runTest(child, test):
         return 0
     child.expect(test["Result"], 1)
 
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 #
 # history test is a special case
 #
@@ -43,7 +38,7 @@ def runHistoryTest(child):
     i = 0
 
     # fill the history with numbers
-    while i < history_size / 10:
+    while i < history_size // 10:
         # add 1 to prevent from parsing as octals
         child.send("1" + str(i).zfill(8) + cmdline_test_data.ENTER)
         # the app will simply print out the number
app/test-cmdline/cmdline_test_data.py +1 -0
index 114d2cb6a0..2d9b3262a6 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
Signature · 1 line
2.25.1
#165 lore raw

PATCH v10 6/11 app/test: support python3 only

In reply to Kevin Laatz

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

---
v6:
- Removed changes to mk/rte.sdktest.mk since the file no longer exists

---
 app/test/autotest.py            |  7 +------
 app/test/autotest_data.py       |  1 +
 app/test/autotest_runner.py     | 21 ++++++++-------------
 app/test/autotest_test_funcs.py |  1 +
 4 files changed, 11 insertions(+), 19 deletions(-)
app/test/autotest.py +1 -6
index cf7584ccd7..9eef1efbe5 100644
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # Script that uses either test app or qemu controlled by python-pexpect
-from __future__ import print_function
 import autotest_data
 import autotest_runner
 import sys
@@ -17,10 +16,6 @@ def usage():
     usage()
     sys.exit(1)
 
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 target = sys.argv[2]
 
 test_whitelist = None
app/test/autotest_data.py +1 -0
index 4b7da45e09..097638941f 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
app/test/autotest_runner.py +8 -13
index 95e74c760d..998fe57a55 100644
@@ -1,10 +1,10 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # The main logic behind running autotests in parallel
 
-from __future__ import print_function
-import StringIO
+import io
 import csv
 from multiprocessing import Pool, Queue
 import pexpect
@@ -50,11 +50,7 @@ def first_cpu_on_node(node_nr):
                 map(os.path.basename, cpu_path)
             )
     )
-    # for compatibility between python 3 and 2 we need to make interable out
-    # of filter return as it returns list in python 2 and a generator in 3
-    m = next(iter(cpu_name))
-    return int(m.group(1))
-
+    return int(next(cpu_name).group(1))
 
 pool_child = None  # per-process child
 
@@ -78,7 +74,7 @@ def pool_init(queue, result_queue):
     cmdline = "%s %s" % (cmdline, prefix_cmdline)
 
     # prepare logging of init
-    startuplog = StringIO.StringIO()
+    startuplog = io.StringIO()
 
     # run test app
     try:
@@ -86,8 +82,7 @@ def pool_init(queue, result_queue):
         print("\n%s %s\n" % ("=" * 20, prefix), file=startuplog)
         print("\ncmdline=%s" % cmdline, file=startuplog)
 
-        pool_child = pexpect.spawn(cmdline, logfile=startuplog)
-
+        pool_child = pexpect.spawn(cmdline, logfile=startuplog, encoding='utf-8')
         # wait for target to boot
         if not wait_prompt(pool_child):
             pool_child.close()
@@ -138,7 +133,7 @@ def run_test(target, test):
     # create log buffer for each test
     # in multiprocessing environment, the logging would be
     # interleaved and will create a mess, hence the buffering
-    logfile = StringIO.StringIO()
+    logfile = io.StringIO()
     pool_child.logfile = logfile
 
     # make a note when the test started
@@ -210,9 +205,9 @@ def __init__(self, cmdline, target, blacklist, whitelist, n_processes):
         # parse the binary for available test commands
         binary = cmdline.split()[0]
         stripped = 'not stripped' not in \
-                   subprocess.check_output(['file', binary])
+                   subprocess.check_output(['file', binary]).decode()
         if not stripped:
-            symbols = subprocess.check_output(['nm', binary]).decode('utf-8')
+            symbols = subprocess.check_output(['nm', binary]).decode()
             self.avail_cmds = re.findall('test_register_(\w+)', symbols)
         else:
             self.avail_cmds = None
app/test/autotest_test_funcs.py +1 -0
index 26688b7132..775dfd1dc5 100644
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
Signature · 1 line
2.25.1
#166 lore raw

PATCH v10 7/11 devtools: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Neil Horman <[email protected]>
Cc Ray Kinsella <[email protected]>
---
 devtools/update_version_map_abi.py | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
devtools/update_version_map_abi.py +1 -6
index 10c3bc8098..3536a54b44 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2019 Intel Corporation
 
@@ -9,7 +9,6 @@
 from the devtools/update-abi.sh utility.
 """
 
-from __future__ import print_function
 import argparse
 import sys
 import re
@@ -160,10 +159,6 @@ def __generate_internal_abi(f_out, lines):
     print("};", file=f_out)
 
 def __main():
-    if sys.version_info.major < 3:
-        print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-        print("Please use Python 3 instead", file=sys.stderr)
-
     arg_parser = argparse.ArgumentParser(
         description='Merge versions in linker version script.')
 
Signature · 1 line
2.25.1
#167 lore raw

PATCH v10 8/11 config/arm: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2.

Cc Thomas Monjalon <[email protected]>
---
 config/arm/armv8_machine.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
config/arm/armv8_machine.py +1 -1
index 404866d2f8..1f689d9a83 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Cavium, Inc
 
Signature · 1 line
2.25.1
#168 lore raw

PATCH v10 9/11 app/test-bbdev: support python3 only

In reply to Kevin Laatz

From: Louise Kilheeney <[email protected]>

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Nicolas Chautru <[email protected]>
---
 app/test-bbdev/test-bbdev.py | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
app/test-bbdev/test-bbdev.py +1 -6
index 5ae2dc6c49..7d67dbe30a 100755
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Intel Corporation
 
-from __future__ import print_function
 import sys
 import os
 import argparse
@@ -16,10 +15,6 @@ def kill(process):
     print("ERROR: Test app timed out")
     process.kill()
 
-if sys.version_info.major < 3:
-    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
-    print("Please use Python 3 instead", file=sys.stderr)
-
 if "RTE_SDK" in os.environ:
     dpdk_path = os.environ["RTE_SDK"]
 else:
Signature · 1 line
2.25.1
#169 lore raw

PATCH v10 10/11 buildtools: support python3 only

In reply to Kevin Laatz

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed deprecation notice.

Cc Dmitry Kozlyuk <[email protected]>
Cc Narcisa Ana Maria Vasile <[email protected]>
Cc Dmitry Malloy <[email protected]>
Cc Pallavi Kadam <[email protected]>

---
v7:
- Cc Maintainers

---
 buildtools/map_to_win.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
buildtools/map_to_win.py +1 -2
index 2990b58634..2a6cb88605 100644
@@ -1,8 +1,7 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2019 Intel Corporation
 
-from __future__ import print_function
 import sys
 from os.path import dirname, basename, join, exists
 
Signature · 1 line
2.25.1
#170 lore raw

PATCH v10 11/11 doc: support python3 only

In reply to Kevin Laatz

Changed script to explicitly use python3 only to avoid maintaining python 2 and removed references to python 2 from the documentation.

The deprecation notice for python 2 support has been removed. A section was added to the rel_notes for this patchset.

Cc Ray Kinsella <[email protected]>
Cc Neil Horman <[email protected]>
Cc John McNamara <[email protected]>
Cc Marko Kovacevic <[email protected]>

---
v10:
- changes to doc/guides/conf.py to only support python3

---
 doc/guides/conf.py                       | 9 ++-------
 doc/guides/contributing/coding_style.rst | 2 +-
 doc/guides/linux_gsg/sys_reqs.rst        | 2 +-
 doc/guides/rel_notes/deprecation.rst     | 6 ------
 doc/guides/rel_notes/release_20_11.rst   | 2 ++
 5 files changed, 6 insertions(+), 15 deletions(-)
doc/guides/conf.py +2 -7
index 9ebc26ed3f..ef550f68c0 100644
@@ -1,7 +1,7 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2015 Intel Corporation
 
-from __future__ import print_function
 from docutils import nodes
 from distutils.version import LooseVersion
 from sphinx import __version__ as sphinx_version
@@ -13,12 +13,7 @@
 from os.path import dirname
 from os.path import join as path_join
 
-try:
-    # Python 2.
-    import ConfigParser as configparser
-except:
-    # Python 3.
-    import configparser
+import configparser
 
 try:
     import sphinx_rtd_theme
doc/guides/contributing/coding_style.rst +1 -1
index 0be9546a6a..4fc010ca49 100644
@@ -765,7 +765,7 @@ specializations, run the ``app/test`` binary, and use the ``dump_log_types``
 Python Code
 -----------
 
-All Python code should work with Python 2.7+ and 3.2+ and be compliant with
+All Python code should work with 3.2+ and be compliant with
 `PEP8 (Style Guide for Python Code) <https://www.python.org/dev/peps/pep-0008/>`_.
 
 The ``pep8`` tool can be used for testing compliance with the guidelines.
doc/guides/linux_gsg/sys_reqs.rst +1 -1
index 6b9f573fce..66d9723dd6 100644
@@ -47,7 +47,7 @@ Compilation of the DPDK
 
     * Python v3.5+ is needed to build DPDK using meson and ninja
 
-    * Python 2.7+ or 3.2+, to use various helper scripts included in the DPDK package.
+    * Python 3.2+, to use various helper scripts included in the DPDK package.
 
 *   Meson (version 0.47.1+) and ninja
 
doc/guides/rel_notes/deprecation.rst +0 -6
index 1f888fa90e..0a7e874b2c 100644
@@ -308,12 +308,6 @@ Deprecation Notices
   In this case the function will return -1 unless the environment is unset first
   (using ``rte_power_unset_env``). Other function usage scenarios will not change.
 
-* python: Since the beginning of 2020, Python 2 has officially reached
-  end-of-support: https://www.python.org/doc/sunset-python-2/.
-  Python 2 support will be completely removed in 20.11.
-  In 20.08, explicit deprecation warnings will be displayed when running
-  scripts with Python 2.
-
 * dpdk-setup.sh: This old script relies on deprecated stuff, and especially
   ``make``. Given environments are too much variables for such a simple script,
   it will be removed in DPDK 20.11.
doc/guides/rel_notes/release_20_11.rst +2 -0
index cc726095a7..2eb83c8137 100644
@@ -68,6 +68,8 @@ Removed Items
    Also, make sure to start the actual text at the margin.
    =======================================================
 
+* Removed python 2 support since it was EOL'd in January 2020.
+
 
 API Changes
 -----------
Signature · 1 line
2.25.1
#171 lore raw

Re: [dpdk-dev] [PATCH v10 11/11] doc: support python3 only

Quoted 14 lines · Kevin Laatz
doc/guides/conf.py +1 -1
index 9ebc26ed3f..ef550f68c0 100644
@@ -1,7 +1,7 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2015 Intel Corporation
 
-from __future__ import print_function
 from docutils import nodes
 from distutils.version import LooseVersion
 from sphinx import __version__ as sphinx_version

doc/guides/conf.py line 16

Quoted 10 lines
doc/guides/conf.py +1 -6
@@ -13,12 +13,7 @@
 from os.path import dirname
 from os.path import join as path_join
 
-try:
-    # Python 2.
-    import ConfigParser as configparser
-except:
-    # Python 3.
-    import configparser
+import configparser

I am ok with dropping this part: the doc generation with python 2 / an old sphinx is already broken since 20.02 with meson.
I can give details for people interested.

Quoted 8 lines
doc/guides/conf.py +0 -0
 try:
     import sphinx_rtd_theme
doc/guides/contributing/coding_style.rst +0 -0
index 0be9546a6a..4fc010ca49 100644

doc/guides/contributing/coding_style.rst specializations, run the ``app/test`` binary, and use the ``dump_log_types`` · line 769

Quoted 6 lines
doc/guides/contributing/coding_style.rst +1 -1
@@ -765,7 +765,7 @@ specializations, run the ``app/test`` binary, and use the ``dump_log_types``
 Python Code
 -----------
 
-All Python code should work with Python 2.7+ and 3.2+ and be compliant with
+All Python code should work with 3.2+ and be compliant with
 `PEP8 (Style Guide for Python Code) <https://www.python.org/dev/peps/pep-0008/>`_.

I am for dropping any mention of a version here.
On current HEAD of the main branch, the requirements already ask for a 3.5 version.
I would simply ask for PEP8 compliance in this doc.

Opinion?

doc/guides/linux_gsg/sys_reqs.rst Compilation of the DPDK · line 50

Quoted 12 lines
doc/guides/contributing/coding_style.rst +0 -0
 The ``pep8`` tool can be used for testing compliance with the guidelines.
doc/guides/linux_gsg/sys_reqs.rst +1 -1
index 6b9f573fce..66d9723dd6 100644
@@ -47,7 +47,7 @@ Compilation of the DPDK
 
     * Python v3.5+ is needed to build DPDK using meson and ninja
 
-    * Python 2.7+ or 3.2+, to use various helper scripts included in the DPDK package.
+    * Python 3.2+, to use various helper scripts included in the DPDK package.

Dropped this hunk, as it has already been updated with the "make removal" series.

Signature · 1 line
David Marchand
#172 lore raw

Re: [dpdk-dev] [PATCH v10 11/11] doc: support python3 only

Quoted 2 lines · David Marchand

On Wed, Sep 30, 2020 at 1:45 PM Kevin Laatz <[email protected]> wrote:

[...]

Quoted 18 lines

doc/guides/contributing/coding_style.rst specializations, run the ``app/test`` binary, and use the ``dump_log_types`` · line 769

Quoted 13 lines

try:
import sphinx_rtd_theme

doc/guides/contributing/coding_style.rst +1 -1
index 0be9546a6a..4fc010ca49 100644
@@ -765,7 +765,7 @@ specializations, run the ``app/test`` binary, and use the ``dump_log_types``
  Python Code
  -----------
 
-All Python code should work with Python 2.7+ and 3.2+ and be compliant with
+All Python code should work with 3.2+ and be compliant with
  `PEP8 (Style Guide for Python Code) <https://www.python.org/dev/peps/pep-0008/>`_.

I am for dropping any mention of a version here.
On current HEAD of the main branch, the requirements already ask for a 3.5 version.
I would simply ask for PEP8 compliance in this doc.

Quoted 2 lines

Opinion?

+1 That was something i concidered while making the patch - saves someone maitaining the versions here in future.

- Kevin

#173 lore raw

Re: [dpdk-dev] [PATCH v10 11/11] doc: support python3 only

In reply to David Marchand
Quoted 18 lines · David Marchand
Quoted 17 lines · Kevin Laatz
doc/guides/conf.py +1 -1
index 9ebc26ed3f..ef550f68c0 100644
@@ -1,7 +1,7 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2015 Intel Corporation
 
-from __future__ import print_function
 from docutils import nodes
 from distutils.version import LooseVersion
 from sphinx import __version__ as sphinx_version
@@ -13,12 +13,7 @@
 from os.path import dirname
 from os.path import join as path_join
 
Quoted 12 lines

doc/guides/conf.py line 16

Quoted 7 lines
doc/guides/conf.py +1 -6
-try:
-    # Python 2.
-    import ConfigParser as configparser
-except:
-    # Python 3.
-    import configparser
+import configparser

I am ok with dropping this part: the doc generation with python 2 / an old sphinx is already broken since 20.02 with meson.
I can give details for people interested.

I'm ok with this. I suspect most non-developers of DPDK just use the published docs on the website.

Quoted 21 lines
Quoted 9 lines
doc/guides/conf.py +0 -0
 try:
     import sphinx_rtd_theme
doc/guides/contributing/coding_style.rst +0 -0
index 0be9546a6a..4fc010ca49 100644

doc/guides/contributing/coding_style.rst specializations, run the ``app/test`` binary, and use the ``dump_log_types`` · line 769

Quoted 6 lines
doc/guides/contributing/coding_style.rst +1 -1
@@ -765,7 +765,7 @@ specializations, run the ``app/test`` binary, and use the ``dump_log_types``
 Python Code
 -----------
 
-All Python code should work with Python 2.7+ and 3.2+ and be compliant with
+All Python code should work with 3.2+ and be compliant with
 `PEP8 (Style Guide for Python Code) <https://www.python.org/dev/peps/pep-0008/>`_.

I am for dropping any mention of a version here.
On current HEAD of the main branch, the requirements already ask for a 3.5 version.
I would simply ask for PEP8 compliance in this doc.

Quoted 2 lines

Opinion?

Ok with this too.

#174 lore raw

Re: [dpdk-dev] [PATCH v10 00/11] adding support for python 3 only

In reply to Kevin Laatz
Quoted 5 lines · Kevin Laatz

This patch set converts all python scripts in the project to use python3 only and removes all deprecation notices associated with these changes. This is due to python2 being EOL in January 2020.

Squashed all as one patch (all commit logs were clones and the changes on the files were mechanical).
Updated coding style guide as commented in the thread.

Applied, thanks Kevin and Louise!

Signature · 1 line
David Marchand