Skip to contentSkip to conversation

tests:py: conversion to python3

PATCH In review Shekhar Sharma 3 participants 11 messages Jun 14, 2019 – Jun 19, 2019 netfilter

lore
Share & tools
lore
#0 lore raw

PATCH v7 1/2 tests:py: conversion to python3

This patch converts the 'nft-test.py' file to run on both python 2 and python3.

---
The version hystory of this patch is:
v1:conversion to py3 by changing the print statements. v2:add the '__future__' package for compatibility with py2 and py3. v3:solves the 'version' problem in argparse by adding a new argument. v4:uses .format() method to make print statements clearer. v5:updated the shebang and corrected the sequence of import statements. v6:resent the same with small changes
v7:resent with small changes

 tests/py/nft-test.py | 44 +++++++++++++++++++++++---------------------
 1 file changed, 23 insertions(+), 21 deletions(-)
tests/py/nft-test.py +23 -21
index 09d00dba..f80517e6 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/python2
+#!/usr/bin/env python
 #
 # (C) 2014 by Ana Rey Botello <[email protected]>
 #
@@ -13,6 +13,7 @@
 # Thanks to the Outreach Program for Women (OPW) for sponsoring this test
 # infrastructure.
 
+from __future__ import print_function
 import sys
 import os
 import argparse
@@ -1016,9 +1017,9 @@ def execute_cmd(cmd, filename, lineno, stdout_log=False, debug=False):
     :param debug: temporarily set these debug flags
     '''
     global log_file
-    print >> log_file, "command: %s" % cmd
+    print("command: {}".format(cmd), file=log_file)
     if debug_option:
-        print cmd
+        print(cmd)
 
     if debug:
         debug_old = nftables.get_debug()
@@ -1212,7 +1213,7 @@ def run_test_file(filename, force_all_family_option, specific_file):
         sys.stdout.flush()
 
         if signal_received == 1:
-            print "\nSignal received. Cleaning up and Exitting..."
+            print("\nSignal received. Cleaning up and Exitting...")
             cleanup_on_exit()
             sys.exit(0)
 
@@ -1319,13 +1320,13 @@ def run_test_file(filename, force_all_family_option, specific_file):
 
     if specific_file:
         if force_all_family_option:
-            print print_result_all(filename, tests, total_warning, total_error,
-                                   total_unit_run)
+            print(print_result_all(filename, tests, total_warning, total_error,
+                                   total_unit_run))
         else:
-            print print_result(filename, tests, total_warning, total_error)
+            print(print_result(filename, tests, total_warning, total_error))
     else:
         if tests == passed and tests > 0:
-            print filename + ": " + Colors.GREEN + "OK" + Colors.ENDC
+            print(filename + ": " + Colors.GREEN + "OK" + Colors.ENDC)
 
     f.close()
     del table_list[:]
@@ -1336,7 +1337,7 @@ def run_test_file(filename, force_all_family_option, specific_file):
 
 
 def main():
-    parser = argparse.ArgumentParser(description='Run nft tests', version='1.0')
+    parser = argparse.ArgumentParser(description='Run nft tests')
 
     parser.add_argument('filenames', nargs='*', metavar='path/to/file.t',
                         help='Run only these tests')
@@ -1359,6 +1360,10 @@ def main():
                         dest='enable_schema',
                         help='verify json input/output against schema')
 
+    parser.add_argument('-v', '--version', action='version',
+                        version='1.0',
+                        help='Print the version information')
+
     args = parser.parse_args()
     global debug_option, need_fix_option, enable_json_option, enable_json_schema
     debug_option = args.debug
@@ -1372,15 +1377,15 @@ def main():
     signal.signal(signal.SIGTERM, signal_handler)
 
     if os.getuid() != 0:
-        print "You need to be root to run this, sorry"
+        print("You need to be root to run this, sorry")
         return
 
     # Change working directory to repository root
     os.chdir(TESTS_PATH + "/../..")
 
     if not os.path.exists('src/.libs/libnftables.so'):
-        print "The nftables library does not exist. " \
-              "You need to build the project."
+        print("The nftables library does not exist. "
+              "You need to build the project.")
         return
 
     if args.enable_schema and not args.enable_json:
@@ -1434,19 +1439,16 @@ def main():
             run_total += file_unit_run
 
     if test_files == 0:
-        print "No test files to run"
+        print("No test files to run")
     else:
         if not specific_file:
             if force_all_family_option:
-                print "%d test files, %d files passed, %d unit tests, " \
-                      "%d total executed, %d error, %d warning" \
-                      % (test_files, files_ok, tests, run_total, errors,
-                         warnings)
-            else:
-                print "%d test files, %d files passed, %d unit tests, " \
-                      "%d error, %d warning" \
-                      % (test_files, files_ok, tests, errors, warnings)
+                print("{} test files, {} files passed, {} unit tests, ".format(test_files,files_ok,tests))
+                print("{} total executed, {} error, {} warning".format(run_total, errors, warnings))
 
+            else:
+                print("{} test files, {} files passed, {} unit tests, ".format(test_files,files_ok,tests))
+                print("{} error, {} warning".format(errors, warnings))
 
 if __name__ == '__main__':
     main()
Signature · 1 line
2.17.1
Acked-by on v7 1/2
#1 lore raw
Quoted 12 lines · Shekhar Sharma

This patch converts the 'nft-test.py' file to run on both python 2 and python3.

---
The version hystory of this patch is:
v1:conversion to py3 by changing the print statements. v2:add the '__future__' package for compatibility with py2 and py3. v3:solves the 'version' problem in argparse by adding a new argument. v4:uses .format() method to make print statements clearer. v5:updated the shebang and corrected the sequence of import statements. v6:resent the same with small changes
v7:resent with small changes

"with small changes" is not helpful. In the future please list what was actually changed so reviewers know what to focus on.

Patch looks good though.

#2 lore raw
Quoted 13 lines · Eric Garver
Quoted 12 lines · Shekhar Sharma

This patch converts the 'nft-test.py' file to run on both python 2 and python3.

---
The version hystory of this patch is:
v1:conversion to py3 by changing the print statements. v2:add the '__future__' package for compatibility with py2 and py3. v3:solves the 'version' problem in argparse by adding a new argument. v4:uses .format() method to make print statements clearer. v5:updated the shebang and corrected the sequence of import statements. v6:resent the same with small changes
v7:resent with small changes

I apply this patch, then, from the nftables/tests/py/ folder I run:

# python3 nft-test.py

I get:

INFO: Log will be available at /tmp/nftables-test.log Traceback (most recent call last):
File "nft-test.py", line 1454, in <module>

    main()

File "nft-test.py", line 1422, in main
result = run_test_file(filename, force_all_family_option, specific_file)
File "nft-test.py", line 1290, in run_test_file
filename_path)
File "nft-test.py", line 774, in rule_add
payload_log = os.tmpfile()
AttributeError: module 'os' has no attribute 'tmpfile'

#3 lore raw

Hi Pablo!

Quoted 25 lines · Pablo Neira Ayuso
Quoted 13 lines · Eric Garver
Quoted 12 lines · Shekhar Sharma

This patch converts the 'nft-test.py' file to run on both python 2 and python3.

---
The version hystory of this patch is:
v1:conversion to py3 by changing the print statements. v2:add the '__future__' package for compatibility with py2 and py3. v3:solves the 'version' problem in argparse by adding a new argument. v4:uses .format() method to make print statements clearer. v5:updated the shebang and corrected the sequence of import statements. v6:resent the same with small changes
v7:resent with small changes

I apply this patch, then, from the nftables/tests/py/ folder I run:

# python3 nft-test.py

I get:

INFO: Log will be available at /tmp/nftables-test.log Traceback (most recent call last):
File "nft-test.py", line 1454, in <module>

Quoted 8 lines
    main()

File "nft-test.py", line 1422, in main
result = run_test_file(filename, force_all_family_option, specific_file)
File "nft-test.py", line 1290, in run_test_file
filename_path)
File "nft-test.py", line 774, in rule_add
payload_log = os.tmpfile()
AttributeError: module 'os' has no attribute 'tmpfile'

I do not know why this error is occurring but may i suggest
you to try the v8 of the netns patch, (as it is a continuation of this patch), if that works, we will know that there is some problem in this patch specifically.

Thanks!
Shekhar

#4 lore raw
Quoted 37 lines · shekhar sharma

Hi Pablo!

Quoted 25 lines · Pablo Neira Ayuso
Quoted 13 lines · Eric Garver
Quoted 12 lines · Shekhar Sharma

This patch converts the 'nft-test.py' file to run on both python 2 and python3.

---
The version hystory of this patch is:
v1:conversion to py3 by changing the print statements. v2:add the '__future__' package for compatibility with py2 and py3. v3:solves the 'version' problem in argparse by adding a new argument. v4:uses .format() method to make print statements clearer. v5:updated the shebang and corrected the sequence of import statements. v6:resent the same with small changes
v7:resent with small changes

I apply this patch, then, from the nftables/tests/py/ folder I run:

# python3 nft-test.py

I get:

INFO: Log will be available at /tmp/nftables-test.log Traceback (most recent call last):
File "nft-test.py", line 1454, in <module>

Quoted 8 lines
    main()

File "nft-test.py", line 1422, in main
result = run_test_file(filename, force_all_family_option, specific_file)
File "nft-test.py", line 1290, in run_test_file
filename_path)
File "nft-test.py", line 774, in rule_add
payload_log = os.tmpfile()
AttributeError: module 'os' has no attribute 'tmpfile'

Quoted 4 lines

I do not know why this error is occurring but may i suggest
you to try the v8 of the netns patch, (as it is a continuation of this patch), if that works, we will know that there is some problem in this patch specifically.

Still the same problem with v8:

Date: Mon Jun 17 19:45:58 2019 +0530

tests: py: add netns feature

This patch adds the netns feature to the 'nft-test.py' file.

#5 lore raw
Quoted 44 lines · Pablo Neira Ayuso
Quoted 37 lines · shekhar sharma

Hi Pablo!

Quoted 25 lines · Pablo Neira Ayuso
Quoted 13 lines · Eric Garver
Quoted 12 lines · Shekhar Sharma

This patch converts the 'nft-test.py' file to run on both python 2 and python3.

---
The version hystory of this patch is:
v1:conversion to py3 by changing the print statements. v2:add the '__future__' package for compatibility with py2 and py3. v3:solves the 'version' problem in argparse by adding a new argument. v4:uses .format() method to make print statements clearer. v5:updated the shebang and corrected the sequence of import statements. v6:resent the same with small changes
v7:resent with small changes

I apply this patch, then, from the nftables/tests/py/ folder I run:

# python3 nft-test.py

I get:

INFO: Log will be available at /tmp/nftables-test.log Traceback (most recent call last):
File "nft-test.py", line 1454, in <module>

Quoted 8 lines
    main()

File "nft-test.py", line 1422, in main
result = run_test_file(filename, force_all_family_option, specific_file)
File "nft-test.py", line 1290, in run_test_file
filename_path)
File "nft-test.py", line 774, in rule_add
payload_log = os.tmpfile()
AttributeError: module 'os' has no attribute 'tmpfile'

Quoted 4 lines

I do not know why this error is occurring but may i suggest
you to try the v8 of the netns patch, (as it is a continuation of this patch), if that works, we will know that there is some problem in this patch specifically.

Quoted 7 lines

Still the same problem with v8:

Date: Mon Jun 17 19:45:58 2019 +0530

tests: py: add netns feature

This patch adds the netns feature to the 'nft-test.py' file.

Ok. I am trying to find out why this is happening.

Shekhar

#6 lore raw

PATCH 1/1 nft-test.py: use tempfile module

os.tmpfile() is not in python3.

---
 tests/py/nft-test.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
tests/py/nft-test.py +3 -2
index f80517e67bfd..4da6fa650f6d 100755
@@ -20,6 +20,7 @@ import argparse
 import signal
 import json
 import traceback
+import tempfile
 
 TESTS_PATH = os.path.dirname(os.path.abspath(__file__))
 sys.path.insert(0, os.path.join(TESTS_PATH, '../../py/'))
@@ -771,7 +772,7 @@ def rule_add(rule, filename, lineno, force_all_family_option, filename_path):
             unit_tests += 1
             table_flush(table, filename, lineno)
 
-            payload_log = os.tmpfile()
+            payload_log = tempfile.TemporaryFile(mode="w+")
 
             # Add rule and check return code
             cmd = "add rule %s %s %s" % (table, chain, rule[0])
@@ -911,7 +912,7 @@ def rule_add(rule, filename, lineno, force_all_family_option, filename_path):
                               gotf.name, 1)
 
             table_flush(table, filename, lineno)
-            payload_log = os.tmpfile()
+            payload_log = tempfile.TemporaryFile(mode="w+")
 
             # Add rule in JSON format
             cmd = json.dumps({ "nftables": [{ "add": { "rule": {
Signature · 1 line
2.20.1
#7 lore raw

Re: [PATCH nft] nft-test.py: use tempfile module

Hi Eric!

Quoted 3 lines · Eric Garver

os.tmpfile() is not in python3.

Did not know that. It should resolve the problem. Thanks!

---

Quoted 34 lines
 tests/py/nft-test.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
tests/py/nft-test.py +3 -3
index f80517e67bfd..4da6fa650f6d 100755
@@ -20,6 +20,7 @@ import argparse
 import signal
 import json
 import traceback
+import tempfile
 
 TESTS_PATH = os.path.dirname(os.path.abspath(__file__))
 sys.path.insert(0, os.path.join(TESTS_PATH, '../../py/'))
@@ -771,7 +772,7 @@ def rule_add(rule, filename, lineno, force_all_family_option, filename_path):
             unit_tests += 1
             table_flush(table, filename, lineno)
 
-            payload_log = os.tmpfile()
+            payload_log = tempfile.TemporaryFile(mode="w+")
 
             # Add rule and check return code
             cmd = "add rule %s %s %s" % (table, chain, rule[0])
@@ -911,7 +912,7 @@ def rule_add(rule, filename, lineno, force_all_family_option, filename_path):
                               gotf.name, 1)
 
             table_flush(table, filename, lineno)
-            payload_log = os.tmpfile()
+            payload_log = tempfile.TemporaryFile(mode="w+")
 
             # Add rule in JSON format
             cmd = json.dumps({ "nftables": [{ "add": { "rule": {
--
Quoted 2 lines

2.20.1

Shekhar

#8 lore raw

Re: [PATCH nft] nft-test.py: use tempfile module

In reply to Eric Garver
Quoted 1 line · Eric Garver

os.tmpfile() is not in python3.

If I apply:

https://patchwork.ozlabs.org/patch/1116034/

and this patch, it's getting better, but still I hit one problem:

# python3 nft-test.py
INFO: Log will be available at /tmp/nftables-test.log
any/fwd.t: OK
any/rt.t: OK
any/queue.t: OK
any/dup.t: OK
any/log.t: OK
Traceback (most recent call last):
  File "nft-test.py", line 1455, in <module>
    main()
  File "nft-test.py", line 1423, in main
    result = run_test_file(filename, force_all_family_option,
specific_file)
  File "nft-test.py", line 1291, in run_test_file
    filename_path)
  File "nft-test.py", line 846, in rule_add
    rule_output.rstrip()) != 0:
  File "nft-test.py", line 495, in set_check_element
    if (cmp(rule1[:pos1], rule2[:pos2]) != 0):
NameError: name 'cmp' is not defined
#9 lore raw

Re: [PATCH nft] nft-test.py: use tempfile module

Quoted 21 lines · Pablo Neira Ayuso
Quoted 1 line · Eric Garver

os.tmpfile() is not in python3.

If I apply:

https://patchwork.ozlabs.org/patch/1116034/

and this patch, it's getting better, but still I hit one problem:

# python3 nft-test.py
INFO: Log will be available at /tmp/nftables-test.log
any/fwd.t: OK
any/rt.t: OK
any/queue.t: OK
any/dup.t: OK
any/log.t: OK
Traceback (most recent call last):
  File "nft-test.py", line 1455, in <module>
    main()
  File "nft-test.py", line 1423, in main
    result = run_test_file(filename, force_all_family_option,
Quoted 8 lines
specific_file)
  File "nft-test.py", line 1291, in run_test_file
    filename_path)
  File "nft-test.py", line 846, in rule_add
    rule_output.rstrip()) != 0:
  File "nft-test.py", line 495, in set_check_element
    if (cmp(rule1[:pos1], rule2[:pos2]) != 0):
NameError: name 'cmp' is not defined

I will post a short series today to address this as well.

#10 lore raw
In reply to Eric Garver
Quoted 15 lines · Eric Garver
Quoted 12 lines · Shekhar Sharma

This patch converts the 'nft-test.py' file to run on both python 2 and python3.

---
The version hystory of this patch is:
v1:conversion to py3 by changing the print statements. v2:add the '__future__' package for compatibility with py2 and py3. v3:solves the 'version' problem in argparse by adding a new argument. v4:uses .format() method to make print statements clearer. v5:updated the shebang and corrected the sequence of import statements. v6:resent the same with small changes
v7:resent with small changes

Quoted 3 lines

"with small changes" is not helpful. In the future please list what was actually changed so reviewers know what to focus on.

Sorry, will be more specific next time. :-)

Quoted 3 lines

Patch looks good though.