default runtime test case for python packages
cover default runtime test case for python packages
Hello,
This series creates a standard runtime test case for python packages and is based on ideas from Thomas and Arnout.
The sample scripts that run on the target (fixture for the test case) are stored in a separate file (so it becomes more readable than inline in the test case). For the time being, they are stored on support/testing.
Those sample scripts are copied by the default test case to the target image in build time. The test infra builds the image, and the default test case logs in the target, checks the script is in the rootfs and calls the Python interpreter passing the sample script.
The first patch adds the common logic to copy the sample script and execute it.
Each one of the following patches changes one of the 6 current test cases for python packages to use the common logic.
Since the sample scripts are now stored on separate files they are automatically picked by the check-flake8 job on GitLab CI, so I formatted them to comply to flake8 (only empty lines or adding "# noqa" for the script that only import the module but don't use it).
Here a complete run on Gitlab CI:
https://gitlab.com/RicardoMartincoski/buildroot/pipelines/32931721
Regards,
Ricardo
Ricardo Martincoski (7):
support/testing: create default test case for python packages support/testing: use default test_run for python-autobahn support/testing: use default test_run for python-cryptography support/testing: use default test_run for python-incremental support/testing: use default test_run for python-twisted support/testing: use default test_run for python-txaio support/testing: use default test_run for python-txtorcon
.../package/copy-sample-script-to-target.sh | 7 +++ .../tests/package/sample_python_autobahn.py | 1 + .../package/sample_python_cryptography.py | 3 ++ .../package/sample_python_incremental.py | 3 ++ .../tests/package/sample_python_twisted.py | 9 ++++ .../package/sample_python_txaio_asyncio.py | 3 ++ .../package/sample_python_txaio_twisted.py | 3 ++ .../tests/package/sample_python_txtorcon.py | 1 + support/testing/tests/package/test_python.py | 43 ++++++++++++++++++ .../tests/package/test_python_autobahn.py | 31 ++++--------- .../tests/package/test_python_cryptography.py | 34 ++++---------- .../tests/package/test_python_incremental.py | 34 ++++---------- .../tests/package/test_python_twisted.py | 45 ++++++------------- .../tests/package/test_python_txaio.py | 30 +++---------- .../tests/package/test_python_txtorcon.py | 32 ++++--------- 15 files changed, 129 insertions(+), 150 deletions(-) create mode 100755 support/testing/tests/package/copy-sample-script-to-target.sh create mode 100644 support/testing/tests/package/sample_python_autobahn.py create mode 100644 support/testing/tests/package/sample_python_cryptography.py create mode 100644 support/testing/tests/package/sample_python_incremental.py create mode 100644 support/testing/tests/package/sample_python_twisted.py create mode 100644 support/testing/tests/package/sample_python_txaio_asyncio.py create mode 100644 support/testing/tests/package/sample_python_txaio_twisted.py create mode 100644 support/testing/tests/package/sample_python_txtorcon.py
Signature · 1 line
2.17.1
PATCH 1/7 support/testing: create default test case for python packages
Test cases for python packages are very similar among each other: run a simple script in the target that minimally tests the package. So move some common logic to the TestPythonBase. Also move the test script to be run on the target from inline in the test case to a separate file.
After this change:
TestPythonBase adds in build time one or more sample scripts to be run on the target. The test case for the python package must explicitly list them in the "sample_scripts" property. The test case then logins to the target, checks the scripts are really in the rootfs (it calls "md5sum" instead of "ls" or "test" in an attempt to make the logfile more friendly, since someone analysing a failure can easily check the expected script was executed) and then calls the python interpreter passing the sample script as parameter.
An optional property "timeout" exists for the case the sample script needs more time to run than the default timeout from the test infra (currently 5 seconds).
Instead of providing a defconfig fragment containing the python version, the test case for a python package must provide only its own config set, and inherit directly from TestPython2 or TestPython3 (that in turn inherit from TestPythonBase).
A simple test case for a package that only supports Python 2 will look like this:
|import tests.package.test_python | | |class TestPythonPy2<Package>(tests.package.test_python.TestPython2): | config_package = \ | """ | BR2_PACKAGE_PYTHON_<PACKAGE>=y | """ | sample_scripts = ["tests/package/sample_python_<package>.py"] | timeout = 15
When the python package supports both Python 2 and Python 3 a helper
class can be used to hold the common properties:
|class TestPython<Package>():
| config_package = \
...
|class TestPythonPy2<Package>(TestPython<Package>, tests.package.test_python.TestPython2):
...
|class TestPythonPy3<Package>(TestPython<Package>, tests.package.test_python.TestPython3):
One catch when inheriting from another class that contains a method test_run is that if "from tests.package.test_python import TestPython2" is used, nose2 would list the test case TestPython2 twice because it looks for symbols in the module namespace to discover test cases to run. So any test cases that inherits from TestPython2 or TestPython3 should instead import the module and use the full name tests.package.test_python.TestPython2.
--- .../package/copy-sample-script-to-target.sh | 7 +++ support/testing/tests/package/test_python.py | 43 +++++++++++++++++++ 2 files changed, 50 insertions(+) create mode 100755 support/testing/tests/package/copy-sample-script-to-target.sh
support/testing/tests/package/copy-sample-script-to-target.sh new file +7 -0
@@ -0,0 +1,7 @@ +#!/bin/sh +set -e + +shift +for file in "$@"; do + cp -f "${file}" "${TARGET_DIR}/root/" +done
support/testing/tests/package/test_python.py +43 -0
67 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH 1/7 support/testing: create default test case for python packages
Hello Ricardo,
Overall, I find the patch series good (thanks for working on this!), but there is one thing that I dislike a bit see below.
Quoted 13 lines · Ricardo Martincoski
support/testing/tests/package/test_python.py +6
-0+ if self.__class__.__name__ != "TestPython2": + # default test_run for TestPythonPy2<Package> that inherits from this one + self.check_sample_scripts_exist() + self.run_sample_scripts() + return + self.version_test("Python 2") self.math_floor_test() self.libc_time_test()@@ -64,6 +100,13 @@ class TestPython3(TestPythonBase): def test_run(self): self.login()
support/testing/tests/package/test_python.py class TestPython3(TestPythonBase): · line 112
Quoted 10 lines
support/testing/tests/package/test_python.py +7
-0+ + if self.__class__.__name__ != "TestPython3": + # default test_run for TestPythonPy3<Package> that inherits from this one + self.check_sample_scripts_exist() + self.run_sample_scripts() + return + self.version_test("Python 3") self.math_floor_test() self.libc_time_test()
I don't really like that TestPython2 and TestPython3 are used for two entirely separate things:
- As a base class for testing individual Python packages
- As test cases for the Python interpreter itself
So here is the class hierarchy that I think would make more sense:
- TestPythonBase, defines the base configuration, login() method and that's it
- TestPythonBase2, that appends the configuration with BR2_PACKAGE_PYTHON=y
- TestPythonBase3, that appends the configuration with BR2_PACKAGE_PYTHON3=y
- TestPythonInterpreter, defines the version_test(), math_floor_test(), etc. methods that test the interpreter, and a test_run() method that calls all those tests. Indeed, calling those tests is currently repeated between Python2/Python3.
- TestPythonInterpreter2, which inherits from TestPythonInterpreter and TestPythonBase2
- TestPythonInterpreter3, which inherits from TestPythonInterpreter and TestPythonBase3
- TestPythonPackageBase, which has all the logic to copy sample scripts, run the scripts on the target, etc
- TestPythonAutobahn that defines the sample script to use, and inherits from TestPythonPackageBase
- TestPython2Autobahn that inherits from TestPythonAutobahn and TestPythonBase2
etc.
Does that make sense ? Basically, I'd like the test case of the interpreter to not be a "special case".
Does that make sense ?
Best regards,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH 1/7 support/testing: create default test case for python packages
Hello,
On Mon, Oct 22, 2018 at 04:55 AM, Thomas Petazzoni wrote:
[snip]
Quoted 50 lines
support/testing/tests/package/test_python.py class TestPython3(TestPythonBase): · line 112
Quoted 12 lines
def
test_run(self):
self.login()support/testing/tests/package/test_python.py +7
-0+ + if self.__class__.__name__ != "TestPython3": + # default test_run for TestPythonPy3<Package> that inherits from this one + self.check_sample_scripts_exist() + self.run_sample_scripts() + return + self.version_test("Python 3") self.math_floor_test() self.libc_time_test()I don't really like that TestPython2 and TestPython3 are used for two entirely separate things:
- As a base class for testing individual Python packages
- As test cases for the Python interpreter itselfSo here is the class hierarchy that I think would make more sense:
- TestPythonBase, defines the base configuration,
login()method and that's it- TestPythonBase2, that appends the configuration with BR2_PACKAGE_PYTHON=y
- TestPythonBase3, that appends the configuration with BR2_PACKAGE_PYTHON3=y
- TestPythonInterpreter, defines the
version_test(),math_floor_test(), etc. methods that test the interpreter, and atest_run()method that calls all those tests. Indeed, calling those tests is currently repeated between Python2/Python3.- TestPythonInterpreter2, which inherits from TestPythonInterpreter and TestPythonBase2
- TestPythonInterpreter3, which inherits from TestPythonInterpreter and TestPythonBase3
- TestPythonPackageBase, which has all the logic to copy sample scripts, run the scripts on the target, etc
- TestPythonAutobahn that defines the sample script to use, and inherits from TestPythonPackageBase
- TestPython2Autobahn that inherits from TestPythonAutobahn and TestPythonBase2
Quoted 6 lines
etc.
Does that make sense ? Basically, I'd like the test case of the interpreter to not be a "special case".
Does that make sense ?
Sure it does make sense. I will rework.
Thank you for your review.
I will set this series as Changes Requested.
Regards,
Ricardo
PATCH 2/7 support/testing: use default test_run for python-autobahn
In reply to Ricardo Martincoski--- .../tests/package/sample_python_autobahn.py | 1 + .../tests/package/test_python_autobahn.py | 31 +++++-------------- 2 files changed, 9 insertions(+), 23 deletions(-) create mode 100644 support/testing/tests/package/sample_python_autobahn.py
support/testing/tests/package/sample_python_autobahn.py new file +1 -0
@@ -0,0 +1 @@ +import autobahn.wamp # noqa
support/testing/tests/package/test_python_autobahn.py +8 -23
@@ -1,32 +1,17 @@ -from tests.package.test_python import TestPythonBase +import tests.package.test_python -class TestPythonAutobahn(TestPythonBase): - def import_test(self): - cmd = self.interpreter + " -c 'import autobahn.wamp'" - _, exit_code = self.emulator.run(cmd) - self.assertEqual(exit_code, 0) - - -class TestPythonPy2Autobahn(TestPythonAutobahn): - config = TestPythonBase.config + \ +class TestPythonAutobahn(): + config_package = \ """ - BR2_PACKAGE_PYTHON=y BR2_PACKAGE_PYTHON_AUTOBAHN=y """ + sample_scripts = ["tests/package/sample_python_autobahn.py"] - def test_run(self): - self.login() - self.import_test() +class TestPythonPy2Autobahn(TestPythonAutobahn, tests.package.test_python.TestPython2): + pass -class TestPythonPy3Autobahn(TestPythonAutobahn): - config = TestPythonBase.config + \ - """ - BR2_PACKAGE_PYTHON3=y - BR2_PACKAGE_PYTHON_AUTOBAHN=y - """ - def test_run(self): - self.login() - self.import_test() +class TestPythonPy3Autobahn(TestPythonAutobahn, tests.package.test_python.TestPython3): + pass
Signature · 1 line
2.17.1
PATCH 3/7 support/testing: use default test_run for python-cryptography
In reply to Ricardo Martincoski--- .../package/sample_python_cryptography.py | 3 ++ .../tests/package/test_python_cryptography.py | 34 +++++-------------- 2 files changed, 12 insertions(+), 25 deletions(-) create mode 100644 support/testing/tests/package/sample_python_cryptography.py
support/testing/tests/package/sample_python_cryptography.py new file +3 -0
@@ -0,0 +1,3 @@ +from cryptography.fernet import Fernet +key = Fernet.generate_key() +f = Fernet(key)
support/testing/tests/package/test_python_cryptography.py +9 -25
@@ -1,34 +1,18 @@ -from tests.package.test_python import TestPythonBase +import tests.package.test_python -class TestPythonCryptography(TestPythonBase): - def fernet_test(self, timeout=-1): - cmd = self.interpreter + " -c 'from cryptography.fernet import Fernet;" - cmd += "key = Fernet.generate_key();" - cmd += "f = Fernet(key)'" - _, exit_code = self.emulator.run(cmd, timeout) - self.assertEqual(exit_code, 0) - - -class TestPythonPy2Cryptography(TestPythonCryptography): - config = TestPythonBase.config + \ +class TestPythonCryptography(): + config_package = \ """ - BR2_PACKAGE_PYTHON=y BR2_PACKAGE_PYTHON_CRYPTOGRAPHY=y """ + sample_scripts = ["tests/package/sample_python_cryptography.py"] + timeout = 40 - def test_run(self): - self.login() - self.fernet_test(40) +class TestPythonPy2Cryptography(TestPythonCryptography, tests.package.test_python.TestPython2): + pass -class TestPythonPy3Cryptography(TestPythonCryptography): - config = TestPythonBase.config + \ - """ - BR2_PACKAGE_PYTHON3=y - BR2_PACKAGE_PYTHON_CRYPTOGRAPHY=y - """ - def test_run(self): - self.login() - self.fernet_test(40) +class TestPythonPy3Cryptography(TestPythonCryptography, tests.package.test_python.TestPython3): + pass
Signature · 1 line
2.17.1
PATCH 4/7 support/testing: use default test_run for python-incremental
In reply to Ricardo Martincoski--- .../package/sample_python_incremental.py | 3 ++ .../tests/package/test_python_incremental.py | 34 +++++-------------- 2 files changed, 12 insertions(+), 25 deletions(-) create mode 100644 support/testing/tests/package/sample_python_incremental.py
support/testing/tests/package/sample_python_incremental.py new file +3 -0
@@ -0,0 +1,3 @@ +import incremental +v = incremental.Version("package", 1, 2, 3, release_candidate=4) +assert(str(v) == "[package, version 1.2.3rc4]")
support/testing/tests/package/test_python_incremental.py +9 -25
@@ -1,34 +1,18 @@ -from tests.package.test_python import TestPythonBase +import tests.package.test_python -class TestPythonIncremental(TestPythonBase): - def str_test(self): - cmd = self.interpreter + " -c 'import incremental;" - cmd += "v = incremental.Version(\"package\", 1, 2, 3, release_candidate=4);" - cmd += "assert(str(v) == \"[package, version 1.2.3rc4]\")'" - _, exit_code = self.emulator.run(cmd, timeout=30) - self.assertEqual(exit_code, 0) - - -class TestPythonPy2Incremental(TestPythonIncremental): - config = TestPythonBase.config + \ +class TestPythonIncremental(): + config_package = \ """ - BR2_PACKAGE_PYTHON=y BR2_PACKAGE_PYTHON_INCREMENTAL=y """ + sample_scripts = ["tests/package/sample_python_incremental.py"] + timeout = 30 - def test_run(self): - self.login() - self.str_test() +class TestPythonPy2Incremental(TestPythonIncremental, tests.package.test_python.TestPython2): + pass -class TestPythonPy3Incremental(TestPythonIncremental): - config = TestPythonBase.config + \ - """ - BR2_PACKAGE_PYTHON3=y - BR2_PACKAGE_PYTHON_INCREMENTAL=y - """ - def test_run(self): - self.login() - self.str_test() +class TestPythonPy3Incremental(TestPythonIncremental, tests.package.test_python.TestPython3): + pass
Signature · 1 line
2.17.1
PATCH 5/7 support/testing: use default test_run for python-twisted
In reply to Ricardo Martincoski--- .../tests/package/sample_python_twisted.py | 9 ++++ .../tests/package/test_python_twisted.py | 45 ++++++------------- 2 files changed, 23 insertions(+), 31 deletions(-) create mode 100644 support/testing/tests/package/sample_python_twisted.py
support/testing/tests/package/sample_python_twisted.py new file +9 -0
@@ -0,0 +1,9 @@ +from twisted.internet import protocol, reactor, endpoints + + +class F(protocol.Factory): + pass + + +endpoints.serverFromString(reactor, "tcp:1234").listen(F()) +reactor.run()
support/testing/tests/package/test_python_twisted.py +14 -31
65 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH 6/7 support/testing: use default test_run for python-txaio
In reply to Ricardo Martincoski--- .../package/sample_python_txaio_asyncio.py | 3 ++ .../package/sample_python_txaio_twisted.py | 3 ++ .../tests/package/test_python_txaio.py | 30 +++++-------------- 3 files changed, 13 insertions(+), 23 deletions(-) create mode 100644 support/testing/tests/package/sample_python_txaio_asyncio.py create mode 100644 support/testing/tests/package/sample_python_txaio_twisted.py
support/testing/tests/package/sample_python_txaio_asyncio.py new file +3 -0
@@ -0,0 +1,3 @@ +import txaio +txaio.use_asyncio() +f0 = txaio.create_future()
support/testing/tests/package/sample_python_txaio_twisted.py new file +3 -0
@@ -0,0 +1,3 @@ +import txaio +txaio.use_twisted() +f0 = txaio.create_future()
support/testing/tests/package/test_python_txaio.py +7 -23
@@ -1,34 +1,18 @@ -from tests.package.test_python import TestPythonBase +import tests.package.test_python -class TestPythonPy2Txaio(TestPythonBase): - config = TestPythonBase.config + \ +class TestPythonPy2Txaio(tests.package.test_python.TestPython2): + config_package = \ """ - BR2_PACKAGE_PYTHON=y BR2_PACKAGE_PYTHON_TXAIO=y BR2_PACKAGE_PYTHON_TWISTED=y """ + sample_scripts = ["tests/package/sample_python_txaio_twisted.py"] - def test_run(self): - self.login() - cmd = self.interpreter + " -c 'import txaio;" - cmd += "txaio.use_twisted();" - cmd += "f0 = txaio.create_future()'" - _, exit_code = self.emulator.run(cmd) - self.assertEqual(exit_code, 0) - -class TestPythonPy3Txaio(TestPythonBase): - config = TestPythonBase.config + \ +class TestPythonPy3Txaio(tests.package.test_python.TestPython3): + config_package = \ """ - BR2_PACKAGE_PYTHON3=y BR2_PACKAGE_PYTHON_TXAIO=y """ - - def test_run(self): - self.login() - cmd = self.interpreter + " -c 'import txaio;" - cmd += "txaio.use_asyncio();" - cmd += "f0 = txaio.create_future()'" - _, exit_code = self.emulator.run(cmd) - self.assertEqual(exit_code, 0) + sample_scripts = ["tests/package/sample_python_txaio_asyncio.py"]
Signature · 1 line
2.17.1
PATCH 7/7 support/testing: use default test_run for python-txtorcon
In reply to Ricardo Martincoski--- .../tests/package/sample_python_txtorcon.py | 1 + .../tests/package/test_python_txtorcon.py | 32 ++++++------------- 2 files changed, 10 insertions(+), 23 deletions(-) create mode 100644 support/testing/tests/package/sample_python_txtorcon.py
support/testing/tests/package/sample_python_txtorcon.py new file +1 -0
@@ -0,0 +1 @@ +import txtorcon # noqa
support/testing/tests/package/test_python_txtorcon.py +9 -23
@@ -1,32 +1,18 @@ -from tests.package.test_python import TestPythonBase +import tests.package.test_python -class TestPythonTxtorcon(TestPythonBase): - def import_test(self): - cmd = self.interpreter + " -c 'import txtorcon'" - _, exit_code = self.emulator.run(cmd, timeout=30) - self.assertEqual(exit_code, 0) - - -class TestPythonPy2Txtorcon(TestPythonTxtorcon): - config = TestPythonBase.config + \ +class TestPythonTxtorcon(): + config_package = \ """ - BR2_PACKAGE_PYTHON=y BR2_PACKAGE_PYTHON_TXTORCON=y """ + sample_scripts = ["tests/package/sample_python_txtorcon.py"] + timeout = 30 - def test_run(self): - self.login() - self.import_test() +class TestPythonPy2Txtorcon(TestPythonTxtorcon, tests.package.test_python.TestPython2): + pass -class TestPythonPy3Txtorcon(TestPythonTxtorcon): - config = TestPythonBase.config + \ - """ - BR2_PACKAGE_PYTHON3=y - BR2_PACKAGE_PYTHON_TXTORCON=y - """ - def test_run(self): - self.login() - self.import_test() +class TestPythonPy3Txtorcon(TestPythonTxtorcon, tests.package.test_python.TestPython3): + pass
Signature · 1 line
2.17.1
v2 cover default runtime test case for python packages v2
In reply to Ricardo MartincoskiHello,
This series creates a standard runtime test case for python packages and is based on ideas from Thomas and Arnout.
The sample scripts that run on the target (fixture for the test case) are stored in a separate file (so it becomes more readable than inline in the test case). For the time being, they are stored on support/testing.
Those sample scripts are copied by the default test case to the target image in build time. The test infra builds the image, and the default test case logs in the target, checks the script is in the rootfs and calls the Python interpreter passing the sample script.
Patches 1 to 3 gradually prepare the current python test cases to receive the new class TestPythonPackageBase.
Patch 4 adds the new class with common logic to copy the sample script and execute it.
Patches 5 to 10 change one by one the 6 current test cases for python packages to use the new class.
Since the sample scripts are now stored on separate files they are automatically picked by the check-flake8 job on GitLab CI, so I formatted them to comply to flake8 (only empty lines or adding "# noqa" for the script that only import the module but don't use it).
Patch 11 use a new naming convention for test cases related to Python.
Patch 12 refreshes a test case that Yegor created for python-crossbar.
Here a complete run of runtime tests on Gitlab CI:
https://gitlab.com/RicardoMartincoski/buildroot/pipelines/35168102
Changes v1 -> v2:
- do not reuse TestPython2 and TestPython3 for two entirely separate things:
(Thomas)
- As a base class for testing individual Python packages;
- As test cases for the Python interpreter itself;
- use a better class hierarchy (Thomas). I did this in various patches (1, 2, 3, 4, 11) trying to make the review easier.
- with the new class hierarchy the trick that allows the defconfig to be changed by each test case without explicitly naming the class that contains the base defconfig is moved from every test case to the new base class;
- new patch to the series, resending a patch from Yegor;
Regards,
Ricardo
Ricardo Martincoski (11):
support/testing: use helper class in IPython test support/testing: use helper class in Python test support/testing: create intermediate class per Python version support/testing: create default test case for python packages support/testing: use TestPythonPackageBase for python-autobahn support/testing: use TestPythonPackageBase for python-cryptography support/testing: use TestPythonPackageBase for python-incremental support/testing: use TestPythonPackageBase for python-twisted support/testing: use TestPythonPackageBase for python-txaio support/testing: use TestPythonPackageBase for python-txtorcon support/testing: rename python* test cases
Yegor Yefremov (1):
testing: add python-crossbar tests
.gitlab-ci.yml | 29 +++---- .../package/copy-sample-script-to-target.sh | 7 ++ .../tests/package/sample_python_autobahn.py | 1 + .../package/sample_python_cryptography.py | 3 + .../package/sample_python_incremental.py | 3 + .../tests/package/sample_python_twisted.py | 9 +++ .../package/sample_python_txaio_asyncio.py | 3 + .../package/sample_python_txaio_twisted.py | 3 + .../tests/package/sample_python_txtorcon.py | 1 + support/testing/tests/package/test_ipython.py | 25 +++---- support/testing/tests/package/test_python.py | 75 +++++++++++++++---- .../tests/package/test_python_autobahn.py | 31 ++------ .../tests/package/test_python_crossbar.py | 20 +++++ .../tests/package/test_python_cryptography.py | 34 +++------ .../tests/package/test_python_incremental.py | 34 +++------ .../tests/package/test_python_twisted.py | 48 ++++-------- .../tests/package/test_python_txaio.py | 30 ++------ .../tests/package/test_python_txtorcon.py | 32 +++----- 18 files changed, 189 insertions(+), 199 deletions(-) create mode 100755 support/testing/tests/package/copy-sample-script-to-target.sh create mode 100644 support/testing/tests/package/sample_python_autobahn.py create mode 100644 support/testing/tests/package/sample_python_cryptography.py create mode 100644 support/testing/tests/package/sample_python_incremental.py create mode 100644 support/testing/tests/package/sample_python_twisted.py create mode 100644 support/testing/tests/package/sample_python_txaio_asyncio.py create mode 100644 support/testing/tests/package/sample_python_txaio_twisted.py create mode 100644 support/testing/tests/package/sample_python_txtorcon.py create mode 100644 support/testing/tests/package/test_python_crossbar.py
Signature · 1 line
2.17.1
PATCH v2 1/12 support/testing: use helper class in IPython test
Test cases for Python 2 and Python 3 are very similar.
Create a helper class named TestIPython to hold all commonalities. This new class is not a subclass of unittest.TestCase and therefore nose2 ignores it, avoiding to create a bogus test case.
---
Changes v1 -> v2:
- new patch to gradually prepare the current python test cases to receive the new class TestPythonPackageBase (see review of
http://patchwork.ozlabs.org/patch/984425/);
--- support/testing/tests/package/test_ipython.py | 23 ++++++++----------- 1 file changed, 10 insertions(+), 13 deletions(-)
support/testing/tests/package/test_ipython.py +10 -13
@@ -8,12 +8,7 @@ from tests.package.test_python import TestPythonBase # does, so this test ends up being a false-negative -class TestIPythonPy2(TestPythonBase): - config = TestPythonBase.config + \ - """ - BR2_PACKAGE_PYTHON=y - BR2_PACKAGE_PYTHON_IPYTHON=y - """ +class TestIPython(): interpreter = "ipython" def test_run(self):
@@ -22,15 +17,17 @@ class TestIPythonPy2(TestPythonBase): self.libc_time_test(40) -class TestIPythonPy3(TestPythonBase): +class TestIPythonPy2(TestIPython, TestPythonBase): config = TestPythonBase.config + \ """ - BR2_PACKAGE_PYTHON3=y + BR2_PACKAGE_PYTHON=y BR2_PACKAGE_PYTHON_IPYTHON=y """ - interpreter = "ipython" - def test_run(self): - self.login() - self.math_floor_test(40) - self.libc_time_test(40) + +class TestIPythonPy3(TestIPython, TestPythonBase): + config = TestPythonBase.config + \ + """ + BR2_PACKAGE_PYTHON3=y + BR2_PACKAGE_PYTHON_IPYTHON=y + """
Signature · 1 line
2.17.1
PATCH v2 2/12 support/testing: use helper class in Python test
In reply to Ricardo MartincoskiTest cases for Python 2 and Python 3 are very similar.
Create a helper class named TestPythonInterpreter to hold all commonalities. This new class is not a subclass of unittest.TestCase and therefore nose2 ignores it, avoiding to create a bogus test case.
---
Changes v1 -> v2:
- new patch to gradually prepare the current python test cases to receive the new class TestPythonPackageBase (see review of
http://patchwork.ozlabs.org/patch/984425/);
--- support/testing/tests/package/test_ipython.py | 4 +-- support/testing/tests/package/test_python.py | 31 +++++++++---------- 2 files changed, 17 insertions(+), 18 deletions(-)
support/testing/tests/package/test_ipython.py +2 -2
@@ -1,4 +1,4 @@ -from tests.package.test_python import TestPythonBase +from tests.package.test_python import TestPythonInterpreter, TestPythonBase # # The following pythong tests are not being used here: #
@@ -8,7 +8,7 @@ from tests.package.test_python import TestPythonBase # does, so this test ends up being a false-negative -class TestIPython(): +class TestIPython(TestPythonInterpreter): interpreter = "ipython" def test_run(self):
support/testing/tests/package/test_python.py +15 -16
@@ -18,6 +18,10 @@ class TestPythonBase(infra.basetest.BRTest): options=["-initrd", cpio_file]) self.emulator.login() + +class TestPythonInterpreter(): + version_string = None + def version_test(self, version, timeout=-1): cmd = self.interpreter + " --version 2>&1 | grep '^{}'".format(version) _, exit_code = self.emulator.run(cmd, timeout)
@@ -41,30 +45,25 @@ class TestPythonBase(infra.basetest.BRTest): _, exit_code = self.emulator.run(cmd, timeout) self.assertEqual(exit_code, 1) - -class TestPython2(TestPythonBase): - config = TestPythonBase.config + \ - """ - BR2_PACKAGE_PYTHON=y - """ - def test_run(self): self.login() - self.version_test("Python 2") + self.version_test(self.version_string) self.math_floor_test() self.libc_time_test() self.zlib_test() -class TestPython3(TestPythonBase): +class TestPython2(TestPythonInterpreter, TestPythonBase): config = TestPythonBase.config + \ """ - BR2_PACKAGE_PYTHON3=y + BR2_PACKAGE_PYTHON=y """ + version_string = "Python 2" - def test_run(self): - self.login() - self.version_test("Python 3") - self.math_floor_test() - self.libc_time_test() - self.zlib_test() + +class TestPython3(TestPythonInterpreter, TestPythonBase): + config = TestPythonBase.config + \ + """ + BR2_PACKAGE_PYTHON3=y + """ + version_string = "Python 3"
Signature · 1 line
2.17.1
PATCH v2 3/12 support/testing: create intermediate class per Python version
In reply to Ricardo MartincoskiCurrently all test cases for python packages add BR2_PACKAGE_PYTHON=y or BR2_PACKAGE_PYTHON3=y by their own.
Move those configs to two intermediate classes named TestPythonBase2 and TestPythonBase3, and change all test cases that need those configs to get them through the new classes.
A few classes still need to inherit directly from TestPythonBase.
---
Changes v1 -> v2:
- new patch to gradually prepare the current python test cases to receive the new class TestPythonPackageBase (see review of
http://patchwork.ozlabs.org/patch/984425/);
--- support/testing/tests/package/test_ipython.py | 12 ++++----- support/testing/tests/package/test_python.py | 26 ++++++++++++------- .../tests/package/test_python_autobahn.py | 12 ++++----- .../tests/package/test_python_cryptography.py | 12 ++++----- .../tests/package/test_python_incremental.py | 12 ++++----- .../tests/package/test_python_twisted.py | 12 ++++----- .../tests/package/test_python_txaio.py | 12 ++++----- .../tests/package/test_python_txtorcon.py | 12 ++++----- 8 files changed, 51 insertions(+), 59 deletions(-)
support/testing/tests/package/test_ipython.py +5 -7
@@ -1,4 +1,4 @@ -from tests.package.test_python import TestPythonInterpreter, TestPythonBase +from tests.package.test_python import TestPythonInterpreter, TestPythonBase2, TestPythonBase3 # # The following pythong tests are not being used here: #
@@ -17,17 +17,15 @@ class TestIPython(TestPythonInterpreter): self.libc_time_test(40) -class TestIPythonPy2(TestIPython, TestPythonBase): - config = TestPythonBase.config + \ +class TestIPythonPy2(TestIPython, TestPythonBase2): + config = TestPythonBase2.config + \ """ - BR2_PACKAGE_PYTHON=y BR2_PACKAGE_PYTHON_IPYTHON=y """ -class TestIPythonPy3(TestIPython, TestPythonBase): - config = TestPythonBase.config + \ +class TestIPythonPy3(TestIPython, TestPythonBase3): + config = TestPythonBase3.config + \ """ - BR2_PACKAGE_PYTHON3=y BR2_PACKAGE_PYTHON_IPYTHON=y """
support/testing/tests/package/test_python.py +16 -10
@@ -19,6 +19,20 @@ class TestPythonBase(infra.basetest.BRTest): self.emulator.login() +class TestPythonBase2(TestPythonBase): + config = TestPythonBase.config + \ + """ + BR2_PACKAGE_PYTHON=y + """ + + +class TestPythonBase3(TestPythonBase): + config = TestPythonBase.config + \ + """ + BR2_PACKAGE_PYTHON3=y + """ + + class TestPythonInterpreter(): version_string = None
@@ -53,17 +67,9 @@ class TestPythonInterpreter(): self.zlib_test() -class TestPython2(TestPythonInterpreter, TestPythonBase): - config = TestPythonBase.config + \ - """ - BR2_PACKAGE_PYTHON=y - """ +class TestPython2(TestPythonInterpreter, TestPythonBase2): version_string = "Python 2" -class TestPython3(TestPythonInterpreter, TestPythonBase): - config = TestPythonBase.config + \ - """ - BR2_PACKAGE_PYTHON3=y - """ +class TestPython3(TestPythonInterpreter, TestPythonBase3): version_string = "Python 3"
support/testing/tests/package/test_python_autobahn.py +5 -7
@@ -1,4 +1,4 @@ -from tests.package.test_python import TestPythonBase +from tests.package.test_python import TestPythonBase, TestPythonBase2, TestPythonBase3 class TestPythonAutobahn(TestPythonBase):
@@ -8,10 +8,9 @@ class TestPythonAutobahn(TestPythonBase): self.assertEqual(exit_code, 0) -class TestPythonPy2Autobahn(TestPythonAutobahn): - config = TestPythonBase.config + \ +class TestPythonPy2Autobahn(TestPythonAutobahn, TestPythonBase2): + config = TestPythonBase2.config + \ """ - BR2_PACKAGE_PYTHON=y BR2_PACKAGE_PYTHON_AUTOBAHN=y """
@@ -20,10 +19,9 @@ class TestPythonPy2Autobahn(TestPythonAutobahn): self.import_test() -class TestPythonPy3Autobahn(TestPythonAutobahn): - config = TestPythonBase.config + \ +class TestPythonPy3Autobahn(TestPythonAutobahn, TestPythonBase3): + config = TestPythonBase3.config + \ """ - BR2_PACKAGE_PYTHON3=y BR2_PACKAGE_PYTHON_AUTOBAHN=y """
support/testing/tests/package/test_python_cryptography.py +5 -7
@@ -1,4 +1,4 @@ -from tests.package.test_python import TestPythonBase +from tests.package.test_python import TestPythonBase, TestPythonBase2, TestPythonBase3 class TestPythonCryptography(TestPythonBase):
@@ -10,10 +10,9 @@ class TestPythonCryptography(TestPythonBase): self.assertEqual(exit_code, 0) -class TestPythonPy2Cryptography(TestPythonCryptography): - config = TestPythonBase.config + \ +class TestPythonPy2Cryptography(TestPythonCryptography, TestPythonBase2): + config = TestPythonBase2.config + \ """ - BR2_PACKAGE_PYTHON=y BR2_PACKAGE_PYTHON_CRYPTOGRAPHY=y """
@@ -22,10 +21,9 @@ class TestPythonPy2Cryptography(TestPythonCryptography): self.fernet_test(40) -class TestPythonPy3Cryptography(TestPythonCryptography): - config = TestPythonBase.config + \ +class TestPythonPy3Cryptography(TestPythonCryptography, TestPythonBase3): + config = TestPythonBase3.config + \ """ - BR2_PACKAGE_PYTHON3=y BR2_PACKAGE_PYTHON_CRYPTOGRAPHY=y """
support/testing/tests/package/test_python_incremental.py +5 -7
@@ -1,4 +1,4 @@ -from tests.package.test_python import TestPythonBase +from tests.package.test_python import TestPythonBase, TestPythonBase2, TestPythonBase3 class TestPythonIncremental(TestPythonBase):
@@ -10,10 +10,9 @@ class TestPythonIncremental(TestPythonBase): self.assertEqual(exit_code, 0) -class TestPythonPy2Incremental(TestPythonIncremental): - config = TestPythonBase.config + \ +class TestPythonPy2Incremental(TestPythonIncremental, TestPythonBase2): + config = TestPythonBase2.config + \ """ - BR2_PACKAGE_PYTHON=y BR2_PACKAGE_PYTHON_INCREMENTAL=y """
@@ -22,10 +21,9 @@ class TestPythonPy2Incremental(TestPythonIncremental): self.str_test() -class TestPythonPy3Incremental(TestPythonIncremental): - config = TestPythonBase.config + \ +class TestPythonPy3Incremental(TestPythonIncremental, TestPythonBase3): + config = TestPythonBase3.config + \ """ - BR2_PACKAGE_PYTHON3=y BR2_PACKAGE_PYTHON_INCREMENTAL=y """
support/testing/tests/package/test_python_twisted.py +5 -7
@@ -1,4 +1,4 @@ -from tests.package.test_python import TestPythonBase +from tests.package.test_python import TestPythonBase, TestPythonBase2, TestPythonBase3 TEST_SCRIPT = """ from twisted.internet import protocol, reactor, endpoints
@@ -30,10 +30,9 @@ class TestPythonTwisted(TestPythonBase): self.assertEqual(exit_code, 0) -class TestPythonPy2Twisted(TestPythonTwisted): - config = TestPythonBase.config + \ +class TestPythonPy2Twisted(TestPythonTwisted, TestPythonBase2): + config = TestPythonBase2.config + \ """ - BR2_PACKAGE_PYTHON=y BR2_PACKAGE_PYTHON_TWISTED=y """
@@ -42,10 +41,9 @@ class TestPythonPy2Twisted(TestPythonTwisted): self.import_test() -class TestPythonPy3Twisted(TestPythonTwisted): - config = TestPythonBase.config + \ +class TestPythonPy3Twisted(TestPythonTwisted, TestPythonBase3): + config = TestPythonBase3.config + \ """ - BR2_PACKAGE_PYTHON3=y BR2_PACKAGE_PYTHON_TWISTED=y """
support/testing/tests/package/test_python_txaio.py +5 -7
@@ -1,10 +1,9 @@ -from tests.package.test_python import TestPythonBase +from tests.package.test_python import TestPythonBase2, TestPythonBase3 -class TestPythonPy2Txaio(TestPythonBase): - config = TestPythonBase.config + \ +class TestPythonPy2Txaio(TestPythonBase2): + config = TestPythonBase2.config + \ """ - BR2_PACKAGE_PYTHON=y BR2_PACKAGE_PYTHON_TXAIO=y BR2_PACKAGE_PYTHON_TWISTED=y """
@@ -18,10 +17,9 @@ class TestPythonPy2Txaio(TestPythonBase): self.assertEqual(exit_code, 0) -class TestPythonPy3Txaio(TestPythonBase): - config = TestPythonBase.config + \ +class TestPythonPy3Txaio(TestPythonBase3): + config = TestPythonBase3.config + \ """ - BR2_PACKAGE_PYTHON3=y BR2_PACKAGE_PYTHON_TXAIO=y """
support/testing/tests/package/test_python_txtorcon.py +5 -7
@@ -1,4 +1,4 @@ -from tests.package.test_python import TestPythonBase +from tests.package.test_python import TestPythonBase, TestPythonBase2, TestPythonBase3 class TestPythonTxtorcon(TestPythonBase):
@@ -8,10 +8,9 @@ class TestPythonTxtorcon(TestPythonBase): self.assertEqual(exit_code, 0) -class TestPythonPy2Txtorcon(TestPythonTxtorcon): - config = TestPythonBase.config + \ +class TestPythonPy2Txtorcon(TestPythonTxtorcon, TestPythonBase2): + config = TestPythonBase2.config + \ """ - BR2_PACKAGE_PYTHON=y BR2_PACKAGE_PYTHON_TXTORCON=y """
@@ -20,10 +19,9 @@ class TestPythonPy2Txtorcon(TestPythonTxtorcon): self.import_test() -class TestPythonPy3Txtorcon(TestPythonTxtorcon): - config = TestPythonBase.config + \ +class TestPythonPy3Txtorcon(TestPythonTxtorcon, TestPythonBase3): + config = TestPythonBase3.config + \ """ - BR2_PACKAGE_PYTHON3=y BR2_PACKAGE_PYTHON_TXTORCON=y """
Signature · 1 line
2.17.1
PATCH v2 4/12 support/testing: create default test case for python packages
In reply to Ricardo MartincoskiTest cases for python packages are very similar among each other: run a simple script in the target that minimally tests the package. So create a new helper class named TestPythonPackageBase that holds all the logic to run a script on the target. This new class is not a subclass of unittest.TestCase and therefore nose2 ignores it, avoiding to create a bogus test case.
TestPythonPackageBase adds in build time one or more sample scripts to be run on the target. The test case for the python package must explicitly list them in the "sample_scripts" property. The test case then automatically logins to the target, checks the scripts are really in the rootfs (it calls "md5sum" instead of "ls" or "test" in an attempt to make the logfile more friendly, since someone analysing a failure can easily check the expected script was executed) and then calls the python interpreter passing the sample script as parameter.
An optional property "timeout" exists for the case the sample script needs more time to run than the default timeout from the test infra (currently 5 seconds).
A simple test case for a package that only supports Python 2 will look like this:
|from tests.package.test_python import TestPythonPackageBase, TestPythonBase2 | | |class TestPython2<Package>(TestPythonPackageBase, TestPythonBase2): | config_package = \ | """ | BR2_PACKAGE_PYTHON_<PACKAGE>=y | """ | sample_scripts = ["tests/package/sample_python_<package>.py"] | timeout = 15
When the python package supports both Python 2 and Python 3 a helper class can be used to hold the common properties:
|class TestPython<Package>(TestPythonPackageBase):
| config_package = \
...
|class TestPythonPy2<Package>(TestPython<Package>, TestPythonBase2):
...
|class TestPythonPy3<Package>(TestPython<Package>, TestPythonBase3):
A trick is used in order to allow this new class to change the defconfig used in the build of the image for the testcase: this class' __init__ method calls the __init__ method from a parent class that in turn inherits from unittest.TestCase. This would be a normal usage ... if this class actually inherited from another class!
This is done to make nose2 to ignore this class when looking for test cases. And it works because all classes that inherit from it also are subclasses of unittest.TestCase.
An alternative solution to this would be to create yet another help class in test_python.py that doesn't inherit from unittest.TestCase to hold only the test_run method and make every test case for python package (not the helper class, i.e. TestPythonArgh) to inherit from this class too.
---
NOTE: example of the alternative solution without the mentioned trick:
https://gitlab.com/RicardoMartincoski/buildroot/commit/ea7177c7a233a32129230189f609ecd36853d7e2
Changes v1 -> v2:
- do not reuse TestPython2 and TestPython3 to for two entirely separate things: (Thomas)
- As a base class for testing individual Python packages;
- As test cases for the Python interpreter itself;
- use a better class hierarchy (Thomas). I did this in various patches (before and after this one) trying to make the review easier.
- with the new class hierarchy the trick that allows the defconfig to be changed by each test case without explicitly naming the class that contains the base defconfig is moved from every test case to the new base class.
v1: http://patchwork.ozlabs.org/patch/984425/
--- .../package/copy-sample-script-to-target.sh | 7 ++++ support/testing/tests/package/test_python.py | 38 +++++++++++++++++++ 2 files changed, 45 insertions(+) create mode 100755 support/testing/tests/package/copy-sample-script-to-target.sh
support/testing/tests/package/copy-sample-script-to-target.sh new file +7 -0
@@ -0,0 +1,7 @@ +#!/bin/sh +set -e + +shift +for file in "$@"; do + cp -f "${file}" "${TARGET_DIR}/root/" +done
support/testing/tests/package/test_python.py +38 -0
@@ -73,3 +73,41 @@ class TestPython2(TestPythonInterpreter, TestPythonBase2): class TestPython3(TestPythonInterpreter, TestPythonBase3): version_string = "Python 3" + + +class TestPythonPackageBase(): + config_package = None + config_sample_scripts = \ + """ + BR2_ROOTFS_POST_BUILD_SCRIPT="{}" + BR2_ROOTFS_POST_SCRIPT_ARGS="{}" + """.format(infra.filepath("tests/package/copy-sample-script-to-target.sh"), + "{sample_scripts}") + sample_scripts = None + timeout = -1 + + def __init__(self, names): + # every class that inherits from this one will inherit from TestPythonBase too + super(TestPythonBase, self).__init__(names) + if self.config_package: + self.config += self.config_package + if self.sample_scripts: + scripts = [infra.filepath(s) for s in self.sample_scripts] + self.config += self.config_sample_scripts.format(sample_scripts=" ".join(scripts)) + + def check_sample_scripts_exist(self): + scripts = [os.path.basename(s) for s in self.sample_scripts] + cmd = "md5sum " + " ".join(scripts) + _, exit_code = self.emulator.run(cmd) + self.assertEqual(exit_code, 0) + + def run_sample_scripts(self): + for script in self.sample_scripts: + cmd = self.interpreter + " " + os.path.basename(script) + _, exit_code = self.emulator.run(cmd, timeout=self.timeout) + self.assertEqual(exit_code, 0) + + def test_run(self): + self.login() + self.check_sample_scripts_exist() + self.run_sample_scripts()
Signature · 1 line
2.17.1
PATCH v2 5/12 support/testing: use TestPythonPackageBase for python-autobahn
In reply to Ricardo MartincoskiMove the test script to be run on the target from inline in the test case to a separate file.
---
Changes v1 -> v2:
- refresh after creating TestPythonPackageBase;
--- .../tests/package/sample_python_autobahn.py | 1 + .../tests/package/test_python_autobahn.py | 29 +++++-------------- 2 files changed, 9 insertions(+), 21 deletions(-) create mode 100644 support/testing/tests/package/sample_python_autobahn.py
support/testing/tests/package/sample_python_autobahn.py new file +1 -0
@@ -0,0 +1 @@ +import autobahn.wamp # noqa
support/testing/tests/package/test_python_autobahn.py +8 -21
@@ -1,30 +1,17 @@ -from tests.package.test_python import TestPythonBase, TestPythonBase2, TestPythonBase3 +from tests.package.test_python import TestPythonPackageBase, TestPythonBase2, TestPythonBase3 -class TestPythonAutobahn(TestPythonBase): - def import_test(self): - cmd = self.interpreter + " -c 'import autobahn.wamp'" - _, exit_code = self.emulator.run(cmd) - self.assertEqual(exit_code, 0) - - -class TestPythonPy2Autobahn(TestPythonAutobahn, TestPythonBase2): - config = TestPythonBase2.config + \ +class TestPythonAutobahn(TestPythonPackageBase): + config_package = \ """ BR2_PACKAGE_PYTHON_AUTOBAHN=y """ + sample_scripts = ["tests/package/sample_python_autobahn.py"] - def test_run(self): - self.login() - self.import_test() +class TestPythonPy2Autobahn(TestPythonAutobahn, TestPythonBase2): + pass -class TestPythonPy3Autobahn(TestPythonAutobahn, TestPythonBase3): - config = TestPythonBase3.config + \ - """ - BR2_PACKAGE_PYTHON_AUTOBAHN=y - """ - def test_run(self): - self.login() - self.import_test() +class TestPythonPy3Autobahn(TestPythonAutobahn, TestPythonBase3): + pass
Signature · 1 line
2.17.1
PATCH v2 6/12 support/testing: use TestPythonPackageBase for python-cryptography
In reply to Ricardo MartincoskiMove the test script to be run on the target from inline in the test case to a separate file.
---
Changes v1 -> v2:
- refresh after creating TestPythonPackageBase;
--- .../package/sample_python_cryptography.py | 3 ++ .../tests/package/test_python_cryptography.py | 32 ++++++------------- 2 files changed, 12 insertions(+), 23 deletions(-) create mode 100644 support/testing/tests/package/sample_python_cryptography.py
support/testing/tests/package/sample_python_cryptography.py new file +3 -0
@@ -0,0 +1,3 @@ +from cryptography.fernet import Fernet +key = Fernet.generate_key() +f = Fernet(key)
support/testing/tests/package/test_python_cryptography.py +9 -23
@@ -1,32 +1,18 @@ -from tests.package.test_python import TestPythonBase, TestPythonBase2, TestPythonBase3 +from tests.package.test_python import TestPythonPackageBase, TestPythonBase2, TestPythonBase3 -class TestPythonCryptography(TestPythonBase): - def fernet_test(self, timeout=-1): - cmd = self.interpreter + " -c 'from cryptography.fernet import Fernet;" - cmd += "key = Fernet.generate_key();" - cmd += "f = Fernet(key)'" - _, exit_code = self.emulator.run(cmd, timeout) - self.assertEqual(exit_code, 0) - - -class TestPythonPy2Cryptography(TestPythonCryptography, TestPythonBase2): - config = TestPythonBase2.config + \ +class TestPythonCryptography(TestPythonPackageBase): + config_package = \ """ BR2_PACKAGE_PYTHON_CRYPTOGRAPHY=y """ + sample_scripts = ["tests/package/sample_python_cryptography.py"] + timeout = 40 - def test_run(self): - self.login() - self.fernet_test(40) +class TestPythonPy2Cryptography(TestPythonCryptography, TestPythonBase2): + pass -class TestPythonPy3Cryptography(TestPythonCryptography, TestPythonBase3): - config = TestPythonBase3.config + \ - """ - BR2_PACKAGE_PYTHON_CRYPTOGRAPHY=y - """ - def test_run(self): - self.login() - self.fernet_test(40) +class TestPythonPy3Cryptography(TestPythonCryptography, TestPythonBase3): + pass
Signature · 1 line
2.17.1
PATCH v2 7/12 support/testing: use TestPythonPackageBase for python-incremental
In reply to Ricardo MartincoskiMove the test script to be run on the target from inline in the test case to a separate file.
---
Changes v1 -> v2:
- refresh after creating TestPythonPackageBase;
--- .../package/sample_python_incremental.py | 3 ++ .../tests/package/test_python_incremental.py | 32 ++++++------------- 2 files changed, 12 insertions(+), 23 deletions(-) create mode 100644 support/testing/tests/package/sample_python_incremental.py
support/testing/tests/package/sample_python_incremental.py new file +3 -0
@@ -0,0 +1,3 @@ +import incremental +v = incremental.Version("package", 1, 2, 3, release_candidate=4) +assert(str(v) == "[package, version 1.2.3rc4]")
support/testing/tests/package/test_python_incremental.py +9 -23
@@ -1,32 +1,18 @@ -from tests.package.test_python import TestPythonBase, TestPythonBase2, TestPythonBase3 +from tests.package.test_python import TestPythonPackageBase, TestPythonBase2, TestPythonBase3 -class TestPythonIncremental(TestPythonBase): - def str_test(self): - cmd = self.interpreter + " -c 'import incremental;" - cmd += "v = incremental.Version(\"package\", 1, 2, 3, release_candidate=4);" - cmd += "assert(str(v) == \"[package, version 1.2.3rc4]\")'" - _, exit_code = self.emulator.run(cmd, timeout=30) - self.assertEqual(exit_code, 0) - - -class TestPythonPy2Incremental(TestPythonIncremental, TestPythonBase2): - config = TestPythonBase2.config + \ +class TestPythonIncremental(TestPythonPackageBase): + config_package = \ """ BR2_PACKAGE_PYTHON_INCREMENTAL=y """ + sample_scripts = ["tests/package/sample_python_incremental.py"] + timeout = 30 - def test_run(self): - self.login() - self.str_test() +class TestPythonPy2Incremental(TestPythonIncremental, TestPythonBase2): + pass -class TestPythonPy3Incremental(TestPythonIncremental, TestPythonBase3): - config = TestPythonBase3.config + \ - """ - BR2_PACKAGE_PYTHON_INCREMENTAL=y - """ - def test_run(self): - self.login() - self.str_test() +class TestPythonPy3Incremental(TestPythonIncremental, TestPythonBase3): + pass
Signature · 1 line
2.17.1
PATCH v2 8/12 support/testing: use TestPythonPackageBase for python-twisted
In reply to Ricardo MartincoskiMove the test script to be run on the target from inline in the test case to a separate file.
---
Changes v1 -> v2:
- refresh after creating TestPythonPackageBase;
- override run_sample_scripts instead of test_run;
--- .../tests/package/sample_python_twisted.py | 9 ++++ .../tests/package/test_python_twisted.py | 42 +++++-------------- 2 files changed, 20 insertions(+), 31 deletions(-) create mode 100644 support/testing/tests/package/sample_python_twisted.py
support/testing/tests/package/sample_python_twisted.py new file +9 -0
@@ -0,0 +1,9 @@ +from twisted.internet import protocol, reactor, endpoints + + +class F(protocol.Factory): + pass + + +endpoints.serverFromString(reactor, "tcp:1234").listen(F()) +reactor.run()
support/testing/tests/package/test_python_twisted.py +11 -31
58 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH v2 9/12 support/testing: use TestPythonPackageBase for python-txaio
In reply to Ricardo MartincoskiMove the test scripts to be run on the target from inline in the test case to a separate file.
---
Changes v1 -> v2:
- refresh after creating TestPythonPackageBase;
--- .../package/sample_python_txaio_asyncio.py | 3 ++ .../package/sample_python_txaio_twisted.py | 3 ++ .../tests/package/test_python_txaio.py | 28 +++++-------------- 3 files changed, 13 insertions(+), 21 deletions(-) create mode 100644 support/testing/tests/package/sample_python_txaio_asyncio.py create mode 100644 support/testing/tests/package/sample_python_txaio_twisted.py
support/testing/tests/package/sample_python_txaio_asyncio.py new file +3 -0
@@ -0,0 +1,3 @@ +import txaio +txaio.use_asyncio() +f0 = txaio.create_future()
support/testing/tests/package/sample_python_txaio_twisted.py new file +3 -0
@@ -0,0 +1,3 @@ +import txaio +txaio.use_twisted() +f0 = txaio.create_future()
support/testing/tests/package/test_python_txaio.py +7 -21
@@ -1,32 +1,18 @@ -from tests.package.test_python import TestPythonBase2, TestPythonBase3 +from tests.package.test_python import TestPythonPackageBase, TestPythonBase2, TestPythonBase3 -class TestPythonPy2Txaio(TestPythonBase2): - config = TestPythonBase2.config + \ +class TestPythonPy2Txaio(TestPythonPackageBase, TestPythonBase2): + config_package = \ """ BR2_PACKAGE_PYTHON_TXAIO=y BR2_PACKAGE_PYTHON_TWISTED=y """ + sample_scripts = ["tests/package/sample_python_txaio_twisted.py"] - def test_run(self): - self.login() - cmd = self.interpreter + " -c 'import txaio;" - cmd += "txaio.use_twisted();" - cmd += "f0 = txaio.create_future()'" - _, exit_code = self.emulator.run(cmd) - self.assertEqual(exit_code, 0) - -class TestPythonPy3Txaio(TestPythonBase3): - config = TestPythonBase3.config + \ +class TestPythonPy3Txaio(TestPythonPackageBase, TestPythonBase3): + config_package = \ """ BR2_PACKAGE_PYTHON_TXAIO=y """ - - def test_run(self): - self.login() - cmd = self.interpreter + " -c 'import txaio;" - cmd += "txaio.use_asyncio();" - cmd += "f0 = txaio.create_future()'" - _, exit_code = self.emulator.run(cmd) - self.assertEqual(exit_code, 0) + sample_scripts = ["tests/package/sample_python_txaio_asyncio.py"]
Signature · 1 line
2.17.1
PATCH v2 10/12 support/testing: use TestPythonPackageBase for python-txtorcon
In reply to Ricardo MartincoskiMove the test script to be run on the target from inline in the test case to a separate file.
---
Changes v1 -> v2:
- refresh after creating TestPythonPackageBase;
--- .../tests/package/sample_python_txtorcon.py | 1 + .../tests/package/test_python_txtorcon.py | 30 ++++++------------- 2 files changed, 10 insertions(+), 21 deletions(-) create mode 100644 support/testing/tests/package/sample_python_txtorcon.py
support/testing/tests/package/sample_python_txtorcon.py new file +1 -0
@@ -0,0 +1 @@ +import txtorcon # noqa
support/testing/tests/package/test_python_txtorcon.py +9 -21
@@ -1,30 +1,18 @@ -from tests.package.test_python import TestPythonBase, TestPythonBase2, TestPythonBase3 +from tests.package.test_python import TestPythonPackageBase, TestPythonBase2, TestPythonBase3 -class TestPythonTxtorcon(TestPythonBase): - def import_test(self): - cmd = self.interpreter + " -c 'import txtorcon'" - _, exit_code = self.emulator.run(cmd, timeout=30) - self.assertEqual(exit_code, 0) - - -class TestPythonPy2Txtorcon(TestPythonTxtorcon, TestPythonBase2): - config = TestPythonBase2.config + \ +class TestPythonTxtorcon(TestPythonPackageBase): + config_package = \ """ BR2_PACKAGE_PYTHON_TXTORCON=y """ + sample_scripts = ["tests/package/sample_python_txtorcon.py"] + timeout = 30 - def test_run(self): - self.login() - self.import_test() +class TestPythonPy2Txtorcon(TestPythonTxtorcon, TestPythonBase2): + pass -class TestPythonPy3Txtorcon(TestPythonTxtorcon, TestPythonBase3): - config = TestPythonBase3.config + \ - """ - BR2_PACKAGE_PYTHON_TXTORCON=y - """ - def test_run(self): - self.login() - self.import_test() +class TestPythonPy3Txtorcon(TestPythonTxtorcon, TestPythonBase3): + pass
Signature · 1 line
2.17.1
PATCH v2 11/12 support/testing: rename python* test cases
In reply to Ricardo MartincoskiTestPython[23] actually test the interpreter, so rename them to TestPythonInterpreter[23].
There is no need to repeat "PythonPy" for test cases that test python packages, so rename TestPythonPy[23]Autobahn to TestPython[23]Autobahn and do the same for the other test cases.
---
Changes v1 -> v2:
- new patch (see review of http://patchwork.ozlabs.org/patch/984425/);
--- .gitlab-ci.yml | 28 +++++++++---------- support/testing/tests/package/test_python.py | 4 +-- .../tests/package/test_python_autobahn.py | 4 +-- .../tests/package/test_python_cryptography.py | 4 +-- .../tests/package/test_python_incremental.py | 4 +-- .../tests/package/test_python_twisted.py | 4 +-- .../tests/package/test_python_txaio.py | 4 +-- .../tests/package/test_python_txtorcon.py | 4 +-- 8 files changed, 28 insertions(+), 28 deletions(-)
.gitlab-ci.yml +14 -14
@@ -311,20 +311,20 @@ tests.init.test_systemd.TestInitSystemSystemdRwNetworkd: *runtime_test tests.package.test_dropbear.TestDropbear: *runtime_test tests.package.test_ipython.TestIPythonPy2: *runtime_test tests.package.test_ipython.TestIPythonPy3: *runtime_test -tests.package.test_python.TestPython2: *runtime_test -tests.package.test_python.TestPython3: *runtime_test -tests.package.test_python_autobahn.TestPythonPy2Autobahn: *runtime_test -tests.package.test_python_autobahn.TestPythonPy3Autobahn: *runtime_test -tests.package.test_python_cryptography.TestPythonPy2Cryptography: *runtime_test -tests.package.test_python_cryptography.TestPythonPy3Cryptography: *runtime_test -tests.package.test_python_incremental.TestPythonPy2Incremental: *runtime_test -tests.package.test_python_incremental.TestPythonPy3Incremental: *runtime_test -tests.package.test_python_twisted.TestPythonPy2Twisted: *runtime_test -tests.package.test_python_twisted.TestPythonPy3Twisted: *runtime_test -tests.package.test_python_txaio.TestPythonPy2Txaio: *runtime_test -tests.package.test_python_txaio.TestPythonPy3Txaio: *runtime_test -tests.package.test_python_txtorcon.TestPythonPy2Txtorcon: *runtime_test -tests.package.test_python_txtorcon.TestPythonPy3Txtorcon: *runtime_test +tests.package.test_python.TestPythonInterpreter2: *runtime_test +tests.package.test_python.TestPythonInterpreter3: *runtime_test +tests.package.test_python_autobahn.TestPython2Autobahn: *runtime_test +tests.package.test_python_autobahn.TestPython3Autobahn: *runtime_test +tests.package.test_python_cryptography.TestPython2Cryptography: *runtime_test +tests.package.test_python_cryptography.TestPython3Cryptography: *runtime_test +tests.package.test_python_incremental.TestPython2Incremental: *runtime_test +tests.package.test_python_incremental.TestPython3Incremental: *runtime_test +tests.package.test_python_twisted.TestPython2Twisted: *runtime_test +tests.package.test_python_twisted.TestPython3Twisted: *runtime_test +tests.package.test_python_txaio.TestPython2Txaio: *runtime_test +tests.package.test_python_txaio.TestPython3Txaio: *runtime_test +tests.package.test_python_txtorcon.TestPython2Txtorcon: *runtime_test +tests.package.test_python_txtorcon.TestPython3Txtorcon: *runtime_test tests.package.test_rust.TestRust: *runtime_test tests.package.test_rust.TestRustBin: *runtime_test tests.package.test_syslog_ng.TestSyslogNg: *runtime_test
support/testing/tests/package/test_python.py +2 -2
@@ -67,11 +67,11 @@ class TestPythonInterpreter(): self.zlib_test() -class TestPython2(TestPythonInterpreter, TestPythonBase2): +class TestPythonInterpreter2(TestPythonInterpreter, TestPythonBase2): version_string = "Python 2" -class TestPython3(TestPythonInterpreter, TestPythonBase3): +class TestPythonInterpreter3(TestPythonInterpreter, TestPythonBase3): version_string = "Python 3"
support/testing/tests/package/test_python_autobahn.py +2 -2
@@ -9,9 +9,9 @@ class TestPythonAutobahn(TestPythonPackageBase): sample_scripts = ["tests/package/sample_python_autobahn.py"] -class TestPythonPy2Autobahn(TestPythonAutobahn, TestPythonBase2): +class TestPython2Autobahn(TestPythonAutobahn, TestPythonBase2): pass -class TestPythonPy3Autobahn(TestPythonAutobahn, TestPythonBase3): +class TestPython3Autobahn(TestPythonAutobahn, TestPythonBase3): pass
support/testing/tests/package/test_python_cryptography.py +2 -2
@@ -10,9 +10,9 @@ class TestPythonCryptography(TestPythonPackageBase): timeout = 40 -class TestPythonPy2Cryptography(TestPythonCryptography, TestPythonBase2): +class TestPython2Cryptography(TestPythonCryptography, TestPythonBase2): pass -class TestPythonPy3Cryptography(TestPythonCryptography, TestPythonBase3): +class TestPython3Cryptography(TestPythonCryptography, TestPythonBase3): pass
support/testing/tests/package/test_python_incremental.py +2 -2
@@ -10,9 +10,9 @@ class TestPythonIncremental(TestPythonPackageBase): timeout = 30 -class TestPythonPy2Incremental(TestPythonIncremental, TestPythonBase2): +class TestPython2Incremental(TestPythonIncremental, TestPythonBase2): pass -class TestPythonPy3Incremental(TestPythonIncremental, TestPythonBase3): +class TestPython3Incremental(TestPythonIncremental, TestPythonBase3): pass
support/testing/tests/package/test_python_twisted.py +2 -2
@@ -24,9 +24,9 @@ class TestPythonTwisted(TestPythonPackageBase): self.assertEqual(exit_code, 0) -class TestPythonPy2Twisted(TestPythonTwisted, TestPythonBase2): +class TestPython2Twisted(TestPythonTwisted, TestPythonBase2): pass -class TestPythonPy3Twisted(TestPythonTwisted, TestPythonBase3): +class TestPython3Twisted(TestPythonTwisted, TestPythonBase3): pass
support/testing/tests/package/test_python_txaio.py +2 -2
@@ -1,7 +1,7 @@ from tests.package.test_python import TestPythonPackageBase, TestPythonBase2, TestPythonBase3 -class TestPythonPy2Txaio(TestPythonPackageBase, TestPythonBase2): +class TestPython2Txaio(TestPythonPackageBase, TestPythonBase2): config_package = \ """ BR2_PACKAGE_PYTHON_TXAIO=y
@@ -10,7 +10,7 @@ class TestPythonPy2Txaio(TestPythonPackageBase, TestPythonBase2): sample_scripts = ["tests/package/sample_python_txaio_twisted.py"] -class TestPythonPy3Txaio(TestPythonPackageBase, TestPythonBase3): +class TestPython3Txaio(TestPythonPackageBase, TestPythonBase3): config_package = \ """ BR2_PACKAGE_PYTHON_TXAIO=y
support/testing/tests/package/test_python_txtorcon.py +2 -2
@@ -10,9 +10,9 @@ class TestPythonTxtorcon(TestPythonPackageBase): timeout = 30 -class TestPythonPy2Txtorcon(TestPythonTxtorcon, TestPythonBase2): +class TestPython2Txtorcon(TestPythonTxtorcon, TestPythonBase2): pass -class TestPythonPy3Txtorcon(TestPythonTxtorcon, TestPythonBase3): +class TestPython3Txtorcon(TestPythonTxtorcon, TestPythonBase3): pass
Signature · 1 line
2.17.1
PATCH v2 12/12 testing: add python-crossbar tests
In reply to Ricardo MartincoskiFrom: Yegor Yefremov <[email protected]>
This test invokes "crossbar version" command, that checks all dependencies found in setup.py files and prints some system related information.
Add haveged to the target to generate enough entropy so crossbar -> pynacl -> libsodium don't hang waiting for /dev/random.
[Ricardo: update code-style and .gitlab-ci.yml, call the command without
'python -m' as it is not supported anymore, remove Python 2 variant, add haveged to target to add entropy and avoid hanging]
---
Changes v1 -> v2:
- new patch to the series, resending a patch from Yegor;
- call 'crossbar version' instead of 'python -m "crossbar version"'
because it is not supported anymore, giving this message:
/usr/bin/python3: No module named crossbar.__main__; 'crossbar' is a package and cannot be directly executed
- add haveged to the target to provide enough entropy, avoiding
hanging forever (on our current kernel 3.11.0 default image) or for
a long time (when the default kernel image is upgraded to a newer
kernel);
- remove test for python 2 because upstream dropped support;
- do not pass timeout parameter since version_test is now called once;
- use new naming convention and classes for tests of python packages;
- rewrap commit message to 72;
- update code-style (we adopted flake8 after v1 submission);
- update .gitlab-ci.yml;
NOTE: please mark below patch as Superseeded.
v1: http://patchwork.ozlabs.org/patch/873813/
--- .gitlab-ci.yml | 1 + .../tests/package/test_python_crossbar.py | 20 +++++++++++++++++++ 2 files changed, 21 insertions(+) create mode 100644 support/testing/tests/package/test_python_crossbar.py
.gitlab-ci.yml +1 -0
@@ -315,6 +315,7 @@ tests.package.test_python.TestPythonInterpreter2: *runtime_test tests.package.test_python.TestPythonInterpreter3: *runtime_test tests.package.test_python_autobahn.TestPython2Autobahn: *runtime_test tests.package.test_python_autobahn.TestPython3Autobahn: *runtime_test +tests.package.test_python_crossbar.TestPython3Crossbar: *runtime_test tests.package.test_python_cryptography.TestPython2Cryptography: *runtime_test tests.package.test_python_cryptography.TestPython3Cryptography: *runtime_test tests.package.test_python_incremental.TestPython2Incremental: *runtime_test
support/testing/tests/package/test_python_crossbar.py new file +20 -0
@@ -0,0 +1,20 @@ +from tests.package.test_python import TestPythonPackageBase, TestPythonBase3 + + +class TestPython3Crossbar(TestPythonPackageBase, TestPythonBase3): + # use haveged to generate enough entropy so crossbar -> pynacl -> libsodium don't hang waiting for /dev/random + config_package = \ + """ + BR2_PACKAGE_PYTHON_CROSSBAR=y + BR2_PACKAGE_HAVEGED=y + """ + + def version_test(self): + # "python -m crossbar version" can't be used + cmd = "crossbar version" + _, exit_code = self.emulator.run(cmd, 60) + self.assertEqual(exit_code, 0) + + def test_run(self): + self.login() + self.version_test()
Signature · 1 line
2.17.1
v2 cover default runtime test case for python packages v2
In reply to Ricardo MartincoskiHello Ricardo,
Quoted 4 lines · Ricardo Martincoski
Ricardo Martincoski (11):
support/testing: use helper class in IPython test support/testing: use helper class in Python test support/testing: create intermediate class per Python version
Quoted 8 lines
support/testing: create default test case for python packages support/testing: use TestPythonPackageBase for python-autobahn support/testing: use TestPythonPackageBase for python-cryptography support/testing: use TestPythonPackageBase for python-incremental support/testing: use TestPythonPackageBase for python-twisted support/testing: use TestPythonPackageBase for python-txaio support/testing: use TestPythonPackageBase for python-txtorcon support/testing: rename python* test cases
I was about to apply this patch series, but the logic in PATCH 04/12 to call the TestPythonBase constructor from a class that isn't a child class of TestPythonBase really looked awkward.
So I google a bit for people having the same issue, i.e defining a base class for unit tests, but wanting this base class to be ignored.
This answer looks particular useful:
https://stackoverflow.com/a/25695512/643208
Alternatively this one, which looks less nice to me:
https://stackoverflow.com/a/22836015/643208
Also, the first answer points to an article that describe potential issues when using multiple inheritance like we're doing:
https://nedbatchelder.com/blog/201210/multiple_inheritance_is_hard.html.
What do you think about this ?
Best regards,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
v2 cover default runtime test case for python packages v2
Hello,
Thank you again for your review.
Quoted 19 lines · Thomas Petazzoni
Quoted 4 lines · Ricardo Martincoski
Ricardo Martincoski (11):
support/testing: use helper class in IPython test support/testing: use helper class in Python test support/testing: create intermediate class per Python versionQuoted 8 lines
support/testing: create default test case for python packages support/testing: use TestPythonPackageBase for python-autobahn support/testing: use TestPythonPackageBase for python-cryptography support/testing: use TestPythonPackageBase for python-incremental support/testing: use TestPythonPackageBase for python-twisted support/testing: use TestPythonPackageBase for python-txaio support/testing: use TestPythonPackageBase for python-txtorcon support/testing: rename python* test cases
I was about to apply this patch series, but the logic in PATCH 04/12 to call the TestPythonBase constructor from a class that isn't a child class of TestPythonBase really looked awkward.
Quoted 6 lines
So I google a bit for people having the same issue, i.e defining a base class for unit tests, but wanting this base class to be ignored.
This answer looks particular useful:
There are 2 suggestions there. So let me answer both.
"move your base class into the separate module"
This should work, but I think we would still need to use the same construct that I used in v1 (I omitted TestPythonBase2 here to make the comparison easier):
import tests.package.new_module
class TestPython2<Package>(tests.package.new_module.TestPythonPackageBase):
instead of the nicer IMO:
from tests.package.new_module import TestPythonPackageBase
class TestPython2<Package>(TestPythonPackageBase):
otherwise the class that is not discovered because the new module doesn't have 'test_' in its name would be discovered later in the symbols table of the module that imports it, unless we use 'del' at the end in every module that imports it.
"wrap it with the blank class"
Using this approach we could even wrap all base classes that are imported by another files (TestPythonBase2, TestPythonBase3, TestPythonPackageBase, TestPythonInterpreter). See this applied on top of v2 in [1]. But we would still use multiple inheritance.
Quoted 4 lines
Alternatively this one, which looks less nice to me:
I am afraid using only 'del' will not work well (maybe won't work at all) in our case, because the base class is in another file.
Quoted 4 lines
Also, the first answer points to an article that describe potential issues when using multiple inheritance like we're doing:
https://nedbatchelder.com/blog/201210/multiple_inheritance_is_hard.html.
Yes, there will be always the catches about the order of the classes. There is (at least) one way out of this: if we don't create the classes TestPythonBase2 and TestPythonBase3. We don't need to keep the BR2_PACKAGE_PYTHON=y in every test case, we can add a python_version variable and a __init__ method to TestPythonBase.
class TestPythonBase(infra.basetest.BRTest):
...
python_version = None
def __init__(self, names):
super(TestPythonBase, self).__init__(names)
if self.python_version == "2":
self.config += \
"""
BR2_PACKAGE_PYTHON=y
"""
elif self.python_version == "3":
self.config += \
"""
BR2_PACKAGE_PYTHON3=y
"""See how the test cases for python packages would look like in [2]. It's a patch on top of v2.
Quoted 2 lines
What do you think about this ?
I am really unsure about the best approach here.
TBH all (my v1 and v2, and the 2 links above) look a bit hackish to me.
But I just found deeper on nose2 docs [3] the __test__ attribute. It seems the most correct solution as it depends on a feature that was implemented and documented. I just failed to find it earlier.
In the multi inheritance version it would look like this:
class TestPythonPackageBase(TestPythonBase):
...
def __init__(self, names):
if not issubclass(self.__class__, TestPythonBase2) and not issubclass(self.__class__, TestPythonBase3):
self.__test__ = False
super(TestPythonBase, self).__init__(names)
if self.config_package:...
And without multi inheritance you can see a hackish patch on top of v2 in [4].
I know I changed my mind a few times.
But now I prefer to resend using [4] (reworking each patch, of course) because:
- it does not use multi inheritance that can potentially lead to future problems as the article you found pointed. Let's avoid it while we can;
- the code for each test case for a python package looks nice, i.e. [2];
But it is not a strong preference.
What do you think about this?
[1] https://gitlab.com/RicardoMartincoski/buildroot/commit/cc2f227de49b256d9023e59bca37b8a89622bc60
[2] https://gitlab.com/RicardoMartincoski/buildroot/blob/ddf45cc7ca6d61a85b6642a246cb9b4625160d1c/support/testing/tests/package/test_python_autobahn.py
[3] https://nose2.readthedocs.io/en/latest/plugins/dundertests.html
[4] https://gitlab.com/RicardoMartincoski/buildroot/commit/ddf45cc7ca6d61a85b6642a246cb9b4625160d1c
Regards,
Ricardo
v2 cover default runtime test case for python packages v2
Hello Ricardo,
Quoted 19 lines · Ricardo Martincoski
I am really unsure about the best approach here.
TBH all (my v1 and v2, and the 2 links above) look a bit hackish to me.But I just found deeper on nose2 docs [3] the
__test__attribute. It seems the most correct solution as it depends on a feature that was implemented and documented. I just failed to find it earlier.In the multi inheritance version it would look like this:
class TestPythonPackageBase(TestPythonBase):
...def __init__(self, names): if not issubclass(self.__class__, TestPythonBase2) and not issubclass(self.__class__, TestPythonBase3): self.__test__ = False super(TestPythonBase, self).__init__(names) if self.config_package:...
And without multi inheritance you can see a hackish patch on top of v2 in [4].
Quoted 8 lines
I know I changed my mind a few times.
But now I prefer to resend using [4] (reworking each patch, of course) because:
- it does not use multi inheritance that can potentially lead to future problems as the article you found pointed. Let's avoid it while we can;
- the code for each test case for a python package looks nice, i.e. [2];
But it is not a strong preference.What do you think about this?
First of all, thanks a lot for this additional research. self.__test__ = False is definitely what we need here.
I also like the proposal that doesn't use multiple inheritance.
Let me make two possible additional proposals:
- Use an integer for the Python version variable, rather than a string, there is really no need for this to be a string I believe ?
- Alternatively, you could derive the version of the Python interpreter to use from the child class name. Maybe this is too "implicit" and a bit tricky, but I wanted to mention this possibility. I.e, in the base class, you use self.__class__.__name__, and it gives you the actual name of the class that is instantiated. You can then check if the string contains Python2 or Python3, and decide which interpreter to use according to this. I am not saying I absolutely want this, I'm just offering this as an alternative solution.
Thanks!
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
v2 cover default runtime test case for python packages v2
Hello Thomas,
Quoted 1 line · Thomas Petazzoni
On Sun, 04 Nov 2018 20:42:29 -0200, Ricardo Martincoski wrote:
[snip]
Quoted 11 lines
Quoted 7 lines
But now I prefer to resend using [4] (reworking each patch, of course) because:
- it does not use multi inheritance that can potentially lead to future problems as the article you found pointed. Let's avoid it while we can;
- the code for each test case for a python package looks nice, i.e. [2];
But it is not a strong preference.What do you think about this?
First of all, thanks a lot for this additional research. self.__test__ = False is definitely what we need here.
Quoted 6 lines
I also like the proposal that doesn't use multiple inheritance.
Let me make two possible additional proposals:
- Use an integer for the Python version variable, rather than a string, there is really no need for this to be a string I believe ?
You are correct. There is no need for a string. I will use an integer.
Quoted 10 lines
- Alternatively, you could derive the version of the Python interpreter to use from the child class name. Maybe this is too "implicit" and a bit tricky, but I wanted to mention this possibility. I.e, in the base class, you use self.__class__.__name__, and it gives you the actual name of the class that is instantiated. You can then check if the string contains Python2 or Python3, and decide which interpreter to use according to this. I am not saying I absolutely want this, I'm just offering this as an alternative solution.
For comparison only, I created a patch on top of the wip v3:
https://gitlab.com/RicardoMartincoski/buildroot/commit/c824c5dfe9c1b0953088c9a818575287fe984fc4
I guess/hope that using the explicit python_version is easier to understand for a newcomer.
Maybe a matter of taste.
I will send v3 using the explicit attribute.
Regards,
Ricardo
v2 cover default runtime test case for python packages v2
Hello Ricardo,
Quoted 4 lines · Ricardo Martincoski
Quoted 2 lines
- Use an integer for the Python version variable, rather than a string, there is really no need for this to be a string I believe ?
You are correct. There is no need for a string. I will use an integer.
On the other hand, this python_version = 2/3 is only one line, and it saves only one line in the config fragment, BR2_PACKAGE_PYTHON=y or BR2_PACKAGE_PYTHON3=y.
So we replace one fairly explicit line:
BR2_PACKAGE_PYTHON=y
by just another line, which requires going to the base class to understand what is does:
python_version = 2
I.e, I am wondering if this refactoring is really useful in the end ?
Quoted 10 lines
Quoted 9 lines
- Alternatively, you could derive the version of the Python interpreter to use from the child class name. Maybe this is too "implicit" and a bit tricky, but I wanted to mention this possibility. I.e, in the base class, you use self.__class__.__name__, and it gives you the actual name of the class that is instantiated. You can then check if the string contains Python2 or Python3, and decide which interpreter to use according to this. I am not saying I absolutely want this, I'm just offering this as an alternative solution.
Quoted 6 lines
For comparison only, I created a patch on top of the wip v3:
https://gitlab.com/RicardoMartincoski/buildroot/commit/c824c5dfe9c1b0953088c9a818575287fe984fc4I guess/hope that using the explicit
python_versionis easier to understand for a newcomer.
Maybe a matter of taste.
Yeah, I don't have a solid opinion on this. Perhaps the explicit solution is more readable/clearer.
Thanks,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
v2 cover default runtime test case for python packages v2
Hello Thomas,
Quoted 13 lines · Thomas Petazzoni
Quoted 4 lines · Ricardo Martincoski
Quoted 2 lines
- Use an integer for the Python version variable, rather than a string, there is really no need for this to be a string I believe ?
You are correct. There is no need for a string. I will use an integer.
On the other hand, this
python_version= 2/3 is only one line, and it saves only one line in the config fragment, BR2_PACKAGE_PYTHON=y or BR2_PACKAGE_PYTHON3=y.So we replace one fairly explicit line:
Quoted 8 lines
BR2_PACKAGE_PYTHON=y
by just another line, which requires going to the base class to understand what is does:
python_version= 2I.e, I am wondering if this refactoring is really useful in the end ?
Please let me send a v3, based from v1, doing the minimum to accomplish this goal:
- add common logic to test python packages with a script in a separate file instead of inline in the testcase
... and following these ideas:
- use separate class
- use no hacks/tricks
- use explicit code
From this v3 we can decide the direction to follow: whether to drop the series or do some additional change, either as v4 or as followup patches.
Regards,
Ricardo
v3 cover default runtime test case for python packages
In reply to Ricardo MartincoskiHello,
This series creates a standard runtime test case for python packages and is based on ideas from Thomas and Arnout.
The sample scripts that run on the target (fixture for the test case) are stored in a separate file (so it becomes more readable than inline in the test case). For the time being, they are stored on support/testing.
Those sample scripts are copied by the default test case to the target image in build time. The test infra builds the image, and the default test case logs in the target, checks the script is in the rootfs and calls the Python interpreter passing the sample script.
Patch 1 adds the new class with common logic to copy the sample script and execute it.
Patches 2 to 7 change one by one the 6 current test cases for python packages to use the new class.
Since the sample scripts are now stored on separate files they are automatically picked by the check-flake8 job on GitLab CI, so I formatted them to comply to flake8 (only empty lines or adding "# noqa" for the script that only imports the module but don't use it).
Patch 8 refreshes a test case that Yegor created for python-crossbar.
All python test cases on Gitlab CI:
https://gitlab.com/RicardoMartincoski/buildroot/pipelines/36090843
Changes v2 -> v3:
- back to v1 but without any tricks and using a separate class;
- fix typo when using super() TestPythonBase -> TestPythonPackageBase,
in v2 it ended up calling the same method, but it is wrong;
- don't use a hack, use __test__ = False from nose2 to declare a class that looks like a test case but is not a test case, as consequence all classes that inherits from it and are test cases must set __test__ = True;
- add docstring;
- refresh patches changing each test case;
- in the python-twisted test, get the config from immediate parent class;
- in the python-crossbar test, move test script to a separate file;
Changes v1 -> v2:
- do not reuse TestPython2 and TestPython3 for two entirely separate things:
(Thomas)
- As a base class for testing individual Python packages;
- As test cases for the Python interpreter itself;
- use a better class hierarchy (Thomas). I did this in various patches (1, 2, 3, 4, 11) trying to make the review easier.
- with the new class hierarchy the trick that allows the defconfig to be changed by each test case without explicitly naming the class that contains the base defconfig is moved from every test case to the new base class;
- new patch to the series, resending a patch from Yegor;
Regards,
Ricardo
Ricardo Martincoski (7):
support/testing: create default test case for python packages support/testing: use TestPythonPackageBase for python-autobahn support/testing: use TestPythonPackageBase for python-cryptography support/testing: use TestPythonPackageBase for python-incremental support/testing: use TestPythonPackageBase for python-twisted support/testing: use TestPythonPackageBase for python-txaio support/testing: use TestPythonPackageBase for python-txtorcon
Yegor Yefremov (1):
testing: add python-crossbar tests
.gitlab-ci.yml | 1 + .../package/copy-sample-script-to-target.sh | 7 +++ .../tests/package/sample_python_autobahn.py | 1 + .../tests/package/sample_python_crossbar.py | 3 + .../package/sample_python_cryptography.py | 3 + .../package/sample_python_incremental.py | 3 + .../tests/package/sample_python_twisted.py | 9 +++ .../package/sample_python_txaio_asyncio.py | 3 + .../package/sample_python_txaio_twisted.py | 3 + .../tests/package/sample_python_txtorcon.py | 1 + support/testing/tests/package/test_python.py | 56 +++++++++++++++++++ .../tests/package/test_python_autobahn.py | 29 +++------- .../tests/package/test_python_crossbar.py | 14 +++++ .../tests/package/test_python_cryptography.py | 33 ++++------- .../tests/package/test_python_incremental.py | 33 ++++------- .../tests/package/test_python_twisted.py | 35 ++++-------- .../tests/package/test_python_txaio.py | 30 +++------- .../tests/package/test_python_txtorcon.py | 31 ++++------ 18 files changed, 165 insertions(+), 130 deletions(-) create mode 100755 support/testing/tests/package/copy-sample-script-to-target.sh create mode 100644 support/testing/tests/package/sample_python_autobahn.py create mode 100644 support/testing/tests/package/sample_python_crossbar.py create mode 100644 support/testing/tests/package/sample_python_cryptography.py create mode 100644 support/testing/tests/package/sample_python_incremental.py create mode 100644 support/testing/tests/package/sample_python_twisted.py create mode 100644 support/testing/tests/package/sample_python_txaio_asyncio.py create mode 100644 support/testing/tests/package/sample_python_txaio_twisted.py create mode 100644 support/testing/tests/package/sample_python_txtorcon.py create mode 100644 support/testing/tests/package/test_python_crossbar.py
Signature · 1 line
2.17.1
PATCH v3 1/8 support/testing: create default test case for python packages
Test cases for python packages are very similar among each other: run a simple script in the target that minimally tests the package. So create a new helper class named TestPythonPackageBase that holds all the logic to run a script on the target.
TestPythonPackageBase adds in build time one or more sample scripts to be run on the target. The test case for the python package must explicitly list them in the "sample_scripts" property. The test case then automatically logins to the target, checks the scripts are really in the rootfs (it calls "md5sum" instead of "ls" or "test" in an attempt to make the logfile more friendly, since someone analysing a failure can easily check the expected script was executed) and then calls the python interpreter passing the sample script as parameter.
An optional property "timeout" exists for the case the sample script needs more time to run than the default timeout from the test infra (currently 5 seconds).
A simple test case for a package that only supports Python 2 will look like this:
|from tests.package.test_python import TestPythonPackageBase | | |class TestPythonPy2<Package>(TestPythonPackageBase): | __test__ = True | config = TestPythonPackageBase.config + \ | """ | BR2_PACKAGE_PYTHON=y | BR2_PACKAGE_PYTHON_<PACKAGE>=y | """ | sample_scripts = ["tests/package/sample_python_<package>.py"] | timeout = 15
---
Changes v2 -> v3:
- back to v1 but without any tricks and using a separate class;
- fix typo when using super() TestPythonBase -> TestPythonPackageBase,
in v2 it ended up calling the same method, but it is wrong;
Changes v1 -> v3:
- don't use a hack, use __test__ = False from nose2 to declare a class that looks like a test case but is not a test case, as consequence all classes that inherits from it and are test cases must set __test__ = True;
- create a new class TestPythonPackageBase;
- add docstring;
Changes v1 -> v2:
- do not reuse TestPython2 and TestPython3 to for two entirely separate things: (Thomas)
- As a base class for testing individual Python packages;
- As test cases for the Python interpreter itself;
- use a better class hierarchy (Thomas). I did this in various patches (before and after this one) trying to make the review easier.
- with the new class hierarchy the trick that allows the defconfig to be changed by each test case without explicitly naming the class that contains the base defconfig is moved from every test case to the new base class.
v1: http://patchwork.ozlabs.org/patch/984425/
--- .../package/copy-sample-script-to-target.sh | 7 +++ support/testing/tests/package/test_python.py | 56 +++++++++++++++++++ 2 files changed, 63 insertions(+) create mode 100755 support/testing/tests/package/copy-sample-script-to-target.sh
support/testing/tests/package/copy-sample-script-to-target.sh new file +7 -0
@@ -0,0 +1,7 @@ +#!/bin/sh +set -e + +shift +for file in "$@"; do + cp -f "${file}" "${TARGET_DIR}/root/" +done
support/testing/tests/package/test_python.py +56 -0
59 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH v3 2/8 support/testing: use TestPythonPackageBase for python-autobahn
In reply to Ricardo MartincoskiMove the test script to be run on the target from inline in the test case to a separate file.
---
Changes v2 -> v3:
- refresh;
Changes v1 -> v2:
- refresh after creating TestPythonPackageBase;
--- .../tests/package/sample_python_autobahn.py | 1 + .../tests/package/test_python_autobahn.py | 29 ++++++------------- 2 files changed, 10 insertions(+), 20 deletions(-) create mode 100644 support/testing/tests/package/sample_python_autobahn.py
support/testing/tests/package/sample_python_autobahn.py new file +1 -0
@@ -0,0 +1 @@ +import autobahn.wamp # noqa
support/testing/tests/package/test_python_autobahn.py +9 -20
@@ -1,32 +1,21 @@ -from tests.package.test_python import TestPythonBase +from tests.package.test_python import TestPythonPackageBase -class TestPythonAutobahn(TestPythonBase): - def import_test(self): - cmd = self.interpreter + " -c 'import autobahn.wamp'" - _, exit_code = self.emulator.run(cmd) - self.assertEqual(exit_code, 0) - - -class TestPythonPy2Autobahn(TestPythonAutobahn): - config = TestPythonBase.config + \ +class TestPythonPy2Autobahn(TestPythonPackageBase): + __test__ = True + config = TestPythonPackageBase.config + \ """ BR2_PACKAGE_PYTHON=y BR2_PACKAGE_PYTHON_AUTOBAHN=y """ + sample_scripts = ["tests/package/sample_python_autobahn.py"] - def test_run(self): - self.login() - self.import_test() - -class TestPythonPy3Autobahn(TestPythonAutobahn): - config = TestPythonBase.config + \ +class TestPythonPy3Autobahn(TestPythonPackageBase): + __test__ = True + config = TestPythonPackageBase.config + \ """ BR2_PACKAGE_PYTHON3=y BR2_PACKAGE_PYTHON_AUTOBAHN=y """ - - def test_run(self): - self.login() - self.import_test() + sample_scripts = ["tests/package/sample_python_autobahn.py"]
Signature · 1 line
2.17.1
PATCH v3 3/8 support/testing: use TestPythonPackageBase for python-cryptography
In reply to Ricardo MartincoskiMove the test script to be run on the target from inline in the test case to a separate file.
---
Changes v2 -> v3:
- refresh;
Changes v1 -> v2:
- refresh after creating TestPythonPackageBase;
--- .../package/sample_python_cryptography.py | 3 ++ .../tests/package/test_python_cryptography.py | 33 +++++++------------ 2 files changed, 14 insertions(+), 22 deletions(-) create mode 100644 support/testing/tests/package/sample_python_cryptography.py
support/testing/tests/package/sample_python_cryptography.py new file +3 -0
@@ -0,0 +1,3 @@ +from cryptography.fernet import Fernet +key = Fernet.generate_key() +f = Fernet(key)
support/testing/tests/package/test_python_cryptography.py +11 -22
@@ -1,34 +1,23 @@ -from tests.package.test_python import TestPythonBase +from tests.package.test_python import TestPythonPackageBase -class TestPythonCryptography(TestPythonBase): - def fernet_test(self, timeout=-1): - cmd = self.interpreter + " -c 'from cryptography.fernet import Fernet;" - cmd += "key = Fernet.generate_key();" - cmd += "f = Fernet(key)'" - _, exit_code = self.emulator.run(cmd, timeout) - self.assertEqual(exit_code, 0) - - -class TestPythonPy2Cryptography(TestPythonCryptography): - config = TestPythonBase.config + \ +class TestPythonPy2Cryptography(TestPythonPackageBase): + __test__ = True + config = TestPythonPackageBase.config + \ """ BR2_PACKAGE_PYTHON=y BR2_PACKAGE_PYTHON_CRYPTOGRAPHY=y """ + sample_scripts = ["tests/package/sample_python_cryptography.py"] + timeout = 40 - def test_run(self): - self.login() - self.fernet_test(40) - -class TestPythonPy3Cryptography(TestPythonCryptography): - config = TestPythonBase.config + \ +class TestPythonPy3Cryptography(TestPythonPackageBase): + __test__ = True + config = TestPythonPackageBase.config + \ """ BR2_PACKAGE_PYTHON3=y BR2_PACKAGE_PYTHON_CRYPTOGRAPHY=y """ - - def test_run(self): - self.login() - self.fernet_test(40) + sample_scripts = ["tests/package/sample_python_cryptography.py"] + timeout = 40
Signature · 1 line
2.17.1
PATCH v3 4/8 support/testing: use TestPythonPackageBase for python-incremental
In reply to Ricardo MartincoskiMove the test script to be run on the target from inline in the test case to a separate file.
---
Changes v2 -> v3:
- refresh;
Changes v1 -> v2:
- refresh after creating TestPythonPackageBase;
--- .../package/sample_python_incremental.py | 3 ++ .../tests/package/test_python_incremental.py | 33 +++++++------------ 2 files changed, 14 insertions(+), 22 deletions(-) create mode 100644 support/testing/tests/package/sample_python_incremental.py
support/testing/tests/package/sample_python_incremental.py new file +3 -0
@@ -0,0 +1,3 @@ +import incremental +v = incremental.Version("package", 1, 2, 3, release_candidate=4) +assert(str(v) == "[package, version 1.2.3rc4]")
support/testing/tests/package/test_python_incremental.py +11 -22
@@ -1,34 +1,23 @@ -from tests.package.test_python import TestPythonBase +from tests.package.test_python import TestPythonPackageBase -class TestPythonIncremental(TestPythonBase): - def str_test(self): - cmd = self.interpreter + " -c 'import incremental;" - cmd += "v = incremental.Version(\"package\", 1, 2, 3, release_candidate=4);" - cmd += "assert(str(v) == \"[package, version 1.2.3rc4]\")'" - _, exit_code = self.emulator.run(cmd, timeout=30) - self.assertEqual(exit_code, 0) - - -class TestPythonPy2Incremental(TestPythonIncremental): - config = TestPythonBase.config + \ +class TestPythonPy2Incremental(TestPythonPackageBase): + __test__ = True + config = TestPythonPackageBase.config + \ """ BR2_PACKAGE_PYTHON=y BR2_PACKAGE_PYTHON_INCREMENTAL=y """ + sample_scripts = ["tests/package/sample_python_incremental.py"] + timeout = 30 - def test_run(self): - self.login() - self.str_test() - -class TestPythonPy3Incremental(TestPythonIncremental): - config = TestPythonBase.config + \ +class TestPythonPy3Incremental(TestPythonPackageBase): + __test__ = True + config = TestPythonPackageBase.config + \ """ BR2_PACKAGE_PYTHON3=y BR2_PACKAGE_PYTHON_INCREMENTAL=y """ - - def test_run(self): - self.login() - self.str_test() + sample_scripts = ["tests/package/sample_python_incremental.py"] + timeout = 30
Signature · 1 line
2.17.1
PATCH v3 5/8 support/testing: use TestPythonPackageBase for python-twisted
In reply to Ricardo MartincoskiMove the test script to be run on the target from inline in the test case to a separate file.
Get the base defconfig fragment from the immediate parent class and not directly from TestPythonBase because it is the correct way of doing this. This way the base class TestPythonTwisted could even be placed in a separate file.
---
Changes v2 -> v3:
- refresh;
- get the config from immediate parent class;
Changes v1 -> v2:
- refresh after creating TestPythonPackageBase;
- override run_sample_scripts instead of test_run;
--- .../tests/package/sample_python_twisted.py | 9 +++++ .../tests/package/test_python_twisted.py | 35 ++++++------------- 2 files changed, 19 insertions(+), 25 deletions(-) create mode 100644 support/testing/tests/package/sample_python_twisted.py
support/testing/tests/package/sample_python_twisted.py new file +9 -0
@@ -0,0 +1,9 @@ +from twisted.internet import protocol, reactor, endpoints + + +class F(protocol.Factory): + pass + + +endpoints.serverFromString(reactor, "tcp:1234").listen(F()) +reactor.run()
support/testing/tests/package/test_python_twisted.py +10 -25
@@ -1,25 +1,16 @@ -from tests.package.test_python import TestPythonBase +from tests.package.test_python import TestPythonPackageBase -TEST_SCRIPT = """ -from twisted.internet import protocol, reactor, endpoints -class F(protocol.Factory): - pass -endpoints.serverFromString(reactor, "tcp:1234").listen(F()) -reactor.run() -""" +class TestPythonTwisted(TestPythonPackageBase): + config = TestPythonPackageBase.config + sample_scripts = ["tests/package/sample_python_twisted.py"] -class TestPythonTwisted(TestPythonBase): - def import_test(self): - cmd = "printf '{}' > test.py".format(TEST_SCRIPT) - _, exit_code = self.emulator.run(cmd) - self.assertEqual(exit_code, 0) - + def run_sample_scripts(self): cmd = "netstat -ltn 2>/dev/null | grep 0.0.0.0:1234" _, exit_code = self.emulator.run(cmd) self.assertEqual(exit_code, 1) - cmd = self.interpreter + " test.py &" + cmd = self.interpreter + " sample_python_twisted.py &" # give some time to setup the server cmd += "sleep 30" _, exit_code = self.emulator.run(cmd, timeout=35)
@@ -31,24 +22,18 @@ class TestPythonTwisted(TestPythonBase): class TestPythonPy2Twisted(TestPythonTwisted): - config = TestPythonBase.config + \ + __test__ = True + config = TestPythonTwisted.config + \ """ BR2_PACKAGE_PYTHON=y BR2_PACKAGE_PYTHON_TWISTED=y """ - def test_run(self): - self.login() - self.import_test() - class TestPythonPy3Twisted(TestPythonTwisted): - config = TestPythonBase.config + \ + __test__ = True + config = TestPythonTwisted.config + \ """ BR2_PACKAGE_PYTHON3=y BR2_PACKAGE_PYTHON_TWISTED=y """ - - def test_run(self): - self.login() - self.import_test()
Signature · 1 line
2.17.1
PATCH v3 6/8 support/testing: use TestPythonPackageBase for python-txaio
In reply to Ricardo MartincoskiMove the test scripts to be run on the target from inline in the test case to a separate file.
---
Changes v2 -> v3:
- refresh;
Changes v1 -> v2:
- refresh after creating TestPythonPackageBase;
--- .../package/sample_python_txaio_asyncio.py | 3 ++ .../package/sample_python_txaio_twisted.py | 3 ++ .../tests/package/test_python_txaio.py | 30 ++++++------------- 3 files changed, 15 insertions(+), 21 deletions(-) create mode 100644 support/testing/tests/package/sample_python_txaio_asyncio.py create mode 100644 support/testing/tests/package/sample_python_txaio_twisted.py
support/testing/tests/package/sample_python_txaio_asyncio.py new file +3 -0
@@ -0,0 +1,3 @@ +import txaio +txaio.use_asyncio() +f0 = txaio.create_future()
support/testing/tests/package/sample_python_txaio_twisted.py new file +3 -0
@@ -0,0 +1,3 @@ +import txaio +txaio.use_twisted() +f0 = txaio.create_future()
support/testing/tests/package/test_python_txaio.py +9 -21
@@ -1,34 +1,22 @@ -from tests.package.test_python import TestPythonBase +from tests.package.test_python import TestPythonPackageBase -class TestPythonPy2Txaio(TestPythonBase): - config = TestPythonBase.config + \ +class TestPythonPy2Txaio(TestPythonPackageBase): + __test__ = True + config = TestPythonPackageBase.config + \ """ BR2_PACKAGE_PYTHON=y BR2_PACKAGE_PYTHON_TXAIO=y BR2_PACKAGE_PYTHON_TWISTED=y """ + sample_scripts = ["tests/package/sample_python_txaio_twisted.py"] - def test_run(self): - self.login() - cmd = self.interpreter + " -c 'import txaio;" - cmd += "txaio.use_twisted();" - cmd += "f0 = txaio.create_future()'" - _, exit_code = self.emulator.run(cmd) - self.assertEqual(exit_code, 0) - -class TestPythonPy3Txaio(TestPythonBase): - config = TestPythonBase.config + \ +class TestPythonPy3Txaio(TestPythonPackageBase): + __test__ = True + config = TestPythonPackageBase.config + \ """ BR2_PACKAGE_PYTHON3=y BR2_PACKAGE_PYTHON_TXAIO=y """ - - def test_run(self): - self.login() - cmd = self.interpreter + " -c 'import txaio;" - cmd += "txaio.use_asyncio();" - cmd += "f0 = txaio.create_future()'" - _, exit_code = self.emulator.run(cmd) - self.assertEqual(exit_code, 0) + sample_scripts = ["tests/package/sample_python_txaio_asyncio.py"]
Signature · 1 line
2.17.1
PATCH v3 7/8 support/testing: use TestPythonPackageBase for python-txtorcon
In reply to Ricardo MartincoskiMove the test script to be run on the target from inline in the test case to a separate file.
---
Changes v2 -> v3:
- refresh;
Changes v1 -> v2:
- refresh after creating TestPythonPackageBase;
--- .../tests/package/sample_python_txtorcon.py | 1 + .../tests/package/test_python_txtorcon.py | 31 +++++++------------ 2 files changed, 12 insertions(+), 20 deletions(-) create mode 100644 support/testing/tests/package/sample_python_txtorcon.py
support/testing/tests/package/sample_python_txtorcon.py new file +1 -0
@@ -0,0 +1 @@ +import txtorcon # noqa
support/testing/tests/package/test_python_txtorcon.py +11 -20
@@ -1,32 +1,23 @@ -from tests.package.test_python import TestPythonBase +from tests.package.test_python import TestPythonPackageBase -class TestPythonTxtorcon(TestPythonBase): - def import_test(self): - cmd = self.interpreter + " -c 'import txtorcon'" - _, exit_code = self.emulator.run(cmd, timeout=30) - self.assertEqual(exit_code, 0) - - -class TestPythonPy2Txtorcon(TestPythonTxtorcon): - config = TestPythonBase.config + \ +class TestPythonPy2Txtorcon(TestPythonPackageBase): + __test__ = True + config = TestPythonPackageBase.config + \ """ BR2_PACKAGE_PYTHON=y BR2_PACKAGE_PYTHON_TXTORCON=y """ + sample_scripts = ["tests/package/sample_python_txtorcon.py"] + timeout = 30 - def test_run(self): - self.login() - self.import_test() - -class TestPythonPy3Txtorcon(TestPythonTxtorcon): - config = TestPythonBase.config + \ +class TestPythonPy3Txtorcon(TestPythonPackageBase): + __test__ = True + config = TestPythonPackageBase.config + \ """ BR2_PACKAGE_PYTHON3=y BR2_PACKAGE_PYTHON_TXTORCON=y """ - - def test_run(self): - self.login() - self.import_test() + sample_scripts = ["tests/package/sample_python_txtorcon.py"] + timeout = 30
Signature · 1 line
2.17.1
PATCH v3 8/8 testing: add python-crossbar tests
In reply to Ricardo MartincoskiFrom: Yegor Yefremov <[email protected]>
This test invokes "crossbar version" command, that checks all dependencies found in setup.py files and prints some system related information.
Add haveged to the target to generate enough entropy so crossbar -> pynacl -> libsodium don't hang waiting for /dev/random.
[Ricardo: move test script to a separate file, remove Python 2 variant,
add haveged to target to add entropy and avoid hanging]
---
Changes v2 -> v3:
- refresh;
- move test script to a separate file;
Changes v1 -> v2:
- new patch to the series, resending a patch from Yegor;
- call 'crossbar version' instead of 'python -m "crossbar version"'
because it is not supported anymore, giving this message:
/usr/bin/python3: No module named crossbar.__main__; 'crossbar' is a package and cannot be directly executed
- add haveged to the target to provide enough entropy, avoiding
hanging forever (on our current kernel 3.11.0 default image) or for
a long time (when the default kernel image is upgraded to a newer
kernel);
- remove test for python 2 because upstream dropped support;
- do not pass timeout parameter since version_test is now called once;
- use new naming convention and classes for tests of python packages;
- rewrap commit message to 72;
- update code-style (we adopted flake8 after v1 submission);
- update .gitlab-ci.yml;
NOTE: please mark below patch as Superseded.
v1: http://patchwork.ozlabs.org/patch/873813/
--- .gitlab-ci.yml | 1 + .../tests/package/sample_python_crossbar.py | 3 +++ .../testing/tests/package/test_python_crossbar.py | 14 ++++++++++++++ 3 files changed, 18 insertions(+) create mode 100644 support/testing/tests/package/sample_python_crossbar.py create mode 100644 support/testing/tests/package/test_python_crossbar.py
.gitlab-ci.yml +1 -0
@@ -316,6 +316,7 @@ tests.package.test_python.TestPython2: *runtime_test tests.package.test_python.TestPython3: *runtime_test tests.package.test_python_autobahn.TestPythonPy2Autobahn: *runtime_test tests.package.test_python_autobahn.TestPythonPy3Autobahn: *runtime_test +tests.package.test_python_crossbar.TestPythonPy3Crossbar: *runtime_test tests.package.test_python_cryptography.TestPythonPy2Cryptography: *runtime_test tests.package.test_python_cryptography.TestPythonPy3Cryptography: *runtime_test tests.package.test_python_incremental.TestPythonPy2Incremental: *runtime_test
support/testing/tests/package/sample_python_crossbar.py new file +3 -0
@@ -0,0 +1,3 @@ +import crossbar + +crossbar.run(["version"])
support/testing/tests/package/test_python_crossbar.py new file +14 -0
@@ -0,0 +1,14 @@ +from tests.package.test_python import TestPythonPackageBase + + +class TestPythonPy3Crossbar(TestPythonPackageBase): + __test__ = True + # use haveged to generate enough entropy so crossbar -> pynacl -> libsodium don't hang waiting for /dev/random + config = TestPythonPackageBase.config + \ + """ + BR2_PACKAGE_PYTHON3=y + BR2_PACKAGE_PYTHON_CROSSBAR=y + BR2_PACKAGE_HAVEGED=y + """ + sample_scripts = ["tests/package/sample_python_crossbar.py"] + timeout = 60
Signature · 1 line
2.17.1
v3 cover default runtime test case for python packages
In reply to Ricardo MartincoskiHello,
Quoted 11 lines · Ricardo Martincoski
Ricardo Martincoski (7):
support/testing: create default test case for python packages support/testing: use TestPythonPackageBase for python-autobahn support/testing: use TestPythonPackageBase for python-cryptography support/testing: use TestPythonPackageBase for python-incremental support/testing: use TestPythonPackageBase for python-twisted support/testing: use TestPythonPackageBase for python-txaio support/testing: use TestPythonPackageBase for python-txtorconYegor Yefremov (1):
testing: add python-crossbar tests
Thanks a lot for all this work, I've applied your series to the next branch. Thanks!
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com