test/py: Implement pytest infrastructure
PATCH v3 1/7 test/py: Implement pytest infrastructure
This tool aims to test U-Boot by executing U-Boot shell commands using the console interface. A single top-level script exists to execute or attach to the U-Boot console, run the entire script of tests against it, and summarize the results. Advantages of this approach are:
- Testing is performed in the same way a user or script would interact with U-Boot; there can be no disconnect.
- There is no need to write or embed test-related code into U-Boot itself. It is asserted that writing test-related code in Python is simpler and more flexible that writing it all in C.
- It is reasonably simple to interact with U-Boot in this way.
A few simple tests are provided as examples. Soon, we should convert as many as possible of the other tests in test/* and test/cmd_ut.c too.
The hook scripts, relay control utilities, and udev rules I use for my own HW setup are published at https://github.com/swarren/uboot-test-hooks.
See README.md for more details!
---
v3:
- Rework HTML log generation so that TAB characters render as expected. Suggested by Michal Simek.
- Move test scripts into a sub-directory.
Suggested by Michal Simek.
- s/uboot/u[-_]boot/g. Suggested by Simon Glass.
- s/"/'/g. Suggested by Simon Glass.
- Typo fixes.
- Add more documentation. Suggested by Simon Glass.
- Make "notes" in the log file be <pre> so that their formatting is preserved. This is useful for large notes such as exception dumps.
v2:
- Many fixes and tweaks have been squashed in. Separated out some of the tests into separate commits, and added some more tests.
--- test/py/.gitignore | 1 + test/py/README.md | 300 ++++++++++++++++++++++++++++++ test/py/conftest.py | 335 ++++++++++++++++++++++++++++++++++ test/py/multiplexed_log.css | 88 +++++++++ test/py/multiplexed_log.py | 335 ++++++++++++++++++++++++++++++++++ test/py/pytest.ini | 11 ++ test/py/test.py | 32 ++++ test/py/tests/test_000_version.py | 20 ++ test/py/tests/test_help.py | 9 + test/py/tests/test_unknown_cmd.py | 14 ++ test/py/u_boot_console_base.py | 260 ++++++++++++++++++++++++++ test/py/u_boot_console_exec_attach.py | 51 ++++++ test/py/u_boot_console_sandbox.py | 48 +++++ test/py/u_boot_spawn.py | 123 +++++++++++++ 14 files changed, 1627 insertions(+) create mode 100644 test/py/.gitignore create mode 100644 test/py/README.md create mode 100644 test/py/conftest.py create mode 100644 test/py/multiplexed_log.css create mode 100644 test/py/multiplexed_log.py create mode 100644 test/py/pytest.ini create mode 100755 test/py/test.py create mode 100644 test/py/tests/test_000_version.py create mode 100644 test/py/tests/test_help.py create mode 100644 test/py/tests/test_unknown_cmd.py create mode 100644 test/py/u_boot_console_base.py create mode 100644 test/py/u_boot_console_exec_attach.py create mode 100644 test/py/u_boot_console_sandbox.py create mode 100644 test/py/u_boot_spawn.py
test/py/.gitignore new file +1 -0
@@ -0,0 +1 @@ +*.pyc
test/py/README.md new file +300 -0
300 lines not on this page · raw message
test/py/conftest.py new file +335 -0
335 lines not on this page · raw message
test/py/multiplexed_log.css new file +88 -0
88 lines not on this page · raw message
test/py/multiplexed_log.py new file +335 -0
335 lines not on this page · raw message
test/py/pytest.ini new file +11 -0
@@ -0,0 +1,11 @@ +# Copyright (c) 2015 Stephen Warren +# Copyright (c) 2015-2016, NVIDIA CORPORATION. All rights reserved. +# +# SPDX-License-Identifier: GPL-2.0 + +# Static configuration data for pytest. pytest reads this at startup time. + +[pytest] +markers = + boardspec: U-Boot: Describes the set of boards a test can/can't run on. + buildconfigspec: U-Boot: Describes Kconfig/config-header constraints.
test/py/test.py new file +32 -0
@@ -0,0 +1,32 @@ +#!/usr/bin/env python + +# Copyright (c) 2015 Stephen Warren +# Copyright (c) 2015-2016, NVIDIA CORPORATION. All rights reserved. +# +# SPDX-License-Identifier: GPL-2.0 + +# Wrapper script to invoke pytest with the directory name that contains the +# U-Boot tests. + +import os +import os.path +import sys + +# Get rid of argv[0] +sys.argv.pop(0) + +# argv; py.test test_directory_name user-supplied-arguments +args = ["py.test", os.path.dirname(__file__) + "/tests"] +args.extend(sys.argv) + +try: + os.execvp("py.test", args) +except: + # Log full details of any exception for detailed analysis + import traceback + traceback.print_exc() + # Hint to the user that they likely simply haven't installed the required + # dependencies. + print >>sys.stderr, """ +exec(py.test) failed; perhaps you are missing some dependencies? +See test/py/README.md for the list."""
test/py/tests/test_000_version.py new file +20 -0
@@ -0,0 +1,20 @@ +# Copyright (c) 2015 Stephen Warren +# Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved. +# +# SPDX-License-Identifier: GPL-2.0 + +# pytest runs tests the order of their module path, which is related to the +# filename containing the test. This file is named such that it is sorted +# first, simply as a very basic sanity check of the functionality of the U-Boot +# command prompt. + +def test_version(u_boot_console): + '''Test that the "version" command prints the U-Boot version.''' + + # "version" prints the U-Boot sign-on message. This is usually considered + # an error, so that any unexpected reboot causes an error. Here, this + # error detection is disabled since the sign-on message is expected. + with u_boot_console.disable_check('main_signon'): + response = u_boot_console.run_command('version') + # Ensure "version" printed what we expected. + u_boot_console.validate_main_signon_in_text(response)
test/py/tests/test_help.py new file +9 -0
@@ -0,0 +1,9 @@ +# Copyright (c) 2015 Stephen Warren +# Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved. +# +# SPDX-License-Identifier: GPL-2.0 + +def test_help(u_boot_console): + '''Test that the "help" command can be executed.''' + + u_boot_console.run_command('help')
test/py/tests/test_unknown_cmd.py new file +14 -0
@@ -0,0 +1,14 @@ +# Copyright (c) 2015 Stephen Warren +# Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved. +# +# SPDX-License-Identifier: GPL-2.0 + +def test_unknown_command(u_boot_console): + '''Test that executing an unknown command causes U-Boot to print an + error.''' + + # The "unknown command" error is actively expected here, + # so error detection for it is disabled. + with u_boot_console.disable_check('unknown_command'): + response = u_boot_console.run_command('non_existent_cmd') + assert('Unknown command \'non_existent_cmd\' - try \'help\'' in response)
test/py/u_boot_console_base.py new file +260 -0
260 lines not on this page · raw message
test/py/u_boot_console_exec_attach.py new file +51 -0
51 lines not on this page · raw message
test/py/u_boot_console_sandbox.py new file +48 -0
48 lines not on this page · raw message
test/py/u_boot_spawn.py new file +123 -0
123 lines not on this page · raw message
Signature · 1 line
2.6.4
PATCH v3 1/7 test/py: Implement pytest infrastructure
Quoted 12 lines · Stephen Warren
This tool aims to test U-Boot by executing U-Boot shell commands using the console interface. A single top-level script exists to execute or attach to the U-Boot console, run the entire script of tests against it, and summarize the results. Advantages of this approach are:
- Testing is performed in the same way a user or script would interact with U-Boot; there can be no disconnect.
- There is no need to write or embed test-related code into U-Boot itself. It is asserted that writing test-related code in Python is simpler and more flexible that writing it all in C.
- It is reasonably simple to interact with U-Boot in this way.
Quoted 7 lines
A few simple tests are provided as examples. Soon, we should convert as many as possible of the other tests in test/* and
test/cmd_ut.ctoo.The hook scripts, relay control utilities, and udev rules I use for my own HW setup are published at https://github.com/swarren/uboot-test-hooks.
See README.md for more details!
It looks like I need to send a v4 of this, since I renamed a Python class but forgot to update all users of it. I didn't notice this, since I had the old module lying around as a *.pyc file, so the old name worked:-(
I also have a couple of minor fixes to roll in that make the scripts work better under a continuous integration environment (which doesn't have a controlling TTY set when the scripts run, which need a minor tweak to the Spawn code).
PATCH v3 1/7 test/py: Implement pytest infrastructure
Quoted 29 lines · Stephen Warren
Quoted 14 lines · Stephen Warren
This tool aims to test U-Boot by executing U-Boot shell commands using the
console interface. A single top-level script exists to execute or attach to the U-Boot console, run the entire script of tests against it, and summarize the results. Advantages of this approach are:- Testing is performed in the same way a user or script would interact with U-Boot; there can be no disconnect.
- There is no need to write or embed test-related code into U-Boot itself.
It is asserted that writing test-related code in Python is simpler and more flexible that writing it all in C.
- It is reasonably simple to interact with U-Boot in this way.Quoted 8 lines
A few simple tests are provided as examples. Soon, we should convert as many as possible of the other tests in test/* and
test/cmd_ut.ctoo.The hook scripts, relay control utilities, and udev rules I use for my own HW setup are published at
https://github.com/swarren/uboot-test-hooks.See README.md for more details!
It looks like I need to send a v4 of this, since I renamed a Python class but forgot to update all users of it. I didn't notice this, since I had the old module lying around as a *.pyc file, so the old name worked:-(
Quoted 4 lines
I also have a couple of minor fixes to roll in that make the scripts work better under a continuous integration environment (which doesn't have a controlling TTY set when the scripts run, which need a minor tweak to the Spawn code).
btw: You can also add a note about kermit whcih I do use and works good.
Thanks,
Michal
Signature · 14 lines
Michal Simek, Ing. (M.Eng), OpenPGP -> KeyID: FE3D1F91 w: www.monstr.eu p: +42-0-721842854 Maintainer of Linux kernel - Microblaze cpu - http://www.monstr.eu/fdt/ Maintainer of Linux kernel - Xilinx Zynq ARM architecture Microblaze U-BOOT custodian and responsible for u-boot arm zynq platform -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: <http://lists.denx.de/pipermail/u-boot/attachments/20160108/c21bbd67/attachment.sig>
PATCH v3 1/7 test/py: Implement pytest infrastructure
Hi Stephen,
Quoted 31 lines · Michal Simek
Quoted 29 lines · Stephen Warren
Quoted 14 lines · Stephen Warren
This tool aims to test U-Boot by executing U-Boot shell commands using the
console interface. A single top-level script exists to execute or attach to the U-Boot console, run the entire script of tests against it, and summarize the results. Advantages of this approach are:- Testing is performed in the same way a user or script would interact with U-Boot; there can be no disconnect.
- There is no need to write or embed test-related code into U-Boot itself.
It is asserted that writing test-related code in Python is simpler and more flexible that writing it all in C.
- It is reasonably simple to interact with U-Boot in this way.Quoted 8 lines
A few simple tests are provided as examples. Soon, we should convert as many as possible of the other tests in test/* and
test/cmd_ut.ctoo.The hook scripts, relay control utilities, and udev rules I use for my own HW setup are published at
https://github.com/swarren/uboot-test-hooks.See README.md for more details!
It looks like I need to send a v4 of this, since I renamed a Python class but forgot to update all users of it. I didn't notice this, since I had the old module lying around as a *.pyc file, so the old name worked:-(
Quoted 4 lines
Quoted 4 lines
I also have a couple of minor fixes to roll in that make the scripts work better under a continuous integration environment (which doesn't have a controlling TTY set when the scripts run, which need a minor tweak to the Spawn code).
I see this now. Do I need another dependency?
/test/py/test.py --bd sandbox --build
+make O=/usr/local/google/c/cosarm/src/third_party/u-boot/files/build-sandbox -s sandbox_defconfig
+make O=/usr/local/google/c/cosarm/src/third_party/u-boot/files/build-sandbox -s -j8
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File
"/usr/lib/python2.7/dist-packages/_pytest/main.py", line 77, inwrap_session
INTERNALERROR> config.do_configure()
INTERNALERROR> File
"/usr/lib/python2.7/dist-packages/_pytest/config.py", line 607, in do_configure
INTERNALERROR> self.hook.pytest_configure(config=self)
INTERNALERROR> File
"/usr/lib/python2.7/dist-packages/_pytest/core.py", line 376, in__call__
INTERNALERROR> return self._docall(methods, kwargs)
INTERNALERROR> File
"/usr/lib/python2.7/dist-packages/_pytest/core.py", line 387, in_docall
INTERNALERROR> res = mc.execute()
INTERNALERROR> File
"/usr/lib/python2.7/dist-packages/_pytest/core.py", line 288, in
execute
INTERNALERROR> res = method(**kwargs)
INTERNALERROR> File
"/usr/local/google/c/cosarm/src/third_party/u-boot/files/test/py/conftest.py", line 163, in pytest_configure
INTERNALERROR> import u_boot_console_sandbox
INTERNALERROR> File
"/usr/local/google/c/cosarm/src/third_party/u-boot/files/test/py/u_boot_console_sandbox.py", line 9, in <module>
INTERNALERROR> from ubspawn import Spawn
Regards,
Simon
PATCH v3 1/7 test/py: Implement pytest infrastructure
Quoted 42 lines · Simon Glass
Hi Stephen,
Quoted 31 lines · Michal Simek
Quoted 29 lines · Stephen Warren
Quoted 14 lines · Stephen Warren
This tool aims to test U-Boot by executing U-Boot shell commands using the
console interface. A single top-level script exists to execute or attach to the U-Boot console, run the entire script of tests against it, and summarize the results. Advantages of this approach are:- Testing is performed in the same way a user or script would interact with U-Boot; there can be no disconnect.
- There is no need to write or embed test-related code into U-Boot itself.
It is asserted that writing test-related code in Python is simpler and more flexible that writing it all in C.
- It is reasonably simple to interact with U-Boot in this way.Quoted 8 lines
A few simple tests are provided as examples. Soon, we should convert as many as possible of the other tests in test/* and
test/cmd_ut.ctoo.The hook scripts, relay control utilities, and udev rules I use for my own HW setup are published at
https://github.com/swarren/uboot-test-hooks.See README.md for more details!
It looks like I need to send a v4 of this, since I renamed a Python class but forgot to update all users of it. I didn't notice this, since I had the old module lying around as a *.pyc file, so the old name worked:-(
Quoted 4 lines
Quoted 4 lines
I also have a couple of minor fixes to roll in that make the scripts work better under a continuous integration environment (which doesn't have a controlling TTY set when the scripts run, which need a minor tweak to the Spawn code).
I see this now. Do I need another dependency?
/test/py/test.py --bd sandbox --build
...
Quoted 1 line
INTERNALERROR> from ubspawn import Spawn
No, I renamed the ubspawn class (to u_boot_spawn) but forgot to update the users of the module to use the new name. My local testing didn't notice this since I still had the .pyc file present with the old name, but you evidently don't.
I think something like the following should fix it for you before I post v4:
sed -i 's/ubspawn/u_boot_spawn/' test/py/*.py
PATCH v3 1/7 test/py: Implement pytest infrastructure
In reply to Stephen WarrenHi Stephen,
Quoted 22 lines · Stephen Warren
Quoted 21 lines · Stephen Warren
This tool aims to test U-Boot by executing U-Boot shell commands using the
console interface. A single top-level script exists to execute or attach to the U-Boot console, run the entire script of tests against it, and summarize the results. Advantages of this approach are:- Testing is performed in the same way a user or script would interact with U-Boot; there can be no disconnect.
- There is no need to write or embed test-related code into U-Boot itself.
It is asserted that writing test-related code in Python is simpler and more flexible that writing it all in C.
- It is reasonably simple to interact with U-Boot in this way.A few simple tests are provided as examples. Soon, we should convert as many as possible of the other tests in test/* and
test/cmd_ut.ctoo.The hook scripts, relay control utilities, and udev rules I use for my own HW setup are published at
https://github.com/swarren/uboot-test-hooks.
Quoted 6 lines
Quoted 1 line
See README.md for more details!
It looks like I need to send a v4 of this, since I renamed a Python class but forgot to update all users of it. I didn't notice this, since I had the old module lying around as a *.pyc file, so the old name worked:-(
yep.
Quoted 5 lines
I also have a couple of minor fixes to roll in that make the scripts work better under a continuous integration environment (which doesn't have a controlling TTY set when the scripts run, which need a minor tweak to the Spawn code).
I have cherry picked 3 patches from your repo. 7813ccad9ed2 test/py: reset SIGHUP handler in child processes a2ccb34de3f5 test/py: fix ubspawn rename fallout 6dbcd7408f9e test/py: add a test for the sleep command
sleep one is missing test for cmd_misc which enables that.
There is still problem with handling special characters. For MB I have compilation error and %5e is shown instead of ^
For example:
+make O=/home/monstr/data/disk/u-boot/build-microblaze-generic -s microblaze-generic_defconfig
+make O=/home/monstr/data/disk/u-boot/build-microblaze-generic -s -j8 ../drivers/gpio/xilinx_gpio.c: In function 'xilinx_gpio_ofdata_to_platdata':../drivers/gpio/xilinx_gpio.c:400:13: warning: assignment makes pointer from integer without a cast
priv->regs = dev_get_addr(dev);
%5eBut the rest looks good.
Thanks,
Michal
Signature · 14 lines
Michal Simek, Ing. (M.Eng), OpenPGP -> KeyID: FE3D1F91 w: www.monstr.eu p: +42-0-721842854 Maintainer of Linux kernel - Microblaze cpu - http://www.monstr.eu/fdt/ Maintainer of Linux kernel - Xilinx Zynq ARM architecture Microblaze U-BOOT custodian and responsible for u-boot arm zynq platform -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: <http://lists.denx.de/pipermail/u-boot/attachments/20160111/0651d0d1/attachment.sig>
PATCH v3 1/7 test/py: Implement pytest infrastructure
Quoted 52 lines · Michal Simek
Hi Stephen,
Quoted 22 lines · Stephen Warren
Quoted 21 lines · Stephen Warren
This tool aims to test U-Boot by executing U-Boot shell commands using the
console interface. A single top-level script exists to execute or attach to the U-Boot console, run the entire script of tests against it, and summarize the results. Advantages of this approach are:- Testing is performed in the same way a user or script would interact with U-Boot; there can be no disconnect.
- There is no need to write or embed test-related code into U-Boot itself.
It is asserted that writing test-related code in Python is simpler and more flexible that writing it all in C.
- It is reasonably simple to interact with U-Boot in this way.A few simple tests are provided as examples. Soon, we should convert as many as possible of the other tests in test/* and
test/cmd_ut.ctoo.The hook scripts, relay control utilities, and udev rules I use for my own HW setup are published at
https://github.com/swarren/uboot-test-hooks.Quoted 6 lines
Quoted 1 line
See README.md for more details!
It looks like I need to send a v4 of this, since I renamed a Python class but forgot to update all users of it. I didn't notice this, since I had the old module lying around as a *.pyc file, so the old name worked:-(
yep.
Quoted 5 lines
I also have a couple of minor fixes to roll in that make the scripts work better under a continuous integration environment (which doesn't have a controlling TTY set when the scripts run, which need a minor tweak to the Spawn code).
I have cherry picked 3 patches from your repo.
7813ccad9ed2test/py: reset SIGHUP handler in child processesa2ccb34de3f5test/py: fix ubspawn rename fallout6dbcd7408f9etest/py: add a test for the sleep commandsleep one is missing test for
cmd_miscwhich enables that.There is still problem with handling special characters. For MB I have compilation error and %5e is shown instead of ^
For example:
Quoted 8 lines
+make O=/home/monstr/data/disk/u-boot/build-microblaze-generic -s microblaze-generic_defconfig
+make O=/home/monstr/data/disk/u-boot/build-microblaze-generic -s -j8../drivers/gpio/xilinx_gpio.c: In function 'xilinx_gpio_ofdata_to_platdata':../drivers/gpio/xilinx_gpio.c:400:13: warning: assignment makes pointer from integer without a castpriv->regs = dev_get_addr(dev); %5e
Ah yes. When I first wrote the code, the log file wasn't HTML format, but used ^ as a delimiter for some markup, so I escaped that character so that log data wouldn't corrupt the file format. When I switched to HTML, I forgot to remove that character from the escape list. I'll fold the fix into v4.
PATCH v3 2/7 test/py: test that sandbox exits when asked
In reply to Stephen WarrenTest the sandbox port's implementation of the reset command and SIGHUP handling. These should both cause the U-Boot process to exit gracefully.
---
v3:
- Move test scripts into a sub-directory. Suggested by Michal Simek.
- s/uboot/u[-_]boot/g. Suggested by Simon Glass.
- Typo.
- s/"/'/g. Suggested by Simon Glass.
- Add more documentation. Suggested by Simon Glass.
--- test/py/tests/test_sandbox_exit.py | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) create mode 100644 test/py/tests/test_sandbox_exit.py
test/py/tests/test_sandbox_exit.py new file +24 -0
@@ -0,0 +1,24 @@ +# Copyright (c) 2015 Stephen Warren +# Copyright (c) 2015-2016, NVIDIA CORPORATION. All rights reserved. +# +# SPDX-License-Identifier: GPL-2.0 + +import pytest +import signal + + at pytest.mark.boardspec('sandbox') + at pytest.mark.buildconfigspec('reset') +def test_reset(u_boot_console): + '''Test that the "reset" command exits sandbox process.''' + + u_boot_console.run_command('reset', wait_for_prompt=False) + assert(u_boot_console.validate_exited()) + u_boot_console.ensure_spawned() + + at pytest.mark.boardspec('sandbox') +def test_ctrl_c(u_boot_console): + '''Test that sending SIGINT to sandbox causes it to exit.''' + + u_boot_console.kill(signal.SIGINT) + assert(u_boot_console.validate_exited()) + u_boot_console.ensure_spawned()
Signature · 1 line
2.6.4
PATCH v3 2/7 test/py: test that sandbox exits when asked
Quoted 13 lines · Stephen Warren
Test the sandbox port's implementation of the reset command and SIGHUP handling. These should both cause the U-Boot process to exit gracefully.
Signed-off-by Stephen Warren <[email protected]>Signed-off-by Stephen Warren <[email protected]>---
v3:
- Move test scripts into a sub-directory. Suggested by Michal Simek.
- s/uboot/u[-_]boot/g. Suggested by Simon Glass.
- Typo.
- s/"/'/g. Suggested by Simon Glass.
- Add more documentation. Suggested by Simon Glass.
Quoted 5 lines
--- test/py/tests/test_sandbox_exit.py | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) create mode 100644 test/py/tests/test_sandbox_exit.py
PATCH v3 3/7 test/py: add test of setenv/printenv/echo
In reply to Stephen WarrenThis tests basic environment variable functionality.
---
v3:
- Handle environment values that contain an = sign.
Reported by Michal Simek.
- Move test scripts into a sub-directory.
Suggested by Michal Simek.
- s/uboot/u[-_]boot/g. Suggested by Simon Glass.
- s/"/'/g. Suggested by Simon Glass.
- Add more documentation. Suggested by Simon Glass.
- Simplify command string construction. Suggested by Simon Glass.
- Move relevant edits to command_ut.c into this patch from a later one. Suggested by Simon Glass.
--- test/command_ut.c | 4 -- test/py/tests/test_env.py | 169 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 169 insertions(+), 4 deletions(-) create mode 100644 test/py/tests/test_env.py
test/command_ut.c +0 -4
@@ -20,10 +20,6 @@ static int do_ut_cmd(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[])printf ("%s: Testing commands\n" , __func__);run_command ("env default -f -a" ,0 ); -/* run a single command */ -run_command ("setenv single 1" ,0 ); -assert (!strcmp ("1" ,getenv ("single" ))); -/* make sure that compound statements work */ #ifdef CONFIG_SYS_HUSH_PARSER run_command ("if test -n ${single} ; then setenv check 1; fi" ,0 );
test/py/tests/test_env.py new file +169 -0
169 lines not on this page · raw message
Signature · 1 line
2.6.4
PATCH v3 3/7 test/py: add test of setenv/printenv/echo
Quoted 16 lines · Stephen Warren
This tests basic environment variable functionality.
Signed-off-by Stephen Warren <[email protected]>Signed-off-by Stephen Warren <[email protected]>---
v3:
- Handle environment values that contain an = sign.
Reported by Michal Simek.
- Move test scripts into a sub-directory.
Suggested by Michal Simek.
- s/uboot/u[-_]boot/g. Suggested by Simon Glass.
- s/"/'/g. Suggested by Simon Glass.
- Add more documentation. Suggested by Simon Glass.
- Simplify command string construction. Suggested by Simon Glass.
- Move relevant edits tocommand_ut.cinto this patch from a later one. Suggested by Simon Glass.
Quoted 5 lines
--- test/command_ut.c | 4 -- test/py/tests/test_env.py | 169 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 169 insertions(+), 4 deletions(-) create mode 100644 test/py/tests/test_env.py
PATCH v3 4/7 test/py: test the md/mw commands
In reply to Stephen WarrenThis tests whether md/mw work, and affect each-other.
Command repeat is also tested.
test/cmd_repeat.sh is removed, since the new Python-based test does everything it used to.
---
v3:
- Add extra mw during md test to account for the possibility that the test's write data may already be present in RAM. Suggested by Michal Simek.
- Move test scripts into a sub-directory.
Suggested by Michal Simek.
- s/uboot/u[-_]boot/g. Suggested by Simon Glass.
- s/"/'/g. Suggested by Simon Glass.
- Add more documentation. Suggested by Simon Glass.
--- test/cmd_repeat.sh | 29 ----------------------------- test/py/tests/test_md.py | 36 ++++++++++++++++++++++++++++++++++++ 2 files changed, 36 insertions(+), 29 deletions(-) delete mode 100755 test/cmd_repeat.sh create mode 100644 test/py/tests/test_md.py
test/cmd_repeat.sh deleted +0 -29
@@ -1,29 +0,0 @@ -#!/bin/sh - -# Test for U-Boot cli including command repeat - -BASE="$(dirname $0)" -. $BASE/common.sh - -run_test() { - ./${OUTPUT_DIR}/u-boot <<END -setenv ctrlc_ignore y -md 0 - -reset -END -} -check_results() { - echo "Check results" - - grep -q 00000100 ${tmp} || fail "Command did not repeat" -} - -echo "Test CLI repeat" -echo -tmp="$(tempfile)" -build_uboot -run_test >${tmp} -check_results ${tmp} -rm ${tmp} -echo "Test passed"
test/py/tests/test_md.py new file +36 -0
@@ -0,0 +1,36 @@ +# Copyright (c) 2015 Stephen Warren +# Copyright (c) 2015-2016, NVIDIA CORPORATION. All rights reserved. +# +# SPDX-License-Identifier: GPL-2.0 + +import pytest + + at pytest.mark.buildconfigspec('cmd_memory') +def test_md(u_boot_console): + '''Test that md reads memory as expected, and that memory can be modified + using the mw command.''' + + ram_base = u_boot_console.find_ram_base() + addr = '%08x' % ram_base + val = 'a5f09876' + expected_response = addr + ': ' + val + u_boot_console.run_command('mw ' + addr + ' 0 10') + response = u_boot_console.run_command('md ' + addr + ' 10') + assert(not (expected_response in response)) + u_boot_console.run_command('mw ' + addr + ' ' + val) + response = u_boot_console.run_command('md ' + addr + ' 10') + assert(expected_response in response) + + at pytest.mark.buildconfigspec('cmd_memory') +def test_md_repeat(u_boot_console): + '''Test command repeat (via executing an empty command) operates correctly + for "md"; the command must repeat and dump an incrementing address.''' + + ram_base = u_boot_console.find_ram_base() + addr_base = '%08x' % ram_base + words = 0x10 + addr_repeat = '%08x' % (ram_base + (words * 4)) + u_boot_console.run_command('md %s %x' % (addr_base, words)) + response = u_boot_console.run_command('') + expected_response = addr_repeat + ': ' + assert(expected_response in response)
Signature · 1 line
2.6.4
PATCH v3 4/7 test/py: test the md/mw commands
Quoted 19 lines · Stephen Warren
This tests whether md/mw work, and affect each-other.
Command repeat is also tested.
test/cmd_repeat.shis removed, since the new Python-based test does everything it used to.Signed-off-by Stephen Warren <[email protected]>Signed-off-by Stephen Warren <[email protected]>---
v3:
- Add extra mw during md test to account for the possibility that the test's write data may already be present in RAM. Suggested by Michal Simek.
- Move test scripts into a sub-directory.
Suggested by Michal Simek.
- s/uboot/u[-_]boot/g. Suggested by Simon Glass.
- s/"/'/g. Suggested by Simon Glass.
- Add more documentation. Suggested by Simon Glass.
Quoted 6 lines
--- test/cmd_repeat.sh | 29 ----------------------------- test/py/tests/test_md.py | 36 ++++++++++++++++++++++++++++++++++++ 2 files changed, 36 insertions(+), 29 deletions(-) delete mode 100755 test/cmd_repeat.sh create mode 100644 test/py/tests/test_md.py
PATCH v3 5/7 test/py: add test of basic shell functionality
In reply to Stephen WarrenFrom: Stephen Warren <[email protected]>
This tests whether the following features of the U-Boot shell:
- Execution of a directly entered command.
- Compound commands (; delimiter).
- Quoting of arguments containing spaces.
- Executing commands from environment variables.
---
v3:
- Move test scripts into a sub-directory.
Suggested by Michal Simek.
- s/uboot/u[-_]boot/g. Suggested by Simon Glass.
- s/"/'/g. Suggested by Simon Glass.
- Add more documentation. Suggested by Simon Glass.
- Move relevant edits to command_ut.c into this patch from a later one. Suggested by Simon Glass.
--- test/command_ut.c | 16 --------------- test/py/tests/test_shell_basics.py | 42 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 42 insertions(+), 16 deletions(-) create mode 100644 test/py/tests/test_shell_basics.py
test/command_ut.c +0 -16
@@ -27,10 +27,6 @@ static int do_ut_cmd(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[])run_command ("setenv check" ,0 );#endif -/* commands separated by ; */ -run_command_list ("setenv list 1; setenv list ${list}1" ,-1 ,0 ); -assert (!strcmp ("11" ,getenv ("list" ))); -/* commands separated by \n */ run_command_list ("setenv list 1\n setenv list ${list}1" ,-1 ,0 );assert (!strcmp ("11" ,getenv ("list" )));
@@ -39,11 +35,6 @@ static int do_ut_cmd(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[])run_command_list ("setenv list 1${list}\n" ,-1 ,0 );assert (!strcmp ("111" ,getenv ("list" ))); -/* three commands in a row */ -run_command_list ("setenv list 1\n setenv list ${list}2; " -"setenv list ${list}3" ,-1 ,0 ); -assert (!strcmp ("123" ,getenv ("list" ))); -/* a command string with \0 in it. Stuff after \0 should be ignored */ run_command ("setenv list" ,0 );run_command_list (test_cmd,sizeof (test_cmd),0 );
@@ -62,13 +53,6 @@ static int do_ut_cmd(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[])assert (run_command_list ("false" ,-1 ,0 ) ==1 );assert (run_command_list ("echo" ,-1 ,0 ) ==0 ); -run_command ("setenv foo 'setenv monty 1; setenv python 2'" ,0 ); -run_command ("run foo" ,0 ); -assert (getenv ("monty" ) !=NULL ); -assert (!strcmp ("1" ,getenv ("monty" ))); -assert (getenv ("python" ) !=NULL ); -assert (!strcmp ("2" ,getenv ("python" ))); -#ifdef CONFIG_SYS_HUSH_PARSER run_command ("setenv foo 'setenv black 1\nsetenv adder 2'" ,0 );run_command ("run foo" ,0 );
test/py/tests/test_shell_basics.py new file +42 -0
42 lines not on this page · raw message
Signature · 1 line
2.6.4
PATCH v3 5/7 test/py: add test of basic shell functionality
Quoted 18 lines · Stephen Warren
From: Stephen Warren <[email protected]>
This tests whether the following features of the U-Boot shell:
- Execution of a directly entered command.
- Compound commands (; delimiter).
- Quoting of arguments containing spaces.
- Executing commands from environment variables.Signed-off-by Stephen Warren <[email protected]>---
v3:
- Move test scripts into a sub-directory.
Suggested by Michal Simek.
- s/uboot/u[-_]boot/g. Suggested by Simon Glass.
- s/"/'/g. Suggested by Simon Glass.
- Add more documentation. Suggested by Simon Glass.
- Move relevant edits tocommand_ut.cinto this patch from a later one. Suggested by Simon Glass.
Quoted 5 lines
--- test/command_ut.c | 16 --------------- test/py/tests/test_shell_basics.py | 42 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 42 insertions(+), 16 deletions(-) create mode 100644 test/py/tests/test_shell_basics.py
PATCH v3 6/7 test/py: test the shell if command
In reply to Stephen WarrenFrom: Stephen Warren <[email protected]>
Migrate all most tests from command_ut.c into the Python test system. This allows the tests to be run against any U-Boot binary that supports the if command (i.e. where hush is enabled) without requiring that binary to be permanently bloated with the code from command_ut.
Some tests in command_ut.c can only be executed from C code, since they test internal (more unit-level) features of various U-Boot APIs. The migrated tests can all operate directly from the U-Boot console.
---
v3:
- Move test scripts into a sub-directory. Suggested by Michal Simek.
- s/uboot/u[-_]boot/g. Suggested by Simon Glass.
- s/"/'/g. Suggested by Simon Glass.
- Add more documentation. Suggested by Simon Glass.
- Remove inclusion of <os.h>. Suggested by Simon Glass.
--- test/command_ut.c | 116 ---------------------------- test/py/tests/test_hush_if_test.py | 154 +++++++++++++++++++++++++++++++++++++ 2 files changed, 154 insertions(+), 116 deletions(-) create mode 100644 test/py/tests/test_hush_if_test.py
test/command_ut.c +0 -116
@@ -7,9 +7,6 @@#define DEBUG #include <common.h> -#ifdef CONFIG_SANDBOX -#include <os.h> -#endif static const char test_cmd[] ="setenv list 1\n setenv list ${list}2; " "setenv list ${list}3\0"
@@ -20,13 +17,6 @@ static int do_ut_cmd(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[])printf ("%s: Testing commands\n" , __func__);run_command ("env default -f -a" ,0 ); -/* make sure that compound statements work */ -#ifdef CONFIG_SYS_HUSH_PARSER -run_command ("if test -n ${single} ; then setenv check 1; fi" ,0 ); -assert (!strcmp ("1" ,getenv ("check" ))); -run_command ("setenv check" ,0 ); -#endif -/* commands separated by \n */ run_command_list ("setenv list 1\n setenv list ${list}1" ,-1 ,0 );assert (!strcmp ("11" ,getenv ("list" )));
@@ -60,112 +50,6 @@ static int do_ut_cmd(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[])assert (!strcmp ("1" ,getenv ("black" )));assert (getenv ("adder" ) !=NULL );assert (!strcmp ("2" ,getenv ("adder" ))); - -/* Test the 'test' command */ - -#define HUSH_TEST (name, expr, expected_result) \ - run_command("if test " expr " ; then " \ - "setenv " #name "_" #expected_result " y; else " \ - "setenv " #name "_" #expected_result " n; fi", 0); \ - assert(!strcmp(#expected_result, getenv(#name "_" #expected_result))); \ - setenv(#name "_" #expected_result, NULL); - -/* Basic operators */ -HUSH_TEST (streq,"aaa = aaa" , y); -HUSH_TEST (streq,"aaa = bbb" , n); - -HUSH_TEST (strneq,"aaa != bbb" , y); -HUSH_TEST (strneq,"aaa != aaa" , n); - -HUSH_TEST (strlt,"aaa < bbb" , y); -HUSH_TEST (strlt,"bbb < aaa" , n); - -HUSH_TEST (strgt,"bbb > aaa" , y); -HUSH_TEST (strgt,"aaa > bbb" , n); - -HUSH_TEST (eq,"123 -eq 123" , y); -HUSH_TEST (eq,"123 -eq 456" , n); - -HUSH_TEST (ne,"123 -ne 456" , y); -HUSH_TEST (ne,"123 -ne 123" , n); - -HUSH_TEST (lt,"123 -lt 456" , y); -HUSH_TEST (lt_eq,"123 -lt 123" , n); -HUSH_TEST (lt,"456 -lt 123" , n); - -HUSH_TEST (le,"123 -le 456" , y); -HUSH_TEST (le_eq,"123 -le 123" , y); -HUSH_TEST (le,"456 -le 123" , n); - -HUSH_TEST (gt,"456 -gt 123" , y); -HUSH_TEST (gt_eq,"123 -gt 123" , n); -HUSH_TEST (gt,"123 -gt 456" , n); - -HUSH_TEST (ge,"456 -ge 123" , y); -HUSH_TEST (ge_eq,"123 -ge 123" , y); -HUSH_TEST (ge,"123 -ge 456" , n); - -HUSH_TEST (z,"-z \"\"" , y); -HUSH_TEST (z,"-z \"aaa\"" , n); - -HUSH_TEST (n,"-n \"aaa\"" , y); -HUSH_TEST (n,"-n \"\"" , n); - -/* Inversion of simple tests */ -HUSH_TEST (streq_inv,"! aaa = aaa" , n); -HUSH_TEST (streq_inv,"! aaa = bbb" , y); - -HUSH_TEST (streq_inv_inv,"! ! aaa = aaa" , y); -HUSH_TEST (streq_inv_inv,"! ! aaa = bbb" , n); - -/* Binary operators */ -HUSH_TEST (or_0_0,"aaa != aaa -o bbb != bbb" , n); -HUSH_TEST (or_0_1,"aaa != aaa -o bbb = bbb" , y); -HUSH_TEST (or_1_0,"aaa = aaa -o bbb != bbb" , y); -HUSH_TEST (or_1_1,"aaa = aaa -o bbb = bbb" , y); - -HUSH_TEST (and_0_0,"aaa != aaa -a bbb != bbb" , n); -HUSH_TEST (and_0_1,"aaa != aaa -a bbb = bbb" , n); -HUSH_TEST (and_1_0,"aaa = aaa -a bbb != bbb" , n); -HUSH_TEST (and_1_1,"aaa = aaa -a bbb = bbb" , y); - -/* Inversion within binary operators */ -HUSH_TEST (or_0_0_inv,"! aaa != aaa -o ! bbb != bbb" , y); -HUSH_TEST (or_0_1_inv,"! aaa != aaa -o ! bbb = bbb" , y); -HUSH_TEST (or_1_0_inv,"! aaa = aaa -o ! bbb != bbb" , y); -HUSH_TEST (or_1_1_inv,"! aaa = aaa -o ! bbb = bbb" , n); - -HUSH_TEST (or_0_0_inv_inv,"! ! aaa != aaa -o ! ! bbb != bbb" , n); -HUSH_TEST (or_0_1_inv_inv,"! ! aaa != aaa -o ! ! bbb = bbb" , y); -HUSH_TEST (or_1_0_inv_inv,"! ! aaa = aaa -o ! ! bbb != bbb" , y); -HUSH_TEST (or_1_1_inv_inv,"! ! aaa = aaa -o ! ! bbb = bbb" , y); - -setenv ("ut_var_nonexistent" ,NULL ); -setenv ("ut_var_exists" ,"1" ); -HUSH_TEST (z_varexp_quoted,"-z \"$ut_var_nonexistent\"" , y); -HUSH_TEST (z_varexp_quoted,"-z \"$ut_var_exists\"" , n); -setenv ("ut_var_exists" ,NULL ); - -run_command ("setenv ut_var_space \" \"" ,0 ); -assert (!strcmp (getenv ("ut_var_space" )," " )); -run_command ("setenv ut_var_test $ut_var_space" ,0 ); -assert (!getenv ("ut_var_test" )); -run_command ("setenv ut_var_test \"$ut_var_space\"" ,0 ); -assert (!strcmp (getenv ("ut_var_test" )," " )); -run_command ("setenv ut_var_test \" 1${ut_var_space}${ut_var_space} 2 \"" ,0 ); -assert (!strcmp (getenv ("ut_var_test" )," 1 2 " )); -setenv ("ut_var_space" ,NULL ); -setenv ("ut_var_test" ,NULL ); - -#ifdef CONFIG_SANDBOX -/* File existence */ -HUSH_TEST (e,"-e hostfs - creating_this_file_breaks_uboot_unit_test" , n); -run_command ("sb save hostfs - creating_this_file_breaks_uboot_unit_test 0 1" ,0 ); -HUSH_TEST (e,"-e hostfs - creating_this_file_breaks_uboot_unit_test" , y); -/* Perhaps this could be replaced by an "rm" shell command one day */ -assert (!os_unlink ("creating_this_file_breaks_uboot_unit_test" )); -HUSH_TEST (e,"-e hostfs - creating_this_file_breaks_uboot_unit_test" , n); -#endif #endif assert (run_command ("" ,0 ) ==0 );
test/py/tests/test_hush_if_test.py new file +154 -0
154 lines not on this page · raw message
Signature · 1 line
2.6.4
PATCH v3 6/7 test/py: test the shell if command
Quoted 20 lines · Stephen Warren
From: Stephen Warren <[email protected]>
Migrate all most tests from
command_ut.cinto the Python test system. This allows the tests to be run against any U-Boot binary that supports the if command (i.e. where hush is enabled) without requiring that binary to be permanently bloated with the code fromcommand_ut.Some tests in
command_ut.ccan only be executed from C code, since they test internal (more unit-level) features of various U-Boot APIs. The migrated tests can all operate directly from the U-Boot console.Signed-off-by Stephen Warren <[email protected]>---
v3:
- Move test scripts into a sub-directory. Suggested by Michal Simek.
- s/uboot/u[-_]boot/g. Suggested by Simon Glass.
- s/"/'/g. Suggested by Simon Glass.
- Add more documentation. Suggested by Simon Glass.
- Remove inclusion of <os.h>. Suggested by Simon Glass.
Quoted 5 lines
--- test/command_ut.c | 116 ---------------------------- test/py/tests/test_hush_if_test.py | 154 +++++++++++++++++++++++++++++++++++++ 2 files changed, 154 insertions(+), 116 deletions(-) create mode 100644 test/py/tests/test_hush_if_test.py
PATCH v3 7/7 test/py: test the ums command
In reply to Stephen WarrenFrom: Stephen Warren <[email protected]>
This test invokes the "ums" command in U-Boot, and validates that a USB storage device is enumerated on the test host system, and can be read from.
---
v3:
- Move test scripts into a sub-directory.
Suggested by Michal Simek.
- s/uboot/u[-_]boot/g. Suggested by Simon Glass.
- s/"/'/g. Suggested by Simon Glass.
- Add more documentation. Suggested by Simon Glass.
- Rename boardenv USB device node variable to avoid naming conflicts if we add support for other types of USB device later (e.g. CDC ACM, CDC Ethernet, DFU).
--- test/py/tests/test_ums.py | 94 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 94 insertions(+) create mode 100644 test/py/tests/test_ums.py
test/py/tests/test_ums.py new file +94 -0
@@ -0,0 +1,94 @@ +# Copyright (c) 2015-2016, NVIDIA CORPORATION. All rights reserved. +# +# SPDX-License-Identifier: GPL-2.0 + +# Test U-Boot's "ums" command. At present, this test only ensures that a UMS +# device can be enumerated by the host/test machine. In the future, this test +# should be enhanced to validate disk IO. + +import os +import pytest +import time + +''' +Note: This test relies on: + +a) boardenv_* to contain configuration values to define which USB ports are +available for testing. Without this, this test will be automatically skipped. +For example: + +env__usb_dev_ports = ( + {'tgt_usb_ctlr': '0', 'host_ums_dev_node': '/dev/disk/by-path/pci-0000:00:14.0-usb-0:13:1.0-scsi-0:0:0:0'}, +) + +env__block_devs = ( + {'type': 'mmc', 'id': '0'}, # eMMC; always present + {'type': 'mmc', 'id': '1'}, # SD card; present since I plugged one in +) + +b) udev rules to set permissions on devices nodes, so that sudo is not +required. For example: + +ACTION=="add", SUBSYSTEM=="block", SUBSYSTEMS=="usb", KERNELS=="3-13", MODE:="666" + +(You may wish to change the group ID instead of setting the permissions wide +open. All that matters is that the user ID running the test can access the +device.) +''' + +def open_ums_device(host_ums_dev_node): + '''Attempt to open a device node, returning either the opened file handle, + or None on any error.''' + + try: + return open(host_ums_dev_node, 'rb') + except: + return None + +def wait_for_ums_device(host_ums_dev_node): + '''Continually attempt to open the device node exported by the "ums" + command, and either return the opened file handle, or raise an exception + after a timeout.''' + + for i in xrange(100): + fh = open_ums_device(host_ums_dev_node) + if fh: + return fh + time.sleep(0.1) + raise Exception('UMS device did not appear') + +def wait_for_ums_device_gone(host_ums_dev_node): + '''Continually attempt to open the device node exported by the "ums" + command, and either return once the device has disappeared, or raise an + exception if it does not before a timeout occurs.''' + + for i in xrange(100): + fh = open_ums_device(host_ums_dev_node) + if not fh: + return + fh.close() + time.sleep(0.1) + raise Exception('UMS device did not disappear') + + at pytest.mark.buildconfigspec('cmd_usb_mass_storage') +def test_ums(u_boot_console, env__usb_dev_port, env__block_devs): + '''Test the "ums" command; the host system must be able to enumerate a UMS + device when "ums" is running, and this device must disappear when "ums" is + aborted.''' + + tgt_usb_ctlr = env__usb_dev_port['tgt_usb_ctlr'] + host_ums_dev_node = env__usb_dev_port['host_ums_dev_node'] + + # We're interested in testing USB device mode on each port, not the cross- + # product of that with each device. So, just pick the first entry in the + # device list here. We'll test each block device somewhere else. + tgt_dev_type = env__block_devs[0]['type'] + tgt_dev_id = env__block_devs[0]['id'] + + cmd = 'ums %s %s %s' % (tgt_usb_ctlr, tgt_dev_type, tgt_dev_id) + u_boot_console.run_command('ums 0 mmc 0', wait_for_prompt=False) + fh = wait_for_ums_device(host_ums_dev_node) + fh.read(4096) + fh.close() + u_boot_console.ctrlc() + wait_for_ums_device_gone(host_ums_dev_node)
Signature · 1 line
2.6.4
PATCH v3 7/7 test/py: test the ums command
Quoted 17 lines · Stephen Warren
From: Stephen Warren <[email protected]>
This test invokes the "ums" command in U-Boot, and validates that a USB storage device is enumerated on the test host system, and can be read from.
Signed-off-by Stephen Warren <[email protected]>---
v3:
- Move test scripts into a sub-directory.
Suggested by Michal Simek.
- s/uboot/u[-_]boot/g. Suggested by Simon Glass.
- s/"/'/g. Suggested by Simon Glass.
- Add more documentation. Suggested by Simon Glass.
- Rename boardenv USB device node variable to avoid naming conflicts if we add support for other types of USB device later (e.g. CDC ACM, CDC Ethernet, DFU).
Quoted 4 lines
--- test/py/tests/test_ums.py | 94 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 94 insertions(+) create mode 100644 test/py/tests/test_ums.py
PATCH v3 1/7 test/py: Implement pytest infrastructure
In reply to Stephen WarrenQuoted 62 lines · Stephen Warren
This tool aims to test U-Boot by executing U-Boot shell commands using the console interface. A single top-level script exists to execute or attach to the U-Boot console, run the entire script of tests against it, and summarize the results. Advantages of this approach are:
- Testing is performed in the same way a user or script would interact with U-Boot; there can be no disconnect.
- There is no need to write or embed test-related code into U-Boot itself. It is asserted that writing test-related code in Python is simpler and more flexible that writing it all in C.
- It is reasonably simple to interact with U-Boot in this way.A few simple tests are provided as examples. Soon, we should convert as many as possible of the other tests in test/* and
test/cmd_ut.ctoo.The hook scripts, relay control utilities, and udev rules I use for my own HW setup are published at https://github.com/swarren/uboot-test-hooks.
See README.md for more details!
Signed-off-by Stephen Warren <[email protected]>Signed-off-by Stephen Warren <[email protected]>Tested-by Michal Simek <[email protected]>Tested-by Simon Glass <[email protected]>---
v3:
- Rework HTML log generation so that TAB characters render as expected. Suggested by Michal Simek.
- Move test scripts into a sub-directory.
Suggested by Michal Simek.
- s/uboot/u[-_]boot/g. Suggested by Simon Glass.
- s/"/'/g. Suggested by Simon Glass.
- Typo fixes.
- Add more documentation. Suggested by Simon Glass.
- Make "notes" in the log file be <pre> so that their formatting is preserved. This is useful for large notes such as exception dumps.v2:
- Many fixes and tweaks have been squashed in. Separated out some of the tests into separate commits, and added some more tests.--- test/py/.gitignore | 1 + test/py/README.md | 300 ++++++++++++++++++++++++++++++ test/py/conftest.py | 335 ++++++++++++++++++++++++++++++++++ test/py/multiplexed_log.css | 88 +++++++++ test/py/multiplexed_log.py | 335 ++++++++++++++++++++++++++++++++++ test/py/pytest.ini | 11 ++ test/py/test.py | 32 ++++ test/py/tests/test_000_version.py | 20 ++ test/py/tests/test_help.py | 9 + test/py/tests/test_unknown_cmd.py | 14 ++ test/py/u_boot_console_base.py | 260 ++++++++++++++++++++++++++ test/py/u_boot_console_exec_attach.py | 51 ++++++ test/py/u_boot_console_sandbox.py | 48 +++++ test/py/u_boot_spawn.py | 123 +++++++++++++ 14 files changed, 1627 insertions(+) create mode 100644 test/py/.gitignore create mode 100644 test/py/README.md create mode 100644 test/py/conftest.py create mode 100644 test/py/multiplexed_log.css create mode 100644 test/py/multiplexed_log.py create mode 100644 test/py/pytest.ini
Quoted 8 lines
create mode 100755 test/py/test.py create mode 100644 test/py/tests/test_000_version.py create mode 100644 test/py/tests/test_help.py create mode 100644 test/py/tests/test_unknown_cmd.py create mode 100644 test/py/u_boot_console_base.py create mode 100644 test/py/u_boot_console_exec_attach.py create mode 100644 test/py/u_boot_console_sandbox.py create mode 100644 test/py/u_boot_spawn.py
Looks good!
One comment about function / class comments. If you like at buildman/patman they use this sort of format:
def RunCommit(self, commit_upto, brd, work_dir, do_config, force_build,
force_build_failures):
"""Build a particular commit.
If the build is already done, and we are not forcing a build, we skip the build and just return the previously-saved results.
Args:
commit_upto: Commit number to build (0...n-1)
brd: Board object to build
work_dir: Directory to which the source will be checked out
do_config: True to run a make <board>_defconfig on the source
force_build: Force a build even if one was previously done
force_build_failures: Force a bulid if the previous result showed
failure
Returns:
tuple containing:
- CommandResult object containing the results of the build
- boolean indicating whether 'make config' is still needed
"""
So double quotes, and the first line describes the function in a very simple way, a bit like a git commit subject. The arguments and return value are specifically called out, as we do in the C code.
For consistency I'd suggest doing the same.
Regards,
Simon