Skip to contentSkip to conversation

test/py: Implement pytest infrastructure

PATCH New Stephen Warren 3 participants 17 messages Jan 15, 2016 – Jan 21, 2016

lore
Share & tools
lore
#0 lore raw

PATCH v4 1/8 test/py: Implement pytest infrastructure

This tool aims to test U-Boot by executing U-Boot shell commands using the console interface. A single top-level script exists to execute or attach to the U-Boot console, run the entire script of tests against it, and summarize the results. Advantages of this approach are:

- Testing is performed in the same way a user or script would interact with U-Boot; there can be no disconnect.
- There is no need to write or embed test-related code into U-Boot itself. It is asserted that writing test-related code in Python is simpler and more flexible that writing it all in C.
- It is reasonably simple to interact with U-Boot in this way.

A few simple tests are provided as examples. Soon, we should convert as many as possible of the other tests in test/* and test/cmd_ut.c too.

The hook scripts, relay control utilities, and udev rules I use for my own HW setup are published at https://github.com/swarren/uboot-test-hooks.

See README.md for more details!

---
Apologies for the resend; I forgot to CC the list last time.

v4:
- Fix all users after ubspawn -> u_boot_spawn module rename.
- Don't escape ^ in the HTML log file.
- Fix u_boot_spawn.py Spawn.__init__ to set SIGHUP handler is set to SIG_DFL. This ensures that sandbox responds to SIGHUP when run non- interactively (e.g. under a daemon continuous integration server).
- Fix logging of all child processes to make sure that all stdout/err, exit code, and exception data from all processes makes its way to both test stdout (for pytest to log to its stdout in case of error) and the HTML test log. This changed multiplexed_log.py RunAndLog.run() (a fair bit of change) and u_boot_console_exec_attach.py ConseolExecAttach.* (just passing sys.stdout to log.get_runner() everywhere).
- Also mention kermit as a possible app to attach to a serial console.
- Fix test_version to work correctly when BUILD_TAG is set, which causes U-Boot to print both the version string and BUILD_TAG value at startup, whereas the "version" shell command only prints the version string.
- Reformat function-level documentation.

v3:
- Rework HTML log generation so that TAB characters render as expected. Suggested by Michal Simek.
- Move test scripts into a sub-directory.
Suggested by Michal Simek.
- s/uboot/u[-_]boot/g. Suggested by Simon Glass.
- s/"/'/g. Suggested by Simon Glass.
- Typo fixes.
- Add more documentation. Suggested by Simon Glass.
- Make "notes" in the log file be <pre> so that their formatting is preserved. This is useful for large notes such as exception dumps.

v2:
- Many fixes and tweaks have been squashed in. Separated out some of the tests into separate commits, and added some more tests.

---
 test/py/.gitignore                    |   1 +
 test/py/README.md                     | 300 ++++++++++++++++++++
 test/py/conftest.py                   | 422 ++++++++++++++++++++++++++++
 test/py/multiplexed_log.css           |  88 ++++++
 test/py/multiplexed_log.py            | 515 ++++++++++++++++++++++++++++++++++
 test/py/pytest.ini                    |  11 +
 test/py/test.py                       |  32 +++
 test/py/tests/test_000_version.py     |  20 ++
 test/py/tests/test_help.py            |   9 +
 test/py/tests/test_unknown_cmd.py     |  14 +
 test/py/u_boot_console_base.py        | 360 ++++++++++++++++++++++++
 test/py/u_boot_console_exec_attach.py |  65 +++++
 test/py/u_boot_console_sandbox.py     |  79 ++++++
 test/py/u_boot_spawn.py               | 174 ++++++++++++
 14 files changed, 2090 insertions(+)
 create mode 100644 test/py/.gitignore
 create mode 100644 test/py/README.md
 create mode 100644 test/py/conftest.py
 create mode 100644 test/py/multiplexed_log.css
 create mode 100644 test/py/multiplexed_log.py
 create mode 100644 test/py/pytest.ini
 create mode 100755 test/py/test.py
 create mode 100644 test/py/tests/test_000_version.py
 create mode 100644 test/py/tests/test_help.py
 create mode 100644 test/py/tests/test_unknown_cmd.py
 create mode 100644 test/py/u_boot_console_base.py
 create mode 100644 test/py/u_boot_console_exec_attach.py
 create mode 100644 test/py/u_boot_console_sandbox.py
 create mode 100644 test/py/u_boot_spawn.py
test/py/.gitignore new file +1 -0
new file mode 100644
index 000000000000..0d20b6487c61
@@ -0,0 +1 @@
+*.pyc
test/py/README.md new file +300 -0

300 lines not on this page · raw message

test/py/conftest.py new file +422 -0

422 lines not on this page · raw message

test/py/multiplexed_log.css new file +88 -0

88 lines not on this page · raw message

test/py/multiplexed_log.py new file +515 -0

515 lines not on this page · raw message

test/py/pytest.ini new file +11 -0
new file mode 100644
index 000000000000..67e514f42058
@@ -0,0 +1,11 @@
+# Copyright (c) 2015 Stephen Warren
+# Copyright (c) 2015-2016, NVIDIA CORPORATION. All rights reserved.
+#
+# SPDX-License-Identifier: GPL-2.0
+
+# Static configuration data for pytest. pytest reads this at startup time.
+
+[pytest]
+markers =
+    boardspec: U-Boot: Describes the set of boards a test can/can't run on.
+    buildconfigspec: U-Boot: Describes Kconfig/config-header constraints.
test/py/test.py new file +32 -0
new file mode 100755
index 000000000000..9c23898774ed
@@ -0,0 +1,32 @@
+#!/usr/bin/env python
+
+# Copyright (c) 2015 Stephen Warren
+# Copyright (c) 2015-2016, NVIDIA CORPORATION. All rights reserved.
+#
+# SPDX-License-Identifier: GPL-2.0
+
+# Wrapper script to invoke pytest with the directory name that contains the
+# U-Boot tests.
+
+import os
+import os.path
+import sys
+
+# Get rid of argv[0]
+sys.argv.pop(0)
+
+# argv; py.test test_directory_name user-supplied-arguments
+args = ["py.test", os.path.dirname(__file__) + "/tests"]
+args.extend(sys.argv)
+
+try:
+    os.execvp("py.test", args)
+except:
+    # Log full details of any exception for detailed analysis
+    import traceback
+    traceback.print_exc()
+    # Hint to the user that they likely simply haven't installed the required
+    # dependencies.
+    print >>sys.stderr, """
+exec(py.test) failed; perhaps you are missing some dependencies?
+See test/py/README.md for the list."""
test/py/tests/test_000_version.py new file +20 -0
new file mode 100644
index 000000000000..d262f0534efd
@@ -0,0 +1,20 @@
+# Copyright (c) 2015 Stephen Warren
+# Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
+#
+# SPDX-License-Identifier: GPL-2.0
+
+# pytest runs tests the order of their module path, which is related to the
+# filename containing the test. This file is named such that it is sorted
+# first, simply as a very basic sanity check of the functionality of the U-Boot
+# command prompt.
+
+def test_version(u_boot_console):
+    '''Test that the "version" command prints the U-Boot version.'''
+
+    # "version" prints the U-Boot sign-on message. This is usually considered
+    # an error, so that any unexpected reboot causes an error. Here, this
+    # error detection is disabled since the sign-on message is expected.
+    with u_boot_console.disable_check('main_signon'):
+        response = u_boot_console.run_command('version')
+    # Ensure "version" printed what we expected.
+    u_boot_console.validate_version_string_in_text(response)
test/py/tests/test_help.py new file +9 -0
new file mode 100644
index 000000000000..894f3b5f1700
@@ -0,0 +1,9 @@
+# Copyright (c) 2015 Stephen Warren
+# Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
+#
+# SPDX-License-Identifier: GPL-2.0
+
+def test_help(u_boot_console):
+    '''Test that the "help" command can be executed.'''
+
+    u_boot_console.run_command('help')
test/py/tests/test_unknown_cmd.py new file +14 -0
new file mode 100644
index 000000000000..2de93e0026ff
@@ -0,0 +1,14 @@
+# Copyright (c) 2015 Stephen Warren
+# Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
+#
+# SPDX-License-Identifier: GPL-2.0
+
+def test_unknown_command(u_boot_console):
+    '''Test that executing an unknown command causes U-Boot to print an
+    error.'''
+
+    # The "unknown command" error is actively expected here,
+    # so error detection for it is disabled.
+    with u_boot_console.disable_check('unknown_command'):
+        response = u_boot_console.run_command('non_existent_cmd')
+    assert('Unknown command \'non_existent_cmd\' - try \'help\'' in response)
test/py/u_boot_console_base.py new file +360 -0

360 lines not on this page · raw message

test/py/u_boot_console_exec_attach.py new file +65 -0

65 lines not on this page · raw message

test/py/u_boot_console_sandbox.py new file +79 -0

79 lines not on this page · raw message

test/py/u_boot_spawn.py new file +174 -0

174 lines not on this page · raw message

Signature · 1 line
2.7.0
#1 lore raw

PATCH v4 2/8 test/py: test that sandbox exits when asked

Test the sandbox port's implementation of the reset command and SIGHUP handling. These should both cause the U-Boot process to exit gracefully.

---
v4:
- No changes.

v3:
- Move test scripts into a sub-directory. Suggested by Michal Simek.
- s/uboot/u[-_]boot/g. Suggested by Simon Glass.
- Typo.
- s/"/'/g. Suggested by Simon Glass.
- Add more documentation. Suggested by Simon Glass.

---
 test/py/tests/test_sandbox_exit.py | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
 create mode 100644 test/py/tests/test_sandbox_exit.py
test/py/tests/test_sandbox_exit.py new file +24 -0
new file mode 100644
index 000000000000..2aa8eb4abc68
@@ -0,0 +1,24 @@
+# Copyright (c) 2015 Stephen Warren
+# Copyright (c) 2015-2016, NVIDIA CORPORATION. All rights reserved.
+#
+# SPDX-License-Identifier: GPL-2.0
+
+import pytest
+import signal
+
+ at pytest.mark.boardspec('sandbox')
+ at pytest.mark.buildconfigspec('reset')
+def test_reset(u_boot_console):
+    '''Test that the "reset" command exits sandbox process.'''
+
+    u_boot_console.run_command('reset', wait_for_prompt=False)
+    assert(u_boot_console.validate_exited())
+    u_boot_console.ensure_spawned()
+
+ at pytest.mark.boardspec('sandbox')
+def test_ctrl_c(u_boot_console):
+    '''Test that sending SIGINT to sandbox causes it to exit.'''
+
+    u_boot_console.kill(signal.SIGINT)
+    assert(u_boot_console.validate_exited())
+    u_boot_console.ensure_spawned()
Signature · 1 line
2.7.0
#2 lore raw

PATCH v4 2/8 test/py: test that sandbox exits when asked

Quoted 17 lines · Stephen Warren

Test the sandbox port's implementation of the reset command and SIGHUP handling. These should both cause the U-Boot process to exit gracefully.

---
v4:
- No changes.

v3:
- Move test scripts into a sub-directory. Suggested by Michal Simek.
- s/uboot/u[-_]boot/g. Suggested by Simon Glass.
- Typo.
- s/"/'/g. Suggested by Simon Glass.
- Add more documentation. Suggested by Simon Glass.

Quoted 4 lines
---
 test/py/tests/test_sandbox_exit.py | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
 create mode 100644 test/py/tests/test_sandbox_exit.py

Applied to u-boot-dm, thanks!

#3 lore raw

PATCH v4 3/8 test/py: add test of setenv/printenv/echo

In reply to Stephen Warren

This tests basic environment variable functionality.

---
v4:
- Reformat function-level documentation.

v3:
- Handle environment values that contain an = sign.
Reported by Michal Simek.
- Move test scripts into a sub-directory.
Suggested by Michal Simek.
- s/uboot/u[-_]boot/g. Suggested by Simon Glass.
- s/"/'/g. Suggested by Simon Glass.
- Add more documentation. Suggested by Simon Glass.
- Simplify command string construction. Suggested by Simon Glass.
- Move relevant edits to command_ut.c into this patch from a later one. Suggested by Simon Glass.

---
 test/command_ut.c         |   4 -
 test/py/tests/test_env.py | 221 ++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 221 insertions(+), 4 deletions(-)
 create mode 100644 test/py/tests/test_env.py
test/command_ut.c +0 -4
index 926573a39543..c086abe3ed3e 100644
@@ -20,10 +20,6 @@ static int do_ut_cmd(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[])
 	printf("%s: Testing commands\n", __func__);
 	run_command("env default -f -a", 0);
 
-	/* run a single command */
-	run_command("setenv single 1", 0);
-	assert(!strcmp("1", getenv("single")));
-
 	/* make sure that compound statements work */
 #ifdef CONFIG_SYS_HUSH_PARSER
 	run_command("if test -n ${single} ; then setenv check 1; fi", 0);
test/py/tests/test_env.py new file +221 -0

221 lines not on this page · raw message

Signature · 1 line
2.7.0
#4 lore raw

PATCH v4 3/8 test/py: add test of setenv/printenv/echo

Quoted 20 lines · Stephen Warren

This tests basic environment variable functionality.

---
v4:
- Reformat function-level documentation.

v3:
- Handle environment values that contain an = sign.
Reported by Michal Simek.
- Move test scripts into a sub-directory.
Suggested by Michal Simek.
- s/uboot/u[-_]boot/g. Suggested by Simon Glass.
- s/"/'/g. Suggested by Simon Glass.
- Add more documentation. Suggested by Simon Glass.
- Simplify command string construction. Suggested by Simon Glass.
- Move relevant edits to command_ut.c into this patch from a later one. Suggested by Simon Glass.

Quoted 5 lines
---
 test/command_ut.c         |   4 -
 test/py/tests/test_env.py | 221 ++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 221 insertions(+), 4 deletions(-)
 create mode 100644 test/py/tests/test_env.py

Applied to u-boot-dm, thanks!

#5 lore raw

PATCH v4 4/8 test/py: test the md/mw commands

In reply to Stephen Warren

This tests whether md/mw work, and affect each-other.

Command repeat is also tested.

test/cmd_repeat.sh is removed, since the new Python-based test does everything it used to.

---
v4:
- No changes.

v3:
- Add extra mw during md test to account for the possibility that the test's write data may already be present in RAM. Suggested by Michal Simek.
- Move test scripts into a sub-directory.
Suggested by Michal Simek.
- s/uboot/u[-_]boot/g. Suggested by Simon Glass.
- s/"/'/g. Suggested by Simon Glass.
- Add more documentation. Suggested by Simon Glass.

---
 test/cmd_repeat.sh       | 29 -----------------------------
 test/py/tests/test_md.py | 36 ++++++++++++++++++++++++++++++++++++
 2 files changed, 36 insertions(+), 29 deletions(-)
 delete mode 100755 test/cmd_repeat.sh
 create mode 100644 test/py/tests/test_md.py
test/cmd_repeat.sh deleted +0 -29
deleted file mode 100755
index 990e79900f47..000000000000
@@ -1,29 +0,0 @@
-#!/bin/sh
-
-# Test for U-Boot cli including command repeat
-
-BASE="$(dirname $0)"
-. $BASE/common.sh
-
-run_test() {
-	./${OUTPUT_DIR}/u-boot <<END
-setenv ctrlc_ignore y
-md 0
-
-reset
-END
-}
-check_results() {
-	echo "Check results"
-
-	grep -q 00000100 ${tmp} || fail "Command did not repeat"
-}
-
-echo "Test CLI repeat"
-echo
-tmp="$(tempfile)"
-build_uboot
-run_test >${tmp}
-check_results ${tmp}
-rm ${tmp}
-echo "Test passed"
test/py/tests/test_md.py new file +36 -0
new file mode 100644
index 000000000000..94603c7df609
@@ -0,0 +1,36 @@
+# Copyright (c) 2015 Stephen Warren
+# Copyright (c) 2015-2016, NVIDIA CORPORATION. All rights reserved.
+#
+# SPDX-License-Identifier: GPL-2.0
+
+import pytest
+
+ at pytest.mark.buildconfigspec('cmd_memory')
+def test_md(u_boot_console):
+    '''Test that md reads memory as expected, and that memory can be modified
+    using the mw command.'''
+
+    ram_base = u_boot_console.find_ram_base()
+    addr = '%08x' % ram_base
+    val = 'a5f09876'
+    expected_response = addr + ': ' + val
+    u_boot_console.run_command('mw ' + addr + ' 0 10')
+    response = u_boot_console.run_command('md ' + addr + ' 10')
+    assert(not (expected_response in response))
+    u_boot_console.run_command('mw ' + addr + ' ' + val)
+    response = u_boot_console.run_command('md ' + addr + ' 10')
+    assert(expected_response in response)
+
+ at pytest.mark.buildconfigspec('cmd_memory')
+def test_md_repeat(u_boot_console):
+    '''Test command repeat (via executing an empty command) operates correctly
+    for "md"; the command must repeat and dump an incrementing address.'''
+
+    ram_base = u_boot_console.find_ram_base()
+    addr_base = '%08x' % ram_base
+    words = 0x10
+    addr_repeat = '%08x' % (ram_base + (words * 4))
+    u_boot_console.run_command('md %s %x' % (addr_base, words))
+    response = u_boot_console.run_command('')
+    expected_response = addr_repeat + ': '
+    assert(expected_response in response)
Signature · 1 line
2.7.0
#6 lore raw

PATCH v4 4/8 test/py: test the md/mw commands

Quoted 23 lines · Stephen Warren

This tests whether md/mw work, and affect each-other.

Command repeat is also tested.

test/cmd_repeat.sh is removed, since the new Python-based test does everything it used to.

---
v4:
- No changes.

v3:
- Add extra mw during md test to account for the possibility that the test's write data may already be present in RAM. Suggested by Michal Simek.
- Move test scripts into a sub-directory.
Suggested by Michal Simek.
- s/uboot/u[-_]boot/g. Suggested by Simon Glass.
- s/"/'/g. Suggested by Simon Glass.
- Add more documentation. Suggested by Simon Glass.

Quoted 6 lines
---
 test/cmd_repeat.sh       | 29 -----------------------------
 test/py/tests/test_md.py | 36 ++++++++++++++++++++++++++++++++++++
 2 files changed, 36 insertions(+), 29 deletions(-)
 delete mode 100755 test/cmd_repeat.sh
 create mode 100644 test/py/tests/test_md.py

Applied to u-boot-dm, thanks!

#7 lore raw

PATCH v4 5/8 test/py: add test of basic shell functionality

In reply to Stephen Warren

From: Stephen Warren <[email protected]>

This tests whether the following features of the U-Boot shell:
- Execution of a directly entered command.
- Compound commands (; delimiter).
- Quoting of arguments containing spaces.
- Executing commands from environment variables.

---
v4:
- No changes.

v3:
- Move test scripts into a sub-directory.
Suggested by Michal Simek.
- s/uboot/u[-_]boot/g. Suggested by Simon Glass.
- s/"/'/g. Suggested by Simon Glass.
- Add more documentation. Suggested by Simon Glass.
- Move relevant edits to command_ut.c into this patch from a later one. Suggested by Simon Glass.

---
 test/command_ut.c                  | 16 ---------------
 test/py/tests/test_shell_basics.py | 42 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 42 insertions(+), 16 deletions(-)
 create mode 100644 test/py/tests/test_shell_basics.py
test/command_ut.c +0 -16
index c086abe3ed3e..43bd2c1771fe 100644
@@ -27,10 +27,6 @@ static int do_ut_cmd(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[])
 	run_command("setenv check", 0);
 #endif
 
-	/* commands separated by ; */
-	run_command_list("setenv list 1; setenv list ${list}1", -1, 0);
-	assert(!strcmp("11", getenv("list")));
-
 	/* commands separated by \n */
 	run_command_list("setenv list 1\n setenv list ${list}1", -1, 0);
 	assert(!strcmp("11", getenv("list")));
@@ -39,11 +35,6 @@ static int do_ut_cmd(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[])
 	run_command_list("setenv list 1${list}\n", -1, 0);
 	assert(!strcmp("111", getenv("list")));
 
-	/* three commands in a row */
-	run_command_list("setenv list 1\n setenv list ${list}2; "
-		"setenv list ${list}3", -1, 0);
-	assert(!strcmp("123", getenv("list")));
-
 	/* a command string with \0 in it. Stuff after \0 should be ignored */
 	run_command("setenv list", 0);
 	run_command_list(test_cmd, sizeof(test_cmd), 0);
@@ -62,13 +53,6 @@ static int do_ut_cmd(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[])
 	assert(run_command_list("false", -1, 0) == 1);
 	assert(run_command_list("echo", -1, 0) == 0);
 
-	run_command("setenv foo 'setenv monty 1; setenv python 2'", 0);
-	run_command("run foo", 0);
-	assert(getenv("monty") != NULL);
-	assert(!strcmp("1", getenv("monty")));
-	assert(getenv("python") != NULL);
-	assert(!strcmp("2", getenv("python")));
-
 #ifdef CONFIG_SYS_HUSH_PARSER
 	run_command("setenv foo 'setenv black 1\nsetenv adder 2'", 0);
 	run_command("run foo", 0);
test/py/tests/test_shell_basics.py new file +42 -0

42 lines not on this page · raw message

Signature · 1 line
2.7.0
#8 lore raw

PATCH v4 5/8 test/py: add test of basic shell functionality

Quoted 22 lines · Stephen Warren

From: Stephen Warren <[email protected]>

This tests whether the following features of the U-Boot shell:
- Execution of a directly entered command.
- Compound commands (; delimiter).
- Quoting of arguments containing spaces.
- Executing commands from environment variables.

---
v4:
- No changes.

v3:
- Move test scripts into a sub-directory.
Suggested by Michal Simek.
- s/uboot/u[-_]boot/g. Suggested by Simon Glass.
- s/"/'/g. Suggested by Simon Glass.
- Add more documentation. Suggested by Simon Glass.
- Move relevant edits to command_ut.c into this patch from a later one. Suggested by Simon Glass.

Quoted 5 lines
---
 test/command_ut.c                  | 16 ---------------
 test/py/tests/test_shell_basics.py | 42 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 42 insertions(+), 16 deletions(-)
 create mode 100644 test/py/tests/test_shell_basics.py

Applied to u-boot-dm, thanks!

#9 lore raw

PATCH v4 6/8 test/py: test the shell if command

In reply to Stephen Warren

From: Stephen Warren <[email protected]>

Migrate all most tests from command_ut.c into the Python test system. This allows the tests to be run against any U-Boot binary that supports the if command (i.e. where hush is enabled) without requiring that binary to be permanently bloated with the code from command_ut.

Some tests in command_ut.c can only be executed from C code, since they test internal (more unit-level) features of various U-Boot APIs. The migrated tests can all operate directly from the U-Boot console.

---
v4:
- No changes.

v3:
- Move test scripts into a sub-directory. Suggested by Michal Simek.
- s/uboot/u[-_]boot/g. Suggested by Simon Glass.
- s/"/'/g. Suggested by Simon Glass.
- Add more documentation. Suggested by Simon Glass.
- Remove inclusion of <os.h>. Suggested by Simon Glass.

---
 test/command_ut.c                  | 116 ----------------------------
 test/py/tests/test_hush_if_test.py | 154 +++++++++++++++++++++++++++++++++++++
 2 files changed, 154 insertions(+), 116 deletions(-)
 create mode 100644 test/py/tests/test_hush_if_test.py
test/command_ut.c +0 -116
index 43bd2c1771fe..54bf62b9bc30 100644
@@ -7,9 +7,6 @@
 #define DEBUG
 
 #include <common.h>
-#ifdef CONFIG_SANDBOX
-#include <os.h>
-#endif
 
 static const char test_cmd[] = "setenv list 1\n setenv list ${list}2; "
 		"setenv list ${list}3\0"
@@ -20,13 +17,6 @@ static int do_ut_cmd(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[])
 	printf("%s: Testing commands\n", __func__);
 	run_command("env default -f -a", 0);
 
-	/* make sure that compound statements work */
-#ifdef CONFIG_SYS_HUSH_PARSER
-	run_command("if test -n ${single} ; then setenv check 1; fi", 0);
-	assert(!strcmp("1", getenv("check")));
-	run_command("setenv check", 0);
-#endif
-
 	/* commands separated by \n */
 	run_command_list("setenv list 1\n setenv list ${list}1", -1, 0);
 	assert(!strcmp("11", getenv("list")));
@@ -60,112 +50,6 @@ static int do_ut_cmd(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[])
 	assert(!strcmp("1", getenv("black")));
 	assert(getenv("adder") != NULL);
 	assert(!strcmp("2", getenv("adder")));
-
-	/* Test the 'test' command */
-
-#define HUSH_TEST(name, expr, expected_result) \
-	run_command("if test " expr " ; then " \
-			"setenv " #name "_" #expected_result " y; else " \
-			"setenv " #name "_" #expected_result " n; fi", 0); \
-	assert(!strcmp(#expected_result, getenv(#name "_" #expected_result))); \
-	setenv(#name "_" #expected_result, NULL);
-
-	/* Basic operators */
-	HUSH_TEST(streq, "aaa = aaa", y);
-	HUSH_TEST(streq, "aaa = bbb", n);
-
-	HUSH_TEST(strneq, "aaa != bbb", y);
-	HUSH_TEST(strneq, "aaa != aaa", n);
-
-	HUSH_TEST(strlt, "aaa < bbb", y);
-	HUSH_TEST(strlt, "bbb < aaa", n);
-
-	HUSH_TEST(strgt, "bbb > aaa", y);
-	HUSH_TEST(strgt, "aaa > bbb", n);
-
-	HUSH_TEST(eq, "123 -eq 123", y);
-	HUSH_TEST(eq, "123 -eq 456", n);
-
-	HUSH_TEST(ne, "123 -ne 456", y);
-	HUSH_TEST(ne, "123 -ne 123", n);
-
-	HUSH_TEST(lt, "123 -lt 456", y);
-	HUSH_TEST(lt_eq, "123 -lt 123", n);
-	HUSH_TEST(lt, "456 -lt 123", n);
-
-	HUSH_TEST(le, "123 -le 456", y);
-	HUSH_TEST(le_eq, "123 -le 123", y);
-	HUSH_TEST(le, "456 -le 123", n);
-
-	HUSH_TEST(gt, "456 -gt 123", y);
-	HUSH_TEST(gt_eq, "123 -gt 123", n);
-	HUSH_TEST(gt, "123 -gt 456", n);
-
-	HUSH_TEST(ge, "456 -ge 123", y);
-	HUSH_TEST(ge_eq, "123 -ge 123", y);
-	HUSH_TEST(ge, "123 -ge 456", n);
-
-	HUSH_TEST(z, "-z \"\"", y);
-	HUSH_TEST(z, "-z \"aaa\"", n);
-
-	HUSH_TEST(n, "-n \"aaa\"", y);
-	HUSH_TEST(n, "-n \"\"", n);
-
-	/* Inversion of simple tests */
-	HUSH_TEST(streq_inv, "! aaa = aaa", n);
-	HUSH_TEST(streq_inv, "! aaa = bbb", y);
-
-	HUSH_TEST(streq_inv_inv, "! ! aaa = aaa", y);
-	HUSH_TEST(streq_inv_inv, "! ! aaa = bbb", n);
-
-	/* Binary operators */
-	HUSH_TEST(or_0_0, "aaa != aaa -o bbb != bbb", n);
-	HUSH_TEST(or_0_1, "aaa != aaa -o bbb = bbb", y);
-	HUSH_TEST(or_1_0, "aaa = aaa -o bbb != bbb", y);
-	HUSH_TEST(or_1_1, "aaa = aaa -o bbb = bbb", y);
-
-	HUSH_TEST(and_0_0, "aaa != aaa -a bbb != bbb", n);
-	HUSH_TEST(and_0_1, "aaa != aaa -a bbb = bbb", n);
-	HUSH_TEST(and_1_0, "aaa = aaa -a bbb != bbb", n);
-	HUSH_TEST(and_1_1, "aaa = aaa -a bbb = bbb", y);
-
-	/* Inversion within binary operators */
-	HUSH_TEST(or_0_0_inv, "! aaa != aaa -o ! bbb != bbb", y);
-	HUSH_TEST(or_0_1_inv, "! aaa != aaa -o ! bbb = bbb", y);
-	HUSH_TEST(or_1_0_inv, "! aaa = aaa -o ! bbb != bbb", y);
-	HUSH_TEST(or_1_1_inv, "! aaa = aaa -o ! bbb = bbb", n);
-
-	HUSH_TEST(or_0_0_inv_inv, "! ! aaa != aaa -o ! ! bbb != bbb", n);
-	HUSH_TEST(or_0_1_inv_inv, "! ! aaa != aaa -o ! ! bbb = bbb", y);
-	HUSH_TEST(or_1_0_inv_inv, "! ! aaa = aaa -o ! ! bbb != bbb", y);
-	HUSH_TEST(or_1_1_inv_inv, "! ! aaa = aaa -o ! ! bbb = bbb", y);
-
-	setenv("ut_var_nonexistent", NULL);
-	setenv("ut_var_exists", "1");
-	HUSH_TEST(z_varexp_quoted, "-z \"$ut_var_nonexistent\"", y);
-	HUSH_TEST(z_varexp_quoted, "-z \"$ut_var_exists\"", n);
-	setenv("ut_var_exists", NULL);
-
-	run_command("setenv ut_var_space \" \"", 0);
-	assert(!strcmp(getenv("ut_var_space"), " "));
-	run_command("setenv ut_var_test $ut_var_space", 0);
-	assert(!getenv("ut_var_test"));
-	run_command("setenv ut_var_test \"$ut_var_space\"", 0);
-	assert(!strcmp(getenv("ut_var_test"), " "));
-	run_command("setenv ut_var_test \" 1${ut_var_space}${ut_var_space} 2 \"", 0);
-	assert(!strcmp(getenv("ut_var_test"), " 1   2 "));
-	setenv("ut_var_space", NULL);
-	setenv("ut_var_test", NULL);
-
-#ifdef CONFIG_SANDBOX
-	/* File existence */
-	HUSH_TEST(e, "-e hostfs - creating_this_file_breaks_uboot_unit_test", n);
-	run_command("sb save hostfs - creating_this_file_breaks_uboot_unit_test 0 1", 0);
-	HUSH_TEST(e, "-e hostfs - creating_this_file_breaks_uboot_unit_test", y);
-	/* Perhaps this could be replaced by an "rm" shell command one day */
-	assert(!os_unlink("creating_this_file_breaks_uboot_unit_test"));
-	HUSH_TEST(e, "-e hostfs - creating_this_file_breaks_uboot_unit_test", n);
-#endif
 #endif
 
 	assert(run_command("", 0) == 0);
test/py/tests/test_hush_if_test.py new file +154 -0

154 lines not on this page · raw message

Signature · 1 line
2.7.0
#10 lore raw

PATCH v4 6/8 test/py: test the shell if command

Quoted 24 lines · Stephen Warren

From: Stephen Warren <[email protected]>

Migrate all most tests from command_ut.c into the Python test system. This allows the tests to be run against any U-Boot binary that supports the if command (i.e. where hush is enabled) without requiring that binary to be permanently bloated with the code from command_ut.

Some tests in command_ut.c can only be executed from C code, since they test internal (more unit-level) features of various U-Boot APIs. The migrated tests can all operate directly from the U-Boot console.

---
v4:
- No changes.

v3:
- Move test scripts into a sub-directory. Suggested by Michal Simek.
- s/uboot/u[-_]boot/g. Suggested by Simon Glass.
- s/"/'/g. Suggested by Simon Glass.
- Add more documentation. Suggested by Simon Glass.
- Remove inclusion of <os.h>. Suggested by Simon Glass.

Quoted 5 lines
---
 test/command_ut.c                  | 116 ----------------------------
 test/py/tests/test_hush_if_test.py | 154 +++++++++++++++++++++++++++++++++++++
 2 files changed, 154 insertions(+), 116 deletions(-)
 create mode 100644 test/py/tests/test_hush_if_test.py

Applied to u-boot-dm, thanks!

#11 lore raw

PATCH v4 7/8 test/py: test the ums command

In reply to Stephen Warren

From: Stephen Warren <[email protected]>

This test invokes the "ums" command in U-Boot, and validates that a USB storage device is enumerated on the test host system, and can be read from.

---
v4:
- No changes.

v3:
- Move test scripts into a sub-directory.
Suggested by Michal Simek.
- s/uboot/u[-_]boot/g. Suggested by Simon Glass.
- s/"/'/g. Suggested by Simon Glass.
- Add more documentation. Suggested by Simon Glass.
- Rename boardenv USB device node variable to avoid naming conflicts if we add support for other types of USB device later (e.g. CDC ACM, CDC Ethernet, DFU).

---
 test/py/tests/test_ums.py | 94 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 94 insertions(+)
 create mode 100644 test/py/tests/test_ums.py
test/py/tests/test_ums.py new file +94 -0
new file mode 100644
index 000000000000..a137221c7a5b
@@ -0,0 +1,94 @@
+# Copyright (c) 2015-2016, NVIDIA CORPORATION. All rights reserved.
+#
+# SPDX-License-Identifier: GPL-2.0
+
+# Test U-Boot's "ums" command. At present, this test only ensures that a UMS
+# device can be enumerated by the host/test machine. In the future, this test
+# should be enhanced to validate disk IO.
+
+import os
+import pytest
+import time
+
+'''
+Note: This test relies on:
+
+a) boardenv_* to contain configuration values to define which USB ports are
+available for testing. Without this, this test will be automatically skipped.
+For example:
+
+env__usb_dev_ports = (
+    {'tgt_usb_ctlr': '0', 'host_ums_dev_node': '/dev/disk/by-path/pci-0000:00:14.0-usb-0:13:1.0-scsi-0:0:0:0'},
+)
+
+env__block_devs = (
+    {'type': 'mmc', 'id': '0'}, # eMMC; always present
+    {'type': 'mmc', 'id': '1'}, # SD card; present since I plugged one in
+)
+
+b) udev rules to set permissions on devices nodes, so that sudo is not
+required. For example:
+
+ACTION=="add", SUBSYSTEM=="block", SUBSYSTEMS=="usb", KERNELS=="3-13", MODE:="666"
+
+(You may wish to change the group ID instead of setting the permissions wide
+open. All that matters is that the user ID running the test can access the
+device.)
+'''
+
+def open_ums_device(host_ums_dev_node):
+    '''Attempt to open a device node, returning either the opened file handle,
+    or None on any error.'''
+
+    try:
+        return open(host_ums_dev_node, 'rb')
+    except:
+        return None
+
+def wait_for_ums_device(host_ums_dev_node):
+    '''Continually attempt to open the device node exported by the "ums"
+    command, and either return the opened file handle, or raise an exception
+    after a timeout.'''
+
+    for i in xrange(100):
+        fh = open_ums_device(host_ums_dev_node)
+        if fh:
+            return fh
+        time.sleep(0.1)
+    raise Exception('UMS device did not appear')
+
+def wait_for_ums_device_gone(host_ums_dev_node):
+    '''Continually attempt to open the device node exported by the "ums"
+    command, and either return once the device has disappeared, or raise an
+    exception if it does not before a timeout occurs.'''
+
+    for i in xrange(100):
+        fh = open_ums_device(host_ums_dev_node)
+        if not fh:
+            return
+        fh.close()
+        time.sleep(0.1)
+    raise Exception('UMS device did not disappear')
+
+ at pytest.mark.buildconfigspec('cmd_usb_mass_storage')
+def test_ums(u_boot_console, env__usb_dev_port, env__block_devs):
+    '''Test the "ums" command; the host system must be able to enumerate a UMS
+    device when "ums" is running, and this device must disappear when "ums" is
+    aborted.'''
+
+    tgt_usb_ctlr = env__usb_dev_port['tgt_usb_ctlr']
+    host_ums_dev_node = env__usb_dev_port['host_ums_dev_node']
+
+    # We're interested in testing USB device mode on each port, not the cross-
+    # product of that with each device. So, just pick the first entry in the
+    # device list here. We'll test each block device somewhere else.
+    tgt_dev_type = env__block_devs[0]['type']
+    tgt_dev_id = env__block_devs[0]['id']
+
+    cmd = 'ums %s %s %s' % (tgt_usb_ctlr, tgt_dev_type, tgt_dev_id)
+    u_boot_console.run_command('ums 0 mmc 0', wait_for_prompt=False)
+    fh = wait_for_ums_device(host_ums_dev_node)
+    fh.read(4096)
+    fh.close()
+    u_boot_console.ctrlc()
+    wait_for_ums_device_gone(host_ums_dev_node)
Signature · 1 line
2.7.0
#12 lore raw

PATCH v4 7/8 test/py: test the ums command

Quoted 21 lines · Stephen Warren

From: Stephen Warren <[email protected]>

This test invokes the "ums" command in U-Boot, and validates that a USB storage device is enumerated on the test host system, and can be read from.

---
v4:
- No changes.

v3:
- Move test scripts into a sub-directory.
Suggested by Michal Simek.
- s/uboot/u[-_]boot/g. Suggested by Simon Glass.
- s/"/'/g. Suggested by Simon Glass.
- Add more documentation. Suggested by Simon Glass.
- Rename boardenv USB device node variable to avoid naming conflicts if we add support for other types of USB device later (e.g. CDC ACM, CDC Ethernet, DFU).

Quoted 4 lines
---
 test/py/tests/test_ums.py | 94 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 94 insertions(+)
 create mode 100644 test/py/tests/test_ums.py

Applied to u-boot-dm, thanks!

#13 lore raw

PATCH v4 7/8 test/py: test the ums command

In reply to Stephen Warren

Hi Stephen,

Quoted 125 lines

From: Stephen Warren <[email protected]>

This test invokes the "ums" command in U-Boot, and validates that a USB storage device is enumerated on the test host system, and can be read from.

---
v4:
- No changes.

v3:
- Move test scripts into a sub-directory.
Suggested by Michal Simek.
- s/uboot/u[-_]boot/g. Suggested by Simon Glass.
- s/"/'/g. Suggested by Simon Glass.
- Add more documentation. Suggested by Simon Glass.
- Rename boardenv USB device node variable to avoid naming conflicts if we add support for other types of USB device later (e.g. CDC ACM, CDC Ethernet, DFU).

---
 test/py/tests/test_ums.py | 94

+++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 94 insertions(+) create mode 100644 test/py/tests/test_ums.py

test/py/tests/test_ums.py new file +5 -0
new file mode 100644
index 000000000000..a137221c7a5b
@@ -0,0 +1,94 @@
+# Copyright (c) 2015-2016, NVIDIA CORPORATION. All rights reserved.
+#
+# SPDX-License-Identifier: GPL-2.0
+
+# Test U-Boot's "ums" command. At present, this test only ensures

that a UMS +# device can be enumerated by the host/test machine. In the future, this test +# should be enhanced to validate disk IO.

test/py/tests/test_ums.py new file +9 -0
+
+import os
+import pytest
+import time
+
+'''
+Note: This test relies on:
+
+a) boardenv_* to contain configuration values to define which USB

ports are +available for testing. Without this, this test will be automatically skipped. +For example:

test/py/tests/test_ums.py new file +3 -0
+
+env__usb_dev_ports = (
+    {'tgt_usb_ctlr': '0', 'host_ums_dev_node':

'/dev/disk/by-path/pci-0000:00:14.0-usb-0:13:1.0-scsi-0:0:0:0'}, +)

test/py/tests/test_ums.py new file +4 -0
+
+env__block_devs = (
+    {'type': 'mmc', 'id': '0'}, # eMMC; always present
+    {'type': 'mmc', 'id': '1'}, # SD card; present since I plugged

one in +)

test/py/tests/test_ums.py new file +2 -0
+
+b) udev rules to set permissions on devices nodes, so that sudo is

not +required. For example:

test/py/tests/test_ums.py new file +2 -0
+
+ACTION=="add", SUBSYSTEM=="block", SUBSYSTEMS=="usb",

KERNELS=="3-13", MODE:="666" +
+(You may wish to change the group ID instead of setting the permissions wide +open. All that matters is that the user ID running the test can access the +device.)
+'''

test/py/tests/test_ums.py new file +3 -0
+
+def open_ums_device(host_ums_dev_node):
+    '''Attempt to open a device node, returning either the opened

file handle,

test/py/tests/test_ums.py new file +9 -0
+    or None on any error.'''
+
+    try:
+        return open(host_ums_dev_node, 'rb')
+    except:
+        return None
+
+def wait_for_ums_device(host_ums_dev_node):
+    '''Continually attempt to open the device node exported by the

"ums"
+ command, and either return the opened file handle, or raise an
exception

test/py/tests/test_ums.py new file +11 -0
+    after a timeout.'''
+
+    for i in xrange(100):
+        fh = open_ums_device(host_ums_dev_node)
+        if fh:
+            return fh
+        time.sleep(0.1)
+    raise Exception('UMS device did not appear')
+
+def wait_for_ums_device_gone(host_ums_dev_node):
+    '''Continually attempt to open the device node exported by the

"ums"
+ command, and either return once the device has disappeared, or
raise an

test/py/tests/test_ums.py new file +13 -0
+    exception if it does not before a timeout occurs.'''
+
+    for i in xrange(100):
+        fh = open_ums_device(host_ums_dev_node)
+        if not fh:
+            return
+        fh.close()
+        time.sleep(0.1)
+    raise Exception('UMS device did not disappear')
+
+ at pytest.mark.buildconfigspec('cmd_usb_mass_storage')
+def test_ums(u_boot_console, env__usb_dev_port, env__block_devs):
+    '''Test the "ums" command; the host system must be able to

enumerate a UMS
+ device when "ums" is running, and this device must disappear
when "ums" is

test/py/tests/test_ums.py new file +6 -0
+    aborted.'''
+
+    tgt_usb_ctlr = env__usb_dev_port['tgt_usb_ctlr']
+    host_ums_dev_node = env__usb_dev_port['host_ums_dev_node']
+
+    # We're interested in testing USB device mode on each port, not

the cross-
+ # product of that with each device. So, just pick the first
entry in the

test/py/tests/test_ums.py new file +1 -0
+    # device list here. We'll test each block device somewhere else.

test/py/tests/test_ums.py line 94

Quoted 10 lines
test/py/tests/test_ums.py new file +10 -0
+    tgt_dev_type = env__block_devs[0]['type']
+    tgt_dev_id = env__block_devs[0]['id']
+
+    cmd = 'ums %s %s %s' % (tgt_usb_ctlr, tgt_dev_type, tgt_dev_id)
+    u_boot_console.run_command('ums 0 mmc 0', wait_for_prompt=False)
+    fh = wait_for_ums_device(host_ums_dev_node)
+    fh.read(4096)
+    fh.close()
+    u_boot_console.ctrlc()
+    wait_for_ums_device_gone(host_ums_dev_node)
Signature · 5 lines
Best regards,

Lukasz Majewski

Samsung R&D Institute Poland (SRPOL) | Linux Platform Group
#14 lore raw

PATCH v4 8/8 test/py: add a test for the sleep command

In reply to Stephen Warren

From: Stephen Warren <[email protected]>

Execute "sleep", and validate that it sleeps for approximately the correct amount of time.

---
v4: New patch.

Note that this test won't pass for sandbox until the following patches is applied: timer: sandbox: work without device tree

---
 test/py/tests/test_sleep.py | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
 create mode 100644 test/py/tests/test_sleep.py
test/py/tests/test_sleep.py new file +24 -0
new file mode 100644
index 000000000000..64f1ddf9a09f
@@ -0,0 +1,24 @@
+# Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
+#
+# SPDX-License-Identifier: GPL-2.0
+
+import pytest
+import time
+
+def test_sleep(u_boot_console):
+    '''Test the sleep command, and validate that it sleeps for approximately
+    the correct amount of time.'''
+
+    # Do this before we time anything, to make sure U-Boot is already running.
+    # Otherwise, the system boot time is included in the time measurement.
+    u_boot_console.ensure_spawned()
+
+    # 3s isn't too long, but is enough to cross a few second boundaries.
+    sleep_time = 3
+    tstart = time.time()
+    u_boot_console.run_command('sleep %d' % sleep_time)
+    tend = time.time()
+    elapsed = tend - tstart
+    delta_to_expected = abs(elapsed - sleep_time)
+    # 0.25s margin is hopefully enough to account for any system overhead.
+    assert delta_to_expected < 0.25
Signature · 1 line
2.7.0
#15 lore raw

PATCH v4 8/8 test/py: add a test for the sleep command

Quoted 10 lines · Stephen Warren

From: Stephen Warren <[email protected]>

Execute "sleep", and validate that it sleeps for approximately the correct amount of time.

---
v4: New patch.

Quoted 6 lines

Note that this test won't pass for sandbox until the following patches is applied: timer: sandbox: work without device tree

---
 test/py/tests/test_sleep.py | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
 create mode 100644 test/py/tests/test_sleep.py

Applied to u-boot-dm, thanks!

#16 lore raw

PATCH v4 1/8 test/py: Implement pytest infrastructure

In reply to Stephen Warren
Quoted 83 lines · Stephen Warren

This tool aims to test U-Boot by executing U-Boot shell commands using the console interface. A single top-level script exists to execute or attach to the U-Boot console, run the entire script of tests against it, and summarize the results. Advantages of this approach are:

- Testing is performed in the same way a user or script would interact with U-Boot; there can be no disconnect.
- There is no need to write or embed test-related code into U-Boot itself. It is asserted that writing test-related code in Python is simpler and more flexible that writing it all in C.
- It is reasonably simple to interact with U-Boot in this way.

A few simple tests are provided as examples. Soon, we should convert as many as possible of the other tests in test/* and test/cmd_ut.c too.

The hook scripts, relay control utilities, and udev rules I use for my own HW setup are published at https://github.com/swarren/uboot-test-hooks.

See README.md for more details!

---
Apologies for the resend; I forgot to CC the list last time.

v4:
- Fix all users after ubspawn -> u_boot_spawn module rename.
- Don't escape ^ in the HTML log file.
- Fix u_boot_spawn.py Spawn.__init__ to set SIGHUP handler is set to SIG_DFL. This ensures that sandbox responds to SIGHUP when run non- interactively (e.g. under a daemon continuous integration server).
- Fix logging of all child processes to make sure that all stdout/err, exit code, and exception data from all processes makes its way to both test stdout (for pytest to log to its stdout in case of error) and the HTML test log. This changed multiplexed_log.py RunAndLog.run() (a fair bit of change) and u_boot_console_exec_attach.py ConseolExecAttach.* (just passing sys.stdout to log.get_runner() everywhere).
- Also mention kermit as a possible app to attach to a serial console.
- Fix test_version to work correctly when BUILD_TAG is set, which causes U-Boot to print both the version string and BUILD_TAG value at startup, whereas the "version" shell command only prints the version string.
- Reformat function-level documentation.

v3:
- Rework HTML log generation so that TAB characters render as expected. Suggested by Michal Simek.
- Move test scripts into a sub-directory.
Suggested by Michal Simek.
- s/uboot/u[-_]boot/g. Suggested by Simon Glass.
- s/"/'/g. Suggested by Simon Glass.
- Typo fixes.
- Add more documentation. Suggested by Simon Glass.
- Make "notes" in the log file be <pre> so that their formatting is preserved. This is useful for large notes such as exception dumps.

v2:
- Many fixes and tweaks have been squashed in. Separated out some of the tests into separate commits, and added some more tests.

---
 test/py/.gitignore                    |   1 +
 test/py/README.md                     | 300 ++++++++++++++++++++
 test/py/conftest.py                   | 422 ++++++++++++++++++++++++++++
 test/py/multiplexed_log.css           |  88 ++++++
 test/py/multiplexed_log.py            | 515 ++++++++++++++++++++++++++++++++++
 test/py/pytest.ini                    |  11 +
 test/py/test.py                       |  32 +++
 test/py/tests/test_000_version.py     |  20 ++
 test/py/tests/test_help.py            |   9 +
 test/py/tests/test_unknown_cmd.py     |  14 +
 test/py/u_boot_console_base.py        | 360 ++++++++++++++++++++++++
 test/py/u_boot_console_exec_attach.py |  65 +++++
 test/py/u_boot_console_sandbox.py     |  79 ++++++
 test/py/u_boot_spawn.py               | 174 ++++++++++++
 14 files changed, 2090 insertions(+)
 create mode 100644 test/py/.gitignore
 create mode 100644 test/py/README.md
 create mode 100644 test/py/conftest.py
 create mode 100644 test/py/multiplexed_log.css
 create mode 100644 test/py/multiplexed_log.py
 create mode 100644 test/py/pytest.ini
Quoted 8 lines
 create mode 100755 test/py/test.py
 create mode 100644 test/py/tests/test_000_version.py
 create mode 100644 test/py/tests/test_help.py
 create mode 100644 test/py/tests/test_unknown_cmd.py
 create mode 100644 test/py/u_boot_console_base.py
 create mode 100644 test/py/u_boot_console_exec_attach.py
 create mode 100644 test/py/u_boot_console_sandbox.py
 create mode 100644 test/py/u_boot_spawn.py

Applied to u-boot-dm, thanks!