Kunit fixes update for Linux 5.8-rc4
Subsystems (4)
Shuah Khan
maintainerHi Linus,
Please pull the following Kunit fixes update for Linux 5.8-rc4.
This kunit fixes update for Linux 5.8-rc4 consists of fixes to build and run-times failures. Also includes troubleshooting tips updates to kunit user documentation.
These tips in the doc patch helped me with my test runs.
diff is included.
thanks,
-- Shuah
----------------------------------------------------------------
The following changes since commit 48778464bb7d346b47157d21ffde2af6b2d39110:
Linux 5.8-rc2 (2020-06-21 15:45:29 -0700)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
tags/linux-kselftest-kunit-fixes-5.8-rc4
for you to fetch changes up to c63d2dd7e134ebddce4745c51f9572b3f0d92b26:
Documentation: kunit: Add some troubleshooting tips to the FAQ
(2020-06-26 14:29:55 -0600)
----------------------------------------------------------------
linux-kselftest-kunit-fixes-5.8-rc4
This kunit fixes update for Linux 5.8-rc4 consists of fixes to build and run-times failures. Also includes troubleshooting tips updates to kunit user documentation.
----------------------------------------------------------------
David Gow (2):
kunit: kunit_tool: Fix invalid result when build fails
Documentation: kunit: Add some troubleshooting tips to the FAQ
Rikard Falkeborn (1):
kunit: kunit_config: Fix parsing of CONFIG options with space
Uriel Guajardo (1):
kunit: show error if kunit results are not present
Documentation/dev-tools/kunit/faq.rst | 40 +++++++++++++++++++++ tools/testing/kunit/kunit.py | 4 ++- tools/testing/kunit/kunit_config.py | 2 +- tools/testing/kunit/kunit_parser.py | 8 ++--- tools/testing/kunit/kunit_tool_test.py | 11 ++++++ .../kunit/test_data/test_insufficient_memory.log | Bin 6 files changed, 59 insertions(+), 6 deletions(-) create mode 100644
tools/testing/kunit/test_data/test_insufficient_memory.log
----------------------------------------------------------------
Documentation/dev-tools/kunit/faq.rst +40 -0
@@ -61,3 +61,43 @@ test, or an end-to-end test. kernel by installing a production configuration of the kernel on production hardware with a production userspace and then trying to exercise some behavior that depends on interactions between the hardware, the kernel, and userspace. + +KUnit isn't working, what should I do? +====================================== + +Unfortunately, there are a number of things which can break, but here are some +things to try. + +1. Try running ``./tools/testing/kunit/kunit.py run`` with the ``--raw_output`` + parameter. This might show details or error messages hidden by the kunit_tool + parser. +2. Instead of running ``kunit.py run``, try running ``kunit.py config``, + ``kunit.py build``, and ``kunit.py exec`` independently. This can help track + down where an issue is occurring. (If you think the parser is at fault, you + can run it manually against stdin or a file with ``kunit.py parse``.) +3. Running the UML kernel directly can often reveal issues or error messages + kunit_tool ignores. This should be as simple as running ``./vmlinux`` after + building the UML kernel (e.g., by using ``kunit.py build``). Note that UML + has some unusual requirements (such as the host having a tmpfs filesystem + mounted), and has had issues in the past when built statically and the host + has KASLR enabled. (On older host kernels, you may need to run ``setarch + `uname -m` -R ./vmlinux`` to disable KASLR.) +4. Make sure the kernel .config has ``CONFIG_KUNIT=y`` and at least one test + (e.g. ``CONFIG_KUNIT_EXAMPLE_TEST=y``). kunit_tool will keep its .config + around, so you can see what config was used after running ``kunit.py run``. + It also preserves any config changes you might make, so you can + enable/disable things with ``make ARCH=um menuconfig`` or similar, and then + re-run kunit_tool. +5. Try to run ``make ARCH=um defconfig`` before running ``kunit.py run``. This + may help clean up any residual config items which could be causing problems. +6. Finally, try running KUnit outside UML. KUnit and KUnit tests can run be + built into any kernel, or can be built as a module and loaded at runtime. + Doing so should allow you to determine if UML is causing the issue you're + seeing. When tests are built-in, they will execute when the kernel boots, and + modules will automatically execute associated tests when loaded. Test results + can be collected from ``/sys/kernel/debug/kunit/<test suite>/results``, and + can be parsed with ``kunit.py parse``. For more details, see "KUnit on + non-UML architectures" in :doc:`usage`. + +If none of the above tricks help, you are always welcome to email any issues to [email protected].
tools/testing/kunit/kunit.py +3 -1
@@ -82,7 +82,9 @@ def build_tests(linux: kunit_kernel.LinuxSourceTree, request.make_options) build_end = time.time() if not success: - return KunitResult(KunitStatus.BUILD_FAILURE, 'could not build kernel') + return KunitResult(KunitStatus.BUILD_FAILURE, + 'could not build kernel', + build_end - build_start) if not success: return KunitResult(KunitStatus.BUILD_FAILURE, 'could not build kernel',
tools/testing/kunit/kunit_config.py +1 -1
@@ -10,7 +10,7 @@ import collections import re CONFIG_IS_NOT_SET_PATTERN = r'^# CONFIG_(\w+) is not set$' -CONFIG_PATTERN = r'^CONFIG_(\w+)=(\S+)$' +CONFIG_PATTERN = r'^CONFIG_(\w+)=(\S+|".*")$' KconfigEntryBase = collections.namedtuple('KconfigEntry', ['name', 'value'])
tools/testing/kunit/kunit_parser.py +4 -4
@@ -265,11 +265,9 @@ def bubble_up_suite_errors(test_suite_list: List[TestSuite]) -> TestStatus: return bubble_up_errors(lambda x: x.status, test_suite_list) def parse_test_result(lines: List[str]) -> TestResult: - if not lines: - return TestResult(TestStatus.NO_TESTS, [], lines) consume_non_diagnositic(lines) - if not parse_tap_header(lines): - return None + if not lines or not parse_tap_header(lines): + return TestResult(TestStatus.NO_TESTS, [], lines) test_suites = [] test_suite = parse_test_suite(lines) while test_suite:
@@ -282,6 +280,8 @@ def parse_run_tests(kernel_output) -> TestResult: failed_tests = 0 crashed_tests = 0 test_result = parse_test_result(list(isolate_kunit_output(kernel_output))) + if test_result.status == TestStatus.NO_TESTS: + print_with_timestamp(red('[ERROR] ') + 'no kunit output detected') for test_suite in test_result.suites: if test_suite.status == TestStatus.SUCCESS: print_suite_divider(green('[PASSED] ') + test_suite.name)
tools/testing/kunit/kunit_tool_test.py +11 -0
@@ -170,6 +170,17 @@ class KUnitParserTest(unittest.TestCase): result.status) file.close() + def test_no_kunit_output(self): + crash_log = get_absolute_path( + 'test_data/test_insufficient_memory.log') + file = open(crash_log) + print_mock = mock.patch('builtins.print').start() + result = kunit_parser.parse_run_tests( + kunit_parser.isolate_kunit_output(file.readlines())) + print_mock.assert_any_call(StrContains("no kunit output detected")) + print_mock.stop() + file.close() + def test_crashed_test(self): crashed_log = get_absolute_path( 'test_data/test_is_test_passed-crash.log')
tools/testing/kunit/test_data/test_insufficient_memory.log new file +0 -0
The pull request you sent on Thu, 2 Jul 2020 09:23:55 -0600:
The pull request you sent on Thu, 2 Jul 2020 09:23:55 -0600:
Quoted 1 line
git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftesttags/linux-kselftest-kunit-fixes-5.8-rc4
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/55844741a1e74bd41b4cea57502c2efedc99bf47
Thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.wiki.kernel.org/userdoc/prtracker