kunit: introduce KUnit, the Linux kernel unit testing framework
Subsystems (11)
v18 cover kunit: introduce KUnit, the Linux kernel unit testing framework
## TL;DR
This revision addresses comments from Linus[1] and Randy[2], by moving top level kunit/ directory to lib/kunit/ and likewise moves top level Kconfig entry under lib/Kconfig.debug, so the KUnit submenu now shows up under the "Kernel Hacking" menu.
As a consequence of this, I rewrote patch 06/18 (kbuild: enable building KUnit) - now 06/19 (lib: enable building KUnit in lib/), and now needs to be re-acked/reviewed.
## Background
This patch set proposes KUnit, a lightweight unit testing and mocking framework for the Linux kernel.
Unlike Autotest and kselftest, KUnit is a true unit testing framework; it does not require installing the kernel on a test machine or in a VM (however, KUnit still allows you to run tests on test machines or in VMs if you want[3]) and does not require tests to be written in userspace running on a host kernel. Additionally, KUnit is fast: From invocation to completion KUnit can run several dozen tests in about a second. Currently, the entire KUnit test suite for KUnit runs in under a second from the initial invocation (build time excluded).
KUnit is heavily inspired by JUnit, Python's unittest.mock, and Googletest/Googlemock for C++. KUnit provides facilities for defining unit test cases, grouping related test cases into test suites, providing common infrastructure for running tests, mocking, spying, and much more.
### What's so special about unit testing?
A unit test is supposed to test a single unit of code in isolation, hence the name. There should be no dependencies outside the control of the test; this means no external dependencies, which makes tests orders of magnitudes faster. Likewise, since there are no external dependencies, there are no hoops to jump through to run the tests. Additionally, this makes unit tests deterministic: a failing unit test always indicates a problem. Finally, because unit tests necessarily have finer granularity, they are able to test all code paths easily solving the classic problem of difficulty in exercising error handling code.
### Is KUnit trying to replace other testing frameworks for the kernel?
No. Most existing tests for the Linux kernel are end-to-end tests, which have their place. A well tested system has lots of unit tests, a reasonable number of integration tests, and some end-to-end tests. KUnit is just trying to address the unit test space which is currently not being addressed.
### More information on KUnit
There is a bunch of documentation near the end of this patch set that describes how to use KUnit and best practices for writing unit tests. For convenience I am hosting the compiled docs here[4].
Additionally for convenience, I have applied these patches to a branch[5]. The repo may be cloned with:
git clone https://kunit.googlesource.com/linux
This patchset is on the kunit/initial/v5.3/v18 branch.
## History since v15
### v18
- Addrssed comments on 07/19 (kunit: test: add initial tests) from Randy Dunlap by removing redundant dependencies from Kconfig entries.
### v17
- Addressed comments on 06/19 (lib: enable building KUnit in lib/) from Stephen Boyd by moving KUnit submenu ahead of Runtime Testing submenu.
### v16
- Addressed comments from Linus Torvalds by moving all kunit/ paths to lib/kunit/.
- Addressed comments by Randy Dunlap by moving KUnit Kconfig under lib/Kconfig.debug so the KUnit submenu shows up under the "Kernel Hacking" menu.
[1] https://www.lkml.org/lkml/2019/9/20/696
[2] https://www.lkml.org/lkml/2019/9/20/738
[3] https://google.github.io/kunit-docs/third_party/kernel/docs/usage.html#kunit-on-non-uml-architectures
[4] https://google.github.io/kunit-docs/third_party/kernel/docs/
[5] https://kunit.googlesource.com/linux/+/kunit/initial/v5.3/v18
---
Avinash Kondareddy (1):
kunit: test: add tests for KUnit managed resources
Brendan Higgins (16):
kunit: test: add KUnit test runner core
kunit: test: add test resource management API
kunit: test: add string_stream a std::stream like string builder
kunit: test: add assertion printing library
kunit: test: add the concept of expectations
lib: enable building KUnit in lib/
kunit: test: add initial tests
objtool: add kunit_try_catch_throw to the noreturn list
kunit: test: add support for test abort
kunit: test: add tests for kunit test abort
kunit: test: add the concept of assertions
kunit: defconfig: add defconfigs for building KUnit tests
Documentation: kunit: add documentation for KUnit
MAINTAINERS: add entry for KUnit the unit testing framework
MAINTAINERS: add proc sysctl KUnit test to PROC SYSCTL section
kunit: fix failure to build without printk
Felix Guo (1):
kunit: tool: add Python wrappers for running KUnit tests
Iurii Zaikin (1):
kernel/sysctl-test: Add null pointer test for sysctl.c:proc_dointvec()
Documentation/dev-tools/index.rst | 1 + Documentation/dev-tools/kunit/api/index.rst | 16 + Documentation/dev-tools/kunit/api/test.rst | 11 + Documentation/dev-tools/kunit/faq.rst | 62 + Documentation/dev-tools/kunit/index.rst | 79 + Documentation/dev-tools/kunit/start.rst | 180 ++ Documentation/dev-tools/kunit/usage.rst | 576 +++++++ MAINTAINERS | 13 + arch/um/configs/kunit_defconfig | 3 + include/kunit/assert.h | 356 ++++ include/kunit/string-stream.h | 51 + include/kunit/test.h | 1490 +++++++++++++++++ include/kunit/try-catch.h | 75 + kernel/Makefile | 2 + kernel/sysctl-test.c | 392 +++++ lib/Kconfig.debug | 13 + lib/Makefile | 2 + lib/kunit/Kconfig | 36 + lib/kunit/Makefile | 9 + lib/kunit/assert.c | 141 ++ lib/kunit/example-test.c | 88 + lib/kunit/string-stream-test.c | 52 + lib/kunit/string-stream.c | 217 +++ lib/kunit/test-test.c | 331 ++++ lib/kunit/test.c | 478 ++++++ lib/kunit/try-catch.c | 118 ++ tools/objtool/check.c | 1 + tools/testing/kunit/.gitignore | 3 + tools/testing/kunit/configs/all_tests.config | 3 + tools/testing/kunit/kunit.py | 136 ++ tools/testing/kunit/kunit_config.py | 66 + tools/testing/kunit/kunit_kernel.py | 149 ++ tools/testing/kunit/kunit_parser.py | 310 ++++ tools/testing/kunit/kunit_tool_test.py | 206 +++ .../test_is_test_passed-all_passed.log | 32 + .../test_data/test_is_test_passed-crash.log | 69 + .../test_data/test_is_test_passed-failure.log | 36 + .../test_is_test_passed-no_tests_run.log | 75 + .../test_output_isolated_correctly.log | 106 ++ .../test_data/test_read_from_file.kconfig | 17 + 40 files changed, 6001 insertions(+) create mode 100644 Documentation/dev-tools/kunit/api/index.rst create mode 100644 Documentation/dev-tools/kunit/api/test.rst create mode 100644 Documentation/dev-tools/kunit/faq.rst create mode 100644 Documentation/dev-tools/kunit/index.rst create mode 100644 Documentation/dev-tools/kunit/start.rst create mode 100644 Documentation/dev-tools/kunit/usage.rst create mode 100644 arch/um/configs/kunit_defconfig create mode 100644 include/kunit/assert.h create mode 100644 include/kunit/string-stream.h create mode 100644 include/kunit/test.h create mode 100644 include/kunit/try-catch.h create mode 100644 kernel/sysctl-test.c create mode 100644 lib/kunit/Kconfig create mode 100644 lib/kunit/Makefile create mode 100644 lib/kunit/assert.c create mode 100644 lib/kunit/example-test.c create mode 100644 lib/kunit/string-stream-test.c create mode 100644 lib/kunit/string-stream.c create mode 100644 lib/kunit/test-test.c create mode 100644 lib/kunit/test.c create mode 100644 lib/kunit/try-catch.c create mode 100644 tools/testing/kunit/.gitignore create mode 100644 tools/testing/kunit/configs/all_tests.config create mode 100755 tools/testing/kunit/kunit.py create mode 100644 tools/testing/kunit/kunit_config.py create mode 100644 tools/testing/kunit/kunit_kernel.py create mode 100644 tools/testing/kunit/kunit_parser.py create mode 100755 tools/testing/kunit/kunit_tool_test.py create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-all_passed.log create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-crash.log create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-failure.log create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-no_tests_run.log create mode 100644 tools/testing/kunit/test_data/test_output_isolated_correctly.log create mode 100644 tools/testing/kunit/test_data/test_read_from_file.kconfig
Signature · 1 line
2.23.0.351.gc4317032e6-goog
PATCH v18 1/19 kunit: test: add KUnit test runner core
Add core facilities for defining unit tests; this provides a common way to define test cases, functions that execute code which is under test and determine whether the code under test behaves as expected; this also provides a way to group together related test cases in test suites (here we call them test_modules).
Just define test cases and how to execute them for now; setting expectations on code will be defined later.
--- include/kunit/test.h | 188 ++++++++++++++++++++++++++++++++++++++++++ lib/kunit/Kconfig | 13 +++ lib/kunit/Makefile | 1 + lib/kunit/test.c | 191 +++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 393 insertions(+) create mode 100644 include/kunit/test.h create mode 100644 lib/kunit/Kconfig create mode 100644 lib/kunit/Makefile create mode 100644 lib/kunit/test.c
include/kunit/test.h new file +188 -0
@@ -0,0 +1,188 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Base unit test (KUnit) API. + * + * Copyright (C) 2019, Google LLC. + * Author: Brendan Higgins <[email protected]> + */ + +#ifndef _KUNIT_TEST_H +#define _KUNIT_TEST_H + +#include <linux/types.h> + +struct kunit; + +/** + * struct kunit_case - represents an individual test case. + * + * @run_case: the function representing the actual test case. + * @name: the name of the test case. + * + * A test case is a function with the signature, + * ``void (*)(struct kunit *)`` that makes expectations (see + * KUNIT_EXPECT_TRUE()) about code under test. Each test case is associated + * with a &struct kunit_suite and will be run after the suite's init + * function and followed by the suite's exit function. + * + * A test case should be static and should only be created with the + * KUNIT_CASE() macro; additionally, every array of test cases should be + * terminated with an empty test case. + * + * Example: + * + * .. code-block:: c + * + * void add_test_basic(struct kunit *test) + * { + * KUNIT_EXPECT_EQ(test, 1, add(1, 0)); + * KUNIT_EXPECT_EQ(test, 2, add(1, 1)); + * KUNIT_EXPECT_EQ(test, 0, add(-1, 1)); + * KUNIT_EXPECT_EQ(test, INT_MAX, add(0, INT_MAX)); + * KUNIT_EXPECT_EQ(test, -1, add(INT_MAX, INT_MIN)); + * } + * + * static struct kunit_case example_test_cases[] = { + * KUNIT_CASE(add_test_basic), + * {} + * }; + * + */ +struct kunit_case { + void (*run_case)(struct kunit *test); + const char *name; + + /* private: internal use only. */ + bool success; +}; + +/** + * KUNIT_CASE - A helper for creating a &struct kunit_case + * + * @test_name: a reference to a test case function. + * + * Takes a symbol for a function representing a test case and creates a + * &struct kunit_case object from it. See the documentation for + * &struct kunit_case for an example on how to use it. + */ +#define KUNIT_CASE(test_name) { .run_case = test_name, .name = #test_name } + +/** + * struct kunit_suite - describes a related collection of &struct kunit_case + * + * @name: the name of the test. Purely informational. + * @init: called before every test case. + * @exit: called after every test case. + * @test_cases: a null terminated array of test cases. + * + * A kunit_suite is a collection of related &struct kunit_case s, such that + * @init is called before every test case and @exit is called after every + * test case, similar to the notion of a *test fixture* or a *test class* + * in other unit testing frameworks like JUnit or Googletest. + * + * Every &struct kunit_case must be associated with a kunit_suite for KUnit + * to run it. + */ +struct kunit_suite { + const char name[256]; + int (*init)(struct kunit *test); + void (*exit)(struct kunit *test); + struct kunit_case *test_cases; +}; + +/** + * struct kunit - represents a running instance of a test. + * + * @priv: for user to store arbitrary data. Commonly used to pass data + * created in the init function (see &struct kunit_suite). + * + * Used to store information about the current context under which the test + * is running. Most of this data is private and should only be accessed + * indirectly via public functions; the one exception is @priv which can be + * used by the test writer to store arbitrary data. + */ +struct kunit { + void *priv; + + /* private: internal use only. */ + const char *name; /* Read only after initialization! */ + /* + * success starts as true, and may only be set to false during a + * test case; thus, it is safe to update this across multiple + * threads using WRITE_ONCE; however, as a consequence, it may only + * be read after the test case finishes once all threads associated + * with the test case have terminated. + */ + bool success; /* Read only after test_case finishes! */ +}; + +void kunit_init_test(struct kunit *test, const char *name); + +int kunit_run_tests(struct kunit_suite *suite); + +/** + * kunit_test_suite() - used to register a &struct kunit_suite with KUnit. + * + * @suite: a statically allocated &struct kunit_suite. + * + * Registers @suite with the test framework. See &struct kunit_suite for + * more information. + * + * NOTE: Currently KUnit tests are all run as late_initcalls; this means + * that they cannot test anything where tests must run at a different init + * phase. One significant restriction resulting from this is that KUnit + * cannot reliably test anything that is initialize in the late_init phase; + * another is that KUnit is useless to test things that need to be run in + * an earlier init phase. + * + * TODO([email protected]): Don't run all KUnit tests as + * late_initcalls. I have some future work planned to dispatch all KUnit + * tests from the same place, and at the very least to do so after + * everything else is definitely initialized. + */ +#define kunit_test_suite(suite) \ + static int kunit_suite_init##suite(void) \ + { \ + return kunit_run_tests(&suite); \ + } \ + late_initcall(kunit_suite_init##suite) + +void __printf(3, 4) kunit_printk(const char *level, + const struct kunit *test, + const char *fmt, ...); + +/** + * kunit_info() - Prints an INFO level message associated with @test. + * + * @test: The test context object. + * @fmt: A printk() style format string. + * + * Prints an info level message associated with the test suite being run. + * Takes a variable number of format parameters just like printk(). + */ +#define kunit_info(test, fmt, ...) \ + kunit_printk(KERN_INFO, test, fmt, ##__VA_ARGS__) + +/** + * kunit_warn() - Prints a WARN level message associated with @test. + * + * @test: The test context object. + * @fmt: A printk() style format string. + * + * Prints a warning level message. + */ +#define kunit_warn(test, fmt, ...) \ + kunit_printk(KERN_WARNING, test, fmt, ##__VA_ARGS__) + +/** + * kunit_err() - Prints an ERROR level message associated with @test. + * + * @test: The test context object. + * @fmt: A printk() style format string. + * + * Prints an error level message. + */ +#define kunit_err(test, fmt, ...) \ + kunit_printk(KERN_ERR, test, fmt, ##__VA_ARGS__) + +#endif /* _KUNIT_TEST_H */
lib/kunit/Kconfig new file +13 -0
@@ -0,0 +1,13 @@ +# +# KUnit base configuration +# + +menuconfig KUNIT + bool "KUnit - Enable support for unit tests" + help + Enables support for kernel unit tests (KUnit), a lightweight unit + testing and mocking framework for the Linux kernel. These tests are + able to be run locally on a developer's workstation without a VM or + special hardware when using UML. Can also be used on most other + architectures. For more information, please see + Documentation/dev-tools/kunit/.
lib/kunit/Makefile new file +1 -0
@@ -0,0 +1 @@ +obj-$(CONFIG_KUNIT) += test.o
lib/kunit/test.c new file +191 -0
191 lines not on this page · raw message
Signature · 1 line
2.23.0.351.gc4317032e6-goog
PATCH v18 2/19 kunit: test: add test resource management API
In reply to Brendan HigginsCreate a common API for test managed resources like memory and test objects. A lot of times a test will want to set up infrastructure to be used in test cases; this could be anything from just wanting to allocate some memory to setting up a driver stack; this defines facilities for creating "test resources" which are managed by the test infrastructure and are automatically cleaned up at the conclusion of the test.
--- include/kunit/test.h | 187 +++++++++++++++++++++++++++++++++++++++++++ lib/kunit/test.c | 163 +++++++++++++++++++++++++++++++++++++ 2 files changed, 350 insertions(+)
include/kunit/test.h +187 -0
@@ -9,8 +9,72 @@#ifndef _KUNIT_TEST_H#define _KUNIT_TEST_H +#include <linux/slab.h> #include <linux/types.h> +struct kunit_resource ; + +typedef int (*kunit_resource_init_t )(struct kunit_resource *,void *); +typedef void (*kunit_resource_free_t )(struct kunit_resource *); + +/** +* struct kunit_resource - represents a *test managed resource* +* @allocation: for the user to store arbitrary data. +* @free: a user supplied function to free the resource. Populated by +* kunit_alloc_resource(). +* +* Represents a *test managed resource*, a resource which will automatically be +* cleaned up at the end of a test case. +* +* Example: +* +* .. code-block:: c +* +* struct kunit_kmalloc_params { +* size_t size; +* gfp_t gfp; +* }; +* +* static int kunit_kmalloc_init(struct kunit_resource *res, void *context) +* { +* struct kunit_kmalloc_params *params = context; +* res->allocation = kmalloc(params->size, params->gfp); +* +* if (!res->allocation) +* return -ENOMEM; +* +* return 0; +* } +* +* static void kunit_kmalloc_free(struct kunit_resource *res) +* { +* kfree(res->allocation); +* } +* +* void *kunit_kmalloc(struct kunit *test, size_t size, gfp_t gfp) +* { +* struct kunit_kmalloc_params params; +* struct kunit_resource *res; +* +* params.size = size; +* params.gfp = gfp; +* +* res = kunit_alloc_resource(test, kunit_kmalloc_init, +* kunit_kmalloc_free, ¶ms); +* if (res) +* return res->allocation; +* +* return NULL; +* } +*/ +struct kunit_resource { +void *allocation; +kunit_resource_free_t free; + +/* private: internal use only. */ +struct list_head node; +}; +struct kunit ;/**
@@ -114,6 +178,13 @@ struct kunit {* with the test case have terminated. */ bool success;/* Read only after test_case finishes! */ +spinlock_t lock;/* Guards all mutable test state. */ +/* +* Because resources is a list that may be updated multiple times (with +* new resources) from any thread associated with a test case, we must +* protect it with some type of lock. +*/ +struct list_head resources;/* Protected by lock. */ };void kunit_init_test (struct kunit *test,const char *name);
@@ -147,6 +218,122 @@ int kunit_run_tests(struct kunit_suite *suite); } \ late_initcall(kunit_suite_init##suite) +/* +* Like kunit_alloc_resource() below, but returns the struct kunit_resource +* object that contains the allocation. This is mostly for testing purposes. +*/ +struct kunit_resource *kunit_alloc_and_get_resource (struct kunit *test, +kunit_resource_init_t init, +kunit_resource_free_t free, +gfp_t internal_gfp, +void *context); + +/** +* kunit_alloc_resource() - Allocates a *test managed resource*. +* @test: The test context object. +* @init: a user supplied function to initialize the resource. +* @free: a user supplied function to free the resource. +* @internal_gfp: gfp to use for internal allocations, if unsure, use GFP_KERNEL +* @context: for the user to pass in arbitrary data to the init function. +* +* Allocates a *test managed resource*, a resource which will automatically be +* cleaned up at the end of a test case. See &struct kunit_resource for an +* example. +* +* NOTE: KUnit needs to allocate memory for each kunit_resource object. You must +* specify an @internal_gfp that is compatible with the use context of your +* resource. +*/ +static inline void *kunit_alloc_resource (struct kunit *test, +kunit_resource_init_t init, +kunit_resource_free_t free, +gfp_t internal_gfp, +void *context) +{ +struct kunit_resource *res; + + res =kunit_alloc_and_get_resource (test, init, free, internal_gfp, + context); + +if (res) +return res->allocation; + +return NULL ; +} + +typedef bool (*kunit_resource_match_t )(struct kunit *test, +const void *res, +void *match_data); + +/** +* kunit_resource_instance_match() - Match a resource with the same instance. +* @test: Test case to which the resource belongs. +* @res: The data stored in kunit_resource->allocation. +* @match_data: The resource pointer to match against. +* +* An instance of kunit_resource_match_t that matches a resource whose +* allocation matches @match_data. +*/ +static inline bool kunit_resource_instance_match (struct kunit *test, +const void *res, +void *match_data) +{ +return res == match_data; +} + +/** +* kunit_resource_destroy() - Find a kunit_resource and destroy it. +* @test: Test case to which the resource belongs. +* @match: Match function. Returns whether a given resource matches @match_data. +* @free: Must match free on the kunit_resource to free. +* @match_data: Data passed into @match. +* +* Free the latest kunit_resource of @test for which @free matches the +* kunit_resource_free_t associated with the resource and for which @match +* returns true. +* +* RETURNS: +* 0 if kunit_resource is found and freed, -ENOENT if not found. +*/ +int kunit_resource_destroy (struct kunit *test, +kunit_resource_match_t match, +kunit_resource_free_t free, +void *match_data); + +/** +* kunit_kmalloc() - Like kmalloc() except the allocation is *test managed*. +* @test: The test context object. +* @size: The size in bytes of the desired memory. +* @gfp: flags passed to underlying kmalloc(). +* +* Just like `kmalloc(...)`, except the allocation is managed by the test case +* and is automatically cleaned up after the test case concludes. See &struct +* kunit_resource for more information. +*/ +void *kunit_kmalloc (struct kunit *test,size_t size,gfp_t gfp); + +/** +* kunit_kfree() - Like kfree except for allocations managed by KUnit. +* @test: The test case to which the resource belongs. +* @ptr: The memory allocation to free. +*/ +void kunit_kfree (struct kunit *test,const void *ptr); + +/** +* kunit_kzalloc() - Just like kunit_kmalloc(), but zeroes the allocation. +* @test: The test context object. +* @size: The size in bytes of the desired memory. +* @gfp: flags passed to underlying kmalloc(). +* +* See kzalloc() and kunit_kmalloc() for more information. +*/ +static inline void *kunit_kzalloc (struct kunit *test,size_t size,gfp_t gfp) +{ +return kunit_kmalloc (test, size, gfp | __GFP_ZERO); +} + +void kunit_cleanup (struct kunit *test); + void __printf(3, 4) kunit_printk(const char *level, const struct kunit *test, const char *fmt, ...);
lib/kunit/test.c +163 -0
181 lines not on this page · raw message
Signature · 1 line
2.23.0.351.gc4317032e6-goog
PATCH v18 3/19 kunit: test: add string_stream a std::stream like string builder
In reply to Brendan HigginsA number of test features need to do pretty complicated string printing where it may not be possible to rely on a single preallocated string with parameters.
So provide a library for constructing the string as you go similar to C++'s std::string. string_stream is really just a string builder, nothing more.
--- include/kunit/string-stream.h | 51 ++++++++ lib/kunit/Makefile | 3 +- lib/kunit/string-stream.c | 217 ++++++++++++++++++++++++++++++++++ 3 files changed, 270 insertions(+), 1 deletion(-) create mode 100644 include/kunit/string-stream.h create mode 100644 lib/kunit/string-stream.c
include/kunit/string-stream.h new file +51 -0
@@ -0,0 +1,51 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* C++ stream style string builder used in KUnit for building messages. +* +* Copyright (C) 2019, Google LLC. +* Author: Brendan Higgins <[email protected]> +*/ + +#ifndef _KUNIT_STRING_STREAM_H +#define _KUNIT_STRING_STREAM_H + +#include <linux/spinlock.h> +#include <linux/types.h> +#include <stdarg.h> + +struct string_stream_fragment { +struct kunit *test; +struct list_head node; +char *fragment; +}; + +struct string_stream { +size_t length; +struct list_head fragments; +/* length and fragments are protected by this lock */ +spinlock_t lock; +struct kunit *test; +gfp_t gfp; +}; + +struct kunit ; + +struct string_stream *alloc_string_stream (struct kunit *test,gfp_t gfp); + +int __printf(2, 3) string_stream_add(struct string_stream *stream, + const char *fmt, ...); + +int string_stream_vadd(struct string_stream *stream, + const char *fmt, + va_list args); + +char *string_stream_get_string(struct string_stream *stream); + +int string_stream_append(struct string_stream *stream, + struct string_stream *other); + +bool string_stream_is_empty(struct string_stream *stream); + +int string_stream_destroy(struct string_stream *stream); + +#endif /* _KUNIT_STRING_STREAM_H */
lib/kunit/Makefile +2 -1
@@ -1 +1,2 @@ -obj-$(CONFIG_KUNIT) += test.o +obj-$(CONFIG_KUNIT) += test.o \ + string-stream.o
lib/kunit/string-stream.c new file +217 -0
217 lines not on this page · raw message
Signature · 6 lines
2.23.0.351.gc4317032e6-goog _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
PATCH v18 4/19 kunit: test: add assertion printing library
In reply to Brendan HigginsAdd struct kunit_assert and friends which provide a structured way to capture data from an expectation or an assertion (introduced later in the series) so that it may be printed out in the event of a failure.
--- include/kunit/assert.h | 356 +++++++++++++++++++++++++++++++++++++++++ lib/kunit/Makefile | 3 +- lib/kunit/assert.c | 141 ++++++++++++++++ 3 files changed, 499 insertions(+), 1 deletion(-) create mode 100644 include/kunit/assert.h create mode 100644 lib/kunit/assert.c
include/kunit/assert.h new file +356 -0
@@ -0,0 +1,356 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Assertion and expectation serialization API. +* +* Copyright (C) 2019, Google LLC. +* Author: Brendan Higgins <[email protected]> +*/ + +#ifndef _KUNIT_ASSERT_H +#define _KUNIT_ASSERT_H + +#include <kunit/string-stream.h> +#include <linux/err.h> + +struct kunit ; + +/** +* enum kunit_assert_type - Type of expectation/assertion. +* @KUNIT_ASSERTION: Used to denote that a kunit_assert represents an assertion. +* @KUNIT_EXPECTATION: Denotes that a kunit_assert represents an expectation. +* +* Used in conjunction with a &struct kunit_assert to denote whether it +* represents an expectation or an assertion. +*/ +enum kunit_assert_type { +KUNIT_ASSERTION , +KUNIT_EXPECTATION , +}; + +/** +* struct kunit_assert - Data for printing a failed assertion or expectation. +* @test: the test case this expectation/assertion is associated with. +* @type: the type (either an expectation or an assertion) of this kunit_assert. +* @line: the source code line number that the expectation/assertion is at. +* @file: the file path of the source file that the expectation/assertion is in. +* @message: an optional message to provide additional context. +* @format: a function which formats the data in this kunit_assert to a string. +* +* Represents a failed expectation/assertion. Contains all the data necessary to +* format a string to a user reporting the failure. +*/ +struct kunit_assert { +struct kunit *test; +enum kunit_assert_type type; +int line; +const char *file; +struct va_format message; +void (*format)(const struct kunit_assert *assert, +struct string_stream *stream); +}; + +/** +* KUNIT_INIT_VA_FMT_NULL - Default initializer for struct va_format. +* +* Used inside a struct initialization block to initialize struct va_format to +* default values where fmt and va are null. +*/ +#define KUNIT_INIT_VA_FMT_NULL { .fmt = NULL, .va = NULL } + +/** +* KUNIT_INIT_ASSERT_STRUCT() - Initializer for a &struct kunit_assert. +* @kunit: The test case that this expectation/assertion is associated with. +* @assert_type: The type (assertion or expectation) of this kunit_assert. +* @fmt: The formatting function which builds a string out of this kunit_assert. +* +* The base initializer for a &struct kunit_assert. +*/ +#define KUNIT_INIT_ASSERT_STRUCT (kunit, assert_type, fmt) { \ + .test = kunit, \ + .type = assert_type, \ + .file = __FILE__, \ + .line = __LINE__, \ + .message = KUNIT_INIT_VA_FMT_NULL, \ + .format = fmt \ +} + +void kunit_base_assert_format (const struct kunit_assert *assert, +struct string_stream *stream); + +void kunit_assert_print_msg (const struct kunit_assert *assert, +struct string_stream *stream); + +/** +* struct kunit_fail_assert - Represents a plain fail expectation/assertion. +* @assert: The parent of this type. +* +* Represents a simple KUNIT_FAIL/KUNIT_ASSERT_FAILURE that always fails. +*/ +struct kunit_fail_assert { +struct kunit_assert assert; +}; + +void kunit_fail_assert_format (const struct kunit_assert *assert, +struct string_stream *stream); + +/** +* KUNIT_INIT_FAIL_ASSERT_STRUCT() - Initializer for &struct kunit_fail_assert. +* @test: The test case that this expectation/assertion is associated with. +* @type: The type (assertion or expectation) of this kunit_assert. +* +* Initializes a &struct kunit_fail_assert. Intended to be used in +* KUNIT_EXPECT_* and KUNIT_ASSERT_* macros. +*/ +#define KUNIT_INIT_FAIL_ASSERT_STRUCT (test, type) { \ + .assert = KUNIT_INIT_ASSERT_STRUCT(test, \ + type, \ + kunit_fail_assert_format) \ +} + +/** +* struct kunit_unary_assert - Represents a KUNIT_{EXPECT|ASSERT}_{TRUE|FALSE} +* @assert: The parent of this type. +* @condition: A string representation of a conditional expression. +* @expected_true: True if of type KUNIT_{EXPECT|ASSERT}_TRUE, false otherwise. +* +* Represents a simple expectation or assertion that simply asserts something is +* true or false. In other words, represents the expectations: +* KUNIT_{EXPECT|ASSERT}_{TRUE|FALSE} +*/ +struct kunit_unary_assert { +struct kunit_assert assert; +const char *condition; +bool expected_true; +}; + +void kunit_unary_assert_format (const struct kunit_assert *assert, +struct string_stream *stream); + +/** +* KUNIT_INIT_UNARY_ASSERT_STRUCT() - Initializes &struct kunit_unary_assert. +* @test: The test case that this expectation/assertion is associated with. +* @type: The type (assertion or expectation) of this kunit_assert. +* @cond: A string representation of the expression asserted true or false. +* @expect_true: True if of type KUNIT_{EXPECT|ASSERT}_TRUE, false otherwise. +* +* Initializes a &struct kunit_unary_assert. Intended to be used in +* KUNIT_EXPECT_* and KUNIT_ASSERT_* macros. +*/ +#define KUNIT_INIT_UNARY_ASSERT_STRUCT (test, type, cond, expect_true) { \ + .assert = KUNIT_INIT_ASSERT_STRUCT(test, \ + type, \ + kunit_unary_assert_format), \ + .condition = cond, \ + .expected_true = expect_true \ +} + +/** +* struct kunit_ptr_not_err_assert - An expectation/assertion that a pointer is +* not NULL and not a -errno. +* @assert: The parent of this type. +* @text: A string representation of the expression passed to the expectation. +* @value: The actual evaluated pointer value of the expression. +* +* Represents an expectation/assertion that a pointer is not null and is does +* not contain a -errno. (See IS_ERR_OR_NULL().) +*/ +struct kunit_ptr_not_err_assert { +struct kunit_assert assert; +const char *text; +const void *value; +}; + +void kunit_ptr_not_err_assert_format (const struct kunit_assert *assert, +struct string_stream *stream); + +/** +* KUNIT_INIT_PTR_NOT_ERR_ASSERT_STRUCT() - Initializes a +* &struct kunit_ptr_not_err_assert. +* @test: The test case that this expectation/assertion is associated with. +* @type: The type (assertion or expectation) of this kunit_assert. +* @txt: A string representation of the expression passed to the expectation. +* @val: The actual evaluated pointer value of the expression. +* +* Initializes a &struct kunit_ptr_not_err_assert. Intended to be used in +* KUNIT_EXPECT_* and KUNIT_ASSERT_* macros. +*/ +#define KUNIT_INIT_PTR_NOT_ERR_STRUCT (test, type, txt, val) { \ + .assert = KUNIT_INIT_ASSERT_STRUCT(test, \ + type, \ + kunit_ptr_not_err_assert_format), \ + .text = txt, \ + .value = val \ +} + +/** +* struct kunit_binary_assert - An expectation/assertion that compares two +* non-pointer values (for example, KUNIT_EXPECT_EQ(test, 1 + 1, 2)). +* @assert: The parent of this type. +* @operation: A string representation of the comparison operator (e.g. "=="). +* @left_text: A string representation of the expression in the left slot. +* @left_value: The actual evaluated value of the expression in the left slot. +* @right_text: A string representation of the expression in the right slot. +* @right_value: The actual evaluated value of the expression in the right slot. +* +* Represents an expectation/assertion that compares two non-pointer values. For +* example, to expect that 1 + 1 == 2, you can use the expectation +* KUNIT_EXPECT_EQ(test, 1 + 1, 2); +*/ +struct kunit_binary_assert { +struct kunit_assert assert; +const char *operation; +const char *left_text; +long long left_value; +const char *right_text; +long long right_value; +}; + +void kunit_binary_assert_format (const struct kunit_assert *assert, +struct string_stream *stream); + +/** +* KUNIT_INIT_BINARY_ASSERT_STRUCT() - Initializes a +* &struct kunit_binary_assert. +* @test: The test case that this expectation/assertion is associated with. +* @type: The type (assertion or expectation) of this kunit_assert. +* @op_str: A string representation of the comparison operator (e.g. "=="). +* @left_str: A string representation of the expression in the left slot. +* @left_val: The actual evaluated value of the expression in the left slot. +* @right_str: A string representation of the expression in the right slot. +* @right_val: The actual evaluated value of the expression in the right slot. +* +* Initializes a &struct kunit_binary_assert. Intended to be used in +* KUNIT_EXPECT_* and KUNIT_ASSERT_* macros. +*/ +#define KUNIT_INIT_BINARY_ASSERT_STRUCT (test, \ + type, \ + op_str, \ + left_str, \ + left_val, \ + right_str, \ + right_val) { \ + .assert = KUNIT_INIT_ASSERT_STRUCT(test, \ + type, \ + kunit_binary_assert_format), \ + .operation = op_str, \ + .left_text = left_str, \ + .left_value = left_val, \ + .right_text = right_str, \ + .right_value = right_val \ +} + +/** +* struct kunit_binary_ptr_assert - An expectation/assertion that compares two +* pointer values (for example, KUNIT_EXPECT_PTR_EQ(test, foo, bar)). +* @assert: The parent of this type. +* @operation: A string representation of the comparison operator (e.g. "=="). +* @left_text: A string representation of the expression in the left slot. +* @left_value: The actual evaluated value of the expression in the left slot. +* @right_text: A string representation of the expression in the right slot. +* @right_value: The actual evaluated value of the expression in the right slot. +* +* Represents an expectation/assertion that compares two pointer values. For +* example, to expect that foo and bar point to the same thing, you can use the +* expectation KUNIT_EXPECT_PTR_EQ(test, foo, bar); +*/ +struct kunit_binary_ptr_assert { +struct kunit_assert assert; +const char *operation; +const char *left_text; +const void *left_value; +const char *right_text; +const void *right_value; +}; + +void kunit_binary_ptr_assert_format (const struct kunit_assert *assert, +struct string_stream *stream); + +/** +* KUNIT_INIT_BINARY_PTR_ASSERT_STRUCT() - Initializes a +* &struct kunit_binary_ptr_assert. +* @test: The test case that this expectation/assertion is associated with. +* @type: The type (assertion or expectation) of this kunit_assert. +* @op_str: A string representation of the comparison operator (e.g. "=="). +* @left_str: A string representation of the expression in the left slot. +* @left_val: The actual evaluated value of the expression in the left slot. +* @right_str: A string representation of the expression in the right slot. +* @right_val: The actual evaluated value of the expression in the right slot. +* +* Initializes a &struct kunit_binary_ptr_assert. Intended to be used in +* KUNIT_EXPECT_* and KUNIT_ASSERT_* macros. +*/ +#define KUNIT_INIT_BINARY_PTR_ASSERT_STRUCT (test, \ + type, \ + op_str, \ + left_str, \ + left_val, \ + right_str, \ + right_val) { \ + .assert = KUNIT_INIT_ASSERT_STRUCT(test, \ + type, \ + kunit_binary_ptr_assert_format), \ + .operation = op_str, \ + .left_text = left_str, \ + .left_value = left_val, \ + .right_text = right_str, \ + .right_value = right_val \ +} + +/** +* struct kunit_binary_str_assert - An expectation/assertion that compares two +* string values (for example, KUNIT_EXPECT_STREQ(test, foo, "bar")). +* @assert: The parent of this type. +* @operation: A string representation of the comparison operator (e.g. "=="). +* @left_text: A string representation of the expression in the left slot. +* @left_value: The actual evaluated value of the expression in the left slot. +* @right_text: A string representation of the expression in the right slot. +* @right_value: The actual evaluated value of the expression in the right slot. +* +* Represents an expectation/assertion that compares two string values. For +* example, to expect that the string in foo is equal to "bar", you can use the +* expectation KUNIT_EXPECT_STREQ(test, foo, "bar"); +*/ +struct kunit_binary_str_assert { +struct kunit_assert assert; +const char *operation; +const char *left_text; +const char *left_value; +const char *right_text; +const char *right_value; +}; + +void kunit_binary_str_assert_format (const struct kunit_assert *assert, +struct string_stream *stream); + +/** +* KUNIT_INIT_BINARY_STR_ASSERT_STRUCT() - Initializes a +* &struct kunit_binary_str_assert. +* @test: The test case that this expectation/assertion is associated with. +* @type: The type (assertion or expectation) of this kunit_assert. +* @op_str: A string representation of the comparison operator (e.g. "=="). +* @left_str: A string representation of the expression in the left slot. +* @left_val: The actual evaluated value of the expression in the left slot. +* @right_str: A string representation of the expression in the right slot. +* @right_val: The actual evaluated value of the expression in the right slot. +* +* Initializes a &struct kunit_binary_str_assert. Intended to be used in +* KUNIT_EXPECT_* and KUNIT_ASSERT_* macros. +*/ +#define KUNIT_INIT_BINARY_STR_ASSERT_STRUCT (test, \ + type, \ + op_str, \ + left_str, \ + left_val, \ + right_str, \ + right_val) { \ + .assert = KUNIT_INIT_ASSERT_STRUCT(test, \ + type, \ + kunit_binary_str_assert_format), \ + .operation = op_str, \ + .left_text = left_str, \ + .left_value = left_val, \ + .right_text = right_str, \ + .right_value = right_val \ +} + +#endif /* _KUNIT_ASSERT_H */
lib/kunit/Makefile +2 -1
@@ -1,2 +1,3 @@ obj-$(CONFIG_KUNIT) += test.o \ - string-stream.o + string-stream.o \ + assert.o
lib/kunit/assert.c new file +141 -0
141 lines not on this page · raw message
Signature · 1 line
2.23.0.351.gc4317032e6-goog
PATCH v18 5/19 kunit: test: add the concept of expectations
In reply to Brendan HigginsAdd support for expectations, which allow properties to be specified and then verified in tests.
--- include/kunit/test.h | 836 +++++++++++++++++++++++++++++++++++++++++++ lib/kunit/test.c | 62 ++++ 2 files changed, 898 insertions(+)
include/kunit/test.h +836 -0
@@ -9,6 +9,8 @@#ifndef _KUNIT_TEST_H#define _KUNIT_TEST_H +#include <kunit/assert.h> +#include <linux/kernel.h> #include <linux/slab.h> #include <linux/types.h>
@@ -372,4 +374,838 @@ void __printf(3, 4) kunit_printk(const char *level,#define kunit_err (test, fmt, ...) \ kunit_printk(KERN_ERR, test, fmt, ##__VA_ARGS__) +/** +* KUNIT_SUCCEED() - A no-op expectation. Only exists for code clarity. +* @test: The test context object. +* +* The opposite of KUNIT_FAIL(), it is an expectation that cannot fail. In other +* words, it does nothing and only exists for code clarity. See +* KUNIT_EXPECT_TRUE() for more information. +*/ +#define KUNIT_SUCCEED (test) do {} while (0) + +void kunit_do_assertion (struct kunit *test, +struct kunit_assert *assert, +bool pass, +const char *fmt, ...); + +#define KUNIT_ASSERTION (test, pass, assert_class,INITIALIZER , fmt, ...) do { \ + struct assert_class __assertion = INITIALIZER; \ + kunit_do_assertion(test, \ + &__assertion.assert, \ + pass, \ + fmt, \ + ##__VA_ARGS__); \ +} while (0) + + +#define KUNIT_FAIL_ASSERTION (test, assert_type, fmt, ...) \ + KUNIT_ASSERTION(test, \ + false, \ + kunit_fail_assert, \ + KUNIT_INIT_FAIL_ASSERT_STRUCT(test, assert_type), \ + fmt, \ + ##__VA_ARGS__) + +/** +* KUNIT_FAIL() - Always causes a test to fail when evaluated. +* @test: The test context object. +* @fmt: an informational message to be printed when the assertion is made. +* @...: string format arguments. +* +* The opposite of KUNIT_SUCCEED(), it is an expectation that always fails. In +* other words, it always results in a failed expectation, and consequently +* always causes the test case to fail when evaluated. See KUNIT_EXPECT_TRUE() +* for more information. +*/ +#define KUNIT_FAIL (test, fmt, ...) \ + KUNIT_FAIL_ASSERTION(test, \ + KUNIT_EXPECTATION, \ + fmt, \ + ##__VA_ARGS__) + +#define KUNIT_UNARY_ASSERTION (test, \ + assert_type, \ + condition, \ + expected_true, \ + fmt, \ + ...) \ + KUNIT_ASSERTION(test, \ + !!(condition) == !!expected_true, \ + kunit_unary_assert, \ + KUNIT_INIT_UNARY_ASSERT_STRUCT(test, \ + assert_type, \ + #condition, \ + expected_true), \ + fmt, \ + ##__VA_ARGS__) + +#define KUNIT_TRUE_MSG_ASSERTION (test, assert_type, condition, fmt, ...) \ + KUNIT_UNARY_ASSERTION(test, \ + assert_type, \ + condition, \ + true, \ + fmt, \ + ##__VA_ARGS__) + +#define KUNIT_TRUE_ASSERTION (test, assert_type, condition) \ + KUNIT_TRUE_MSG_ASSERTION(test, assert_type, condition, NULL) + +#define KUNIT_FALSE_MSG_ASSERTION (test, assert_type, condition, fmt, ...) \ + KUNIT_UNARY_ASSERTION(test, \ + assert_type, \ + condition, \ + false, \ + fmt, \ + ##__VA_ARGS__) + +#define KUNIT_FALSE_ASSERTION (test, assert_type, condition) \ + KUNIT_FALSE_MSG_ASSERTION(test, assert_type, condition, NULL) + +/* +* A factory macro for defining the assertions and expectations for the basic +* comparisons defined for the built in types. +* +* Unfortunately, there is no common type that all types can be promoted to for +* which all the binary operators behave the same way as for the actual types +* (for example, there is no type that long long and unsigned long long can +* both be cast to where the comparison result is preserved for all values). So +* the best we can do is do the comparison in the original types and then coerce +* everything to long long for printing; this way, the comparison behaves +* correctly and the printed out value usually makes sense without +* interpretation, but can always be interpreted to figure out the actual +* value. +*/ +#define KUNIT_BASE_BINARY_ASSERTION (test, \ + assert_class, \ +ASSERT_CLASS_INIT , \ + assert_type, \ + left, \ + op, \ + right, \ + fmt, \ + ...) \ +do { \ + typeof(left) __left = (left); \ + typeof(right) __right = (right); \ + ((void)__typecheck(__left, __right)); \ + \ + KUNIT_ASSERTION(test, \ + __left op __right, \ + assert_class, \ + ASSERT_CLASS_INIT(test, \ + assert_type, \ + #op, \ + #left, \ + __left, \ + #right, \ + __right), \ + fmt, \ + ##__VA_ARGS__); \ +} while (0) + +#define KUNIT_BASE_EQ_MSG_ASSERTION (test, \ + assert_class, \ +ASSERT_CLASS_INIT , \ + assert_type, \ + left, \ + right, \ + fmt, \ + ...) \ + KUNIT_BASE_BINARY_ASSERTION(test, \ + assert_class, \ + ASSERT_CLASS_INIT, \ + assert_type, \ + left, ==, right, \ + fmt, \ + ##__VA_ARGS__) + +#define KUNIT_BASE_NE_MSG_ASSERTION (test, \ + assert_class, \ +ASSERT_CLASS_INIT , \ + assert_type, \ + left, \ + right, \ + fmt, \ + ...) \ + KUNIT_BASE_BINARY_ASSERTION(test, \ + assert_class, \ + ASSERT_CLASS_INIT, \ + assert_type, \ + left, !=, right, \ + fmt, \ + ##__VA_ARGS__) + +#define KUNIT_BASE_LT_MSG_ASSERTION (test, \ + assert_class, \ +ASSERT_CLASS_INIT , \ + assert_type, \ + left, \ + right, \ + fmt, \ + ...) \ + KUNIT_BASE_BINARY_ASSERTION(test, \ + assert_class, \ + ASSERT_CLASS_INIT, \ + assert_type, \ + left, <, right, \ + fmt, \ + ##__VA_ARGS__) + +#define KUNIT_BASE_LE_MSG_ASSERTION (test, \ + assert_class, \ +ASSERT_CLASS_INIT , \ + assert_type, \ + left, \ + right, \ + fmt, \ + ...) \ + KUNIT_BASE_BINARY_ASSERTION(test, \ + assert_class, \ + ASSERT_CLASS_INIT, \ + assert_type, \ + left, <=, right, \ + fmt, \ + ##__VA_ARGS__) + +#define KUNIT_BASE_GT_MSG_ASSERTION (test, \ + assert_class, \ +ASSERT_CLASS_INIT , \ + assert_type, \ + left, \ + right, \ + fmt, \ + ...) \ + KUNIT_BASE_BINARY_ASSERTION(test, \ + assert_class, \ + ASSERT_CLASS_INIT, \ + assert_type, \ + left, >, right, \ + fmt, \ + ##__VA_ARGS__) + +#define KUNIT_BASE_GE_MSG_ASSERTION (test, \ + assert_class, \ +ASSERT_CLASS_INIT , \ + assert_type, \ + left, \ + right, \ + fmt, \ + ...) \ + KUNIT_BASE_BINARY_ASSERTION(test, \ + assert_class, \ + ASSERT_CLASS_INIT, \ + assert_type, \ + left, >=, right, \ + fmt, \ + ##__VA_ARGS__) + +#define KUNIT_BINARY_EQ_MSG_ASSERTION (test, assert_type, left, right, fmt, ...)\ + KUNIT_BASE_EQ_MSG_ASSERTION(test, \ + kunit_binary_assert, \ + KUNIT_INIT_BINARY_ASSERT_STRUCT, \ + assert_type, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +#define KUNIT_BINARY_EQ_ASSERTION (test, assert_type, left, right) \ + KUNIT_BINARY_EQ_MSG_ASSERTION(test, \ + assert_type, \ + left, \ + right, \ + NULL) + +#define KUNIT_BINARY_PTR_EQ_MSG_ASSERTION (test, \ + assert_type, \ + left, \ + right, \ + fmt, \ + ...) \ + KUNIT_BASE_EQ_MSG_ASSERTION(test, \ + kunit_binary_ptr_assert, \ + KUNIT_INIT_BINARY_PTR_ASSERT_STRUCT, \ + assert_type, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +#define KUNIT_BINARY_PTR_EQ_ASSERTION (test, assert_type, left, right) \ + KUNIT_BINARY_PTR_EQ_MSG_ASSERTION(test, \ + assert_type, \ + left, \ + right, \ + NULL) + +#define KUNIT_BINARY_NE_MSG_ASSERTION (test, assert_type, left, right, fmt, ...)\ + KUNIT_BASE_NE_MSG_ASSERTION(test, \ + kunit_binary_assert, \ + KUNIT_INIT_BINARY_ASSERT_STRUCT, \ + assert_type, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +#define KUNIT_BINARY_NE_ASSERTION (test, assert_type, left, right) \ + KUNIT_BINARY_NE_MSG_ASSERTION(test, \ + assert_type, \ + left, \ + right, \ + NULL) + +#define KUNIT_BINARY_PTR_NE_MSG_ASSERTION (test, \ + assert_type, \ + left, \ + right, \ + fmt, \ + ...) \ + KUNIT_BASE_NE_MSG_ASSERTION(test, \ + kunit_binary_ptr_assert, \ + KUNIT_INIT_BINARY_PTR_ASSERT_STRUCT, \ + assert_type, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +#define KUNIT_BINARY_PTR_NE_ASSERTION (test, assert_type, left, right) \ + KUNIT_BINARY_PTR_NE_MSG_ASSERTION(test, \ + assert_type, \ + left, \ + right, \ + NULL) + +#define KUNIT_BINARY_LT_MSG_ASSERTION (test, assert_type, left, right, fmt, ...)\ + KUNIT_BASE_LT_MSG_ASSERTION(test, \ + kunit_binary_assert, \ + KUNIT_INIT_BINARY_ASSERT_STRUCT, \ + assert_type, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +#define KUNIT_BINARY_LT_ASSERTION (test, assert_type, left, right) \ + KUNIT_BINARY_LT_MSG_ASSERTION(test, \ + assert_type, \ + left, \ + right, \ + NULL) + +#define KUNIT_BINARY_PTR_LT_MSG_ASSERTION (test, \ + assert_type, \ + left, \ + right, \ + fmt, \ + ...) \ + KUNIT_BASE_LT_MSG_ASSERTION(test, \ + kunit_binary_ptr_assert, \ + KUNIT_INIT_BINARY_PTR_ASSERT_STRUCT, \ + assert_type, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +#define KUNIT_BINARY_PTR_LT_ASSERTION (test, assert_type, left, right) \ + KUNIT_BINARY_PTR_LT_MSG_ASSERTION(test, \ + assert_type, \ + left, \ + right, \ + NULL) + +#define KUNIT_BINARY_LE_MSG_ASSERTION (test, assert_type, left, right, fmt, ...)\ + KUNIT_BASE_LE_MSG_ASSERTION(test, \ + kunit_binary_assert, \ + KUNIT_INIT_BINARY_ASSERT_STRUCT, \ + assert_type, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +#define KUNIT_BINARY_LE_ASSERTION (test, assert_type, left, right) \ + KUNIT_BINARY_LE_MSG_ASSERTION(test, \ + assert_type, \ + left, \ + right, \ + NULL) + +#define KUNIT_BINARY_PTR_LE_MSG_ASSERTION (test, \ + assert_type, \ + left, \ + right, \ + fmt, \ + ...) \ + KUNIT_BASE_LE_MSG_ASSERTION(test, \ + kunit_binary_ptr_assert, \ + KUNIT_INIT_BINARY_PTR_ASSERT_STRUCT, \ + assert_type, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +#define KUNIT_BINARY_PTR_LE_ASSERTION (test, assert_type, left, right) \ + KUNIT_BINARY_PTR_LE_MSG_ASSERTION(test, \ + assert_type, \ + left, \ + right, \ + NULL) + +#define KUNIT_BINARY_GT_MSG_ASSERTION (test, assert_type, left, right, fmt, ...)\ + KUNIT_BASE_GT_MSG_ASSERTION(test, \ + kunit_binary_assert, \ + KUNIT_INIT_BINARY_ASSERT_STRUCT, \ + assert_type, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +#define KUNIT_BINARY_GT_ASSERTION (test, assert_type, left, right) \ + KUNIT_BINARY_GT_MSG_ASSERTION(test, \ + assert_type, \ + left, \ + right, \ + NULL) + +#define KUNIT_BINARY_PTR_GT_MSG_ASSERTION (test, \ + assert_type, \ + left, \ + right, \ + fmt, \ + ...) \ + KUNIT_BASE_GT_MSG_ASSERTION(test, \ + kunit_binary_ptr_assert, \ + KUNIT_INIT_BINARY_PTR_ASSERT_STRUCT, \ + assert_type, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +#define KUNIT_BINARY_PTR_GT_ASSERTION (test, assert_type, left, right) \ + KUNIT_BINARY_PTR_GT_MSG_ASSERTION(test, \ + assert_type, \ + left, \ + right, \ + NULL) + +#define KUNIT_BINARY_GE_MSG_ASSERTION (test, assert_type, left, right, fmt, ...)\ + KUNIT_BASE_GE_MSG_ASSERTION(test, \ + kunit_binary_assert, \ + KUNIT_INIT_BINARY_ASSERT_STRUCT, \ + assert_type, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +#define KUNIT_BINARY_GE_ASSERTION (test, assert_type, left, right) \ + KUNIT_BINARY_GE_MSG_ASSERTION(test, \ + assert_type, \ + left, \ + right, \ + NULL) + +#define KUNIT_BINARY_PTR_GE_MSG_ASSERTION (test, \ + assert_type, \ + left, \ + right, \ + fmt, \ + ...) \ + KUNIT_BASE_GE_MSG_ASSERTION(test, \ + kunit_binary_ptr_assert, \ + KUNIT_INIT_BINARY_PTR_ASSERT_STRUCT, \ + assert_type, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +#define KUNIT_BINARY_PTR_GE_ASSERTION (test, assert_type, left, right) \ + KUNIT_BINARY_PTR_GE_MSG_ASSERTION(test, \ + assert_type, \ + left, \ + right, \ + NULL) + +#define KUNIT_BINARY_STR_ASSERTION (test, \ + assert_type, \ + left, \ + op, \ + right, \ + fmt, \ + ...) \ +do { \ + typeof(left) __left = (left); \ + typeof(right) __right = (right); \ + \ + KUNIT_ASSERTION(test, \ + strcmp(__left, __right) op 0, \ + kunit_binary_str_assert, \ + KUNIT_INIT_BINARY_ASSERT_STRUCT(test, \ + assert_type, \ + #op, \ + #left, \ + __left, \ + #right, \ + __right), \ + fmt, \ + ##__VA_ARGS__); \ +} while (0) + +#define KUNIT_BINARY_STR_EQ_MSG_ASSERTION (test, \ + assert_type, \ + left, \ + right, \ + fmt, \ + ...) \ + KUNIT_BINARY_STR_ASSERTION(test, \ + assert_type, \ + left, ==, right, \ + fmt, \ + ##__VA_ARGS__) + +#define KUNIT_BINARY_STR_EQ_ASSERTION (test, assert_type, left, right) \ + KUNIT_BINARY_STR_EQ_MSG_ASSERTION(test, \ + assert_type, \ + left, \ + right, \ + NULL) + +#define KUNIT_BINARY_STR_NE_MSG_ASSERTION (test, \ + assert_type, \ + left, \ + right, \ + fmt, \ + ...) \ + KUNIT_BINARY_STR_ASSERTION(test, \ + assert_type, \ + left, !=, right, \ + fmt, \ + ##__VA_ARGS__) + +#define KUNIT_BINARY_STR_NE_ASSERTION (test, assert_type, left, right) \ + KUNIT_BINARY_STR_NE_MSG_ASSERTION(test, \ + assert_type, \ + left, \ + right, \ + NULL) + +#define KUNIT_PTR_NOT_ERR_OR_NULL_MSG_ASSERTION (test, \ + assert_type, \ + ptr, \ + fmt, \ + ...) \ +do { \ + typeof(ptr) __ptr = (ptr); \ + \ + KUNIT_ASSERTION(test, \ + !IS_ERR_OR_NULL(__ptr), \ + kunit_ptr_not_err_assert, \ + KUNIT_INIT_PTR_NOT_ERR_STRUCT(test, \ + assert_type, \ + #ptr, \ + __ptr), \ + fmt, \ + ##__VA_ARGS__); \ +} while (0) + +#define KUNIT_PTR_NOT_ERR_OR_NULL_ASSERTION (test, assert_type, ptr) \ + KUNIT_PTR_NOT_ERR_OR_NULL_MSG_ASSERTION(test, \ + assert_type, \ + ptr, \ + NULL) + +/** +* KUNIT_EXPECT_TRUE() - Causes a test failure when the expression is not true. +* @test: The test context object. +* @condition: an arbitrary boolean expression. The test fails when this does +* not evaluate to true. +* +* This and expectations of the form `KUNIT_EXPECT_*` will cause the test case +* to fail when the specified condition is not met; however, it will not prevent +* the test case from continuing to run; this is otherwise known as an +* *expectation failure*. +*/ +#define KUNIT_EXPECT_TRUE (test, condition) \ + KUNIT_TRUE_ASSERTION(test, KUNIT_EXPECTATION, condition) + +#define KUNIT_EXPECT_TRUE_MSG (test, condition, fmt, ...) \ + KUNIT_TRUE_MSG_ASSERTION(test, \ + KUNIT_EXPECTATION, \ + condition, \ + fmt, \ + ##__VA_ARGS__) + +/** +* KUNIT_EXPECT_FALSE() - Makes a test failure when the expression is not false. +* @test: The test context object. +* @condition: an arbitrary boolean expression. The test fails when this does +* not evaluate to false. +* +* Sets an expectation that @condition evaluates to false. See +* KUNIT_EXPECT_TRUE() for more information. +*/ +#define KUNIT_EXPECT_FALSE (test, condition) \ + KUNIT_FALSE_ASSERTION(test, KUNIT_EXPECTATION, condition) + +#define KUNIT_EXPECT_FALSE_MSG (test, condition, fmt, ...) \ + KUNIT_FALSE_MSG_ASSERTION(test, \ + KUNIT_EXPECTATION, \ + condition, \ + fmt, \ + ##__VA_ARGS__) + +/** +* KUNIT_EXPECT_EQ() - Sets an expectation that @left and @right are equal. +* @test: The test context object. +* @left: an arbitrary expression that evaluates to a primitive C type. +* @right: an arbitrary expression that evaluates to a primitive C type. +* +* Sets an expectation that the values that @left and @right evaluate to are +* equal. This is semantically equivalent to +* KUNIT_EXPECT_TRUE(@test, (@left) == (@right)). See KUNIT_EXPECT_TRUE() for +* more information. +*/ +#define KUNIT_EXPECT_EQ (test, left, right) \ + KUNIT_BINARY_EQ_ASSERTION(test, KUNIT_EXPECTATION, left, right) + +#define KUNIT_EXPECT_EQ_MSG (test, left, right, fmt, ...) \ + KUNIT_BINARY_EQ_MSG_ASSERTION(test, \ + KUNIT_EXPECTATION, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +/** +* KUNIT_EXPECT_PTR_EQ() - Expects that pointers @left and @right are equal. +* @test: The test context object. +* @left: an arbitrary expression that evaluates to a pointer. +* @right: an arbitrary expression that evaluates to a pointer. +* +* Sets an expectation that the values that @left and @right evaluate to are +* equal. This is semantically equivalent to +* KUNIT_EXPECT_TRUE(@test, (@left) == (@right)). See KUNIT_EXPECT_TRUE() for +* more information. +*/ +#define KUNIT_EXPECT_PTR_EQ (test, left, right) \ + KUNIT_BINARY_PTR_EQ_ASSERTION(test, \ + KUNIT_EXPECTATION, \ + left, \ + right) + +#define KUNIT_EXPECT_PTR_EQ_MSG (test, left, right, fmt, ...) \ + KUNIT_BINARY_PTR_EQ_MSG_ASSERTION(test, \ + KUNIT_EXPECTATION, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +/** +* KUNIT_EXPECT_NE() - An expectation that @left and @right are not equal. +* @test: The test context object. +* @left: an arbitrary expression that evaluates to a primitive C type. +* @right: an arbitrary expression that evaluates to a primitive C type. +* +* Sets an expectation that the values that @left and @right evaluate to are not +* equal. This is semantically equivalent to +* KUNIT_EXPECT_TRUE(@test, (@left) != (@right)). See KUNIT_EXPECT_TRUE() for +* more information. +*/ +#define KUNIT_EXPECT_NE (test, left, right) \ + KUNIT_BINARY_NE_ASSERTION(test, KUNIT_EXPECTATION, left, right) + +#define KUNIT_EXPECT_NE_MSG (test, left, right, fmt, ...) \ + KUNIT_BINARY_NE_MSG_ASSERTION(test, \ + KUNIT_EXPECTATION, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +/** +* KUNIT_EXPECT_PTR_NE() - Expects that pointers @left and @right are not equal. +* @test: The test context object. +* @left: an arbitrary expression that evaluates to a pointer. +* @right: an arbitrary expression that evaluates to a pointer. +* +* Sets an expectation that the values that @left and @right evaluate to are not +* equal. This is semantically equivalent to +* KUNIT_EXPECT_TRUE(@test, (@left) != (@right)). See KUNIT_EXPECT_TRUE() for +* more information. +*/ +#define KUNIT_EXPECT_PTR_NE (test, left, right) \ + KUNIT_BINARY_PTR_NE_ASSERTION(test, \ + KUNIT_EXPECTATION, \ + left, \ + right) + +#define KUNIT_EXPECT_PTR_NE_MSG (test, left, right, fmt, ...) \ + KUNIT_BINARY_PTR_NE_MSG_ASSERTION(test, \ + KUNIT_EXPECTATION, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +/** +* KUNIT_EXPECT_LT() - An expectation that @left is less than @right. +* @test: The test context object. +* @left: an arbitrary expression that evaluates to a primitive C type. +* @right: an arbitrary expression that evaluates to a primitive C type. +* +* Sets an expectation that the value that @left evaluates to is less than the +* value that @right evaluates to. This is semantically equivalent to +* KUNIT_EXPECT_TRUE(@test, (@left) < (@right)). See KUNIT_EXPECT_TRUE() for +* more information. +*/ +#define KUNIT_EXPECT_LT (test, left, right) \ + KUNIT_BINARY_LT_ASSERTION(test, KUNIT_EXPECTATION, left, right) + +#define KUNIT_EXPECT_LT_MSG (test, left, right, fmt, ...) \ + KUNIT_BINARY_LT_MSG_ASSERTION(test, \ + KUNIT_EXPECTATION, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +/** +* KUNIT_EXPECT_LE() - Expects that @left is less than or equal to @right. +* @test: The test context object. +* @left: an arbitrary expression that evaluates to a primitive C type. +* @right: an arbitrary expression that evaluates to a primitive C type. +* +* Sets an expectation that the value that @left evaluates to is less than or +* equal to the value that @right evaluates to. Semantically this is equivalent +* to KUNIT_EXPECT_TRUE(@test, (@left) <= (@right)). See KUNIT_EXPECT_TRUE() for +* more information. +*/ +#define KUNIT_EXPECT_LE (test, left, right) \ + KUNIT_BINARY_LE_ASSERTION(test, KUNIT_EXPECTATION, left, right) + +#define KUNIT_EXPECT_LE_MSG (test, left, right, fmt, ...) \ + KUNIT_BINARY_LE_MSG_ASSERTION(test, \ + KUNIT_EXPECTATION, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +/** +* KUNIT_EXPECT_GT() - An expectation that @left is greater than @right. +* @test: The test context object. +* @left: an arbitrary expression that evaluates to a primitive C type. +* @right: an arbitrary expression that evaluates to a primitive C type. +* +* Sets an expectation that the value that @left evaluates to is greater than +* the value that @right evaluates to. This is semantically equivalent to +* KUNIT_EXPECT_TRUE(@test, (@left) > (@right)). See KUNIT_EXPECT_TRUE() for +* more information. +*/ +#define KUNIT_EXPECT_GT (test, left, right) \ + KUNIT_BINARY_GT_ASSERTION(test, KUNIT_EXPECTATION, left, right) + +#define KUNIT_EXPECT_GT_MSG (test, left, right, fmt, ...) \ + KUNIT_BINARY_GT_MSG_ASSERTION(test, \ + KUNIT_EXPECTATION, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +/** +* KUNIT_EXPECT_GE() - Expects that @left is greater than or equal to @right. +* @test: The test context object. +* @left: an arbitrary expression that evaluates to a primitive C type. +* @right: an arbitrary expression that evaluates to a primitive C type. +* +* Sets an expectation that the value that @left evaluates to is greater than +* the value that @right evaluates to. This is semantically equivalent to +* KUNIT_EXPECT_TRUE(@test, (@left) >= (@right)). See KUNIT_EXPECT_TRUE() for +* more information. +*/ +#define KUNIT_EXPECT_GE (test, left, right) \ + KUNIT_BINARY_GE_ASSERTION(test, KUNIT_EXPECTATION, left, right) + +#define KUNIT_EXPECT_GE_MSG (test, left, right, fmt, ...) \ + KUNIT_BINARY_GE_MSG_ASSERTION(test, \ + KUNIT_EXPECTATION, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +/** +* KUNIT_EXPECT_STREQ() - Expects that strings @left and @right are equal. +* @test: The test context object. +* @left: an arbitrary expression that evaluates to a null terminated string. +* @right: an arbitrary expression that evaluates to a null terminated string. +* +* Sets an expectation that the values that @left and @right evaluate to are +* equal. This is semantically equivalent to +* KUNIT_EXPECT_TRUE(@test, !strcmp((@left), (@right))). See KUNIT_EXPECT_TRUE() +* for more information. +*/ +#define KUNIT_EXPECT_STREQ (test, left, right) \ + KUNIT_BINARY_STR_EQ_ASSERTION(test, KUNIT_EXPECTATION, left, right) + +#define KUNIT_EXPECT_STREQ_MSG (test, left, right, fmt, ...) \ + KUNIT_BINARY_STR_EQ_MSG_ASSERTION(test, \ + KUNIT_EXPECTATION, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +/** +* KUNIT_EXPECT_STRNEQ() - Expects that strings @left and @right are not equal. +* @test: The test context object. +* @left: an arbitrary expression that evaluates to a null terminated string. +* @right: an arbitrary expression that evaluates to a null terminated string. +* +* Sets an expectation that the values that @left and @right evaluate to are +* not equal. This is semantically equivalent to +* KUNIT_EXPECT_TRUE(@test, strcmp((@left), (@right))). See KUNIT_EXPECT_TRUE() +* for more information. +*/ +#define KUNIT_EXPECT_STRNEQ (test, left, right) \ + KUNIT_BINARY_STR_NE_ASSERTION(test, KUNIT_EXPECTATION, left, right) + +#define KUNIT_EXPECT_STRNEQ_MSG (test, left, right, fmt, ...) \ + KUNIT_BINARY_STR_NE_MSG_ASSERTION(test, \ + KUNIT_EXPECTATION, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +/** +* KUNIT_EXPECT_NOT_ERR_OR_NULL() - Expects that @ptr is not null and not err. +* @test: The test context object. +* @ptr: an arbitrary pointer. +* +* Sets an expectation that the value that @ptr evaluates to is not null and not +* an errno stored in a pointer. This is semantically equivalent to +* KUNIT_EXPECT_TRUE(@test, !IS_ERR_OR_NULL(@ptr)). See KUNIT_EXPECT_TRUE() for +* more information. +*/ +#define KUNIT_EXPECT_NOT_ERR_OR_NULL (test, ptr) \ + KUNIT_PTR_NOT_ERR_OR_NULL_ASSERTION(test, KUNIT_EXPECTATION, ptr) + +#define KUNIT_EXPECT_NOT_ERR_OR_NULL_MSG (test, ptr, fmt, ...) \ + KUNIT_PTR_NOT_ERR_OR_NULL_MSG_ASSERTION(test, \ + KUNIT_EXPECTATION, \ + ptr, \ + fmt, \ + ##__VA_ARGS__) +#endif /* _KUNIT_TEST_H */
lib/kunit/test.c +62 -0
68 lines not on this page · raw message
Signature · 1 line
2.23.0.351.gc4317032e6-goog
PATCH v18 6/19 lib: enable building KUnit in lib/
In reply to Brendan HigginsKUnit is a new unit testing framework for the kernel and when used is built into the kernel as a part of it. Add KUnit to the lib Kconfig and Makefile to allow it to be actually built.
--- lib/Kconfig.debug | 2 ++ lib/Makefile | 2 ++ 2 files changed, 4 insertions(+)
lib/Kconfig.debug +2 -0
@@ -1681,6 +1681,8 @@ config PROVIDE_OHCI1394_DMA_INIT See Documentation/debugging-via-ohci1394.txt for more information. +source "lib/kunit/Kconfig" + menuconfig RUNTIME_TESTING_MENU bool "Runtime Testing" def_bool y
lib/Makefile +2 -0
@@ -96,6 +96,8 @@ obj-$(CONFIG_TEST_MEMINIT) += test_meminit.o obj-$(CONFIG_TEST_LIVEPATCH) += livepatch/ +obj-$(CONFIG_KUNIT) += kunit/ + ifeq ($(CONFIG_DEBUG_KOBJECT),y) CFLAGS_kobject.o += -DDEBUG CFLAGS_kobject_uevent.o += -DDEBUG
Signature · 1 line
2.23.0.351.gc4317032e6-goog
Re: [PATCH v18 06/19] lib: enable building KUnit in lib/
Quoting Brendan Higgins (2019-09-23 02:02:36)
Quoted 10 lines
KUnit is a new unit testing framework for the kernel and when used is built into the kernel as a part of it. Add KUnit to the lib
KconfigandMakefileto allow it to be actually built.Signed-off-by Brendan Higgins <[email protected]>Cc Randy Dunlap <[email protected]>Cc Andrew Morton <[email protected]>Cc Masahiro Yamada <[email protected]>Cc Kees Cook <[email protected]>---
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
PATCH v18 7/19 kunit: test: add initial tests
In reply to Brendan HigginsAdd a test for string stream along with a simpler example.
--- lib/kunit/Kconfig | 23 +++++++++ lib/kunit/Makefile | 4 ++ lib/kunit/example-test.c | 88 ++++++++++++++++++++++++++++++++++ lib/kunit/string-stream-test.c | 52 ++++++++++++++++++++ 4 files changed, 167 insertions(+) create mode 100644 lib/kunit/example-test.c create mode 100644 lib/kunit/string-stream-test.c
lib/kunit/Kconfig +23 -0
@@ -11,3 +11,26 @@ menuconfig KUNIT special hardware when using UML. Can also be used on most other architectures. For more information, please see Documentation/dev-tools/kunit/. + +if KUNIT + +config KUNIT_TEST + bool "KUnit test for KUnit" + help + Enables the unit tests for the KUnit test framework. These tests test + the KUnit test framework itself; the tests are both written using + KUnit and test KUnit. This option should only be enabled for testing + purposes by developers interested in testing that KUnit works as + expected. + +config KUNIT_EXAMPLE_TEST + bool "Example test for KUnit" + help + Enables an example unit test that illustrates some of the basic + features of KUnit. This test only exists to help new users understand + what KUnit is and how it is used. Please refer to the example test + itself, lib/kunit/example-test.c, for more information. This option + is intended for curious hackers who would like to understand how to + use KUnit for kernel development. + +endif # KUNIT
lib/kunit/Makefile +4 -0
@@ -1,3 +1,7 @@ obj-$(CONFIG_KUNIT) += test.o \ string-stream.o \ assert.o + +obj-$(CONFIG_KUNIT_TEST) += string-stream-test.o + +obj-$(CONFIG_KUNIT_EXAMPLE_TEST) += example-test.o
lib/kunit/example-test.c new file +88 -0
88 lines not on this page · raw message
lib/kunit/string-stream-test.c new file +52 -0
52 lines not on this page · raw message
Signature · 1 line
2.23.0.351.gc4317032e6-goog
PATCH v18 8/19 objtool: add kunit_try_catch_throw to the noreturn list
In reply to Brendan HigginsFix the following warning seen on GCC 7.3:
kunit/test-test.o: warning: objtool: kunit_test_unsuccessful_try() falls through to next function kunit_test_catch()
kunit_try_catch_throw is a function added in the following patch in this series; it allows KUnit, a unit testing framework for the kernel, to bail out of a broken test. As a consequence, it is a new __noreturn function that objtool thinks is broken (as seen above). So fix this warning by adding kunit_try_catch_throw to objtool's noreturn list.
--- tools/objtool/check.c | 1 + 1 file changed, 1 insertion(+)
tools/objtool/check.c +1 -0
@@ -145,6 +145,7 @@ static bool __dead_end_function(struct objtool_file *file, struct symbol *func,"usercopy_abort" ,"machine_real_restart" ,"rewind_stack_do_exit" , + "kunit_try_catch_throw", };if (!func)
Signature · 1 line
2.23.0.351.gc4317032e6-goog
PATCH v18 9/19 kunit: test: add support for test abort
In reply to Brendan HigginsAdd support for aborting/bailing out of test cases, which is needed for implementing assertions.
An assertion is like an expectation, but bails out of the test case early if the assertion is not met. The idea with assertions is that you use them to state all the preconditions for your test. Logically speaking, these are the premises of the test case, so if a premise isn't true, there is no point in continuing the test case because there are no conclusions that can be drawn without the premises. Whereas, the expectation is the thing you are trying to prove.
--- include/kunit/test.h | 2 + include/kunit/try-catch.h | 75 +++++++++++++++++++++ lib/kunit/Makefile | 3 +- lib/kunit/test.c | 137 +++++++++++++++++++++++++++++++++----- lib/kunit/try-catch.c | 118 ++++++++++++++++++++++++++++++++ 5 files changed, 319 insertions(+), 16 deletions(-) create mode 100644 include/kunit/try-catch.h create mode 100644 lib/kunit/try-catch.c
include/kunit/test.h +2 -0
@@ -10,6 +10,7 @@#define _KUNIT_TEST_H#include <kunit/assert.h> +#include <kunit/try-catch.h> #include <linux/kernel.h> #include <linux/slab.h> #include <linux/types.h>
@@ -172,6 +173,7 @@ struct kunit {/* private: internal use only. */ const char *name;/* Read only after initialization! */ +struct kunit_try_catch try_catch;/* * success starts as true, and may only be set to false during a * test case; thus, it is safe to update this across multiple
include/kunit/try-catch.h new file +75 -0
75 lines not on this page · raw message
lib/kunit/Makefile +2 -1
@@ -1,6 +1,7 @@ obj-$(CONFIG_KUNIT) += test.o \ string-stream.o \ - assert.o + assert.o \ + try-catch.o obj-$(CONFIG_KUNIT_TEST) += string-stream-test.o
lib/kunit/test.c +122 -15
181 lines not on this page · raw message
lib/kunit/try-catch.c new file +118 -0
118 lines not on this page · raw message
Signature · 1 line
2.23.0.351.gc4317032e6-goog
PATCH v18 10/19 kunit: test: add tests for kunit test abort
In reply to Brendan HigginsAdd KUnit tests for the KUnit test abort mechanism (see preceding commit). Add tests both for general try catch mechanism as well as non-architecture specific mechanism.
--- lib/kunit/Makefile | 3 +- lib/kunit/test-test.c | 106 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 108 insertions(+), 1 deletion(-) create mode 100644 lib/kunit/test-test.c
lib/kunit/Makefile +2 -1
@@ -3,6 +3,7 @@ obj-$(CONFIG_KUNIT) += test.o \ assert.o \ try-catch.o -obj-$(CONFIG_KUNIT_TEST) += string-stream-test.o +obj-$(CONFIG_KUNIT_TEST) += test-test.o \ + string-stream-test.o obj-$(CONFIG_KUNIT_EXAMPLE_TEST) += example-test.o
lib/kunit/test-test.c new file +106 -0
106 lines not on this page · raw message
Signature · 1 line
2.23.0.351.gc4317032e6-goog
PATCH v18 11/19 kunit: test: add the concept of assertions
In reply to Brendan HigginsAdd support for assertions which are like expectations except the test terminates if the assertion is not satisfied.
The idea with assertions is that you use them to state all the preconditions for your test. Logically speaking, these are the premises of the test case, so if a premise isn't true, there is no point in continuing the test case because there are no conclusions that can be drawn without the premises. Whereas, the expectation is the thing you are trying to prove. It is not used universally in x-unit style test frameworks, but I really like it as a convention. You could still express the idea of a premise using the above idiom, but I think KUNIT_ASSERT_* states the intended idea perfectly.
--- include/kunit/test.h | 282 ++++++++++++++++++++++++++++++++- lib/kunit/string-stream-test.c | 2 +- lib/kunit/test-test.c | 7 +- 3 files changed, 283 insertions(+), 8 deletions(-)
include/kunit/test.h +280 -2
@@ -87,8 +87,9 @@ struct kunit;* @name: the name of the test case. * * A test case is a function with the signature, -* ``void (*)(struct kunit *)`` that makes expectations (see -* KUNIT_EXPECT_TRUE()) about code under test. Each test case is associated +* ``void (*)(struct kunit *)`` +* that makes expectations and assertions (see KUNIT_EXPECT_TRUE() and +* KUNIT_ASSERT_TRUE()) about code under test. Each test case is associated * with a &struct kunit_suite and will be run after the suite's init * function and followed by the suite's exit function. *
@@ -1210,4 +1211,281 @@ do { \ fmt, \ ##__VA_ARGS__) +#define KUNIT_ASSERT_FAILURE (test, fmt, ...) \ + KUNIT_FAIL_ASSERTION(test, KUNIT_ASSERTION, fmt, ##__VA_ARGS__) + +/** +* KUNIT_ASSERT_TRUE() - Sets an assertion that @condition is true. +* @test: The test context object. +* @condition: an arbitrary boolean expression. The test fails and aborts when +* this does not evaluate to true. +* +* This and assertions of the form `KUNIT_ASSERT_*` will cause the test case to +* fail *and immediately abort* when the specified condition is not met. Unlike +* an expectation failure, it will prevent the test case from continuing to run; +* this is otherwise known as an *assertion failure*. +*/ +#define KUNIT_ASSERT_TRUE (test, condition) \ + KUNIT_TRUE_ASSERTION(test, KUNIT_ASSERTION, condition) + +#define KUNIT_ASSERT_TRUE_MSG (test, condition, fmt, ...) \ + KUNIT_TRUE_MSG_ASSERTION(test, \ + KUNIT_ASSERTION, \ + condition, \ + fmt, \ + ##__VA_ARGS__) + +/** +* KUNIT_ASSERT_FALSE() - Sets an assertion that @condition is false. +* @test: The test context object. +* @condition: an arbitrary boolean expression. +* +* Sets an assertion that the value that @condition evaluates to is false. This +* is the same as KUNIT_EXPECT_FALSE(), except it causes an assertion failure +* (see KUNIT_ASSERT_TRUE()) when the assertion is not met. +*/ +#define KUNIT_ASSERT_FALSE (test, condition) \ + KUNIT_FALSE_ASSERTION(test, KUNIT_ASSERTION, condition) + +#define KUNIT_ASSERT_FALSE_MSG (test, condition, fmt, ...) \ + KUNIT_FALSE_MSG_ASSERTION(test, \ + KUNIT_ASSERTION, \ + condition, \ + fmt, \ + ##__VA_ARGS__) + +/** +* KUNIT_ASSERT_EQ() - Sets an assertion that @left and @right are equal. +* @test: The test context object. +* @left: an arbitrary expression that evaluates to a primitive C type. +* @right: an arbitrary expression that evaluates to a primitive C type. +* +* Sets an assertion that the values that @left and @right evaluate to are +* equal. This is the same as KUNIT_EXPECT_EQ(), except it causes an assertion +* failure (see KUNIT_ASSERT_TRUE()) when the assertion is not met. +*/ +#define KUNIT_ASSERT_EQ (test, left, right) \ + KUNIT_BINARY_EQ_ASSERTION(test, KUNIT_ASSERTION, left, right) + +#define KUNIT_ASSERT_EQ_MSG (test, left, right, fmt, ...) \ + KUNIT_BINARY_EQ_MSG_ASSERTION(test, \ + KUNIT_ASSERTION, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +/** +* KUNIT_ASSERT_PTR_EQ() - Asserts that pointers @left and @right are equal. +* @test: The test context object. +* @left: an arbitrary expression that evaluates to a pointer. +* @right: an arbitrary expression that evaluates to a pointer. +* +* Sets an assertion that the values that @left and @right evaluate to are +* equal. This is the same as KUNIT_EXPECT_EQ(), except it causes an assertion +* failure (see KUNIT_ASSERT_TRUE()) when the assertion is not met. +*/ +#define KUNIT_ASSERT_PTR_EQ (test, left, right) \ + KUNIT_BINARY_PTR_EQ_ASSERTION(test, KUNIT_ASSERTION, left, right) + +#define KUNIT_ASSERT_PTR_EQ_MSG (test, left, right, fmt, ...) \ + KUNIT_BINARY_PTR_EQ_MSG_ASSERTION(test, \ + KUNIT_ASSERTION, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +/** +* KUNIT_ASSERT_NE() - An assertion that @left and @right are not equal. +* @test: The test context object. +* @left: an arbitrary expression that evaluates to a primitive C type. +* @right: an arbitrary expression that evaluates to a primitive C type. +* +* Sets an assertion that the values that @left and @right evaluate to are not +* equal. This is the same as KUNIT_EXPECT_NE(), except it causes an assertion +* failure (see KUNIT_ASSERT_TRUE()) when the assertion is not met. +*/ +#define KUNIT_ASSERT_NE (test, left, right) \ + KUNIT_BINARY_NE_ASSERTION(test, KUNIT_ASSERTION, left, right) + +#define KUNIT_ASSERT_NE_MSG (test, left, right, fmt, ...) \ + KUNIT_BINARY_NE_MSG_ASSERTION(test, \ + KUNIT_ASSERTION, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +/** +* KUNIT_ASSERT_PTR_NE() - Asserts that pointers @left and @right are not equal. +* KUNIT_ASSERT_PTR_EQ() - Asserts that pointers @left and @right are equal. +* @test: The test context object. +* @left: an arbitrary expression that evaluates to a pointer. +* @right: an arbitrary expression that evaluates to a pointer. +* +* Sets an assertion that the values that @left and @right evaluate to are not +* equal. This is the same as KUNIT_EXPECT_NE(), except it causes an assertion +* failure (see KUNIT_ASSERT_TRUE()) when the assertion is not met. +*/ +#define KUNIT_ASSERT_PTR_NE (test, left, right) \ + KUNIT_BINARY_PTR_NE_ASSERTION(test, KUNIT_ASSERTION, left, right) + +#define KUNIT_ASSERT_PTR_NE_MSG (test, left, right, fmt, ...) \ + KUNIT_BINARY_PTR_NE_MSG_ASSERTION(test, \ + KUNIT_ASSERTION, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) +/** +* KUNIT_ASSERT_LT() - An assertion that @left is less than @right. +* @test: The test context object. +* @left: an arbitrary expression that evaluates to a primitive C type. +* @right: an arbitrary expression that evaluates to a primitive C type. +* +* Sets an assertion that the value that @left evaluates to is less than the +* value that @right evaluates to. This is the same as KUNIT_EXPECT_LT(), except +* it causes an assertion failure (see KUNIT_ASSERT_TRUE()) when the assertion +* is not met. +*/ +#define KUNIT_ASSERT_LT (test, left, right) \ + KUNIT_BINARY_LT_ASSERTION(test, KUNIT_ASSERTION, left, right) + +#define KUNIT_ASSERT_LT_MSG (test, left, right, fmt, ...) \ + KUNIT_BINARY_LT_MSG_ASSERTION(test, \ + KUNIT_ASSERTION, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) +/** +* KUNIT_ASSERT_LE() - An assertion that @left is less than or equal to @right. +* @test: The test context object. +* @left: an arbitrary expression that evaluates to a primitive C type. +* @right: an arbitrary expression that evaluates to a primitive C type. +* +* Sets an assertion that the value that @left evaluates to is less than or +* equal to the value that @right evaluates to. This is the same as +* KUNIT_EXPECT_LE(), except it causes an assertion failure (see +* KUNIT_ASSERT_TRUE()) when the assertion is not met. +*/ +#define KUNIT_ASSERT_LE (test, left, right) \ + KUNIT_BINARY_LE_ASSERTION(test, KUNIT_ASSERTION, left, right) + +#define KUNIT_ASSERT_LE_MSG (test, left, right, fmt, ...) \ + KUNIT_BINARY_LE_MSG_ASSERTION(test, \ + KUNIT_ASSERTION, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +/** +* KUNIT_ASSERT_GT() - An assertion that @left is greater than @right. +* @test: The test context object. +* @left: an arbitrary expression that evaluates to a primitive C type. +* @right: an arbitrary expression that evaluates to a primitive C type. +* +* Sets an assertion that the value that @left evaluates to is greater than the +* value that @right evaluates to. This is the same as KUNIT_EXPECT_GT(), except +* it causes an assertion failure (see KUNIT_ASSERT_TRUE()) when the assertion +* is not met. +*/ +#define KUNIT_ASSERT_GT (test, left, right) \ + KUNIT_BINARY_GT_ASSERTION(test, KUNIT_ASSERTION, left, right) + +#define KUNIT_ASSERT_GT_MSG (test, left, right, fmt, ...) \ + KUNIT_BINARY_GT_MSG_ASSERTION(test, \ + KUNIT_ASSERTION, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +/** +* KUNIT_ASSERT_GE() - Assertion that @left is greater than or equal to @right. +* @test: The test context object. +* @left: an arbitrary expression that evaluates to a primitive C type. +* @right: an arbitrary expression that evaluates to a primitive C type. +* +* Sets an assertion that the value that @left evaluates to is greater than the +* value that @right evaluates to. This is the same as KUNIT_EXPECT_GE(), except +* it causes an assertion failure (see KUNIT_ASSERT_TRUE()) when the assertion +* is not met. +*/ +#define KUNIT_ASSERT_GE (test, left, right) \ + KUNIT_BINARY_GE_ASSERTION(test, KUNIT_ASSERTION, left, right) + +#define KUNIT_ASSERT_GE_MSG (test, left, right, fmt, ...) \ + KUNIT_BINARY_GE_MSG_ASSERTION(test, \ + KUNIT_ASSERTION, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +/** +* KUNIT_ASSERT_STREQ() - An assertion that strings @left and @right are equal. +* @test: The test context object. +* @left: an arbitrary expression that evaluates to a null terminated string. +* @right: an arbitrary expression that evaluates to a null terminated string. +* +* Sets an assertion that the values that @left and @right evaluate to are +* equal. This is the same as KUNIT_EXPECT_STREQ(), except it causes an +* assertion failure (see KUNIT_ASSERT_TRUE()) when the assertion is not met. +*/ +#define KUNIT_ASSERT_STREQ (test, left, right) \ + KUNIT_BINARY_STR_EQ_ASSERTION(test, KUNIT_ASSERTION, left, right) + +#define KUNIT_ASSERT_STREQ_MSG (test, left, right, fmt, ...) \ + KUNIT_BINARY_STR_EQ_MSG_ASSERTION(test, \ + KUNIT_ASSERTION, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +/** +* KUNIT_ASSERT_STRNEQ() - Expects that strings @left and @right are not equal. +* @test: The test context object. +* @left: an arbitrary expression that evaluates to a null terminated string. +* @right: an arbitrary expression that evaluates to a null terminated string. +* +* Sets an expectation that the values that @left and @right evaluate to are +* not equal. This is semantically equivalent to +* KUNIT_ASSERT_TRUE(@test, strcmp((@left), (@right))). See KUNIT_ASSERT_TRUE() +* for more information. +*/ +#define KUNIT_ASSERT_STRNEQ (test, left, right) \ + KUNIT_BINARY_STR_NE_ASSERTION(test, KUNIT_ASSERTION, left, right) + +#define KUNIT_ASSERT_STRNEQ_MSG (test, left, right, fmt, ...) \ + KUNIT_BINARY_STR_NE_MSG_ASSERTION(test, \ + KUNIT_ASSERTION, \ + left, \ + right, \ + fmt, \ + ##__VA_ARGS__) + +/** +* KUNIT_ASSERT_NOT_ERR_OR_NULL() - Assertion that @ptr is not null and not err. +* @test: The test context object. +* @ptr: an arbitrary pointer. +* +* Sets an assertion that the value that @ptr evaluates to is not null and not +* an errno stored in a pointer. This is the same as +* KUNIT_EXPECT_NOT_ERR_OR_NULL(), except it causes an assertion failure (see +* KUNIT_ASSERT_TRUE()) when the assertion is not met. +*/ +#define KUNIT_ASSERT_NOT_ERR_OR_NULL (test, ptr) \ + KUNIT_PTR_NOT_ERR_OR_NULL_ASSERTION(test, KUNIT_ASSERTION, ptr) + +#define KUNIT_ASSERT_NOT_ERR_OR_NULL_MSG (test, ptr, fmt, ...) \ + KUNIT_PTR_NOT_ERR_OR_NULL_MSG_ASSERTION(test, \ + KUNIT_ASSERTION, \ + ptr, \ + fmt, \ + ##__VA_ARGS__) +#endif /* _KUNIT_TEST_H */
lib/kunit/string-stream-test.c +1 -1
@@ -35,7 +35,7 @@ static void string_stream_test_get_string(struct kunit *test)string_stream_add (stream," %s" ,"bar" ); output =string_stream_get_string (stream); -KUNIT_EXPECT_STREQ (test, output,"Foo bar" ); +KUNIT_ASSERT_STREQ (test, output,"Foo bar" ); }static struct kunit_case string_stream_test_cases[] = {
lib/kunit/test-test.c +2 -5
@@ -78,16 +78,13 @@ static int kunit_try_catch_test_init(struct kunit *test)struct kunit_try_catch_test_context *ctx; ctx =kunit_kzalloc (test,sizeof (*ctx),GFP_KERNEL ); -if (!ctx) -return -ENOMEM ; - +KUNIT_ASSERT_NOT_ERR_OR_NULL (test, ctx); test->priv = ctx; ctx->try_catch =kunit_kmalloc (test,sizeof (*ctx->try_catch),GFP_KERNEL ); -if (!ctx->try_catch) -return -ENOMEM ; +KUNIT_ASSERT_NOT_ERR_OR_NULL (test, ctx->try_catch);return 0 ; }
Signature · 1 line
2.23.0.351.gc4317032e6-goog
PATCH v18 12/19 kunit: test: add tests for KUnit managed resources
In reply to Brendan HigginsFrom: Avinash Kondareddy <[email protected]>
Add unit tests for KUnit managed resources. KUnit managed resources (struct kunit_resource) are resources that are automatically cleaned up at the end of a KUnit test, similar to the concept of devm_* managed resources.
--- lib/kunit/test-test.c | 228 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 228 insertions(+)
lib/kunit/test-test.c +228 -0
@@ -101,3 +101,231 @@ static struct kunit_suite kunit_try_catch_test_suite = { .test_cases = kunit_try_catch_test_cases, };kunit_test_suite (kunit_try_catch_test_suite); + +/* +* Context for testing test managed resources +* is_resource_initialized is used to test arbitrary resources +*/ +struct kunit_test_resource_context { +struct kunit test; +bool is_resource_initialized; +int allocate_order[2 ]; +int free_order[2 ]; +}; + +static int fake_resource_init (struct kunit_resource *res,void *context) +{ +struct kunit_test_resource_context *ctx = context; + + res->allocation = &ctx->is_resource_initialized; + ctx->is_resource_initialized = true; +return 0 ; +} + +static void fake_resource_free (struct kunit_resource *res) +{ +bool *is_resource_initialized = res->allocation; + + *is_resource_initialized = false; +} + +static void kunit_resource_test_init_resources (struct kunit *test) +{ +struct kunit_test_resource_context *ctx = test->priv; + +kunit_init_test (&ctx->test,"testing_test_init_test" ); + +KUNIT_EXPECT_TRUE (test,list_empty (&ctx->test.resources)); +} + +static void kunit_resource_test_alloc_resource (struct kunit *test) +{ +struct kunit_test_resource_context *ctx = test->priv; +struct kunit_resource *res; +kunit_resource_free_t free = fake_resource_free; + + res =kunit_alloc_and_get_resource (&ctx->test, + fake_resource_init, + fake_resource_free, +GFP_KERNEL , + ctx); + +KUNIT_ASSERT_NOT_ERR_OR_NULL (test, res); +KUNIT_EXPECT_PTR_EQ (test, + &ctx->is_resource_initialized, + (bool *) res->allocation); +KUNIT_EXPECT_TRUE (test,list_is_last (&res->node, &ctx->test.resources)); +KUNIT_EXPECT_PTR_EQ (test, free, res->free); +} + +static void kunit_resource_test_destroy_resource (struct kunit *test) +{ +struct kunit_test_resource_context *ctx = test->priv; +struct kunit_resource *res =kunit_alloc_and_get_resource ( + &ctx->test, + fake_resource_init, + fake_resource_free, +GFP_KERNEL , + ctx); + +KUNIT_ASSERT_FALSE (test, +kunit_resource_destroy (&ctx->test, + kunit_resource_instance_match, + res->free, + res->allocation)); + +KUNIT_EXPECT_FALSE (test, ctx->is_resource_initialized); +KUNIT_EXPECT_TRUE (test,list_empty (&ctx->test.resources)); +} + +static void kunit_resource_test_cleanup_resources (struct kunit *test) +{ +int i; +struct kunit_test_resource_context *ctx = test->priv; +struct kunit_resource *resources[5 ]; + +for (i =0 ; i <ARRAY_SIZE (resources); i++) { + resources[i] =kunit_alloc_and_get_resource (&ctx->test, + fake_resource_init, + fake_resource_free, +GFP_KERNEL , + ctx); + } + +kunit_cleanup (&ctx->test); + +KUNIT_EXPECT_TRUE (test,list_empty (&ctx->test.resources)); +} + +static void kunit_resource_test_mark_order (int order_array[], +size_t order_size, +int key) +{ +int i; + +for (i =0 ; i < order_size && order_array[i]; i++) + ; + + order_array[i] = key; +} + +#define KUNIT_RESOURCE_TEST_MARK_ORDER (ctx, order_field, key) \ + kunit_resource_test_mark_order(ctx->order_field, \ + ARRAY_SIZE(ctx->order_field), \ + key) + +static int fake_resource_2_init (struct kunit_resource *res,void *context) +{ +struct kunit_test_resource_context *ctx = context; + +KUNIT_RESOURCE_TEST_MARK_ORDER (ctx, allocate_order,2 ); + + res->allocation = ctx; + +return 0 ; +} + +static void fake_resource_2_free (struct kunit_resource *res) +{ +struct kunit_test_resource_context *ctx = res->allocation; + +KUNIT_RESOURCE_TEST_MARK_ORDER (ctx, free_order,2 ); +} + +static int fake_resource_1_init (struct kunit_resource *res,void *context) +{ +struct kunit_test_resource_context *ctx = context; + +kunit_alloc_and_get_resource (&ctx->test, + fake_resource_2_init, + fake_resource_2_free, +GFP_KERNEL , + ctx); + +KUNIT_RESOURCE_TEST_MARK_ORDER (ctx, allocate_order,1 ); + + res->allocation = ctx; + +return 0 ; +} + +static void fake_resource_1_free (struct kunit_resource *res) +{ +struct kunit_test_resource_context *ctx = res->allocation; + +KUNIT_RESOURCE_TEST_MARK_ORDER (ctx, free_order,1 ); +} + +/* +* TODO([email protected]): replace the arrays that keep track of the +* order of allocation and freeing with strict mocks using the IN_SEQUENCE macro +* to assert allocation and freeing order when the feature becomes available. +*/ +static void kunit_resource_test_proper_free_ordering (struct kunit *test) +{ +struct kunit_test_resource_context *ctx = test->priv; + +/* fake_resource_1 allocates a fake_resource_2 in its init. */ +kunit_alloc_and_get_resource (&ctx->test, + fake_resource_1_init, + fake_resource_1_free, +GFP_KERNEL , + ctx); + +/* +* Since fake_resource_2_init calls KUNIT_RESOURCE_TEST_MARK_ORDER +* before returning to fake_resource_1_init, it should be the first to +* put its key in the allocate_order array. +*/ +KUNIT_EXPECT_EQ (test, ctx->allocate_order[0 ],2 ); +KUNIT_EXPECT_EQ (test, ctx->allocate_order[1 ],1 ); + +kunit_cleanup (&ctx->test); + +/* +* Because fake_resource_2 finishes allocation before fake_resource_1, +* fake_resource_1 should be freed first since it could depend on +* fake_resource_2. +*/ +KUNIT_EXPECT_EQ (test, ctx->free_order[0 ],1 ); +KUNIT_EXPECT_EQ (test, ctx->free_order[1 ],2 ); +} + +static int kunit_resource_test_init (struct kunit *test) +{ +struct kunit_test_resource_context *ctx = +kzalloc (sizeof (*ctx),GFP_KERNEL ); + +KUNIT_ASSERT_NOT_ERR_OR_NULL (test, ctx); + + test->priv = ctx; + +kunit_init_test (&ctx->test,"test_test_context" ); + +return 0 ; +} + +static void kunit_resource_test_exit (struct kunit *test) +{ +struct kunit_test_resource_context *ctx = test->priv; + +kunit_cleanup (&ctx->test); +kfree (ctx); +} + +static struct kunit_case kunit_resource_test_cases[] = { +KUNIT_CASE (kunit_resource_test_init_resources), +KUNIT_CASE (kunit_resource_test_alloc_resource), +KUNIT_CASE (kunit_resource_test_destroy_resource), +KUNIT_CASE (kunit_resource_test_cleanup_resources), +KUNIT_CASE (kunit_resource_test_proper_free_ordering), + {} +}; + +static struct kunit_suite kunit_resource_test_suite = { + .name ="kunit-resource-test" , + .init = kunit_resource_test_init, + .exit = kunit_resource_test_exit, + .test_cases = kunit_resource_test_cases, +}; +kunit_test_suite (kunit_resource_test_suite);
Signature · 1 line
2.23.0.351.gc4317032e6-goog
PATCH v18 13/19 kunit: tool: add Python wrappers for running KUnit tests
In reply to Brendan HigginsFrom: Felix Guo <[email protected]>
The ultimate goal is to create minimal isolated test binaries; in the meantime we are using UML to provide the infrastructure to run tests, so define an abstract way to configure and run tests that allow us to change the context in which tests are built without affecting the user. This also makes pretty and dynamic error reporting, and a lot of other nice features easier.
kunit_config.py:
- parse .config and Kconfig files.
kunit_kernel.py: provides helper functions to:
- configure the kernel using kunitconfig.
- build the kernel with the appropriate configuration.
- provide function to invoke the kernel and stream the output back.
kunit_parser.py: parses raw logs returned out by kunit_kernel and displays them in a user friendly way.
test_data/*: samples of test data for testing kunit.py, kunit_config.py, etc.
--- tools/testing/kunit/.gitignore | 3 + tools/testing/kunit/kunit.py | 116 +++++++ tools/testing/kunit/kunit_config.py | 66 ++++ tools/testing/kunit/kunit_kernel.py | 148 +++++++++ tools/testing/kunit/kunit_parser.py | 310 ++++++++++++++++++ tools/testing/kunit/kunit_tool_test.py | 206 ++++++++++++ .../test_is_test_passed-all_passed.log | 32 ++ .../test_data/test_is_test_passed-crash.log | 69 ++++ .../test_data/test_is_test_passed-failure.log | 36 ++ .../test_is_test_passed-no_tests_run.log | 75 +++++ .../test_output_isolated_correctly.log | 106 ++++++ .../test_data/test_read_from_file.kconfig | 17 + 12 files changed, 1184 insertions(+) create mode 100644 tools/testing/kunit/.gitignore create mode 100755 tools/testing/kunit/kunit.py create mode 100644 tools/testing/kunit/kunit_config.py create mode 100644 tools/testing/kunit/kunit_kernel.py create mode 100644 tools/testing/kunit/kunit_parser.py create mode 100755 tools/testing/kunit/kunit_tool_test.py create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-all_passed.log create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-crash.log create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-failure.log create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-no_tests_run.log create mode 100644 tools/testing/kunit/test_data/test_output_isolated_correctly.log create mode 100644 tools/testing/kunit/test_data/test_read_from_file.kconfig
tools/testing/kunit/.gitignore new file +3 -0
@@ -0,0 +1,3 @@ +# Byte-compiled / optimized / DLL files +__pycache__/ +*.py[cod]
tools/testing/kunit/kunit.py new file +116 -0
116 lines not on this page · raw message
tools/testing/kunit/kunit_config.py new file +66 -0
66 lines not on this page · raw message
tools/testing/kunit/kunit_kernel.py new file +148 -0
148 lines not on this page · raw message
tools/testing/kunit/kunit_parser.py new file +310 -0
310 lines not on this page · raw message
tools/testing/kunit/kunit_tool_test.py new file +206 -0
206 lines not on this page · raw message
tools/testing/kunit/test_data/test_is_test_passed-all_passed.log new file +32 -0
@@ -0,0 +1,32 @@ +TAP version 14 + # Subtest: sysctl_test + 1..8 + # sysctl_test_dointvec_null_tbl_data: sysctl_test_dointvec_null_tbl_data passed + ok 1 - sysctl_test_dointvec_null_tbl_data + # sysctl_test_dointvec_table_maxlen_unset: sysctl_test_dointvec_table_maxlen_unset passed + ok 2 - sysctl_test_dointvec_table_maxlen_unset + # sysctl_test_dointvec_table_len_is_zero: sysctl_test_dointvec_table_len_is_zero passed + ok 3 - sysctl_test_dointvec_table_len_is_zero + # sysctl_test_dointvec_table_read_but_position_set: sysctl_test_dointvec_table_read_but_position_set passed + ok 4 - sysctl_test_dointvec_table_read_but_position_set + # sysctl_test_dointvec_happy_single_positive: sysctl_test_dointvec_happy_single_positive passed + ok 5 - sysctl_test_dointvec_happy_single_positive + # sysctl_test_dointvec_happy_single_negative: sysctl_test_dointvec_happy_single_negative passed + ok 6 - sysctl_test_dointvec_happy_single_negative + # sysctl_test_dointvec_single_less_int_min: sysctl_test_dointvec_single_less_int_min passed + ok 7 - sysctl_test_dointvec_single_less_int_min + # sysctl_test_dointvec_single_greater_int_max: sysctl_test_dointvec_single_greater_int_max passed + ok 8 - sysctl_test_dointvec_single_greater_int_max +kunit sysctl_test: all tests passed +ok 1 - sysctl_test + # Subtest: example + 1..2 +init_suite + # example_simple_test: initializing + # example_simple_test: example_simple_test passed + ok 1 - example_simple_test + # example_mock_test: initializing + # example_mock_test: example_mock_test passed + ok 2 - example_mock_test +kunit example: all tests passed +ok 2 - example
tools/testing/kunit/test_data/test_is_test_passed-crash.log new file +69 -0
69 lines not on this page · raw message
tools/testing/kunit/test_data/test_is_test_passed-failure.log new file +36 -0
@@ -0,0 +1,36 @@ +TAP version 14 + # Subtest: sysctl_test + 1..8 + # sysctl_test_dointvec_null_tbl_data: sysctl_test_dointvec_null_tbl_data passed + ok 1 - sysctl_test_dointvec_null_tbl_data + # sysctl_test_dointvec_table_maxlen_unset: sysctl_test_dointvec_table_maxlen_unset passed + ok 2 - sysctl_test_dointvec_table_maxlen_unset + # sysctl_test_dointvec_table_len_is_zero: sysctl_test_dointvec_table_len_is_zero passed + ok 3 - sysctl_test_dointvec_table_len_is_zero + # sysctl_test_dointvec_table_read_but_position_set: sysctl_test_dointvec_table_read_but_position_set passed + ok 4 - sysctl_test_dointvec_table_read_but_position_set + # sysctl_test_dointvec_happy_single_positive: sysctl_test_dointvec_happy_single_positive passed + ok 5 - sysctl_test_dointvec_happy_single_positive + # sysctl_test_dointvec_happy_single_negative: sysctl_test_dointvec_happy_single_negative passed + ok 6 - sysctl_test_dointvec_happy_single_negative + # sysctl_test_dointvec_single_less_int_min: sysctl_test_dointvec_single_less_int_min passed + ok 7 - sysctl_test_dointvec_single_less_int_min + # sysctl_test_dointvec_single_greater_int_max: sysctl_test_dointvec_single_greater_int_max passed + ok 8 - sysctl_test_dointvec_single_greater_int_max +kunit sysctl_test: all tests passed +ok 1 - sysctl_test + # Subtest: example + 1..2 +init_suite + # example_simple_test: initializing + # example_simple_test: EXPECTATION FAILED at lib/kunit/example-test.c:30 + Expected 1 + 1 == 3, but + 1 + 1 == 2 + 3 == 3 + # example_simple_test: example_simple_test failed + not ok 1 - example_simple_test + # example_mock_test: initializing + # example_mock_test: example_mock_test passed + ok 2 - example_mock_test +kunit example: one or more tests failed +not ok 2 - example
tools/testing/kunit/test_data/test_is_test_passed-no_tests_run.log new file +75 -0
75 lines not on this page · raw message
tools/testing/kunit/test_data/test_output_isolated_correctly.log new file +106 -0
106 lines not on this page · raw message
tools/testing/kunit/test_data/test_read_from_file.kconfig new file +17 -0
@@ -0,0 +1,17 @@ +# +# Automatically generated file; DO NOT EDIT. +# User Mode Linux/x86 4.12.0-rc3 Kernel Configuration +# +CONFIG_UML=y +CONFIG_MMU=y + +# +# UML-specific options +# + +# +# Host processor type and features +# +# CONFIG_MK8 is not set +CONFIG_TEST=y +CONFIG_EXAMPLE_TEST=y
Signature · 1 line
2.23.0.351.gc4317032e6-goog
PATCH v18 14/19 kunit: defconfig: add defconfigs for building KUnit tests
In reply to Brendan HigginsAdd defconfig for UML and a fragment that can be used to configure other architectures for building KUnit tests. Add option to kunit_tool to use a defconfig to create the kunitconfig.
--- arch/um/configs/kunit_defconfig | 3 +++ tools/testing/kunit/configs/all_tests.config | 3 +++ tools/testing/kunit/kunit.py | 28 +++++++++++++++++--- tools/testing/kunit/kunit_kernel.py | 3 ++- 4 files changed, 32 insertions(+), 5 deletions(-) create mode 100644 arch/um/configs/kunit_defconfig create mode 100644 tools/testing/kunit/configs/all_tests.config
arch/um/configs/kunit_defconfig new file +3 -0
@@ -0,0 +1,3 @@ +CONFIG_KUNIT=y +CONFIG_KUNIT_TEST=y +CONFIG_KUNIT_EXAMPLE_TEST=y
tools/testing/kunit/configs/all_tests.config new file +3 -0
@@ -0,0 +1,3 @@ +CONFIG_KUNIT=y +CONFIG_KUNIT_TEST=y +CONFIG_KUNIT_EXAMPLE_TEST=y
tools/testing/kunit/kunit.py +24 -4
@@ -11,6 +11,7 @@ import argparse import sys import os import time +import shutil from collections import namedtuple from enum import Enum, auto
@@ -21,7 +22,7 @@ import kunit_parser KunitResult = namedtuple('KunitResult', ['status','result']) -KunitRequest = namedtuple('KunitRequest', ['raw_output','timeout', 'jobs', 'build_dir']) +KunitRequest = namedtuple('KunitRequest', ['raw_output','timeout', 'jobs', 'build_dir', 'defconfig']) class KunitStatus(Enum): SUCCESS = auto()
@@ -29,8 +30,16 @@ class KunitStatus(Enum): BUILD_FAILURE = auto() TEST_FAILURE = auto() +def create_default_kunitconfig(): + if not os.path.exists(kunit_kernel.KUNITCONFIG_PATH): + shutil.copyfile('arch/um/configs/kunit_defconfig', + kunit_kernel.KUNITCONFIG_PATH) + def run_tests(linux: kunit_kernel.LinuxSourceTree, request: KunitRequest) -> KunitResult: + if request.defconfig: + create_default_kunitconfig() + config_start = time.time() success = linux.build_reconfig(request.build_dir) config_end = time.time()
@@ -72,7 +81,7 @@ def run_tests(linux: kunit_kernel.LinuxSourceTree, else: return KunitResult(KunitStatus.SUCCESS, test_result) -def main(argv, linux): +def main(argv, linux=None): parser = argparse.ArgumentParser( description='Helps writing and running KUnit tests.') subparser = parser.add_subparsers(dest='subcommand')
@@ -99,13 +108,24 @@ def main(argv, linux): 'directory.', type=str, default=None, metavar='build_dir') + run_parser.add_argument('--defconfig', + help='Uses a default kunitconfig.', + action='store_true') + cli_args = parser.parse_args(argv) if cli_args.subcommand == 'run': + if cli_args.defconfig: + create_default_kunitconfig() + + if not linux: + linux = kunit_kernel.LinuxSourceTree() + request = KunitRequest(cli_args.raw_output, cli_args.timeout, cli_args.jobs, - cli_args.build_dir) + cli_args.build_dir, + cli_args.defconfig) result = run_tests(linux, request) if result.status != KunitStatus.SUCCESS: sys.exit(1)
@@ -113,4 +133,4 @@ def main(argv, linux): parser.print_help() if __name__ == '__main__': - main(sys.argv[1:], kunit_kernel.LinuxSourceTree()) + main(sys.argv[1:])
tools/testing/kunit/kunit_kernel.py +2 -1
@@ -14,6 +14,7 @@ import os import kunit_config KCONFIG_PATH = '.config' +KUNITCONFIG_PATH = 'kunitconfig' class ConfigError(Exception): """Represents an error trying to configure the Linux kernel."""
@@ -81,7 +82,7 @@ class LinuxSourceTree(object): def __init__(self): self._kconfig = kunit_config.Kconfig() - self._kconfig.read_from_file('kunitconfig') + self._kconfig.read_from_file(KUNITCONFIG_PATH) self._ops = LinuxSourceTreeOperations() def clean(self):
Signature · 1 line
2.23.0.351.gc4317032e6-goog
PATCH v18 15/19 Documentation: kunit: add documentation for KUnit
In reply to Brendan HigginsAdd documentation for KUnit, the Linux kernel unit testing framework.
- Add intro and usage guide for KUnit
- Add API reference
--- Documentation/dev-tools/index.rst | 1 + Documentation/dev-tools/kunit/api/index.rst | 16 + Documentation/dev-tools/kunit/api/test.rst | 11 + Documentation/dev-tools/kunit/faq.rst | 62 +++ Documentation/dev-tools/kunit/index.rst | 79 +++ Documentation/dev-tools/kunit/start.rst | 180 ++++++ Documentation/dev-tools/kunit/usage.rst | 576 ++++++++++++++++++++ 7 files changed, 925 insertions(+) create mode 100644 Documentation/dev-tools/kunit/api/index.rst create mode 100644 Documentation/dev-tools/kunit/api/test.rst create mode 100644 Documentation/dev-tools/kunit/faq.rst create mode 100644 Documentation/dev-tools/kunit/index.rst create mode 100644 Documentation/dev-tools/kunit/start.rst create mode 100644 Documentation/dev-tools/kunit/usage.rst
Documentation/dev-tools/index.rst +1 -0
@@ -24,6 +24,7 @@ whole; patches welcome! gdb-kernel-debugging kgdb kselftest + kunit/index .. only:: subproject and html
Documentation/dev-tools/kunit/api/index.rst new file +16 -0
@@ -0,0 +1,16 @@ +.. SPDX-License-Identifier: GPL-2.0 + +============= +API Reference +============= +.. toctree:: + + test + +This section documents the KUnit kernel testing API. It is divided into the +following sections: + +================================= ============================================== +:doc:`test` documents all of the standard testing API + excluding mocking or mocking related features. +================================= ==============================================
Documentation/dev-tools/kunit/api/test.rst new file +11 -0
@@ -0,0 +1,11 @@ +.. SPDX-License-Identifier: GPL-2.0 + +======== +Test API +======== + +This file documents all of the standard testing API excluding mocking or mocking +related features. + +.. kernel-doc:: include/kunit/test.h + :internal:
Documentation/dev-tools/kunit/faq.rst new file +62 -0
62 lines not on this page · raw message
Documentation/dev-tools/kunit/index.rst new file +79 -0
79 lines not on this page · raw message
Documentation/dev-tools/kunit/start.rst new file +180 -0
180 lines not on this page · raw message
Documentation/dev-tools/kunit/usage.rst new file +576 -0
576 lines not on this page · raw message
Signature · 1 line
2.23.0.351.gc4317032e6-goog
Randy Dunlap
reviewerRe: [PATCH v18 15/19] Documentation: kunit: add documentation for KUnit
Quoted 49 lines · Brendan Higgins
Add documentation for KUnit, the Linux kernel unit testing framework.
- Add intro and usage guide for KUnit
- Add API referenceSigned-off-by Felix Guo <[email protected]>Signed-off-by Brendan Higgins <[email protected]>Cc Jonathan Corbet <[email protected]>Reviewed-by Greg Kroah-Hartman <[email protected]>Reviewed-by Stephen Boyd <[email protected]>--- Documentation/dev-tools/index.rst | 1 + Documentation/dev-tools/kunit/api/index.rst | 16 + Documentation/dev-tools/kunit/api/test.rst | 11 + Documentation/dev-tools/kunit/faq.rst | 62 +++ Documentation/dev-tools/kunit/index.rst | 79 +++ Documentation/dev-tools/kunit/start.rst | 180 ++++++ Documentation/dev-tools/kunit/usage.rst | 576 ++++++++++++++++++++ 7 files changed, 925 insertions(+) create mode 100644 Documentation/dev-tools/kunit/api/index.rst create mode 100644 Documentation/dev-tools/kunit/api/test.rst create mode 100644 Documentation/dev-tools/kunit/faq.rst create mode 100644 Documentation/dev-tools/kunit/index.rst create mode 100644 Documentation/dev-tools/kunit/start.rst create mode 100644 Documentation/dev-tools/kunit/usage.rstDocumentation/dev-tools/kunit/start.rst new file +16
-0@@ -0,0 +1,180 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=============== +Getting Started +=============== + +Installing dependencies +======================= +KUnit has the same dependencies as the Linux kernel. As long as you can build +the kernel, you can run KUnit. + +KUnit Wrapper +============= +Included with KUnit is a simple Python wrapper that helps format the output to +easily use and read KUnit output. It handles building and running the kernel, as +well as formatting the output.
Documentation/dev-tools/kunit/start.rst line 26
Quoted 10 lines
Documentation/dev-tools/kunit/start.rst new file +10
-0+ +The wrapper can be run with: + +.. code-block:: bash + + ./tools/testing/kunit/kunit.py run + +Creating a kunitconfig +====================== +The Python script is a thin wrapper around Kbuild as such, it needs to be
around Kbuild. As such,
Quoted 13 lines
Documentation/dev-tools/kunit/start.rst new file +13
-0+configured with a ``kunitconfig`` file. This file essentially contains the +regular Kernel config, with the specific test targets as well. + +.. code-block:: bash + + git clone -b master https://kunit.googlesource.com/kunitconfig $PATH_TO_KUNITCONFIG_REPO + cd $PATH_TO_LINUX_REPO + ln -s $PATH_TO_KUNIT_CONFIG_REPO/kunitconfig kunitconfig + +You may want to add kunitconfig to your local gitignore. + +Verifying KUnit Works +---------------------
Documentation/dev-tools/kunit/start.rst line 49
Quoted 10 lines
Documentation/dev-tools/kunit/start.rst new file +10
-0+ +To make sure that everything is set up correctly, simply invoke the Python +wrapper from your kernel repo: + +.. code-block:: bash + + ./tools/testing/kunit/kunit.py + +.. note:: + You may want to run ``make mrproper`` first.
I normally use O=builddir when building kernels.
Does this support using O=builddir ?
Quoted 17 lines
Documentation/dev-tools/kunit/start.rst new file +17
-0+ +If everything worked correctly, you should see the following: + +.. code-block:: bash + + Generating .config ... + Building KUnit Kernel ... + Starting KUnit Kernel ... + +followed by a list of tests that are run. All of them should be passing. + +.. note:: + Because it is building a lot of sources for the first time, the ``Building + kunit kernel`` step may take a while. + +Writing your first test +=======================
[snip]
Documentation/dev-tools/kunit/usage.rst
Quoted 5 lines
Documentation/dev-tools/kunit/usage.rst new file +0
-0
TBD...
Signature · 1 line
~Randy
Re: [PATCH v18 15/19] Documentation: kunit: add documentation for KUnit
Quoted 55 lines · Randy Dunlap
Quoted 53 lines · Brendan Higgins
Add documentation for KUnit, the Linux kernel unit testing framework.
- Add intro and usage guide for KUnit
- Add API referenceSigned-off-by Felix Guo <[email protected]>Signed-off-by Brendan Higgins <[email protected]>Cc Jonathan Corbet <[email protected]>Reviewed-by Greg Kroah-Hartman <[email protected]>Reviewed-by Logan Gunthorpe <[email protected]>Reviewed-by Stephen Boyd <[email protected]>--- Documentation/dev-tools/index.rst | 1 + Documentation/dev-tools/kunit/api/index.rst | 16 + Documentation/dev-tools/kunit/api/test.rst | 11 + Documentation/dev-tools/kunit/faq.rst | 62 +++ Documentation/dev-tools/kunit/index.rst | 79 +++ Documentation/dev-tools/kunit/start.rst | 180 ++++++ Documentation/dev-tools/kunit/usage.rst | 576 ++++++++++++++++++++ 7 files changed, 925 insertions(+) create mode 100644 Documentation/dev-tools/kunit/api/index.rst create mode 100644 Documentation/dev-tools/kunit/api/test.rst create mode 100644 Documentation/dev-tools/kunit/faq.rst create mode 100644 Documentation/dev-tools/kunit/index.rst create mode 100644 Documentation/dev-tools/kunit/start.rst create mode 100644 Documentation/dev-tools/kunit/usage.rstDocumentation/dev-tools/kunit/start.rst new file +20
-0@@ -0,0 +1,180 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=============== +Getting Started +=============== + +Installing dependencies +======================= +KUnit has the same dependencies as the Linux kernel. As long as you can build +the kernel, you can run KUnit. + +KUnit Wrapper +============= +Included with KUnit is a simple Python wrapper that helps format the output to +easily use and read KUnit output. It handles building and running the kernel, as +well as formatting the output. + +The wrapper can be run with: + +.. code-block:: bash
Quoted 8 lines
Documentation/dev-tools/kunit/start.rst line 26
Quoted 6 lines
Documentation/dev-tools/kunit/start.rst new file +6
-0+ + ./tools/testing/kunit/kunit.py run + +Creating a kunitconfig +====================== +The Python script is a thin wrapper around Kbuild as such, it needs to bearound
Kbuild. As such,
Thanks for pointing this out.
Quoted 19 lines
Quoted 19 lines
Documentation/dev-tools/kunit/start.rst new file +18
-0+configured with a ``kunitconfig`` file. This file essentially contains the +regular Kernel config, with the specific test targets as well. + +.. code-block:: bash + + git clone -b master https://kunit.googlesource.com/kunitconfig $PATH_TO_KUNITCONFIG_REPO + cd $PATH_TO_LINUX_REPO + ln -s $PATH_TO_KUNIT_CONFIG_REPO/kunitconfig kunitconfig + +You may want to add kunitconfig to your local gitignore. + +Verifying KUnit Works +--------------------- + +To make sure that everything is set up correctly, simply invoke the Python +wrapper from your kernel repo: + +.. code-block:: bash
Quoted 8 lines
Documentation/dev-tools/kunit/start.rst line 49
Quoted 5 lines
Documentation/dev-tools/kunit/start.rst new file +5
-0+ + ./tools/testing/kunit/kunit.py + +.. note:: + You may want to run ``make mrproper`` first.I normally use O=builddir when building kernels.
Does this support using O=builddir ?
Yep, it supports specifying a separate build directory.
Quoted 20 lines
Quoted 17 lines
Documentation/dev-tools/kunit/start.rst new file +17
-0+ +If everything worked correctly, you should see the following: + +.. code-block:: bash + + Generating .config ... + Building KUnit Kernel ... + Starting KUnit Kernel ... + +followed by a list of tests that are run. All of them should be passing. + +.. note:: + Because it is building a lot of sources for the first time, the ``Building + kunit kernel`` step may take a while. + +Writing your first test +=======================[snip]
Quoted 7 lines
Documentation/dev-tools/kunit/usage.rst
Quoted 5 lines
Documentation/dev-tools/kunit/usage.rst new file +0
-0TBD...
What did you mean by this comment?
Cheers
Randy Dunlap
reviewerRe: [PATCH v18 15/19] Documentation: kunit: add documentation for KUnit
Quoted 125 lines · Brendan Higgins
Quoted 55 lines · Randy Dunlap
Quoted 53 lines · Brendan Higgins
Add documentation for KUnit, the Linux kernel unit testing framework.
- Add intro and usage guide for KUnit
- Add API referenceSigned-off-by Felix Guo <[email protected]>Signed-off-by Brendan Higgins <[email protected]>Cc Jonathan Corbet <[email protected]>Reviewed-by Greg Kroah-Hartman <[email protected]>Reviewed-by Logan Gunthorpe <[email protected]>Reviewed-by Stephen Boyd <[email protected]>--- Documentation/dev-tools/index.rst | 1 + Documentation/dev-tools/kunit/api/index.rst | 16 + Documentation/dev-tools/kunit/api/test.rst | 11 + Documentation/dev-tools/kunit/faq.rst | 62 +++ Documentation/dev-tools/kunit/index.rst | 79 +++ Documentation/dev-tools/kunit/start.rst | 180 ++++++ Documentation/dev-tools/kunit/usage.rst | 576 ++++++++++++++++++++ 7 files changed, 925 insertions(+) create mode 100644 Documentation/dev-tools/kunit/api/index.rst create mode 100644 Documentation/dev-tools/kunit/api/test.rst create mode 100644 Documentation/dev-tools/kunit/faq.rst create mode 100644 Documentation/dev-tools/kunit/index.rst create mode 100644 Documentation/dev-tools/kunit/start.rst create mode 100644 Documentation/dev-tools/kunit/usage.rstDocumentation/dev-tools/kunit/start.rst new file +20
-0@@ -0,0 +1,180 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=============== +Getting Started +=============== + +Installing dependencies +======================= +KUnit has the same dependencies as the Linux kernel. As long as you can build +the kernel, you can run KUnit. + +KUnit Wrapper +============= +Included with KUnit is a simple Python wrapper that helps format the output to +easily use and read KUnit output. It handles building and running the kernel, as +well as formatting the output. + +The wrapper can be run with: + +.. code-block:: bashQuoted 8 lines
Documentation/dev-tools/kunit/start.rst line 26
Quoted 6 lines
Documentation/dev-tools/kunit/start.rst new file +6
-0+ + ./tools/testing/kunit/kunit.py run + +Creating a kunitconfig +====================== +The Python script is a thin wrapper around Kbuild as such, it needs to bearound
Kbuild. As such,Thanks for pointing this out.
Quoted 19 lines
Quoted 19 lines
Documentation/dev-tools/kunit/start.rst new file +18
-0+configured with a ``kunitconfig`` file. This file essentially contains the +regular Kernel config, with the specific test targets as well. + +.. code-block:: bash + + git clone -b master https://kunit.googlesource.com/kunitconfig $PATH_TO_KUNITCONFIG_REPO + cd $PATH_TO_LINUX_REPO + ln -s $PATH_TO_KUNIT_CONFIG_REPO/kunitconfig kunitconfig + +You may want to add kunitconfig to your local gitignore. + +Verifying KUnit Works +--------------------- + +To make sure that everything is set up correctly, simply invoke the Python +wrapper from your kernel repo: + +.. code-block:: bashQuoted 8 lines
Documentation/dev-tools/kunit/start.rst line 49
Quoted 5 lines
Documentation/dev-tools/kunit/start.rst new file +5
-0+ + ./tools/testing/kunit/kunit.py + +.. note:: + You may want to run ``make mrproper`` first.I normally use O=builddir when building kernels.
Does this support using O=builddir ?Yep, it supports specifying a separate build directory.
Quoted 20 lines
Quoted 17 lines
Documentation/dev-tools/kunit/start.rst new file +17
-0+ +If everything worked correctly, you should see the following: + +.. code-block:: bash + + Generating .config ... + Building KUnit Kernel ... + Starting KUnit Kernel ... + +followed by a list of tests that are run. All of them should be passing. + +.. note:: + Because it is building a lot of sources for the first time, the ``Building + kunit kernel`` step may take a while. + +Writing your first test +=======================[snip]
Quoted 7 lines
Documentation/dev-tools/kunit/usage.rst
Quoted 5 lines
Documentation/dev-tools/kunit/usage.rst new file +0
-0TBD...
Quoted 1 line
What did you mean by this comment?
I plan to review usage.rst soon... (To Be Done :)
Signature · 1 line
~Randy
shuah
maintainerRe: [PATCH v18 15/19] Documentation: kunit: add documentation for KUnit
Quoted 126 lines · Randy Dunlap
Quoted 125 lines · Brendan Higgins
Quoted 55 lines · Randy Dunlap
Quoted 53 lines · Brendan Higgins
Add documentation for KUnit, the Linux kernel unit testing framework.
- Add intro and usage guide for KUnit
- Add API referenceSigned-off-by Felix Guo <[email protected]>Signed-off-by Brendan Higgins <[email protected]>Cc Jonathan Corbet <[email protected]>Reviewed-by Greg Kroah-Hartman <[email protected]>Reviewed-by Stephen Boyd <[email protected]>--- Documentation/dev-tools/index.rst | 1 + Documentation/dev-tools/kunit/api/index.rst | 16 + Documentation/dev-tools/kunit/api/test.rst | 11 + Documentation/dev-tools/kunit/faq.rst | 62 +++ Documentation/dev-tools/kunit/index.rst | 79 +++ Documentation/dev-tools/kunit/start.rst | 180 ++++++ Documentation/dev-tools/kunit/usage.rst | 576 ++++++++++++++++++++ 7 files changed, 925 insertions(+)create mode 100644
Documentation/dev-tools/kunit/api/index.rstcreate mode 100644Documentation/dev-tools/kunit/api/test.rstcreate mode 100644Documentation/dev-tools/kunit/faq.rstcreate mode 100644Documentation/dev-tools/kunit/index.rstcreate mode 100644Documentation/dev-tools/kunit/start.rstcreate mode 100644Documentation/dev-tools/kunit/usage.rstDocumentation/dev-tools/kunit/start.rst new file +20
-0@@ -0,0 +1,180 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=============== +Getting Started +=============== + +Installing dependencies +======================= +KUnit has the same dependencies as the Linux kernel. As long as you can build +the kernel, you can run KUnit. + +KUnit Wrapper +============= +Included with KUnit is a simple Python wrapper that helps format the output to +easily use and read KUnit output. It handles building and running the kernel, as +well as formatting the output. + +The wrapper can be run with: + +.. code-block:: bashQuoted 8 lines
Documentation/dev-tools/kunit/start.rst line 26
Quoted 6 lines
Documentation/dev-tools/kunit/start.rst new file +6
-0+ + ./tools/testing/kunit/kunit.py run + +Creating a kunitconfig +====================== +The Python script is a thin wrapper around Kbuild as such, it needs to bearound
Kbuild. As such,Thanks for pointing this out.
Quoted 19 lines
Quoted 19 lines
Documentation/dev-tools/kunit/start.rst new file +18
-0+configured with a ``kunitconfig`` file. This file essentially contains the +regular Kernel config, with the specific test targets as well. + +.. code-block:: bash + + git clone -b master https://kunit.googlesource.com/kunitconfig $PATH_TO_KUNITCONFIG_REPO + cd $PATH_TO_LINUX_REPO + ln -s $PATH_TO_KUNIT_CONFIG_REPO/kunitconfig kunitconfig + +You may want to add kunitconfig to your local gitignore. + +Verifying KUnit Works +--------------------- + +To make sure that everything is set up correctly, simply invoke the Python +wrapper from your kernel repo: + +.. code-block:: bashQuoted 8 lines
Documentation/dev-tools/kunit/start.rst line 49
Quoted 5 lines
Documentation/dev-tools/kunit/start.rst new file +5
-0+ + ./tools/testing/kunit/kunit.py + +.. note:: + You may want to run ``make mrproper`` first.I normally use O=builddir when building kernels.
Does this support using O=builddir ?Yep, it supports specifying a separate build directory.
Quoted 27 lines
Quoted 17 lines
Documentation/dev-tools/kunit/start.rst new file +17
-0+ +If everything worked correctly, you should see the following: + +.. code-block:: bash + + Generating .config ... + Building KUnit Kernel ... + Starting KUnit Kernel ... + +followed by a list of tests that are run. All of them should be passing. + +.. note:: + Because it is building a lot of sources for the first time, the ``Building + kunit kernel`` step may take a while. + +Writing your first test +=======================[snip]
Documentation/dev-tools/kunit/usage.rst
Quoted 5 lines
Documentation/dev-tools/kunit/usage.rst new file +0
-0TBD...
Quoted 4 lines
Quoted 1 line
What did you mean by this comment?
I plan to review
usage.rstsoon... (To Be Done :)
I would like to apply the series very soon so it gets some soak time after this move in linux-next and it can still make the rc1.
Since there changes can be addressed after rc1, I would like to not require Brendan to do another version before I apply.
Hope you are okay with that Randy!
thanks,
-- Shuah
Randy Dunlap
reviewerRe: [PATCH v18 15/19] Documentation: kunit: add documentation for KUnit
Quoted 10 lines · shuah
I would like to apply the series very soon so it gets some soak time after this move in linux-next and it can still make the rc1.
Since there changes can be addressed after rc1, I would like to not require Brendan to do another version before I apply.
Hope you are okay with that Randy!
thanks,
-- Shuah
Sure, no problem, go for it .
Signature · 1 line
~Randy
Randy Dunlap
reviewerRe: [PATCH v18 15/19] Documentation: kunit: add documentation for KUnit
In reply to Brendan HigginsQuoted 15 lines · Brendan Higgins
Documentation/dev-tools/kunit/usage.rst new file +9
-0@@ -0,0 +1,576 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=========== +Using KUnit +=========== + +The purpose of this document is to describe what KUnit is, how it works, how it +is intended to be used, and all the concepts and terminology that are needed to +understand it. This guide assumes a working knowledge of the Linux kernel and
Documentation/dev-tools/kunit/usage.rst line 19
Quoted 10 lines
Documentation/dev-tools/kunit/usage.rst new file +10
-0+some basic knowledge of testing. + +For a high level introduction to KUnit, including setting up KUnit for your +project, see :doc:`start`. + +Organization of this document +============================= + +This document is organized into two main sections: Testing and Isolating +Behavior. The first covers what a unit test is and how to use KUnit to write
what unit tests are
would agree with the following "them."
Quoted 6 lines
Documentation/dev-tools/kunit/usage.rst new file +6
-0+them. The second covers how to use KUnit to isolate code and make it possible +to unit test code that was otherwise un-unit-testable. + +Testing +======= +
[snip]
Documentation/dev-tools/kunit/usage.rst line 177
Quoted 6 lines
Documentation/dev-tools/kunit/usage.rst new file +6
-0+ +Test Suites +~~~~~~~~~~~ + +Now obviously one unit test isn't very helpful; the power comes from having +many test cases covering all of your behaviors. Consequently it is common to
covering all of a unit's behaviors.
Documentation/dev-tools/kunit/usage.rst line 180
Quoted 3 lines
Documentation/dev-tools/kunit/usage.rst new file +3
-0+have many *similar* tests; in order to reduce duplication in these closely +related tests most unit testing frameworks provide the concept of a *test +suite*, in KUnit we call it a *test suite*; all it is is just a collection of
. This is just a collection of
Documentation/dev-tools/kunit/usage.rst line 182
Quoted 2 lines
Documentation/dev-tools/kunit/usage.rst new file +2
-0+test cases for a unit of code with a set up function that gets invoked before +every test cases and then a tear down function that gets invoked after every
every test case
Quoted 22 lines
Documentation/dev-tools/kunit/usage.rst new file +22
-0+test case completes. + +Example: + +.. code-block:: c + + static struct kunit_case example_test_cases[] = { + KUNIT_CASE(example_test_foo), + KUNIT_CASE(example_test_bar), + KUNIT_CASE(example_test_baz), + {} + }; + + static struct kunit_suite example_test_suite = { + .name = "example", + .init = example_test_init, + .exit = example_test_exit, + .test_cases = example_test_cases, + }; + kunit_test_suite(example_test_suite); + +In the above example the test suite, ``example_test_suite``, would run the test
Documentation/dev-tools/kunit/usage.rst line 214
Quoted 10 lines
Documentation/dev-tools/kunit/usage.rst new file +10
-0+cases ``example_test_foo``, ``example_test_bar``, and ``example_test_baz``, +each would have ``example_test_init`` called immediately before it and would +have ``example_test_exit`` called immediately after it. +``kunit_test_suite(example_test_suite)`` registers the test suite with the +KUnit test framework. + +.. note:: + A test case will only be run if it is associated with a test suite. + +For a more information on these types of things see the :doc:`api/test`.
For more
Quoted 4 lines
Documentation/dev-tools/kunit/usage.rst new file +4
-0+ +Isolating Behavior +================== +
[snip]
Documentation/dev-tools/kunit/usage.rst line 457
Quoted 10 lines
Documentation/dev-tools/kunit/usage.rst new file +10
-0+ +.. _kunit-on-non-uml: + +KUnit on non-UML architectures +============================== + +By default KUnit uses UML as a way to provide dependencies for code under test. +Under most circumstances KUnit's usage of UML should be treated as an +implementation detail of how KUnit works under the hood. Nevertheless, there +are instances where being able to run architecture specific code, or test
I would drop the comma above.
Quoted 6 lines
Documentation/dev-tools/kunit/usage.rst new file +6
-0+against real hardware is desirable. For these reasons KUnit supports running on +other architectures. + +Running existing KUnit tests on non-UML architectures +----------------------------------------------------- +
[snip]
Quoted 9 lines
Documentation/dev-tools/kunit/usage.rst new file +9
-0+Writing new tests for other architectures +----------------------------------------- + +The first thing you must do is ask yourself whether it is necessary to write a +KUnit test for a specific architecture, and then whether it is necessary to +write that test for a particular piece of hardware. In general, writing a test +that depends on having access to a particular piece of hardware or software (not +included in the Linux source repo) should be avoided at all costs. +
Documentation/dev-tools/kunit/usage.rst line 560
Quoted 10 lines
Documentation/dev-tools/kunit/usage.rst new file +10
-0+Even if you only ever plan on running your KUnit test on your hardware +configuration, other people may want to run your tests and may not have access +to your hardware. If you write your test to run on UML, then anyone can run your +tests without knowing anything about your particular setup, and you can still +run your tests on your hardware setup just by compiling for your architecture. + +.. important:: + Always prefer tests that run on UML to tests that only run under a particular + architecture, and always prefer tests that run under QEMU or another easy + (and monitarily free) to obtain software environment to a specific piece of
monetarily
Quoted 16 lines
Documentation/dev-tools/kunit/usage.rst new file +16
-0+ hardware. + +Nevertheless, there are still valid reasons to write an architecture or hardware +specific test: for example, you might want to test some code that really belongs +in ``arch/some-arch/*``. Even so, try your best to write the test so that it +does not depend on physical hardware: if some of your test cases don't need the +hardware, only require the hardware for tests that actually need it. + +Now that you have narrowed down exactly what bits are hardware specific, the +actual procedure for writing and running the tests is pretty much the same as +writing normal KUnit tests. One special caveat is that you have to reset +hardware state in between test cases; if this is not possible, you may only be +able to run one test case per invocation. + +.. TODO([email protected]): Add an actual example of an architecture + dependent KUnit test.
Signature · 1 line
~Randy
PATCH v18 16/19 MAINTAINERS: add entry for KUnit the unit testing framework
In reply to Brendan HigginsAdd myself as maintainer of KUnit, the Linux kernel's unit testing framework.
--- MAINTAINERS | 11 +++++++++++ 1 file changed, 11 insertions(+)
MAINTAINERS +11 -0
@@ -8802,6 +8802,17 @@ S: Maintained F: tools/testing/selftests/ F: Documentation/dev-tools/kselftest* +KERNEL UNIT TESTING FRAMEWORK (KUnit) +M: Brendan Higgins <[email protected]> +L: [email protected] +L: kunit-dev-/[email protected] +W: https://google.github.io/kunit-docs/third_party/kernel/docs/ +S: Maintained +F: Documentation/dev-tools/kunit/ +F: include/kunit/ +F: lib/kunit/ +F: tools/testing/kunit/ + KERNEL USERMODE HELPER M: Luis Chamberlain <[email protected]> L: [email protected]
Signature · 1 line
2.23.0.351.gc4317032e6-goog
PATCH v18 17/19 kernel/sysctl-test: Add null pointer test for sysctl.c:proc_dointvec()
In reply to Brendan HigginsFrom: Iurii Zaikin <[email protected]>
KUnit tests for initialized data behavior of proc_dointvec that is explicitly checked in the code. Includes basic parsing tests including int min/max overflow.
--- kernel/Makefile | 2 + kernel/sysctl-test.c | 392 +++++++++++++++++++++++++++++++++++++++++++ lib/Kconfig.debug | 11 ++ 3 files changed, 405 insertions(+) create mode 100644 kernel/sysctl-test.c
kernel/Makefile +2 -0
@@ -113,6 +113,8 @@ obj-$(CONFIG_TORTURE_TEST) += torture.o obj-$(CONFIG_HAS_IOMEM) += iomem.o obj-$(CONFIG_RSEQ) += rseq.o +obj-$(CONFIG_SYSCTL_KUNIT_TEST) += sysctl-test.o + obj-$(CONFIG_GCC_PLUGIN_STACKLEAK) += stackleak.o KASAN_SANITIZE_stackleak.o := n KCOV_INSTRUMENT_stackleak.o := n
kernel/sysctl-test.c new file +392 -0
392 lines not on this page · raw message
lib/Kconfig.debug +11 -0
@@ -1967,6 +1967,17 @@ config TEST_SYSCTL If unsure, say N. +config SYSCTL_KUNIT_TEST + bool "KUnit test for sysctl" + depends on KUNIT + help + This builds the proc sysctl unit test, which runs on boot. + Tests the API contract and implementation correctness of sysctl. + For more information on KUnit and unit tests in general please refer + to the KUnit documentation in Documentation/dev-tools/kunit/. + + If unsure, say N. + config TEST_UDELAY tristate "udelay test driver" help
Signature · 1 line
2.23.0.351.gc4317032e6-goog
PATCH v18 18/19 MAINTAINERS: add proc sysctl KUnit test to PROC SYSCTL section
In reply to Brendan HigginsAdd entry for the new proc sysctl KUnit test to the PROC SYSCTL section, and add Iurii as a maintainer.
--- MAINTAINERS | 2 ++ 1 file changed, 2 insertions(+)
MAINTAINERS +2 -0
@@ -12983,12 +12983,14 @@ F: Documentation/filesystems/proc.txt PROC SYSCTL M: Luis Chamberlain <[email protected]> M: Kees Cook <[email protected]> +M: Iurii Zaikin <[email protected]> L: [email protected] L: [email protected] S: Maintained F: fs/proc/proc_sysctl.c F: include/linux/sysctl.h F: kernel/sysctl.c +F: kernel/sysctl-test.c F: tools/testing/selftests/sysctl/ PS3 NETWORK SUPPORT
Signature · 1 line
2.23.0.351.gc4317032e6-goog
PATCH v18 19/19 kunit: fix failure to build without printk
In reply to Brendan HigginsPreviously KUnit assumed that printk would always be present, which is not a valid assumption to make. Fix that by removing call to vprintk_emit, and calling printk directly.
This fixes a build error[1] reported by Randy.
For context this change comes after much discussion. My first stab[2] at this was just to make the KUnit logging code compile out; however, it was agreed that if we were going to use vprintk_emit, then vprintk_emit should provide a no-op stub, which lead to my second attempt[3]. In response to me trying to stub out vprintk_emit, Sergey Senozhatsky suggested a way for me to remove our usage of vprintk_emit, which led to my third attempt at solving this[4].
In my third version of this patch[4], I completely removed vprintk_emit, as suggested by Sergey; however, there was a bit of debate over whether Sergey's solution was the best. The debate arose due to Sergey's version resulting in a checkpatch warning, which resulted in a debate over correct printk usage. Joe Perches offered an alternative fix which was somewhat less far reaching than what Sergey had suggested and importantly relied on continuing to use %pV. Much of the debated centered around whether %pV should be widely used, and whether Sergey's version would result in object size bloat. Ultimately, we decided to go with Sergey's version.
Link[1]: https://lore.kernel.org/linux-kselftest/c7229254-0d90-d90e-f3df-5b6d6fc0b51f-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org/
Link[2]: https://lore.kernel.org/linux-kselftest/20190827174932.44177-1-brendanhiggins-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org/
Link[3]: https://lore.kernel.org/linux-kselftest/20190827234835.234473-1-brendanhiggins-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org/
Link[4]: https://lore.kernel.org/linux-kselftest/20190828093143.163302-1-brendanhiggins-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org/
--- include/kunit/test.h | 5 ++-- lib/kunit/test.c | 57 +++++--------------------------------------- 2 files changed, 8 insertions(+), 54 deletions(-)
include/kunit/test.h +2 -3
@@ -339,9 +339,8 @@ static inline void *kunit_kzalloc(struct kunit *test, size_t size, gfp_t gfp)void kunit_cleanup (struct kunit *test); -void __printf(3, 4) kunit_printk(const char *level, - const struct kunit *test, - const char *fmt, ...); +#define kunit_printk (lvl, test, fmt, ...) \ + printk(lvl "\t# %s: " fmt, (test)->name, ##__VA_ARGS__)/** * kunit_info() - Prints an INFO level message associated with @test.
lib/kunit/test.c +6 -51
92 lines not on this page · raw message
Signature · 1 line
2.23.0.351.gc4317032e6-goog
Quoted 6 lines · Brendan Higgins
## TL;DR
This revision addresses comments from Linus[1] and Randy[2], by moving top level
kunit/directory tolib/kunit/and likewise moves top levelKconfigentry underlib/Kconfig.debug, so the KUnit submenu now shows up under the "Kernel Hacking" menu.
This question is primarily directed at Shuah and Linus....
What's the current status of the kunit series now that Brendan has moved it out of the top-level kunit directory as Linus has requested?
There doesn't appear to have been many comments or changes since since September 23rd, and I was very much hoping they could land before -rc2, since I've been hoping to add unit tests for ext4.
Is kunit likely to be able to be landed in Linus's tree during this development cycle?
Many thanks!
- Ted
Quoted 5 lines · Theodore Y. Ts'o
This question is primarily directed at Shuah and Linus....
What's the current status of the kunit series now that Brendan has moved it out of the top-level kunit directory as Linus has requested?
We seemed to decide to just wait for 5.5, but there is nothing that looks to block that. And I encouraged Shuah to find more kunit cases for when it _does_ get merged.
So if the kunit branch is stable, and people want to start using it for their unit tests, then I think that would be a good idea, and then during the 5.5 merge window we'll not just get the infrastructure, we'll get a few more users too and not just examples.
Linus
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
shuah
maintainerQuoted 7 lines · Linus Torvalds
Quoted 5 lines · Theodore Y. Ts'o
This question is primarily directed at Shuah and Linus....
What's the current status of the kunit series now that Brendan has moved it out of the top-level kunit directory as Linus has requested?
The move happened smack in the middle of merge window and landed in linux-next towards the end of the merge window.
Quoted 4 lines
We seemed to decide to just wait for 5.5, but there is nothing that looks to block that. And I encouraged Shuah to find more kunit cases for when it _does_ get merged.
Right. I communicated that to Brendan that we could work on adding more kunit based tests which would help get more mileage on the kunit.
Quoted 5 lines
So if the kunit branch is stable, and people want to start using it for their unit tests, then I think that would be a good idea, and then during the 5.5 merge window we'll not just get the infrastructure, we'll get a few more users too and not just examples.
thanks,
-- Shuah
Quoted 12 lines · shuah
Quoted 7 lines · Linus Torvalds
Quoted 5 lines · Theodore Y. Ts'o
This question is primarily directed at Shuah and Linus....
What's the current status of the kunit series now that Brendan has moved it out of the top-level kunit directory as Linus has requested?
The move happened smack in the middle of merge window and landed in linux-next towards the end of the merge window.
Quoted 12 lines
Quoted 4 lines
We seemed to decide to just wait for 5.5, but there is nothing that looks to block that. And I encouraged Shuah to find more kunit cases for when it _does_ get merged.
Right. I communicated that to Brendan that we could work on adding more kunit based tests which would help get more mileage on the kunit.
Quoted 4 lines
So if the kunit branch is stable, and people want to start using it for their unit tests, then I think that would be a good idea, and then during the 5.5 merge window we'll not just get the infrastructure, we'll get a few more users too and not just examples.
I was planning on holding off on accepting more tests/changes until KUnit is in torvalds/master. As much as I would like to go around promoting it, I don't really want to promote too much complexity in a non-upstream branch before getting it upstream because I don't want to risk adding something that might cause it to get rejected again.
To be clear, I can understand from your perspective why getting more tests/usage before accepting it is a good thing. The more people that play around with it, the more likely that someone will find an issue with it, and more likely that what is accepted into torvalds/master is of high quality.
However, if I encourage arbitrary tests/improvements into my KUnit branch, it further diverges away from torvalds/master, and is more likely that there will be a merge conflict or issue that is not related to the core KUnit changes that will cause the whole thing to be rejected again in v5.5.
I don't know. I guess we could maybe address that situation by splitting up the pull request into features and tests when we go to send it in, but that seems to invite a lot of unnecessary complexity. I actually already had some other tests/changes ready to send for review, but was holding off until the initial set of patches mad it in.
Looking forward to hearing other people's thoughts.
_______________________________________________
dri-devel mailing list [email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
shuah
maintainerQuoted 38 lines · Brendan Higgins
Quoted 12 lines · shuah
Quoted 7 lines · Linus Torvalds
Quoted 5 lines · Theodore Y. Ts'o
This question is primarily directed at Shuah and Linus....
What's the current status of the kunit series now that Brendan has moved it out of the top-level kunit directory as Linus has requested?
The move happened smack in the middle of merge window and landed in linux-next towards the end of the merge window.
Quoted 12 lines
Quoted 4 lines
We seemed to decide to just wait for 5.5, but there is nothing that looks to block that. And I encouraged Shuah to find more kunit cases for when it _does_ get merged.
Right. I communicated that to Brendan that we could work on adding more kunit based tests which would help get more mileage on the kunit.
Quoted 4 lines
So if the kunit branch is stable, and people want to start using it for their unit tests, then I think that would be a good idea, and then during the 5.5 merge window we'll not just get the infrastructure, we'll get a few more users too and not just examples.
I was planning on holding off on accepting more tests/changes until KUnit is in torvalds/master. As much as I would like to go around promoting it, I don't really want to promote too much complexity in a non-upstream branch before getting it upstream because I don't want to risk adding something that might cause it to get rejected again.
To be clear, I can understand from your perspective why getting more tests/usage before accepting it is a good thing. The more people that play around with it, the more likely that someone will find an issue with it, and more likely that what is accepted into torvalds/master is of high quality.
Quoted 6 lines
However, if I encourage arbitrary tests/improvements into my KUnit branch, it further diverges away from torvalds/master, and is more likely that there will be a merge conflict or issue that is not related to the core KUnit changes that will cause the whole thing to be rejected again in v5.5.
The idea is that the new development will happen based on kunit in linux-kselftest next. It will work just fine. As we accepts patches, they will go on top of kunit that is in linux-next now.
thanks,
-- Shuah
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 47 lines · shuah
Quoted 38 lines · Brendan Higgins
Quoted 12 lines · shuah
Quoted 7 lines · Linus Torvalds
Quoted 5 lines · Theodore Y. Ts'o
This question is primarily directed at Shuah and Linus....
What's the current status of the kunit series now that Brendan has moved it out of the top-level kunit directory as Linus has requested?
The move happened smack in the middle of merge window and landed in linux-next towards the end of the merge window.
Quoted 12 lines
Quoted 4 lines
We seemed to decide to just wait for 5.5, but there is nothing that looks to block that. And I encouraged Shuah to find more kunit cases for when it _does_ get merged.
Right. I communicated that to Brendan that we could work on adding more kunit based tests which would help get more mileage on the kunit.
Quoted 4 lines
So if the kunit branch is stable, and people want to start using it for their unit tests, then I think that would be a good idea, and then during the 5.5 merge window we'll not just get the infrastructure, we'll get a few more users too and not just examples.
I was planning on holding off on accepting more tests/changes until KUnit is in torvalds/master. As much as I would like to go around promoting it, I don't really want to promote too much complexity in a non-upstream branch before getting it upstream because I don't want to risk adding something that might cause it to get rejected again.
To be clear, I can understand from your perspective why getting more tests/usage before accepting it is a good thing. The more people that play around with it, the more likely that someone will find an issue with it, and more likely that what is accepted into torvalds/master is of high quality.
Quoted 6 lines
However, if I encourage arbitrary tests/improvements into my KUnit branch, it further diverges away from torvalds/master, and is more likely that there will be a merge conflict or issue that is not related to the core KUnit changes that will cause the whole thing to be rejected again in v5.5.
Quoted 3 lines
The idea is that the new development will happen based on kunit in linux-kselftest next. It will work just fine. As we accepts patches, they will go on top of kunit that is in linux-next now.
But then wouldn't we want to limit what KUnit changes are going into linux-kselftest next for v5.5? For example, we probably don't want to do anymore feature development on it until it is in v5.5, since the goal is to make it more stable, right?
I am guessing that it will probably be fine, but it still sounds like we need to establish some ground rules, and play it *very* safe.
shuah
maintainerQuoted 52 lines · Brendan Higgins
Quoted 47 lines · shuah
Quoted 38 lines · Brendan Higgins
Quoted 12 lines · shuah
Quoted 7 lines · Linus Torvalds
Quoted 5 lines · Theodore Y. Ts'o
This question is primarily directed at Shuah and Linus....
What's the current status of the kunit series now that Brendan has moved it out of the top-level kunit directory as Linus has requested?
The move happened smack in the middle of merge window and landed in linux-next towards the end of the merge window.
Quoted 12 lines
Quoted 4 lines
We seemed to decide to just wait for 5.5, but there is nothing that looks to block that. And I encouraged Shuah to find more kunit cases for when it _does_ get merged.
Right. I communicated that to Brendan that we could work on adding more kunit based tests which would help get more mileage on the kunit.
Quoted 4 lines
So if the kunit branch is stable, and people want to start using it for their unit tests, then I think that would be a good idea, and then during the 5.5 merge window we'll not just get the infrastructure, we'll get a few more users too and not just examples.
I was planning on holding off on accepting more tests/changes until KUnit is in torvalds/master. As much as I would like to go around promoting it, I don't really want to promote too much complexity in a non-upstream branch before getting it upstream because I don't want to risk adding something that might cause it to get rejected again.
To be clear, I can understand from your perspective why getting more tests/usage before accepting it is a good thing. The more people that play around with it, the more likely that someone will find an issue with it, and more likely that what is accepted into torvalds/master is of high quality.
Quoted 6 lines
However, if I encourage arbitrary tests/improvements into my KUnit branch, it further diverges away from torvalds/master, and is more likely that there will be a merge conflict or issue that is not related to the core KUnit changes that will cause the whole thing to be rejected again in v5.5.
Quoted 3 lines
The idea is that the new development will happen based on kunit in linux-kselftest next. It will work just fine. As we accepts patches, they will go on top of kunit that is in linux-next now.
Quoted 8 lines
But then wouldn't we want to limit what KUnit changes are going into linux-kselftest next for v5.5? For example, we probably don't want to do anymore feature development on it until it is in v5.5, since the goal is to make it more stable, right?
I am guessing that it will probably be fine, but it still sounds like we need to establish some ground rules, and play it *very* safe.
How about we identify a small number tests that can add value and focus on them. I am thinking a number between 2 and 5. This way we get a feel for the API, if it changes for the better great, if it doesn't have to, then you know you already did a great job.
Does that sound reasonable to you?
thanks,
-- Shuah
Quoted 9 lines · shuah
Quoted 5 lines
However, if I encourage arbitrary tests/improvements into my KUnit branch, it further diverges away from torvalds/master, and is more likely that there will be a merge conflict or issue that is not related to the core KUnit changes that will cause the whole thing to be rejected again in v5.5.
The idea is that the new development will happen based on kunit in linux-kselftest next. It will work just fine. As we accepts patches, they will go on top of kunit that is in linux-next now.
I don't see how this would work. If I add unit tests to ext4, they would be in fs/ext4. And to the extent that I need to add test mocks to allow the unit tests to work, they will involve changes to existing files in fs/ext4. I can't put them in the ext4.git tree without pulling in the kunit changes into the ext4 git tree. And if they ext4 unit tests land in the kunit tree, it would be a receipe for large numbers of merge conflicts.
- Ted
_______________________________________________
dri-devel mailing list [email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 12 lines · Theodore Y. Ts'o
Quoted 9 lines · shuah
Quoted 5 lines
However, if I encourage arbitrary tests/improvements into my KUnit branch, it further diverges away from torvalds/master, and is more likely that there will be a merge conflict or issue that is not related to the core KUnit changes that will cause the whole thing to be rejected again in v5.5.
The idea is that the new development will happen based on kunit in linux-kselftest next. It will work just fine. As we accepts patches, they will go on top of kunit that is in linux-next now.
Quoted 7 lines
I don't see how this would work. If I add unit tests to ext4, they would be in fs/ext4. And to the extent that I need to add test mocks to allow the unit tests to work, they will involve changes to existing files in fs/ext4. I can't put them in the ext4.git tree without pulling in the kunit changes into the ext4 git tree. And if they ext4 unit tests land in the kunit tree, it would be a receipe for large numbers of merge conflicts.
That's where I was originally coming from.
So here's a dumb idea: what if we merged KUnit through the ext4 tree? I imagine that could potentially get very confusing when we go back to sending changes in through the kselftest tree, but at least it means that ext4 can use it in the meantime, which means that it at least gets to be useful to one group of people. Also, since Ted seems pretty keen on using this, I imagine it is much more likely to produce real world, immediately useful tests not written by me (I'm not being lazy, I just think it is better to get other people's experiences other than my own).
Thoughts?
shuah
maintainerQuoted 23 lines · Brendan Higgins
Quoted 12 lines · Theodore Y. Ts'o
Quoted 9 lines · shuah
Quoted 5 lines
However, if I encourage arbitrary tests/improvements into my KUnit branch, it further diverges away from torvalds/master, and is more likely that there will be a merge conflict or issue that is not related to the core KUnit changes that will cause the whole thing to be rejected again in v5.5.
The idea is that the new development will happen based on kunit in linux-kselftest next. It will work just fine. As we accepts patches, they will go on top of kunit that is in linux-next now.
Quoted 7 lines
I don't see how this would work. If I add unit tests to ext4, they would be in fs/ext4. And to the extent that I need to add test mocks to allow the unit tests to work, they will involve changes to existing files in fs/ext4. I can't put them in the ext4.git tree without pulling in the kunit changes into the ext4 git tree. And if they ext4 unit tests land in the kunit tree, it would be a receipe for large numbers of merge conflicts.
That's where I was originally coming from.
Quoted 10 lines
So here's a dumb idea: what if we merged KUnit through the ext4 tree? I imagine that could potentially get very confusing when we go back to sending changes in through the kselftest tree, but at least it means that ext4 can use it in the meantime, which means that it at least gets to be useful to one group of people. Also, since Ted seems pretty keen on using this, I imagine it is much more likely to produce real world, immediately useful tests not written by me (I'm not being lazy, I just think it is better to get other people's experiences other than my own).
That doesn't make sense does it? The tests might not be limited to fs/ext4. We might have other sub-systems that add tests.
So, we will have to work to make linux-next as the base for new development and limit the number of tests to where it will be easier work in this mode for 5.5. We can stage the pull requests so that kunit lands first followed by tests.
We have a similar situation with kselftest as well. Sub-systems send tests that depend on their tress separately.
thanks,
-- Shuah
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 36 lines · shuah
Quoted 23 lines · Brendan Higgins
Quoted 12 lines · Theodore Y. Ts'o
Quoted 9 lines · shuah
Quoted 5 lines
However, if I encourage arbitrary tests/improvements into my KUnit branch, it further diverges away from torvalds/master, and is more likely that there will be a merge conflict or issue that is not related to the core KUnit changes that will cause the whole thing to be rejected again in v5.5.
The idea is that the new development will happen based on kunit in linux-kselftest next. It will work just fine. As we accepts patches, they will go on top of kunit that is in linux-next now.
Quoted 7 lines
I don't see how this would work. If I add unit tests to ext4, they would be in fs/ext4. And to the extent that I need to add test mocks to allow the unit tests to work, they will involve changes to existing files in fs/ext4. I can't put them in the ext4.git tree without pulling in the kunit changes into the ext4 git tree. And if they ext4 unit tests land in the kunit tree, it would be a receipe for large numbers of merge conflicts.
That's where I was originally coming from.
Quoted 10 lines
So here's a dumb idea: what if we merged KUnit through the ext4 tree? I imagine that could potentially get very confusing when we go back to sending changes in through the kselftest tree, but at least it means that ext4 can use it in the meantime, which means that it at least gets to be useful to one group of people. Also, since Ted seems pretty keen on using this, I imagine it is much more likely to produce real world, immediately useful tests not written by me (I'm not being lazy, I just think it is better to get other people's experiences other than my own).
Quoted 2 lines
That doesn't make sense does it? The tests might not be limited to fs/ext4. We might have other sub-systems that add tests.
Well, I have some small isolated examples that I think would probably work no matter what, so we can get some usage outside of ext4. Also, if we want to limit the number of tests, then we don't expect to get much usage outside of ext4 before v5.5 anyway. I just figure, it's better to make it work for one person than no one, right?
In any case, I admit it is not a great idea. I just thought it had some interesting advantages over going in through linux-kselftest that were worth exploring.
Quoted 4 lines
So, we will have to work to make linux-next as the base for new development and limit the number of tests to where it will be easier work in this mode for 5.5. We can stage the pull requests so that kunit lands first followed by tests.
So we are going to encourage maintainers to allow tests in their tree based on KUnit on the assumption that KUnit will get merged before there changes? That sounds like a huge burden, not just on us, but on other maintainers and users.
I think if we are going to allow tests before KUnit is merged, we should have the tests come in through the same tree as KUnit.
Quoted 2 lines
We have a similar situation with kselftest as well. Sub-systems send tests that depend on their tress separately.
Well it is different if the maintainer wants to send the test in through their tree, right? Otherwise, it won't matter what the state of linux-next is and it won't matter when linux-kselftest gets merged. Or am I not understanding you?
shuah
maintainerQuoted 63 lines · Brendan Higgins
Quoted 36 lines · shuah
Quoted 23 lines · Brendan Higgins
Quoted 12 lines · Theodore Y. Ts'o
Quoted 9 lines · shuah
Quoted 5 lines
However, if I encourage arbitrary tests/improvements into my KUnit branch, it further diverges away from torvalds/master, and is more likely that there will be a merge conflict or issue that is not related to the core KUnit changes that will cause the whole thing to be rejected again in v5.5.
The idea is that the new development will happen based on kunit in linux-kselftest next. It will work just fine. As we accepts patches, they will go on top of kunit that is in linux-next now.
Quoted 7 lines
I don't see how this would work. If I add unit tests to ext4, they would be in fs/ext4. And to the extent that I need to add test mocks to allow the unit tests to work, they will involve changes to existing files in fs/ext4. I can't put them in the ext4.git tree without pulling in the kunit changes into the ext4 git tree. And if they ext4 unit tests land in the kunit tree, it would be a receipe for large numbers of merge conflicts.
That's where I was originally coming from.
Quoted 10 lines
So here's a dumb idea: what if we merged KUnit through the ext4 tree? I imagine that could potentially get very confusing when we go back to sending changes in through the kselftest tree, but at least it means that ext4 can use it in the meantime, which means that it at least gets to be useful to one group of people. Also, since Ted seems pretty keen on using this, I imagine it is much more likely to produce real world, immediately useful tests not written by me (I'm not being lazy, I just think it is better to get other people's experiences other than my own).
Quoted 2 lines
That doesn't make sense does it? The tests might not be limited to fs/ext4. We might have other sub-systems that add tests.
Well, I have some small isolated examples that I think would probably work no matter what, so we can get some usage outside of ext4. Also, if we want to limit the number of tests, then we don't expect to get much usage outside of ext4 before v5.5 anyway. I just figure, it's better to make it work for one person than no one, right?
In any case, I admit it is not a great idea. I just thought it had some interesting advantages over going in through linux-kselftest that were worth exploring.
Quoted 4 lines
So, we will have to work to make linux-next as the base for new development and limit the number of tests to where it will be easier work in this mode for 5.5. We can stage the pull requests so that kunit lands first followed by tests.
So we are going to encourage maintainers to allow tests in their tree based on KUnit on the assumption that KUnit will get merged before there changes? That sounds like a huge burden, not just on us, but on other maintainers and users.
I think if we are going to allow tests before KUnit is merged, we should have the tests come in through the same tree as KUnit.
Quoted 8 lines
Quoted 2 lines
We have a similar situation with kselftest as well. Sub-systems send tests that depend on their tress separately.
Well it is different if the maintainer wants to send the test in through their tree, right? Otherwise, it won't matter what the state of linux-next is and it won't matter when linux-kselftest gets merged. Or am I not understanding you?
Let's talk about current state. Right now kunit is in linux-next and we want to add a few more tests. We will have to coordinate the effort. Once kunit get into mainline, then the need for this coordination goes down.
Let's focus on the next few weeks first so we can get this into mainline in 5.5.
The two of us can chat next week and come up with a plan.
thanks,
-- Shuah
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 74 lines · shuah
Quoted 63 lines · Brendan Higgins
Quoted 36 lines · shuah
Quoted 23 lines · Brendan Higgins
Quoted 12 lines · Theodore Y. Ts'o
Quoted 9 lines · shuah
Quoted 5 lines
However, if I encourage arbitrary tests/improvements into my KUnit branch, it further diverges away from torvalds/master, and is more likely that there will be a merge conflict or issue that is not related to the core KUnit changes that will cause the whole thing to be rejected again in v5.5.
The idea is that the new development will happen based on kunit in linux-kselftest next. It will work just fine. As we accepts patches, they will go on top of kunit that is in linux-next now.
Quoted 7 lines
I don't see how this would work. If I add unit tests to ext4, they would be in fs/ext4. And to the extent that I need to add test mocks to allow the unit tests to work, they will involve changes to existing files in fs/ext4. I can't put them in the ext4.git tree without pulling in the kunit changes into the ext4 git tree. And if they ext4 unit tests land in the kunit tree, it would be a receipe for large numbers of merge conflicts.
That's where I was originally coming from.
Quoted 10 lines
So here's a dumb idea: what if we merged KUnit through the ext4 tree? I imagine that could potentially get very confusing when we go back to sending changes in through the kselftest tree, but at least it means that ext4 can use it in the meantime, which means that it at least gets to be useful to one group of people. Also, since Ted seems pretty keen on using this, I imagine it is much more likely to produce real world, immediately useful tests not written by me (I'm not being lazy, I just think it is better to get other people's experiences other than my own).
Quoted 2 lines
That doesn't make sense does it? The tests might not be limited to fs/ext4. We might have other sub-systems that add tests.
Well, I have some small isolated examples that I think would probably work no matter what, so we can get some usage outside of ext4. Also, if we want to limit the number of tests, then we don't expect to get much usage outside of ext4 before v5.5 anyway. I just figure, it's better to make it work for one person than no one, right?
In any case, I admit it is not a great idea. I just thought it had some interesting advantages over going in through linux-kselftest that were worth exploring.
Quoted 4 lines
So, we will have to work to make linux-next as the base for new development and limit the number of tests to where it will be easier work in this mode for 5.5. We can stage the pull requests so that kunit lands first followed by tests.
So we are going to encourage maintainers to allow tests in their tree based on KUnit on the assumption that KUnit will get merged before there changes? That sounds like a huge burden, not just on us, but on other maintainers and users.
I think if we are going to allow tests before KUnit is merged, we should have the tests come in through the same tree as KUnit.
Quoted 8 lines
Quoted 2 lines
We have a similar situation with kselftest as well. Sub-systems send tests that depend on their tress separately.
Well it is different if the maintainer wants to send the test in through their tree, right? Otherwise, it won't matter what the state of linux-next is and it won't matter when linux-kselftest gets merged. Or am I not understanding you?
Quoted 4 lines
Let's talk about current state. Right now kunit is in linux-next and we want to add a few more tests. We will have to coordinate the effort. Once kunit get into mainline, then the need for this coordination goes down.
Sure, I was just thinking that getting other people to write the tests would be better. Since not only is it then useful to someone else, it provides the best possible exercise of KUnit.
Nevertheless, it would probably just be easier to get a handful of example tests, and it is less likely to result in any issues for v5.5, so that's probably better. (I think that is what you are hinting at here. ;-) )
Hey Ted, do you know if that ext4 timestamp test can go in through linux-kselftest? It seemed fairly self-contained. Or is that what you were saying wouldn't work for you?
Quoted 2 lines
Let's focus on the next few weeks first so we can get this into mainline in 5.5.
I agree. That is the most important thing.
Quoted 1 line
The two of us can chat next week and come up with a plan.
Sure.
Cheers!
Quoted 8 lines · Brendan Higgins
Quoted 4 lines
Let's talk about current state. Right now kunit is in linux-next and we want to add a few more tests. We will have to coordinate the effort. Once kunit get into mainline, then the need for this coordination goes down.
Sure, I was just thinking that getting other people to write the tests would be better. Since not only is it then useful to someone else, it provides the best possible exercise of KUnit.
Well, one thing we *can* do is if (a) if we can create a kselftest branch which we know is stable and won't change, and (b) we can get assurances that Linus *will* accept that branch during the next merge window, those subsystems which want to use kself test can simply pull it into their tree.
We've done this before in the file system world, when there has been some common set of changes needed to improve, say, Direct I/O, where the changes are put into a standalone branch, say, in the xfs tree, and those file systems which need it as a building block can pull it into their tree, and then add the changes needed to use those changes into their file system git tree. These changes are generally not terribly controversial, and we've not had to worry about people want to bikeshed the changes.
There is a risk with doing this of course, which is that if the branch *is* controversial, or gets bike-shedded for some reason, then Linus gets upset and any branches which depended on said branch will get rejected at the next merge window. Which is the requirement for (a) and (b) above. Presumably, the fact that people were unwilling to let Kunit land during this merge window might will *because* we think more changes might be pending?
The other thing I suppose I can do is to let the ext4 kunit tests land in ext4 tree, but with the necessary #ifdef's around things like "#include <kunit/test.h>" so that the build won't blow up w/o kunit changes being in the tree that I'm building. It means I won't be able to run the tests without creating a test integration branch and merging in kunit by hand, which will super-annoying, of course. And if some of the bike-shedding is in Kunit's interfaces, then that becomes problematic as well, since any tests that are in ext4.git tree might change if people want to rename Kunit's publically exported functions (for example).
Quoted 3 lines
Hey Ted, do you know if that ext4 timestamp test can go in through linux-kselftest? It seemed fairly self-contained. Or is that what you were saying wouldn't work for you?
Well, I was hoping that we might start creating more tests beyond just the ext4 timestamp tests....
- Ted
_______________________________________________
dri-devel mailing list [email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 6 lines · Theodore Y. Ts'o
Well, one thing we *can* do is if (a) if we can create a kselftest branch which we know is stable and won't change, and (b) we can get assurances that Linus *will* accept that branch during the next merge window, those subsystems which want to use kself test can simply pull it into their tree.
Yes.
At the same time, I don't think it needs to be even that fancy. Even if it's not a stable branch that gets shared between different developers, it would be good to just have people do a "let's try this" throw-away branch to use the kunit functionality and verify that "yeah, this is fairly convenient for ext4".
It doesn't have to be merged in that form, but just confirmation that the infrastructure is helpful before it gets merged would be good.
Linus
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 11 lines · Linus Torvalds
Quoted 6 lines · Theodore Y. Ts'o
Well, one thing we *can* do is if (a) if we can create a kselftest branch which we know is stable and won't change, and (b) we can get assurances that Linus *will* accept that branch during the next merge window, those subsystems which want to use kself test can simply pull it into their tree.
Yes.
Quoted 8 lines
At the same time, I don't think it needs to be even that fancy. Even if it's not a stable branch that gets shared between different developers, it would be good to just have people do a "let's try this" throw-away branch to use the kunit functionality and verify that "yeah, this is fairly convenient for ext4".
It doesn't have to be merged in that form, but just confirmation that the infrastructure is helpful before it gets merged would be good.
I thought we already had done this satisfactorily.
We have one proof-of-concept test in the branch in the kselftest repo (proc sysctl test) that went out in the pull request, and we also had some other tests that were not in the pull request (there is the ext4 timestamp stuff mentioned above, and we also had one against the list data structure), which we were planning on sending out for review once Shuah's pull request was accepted. I know the apparmor people also wrote some tests that they said were useful; however, I have not coordinated with them on upstreaming their tests. I know of some other people who are using it, but I don't think the tests are as far along for upstreaming.
The point is: I thought we had plenty of signal that KUnit would be useful to have merged into the mainline kernel. I thought the only reason it was rejected for 5.4 was due to the directory name issue combined with bad timing.
Please correct me if I missed anything.
Thanks!
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
shuah
maintainerQuoted 19 lines · Brendan Higgins
Quoted 17 lines · Linus Torvalds
Quoted 6 lines · Theodore Y. Ts'o
Well, one thing we *can* do is if (a) if we can create a kselftest branch which we know is stable and won't change, and (b) we can get assurances that Linus *will* accept that branch during the next merge window, those subsystems which want to use kself test can simply pull it into their tree.
Yes.
At the same time, I don't think it needs to be even that fancy. Even if it's not a stable branch that gets shared between different developers, it would be good to just have people do a "let's try this" throw-away branch to use the kunit functionality and verify that "yeah, this is fairly convenient for ext4".
Quoted 5 lines
Quoted 2 lines
It doesn't have to be merged in that form, but just confirmation that the infrastructure is helpful before it gets merged would be good.
I thought we already had done this satisfactorily.
Adding a couple more tests will only help in the long run. The idea is to see can this help
Quoted 11 lines
We have one proof-of-concept test in the branch in the kselftest repo (proc sysctl test) that went out in the pull request, and we also had some other tests that were not in the pull request (there is the ext4 timestamp stuff mentioned above, and we also had one against the list data structure), which we were planning on sending out for review once Shuah's pull request was accepted. I know the apparmor people also wrote some tests that they said were useful; however, I have not coordinated with them on upstreaming their tests. I know of some other people who are using it, but I don't think the tests are as far along for upstreaming.
Maybe that is a good start. To get the tests that are already in use and get them in shape for upstream.
Quoted 5 lines
The point is: I thought we had plenty of signal that KUnit would be useful to have merged into the mainline kernel. I thought the only reason it was rejected for 5.4 was due to the directory name issue combined with bad timing.
That is probably the initial thought. However, it makes perfect sense to add a couple of tests in. We have a few weeks anyway and it gives us more confidence on kunit.
I already have a branch that is in linux-next and it just has kunit in it and I will rebase it to 5.4-rc1.
https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git/log/?h=test
Let's use that for kunit work for 5.5. I won't add any kselftest patches to it and keep it dedicated for kunit work. When tests are ready for upstream, I can keep adding them to this branch.
thanks,
-- Shuah
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 10 lines · On Sun, 6 Oct 2019 10:18:11 -0700 Linus Torvalds…
Quoted 6 lines · Theodore Y. Ts'o
Well, one thing we *can* do is if (a) if we can create a kselftest branch which we know is stable and won't change, and (b) we can get assurances that Linus *will* accept that branch during the next merge window, those subsystems which want to use kself test can simply pull it into their tree.
Yes.
Quoted 8 lines
At the same time, I don't think it needs to be even that fancy. Even if it's not a stable branch that gets shared between different developers, it would be good to just have people do a "let's try this" throw-away branch to use the kunit functionality and verify that "yeah, this is fairly convenient for ext4".
It doesn't have to be merged in that form, but just confirmation that the infrastructure is helpful before it gets merged would be good.
Can't you just create an ext4 branch that has the kselftest-next branch in it, that you build upon. And push that after the kunit test is merged?
In the past I've had to rely on other branches in next, and would just hold two branches myself. One with everything not dependent on the other developer's branch, and one with the work that was. At the merge window, I would either merge the two or just send two pull requests with the two branches.
-- Steve
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
shuah
maintainerQuoted 26 lines · Steven Rostedt
Quoted 10 lines · On Sun, 6 Oct 2019 10:18:11 -0700 Linus Torvalds…
Quoted 6 lines · Theodore Y. Ts'o
Well, one thing we *can* do is if (a) if we can create a kselftest branch which we know is stable and won't change, and (b) we can get assurances that Linus *will* accept that branch during the next merge window, those subsystems which want to use kself test can simply pull it into their tree.
Yes.
Quoted 8 lines
At the same time, I don't think it needs to be even that fancy. Even if it's not a stable branch that gets shared between different developers, it would be good to just have people do a "let's try this" throw-away branch to use the kunit functionality and verify that "yeah, this is fairly convenient for ext4".
It doesn't have to be merged in that form, but just confirmation that the infrastructure is helpful before it gets merged would be good.
Can't you just create an ext4 branch that has the kselftest-next branch in it, that you build upon. And push that after the kunit test is merged?
Quoted 6 lines
In the past I've had to rely on other branches in next, and would just hold two branches myself. One with everything not dependent on the other developer's branch, and one with the work that was. At the merge window, I would either merge the two or just send two pull requests with the two branches.
I do something similar when I am working on top of a branch that isn't already in the mainline. In any case, repeating myself
Let's work on top of - it is rebased to 5.4-rc1 and ready for use.
https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git/log/?h=test
Let's use that for kunit work for 5.5. I won't add any kselftest patches to it and keep it dedicated for kunit work. When tests are ready for upstream, I can keep adding them to this branch.
thanks,
-- Shuah
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 11 lines · Theodore Y. Ts'o
Quoted 8 lines · Brendan Higgins
Quoted 4 lines
Let's talk about current state. Right now kunit is in linux-next and we want to add a few more tests. We will have to coordinate the effort. Once kunit get into mainline, then the need for this coordination goes down.
Sure, I was just thinking that getting other people to write the tests would be better. Since not only is it then useful to someone else, it provides the best possible exercise of KUnit.
Quoted 5 lines
Well, one thing we *can* do is if (a) if we can create a kselftest branch which we know is stable and won't change, and (b) we can get assurances that Linus *will* accept that branch during the next merge window, those subsystems which want to use kself test can simply pull it into their tree.
Yeah, I can't think of any reason that you haven't outlined already why that might not work, but that seems kind of like circumventing Linus.
Quoted 9 lines
We've done this before in the file system world, when there has been some common set of changes needed to improve, say, Direct I/O, where the changes are put into a standalone branch, say, in the xfs tree, and those file systems which need it as a building block can pull it into their tree, and then add the changes needed to use those changes into their file system git tree. These changes are generally not terribly controversial, and we've not had to worry about people want to bikeshed the changes.
Quoted 7 lines
There is a risk with doing this of course, which is that if the branch *is* controversial, or gets bike-shedded for some reason, then Linus gets upset and any branches which depended on said branch will get rejected at the next merge window. Which is the requirement for (a) and (b) above. Presumably, the fact that people were unwilling to let Kunit land during this merge window might will *because* we think more changes might be pending?
My understanding, based on what I have been told, is that we were simply unlucky with the timing when Linus pulled the branch in the first week of the 5.4 merge window (Friday), such that once I fixed the directory naming issue, the updated changes didn't spend enough time in linux-next. And now with this issue fixed and KUnit back in linux-next, if nothing interesting happens between now and 5.5, it will be accepted in the 5.5 merge window. I do not think that anyone is expecting anymore changes before merging.
Shuah, Linus, is my understanding correct?
Quoted 10 lines
The other thing I suppose I can do is to let the ext4 kunit tests land in ext4 tree, but with the necessary #ifdef's around things like "#include <
kunit/test.h>" so that the build won't blow up w/o kunit changes being in the tree that I'm building. It means I won't be able to run the tests without creating a test integration branch and merging in kunit by hand, which will super-annoying, of course. And if some of the bike-shedding is in Kunit's interfaces, then that becomes problematic as well, since any tests that are in ext4.git tree might change if people want to rename Kunit's publically exported functions (for example).
Yeah, that seems even worse. I'm sorry to have caused this frustration.
Quoted 6 lines
Quoted 3 lines
Hey Ted, do you know if that ext4 timestamp test can go in through linux-kselftest? It seemed fairly self-contained. Or is that what you were saying wouldn't work for you?
Well, I was hoping that we might start creating more tests beyond just the ext4 timestamp tests....
Okay, that's what I thought (and what I hoped) you were saying :-)
I hope we can figure out something that will work for you. Or otherwise that you won't mind waiting until 5.5.
Sorry
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel