Skip to contentSkip to conversation

Documentation: KUnit: make usage.rst a superset of tips.rst, remove duplication

PATCH In review Daniel Latypov 2 participants 5 messages Nov 11, 2022 – Nov 19, 2022

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

PATCH v3 1/3 Documentation: KUnit: make usage.rst a superset of tips.rst, remove duplication

usage.rst had most of the content of the tips.rst page copied over.
But it's missing https://www.kernel.org/doc/html/v6.0/dev-tools/kunit/tips.html#customizing-error-messages
Copy it over so we can retire tips.rst w/o losing content.

And in that process, it also gained a duplicate section about how KUNIT_ASSERT_*() exit the test case early. Remove that.

---
 Documentation/dev-tools/kunit/usage.rst | 49 ++++++++++++++++---------
 1 file changed, 31 insertions(+), 18 deletions(-)
Documentation/dev-tools/kunit/usage.rst +31 -18
index 2737863ef365..b0a6c3bc0eeb 100644
@@ -118,6 +118,37 @@ expectation could crash the test case. `ASSERT_NOT_ERR_OR_NULL(...)` allows us
 to bail out of the test case if the appropriate conditions are not satisfied to
 complete the test.
 
+Customizing error messages
+--------------------------
+
+Each of the ``KUNIT_EXPECT`` and ``KUNIT_ASSERT`` macros have a ``_MSG``
+variant.  These take a format string and arguments to provide additional
+context to the automatically generated error messages.
+
+.. code-block:: c
+
+	char some_str[41];
+	generate_sha1_hex_string(some_str);
+
+	/* Before. Not easy to tell why the test failed. */
+	KUNIT_EXPECT_EQ(test, strlen(some_str), 40);
+
+	/* After. Now we see the offending string. */
+	KUNIT_EXPECT_EQ_MSG(test, strlen(some_str), 40, "some_str='%s'", some_str);
+
+Alternatively, one can take full control over the error message by using
+``KUNIT_FAIL()``, e.g.
+
+.. code-block:: c
+
+	/* Before */
+	KUNIT_EXPECT_EQ(test, some_setup_function(), 0);
+
+	/* After: full control over the failure message. */
+	if (some_setup_function())
+		KUNIT_FAIL(test, "Failed to setup thing for testing");
+
+
 Test Suites
 ~~~~~~~~~~~
 
@@ -546,24 +577,6 @@ By reusing the same ``cases`` array from above, we can write the test as a
 		{}
 	};
 
-Exiting Early on Failed Expectations
-------------------------------------
-
-We can use ``KUNIT_EXPECT_EQ`` to mark the test as failed and continue
-execution.  In some cases, it is unsafe to continue. We can use the
-``KUNIT_ASSERT`` variant to exit on failure.
-
-.. code-block:: c
-
-	void example_test_user_alloc_function(struct kunit *test)
-	{
-		void *object = alloc_some_object_for_me();
-
-		/* Make sure we got a valid pointer back. */
-		KUNIT_ASSERT_NOT_ERR_OR_NULL(test, object);
-		do_something_with_object(object);
-	}
-
 Allocating Memory
 -----------------
 

base-commit: 870f63b7cd78d0055902d839a60408f7428b4e84

Signature · 1 line
2.38.1.431.g37b22c650d-goog
#1 lore raw

PATCH v3 2/3 Documentation: KUnit: reword description of assertions

The existing wording implies that kunit_kmalloc_array() is "the method under test". We're actually testing the sort() function in that example. This is because the example was changed in commit 953574390634 ("Documentation: KUnit: Rework writing page to focus on writing tests"), but the wording was not.

Also add a note telling people they can use the KUNIT_ASSERT_EQ() macros from any function. Some users might be coming from a framework like gUnit where that'll compile but silently do the wrong thing.

---
 Documentation/dev-tools/kunit/usage.rst | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)
Documentation/dev-tools/kunit/usage.rst +8 -5
index b0a6c3bc0eeb..22416ebb94ab 100644
@@ -112,11 +112,14 @@ terminates the test case if the condition is not satisfied. For example:
 			KUNIT_EXPECT_LE(test, a[i], a[i + 1]);
 	}
 
-In this example, the method under test should return pointer to a value. If the
-pointer returns null or an errno, we want to stop the test since the following
-expectation could crash the test case. `ASSERT_NOT_ERR_OR_NULL(...)` allows us
-to bail out of the test case if the appropriate conditions are not satisfied to
-complete the test.
+In this example, we need to be able to allocate an array to test the ``sort()``
+function. So we use ``KUNIT_ASSERT_NOT_ERR_OR_NULL()`` to abort the test if
+there's an allocation error.
+
+.. note::
+   In other test frameworks, ``ASSERT`` macros are often implemented by calling
+   ``return`` so they only work from the test function. In KUnit, we stop the
+   current kthread on failure, so you can call them from anywhere.
 
 Customizing error messages
 --------------------------
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Reviewed-by on v3 2/3
#2 lore raw

Re: [PATCH v3 2/3] Documentation: KUnit: reword description of assertions

Quoted 7 lines · Daniel Latypov

The existing wording implies that kunit_kmalloc_array() is "the method under test". We're actually testing the sort() function in that example. This is because the example was changed in commit 953574390634 ("Documentation: KUnit: Rework writing page to focus on writing tests"), but the wording was not.

Quoted 7 lines

Also add a note telling people they can use the KUNIT_ASSERT_EQ() macros from any function. Some users might be coming from a framework like gUnit where that'll compile but silently do the wrong thing.

---

Looking good!

Cheers,
-- David

Quoted 30 lines
 Documentation/dev-tools/kunit/usage.rst | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)
Documentation/dev-tools/kunit/usage.rst +8 -6
index b0a6c3bc0eeb..22416ebb94ab 100644
@@ -112,11 +112,14 @@ terminates the test case if the condition is not satisfied. For example:
                        KUNIT_EXPECT_LE(test, a[i], a[i + 1]);
        }
 
-In this example, the method under test should return pointer to a value. If the
-pointer returns null or an errno, we want to stop the test since the following
-expectation could crash the test case. `ASSERT_NOT_ERR_OR_NULL(...)` allows us
-to bail out of the test case if the appropriate conditions are not satisfied to
-complete the test.
+In this example, we need to be able to allocate an array to test the ``sort()``
+function. So we use ``KUNIT_ASSERT_NOT_ERR_OR_NULL()`` to abort the test if
+there's an allocation error.
+
+.. note::
+   In other test frameworks, ``ASSERT`` macros are often implemented by calling
+   ``return`` so they only work from the test function. In KUnit, we stop the
+   current kthread on failure, so you can call them from anywhere.
 
 Customizing error messages
 --------------------------
--

2.38.1.431.g37b22c650d-goog

#3 lore raw

PATCH v3 3/3 Documentation: kunit: Remove redundant 'tips.rst' page

In reply to Daniel Latypov

From: David Gow <[email protected]>

The contents of 'tips.rst' was mostly included in 'usage.rst' way back in commit 953574390634 ("Documentation: KUnit: Rework writing page to focus on writing tests"), but the tips page remained behind as well.

The parent patches in this series fill in the gaps, so now 'tips.rst' is redundant.
Therefore, delete 'tips.rst'.

While I regret breaking any links to 'tips' which might exist externally, it's confusing to have two subtly different versions of the same content around.

---
v1 -> v2: rebased onto some parent patches to fix the missing sections in usage.rst and tweaked the commit message to reflect that.

---
 Documentation/dev-tools/kunit/index.rst |   1 -
 Documentation/dev-tools/kunit/tips.rst  | 190 ------------------------
 2 files changed, 191 deletions(-)
 delete mode 100644 Documentation/dev-tools/kunit/tips.rst
Documentation/dev-tools/kunit/index.rst +0 -1
index f5d13f1d37be..d5629817cd72 100644
@@ -16,7 +16,6 @@ KUnit - Linux Kernel Unit Testing
 	api/index
 	style
 	faq
-	tips
 	running_tips
 
 This section details the kernel unit testing framework.
Documentation/dev-tools/kunit/tips.rst deleted +0 -190

190 lines not on this page · raw message

Signature · 1 line
2.38.1.431.g37b22c650d-goog
Reviewed-by on v3 1/3
#4 lore raw
In reply to Daniel Latypov
Quoted 11 lines · Daniel Latypov' via KUnit Development

usage.rst had most of the content of the tips.rst page copied over.
But it's missing https://www.kernel.org/doc/html/v6.0/dev-tools/kunit/tips.html#customizing-error-messages
Copy it over so we can retire tips.rst w/o losing content.

And in that process, it also gained a duplicate section about how KUNIT_ASSERT_*() exit the test case early. Remove that.

---

Looks good to me, thanks!

Cheers,
-- David

Quoted 79 lines
 Documentation/dev-tools/kunit/usage.rst | 49 ++++++++++++++++---------
 1 file changed, 31 insertions(+), 18 deletions(-)
Documentation/dev-tools/kunit/usage.rst +31 -18
index 2737863ef365..b0a6c3bc0eeb 100644
@@ -118,6 +118,37 @@ expectation could crash the test case. `ASSERT_NOT_ERR_OR_NULL(...)` allows us
 to bail out of the test case if the appropriate conditions are not satisfied to
 complete the test.
 
+Customizing error messages
+--------------------------
+
+Each of the ``KUNIT_EXPECT`` and ``KUNIT_ASSERT`` macros have a ``_MSG``
+variant.  These take a format string and arguments to provide additional
+context to the automatically generated error messages.
+
+.. code-block:: c
+
+       char some_str[41];
+       generate_sha1_hex_string(some_str);
+
+       /* Before. Not easy to tell why the test failed. */
+       KUNIT_EXPECT_EQ(test, strlen(some_str), 40);
+
+       /* After. Now we see the offending string. */
+       KUNIT_EXPECT_EQ_MSG(test, strlen(some_str), 40, "some_str='%s'", some_str);
+
+Alternatively, one can take full control over the error message by using
+``KUNIT_FAIL()``, e.g.
+
+.. code-block:: c
+
+       /* Before */
+       KUNIT_EXPECT_EQ(test, some_setup_function(), 0);
+
+       /* After: full control over the failure message. */
+       if (some_setup_function())
+               KUNIT_FAIL(test, "Failed to setup thing for testing");
+
+
 Test Suites
 ~~~~~~~~~~~
 
@@ -546,24 +577,6 @@ By reusing the same ``cases`` array from above, we can write the test as a
                {}
        };
 
-Exiting Early on Failed Expectations
-------------------------------------
-
-We can use ``KUNIT_EXPECT_EQ`` to mark the test as failed and continue
-execution.  In some cases, it is unsafe to continue. We can use the
-``KUNIT_ASSERT`` variant to exit on failure.
-
-.. code-block:: c
-
-       void example_test_user_alloc_function(struct kunit *test)
-       {
-               void *object = alloc_some_object_for_me();
-
-               /* Make sure we got a valid pointer back. */
-               KUNIT_ASSERT_NOT_ERR_OR_NULL(test, object);
-               do_something_with_object(object);
-       }
-
 Allocating Memory
 -----------------
 

base-commit: 870f63b7cd78d0055902d839a60408f7428b4e84 --
2.38.1.431.g37b22c650d-goog

--
You received this message because you are subscribed to the Google Groups "KUnit Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/kunit-dev/20221111182906.1377191-1-dlatypov%40google.com.