Add support for inlined documentation for kunit and kselftests

As an example for the new documentation tool, add a documentation for drm_buddy_test.

I opted to place this on a completely different directory, in order to make easier to test the feature with:

	$ make SPHINXDIRS="tests" htmldocs

---
 Documentation/index.rst                |  2 +-
 Documentation/tests/index.rst          |  6 ++++++
 Documentation/tests/kunit.rst          |  5 +++++
 drivers/gpu/drm/tests/drm_buddy_test.c | 12 ++++++++++++
 4 files changed, 24 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/tests/index.rst
 create mode 100644 Documentation/tests/kunit.rst

index 9dfdc826618c..80a6ce14a61a 100644
@@ -60,7 +60,7 @@ Various other manuals with useful information for all kernel developers.
    fault-injection/index
    livepatch/index
    rust/index
-
+   test/index
 
 User-oriented documentation
 ===========================

new file mode 100644
index 000000000000..bfc39eb5c0aa
@@ -0,0 +1,6 @@
+========================
+Kunit documentation test
+========================
+
+.. toctree::
+   kunit

new file mode 100644
index 000000000000..6ffc151988a0
@@ -0,0 +1,5 @@
+Kunit tests
+-----------
+
+.. include-test:: drivers/gpu/drm/tests/drm_buddy_test.c
+

index 09ee6f6af896..dd6c5afd6cd6 100644
@@ -737,6 +737,18 @@ static int drm_buddy_suite_init(struct kunit_suite *suite)
 	return 0;
 }
 
+/**
+ * KTEST_SUITE: set of tests for drm buddy alloc
+ * Scope: drm subsystem
+ * Mega feature: drm
+ * Feature: buddy_alloc
+ *
+ * KTEST_TEST: drm_test_buddy_alloc_%s
+ * Description: Run DRM buddy allocation %arg[1] test
+ *
+ * arg[1].values: limit, range, optimistic, smoke, pathological
+ */
+
 static struct kunit_case drm_buddy_tests[] = {
 	KUNIT_CASE(drm_test_buddy_alloc_limit),
 	KUNIT_CASE(drm_test_buddy_alloc_range),

2.41.0

This is a follow-up of the discussions taken here:

https://lore.kernel.org/linux-doc/20230704132812.02ba97ba@maurocar-mobl2/T/#t

I sent a previous version as RFC. This is basically what we had there, with some improvements at test_list.py.

It adds a new extension that allows documenting tests using the same tool we're using for DRM unit tests at IGT GPU tools: https://gitlab.freedesktop.org/drm/igt-gpu-tools.

While kernel-doc has provided documentation for in-lined functions/struct comments, it was not meant to document tests.

Tests need to be grouped by the test functions. It should also be possible to produce other outputs from the documentation, to integrate it with test suites. For instance, Internally at Intel, we use the comments to generate DOT files hierarchically grouped per feature categories.

This is meant to be an initial series to start documenting kunit.

It comes with a documenation for a kunit suite from drm at drm_buddy_test.c.

My plan is that, once we get this merged, start documenting other unit tests and work to add extra requirements at the tool needed by Linux Kernel.

On this series, we have:

- patch 1:
adds test_list.py - which is identical to the file with the same name I developed for IGT tree;
- patch 2:
adds test documentation from a single DRM kunit file (drm_buddy_test.c).

Mauro Carvalho Chehab (2):
docs: add support for documenting kUnit and kSelftests
drm: add documentation for drm_buddy_test kUnit test

 Documentation/conf.py                  |    2 +-
 Documentation/index.rst                |    2 +-
 Documentation/sphinx/test_kdoc.py      |  108 ++
 Documentation/sphinx/test_list.py      | 1314 ++++++++++++++++++++++++
 Documentation/tests/index.rst          |    6 +
 Documentation/tests/kunit.rst          |    5 +
 drivers/gpu/drm/tests/drm_buddy_test.c |   12 +
 7 files changed, 1447 insertions(+), 2 deletions(-)
 create mode 100755 Documentation/sphinx/test_kdoc.py
 create mode 100644 Documentation/sphinx/test_list.py
 create mode 100644 Documentation/tests/index.rst
 create mode 100644 Documentation/tests/kunit.rst

2.41.0

Add an extension at Sphinx to allow documenting tests. Such extension uses a class developed for me to document GPU tests as part of the IGT test suite: https://gitlab.freedesktop.org/drm/igt-gpu-tools,
meaning that sphinx/test_list.py file is identical to the IGT tree file named scripts/test_list.py.

The idea is to keep the test_list.py in sync on both trees, while adding any kernel-specific logic inside test_kdoc.py.

---
 Documentation/conf.py             |    2 +-
 Documentation/sphinx/test_kdoc.py |  108 +++
 Documentation/sphinx/test_list.py | 1314 +++++++++++++++++++++++++++++
 3 files changed, 1423 insertions(+), 1 deletion(-)
 create mode 100755 Documentation/sphinx/test_kdoc.py
 create mode 100644 Documentation/sphinx/test_list.py

index d4fdf6a3875a..cec017a6cab5 100644
@@ -55,7 +55,7 @@ needs_sphinx = '1.7'
 extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include',
               'kfigure', 'sphinx.ext.ifconfig', 'automarkup',
               'maintainers_include', 'sphinx.ext.autosectionlabel',
-              'kernel_abi', 'kernel_feat']
+              'kernel_abi', 'kernel_feat', 'test_kdoc']
 
 if major >= 3:
     if (major > 3) or (minor > 0 or patch >= 2):

2.41.0

I've played with this a bit...a couple of quick impressions:

- That's quite a chunk of Python code to be adding. I've not yet had the chance to read it through properly, will hopefully be able to do so soon. A bit more commenting would not have gone amiss here...

- I kind of think that this should go under dev-tools rather than being a new top-level directory. Is there a reason not to put it there?

Thanks,

jon

One other little thing...

Em Tue, 03 Oct 2023 11:00:20 -0600
Jonathan Corbet <[email protected]> escreveu:

I'll try to add more comments when respin this series.

I guess I should also add a documentation similar to the one I wrote for IGT [1]:
https://gitlab.freedesktop.org/drm/igt-gpu-tools/-/blob/master/docs/test_documentation.md?ref_type=heads#documenting-tests-via-testplan

[1] this document is specific for the way IGT uses it; I'll write something similar to it considering the names we've agreed for KUnit.

No particular reason. I'll change it to be under dev-tools/tests at the next submission.

Should I wait for you to take a look at patch 1/2 before sending a new version?

Regards,
Mauro

Go ahead and resend whenever...I'm still digging out from the last few weeks.

Thanks,

jon