Add support for inlined documentation for kunit and kselftests
Subsystems (5)
Mauro Carvalho Chehab
maintainercover Add support for inlined documentation for kunit and kselftests
This RFC is a follow-up of the discussions taken here:
https://lore.kernel.org/linux-doc/20230704132812.02ba97ba@maurocar-mobl2/T/#t
It adds a new extension that allows documenting tests using the same tool we're using for DRM unit tests at IGT GPU tools: https://gitlab.freedesktop.org/drm/igt-gpu-tools.
While kernel-doc has provided documentation for in-lined functions/struct comments, it was not meant to document tests.
Tests need to be grouped by the test functions. It should also be possible to produce other outputs from the documentation, to integrate it with test suites. For instance, Internally at Intel, we use the comments to generate DOT files hierarchically grouped per feature categories.
This is just an initial RFC to start discussions around the solution. Before being merged upstream, we need to define what tags will be used to identify test markups and add a simple change at kernel-doc to let it ignore such markups.
On this series, we have:
- patch 1:
adding test_list.py as present at the IGT tree, after a patch series to make it more generic: https://patchwork.freedesktop.org/series/120622/
- patch 2:
adds an example about how tests could be documented. This is a really simple example, just to test the feature, specially designed to make easy to build just the test documentation from a single DRM kunit file.
After discussions, my plan is to send a new version addressing the issues, and add some documentation for DRM and/or i915 kunit tests.
Mauro Carvalho Chehab (2):
docs: add support for documenting kUnit and kSelftests
drm: add documentation for drm_buddy_test kUnit test
Documentation/conf.py | 2 +- Documentation/index.rst | 2 +- Documentation/sphinx/test_kdoc.py | 108 ++ Documentation/sphinx/test_list.py | 1288 ++++++++++++++++++++++++ Documentation/tests/index.rst | 6 + Documentation/tests/kunit.rst | 5 + drivers/gpu/drm/tests/drm_buddy_test.c | 12 + 7 files changed, 1421 insertions(+), 2 deletions(-) create mode 100644 Documentation/sphinx/test_kdoc.py create mode 100644 Documentation/sphinx/test_list.py create mode 100644 Documentation/tests/index.rst create mode 100644 Documentation/tests/kunit.rst
Signature · 1 line
2.40.1
Mauro Carvalho Chehab
maintainerRFC PATCH 1/2 docs: add support for documenting kUnit and kSelftests
Add an extension at Sphinx to allow documenting tests. Such extension uses a class developed for me to document GPU tests as part of the IGT test suite: https://gitlab.freedesktop.org/drm/igt-gpu-tools
The idea is to keep the test_list.py in sync on both trees, while adding any kernel-specific logic inside test_kdoc.py.
---
To avoid mailbombing on a large number of people, only mailing lists were C/C on the cover. See [PATCH RFC 0/2] at: https://lore.kernel.org/all/[email protected]/
Documentation/conf.py | 2 +- Documentation/sphinx/test_kdoc.py | 108 +++ Documentation/sphinx/test_list.py | 1288 +++++++++++++++++++++++++++++ 3 files changed, 1397 insertions(+), 1 deletion(-) create mode 100644 Documentation/sphinx/test_kdoc.py create mode 100644 Documentation/sphinx/test_list.py
Documentation/conf.py +1 -1
@@ -55,7 +55,7 @@ needs_sphinx = '1.7' extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'kfigure', 'sphinx.ext.ifconfig', 'automarkup', 'maintainers_include', 'sphinx.ext.autosectionlabel', - 'kernel_abi', 'kernel_feat'] + 'kernel_abi', 'kernel_feat', 'test_kdoc'] if major >= 3: if (major > 3) or (minor > 0 or patch >= 2):
Documentation/sphinx/test_kdoc.py new file +108 -0
108 lines not on this page · raw message
Documentation/sphinx/test_list.py new file +1288 -0
1,288 lines not on this page · raw message
Signature · 1 line
2.40.1
Mauro Carvalho Chehab
maintainerRFC PATCH 2/2 drm: add documentation for drm_buddy_test kUnit test
In reply to Mauro Carvalho ChehabAs an example for the new documentation tool, add a documentation for drm_buddy_test.
I opted to place this on a completely different directory, in order to make easier to test the feature with:
$ make SPHINXDIRS="tests" htmldocs
---
To avoid mailbombing on a large number of people, only mailing lists were C/C on the cover. See [PATCH RFC 0/2] at: https://lore.kernel.org/all/[email protected]/
Documentation/index.rst | 2 +- Documentation/tests/index.rst | 6 ++++++ Documentation/tests/kunit.rst | 5 +++++ drivers/gpu/drm/tests/drm_buddy_test.c | 12 ++++++++++++ 4 files changed, 24 insertions(+), 1 deletion(-) create mode 100644 Documentation/tests/index.rst create mode 100644 Documentation/tests/kunit.rst
Documentation/index.rst +1 -1
@@ -60,7 +60,7 @@ Various other manuals with useful information for all kernel developers. fault-injection/index livepatch/index rust/index - + test/index User-oriented documentation ===========================
Documentation/tests/index.rst new file +6 -0
@@ -0,0 +1,6 @@ +======================== +Kunit documentation test +======================== + +.. toctree:: + kunit
Documentation/tests/kunit.rst new file +5 -0
@@ -0,0 +1,5 @@ +Kunit tests +----------- + +.. include-test:: drivers/gpu/drm/tests/drm_buddy_test.c +
drivers/gpu/drm/tests/drm_buddy_test.c +12 -0
@@ -737,6 +737,18 @@ static int drm_buddy_suite_init(struct kunit_suite *suite)return 0 ; } +/** +* KTEST_SUITE: set of tests for drm buddy alloc +* Scope: drm subsystem +* Mega feature: drm +* Feature: buddy_alloc +* +* KTEST_TEST: drm_test_buddy_alloc_%s +* Description: Run DRM buddy allocation %arg[1] test +* +* arg[1].values: limit, range, optimistic, smoke, pathological +*/ +static struct kunit_case drm_buddy_tests[] = {KUNIT_CASE (drm_test_buddy_alloc_limit),KUNIT_CASE (drm_test_buddy_alloc_range),
Signature · 1 line
2.40.1
Re: [PATCH RFC 2/2] drm: add documentation for drm_buddy_test kUnit test
Quoted 10 lines · Mauro Carvalho Chehab
drivers/gpu/drm/tests/drm_buddy_test.c +2
-0@@ -737,6 +737,18 @@ static int drm_buddy_suite_init(struct kunit_suite *suite)return 0 ; } +/** +* KTEST_SUITE: set of tests for drm buddy alloc
drivers/gpu/drm/tests/drm_buddy_test.c static int drm_buddy_suite_init(struct kunit_suite *suite) · line 751
Quoted 10 lines
drivers/gpu/drm/tests/drm_buddy_test.c +10
-0+* Scope: drm subsystem +* Mega feature: drm +* Feature: buddy_alloc +* +* KTEST_TEST: drm_test_buddy_alloc_%s +* Description: Run DRM buddy allocation %arg[1] test +* +* arg[1].values: limit, range, optimistic, smoke, pathological +*/ +
"/**" indicates a kernel-doc comment, and this is not a kernel-doc comment.
$ scripts/kernel-doc -none drivers/gpu/drm/tests/drm_buddy_test.c drivers/gpu/drm/tests/drm_buddy_test.c:752: warning: cannot understand function prototype: 'struct kunit_case drm_buddy_tests[] = '
Nowadays kernel-doc is part of W=1 builds.
BR,
Jani.
Quoted 3 lines
drivers/gpu/drm/tests/drm_buddy_test.c +0
-0static struct kunit_case drm_buddy_tests[] = {KUNIT_CASE (drm_test_buddy_alloc_limit),KUNIT_CASE (drm_test_buddy_alloc_range),
Signature · 1 line
Jani Nikula, Intel Open Source Graphics Center
Mauro Carvalho Chehab
maintainerRe: [PATCH RFC 2/2] drm: add documentation for drm_buddy_test kUnit test
Em Wed, 12 Jul 2023 18:03:00 +0300
Jani Nikula <[email protected]> escreveu:
Quoted 25 lines · Mauro Carvalho Chehab
Quoted 10 lines
drivers/gpu/drm/tests/drm_buddy_test.c +2
-0@@ -737,6 +737,18 @@ static int drm_buddy_suite_init(struct kunit_suite *suite)return 0 ; } +/** +* KTEST_SUITE: set of tests for drm buddy alloc drivers/gpu/drm/tests/drm_buddy_test.c static int drm_buddy_suite_init(struct kunit_suite *suite) · line 751
Quoted 10 lines
drivers/gpu/drm/tests/drm_buddy_test.c +10
-0+* Scope: drm subsystem +* Mega feature: drm +* Feature: buddy_alloc +* +* KTEST_TEST: drm_test_buddy_alloc_%s +* Description: Run DRM buddy allocation %arg[1] test +* +* arg[1].values: limit, range, optimistic, smoke, pathological +*/ +"/**" indicates a kernel-doc comment, and this is not a kernel-doc comment.
Quoted 5 lines
$ scripts/kernel-doc -none drivers/gpu/drm/tests/drm_buddy_test.c drivers/gpu/drm/tests/drm_buddy_test.c:752: warning: cannot understand function prototype: 'struct kunit_case drm_buddy_tests[] = 'Nowadays kernel-doc is part of W=1 builds.
True. I already told it at patch 0. I opted to not add a patch for it on this RFC series, to make it simpler. A simple logic at kernel-doc is enough to tell its state machine to ignore blocks that contain the KTEST_\w+: pattern:
scripts/kernel-doc +5 -0
@@ -259,6 +259,7 @@ my $doc_sect = $doc_com . '\s*(\@[.\w]+|\@\.\.\.|description|context|returns?|notes?|examples?)\s*:([^:].*)?$'; my $doc_content = $doc_com_body . '(.*)'; my $doc_block = $doc_com . 'DOC:\s*(.*)?'; +my $ktest_block = $doc_com . 'KTEST_\w+:\s*(.*)?'; my $doc_inline_start = '^\s*/\*\*\s*$'; my $doc_inline_sect = '\s*\*\s*(@\s*[\w][\w\.]*\s*):(.*)'; my $doc_inline_end = '^\s*\*/\s*$';
@@ -2015,6 +2016,10 @@ sub process_name($$) { my $file = shift; my $descr; + if (/$ktest_block/o) { + $state = STATE_NORMAL; + return; + } if (/$doc_block/o) { $state = STATE_DOCBLOCK; $contents = "";
Thanks,
Mauro
Re: [PATCH RFC 2/2] drm: add documentation for drm_buddy_test kUnit test
In reply to Mauro Carvalho Chehab[Adding Arun]
Am 12.07.23 um 16:28 schrieb Mauro Carvalho Chehab:
Quoted 9 lines
As an example for the new documentation tool, add a documentation for
drm_buddy_test.I opted to place this on a completely different directory, in order to make easier to test the feature with:
$ make SPHINXDIRS="tests" htmldocsSigned-off-by Mauro Carvalho Chehab <[email protected]>
Arun please take a look as well.
Thanks,
Christian.
Quoted 72 lines
---
To avoid mailbombing on a large number of people, only mailing lists were C/C on the cover. See [PATCH RFC 0/2] at: https://lore.kernel.org/all/[email protected]/
Documentation/index.rst | 2 +- Documentation/tests/index.rst | 6 ++++++ Documentation/tests/kunit.rst | 5 +++++ drivers/gpu/drm/tests/drm_buddy_test.c | 12 ++++++++++++ 4 files changed, 24 insertions(+), 1 deletion(-)create mode 100644
Documentation/tests/index.rstcreate mode 100644Documentation/tests/kunit.rstDocumentation/index.rst +1
-1@@ -60,7 +60,7 @@ Various other manuals with useful information for all kernel developers. fault-injection/index livepatch/index rust/index - + test/index User-oriented documentation ===========================Documentation/tests/index.rst new file +6
-0@@ -0,0 +1,6 @@ +======================== +Kunit documentation test +======================== + +.. toctree:: + kunitDocumentation/tests/kunit.rst new file +5
-0@@ -0,0 +1,5 @@ +Kunit tests +----------- + +.. include-test:: drivers/gpu/drm/tests/drm_buddy_test.c +drivers/gpu/drm/tests/drm_buddy_test.c +12
-0@@ -737,6 +737,18 @@ static int drm_buddy_suite_init(struct kunit_suite *suite)return 0 ; } +/** +* KTEST_SUITE: set of tests for drm buddy alloc +* Scope: drm subsystem +* Mega feature: drm +* Feature: buddy_alloc +* +* KTEST_TEST: drm_test_buddy_alloc_%s +* Description: Run DRM buddy allocation %arg[1] test +* +* arg[1].values: limit, range, optimistic, smoke, pathological +*/ +static struct kunit_case drm_buddy_tests[] = {KUNIT_CASE (drm_test_buddy_alloc_limit),KUNIT_CASE (drm_test_buddy_alloc_range),
Re: [PATCH RFC 2/2] drm: add documentation for drm_buddy_test kUnit test
Looks good, Feel free to add Reviewed-by: Arunpravin Paneer Selvam <[email protected]>
Regards,
Arun.
Quoted 93 lines · Christian König
[Adding Arun]
Am 12.07.23 um 16:28 schrieb Mauro Carvalho Chehab:
Quoted 9 lines
As an example for the new documentation tool, add a documentation for
drm_buddy_test.I opted to place this on a completely different directory, in order to make easier to test the feature with:
$ make SPHINXDIRS="tests" htmldocsSigned-off-by Mauro Carvalho Chehab <[email protected]>Acked-by Christian König <[email protected]>Arun please take a look as well.
Thanks,
Christian.Quoted 72 lines
---
To avoid mailbombing on a large number of people, only mailing lists were C/C on the cover. See [PATCH RFC 0/2] at: https://lore.kernel.org/all/[email protected]/
Documentation/index.rst | 2 +- Documentation/tests/index.rst | 6 ++++++ Documentation/tests/kunit.rst | 5 +++++
drivers/gpu/drm/tests/drm_buddy_test.c| 12 ++++++++++++4 files changed, 24 insertions(+), 1 deletion(-)create mode 100644
Documentation/tests/index.rstcreate mode 100644Documentation/tests/kunit.rstDocumentation/index.rst +0
-0@@ -60,7 +60,7 @@ Various other manuals with useful information forall kernel developers.
fault-injection/index
livepatch/index
rust/index
-
+ test/index
User-oriented documentation
===========================Documentation/tests/index.rst new file +6
-0@@ -0,0 +1,6 @@ +======================== +Kunit documentation test +======================== + +.. toctree:: + kunitDocumentation/tests/kunit.rst new file +5
-0@@ -0,0 +1,5 @@ +Kunit tests +----------- + +.. include-test:: drivers/gpu/drm/tests/drm_buddy_test.c +drivers/gpu/drm/tests/drm_buddy_test.c +0
-0@@ -737,6 +737,18 @@ static int drm_buddy_suite_init(struct kunit_suite *suite) return 0; }+/**
+ *KTEST_SUITE: set of tests for drm buddy alloc
+ * Scope: drm subsystem
+ * Mega feature: drm
+ * Feature:buddy_alloc
+ *
+ *KTEST_TEST: drm_test_buddy_alloc_%s
+ * Description: Run DRM buddy allocation %arg[1] test
+ *
+ * arg[1].values: limit, range, optimistic, smoke, pathological
+ */
+
static structkunit_casedrm_buddy_tests[] = {KUNIT_CASE(drm_test_buddy_alloc_limit),KUNIT_CASE(drm_test_buddy_alloc_range),
Re: [PATCH RFC 2/2] drm: add documentation for drm_buddy_test kUnit test
In reply to Mauro Carvalho ChehabQuoted 72 lines · Mauro Carvalho Chehab
As an example for the new documentation tool, add a documentation for
drm_buddy_test.I opted to place this on a completely different directory, in order to make easier to test the feature with:
$ make SPHINXDIRS="tests" htmldocsSigned-off-by Mauro Carvalho Chehab <[email protected]>---
To avoid mailbombing on a large number of people, only mailing lists were C/C on the cover.
See [PATCH RFC 0/2] at:
https://lore.kernel.org/all/[email protected]/Documentation/index.rst | 2 +- Documentation/tests/index.rst | 6 ++++++ Documentation/tests/kunit.rst | 5 +++++ drivers/gpu/drm/tests/drm_buddy_test.c | 12 ++++++++++++ 4 files changed, 24 insertions(+), 1 deletion(-) create mode 100644 Documentation/tests/index.rst create mode 100644 Documentation/tests/kunit.rstDocumentation/index.rst +0
-0@@ -60,7 +60,7 @@ Various other manuals with useful information for allkernel developers.
fault-injection/index
livepatch/index
rust/indexDocumentation/index.rst +1
-1- + test/index User-oriented documentation ===========================Documentation/tests/index.rst new file +6
-0@@ -0,0 +1,6 @@ +======================== +Kunit documentation test +======================== + +.. toctree:: + kunitDocumentation/tests/kunit.rst new file +5
-0@@ -0,0 +1,5 @@ +Kunit tests +----------- + +.. include-test:: drivers/gpu/drm/tests/drm_buddy_test.c +drivers/gpu/drm/tests/drm_buddy_test.c +1
-0@@ -737,6 +737,18 @@ static int drm_buddy_suite_init(struct kunit_suite *suite)return 0 ; } +/**
drivers/gpu/drm/tests/drm_buddy_test.c static int drm_buddy_suite_init(struct kunit_suite · line 751
Quoted 10 lines
drivers/gpu/drm/tests/drm_buddy_test.c +10
-0+* KTEST_SUITE: set of tests for drm buddy alloc +* Scope: drm subsystem +* Mega feature: drm +* Feature: buddy_alloc +* +* KTEST_TEST: drm_test_buddy_alloc_%s +* Description: Run DRM buddy allocation %arg[1] test +* +* arg[1].values: limit, range, optimistic, smoke, pathological +*/
Hi!
This is such a cool patch series. I just have a few comments related to the output.
In the html output the tests are listed as:
ktest@drm_buddy_test@…
I wonder if instead of using the file name of “drm_buddy_test” this could possibly be the suite name, “drm_buddy”, as this is what users will call when using kunit.py to run the tests. Although "drm_buddy_test" is also the module name so I don't mind it too much. But in the future the file name and module name are not guaranteed to be the same for other tests.
Most preferably, there would be a reference to the kunit suite name, file name, and the module name.
This may be difficult to implement as these can all differ. I am currently working on the KUnit Attribute framework which saves the module name and I am thinking about also saving the file path as a future attribute. This could be a helpful framework for the KUnit tests specifically.
I am not sure how easy it would be to access c objects/functions using this system.
Finally, I was wondering if it is the intention to put a list of all kunit tests that use this new feature into tests/kunit.rst or would this be broken up in some way
Thanks!
-Rae
Quoted 8 lines
drivers/gpu/drm/tests/drm_buddy_test.c +1
-0+static struct kunit_case drm_buddy_tests[] = {KUNIT_CASE (drm_test_buddy_alloc_limit),
KUNIT_CASE(drm_test_buddy_alloc_range),
--
2.40.1
Mauro Carvalho Chehab
maintainerRe: [PATCH RFC 2/2] drm: add documentation for drm_buddy_test kUnit test
Hi Rae,
Em Thu, 13 Jul 2023 17:31:19 -0400
Rae Moar <[email protected]> escreveu:
Quoted 89 lines
Quoted 72 lines · Mauro Carvalho Chehab
As an example for the new documentation tool, add a documentation for
drm_buddy_test.I opted to place this on a completely different directory, in order to make easier to test the feature with:
$ make SPHINXDIRS="tests" htmldocsSigned-off-by Mauro Carvalho Chehab <[email protected]>---
To avoid mailbombing on a large number of people, only mailing lists were C/C on the cover.
See [PATCH RFC 0/2] at:
https://lore.kernel.org/all/[email protected]/Documentation/index.rst | 2 +- Documentation/tests/index.rst | 6 ++++++ Documentation/tests/kunit.rst | 5 +++++ drivers/gpu/drm/tests/drm_buddy_test.c | 12 ++++++++++++ 4 files changed, 24 insertions(+), 1 deletion(-) create mode 100644 Documentation/tests/index.rst create mode 100644 Documentation/tests/kunit.rstDocumentation/index.rst +0
-0@@ -60,7 +60,7 @@ Various other manuals with useful information for allkernel developers.
fault-injection/index
livepatch/index
rust/indexDocumentation/index.rst +1
-1- + test/index User-oriented documentation ===========================Documentation/tests/index.rst new file +6
-0@@ -0,0 +1,6 @@ +======================== +Kunit documentation test +======================== + +.. toctree:: + kunitDocumentation/tests/kunit.rst new file +5
-0@@ -0,0 +1,5 @@ +Kunit tests +----------- + +.. include-test:: drivers/gpu/drm/tests/drm_buddy_test.c +drivers/gpu/drm/tests/drm_buddy_test.c +1
-0@@ -737,6 +737,18 @@ static int drm_buddy_suite_init(struct kunit_suite *suite)return 0 ; } +/** drivers/gpu/drm/tests/drm_buddy_test.c static int drm_buddy_suite_init(struct kunit_suite · line 751
Quoted 10 lines
drivers/gpu/drm/tests/drm_buddy_test.c +10
-0+* KTEST_SUITE: set of tests for drm buddy alloc +* Scope: drm subsystem +* Mega feature: drm +* Feature: buddy_alloc +* +* KTEST_TEST: drm_test_buddy_alloc_%s +* Description: Run DRM buddy allocation %arg[1] test +* +* arg[1].values: limit, range, optimistic, smoke, pathological +*/ Hi!
Quoted 2 lines
This is such a cool patch series. I just have a few comments related to the output.
Thank you for your comments! Sorry for not answering earlier. I took some vacations and this series ended sleeping over other tasks on my todo list.
Also, before sending another version, I wanted to have the test_list.py changes to make it generic enough to be merged on IGT, to avoid having a fork of it. Those got merged today.
Quoted 11 lines
In the html output the tests are listed as:
ktest@drm_buddy_test@…I wonder if instead of using the file name of “drm_buddy_test” this could possibly be the suite name, “drm_buddy”, as this is what users will call when using
kunit.pyto run the tests. Although "drm_buddy_test" is also the module name so I don't mind it too much. But in the future the file name and module name are not guaranteed to be the same for other tests.Most preferably, there would be a reference to the kunit suite name, file name, and the module name.
I guess it shouldn't be hard to do such change in a way that it won't affect its usage on IGT. We need to define what would be the best pattern. As this can be used for both kunit and selftests, I would place kunit at the beginning.
Currently, we're using:
kunit@<base file name without .c>@<test_name>
Some possible patterns would be:
kunit@<base file name without .c>@<suite name>@<test_name> kunit@<subsystem>@<base file name without .c>@<suite name>@<test_name> kunit@<subsystem>@<suite name>@<test_name>
Would do you think it would work best?
Quoted 7 lines
This may be difficult to implement as these can all differ. I am currently working on the KUnit Attribute framework which saves the module name and I am thinking about also saving the file path as a future attribute. This could be a helpful framework for the KUnit tests specifically.
I am not sure how easy it would be to access c objects/functions using this system.
I would prefer not. C language allows lots of flexibility with macros, making it hard to write a parser to read those C objects from the source. We have this at kernel-doc. As one of the people that did some work there, I can say that that several tricks are needed to keep this working.
On the other hand, it should be easy to use the TestList class from test_list.py at kunit.py.
So, kunit.py could use the data that came from the documentation directly.
Quoted 3 lines
Finally, I was wondering if it is the intention to put a list of all kunit tests that use this new feature into
tests/kunit.rstor would this be broken up in some way
IMO, it makes sense to break this per subsystem, and have an auto-generated index.rst pointing to the entire set of documents.
We're already storing the subsystem at the documentation macros, so, IMO, it should shouldn't be hard to implement it.
Regards,
Mauro
Re: [PATCH RFC 2/2] drm: add documentation for drm_buddy_test kUnit test
Quoted 98 lines · Mauro Carvalho Chehab
Hi Rae,
Em Thu, 13 Jul 2023 17:31:19 -0400
Rae Moar <[email protected]> escreveu:Quoted 89 lines
Quoted 72 lines · Mauro Carvalho Chehab
As an example for the new documentation tool, add a documentation for
drm_buddy_test.I opted to place this on a completely different directory, in order to make easier to test the feature with:
$ make SPHINXDIRS="tests" htmldocsSigned-off-by Mauro Carvalho Chehab <[email protected]>---
To avoid mailbombing on a large number of people, only mailing lists were C/C on the cover.
See [PATCH RFC 0/2] at:
https://lore.kernel.org/all/[email protected]/Documentation/index.rst | 2 +- Documentation/tests/index.rst | 6 ++++++ Documentation/tests/kunit.rst | 5 +++++ drivers/gpu/drm/tests/drm_buddy_test.c | 12 ++++++++++++ 4 files changed, 24 insertions(+), 1 deletion(-) create mode 100644 Documentation/tests/index.rst create mode 100644 Documentation/tests/kunit.rstDocumentation/index.rst +0
-0@@ -60,7 +60,7 @@ Various other manuals with useful information for allkernel developers.
fault-injection/index
livepatch/index
rust/indexDocumentation/index.rst +1
-1- + test/index User-oriented documentation ===========================Documentation/tests/index.rst new file +6
-0@@ -0,0 +1,6 @@ +======================== +Kunit documentation test +======================== + +.. toctree:: + kunitDocumentation/tests/kunit.rst new file +5
-0@@ -0,0 +1,5 @@ +Kunit tests +----------- + +.. include-test:: drivers/gpu/drm/tests/drm_buddy_test.c +drivers/gpu/drm/tests/drm_buddy_test.c +1
-0@@ -737,6 +737,18 @@ static int drm_buddy_suite_init(struct kunit_suite *suite)return 0 ; } +/** drivers/gpu/drm/tests/drm_buddy_test.c static int drm_buddy_suite_init(struct kunit_suite · line 751
Quoted 10 lines
drivers/gpu/drm/tests/drm_buddy_test.c +10
-0+* KTEST_SUITE: set of tests for drm buddy alloc +* Scope: drm subsystem +* Mega feature: drm +* Feature: buddy_alloc +* +* KTEST_TEST: drm_test_buddy_alloc_%s +* Description: Run DRM buddy allocation %arg[1] test +* +* arg[1].values: limit, range, optimistic, smoke, pathological +*/ Hi!
Quoted 2 lines
This is such a cool patch series. I just have a few comments related to the output.
Quoted 7 lines
Thank you for your comments! Sorry for not answering earlier. I took some vacations and this series ended sleeping over other tasks on my todo list.
Also, before sending another version, I wanted to have the
test_list.pychanges to make it generic enough to be merged on IGT, to avoid having a fork of it. Those got merged today.
Hi Mauro!
No worries at all!
Quoted 22 lines
Quoted 4 lines
In the html output the tests are listed as:
ktest@drm_buddy_test@…Quoted 8 lines
I wonder if instead of using the file name of “drm_buddy_test” this could possibly be the suite name, “drm_buddy”, as this is what users will call when using
kunit.pyto run the tests. Although "drm_buddy_test" is also the module name so I don't mind it too much. But in the future the file name and module name are not guaranteed to be the same for other tests.Most preferably, there would be a reference to the kunit suite name, file name, and the module name.
I guess it shouldn't be hard to do such change in a way that it won't affect its usage on IGT. We need to define what would be the best pattern. As this can be used for both kunit and selftests, I would place kunit at the beginning.
Currently, we're using:
kunit@<base file name without .c>@<test_name>
Quoted 7 lines
Some possible patterns would be:
kunit@<base file name without .c>@<suite name>@<test_name> kunit@<subsystem>@<base file name without .c>@<suite name>@<test_name> kunit@<subsystem>@<suite name>@<test_name>
Would do you think it would work best?
How possible is it to separate out the file and suite name as headers? I think that this could reduce some of the repetition.
If we are already separating documentation pages by subsystem, a potential format could be:
File: <base file>
<kunit_or_kselftest> suite: <suite name>
List of Tests:
<test name>
<test name>
...
What do you think?
Quoted 14 lines
Quoted 8 lines
This may be difficult to implement as these can all differ. I am currently working on the KUnit Attribute framework which saves the module name and I am thinking about also saving the file path as a future attribute. This could be a helpful framework for the KUnit tests specifically.
I am not sure how easy it would be to access c objects/functions using this system.
I would prefer not. C language allows lots of flexibility with macros, making it hard to write a parser to read those C objects from the source. We have this at kernel-doc. As one of the people that did some work there, I can say that that several tricks are needed to keep this working.
Quoted 6 lines
On the other hand, it should be easy to use the TestList class from
test_list.pyatkunit.py.So,
kunit.pycould use the data that came from the documentation directly.
Got it. So it is possible to get some of this info. Thanks!
Quoted 4 lines
Quoted 3 lines
Finally, I was wondering if it is the intention to put a list of all kunit tests that use this new feature into
tests/kunit.rstor would this be broken up in some way
Quoted 8 lines
IMO, it makes sense to break this per subsystem, and have an auto-generated
index.rstpointing to the entire set of documents.We're already storing the subsystem at the documentation macros, so, IMO, it should shouldn't be hard to implement it.
Regards,
Mauro
I think breaking this up by subsystems sounds like a good idea, especially since we still have them documented already.
Thanks for your responses!
-Rae
Quoted 1 line
Re: [PATCH RFC 2/2] drm: add documentation for drm_buddy_test kUnit test
In reply to Mauro Carvalho ChehabQuoted 69 lines · Mauro Carvalho Chehab
As an example for the new documentation tool, add a documentation for
drm_buddy_test.I opted to place this on a completely different directory, in order to make easier to test the feature with:
$ make SPHINXDIRS="tests" htmldocsSigned-off-by Mauro Carvalho Chehab <[email protected]>---
To avoid mailbombing on a large number of people, only mailing lists were C/C on the cover. See [PATCH RFC 0/2] at: https://lore.kernel.org/all/[email protected]/
Documentation/index.rst | 2 +- Documentation/tests/index.rst | 6 ++++++ Documentation/tests/kunit.rst | 5 +++++ drivers/gpu/drm/tests/drm_buddy_test.c | 12 ++++++++++++ 4 files changed, 24 insertions(+), 1 deletion(-) create mode 100644 Documentation/tests/index.rst create mode 100644 Documentation/tests/kunit.rstDocumentation/index.rst +1
-1@@ -60,7 +60,7 @@ Various other manuals with useful information for all kernel developers. fault-injection/index livepatch/index rust/index - + test/index User-oriented documentation ===========================Documentation/tests/index.rst new file +6
-0@@ -0,0 +1,6 @@ +======================== +Kunit documentation test +======================== + +.. toctree:: + kunitDocumentation/tests/kunit.rst new file +5
-0@@ -0,0 +1,5 @@ +Kunit tests +----------- + +.. include-test:: drivers/gpu/drm/tests/drm_buddy_test.c +drivers/gpu/drm/tests/drm_buddy_test.c +2
-0@@ -737,6 +737,18 @@ static int drm_buddy_suite_init(struct kunit_suite *suite)return 0 ; } +/** +* KTEST_SUITE: set of tests for drm buddy alloc
drivers/gpu/drm/tests/drm_buddy_test.c static int drm_buddy_suite_init(struct kunit_suite *suite) · line 751
Quoted 10 lines
drivers/gpu/drm/tests/drm_buddy_test.c +10
-0+* Scope: drm subsystem +* Mega feature: drm +* Feature: buddy_alloc +* +* KTEST_TEST: drm_test_buddy_alloc_%s +* Description: Run DRM buddy allocation %arg[1] test +* +* arg[1].values: limit, range, optimistic, smoke, pathological +*/ +
I apologize that the last email included a HTML attachment of the message.
Just in case anyone was unable to receive the last email here is a copy of the message:
Hi!
This is such a cool patch series. I just have a few comments related to the output.
In the html output the tests are listed as:
ktest@drm_buddy_test@...
I wonder if instead of using the file name of "drm_buddy_test" this could possibly be the suite name, "drm_buddy", as this is what users will call when using kunit.py to run the tests. Although "drm_buddy_test" is also the module name so I don't mind it too much. But in the future the file name and module name are not guaranteed to be the same for other tests.
Most preferably, there would be a reference to the kunit suite name, file name, and the module name.
This may be difficult to implement as these can all differ. I am currently working on the KUnit Attribute framework which saves the module name and I am thinking about also saving the file path as a future attribute. This could be a helpful framework for the KUnit tests specifically.
I am not sure how easy it would be to access c objects/functions using this system.
Finally, I was wondering if it is the intention to put a list of all KUnit tests that use this new feature into tests/kunit.rst or would this be broken up in some way.
Thanks!
-Rae
Quoted 6 lines
drivers/gpu/drm/tests/drm_buddy_test.c +0
-1static struct kunit_case drm_buddy_tests[] = {KUNIT_CASE (drm_test_buddy_alloc_limit),KUNIT_CASE (drm_test_buddy_alloc_range), --2.40.1