doc: kselftest: update for clarity on running kselftests in CI rings
Subsystems (3)
Shuah Khan
maintainerPATCH v2 1/1 doc: kselftest: update for clarity on running kselftests in CI rings
Update to add clarity and recommendations on running newer kselftests on older kernels vs. matching the kernel and kselftest revisions.
The recommendation is "Match kernel revision and kselftest."
---
Changes since v1: Fixed "WARNING: Title underline too short."
Documentation/dev-tools/kselftest.rst | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+)
Documentation/dev-tools/kselftest.rst +25 -0
@@ -12,6 +12,31 @@ write new tests using the framework on Kselftest wiki: https://kselftest.wiki.kernel.org/ +Recommendations on running kselftests in Continuous Integration test rings +========================================================================== + +It is recommended that users run Kselftest from the same release. Running +newer Kselftest on older kernels isn't recommended for the following +reasons: + +- Kselftest from mainline and linux-next might not be stable enough to run + on stable kernels. +- Kselftests detect feature dependencies at run-time and skip tests if a + feature and/or configuration they test aren't enabled. Running newer + tests on older kernels could result in a few too many skipped/failed + conditions. It becomes difficult to evaluate the results. +- Newer tests provide better coverage. However, users should make a judgement + call on coverage vs. run to run consistency and being able to compare + run to run results on older kernels. + +Recommendations: + +Match kernel revision and kselftest. Especially important for LTS and +Stable kernel Continuous Integration test rings. + +Hot-plug tests +============== + On some systems, hot-plug tests could hang forever waiting for cpu and memory to be ready to be offlined. A special hot-plug target is created to run the full range of hot-plug tests. In default mode, hot-plug tests run
Signature · 1 line
2.20.1
shuah
maintainerQuoted 8 lines · Shuah Khan
Update to add clarity and recommendations on running newer kselftests on older kernels vs. matching the kernel and kselftest revisions.
The recommendation is "Match kernel revision and kselftest."
Signed-off-by Shuah Khan <[email protected]>---
Changes since v1: Fixed "WARNING: Title underline too short."
I have a few more changes and would like to make and send a v3 after the LPC's Testing and Fuzzing kselftest discussion.
Holding off on this patch for now.
thanks,
-- Shuah
Quoted 7 lines · shuah
Quoted 5 lines · Shuah Khan
Update to add clarity and recommendations on running newer kselftests on older kernels vs. matching the kernel and kselftest revisions.
The recommendation is "Match kernel revision and kselftest."
Quoted 8 lines
Quoted 3 lines
Signed-off-by Shuah Khan <[email protected]>---
Changes since v1: Fixed "WARNING: Title underline too short."I have a few more changes and would like to make and send a v3 after the LPC's Testing and Fuzzing kselftest discussion.
Holding off on this patch for now.
Is this just because you are busy, or because you expect what you want to say to change after the discussion?
If it is because you expect what you want to say here to change, I am surprised. This seems like pretty good, straightforward advice. From where I stand this seems like it makes the documentation better without making anything worse, and so this change should probably be included.
In anycase, your call. I just don't think anyone is going to dispute what you are saying here :-)
Cheers!
Quoted 7 lines · Shuah Khan
Update to add clarity and recommendations on running newer kselftests on older kernels vs. matching the kernel and kselftest revisions.
The recommendation is "Match kernel revision and kselftest."
Signed-off-by Shuah Khan <[email protected]>
One minor comment below.
Quoted 22 lines
---
Changes since v1: Fixed "WARNING: Title underline too short."Documentation/dev-tools/kselftest.rst | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+)Documentation/dev-tools/kselftest.rst +8
-0@@ -12,6 +12,31 @@ write new tests using the framework on Kselftest wiki: https://kselftest.wiki.kernel.org/ +Recommendations on running kselftests in Continuous Integration test rings +========================================================================== + +It is recommended that users run Kselftest from the same release. Running +newer Kselftest on older kernels isn't recommended for the following +reasons: + +- Kselftest from mainline and linux-next might not be stable enough to run
Documentation/dev-tools/kselftest.rst write new tests using the framework on Kselftest wiki: · line 32
Quoted 10 lines
Documentation/dev-tools/kselftest.rst +10
-0+ on stable kernels. +- Kselftests detect feature dependencies at run-time and skip tests if a + feature and/or configuration they test aren't enabled. Running newer + tests on older kernels could result in a few too many skipped/failed + conditions. It becomes difficult to evaluate the results. +- Newer tests provide better coverage. However, users should make a judgement + call on coverage vs. run to run consistency and being able to compare + run to run results on older kernels. + +Recommendations:
I think this should be the "note" directive. You already have a list of recommendations above; this seems like a TLDR section, or an important note or tip.
Quoted 13 lines
Documentation/dev-tools/kselftest.rst +7
-1+ +Match kernel revision and kselftest. Especially important for LTS and +Stable kernel Continuous Integration test rings. + +Hot-plug tests +============== + On some systems, hot-plug tests could hang forever waiting for cpu and memory to be ready to be offlined. A special hot-plug target is created to run the full range of hot-plug tests. In default mode, hot-plug tests run --2.20.1
shuah
maintainerQuoted 50 lines · Brendan Higgins
Quoted 7 lines · Shuah Khan
Update to add clarity and recommendations on running newer kselftests on older kernels vs. matching the kernel and kselftest revisions.
The recommendation is "Match kernel revision and kselftest."
Signed-off-by Shuah Khan <[email protected]>Reviewed-by Brendan Higgins <[email protected]>One minor comment below.
Quoted 22 lines
---
Changes since v1: Fixed "WARNING: Title underline too short."Documentation/dev-tools/kselftest.rst | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+)Documentation/dev-tools/kselftest.rst +8
-0@@ -12,6 +12,31 @@ write new tests using the framework on Kselftest wiki: https://kselftest.wiki.kernel.org/ +Recommendations on running kselftests in Continuous Integration test rings +========================================================================== + +It is recommended that users run Kselftest from the same release. Running +newer Kselftest on older kernels isn't recommended for the following +reasons: + +- Kselftest from mainline and linux-next might not be stable enough to runDocumentation/dev-tools/kselftest.rst write new tests using the framework on Kselftest wiki: · line 32
Quoted 10 lines
Documentation/dev-tools/kselftest.rst +10
-0+ on stable kernels. +- Kselftests detect feature dependencies at run-time and skip tests if a + feature and/or configuration they test aren't enabled. Running newer + tests on older kernels could result in a few too many skipped/failed + conditions. It becomes difficult to evaluate the results. +- Newer tests provide better coverage. However, users should make a judgement + call on coverage vs. run to run consistency and being able to compare + run to run results on older kernels. + +Recommendations:I think this should be the "note" directive. You already have a list of recommendations above; this seems like a TLDR section, or an important note or tip.
Quoted 14 lines
Quoted 11 lines
Documentation/dev-tools/kselftest.rst +7
-1+ +Match kernel revision and kselftest. Especially important for LTS and +Stable kernel Continuous Integration test rings. + +Hot-plug tests +============== + On some systems, hot-plug tests could hang forever waiting for cpu and memory to be ready to be offlined. A special hot-plug target is created to run the full range of hot-plug tests. In default mode, hot-plug tests run --Quoted 2 lines
2.20.1
Thanks Brendan. I am holding off on this patch for now. I will have to make some updates. I will fold your comment in when I send an updated patch.
thanks,
-- Shuah