Skip to contentSkip to conversation

docs/kselftest: clarify running mainline tests on stables

PATCH In review Shuah Khan 2 participants 5 messages Jan 26, 2022 – Jan 31, 2022

Subsystems (3)
lore
Share & tools
lore

Shuah Khan

author maintainer
#0 lore raw

PATCH 1/1 docs/kselftest: clarify running mainline tests on stables

Update the document to clarifiy support for running mainline kselftest on stable releases and the reasons for not removing test code that can test older kernels.

---
 Documentation/dev-tools/kselftest.rst | 8 ++++++++
 1 file changed, 8 insertions(+)
Documentation/dev-tools/kselftest.rst +8 -0
index dcefee707ccd..a833ecf12fbc 100644
@@ -7,6 +7,14 @@ directory. These are intended to be small tests to exercise individual code
 paths in the kernel. Tests are intended to be run after building, installing
 and booting a kernel.
 
+Kselftest from mainline can be run on older stable kernels. Running tests
+from mainline offers the best coverage. Several test rings run mainline
+kselftest suite on stable releases. The reason is that when a new test
+gets added to test existing code to regression test a bug, we should be
+able to run that test on an older kernel. Hence, it is important to keep
+code that can still test an older kernel and make sure it skips the test
+gracefully on newer releases.
+
 You can find additional information on Kselftest framework, how to
 write new tests using the framework on Kselftest wiki:
 
Signature · 1 line
2.32.0
#1 lore raw

Hi Shuah,

Quoted 18 lines · Shuah Khan

Update the document to clarifiy support for running mainline kselftest on stable releases and the reasons for not removing test code that can test older kernels.

---
 Documentation/dev-tools/kselftest.rst | 8 ++++++++
 1 file changed, 8 insertions(+)
Documentation/dev-tools/kselftest.rst +1 -0
index dcefee707ccd..a833ecf12fbc 100644
@@ -7,6 +7,14 @@ directory. These are intended to be small tests to exercise individual code
 paths in the kernel. Tests are intended to be run after building, installing
 and booting a kernel.
 
+Kselftest from mainline can be run on older stable kernels. Running tests

Documentation/dev-tools/kselftest.rst directory. These are intended to be small tests to exercise individual code · line 20

Quoted 10 lines
Documentation/dev-tools/kselftest.rst +7 -0
+from mainline offers the best coverage. Several test rings run mainline
+kselftest suite on stable releases. The reason is that when a new test
+gets added to test existing code to regression test a bug, we should be
+able to run that test on an older kernel. Hence, it is important to keep
+code that can still test an older kernel and make sure it skips the test
+gracefully on newer releases.
+
 You can find additional information on Kselftest framework, how to
 write new tests using the framework on Kselftest wiki:
 

(My apologies if this is already documented, I was not able to find this guidance in Documentation/dev-tools/kselftest.rst nor when looking at the "Kselftest use-cases..." slides linked from https://kselftest.wiki.kernel.org/)

Could you please clarify what the requirement/expectation is regarding fixes to tests? Since the recommendation in the above change is that Kselftest from mainline should be run on older stable kernels, is it required to backport fixes to the tests themselves to stable kernels?

Thank you very much

Reinette

author maintainer
#2 lore raw

Hi Reinette,

Quoted 36 lines · Reinette Chatre

Hi Shuah,

Quoted 18 lines · Shuah Khan

Update the document to clarifiy support for running mainline kselftest on stable releases and the reasons for not removing test code that can test older kernels.

---
  Documentation/dev-tools/kselftest.rst | 8 ++++++++
  1 file changed, 8 insertions(+)
Documentation/dev-tools/kselftest.rst +1 -0
index dcefee707ccd..a833ecf12fbc 100644
@@ -7,6 +7,14 @@ directory. These are intended to be small tests to exercise individual code
  paths in the kernel. Tests are intended to be run after building, installing
  and booting a kernel.
 
+Kselftest from mainline can be run on older stable kernels. Running tests

Documentation/dev-tools/kselftest.rst directory. These are intended to be small tests to exercise individual code · line 20

Quoted 10 lines
Documentation/dev-tools/kselftest.rst +7 -0
+from mainline offers the best coverage. Several test rings run mainline
+kselftest suite on stable releases. The reason is that when a new test
+gets added to test existing code to regression test a bug, we should be
+able to run that test on an older kernel. Hence, it is important to keep
+code that can still test an older kernel and make sure it skips the test
+gracefully on newer releases.
+
  You can find additional information on Kselftest framework, how to
  write new tests using the framework on Kselftest wiki:
 

(My apologies if this is already documented, I was not able to find this guidance in Documentation/dev-tools/kselftest.rst nor when looking at the "Kselftest use-cases..." slides linked from https://kselftest.wiki.kernel.org/)

Quoted 5 lines

Could you please clarify what the requirement/expectation is regarding fixes to tests? Since the recommendation in the above change is that Kselftest from mainline should be run on older stable kernels, is it required to backport fixes to the tests themselves to stable kernels?

Couple of things to consider.

- A new test gets added to regression test a bug in stable and mainline
- A new test gets added to test a kernel module/feature/API that has been supported by stable and mainline releases

In both of these cases, running mainline kselftest on stables gives you the best coverage.

Kselftest fixes get pulled into stables like any other kernel fixes. If a few fixes are missing, it is a good idea to back-port if they fall into above two categories. If the test is for a new feature then, it doesn't make sense to back-port.

Hope this is helpful.

thanks,
-- Shuah

#3 lore raw

Hi Shuah,

Quoted 49 lines · Shuah Khan
Quoted 33 lines · Reinette Chatre
Quoted 24 lines · Shuah Khan

Update the document to clarifiy support for running mainline kselftest on stable releases and the reasons for not removing test code that can test older kernels.

---
  Documentation/dev-tools/kselftest.rst | 8 ++++++++

  1 file changed, 8 insertions(+)
Documentation/dev-tools/kselftest.rst +0 -0
index dcefee707ccd..a833ecf12fbc 100644
@@ -7,6 +7,14 @@ directory. These are intended to be small tests to exercise individual code

  paths in the kernel. Tests are intended to be run after building, installing and booting a kernel.
  +Kselftest from mainline can be run on older stable kernels. Running tests

Documentation/dev-tools/kselftest.rst +7 -0
+from mainline offers the best coverage. Several test rings run mainline
+kselftest suite on stable releases. The reason is that when a new test
+gets added to test existing code to regression test a bug, we should be
+able to run that test on an older kernel. Hence, it is important to keep
+code that can still test an older kernel and make sure it skips the test
+gracefully on newer releases.
+
Quoted 3 lines

  You can find additional information on Kselftest framework, how to write new tests using the framework on Kselftest wiki:

(My apologies if this is already documented, I was not able to find this guidance in Documentation/dev-tools/kselftest.rst nor when looking at the "Kselftest use-cases..." slides linked from https://kselftest.wiki.kernel.org/)

Quoted 5 lines

Could you please clarify what the requirement/expectation is regarding fixes to tests? Since the recommendation in the above change is that Kselftest from mainline should be run on older stable kernels, is it required to backport fixes to the tests themselves to stable kernels?

Couple of things to consider.

- A new test gets added to regression test a bug in stable and mainline
- A new test gets added to test a kernel module/feature/API that has been supported by stable and mainline releases

In both of these cases, running mainline kselftest on stables gives you the best coverage.

Quoted 6 lines

Kselftest fixes get pulled into stables like any other kernel fixes. If a few fixes are missing, it is a good idea to back-port if they fall into above two categories. If the test is for a new feature then, it doesn't make sense to back-port.

Hope this is helpful.

This is helpful, thank you very much. In summary I understand this to mean that when testing a stable kernel it is recommended to run tests from mainline, but running the tests from the same stable kernel version as the kernel being tested is also a supported use case and thus fixes to tests should be back-ported.

Thank you

Reinette

author maintainer
#4 lore raw

Hi Reinette,

Quoted 57 lines · Reinette Chatre

Hi Shuah,

Quoted 54 lines · Shuah Khan
Quoted 33 lines · Reinette Chatre
Quoted 17 lines · Shuah Khan

Update the document to clarifiy support for running mainline kselftest on stable releases and the reasons for not removing test code that can test older kernels.

---
   Documentation/dev-tools/kselftest.rst | 8 ++++++++
   1 file changed, 8 insertions(+)
Documentation/dev-tools/kselftest.rst +0 -0
index dcefee707ccd..a833ecf12fbc 100644
@@ -7,6 +7,14 @@ directory. These are intended to be small tests to exercise individual code
   paths in the kernel. Tests are intended to be run after building, installing
   and booting a kernel.
   +Kselftest from mainline can be run on older stable kernels. Running tests

Documentation/dev-tools/kselftest.rst directory. These are intended to be small tests to exercise individual code · line 19

Quoted 10 lines
Documentation/dev-tools/kselftest.rst +7 -0
+from mainline offers the best coverage. Several test rings run mainline
+kselftest suite on stable releases. The reason is that when a new test
+gets added to test existing code to regression test a bug, we should be
+able to run that test on an older kernel. Hence, it is important to keep
+code that can still test an older kernel and make sure it skips the test
+gracefully on newer releases.
+
   You can find additional information on Kselftest framework, how to
   write new tests using the framework on Kselftest wiki:
 

(My apologies if this is already documented, I was not able to find this guidance in Documentation/dev-tools/kselftest.rst nor when looking at the "Kselftest use-cases..." slides linked from https://kselftest.wiki.kernel.org/)

Quoted 5 lines

Could you please clarify what the requirement/expectation is regarding fixes to tests? Since the recommendation in the above change is that Kselftest from mainline should be run on older stable kernels, is it required to backport fixes to the tests themselves to stable kernels?

Couple of things to consider.

- A new test gets added to regression test a bug in stable and mainline
- A new test gets added to test a kernel module/feature/API that has been supported by stable and mainline releases

In both of these cases, running mainline kselftest on stables gives you the best coverage.

Kselftest fixes get pulled into stables like any other kernel fixes. If a few fixes are missing, it is a good idea to back-port if they fall into above two categories. If the test is for a new feature then, it doesn't make sense to back-port.

Quoted 7 lines
Quoted 1 line

Hope this is helpful.

This is helpful, thank you very much. In summary I understand this to mean that when testing a stable kernel it is recommended to run tests from mainline, but running the tests from the same stable kernel version as the kernel being tested is also a supported use case and thus fixes to tests should be back-ported.

Correct. Right. In the slide set you referenced, I included a slide that shows the combination and one of them is indeed rev matching kselftest and kernel.

Some tests such as bpf require rev match with the kernel i.e running kselftest from the same release.

thanks,
-- Shuah