Baseline test plan and bootrr
Quoted 15 lines · Guillaume Tucker via Groups.Io
Hi,
Currently, boot tests entirely rely on the LAVA logic to detect when a login prompt has been reached. While this mostly works, it does not guarantee that the platform is actually usable or that everything went smoothly during the boot.
For this reason, it would seem useful to introduce a "baseline" test plan which would essentially boot and then do some fast checks to verify that nothing is obviously broken. This can include things like grepping the kernel log for any errors and checking that drivers have been initialised correctly. There's a lot that can be done in a few seconds with a basic ramdisk.
Quoted 8 lines
So we could have a list of regular expressions to detect any issues in the kernel log and report a LAVA test case result for each of them. The log fragment associated with each match should also be available to include the actual errors in a report. Doing this on the device means we keep the test definition in the same location as the other test plans, and we can run it like any test suite.
Any thoughts on this part? Does anyone see any issue with having a series of patterns to grep the kernel log and have a test case result for each of them in LAVA? (i.e. pass if pattern was not matched...)
Guillaume
Quoted 27 lines · Guillaume Tucker
Quoted 15 lines · Guillaume Tucker via Groups.Io
Hi,
Currently, boot tests entirely rely on the LAVA logic to detect when a login prompt has been reached. While this mostly works, it does not guarantee that the platform is actually usable or that everything went smoothly during the boot.
For this reason, it would seem useful to introduce a "baseline" test plan which would essentially boot and then do some fast checks to verify that nothing is obviously broken. This can include things like grepping the kernel log for any errors and checking that drivers have been initialised correctly. There's a lot that can be done in a few seconds with a basic ramdisk.
Quoted 8 lines
So we could have a list of regular expressions to detect any issues in the kernel log and report a LAVA test case result for each of them. The log fragment associated with each match should also be available to include the actual errors in a report. Doing this on the device means we keep the test definition in the same location as the other test plans, and we can run it like any test suite.
Quoted 4 lines
Any thoughts on this part? Does anyone see any issue with having a series of patterns to grep the kernel log and have a test case result for each of them in LAVA? (i.e. pass if pattern was not matched...)
I think this is a good idea.
The difficulty comes in looking for device/board specific stuff, but that shouldn't stop us from doing something generic that looks for the obvious stuff.
It could also look for (and count) kernel errors, warnings, backtraces, etc. etc.
Kevin
Quoted 34 lines · Kevin Hilman
Quoted 27 lines · Guillaume Tucker
Quoted 15 lines · Guillaume Tucker via Groups.Io
Hi,
Currently, boot tests entirely rely on the LAVA logic to detect when a login prompt has been reached. While this mostly works, it does not guarantee that the platform is actually usable or that everything went smoothly during the boot.
For this reason, it would seem useful to introduce a "baseline" test plan which would essentially boot and then do some fast checks to verify that nothing is obviously broken. This can include things like grepping the kernel log for any errors and checking that drivers have been initialised correctly. There's a lot that can be done in a few seconds with a basic ramdisk.
Quoted 8 lines
So we could have a list of regular expressions to detect any issues in the kernel log and report a LAVA test case result for each of them. The log fragment associated with each match should also be available to include the actual errors in a report. Doing this on the device means we keep the test definition in the same location as the other test plans, and we can run it like any test suite.
Quoted 4 lines
Any thoughts on this part? Does anyone see any issue with having a series of patterns to grep the kernel log and have a test case result for each of them in LAVA? (i.e. pass if pattern was not matched...)
Quoted 9 lines
I think this is a good idea.
The difficulty comes in looking for device/board specific stuff, but that shouldn't stop us from doing something generic that looks for the obvious stuff.
It could also look for (and count) kernel errors, warnings, backtraces, etc. etc.
Thanks for the feedback so far. I've now created an initial PR to start testing this on staging:
https://github.com/kernelci/kernelci-core/pull/55
I believe the bootrr script should be part of the rootfs before we can merge this and roll it out in production.
Guillaume