Skip to contentSkip to conversation

test-manual/ptest: Add info about fail state of ptest-runner/ptests

PATCH Superseded Anibal Limon 2 participants 3 messages Jun 11, 2025 – Jun 19, 2025

v4 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

PATCH v4 1/1 test-manual/ptest: Add info about fail state of ptest-runner/ptests

On the ML was pointed about no specific information about ptest-runner/ptests so improve docs around.

https://lists.yoctoproject.org/g/yocto-patches/message/1622

[YOCTO #15832]

---
 documentation/test-manual/ptest.rst | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)
documentation/test-manual/ptest.rst +12 -5
index 2c021af51..9442387f7 100644
@@ -46,13 +46,19 @@ Running ptest
 =============
 
 The ``ptest-runner`` package installs a shell script that loops through
-all installed ptest test suites and runs them in sequence. Consequently,
-you might want to add this package to your image.
+all installed ptest test suites and runs them in sequence.
+
+During the execution ``ptest-runner`` keeps counter of total and failed
+``ptests``, At end the execution summary is written to the console. If
+any of the ``run-ptest`` fails returns '1'.
+
+Consequently, you might want to add this package to your image.
+
 
 Getting Your Package Ready
 ==========================
 
-In order to enable a recipe to run installed ptests on target hardware,
+In order to enable a recipe to run installed ``ptests`` on target hardware,
 you need to prepare the recipes that build the packages you want to
 test. Here is what you have to do for each recipe:
 
@@ -77,8 +83,9 @@ test. Here is what you have to do for each recipe:
 
 -  *Create run-ptest:* This script starts your test. Locate the
    script where you will refer to it using
-   :term:`SRC_URI`. Here is an
-   example that starts a test for ``dbus``::
+   :term:`SRC_URI`. Be sure ``run-ptest`` exit with 0 to mark it
+   as sucessfully executed otherwise will be marked as fail.
+   Here is an example that starts a test for ``dbus``::
 
       #!/bin/sh
       cd test
Signature · 1 line
2.39.5
#1 lore raw

Hi,

Quoted 17 lines · Anibal Limon

On the ML was pointed about no specific information about ptest-runner/ptests so improve docs around.

https://lists.yoctoproject.org/g/yocto-patches/message/1622

[YOCTO #15832]

---
 documentation/test-manual/ptest.rst | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)
documentation/test-manual/ptest.rst +0 -0
index 2c021af51..9442387f7 100644

documentation/test-manual/ptest.rst Running ptest · line 51

Quoted 8 lines
documentation/test-manual/ptest.rst +3 -2
@@ -46,13 +46,19 @@ Running ptest
 =============
 
 The ``ptest-runner`` package installs a shell script that loops through
-all installed ptest test suites and runs them in sequence. Consequently,
-you might want to add this package to your image.
+all installed ptest test suites and runs them in sequence.
+
+During the execution ``ptest-runner`` keeps counter of total and failed

s/keeps counter of/keeps count of/

documentation/test-manual/ptest.rst Running ptest · line 52

Quoted 1 line
documentation/test-manual/ptest.rst +1 -0
+``ptests``, At end the execution summary is written to the console. If

Should be a "." not "," after ``ptests``.

documentation/test-manual/ptest.rst Running ptest · line 53

Quoted 1 line
documentation/test-manual/ptest.rst +1 -0
+any of the ``run-ptest`` fails returns '1'.

Missing a word?

s/``run-ptest`` fails returns '1'/``run-ptest``, ``ptest-runner`` returns '1'/

documentation/test-manual/ptest.rst Running ptest · line 55

Quoted 2 lines
documentation/test-manual/ptest.rst +2 -0
+
+Consequently, you might want to add this package to your image.

I would suggest:

"Consequently, you might want to add ``ptest-runner`` to your image."

So users know they can add the package "ptest-runner" in IMAGE_INSTALL directly.

Quoted 11 lines
documentation/test-manual/ptest.rst +2 -1
+
 
 Getting Your Package Ready
 ==========================
 
-In order to enable a recipe to run installed ptests on target hardware,
+In order to enable a recipe to run installed ``ptests`` on target hardware,
 you need to prepare the recipes that build the packages you want to
 test. Here is what you have to do for each recipe:
 

documentation/test-manual/ptest.rst test. Here is what you have to do for each recipe: · line 86

Quoted 6 lines
documentation/test-manual/ptest.rst +1 -2
@@ -77,8 +83,9 @@ test. Here is what you have to do for each recipe:
 
 -  *Create run-ptest:* This script starts your test. Locate the
    script where you will refer to it using
-   :term:`SRC_URI`. Here is an
-   example that starts a test for ``dbus``::
+   :term:`SRC_URI`. Be sure ``run-ptest`` exit with 0 to mark it

s/exit/exits/

documentation/test-manual/ptest.rst test. Here is what you have to do for each recipe: · line 87

Quoted 1 line
documentation/test-manual/ptest.rst +1 -0
+   as sucessfully executed otherwise will be marked as fail.

Sorry, I corrected your typo with a typo the first time. It's actually:

s/sucessfully/successfully/

documentation/test-manual/ptest.rst test. Here is what you have to do for each recipe: · line 91

Quoted 4 lines
documentation/test-manual/ptest.rst +1 -0
+   Here is an example that starts a test for ``dbus``::
 
       #!/bin/sh
       cd test

Thanks,
Antonin

Signature · 3 lines
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#2 lore raw

Hi Antonin,

Thanks for the corrections, sending the next PATCH version.

Cheers!,
Anibal

Quoted 94 lines · Antonin Godard

Hi,

Quoted 5 lines · Anibal Limon

On the ML was pointed about no specific information about ptest-runner/ptests so improve docs around.

https://lists.yoctoproject.org/g/yocto-patches/message/1622

Review

Quoted 8 lines

[YOCTO #15832]

---
 documentation/test-manual/ptest.rst | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)
(quoted diff) +0 -0

b/documentation/test-manual/ptest.rst

documentation/test-manual/ptest.rst Running ptest · line 51

Quoted 12 lines

index 2c021af51..9442387f7 100644

documentation/test-manual/ptest.rst +3 -2
@@ -46,13 +46,19 @@ Running ptest
 =============
 
 The ``ptest-runner`` package installs a shell script that loops through
-all installed ptest test suites and runs them in sequence. Consequently,
-you might want to add this package to your image.
+all installed ptest test suites and runs them in sequence.
+
+During the execution ``ptest-runner`` keeps counter of total and failed

s/keeps counter of/keeps count of/

documentation/test-manual/ptest.rst Running ptest · line 52

Quoted 1 line
documentation/test-manual/ptest.rst +1 -0
+``ptests``, At end the execution summary is written to the console. If

Should be a "." not "," after ``ptests``.

documentation/test-manual/ptest.rst Running ptest · line 53

Quoted 1 line
documentation/test-manual/ptest.rst +1 -0
+any of the ``run-ptest`` fails returns '1'.

Missing a word?

s/``run-ptest`` fails returns '1'/``run-ptest``, ``ptest-runner`` returns '1'/

documentation/test-manual/ptest.rst Running ptest · line 55

Quoted 2 lines
documentation/test-manual/ptest.rst +2 -0
+
+Consequently, you might want to add this package to your image.

I would suggest:

"Consequently, you might want to add ``ptest-runner`` to your image."

So users know they can add the package "ptest-runner" in IMAGE_INSTALL directly.

documentation/test-manual/ptest.rst Running ptest

Quoted 7 lines
documentation/test-manual/ptest.rst +2 -1
+
 
 Getting Your Package Ready
 ==========================
 
-In order to enable a recipe to run installed ptests on target hardware,
+In order to enable a recipe to run installed ``ptests`` on target

hardware,

documentation/test-manual/ptest.rst test. Here is what you have to do for each recipe: · line 86

Quoted 10 lines
documentation/test-manual/ptest.rst +1 -2
 you need to prepare the recipes that build the packages you want to
 test. Here is what you have to do for each recipe:
 
@@ -77,8 +83,9 @@ test. Here is what you have to do for each recipe:
 
 -  *Create run-ptest:* This script starts your test. Locate the
    script where you will refer to it using
-   :term:`SRC_URI`. Here is an
-   example that starts a test for ``dbus``::
+   :term:`SRC_URI`. Be sure ``run-ptest`` exit with 0 to mark it

s/exit/exits/

documentation/test-manual/ptest.rst test. Here is what you have to do for each recipe: · line 87

Quoted 1 line
documentation/test-manual/ptest.rst +1 -0
+   as sucessfully executed otherwise will be marked as fail.

Sorry, I corrected your typo with a typo the first time. It's actually:

s/sucessfully/successfully/

documentation/test-manual/ptest.rst test. Here is what you have to do for each recipe: · line 91

Quoted 4 lines
documentation/test-manual/ptest.rst +1 -0
+   Here is an example that starts a test for ``dbus``::
 
       #!/bin/sh
       cd test

Thanks,
Antonin

--
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com