Skip to contentSkip to conversation

oeqa/runtime/cases/ptest.py: use to_boolean for PTEST_EXPECT_FAILURE

PATCH Superseded [email protected] 2 participants 4 messages Dec 5, 2025

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 oeqa/runtime/cases/ptest.py: use to_boolean for PTEST_EXPECT_FAILURE

From: Chen Qi <[email protected]>

When PTEST_EXPECT_FAILURE is set to "0", the expected behavior is that ptest should succeed. So we need to use to_boolean.

---
 meta/lib/oeqa/runtime/cases/ptest.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
meta/lib/oeqa/runtime/cases/ptest.py +3 -3
index fbaeb84d00..4bbed290b4 100644
@@ -22,7 +22,7 @@ class PtestRunnerTest(OERuntimeTestCase):
     @OEHasPackage(['ptest-runner'])
     @unittest.expectedFailure
     def test_ptestrunner_expectfail(self):
-        if not self.td.get('PTEST_EXPECT_FAILURE'):
+        if not bb.utils.to_boolean(self.td.get('PTEST_EXPECT_FAILURE')):
             self.skipTest('Cannot run ptests with @expectedFailure as ptests are required to pass')
         self.do_ptestrunner()
 
@@ -30,8 +30,8 @@ class PtestRunnerTest(OERuntimeTestCase):
     @OETestDepends(['ssh.SSHTest.test_ssh'])
     @OEHasPackage(['ptest-runner'])
     def test_ptestrunner_expectsuccess(self):
-        if self.td.get('PTEST_EXPECT_FAILURE'):
-            self.skipTest('Cannot run ptests without @expectedFailure as ptests are expected to fail')
+        if bb.utils.to_boolean(self.td.get('PTEST_EXPECT_FAILURE')):
+             self.skipTest('Cannot run ptests without @expectedFailure as ptests are expected to fail')
         self.do_ptestrunner()
 
     def do_ptestrunner(self):
Signature · 1 line
2.43.0
#1 lore raw

PATCH 2/2 core-image-ptest.bb: use ?= for PTEST_EXPECT_FAILURE

From: Chen Qi <[email protected]>

We need to allow users to easily override this value in local.conf. This has the benefit that 'bitbake core-image-ptest-xxx:do_testimage' fails when expected.

As an example, I used 'bitbake core-image-ptest-util-linux:do_testimage'. It succeeded with warning message. I didn't notice the warning message. I saw the command succeeded and I tought util-linux ptest is OK. But in actual fact, the ptest failed. It's the PTEST_EXPECT_FAILURE setting in this core-image-ptest.bb that is not giving me error.

With this change, I can put PTEST_EXPECT_FAILURE = "1" in my local.conf, and I can easily see if 'core-image-ptest-xxx:do_testimage' succeeds or not.

The PTEST_EXPECT_FAILURE change should result in testdata regenerated because ptest.py test case's behavior is affected by it.

---
 meta/recipes-core/images/core-image-ptest.bb | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
meta/recipes-core/images/core-image-ptest.bb +2 -1
index 017f05f81b..b554964d20 100644
@@ -42,7 +42,8 @@ QB_MEM:virtclass-mcextend-tcl = "-m 5100"
 TEST_SUITES = "ping ssh parselogs ptest"
 
 # Sadly at the moment the full set of ptests is not robust enough and sporadically fails in random places
-PTEST_EXPECT_FAILURE = "1"
+PTEST_EXPECT_FAILURE ?= "1"
+write_image_test_data[vardeps] += "PTEST_EXPECT_FAILURE"
 
 python () {
     if not d.getVar("MCNAME"):
Signature · 1 line
2.43.0
#2 lore raw

RE: [OE-core][PATCH 2/2] core-image-ptest.bb: use ?= for PTEST_EXPECT_FAILURE

Quoted 17 lines

-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Chen Qi via lists.openembedded.org
Sent: Friday, December 5, 2025 9:10
To: [email protected]
Subject: [OE-core][PATCH 2/2] core-image-ptest.bb: use ?= for PTEST_EXPECT_FAILURE

From: Chen Qi <[email protected]>

We need to allow users to easily override this value in local.conf. This has the benefit that 'bitbake core-image-ptest-xxx:do_testimage' fails when expected.

Quoted 7 lines

As an example, I used 'bitbake core-image-ptest-util-linux:do_testimage'. It succeeded with warning message. I didn't notice the warning message. I saw the command succeeded and I tought util-linux ptest is OK. But in actual fact, the ptest failed. It's the PTEST_EXPECT_FAILURE setting in this core-image-ptest.bb that is not giving me error.

With this change, I can put PTEST_EXPECT_FAILURE = "1" in my local.conf,

This comment should be probably "0".

Quoted 29 lines

and I can easily see if 'core-image-ptest-xxx:do_testimage' succeeds or not.

The PTEST_EXPECT_FAILURE change should result in testdata regenerated because ptest.py test case's behavior is affected by it.

---
 meta/recipes-core/images/core-image-ptest.bb | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
meta/recipes- renamed from meta/recipes-core/images/core-image-ptest.bb +0 -0

core/images/core-image-ptest.bb
index 017f05f81b..b554964d20 100644

meta/recipes-core/images/core-image-ptest.bb +0 -0
@@ -42,7 +42,8 @@ QB_MEM:virtclass-mcextend-tcl = "-m 5100"
 TEST_SUITES = "ping ssh parselogs ptest"
 
 # Sadly at the moment the full set of ptests is not robust enough and sporadically

fails in random places

meta/recipes-core/images/core-image-ptest.bb +2 -2
-PTEST_EXPECT_FAILURE = "1"
+PTEST_EXPECT_FAILURE ?= "1"
+write_image_test_data[vardeps] += "PTEST_EXPECT_FAILURE"
 
 python () {
     if not d.getVar("MCNAME"):
--

2.43.0

author
#3 lore raw

Re: [OE-core][PATCH 2/2] core-image-ptest.bb: use ?= for PTEST_EXPECT_FAILURE

Quoted 16 lines · Peter
Quoted 16 lines

-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Chen Qi via lists.openembedded.org
Sent: Friday, December 5, 2025 9:10
To: [email protected]
Subject: [OE-core][PATCH 2/2] core-image-ptest.bb: use ?= for PTEST_EXPECT_FAILURE

From: Chen Qi <[email protected]>

We need to allow users to easily override this value in local.conf. This has the benefit that 'bitbake core-image-ptest-xxx:do_testimage' fails when expected.

Quoted 8 lines
Quoted 7 lines

As an example, I used 'bitbake core-image-ptest-util-linux:do_testimage'. It succeeded with warning message. I didn't notice the warning message. I saw the command succeeded and I tought util-linux ptest is OK. But in actual fact, the ptest failed. It's the PTEST_EXPECT_FAILURE setting in this core-image-ptest.bb that is not giving me error.

With this change, I can put PTEST_EXPECT_FAILURE = "1" in my local.conf,

This comment should be probably "0".

You're right. I'll send out V2. Thanks for spotting it.

Regards,
Qi

Quoted 30 lines
Quoted 30 lines

and I can easily see if 'core-image-ptest-xxx:do_testimage' succeeds or not.

The PTEST_EXPECT_FAILURE change should result in testdata regenerated because ptest.py test case's behavior is affected by it.

---
  meta/recipes-core/images/core-image-ptest.bb | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)
meta/recipes- renamed from meta/recipes-core/images/core-image-ptest.bb +0 -0

core/images/core-image-ptest.bb
index 017f05f81b..b554964d20 100644

meta/recipes-core/images/core-image-ptest.bb +0 -0
@@ -42,7 +42,8 @@ QB_MEM:virtclass-mcextend-tcl = "-m 5100"
  TEST_SUITES = "ping ssh parselogs ptest"
 
  # Sadly at the moment the full set of ptests is not robust enough and sporadically

fails in random places

meta/recipes-core/images/core-image-ptest.bb +2 -2
-PTEST_EXPECT_FAILURE = "1"
+PTEST_EXPECT_FAILURE ?= "1"
+write_image_test_data[vardeps] += "PTEST_EXPECT_FAILURE"
 
  python () {
      if not d.getVar("MCNAME"):
--

2.43.0