Skip to contentSkip to conversation

insane.bbclass: Make do_qa_patch() depend on if patch-fuzz is in ERROR_QA

PATCH In review Peter Kjellerstedt 2 participants 5 messages Apr 4, 2022 1 message not in the archive

lore
Share & tools
lore
#0 lore raw

PATCH 1/3 insane.bbclass: Make do_qa_patch() depend on if patch-fuzz is in ERROR_QA

Adding "patch-fuzz" to ERROR_QA should trigger the patch tasks to rerun to make sure any already existing fuzz is caught. This is achieved by using bb.utils.filter() to see if "patch-fuzz" is in ERROR_QA/WARN_QA as it adds whether the filtered strings are set or not to the task hash.

Since the mechanism used above ignores that a variable is excluded (which WARN_QA is), the value for whether "patch-fuzz" is in WARN_QA or not has to be explicitly excluded so that adding/removing "patch-fuzz" to/from WARN_QA does not trigger the patch tasks to rerun.

---
 meta/classes/insane.bbclass | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
meta/classes/insane.bbclass +4 -2
index 0deebdb148..873706952b 100644
@@ -1142,6 +1142,8 @@ python do_qa_staging() {
     oe.qa.exit_with_message_if_errors("QA staging was broken by the package built above", d)
 }
 
+do_qa_patch[vardepvalueexclude] .= "|WARN_QA{patch-fuzz} = Set"
+do_qa_patch[vardepvalueexclude] .= "|WARN_QA{patch-fuzz} = Unset"
 python do_qa_patch() {
     import subprocess
 
@@ -1182,9 +1184,9 @@ python do_qa_patch() {
             msg += "    devtool modify %s\n" % d.getVar('PN')
             msg += "    devtool finish --force-patch-refresh %s <layer_path>\n\n" % d.getVar('PN')
             msg += "Don't forget to review changes done by devtool!\n"
-            if 'patch-fuzz' in d.getVar('ERROR_QA'):
+            if bb.utils.filter('ERROR_QA', 'patch-fuzz', d):
                 bb.error(msg)
-            elif 'patch-fuzz' in d.getVar('WARN_QA'):
+            elif bb.utils.filter('WARN_QA', 'patch-fuzz', d):
                 bb.warn(msg)
             msg = "Patch log indicates that patches do not apply cleanly."
             oe.qa.handle_error("patch-fuzz", msg, d)
#1 lore raw

PATCH 2/3 bitbake.conf: Remove ERROR_QA from BB_HASHEXCLUDE_COMMON

If a QA test is added to ERROR_QA, all package_qa tasks should rerun to make sure any already existing errors are caught.

---
 meta/conf/bitbake.conf | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/conf/bitbake.conf +1 -1
index 7705415a4f..3b8760a3f8 100644
@@ -949,7 +949,7 @@ BB_HASHEXCLUDE_COMMON ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH BBSERVER DL_DI
     USER FILESPATH STAGING_DIR_HOST STAGING_DIR_TARGET COREBASE PRSERV_HOST \
     STAMPS_DIR PRSERV_DUMPDIR PRSERV_DUMPFILE PRSERV_LOCKDOWN PARALLEL_MAKE \
     CCACHE_DIR EXTERNAL_TOOLCHAIN CCACHE CCACHE_NOHASHDIR LICENSE_PATH SDKPKGSUFFIX \
-    WARN_QA ERROR_QA WORKDIR STAMPCLEAN PKGDATA_DIR BUILD_ARCH SSTATE_PKGARCH \
+    WARN_QA WORKDIR STAMPCLEAN PKGDATA_DIR BUILD_ARCH SSTATE_PKGARCH \
     BB_WORKERCONTEXT BB_LIMITEDDEPS BB_UNIHASH extend_recipe_sysroot DEPLOY_DIR \
     SSTATE_HASHEQUIV_METHOD SSTATE_HASHEQUIV_REPORT_TASKDATA \
     SSTATE_HASHEQUIV_OWNER CCACHE_TOP_DIR BB_HASHSERVE GIT_CEILING_DIRECTORIES \
#2 lore raw

PATCH 3/3 insane.bbclass: Make changes to QA_EMPTY_DIRS trigger package_qa to rerun

In reply to Peter Kjellerstedt

If "empty-dirs" is in ERROR_QA, then changes to QA_EMPTY_DIRS should trigger the package_qa tasks to be rerun.

---
 meta/classes/insane.bbclass | 1 +
 1 file changed, 1 insertion(+)
meta/classes/insane.bbclass +1 -0
index 873706952b..ca67a70c43 100644
@@ -1117,6 +1117,7 @@ python do_package_qa () {
 # binutils is used for most checks, so need to set as dependency
 # POPULATESYSROOTDEPS is defined in staging class.
 do_package_qa[depends] += "${POPULATESYSROOTDEPS}"
+do_package_qa[vardeps] = "${@bb.utils.contains('ERROR_QA', 'empty-dirs', 'QA_EMPTY_DIRS', '', d)}"
 do_package_qa[vardepsexclude] = "BB_TASKDEPDATA"
 do_package_qa[rdeptask] = "do_packagedata"
 addtask do_package_qa after do_packagedata do_package before do_build
#3 lore raw
In reply to Peter Kjellerstedt
Quoted 22 lines · Peter Kjellerstedt

Adding "patch-fuzz" to ERROR_QA should trigger the patch tasks to rerun to make sure any already existing fuzz is caught. This is achieved by using bb.utils.filter() to see if "patch-fuzz" is in ERROR_QA/WARN_QA as it adds whether the filtered strings are set or not to the task hash.

Since the mechanism used above ignores that a variable is excluded (which WARN_QA is), the value for whether "patch-fuzz" is in WARN_QA or not has to be explicitly excluded so that adding/removing "patch-fuzz" to/from WARN_QA does not trigger the patch tasks to rerun.

---
 meta/classes/insane.bbclass | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
meta/classes/insane.bbclass +0 -0
index 0deebdb148..873706952b 100644

meta/classes/insane.bbclass python do_qa_staging() { · line 1146

Quoted 5 lines
meta/classes/insane.bbclass +2 -0
@@ -1142,6 +1142,8 @@ python do_qa_staging() {
     oe.qa.exit_with_message_if_errors("QA staging was broken by the package built above", d)
 }
 
+do_qa_patch[vardepvalueexclude] .= "|WARN_QA{patch-fuzz} = Set"
+do_qa_patch[vardepvalueexclude] .= "|WARN_QA{patch-fuzz} = Unset"

I'd never seen that syntax before, it certainly wasn't intentional. Looking at the code, I can see how it happens to work though. It does assume a bit more internal implementation knowledge than I'd prefer...

Cheers,

Richard

#5 lore raw
In reply to a message not in the archive
Quoted 26 lines · Richard Purdie via lists.openembedded.org
Quoted 25 lines · Peter Kjellerstedt

Adding "patch-fuzz" to ERROR_QA should trigger the patch tasks to rerun to make sure any already existing fuzz is caught. This is achieved by using bb.utils.filter() to see if "patch-fuzz" is in ERROR_QA/WARN_QA as it adds whether the filtered strings are set or not to the task hash.

Since the mechanism used above ignores that a variable is excluded (which WARN_QA is), the value for whether "patch-fuzz" is in WARN_QA or not has to be explicitly excluded so that adding/removing "patch-fuzz" to/from WARN_QA does not trigger the patch tasks to rerun.

---
 meta/classes/insane.bbclass | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
meta/classes/insane.bbclass +0 -0
index 0deebdb148..873706952b 100644
@@ -1142,6 +1142,8 @@ python do_qa_staging() {
     oe.qa.exit_with_message_if_errors("QA staging was broken by the package built above", d)
 }
 
Quoted 6 lines

meta/classes/insane.bbclass python do_qa_staging() { · line 1146

Quoted 2 lines
meta/classes/insane.bbclass +2 -0
+do_qa_patch[vardepvalueexclude] .= "|WARN_QA{patch-fuzz} = Set"
+do_qa_patch[vardepvalueexclude] .= "|WARN_QA{patch-fuzz} = Unset"

I'd never seen that syntax before, it certainly wasn't intentional. Looking at the code, I can see how it happens to work though. It does assume a bit more internal implementation knowledge than I'd prefer...

I've a patch I'm testing in master-next to try and address this and remove the need for this piece of patch.

I need to be a little careful as the loop in question is one of the most time sensitive we have in the parsing code being iterated millions of times, so changes need to be mindful of that.

Cheers,

Richard