Skip to contentSkip to conversation

Simplify conditional operations with bb.utils.filter

PATCH In review João Marcos Costa 3 participants 8 messages Apr 27 – May 4

lore
Share & tools
lore
#0 lore raw

v2 cover Simplify conditional operations with bb.utils.filter

Hello,

This patch series applies overall the same change, but to specific scopes:
the recipes, insane.bbclass, and a couple .inc files. The idea here is to address the comments and concerns raised in v1.

Considering now I'm providing separate patches, the review process can be focused on points where the debate is still ongoing, while the rest (i.e., the recipes) can be picked.

Changes since v1:
- v1 was actually one single patch providing all of the changes
- the .strip() workaround was not provided in feature-arm-vfp.inc.

João Marcos Costa (3):
meta: simplify conditional operations with bb.utils.filter
classes-global: insane.bbclass : simplify conditional operations with bb.utils.filter
machine: include: arm: simplify conditional operations with bb.utils.filter

 meta/classes-global/insane.bbclass                        | 2 +-
 meta/conf/machine/include/arm/feature-arm-neon.inc        | 8 ++++----
 meta/conf/machine/include/arm/feature-arm-vfp.inc         | 8 ++++----
 meta/recipes-connectivity/connman/connman_2.0.bb          | 2 +-
 meta/recipes-core/coreutils/coreutils_9.10.bb             | 2 +-
 meta/recipes-core/ovmf/ovmf_git.bb                        | 2 +-
 meta/recipes-extended/at/at_3.2.5.bb                      | 2 +-
 meta/recipes-graphics/libglvnd/libglvnd_1.7.0.bb          | 2 +-
 meta/recipes-graphics/waffle/waffle_1.8.1.bb              | 2 +-
 .../gstreamer/gstreamer1.0-plugins-bad_1.28.2.bb          | 2 +-
 10 files changed, 16 insertions(+), 16 deletions(-)
Signature · 1 line
2.47.0
#1 lore raw

PATCH v2 1/3 meta: simplify conditional operations with bb.utils.filter

Some recipes use bb.utils.contains to check for a string inside a variable, and return the exact same string if true.

This can be simplified by a call to bb.utils.filter, since the result is the same, and the inline is shorter.

Replace "bb.utils.contains(A, 'a', 'a', '', d)" by "bb.utils.filter(A, 'a', d)".

---
 meta/recipes-connectivity/connman/connman_2.0.bb                | 2 +-
 meta/recipes-core/coreutils/coreutils_9.10.bb                   | 2 +-
 meta/recipes-core/ovmf/ovmf_git.bb                              | 2 +-
 meta/recipes-extended/at/at_3.2.5.bb                            | 2 +-
 meta/recipes-graphics/libglvnd/libglvnd_1.7.0.bb                | 2 +-
 meta/recipes-graphics/waffle/waffle_1.8.1.bb                    | 2 +-
 .../gstreamer/gstreamer1.0-plugins-bad_1.28.2.bb                | 2 +-
 7 files changed, 7 insertions(+), 7 deletions(-)
meta/recipes-connectivity/connman/connman_2.0.bb +1 -1
index 6f7093301b..e9873f3163 100644
@@ -169,7 +169,7 @@ FILES:${PN}-tools = "${bindir}/wispr"
 RDEPENDS:${PN}-tools = "${PN}"
 
 FILES:${PN}-tests = "${bindir}/*-test"
-RDEPENDS:${PN}-tests = "${@bb.utils.contains('PACKAGECONFIG', 'iptables', 'iptables', '', d)}"
+RDEPENDS:${PN}-tests = "${@bb.utils.filter('PACKAGECONFIG', 'iptables', d)}"
 
 FILES:${PN}-client = "${bindir}/connmanctl"
 RDEPENDS:${PN}-client = "${PN}"
meta/recipes-core/coreutils/coreutils_9.10.bb +1 -1
index 984c5b5292..744d930272 100644
@@ -222,6 +222,6 @@ do_install_ptest:append:libc-musl () {
 }
 
 RDEPENDS:${PN}-ptest += "xz  \
-                         ${@bb.utils.contains('PACKAGECONFIG', 'acl', 'acl', '', d)} \
+                         ${@bb.utils.filter('PACKAGECONFIG', 'acl', d)} \
                          ${@bb.utils.contains('PACKAGECONFIG', 'xattr', 'attr', '', d)}"
 FILES:${PN}-ptest += "${bindir}/getlimits"
meta/recipes-core/ovmf/ovmf_git.bb +1 -1
index d731bca7f2..38d5d090b4 100644
@@ -10,7 +10,7 @@ LIC_FILES_CHKSUM = "file://OvmfPkg/License.txt;md5=06357ddc23f46577c2aeaeaf7b776
 # compiling OVMF twice, so it is disabled by default. Distros
 # may change that default.
 PACKAGECONFIG ??= ""
-PACKAGECONFIG += "${@bb.utils.contains('MACHINE_FEATURES', 'tpm', 'tpm', '', d)}"
+PACKAGECONFIG += "${@bb.utils.filter('MACHINE_FEATURES', 'tpm', d)}"
 PACKAGECONFIG += "${@bb.utils.contains('MACHINE_FEATURES', 'tpm2', 'tpm', '', d)}"
 PACKAGECONFIG[debug] = ",,,"
 PACKAGECONFIG[secureboot] = ",,,"
meta/recipes-extended/at/at_3.2.5.bb +1 -1
index 112d1c4adc..ee485f67ec 100644
@@ -9,7 +9,7 @@ DEPENDS = "flex flex-native bison-native \
            ${@bb.utils.contains('DISTRO_FEATURES', 'pam', 'libpam', '', d)}"
 
 PACKAGECONFIG ?= "\
-    ${@bb.utils.contains('DISTRO_FEATURES', 'selinux', 'selinux', '', d)} \
+    ${@bb.utils.filter('DISTRO_FEATURES', 'selinux', d)} \
 "
 
 PACKAGECONFIG[selinux] = "--with-selinux,--without-selinux,libselinux,"
meta/recipes-graphics/libglvnd/libglvnd_1.7.0.bb +1 -1
index 18eeaa5523..c76763f811 100644
@@ -14,7 +14,7 @@ REQUIRED_DISTRO_FEATURES = "opengl glvnd"
 inherit meson pkgconfig features_check
 
 PACKAGECONFIG ?= "\
-  ${@bb.utils.contains('DISTRO_FEATURES', 'x11', 'x11', '', d)} \
+  ${@bb.utils.filter('DISTRO_FEATURES', 'x11', d)} \
   ${@bb.utils.contains('DISTRO_FEATURES', 'opengl', 'egl gles1 gles2', '', d)} \
   ${@bb.utils.contains('DISTRO_FEATURES', 'opengl x11', 'glx', '', d)} \
   "
meta/recipes-graphics/waffle/waffle_1.8.1.bb +1 -1
index aefa0069cf..5d993cf254 100644
@@ -21,7 +21,7 @@ DEPENDS:append = " python3"
 # This should be overridden per-machine to reflect the capabilities of the GL
 # stack.
 PACKAGECONFIG ??= "${@bb.utils.contains('DISTRO_FEATURES', 'x11', 'glx x11-egl', '', d)} \
-                   ${@bb.utils.contains('DISTRO_FEATURES', 'wayland', 'wayland', '', d)} \
+                   ${@bb.utils.filter('DISTRO_FEATURES', 'wayland', d)} \
                    ${@bb.utils.contains('DISTRO_FEATURES', 'opengl', 'gbm surfaceless-egl', '', d)} \
 "
 
meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_1.28.2.bb +1 -1
index cdf3a20dff..9945e79bf7 100644
@@ -25,7 +25,7 @@ PACKAGECONFIG ??= " \
     ${GSTREAMER_ORC} \
     ${@bb.utils.contains('DISTRO_FEATURES', 'bluetooth', 'bluez', '', d)} \
     ${@bb.utils.filter('DISTRO_FEATURES', 'directfb vulkan x11', d)} \
-    ${@bb.utils.contains('DISTRO_FEATURES', 'wayland', 'wayland', '', d)} \
+    ${@bb.utils.filter('DISTRO_FEATURES', 'wayland', d)} \
     ${@bb.utils.contains('DISTRO_FEATURES', 'opengl', 'gl', '', d)} \
     bz2 closedcaption curl dash dtls hls openssl sbc smoothstreaming \
     sndfile ttml uvch264 webp analytics \
Signature · 1 line
2.47.0
Reviewed-by on v2 1/3
#2 lore raw

Re: [PATCH v2 1/3] meta: simplify conditional operations with bb.utils.filter

Hi João,

Quoted 49 lines · João Marcos Costa

Some recipes use bb.utils.contains to check for a string inside a variable, and return the exact same string if true.

This can be simplified by a call to bb.utils.filter, since the result is the same, and the inline is shorter.

Replace "bb.utils.contains(A, 'a', 'a', '', d)" by "bb.utils.filter(A, 'a', d)".

---
  meta/recipes-connectivity/connman/connman_2.0.bb                | 2 +-
  meta/recipes-core/coreutils/coreutils_9.10.bb                   | 2 +-
  meta/recipes-core/ovmf/ovmf_git.bb                              | 2 +-
  meta/recipes-extended/at/at_3.2.5.bb                            | 2 +-
  meta/recipes-graphics/libglvnd/libglvnd_1.7.0.bb                | 2 +-
  meta/recipes-graphics/waffle/waffle_1.8.1.bb                    | 2 +-
  .../gstreamer/gstreamer1.0-plugins-bad_1.28.2.bb                | 2 +-
  7 files changed, 7 insertions(+), 7 deletions(-)
meta/recipes-connectivity/connman/connman_2.0.bb +1 -1
index 6f7093301b..e9873f3163 100644
@@ -169,7 +169,7 @@ FILES:${PN}-tools = "${bindir}/wispr"
  RDEPENDS:${PN}-tools = "${PN}"
 
  FILES:${PN}-tests = "${bindir}/*-test"
-RDEPENDS:${PN}-tests = "${@bb.utils.contains('PACKAGECONFIG', 'iptables', 'iptables', '', d)}"
+RDEPENDS:${PN}-tests = "${@bb.utils.filter('PACKAGECONFIG', 'iptables', d)}"
 
  FILES:${PN}-client = "${bindir}/connmanctl"
  RDEPENDS:${PN}-client = "${PN}"
meta/recipes-core/coreutils/coreutils_9.10.bb +1 -1
index 984c5b5292..744d930272 100644
@@ -222,6 +222,6 @@ do_install_ptest:append:libc-musl () {
  }
 
  RDEPENDS:${PN}-ptest += "xz  \
-                         ${@bb.utils.contains('PACKAGECONFIG', 'acl', 'acl', '', d)} \
+                         ${@bb.utils.filter('PACKAGECONFIG', 'acl', d)} \
                           ${@bb.utils.contains('PACKAGECONFIG', 'xattr', 'attr', '', d)}"
  FILES:${PN}-ptest += "${bindir}/getlimits"
meta/recipes-core/ovmf/ovmf_git.bb +0 -0
index d731bca7f2..38d5d090b4 100644

meta/recipes-core/ovmf/ovmf_git.bb LIC_FILES_CHKSUM = "file://OvmfPkg/License.txt;md5=06357ddc23f46577c2aeaeaf7b776 · line 14

Quoted 6 lines
meta/recipes-core/ovmf/ovmf_git.bb +1 -1
@@ -10,7 +10,7 @@ LIC_FILES_CHKSUM = "file://OvmfPkg/License.txt;md5=06357ddc23f46577c2aeaeaf7b776
  # compiling OVMF twice, so it is disabled by default. Distros
  # may change that default.
  PACKAGECONFIG ??= ""
-PACKAGECONFIG += "${@bb.utils.contains('MACHINE_FEATURES', 'tpm', 'tpm', '', d)}"
+PACKAGECONFIG += "${@bb.utils.filter('MACHINE_FEATURES', 'tpm', d)}"
  PACKAGECONFIG += "${@bb.utils.contains('MACHINE_FEATURES', 'tpm2', 'tpm', '', d)}"

I believe we could actually use

bb.utils.contains_any('MACHINE_FEATURES', 'tpm tpm2', 'tpm', '', d)

to merge the two tpm MACHINE_FEATURES that enable the same PACKAGECONFIG.

It also irks me that we're using += here instead of

PACKAGECONFIG ?= "${@bb.utils.contains_any('MACHINE_FEATURES', 'tpm tpm2', 'tpm', '', d)}"

(or ??=), but maybe that's fine (or in a separate commit anyway).

Thanks!
Quentin

#3 lore raw

PATCH v2 2/3 classes-global: insane.bbclass : simplify conditional operations with bb.utils.filter

In reply to João Marcos Costa

The append override on ERROR_QA uses bb.utils.contains to check for a string inside a variable, and return the exact same string if true.

This can be simplified by a call to bb.utils.filter, since the result is the same, and the inline is shorter.

Replace "bb.utils.contains(A, 'a', 'a', '', d)" by "bb.utils.filter(A, 'a', d)".

---
 meta/classes-global/insane.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes-global/insane.bbclass +1 -1
index feddfe0335..04700be71c 100644
@@ -46,7 +46,7 @@ ERROR_QA ?= "\
     ${CHECKLAYER_REQUIRED_TESTS}"
 
 # Add usrmerge QA check based on distro feature
-ERROR_QA:append = "${@bb.utils.contains('DISTRO_FEATURES', 'usrmerge', ' usrmerge', '', d)}"
+ERROR_QA:append = " ${@bb.utils.filter('DISTRO_FEATURES', 'usrmerge', d)}"
 WARN_QA:append:layer-core = " missing-metadata missing-maintainer"
 
 FAKEROOT_QA = "host-user-contaminated"
Signature · 1 line
2.47.0
Reviewed-by on v2 2/3
#4 lore raw

Re: [PATCH v2 2/3] classes-global: insane.bbclass : simplify conditional operations with bb.utils.filter

Hi João,

Quoted 8 lines · João Marcos Costa

The append override on ERROR_QA uses bb.utils.contains to check for a string inside a variable, and return the exact same string if true.

This can be simplified by a call to bb.utils.filter, since the result is the same, and the inline is shorter.

Replace "bb.utils.contains(A, 'a', 'a', '', d)" by "bb.utils.filter(A, 'a', d)".

*Technically*, you are replacing

bb.utils.contains(A, 'a', ' a', '', d)
with
bb.utils.filter(A, 'a', d)

You could have also said that you checked that ERROR_QA doesn't care about spurious whitespaces so that is a fine change.

Thanks!
Quentin

#5 lore raw

PATCH v2 3/3 machine: include: arm: simplify conditional operations with bb.utils.filter

In reply to João Marcos Costa

Some configuration files use bb.utils.contains to check for a string inside a variable, and return the exact same string if true.

This can be simplified by a call to bb.utils.filter, since the result is the same, and the inline is shorter.

Replace "bb.utils.contains(A, 'a', 'a', '', d)" by "bb.utils.filter(A, 'a', d)".

bb.utils.filter() does not return the string with a leading space, and this is handled by a leading space outside of the helper. This workaround, however, has its limitations: the leading space is always added. To avoid any potential issues, use .strip() when dereferencing TUNE_CCARGS_MFPU in the if statements.

---
 meta/conf/machine/include/arm/feature-arm-neon.inc | 8 ++++----
 meta/conf/machine/include/arm/feature-arm-vfp.inc  | 8 ++++----
 2 files changed, 8 insertions(+), 8 deletions(-)
meta/conf/machine/include/arm/feature-arm-neon.inc +4 -4
index 174b9b9f2a..2ec354bfeb 100644
@@ -5,16 +5,16 @@
 # 'vfp', -mfloat-abi parameter and 'hf' suffix is implemented in feature-arm-vfp.inc
 
 TUNEVALID[neon] = "Enable Neon SIMD accelerator unit."
-TUNE_CCARGS_MFPU .= "${@bb.utils.contains('TUNE_FEATURES', 'neon', ' neon', '', d)}"
+TUNE_CCARGS_MFPU .= " ${@bb.utils.filter('TUNE_FEATURES', 'neon', d)}"
 
 TUNEVALID[vfpv3d16] = "Enable Vector Floating Point Version 3 with 16 registers (vfpv3-d16) unit."
-TUNE_CCARGS_MFPU .= "${@bb.utils.contains('TUNE_FEATURES', 'vfpv3d16', ' vfpv3-d16', '', d)}"
+TUNE_CCARGS_MFPU .= " ${@bb.utils.contains('TUNE_FEATURES', 'vfpv3d16', ' vfpv3-d16', '', d)}"
 
 TUNEVALID[vfpv3] = "Enable Vector Floating Point Version 3 with 32 registers (vfpv3) unit."
-TUNE_CCARGS_MFPU .= "${@bb.utils.contains('TUNE_FEATURES', 'vfpv3', ' vfpv3', '' , d)}"
+TUNE_CCARGS_MFPU .= " ${@bb.utils.filter('TUNE_FEATURES', 'vfpv3', d)}"
 
 TUNEVALID[vfpv4] = "Enable Vector Floating Point Version 4 (vfpv4) unit."
-TUNE_CCARGS_MFPU .= "${@bb.utils.contains('TUNE_FEATURES', 'vfpv4', ' vfpv4', '', d)}"
+TUNE_CCARGS_MFPU .= " ${@bb.utils.filter('TUNE_FEATURES', 'vfpv4', d)}"
 TUNE_CCARGS_MFPU .= "${@bb.utils.contains('TUNE_FEATURES', [ 'vfpv4', 'neon' ], ' neon-vfpv4', '', d)}"
 
 TUNEVALID[vfpv4d16] = "Enable Vector Floating Point Version 4 with 16 registers (vfpv4-d16) unit."
meta/conf/machine/include/arm/feature-arm-vfp.inc +4 -4
index d020100daa..22cd5e1767 100644
@@ -3,14 +3,14 @@
 # and this .inc file is included from armv5
 
 TUNEVALID[vfp] = "Enable Vector Floating Point (vfp) unit."
-TUNE_CCARGS_MFPU .= "${@bb.utils.contains('TUNE_FEATURES', 'vfp', ' vfp', '', d)}"
+TUNE_CCARGS_MFPU .= " ${@bb.utils.filter('TUNE_FEATURES', 'vfp', d)}"
 
 # simd is special, we don't pass this to the -mfpu, it's implied
-TUNE_CCARGS  .= "${@ (' -mfpu=%s' % d.getVar('TUNE_CCARGS_MFPU').split()[-1]) if (d.getVar('TUNE_CCARGS_MFPU') != '') else ''}"
+TUNE_CCARGS  .= "${@ (' -mfpu=%s' % d.getVar('TUNE_CCARGS_MFPU').split()[-1]) if (d.getVar('TUNE_CCARGS_MFPU').strip() != '') else ''}"
 # The following deals with both vfpv3-d16 and vfpv4-d16
-ARMPKGSFX_FPU = "${@ ('-%s'       % d.getVar('TUNE_CCARGS_MFPU').split()[-1].replace('-d16', 'd16')) if (d.getVar('TUNE_CCARGS_MFPU') != '') else ''}"
+ARMPKGSFX_FPU = "${@ ('-%s'       % d.getVar('TUNE_CCARGS_MFPU').split()[-1].replace('-d16', 'd16')) if (d.getVar('TUNE_CCARGS_MFPU').strip() != '') else ''}"
 
 TUNEVALID[callconvention-hard] = "Enable EABI hard float call convention, requires VFP."
-TUNE_CCARGS_MFLOAT = "${@ bb.utils.contains('TUNE_FEATURES', 'callconvention-hard', 'hard', 'softfp', d) if (d.getVar('TUNE_CCARGS_MFPU') != '' or bb.utils.contains('TUNE_FEATURES', 'simd', True, False, d)) else '' }"
+TUNE_CCARGS_MFLOAT = "${@ bb.utils.contains('TUNE_FEATURES', 'callconvention-hard', 'hard', 'softfp', d) if (d.getVar('TUNE_CCARGS_MFPU').strip() != '' or bb.utils.contains('TUNE_FEATURES', 'simd', True, False, d)) else '' }"
 TUNE_CCARGS   .= "${@ ' -mfloat-abi=${TUNE_CCARGS_MFLOAT}' if (d.getVar('TUNE_CCARGS_MFLOAT') != '')     else ''}"
 ARMPKGSFX_EABI = "${@ 'hf'                                 if (d.getVar('TUNE_CCARGS_MFLOAT') == 'hard') else ''}"
Signature · 1 line
2.47.0
Reviewed-by on v2 3/3
#6 lore raw

Re: [PATCH v2 3/3] machine: include: arm: simplify conditional operations with bb.utils.filter

Hi João,

Quoted 6 lines · João Marcos Costa

Some configuration files use bb.utils.contains to check for a string inside a variable, and return the exact same string if true.

This can be simplified by a call to bb.utils.filter, since the result is the same, and the inline is shorter.

Quoted 8 lines

Replace "bb.utils.contains(A, 'a', 'a', '', d)" by "bb.utils.filter(A, 'a', d)".

bb.utils.filter() does not return the string with a leading space, and this is handled by a leading space outside of the helper. This workaround, however, has its limitations: the leading space is always added. To avoid any potential issues, use .strip() when dereferencing TUNE_CCARGS_MFPU in the if statements.

I'm not sure this is worth it as this pattern means next time we add a new tunefeatures we'll likely follow the same pattern. This means that even though the existing machines won't set this new tunefeature, they'll likely rebuild a few things because TUNE_CCARGS_MFPU will have an additional space in it (but maybe BitBake parsing is smart enough to realize that only

TUNE_CCARGS, ARMPKGSFX_FPU and TUNE_CCARGS_MFLOAT use it and they won't change if there's only one space difference, so:

Thanks!
Quentin

#7 lore raw

Re: [OE-core] [PATCH v2 3/3] machine: include: arm: simplify conditional operations with bb.utils.filter

In reply to João Marcos Costa
Quoted 14 lines · Joao Marcos Costa via lists.openembedded.org

Some configuration files use bb.utils.contains to check for a string inside a variable, and return the exact same string if true.

This can be simplified by a call to bb.utils.filter, since the result is the same, and the inline is shorter.

Replace "bb.utils.contains(A, 'a', 'a', '', d)" by "bb.utils.filter(A, 'a', d)".

bb.utils.filter() does not return the string with a leading space, and this is handled by a leading space outside of the helper. This workaround, however, has its limitations: the leading space is always added. To avoid any potential issues, use .strip() when dereferencing TUNE_CCARGS_MFPU in the if statements.

Quoted 5 lines

---
 meta/conf/machine/include/arm/feature-arm-neon.inc | 8 ++++---- meta/conf/machine/include/arm/feature-arm-vfp.inc  | 8 ++++----

 2 files changed, 8 insertions(+), 8 deletions(-)

We've taken the other patches but not this one since we don't think this one is a net gain in readability or usability and there is no pressing reason we have to change.

Cheers,

Richard