Skip to contentSkip to conversation

Implement variable typing

RFC PATCH In review Chris Larson 1 participant 4 messages Nov 9, 2010 – Nov 10, 2010

lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/2 Implement variable typing

From: Chris Larson <[email protected]>

This implementation consists of two components:

- oe.types python module, whose job it is to construct objects of the defined type for a given variable in the metadata
- typecheck.bbclass, which iterates over all configuration variables with a type defined and uses oe.types to check the validity of the values

This gives us a few benefits:

- Automatic sanity checking of all configuration variables with a defined type
- Avoid duplicating the "how do I make use of the value of this variable" logic between its users. For variables like PATH, this is simply a split(), for boolean variables, the duplication can result in confusing, or even mismatched semantics (is this 0/1, empty/nonempty, what?)
- Make it easier to create a configuration UI, as the type information could be used to provide a better interface than a text edit box (e.g checkbox for 'boolean', dropdown for 'choice')

This functionality is entirely opt-in right now. To enable the configuration variable type checking, simply INHERIT += "typecheck". Example of a failing type check:

BAZ = "foo"
BAZ[type] = "boolean"

$ bitbake -p
FATAL: BAZ: Invalid boolean value 'foo'
$

Examples of leveraging oe.types in a python snippet:

PACKAGES[type] = "list"

python () {
import oe.types
for pkg in oe.types.value("PACKAGES", d):
bb.note("package: %s" % pkg)
}

LIBTOOL_HAS_SYSROOT = "yes"
LIBTOOL_HAS_SYSROOT[type] = "boolean"

python () {
    import oe.types
    assert(oe.types.value("LIBTOOL_HAS_SYSROOT", d) == True)
}
---
 classes/typecheck.bbclass |   12 +++++++
 lib/oe/types.py           |   77 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 89 insertions(+), 0 deletions(-)
 create mode 100644 classes/typecheck.bbclass
 create mode 100644 lib/oe/types.py
classes/typecheck.bbclass new file +12 -0
new file mode 100644
index 0000000..646cd4e
@@ -0,0 +1,12 @@
+# Check types of bitbake configuration variables
+#
+# See oe.types for details.
+
+python check_types() {
+    import oe.types
+    if isinstance(e, bb.event.ConfigParsed):
+        for key in e.data.keys():
+            if e.data.getVarFlag(key, "type"):
+                oe.types.value(key, e.data)
+}
+addhandler check_types
lib/oe/types.py new file +77 -0

77 lines not on this page · raw message

Signature · 1 line
1.7.2.3
#1 lore raw

RFC PATCH 2/2 Add first pass type flags for bitbake.conf

From: Chris Larson <[email protected]>

---
 classes/patch.bbclass |    5 +++++
 conf/bitbake.conf     |   33 +++++++++++++++++++++++++++++++++
 2 files changed, 38 insertions(+), 0 deletions(-)
classes/patch.bbclass +5 -0
index 7629e9a..bc52ae7 100644
@@ -5,6 +5,11 @@ QUILTRCFILE ?= "${STAGING_BINDIR_NATIVE}/quiltrc"
 
 PATCHDEPENDENCY = "${PATCHTOOL}-native:do_populate_sysroot"
 
+PATCHTOOL[type] = "choice"
+PATCHTOOL[choices] = "patch quilt git"
+PATCHRESOLVE[type] = "choice"
+PATCHRESOLVE[choices] = "noop user"
+
 python patch_do_patch() {
 	import oe.patch
 	import oe.unpack
conf/bitbake.conf +33 -0
index f1dc0ff..86a5cc8 100644
@@ -2,6 +2,7 @@
 # Recipe Parsing
 ##################################################################
 BBMASK = "/(nonworking|obsolete)/"
+BBMASK[type] = "regex"
 
 ##################################################################
 # Standard target filesystem layout.
@@ -172,6 +173,7 @@ ASSUME_PROVIDED = "\
     texinfo-native \
     util-linux-native \
     "
+ASSUME_PROVIDED[type] = "list"
 
 ##################################################################
 # Package default variables.
@@ -211,7 +213,9 @@ HOMEPAGE = "unknown"
 # Ensure that -dev packages recommend the corresponding -dev packages of their
 # deps, and the same for -dbg.
 DEPCHAIN_PRE  = ""
+DEPCHAIN_PRE[type] = "list"
 DEPCHAIN_POST = "-dev -dbg"
+DEPCHAIN_POST[type] = "list"
 
 DEPENDS = ""
 RDEPENDS = ""
@@ -232,8 +236,10 @@ SOLIBSDEV_darwin8 = ".dylib"
 SOLIBSDEV_darwin9 = ".dylib"
 
 PACKAGES = "${PN}-dbg ${PN} ${PN}-doc ${PN}-dev ${PN}-static ${PN}-locale"
+PACKAGES[type] = "list"
 #enable this when bitbake is upgraded to cope with ++ values in the field
 PACKAGES_DYNAMIC = "${PN}-locale-*"
+PACKAGES_DYNAMIC[type] = "list"
 FILES = ""
 
 FILES_${PN} = "${bindir}/* ${sbindir}/* ${libexecdir}/* ${libdir}/lib*${SOLIBS} \
@@ -270,8 +276,14 @@ FILES_${PN}-locale = "${datadir}/locale"
 # file:// search paths
 FILE_DIRNAME = "${@os.path.dirname(bb.data.getVar('FILE', d))}"
 FILESPATHBASE = "${FILE_DIRNAME}"
+FILESPATHBASE[type] = "list"
+FILESPATHBASE[separator] = ":"
 FILESPATHPKG = "${PF}:${P}:${PN}:${BP}:${BPN}:files:."
+FILESPATHPKG[type] = "list"
+FILESPATHPKG[separator] = ":"
 FILESPATH = "${@':'.join(uniq(os.path.normpath(os.path.join(fp, p, o)) for fp in '${FILESPATHBASE}'.split(':') for p in '${FILESPATHPKG}'.split(':') for o in reversed([''] + filter(None, '${OVERRIDES}'.split(':')))))}"
+FILESPATH[type] = "list"
+FILESPATH[separator] = ":"
 FILESDIR = "${@bb.which(d.getVar('FILESPATH', 1), '.')}"
 
 ##################################################################
@@ -420,8 +432,11 @@ EXTRA_IMAGEDEPENDS = ""
 # by default this is unset and should be so for libtools < 2.4
 
 LIBTOOL_HAS_SYSROOT ?= "no"
+LIBTOOL_HAS_SYSROOT[type] = "boolean"
 
 PATH_prepend = "${STAGING_BINDIR_CROSS}:${STAGING_DIR_NATIVE}${sbindir_native}:${STAGING_BINDIR_NATIVE}:${STAGING_DIR_NATIVE}${base_sbindir_native}:${STAGING_DIR_NATIVE}/${base_bindir_native}:"
+PATH[type] = "list"
+PATH[separator] = ":"
 export PATH
 
 ##################################################################
@@ -594,6 +609,7 @@ BB_LOCALCOUNT_OVERRIDE = "1"
 
 # Value for revision counter in GIT recipes (NNNN in PV="1.0+gitrNNNN+a7401084a72285c8d3485bacf4eed593f303b1b7-r4.4")
 LOCALCOUNT = "0"
+LOCALCOUNT[type] = "integer"
 
 SRC_URI = "file://${FILE}"
 
@@ -647,6 +663,8 @@ export HOSTNAME := "${@os.uname()[1]}"
 PKG_CONFIG_DIR = "${STAGING_LIBDIR}/pkgconfig"
 export PKG_CONFIG_PATH = "${STAGING_LIBDIR}/pkgconfig:${STAGING_DATADIR}/pkgconfig"
 PKG_CONFIG_PATH .= ":${STAGING_LIBDIR_NATIVE}/pkgconfig:${STAGING_DATADIR_NATIVE}/pkgconfig"
+PKG_CONFIG_PATH[type] = "list"
+PKG_CONFIG_PATH[separator] = ":"
 
 export PKG_CONFIG_SYSROOT_DIR = "${STAGING_DIR}/${BASE_PACKAGE_ARCH}${TARGET_VENDOR}-${TARGET_OS}"
 export PKG_CONFIG_DISABLE_UNINSTALLED = "yes"
@@ -681,6 +699,8 @@ MACHINE_OVERRIDES ?= ""
 OVERRIDES = "pn-${PN}:fail-fast:build-${BUILD_OS}:${TARGET_ARCH}:${TARGET_OS}:${DISTRO}:\
 ${@[':'.join(d.getVar('MACHINE_OVERRIDES', True).split())+':',''][d.getVar('MACHINE_OVERRIDES', True).strip() == '']}\
 ${MACHINE}:local"
+OVERRIDES[type] = "list"
+OVERRIDES[separator] = ":"
 
 ##################################################################
 # Include the rest of the config files.
@@ -714,6 +734,7 @@ PCMCIA_MANAGER ?= "pcmcia-cs"
 DEFAULT_TASK_PROVIDER ?= "task-base"
 MACHINE_TASK_PROVIDER ?= "${DEFAULT_TASK_PROVIDER}"
 IMAGE_ROOTFS_SIZE ?= "65536"
+IMAGE_ROOTFS_SIZE[type] = "integer"
 IMAGE_ROOTFS_SIZE_ext2 ?= "65536"
 IMAGE_ROOTFS_SIZE_ext2.gz ?= "65536"
 IMAGE_ROOTFS_SIZE_ext3 ?= "65536"
@@ -726,6 +747,7 @@ CACHE := "${CACHE}"
 
 # Default to disabling packaged staging code
 export PSTAGING_ACTIVE = "0"
+PSTAGING_ACTIVE[type] = "boolean"
 
 ##################################################################
 # Magic Cookie for SANITY CHECK
@@ -743,14 +765,21 @@ DISTRO_FEATURES += ' ${@["", "thumb-interwork"][bb.data.getVar('THUMB_INTERWORK'
 
 # This is used to limit what packages goes into images built, so set big by default
 ROOT_FLASH_SIZE ?= "256"
+ROOT_FLASH_SIZE[type] = "integer"
 
 MACHINE_GUI_CLASS ?= "smallscreen"
+MACHINE_GUI_CLASS[type] = "choice"
+MACHINE_GUI_CLASS[choices] = "bigscreen smallscreen"
 # GUI_MACHINE_CLASS is deprecated, please use MACHINE_GUI_CLASS instead
 GUI_MACHINE_CLASS ?= "${MACHINE_GUI_CLASS}"
 MACHINE_DISPLAY_WIDTH_PIXELS ?= "240"
+MACHINE_DISPLAY_WIDTH_PIXELS[type] = "integer"
 MACHINE_DISPLAY_HEIGHT_PIXELS ?= "320"
+MACHINE_DISPLAY_HEIGHT_PIXELS[type] = "integer"
 MACHINE_DISPLAY_ORIENTATION ?= "0"
+MACHINE_DISPLAY_ORIENTATION[type] = "boolean"
 MACHINE_DISPLAY_BPP ?= "16"
+MACHINE_DISPLAY_BPP[type] = "integer"
 
 DISTRO_EXTRA_RDEPENDS ?= ""
 DISTRO_EXTRA_RRECOMMENDS ?= ""
@@ -759,6 +788,9 @@ MACHINE_EXTRA_RRECOMMENDS ?= ""
 MACHINE_ESSENTIAL_EXTRA_RDEPENDS ?= ""
 MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS ?= ""
 
+DISTRO_FEATURES[type] = "list"
+MACHINE_FEATURES[type] = "list"
+
 COMBINED_FEATURES = "\
     ${@base_both_contain("DISTRO_FEATURES", "MACHINE_FEATURES", "alsa", d)} \
     ${@base_both_contain("DISTRO_FEATURES", "MACHINE_FEATURES", "bluetooth", d)} \
@@ -789,6 +821,7 @@ COMBINED_FEATURES += "${@base_ifelse( \
      (base_contains('MACHINE_FEATURES', 'pcmcia', True, False, d) and \
       base_contains('DISTRO_FEATURES', 'pcmcia', True, False, d))), \
     'hostap', '')}"
+COMBINED_FEATURES[type] = "list"
 
 # Make sure MACHINE isn't exported
 # (breaks binutils at least)
Signature · 1 line
1.7.2.3
#2 lore raw
In reply to Chris Larson
Quoted 46 lines · Chris Larson

This implementation consists of two components:

- oe.types python module, whose job it is to construct objects of the defined
type for a given variable in the metadata
- typecheck.bbclass, which iterates over all configuration variables with a type defined and uses oe.types to check the validity of the values

This gives us a few benefits:

- Automatic sanity checking of all configuration variables with a defined type
- Avoid duplicating the "how do I make use of the value of this variable" logic between its users. For variables like PATH, this is simply a
split(),
for boolean variables, the duplication can result in confusing, or even mismatched semantics (is this 0/1, empty/nonempty, what?)
- Make it easier to create a configuration UI, as the type information could
be used to provide a better interface than a text edit box (e.g checkbox
for
'boolean', dropdown for 'choice')

This functionality is entirely opt-in right now. To enable the configuration
variable type checking, simply INHERIT += "typecheck". Example of a failing
type check:

BAZ = "foo"
BAZ[type] = "boolean"

$ bitbake -p
FATAL: BAZ: Invalid boolean value 'foo'
$

Examples of leveraging oe.types in a python snippet:

PACKAGES[type] = "list"

python () {
import oe.types
for pkg in oe.types.value("PACKAGES", d):

       bb.note("package: %s" % pkg)
}
Quoted 8 lines

LIBTOOL_HAS_SYSROOT = "yes"
LIBTOOL_HAS_SYSROOT[type] = "boolean"

python () {
   import oe.types
   assert(oe.types.value("LIBTOOL_HAS_SYSROOT", d) == True)
}

See https://github.com/kergoth/openembedded/compare/master...typing for the current version, just reworked to decouple the types from the metadata to make their unit testing a bit cleaner. Note that the __new__ use for Choice and Regex is pretty ugly, really only using those classes as a container to make setting the name and flags attributes on the callable less ugly :P
Comments welcome.

Signature · 5 lines
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics
#3 lore raw
Quoted 7 lines · Chris Larson

See https://github.com/kergoth/openembedded/compare/master...typing for the current version, just reworked to decouple the types from the metadata to make their unit testing a bit cleaner. Note that the __new__ use for Choice and Regex is pretty ugly, really only using those classes as a container to make setting the name and flags attributes on the callable less ugly :P Comments welcome.

Fixed. I'm fairly happy with that branch now, so am going to stop messing with it, other than adding more type flags to the metadata. Let me know what you think.

Signature · 5 lines
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics