Implement variable typing
RFC PATCH 1/2 Implement variable typing
From: Chris Larson <[email protected]>
This implementation consists of two components:
- oe.types python module, whose job it is to construct objects of the defined type for a given variable in the metadata
- typecheck.bbclass, which iterates over all configuration variables with a type defined and uses oe.types to check the validity of the values
This gives us a few benefits:
- Automatic sanity checking of all configuration variables with a defined type
- Avoid duplicating the "how do I make use of the value of this variable" logic between its users. For variables like PATH, this is simply a split(), for boolean variables, the duplication can result in confusing, or even mismatched semantics (is this 0/1, empty/nonempty, what?)
- Make it easier to create a configuration UI, as the type information could be used to provide a better interface than a text edit box (e.g checkbox for 'boolean', dropdown for 'choice')
This functionality is entirely opt-in right now. To enable the configuration variable type checking, simply INHERIT += "typecheck". Example of a failing type check:
BAZ = "foo"
BAZ[type] = "boolean"
$ bitbake -p FATAL: BAZ: Invalid boolean value 'foo' $
Examples of leveraging oe.types in a python snippet:
PACKAGES[type] = "list"
python () {
import oe.types
for pkg in oe.types.value("PACKAGES", d):
bb.note("package: %s" % pkg)
}
LIBTOOL_HAS_SYSROOT = "yes"
LIBTOOL_HAS_SYSROOT[type] = "boolean"
python () {
import oe.types
assert(oe.types.value("LIBTOOL_HAS_SYSROOT", d) == True)
}--- classes/typecheck.bbclass | 12 +++++++ lib/oe/types.py | 77 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 89 insertions(+), 0 deletions(-) create mode 100644 classes/typecheck.bbclass create mode 100644 lib/oe/types.py
classes/typecheck.bbclass new file +12 -0
@@ -0,0 +1,12 @@ +# Check types of bitbake configuration variables +# +# See oe.types for details. + +python check_types() { + import oe.types + if isinstance(e, bb.event.ConfigParsed): + for key in e.data.keys(): + if e.data.getVarFlag(key, "type"): + oe.types.value(key, e.data) +} +addhandler check_types
lib/oe/types.py new file +77 -0
77 lines not on this page · raw message
Signature · 1 line
1.7.2.3
RFC PATCH 2/2 Add first pass type flags for bitbake.conf
From: Chris Larson <[email protected]>
--- classes/patch.bbclass | 5 +++++ conf/bitbake.conf | 33 +++++++++++++++++++++++++++++++++ 2 files changed, 38 insertions(+), 0 deletions(-)
classes/patch.bbclass +5 -0
@@ -5,6 +5,11 @@ QUILTRCFILE ?= "${STAGING_BINDIR_NATIVE}/quiltrc" PATCHDEPENDENCY = "${PATCHTOOL}-native:do_populate_sysroot" +PATCHTOOL[type] = "choice" +PATCHTOOL[choices] = "patch quilt git" +PATCHRESOLVE[type] = "choice" +PATCHRESOLVE[choices] = "noop user" + python patch_do_patch() { import oe.patch import oe.unpack
conf/bitbake.conf +33 -0
@@ -2,6 +2,7 @@ # Recipe Parsing ################################################################## BBMASK = "/(nonworking|obsolete)/" +BBMASK[type] = "regex" ################################################################## # Standard target filesystem layout.
@@ -172,6 +173,7 @@ ASSUME_PROVIDED = "\ texinfo-native \ util-linux-native \ " +ASSUME_PROVIDED[type] = "list" ################################################################## # Package default variables.
@@ -211,7 +213,9 @@ HOMEPAGE = "unknown" # Ensure that -dev packages recommend the corresponding -dev packages of their # deps, and the same for -dbg. DEPCHAIN_PRE = "" +DEPCHAIN_PRE[type] = "list" DEPCHAIN_POST = "-dev -dbg" +DEPCHAIN_POST[type] = "list" DEPENDS = "" RDEPENDS = ""
@@ -232,8 +236,10 @@ SOLIBSDEV_darwin8 = ".dylib" SOLIBSDEV_darwin9 = ".dylib" PACKAGES = "${PN}-dbg ${PN} ${PN}-doc ${PN}-dev ${PN}-static ${PN}-locale" +PACKAGES[type] = "list" #enable this when bitbake is upgraded to cope with ++ values in the field PACKAGES_DYNAMIC = "${PN}-locale-*" +PACKAGES_DYNAMIC[type] = "list" FILES = "" FILES_${PN} = "${bindir}/* ${sbindir}/* ${libexecdir}/* ${libdir}/lib*${SOLIBS} \
@@ -270,8 +276,14 @@ FILES_${PN}-locale = "${datadir}/locale" # file:// search paths FILE_DIRNAME = "${@os.path.dirname(bb.data.getVar('FILE', d))}" FILESPATHBASE = "${FILE_DIRNAME}" +FILESPATHBASE[type] = "list" +FILESPATHBASE[separator] = ":" FILESPATHPKG = "${PF}:${P}:${PN}:${BP}:${BPN}:files:." +FILESPATHPKG[type] = "list" +FILESPATHPKG[separator] = ":" FILESPATH = "${@':'.join(uniq(os.path.normpath(os.path.join(fp, p, o)) for fp in '${FILESPATHBASE}'.split(':') for p in '${FILESPATHPKG}'.split(':') for o in reversed([''] + filter(None, '${OVERRIDES}'.split(':')))))}" +FILESPATH[type] = "list" +FILESPATH[separator] = ":" FILESDIR = "${@bb.which(d.getVar('FILESPATH', 1), '.')}" ##################################################################
@@ -420,8 +432,11 @@ EXTRA_IMAGEDEPENDS = "" # by default this is unset and should be so for libtools < 2.4 LIBTOOL_HAS_SYSROOT ?= "no" +LIBTOOL_HAS_SYSROOT[type] = "boolean" PATH_prepend = "${STAGING_BINDIR_CROSS}:${STAGING_DIR_NATIVE}${sbindir_native}:${STAGING_BINDIR_NATIVE}:${STAGING_DIR_NATIVE}${base_sbindir_native}:${STAGING_DIR_NATIVE}/${base_bindir_native}:" +PATH[type] = "list" +PATH[separator] = ":" export PATH ##################################################################
@@ -594,6 +609,7 @@ BB_LOCALCOUNT_OVERRIDE = "1" # Value for revision counter in GIT recipes (NNNN in PV="1.0+gitrNNNN+a7401084a72285c8d3485bacf4eed593f303b1b7-r4.4") LOCALCOUNT = "0" +LOCALCOUNT[type] = "integer" SRC_URI = "file://${FILE}"
@@ -647,6 +663,8 @@ export HOSTNAME := "${@os.uname()[1]}" PKG_CONFIG_DIR = "${STAGING_LIBDIR}/pkgconfig" export PKG_CONFIG_PATH = "${STAGING_LIBDIR}/pkgconfig:${STAGING_DATADIR}/pkgconfig" PKG_CONFIG_PATH .= ":${STAGING_LIBDIR_NATIVE}/pkgconfig:${STAGING_DATADIR_NATIVE}/pkgconfig" +PKG_CONFIG_PATH[type] = "list" +PKG_CONFIG_PATH[separator] = ":" export PKG_CONFIG_SYSROOT_DIR = "${STAGING_DIR}/${BASE_PACKAGE_ARCH}${TARGET_VENDOR}-${TARGET_OS}" export PKG_CONFIG_DISABLE_UNINSTALLED = "yes"
@@ -681,6 +699,8 @@ MACHINE_OVERRIDES ?= "" OVERRIDES = "pn-${PN}:fail-fast:build-${BUILD_OS}:${TARGET_ARCH}:${TARGET_OS}:${DISTRO}:\ ${@[':'.join(d.getVar('MACHINE_OVERRIDES', True).split())+':',''][d.getVar('MACHINE_OVERRIDES', True).strip() == '']}\ ${MACHINE}:local" +OVERRIDES[type] = "list" +OVERRIDES[separator] = ":" ################################################################## # Include the rest of the config files.
@@ -714,6 +734,7 @@ PCMCIA_MANAGER ?= "pcmcia-cs" DEFAULT_TASK_PROVIDER ?= "task-base" MACHINE_TASK_PROVIDER ?= "${DEFAULT_TASK_PROVIDER}" IMAGE_ROOTFS_SIZE ?= "65536" +IMAGE_ROOTFS_SIZE[type] = "integer" IMAGE_ROOTFS_SIZE_ext2 ?= "65536" IMAGE_ROOTFS_SIZE_ext2.gz ?= "65536" IMAGE_ROOTFS_SIZE_ext3 ?= "65536"
@@ -726,6 +747,7 @@ CACHE := "${CACHE}" # Default to disabling packaged staging code export PSTAGING_ACTIVE = "0" +PSTAGING_ACTIVE[type] = "boolean" ################################################################## # Magic Cookie for SANITY CHECK
@@ -743,14 +765,21 @@ DISTRO_FEATURES += ' ${@["", "thumb-interwork"][bb.data.getVar('THUMB_INTERWORK' # This is used to limit what packages goes into images built, so set big by default ROOT_FLASH_SIZE ?= "256" +ROOT_FLASH_SIZE[type] = "integer" MACHINE_GUI_CLASS ?= "smallscreen" +MACHINE_GUI_CLASS[type] = "choice" +MACHINE_GUI_CLASS[choices] = "bigscreen smallscreen" # GUI_MACHINE_CLASS is deprecated, please use MACHINE_GUI_CLASS instead GUI_MACHINE_CLASS ?= "${MACHINE_GUI_CLASS}" MACHINE_DISPLAY_WIDTH_PIXELS ?= "240" +MACHINE_DISPLAY_WIDTH_PIXELS[type] = "integer" MACHINE_DISPLAY_HEIGHT_PIXELS ?= "320" +MACHINE_DISPLAY_HEIGHT_PIXELS[type] = "integer" MACHINE_DISPLAY_ORIENTATION ?= "0" +MACHINE_DISPLAY_ORIENTATION[type] = "boolean" MACHINE_DISPLAY_BPP ?= "16" +MACHINE_DISPLAY_BPP[type] = "integer" DISTRO_EXTRA_RDEPENDS ?= "" DISTRO_EXTRA_RRECOMMENDS ?= ""
@@ -759,6 +788,9 @@ MACHINE_EXTRA_RRECOMMENDS ?= "" MACHINE_ESSENTIAL_EXTRA_RDEPENDS ?= "" MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS ?= "" +DISTRO_FEATURES[type] = "list" +MACHINE_FEATURES[type] = "list" + COMBINED_FEATURES = "\ ${@base_both_contain("DISTRO_FEATURES", "MACHINE_FEATURES", "alsa", d)} \ ${@base_both_contain("DISTRO_FEATURES", "MACHINE_FEATURES", "bluetooth", d)} \
@@ -789,6 +821,7 @@ COMBINED_FEATURES += "${@base_ifelse( \ (base_contains('MACHINE_FEATURES', 'pcmcia', True, False, d) and \ base_contains('DISTRO_FEATURES', 'pcmcia', True, False, d))), \ 'hostap', '')}" +COMBINED_FEATURES[type] = "list" # Make sure MACHINE isn't exported # (breaks binutils at least)
Signature · 1 line
1.7.2.3
Quoted 46 lines · Chris Larson
This implementation consists of two components:
- oe.types python module, whose job it is to construct objects of the defined
type for a given variable in the metadata
- typecheck.bbclass, which iterates over all configuration variables with a type defined and uses oe.types to check the validity of the valuesThis gives us a few benefits:
- Automatic sanity checking of all configuration variables with a defined type
- Avoid duplicating the "how do I make use of the value of this variable" logic between its users. For variables like PATH, this is simply asplit(),
for boolean variables, the duplication can result in confusing, or even mismatched semantics (is this 0/1, empty/nonempty, what?)
- Make it easier to create a configuration UI, as the type information could
be used to provide a better interface than a text edit box (e.g checkbox
for
'boolean', dropdown for 'choice')This functionality is entirely opt-in right now. To enable the configuration
variable type checking, simply INHERIT += "typecheck". Example of a failing
type check:BAZ = "foo"
BAZ[type] = "boolean"$ bitbake -p FATAL: BAZ: Invalid boolean value 'foo' $Examples of leveraging oe.types in a python snippet:
PACKAGES[type] = "list"
python () {
import oe.types
for pkg inoe.types.value("PACKAGES", d):bb.note("package: %s" % pkg) }
Quoted 8 lines
LIBTOOL_HAS_SYSROOT= "yes"
LIBTOOL_HAS_SYSROOT[type] = "boolean"python () { import oe.types assert(oe.types.value("LIBTOOL_HAS_SYSROOT", d) == True) }
See https://github.com/kergoth/openembedded/compare/master...typing for the current version, just reworked to decouple the types from the metadata to make their unit testing a bit cleaner. Note that the __new__ use for Choice and Regex is pretty ugly, really only using those classes as a container to make setting the name and flags attributes on the callable less ugly :P
Comments welcome.
Signature · 5 lines
Christopher Larson clarson at kergoth dot com Founder - BitBake, OpenEmbedded, OpenZaurus Maintainer - Tslib Senior Software Engineer, Mentor Graphics
Quoted 7 lines · Chris Larson
See https://github.com/kergoth/openembedded/compare/master...typing for the current version, just reworked to decouple the types from the metadata to make their unit testing a bit cleaner. Note that the
__new__use for Choice and Regex is pretty ugly, really only using those classes as a container to make setting the name and flags attributes on the callable less ugly :P Comments welcome.
Fixed. I'm fairly happy with that branch now, so am going to stop messing with it, other than adding more type flags to the metadata. Let me know what you think.
Signature · 5 lines
Christopher Larson clarson at kergoth dot com Founder - BitBake, OpenEmbedded, OpenZaurus Maintainer - Tslib Senior Software Engineer, Mentor Graphics