Skip to contentSkip to conversation

Add barebox bootloader support (and testing)

PATCH Superseded Enrico Jörns 3 participants 15 messages Sep 19, 2024 – Oct 9, 2024

v7 of 9 Superseded Latest is v9 →

lore
Share & tools
lore
#0 lore raw

v7 cover Add barebox bootloader support (and testing)

This adds support for the barebox bootloader (and tools) to oe-core.

In order to have proper testing, this extends oe-selftest to allow basic testing of bootloaders. While at it, cover both u-boot and barebox.

v7:
* add missing HOMEPAGE to barebox recipe
* ignore barebox for maintainers check since the recipe is skipped during parsing (because of PREFERRED_PROVIDER mechanism).

Enrico Jorns (5):
barebox: set default BAREBOX_CONFIG for qemu machines oeqa/utils/qemurunner: support ignoring vt100 escape sequences
oeqa: support passing custom boot patterns to runqemu oeqa/selftest/cases: add basic u-boot test oeqa/selftest/cases: add basic barebox tests

Marco Felsch (2):
barebox: add initial support
barebox-tools: add initial barebox tools support

 meta/classes-recipe/barebox.bbclass           | 160 ++++++++++++++++++
 meta/conf/distro/include/maintainers.inc      |   2 +
 meta/conf/machine/qemuarm.conf                |   1 +
 meta/conf/machine/qemuarm64.conf              |   1 +
 meta/conf/machine/qemux86-64.conf             |   1 +
 meta/lib/oeqa/selftest/cases/barebox.py       |  43 +++++
 meta/lib/oeqa/selftest/cases/distrodata.py    |   2 +-
 meta/lib/oeqa/selftest/cases/uboot.py         |  42 +++++
 meta/lib/oeqa/targetcontrol.py                |   5 +-
 meta/lib/oeqa/utils/commands.py               |   5 +-
 meta/lib/oeqa/utils/qemurunner.py             |   6 +-
 .../barebox/barebox-tools_2024.08.0.bb        |  59 +++++++
 meta/recipes-bsp/barebox/barebox_2024.08.0.bb |  15 ++
 13 files changed, 335 insertions(+), 7 deletions(-)
 create mode 100644 meta/classes-recipe/barebox.bbclass
 create mode 100644 meta/lib/oeqa/selftest/cases/barebox.py
 create mode 100644 meta/lib/oeqa/selftest/cases/uboot.py
 create mode 100644 meta/recipes-bsp/barebox/barebox-tools_2024.08.0.bb
 create mode 100644 meta/recipes-bsp/barebox/barebox_2024.08.0.bb
Signature · 1 line
2.39.5
#1 lore raw

PATCH v7 1/7 barebox: add initial support

From: Marco Felsch <[email protected]>

This adds the support for the barebox bootloader to oe-core. The recipe is inspired by meta-ptx [1] but is a major rework of the one found there.

Barebox comes with a wide range of supported architectures and follows the concepts of Linux in various aspects like the driver model, the shell, or the virtual file system.
This not only eases porting Linux drivers but also makes barebox a developer-friendly and feature-rich bootloader alternative [2].

For barebox (like for the kernel or other bootloaders) it is quite likely that people will not just build the original recipe but need to adapt it, point to custom repositories, apply patch stacks, COMPATIBLE_MACHINE etc. They may also choose to have different recipe names for different variants.
Having only a single .bb file and requiring to copy or .bbappend it is inconvenient and results in unnecessary code duplication. Therefore, the base support for building barebox is encapsulated in barebox.bbclass (like kernel.bbclass for the kernel).

Adds barebox to maintainers.inc but excludes it from the maintainers check since with the current check mechanism barebox would be skipped (and making the check fail) due to not being the PREFERRED_PROVIDER for virtual/bootloader.

[1] https://github.com/pengutronix/meta-ptx/tree/master/recipes-bsp/barebox
[2] https://www.barebox.org/demo/?graphic=0

---
 meta/classes-recipe/barebox.bbclass           | 160 ++++++++++++++++++
 meta/conf/distro/include/maintainers.inc      |   1 +
 meta/lib/oeqa/selftest/cases/distrodata.py    |   2 +-
 meta/recipes-bsp/barebox/barebox_2024.08.0.bb |  15 ++
 4 files changed, 177 insertions(+), 1 deletion(-)
 create mode 100644 meta/classes-recipe/barebox.bbclass
 create mode 100644 meta/recipes-bsp/barebox/barebox_2024.08.0.bb
meta/classes-recipe/barebox.bbclass new file +160 -0
new file mode 100644
index 0000000000..200ba08326
@@ -0,0 +1,160 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+inherit kernel-arch deploy cml1 pkgconfig
+
+LICENSE ?= "GPL-2.0-only"
+
+PROVIDES += "virtual/bootloader"
+
+PACKAGE_ARCH = "${MACHINE_ARCH}"
+
+DEPENDS += "bison-native flex-native"
+
+S = "${WORKDIR}/barebox-${PV}"
+B = "${WORKDIR}/build"
+
+require conf/image-uefi.conf
+
+# For some platforms and configuration, the barebox build process will require
+# additional host tools that can be activated/deactivated here.
+PACKAGECONFIG ??= "openssl libusb fit"
+
+PACKAGECONFIG[openssl] = ",,openssl-native"
+PACKAGECONFIG[libusb] = ",,libusb1-native"
+PACKAGECONFIG[fit] = ",,u-boot-tools-native dtc-native"
+
+export KBUILD_BUILD_USER ?= "oe-user"
+export KBUILD_BUILD_HOST ?= "oe-host"
+
+# unlike the kernel, barebox may build against host tools like openssl
+export HOST_EXTRACFLAGS
+
+def get_layer_rev(path):
+    try:
+        rev, _ = bb.process.run("git describe --match='' --always --dirty --broken", cwd=path)
+    except bb.process.ExecutionError:
+        rev = ""
+    return rev.strip()
+
+BAREBOX_BUILDSYSTEM_VERSION[doc] = "Build system version to add to the barebox image. By default this is the git description of the containing layer."
+BAREBOX_BUILDSYSTEM_VERSION ??= "${@get_layer_rev(os.path.dirname(d.getVar('FILE')))}"
+
+BAREBOX_FIRMWARE_DIR[doc] = "Overwrite barebox' firmware blobs search directory (CONFIG_EXTRA_FIRMWARE_DIR) with this path, default ${B}/firmware"
+BAREBOX_FIRMWARE_DIR ??= "${B}/firmware"
+
+EXTRA_OEMAKE = " \
+    CROSS_COMPILE=${TARGET_PREFIX} -C ${S} O=${B} \
+    BUILDSYSTEM_VERSION=${BAREBOX_BUILDSYSTEM_VERSION} \
+    CONFIG_EXTRA_FIRMWARE_DIR=${BAREBOX_FIRMWARE_DIR} \
+    PKG_CONFIG=pkg-config-native \
+    CROSS_PKG_CONFIG=pkg-config \
+"
+
+BAREBOX_CONFIG[doc] = "The barebox kconfig defconfig file. Not used if a file called defconfig is added to the SRC_URI."
+BAREBOX_CONFIG ?= ""
+
+# set sensible default configs for some of oe-core's QEMU MACHINEs
+BAREBOX_CONFIG:qemuarm = "multi_v7_defconfig"
+BAREBOX_CONFIG:qemuarm64 = "multi_v8_defconfig"
+BAREBOX_CONFIG:qemux86-64 = "efi_defconfig"
+
+# prevent from acting as non-buildable provider
+python () {
+    bareboxconfig = d.getVar('BAREBOX_CONFIG')
+    bareboxdefconfig = 'file://defconfig' in d.getVar('SRC_URI')
+
+    if not bareboxconfig and not bareboxdefconfig:
+        raise bb.parse.SkipRecipe("BAREBOX_CONFIG must be set in the %s machine configuration or file://defconfig must be given in SRC_URI." % d.getVar("MACHINE"))
+}
+
+barebox_do_configure() {
+        if [ -e ${UNPACKDIR}/defconfig ]; then
+                cp ${UNPACKDIR}/defconfig ${B}/.config
+        else
+                if [ -n "${BAREBOX_CONFIG}" ]; then
+                        oe_runmake ${BAREBOX_CONFIG}
+                else
+                        bbfatal "No defconfig given. Either add file 'file://defconfig' to SRC_URI or set BAREBOX_CONFIG"
+                fi
+        fi
+
+        ${S}/scripts/kconfig/merge_config.sh -m .config ${@" ".join(find_cfgs(d))}
+        cml1_do_configure
+}
+
+BAREBOX_ENV_DIR[doc] = "Overlay the barebox built-in environment with the environment provided by the BSP if specified."
+BAREBOX_ENV_DIR ??= "${UNPACKDIR}/env/"
+
+barebox_do_compile () {
+        export userccflags="${TARGET_LDFLAGS}${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
+        unset LDFLAGS
+        unset CFLAGS
+        unset CPPFLAGS
+        unset CXXFLAGS
+        unset MACHINE
+        # Allow to use ${UNPACKDIR} in kconfig options to include additionally fetched files
+        export UNPACKDIR=${UNPACKDIR}
+
+        if [ -d ${BAREBOX_ENV_DIR} ]; then
+                BAREBOX_DEFAULT_ENV="$(grep ^CONFIG_DEFAULT_ENVIRONMENT_PATH .config | cut -d '=' -f 2 | tr -d '"')"
+                oe_runmake CONFIG_DEFAULT_ENVIRONMENT_PATH="\"${BAREBOX_DEFAULT_ENV} ${BAREBOX_ENV_DIR}\""
+        else
+                oe_runmake
+        fi
+}
+
+BAREBOX_BINARY[doc] = "Specify the barebox binary to install. If not specified all barebox artifacts are installed."
+BAREBOX_BINARY ??= "${@'barebox.efi' if d.getVar('EFI_PROVIDER') == 'barebox' else ''}"
+BAREBOX_SUFFIX[doc] = "Specify the suffix for ${BAREBOX_IMAGE}."
+BAREBOX_SUFFIX ??= "img"
+BAREBOX_IMAGE[doc] = "A unique barebox image name. Unused if ${BAREBOX_BINARY} is not set."
+BAREBOX_IMAGE_DEFAULT ?= "${PN}-${MACHINE}-${PV}-${PR}.${BAREBOX_SUFFIX}"
+BAREBOX_IMAGE ?= "${@'${EFI_BOOT_IMAGE}' if d.getVar('EFI_PROVIDER') == 'barebox' else '${BAREBOX_IMAGE_DEFAULT}'}"
+
+BAREBOX_INSTALL_PATH ?= "${@'${EFI_FILES_PATH}' if d.getVar('EFI_PROVIDER') == 'barebox' else '/boot'}"
+
+barebox_do_install () {
+        if [ -n "${BAREBOX_BINARY}" ]; then
+
+                BAREBOX_BIN=${B}/${BAREBOX_BINARY}
+                if [ ! -f "${BAREBOX_BIN}" ]; then
+                        BAREBOX_BIN=${B}/images/${BAREBOX_BINARY}
+                fi
+                if [ ! -f "${BAREBOX_BIN}" ]; then
+                        bbfatal "Failed to locate ${BAREBOX_BINARY}"
+                fi
+
+                install -D -m 644 ${BAREBOX_BIN} ${D}${BAREBOX_INSTALL_PATH}/${BAREBOX_IMAGE}
+                ln -sf ${BAREBOX_IMAGE} ${D}${BAREBOX_INSTALL_PATH}/${BAREBOX_BINARY}
+        else
+                install -d ${D}${BAREBOX_INSTALL_PATH}/
+                for image in $(cat ${B}/barebox-flash-images); do
+                        install -m 644 ${B}/${image} ${D}${BAREBOX_INSTALL_PATH}/
+                done
+        fi
+}
+FILES:${PN} = "${BAREBOX_INSTALL_PATH}"
+
+barebox_do_deploy () {
+        if [ -n "${BAREBOX_BINARY}" ]; then
+
+                BAREBOX_BIN=${B}/${BAREBOX_BINARY}
+                if [ ! -f "${BAREBOX_BIN}" ]; then
+                        BAREBOX_BIN=${B}/images/${BAREBOX_BINARY}
+                fi
+
+                install -D -m 644 ${BAREBOX_BIN} ${DEPLOYDIR}/${BAREBOX_IMAGE}
+                ln -sf ${BAREBOX_IMAGE} ${DEPLOYDIR}/${BAREBOX_BINARY}
+        else
+                for image in $(cat ${B}/barebox-flash-images); do
+                        cp ${B}/${image} ${DEPLOYDIR}
+                done
+        fi
+}
+addtask deploy after do_compile
+
+EXPORT_FUNCTIONS do_configure do_compile do_install do_deploy
meta/conf/distro/include/maintainers.inc +1 -0
index 60f36a9449..1b3d485e48 100644
@@ -54,6 +54,7 @@ RECIPE_MAINTAINER:pn-automake = "Robert Yang <[email protected]>"
 RECIPE_MAINTAINER:pn-avahi = "Yi Zhao <[email protected]>"
 RECIPE_MAINTAINER:pn-babeltrace = "Alexander Kanavin <[email protected]>"
 RECIPE_MAINTAINER:pn-babeltrace2 = "Alexander Kanavin <[email protected]>"
+RECIPE_MAINTAINER:pn-barebox = "Enrico Jörns <[email protected]>"
 RECIPE_MAINTAINER:pn-baremetal-helloworld = "Alejandro Hernandez <[email protected]>"
 RECIPE_MAINTAINER:pn-base-files = "Anuj Mittal <[email protected]>"
 RECIPE_MAINTAINER:pn-base-passwd = "Anuj Mittal <[email protected]>"
meta/lib/oeqa/selftest/cases/distrodata.py +1 -1
index 7771a42e2b..1e88ea82e6 100644
@@ -56,7 +56,7 @@ but their recipes claim otherwise by setting UPSTREAM_VERSION_UNKNOWN. Please re
 
         def is_maintainer_exception(entry):
             exceptions = ["musl", "newlib", "picolibc", "linux-yocto", "linux-dummy", "mesa-gl", "libgfortran", "libx11-compose-data",
-                          "cve-update-nvd2-native",]
+                          "cve-update-nvd2-native", "barebox"]
             for i in exceptions:
                  if i in entry:
                      return True
meta/recipes-bsp/barebox/barebox_2024.08.0.bb new file +15 -0
new file mode 100644
index 0000000000..98e8aa053b
@@ -0,0 +1,15 @@
+SUMMARY = "barebox is a bootloader designed for embedded systems. It runs on a variety of architectures including x86, ARM, MIPS, PowerPC and others."
+DESCRIPTION = "barebox aims to be a versatile and flexible bootloader not only for booting embedded Linux systems, \
+but also for initial hardware bringup and development. \
+Users should feel right at home with a shell with UNIX-like virtual file system access to hardware, \
+Linux kernel driver API for making driver porting easier, \
+and a subset of the POSIX C library for writing more command-line utilities."
+HOMEPAGE = "https://barebox.org/"
+SECTION = "bootloaders"
+
+LIC_FILES_CHKSUM = "file://COPYING;md5=f5125d13e000b9ca1f0d3364286c4192"
+
+inherit barebox
+
+SRC_URI = "https://barebox.org/download/barebox-${PV}.tar.bz2"
+SRC_URI[sha256sum] = "b08a762da8d63dd18b4f2d9f5d0a8da001b6e608d1b3eff6dcebc6a2e575d535"
Signature · 1 line
2.39.5
#2 lore raw

PATCH v7 2/7 barebox-tools: add initial barebox tools support

In reply to Enrico Jörns

From: Marco Felsch <[email protected]>

Add initial support to build the barebox tools for the host and the target.

Some of the tools are generic barebox utilities (like 'bareboximd' for image meta data inspection or 'bareboxenv' for accessing the barebox environment) some are SoC family-specific specific utilities (like 'imx-usb-loader' or 'omap3-usb-loader').

---
 meta/conf/distro/include/maintainers.inc      |  1 +
 .../barebox/barebox-tools_2024.08.0.bb        | 59 +++++++++++++++++++
 2 files changed, 60 insertions(+)
 create mode 100644 meta/recipes-bsp/barebox/barebox-tools_2024.08.0.bb
meta/conf/distro/include/maintainers.inc +1 -0
index 1b3d485e48..067c4af096 100644
@@ -55,6 +55,7 @@ RECIPE_MAINTAINER:pn-avahi = "Yi Zhao <[email protected]>"
 RECIPE_MAINTAINER:pn-babeltrace = "Alexander Kanavin <[email protected]>"
 RECIPE_MAINTAINER:pn-babeltrace2 = "Alexander Kanavin <[email protected]>"
 RECIPE_MAINTAINER:pn-barebox = "Enrico Jörns <[email protected]>"
+RECIPE_MAINTAINER:pn-barebox-tools = "Enrico Jörns <[email protected]>"
 RECIPE_MAINTAINER:pn-baremetal-helloworld = "Alejandro Hernandez <[email protected]>"
 RECIPE_MAINTAINER:pn-base-files = "Anuj Mittal <[email protected]>"
 RECIPE_MAINTAINER:pn-base-passwd = "Anuj Mittal <[email protected]>"
meta/recipes-bsp/barebox/barebox-tools_2024.08.0.bb new file +59 -0

59 lines not on this page · raw message

Signature · 1 line
2.39.5
#3 lore raw

PATCH v7 3/7 barebox: set default BAREBOX_CONFIG for qemu machines

In reply to Enrico Jörns

From: Enrico Jorns <[email protected]>

These are set in the barebox recipe rather in the corresponding machines (where they would belong otherwise) to keep the impact of barebox to oe-core minimal for now.

"multi_v7_defconfig" is used for qemuarm since this is the default barebox armv7 config that just enables all supported platforms.

"multi_v8_defconfig" is used for qemuarm64 sine this is the default barebox armv8 config that just enables all supported platforms.

"efi_defconfig" is used for qemux86-64 which is the primary platform where barebox will not be the first stage bootloader but an EFI payload.

Since these changes make barebox a provider for virtual/bootloader, explicitly default to u-boot in the corresponding MACHINE configs to not unnecessarily surprise users.

---
 meta/conf/machine/qemuarm.conf    | 1 +
 meta/conf/machine/qemuarm64.conf  | 1 +
 meta/conf/machine/qemux86-64.conf | 1 +
 3 files changed, 3 insertions(+)
meta/conf/machine/qemuarm.conf +1 -0
index 943ce7c16a..af76a53b8e 100644
@@ -7,6 +7,7 @@ require conf/machine/include/qemu.inc
 
 KERNEL_IMAGETYPE = "zImage"
 
+PREFERRED_PROVIDER_virtual/bootloader ?= "u-boot"
 UBOOT_MACHINE ?= "qemu_arm_defconfig"
 
 SERIAL_CONSOLES ?= "115200;ttyAMA0 115200;hvc0"
meta/conf/machine/qemuarm64.conf +1 -0
index a096d964db..d310445a34 100644
@@ -7,6 +7,7 @@ require conf/machine/include/qemu.inc
 
 KERNEL_IMAGETYPE = "Image"
 
+PREFERRED_PROVIDER_virtual/bootloader ?= "u-boot"
 UBOOT_MACHINE ?= "qemu_arm64_defconfig"
 
 SERIAL_CONSOLES ?= "115200;ttyAMA0 115200;hvc0"
meta/conf/machine/qemux86-64.conf +1 -0
index 14873a3b4f..4a9c6d364b 100644
@@ -13,6 +13,7 @@ DEFAULTTUNE ?= "core2-64"
 require conf/machine/include/x86/tune-x86-64-v3.inc
 require conf/machine/include/x86/qemuboot-x86.inc
 
+PREFERRED_PROVIDER_virtual/bootloader ?= "u-boot"
 UBOOT_MACHINE ?= "qemu-x86_64_defconfig"
 
 KERNEL_IMAGETYPE = "bzImage"
Signature · 1 line
2.39.5
#4 lore raw

PATCH v7 4/7 oeqa/utils/qemurunner: support ignoring vt100 escape sequences

In reply to Enrico Jörns

From: Enrico Jorns <[email protected]>

If we talk to terminals that like colors, we need to ignore the vt100 escape sequences when matching strings.

An unprocessed barebox console prompt would e.g. look like:

ESC[1;32mbarebox@ESC[1;36mARM QEMU virt64:/ESC[0m

where we cannot match for something like "barebox@ARM QEMU virt64:/". The same applies to colored Linux terminal output of course.

The "\x1b\[" from the regex catches the standard start of ANSI escape sequence while the rest catches the actual command code executed.

---
 meta/lib/oeqa/utils/qemurunner.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
meta/lib/oeqa/utils/qemurunner.py +3 -1
index cda43aad8c..63fc6f6b53 100644
@@ -30,6 +30,8 @@ control_range = list(range(0,32))+list(range(127,160))
 control_chars = [chr(x) for x in control_range
                 if chr(x) not in string.printable]
 re_control_char = re.compile('[%s]' % re.escape("".join(control_chars)))
+# Regex to remove the ANSI (color) control codes from console strings in order to match the text only
+re_vt100 = re.compile(r'(\x1b\[|\x9b)[^@-_a-z]*[@-_a-z]|\x1b[@-_a-z]')
 
 def getOutput(o):
     import fcntl
@@ -681,7 +683,7 @@ class QemuRunner:
                     time.sleep(0.1)
                     answer = self.server_socket.recv(1024)
                     if answer:
-                        data += answer.decode('utf-8')
+                        data += re_vt100.sub("", answer.decode('utf-8'))
                         # Search the prompt to stop
                         if re.search(self.boot_patterns['search_cmd_finished'], data):
                             break
Signature · 1 line
2.39.5
#5 lore raw

PATCH v7 5/7 oeqa: support passing custom boot patterns to runqemu

In reply to Enrico Jörns

From: Enrico Jorns <[email protected]>

This allows defining non-standard patterns from QEMU tests just as they are already supported by the testimage.bbclass.

Will allow testing non-Linux shells in QEMU, too (e.g. a U-Boot shell or another bootloader shell).

---
 meta/lib/oeqa/targetcontrol.py    | 5 +++--
 meta/lib/oeqa/utils/commands.py   | 5 +++--
 meta/lib/oeqa/utils/qemurunner.py | 2 +-
 3 files changed, 7 insertions(+), 5 deletions(-)
meta/lib/oeqa/targetcontrol.py +3 -2
index 6e8b781973..cdf382ee21 100644
@@ -88,7 +88,7 @@ class QemuTarget(BaseTarget):
 
     supported_image_fstypes = ['ext3', 'ext4', 'cpio.gz', 'wic']
 
-    def __init__(self, d, logger, image_fstype=None):
+    def __init__(self, d, logger, image_fstype=None, boot_patterns=None):
 
         import oe.types
 
@@ -141,7 +141,8 @@ class QemuTarget(BaseTarget):
                             dump_dir = dump_dir,
                             logger = logger,
                             tmpfsdir = d.getVar("RUNQEMU_TMPFS_DIR"),
-                            serial_ports = len(d.getVar("SERIAL_CONSOLES").split()))
+                            serial_ports = len(d.getVar("SERIAL_CONSOLES").split()),
+                            boot_patterns = boot_patterns)
 
         self.monitor_dumper = MonitorDumper(dump_monitor_cmds, dump_dir, self.runner)
         if (self.monitor_dumper):
meta/lib/oeqa/utils/commands.py +3 -2
index bf2f49d0c0..ca22d69f29 100644
@@ -312,7 +312,7 @@ def create_temp_layer(templayerdir, templayername, priority=999, recipepathspec=
         f.write('LAYERSERIES_COMPAT_%s = "%s"\n' % (templayername, corenames))
 
 @contextlib.contextmanager
-def runqemu(pn, ssh=True, runqemuparams='', image_fstype=None, launch_cmd=None, qemuparams=None, overrides={}, discard_writes=True):
+def runqemu(pn, ssh=True, runqemuparams='', image_fstype=None, launch_cmd=None, qemuparams=None, overrides={}, boot_patterns = {}, discard_writes=True):
     """
     Starts a context manager for a 'oeqa.targetcontrol.QemuTarget' resource.
     The underlying Qemu will be booted into a shell when the generator yields
@@ -330,6 +330,7 @@ def runqemu(pn, ssh=True, runqemuparams='', image_fstype=None, launch_cmd=None,
         image_fstype (str): IMAGE_FSTYPE to use
         launch_cmd (str): directly run this command and bypass automatic runqemu parameter generation
         overrides (dict): dict of "'<bitbake-variable>': value" pairs that allows overriding bitbake variables
+        boot_patterns (dict): dict of "'<pattern-name>': value" pairs to override default boot patterns, e.g. when not booting Linux
         discard_writes (boolean): enables qemu -snapshot feature to prevent modifying original image
     """
 
@@ -361,7 +362,7 @@ def runqemu(pn, ssh=True, runqemuparams='', image_fstype=None, launch_cmd=None,
 
         logdir = recipedata.getVar("TEST_LOG_DIR")
 
-        qemu = oeqa.targetcontrol.QemuTarget(recipedata, targetlogger, image_fstype)
+        qemu = oeqa.targetcontrol.QemuTarget(recipedata, targetlogger, image_fstype, boot_patterns=boot_patterns)
     finally:
         # We need to shut down tinfoil early here in case we actually want
         # to run tinfoil-using utilities with the running QEMU instance.
meta/lib/oeqa/utils/qemurunner.py +1 -1
index 63fc6f6b53..5c3a8e5999 100644
@@ -103,7 +103,7 @@ class QemuRunner:
 
         # Only override patterns that were set e.g. login user TESTIMAGE_BOOT_PATTERNS[send_login_user] = "webserver\n"
         for pattern in accepted_patterns:
-            if not self.boot_patterns[pattern]:
+            if pattern not in self.boot_patterns or not self.boot_patterns[pattern]:
                 self.boot_patterns[pattern] = default_boot_patterns[pattern]
 
     def create_socket(self):
Signature · 1 line
2.39.5
#6 lore raw

PATCH v7 6/7 oeqa/selftest/cases: add basic u-boot test

In reply to Enrico Jörns

From: Enrico Jorns <[email protected]>

This adds basic tests for qemuarm and qemuarm64.

---
 meta/lib/oeqa/selftest/cases/uboot.py | 42 +++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/cases/uboot.py
meta/lib/oeqa/selftest/cases/uboot.py new file +42 -0
new file mode 100644
index 0000000000..936dfa655f
@@ -0,0 +1,42 @@
+# Qemu-based u-boot bootloader integration testing
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import bitbake, runqemu
+from oeqa.core.decorator.data import skipIfNotArch
+from oeqa.core.decorator import OETestTag
+
+uboot_boot_patterns = {
+        'search_reached_prompt': "stop autoboot",
+        'search_login_succeeded': "=>",
+        'search_cmd_finished': "=>"
+        }
+
+
+class UBootTest(OESelftestTestCase):
+
+    @skipIfNotArch(['arm', 'aarch64'])
+    @OETestTag("runqemu")
+    def test_boot_uboot(self):
+        """
+        Tests building u-boot and booting it with QEMU
+        """
+
+        self.write_config("""
+QB_DEFAULT_BIOS = "u-boot.bin"
+PREFERRED_PROVIDER_virtual/bootloader = "u-boot"
+""")
+        bitbake("virtual/bootloader core-image-minimal")
+
+        with runqemu('core-image-minimal', ssh=False, runqemuparams='nographic',
+                     boot_patterns=uboot_boot_patterns) as qemu:
+
+            # test if u-boot console works
+            cmd = "version"
+            status, output = qemu.run_serial(cmd)
+            self.assertEqual(status, 1, msg=output)
+            self.assertTrue("U-Boot" in output, msg=output)
Signature · 1 line
2.39.5
#7 lore raw

PATCH v7 7/7 oeqa/selftest/cases: add basic barebox tests

In reply to Enrico Jörns

From: Enrico Jorns <[email protected]>

This adds basic tests for qemuarm and qemuarm64.

---
 meta/lib/oeqa/selftest/cases/barebox.py | 43 +++++++++++++++++++++++++
 1 file changed, 43 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/cases/barebox.py
meta/lib/oeqa/selftest/cases/barebox.py new file +43 -0
new file mode 100644
index 0000000000..773cf6a1c8
@@ -0,0 +1,43 @@
+# Qemu-based barebox bootloader integration testing
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import bitbake, runqemu
+from oeqa.core.decorator.data import skipIfNotArch
+from oeqa.core.decorator import OETestTag
+
+barebox_boot_patterns = {
+        'search_reached_prompt': r"stop autoboot",
+        'search_login_succeeded': r"barebox@[^:]+:[^ ]+ ",
+        'search_cmd_finished': r"barebox@[a-zA-Z0-9\-\s]+:/"
+        }
+
+
+class BareboxTest(OESelftestTestCase):
+
+    @skipIfNotArch(['arm', 'aarch64'])
+    @OETestTag("runqemu")
+    def test_boot_barebox(self):
+        """
+        Tests building barebox and booting it with QEMU
+        """
+
+        self.write_config("""
+QB_DEFAULT_KERNEL = "barebox-dt-2nd.img"
+PREFERRED_PROVIDER_virtual/bootloader = "barebox"
+""")
+
+        bitbake("virtual/bootloader core-image-minimal")
+
+        with runqemu('core-image-minimal', ssh=False, runqemuparams='nographic',
+                     boot_patterns=barebox_boot_patterns) as qemu:
+
+            # test if barebox console works
+            cmd = "version"
+            status, output = qemu.run_serial(cmd)
+            self.assertEqual(status, 1, msg=output)
+            self.assertTrue("barebox" in output, msg=output)
Signature · 1 line
2.39.5
#8 lore raw
In reply to Enrico Jörns
Quoted 18 lines · Enrico Jörns

This adds support for the barebox bootloader (and tools) to oe-core.

In order to have proper testing, this extends oe-selftest to allow basic testing of bootloaders. While at it, cover both u-boot and barebox.

v7:
* add missing HOMEPAGE to barebox recipe
* ignore barebox for maintainers check since the recipe is skipped during parsing (because of PREFERRED_PROVIDER mechanism).

Enrico Jorns (5):
  barebox: set default BAREBOX_CONFIG for qemu machines oeqa/utils/qemurunner: support ignoring vt100 escape sequences
  oeqa: support passing custom boot patterns to runqemu oeqa/selftest/cases: add basic u-boot test oeqa/selftest/cases: add basic barebox tests

Quoted 3 lines

Marco Felsch (2):
  barebox: add initial support
  barebox-tools: add initial barebox tools support

This looked better in testing, thanks. There was:

https://valkyrie.yoctoproject.org/#/builders/23/builds/168/steps/14/logs/stdio

which is from oe-selftest on an ARM server though :/

Cheers,

Richard

#9 lore raw

Am Donnerstag, dem 19.09.2024 um 21:56 +0100 schrieb Richard Purdie:

Quoted 23 lines · Enrico Jörns
Quoted 18 lines

This adds support for the barebox bootloader (and tools) to oe-core.

In order to have proper testing, this extends oe-selftest to allow basic testing of bootloaders. While at it, cover both u-boot and barebox.

v7:
* add missing HOMEPAGE to barebox recipe
* ignore barebox for maintainers check since the recipe is skipped during parsing (because of PREFERRED_PROVIDER mechanism).

Enrico Jorns (5):
  barebox: set default BAREBOX_CONFIG for qemu machines oeqa/utils/qemurunner: support ignoring vt100 escape sequences
  oeqa: support passing custom boot patterns to runqemu oeqa/selftest/cases: add basic u-boot test oeqa/selftest/cases: add basic barebox tests

Quoted 3 lines

Marco Felsch (2):
  barebox: add initial support
  barebox-tools: add initial barebox tools support

Quoted 5 lines

This looked better in testing, thanks. There was:

https://valkyrie.yoctoproject.org/#/builders/23/builds/168/steps/14/logs/stdio

which is from oe-selftest on an ARM server though :/

Thank you for the test log!

I'll try to figure out why it works in my test setup but doesn't in the autobuilder.. Not sure if I'll have the chance to reproduce it on an ARM host.

Barebox doesn't seem to boot at all (or output is missing) and U-Boot got stuck.

U-Boot log in the failed test says

Quoted 10 lines

U-Boot 2024.07 (Jul 01 2024 - 18:07:18 +0000)

DRAM:  256 MiB
Core:  51 devices, 14 uclasses, devicetree: board
Flash: 64 MiB
Loading Environment from Flash... *** Warning - bad CRC, using default environment
In:    serial,usbkbd
Out:   serial,vidconsole
Err:   serial,vidconsole
Bus xhci_pci: Register 8001040 NbrPorts 8

Starting the controller

While my successful run goes further:

Quoted 7 lines

[...]
Starting the controller
USB XHCI 1.00
scanning bus xhci_pci for devices... 3 USB Device(s) found
Net: eth0: virtio-net#32
ESC[?25h
Hit any key to stop autoboot: 2

But not sure if this is an issue with USB emulation or something else...

Regards, Enrico

Quoted 4 lines

Cheers,

Richard

Signature · 4 lines
Pengutronix e.K.                           | Enrico Jörns                |
Embedded Linux Consulting & Support        | https://www.pengutronix.de/ |
Steuerwalder Str. 21                       | Phone: +49-5121-206917-180  |
31137 Hildesheim, Germany                  | Fax:   +49-5121-206917-9    |
#10 lore raw

Hi,

Quoted 10 lines · Enrico Jörns

Am Donnerstag, dem 19.09.2024 um 21:56 +0100 schrieb Richard Purdie:

Quoted 5 lines

This looked better in testing, thanks. There was:

https://valkyrie.yoctoproject.org/#/builders/23/builds/168/steps/14/logs/stdio

which is from oe-selftest on an ARM server though :/

Thank you for the test log!>
I'll try to figure out why it works in my test setup but doesn't in the autobuilder.. Not sure if I'll have the chance to reproduce it on an ARM host.

I tried reproducing the breakage on a Librecomputer Alta (Cortex-A73/A53) with Debian 12 for arm64, but without success. Details below.

I applied this series on top of commit

4fc8427a6c ("systemd: make home directory readable by systemd-coredump")

Quoted 1 line

Barebox doesn't seem to boot at all (or output is missing) and U-Boot got stuck.

Both barebox' oe-selftest:

  a3f@hass:~/poky/build$ oe-selftest -r barebox -v -v -v -K
  [1] 257856
  2024-09-27 15:06:52,672 - oe-selftest - INFO - Adding layer libraries:
  2024-09-27 15:06:52,672 - oe-selftest - INFO -  /home/a3f/poky/meta/lib
  2024-09-27 15:06:52,673 - oe-selftest - INFO -  /home/a3f/poky/meta-yocto-bsp/lib
  2024-09-27 15:06:52,674 - oe-selftest - INFO -  /home/a3f/poky/meta-selftest/lib
  2024-09-27 15:06:52,680 - oe-selftest - INFO - Checking base configuration is valid/parsable
  NOTE: Starting bitbake server...
  2024-09-27 15:06:59,982 - oe-selftest - INFO - Adding: "include selftest.inc" in /home/a3f/poky/build-st/conf/local.conf
  2024-09-27 15:06:59,983 - oe-selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf
  2024-09-27 15:06:59,985 - oe-selftest - INFO - test_boot_barebox (barebox.BareboxTest.test_boot_barebox)
  2024-09-27 15:07:00,024 - oe-selftest - DEBUG - Writing to: /home/a3f/poky/build-st/conf/selftest.inc

QB_DEFAULT_KERNEL = "barebox-dt-2nd.img" PREFERRED_PROVIDER_virtual/bootloader = "barebox"

  2024-09-27 15:23:54,349 - oe-selftest - INFO -  ... ok
  2024-09-27 15:23:54,357 - oe-selftest - INFO - ----------------------------------------------------------------------
  2024-09-27 15:23:54,358 - oe-selftest - INFO - Ran 1 test in 1019.863s
  2024-09-27 15:23:54,359 - oe-selftest - INFO - OK
  2024-09-27 15:24:08,447 - oe-selftest - INFO - RESULTS:
  2024-09-27 15:24:08,450 - oe-selftest - INFO - RESULTS - barebox.BareboxTest.test_boot_barebox: PASSED (1014.37s)
  2024-09-27 15:24:08,513 - oe-selftest - INFO - SUMMARY:
  2024-09-27 15:24:08,514 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 1019.866s
  2024-09-27 15:24:08,515 - oe-selftest - INFO - oe-selftest - OK - All required tests passed (successes=1, skipped=0, failures=0, errors=0)

and U-Boot's conclude successfully:

  a3f@hass:~/poky/build$ oe-selftest -r uboot -v -v -v -K
  2024-09-30 10:41:21,225 - oe-selftest - INFO - Adding layer libraries:
  2024-09-30 10:41:21,225 - oe-selftest - INFO -  /home/a3f/poky/meta/lib
  2024-09-30 10:41:21,225 - oe-selftest - INFO -  /home/a3f/poky/meta-yocto-bsp/lib
  2024-09-30 10:41:21,226 - oe-selftest - INFO -  /home/a3f/poky/meta-selftest/lib
  2024-09-30 10:41:21,229 - oe-selftest - INFO - Checking base configuration is valid/parsable
  NOTE: Starting bitbake server...
  2024-09-30 10:41:24,120 - oe-selftest - INFO - Adding: "include selftest.inc" in /home/a3f/poky/build-st/conf/local.conf
  2024-09-30 10:41:24,125 - oe-selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf
  2024-09-30 10:41:24,131 - oe-selftest - INFO - test_boot_uboot (uboot.UBootTest.test_boot_uboot)
  2024-09-30 10:41:24,214 - oe-selftest - DEBUG - Writing to: /home/a3f/poky/build-st/conf/selftest.inc

QB_DEFAULT_BIOS = "u-boot.bin" PREFERRED_PROVIDER_virtual/bootloader = "u-boot"

  2024-09-30 10:52:59,680 - oe-selftest - INFO -  ... ok
  2024-09-30 10:52:59,692 - oe-selftest - INFO - ----------------------------------------------------------------------
  2024-09-30 10:52:59,693 - oe-selftest - INFO - Ran 1 test in 696.374s
  2024-09-30 10:52:59,695 - oe-selftest - INFO - OK
  2024-09-30 10:53:13,581 - oe-selftest - INFO - RESULTS:
  2024-09-30 10:53:13,584 - oe-selftest - INFO - RESULTS - uboot.UBootTest.test_boot_uboot: PASSED (695.55s)
  2024-09-30 10:53:13,647 - oe-selftest - INFO - SUMMARY:
  2024-09-30 10:53:13,648 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 696.378s
  2024-09-30 10:53:13,649 - oe-selftest - INFO - oe-selftest - OK - All required tests passed (successes=1, skipped=0, failures=0, errors=0)
Quoted 23 lines

U-Boot log in the failed test says

Quoted 10 lines

U-Boot 2024.07 (Jul 01 2024 - 18:07:18 +0000)

DRAM:  256 MiB
Core:  51 devices, 14 uclasses, devicetree: board
Flash: 64 MiB
Loading Environment from Flash... *** Warning - bad CRC, using default environment
In:    serial,usbkbd
Out:   serial,vidconsole
Err:   serial,vidconsole
Bus xhci_pci: Register 8001040 NbrPorts 8

Starting the controller

While my successful run goes further:

Quoted 7 lines

[...]
Starting the controller
USB XHCI 1.00
scanning bus xhci_pci for devices... 3 USB Device(s) found
Net: eth0: virtio-net#32
ESC[?25h
Hit any key to stop autoboot: 2

Quoted 1 line

But not sure if this is an issue with USB emulation or something else...

My successful run looks identical to yours. barebox start up is also successful and uneventful.

Note that barebox' XHCI driver doesn't have PCI support unlike U-Boot, so if the same issue is affecting both barebox and U-Boot breakage in CI, it might not be USB emulation related as barebox doesn't do any USB on that platform...

Is selftest -r uboot -v -v -v -K close enough to what the CI server runs or should I try something else to reproduce this?

Thanks,
Ahmad

Quoted 10 lines

Regards, Enrico

Quoted 4 lines

Cheers,

Richard

Signature · 4 lines
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
#11 lore raw
Quoted 109 lines · Ahmad Fatoum

Hi,

Quoted 10 lines · Enrico Jörns

Am Donnerstag, dem 19.09.2024 um 21:56 +0100 schrieb Richard Purdie:

Quoted 5 lines

This looked better in testing, thanks. There was:

https://valkyrie.yoctoproject.org/#/builders/23/builds/168/steps/14/logs/stdio

which is from oe-selftest on an ARM server though :/

Thank you for the test log!>
I'll try to figure out why it works in my test setup but doesn't in the autobuilder.. Not sure if I'll have the chance to reproduce it on an ARM host.

I tried reproducing the breakage on a Librecomputer Alta (Cortex-A73/A53) with Debian 12 for arm64, but without success. Details below.

I applied this series on top of commit

4fc8427a6c ("systemd: make home directory readable by systemd-coredump")

Quoted 1 line

Barebox doesn't seem to boot at all (or output is missing) and U-Boot got stuck.

Both barebox' oe-selftest:

  a3f@hass:~/poky/build$ oe-selftest -r barebox -v -v -v -K
  [1] 257856
  2024-09-27 15:06:52,672 - oe-selftest - INFO - Adding layer libraries:
  2024-09-27 15:06:52,672 - oe-selftest - INFO -  /home/a3f/poky/meta/lib
  2024-09-27 15:06:52,673 - oe-selftest - INFO -  /home/a3f/poky/meta-yocto-bsp/lib
  2024-09-27 15:06:52,674 - oe-selftest - INFO -  /home/a3f/poky/meta-selftest/lib
  2024-09-27 15:06:52,680 - oe-selftest - INFO - Checking base configuration is valid/parsable
  NOTE: Starting bitbake server...
  2024-09-27 15:06:59,982 - oe-selftest - INFO - Adding: "include selftest.inc" in /home/a3f/poky/build-st/conf/local.conf
  2024-09-27 15:06:59,983 - oe-selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf
  2024-09-27 15:06:59,985 - oe-selftest - INFO - test_boot_barebox (barebox.BareboxTest.test_boot_barebox)
  2024-09-27 15:07:00,024 - oe-selftest - DEBUG - Writing to: /home/a3f/poky/build-st/conf/selftest.inc
  
  QB_DEFAULT_KERNEL = "barebox-dt-2nd.img"
  PREFERRED_PROVIDER_virtual/bootloader = "barebox"
  
  
  2024-09-27 15:23:54,349 - oe-selftest - INFO -  ... ok
  2024-09-27 15:23:54,357 - oe-selftest - INFO - ----------------------------------------------------------------------
  2024-09-27 15:23:54,358 - oe-selftest - INFO - Ran 1 test in 1019.863s
  2024-09-27 15:23:54,359 - oe-selftest - INFO - OK
  2024-09-27 15:24:08,447 - oe-selftest - INFO - RESULTS:
  2024-09-27 15:24:08,450 - oe-selftest - INFO - RESULTS - barebox.BareboxTest.test_boot_barebox: PASSED (1014.37s)
  2024-09-27 15:24:08,513 - oe-selftest - INFO - SUMMARY:
  2024-09-27 15:24:08,514 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 1019.866s
  2024-09-27 15:24:08,515 - oe-selftest - INFO - oe-selftest - OK - All required tests passed (successes=1, skipped=0, failures=0, errors=0)

and U-Boot's conclude successfully:

  a3f@hass:~/poky/build$ oe-selftest -r uboot -v -v -v -K
  2024-09-30 10:41:21,225 - oe-selftest - INFO - Adding layer libraries:
  2024-09-30 10:41:21,225 - oe-selftest - INFO -  /home/a3f/poky/meta/lib
  2024-09-30 10:41:21,225 - oe-selftest - INFO -  /home/a3f/poky/meta-yocto-bsp/lib
  2024-09-30 10:41:21,226 - oe-selftest - INFO -  /home/a3f/poky/meta-selftest/lib
  2024-09-30 10:41:21,229 - oe-selftest - INFO - Checking base configuration is valid/parsable
  NOTE: Starting bitbake server...
  2024-09-30 10:41:24,120 - oe-selftest - INFO - Adding: "include selftest.inc" in /home/a3f/poky/build-st/conf/local.conf
  2024-09-30 10:41:24,125 - oe-selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf
  2024-09-30 10:41:24,131 - oe-selftest - INFO - test_boot_uboot (uboot.UBootTest.test_boot_uboot)
  2024-09-30 10:41:24,214 - oe-selftest - DEBUG - Writing to: /home/a3f/poky/build-st/conf/selftest.inc
  
  QB_DEFAULT_BIOS = "u-boot.bin"
  PREFERRED_PROVIDER_virtual/bootloader = "u-boot"
  
  
  2024-09-30 10:52:59,680 - oe-selftest - INFO -  ... ok
  2024-09-30 10:52:59,692 - oe-selftest - INFO - ----------------------------------------------------------------------
  2024-09-30 10:52:59,693 - oe-selftest - INFO - Ran 1 test in 696.374s
  2024-09-30 10:52:59,695 - oe-selftest - INFO - OK
  2024-09-30 10:53:13,581 - oe-selftest - INFO - RESULTS:
  2024-09-30 10:53:13,584 - oe-selftest - INFO - RESULTS - uboot.UBootTest.test_boot_uboot: PASSED (695.55s)
  2024-09-30 10:53:13,647 - oe-selftest - INFO - SUMMARY:
  2024-09-30 10:53:13,648 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 696.378s
  2024-09-30 10:53:13,649 - oe-selftest - INFO - oe-selftest - OK - All required tests passed (successes=1, skipped=0, failures=0, errors=0)
Quoted 23 lines

U-Boot log in the failed test says

Quoted 10 lines

U-Boot 2024.07 (Jul 01 2024 - 18:07:18 +0000)
DRAM:  256 MiB
Core:  51 devices, 14 uclasses, devicetree: board
Flash: 64 MiB
Loading Environment from Flash... *** Warning - bad CRC, using default environment In:    serial,usbkbd
Out:   serial,vidconsole
Err:   serial,vidconsole
Bus xhci_pci: Register 8001040 NbrPorts 8
Starting the controller

While my successful run goes further:

Quoted 7 lines

[...]
Starting the controller
USB XHCI 1.00
scanning bus xhci_pci for devices... 3 USB Device(s) found Net:   eth0: virtio-net#32
ESC[?25h
Hit any key to stop autoboot:  2

Quoted 1 line

But not sure if this is an issue with USB emulation or something else...

My successful run looks identical to yours. barebox start up is also successful and uneventful.

Quoted 7 lines

Note that barebox' XHCI driver doesn't have PCI support unlike U-Boot, so if the same issue is affecting both barebox and U-Boot breakage in CI, it might not be USB emulation related as barebox doesn't do any USB on that platform...

Is selftest -r uboot -v -v -v -K close enough to what the CI server runs or should I try something else to reproduce this?

That should have worked. "selftest -r barebox" should also fail.

The config used on the autobuilder for that can be seen here:

https://valkyrie.yoctoproject.org/#/builders/23/builds/168/steps/13/logs/stdio

in case the MACHINE value in particular makes a difference.

Cheers,

Richard

#12 lore raw

Hello Richard,

Quoted 12 lines · Richard Purdie
Quoted 10 lines · Ahmad Fatoum

My successful run looks identical to yours. barebox start up is also successful and uneventful.

Note that barebox' XHCI driver doesn't have PCI support unlike U-Boot, so if the same issue is affecting both barebox and U-Boot breakage in CI, it might not be USB emulation related as barebox doesn't do any USB on that platform...

Is selftest -r uboot -v -v -v -K close enough to what the CI server runs or should I try something else to reproduce this?

Quoted 7 lines

That should have worked. "selftest -r barebox" should also fail.

The config used on the autobuilder for that can be seen here:

https://valkyrie.yoctoproject.org/#/builders/23/builds/168/steps/13/logs/stdio

in case the MACHINE value in particular makes a difference.

I am building for MACHINE = "qemuarm64" already, but something else catches my eye: QEMU_USE_KVM = 'True'

I wasn't aware that I need to manually set this. I do this now and it fails for other reasons: qemu-system-aarch64: KVM does not support GICv3 emulation

I'll look into how to resolve this and maybe I am lucky and can reproduce this issue, once KVM is enabled.

Can you share what SoC is used for the ARM CI server? I assume it doesn't run into this problem, because it has a GIC >= v3. Mine is a GIC-400, which apparently only implements GICv2.

Cheers,
Ahmad

Quoted 7 lines

Cheers,

Richard

Signature · 4 lines
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
#13 lore raw

Hello,

Quoted 8 lines · Ahmad Fatoum

I am building for MACHINE = "qemuarm64" already, but something else catches my eye: QEMU_USE_KVM = 'True'

I wasn't aware that I need to manually set this. I do this now and it fails for other reasons: qemu-system-aarch64: KVM does not support GICv3 emulation

I'll look into how to resolve this and maybe I am lucky and can reproduce this issue, once KVM is enabled.

Good news: I managed to reproduce this issue and write a few lines long reproducer that I sent in a bug report to the KVM/QEMU development mailing lists:

    https://lore.kernel.org/all/[email protected]/

I will look after any fixes that may result.

As far as this series here is concerned, both barebox and U-Boot function correctly with KVM disabled, so could we just disable KVM for only the U-Boot/barebox runqemu tests to unblock merge of this series?

Some more info on my debugging below.

I managed to reproduce by enabling KVM and changing:

-QB_CPU_KVM = "-cpu host -machine gic-version=3"
+QB_CPU_KVM = "-cpu host -machine virt"

and by disabling migration across the big-little cluster with taskset -a --cpu-list 2-5

I even got a U-Boot crash register dump:

Flash: 64 MiB
Loading Environment from Flash... *** Warning - bad CRC, using default environment

  In:    serial,usbkbd
  Out:   serial,vidconsole
  Err:   serial,vidconsole

Bus xhci_pci: Register 8001040 NbrPorts 8 Starting the controller
"Synchronous Abort" handler, esr 0x96000010, far 0x10090040
elr: 0000000000055114 lr : 00000000000550f8 (reloc)
elr: 000000004f724114 lr : 000000004f7240f8
x0 : 0000000010090040 x1 : 0000000000000001
x2 : 0000000000000000 x3 : 0000000000003e80
x4 : 0000000000000038 x5 : 000000004e58e7a2
x6 : 0000000000000000 x7 : 0000000000000000
x8 : 000000004e58ec30 x9 : 00000000ffffffd8
x10: 000000000000000d x11: 0000000000000006
x12: 000000004e58ea88 x13: 000000004e58ed90
x14: 0000000000000000 x15: 000000004e58e7a2
x16: 000000004f716d84 x17: 0000000000000000
x18: 000000004e68ed90 x19: 000000004e695b00
x20: 0000000000000000 x21: 0000000010090040
x22: 0000000010090000 x23: 000000004f79553c
x24: 0000000000000000 x25: 0000000000000000

Removing -enable-kvm makes the issue disappear. barebox also reaches the shell normally when -enable-kvm is omitted.

Given that I am more familiar with barebox (and the stack traces there actually walk the stack and contain symbols if the data abort happens after the vector table is installed), I debugged the issue there and narrowed it down to a specific encoding of the LDR instruction operating on an MMIO peripheral.

More information about that in the bug report linked above.

Cheers,
Ahmad

Quoted 17 lines

Can you share what SoC is used for the ARM CI server? I assume it doesn't run into this problem, because it has a GIC >= v3. Mine is a GIC-400, which apparently only implements GICv2.

Cheers,
Ahmad

Quoted 7 lines

Cheers,

Richard

Signature · 4 lines
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
#14 lore raw

Hello,

Quoted 7 lines · Ahmad Fatoum
Quoted 6 lines · Ahmad Fatoum

I am building for MACHINE = "qemuarm64" already, but something else catches my eye: QEMU_USE_KVM = 'True'

I wasn't aware that I need to manually set this. I do this now and it fails for other reasons: qemu-system-aarch64: KVM does not support GICv3 emulation

Quoted 7 lines
Quoted 2 lines

I'll look into how to resolve this and maybe I am lucky and can reproduce this issue, once KVM is enabled.

Good news: I managed to reproduce this issue and write a few lines long reproducer that I sent in a bug report to the KVM/QEMU development mailing lists:

    https://lore.kernel.org/all/[email protected]/

TL;DR: Turns out this is not a bug, but a known quirk of KVM and software that wishes to run under KVM needs to adapt the instructions it uses to access MMIO regions accordingly.

For barebox, I sent a series adding KVM compatibility here:
https://lore.kernel.org/barebox/[email protected]

For U-Boot, I reported the issue here:
https://lore.kernel.org/u-boot/[email protected]

Quoted 3 lines

As far as this series here is concerned, both barebox and U-Boot function correctly with KVM disabled, so could we just disable KVM for only the U-Boot/barebox runqemu tests to unblock merge of this series?

I am more certain now that we should disable KVM for bootloader testing. Full system tests with Linux are much more demanding, so KVM absolutely makes sense there.

In the bootloader however, normal emulation should suffice, especially when their upstream isn't KVM-compatible yet anyway.

Cheers,
Ahmad

Signature · 4 lines
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |