Add barebox bootloader support (and testing)
v2 cover Add barebox bootloader support (and testing)
This adds support for the barebox bootloader (and tools) to oe-core.
In order to have proper testing, this extends oe-selftest to allow basic testing of bootloaders. While at it, cover both barebox and u-boot.
v2:
* set myself as maintainer in maintainers.inc
* move doc from documentation.conf to recipe
* update to barebox v2023.03 (including fixes for musl, etc.)
* set standard configs for qemu machines to allow building them
* add better efi dir support in barebox recipe
* enable testing bootloaders in oeqa
* add test cases for barebox (and u-boot)
Enrico Jorns (7):
barebox: set default BAREBOX_CONFIG for qemu machines oeqa/utils/qemurunner: support ignoring vt100 escape sequences oeqa/utils/qemurunner: simplify output parsing and make
crlf-compatible
oeqa/utils/commands: document runqemu context manager
oeqa: support passing custom boot patterns to runqemu oeqa/selftest/cases: add barebox tests oeqa/selftest/cases: add basic u-boot test
Marco Felsch (2):
barebox: add initial support
barebox-tools: add initial barebox tools support
meta/conf/distro/include/maintainers.inc | 2 + meta/lib/oeqa/selftest/cases/barebox.py | 72 +++++++++ meta/lib/oeqa/selftest/cases/uboot.py | 40 +++++ meta/lib/oeqa/targetcontrol.py | 5 +- meta/lib/oeqa/utils/commands.py | 22 ++- meta/lib/oeqa/utils/qemurunner.py | 37 +++-- .../barebox/barebox-tools_2023.03.0.bb | 56 +++++++ meta/recipes-bsp/barebox/barebox.inc | 138 ++++++++++++++++++ meta/recipes-bsp/barebox/barebox_2023.03.0.bb | 3 + 9 files changed, 351 insertions(+), 24 deletions(-) create mode 100644 meta/lib/oeqa/selftest/cases/barebox.py create mode 100644 meta/lib/oeqa/selftest/cases/uboot.py create mode 100644 meta/recipes-bsp/barebox/barebox-tools_2023.03.0.bb create mode 100644 meta/recipes-bsp/barebox/barebox.inc create mode 100644 meta/recipes-bsp/barebox/barebox_2023.03.0.bb
Signature · 1 line
2.39.2
PATCH v2 1/9 barebox: add initial support
From: Marco Felsch <[email protected]>
This adds the support for the barebox bootloader to oe-core. The recipe is based on the recipe found in meta-ptx [1] with a few minor adaptions.
This basic support includes the bootloader and the target tools to interact with the bootloader. The host tools support is not part of this commit. This will be added later on as separate recipe.
[1] https://github.com/pengutronix/meta-ptx/tree/master/recipes-bsp/barebox
--- meta/conf/distro/include/maintainers.inc | 1 + meta/recipes-bsp/barebox/barebox.inc | 133 ++++++++++++++++++ meta/recipes-bsp/barebox/barebox_2023.03.0.bb | 3 + 3 files changed, 137 insertions(+) create mode 100644 meta/recipes-bsp/barebox/barebox.inc create mode 100644 meta/recipes-bsp/barebox/barebox_2023.03.0.bb
meta/conf/distro/include/maintainers.inc +1 -0
@@ -54,6 +54,7 @@ RECIPE_MAINTAINER:pn-automake = "Robert Yang <[email protected]>" RECIPE_MAINTAINER:pn-avahi = "Yi Zhao <[email protected]>" RECIPE_MAINTAINER:pn-babeltrace = "Alexander Kanavin <[email protected]>" RECIPE_MAINTAINER:pn-babeltrace2 = "Alexander Kanavin <[email protected]>" +RECIPE_MAINTAINER:pn-barebox = "Enrico Jörns <[email protected]>" RECIPE_MAINTAINER:pn-baremetal-helloworld = "Alejandro Hernandez <[email protected]>" RECIPE_MAINTAINER:pn-base-files = "Anuj Mittal <[email protected]>" RECIPE_MAINTAINER:pn-base-passwd = "Anuj Mittal <[email protected]>"
meta/recipes-bsp/barebox/barebox.inc new file +133 -0
133 lines not on this page · raw message
meta/recipes-bsp/barebox/barebox_2023.03.0.bb new file +3 -0
@@ -0,0 +1,3 @@ +require barebox.inc + +SRC_URI[sha256sum] = "d0f78a69ba240327247c8fd0e1d45287e4a0dff99ed847e9a696cc2da0cf388c"
Signature · 1 line
2.39.2
PATCH v2 2/9 barebox-tools: add initial barebox tools support
In reply to Enrico JornsFrom: Marco Felsch <[email protected]>
Add initial support to build the barebox tools for the host and the target.
--- meta/conf/distro/include/maintainers.inc | 1 + .../barebox/barebox-tools_2023.03.0.bb | 56 +++++++++++++++++++ 2 files changed, 57 insertions(+) create mode 100644 meta/recipes-bsp/barebox/barebox-tools_2023.03.0.bb
meta/conf/distro/include/maintainers.inc +1 -0
@@ -55,6 +55,7 @@ RECIPE_MAINTAINER:pn-avahi = "Yi Zhao <[email protected]>" RECIPE_MAINTAINER:pn-babeltrace = "Alexander Kanavin <[email protected]>" RECIPE_MAINTAINER:pn-babeltrace2 = "Alexander Kanavin <[email protected]>" RECIPE_MAINTAINER:pn-barebox = "Enrico Jörns <[email protected]>" +RECIPE_MAINTAINER:pn-barebox-tools = "Enrico Jörns <[email protected]>" RECIPE_MAINTAINER:pn-baremetal-helloworld = "Alejandro Hernandez <[email protected]>" RECIPE_MAINTAINER:pn-base-files = "Anuj Mittal <[email protected]>" RECIPE_MAINTAINER:pn-base-passwd = "Anuj Mittal <[email protected]>"
meta/recipes-bsp/barebox/barebox-tools_2023.03.0.bb new file +56 -0
56 lines not on this page · raw message
Signature · 1 line
2.39.2
PATCH v2 3/9 barebox: set default BAREBOX_CONFIG for qemu machines
In reply to Enrico JornsThese are set in the barebox recipe rather in the corresponding machines (where they would belong otherwise) to keep the impact of barebox to oe-core minimal for now.
--- meta/recipes-bsp/barebox/barebox.inc | 5 +++++ 1 file changed, 5 insertions(+)
meta/recipes-bsp/barebox/barebox.inc +5 -0
@@ -48,6 +48,11 @@ EXTRA_OEMAKE = " \ BAREBOX_CONFIG[doc] = "The barebox kconfig defconfig file. Not used if a file called defconfig is added to the SRC_URI." BAREBOX_CONFIG ?= "" +# set defaults for oe-core MACHINEs to allow building barebox +BAREBOX_CONFIG:qemuarm = "vexpress_defconfig" +BAREBOX_CONFIG:qemuarm64 = "qemu_virt64_defconfig" +BAREBOX_CONFIG:qemux86-64 = "efi_defconfig" + do_configure() { if [ -e ${WORKDIR}/defconfig ]; then cp ${WORKDIR}/defconfig ${B}/.config
Signature · 1 line
2.39.2
Re: [PATCH v2 3/9] barebox: set default BAREBOX_CONFIG for qemu machines
meta/recipes-bsp/barebox/barebox.inc EXTRA_OEMAKE = " \ · line 54
Quoted 4 lines · Enrico Jorns
meta/recipes-bsp/barebox/barebox.inc +4
-0+# set defaults for oe-core MACHINEs to allow building barebox +BAREBOX_CONFIG:qemuarm = "vexpress_defconfig" +BAREBOX_CONFIG:qemuarm64 = "qemu_virt64_defconfig" +BAREBOX_CONFIG:qemux86-64 = "efi_defconfig"
As all three are different perhaps there could be commentary in the recipe explaining why that and how they differ? Note that there's also a qemuall override if there's a single configuration that works for all qemu targets.
Alex
Re: [PATCH v2 3/9] barebox: set default BAREBOX_CONFIG for qemu machines
Hi Alex,
Am Freitag, dem 31.03.2023 um 16:04 +0200 schrieb Alexander Kanavin:
Quoted 10 lines · Enrico Jorns
meta/recipes-bsp/barebox/barebox.inc EXTRA_OEMAKE = " \ · line 54
Quoted 4 lines
meta/recipes-bsp/barebox/barebox.inc +4
-0+# set defaults for oe-core MACHINEs to allow building barebox +BAREBOX_CONFIG:qemuarm = "vexpress_defconfig" +BAREBOX_CONFIG:qemuarm64 = "qemu_virt64_defconfig" +BAREBOX_CONFIG:qemux86-64 = "efi_defconfig"As all three are different perhaps there could be commentary in the recipe explaining why that and how they differ? Note that there's also a qemuall override if there's a single configuration that works for all qemu targets.
as these three are different archs/machines, I do not find it surprising that they require different configs. I assumed that this was self-explaining.
Note that for UBOOT, you have
in qemux86-64.conf:
UBOOT_MACHINE ?= "qemu-x86_64_defconfig"
in qemuarm.conf:
UBOOT_MACHINE ?= "qemu_arm_defconfig"
in qemuarm64.conf:
UBOOT_MACHINE ?= "qemu_arm64_defconfig"
which is pretty much the same as for barebox, isn't it? Only the naming scheme is a bit more consistent probably.
Thanks and regards
Enrico
Quoted 3 lines
Alex
Signature · 4 lines
Pengutronix e.K. | Enrico Jörns | Embedded Linux Consulting & Support | https://www.pengutronix.de/ | Steuerwalder Str. 21 | Phone: +49-5121-206917-180 | 31137 Hildesheim, Germany | Fax: +49-5121-206917-9 |
Re: [PATCH v2 3/9] barebox: set default BAREBOX_CONFIG for qemu machines
Quoted 10 lines · Enrico Jörns
Quoted 9 lines
meta/recipes-bsp/barebox/barebox.inc EXTRA_OEMAKE = " \ · line 54
Quoted 4 lines
meta/recipes-bsp/barebox/barebox.inc +4
-0+# set defaults for oe-core MACHINEs to allow building barebox +BAREBOX_CONFIG:qemuarm = "vexpress_defconfig" +BAREBOX_CONFIG:qemuarm64 = "qemu_virt64_defconfig" +BAREBOX_CONFIG:qemux86-64 = "efi_defconfig"As all three are different perhaps there could be commentary in the recipe explaining why that and how they differ? Note that there's also a qemuall override if there's a single configuration that works for all qemu targets.
Quoted 2 lines
as these three are different archs/machines, I do not find it surprising that they require different configs. I assumed that this was self-explaining.
Not at all. What is this 'vexpress' thingy? What does 'express' mean? Why does it apply to arm, but not arm64? Why arm64 is using 'qemu_virt64', but not any other targets, if that config does not seem arm-specific? Why is x86 using 'efi' but not the arm targets?
The choices you made here should be explained.
Alex
Re: [PATCH v2 3/9] barebox: set default BAREBOX_CONFIG for qemu machines
Am Dienstag, dem 11.04.2023 um 16:16 +0200 schrieb Alexander Kanavin:
Quoted 15 lines · Enrico Jörns
Quoted 10 lines
Quoted 9 lines
meta/recipes-bsp/barebox/barebox.inc EXTRA_OEMAKE = " \ · line 54
Quoted 4 lines
meta/recipes-bsp/barebox/barebox.inc +4
-0+# set defaults for oe-core MACHINEs to allow building barebox +BAREBOX_CONFIG:qemuarm = "vexpress_defconfig" +BAREBOX_CONFIG:qemuarm64 = "qemu_virt64_defconfig" +BAREBOX_CONFIG:qemux86-64 = "efi_defconfig"As all three are different perhaps there could be commentary in the recipe explaining why that and how they differ? Note that there's also a qemuall override if there's a single configuration that works for all qemu targets.
Quoted 3 lines
as these three are different archs/machines, I do not find it surprising that they require different
configs. I assumed that this was self-explaining.
Quoted 6 lines
Not at all. What is this 'vexpress' thingy? What does 'express' mean? Why does it apply to arm, but not arm64? Why arm64 is using '
qemu_virt64', but not any other targets, if that config does not seem arm-specific? Why is x86 using 'efi' but not the arm targets?The choices you made here should be explained.
Vexpress is "Versatile Express" which once was the ARMv7 reference platform and is one of the platforms emulated in QEMU (when using "-machine vexpress-a9" or "-machine vexpress-a15").
virt, or virt64 are the generic QEMU ARM platform. For historical reasons, there is only a dedicated vexpress defconfig in barebox.
EFI is used for x86 only, since on ARM barebox is normally used as the primary loader only.
However, since barebox builds are multi-platform builds by default, it should be possible to use the standard multi_v7_defconfig and multi_v8_defconfig instead (for arm). I'll give it a try.
Regards, Enrico
Quoted 3 lines
Alex
Signature · 4 lines
Pengutronix e.K. | Enrico Jörns | Embedded Linux Consulting & Support | https://www.pengutronix.de/ | Steuerwalder Str. 21 | Phone: +49-5121-206917-180 | 31137 Hildesheim, Germany | Fax: +49-5121-206917-9 |
PATCH v2 4/9 oeqa/utils/qemurunner: support ignoring vt100 escape sequences
In reply to Enrico JornsIf we talk to terminals that like colors, we need to ignore the vt100 escape sequences when matching strings.
--- meta/lib/oeqa/utils/qemurunner.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
meta/lib/oeqa/utils/qemurunner.py +2 -1
@@ -30,6 +30,7 @@ control_range = list(range(0,32))+list(range(127,160)) control_chars = [chr(x) for x in control_range if chr(x) not in string.printable] re_control_char = re.compile('[%s]' % re.escape("".join(control_chars))) +re_vt100 = re.compile(r'(\x1b\[|\x9b)[^@-_a-z]*[@-_a-z]|\x1b[@-_a-z]') class QemuRunner:
@@ -662,7 +663,7 @@ class QemuRunner: time.sleep(0.1) answer = self.server_socket.recv(1024) if answer: - data += answer.decode('utf-8') + data += re_vt100.sub("", answer.decode('utf-8')) # Search the prompt to stop if re.search(self.boot_patterns['search_cmd_finished'], data): break
Signature · 1 line
2.39.2
Re: [PATCH v2 4/9] oeqa/utils/qemurunner: support ignoring vt100 escape sequences
Quoted 1 line · Enrico Jorns
+re_vt100 = re.compile(r'(\x1b\[|\x9b)[^@-_a-z]*[@-_a-z]|\x1b[@-_a-z]')
This piece of 'magic' needs to be explained :) What does it define?
Alex
Re: [PATCH v2 4/9] oeqa/utils/qemurunner: support ignoring vt100 escape sequences
Hi Alex,
Am Freitag, dem 31.03.2023 um 16:05 +0200 schrieb Alexander Kanavin:
Quoted 4 lines · Enrico Jorns
Quoted 1 line
+re_vt100 = re.compile(r'(\x1b\[|\x9b)[^@-_a-z]*[@-_a-z]|\x1b[@-_a-z]')
This piece of 'magic' needs to be explained :) What does it define?
this is actually 'stolen' from labgrid's implementation for barebox[1] or shell[2] drivers which have exactly the same needs: They need to remove the ansii (color) control codes from the strings in order to match the text only.
The unprocessed barebox console prompt would look like:
ESC[1;32mbarebox@ESC[1;36mARM QEMU virt64:/ESC[0m
where we cannot match for something like "barebox@ARM QEMU virt64:/". The same applies to colored linux terminal output of course.
The "\x1b\[" from the regex catches the standard start of ansii escape sequence while the rest catches the actual command code executed.
Regards, Enrico
[1] https://github.com/labgrid-project/labgrid/blob/e05f530c22513cd775b4f84e28f6c8c920b95102/labgrid/driver/bareboxdriver.py#L44
[2] https://github.com/labgrid-project/labgrid/blob/5d3d5714976b37df0f9a9b769c581fe1f83ccbdc/labgrid/driver/shelldriver.py#L61
Quoted 2 lines
Alex
Signature · 4 lines
Pengutronix e.K. | Enrico Jörns | Embedded Linux Consulting & Support | https://www.pengutronix.de/ | Steuerwalder Str. 21 | Phone: +49-5121-206917-180 | 31137 Hildesheim, Germany | Fax: +49-5121-206917-9 |
Re: [PATCH v2 4/9] oeqa/utils/qemurunner: support ignoring vt100 escape sequences
Right, but this information should be recorded in commit message, or the file itself.
Alex
Quoted 38 lines · Enrico Jörns
Hi Alex,
Am Freitag, dem 31.03.2023 um 16:05 +0200 schrieb Alexander Kanavin:
Quoted 4 lines · Enrico Jorns
Quoted 1 line
+re_vt100 = re.compile(r'(\x1b\[|\x9b)[^@-_a-z]*[@-_a-z]|\x1b[@-_a-z]')
This piece of 'magic' needs to be explained :) What does it define?
this is actually 'stolen' from labgrid's implementation for barebox[1] or shell[2] drivers which have exactly the same needs: They need to remove the ansii (color) control codes from the strings in order to match the text only.
The unprocessed barebox console prompt would look like:
ESC[1;32mbarebox@ESC[1;36mARM QEMU virt64:/ESC[0m
where we cannot match for something like "barebox@ARM QEMU virt64:/". The same applies to colored linux terminal output of course.
The "\x1b\[" from the regex catches the standard start of ansii escape sequence while the rest catches the actual command code executed.
Regards, Enrico
[1] https://github.com/labgrid-project/labgrid/blob/e05f530c22513cd775b4f84e28f6c8c920b95102/labgrid/driver/bareboxdriver.py#L44
[2] https://github.com/labgrid-project/labgrid/blob/5d3d5714976b37df0f9a9b769c581fe1f83ccbdc/labgrid/driver/shelldriver.py#L61Quoted 2 lines
Alex
--
Pengutronix e.K. | Enrico Jörns | Embedded Linux Consulting & Support | https://www.pengutronix.de/ | Steuerwalder Str. 21 | Phone: +49-5121-206917-180 | 31137 Hildesheim, Germany | Fax: +49-5121-206917-9 |
PATCH v2 5/9 oeqa/utils/qemurunner: simplify output parsing and make crlf-compatible
In reply to Enrico JornsReduces the nesting depth and replaces the manual newline matching by built-in splitlines() method.
This makes it compatible with shells that use windows-compatible line breaks, e.g. for EFI loaders.
More comments and an early return handling should make the code a bit more readable.
--- meta/lib/oeqa/utils/qemurunner.py | 32 +++++++++++++++---------------- 1 file changed, 15 insertions(+), 17 deletions(-)
meta/lib/oeqa/utils/qemurunner.py +15 -17
@@ -672,23 +672,21 @@ class QemuRunner: return (1, "") raise Exception("No data on serial console socket, connection closed?") - if data: - if raw: - status = 1 - else: - # Remove first line (command line) and last line (prompt) - data = data[data.find('$?\r\n')+4:data.rfind('\r\n')] - index = data.rfind('\r\n') - if index == -1: - status_cmd = data - data = "" - else: - status_cmd = data[index+2:] - data = data[:index] - if (status_cmd == "0"): - status = 1 - return (status, str(data)) - + # If we got no data, we assume something went wrong and return 0 + if not data: + return (0, str(None)) + + # in raw mode, we cannot check exit status output and thus assume success + if raw: + return (1, str(data)) + + # Split lines into array and remove first line (command line) and last line (prompt) + # Also remove empty lines to ease catching results + outlines = list(filter(None, data.splitlines()[1:-1])) + # Remaining last line contains exit code output + if (outlines[-1] == "0"): + status = 1 + return (status, "\n".join(outlines[0:-1])) def _dump_host(self): self.host_dumper.create_dir("qemu")
Signature · 1 line
2.39.2
PATCH v2 6/9 oeqa/utils/commands: document runqemu context manager
In reply to Enrico JornsUsage and arguments are not necessarily self-explaining.
--- meta/lib/oeqa/utils/commands.py | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-)
meta/lib/oeqa/utils/commands.py +16 -1
@@ -314,7 +314,22 @@ def create_temp_layer(templayerdir, templayername, priority=999, recipepathspec= @contextlib.contextmanager def runqemu(pn, ssh=True, runqemuparams='', image_fstype=None, launch_cmd=None, qemuparams=None, overrides={}, discard_writes=True): """ - launch_cmd means directly run the command, don't need set rootfs or env vars. + Starts a context manager for a 'oeqa.targetcontrol.QemuTarget' resource. + The underlying Qemu will be booted into a shell when the generator yields. + + Usage: + + with runqemu('core-image-minimal') as qemu: + qemu.run_serial('cat /proc/cpuinfo') + + Args: + pn (str): (image) recipe to run on + ssh (boolean): whether or not to enable SSH (network access) + runqemuparams (str): space-separated list of params to pass to 'runqemu' script (like 'nographics', 'ovmf', etc.) + image_fstype (str): IMAGE_FSTYPE to use + launch_cmd (str): directly run this command and bypass automatic runqemu parameter generation + overrides (dict): dict of "'<bitbake-variable>': value" pairs that allows overriding bitbake variables + discard_writes (boolean): enables qemu -snapshot feature to prevent modifying original image """ import bb.tinfoil
Signature · 1 line
2.39.2
PATCH v2 7/9 oeqa: support passing custom boot patterns to runqemu
In reply to Enrico JornsThis allows defining non-standard patterns from QEMU tests just as they are already supported by the testimage.bbclass.
Will allow testing non-Linux shells in QEMU, too, e.g. a U-Boot or other bootloader shell.
--- meta/lib/oeqa/targetcontrol.py | 5 +++-- meta/lib/oeqa/utils/commands.py | 5 +++-- meta/lib/oeqa/utils/qemurunner.py | 2 +- 3 files changed, 7 insertions(+), 5 deletions(-)
meta/lib/oeqa/targetcontrol.py +3 -2
@@ -92,7 +92,7 @@ class QemuTarget(BaseTarget): supported_image_fstypes = ['ext3', 'ext4', 'cpio.gz', 'wic'] - def __init__(self, d, logger, image_fstype=None): + def __init__(self, d, logger, image_fstype=None, boot_patterns=None): import oe.types
@@ -148,7 +148,8 @@ class QemuTarget(BaseTarget): dump_host_cmds = d.getVar("testimage_dump_host"), logger = logger, tmpfsdir = d.getVar("RUNQEMU_TMPFS_DIR"), - serial_ports = len(d.getVar("SERIAL_CONSOLES").split())) + serial_ports = len(d.getVar("SERIAL_CONSOLES").split()), + boot_patterns = boot_patterns) self.target_dumper = TargetDumper(dump_target_cmds, dump_dir, self.runner) self.monitor_dumper = MonitorDumper(dump_monitor_cmds, dump_dir, self.runner)
meta/lib/oeqa/utils/commands.py +3 -2
@@ -312,7 +312,7 @@ def create_temp_layer(templayerdir, templayername, priority=999, recipepathspec= f.write('LAYERSERIES_COMPAT_%s = "%s"\n' % (templayername, corenames)) @contextlib.contextmanager -def runqemu(pn, ssh=True, runqemuparams='', image_fstype=None, launch_cmd=None, qemuparams=None, overrides={}, discard_writes=True): +def runqemu(pn, ssh=True, runqemuparams='', image_fstype=None, launch_cmd=None, qemuparams=None, overrides={}, boot_patterns = {}, discard_writes=True): """ Starts a context manager for a 'oeqa.targetcontrol.QemuTarget' resource. The underlying Qemu will be booted into a shell when the generator yields.
@@ -329,6 +329,7 @@ def runqemu(pn, ssh=True, runqemuparams='', image_fstype=None, launch_cmd=None, image_fstype (str): IMAGE_FSTYPE to use launch_cmd (str): directly run this command and bypass automatic runqemu parameter generation overrides (dict): dict of "'<bitbake-variable>': value" pairs that allows overriding bitbake variables + boot_patterns (dict): dict of "'<pattern-name>': value" pairs to override default boot patterns, e.g. when not booting Linux discard_writes (boolean): enables qemu -snapshot feature to prevent modifying original image """
@@ -360,7 +361,7 @@ def runqemu(pn, ssh=True, runqemuparams='', image_fstype=None, launch_cmd=None, logdir = recipedata.getVar("TEST_LOG_DIR") - qemu = oeqa.targetcontrol.QemuTarget(recipedata, targetlogger, image_fstype) + qemu = oeqa.targetcontrol.QemuTarget(recipedata, targetlogger, image_fstype, boot_patterns=boot_patterns) finally: # We need to shut down tinfoil early here in case we actually want # to run tinfoil-using utilities with the running QEMU instance.
meta/lib/oeqa/utils/qemurunner.py +1 -1
@@ -93,7 +93,7 @@ class QemuRunner: # Only override patterns that were set e.g. login user TESTIMAGE_BOOT_PATTERNS[send_login_user] = "webserver\n" for pattern in accepted_patterns: - if not self.boot_patterns[pattern]: + if pattern not in self.boot_patterns or not self.boot_patterns[pattern]: self.boot_patterns[pattern] = default_boot_patterns[pattern] def create_socket(self):
Signature · 1 line
2.39.2
PATCH v2 8/9 oeqa/selftest/cases: add barebox tests
In reply to Enrico Jorns--- meta/lib/oeqa/selftest/cases/barebox.py | 72 +++++++++++++++++++++++++ 1 file changed, 72 insertions(+) create mode 100644 meta/lib/oeqa/selftest/cases/barebox.py
meta/lib/oeqa/selftest/cases/barebox.py new file +72 -0
@@ -0,0 +1,72 @@ +# Qemu-based barebox bootloader integration testing +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +from oeqa.selftest.case import OESelftestTestCase +from oeqa.utils.commands import bitbake, runqemu +from oeqa.core.decorator.data import skipIfNotArch +from oeqa.core.decorator import OETestTag + +barebox_boot_patterns = { + 'search_reached_prompt': r"stop autoboot", + 'search_login_succeeded': r"barebox@[^:]+:[^ ]+ ", + 'search_cmd_finished': r"barebox@[a-zA-Z0-9\-\s]+:/" + } + + +class BareboxTest(OESelftestTestCase): + + @skipIfNotArch(['arm', 'aarch64']) + @OETestTag("runqemu") + def test_boot_barebox(self): + """ + Tests building barebox and booting it with QEMU + """ + + self.write_config(""" +QB_DEFAULT_KERNEL = "barebox-dt-2nd.img" +PREFERRED_PROVIDER_virtual/bootloader = "barebox" +""") + + bitbake("virtual/bootloader core-image-minimal") + + with runqemu('core-image-minimal', ssh=False, runqemuparams='nographic', + boot_patterns=barebox_boot_patterns) as qemu: + + # test if barebox console works + cmd = "version" + status, output = qemu.run_serial(cmd) + self.assertEqual(status, 1, msg=output) + self.assertTrue("barebox" in output, msg=output) + + @skipIfNotArch(['x86_64']) + @OETestTag("runqemu") + def test_boot_barebox_efi(self): + """ + Tests building barebox for UEFI and booting it as EFI payload + with QEMU + OVMF + """ + image = "core-image-minimal" + + self.write_config(""" +IMAGE_INSTALL:append = " barebox" +MACHINE_FEATURES:append = " pcbios efi" +EXTRA_IMAGEDEPENDS += "ovmf" +EFI_PROVIDER = "barebox" +IMAGE_FSTYPES += "wic" +WKS_FILE = "efi-bootdisk.wks.in" +""") + + bitbake(image) + + with runqemu(image, ssh=False, runqemuparams='nographic ovmf', + boot_patterns=barebox_boot_patterns, image_fstype='wic') as qemu: + + # test if barebox console works + cmd = "version" + status, output = qemu.run_serial(cmd) + self.assertEqual(status, 1, msg=output) + self.assertTrue("barebox" in output, msg=output)
Signature · 1 line
2.39.2
PATCH v2 9/9 oeqa/selftest/cases: add basic u-boot test
In reply to Enrico Jorns--- meta/lib/oeqa/selftest/cases/uboot.py | 40 +++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100644 meta/lib/oeqa/selftest/cases/uboot.py
meta/lib/oeqa/selftest/cases/uboot.py new file +40 -0
@@ -0,0 +1,40 @@ +# Qemu-based u-boot bootloader integration testing +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +from oeqa.selftest.case import OESelftestTestCase +from oeqa.utils.commands import bitbake, runqemu +from oeqa.core.decorator import OETestTag + +uboot_boot_patterns = { + 'search_reached_prompt': "stop autoboot", + 'search_login_succeeded': "=>", + 'search_cmd_finished': "=>" + } + + +class UBootTest(OESelftestTestCase): + + @OETestTag("runqemu") + def test_boot_uboot(self): + """ + Tests building u-boot and booting it with QEMU + """ + + self.write_config(""" +QB_DEFAULT_BIOS = "u-boot.bin" +PREFERRED_PROVIDER_virtual/bootloader = "u-boot" +""") + bitbake("virtual/bootloader core-image-minimal") + + with runqemu('core-image-minimal', ssh=False, runqemuparams='nographic', + boot_patterns=uboot_boot_patterns) as qemu: + + # test if u-boot console works + cmd = "version" + status, output = qemu.run_serial(cmd) + self.assertEqual(status, 1, msg=output) + self.assertTrue("U-Boot" in output, msg=output)
Signature · 1 line
2.39.2
Quoted 8 lines · Enrico Jorns
v2:
* set myself as maintainer in maintainers.inc
* move doc from documentation.conf to recipe
* update to barebox v2023.03 (including fixes for musl, etc.)
* set standard configs for qemu machines to allow building them
* add better efi dir support in barebox recipe
* enable testing bootloaders in oeqa
* add test cases for barebox (and u-boot)
Thanks for persevering with this! The patchset is mostly fine (I'll write a couple comments now), and just needs to get a verdict from the autobuilder.
Alex
Quoted 24 lines · Enrico Jorns
This adds support for the barebox bootloader (and tools) to oe-core.
In order to have proper testing, this extends oe-selftest to allow basic testing of bootloaders. While at it, cover both barebox and u-boot.
v2:
* set myself as maintainer in maintainers.inc
* move doc from documentation.conf to recipe
* update to barebox v2023.03 (including fixes for musl, etc.)
* set standard configs for qemu machines to allow building them
* add better efi dir support in barebox recipe
* enable testing bootloaders in oeqa
* add test cases for barebox (and u-boot)Enrico Jorns (7):
barebox: set defaultBAREBOX_CONFIGfor qemu machines oeqa/utils/qemurunner: support ignoring vt100 escape sequences oeqa/utils/qemurunner: simplify output parsing and make
crlf-compatible
oeqa/utils/commands: document runqemu context manager
oeqa: support passing custom boot patterns to runqemu oeqa/selftest/cases: add barebox tests oeqa/selftest/cases: add basic u-boot test
Quoted 3 lines
Marco Felsch (2):
barebox: add initial support
barebox-tools: add initial barebox tools support
Testing on the autobuilder did trigger a few issues:
on musl:
https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/5023
https://autobuilder.yoctoproject.org/typhoon/#/builders/45/builds/6932
and some failing oe-selftests:
https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/4971
https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/5023
[there were more but look similar]
Cheers,
Richard
Hi Richard,
Am Montag, dem 03.04.2023 um 16:49 +0100 schrieb Richard Purdie:
Quoted 33 lines · Enrico Jorns
Quoted 24 lines
This adds support for the barebox bootloader (and tools) to oe-core.
In order to have proper testing, this extends oe-selftest to allow basic testing of bootloaders. While at it, cover both barebox and u-boot.
v2:
* set myself as maintainer in maintainers.inc
* move doc from documentation.conf to recipe
* update to barebox v2023.03 (including fixes for musl, etc.)
* set standard configs for qemu machines to allow building them
* add better efi dir support in barebox recipe
* enable testing bootloaders in oeqa
* add test cases for barebox (and u-boot)Enrico Jorns (7):
barebox: set defaultBAREBOX_CONFIGfor qemu machines oeqa/utils/qemurunner: support ignoring vt100 escape sequences oeqa/utils/qemurunner: simplify output parsing and make crlf-compatible
oeqa/utils/commands: document runqemu context manager
oeqa: support passing custom boot patterns to runqemu oeqa/selftest/cases: add barebox tests oeqa/selftest/cases: add basic u-boot testQuoted 3 lines
Marco Felsch (2):
barebox: add initial support
barebox-tools: add initial barebox tools supportTesting on the autobuilder did trigger a few issues:
on musl:
Quoted 8 lines
https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/5023
https://autobuilder.yoctoproject.org/typhoon/#/builders/45/builds/6932and some failing oe-selftests:
https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/4971
https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/5023
[there were more but look similar]
thank you for the autobuilder logs!
I fear we are running into a conceptional issue that I already feared we could run into (but did not test for yet).
If I traced this down correctly, then the 'issue' is:
| NOTE: Multiple providers are available for virtual/bootloader (barebox, u-boot) | Consider defining a PREFERRED_PROVIDER entry to match virtual/bootloader
The fitimage tests building "virtual/bootloader", silently assuming this is equal to u-boot (which it actually was so far).
Now, the quick fix for this would be to properly set the correct PREFERRED_PROVIDER here, but I could imagine that we run into similar issues in other builds, too.
Do you have a good solution for this at hand? Like defining a standard preferred provider for virtual/bootloader?
I guess one could argue possibly that the mechanism does exactly what it is meant for. And a few machines (like beaglebone-yocto or qemuloongarch) actually set the PREFERRED_PROVIDER, but not all and this might lead to silently switching the bootloader (like in our tests).
I hope you have a better overview on the topic than I have and could give me a hint on how you would like to see this being resolved?
Thanks in advance and best regards
Enrico
Quoted 5 lines
Cheers,
Richard
Signature · 4 lines
Pengutronix e.K. | Enrico Jörns | Embedded Linux Consulting & Support | https://www.pengutronix.de/ | Steuerwalder Str. 21 | Phone: +49-5121-206917-180 | 31137 Hildesheim, Germany | Fax: +49-5121-206917-9 |
Quoted 65 lines · Enrico Jörns
Hi Richard,
Am Montag, dem 03.04.2023 um 16:49 +0100 schrieb Richard Purdie:
Quoted 33 lines · Enrico Jorns
Quoted 24 lines
This adds support for the barebox bootloader (and tools) to oe-core.
In order to have proper testing, this extends oe-selftest to allow basic testing of bootloaders. While at it, cover both barebox and u-boot.
v2:
* set myself as maintainer in maintainers.inc
* move doc from documentation.conf to recipe
* update to barebox v2023.03 (including fixes for musl, etc.)
* set standard configs for qemu machines to allow building them
* add better efi dir support in barebox recipe
* enable testing bootloaders in oeqa
* add test cases for barebox (and u-boot)Enrico Jorns (7):
barebox: set defaultBAREBOX_CONFIGfor qemu machines oeqa/utils/qemurunner: support ignoring vt100 escape sequences oeqa/utils/qemurunner: simplify output parsing and make
crlf-compatible
oeqa/utils/commands: document runqemu context manager
oeqa: support passing custom boot patterns to runqemu oeqa/selftest/cases: add barebox tests oeqa/selftest/cases: add basic u-boot testQuoted 3 lines
Marco Felsch (2):
barebox: add initial support
barebox-tools: add initial barebox tools supportTesting on the autobuilder did trigger a few issues:
on musl:
Quoted 8 lines
https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/5023
https://autobuilder.yoctoproject.org/typhoon/#/builders/45/builds/6932and some failing oe-selftests:
https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/4971
https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/5023
[there were more but look similar]thank you for the autobuilder logs!
I fear we are running into a conceptional issue that I already feared we could run into (but did not test for yet).
If I traced this down correctly, then the 'issue' is:
| NOTE: Multiple providers are available for virtual/bootloader (barebox, u-boot) | Consider defining a
PREFERRED_PROVIDERentry to match virtual/bootloaderThe fitimage tests building "virtual/bootloader", silently assuming this is equal to u-boot (which it actually was so far).
Now, the quick fix for this would be to properly set the correct
PREFERRED_PROVIDERhere, but I could imagine that we run into similar issues in other builds, too.Do you have a good solution for this at hand? Like defining a standard preferred provider for virtual/bootloader?
Quoted 6 lines
I guess one could argue possibly that the mechanism does exactly what it is meant for. And a few machines (like beaglebone-yocto or qemuloongarch) actually set the
PREFERRED_PROVIDER, but not all and this might lead to silently switching the bootloader (like in our tests).I hope you have a better overview on the topic than I have and could give me a hint on how you would like to see this being resolved?
I see that it fails to build with clang
https://errors.yoctoproject.org/Errors/Details/699467/
is it buildable with clang ? if not we can pin it to use gcc always in meta-clang but I want to make sure first.
Quoted 27 lines
Thanks in advance and best regards
Enrico
Quoted 5 lines
Cheers,
Richard
--
Pengutronix e.K. | Enrico Jörns | Embedded Linux Consulting & Support | https://www.pengutronix.de/ | Steuerwalder Str. 21 | Phone: +49-5121-206917-180 | 31137 Hildesheim, Germany | Fax: +49-5121-206917-9 |-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#179655): https://lists.openembedded.org/g/openembedded-core/message/179655
Mute This Topic: https://lists.openembedded.org/mt/97970644/1997914
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Hi Khem,
Quoted 7 lines
Am Montag, dem 03.04.2023 um 19:33 -0700 schrieb Khem Raj:
[...]
I see that it fails to build with clang
https://errors.yoctoproject.org/Errors/Details/699467/
thanks for testing this!
Quoted 2 lines
is it buildable with clang ? if not we can pin it to use gcc always in meta-clang but I want to make sure first.
barebox itself is unfortunately not buildable with clang since on some architectures like ARM32 it uses GCC extensions that are not implemented by clang.
The barebox-tools however that are built for 'sandbox' should be buildable with clang. It's just that when passing CROSS_COMPILE, barebox cannot guess if it has to append 'clang' or 'gcc' suffix. The kernel seems to have an explicit handling for this by checking for "ifneq ($(LLVM),)" and plugging together CC differently.
With barebox we could however still override the Makefile's default "CC = $(CROSS_COMPILE)gcc" when passing CC explicitly to EXTRA_OEMAKE. We have just successfully tested this patch:
meta/recipes-bsp/barebox/barebox-tools_2023.03.0.bb +2 -0
@@ -18,6 +18,8 @@ EXTRA_OEMAKE = " \ ARCH=sandbox \ CROSS_COMPILE=${TARGET_PREFIX} -C ${S} O=${B} \ CROSS_PKG_CONFIG=pkg-config \ + CC='${CC}' \ + LD='${LD}' \ "
If that looks fine for you, I could add it to the recipe.
Best regards, Enrico
Quoted 29 lines
Quoted 28 lines
Thanks in advance and best regards
Enrico
Quoted 5 lines
Cheers,
Richard
--
Pengutronix e.K. | Enrico Jörns | Embedded Linux Consulting & Support | https://www.pengutronix.de/ | Steuerwalder Str. 21 | Phone: +49-5121-206917-180 | 31137 Hildesheim, Germany | Fax: +49-5121-206917-9 |-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#179655): https://lists.openembedded.org/g/openembedded-core/message/179655
Mute This Topic: https://lists.openembedded.org/mt/97970644/1997914
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Signature · 4 lines
Pengutronix e.K. | Enrico Jörns | Embedded Linux Consulting & Support | https://www.pengutronix.de/ | Steuerwalder Str. 21 | Phone: +49-5121-206917-180 | 31137 Hildesheim, Germany | Fax: +49-5121-206917-9 |
Quoted 38 lines · Enrico Jörns
Hi Khem,
Quoted 7 lines
Am Montag, dem 03.04.2023 um 19:33 -0700 schrieb Khem Raj:
[...]
I see that it fails to build with clang
https://errors.yoctoproject.org/Errors/Details/699467/thanks for testing this!
Quoted 2 lines
is it buildable with clang ? if not we can pin it to use gcc always in meta-clang but I want to make sure first.
barebox itself is unfortunately not buildable with clang since on some architectures like ARM32 it uses GCC extensions that are not implemented by clang.
The barebox-tools however that are built for 'sandbox' should be buildable with clang. It's just that when passing
CROSS_COMPILE, barebox cannot guess if it has to append 'clang' or 'gcc' suffix. The kernel seems to have an explicit handling for this by checking for "ifneq ($(LLVM),)" and plugging together CC differently.With barebox we could however still override the Makefile's default "CC = $(CROSS_COMPILE)gcc" when passing CC explicitly to
EXTRA_OEMAKE. We have just successfully tested this patch:meta/recipes-bsp/barebox/barebox-tools_2023.03.0.bb +2
-0@@ -18,6 +18,8 @@ EXTRA_OEMAKE = " \ ARCH=sandbox \ CROSS_COMPILE=${TARGET_PREFIX} -C ${S} O=${B} \ CROSS_PKG_CONFIG=pkg-config \ + CC='${CC}' \ + LD='${LD}' \ "
Quoted 2 lines
If that looks fine for you, I could add it to the recipe.
yeah I was testing one with CC alone and that worked too but also containing LD is good, so please include above
changes in next patchset
Quoted 39 lines
Best regards, Enrico
Quoted 20 lines
Quoted 20 lines
Thanks in advance and best regards
Enrico
Quoted 5 lines
Cheers,
Richard
--
Pengutronix e.K. | Enrico Jörns | Embedded Linux Consulting & Support | https://www.pengutronix.de/ | Steuerwalder Str. 21 | Phone: +49-5121-206917-180 | 31137 Hildesheim, Germany | Fax: +49-5121-206917-9 |Quoted 9 lines
Quoted 8 lines
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#179655): https://lists.openembedded.org/g/openembedded-core/message/179655
Mute This Topic: https://lists.openembedded.org/mt/97970644/1997914
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=---
Pengutronix e.K. | Enrico Jörns | Embedded Linux Consulting & Support | https://www.pengutronix.de/ | Steuerwalder Str. 21 | Phone: +49-5121-206917-180 | 31137 Hildesheim, Germany | Fax: +49-5121-206917-9 |