Move test/py image creation into separate modules
Subsystems (3)
v2 cover Move test/py image creation into separate modules
test_ut.py has accumulated a setup_*_image() helper for each image that the bootflow tests want to boot. Each one is mostly shell script and partition layout, so the file is large and the test logic is hidden among image-creation details.
Move each setup function to its own module under test/py/img/, with a shared common.py for mkdir_cond(), copy_partition() and the make_extlinux_disk() helper. Each patch moves one image at a time so reviewers can see the relocation cleanly. The modules are named after the bootmeth or feature they exercise rather than the distribution they are modelled on.
While here, also redirect the generated disk images to the persistent-data directory instead of writing them under the source tree. The sandbox mmc, scsi and usb-flash drivers now look there first.
Further improvements are planned, such as using the context manager when creating partitions.
Changes in v2:
- Rename the setup_extlinux_image() helper to make_extlinux_disk()
- Rename the module to extlinux and the function to setup_extlinux_image(), naming it after the bootmeth it tests rather than the Fedora distribution it is modelled on
- Rename the module to script and the function to setup_script_image(), naming it after the bootmeth it tests rather than the Armbian distribution it is modelled on
- Reword to drop the per-OS naming, since the modules are now named after the bootmeth or feature they test
Simon Glass (11):
test: Create a common file for image utilities
test: Move extlinux image-creation to its own file
test: Move script image-creation to its own file
test: Move ChromeOS image-creation to its own file
test: Move Android image-creation to its own file
test: Move EFI image-creation to its own file
test: Move the configuration-editor setup to its own file
test: Add Args docstrings to img setup functions
test: Reformat line wraps in img setup functions
sandbox: Find disk images in the persistent-data directory
test: Move disk images to persistent-data directory
MAINTAINERS | 5 + drivers/mmc/sandbox_mmc.c | 13 +- drivers/scsi/sandbox_scsi.c | 15 +- drivers/usb/emul/sandbox_flash.c | 14 +- test/py/img/android.py | 143 ++++++++ test/py/img/cedit.py | 24 ++ test/py/img/chromeos.py | 153 +++++++++ test/py/img/common.py | 86 +++++ test/py/img/efi.py | 37 +++ test/py/img/extlinux.py | 36 ++ test/py/img/script.py | 132 ++++++++ test/py/tests/test_ut.py | 541 +------------------------------ 12 files changed, 666 insertions(+), 533 deletions(-) create mode 100644 test/py/img/android.py create mode 100644 test/py/img/cedit.py create mode 100644 test/py/img/chromeos.py create mode 100644 test/py/img/common.py create mode 100644 test/py/img/efi.py create mode 100644 test/py/img/extlinux.py create mode 100644 test/py/img/script.py
---
base-commit: bacf09599a133f0274bc9fb8e67bfb444f057c16
branch: testc-us2
Signature · 1 line
2.43.0
PATCH v2 1/11 test: Create a common file for image utilities
Move mkdir_cond(), copy_partition(), and make_extlinux_disk() to a common module which can be used by the rest of the image-creation code.
Add myself as a maintainer for this new directory, and the test/py framework itself.
---
Changes in v2:
- Rename the setup_extlinux_image() helper to make_extlinux_disk()
MAINTAINERS | 5 +++ test/py/img/common.py | 86 ++++++++++++++++++++++++++++++++++++++++ test/py/tests/test_ut.py | 77 ++--------------------------------- 3 files changed, 94 insertions(+), 74 deletions(-) create mode 100644 test/py/img/common.py
MAINTAINERS +5 -0
@@ -1829,6 +1829,11 @@ M: Liviu Dudau <[email protected]> S: Maintained F: drivers/video/tda19988.c +TEST FRAMEWORK (PYTHON) +M: Simon Glass <[email protected]> +F: test/py +F: test/py/img + TI LP5562 LED DRIVER M: Rasmus Villemoes <[email protected]> S: Supported
test/py/img/common.py new file +86 -0
86 lines not on this page · raw message
test/py/tests/test_ut.py +3 -74
96 lines not on this page · raw message
Signature · 1 line
2.43.0
Re: [PATCH v2 01/11] test: Create a common file for image utilities
Hi Simon,
Thank you for the patch.
Quoted 23 lines · Simon Glass
Move
mkdir_cond(),copy_partition(), andmake_extlinux_disk()to a common module which can be used by the rest of the image-creation code.Add myself as a maintainer for this new directory, and the test/py framework itself.
Signed-off-by Simon Glass <[email protected]>---
Changes in v2:
- Rename thesetup_extlinux_image()helper tomake_extlinux_disk()MAINTAINERS | 5 +++ test/py/img/common.py | 86 ++++++++++++++++++++++++++++++++++++++++ test/py/tests/test_ut.py | 77 ++--------------------------------- 3 files changed, 94 insertions(+), 74 deletions(-) create mode 100644 test/py/img/common.pyMAINTAINERS +0
-0
MAINTAINERS M: Liviu Dudau <[email protected]> · line 1833
Quoted 5 lines
MAINTAINERS +2
-0@@ -1829,6 +1829,11 @@ M: Liviu Dudau <[email protected]> S: Maintained F: drivers/video/tda19988.c +TEST FRAMEWORK (PYTHON) +M: Simon Glass <[email protected]>
Add a S: line here as well, otherwise get_maintainers.pl reports this as "unknown"
$ ./scripts/get_maintainer.pl -f test/py Simon Glass <[email protected]> (unknown:TEST FRAMEWORK (PYTHON),commit_signer:51/90=57%,authored:42/90=47%)
When S: line is present:
$ ./scripts/get_maintainer.pl -f test/py Simon Glass <[email protected]> (maintainer:TEST FRAMEWORK (PYTHON),commit_signer:51/90=57%,authored:42/90=47%)
With the above change:
Quoted 12 lines
MAINTAINERS +3
-0+F: test/py +F: test/py/img + TI LP5562 LED DRIVER M: Rasmus Villemoes <[email protected]> S: Supportedtest/py/img/common.py new file +0
-0@@ -0,0 +1,86 @@
PATCH v2 2/11 test: Move extlinux image-creation to its own file
In reply to Simon GlassMove setup_extlinux_image() to its own module. The image exercises the extlinux bootmeth and is modelled on Fedora 31, so name the module and function after what they test rather than the distribution.
---
Changes in v2:
- Rename the module to extlinux and the function to setup_extlinux_image(), naming it after the bootmeth it tests rather than the Fedora distribution it is modelled on
test/py/img/extlinux.py | 36 ++++++++++++++++++++++++++++++++++++ test/py/tests/test_ut.py | 32 +++----------------------------- 2 files changed, 39 insertions(+), 29 deletions(-) create mode 100644 test/py/img/extlinux.py
test/py/img/extlinux.py new file +36 -0
@@ -0,0 +1,36 @@ +# SPDX-License-Identifier: GPL-2.0+ +# Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved. + +"""Create extlinux test disk images""" + +from img.common import make_extlinux_disk + + +def setup_extlinux_image(ubman, devnum, basename): + """Create a 20MB disk image for the extlinux bootmeth + + This is modelled on Fedora 31 + + Args: + ubman (ConsoleBase): Console to use + devnum (int): Device number to use, e.g. 1 + basename (str): Base name to use in the filename, e.g. 'mmc' + """ + vmlinux = 'vmlinuz-5.3.7-301.fc31.armv7hl' + initrd = 'initramfs-5.3.7-301.fc31.armv7hl.img' + dtbdir = 'dtb-5.3.7-301.fc31.armv7hl' + script = '''# extlinux.conf generated by appliance-creator +ui menu.c32 +menu autoboot Welcome to Fedora-Workstation-armhfp-31-1.9. Automatic boot in # second{,s}. Press a key for options. +menu title Fedora-Workstation-armhfp-31-1.9 Boot Options. +menu hidden +timeout 20 +totaltimeout 600 + +label Fedora-Workstation-armhfp-31-1.9 (5.3.7-301.fc31.armv7hl) + kernel /%s + append ro root=UUID=9732b35b-4cd5-458b-9b91-80f7047e0b8a rhgb quiet LANG=en_US.UTF-8 cma=192MB cma=256MB + fdtdir /%s/ + initrd /%s''' % (vmlinux, dtbdir, initrd) + make_extlinux_disk(ubman, devnum, basename, vmlinux, initrd, dtbdir, + script)
test/py/tests/test_ut.py +3 -29
@@ -19,7 +19,8 @@ import utils from tests import fs_helper from fs_helper import DiskHelper, FsHelper from test_android import test_abootimg -from img.common import mkdir_cond, copy_partition, make_extlinux_disk +from img.common import mkdir_cond, copy_partition +from img.extlinux import setup_extlinux_image def setup_bootmenu_image(ubman):
@@ -141,33 +142,6 @@ booti ${kernel_addr_r} ${ramdisk_addr_r} ${fdt_addr_r} fsh.cleanup() -def setup_fedora_image(ubman, devnum, basename): - """Create a 20MB Fedora disk image with a single FAT partition - - Args: - ubman (ConsoleBase): Console to use - devnum (int): Device number to use, e.g. 1 - basename (str): Base name to use in the filename, e.g. 'mmc' - """ - vmlinux = 'vmlinuz-5.3.7-301.fc31.armv7hl' - initrd = 'initramfs-5.3.7-301.fc31.armv7hl.img' - dtbdir = 'dtb-5.3.7-301.fc31.armv7hl' - script = '''# extlinux.conf generated by appliance-creator -ui menu.c32 -menu autoboot Welcome to Fedora-Workstation-armhfp-31-1.9. Automatic boot in # second{,s}. Press a key for options. -menu title Fedora-Workstation-armhfp-31-1.9 Boot Options. -menu hidden -timeout 20 -totaltimeout 600 - -label Fedora-Workstation-armhfp-31-1.9 (5.3.7-301.fc31.armv7hl) - kernel /%s - append ro root=UUID=9732b35b-4cd5-458b-9b91-80f7047e0b8a rhgb quiet LANG=en_US.UTF-8 cma=192MB cma=256MB - fdtdir /%s/ - initrd /%s''' % (vmlinux, dtbdir, initrd) - make_extlinux_disk(ubman, devnum, basename, vmlinux, initrd, dtbdir, - script) - def setup_cros_image(ubman): """Create a 20MB disk image with ChromiumOS partitions""" Partition = collections.namedtuple('part', 'start,size,name')
@@ -548,7 +522,7 @@ def setup_rauc_image(ubman): def test_ut_dm_init_bootstd(ubman): """Initialise data for bootflow tests""" - setup_fedora_image(ubman, 1, 'mmc') + setup_extlinux_image(ubman, 1, 'mmc') setup_bootmenu_image(ubman) setup_cedit_file(ubman) setup_cros_image(ubman)
Signature · 1 line
2.43.0
Re: [PATCH v2 02/11] test: Move extlinux image-creation to its own file
Hi Simon,
Thank you for the patch.
Quoted 5 lines · Simon Glass
Move
setup_extlinux_image()to its own module. The image exercises the extlinux bootmeth and is modelled on Fedora 31, so name the module and function after what they test rather than the distribution.Signed-off-by Simon Glass <[email protected]>
Quoted 2 lines
---
PATCH v2 3/11 test: Move script image-creation to its own file
In reply to Simon GlassMove setup_script_image() to its own module. The image exercises the script bootmeth and is modelled on Armbian 22.08 Jammy, so name the module and function after what they test rather than the distribution.
---
Changes in v2:
- Rename the module to script and the function to setup_script_image(), naming it after the bootmeth it tests rather than the Armbian distribution it is modelled on
test/py/img/script.py | 129 +++++++++++++++++++++++++++++++++++++++ test/py/tests/test_ut.py | 122 +----------------------------------- 2 files changed, 131 insertions(+), 120 deletions(-) create mode 100644 test/py/img/script.py
test/py/img/script.py new file +129 -0
@@ -0,0 +1,129 @@ +# SPDX-License-Identifier: GPL-2.0+ +# Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved. + +"""Create script test disk images""" + +import os + +import utils +from fs_helper import DiskHelper, FsHelper +from img.common import mkdir_cond + + +def setup_script_image(ubman): + """Create a 20MB disk image for the script bootmeth + + This is modelled on Armbian 22.08 Jammy + """ + mmc_dev = 4 + + script = '''# DO NOT EDIT THIS FILE +# +# Please edit /boot/armbianEnv.txt to set supported parameters +# + +setenv load_addr "0x9000000" +setenv overlay_error "false" +# default values +setenv rootdev "/dev/mmcblk%dp1" +setenv verbosity "1" +setenv console "both" +setenv bootlogo "false" +setenv rootfstype "ext4" +setenv docker_optimizations "on" +setenv earlycon "off" + +echo "Boot script loaded from ${devtype} ${devnum}" + +if test -e ${devtype} ${devnum} ${prefix}armbianEnv.txt; then + load ${devtype} ${devnum} ${load_addr} ${prefix}armbianEnv.txt + env import -t ${load_addr} ${filesize} +fi + +if test "${logo}" = "disabled"; then setenv logo "logo.nologo"; fi + +if test "${console}" = "display" || test "${console}" = "both"; then setenv consoleargs "console=tty1"; fi +if test "${console}" = "serial" || test "${console}" = "both"; then setenv consoleargs "console=ttyS2,1500000 ${consoleargs}"; fi +if test "${earlycon}" = "on"; then setenv consoleargs "earlycon ${consoleargs}"; fi +if test "${bootlogo}" = "true"; then setenv consoleargs "bootsplash.bootfile=bootsplash.armbian ${consoleargs}"; fi + +# get PARTUUID of first partition on SD/eMMC the boot script was loaded from +if test "${devtype}" = "mmc"; then part uuid mmc ${devnum}:1 partuuid; fi + +setenv bootargs "root=${rootdev} rootwait rootfstype=${rootfstype} ${consoleargs} consoleblank=0 loglevel=${verbosity} ubootpart=${partuuid} usb-storage.quirks=${usbstoragequirks} ${extraargs} ${extraboardargs}" + +if test "${docker_optimizations}" = "on"; then setenv bootargs "${bootargs} cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory swapaccount=1"; fi + +load ${devtype} ${devnum} ${ramdisk_addr_r} ${prefix}uInitrd +load ${devtype} ${devnum} ${kernel_addr_r} ${prefix}Image + +load ${devtype} ${devnum} ${fdt_addr_r} ${prefix}dtb/${fdtfile} +fdt addr ${fdt_addr_r} +fdt resize 65536 +for overlay_file in ${overlays}; do + if load ${devtype} ${devnum} ${load_addr} ${prefix}dtb/rockchip/overlay/${overlay_prefix}-${overlay_file}.dtbo; then + echo "Applying kernel provided DT overlay ${overlay_prefix}-${overlay_file}.dtbo" + fdt apply ${load_addr} || setenv overlay_error "true" + fi +done +for overlay_file in ${user_overlays}; do + if load ${devtype} ${devnum} ${load_addr} ${prefix}overlay-user/${overlay_file}.dtbo; then + echo "Applying user provided DT overlay ${overlay_file}.dtbo" + fdt apply ${load_addr} || setenv overlay_error "true" + fi +done +if test "${overlay_error}" = "true"; then + echo "Error applying DT overlays, restoring original DT" + load ${devtype} ${devnum} ${fdt_addr_r} ${prefix}dtb/${fdtfile} +else + if load ${devtype} ${devnum} ${load_addr} ${prefix}dtb/rockchip/overlay/${overlay_prefix}-fixup.scr; then + echo "Applying kernel provided DT fixup script (${overlay_prefix}-fixup.scr)" + source ${load_addr} + fi + if test -e ${devtype} ${devnum} ${prefix}fixup.scr; then + load ${devtype} ${devnum} ${load_addr} ${prefix}fixup.scr + echo "Applying user provided fixup script (fixup.scr)" + source ${load_addr} + fi +fi +booti ${kernel_addr_r} ${ramdisk_addr_r} ${fdt_addr_r} + +# Recompile with: +# mkimage -C none -A arm -T script -d /boot/boot.cmd /boot/boot.scr +''' + fsh = FsHelper(ubman.config, 'ext4', 18, 'mmc') + fsh.setup() + bootdir = os.path.join(fsh.srcdir, 'boot') + mkdir_cond(bootdir) + cmd_fname = os.path.join(bootdir, 'boot.cmd') + scr_fname = os.path.join(bootdir, 'boot.scr') + with open(cmd_fname, 'w', encoding='ascii') as outf: + print(script, file=outf) + + infname = os.path.join(ubman.config.source_dir, + 'test/py/tests/bootstd/armbian.bmp.xz') + bmp_file = os.path.join(bootdir, 'boot.bmp') + utils.run_and_log( + ubman, + ['sh', '-c', f'xz -dc {infname} >{bmp_file}']) + + mkimage = ubman.config.build_dir + '/tools/mkimage' + utils.run_and_log( + ubman, f'{mkimage} -C none -A arm -T script -d {cmd_fname} {scr_fname}') + + kernel = 'vmlinuz-5.15.63-rockchip64' + target = os.path.join(bootdir, kernel) + with open(target, 'wb') as outf: + print('kernel', outf) + + symlink = os.path.join(bootdir, 'Image') + if os.path.exists(symlink): + os.remove(symlink) + utils.run_and_log( + ubman, f'echo here {kernel} {symlink}') + os.symlink(kernel, symlink) + fsh.mk_fs() + img = DiskHelper(ubman.config, mmc_dev, 'mmc', True) + img.add_fs(fsh, DiskHelper.EXT4) + img.create() + fsh.cleanup()
test/py/tests/test_ut.py +2 -120
134 lines not on this page · raw message
Signature · 1 line
2.43.0
Re: [PATCH v2 03/11] test: Move script image-creation to its own file
Hi Simon,
Thank you for the patch.
Quoted 5 lines · Simon Glass
Move
setup_script_image()to its own module. The image exercises the script bootmeth and is modelled on Armbian 22.08 Jammy, so name the module and function after what they test rather than the distribution.Signed-off-by Simon Glass <[email protected]>
Quoted 2 lines
---
PATCH v2 4/11 test: Move ChromeOS image-creation to its own file
In reply to Simon GlassMove setup_cros_image() to its own module. This function creates disk images with a ChromiumOS partition layout.
---
(no changes since v1)
test/py/img/chromeos.py | 149 +++++++++++++++++++++++++++++++++++++++ test/py/tests/test_ut.py | 140 +----------------------------------- 2 files changed, 150 insertions(+), 139 deletions(-) create mode 100644 test/py/img/chromeos.py
test/py/img/chromeos.py new file +149 -0
@@ -0,0 +1,149 @@ +# SPDX-License-Identifier: GPL-2.0+ +# Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved. + +"""Create ChromeOS test disk images""" + +import collections +import os + +import utils + + +def setup_cros_image(ubman): + """Create a 20MB disk image with ChromiumOS partitions""" + Partition = collections.namedtuple('part', 'start,size,name') + parts = {} + disk_data = None + + def pack_kernel(ubman, arch, kern, dummy): + """Pack a kernel containing some fake data + + Args: + ubman (ConsoleBase): Console to use + arch (str): Architecture to use ('x86' or 'arm') + kern (str): Filename containing kernel + dummy (str): Dummy filename to use for config and bootloader + + Return: + bytes: Packed-kernel data + """ + kern_part = os.path.join(ubman.config.result_dir, + f'kern-part-{arch}.bin') + utils.run_and_log( + ubman, + f'futility vbutil_kernel --pack {kern_part} ' + '--keyblock doc/chromium/files/devkeys/kernel.keyblock ' + '--signprivate doc/chromium/files/devkeys/kernel_data_key.vbprivk ' + f'--version 1 --config {dummy} --bootloader {dummy} ' + f'--vmlinuz {kern}') + + with open(kern_part, 'rb') as inf: + kern_part_data = inf.read() + return kern_part_data + + def set_part_data(partnum, data): + """Set the contents of a disk partition + + This updates disk_data by putting data in the right place + + Args: + partnum (int): Partition number to set + data (bytes): Data for that partition + """ + nonlocal disk_data + + start = parts[partnum].start * sect_size + disk_data = disk_data[:start] + data + disk_data[start + len(data):] + + mmc_dev = 5 + fname = os.path.join(ubman.config.source_dir, f'mmc{mmc_dev}.img') + utils.run_and_log(ubman, f'qemu-img create {fname} 20M') + utils.run_and_log(ubman, f'cgpt create {fname}') + + uuid_state = 'ebd0a0a2-b9e5-4433-87c0-68b6b72699c7' + uuid_kern = 'fe3a2a5d-4f32-41a7-b725-accc3285a309' + uuid_root = '3cb8e202-3b7e-47dd-8a3c-7ff2a13cfcec' + uuid_rwfw = 'cab6e88e-abf3-4102-a07a-d4bb9be3c1d3' + uuid_reserved = '2e0a753d-9e48-43b0-8337-b15192cb1b5e' + uuid_efi = 'c12a7328-f81f-11d2-ba4b-00a0c93ec93b' + + ptr = 40 + + # Number of sectors in 1MB + sect_size = 512 + sect_1mb = (1 << 20) // sect_size + + required_parts = [ + {'num': 0xb, 'label':'RWFW', 'type': uuid_rwfw, 'size': '1'}, + {'num': 6, 'label':'KERN_C', 'type': uuid_kern, 'size': '1'}, + {'num': 7, 'label':'ROOT_C', 'type': uuid_root, 'size': '1'}, + {'num': 9, 'label':'reserved', 'type': uuid_reserved, 'size': '1'}, + {'num': 0xa, 'label':'reserved', 'type': uuid_reserved, 'size': '1'}, + + {'num': 2, 'label':'KERN_A', 'type': uuid_kern, 'size': '1M'}, + {'num': 4, 'label':'KERN_B', 'type': uuid_kern, 'size': '1M'}, + + {'num': 8, 'label':'OEM', 'type': uuid_state, 'size': '1M'}, + {'num': 0xc, 'label':'EFI-SYSTEM', 'type': uuid_efi, 'size': '1M'}, + + {'num': 5, 'label':'ROOT_B', 'type': uuid_root, 'size': '1'}, + {'num': 3, 'label':'ROOT_A', 'type': uuid_root, 'size': '1'}, + {'num': 1, 'label':'STATE', 'type': uuid_state, 'size': '1M'}, + ] + + for part in required_parts: + size_str = part['size'] + if 'M' in size_str: + size = int(size_str[:-1]) * sect_1mb + else: + size = int(size_str) + utils.run_and_log( + ubman, + f"cgpt add -i {part['num']} -b {ptr} -s {size} -t {part['type']} {fname}") + ptr += size + + utils.run_and_log(ubman, f'cgpt boot -p {fname}') + out = utils.run_and_log(ubman, f'cgpt show -q {fname}') + + # We expect something like this: + # 8239 2048 1 Basic data + # 45 2048 2 ChromeOS kernel + # 8238 1 3 ChromeOS rootfs + # 2093 2048 4 ChromeOS kernel + # 8237 1 5 ChromeOS rootfs + # 41 1 6 ChromeOS kernel + # 42 1 7 ChromeOS rootfs + # 4141 2048 8 Basic data + # 43 1 9 ChromeOS reserved + # 44 1 10 ChromeOS reserved + # 40 1 11 ChromeOS firmware + # 6189 2048 12 EFI System Partition + + # Create a dict (indexed by partition number) containing the above info + for line in out.splitlines(): + start, size, num, name = line.split(maxsplit=3) + parts[int(num)] = Partition(int(start), int(size), name) + + # Set up the kernel command-line + dummy = os.path.join(ubman.config.result_dir, 'dummy.txt') + with open(dummy, 'wb') as outf: + outf.write(b'BOOT_IMAGE=/vmlinuz-5.15.0-121-generic root=/dev/nvme0n1p1 ro quiet splash vt.handoff=7') + + # For now we just use dummy kernels. This limits testing to just detecting + # a signed kernel. We could add support for the x86 data structures so that + # testing could cover getting the cmdline, setup.bin and other pieces. + kern = os.path.join(ubman.config.result_dir, 'kern.bin') + with open(kern, 'wb') as outf: + outf.write(b'kernel\n') + + with open(fname, 'rb') as inf: + disk_data = inf.read() + + # put x86 kernel in partition 2 and arm one in partition 4 + set_part_data(2, pack_kernel(ubman, 'x86', kern, dummy)) + set_part_data(4, pack_kernel(ubman, 'arm', kern, dummy)) + + with open(fname, 'wb') as outf: + outf.write(disk_data) + + return fname
test/py/tests/test_ut.py +1 -139
148 lines not on this page · raw message
Signature · 1 line
2.43.0
Re: [PATCH v2 04/11] test: Move ChromeOS image-creation to its own file
Hi Simon,
Thank you for the patch.
Quoted 4 lines · Simon Glass
Move
setup_cros_image()to its own module. This function creates disk images with a ChromiumOS partition layout.Signed-off-by Simon Glass <[email protected]>
Quoted 1 line
---
PATCH v2 5/11 test: Move Android image-creation to its own file
In reply to Simon GlassMove setup_android_image() function to its own module.
---
(no changes since v1)
test/py/img/android.py | 138 +++++++++++++++++++++++++++++++++++++++ test/py/tests/test_ut.py | 128 +----------------------------------- 2 files changed, 139 insertions(+), 127 deletions(-) create mode 100644 test/py/img/android.py
test/py/img/android.py new file +138 -0
@@ -0,0 +1,138 @@ +# SPDX-License-Identifier: GPL-2.0+ +# Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved. + +"""Create Android test disk images""" + +import collections +import os + +import utils +from test_android import test_abootimg + + +def setup_android_image(ubman): + """Create a 20MB disk image with Android partitions""" + Partition = collections.namedtuple('part', 'start,size,name') + parts = {} + disk_data = None + + def set_part_data(partnum, data): + """Set the contents of a disk partition + + This updates disk_data by putting data in the right place + + Args: + partnum (int): Partition number to set + data (bytes): Data for that partition + """ + nonlocal disk_data + + start = parts[partnum].start * sect_size + disk_data = disk_data[:start] + data + disk_data[start + len(data):] + + mmc_dev = 7 + fname = os.path.join(ubman.config.source_dir, f'mmc{mmc_dev}.img') + utils.run_and_log(ubman, f'qemu-img create {fname} 20M') + utils.run_and_log(ubman, f'cgpt create {fname}') + + ptr = 40 + + # Number of sectors in 1MB + sect_size = 512 + sect_1mb = (1 << 20) // sect_size + + required_parts = [ + {'num': 1, 'label':'misc', 'size': '1M'}, + {'num': 2, 'label':'boot_a', 'size': '4M'}, + {'num': 3, 'label':'boot_b', 'size': '4M'}, + {'num': 4, 'label':'vendor_boot_a', 'size': '4M'}, + {'num': 5, 'label':'vendor_boot_b', 'size': '4M'}, + ] + + for part in required_parts: + size_str = part['size'] + if 'M' in size_str: + size = int(size_str[:-1]) * sect_1mb + else: + size = int(size_str) + utils.run_and_log( + ubman, + f"cgpt add -i {part['num']} -b {ptr} -s {size} -l {part['label']} -t basicdata {fname}") + ptr += size + + utils.run_and_log(ubman, f'cgpt boot -p {fname}') + out = utils.run_and_log(ubman, f'cgpt show -q {fname}') + + # Create a dict (indexed by partition number) containing the above info + for line in out.splitlines(): + start, size, num, name = line.split(maxsplit=3) + parts[int(num)] = Partition(int(start), int(size), name) + + with open(fname, 'rb') as inf: + disk_data = inf.read() + + test_abootimg.AbootimgTestDiskImage(ubman, 'bootv4.img', test_abootimg.boot_img_hex) + boot_img = os.path.join(ubman.config.result_dir, 'bootv4.img') + with open(boot_img, 'rb') as inf: + set_part_data(2, inf.read()) + + test_abootimg.AbootimgTestDiskImage(ubman, 'vendor_boot.img', test_abootimg.vboot_img_hex) + vendor_boot_img = os.path.join(ubman.config.result_dir, 'vendor_boot.img') + with open(vendor_boot_img, 'rb') as inf: + set_part_data(4, inf.read()) + + with open(fname, 'wb') as outf: + outf.write(disk_data) + + print(f'wrote to {fname}') + + mmc_dev = 8 + fname = os.path.join(ubman.config.source_dir, f'mmc{mmc_dev}.img') + utils.run_and_log(ubman, f'qemu-img create {fname} 20M') + utils.run_and_log(ubman, f'cgpt create {fname}') + + ptr = 40 + + # Number of sectors in 1MB + sect_size = 512 + sect_1mb = (1 << 20) // sect_size + + required_parts = [ + {'num': 1, 'label':'misc', 'size': '1M'}, + {'num': 2, 'label':'boot_a', 'size': '4M'}, + {'num': 3, 'label':'boot_b', 'size': '4M'}, + ] + + for part in required_parts: + size_str = part['size'] + if 'M' in size_str: + size = int(size_str[:-1]) * sect_1mb + else: + size = int(size_str) + utils.run_and_log( + ubman, + f"cgpt add -i {part['num']} -b {ptr} -s {size} -l {part['label']} -t basicdata {fname}") + ptr += size + + utils.run_and_log(ubman, f'cgpt boot -p {fname}') + out = utils.run_and_log(ubman, f'cgpt show -q {fname}') + + # Create a dict (indexed by partition number) containing the above info + for line in out.splitlines(): + start, size, num, name = line.split(maxsplit=3) + parts[int(num)] = Partition(int(start), int(size), name) + + with open(fname, 'rb') as inf: + disk_data = inf.read() + + test_abootimg.AbootimgTestDiskImage(ubman, 'boot.img', test_abootimg.img_hex) + boot_img = os.path.join(ubman.config.result_dir, 'boot.img') + with open(boot_img, 'rb') as inf: + set_part_data(2, inf.read()) + + with open(fname, 'wb') as outf: + outf.write(disk_data) + + print(f'wrote to {fname}') + + return fname
test/py/tests/test_ut.py +1 -127
136 lines not on this page · raw message
Signature · 1 line
2.43.0
Re: [PATCH v2 05/11] test: Move Android image-creation to its own file
Hi Simon,
Thank you for the patch.
Quoted 3 lines · Simon Glass
Move
setup_android_image()function to its own module.Signed-off-by Simon Glass <[email protected]>
Quoted 2 lines
---
PATCH v2 6/11 test: Move EFI image-creation to its own file
In reply to Simon GlassMove setup_efi_image() to its own module.
---
(no changes since v1)
test/py/img/efi.py | 33 +++++++++++++++++++++++++++++++++ test/py/tests/test_ut.py | 25 +------------------------ 2 files changed, 34 insertions(+), 24 deletions(-) create mode 100644 test/py/img/efi.py
test/py/img/efi.py new file +33 -0
@@ -0,0 +1,33 @@ +# SPDX-License-Identifier: GPL-2.0+ +# Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved. + +"""Create EFI test disk images""" + +import os + +from fs_helper import DiskHelper, FsHelper +from img.common import mkdir_cond + + +def setup_efi_image(ubman): + """Create a 20MB disk image with an EFI app on it""" + devnum = 1 + fsh = FsHelper(ubman.config, 'vfat', 18, 'flash') + fsh.setup() + efi_dir = os.path.join(fsh.srcdir, 'EFI') + mkdir_cond(efi_dir) + bootdir = os.path.join(efi_dir, 'BOOT') + mkdir_cond(bootdir) + efi_src = os.path.join(ubman.config.build_dir, + 'lib/efi_loader/testapp.efi') + efi_dst = os.path.join(bootdir, 'BOOTSBOX.EFI') + with open(efi_src, 'rb') as inf: + with open(efi_dst, 'wb') as outf: + outf.write(inf.read()) + + fsh.mk_fs() + + img = DiskHelper(ubman.config, devnum, 'flash', True) + img.add_fs(fsh, DiskHelper.VFAT) + img.create() + fsh.cleanup()
test/py/tests/test_ut.py +1 -24
@@ -24,6 +24,7 @@ from img.extlinux import setup_extlinux_image from img.script import setup_script_image from img.chromeos import setup_cros_image from img.android import setup_android_image +from img.efi import setup_efi_image def setup_cedit_file(ubman):
@@ -79,30 +80,6 @@ def test_ut_dm_init(ubman): fh.write(data) -def setup_efi_image(ubman): - """Create a 20MB disk image with an EFI app on it""" - devnum = 1 - fsh = FsHelper(ubman.config, 'vfat', 18, 'flash') - fsh.setup() - efi_dir = os.path.join(fsh.srcdir, 'EFI') - mkdir_cond(efi_dir) - bootdir = os.path.join(efi_dir, 'BOOT') - mkdir_cond(bootdir) - efi_src = os.path.join(ubman.config.build_dir, - 'lib/efi_loader/testapp.efi') - efi_dst = os.path.join(bootdir, 'BOOTSBOX.EFI') - with open(efi_src, 'rb') as inf: - with open(efi_dst, 'wb') as outf: - outf.write(inf.read()) - - fsh.mk_fs() - - img = DiskHelper(ubman.config, devnum, 'flash', True) - img.add_fs(fsh, DiskHelper.VFAT) - img.create() - fsh.cleanup() - - def setup_rauc_image(ubman): """Create a 40MB disk image with an A/B RAUC system on it""" mmc_dev = 10
Signature · 1 line
2.43.0
Re: [PATCH v2 06/11] test: Move EFI image-creation to its own file
Hi Simon,
Thank you for the patch.
Quoted 3 lines · Simon Glass
Move
setup_efi_image()to its own module.Signed-off-by Simon Glass <[email protected]>
Quoted 2 lines
---
PATCH v2 7/11 test: Move the configuration-editor setup to its own file
In reply to Simon GlassMove setup_cedit_file() to its own module.
---
(no changes since v1)
test/py/img/cedit.py | 20 ++++++++++++++++++++ test/py/tests/test_ut.py | 12 +----------- 2 files changed, 21 insertions(+), 11 deletions(-) create mode 100644 test/py/img/cedit.py
test/py/img/cedit.py new file +20 -0
@@ -0,0 +1,20 @@ +# SPDX-License-Identifier: GPL-2.0+ +# Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved. + +"""Create configuration editor test files""" + +import os + +import utils + + +def setup_cedit_file(ubman): + """Set up a .dtb file for use with testing expo and configuration editor""" + infname = os.path.join(ubman.config.source_dir, + 'test/boot/files/expo_layout.dts') + inhname = os.path.join(ubman.config.source_dir, + 'test/boot/files/expo_ids.h') + expo_tool = os.path.join(ubman.config.source_dir, 'tools/expo.py') + outfname = 'cedit.dtb' + utils.run_and_log( + ubman, f'{expo_tool} -e {inhname} -l {infname} -o {outfname}')
test/py/tests/test_ut.py +1 -11
@@ -25,19 +25,9 @@ from img.script import setup_script_image from img.chromeos import setup_cros_image from img.android import setup_android_image from img.efi import setup_efi_image +from img.cedit import setup_cedit_file -def setup_cedit_file(ubman): - """Set up a .dtb file for use with testing expo and configuration editor""" - infname = os.path.join(ubman.config.source_dir, - 'test/boot/files/expo_layout.dts') - inhname = os.path.join(ubman.config.source_dir, - 'test/boot/files/expo_ids.h') - expo_tool = os.path.join(ubman.config.source_dir, 'tools/expo.py') - outfname = 'cedit.dtb' - utils.run_and_log( - ubman, f'{expo_tool} -e {inhname} -l {infname} -o {outfname}') - @pytest.mark.buildconfigspec('ut_dm') def test_ut_dm_init(ubman): """Initialize data for ut dm tests."""
Signature · 1 line
2.43.0
Re: [PATCH v2 07/11] test: Move the configuration-editor setup to its own file
Hi Simon,
Thank you for the patch.
Quoted 3 lines · Simon Glass
Move
setup_cedit_file()to its own module.Signed-off-by Simon Glass <[email protected]>
Quoted 1 line
---
PATCH v2 8/11 test: Add Args docstrings to img setup functions
In reply to Simon GlassThe image-setup functions in test/py/img/ have terse docstrings from their original location in test_ut.py
Add an Args block describing the ubman parameter so each function is self-documenting.
---
(no changes since v1)
test/py/img/android.py | 6 +++++- test/py/img/cedit.py | 6 +++++- test/py/img/chromeos.py | 6 +++++- test/py/img/efi.py | 6 +++++- test/py/img/script.py | 3 +++ 5 files changed, 23 insertions(+), 4 deletions(-)
test/py/img/android.py +5 -1
@@ -11,7 +11,11 @@ from test_android import test_abootimg def setup_android_image(ubman): - """Create a 20MB disk image with Android partitions""" + """Create a 20MB disk image with Android partitions + + Args: + ubman (ConsoleBase): Console to use + """ Partition = collections.namedtuple('part', 'start,size,name') parts = {} disk_data = None
test/py/img/cedit.py +5 -1
@@ -9,7 +9,11 @@ import utils def setup_cedit_file(ubman): - """Set up a .dtb file for use with testing expo and configuration editor""" + """Set up a .dtb file for use with testing expo and configuration editor + + Args: + ubman (ConsoleBase): Console to use + """ infname = os.path.join(ubman.config.source_dir, 'test/boot/files/expo_layout.dts') inhname = os.path.join(ubman.config.source_dir,
test/py/img/chromeos.py +5 -1
@@ -10,7 +10,11 @@ import utils def setup_cros_image(ubman): - """Create a 20MB disk image with ChromiumOS partitions""" + """Create a 20MB disk image with ChromiumOS partitions + + Args: + ubman (ConsoleBase): Console to use + """ Partition = collections.namedtuple('part', 'start,size,name') parts = {} disk_data = None
test/py/img/efi.py +5 -1
@@ -10,7 +10,11 @@ from img.common import mkdir_cond def setup_efi_image(ubman): - """Create a 20MB disk image with an EFI app on it""" + """Create a 20MB disk image with an EFI app on it + + Args: + ubman (ConsoleBase): Console to use + """ devnum = 1 fsh = FsHelper(ubman.config, 'vfat', 18, 'flash') fsh.setup()
test/py/img/script.py +3 -0
@@ -14,6 +14,9 @@ def setup_script_image(ubman): """Create a 20MB disk image for the script bootmeth This is modelled on Armbian 22.08 Jammy + + Args: + ubman (ConsoleBase): Console to use """ mmc_dev = 4
Signature · 1 line
2.43.0
Re: [PATCH v2 08/11] test: Add Args docstrings to img setup functions
Hi Simon,
Thank you for the patch.
Quoted 7 lines · Simon Glass
The image-setup functions in test/py/img/ have terse docstrings from their original location in
test_ut.pyAdd an Args block describing the ubman parameter so each function is self-documenting.
Signed-off-by Simon Glass <[email protected]>
PATCH v2 9/11 test: Reformat line wraps in img setup functions
In reply to Simon GlassSome call-sites use awkward line wraps inherited from test_ut.py
Adjust the wrapping so each call uses a more natural format with the first argument on the same line as the function name where possible, and arguments wrapped on natural boundaries elsewhere. No behaviour changes.
---
(no changes since v1)
test/py/img/android.py | 15 ++++++++------- test/py/img/cedit.py | 4 ++-- 2 files changed, 10 insertions(+), 9 deletions(-)
test/py/img/android.py +8 -7
@@ -59,8 +59,7 @@ def setup_android_image(ubman): size = int(size_str[:-1]) * sect_1mb else: size = int(size_str) - utils.run_and_log( - ubman, + utils.run_and_log(ubman, f"cgpt add -i {part['num']} -b {ptr} -s {size} -l {part['label']} -t basicdata {fname}") ptr += size
@@ -75,12 +74,14 @@ def setup_android_image(ubman): with open(fname, 'rb') as inf: disk_data = inf.read() - test_abootimg.AbootimgTestDiskImage(ubman, 'bootv4.img', test_abootimg.boot_img_hex) + test_abootimg.AbootimgTestDiskImage(ubman, 'bootv4.img', + test_abootimg.boot_img_hex) boot_img = os.path.join(ubman.config.result_dir, 'bootv4.img') with open(boot_img, 'rb') as inf: set_part_data(2, inf.read()) - test_abootimg.AbootimgTestDiskImage(ubman, 'vendor_boot.img', test_abootimg.vboot_img_hex) + test_abootimg.AbootimgTestDiskImage(ubman, 'vendor_boot.img', + test_abootimg.vboot_img_hex) vendor_boot_img = os.path.join(ubman.config.result_dir, 'vendor_boot.img') with open(vendor_boot_img, 'rb') as inf: set_part_data(4, inf.read())
@@ -113,8 +114,7 @@ def setup_android_image(ubman): size = int(size_str[:-1]) * sect_1mb else: size = int(size_str) - utils.run_and_log( - ubman, + utils.run_and_log(ubman, f"cgpt add -i {part['num']} -b {ptr} -s {size} -l {part['label']} -t basicdata {fname}") ptr += size
@@ -129,7 +129,8 @@ def setup_android_image(ubman): with open(fname, 'rb') as inf: disk_data = inf.read() - test_abootimg.AbootimgTestDiskImage(ubman, 'boot.img', test_abootimg.img_hex) + test_abootimg.AbootimgTestDiskImage(ubman, 'boot.img', + test_abootimg.img_hex) boot_img = os.path.join(ubman.config.result_dir, 'boot.img') with open(boot_img, 'rb') as inf: set_part_data(2, inf.read())
test/py/img/cedit.py +2 -2
@@ -20,5 +20,5 @@ def setup_cedit_file(ubman): 'test/boot/files/expo_ids.h') expo_tool = os.path.join(ubman.config.source_dir, 'tools/expo.py') outfname = 'cedit.dtb' - utils.run_and_log( - ubman, f'{expo_tool} -e {inhname} -l {infname} -o {outfname}') + utils.run_and_log(ubman, + f'{expo_tool} -e {inhname} -l {infname} -o {outfname}')
Signature · 1 line
2.43.0
Re: [PATCH v2 09/11] test: Reformat line wraps in img setup functions
Hi Simon,
Thank you for the patch.
Quoted 8 lines · Simon Glass
Some call-sites use awkward line wraps inherited from
test_ut.pyAdjust the wrapping so each call uses a more natural format with the first argument on the same line as the function name where possible, and arguments wrapped on natural boundaries elsewhere. No behaviour changes.
Signed-off-by Simon Glass <[email protected]>
Quoted 2 lines
---
PATCH v2 10/11 sandbox: Find disk images in the persistent-data directory
In reply to Simon GlassThe sandbox mmc, scsi and usb-flash drivers open their backing files from plat->pathname as given. Test images are about to move out of the source tree into the persistent-data directory, so the drivers need to find them there.
Try the persistent-data directory first via os_persistent_file() and fall back to the original pathname when the lookup fails, so absolute paths and files already in the current directory keep working.
---
(no changes since v1)
drivers/mmc/sandbox_mmc.c | 13 +++++++++++-- drivers/scsi/sandbox_scsi.c | 15 ++++++++++++--- drivers/usb/emul/sandbox_flash.c | 14 +++++++++++--- 3 files changed, 34 insertions(+), 8 deletions(-)
drivers/mmc/sandbox_mmc.c +11 -2
@@ -170,11 +170,20 @@ static int sandbox_mmc_probe(struct udevice *dev)int ret;if (plat->fname) { - ret =os_map_file (plat->fname,OS_O_RDWR |OS_O_CREAT , +const char *fname = plat->fname; +char buf[256 ]; + +/* +* Try persistent data directory first, then fall back to the +* filename as given (for absolute paths or current directory) +*/ +if (!os_persistent_file (buf,sizeof (buf), plat->fname)) + fname = buf; + ret =os_map_file (fname,OS_O_RDWR |OS_O_CREAT , (void **)&priv->buf, &priv->size);if (ret) {log_err ("%s: Unable to map file '%s'\n" , dev->name, - plat->fname); + fname);return ret; } priv->csize = priv->size /SIZE_MULTIPLE -1 ;
drivers/scsi/sandbox_scsi.c +12 -3
@@ -104,9 +104,18 @@ static int sandbox_scsi_probe(struct udevice *dev) info->block_size =SANDBOX_SCSI_BLOCK_LEN ;if (priv->pathname) { - priv->fd =os_open (priv->pathname,OS_O_RDONLY ); -if (priv->fd !=-1 ) { - ret =os_get_filesize (priv->pathname, &info->file_size); +const char *pathname = priv->pathname; +char buf[256 ]; + +/* +* Try persistent data directory first, then fall back to the +* pathname as given (for absolute paths or current directory) +*/ +if (!os_persistent_file (buf,sizeof (buf), priv->pathname)) + pathname = buf; + priv->fd =os_open (pathname,OS_O_RDONLY ); +if (priv->fd >=0 ) { + ret =os_get_filesize (pathname, &info->file_size);if (ret)return log_msg_ret ("sz" , ret); }
drivers/usb/emul/sandbox_flash.c +11 -3
@@ -339,11 +339,19 @@ static int sandbox_flash_probe(struct udevice *dev)struct sandbox_flash_plat *plat =dev_get_plat (dev);struct sandbox_flash_priv *priv =dev_get_priv (dev);struct scsi_emul_info *info = &priv->eminfo; +const char *pathname = plat->pathname; +char buf[256 ];int ret; - priv->fd =os_open (plat->pathname,OS_O_RDWR ); -if (priv->fd !=-1 ) { - ret =os_get_filesize (plat->pathname, &info->file_size); +/* +* Try persistent data directory first, then fall back to the +* pathname as given (for absolute paths or current directory) +*/ +if (!os_persistent_file (buf,sizeof (buf), plat->pathname)) + pathname = buf; + priv->fd =os_open (pathname,OS_O_RDWR ); +if (priv->fd >=0 ) { + ret =os_get_filesize (pathname, &info->file_size);if (ret)return log_msg_ret ("sz" , ret); }
Signature · 1 line
2.43.0
Re: [PATCH v2 10/11] sandbox: Find disk images in the persistent-data directory
Hi Simon,
Thank you for the patch.
Quoted 24 lines · Simon Glass
The sandbox mmc, scsi and usb-flash drivers open their backing files from
plat->pathnameas given. Test images are about to move out of the source tree into the persistent-data directory, so the drivers need to find them there.Try the persistent-data directory first via
os_persistent_file()and fall back to the original pathname when the lookup fails, so absolute paths and files already in the current directory keep working.Signed-off-by Simon Glass <[email protected]>---
(no changes since v1)
drivers/mmc/sandbox_mmc.c | 13 +++++++++++-- drivers/scsi/sandbox_scsi.c | 15 ++++++++++++--- drivers/usb/emul/sandbox_flash.c | 14 +++++++++++--- 3 files changed, 34 insertions(+), 8 deletions(-)drivers/mmc/sandbox_mmc.c +0
-0
drivers/mmc/sandbox_mmc.c static int sandbox_mmc_probe(struct udevice *dev) · line 174
Quoted 6 lines
drivers/mmc/sandbox_mmc.c +2
-1@@ -170,11 +170,20 @@ static int sandbox_mmc_probe(struct udevice *dev)int ret;if (plat->fname) { - ret = os_map_file(plat->fname, OS_O_RDWR | OS_O_CREAT, +const char *fname = plat->fname; +char buf[256 ];
Why 256? Can't we use a define/constant for this?
I see that fname ends up coming from ofnode_read_string() but it's unclear to me if there is a build-time constant size we could use instead.
Quoted 28 lines
drivers/mmc/sandbox_mmc.c +9
-1+ +/* +* Try persistent data directory first, then fall back to the +* filename as given (for absolute paths or current directory) +*/ +if (!os_persistent_file (buf,sizeof (buf), plat->fname)) + fname = buf; + ret =os_map_file (fname,OS_O_RDWR |OS_O_CREAT , (void **)&priv->buf, &priv->size);if (ret) {log_err ("%s: Unable to map file '%s'\n" , dev->name, - plat->fname); + fname);return ret; } priv->csize = priv->size /SIZE_MULTIPLE -1 ;drivers/scsi/sandbox_scsi.c +1
-3@@ -104,9 +104,18 @@ static int sandbox_scsi_probe(struct udevice *dev) info->block_size =SANDBOX_SCSI_BLOCK_LEN ;if (priv->pathname) { - priv->fd =os_open (priv->pathname,OS_O_RDONLY ); -if (priv->fd !=-1 ) { - ret =os_get_filesize (priv->pathname, &info->file_size); +const char *pathname = priv->pathname;
drivers/scsi/sandbox_scsi.c static int sandbox_scsi_probe(struct udevice *dev) · line 117
Quoted 10 lines
drivers/scsi/sandbox_scsi.c +10
-0+char buf[256 ]; + +/* +* Try persistent data directory first, then fall back to the +* pathname as given (for absolute paths or current directory) +*/ +if (!os_persistent_file (buf,sizeof (buf), priv->pathname)) + pathname = buf; + priv->fd =os_open (pathname,OS_O_RDONLY ); +if (priv->fd >=0 ) {
I've noticed that the error handling gets updated here (from != -1 to >= 0).
Any reason for doing that? If so, can you please mention it in the commit message?
os_open()'s docstring seems to mention:
* Return: file descriptor, or -1 on error
Quoted 17 lines
drivers/scsi/sandbox_scsi.c +1
-0+ ret =os_get_filesize (pathname, &info->file_size);if (ret)return log_msg_ret ("sz" , ret); }drivers/usb/emul/sandbox_flash.c +2
-1@@ -339,11 +339,19 @@ static int sandbox_flash_probe(struct udevice *dev)struct sandbox_flash_plat *plat =dev_get_plat (dev);struct sandbox_flash_priv *priv =dev_get_priv (dev);struct scsi_emul_info *info = &priv->eminfo; +const char *pathname = plat->pathname; +char buf[256 ];int ret; - priv->fd =os_open (plat->pathname,OS_O_RDWR );
drivers/usb/emul/sandbox_flash.c static int sandbox_flash_probe(struct udevice *dev) · line 353
Quoted 10 lines
drivers/usb/emul/sandbox_flash.c +8
-2-if (priv->fd !=-1 ) { - ret =os_get_filesize (plat->pathname, &info->file_size); +/* +* Try persistent data directory first, then fall back to the +* pathname as given (for absolute paths or current directory) +*/ +if (!os_persistent_file (buf,sizeof (buf), plat->pathname)) + pathname = buf; + priv->fd =os_open (pathname,OS_O_RDWR ); +if (priv->fd >=0 ) {
Same here.
Quoted 6 lines
drivers/usb/emul/sandbox_flash.c +1
-0+ ret =os_get_filesize (pathname, &info->file_size);if (ret)return log_msg_ret ("sz" , ret); }Signature · 1 line
2.43.0
PATCH v2 11/11 test: Move disk images to persistent-data directory
In reply to Simon GlassIt is annoying to have disk images in the source directory since it clutters up the working space.
Remove cur_dir=True from DiskHelper calls so disk images are written to the persistent-data directory instead.
Move scsi.img too (used by the bootstd tests) and mmc6.img (used by the MBR tests).
Add a few comments as to where the images are used.
This keeps the source tree clean and puts disk images in the same place as other test data.
---
(no changes since v1)
test/py/img/common.py | 2 +- test/py/img/efi.py | 2 +- test/py/img/script.py | 2 +- test/py/tests/test_ut.py | 11 +++++++---- 4 files changed, 10 insertions(+), 7 deletions(-)
test/py/img/common.py +1 -1
@@ -74,7 +74,7 @@ def make_extlinux_disk(ubman, devnum, basename, vmlinux, initrd, dtbdir, fsh.mk_fs() - img = DiskHelper(ubman.config, devnum, basename, True) + img = DiskHelper(ubman.config, devnum, basename) img.add_fs(fsh, DiskHelper.VFAT, bootable=True) ext4 = FsHelper(ubman.config, 'ext4', 1, prefix=basename)
test/py/img/efi.py +1 -1
@@ -31,7 +31,7 @@ def setup_efi_image(ubman): fsh.mk_fs() - img = DiskHelper(ubman.config, devnum, 'flash', True) + img = DiskHelper(ubman.config, devnum, 'flash') img.add_fs(fsh, DiskHelper.VFAT) img.create() fsh.cleanup()
test/py/img/script.py +1 -1
@@ -126,7 +126,7 @@ booti ${kernel_addr_r} ${ramdisk_addr_r} ${fdt_addr_r} ubman, f'echo here {kernel} {symlink}') os.symlink(kernel, symlink) fsh.mk_fs() - img = DiskHelper(ubman.config, mmc_dev, 'mmc', True) + img = DiskHelper(ubman.config, mmc_dev, 'mmc') img.add_fs(fsh, DiskHelper.EXT4) img.create() fsh.cleanup()
test/py/tests/test_ut.py +7 -4
@@ -32,7 +32,8 @@ from img.cedit import setup_cedit_file def test_ut_dm_init(ubman): """Initialize data for ut dm tests.""" - fn = ubman.config.source_dir + '/testflash.bin' + # This is used by flash-stick@0 in test.py + fn = ubman.config.persistent_data_dir + '/testflash.bin' if not os.path.exists(fn): data = b'this is a test' data += b'\x00' * ((4 * 1024 * 1024) - len(data))
@@ -45,8 +46,8 @@ def test_ut_dm_init(ubman): with open(fn, 'wb') as fh: fh.write(data) - # Create a file with a single partition - fn = ubman.config.source_dir + '/scsi.img' + # Create a file with a single partition (used by /scsi in test.dts) */ + fn = ubman.config.persistent_data_dir + '/scsi.img' if not os.path.exists(fn): data = b'\x00' * (2 * 1024 * 1024) with open(fn, 'wb') as fh:
@@ -54,11 +55,13 @@ def test_ut_dm_init(ubman): utils.run_and_log( ubman, f'sfdisk {fn}', stdin=b'type=83') + # These two are used by test/dm/host.c FsHelper(ubman.config, 'ext2', 2, '2MB').mk_fs() FsHelper(ubman.config, 'fat32', 1, '1MB').mk_fs() + # This is used by test/cmd/mbr.c mmc_dev = 6 - fn = os.path.join(ubman.config.source_dir, f'mmc{mmc_dev}.img') + fn = os.path.join(ubman.config.persistent_data_dir, f'mmc{mmc_dev}.img') data = b'\x00' * (12 * 1024 * 1024) with open(fn, 'wb') as fh: fh.write(data)
Signature · 1 line
2.43.0
Re: [PATCH v2 11/11] test: Move disk images to persistent-data directory
Hi Simon,
Thank you for the patch.
Quoted 9 lines · Simon Glass
It is annoying to have disk images in the source directory since it clutters up the working space.
Remove cur_dir=True from DiskHelper calls so disk images are written to the persistent-data directory instead.
Move scsi.img too (used by the bootstd tests) and mmc6.img (used by the MBR tests).
Quoted 6 lines
Add a few comments as to where the images are used.
This keeps the source tree clean and puts disk images in the same place as other test data.
Signed-off-by Simon Glass <[email protected]>
I've re-ran the following using this series:
$ git clean -xdf $ python3 -m venv venv && . venv/bin/activate && pip install -r test/py/requirements.txt && pip install setuptools && ./test/py/test.py --bd sandbox --build -k ut
Then, if I check the source folder, I still see some disk images:
$ ls *.img mmc10.img mmc1.img mmc5.img mmc7.img mmc8.img mmc9.img
Also see:
$ find -name '*.img' ./build-sandbox/persistent-data/test_efi_capsule.vfat.img ./build-sandbox/persistent-data/scsi.img ./build-sandbox/persistent-data/2MB.ext2.img ./build-sandbox/persistent-data/1MB.fat32.img ./build-sandbox/persistent-data/mmc6.img ./build-sandbox/persistent-data/mmc.vfat.tmp/initramfs-5.3.7-301.fc31.armv7hl.img ./build-sandbox/persistent-data/mmc1.img ./build-sandbox/persistent-data/mmc4.img ./build-sandbox/persistent-data/bootv4.img ./build-sandbox/persistent-data/vendor_boot.img ./build-sandbox/persistent-data/boot.img ./build-sandbox/persistent-data/flash1.img ./build-sandbox/bootv4.img ./build-sandbox/vendor_boot.img ./build-sandbox/boot.img ./mmc1.img ./mmc9.img ./mmc5.img ./mmc7.img ./mmc8.img ./mmc10.img
Is there a reason for not moving all the images?
I think that for this patch, we should at least ensure that all the mmc*.img are moved.
For example, mmc7.img and mmc8.img are used by Android and I believe they should be in build-sandbox/persistent-data/ as well.
Quoted 91 lines
---
(no changes since v1)
test/py/img/common.py | 2 +- test/py/img/efi.py | 2 +- test/py/img/script.py | 2 +- test/py/tests/test_ut.py | 11 +++++++---- 4 files changed, 10 insertions(+), 7 deletions(-)test/py/img/common.py +1
-1@@ -74,7 +74,7 @@ def make_extlinux_disk(ubman, devnum, basename, vmlinux, initrd, dtbdir, fsh.mk_fs() - img = DiskHelper(ubman.config, devnum, basename, True) + img = DiskHelper(ubman.config, devnum, basename) img.add_fs(fsh, DiskHelper.VFAT, bootable=True) ext4 = FsHelper(ubman.config, 'ext4', 1, prefix=basename)test/py/img/efi.py +1
-1@@ -31,7 +31,7 @@ def setup_efi_image(ubman): fsh.mk_fs() - img = DiskHelper(ubman.config, devnum, 'flash', True) + img = DiskHelper(ubman.config, devnum, 'flash') img.add_fs(fsh, DiskHelper.VFAT) img.create() fsh.cleanup()test/py/img/script.py +1
-1@@ -126,7 +126,7 @@ booti ${kernel_addr_r} ${ramdisk_addr_r} ${fdt_addr_r} ubman, f'echo here {kernel} {symlink}') os.symlink(kernel, symlink) fsh.mk_fs() - img = DiskHelper(ubman.config, mmc_dev, 'mmc', True) + img = DiskHelper(ubman.config, mmc_dev, 'mmc') img.add_fs(fsh, DiskHelper.EXT4) img.create() fsh.cleanup()test/py/tests/test_ut.py +7
-4@@ -32,7 +32,8 @@ from img.cedit import setup_cedit_file def test_ut_dm_init(ubman): """Initialize data for ut dm tests.""" - fn = ubman.config.source_dir + '/testflash.bin' + # This is used by flash-stick@0 in test.py + fn = ubman.config.persistent_data_dir + '/testflash.bin' if not os.path.exists(fn): data = b'this is a test' data += b'\x00' * ((4 * 1024 * 1024) - len(data))@@ -45,8 +46,8 @@ def test_ut_dm_init(ubman): with open(fn, 'wb') as fh: fh.write(data) - # Create a file with a single partition - fn = ubman.config.source_dir + '/scsi.img' + # Create a file with a single partition (used by /scsi in test.dts) */ + fn = ubman.config.persistent_data_dir + '/scsi.img' if not os.path.exists(fn): data = b'\x00' * (2 * 1024 * 1024) with open(fn, 'wb') as fh:@@ -54,11 +55,13 @@ def test_ut_dm_init(ubman): utils.run_and_log( ubman, f'sfdisk {fn}', stdin=b'type=83') + # These two are used by test/dm/host.c FsHelper(ubman.config, 'ext2', 2, '2MB').mk_fs() FsHelper(ubman.config, 'fat32', 1, '1MB').mk_fs() + # This is used by test/cmd/mbr.c mmc_dev = 6 - fn = os.path.join(ubman.config.source_dir, f'mmc{mmc_dev}.img') + fn = os.path.join(ubman.config.persistent_data_dir, f'mmc{mmc_dev}.img') data = b'\x00' * (12 * 1024 * 1024) with open(fn, 'wb') as fh: fh.write(data)Signature · 1 line
2.43.0
Hi Simon,
Thank you for the series.
Quoted 12 lines · Simon Glass
test_ut.pyhas accumulated a setup_*_image() helper for each image that the bootflow tests want to boot. Each one is mostly shell script and partition layout, so the file is large and the test logic is hidden among image-creation details.Move each setup function to its own module under test/py/img/, with a shared
common.pyformkdir_cond(),copy_partition()and themake_extlinux_disk()helper. Each patch moves one image at a time so reviewers can see the relocation cleanly. The modules are named after the bootmeth or feature they exercise rather than the distribution they are modelled on.
Quoted 6 lines
While here, also redirect the generated disk images to the persistent-data directory instead of writing them under the source tree. The sandbox mmc, scsi and usb-flash drivers now look there first.
Further improvements are planned, such as using the context manager when creating partitions.
I've tested this on sandbox using:
$ ./test/py/test.py --bd sandbox --build -k ut $ ./test/py/test.py --bd sandbox --build -k bootflow_android
Quoted 1 line
Hi Mattijs,
Quoted 26 lines · Mattijs Korpershoek
Hi Simon,
Thank you for the series.
Quoted 12 lines · Simon Glass
test_ut.pyhas accumulated a setup_*_image() helper for each image that the bootflow tests want to boot. Each one is mostly shell script and partition layout, so the file is large and the test logic is hidden among image-creation details.Move each setup function to its own module under test/py/img/, with a shared
common.pyformkdir_cond(),copy_partition()and themake_extlinux_disk()helper. Each patch moves one image at a time so reviewers can see the relocation cleanly. The modules are named after the bootmeth or feature they exercise rather than the distribution they are modelled on.Quoted 6 lines
While here, also redirect the generated disk images to the persistent-data directory instead of writing them under the source tree. The sandbox mmc, scsi and usb-flash drivers now look there first.
Further improvements are planned, such as using the context manager when creating partitions.
Quoted 6 lines
I've tested this on sandbox using:
$ ./test/py/test.py --bd sandbox --build -k ut $ ./test/py/test.py --bd sandbox --build -k bootflow_androidTested-by Mattijs Korpershoek <[email protected]>
I appreciate you reviewing this - quite tedious work!
Regards,
Simon
Hi Simon,
Quoted 30 lines · Simon Glass
Hi Mattijs,
Quoted 26 lines · Mattijs Korpershoek
Hi Simon,
Thank you for the series.
Quoted 18 lines · Simon Glass
test_ut.pyhas accumulated a setup_*_image() helper for each image that the bootflow tests want to boot. Each one is mostly shell script and partition layout, so the file is large and the test logic is hidden among image-creation details.Move each setup function to its own module under test/py/img/, with a shared
common.pyformkdir_cond(),copy_partition()and themake_extlinux_disk()helper. Each patch moves one image at a time so reviewers can see the relocation cleanly. The modules are named after the bootmeth or feature they exercise rather than the distribution they are modelled on.While here, also redirect the generated disk images to the persistent-data directory instead of writing them under the source tree. The sandbox mmc, scsi and usb-flash drivers now look there first.
Further improvements are planned, such as using the context manager when creating partitions.
Quoted 8 lines
Quoted 6 lines
I've tested this on sandbox using:
$ ./test/py/test.py --bd sandbox --build -k ut $ ./test/py/test.py --bd sandbox --build -k bootflow_androidTested-by Mattijs Korpershoek <[email protected]>I appreciate you reviewing this - quite tedious work!
Happy to help :)
Please also see the individual commits for review remarks. I've found a couple of things I had questions on.
Regards,
Mattijs
Quoted 3 lines
Regards,
Simon