imagemap: on-demand FIT loading from storage
v2 cover imagemap: on-demand FIT loading from storage
This series adds "imagemap", a small layer for reading FIT images directly from storage on demand instead of copying the whole image into RAM first. It keeps a translation table of the byte ranges already loaded, so a header probe, signature verification and the final load can reuse a single read, and it hooks transparently into fit_image_load() so all existing verification and decompression paths run unchanged for both RAM- and storage-backed images.
An imagemap device is created over a partition of a block device. That covers more than raw disks: an MTD partition is reached through mtdblock and a UBI volume through ubiblock, each exposed as a named block-device partition, so imagemap carries no storage-specific code of its own. NAND bad-block handling and wear-levelling stay with UBI, as they do elsewhere in U-Boot; on NOR a partition is read linearly with end-to-end integrity guaranteed by the FIT hash or signature.
To make those two paths reachable and testable, the first patches fill in the missing block devices: a non-NAND MTD master now gets an mtd_blk device automatically (patch 1), and attaching UBI now binds a ubi_blk device on any flash (patch 3, which makes the SPI-NAND-only eager binding redundant, dropped in patch 4).
The read path is built on the SPL struct spl_load_info abstraction and a new spl_load_region() helper. imagemap is full-U-Boot only; the SPL FIT loader is unaffected.
Testing (sandbox): "ut imagemap" (13 tests) covers the translation-table core plus real end-to-end reads of a NOR partition through mtd_blk and a UBI volume through ubiblock; a CONFIG_IMAGEMAP=n build is unaffected. The whole series passes CI [1].
On hardware it has been tested end-to-end on a BananaPi R3 (mt7986), booting a dm-verity-protected rootfs from every storage path the board
exposes: eMMC and SD (block partition), SPI-NAND (UBI volume via ubiblock) and SPI-NOR (raw MTD partition via mtdblock). The first in-tree consumer, a bootmeth for OpenWrt-style FIT firmware, will follow as a separate series.
Changes in v2 (since v1 [2]):
- Add two prerequisite fixes for the UBI block driver that enabling CONFIG_UBI_BLOCK in sandbox exposes: make it depend on CMD_UBI (its ubi_volume_read()/write() live in cmd/ubi.c, so a CONFIG_CMDLINE=n build otherwise fails to link), and stop the UBI partition scan from claiming any readable block device or dereferencing a NULL ubi_device.
- include/spl.h: express the block-length field directly with #if IS_ENABLED(CONFIG_SPL_LOAD_BLOCK) || CONFIG_IS_ENABLED(IMAGEMAP) instead of a SPL_LOAD_INFO_HAS_BL_LEN helper macro, and fold the note into the existing struct comment (per Tom Rini).
- boot: fit: fix a continuation-line alignment in fit_image_load() flagged by checkpatch (per Tom Rini).
This series began as part of a combined RFC that also carried the OpenWrt boot method and its bootdevs [3]. Following the review there it has been reworked substantially:
- Build on U-Boot's existing SPL external-data loading path rather than a new abstraction: imagemap now uses struct spl_load_info and a new spl_load_region() helper (suggested by Marek Vasut).
- Do not extend the bootm command with a storage source; on-demand loading hooks into fit_image_load() transparently, and the in-tree consumer is a bootstd boot method reached through bootflow/bootdev/bootmeth rather than new bootm syntax (per Tom Rini).
- Move the storage-backed path out of the long fit_image_load() into its own helper, gated with if()/tools_build() instead of an inline #if (per Simon Glass).
- Collapse the three storage-specific backends (block, MTD and UBI) into a single block backend: NOR is read through mtd_blk and SPI-NAND through UBI/ubiblock, so imagemap holds no storage-specific code.
- Send the on-demand loading core on its own; the OpenWrt boot method, the MTD/UBI bootdevs and board enablement follow as separate series once this lands.
A note on that follow-up: to tell Linux which FIT it booted, U-Boot uses the install-uuid mechanism of [4], which [5] could supersede. Both have stalled.
[1] https://git.u-boot-project.org/u-boot/contributors/dgolle/u-boot/-/pipelines/1067
[2] https://lore.kernel.org/u-boot/[email protected]/
[3] https://lore.kernel.org/u-boot/[email protected]/
[4] https://github.com/open-source-firmware/flat-image-tree/pull/40
[5] https://github.com/open-source-firmware/flat-image-tree/pull/44
Daniel Golle (11):
mtd: bind an mtd_blk device for non-NAND MTD masters
mtd: nand: sandbox: forbid sub-page writes
cmd: ubi: create a ubi_blk device when attaching UBI
mtd: nand: spi: drop the eager ubi_blk binding
mtd: ubi: block: build only with CONFIG_CMD_UBI
mtd: ubi: block: only claim UBI block devices in the partition scan
boot: add imagemap on-demand loading from storage
boot: fit: support on-demand loading in fit_image_load()
configs: sandbox: enable imagemap and its storage backends
test: boot: add imagemap unit tests
doc: imagemap: document on-demand loading framework
MAINTAINERS | 9 + arch/sandbox/dts/test.dts | 29 ++ boot/Kconfig | 16 + boot/Makefile | 2 + boot/bootm.c | 70 +++- boot/image-fit.c | 109 +++++- boot/imagemap.c | 451 +++++++++++++++++++++ cmd/ubi.c | 27 ++ configs/sandbox_defconfig | 6 + doc/develop/imagemap.rst | 100 +++++ doc/develop/index.rst | 1 + drivers/mtd/mtdcore.c | 24 ++ drivers/mtd/nand/raw/sand_nand.c | 2 + drivers/mtd/nand/spi/core.c | 3 - drivers/mtd/ubi/Kconfig | 1 + drivers/mtd/ubi/part.c | 9 + include/bootm.h | 2 + include/dm/uclass-id.h | 1 + include/image.h | 2 + include/imagemap.h | 136 +++++++ include/spl.h | 38 +- test/boot/Makefile | 2 + test/boot/imagemap.c | 645 +++++++++++++++++++++++++++++++ test/cmd_ut.c | 2 + 24 files changed, 1671 insertions(+), 16 deletions(-) create mode 100644 boot/imagemap.c create mode 100644 doc/develop/imagemap.rst create mode 100644 include/imagemap.h create mode 100644 test/boot/imagemap.c
base-commit: 8841b7b292c74864acf4ed0bd8ce05839252c1f2
Signature · 1 line
2.55.0
PATCH v2 1/11 mtd: bind an mtd_blk device for non-NAND MTD masters
Only the SPI-NAND bind path currently creates an mtd_blk block device, so a bare NOR (parallel NOR, SPI-NOR, ...) has no block-device access at all. Bind an mtd_blk device for every non-NAND MTD master when CONFIG_MTD_BLOCK is enabled, so its partitions become reachable through the block layer (and thus by bootstd, imagemap, ...).
NAND is excluded on purpose: it needs UBI, which brings its own block device. Partitions inherit the master's block device via the "mtd" partition type, so only masters are bound.
mtd_bind() keeps a pointer to the passed mtd_info pointer (SPI-NAND fills it after bind), so the generic caller provides a small heap slot that lives as long as the block device; an MTD master is never removed in practice.
--- drivers/mtd/mtdcore.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+)
drivers/mtd/mtdcore.c +24 -0
@@ -494,6 +494,30 @@ int add_mtd_device(struct mtd_info *mtd)#endif mutex_unlock (&mtd_table_mutex); + +if (IS_ENABLED (CONFIG_MTD_BLOCK ) && mtd->dev && + !mtd_is_partition (mtd) && !mtd_type_is_nand (mtd)) { +/* +* Expose a non-NAND MTD master (parallel NOR, SPI-NOR, ...) as +* a mtd_blk block device, so its partitions are reachable +* through the block layer (and thus by imagemap, bootstd, ...). +* NAND is excluded: it needs UBI, which brings its own block +* device. Partitions inherit the master's block device via the +* "mtd" partition type, so only masters are bound. +* +* mtd_bind() keeps a pointer to the passed mtd_info pointer, so +* it needs storage that lives as long as the block device; the +* MTD master is never removed in practice, so a small heap slot +* is fine. +*/ +struct mtd_info **mtdp =kmalloc (sizeof (*mtdp),GFP_KERNEL ); + +if (mtdp) { + *mtdp = mtd; +mtd_bind (mtd->dev, mtdp); + } + } +/* We _know_ we aren't being removed, because our caller is still holding us here. So none of this try_ nonsense, and no bitching about it
Signature · 1 line
2.55.0
PATCH v2 2/11 mtd: nand: sandbox: forbid sub-page writes
In reply to Daniel GolleThe sandbox NAND model programs each page in a single operation and rejects a second write to an already-programmed page. UBI writes the EC and VID headers into the first page in two separate sub-page operations, which the model fails with -EIO, so UBI cannot attach to a sandbox NAND.
Advertise NAND_NO_SUBPAGE_WRITE so callers place each header in its own page, matching what the model can actually do. This lets the imagemap UBI-volume unit test attach UBI on the sandbox NAND.
--- drivers/mtd/nand/raw/sand_nand.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/mtd/nand/raw/sand_nand.c +2 -0
@@ -602,6 +602,8 @@ static int sand_nand_probe(struct udevice *dev) nand = &chip->nand; nand->options =not_xpl () ?0 :NAND_SKIP_BBTSCAN ; +/* Each page is programmed in a single operation only */ + nand->options |=NAND_NO_SUBPAGE_WRITE ; nand->flash_node = np; nand->dev_ready = sand_nand_dev_ready; nand->cmdfunc = sand_nand_command;
Signature · 1 line
2.55.0
PATCH v2 3/11 cmd: ubi: create a ubi_blk device when attaching UBI
In reply to Daniel GolleA UBI volume can be reached as a partition on the ubi_blk block device, but that device is only bound from spinand_bind(), so it is missing on every other flash type (raw NAND, SPI-NOR, ...).
Bind a ubi_blk device when a UBI partition is attached, unless one has already been created, parented to the MTD device UBI sits on. UBI volumes then become reachable as block-device partitions regardless of the underlying flash. The imagemap block path relies on this to read a UBI volume.
--- cmd/ubi.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+)
cmd/ubi.c +27 -0
@@ -20,7 +20,10 @@#include <mtd.h> #include <nand.h> #include <onenand_uboot.h> +#include <dm/device.h> #include <dm/devres.h> +#include <dm/root.h> +#include <dm/uclass-internal.h> #include <linux/mtd/mtd.h> #include <linux/mtd/partitions.h> #include <linux/err.h>
@@ -670,6 +673,29 @@ int ubi_detach(void)return 0 ; } +#if CONFIG_IS_ENABLED (UBI_BLOCK ) +static void ubi_blk_bind_once (void ) +{ +struct udevice *dev; + +/* +* A single ubi_blk device serves all volumes of the attached UBI +* device, the volume being selected through the block descriptor's +* hwpart. Bind one when UBI is attached -- unless a previous attach +* already did -- parented to the MTD device UBI sits on. +*/ +for (uclass_find_first_device (UCLASS_BLK , &dev); dev; +uclass_find_next_device (&dev)) { +if (dev->driver ==DM_DRIVER_GET (ubi_blk)) +return ; + } + +ubi_bind (ubi && ubi->mtd && ubi->mtd->dev ? ubi->mtd->dev :dm_root ()); +} +#else +static inline void ubi_blk_bind_once (void ) { } +#endif +int ubi_part (const char *part_name,const char *vid_header_offset) {struct mtd_info *mtd;
@@ -698,6 +724,7 @@ int ubi_part(const char *part_name, const char *vid_header_offset) } ubi = ubi_devices[0 ]; +ubi_blk_bind_once ();return 0 ; }
Signature · 1 line
2.55.0
PATCH v2 4/11 mtd: nand: spi: drop the eager ubi_blk binding
In reply to Daniel Gollespinand_bind() binds a ubi_blk device at bind time. Now that "cmd:
ubi: create a ubi_blk device when attaching UBI" binds one generically when UBI is attached -- with an identical result, only lazily and for any flash type -- the SPI-NAND-specific eager call is redundant.
Drop it to avoid the duplicate code path (a ubi_blk is unusable before UBI is attached anyway). The mtd_blk binding is kept: bare mtd_blk access to SPI-NAND remains available for users who want it.
--- drivers/mtd/nand/spi/core.c | 3 --- 1 file changed, 3 deletions(-)
drivers/mtd/nand/spi/core.c +0 -3
@@ -1654,9 +1654,6 @@ static int spinand_bind(struct udevice *dev)if (ret)return ret; } - -if (CONFIG_IS_ENABLED (UBI_BLOCK )) -return ubi_bind (dev); }return 0 ;
Signature · 1 line
2.55.0
PATCH v2 5/11 mtd: ubi: block: build only with CONFIG_CMD_UBI
In reply to Daniel GolleThe UBI block driver calls ubi_volume_read() and ubi_volume_write(), which are defined in cmd/ubi.c and thus only built with CONFIG_CMD_UBI. Selecting UBI_BLOCK without CMD_UBI, for example in a CONFIG_CMDLINE=n build, fails to link:
drivers/mtd/ubi/block.c: undefined reference to ubi_volume_read' drivers/mtd/ubi/block.c: undefined reference to ubi_volume_write'
Make the dependency explicit so UBI_BLOCK is only offered when CMD_UBI is enabled. The only in-tree user, topic_miami_defconfig, already enables CMD_UBI, so no board is affected.
--- drivers/mtd/ubi/Kconfig | 1 + 1 file changed, 1 insertion(+)
drivers/mtd/ubi/Kconfig +1 -0
@@ -116,6 +116,7 @@ config MTD_UBI_FM_DEBUG config UBI_BLOCK bool "Enable UBI block device support" + depends on CMD_UBI select BLK help Enable UBI block device support using blk_ops abstraction.
Signature · 1 line
2.55.0
PATCH v2 6/11 mtd: ubi: block: only claim UBI block devices in the partition scan
In reply to Daniel Gollepart_test_ubi() reads the first block and returns success for any readable device, so a bare block device with no partition table (for example a sandbox "host bind" image) is mis-detected as UBI once CONFIG_UBI_BLOCK is enabled. part_get_info_ubi() then calls ubi_get_volume_by_index(), which dereferences ubi_devices[0] without a NULL check and crashes when no UBI device is attached.
Fail part_test_ubi() when no UBI device is attached, and guard the volume lookup and print helpers against a NULL ubi_device, so the driver only claims devices while UBI is present and never dereferences a missing one.
--- drivers/mtd/ubi/part.c | 9 +++++++++ 1 file changed, 9 insertions(+)
drivers/mtd/ubi/part.c +9 -0
@@ -19,6 +19,9 @@ static struct ubi_volume *ubi_get_volume_by_index(int vol_id)struct ubi_device *ubi =get_ubi_device ();int i; +if (!ubi) +return NULL ; +for (i =0 ; i < (ubi->vtbl_slots +1 ); i++) {struct ubi_volume *volume = ubi->volumes[i];
@@ -66,6 +69,9 @@ static void __maybe_unused part_print_ubi(struct blk_desc *dev_desc)struct ubi_device *ubi =get_ubi_device ();int i; +if (!ubi) +return ; +for (i =0 ; i < (ubi->vtbl_slots +1 ); i++) {struct ubi_volume *volume = ubi->volumes[i];
@@ -83,6 +89,9 @@ static int part_test_ubi(struct blk_desc *dev_desc) { ALLOC_CACHE_ALIGN_BUFFER(unsigned char, buffer, dev_desc->blksz); +if (!get_ubi_device ()) +return -1 ; +if (blk_dread (dev_desc,0 ,1 , (ulong *)buffer) !=1 )return -1 ;
Signature · 1 line
2.55.0
PATCH v2 7/11 boot: add imagemap on-demand loading from storage
In reply to Daniel GolleIntroduce imagemap, a small layer that reads image data from a storage device on demand and keeps a translation table of the byte ranges already loaded into RAM, instead of copying the whole image up front.
An imagemap device (UCLASS_IMAGEMAP) is created over a partition of a block device with imagemap_create(); imagemap_map() then returns a RAM pointer for any byte range, reading it in on first access and reusing it on later accesses -- so a header probe, verification and the final load of the same range share a single read. Block devices cover more than raw disks: an MTD partition is reached through mtdblock and a UBI volume through ubiblock, each exposed as a named block-device partition, so imagemap needs no storage-specific code of its own.
The read path is built on the SPL struct spl_load_info abstraction and a new spl_load_region() helper (added to spl.h): the block reader reports the device block length and spl_load_region() performs the native sector alignment. imagemap_map_to() keeps a payload byte-exact at a caller's load address, reading the aligned middle straight to the destination and bouncing only the partial head/tail block.
Regions that are not placed at a caller-supplied address are allocated through the LMB allocator and released on cleanup; the translation table doubles as the registry of those allocations. imagemap is gated to the full-U-Boot phase, so SPL and TPL carry no new cost.
--- MAINTAINERS | 7 + boot/Kconfig | 16 ++ boot/Makefile | 2 + boot/imagemap.c | 451 +++++++++++++++++++++++++++++++++++++++++ include/dm/uclass-id.h | 1 + include/imagemap.h | 136 +++++++++++++ include/spl.h | 38 +++- 7 files changed, 647 insertions(+), 4 deletions(-) create mode 100644 boot/imagemap.c create mode 100644 include/imagemap.h
MAINTAINERS +7 -0
@@ -1186,6 +1186,13 @@ S: Maintained F: drivers/timer/goldfish_timer.c F: include/goldfish_timer.h +IMAGEMAP +M: Daniel Golle <[email protected]> +L: [email protected] +S: Maintained +F: boot/imagemap.c +F: include/imagemap.h + INTERCONNECT M: Neil Armstrong <[email protected]> S: Maintained
boot/Kconfig +16 -0
@@ -1205,6 +1205,22 @@ config SYS_BOOT_RAMDISK_HIGH endmenu # Boot images +config IMAGEMAP + bool "On-demand image loading from storage" + depends on DM && LMB && BLK && PARTITIONS + help + Read image data from a storage device on demand instead of + copying the whole image into RAM first. A translation table maps + already-loaded byte ranges to their RAM addresses to avoid + redundant reads, and scratch allocations are managed through the + LMB allocator. + + Images are read from a partition on a block device, identified by + number or name. MTD partitions (via mtdblock) and UBI volumes + (via ubiblock) are reached as block-device partitions too. Used + by bootm when a storage device is specified instead of a RAM + address. + config DISTRO_DEFAULTS bool "(deprecated) Script-based booting of Linux distributions" select CMDLINE
boot/Makefile +2 -0
@@ -73,6 +73,8 @@ obj-$(CONFIG_$(PHASE_)BOOTMETH_VBE_SIMPLE_OS) += vbe_simple_os.o obj-$(CONFIG_$(PHASE_)BOOTMETH_ANDROID) += bootmeth_android.o +obj-$(CONFIG_$(PHASE_)IMAGEMAP) += imagemap.o + obj-$(CONFIG_$(PHASE_)BOOTMETH_VBE_ABREC) += vbe_abrec.o vbe_common.o obj-$(CONFIG_$(PHASE_)BOOTMETH_VBE_ABREC_FW) += vbe_abrec_fw.o obj-$(CONFIG_$(PHASE_)BOOTMETH_VBE_ABREC_OS) += vbe_abrec_os.o
boot/imagemap.c new file +451 -0
451 lines not on this page · raw message
include/dm/uclass-id.h +1 -0
@@ -75,6 +75,7 @@ enum uclass_id {UCLASS_HASH ,/* Hash device */ UCLASS_HWSPINLOCK ,/* Hardware semaphores */ UCLASS_HOST ,/* Sandbox host device */ +UCLASS_IMAGEMAP ,/* On-demand image loading from storage */ UCLASS_I2C ,/* I2C bus */ UCLASS_I2C_EEPROM ,/* I2C EEPROM device */ UCLASS_I2C_GENERIC, /* Generic I2C device */
include/imagemap.h new file +136 -0
136 lines not on this page · raw message
include/spl.h +34 -4
@@ -346,14 +346,14 @@ typedef ulong (*spl_load_reader)(struct spl_load_info *load, ulong sector,* * @read: Function to call to read from the device * @priv: Private data for the device -* @bl_len: Block length for reading in bytes +* @bl_len: Block length in bytes (CONFIG_SPL_LOAD_BLOCK, CONFIG_IMAGEMAP) * @phase: Image phase to load * @no_fdt_update: true to update the FDT with any loadables that are loaded */ struct spl_load_info {spl_load_reader read;void *priv; -#if IS_ENABLED (CONFIG_SPL_LOAD_BLOCK ) +#if IS_ENABLED (CONFIG_SPL_LOAD_BLOCK ) ||CONFIG_IS_ENABLED (IMAGEMAP )u16 bl_len;#endif #if CONFIG_IS_ENABLED (BOOTMETH_VBE )
@@ -364,7 +364,7 @@ struct spl_load_info {static inline int spl_get_bl_len (struct spl_load_info *info) { -#if IS_ENABLED (CONFIG_SPL_LOAD_BLOCK ) +#if IS_ENABLED (CONFIG_SPL_LOAD_BLOCK ) ||CONFIG_IS_ENABLED (IMAGEMAP )return info->bl_len;#else return 1 ;
@@ -373,7 +373,7 @@ static inline int spl_get_bl_len(struct spl_load_info *info)static inline void spl_set_bl_len (struct spl_load_info *info,int bl_len) { -#if IS_ENABLED (CONFIG_SPL_LOAD_BLOCK ) +#if IS_ENABLED (CONFIG_SPL_LOAD_BLOCK ) ||CONFIG_IS_ENABLED (IMAGEMAP ) info->bl_len = bl_len;#else if (bl_len !=1 )
@@ -429,6 +429,36 @@ static inline void spl_load_init(struct spl_load_info *load,xpl_set_fdt_update (load, true); } +/** +* spl_load_region() - Read a block-aligned region into a buffer +* +* Reads @size bytes starting at @offset from the device described by @info +* into @dst, rounding the read down/up to the device block length so that +* block media are read on their native boundaries. On byte-addressed media +* (bl_len == 1) every alignment folds to identity. +* +* @info: information about the device to read from +* @offset: byte offset on the device of the first wanted byte +* @size: number of wanted bytes +* @dst: buffer to read the block-aligned region into +* Return: on success, the number of leading padding bytes in @dst that +* precede the wanted data (0 on byte media); a negative error +* number on failure. +*/ +static inline long spl_load_region (struct spl_load_info *info,loff_t offset, +ulong size,void *dst) +{ +ulong bl_len =spl_get_bl_len (info); +ulong overhead = offset & (bl_len -1 ); +loff_t roff =ALIGN_DOWN (offset, bl_len); +ulong rsize =ALIGN (size + overhead, bl_len); + +if (info->read (info, roff, rsize, dst) < rsize) +return -EIO ; + +return overhead; +} +/* * We need to know the position of U-Boot in memory so we can jump to it. We * allow any U-Boot binary to be used (u-boot.bin, u-boot-nodtb.bin,
Signature · 1 line
2.55.0
PATCH v2 8/11 boot: fit: support on-demand loading in fit_image_load()
In reply to Daniel GolleIntegrate storage-backed image loading transparently into the existing FIT flow so that all verification, decompression, and copy-to-load- address code runs unchanged for both RAM-backed and storage-backed images.
Hook imagemap into fit_image_get_data(): when gd->imagemap is set and the sub-image uses external data, call imagemap_map() to bring the payload into RAM instead of returning the unmapped fit + offset pointer. Every existing caller, including fit_image_verify() called from fit_image_select(), transparently gets valid data without any changes to the authentication path.
For uncompressed sub-images with a known load address, pre-populate the translation table via imagemap_map_to() before verification runs. This loads data directly to the final RAM destination; when fit_image_get_data() is later called, the translation table hit returns the same pointer, achieving zero-copy. Compressed images or those without a load address fall through to the lazy imagemap_map() path.
Filesystem sub-images protected by dm-verity are returned early and stay on storage, as the kernel verifies them at block level.
--- boot/bootm.c | 70 +++++++++++++++++++++++++++--- boot/image-fit.c | 109 ++++++++++++++++++++++++++++++++++++++++++++++- include/bootm.h | 2 + include/image.h | 2 + 4 files changed, 174 insertions(+), 9 deletions(-)
boot/bootm.c +63 -7
@@ -14,6 +14,7 @@#include <env.h> #include <errno.h> #include <fdt_support.h> +#include <imagemap.h> #include <irq_func.h> #include <lmb.h> #include <log.h>
@@ -147,7 +148,22 @@ static int boot_get_kernel(const char *addr_fit, struct bootm_headers *images,/* check image type, for FIT images get FIT kernel node */ *os_data = *os_len =0 ; - buf =map_sysmem (img_addr,0 ); +if (IS_ENABLED (CONFIG_IMAGEMAP ) && images->imagemap) { +/* +* Storage path: read enough bytes to detect the image +* format. genimg_get_kernel_addr_fit() above still +* parsed any #config / :subimage suffix so the FIT +* selection variables are populated. +*/ + buf =imagemap_map (images->imagemap,0 ,64 ); +if (IS_ERR (buf)) { +puts ("Cannot read image header from storage\n" ); +return PTR_ERR (buf); + } + img_addr =map_to_sysmem (buf); + }else { + buf =map_sysmem (img_addr,0 ); + }switch (genimg_get_format (buf)) {#if CONFIG_IS_ENABLED (LEGACY_IMAGE_FORMAT )case IMAGE_FORMAT_LEGACY :
@@ -193,6 +209,20 @@ static int boot_get_kernel(const char *addr_fit, struct bootm_headers *images,#endif #if CONFIG_IS_ENABLED (FIT )case IMAGE_FORMAT_FIT : +if (IS_ENABLED (CONFIG_IMAGEMAP ) && images->imagemap) { +/* +* Extend the mapping to cover the full FIT +* FDT structure so all metadata is accessible. +*/ +size_t fdt_sz =fdt_totalsize (buf); + + buf =imagemap_map (images->imagemap,0 , fdt_sz); +if (IS_ERR (buf)) { +puts ("Cannot read FIT header from storage\n" ); +return PTR_ERR (buf); + } + img_addr =map_to_sysmem (buf); + } os_noffset = fit_image_load(images, img_addr, &fit_uname_kernel, &fit_uname_config, IH_ARCH_DEFAULT, IH_TYPE_KERNEL,
@@ -1033,11 +1063,21 @@ int bootm_run_states(struct bootm_info *bmi, int states)* Work through the states and see how far we get. We stop on * any error. */ -if (states &BOOTM_STATE_START ) +if (states &BOOTM_STATE_START ) { ret =bootm_start (); +/* +* bootm_start() zeroes the global images struct. Restore +* the loader pointer so the storage-backed path works. +*/ +if (IS_ENABLED (CONFIG_IMAGEMAP ) && bmi->imagemap) + images->imagemap = bmi->imagemap; + } -if (!ret && (states &BOOTM_STATE_PRE_LOAD )) - ret =bootm_pre_load (bmi->addr_img); +if (!ret && (states &BOOTM_STATE_PRE_LOAD )) { +/* Pre-load verification is not applicable to storage boot */ +if (!IS_ENABLED (CONFIG_IMAGEMAP ) || !images->imagemap) + ret =bootm_pre_load (bmi->addr_img); + }if (!ret && (states &BOOTM_STATE_FINDOS )) ret =bootm_find_os (bmi->cmd_name, bmi->addr_img);
@@ -1045,8 +1085,11 @@ int bootm_run_states(struct bootm_info *bmi, int states)if (!ret && (states &BOOTM_STATE_FINDOTHER )) {ulong img_addr; - img_addr = bmi->addr_img ?hextoul (bmi->addr_img,NULL ) - : image_load_addr; +if (IS_ENABLED (CONFIG_IMAGEMAP ) && images->imagemap) + img_addr = images->os.start; +else + img_addr = bmi->addr_img ?hextoul (bmi->addr_img,NULL ) + : image_load_addr; ret =bootm_find_other (img_addr, bmi->conf_ramdisk, bmi->conf_fdt); }
@@ -1145,11 +1188,24 @@ int bootm_run_states(struct bootm_info *bmi, int states) }/* Now run the OS! We hope this doesn't return */ -if (!ret && (states &BOOTM_STATE_OS_GO )) +if (!ret && (states &BOOTM_STATE_OS_GO )) { +/* Release storage backend before jumping - no return expected */ +if (IS_ENABLED (CONFIG_IMAGEMAP ) && images->imagemap) { +imagemap_cleanup (images->imagemap); + images->imagemap =NULL ; + } + ret =boot_selected_os (BOOTM_STATE_OS_GO , bmi, boot_fn); + }/* Deal with any fallout */ err: +/* Clean up imagemap on error (not reached on successful boot) */ +if (IS_ENABLED (CONFIG_IMAGEMAP ) && images->imagemap) { +imagemap_cleanup (images->imagemap); + images->imagemap =NULL ; + } +if (iflag)enable_interrupts ();
boot/image-fit.c +107 -2
139 lines not on this page · raw message
include/bootm.h +2 -0
@@ -40,6 +40,7 @@ struct cmd_tbl;* boot_get_fdt() for processing, or NULL for none * @boot_progress: true to show boot progress * @images: images information +* @imagemap: imagemap device for storage-backed boot (NULL for in-memory) * @cmd_name: command which invoked this operation, e.g. "bootm" * @argc: Number of arguments to the command (excluding the actual command). * This is 0 if there are no arguments
@@ -51,6 +52,7 @@ struct bootm_info {const char *conf_fdt;bool boot_progress;struct bootm_headers *images; +struct udevice *imagemap;const char *cmd_name;int argc;char *const *argv;
include/image.h +2 -0
@@ -412,6 +412,8 @@ struct bootm_headers {int verify;/* env_get("verify")[0] != 'n' */ +struct udevice *imagemap;/* on-demand storage loader, or NULL */ +#define BOOTM_STATE_START 0x00000001#define BOOTM_STATE_FINDOS 0x00000002#define BOOTM_STATE_FINDOTHER 0x00000004
Signature · 1 line
2.55.0
PATCH v2 9/11 configs: sandbox: enable imagemap and its storage backends
In reply to Daniel GolleEnable CONFIG_IMAGEMAP and, so the end-to-end unit tests can exercise the real block paths, the storage that feeds them: CONFIG_MTD_BLOCK and CONFIG_SPI_FLASH_MTD (a bare non-NAND MTD reached via mtd_blk) and CONFIG_CMD_UBI / CONFIG_UBI_BLOCK (a UBI volume reached via ubi_blk).
Give the sandbox SPI-NOR two device-tree partitions for the mtd_blk test and add a clean sandbox NAND chip (nand@2, no injected errors) so UBI can format on it for the ubiblock test.
--- arch/sandbox/dts/test.dts | 29 +++++++++++++++++++++++++++++ configs/sandbox_defconfig | 6 ++++++ 2 files changed, 35 insertions(+)
arch/sandbox/dts/test.dts +29 -0
@@ -1623,6 +1623,22 @@ compatible = "spansion,m25p16", "jedec,spi-nor"; spi-max-frequency = <40000000>; sandbox,filename = "spi.bin"; + + partitions { + compatible = "fixed-partitions"; + #address-cells = <1>; + #size-cells = <1>; + + partition@0 { + label = "nor-fit"; + reg = <0x0 0x100000>; + }; + + partition@100000 { + label = "nor-spare"; + reg = <0x100000 0x100000>; + }; + }; }; spi.bin@1 { reg = <1>;
@@ -2232,6 +2248,19 @@ sandbox,err-count = <3>; sandbox,err-step-size = <512>; }; + + /* Clean chip (no injected errors) for UBI/ubiblock tests */ + nand@2 { + reg = <2>; + nand-ecc-mode = "soft"; + sandbox,id = [00 e3]; + sandbox,erasesize = <(8 * 1024)>; + sandbox,oobsize = <16>; + sandbox,pagesize = <512>; + sandbox,pages = <0x2000>; + sandbox,err-count = <0>; + sandbox,err-step-size = <512>; + }; }; graph1 {
configs/sandbox_defconfig +6 -0
@@ -30,6 +30,7 @@ CONFIG_BOOTMETH_RAUC=y CONFIG_UPL=y CONFIG_LEGACY_IMAGE_FORMAT=y CONFIG_MEASURED_BOOT=y +CONFIG_IMAGEMAP=y CONFIG_BOOTSTAGE=y CONFIG_BOOTSTAGE_REPORT=y CONFIG_BOOTSTAGE_FDT=y
@@ -399,3 +400,8 @@ CONFIG_UTHREAD=y CONFIG_UNIT_TEST=y CONFIG_UT_TIME=y CONFIG_UT_DM=y +CONFIG_MTD_BLOCK=y +CONFIG_SPI_FLASH_MTD=y +CONFIG_CMD_UBI=y +CONFIG_MTD_UBI=y +CONFIG_UBI_BLOCK=y
Signature · 1 line
2.55.0
PATCH v2 10/11 test: boot: add imagemap unit tests
In reply to Daniel GolleAdd a test suite exercising imagemap:
- the translation-table core (map, lookup, map_to, cleanup, region growth, the block-aligned read path, and edge cases) using a mock RAM-backed reader, so most tests need no real storage;
- a NOR read through mtd_blk: write a known pattern into a device-tree partition of the sandbox SPI-NOR and read it back through imagemap, exercising imagemap -> spl_load_region -> blk_dread -> mtd_blk_read -> mtd_read for real;
- a UBI-volume read through ubiblock: attach UBI on the sandbox NAND, create a volume, write a pattern and read it back through imagemap on the ubi_blk device.
Registered under ut_boot_imagemap in cmd_ut.c.
--- MAINTAINERS | 1 + test/boot/Makefile | 2 + test/boot/imagemap.c | 645 +++++++++++++++++++++++++++++++++++++++++++ test/cmd_ut.c | 2 + 4 files changed, 650 insertions(+) create mode 100644 test/boot/imagemap.c
MAINTAINERS +1 -0
@@ -1192,6 +1192,7 @@ L: [email protected] S: Maintained F: boot/imagemap.c F: include/imagemap.h +F: test/boot/imagemap.c INTERCONNECT M: Neil Armstrong <[email protected]>
test/boot/Makefile +2 -0
@@ -27,3 +27,5 @@ endif obj-$(CONFIG_BOOTMETH_VBE) += vbe_fixup.o obj-$(CONFIG_UPL) += upl.o + +obj-$(CONFIG_IMAGEMAP) += imagemap.o
test/boot/imagemap.c new file +645 -0
645 lines not on this page · raw message
test/cmd_ut.c +2 -0
@@ -73,6 +73,7 @@ SUITE_DECL(optee);SUITE_DECL (pci_mps);SUITE_DECL (seama);SUITE_DECL (setexpr); +SUITE_DECL (imagemap);SUITE_DECL (upl);static struct suite suites[] = {
@@ -102,6 +103,7 @@ static struct suite suites[] = {SUITE (pci_mps,"PCI Express Maximum Payload Size" ),SUITE (seama,"seama command parameters loading and decoding" ),SUITE (setexpr,"setexpr command" ), +SUITE (imagemap,"imagemap on-demand storage loading" ), SUITE(upl, "Universal payload support"), };
Signature · 1 line
2.55.0
PATCH v2 11/11 doc: imagemap: document on-demand loading framework
In reply to Daniel GolleDescribe the imagemap framework architecture: the translation table that tracks mapped regions and its phase-determined caching, reading from a block-device partition (including MTD via mtdblock and UBI via ubiblock), the integration with bootm for on-demand FIT sub-image loading, and the SPL build.
--- MAINTAINERS | 1 + doc/develop/imagemap.rst | 100 +++++++++++++++++++++++++++++++++++++++ doc/develop/index.rst | 1 + 3 files changed, 102 insertions(+) create mode 100644 doc/develop/imagemap.rst
MAINTAINERS +1 -0
@@ -1191,6 +1191,7 @@ M: Daniel Golle <[email protected]> L: [email protected] S: Maintained F: boot/imagemap.c +F: doc/develop/imagemap.rst F: include/imagemap.h F: test/boot/imagemap.c
doc/develop/imagemap.rst new file +100 -0
100 lines not on this page · raw message
doc/develop/index.rst +1 -0
@@ -31,6 +31,7 @@ Implementation directories bloblist + imagemap bootstd/index ci_testing commands
Signature · 1 line
2.55.0
Quoted 9 lines · Daniel Golle
This series adds "imagemap", a small layer for reading FIT images directly from storage on demand instead of copying the whole image into RAM first. It keeps a translation table of the byte ranges already loaded, so a header probe, signature verification and the final load can reuse a single read, and it hooks transparently into
fit_image_load()so all existing verification and decompression paths run unchanged for both RAM- and storage-backed images.[...]
Applied to u-boot/next, thanks!
[01/11] mtd: bind an mtd_blk device for non-NAND MTD masters
commit: e8a7453824b7ec20c722b0268de417058e49a9e0
[02/11] mtd: nand: sandbox: forbid sub-page writes
commit: 7c821fea086014d3a78130a853e017a269d63e1e
[03/11] cmd: ubi: create a ubi_blk device when attaching UBI
commit: dec405d1653a9e6b8bfc9a0f9aaf66ac3de7cb99
[04/11] mtd: nand: spi: drop the eager ubi_blk binding
commit: 790c94a5c40a1b2c8cbcb87bc49c7fe0af5cbf2e
[05/11] mtd: ubi: block: build only with CONFIG_CMD_UBI
commit: 15358294f11268e7998271df6e03127650178aa4
[06/11] mtd: ubi: block: only claim UBI block devices in the partition scan
commit: d6bcfe912757ffc53403578700919a7a427ef94d
[07/11] boot: add imagemap on-demand loading from storage
commit: 9b41c7dbee964fd6ea67c8ef23d30346c1f0f8a8
[08/11] boot: fit: support on-demand loading in fit_image_load()
commit: 46d32e38ee4ef49567255aa9a89a7be44c0f0e04
[09/11] configs: sandbox: enable imagemap and its storage backends
commit: feeef0a69f0f69252e8d06d906b80aaafe0c2cd2
[10/11] test: boot: add imagemap unit tests
commit: 05c1fbbfa79ffe020b51b5d9ac9fd46ca3eb3e74
[11/11] doc: imagemap: document on-demand loading framework
commit: 0f59e0702ae90dab8561e4ff2b32e89ef8390f28Signature · 1 line
Tom