partition parser for U-Boot's uImage.FIT

Add uImage.FIT partition parser and wire it up to allow mounting filesystem sub-images from uImage.FIT in GPT partitions as well as mtdblock and ubiblock devices within Linux (e.g. as root filesystem).

Using uImage.FIT to store the root filesystem besides kernel and dtb has several obvious advantages which are hard to obtain in any other way:
* single image accross different storage types
* dynamically sized partitions for kernel and rootfs
* hash also for rootfs checked by U-Boot before launching kernel
* images may include additional filesystems e.g. for localization or branding

For this to work, the image has to be created with external data and sub-images aligned to the system's memory page boundaries, ie.
mkimage -E -B 0x1000 -p 0x1000 ...

Booting such images has been supported by U-Boot since v2018.01.

A previous version of this partition parser is in production use on some OpenWrt devices, eg. the BananaPi R64 where using the FIT parser allows booting the very same image from eMMC, SD Card or SPI-NAND/UBI and also using it as a firmware-upgrade image at the same time.
The Ubiquiti UniFi 6 LR access point served as a reference board with SPI-NOR flash and use of the partition parser on top of a mtdblock device.

As U-Boot by now also passes down the selected configuration node name via device tree this allows the partition parser (or userspace process via sysfs) to identify the selected image configuration.

Device Tree schema for that:
https://github.com/devicetree-org/dt-schema/commit/a24d97d43491e55d4def006213213a6c4045b646

In most cases this partition parser can be used without relying on the bootloader to pass-down the configuration node name. The default configuration node is used then.

Changes since v4:
* use folio instead of page (requested by Matthew Wilcox)
* remove unneeded checks (requested by Matthew Wilcox)
* use strscpy instead of strlcpy (strlcpy is deprecated)
* fix off-by-one string size limit label_min (discovered during tests)
* use only a single config symbol MTD_BLOCK_PARTITION affecting both ubiblock and mtdblock (requested by Richard Weinberger)

Changes since v3:
* use min_t(size_t, ...)

Changes since v2:
* use returned length to limit all strings read from fit/dt
* use __be32 type for 32-bit values read from fit/dt

* Kconfig: select LIBFDT for FIT_PARTITION

Changes since v1:
* Use again #ifdef's in partitions/efi to only build against FIT parser symbols if it is actually selected. Otherwise the efi/gpt would unconditionally depend on the FTT parser to be present.

Changes since RFC:
* fixed wrong variable used in error path
* introduced dedicated Kconfig options to enable partition parsers on mtdblock and ubiblock
* drop #ifdef'ery, use IS_ENABLED(...) where needed

Daniel Golle (4):
block: add new flag to add partitions read-only
block: add partition parser for U-Boot uImage.FIT partitions/efi: add support for uImage.FIT sub-partitions
mtd: add option to enable scanning for partitions

 MAINTAINERS               |   6 +
 block/blk.h               |   1 +
 block/partitions/Kconfig  |  15 ++
 block/partitions/Makefile |   1 +
 block/partitions/check.h  |   4 +
 block/partitions/core.c   |   6 +
 block/partitions/efi.c    |   9 +
 block/partitions/efi.h    |   3 +
 block/partitions/fit.c    | 346 ++++++++++++++++++++++++++++++++++++++
 drivers/mtd/Kconfig       |  11 ++
 drivers/mtd/mtd_blkdevs.c |   4 +-
 drivers/mtd/ubi/block.c   |   5 +-
 12 files changed, 409 insertions(+), 2 deletions(-)
 create mode 100644 block/partitions/fit.c

2.38.1


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Add flag ADDPART_FLAG_READONLY to allow partition parsers marking a partition to be set read-only.
This is needed for the uImage.FIT partition parser added by a follow-up
commit: we need to be sure the contents of uImage.FIT sub-images remain unaltered they are validated using a hash within the uImage.FIT structure which also serves as partition table.

---
 block/blk.h             | 1 +
 block/partitions/core.c | 3 +++
 2 files changed, 4 insertions(+)

index e85703ae81dd..05ac426350b2 100644
@@ -414,6 +414,7 @@ void blk_free_ext_minor(unsigned int minor);
 #define ADDPART_FLAG_NONE	0
 #define ADDPART_FLAG_RAID	1
 #define ADDPART_FLAG_WHOLEDISK	2
+#define ADDPART_FLAG_READONLY	4
 int bdev_add_partition(struct gendisk *disk, int partno, sector_t start,
 		sector_t length);
 int bdev_del_partition(struct gendisk *disk, int partno);

index b8112f52d388..355646b0707d 100644
@@ -398,6 +398,9 @@ static struct block_device *add_partition(struct gendisk *disk, int partno,
 			goto out_del;
 	}
 
+	if (flags & ADDPART_FLAG_READONLY)
+		bdev->bd_read_only = true;
+
 	/* everything is up and running, commence */
 	err = xa_insert(&disk->part_tbl, partno, bdev, GFP_KERNEL);
 	if (err)

2.38.1

Introduce a new partition parser for U-Boot's Flattened-Image-Tree (FIT) in order to allow Linux to mount the filesystem part of a uImage.FIT.

uImage.FIT needs to be created with external data and aligned to the system's memory page size. e.g.
mkimage -E -B 0x1000 -p 0x1000 ...

---
 MAINTAINERS               |   6 +
 block/partitions/Kconfig  |  15 ++
 block/partitions/Makefile |   1 +
 block/partitions/check.h  |   4 +
 block/partitions/core.c   |   3 +
 block/partitions/fit.c    | 346 ++++++++++++++++++++++++++++++++++++++
 6 files changed, 375 insertions(+)
 create mode 100644 block/partitions/fit.c

index 547ba994ea98..632672eb33ee 100644
@@ -8059,6 +8059,12 @@ F:	Documentation/firmware_class/
 F:	drivers/base/firmware_loader/
 F:	include/linux/firmware.h
 
+FIT PARTITION TABLE (uImage.FIT)
+M:	Daniel Golle <[email protected]>
+L:	[email protected]
+S:	Maintained
+F:	block/partitions/fit.c
+
 FLEXTIMER FTM-QUADDEC DRIVER
 M:	Patrick Havelange <[email protected]>
 L:	[email protected]

index 7aff4eb81c60..77dc85d3797d 100644
@@ -103,6 +103,21 @@ config ATARI_PARTITION
 	  Say Y here if you would like to use hard disks under Linux which
 	  were partitioned under the Atari OS.
 
+config FIT_PARTITION
+	bool "Flattened-Image-Tree (FIT) partition support" if PARTITION_ADVANCED
+	default n
+	select LIBFDT
+	help
+	  Say Y here if your system needs to mount the filesystem part of
+	  a Flattened-Image-Tree (FIT) image commonly used with Das U-Boot.
+
+	  uImage.FIT needs to be created with external data and aligned to
+	  the systems memory page size. e.g.
+	    mkimage -E -B 0x1000 -p 0x1000 ...
+
+	  If your system doesn't use U-Boot or you don't need to mount uImage.FIT
+	  filesystem sub-images in Linux, say N.
+
 config IBM_PARTITION
 	bool "IBM disk label and partition support"
 	depends on PARTITION_ADVANCED && S390

index a7f05cdb02a8..d319eb1deba9 100644
@@ -8,6 +8,7 @@ obj-$(CONFIG_ACORN_PARTITION) += acorn.o
 obj-$(CONFIG_AMIGA_PARTITION) += amiga.o
 obj-$(CONFIG_ATARI_PARTITION) += atari.o
 obj-$(CONFIG_AIX_PARTITION) += aix.o
+obj-$(CONFIG_FIT_PARTITION) += fit.o
 obj-$(CONFIG_CMDLINE_PARTITION) += cmdline.o
 obj-$(CONFIG_MAC_PARTITION) += mac.o
 obj-$(CONFIG_LDM_PARTITION) += ldm.o

index 8d70a880c372..76b006d3cb27 100644
@@ -57,6 +57,7 @@ int amiga_partition(struct parsed_partitions *state);
 int atari_partition(struct parsed_partitions *state);
 int cmdline_partition(struct parsed_partitions *state);
 int efi_partition(struct parsed_partitions *state);
+int fit_partition(struct parsed_partitions *state);
 int ibm_partition(struct parsed_partitions *);
 int karma_partition(struct parsed_partitions *state);
 int ldm_partition(struct parsed_partitions *state);

@@ -67,3 +68,6 @@ int sgi_partition(struct parsed_partitions *state);
 int sun_partition(struct parsed_partitions *state);
 int sysv68_partition(struct parsed_partitions *state);
 int ultrix_partition(struct parsed_partitions *state);
+
+int parse_fit_partitions(struct parsed_partitions *state, u64 fit_start_sector,
+			 u64 sectors, int *slot, int max_slot, bool add_remain);

index 355646b0707d..9d4bb8f48d35 100644
@@ -43,6 +43,9 @@ static int (*check_part[])(struct parsed_partitions *) = {
 #ifdef CONFIG_CMDLINE_PARTITION
 	cmdline_partition,
 #endif
+#ifdef CONFIG_FIT_PARTITION
+	fit_partition,
+#endif
 #ifdef CONFIG_EFI_PARTITION
 	efi_partition,		/* this must come before msdos */
 #endif

2.38.1

Add new GUID allowing to parse uImage.FIT stored in a GPT partition and map filesystem sub-image as sub-partitions.

---
 block/partitions/efi.c | 9 +++++++++
 block/partitions/efi.h | 3 +++
 2 files changed, 12 insertions(+)

index 5e9be13a56a8..f4406b443f04 100644
@@ -716,6 +716,9 @@ int efi_partition(struct parsed_partitions *state)
 	gpt_entry *ptes = NULL;
 	u32 i;
 	unsigned ssz = queue_logical_block_size(state->disk->queue) / 512;
+#ifdef CONFIG_FIT_PARTITION
+	u32 extra_slot = 65;
+#endif
 
 	if (!find_valid_gpt(state, &gpt, &ptes) || !gpt || !ptes) {
 		kfree(gpt);

@@ -749,6 +752,12 @@ int efi_partition(struct parsed_partitions *state)
 				ARRAY_SIZE(ptes[i].partition_name));
 		utf16_le_to_7bit(ptes[i].partition_name, label_max, info->volname);
 		state->parts[i + 1].has_info = true;
+		/* If this is a U-Boot FIT volume it may have subpartitions */
+#ifdef CONFIG_FIT_PARTITION
+		if (!efi_guidcmp(ptes[i].partition_type_guid, PARTITION_LINUX_FIT_GUID))
+			(void) parse_fit_partitions(state, start * ssz, size * ssz,
+						    &extra_slot, 127, 1);
+#endif
 	}
 	kfree(ptes);
 	kfree(gpt);

index 84b9f36b9e47..06c11f6ae398 100644
@@ -51,6 +51,9 @@
 #define PARTITION_LINUX_LVM_GUID \
     EFI_GUID( 0xe6d6d379, 0xf507, 0x44c2, \
               0xa2, 0x3c, 0x23, 0x8f, 0x2a, 0x3d, 0xf9, 0x28)
+#define PARTITION_LINUX_FIT_GUID \
+    EFI_GUID( 0xcae9be83, 0xb15f, 0x49cc, \
+              0x86, 0x3f, 0x08, 0x1b, 0x74, 0x4a, 0x2d, 0x93)
 
 typedef struct _gpt_header {
 	__le64 signature;

2.38.1

NAK, we should not go out from the partition code to parse random weird image formats. If you want to support uImage.FIT just write a tinty stackable block driver or dm table for it.

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Hi Christoph,

While weirdness is certainly subjective, uImage.FIT is not just a random image format but used by a great majority of headless embedded Linux devices out there. It's the default image format of many of the SDKs distributed by chip vendors such as Allwinner, Marvell, MediaTek, NXP, Qualcomm/Atheros, ...

Having better support for it in Linux hence doesn't seem too far-fetched to me, especially given that we got partition parsers for all sorts of historic (Acorn, Amiga, Atari, ...) or actually exotic (Karma?) formats. Even Microsoft Windows' Logical Disk Manager is supported natively by the kernel...

As this is used on rather tiny embedded devices my hope was to keep things simple and not having to enable device mapper on systems which have anyway only very small amounts of storage and won't ever need most of the device mapper features.

Using a tiny block driver instead is an option, I've implemented this approach in the past couple of hours and it works just as fine.

In this case I would introduce a new kernel cmdline option allowing to specify which block device (ie. a partition on eMMC, or mtdblock or ubiblock device) to launch the uImage.FIT parser on.

Allowing this new driver to add block partitions by exporting a new helper functions for that in block/partition/core.c would greatly simplify things, as then the existing partitioning code could still be used (instead of basically having to re-implement loopdev and introduce a whole new type of block devices).

I will post an RFC series illustrating this approach.

Please let me know if this sounds acceptable, so I won't put effort into implementing something which will then be rejected again after 5 iterations on the mailing list for reasons which could have been expressed from the beginning. An RFC for this series was posted on 2022-04-25 [1], I wouldn't have worked months to fix all requests of other maintainers and tested it on a variety of different hardware knowing that the whole approach will be NACK'ed...

And, of course, thank you anyway for reviewing!

Cheers

Daniel

[1]: https://patchwork.kernel.org/project/linux-block/list/?series=635369&state=*

"Look see, my weird format is used by all these companies building crappy SOCs, it is not weird.."

If people ignore something that is obviously broken they might just hope for it to go away, becaue often it does.

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

----- Ursprüngliche Mail -----

Well, FIT is not something strange invented by SoC companies, it comes from u-boot and is more or less a de-facto standard.
While I agree that using the block layer for partition parsing is questionable I think supporting these images in Linux is a worthwhile goal.

Thanks,
//richard

I never argued against that. But it is not a fit for partitions. So write a proper stacked block driver or dm driver for it if you care enough. The format is a complete mess and should be isolated to not affect the rest of the kernel.

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

I didn't invent this, and it's just as broken and yet perdominant as, let's say, MS LDM on x86.

While I'm sure that strategy works seen from your perspective, it does waste resources on the other end. In this case it might not have been obvious to everybody, I did receive feedback from other maintainers, as I said. It's not that everybody ignored this contribution. Hence, looking at it from my end, the picture is a bit different. Anyway. I would have appreciated an earlier explicite NACK, that's all I wanted to say.

Add Kconfig boolean CONFIG_MTD_BLOCK_PARTITIONS and enable block partition parsers on non-NAND mtdblock as well as ubiblock devices in case it is selected.
Never scan partitions on NAND-backed mtdblock devices, ubiblock should be used instead.

---
 drivers/mtd/Kconfig       | 11 +++++++++++
 drivers/mtd/mtd_blkdevs.c |  4 +++-
 drivers/mtd/ubi/block.c   |  5 ++++-
 3 files changed, 18 insertions(+), 2 deletions(-)

index 796a2eccbef0..12874dec1569 100644
@@ -69,6 +69,17 @@ config MTD_BLOCK_RO
 	  You do not need this option for use with the DiskOnChip devices. For
 	  those, enable NFTL support (CONFIG_NFTL) instead.
 
+config MTD_BLOCK_PARTITIONS
+	bool "Scan for partitions on MTD block devices"
+	depends on MTD_BLOCK || MTD_BLOCK_RO
+	default y if FIT_PARTITION
+	help
+	  Scan MTD block devices for partitions (ie. MBR, GPT, uImage.FIT, ...).
+	  (NAND devices are omitted, ubiblock should be used instead when)
+
+	  Unless your MTD partitions contain sub-partitions mapped using a
+	  partition table, say no.
+
 comment "Note that in some cases UBI block is preferred. See MTD_UBI_BLOCK."
 	depends on MTD_BLOCK || MTD_BLOCK_RO
 

index 60b222799871..e6f2e0888246 100644
@@ -359,7 +359,9 @@ int add_mtd_blktrans_dev(struct mtd_blktrans_dev *new)
 	} else {
 		snprintf(gd->disk_name, sizeof(gd->disk_name),
 			 "%s%d", tr->name, new->devnum);
-		gd->flags |= GENHD_FL_NO_PART;
+
+		if (!IS_ENABLED(CONFIG_MTD_BLOCK_PARTITIONS) || mtd_type_is_nand(new->mtd))
+			gd->flags |= GENHD_FL_NO_PART;
 	}
 
 	set_capacity(gd, ((u64)new->size * tr->blksize) >> 9);

index 75eaecc8639f..e617635ff6e6 100644
@@ -430,7 +430,10 @@ int ubiblock_create(struct ubi_volume_info *vi)
 		ret = -ENODEV;
 		goto out_cleanup_disk;
 	}
-	gd->flags |= GENHD_FL_NO_PART;
+
+	if (!IS_ENABLED(CONFIG_MTD_BLOCK_PARTITIONS))
+		gd->flags |= GENHD_FL_NO_PART;
+
 	gd->private_data = dev;
 	sprintf(gd->disk_name, "ubiblock%d_%d", dev->ubi_num, dev->vol_id);
 	set_capacity(gd, disk_capacity);

2.38.1