UBI: support EOF block marking end of ubinized image
Subsystems (2)
PATCH 1/1 UBI: support EOF block marking end of ubinized image
From: Rafał Miłecki <[email protected]>
On home routers UBI is often used with UBI unaware bootloaders. In such case an ubinized image needs to be created & flashed.
The problem is many such bootloaders are poorly written and they don't erase whole flash (partition) before writing an image. Only part of flash required to fit new image gets erased. It results in UBI complaining about flash garbage content, e.g.:
ubi0: attaching mtd2
ubi0 error: ubi_attach: bad image sequence number 1646161998 in PEB 119, expected 524503983 Erase counter header dump:
magic 0x55424923
version 1
ec 1
vid_hdr_offset 2048
data_offset 4096
image_seq 1646161998
hdr_crc 0x1eb28994erase counter header hexdump:
ubi0 error: ubi_attach_mtd_dev: failed to attach mtd2, error -22 UBI error: cannot attach mtd2
To fix up such flashed images it's required to identify the end of ubinized image on flash and erase remaining blocks.
Idea of extending on-disk UBI format to mark end of ubinized image was rejected, see:
[PATCH RFC 2/2] ubi: add support for UBI_EC_FLAG_ERASE_FROM_HERE
https://patchwork.ozlabs.org/project/linux-mtd/patch/[email protected]/
This patch implements support for a simple "EOF" text content (block) appended to the ubinized image instead. It doesn't require changing on-disk format and still allows simple integration into ubi code.
--- drivers/mtd/ubi/attach.c | 19 +++++++++++++++++++ drivers/mtd/ubi/ubi.h | 1 + 2 files changed, 20 insertions(+)
drivers/mtd/ubi/attach.c +19 -0
@@ -958,9 +958,28 @@ static int scan_peb(struct ubi_device *ubi, struct ubi_attach_info *ai,return 0 ; } +if (ai->eof_found) { + ai->empty_peb_count +=1 ; +return add_to_list (ai, pnum,UBI_UNKNOWN ,UBI_UNKNOWN , +UBI_UNKNOWN ,0 , &ai->erase); + } + err =ubi_io_read_ec_hdr (ubi, pnum, ech,0 );if (err <0 )return err; + +if (err ==UBI_IO_BAD_HDR ) { +uint8_t eof[] = {'E' ,'O' ,'F' }; +uint8_t *data = (uint8_t *)ech; + +if (!memcmp (data, eof,sizeof (eof))) { + ai->eof_found = true; + ai->empty_peb_count +=1 ; +return add_to_list (ai, pnum,UBI_UNKNOWN ,UBI_UNKNOWN , +UBI_UNKNOWN ,0 , &ai->erase); + } + } +switch (err) {case 0 :break ;
drivers/mtd/ubi/ubi.h +1 -0
@@ -783,6 +783,7 @@ struct ubi_attach_info {struct kmem_cache *aeb_slab_cache;struct ubi_ec_hdr *ech;struct ubi_vid_io_buf *vidb; +bool eof_found; };/**
Signature · 6 lines
2.34.1 ______________________________________________________ Linux MTD discussion mailing list http://lists.infradead.org/mailman/listinfo/linux-mtd/
Quoted 32 lines · Rafał Miłecki
From: Rafał Miłecki <[email protected]>
On home routers UBI is often used with UBI unaware bootloaders. In such case an ubinized image needs to be created & flashed.
The problem is many such bootloaders are poorly written and they don't erase whole flash (partition) before writing an image. Only part of flash required to fit new image gets erased. It results in UBI complaining about flash garbage content, e.g.:
ubi0: attaching mtd2
ubi0 error:ubi_attach: bad image sequence number 1646161998 in PEB 119, expected 524503983 Erase counter header dump:magic 0x55424923 version 1 ec 1 vid_hdr_offset 2048 data_offset 4096 image_seq 1646161998 hdr_crc 0x1eb28994erase counter header hexdump:
ubi0 error:ubi_attach_mtd_dev: failed to attach mtd2, error -22 UBI error: cannot attach mtd2To fix up such flashed images it's required to identify the end of ubinized image on flash and erase remaining blocks.
Idea of extending on-disk UBI format to mark end of ubinized image was rejected, see:
[PATCH RFC 2/2] ubi: add support forUBI_EC_FLAG_ERASE_FROM_HERE
https://patchwork.ozlabs.org/project/linux-mtd/patch/[email protected]/
Quoted 5 lines
This patch implements support for a simple "EOF" text content (block) appended to the ubinized image instead. It doesn't require changing on-disk format and still allows simple integration into ubi code.
Signed-off-by Rafał Miłecki <[email protected]>
While I personally think that UBI should provide *any* solution for such poor bootloaders it has been stated upstream won't accept this hack. There is no idea for any other / cleaner solution neither.
For details see e-mail thread:
Handling ubinized image flashed on not erased flash space
http://lists.infradead.org/pipermail/linux-mtd/2022-March/091993.html
http://lists.infradead.org/pipermail/linux-mtd/2022-March/092081.html
This patch is meant for downstream usage in projects like OpenWrt / Buildroot / Yocto / foo.
______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/
Zhihao Cheng
reviewerHi, Rafał
Quoted 11 lines
While I personally think that UBI should provide *any* solution for such poor bootloaders it has been stated upstream won't accept this hack. There is no idea for any other / cleaner solution neither.
For details see e-mail thread:
Handling ubinized image flashed on not erased flash space
http://lists.infradead.org/pipermail/linux-mtd/2022-March/091993.html
http://lists.infradead.org/pipermail/linux-mtd/2022-March/092081.htmlThis patch is meant for downstream usage in projects like OpenWrt / Buildroot / Yocto / foo.
Does 'downstream' mean this patch won't be applied to mainline? Eventhough, I don't think this modification is sutiable for common scenarios. The UBI image provided by poor bootloader does not follow the standard UBI format defined in [1], and linux kernel should have recognized bad image and rejected attaching rather than tries fixing bad image.
Maybe there are other better solutions? Such as,
1. For 'EOF' ended image scenarios, DIY an userspace tool to convert bad UBI image to valid UBI image.
2. Erasing/Formatting the whole mtd partition. (BTW, why does booloader only format part of mtd partition? What will the left space in mtd partition be used for?)
[1] http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubi_headers
Quoted 4 lines
______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/
______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/
Quoted 4 lines · Zhihao Cheng
Quoted 4 lines
While I personally think that UBI should provide *any* solution for such poor bootloaders it has been stated upstream won't accept this hack. There is no idea for any other / cleaner solution neither.
Quoted 8 lines
Quoted 7 lines
For details see e-mail thread:
Handling ubinized image flashed on not erased flash space
http://lists.infradead.org/pipermail/linux-mtd/2022-March/091993.html
http://lists.infradead.org/pipermail/linux-mtd/2022-March/092081.htmlThis patch is meant for downstream usage in projects like OpenWrt / Buildroot / Yocto / foo.
Does 'downstream' mean this patch won't be applied to mainline? Eventhough, I don't think this modification is sutiable for common scenarios. The UBI image provided by poor bootloader does not follow the standard UBI format defined in [1], and linux kernel should have recognized bad image and rejected attaching rather than tries fixing bad image.
Yes, downstream means carrying on that patch in projects using Linux kernel (I named few). They have to patch kernel with it before compiling it.
Upstream means simply:git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
I'm not sure what do you mean by "sutiable". Linux doesn't really offer for raw NAND flashes anything except UBI. So that's the only choice.
Of course those poor bootloaders don't "follow" UBI on-disk format. We all know that. The problem is they are there. On millions of devices.
People out of your utopia still need a way to use UBI with them. All crazy ideas like flashing intermediate firmware just to flash a target one properly (with UBI proper flash setup) just don't work in real world
Quoted 2 lines
Maybe there are other better solutions? Such as,
1. For 'EOF' ended image scenarios, DIY an userspace tool to convert bad UBI image to valid UBI image.
How does it make life easier for anyone?
1. Every distro still has to carry on such user-space tool
2. We don't have any intermediate rootfs so it needs to go to initramfs
3. That may confict with other initramfs usages
4. It requires scanning blocks twice (in user-space and then in kernel)
I don't see any gain from moving that task from kernel to user-space.
Quoted 1 line
2. Erasing/Formatting the whole mtd partition. (BTW, why does booloader only format part of mtd partition? What will the left space in mtd partition be used for?)
Don't ask me. Ask ASUS, Linksys, Netgear, TP-Link, whoever. I think you can even try Huawei - they seem to have some Broadcom and Ralink based hw that does the same.
______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/