Bootloader support for UBI images
Hi,
The system I am developing keeps two kernel and root filesystem images to allow easy recovery from any errors. I would like to use UBI to store both the kernel and filesystem, to take advantage of the error recovery, wear leveling, etc. I see searching the list adding support for UBI into a bootloader has been discussed before, but no patches are available so far.
I have now ported a limited read-only version of UBI to redboot, to allow loading static volumes. This is at a stage where I can now boot the system after loading the kernel from the UBI volume. It should be easy to also extend this to work with u-boot (but I have no immediate need for this myself). I will be adding erase and writing for UBI images, but not updating volumes as this is significantly more complex. If people are interested I can post the redboot package.
Before I proceed any further I have some questions. To upgrade the system I would like to do something like:
# Remove backup volumes
ubirmvol /dev/ubi0 -N kernel.old
ubirmvol /dev/ubi0 -N rootfs.old
# Create new volumes
ubimkvol /dev/ubi0 -N kernel.new -s ...
ubimkvol /dev/ubi0 -N rootfs.old -s ...
# Write new volume contents
ubiupdatevol /dev/ubi0 -N kernel.new zImage
ubiupdatevol /dev/ubi0 -N rootfs.new rootfs
# Atomically rename volumes, switch to the new images
ubirenamevol /dev/ubi0 kernel kernel.old kernel.new kernel rootfs rootfs.old rootfs.new rootfs
This would require adding support for -N to ubiupdatevol to allow the volume name to be used (just syntactic sugar) . Also I need support for renaming multiple volumes atomically when switching images after the upgrade. If these sound sensible and useful changes I can look at creating patches to add these features.
The initial scanning of the flash in the bootloader seems to be taking a long time (longer than mounting the volume from linux). I need to look at why this is taking longer, but it would be great to avoid doing two full table scans. Passing the scan information from the bootloader to the kernel seems tricky, especially as the kernel builds multiple trees and lists during the scan. Bootloader support for UBI imagesA possible alternative approach would be to include an LER to PER index table for static volumes only. This table would only need updating when the static volume was rewritten or a PER moved due to a bit error or wear leveling. If the index table and the volume table were always located near the start or end of the flash then a full volume scan could be avoided in the bootloader. Does this sound like a reasonable optimization, or is another solution preferred?
Richard
Hi,
Quoted 16 lines · Richard Titmuss
Before I proceed any further I have some questions. To upgrade the system I would like to do something like:
# Remove backup volumes
ubirmvol/dev/ubi0-N kernel.old
ubirmvol/dev/ubi0-N rootfs.old
# Create new volumes
ubimkvol/dev/ubi0-N kernel.new -s ...
ubimkvol/dev/ubi0-N rootfs.old -s ...
# Write new volume contents
ubiupdatevol/dev/ubi0-N kernel.new zImage ubiupdatevol/dev/ubi0-N rootfs.new rootfs
# Atomically rename volumes, switch to the new images ubirenamevol/dev/ubi0kernel kernel.old kernel.new kernel rootfs rootfs.old rootfs.new rootfs
Quoted 2 lines
This would require adding support for -N to ubiupdatevol to allow the volume name to be used (just syntactic sugar).
Sounds fine and useful.
Quoted 4 lines
Also I need support for
renaming multiple volumes atomically when switching images after the upgrade. If these sound sensible and useful changes I can look at creating patches to add these features.
Also sounds as a good and useful idea. And implementation should not difficult - it is just about changing the volume table. You may count on my assistance.
I'm just not sure about the interface. Renaming one volume is fine. But you need to rename 2, so the interface should be more complex. And if 2, why not 3, or 5? Or 16? Or any amount from 1 to UBI_MAX_VOLUMES?
Quoted 4 lines
The initial scanning of the flash in the bootloader seems to be taking a long time (longer than mounting the volume from linux). I need to look at why this is taking longer, but it would be great to avoid doing two full table scans.
So you use the sub-page read patch BTW?
Quoted 11 lines
Passing the scan information from the bootloader to
the kernel seems tricky, especially as the kernel builds multiple trees and lists during the scan.
Bootloader support for UBI imagesA possible
alternative approach would be to include an LER to PER index table for static volumes only. This table would only need updating when the static volume was rewritten or a PER moved due to a bit error or wear leveling. If the index table and the volume table were always located near the start or end of the flash then a full volume scan could be avoided in the bootloader. Does this sound like a reasonable optimization, or is another solution preferred?
Well, surely the boot-loader may create a per-PEB (physical eraseblock) array and put all information about each PEB there (mapped/unmapped, LEB number, erase-counter, etc)? Then in UBI it will be easy to scan this array instead of the flash media. I am just not aware how such kind of bootloader->kernel data passing is done in Linux. Can you come up with a nice and mainline-acceptible way?
I did not actually catch the second part (LER and PER) tables - this is something I am not aware of. UBI has only volume table on flash, the other things are in-memory and build by scanning.
But in general, the idea not to do double scan but pass the scanning information from boot-loader to the kernel sounds good. We just need to figure out how to do this nicely.
Signature · 2 lines
Best regards, Artem Bityutskiy (Битюцкий Артём)
Quoted 11 lines · Artem Bityutskiy
Quoted 4 lines
Also I need support for
renaming multiple volumes atomically when switching images after the upgrade. If these sound sensible and useful changes I can look at creating patches to add these features.Also sounds as a good and useful idea. And implementation should not difficult - it is just about changing the volume table. You may count on my assistance.
I'm just not sure about the interface. Renaming one volume is fine. But you need to rename 2, so the interface should be more complex. And if 2, why not 3, or 5? Or 16? Or any amount from 1 to
UBI_MAX_VOLUMES?
Just a note: I was talking about how would the corresponding UBI ioctl(s) interface(s) look like. Kernel people are very picky at this.
Signature · 2 lines
Best regards, Artem Bityutskiy (Битюцкий Артём)
Quoted 11 lines · Artem Bityutskiy
Quoted 3 lines
Also I need support for
renaming multiple volumes atomically when switching images after the upgrade. If these sound sensible and useful changes I can look at creating patches to add these features.Also sounds as a good and useful idea. And implementation should not difficult - it is just about changing the volume table. You may count on my assistance.
I'm just not sure about the interface. Renaming one volume is fine. But you need to rename 2, so the interface should be more complex. And if 2, why not 3, or 5? Or 16? Or any amount from 1 to
UBI_MAX_VOLUMES?
Right, I was thinking that it would allow any number of volumes for renaming. I am also not sure about the interface, and was hoping to get some suggestions here. I will think about it and propose something in a few days.
Quoted 1 line
So you use the sub-page read patch BTW?
No, I'd missed that. Where is it?
Quoted 7 lines
Well, surely the boot-loader may create a per-PEB (physical eraseblock) array and put all information about each PEB there (mapped/unmapped, LEB number, erase-counter, etc)? Then in UBI it will be easy to scan this array instead of the flash media. I am just not aware how such kind of
bootloader->kerneldata passing is done in Linux. Can you come up with a nice and mainline-acceptible way?
Ok, that is possible. This would also have to include the bad block and bit flip status for each PEB. At the moment the Redboot flash api does not make this available, so it would mean extending Redboot or writing custom flash drivers to access this information.
Quoted 4 lines
I did not actually catch the second part (LER and PER) tables - this is something I am not aware of. UBI has only volume table on flash, the other things are in-memory and build by scanning.
It was a proposal to _add_ an addition table, that provided a LER to PER mapping for static volumes. If this table was stored near the beginning of the flash, then it could be accessed without a full scan. This could only be used if the UBI volume was mounted read-only, any write operations would require scanning all the flash blocks. This is an alternative approach to passing data from the bootloader to the kernel.
Richard
Quoted 4 lines · Richard Titmuss
Right, I was thinking that it would allow any number of volumes for renaming. I am also not sure about the interface, and was hoping to get some suggestions here. I will think about it and propose something in a few days.
Do you absolutely need multiple renames? Could you live with one at a time?
Quoted 2 lines
Quoted 1 line
So you use the sub-page read patch BTW?
No, I'd missed that. Where is it?
Here is the relevant thread. Some people already use that patch. dwmw2 was positive about it and told he would accept it.
http://lists.infradead.org/pipermail/linux-mtd/2008-April/021412.html
Quoted 4 lines
Ok, that is possible. This would also have to include the bad block and bit flip status for each PEB. At the moment the Redboot flash api does not make this available, so it would mean extending Redboot or writing custom flash drivers to access this information.
Well, bit-flip information is not mandatory and may be done later as an improvement. Bad-block information have to be available.
Quoted 6 lines
Quoted 4 lines
I did not actually catch the second part (LER and PER) tables - this is something I am not aware of. UBI has only volume table on flash, the other things are in-memory and build by scanning.
It was a proposal to _add_ an addition table, that provided a LER to PER mapping for static volumes.
Ah, this is s/LER/LEB/, s/PER/PEB/.
Quoted 5 lines
If this table was stored near the beginning
of the flash, then it could be accessed without a full scan. This could only be used if the UBI volume was mounted read-only, any write operations would require scanning all the flash blocks. This is an alternative approach to passing data from the bootloader to the kernel.
Well, may be done, but the R/O limitation seems to be too hard.
Signature · 2 lines
Best regards, Artem Bityutskiy (Битюцкий Артём)