Fastboot behaviour with sparse images
Hi,
I'm currently writing the support in U-Boot for NAND-backed devices using fastboot [1], and that work derived a bit to supporting the sparse images.
For "regular" images that are being stored, we expect a pair of download and flash commands. Simple.
Things start to get a bit more complex with sparse images that have been split because of a max-download-size lower than the actual image size.
Here, from what I could gather from various random blog posts, the fastboot client implementation and dumping a few USB sessions, the client simply creates several download / flash pairs, always on the same partition, without any way to distinct that from several subsequent writes issued by the user.
So, I'm guessing that the expectation is that the bootloader implementation should store the last offset it wrote to, and simple resume from there if the partition names in the flash commands are the same, which would prevent two subsequent write on the same partition by any client. Am I right?
A related question is when should we erase the NAND partition? Only when doing fastboot erase, or also when doing fastboot write (which, combined with the issue raised above, would also mean that we don't want to do an erase on the whole partition everytime there's a flash command on it).
Thanks!
Maxime
1: http://lists.denx.de/pipermail/u-boot/2015-August/226053.html
Signature · 10 lines
Maxime Ripard, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: Digital signature URL: <http://lists.denx.de/pipermail/u-boot/attachments/20151012/5b7991e0/attachment.sig>
Quoted 25 lines · Maxime Ripard
Hi,
I'm currently writing the support in U-Boot for NAND-backed devices using fastboot [1], and that work derived a bit to supporting the sparse images.
For "regular" images that are being stored, we expect a pair of download and flash commands. Simple.
Things start to get a bit more complex with sparse images that have been split because of a max-download-size lower than the actual image size.
Here, from what I could gather from various random blog posts, the fastboot client implementation and dumping a few USB sessions, the client simply creates several download / flash pairs, always on the same partition, without any way to distinct that from several subsequent writes issued by the user.
So, I'm guessing that the expectation is that the bootloader implementation should store the last offset it wrote to, and simple resume from there if the partition names in the flash commands are the same, which would prevent two subsequent write on the same partition by any client. Am I right?
Quoted 5 lines
A related question is when should we erase the NAND partition? Only when doing fastboot erase, or also when doing fastboot write (which, combined with the issue raised above, would also mean that we don't want to do an erase on the whole partition everytime there's a flash command on it).
I think for this last question, some experimentation with the existing tools might be required. As there's no required explicit erase for MMC, I think it might make sense to say you erase nand up front and then write as anything else starts getting really tricky and we're just second-guessing the user.
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: <http://lists.denx.de/pipermail/u-boot/attachments/20151021/410715c7/attachment.sig>
Hi Tom,
Quoted 32 lines · Tom Rini
Quoted 25 lines · Maxime Ripard
Hi,
I'm currently writing the support in U-Boot for NAND-backed devices using fastboot [1], and that work derived a bit to supporting the sparse images.
For "regular" images that are being stored, we expect a pair of download and flash commands. Simple.
Things start to get a bit more complex with sparse images that have been split because of a max-download-size lower than the actual image size.
Here, from what I could gather from various random blog posts, the fastboot client implementation and dumping a few USB sessions, the client simply creates several download / flash pairs, always on the same partition, without any way to distinct that from several subsequent writes issued by the user.
So, I'm guessing that the expectation is that the bootloader implementation should store the last offset it wrote to, and simple resume from there if the partition names in the flash commands are the same, which would prevent two subsequent write on the same partition by any client. Am I right?
Quoted 5 lines
A related question is when should we erase the NAND partition? Only when doing fastboot erase, or also when doing fastboot write (which, combined with the issue raised above, would also mean that we don't want to do an erase on the whole partition everytime there's a flash command on it).
Quoted 5 lines
I think for this last question, some experimentation with the existing tools might be required. As there's no required explicit erase for MMC, I think it might make sense to say you erase nand up front and then write as anything else starts getting really tricky and we're just second-guessing the user.
Actually, the only FS the fastboot tool seems to be doing it for the moment are ext4 and F2FS. It can probably be extended to UBI and raw partitions, but that won't fix the tools that are bundled by the distros at the moment.
So I guess we can always erase it now using the session counter: if we are writing the first chunk, erase the whole partition, if we're not, then simply flash it at the previous offset.
How does it sound?
Maxime
Signature · 10 lines
Maxime Ripard, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: Digital signature URL: <http://lists.denx.de/pipermail/u-boot/attachments/20151022/80b3135f/attachment.sig>
Quoted 46 lines · Maxime Ripard
Hi Tom,
Quoted 32 lines · Tom Rini
Quoted 25 lines · Maxime Ripard
Hi,
I'm currently writing the support in U-Boot for NAND-backed devices using fastboot [1], and that work derived a bit to supporting the sparse images.
For "regular" images that are being stored, we expect a pair of download and flash commands. Simple.
Things start to get a bit more complex with sparse images that have been split because of a max-download-size lower than the actual image size.
Here, from what I could gather from various random blog posts, the fastboot client implementation and dumping a few USB sessions, the client simply creates several download / flash pairs, always on the same partition, without any way to distinct that from several subsequent writes issued by the user.
So, I'm guessing that the expectation is that the bootloader implementation should store the last offset it wrote to, and simple resume from there if the partition names in the flash commands are the same, which would prevent two subsequent write on the same partition by any client. Am I right?
Quoted 5 lines
A related question is when should we erase the NAND partition? Only when doing fastboot erase, or also when doing fastboot write (which, combined with the issue raised above, would also mean that we don't want to do an erase on the whole partition everytime there's a flash command on it).
Quoted 5 lines
I think for this last question, some experimentation with the existing tools might be required. As there's no required explicit erase for MMC, I think it might make sense to say you erase nand up front and then write as anything else starts getting really tricky and we're just second-guessing the user.
Actually, the only FS the fastboot tool seems to be doing it for the moment are ext4 and F2FS. It can probably be extended to UBI and raw partitions, but that won't fix the tools that are bundled by the distros at the moment.
Quoted 5 lines
So I guess we can always erase it now using the session counter: if we are writing the first chunk, erase the whole partition, if we're not, then simply flash it at the previous offset.
How does it sound?
Sounds workable but testing with the existing tools will be the key and the hard part here :(
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: <http://lists.denx.de/pipermail/u-boot/attachments/20151022/aea21652/attachment.sig>
Quoted 47 lines · Tom Rini
Quoted 46 lines · Maxime Ripard
Hi Tom,
Quoted 32 lines · Tom Rini
Quoted 25 lines · Maxime Ripard
Hi,
I'm currently writing the support in U-Boot for NAND-backed devices using fastboot [1], and that work derived a bit to supporting the sparse images.
For "regular" images that are being stored, we expect a pair of download and flash commands. Simple.
Things start to get a bit more complex with sparse images that have been split because of a max-download-size lower than the actual image size.
Here, from what I could gather from various random blog posts, the fastboot client implementation and dumping a few USB sessions, the client simply creates several download / flash pairs, always on the same partition, without any way to distinct that from several subsequent writes issued by the user.
So, I'm guessing that the expectation is that the bootloader implementation should store the last offset it wrote to, and simple resume from there if the partition names in the flash commands are the same, which would prevent two subsequent write on the same partition by any client. Am I right?
Quoted 5 lines
A related question is when should we erase the NAND partition? Only when doing fastboot erase, or also when doing fastboot write (which, combined with the issue raised above, would also mean that we don't want to do an erase on the whole partition everytime there's a flash command on it).
Quoted 5 lines
I think for this last question, some experimentation with the existing tools might be required. As there's no required explicit erase for MMC, I think it might make sense to say you erase nand up front and then write as anything else starts getting really tricky and we're just second-guessing the user.
Actually, the only FS the fastboot tool seems to be doing it for the moment are ext4 and F2FS. It can probably be extended to UBI and raw partitions, but that won't fix the tools that are bundled by the distros at the moment.
Quoted 8 lines
Quoted 5 lines
So I guess we can always erase it now using the session counter: if we are writing the first chunk, erase the whole partition, if we're not, then simply flash it at the previous offset.
How does it sound?
Sounds workable but testing with the existing tools will be the key and the hard part here :(
Well, if we always erase when we write, the worst case scenario would be one erase too many. It doesn't sound that bad, or hard.
Maxime
Signature · 10 lines
Maxime Ripard, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: Digital signature URL: <http://lists.denx.de/pipermail/u-boot/attachments/20151022/0215fabc/attachment.sig>
Quoted 20 lines · Maxime Ripard
Hi,
I'm currently writing the support in U-Boot for NAND-backed devices using fastboot [1], and that work derived a bit to supporting the sparse images.
For "regular" images that are being stored, we expect a pair of download and flash commands. Simple.
Things start to get a bit more complex with sparse images that have been split because of a max-download-size lower than the actual image size.
Here, from what I could gather from various random blog posts, the fastboot client implementation and dumping a few USB sessions, the client simply creates several download / flash pairs, always on the same partition, without any way to distinct that from several subsequent writes issued by the user.
Quoted 6 lines
So, I'm guessing that the expectation is that the bootloader implementation should store the last offset it wrote to, and simple resume from there if the partition names in the flash commands are the same, which would prevent two subsequent write on the same partition by any client. Am I right?
No, each blob passed to the bootloader will begin with a sparse "skip" chunk that will seek to the correct place to resume writing. The bootloader shouldn't need to store any metadata across commands. Just read in the blob from the data command, then write it out using a port of the Apache-licensed libsparse during the flash command.
Quoted 5 lines
A related question is when should we erase the NAND partition? Only when doing fastboot erase, or also when doing fastboot write (which, combined with the issue raised above, would also mean that we don't want to do an erase on the whole partition everytime there's a flash command on it).
Fastboot should send an erase command before every sequence of writes.
Erase the whole partition on the erase command, and don't erase anything
on the flash command.
Quoted 2 lines
Thanks!
Quoted 9 lines
Maxime
1: http://lists.denx.de/pipermail/u-boot/2015-August/226053.html
Signature · 3 lines
Maxime Ripard, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com
Hi Colin,
Quoted 28 lines · Colin Cross
Quoted 20 lines · Maxime Ripard
Hi,
I'm currently writing the support in U-Boot for NAND-backed devices using fastboot [1], and that work derived a bit to supporting the sparse images.
For "regular" images that are being stored, we expect a pair of download and flash commands. Simple.
Things start to get a bit more complex with sparse images that have been split because of a max-download-size lower than the actual image size.
Here, from what I could gather from various random blog posts, the fastboot client implementation and dumping a few USB sessions, the client simply creates several download / flash pairs, always on the same partition, without any way to distinct that from several subsequent writes issued by the user.
Quoted 6 lines
So, I'm guessing that the expectation is that the bootloader implementation should store the last offset it wrote to, and simple resume from there if the partition names in the flash commands are the same, which would prevent two subsequent write on the same partition by any client. Am I right?
Quoted 5 lines
No, each blob passed to the bootloader will begin with a sparse "skip" chunk that will seek to the correct place to resume writing. The bootloader shouldn't need to store any metadata across commands. Just read in the blob from the data command, then write it out using a port of the Apache-licensed libsparse during the flash command.
Oh, so that's how it works. Great. I guess however that you still need to scan out the area you skip for bad blocks to account them in the offset calculation as well then (when you're using NAND).
Quoted 9 lines
Quoted 5 lines
A related question is when should we erase the NAND partition? Only when doing fastboot erase, or also when doing fastboot write (which, combined with the issue raised above, would also mean that we don't want to do an erase on the whole partition everytime there's a flash command on it).
Fastboot should send an erase command before every sequence of writes.
Erase the whole partition on the erase command, and don't erase anything
on the flash command.
Ack.
Thanks!
Maxime
Signature · 10 lines
Maxime Ripard, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: Digital signature URL: <http://lists.denx.de/pipermail/u-boot/attachments/20151104/033d9ae0/attachment.sig>