lib: sparse: allocate blkcnt instead of arbitrary small number
PATCH 1/1 lib: sparse: allocate blkcnt instead of arbitrary small number
Commit 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned") fixed cache alignment for systems with a D-CACHE.
However it introduced some performance regressions [1] on system flashing huge images, such as Android.
On AM62x SK EVM, we also observe such performance penalty:
Sending sparse 'super' 1/2 (768793 KB) OKAY [ 23.954s] Writing 'super' OKAY [ 75.926s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.641s] Writing 'super' OKAY [ 62.849s]
Finished. Total time: 182.474s
The reason for this is that we use an arbitrary small buffer (info->blksz * 100) for transferring.
Fix it by using a bigger buffer (info->blksz * blkcnt) as suggested in the original's patch review [2].
With this patch, performance impact is mitigated:
Sending sparse 'super' 1/2 (768793 KB) OKAY [ 24.006s] Writing 'super' OKAY [ 15.920s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.651s] Writing 'super' OKAY [ 14.665s]
Finished. Total time: 74.346s
[1] https://lore.kernel.org/r/[email protected]
[2] https://lore.kernel.org/r/all/[email protected]/
--- lib/image-sparse.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/image-sparse.c +1 -1
@@ -55,7 +55,7 @@ static lbaint_t write_sparse_chunk_raw(struct sparse_storage *info,void *data, char *response) { - lbaint_t n = blkcnt, write_blks, blks = 0, aligned_buf_blks = 100; + lbaint_t n = blkcnt, write_blks, blks = 0, aligned_buf_blks = blkcnt;uint32_t *aligned_buf =NULL ;if (CONFIG_IS_ENABLED (SYS_DCACHE_OFF )) {
---
base-commit: 2f4664f5c3edc55b18d8906f256a4c8e303243c0
change-id: 20230616-sparse-flash-fix-9c2852aa8d16
Best regards,
Signature · 1 line
Mattijs Korpershoek <[email protected]>
Quoted 40 lines · Mattijs Korpershoek
Commit
62649165cb02("lib: sparse: MakeCHUNK_TYPE_RAWbuffer aligned") fixed cache alignment for systems with a D-CACHE.However it introduced some performance regressions [1] on system flashing huge images, such as Android.
On AM62x SK EVM, we also observe such performance penalty:
Sending sparse 'super' 1/2 (768793 KB) OKAY [ 23.954s] Writing 'super' OKAY [ 75.926s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.641s] Writing 'super' OKAY [ 62.849s]Finished. Total time: 182.474s
The reason for this is that we use an arbitrary small buffer (
info->blksz* 100) for transferring.Fix it by using a bigger buffer (
info->blksz* blkcnt) as suggested in the original's patch review [2].With this patch, performance impact is mitigated:
Sending sparse 'super' 1/2 (768793 KB) OKAY [ 24.006s] Writing 'super' OKAY [ 15.920s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.651s] Writing 'super' OKAY [ 14.665s]Finished. Total time: 74.346s
[1] https://lore.kernel.org/r/[email protected]
[2] https://lore.kernel.org/r/all/[email protected]/Fixes 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned")Signed-off-by Mattijs Korpershoek <[email protected]>--- lib/image-sparse.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)lib/image-sparse.c +0
-0
lib/image-sparse.c static lbaint_t write_sparse_chunk_raw(struct sparse_storage *info, · line 58
Quoted 5 lines
lib/image-sparse.c +1
-1@@ -55,7 +55,7 @@ static lbaint_t write_sparse_chunk_raw(struct sparse_storage *info,void *data, char *response) { - lbaint_t n = blkcnt, write_blks, blks = 0, aligned_buf_blks = 100; + lbaint_t n = blkcnt, write_blks, blks = 0, aligned_buf_blks = blkcnt;
Hi:
It's a good point that this code report the performance was affected by write large small mmc blks, not memory copy.
And I can not make sure whether memalign can always alloc such huge memory when we change the aligned_buf_blks to blkcnt.
Could you please set aligned_buf_blks to FASTBOOT_MAX_BLK_WRITE(16384) and test again?
Quoted 9 lines
lib/image-sparse.c +0
-1uint32_t *aligned_buf =NULL ;if (CONFIG_IS_ENABLED (SYS_DCACHE_OFF )) { ---base-commit:
2f4664f5c3edc55b18d8906f256a4c8e303243c0
change-id: 20230616-sparse-flash-fix-9c2852aa8d16Best regards,
Hi Qianfan,
Thank you for your review.
Quoted 48 lines · qianfan
Quoted 40 lines · Mattijs Korpershoek
Commit
62649165cb02("lib: sparse: MakeCHUNK_TYPE_RAWbuffer aligned") fixed cache alignment for systems with a D-CACHE.However it introduced some performance regressions [1] on system flashing huge images, such as Android.
On AM62x SK EVM, we also observe such performance penalty:
Sending sparse 'super' 1/2 (768793 KB) OKAY [ 23.954s] Writing 'super' OKAY [ 75.926s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.641s] Writing 'super' OKAY [ 62.849s]Finished. Total time: 182.474s
The reason for this is that we use an arbitrary small buffer (
info->blksz* 100) for transferring.Fix it by using a bigger buffer (
info->blksz* blkcnt) as suggested in the original's patch review [2].With this patch, performance impact is mitigated:
Sending sparse 'super' 1/2 (768793 KB) OKAY [ 24.006s] Writing 'super' OKAY [ 15.920s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.651s] Writing 'super' OKAY [ 14.665s]Finished. Total time: 74.346s
[1] https://lore.kernel.org/r/[email protected]
[2] https://lore.kernel.org/r/all/[email protected]/Fixes 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned")Signed-off-by Mattijs Korpershoek <[email protected]>--- lib/image-sparse.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)lib/image-sparse.c +0
-0lib/image-sparse.c static lbaint_t write_sparse_chunk_raw(struct sparse_storage *info, · line 58
Quoted 5 lines
lib/image-sparse.c +1
-1@@ -55,7 +55,7 @@ static lbaint_t write_sparse_chunk_raw(struct sparse_storage *info,void *data, char *response) { - lbaint_t n = blkcnt, write_blks, blks = 0, aligned_buf_blks = 100; + lbaint_t n = blkcnt, write_blks, blks = 0, aligned_buf_blks = blkcnt;Hi:
Quoted 3 lines
It's a good point that this code report the performance was affected by write large small
mmc blks, not memory copy.
I believe memory copy also affects performance, but in my case, it has less impact than small mmc blks.
With 62649165cb02 reverted:
Sending sparse 'super' 1/2 (768793 KB) OKAY [ 23.947s] Writing 'super' OKAY [ 12.983s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.600s] Writing 'super' OKAY [ 12.796s]
Finished. Total time: 69.430s
With aligned_buf_blks = blkcnt:
Sending sparse 'super' 1/2 (768793 KB) OKAY [ 24.072s] Writing 'super' OKAY [ 16.177s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.681s] Writing 'super' OKAY [ 14.845s]
Finished. Total time: 74.919s
Quoted 4 lines
And I can not make sure whether memalign can always alloc such huge memory when we change the
aligned_buf_blksto blkcnt.
Could you clarify the concern here? I've dumped blkcnt for my board (AM62x SK EVK) and the biggest blkcnt I found was: 131072
With info->blksz = 512, this gives me: 512 * 131072 = 67108864
Which is a memalign (memory alloc) of 64MB. Is 64MB really that big? (I don't realize it's that much)
Quoted 3 lines
Could you please set
aligned_buf_blkstoFASTBOOT_MAX_BLK_WRITE(16384)and test again?
With aligned_buf_blks = FASTBOOT_MAX_BLK_WRITE(16384):
Sending sparse 'super' 1/2 (768793 KB) OKAY [ 23.912s] Writing 'super' OKAY [ 15.780s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.581s] Writing 'super' OKAY [ 17.192s]
Finished. Total time: 76.569s
So using FASTBOOT_MAX_BLK_WRITE is slightly worse than using blkcnt. But allocations (for blksz = 512) are smaller: 8MB instead of 64MB in my example.
I can spin up a v2 with FASTBOOT_MAX_BLK_WRITE but i'm waiting a little more feedback before doing so.
Quoted 9 lines
Quoted 9 lines
lib/image-sparse.c +0
-1uint32_t *aligned_buf =NULL ;if (CONFIG_IS_ENABLED (SYS_DCACHE_OFF )) { ---base-commit:
2f4664f5c3edc55b18d8906f256a4c8e303243c0
change-id: 20230616-sparse-flash-fix-9c2852aa8d16Best regards,
Quoted 97 lines · Mattijs Korpershoek
Hi Qianfan,
Thank you for your review.
Quoted 48 lines · qianfan
Quoted 40 lines · Mattijs Korpershoek
Commit
62649165cb02("lib: sparse: MakeCHUNK_TYPE_RAWbuffer aligned") fixed cache alignment for systems with a D-CACHE.However it introduced some performance regressions [1] on system flashing huge images, such as Android.
On AM62x SK EVM, we also observe such performance penalty:
Sending sparse 'super' 1/2 (768793 KB) OKAY [ 23.954s] Writing 'super' OKAY [ 75.926s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.641s] Writing 'super' OKAY [ 62.849s]Finished. Total time: 182.474s
The reason for this is that we use an arbitrary small buffer (
info->blksz* 100) for transferring.Fix it by using a bigger buffer (
info->blksz* blkcnt) as suggested in the original's patch review [2].With this patch, performance impact is mitigated:
Sending sparse 'super' 1/2 (768793 KB) OKAY [ 24.006s] Writing 'super' OKAY [ 15.920s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.651s] Writing 'super' OKAY [ 14.665s]Finished. Total time: 74.346s
[1] https://lore.kernel.org/r/[email protected]
[2] https://lore.kernel.org/r/all/[email protected]/Fixes 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned")Signed-off-by Mattijs Korpershoek <[email protected]>--- lib/image-sparse.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)lib/image-sparse.c +0
-0lib/image-sparse.c static lbaint_t write_sparse_chunk_raw(struct sparse_storage *info, · line 58
Quoted 5 lines
lib/image-sparse.c +1
-1@@ -55,7 +55,7 @@ static lbaint_t write_sparse_chunk_raw(struct sparse_storage *info,void *data, char *response) { - lbaint_t n = blkcnt, write_blks, blks = 0, aligned_buf_blks = 100; + lbaint_t n = blkcnt, write_blks, blks = 0, aligned_buf_blks = blkcnt;Hi:
Quoted 3 lines
It's a good point that this code report the performance was affected by write large small
mmc blks, not memory copy.I believe memory copy also affects performance, but in my case, it has less impact than small mmc blks.
With
62649165cb02reverted:Sending sparse 'super' 1/2 (768793 KB) OKAY [ 23.947s] Writing 'super' OKAY [ 12.983s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.600s] Writing 'super' OKAY [ 12.796s]Finished. Total time: 69.430s
With
aligned_buf_blks= blkcnt:Sending sparse 'super' 1/2 (768793 KB) OKAY [ 24.072s] Writing 'super' OKAY [ 16.177s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.681s] Writing 'super' OKAY [ 14.845s]Finished. Total time: 74.919s
Quoted 4 lines
And I can not make sure whether memalign can always alloc such huge memory when we change the
aligned_buf_blksto blkcnt.Could you clarify the concern here? I've dumped blkcnt for my board (AM62x SK EVK) and the biggest blkcnt I found was: 131072
With
info->blksz= 512, this gives me: 512 * 131072 = 67108864Which is a memalign (memory alloc) of 64MB. Is 64MB really that big? (I don't realize it's that much)
Quoted 3 lines
Could you please set
aligned_buf_blkstoFASTBOOT_MAX_BLK_WRITE(16384)and test again?With
aligned_buf_blks=FASTBOOT_MAX_BLK_WRITE(16384):Sending sparse 'super' 1/2 (768793 KB) OKAY [ 23.912s] Writing 'super' OKAY [ 15.780s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.581s] Writing 'super' OKAY [ 17.192s]
Quoted 7 lines
Finished. Total time: 76.569s
So using
FASTBOOT_MAX_BLK_WRITEis slightly worse than using blkcnt. But allocations (for blksz = 512) are smaller: 8MB instead of 64MB in my example.I can spin up a v2 with
FASTBOOT_MAX_BLK_WRITEbut i'm waiting a little more feedback before doing so.
Hi Marek, Tom,
What's your take on this ? Can we keep blkcnt or should I respin using FASTBOOT_MAX_BLK_WRITE ?
I have also tested this on VIM3, on
U-Boot 2023.07-rc6-00003-g923de765ee1a:
Sending sparse 'super' 1/13 (114684 KB) OKAY [ 5.442s] Writing 'super' OKAY [ 5.791s] Sending sparse 'super' 2/13 (114684 KB) OKAY [ 5.706s] Writing 'super' OKAY [ 5.607s] Sending sparse 'super' 3/13 (114684 KB) OKAY [ 5.468s] Writing 'super' OKAY [ 5.835s] Sending sparse 'super' 4/13 (114684 KB) OKAY [ 5.703s] Writing 'super' OKAY [ 5.618s] Sending sparse 'super' 5/13 (114684 KB) OKAY [ 6.176s] Writing 'super' OKAY [ 5.421s] Sending sparse 'super' 6/13 (104176 KB) OKAY [ 5.204s] Writing 'super' OKAY [ 5.199s] Sending sparse 'super' 7/13 (108856 KB) OKAY [ 5.456s] Writing 'super' OKAY [ 5.290s] Sending sparse 'super' 8/13 (114684 KB) OKAY [ 6.122s] Writing 'super' OKAY [ 5.838s] Sending sparse 'super' 9/13 (114684 KB) OKAY [ 5.951s] Writing 'super' OKAY [ 5.857s] Sending sparse 'super' 10/13 (100980 KB) OKAY [ 4.902s] Writing 'super' OKAY [ 4.749s] Sending sparse 'super' 11/13 (114681 KB) OKAY [ 6.041s] Writing 'super' OKAY [ 5.779s] Sending sparse 'super' 12/13 (107212 KB) OKAY [ 5.174s] Writing 'super' OKAY [ 6.587s] Sending sparse 'super' 13/13 (71496 KB) OKAY [ 3.717s] Writing 'super' OKAY [ 3.744s]
Finished. Total time: 142.578s
With this patch:
Sending sparse 'super' 1/13 (114684 KB) OKAY [ 7.149s] Writing 'super' OKAY [ 1.639s] Sending sparse 'super' 2/13 (114684 KB) OKAY [ 6.993s] Writing 'super' OKAY [ 1.713s] Sending sparse 'super' 3/13 (114684 KB) OKAY [ 7.029s] Writing 'super' OKAY [ 1.107s] Sending sparse 'super' 4/13 (114684 KB) OKAY [ 7.027s] Writing 'super' OKAY [ 0.162s] Sending sparse 'super' 5/13 (114684 KB) OKAY [ 6.930s] Writing 'super' OKAY [ 1.643s] Sending sparse 'super' 6/13 (104176 KB) OKAY [ 6.253s] Writing 'super' OKAY [ 2.348s] Sending sparse 'super' 7/13 (108856 KB) OKAY [ 6.346s] Writing 'super' OKAY [ 0.723s] Sending sparse 'super' 8/13 (114684 KB) OKAY [ 6.715s] Writing 'super' OKAY [ 2.848s] Sending sparse 'super' 9/13 (114684 KB) OKAY [ 6.888s] Writing 'super' OKAY [ 1.928s] Sending sparse 'super' 10/13 (100980 KB) OKAY [ 5.979s] Writing 'super' OKAY [ 1.178s] Sending sparse 'super' 11/13 (114681 KB) OKAY [ 6.822s] Writing 'super' OKAY [ 2.652s] Sending sparse 'super' 12/13 (107212 KB) OKAY [ 6.414s] Writing 'super' OKAY [ 5.109s] Sending sparse 'super' 13/13 (71496 KB) OKAY [ 4.238s] Writing 'super' OKAY [ 0.252s]
Finished. Total time: 108.151s
It's probably too late for v2023.07 to pick this up but can we consider taking it for next?
Thanks a lot
Mattijs
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
lib/image-sparse.c +0
-1uint32_t *aligned_buf =NULL ;if (CONFIG_IS_ENABLED (SYS_DCACHE_OFF )) { ---base-commit:
2f4664f5c3edc55b18d8906f256a4c8e303243c0
change-id: 20230616-sparse-flash-fix-9c2852aa8d16Best regards,
Quoted 170 lines · Mattijs Korpershoek
Quoted 97 lines · Mattijs Korpershoek
Hi Qianfan,
Thank you for your review.
Quoted 48 lines · qianfan
Quoted 40 lines · Mattijs Korpershoek
Commit
62649165cb02("lib: sparse: MakeCHUNK_TYPE_RAWbuffer aligned") fixed cache alignment for systems with a D-CACHE.However it introduced some performance regressions [1] on system flashing huge images, such as Android.
On AM62x SK EVM, we also observe such performance penalty:
Sending sparse 'super' 1/2 (768793 KB) OKAY [ 23.954s] Writing 'super' OKAY [ 75.926s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.641s] Writing 'super' OKAY [ 62.849s]Finished. Total time: 182.474s
The reason for this is that we use an arbitrary small buffer (
info->blksz* 100) for transferring.Fix it by using a bigger buffer (
info->blksz* blkcnt) as suggested in the original's patch review [2].With this patch, performance impact is mitigated:
Sending sparse 'super' 1/2 (768793 KB) OKAY [ 24.006s] Writing 'super' OKAY [ 15.920s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.651s] Writing 'super' OKAY [ 14.665s]Finished. Total time: 74.346s
[1] https://lore.kernel.org/r/[email protected]
[2] https://lore.kernel.org/r/all/[email protected]/Fixes 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned")Signed-off-by Mattijs Korpershoek <[email protected]>--- lib/image-sparse.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)lib/image-sparse.c +0
-0lib/image-sparse.c static lbaint_t write_sparse_chunk_raw(struct sparse_storage *info, · line 58
Quoted 5 lines
lib/image-sparse.c +1
-1@@ -55,7 +55,7 @@ static lbaint_t write_sparse_chunk_raw(struct sparse_storage *info,void *data, char *response) { - lbaint_t n = blkcnt, write_blks, blks = 0, aligned_buf_blks = 100; + lbaint_t n = blkcnt, write_blks, blks = 0, aligned_buf_blks = blkcnt;Hi:
Quoted 3 lines
It's a good point that this code report the performance was affected by write large small
mmc blks, not memory copy.I believe memory copy also affects performance, but in my case, it has less impact than small mmc blks.
With
62649165cb02reverted:Sending sparse 'super' 1/2 (768793 KB) OKAY [ 23.947s] Writing 'super' OKAY [ 12.983s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.600s] Writing 'super' OKAY [ 12.796s]Finished. Total time: 69.430s
With
aligned_buf_blks= blkcnt:Sending sparse 'super' 1/2 (768793 KB) OKAY [ 24.072s] Writing 'super' OKAY [ 16.177s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.681s] Writing 'super' OKAY [ 14.845s]Finished. Total time: 74.919s
Quoted 4 lines
And I can not make sure whether memalign can always alloc such huge memory when we change the
aligned_buf_blksto blkcnt.Could you clarify the concern here? I've dumped blkcnt for my board (AM62x SK EVK) and the biggest blkcnt I found was: 131072
With
info->blksz= 512, this gives me: 512 * 131072 = 67108864Which is a memalign (memory alloc) of 64MB. Is 64MB really that big? (I don't realize it's that much)
Quoted 3 lines
Could you please set
aligned_buf_blkstoFASTBOOT_MAX_BLK_WRITE(16384)and test again?With
aligned_buf_blks=FASTBOOT_MAX_BLK_WRITE(16384):Sending sparse 'super' 1/2 (768793 KB) OKAY [ 23.912s] Writing 'super' OKAY [ 15.780s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.581s] Writing 'super' OKAY [ 17.192s]Quoted 7 lines
Finished. Total time: 76.569s
So using
FASTBOOT_MAX_BLK_WRITEis slightly worse than using blkcnt. But allocations (for blksz = 512) are smaller: 8MB instead of 64MB in my example.I can spin up a v2 with
FASTBOOT_MAX_BLK_WRITEbut i'm waiting a little more feedback before doing so.Hi Marek, Tom,
What's your take on this ? Can we keep blkcnt or should I respin using
FASTBOOT_MAX_BLK_WRITE?I have also tested this on VIM3, on
U-Boot 2023.07-rc6-00003-g923de765ee1a:Sending sparse 'super' 1/13 (114684 KB) OKAY [ 5.442s] Writing 'super' OKAY [ 5.791s] Sending sparse 'super' 2/13 (114684 KB) OKAY [ 5.706s] Writing 'super' OKAY [ 5.607s] Sending sparse 'super' 3/13 (114684 KB) OKAY [ 5.468s] Writing 'super' OKAY [ 5.835s] Sending sparse 'super' 4/13 (114684 KB) OKAY [ 5.703s] Writing 'super' OKAY [ 5.618s] Sending sparse 'super' 5/13 (114684 KB) OKAY [ 6.176s] Writing 'super' OKAY [ 5.421s] Sending sparse 'super' 6/13 (104176 KB) OKAY [ 5.204s] Writing 'super' OKAY [ 5.199s] Sending sparse 'super' 7/13 (108856 KB) OKAY [ 5.456s] Writing 'super' OKAY [ 5.290s] Sending sparse 'super' 8/13 (114684 KB) OKAY [ 6.122s] Writing 'super' OKAY [ 5.838s] Sending sparse 'super' 9/13 (114684 KB) OKAY [ 5.951s] Writing 'super' OKAY [ 5.857s] Sending sparse 'super' 10/13 (100980 KB) OKAY [ 4.902s] Writing 'super' OKAY [ 4.749s] Sending sparse 'super' 11/13 (114681 KB) OKAY [ 6.041s] Writing 'super' OKAY [ 5.779s] Sending sparse 'super' 12/13 (107212 KB) OKAY [ 5.174s] Writing 'super' OKAY [ 6.587s] Sending sparse 'super' 13/13 (71496 KB) OKAY [ 3.717s] Writing 'super' OKAY [ 3.744s]Finished. Total time: 142.578s
With this patch:
Sending sparse 'super' 1/13 (114684 KB) OKAY [ 7.149s] Writing 'super' OKAY [ 1.639s] Sending sparse 'super' 2/13 (114684 KB) OKAY [ 6.993s] Writing 'super' OKAY [ 1.713s] Sending sparse 'super' 3/13 (114684 KB) OKAY [ 7.029s] Writing 'super' OKAY [ 1.107s] Sending sparse 'super' 4/13 (114684 KB) OKAY [ 7.027s] Writing 'super' OKAY [ 0.162s] Sending sparse 'super' 5/13 (114684 KB) OKAY [ 6.930s] Writing 'super' OKAY [ 1.643s] Sending sparse 'super' 6/13 (104176 KB) OKAY [ 6.253s] Writing 'super' OKAY [ 2.348s] Sending sparse 'super' 7/13 (108856 KB) OKAY [ 6.346s] Writing 'super' OKAY [ 0.723s] Sending sparse 'super' 8/13 (114684 KB) OKAY [ 6.715s] Writing 'super' OKAY [ 2.848s] Sending sparse 'super' 9/13 (114684 KB) OKAY [ 6.888s] Writing 'super' OKAY [ 1.928s] Sending sparse 'super' 10/13 (100980 KB) OKAY [ 5.979s] Writing 'super' OKAY [ 1.178s] Sending sparse 'super' 11/13 (114681 KB) OKAY [ 6.822s] Writing 'super' OKAY [ 2.652s] Sending sparse 'super' 12/13 (107212 KB) OKAY [ 6.414s] Writing 'super' OKAY [ 5.109s] Sending sparse 'super' 13/13 (71496 KB) OKAY [ 4.238s] Writing 'super' OKAY [ 0.252s]
Quoted 4 lines
Finished. Total time: 108.151s
It's probably too late for v2023.07 to pick this up but can we consider taking it for next?
I was waiting for a v2, and yes, it's too late for v2023.07. Sorry for not being clear enough.
Signature · 1 line
Tom
Quoted 173 lines · Tom Rini
Quoted 172 lines · Mattijs Korpershoek
Quoted 97 lines · Mattijs Korpershoek
Hi Qianfan,
Thank you for your review.
Quoted 48 lines · qianfan
Quoted 40 lines · Mattijs Korpershoek
Commit
62649165cb02("lib: sparse: MakeCHUNK_TYPE_RAWbuffer aligned") fixed cache alignment for systems with a D-CACHE.However it introduced some performance regressions [1] on system flashing huge images, such as Android.
On AM62x SK EVM, we also observe such performance penalty:
Sending sparse 'super' 1/2 (768793 KB) OKAY [ 23.954s] Writing 'super' OKAY [ 75.926s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.641s] Writing 'super' OKAY [ 62.849s]Finished. Total time: 182.474s
The reason for this is that we use an arbitrary small buffer (
info->blksz* 100) for transferring.Fix it by using a bigger buffer (
info->blksz* blkcnt) as suggested in the original's patch review [2].With this patch, performance impact is mitigated:
Sending sparse 'super' 1/2 (768793 KB) OKAY [ 24.006s] Writing 'super' OKAY [ 15.920s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.651s] Writing 'super' OKAY [ 14.665s]Finished. Total time: 74.346s
[1] https://lore.kernel.org/r/[email protected]
[2] https://lore.kernel.org/r/all/[email protected]/Fixes 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned")Signed-off-by Mattijs Korpershoek <[email protected]>--- lib/image-sparse.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)lib/image-sparse.c +0
-0lib/image-sparse.c static lbaint_t write_sparse_chunk_raw(struct sparse_storage *info, · line 58
Quoted 5 lines
lib/image-sparse.c +1
-1@@ -55,7 +55,7 @@ static lbaint_t write_sparse_chunk_raw(struct sparse_storage *info,void *data, char *response) { - lbaint_t n = blkcnt, write_blks, blks = 0, aligned_buf_blks = 100; + lbaint_t n = blkcnt, write_blks, blks = 0, aligned_buf_blks = blkcnt;Hi:
Quoted 3 lines
It's a good point that this code report the performance was affected by write large small
mmc blks, not memory copy.I believe memory copy also affects performance, but in my case, it has less impact than small mmc blks.
With
62649165cb02reverted:Sending sparse 'super' 1/2 (768793 KB) OKAY [ 23.947s] Writing 'super' OKAY [ 12.983s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.600s] Writing 'super' OKAY [ 12.796s]Finished. Total time: 69.430s
With
aligned_buf_blks= blkcnt:Sending sparse 'super' 1/2 (768793 KB) OKAY [ 24.072s] Writing 'super' OKAY [ 16.177s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.681s] Writing 'super' OKAY [ 14.845s]Finished. Total time: 74.919s
Quoted 4 lines
And I can not make sure whether memalign can always alloc such huge memory when we change the
aligned_buf_blksto blkcnt.Could you clarify the concern here? I've dumped blkcnt for my board (AM62x SK EVK) and the biggest blkcnt I found was: 131072
With
info->blksz= 512, this gives me: 512 * 131072 = 67108864Which is a memalign (memory alloc) of 64MB. Is 64MB really that big? (I don't realize it's that much)
Quoted 3 lines
Could you please set
aligned_buf_blkstoFASTBOOT_MAX_BLK_WRITE(16384)and test again?With
aligned_buf_blks=FASTBOOT_MAX_BLK_WRITE(16384):Sending sparse 'super' 1/2 (768793 KB) OKAY [ 23.912s] Writing 'super' OKAY [ 15.780s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.581s] Writing 'super' OKAY [ 17.192s]Quoted 7 lines
Finished. Total time: 76.569s
So using
FASTBOOT_MAX_BLK_WRITEis slightly worse than using blkcnt. But allocations (for blksz = 512) are smaller: 8MB instead of 64MB in my example.I can spin up a v2 with
FASTBOOT_MAX_BLK_WRITEbut i'm waiting a little more feedback before doing so.Hi Marek, Tom,
What's your take on this ? Can we keep blkcnt or should I respin using
FASTBOOT_MAX_BLK_WRITE?I have also tested this on VIM3, on
U-Boot 2023.07-rc6-00003-g923de765ee1a:Sending sparse 'super' 1/13 (114684 KB) OKAY [ 5.442s] Writing 'super' OKAY [ 5.791s] Sending sparse 'super' 2/13 (114684 KB) OKAY [ 5.706s] Writing 'super' OKAY [ 5.607s] Sending sparse 'super' 3/13 (114684 KB) OKAY [ 5.468s] Writing 'super' OKAY [ 5.835s] Sending sparse 'super' 4/13 (114684 KB) OKAY [ 5.703s] Writing 'super' OKAY [ 5.618s] Sending sparse 'super' 5/13 (114684 KB) OKAY [ 6.176s] Writing 'super' OKAY [ 5.421s] Sending sparse 'super' 6/13 (104176 KB) OKAY [ 5.204s] Writing 'super' OKAY [ 5.199s] Sending sparse 'super' 7/13 (108856 KB) OKAY [ 5.456s] Writing 'super' OKAY [ 5.290s] Sending sparse 'super' 8/13 (114684 KB) OKAY [ 6.122s] Writing 'super' OKAY [ 5.838s] Sending sparse 'super' 9/13 (114684 KB) OKAY [ 5.951s] Writing 'super' OKAY [ 5.857s] Sending sparse 'super' 10/13 (100980 KB) OKAY [ 4.902s] Writing 'super' OKAY [ 4.749s] Sending sparse 'super' 11/13 (114681 KB) OKAY [ 6.041s] Writing 'super' OKAY [ 5.779s] Sending sparse 'super' 12/13 (107212 KB) OKAY [ 5.174s] Writing 'super' OKAY [ 6.587s] Sending sparse 'super' 13/13 (71496 KB) OKAY [ 3.717s] Writing 'super' OKAY [ 3.744s]Finished. Total time: 142.578s
With this patch:
Sending sparse 'super' 1/13 (114684 KB) OKAY [ 7.149s] Writing 'super' OKAY [ 1.639s] Sending sparse 'super' 2/13 (114684 KB) OKAY [ 6.993s] Writing 'super' OKAY [ 1.713s] Sending sparse 'super' 3/13 (114684 KB) OKAY [ 7.029s] Writing 'super' OKAY [ 1.107s] Sending sparse 'super' 4/13 (114684 KB) OKAY [ 7.027s] Writing 'super' OKAY [ 0.162s] Sending sparse 'super' 5/13 (114684 KB) OKAY [ 6.930s] Writing 'super' OKAY [ 1.643s] Sending sparse 'super' 6/13 (104176 KB) OKAY [ 6.253s] Writing 'super' OKAY [ 2.348s] Sending sparse 'super' 7/13 (108856 KB) OKAY [ 6.346s] Writing 'super' OKAY [ 0.723s] Sending sparse 'super' 8/13 (114684 KB) OKAY [ 6.715s] Writing 'super' OKAY [ 2.848s] Sending sparse 'super' 9/13 (114684 KB) OKAY [ 6.888s] Writing 'super' OKAY [ 1.928s] Sending sparse 'super' 10/13 (100980 KB) OKAY [ 5.979s] Writing 'super' OKAY [ 1.178s] Sending sparse 'super' 11/13 (114681 KB) OKAY [ 6.822s] Writing 'super' OKAY [ 2.652s] Sending sparse 'super' 12/13 (107212 KB) OKAY [ 6.414s] Writing 'super' OKAY [ 5.109s] Sending sparse 'super' 13/13 (71496 KB) OKAY [ 4.238s] Writing 'super' OKAY [ 0.252s]Finished. Total time: 108.151s
Quoted 5 lines
Quoted 2 lines
It's probably too late for v2023.07 to pick this up but can we consider taking it for next?
I was waiting for a v2, and yes, it's too late for v2023.07. Sorry for not being clear enough.
Oh, sorry I did not understand that. I understand for v2023.07. Thank you for the quick answer.
Will send a v2 shortly using FASTBOOT_MAX_BLK_WRITE.
Quoted 3 lines
Signature · 2 lines
-- Tom