Skip to contentSkip to conversation

lib: sparse: allocate FASTBOOT_MAX_BLK_WRITE instead of small number

PATCH In review Mattijs Korpershoek 3 participants 4 messages Jul 7, 2023 – Jul 25, 2023 Library code

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 lib: sparse: allocate FASTBOOT_MAX_BLK_WRITE instead of small number

Commit 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned") fixed cache alignment for systems with a D-CACHE.

However it introduced some performance regressions [1] on system flashing huge images, such as Android.

On AM62x SK EVM, we also observe such performance penalty:

Sending sparse 'super' 1/2 (768793 KB)             OKAY [ 23.954s]
Writing 'super'                                    OKAY [ 75.926s]
Sending sparse 'super' 2/2 (629819 KB)             OKAY [ 19.641s]
Writing 'super'                                    OKAY [ 62.849s]

Finished. Total time: 182.474s

The reason for this is that we use an arbitrary small buffer (info->blksz * 100) for transferring.

Fix it by using a bigger buffer (info->blksz * FASTBOOT_MAX_BLK_WRITE) as suggested in the original's patch review [2].

With this patch, performance impact is mitigated:

Sending sparse 'super' 1/2 (768793 KB)             OKAY [ 23.912s]
Writing 'super'                                    OKAY [ 15.780s]
Sending sparse 'super' 2/2 (629819 KB)             OKAY [ 19.581s]
Writing 'super'                                    OKAY [ 17.192s]

Finished. Total time: 76.569s

[1] https://lore.kernel.org/r/[email protected]
[2] https://lore.kernel.org/r/all/[email protected]/

Fixes 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned")

---
Changes in v2:
- Use FASTBOOT_MAX_BLK_WRITE instead of blkcnt (Qianfan)
- Link to v1: https://lore.kernel.org/r/[email protected]

---
 drivers/fastboot/fb_mmc.c | 2 --
 include/image-sparse.h    | 2 ++
 lib/image-sparse.c        | 3 ++-
 3 files changed, 4 insertions(+), 3 deletions(-)
drivers/fastboot/fb_mmc.c +0 -2
index 9d25c402028a..060918e49109 100644
@@ -19,8 +19,6 @@
 #include <linux/compat.h>
 #include <android_image.h>
 
-#define FASTBOOT_MAX_BLK_WRITE 16384
-
 #define BOOT_PARTITION_NAME "boot"
 
 struct fb_mmc_sparse {
include/image-sparse.h +2 -0
index 0572dbd0a283..282a0b256498 100644
@@ -7,6 +7,8 @@
 #include <part.h>
 #include <sparse_format.h>
 
+#define FASTBOOT_MAX_BLK_WRITE 16384
+
 #define ROUNDUP(x, y)	(((x) + ((y) - 1)) & ~((y) - 1))
 
 struct sparse_storage {
lib/image-sparse.c +2 -1
index 5ec0f94ab3eb..8f8a67e15804 100644
@@ -55,7 +55,8 @@ static lbaint_t write_sparse_chunk_raw(struct sparse_storage *info,
 				       void *data,
 				       char *response)
 {
-	lbaint_t n = blkcnt, write_blks, blks = 0, aligned_buf_blks = 100;
+	lbaint_t n = blkcnt, write_blks, blks = 0;
+	lbaint_t aligned_buf_blks = FASTBOOT_MAX_BLK_WRITE;
 	uint32_t *aligned_buf = NULL;
 
 	if (CONFIG_IS_ENABLED(SYS_DCACHE_OFF)) {

---
base-commit: 923de765ee1a5b26310f02cb42dcbad9e2b011c5
change-id: 20230616-sparse-flash-fix-9c2852aa8d16

Best regards,

Signature · 1 line
Mattijs Korpershoek <[email protected]>
#1 lore raw
Quoted 59 lines · Mattijs Korpershoek

Commit 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned") fixed cache alignment for systems with a D-CACHE.

However it introduced some performance regressions [1] on system flashing huge images, such as Android.

On AM62x SK EVM, we also observe such performance penalty:

Sending sparse 'super' 1/2 (768793 KB)             OKAY [ 23.954s]
Writing 'super'                                    OKAY [ 75.926s]
Sending sparse 'super' 2/2 (629819 KB)             OKAY [ 19.641s]
Writing 'super'                                    OKAY [ 62.849s]

Finished. Total time: 182.474s

The reason for this is that we use an arbitrary small buffer (info->blksz * 100) for transferring.

Fix it by using a bigger buffer (info->blksz * FASTBOOT_MAX_BLK_WRITE) as suggested in the original's patch review [2].

With this patch, performance impact is mitigated:

Sending sparse 'super' 1/2 (768793 KB)             OKAY [ 23.912s]
Writing 'super'                                    OKAY [ 15.780s]
Sending sparse 'super' 2/2 (629819 KB)             OKAY [ 19.581s]
Writing 'super'                                    OKAY [ 17.192s]

Finished. Total time: 76.569s

[1] https://lore.kernel.org/r/[email protected]
[2] https://lore.kernel.org/r/all/[email protected]/

Fixes 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned")

---
Changes in v2:
- Use FASTBOOT_MAX_BLK_WRITE instead of blkcnt (Qianfan)
- Link to v1: https://lore.kernel.org/r/[email protected]

---
  drivers/fastboot/fb_mmc.c | 2 --
  include/image-sparse.h    | 2 ++
  lib/image-sparse.c        | 3 ++-
  3 files changed, 4 insertions(+), 3 deletions(-)
drivers/fastboot/fb_mmc.c +0 -2
index 9d25c402028a..060918e49109 100644
@@ -19,8 +19,6 @@
  #include <linux/compat.h>
  #include <android_image.h>
 
-#define FASTBOOT_MAX_BLK_WRITE 16384
-
  #define BOOT_PARTITION_NAME "boot"
 
  struct fb_mmc_sparse {
include/image-sparse.h +0 -0
index 0572dbd0a283..282a0b256498 100644

include/image-sparse.h line 10

Quoted 4 lines
include/image-sparse.h +1 -0
@@ -7,6 +7,8 @@
  #include <part.h>
  #include <sparse_format.h>
 
+#define FASTBOOT_MAX_BLK_WRITE 16384

Hi:

Just a personal suggestion, define sometings like FASTBOOT_MAX_BLK_WRITE in image-sparse.c is very strange.

Or maybe define a marco such as SPARSE_MAX_BLK_WRITE and set a default value to 16384, and leave some comments for why we choice this value.

Thanks.

Quoted 24 lines
include/image-sparse.h +1 -0
+
  #define ROUNDUP(x, y)	(((x) + ((y) - 1)) & ~((y) - 1))
 
  struct sparse_storage {
lib/image-sparse.c +2 -2
index 5ec0f94ab3eb..8f8a67e15804 100644
@@ -55,7 +55,8 @@ static lbaint_t write_sparse_chunk_raw(struct sparse_storage *info,
  				       void *data,
  				       char *response)
  {
-	lbaint_t n = blkcnt, write_blks, blks = 0, aligned_buf_blks = 100;
+	lbaint_t n = blkcnt, write_blks, blks = 0;
+	lbaint_t aligned_buf_blks = FASTBOOT_MAX_BLK_WRITE;
  	uint32_t *aligned_buf = NULL;
 
  	if (CONFIG_IS_ENABLED(SYS_DCACHE_OFF)) {
 
---

base-commit: 923de765ee1a5b26310f02cb42dcbad9e2b011c5
change-id: 20230616-sparse-flash-fix-9c2852aa8d16

Best regards,

#2 lore raw

Hi Qianfan,

Thank you for your review.

Quoted 63 lines · qianfan
Quoted 62 lines · Mattijs Korpershoek

Commit 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned") fixed cache alignment for systems with a D-CACHE.

However it introduced some performance regressions [1] on system flashing huge images, such as Android.

On AM62x SK EVM, we also observe such performance penalty:

Sending sparse 'super' 1/2 (768793 KB)             OKAY [ 23.954s]
Writing 'super'                                    OKAY [ 75.926s]
Sending sparse 'super' 2/2 (629819 KB)             OKAY [ 19.641s]
Writing 'super'                                    OKAY [ 62.849s]

Finished. Total time: 182.474s

The reason for this is that we use an arbitrary small buffer (info->blksz * 100) for transferring.

Fix it by using a bigger buffer (info->blksz * FASTBOOT_MAX_BLK_WRITE) as suggested in the original's patch review [2].

With this patch, performance impact is mitigated:

Sending sparse 'super' 1/2 (768793 KB)             OKAY [ 23.912s]
Writing 'super'                                    OKAY [ 15.780s]
Sending sparse 'super' 2/2 (629819 KB)             OKAY [ 19.581s]
Writing 'super'                                    OKAY [ 17.192s]

Finished. Total time: 76.569s

[1] https://lore.kernel.org/r/[email protected]
[2] https://lore.kernel.org/r/all/[email protected]/

Fixes 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned")

---
Changes in v2:
- Use FASTBOOT_MAX_BLK_WRITE instead of blkcnt (Qianfan)
- Link to v1: https://lore.kernel.org/r/[email protected]

---
  drivers/fastboot/fb_mmc.c | 2 --
  include/image-sparse.h    | 2 ++
  lib/image-sparse.c        | 3 ++-
  3 files changed, 4 insertions(+), 3 deletions(-)
drivers/fastboot/fb_mmc.c +0 -2
index 9d25c402028a..060918e49109 100644
@@ -19,8 +19,6 @@
  #include <linux/compat.h>
  #include <android_image.h>
 
-#define FASTBOOT_MAX_BLK_WRITE 16384
-
  #define BOOT_PARTITION_NAME "boot"
 
  struct fb_mmc_sparse {
include/image-sparse.h +0 -0
index 0572dbd0a283..282a0b256498 100644
@@ -7,6 +7,8 @@
  #include <part.h>
  #include <sparse_format.h>
 
Quoted 8 lines

include/image-sparse.h line 10

Quoted 1 line
include/image-sparse.h +1 -0
+#define FASTBOOT_MAX_BLK_WRITE 16384

Hi:

Just a personal suggestion, define sometings like FASTBOOT_MAX_BLK_WRITE in image-sparse.c is very strange.

Or maybe define a marco such as SPARSE_MAX_BLK_WRITE and set a default value to 16384, and leave some comments for why we choice this value.

I don't agree with having a duplicating between SPARSE_MAX_BLK_WRITE and FASTBOOT_MAX_BLK_WRITE. code comments can rot as well.

And FASTBOOT_MAX_BLK_WRITE is used for both sparse and unsparsed image, which is why I chose to not rename it.

Quoted 26 lines

Thanks.

Quoted 24 lines
include/image-sparse.h +1 -0
+
  #define ROUNDUP(x, y)	(((x) + ((y) - 1)) & ~((y) - 1))
 
  struct sparse_storage {
lib/image-sparse.c +2 -2
index 5ec0f94ab3eb..8f8a67e15804 100644
@@ -55,7 +55,8 @@ static lbaint_t write_sparse_chunk_raw(struct sparse_storage *info,
  				       void *data,
  				       char *response)
  {
-	lbaint_t n = blkcnt, write_blks, blks = 0, aligned_buf_blks = 100;
+	lbaint_t n = blkcnt, write_blks, blks = 0;
+	lbaint_t aligned_buf_blks = FASTBOOT_MAX_BLK_WRITE;
  	uint32_t *aligned_buf = NULL;
 
  	if (CONFIG_IS_ENABLED(SYS_DCACHE_OFF)) {
 
---

base-commit: 923de765ee1a5b26310f02cb42dcbad9e2b011c5
change-id: 20230616-sparse-flash-fix-9c2852aa8d16

Best regards,

#3 lore raw
In reply to Mattijs Korpershoek
Quoted 24 lines · Mattijs Korpershoek

Commit 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned") fixed cache alignment for systems with a D-CACHE.

However it introduced some performance regressions [1] on system flashing huge images, such as Android.

On AM62x SK EVM, we also observe such performance penalty:

Sending sparse 'super' 1/2 (768793 KB)             OKAY [ 23.954s]
Writing 'super'                                    OKAY [ 75.926s]
Sending sparse 'super' 2/2 (629819 KB)             OKAY [ 19.641s]
Writing 'super'                                    OKAY [ 62.849s]

Finished. Total time: 182.474s

The reason for this is that we use an arbitrary small buffer (info->blksz * 100) for transferring.

Fix it by using a bigger buffer (info->blksz * FASTBOOT_MAX_BLK_WRITE) as suggested in the original's patch review [2].

With this patch, performance impact is mitigated:

Sending sparse 'super' 1/2 (768793 KB)             OKAY [ 23.912s]
Writing 'super'                                    OKAY [ 15.780s]
Sending sparse 'super' 2/2 (629819 KB)             OKAY [ 19.581s]
Writing 'super'                                    OKAY [ 17.192s]
Quoted 7 lines

Finished. Total time: 76.569s

[1] https://lore.kernel.org/r/[email protected]
[2] https://lore.kernel.org/r/all/[email protected]/

Fixes 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned")

Applied to u-boot/master, thanks!

Signature · 1 line
Tom