fs: fat: dynamically allocate memory for temporary buffer
PATCH v3 1/2 fs: fat: dynamically allocate memory for temporary buffer
From: Tien Fong Chee <[email protected]>
Drop the statically allocated get_contents_vfatname_block and dynamically allocate a buffer only if required. This saves 64KiB of memory.
---
changes for v3
- Removed the cast on actsize
--- fs/fat/fat.c | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-)
fs/fat/fat.c +12 -6
@@ -306,9 +306,6 @@ get_cluster(fsdata *mydata, __u32 clustnum, __u8 *buffer, unsigned long size)* into 'buffer'. * Update the number of bytes read in *gotsize or return -1 on fatal errors. */ -__u8 get_contents_vfatname_block[MAX_CLUSTSIZE ] -__aligned(ARCH_DMA_MINALIGN) ; -static int get_contents (fsdata *mydata,dir_entry *dentptr,loff_t pos,__u8 *buffer,loff_t maxsize,loff_t *gotsize) {
@@ -351,15 +348,24 @@ static int get_contents(fsdata *mydata, dir_entry *dentptr, loff_t pos,/* align to beginning of next cluster if any */ if (pos) { +__u8 *tmp_buffer; + actsize =min (filesize, (loff_t )bytesperclust); -if (get_cluster (mydata, curclust, get_contents_vfatname_block, - (int )actsize) !=0 ) { + tmp_buffer =malloc_cache_aligned (actsize); +if (!tmp_buffer) { +debug ("Error: allocating buffer\n" ); +return -ENOMEM ; + } + +if (get_cluster (mydata, curclust, tmp_buffer, actsize) !=0 ) {printf ("Error reading cluster\n" ); +free (tmp_buffer);return -1 ; } filesize -= actsize; actsize -= pos; -memcpy (buffer, get_contents_vfatname_block + pos, actsize); +memcpy (buffer, tmp_buffer + pos, actsize); +free (tmp_buffer); *gotsize += actsize;if (!filesize)return 0 ;
Signature · 1 line
2.2.0
PATCH v3 2/2 fs: fat: Reduce default max clustersize 64KiB from malloc pool
From: Tien Fong Chee <[email protected]>
Release cluster block immediately when no longer use would help to reduce 64KiB memory allocated to the memory pool.
---
changes for v3
- Dropped the if conditional because free(NULL) is valid.
--- fs/fat/fat.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
fs/fat/fat.c +9 -1
@@ -1151,7 +1151,15 @@ int file_fat_read_at(const char *filename, loff_t pos, void *buffer, goto out_free_both;debug ("reading %s at pos %llu\n" , filename, pos); - ret =get_contents (&fsdata, itr->dent, pos, buffer, maxsize, actread); + +/* For saving default max clustersize memory allocated to malloc pool */ +dir_entry *dentptr = itr->dent; + +free (itr); + + itr =NULL ; + + ret =get_contents (&fsdata, dentptr, pos, buffer, maxsize, actread); out_free_both:free (fsdata.fatbuf);
Signature · 1 line
2.2.0
PATCH v3 2/2 fs: fat: Reduce default max clustersize 64KiB from malloc pool
Quoted 6 lines · tien.fong.chee at intel.com
From: Tien Fong Chee <[email protected]>
Release cluster block immediately when no longer use would help to reduce 64KiB memory allocated to the memory pool.
Signed-off-by Tien Fong Chee <[email protected]>
Applied to u-boot/master, thanks!
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: not available URL: <http://lists.denx.de/pipermail/u-boot/attachments/20190219/3418b56c/attachment.sig>
PATCH v3 1/2 fs: fat: dynamically allocate memory for temporary buffer
In reply to tien.fong.chee at intel.comQuoted 8 lines · tien.fong.chee at intel.com
From: Tien Fong Chee <[email protected]>
Drop the statically allocated
get_contents_vfatname_blockand dynamically allocate a buffer only if required. This saves 64KiB of memory.Signed-off-by Stefan Agner <[email protected]>Signed-off-by Tien Fong Chee <[email protected]>
Applied to u-boot/master, thanks!
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: not available URL: <http://lists.denx.de/pipermail/u-boot/attachments/20190219/6ab82a55/attachment.sig>
PATCH v3 1/2 fs: fat: dynamically allocate memory for temporary buffer
Hi Tom,
Quoted 4 lines · Tom Rini
Quoted 2 lines · tien.fong.chee at intel.com
From: Tien Fong Chee <[email protected]>
Quoted 8 lines
Quoted 6 lines
Drop the statically allocated
get_contents_vfatname_blockand dynamically allocate a buffer only if required. This saves 64KiB of memory.Signed-off-by Stefan Agner <[email protected]>Signed-off-by Tien Fong Chee <[email protected]>Applied to u-boot/master, thanks!
please remove this patch (better both of them because they were in series) because they are breaking at least ZynqMP SPL. It is also too late in cycle to create random fix.
You can't simply move 64KB from code to malloc without reflecting this by changing MALLOC space size.
Other boards with SPL fat could be also affected by this if they don't allocate big malloc space.
Thanks,
Michal
Signature · 5 lines
Michal Simek, Ing. (M.Eng), OpenPGP -> KeyID: FE3D1F91 w: www.monstr.eu p: +42-0-721842854 Maintainer of Linux kernel - Xilinx Microblaze Maintainer of Linux kernel - Xilinx Zynq ARM and ZynqMP ARM64 SoCs U-Boot custodian - Xilinx Microblaze/Zynq/ZynqMP/Versal SoCs
Chee
PATCH v3 1/2 fs: fat: dynamically allocate memory for temporary buffer
Quoted 14 lines · Michal Simek
Hi Tom,
Quoted 11 lines · Tom Rini
Quoted 7 lines · tien.fong.chee at intel.com
From: Tien Fong Chee <[email protected]>
Drop the statically allocated
get_contents_vfatname_blockand dynamically allocate a buffer only if required. This saves 64KiB of memory.
Quoted 5 lines
Quoted 3 lines
Quoted 2 lines
Signed-off-by Stefan Agner <[email protected]>Signed-off-by Tien Fong Chee <[email protected]>Applied to u-boot/master, thanks!
please remove this patch (better both of them because they were in series)
I think patch 2/2 should be safe, because no memory size is changed. Basically, it just to release the allocated memory immediately when it's not required, so other can re-use it.
Quoted 10 lines
because they are breaking at least ZynqMP SPL. It is also too late in cycle to create random fix.
You can't simply move 64KB from code to malloc without reflecting this
by changing MALLOC space size.Other boards with SPL fat could be also affected by this if they don't
allocate big malloc space.
So, any suggestion to get the patch 1/2 accepted? inform all board maintainers to test it out?
Quoted 4 lines
Thanks,
Michal
Thanks,
Tien Fong.
PATCH v3 1/2 fs: fat: dynamically allocate memory for temporary buffer
Quoted 31 lines · Tien Fong
Quoted 14 lines · Michal Simek
Hi Tom,
Quoted 11 lines · Tom Rini
Quoted 7 lines · tien.fong.chee at intel.com
From: Tien Fong Chee <[email protected]>
Drop the statically allocated
get_contents_vfatname_blockand dynamically allocate a buffer only if required. This saves 64KiB of memory.Quoted 5 lines
Quoted 3 lines
Quoted 2 lines
Signed-off-by Stefan Agner <[email protected]>Signed-off-by Tien Fong Chee <[email protected]>Applied to u-boot/master, thanks!
please remove this patch (better both of them because they were in series)
I think patch 2/2 should be safe, because no memory size is changed. Basically, it just to release the allocated memory immediately when it's not required, so other can re-use it.
Quoted 7 lines
because they are breaking at least ZynqMP SPL. It is also too late in cycle to create random fix.
You can't simply move 64KB from code to malloc without reflecting this
by changing MALLOC space size.
Quoted 5 lines
Quoted 3 lines
Other boards with SPL fat could be also affected by this if they don't
allocate big malloc space.So, any suggestion to get the patch 1/2 accepted? inform all board maintainers to test it out?
You already received feedback that it does break ZynqMP, so the current approach won't work.
How about you create a new kconfig option that allows you to say whether you want to use malloc or .bss for temporary data in the FAT driver. You can then have an _SPL_ version of that kconfig and check for it with IS_ENABLED() which should automatically tell you the right answer depending on whether you're in an SPL build or not.
Then you can set the SPL version to default malloc and the non-SPL version to default .bss.
That should give you the fix you want, without the problems it introduces for SPL (where malloc space is really constrained, and discouraged to use because you can't check whether it fits at compile time).
Alex
Chee
PATCH v3 1/2 fs: fat: dynamically allocate memory for temporary buffer
Quoted 58 lines · Alexander Graf
Quoted 39 lines · Tien Fong
Quoted 18 lines · Michal Simek
Hi Tom,
Quoted 14 lines · Tom Rini
Quoted 8 lines · tien.fong.chee at intel. com
From: Tien Fong Chee <[email protected]>
Drop the statically allocated
get_contents_vfatname_blockand dynamically allocate a buffer only if required. This saves 64KiB of memory.Quoted 6 lines
Quoted 3 lines
Quoted 2 lines
Signed-off-by Stefan Agner <[email protected]>Signed-off-by Tien Fong Chee <[email protected]>Applied to u-boot/master, thanks!
please remove this patch (better both of them because they were in
series)I think patch 2/2 should be safe, because no memory size is changed.
Basically, it just to release the allocated memory immediately when it's not required, so other can re-use it.Quoted 8 lines
because they are breaking at least ZynqMP SPL. It is also too late in cycle to create random fix.
You can't simply move 64KB from code to malloc without reflecting this
by changing MALLOC space size.Quoted 5 lines
Quoted 3 lines
Other boards with SPL fat could be also affected by this if they don't
allocate big malloc space.So, any suggestion to get the patch 1/2 accepted? inform all board maintainers to test it out?
You already received feedback that it does break ZynqMP, so the current
approach won't work.How about you create a new kconfig option that allows you to say whether
you want to use malloc or .bss for temporary data in the FAT driver. You
can then have an _SPL_ version of that kconfig and check for it withIS_ENABLED()which should automatically tell you the right answer depending on whether you're in an SPL build or not.
Quoted 2 lines
Then you can set the SPL version to default malloc and the non-SPL version to default .bss.
Marek and Tom rini,
Are you guys okay with Alex's suggestion?
Quoted 8 lines
That should give you the fix you want, without the problems it introduces for SPL (where malloc space is really constrained, and discouraged to use because you can't check whether it fits at compile time).
Alex
Thanks.
TF.
PATCH v3 1/2 fs: fat: dynamically allocate memory for temporary buffer
Quoted 59 lines · Tien Fong
Quoted 58 lines · Alexander Graf
Quoted 39 lines · Tien Fong
Quoted 18 lines · Michal Simek
Hi Tom,
Quoted 14 lines · Tom Rini
Quoted 8 lines · tien.fong.chee at intel. com
From: Tien Fong Chee <[email protected]>
Drop the statically allocated
get_contents_vfatname_blockand dynamically allocate a buffer only if required. This saves 64KiB of memory.Quoted 6 lines
Quoted 3 lines
Quoted 2 lines
Signed-off-by Stefan Agner <[email protected]>Signed-off-by Tien Fong Chee <[email protected]>Applied to u-boot/master, thanks!
please remove this patch (better both of them because they were in
series)I think patch 2/2 should be safe, because no memory size is changed.
Basically, it just to release the allocated memory immediately when it's not required, so other can re-use it.Quoted 8 lines
because they are breaking at least ZynqMP SPL. It is also too late in cycle to create random fix.
You can't simply move 64KB from code to malloc without reflecting this
by changing MALLOC space size.Quoted 5 lines
Quoted 3 lines
Other boards with SPL fat could be also affected by this if they don't
allocate big malloc space.So, any suggestion to get the patch 1/2 accepted? inform all board maintainers to test it out?
You already received feedback that it does break ZynqMP, so the current
approach won't work.How about you create a new kconfig option that allows you to say whether
you want to use malloc or .bss for temporary data in the FAT driver. You
can then have an _SPL_ version of that kconfig and check for it withIS_ENABLED()which should automatically tell you the right answer depending on whether you're in an SPL build or not.
Quoted 5 lines
Quoted 2 lines
Then you can set the SPL version to default malloc and the non-SPL version to default .bss.
Marek and Tom rini,
Are you guys okay with Alex's suggestion?
I'm not a big fan of adding more and more ifdeffery. Is there some other option ?
Signature · 2 lines
Best regards, Marek Vasut
PATCH v3 1/2 fs: fat: dynamically allocate memory for temporary buffer
Quoted 60 lines · Marek Vasut
Quoted 59 lines · Tien Fong
Quoted 58 lines · Alexander Graf
Quoted 39 lines · Tien Fong
Quoted 18 lines · Michal Simek
Hi Tom,
Quoted 14 lines · Tom Rini
Quoted 8 lines · tien.fong.chee at intel. com
From: Tien Fong Chee <[email protected]>
Drop the statically allocated
get_contents_vfatname_blockand dynamically allocate a buffer only if required. This saves 64KiB of memory.Quoted 6 lines
Quoted 3 lines
Quoted 2 lines
Signed-off-by Stefan Agner <[email protected]>Signed-off-by Tien Fong Chee <[email protected]>Applied to u-boot/master, thanks!
please remove this patch (better both of them because they were in
series)I think patch 2/2 should be safe, because no memory size is changed.
Basically, it just to release the allocated memory immediately when it's not required, so other can re-use it.Quoted 8 lines
because they are breaking at least ZynqMP SPL. It is also too late in cycle to create random fix.
You can't simply move 64KB from code to malloc without reflecting this
by changing MALLOC space size.Quoted 5 lines
Quoted 3 lines
Other boards with SPL fat could be also affected by this if they don't
allocate big malloc space.So, any suggestion to get the patch 1/2 accepted? inform all board maintainers to test it out?
You already received feedback that it does break ZynqMP, so the current
approach won't work.How about you create a new kconfig option that allows you to say whether
you want to use malloc or .bss for temporary data in the FAT driver. You
can then have an _SPL_ version of that kconfig and check for it withIS_ENABLED()which should automatically tell you the right answer depending on whether you're in an SPL build or not.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Then you can set the SPL version to default malloc and the non-SPL version to default .bss.
Marek and Tom rini,
Are you guys okay with Alex's suggestion?
I'm not a big fan of adding more and more ifdeffery. Is there some other option ?
Is RAM up already at this point? Maybe we could improve the SPL malloc mechanism to move allocations into DRAM once it's available.
Alex
PATCH v3 1/2 fs: fat: dynamically allocate memory for temporary buffer
Quoted 67 lines · Alexander Graf
Quoted 64 lines · Marek Vasut
Quoted 63 lines · Tien Fong
Quoted 58 lines · Alexander Graf
Quoted 39 lines · Tien Fong
Quoted 18 lines · Michal Simek
Hi Tom,
Quoted 14 lines · Tom Rini
Quoted 8 lines · tien.fong.chee at intel. com
From: Tien Fong Chee <[email protected]>
Drop the statically allocated
get_contents_vfatname_blockand dynamically allocate a buffer only if required. This saves 64KiB of memory.Quoted 6 lines
Quoted 3 lines
Quoted 2 lines
Signed-off-by Stefan Agner <[email protected]>Signed-off-by Tien Fong Chee <[email protected]>Applied to u-boot/master, thanks!
please remove this patch (better both of them because they were in
series)I think patch 2/2 should be safe, because no memory size is changed.
Basically, it just to release the allocated memory immediately when it's not required, so other can re-use it.Quoted 8 lines
because they are breaking at least ZynqMP SPL. It is also too late in cycle to create random fix.
You can't simply move 64KB from code to malloc without reflecting this
by changing MALLOC space size.Quoted 5 lines
Quoted 3 lines
Other boards with SPL fat could be also affected by this if they don't
allocate big malloc space.So, any suggestion to get the patch 1/2 accepted? inform all board maintainers to test it out?
You already received feedback that it does break ZynqMP, so the current
approach won't work.How about you create a new kconfig option that allows you to say whether
you want to use malloc or .bss for temporary data in the FAT driver. You
can then have an _SPL_ version of that kconfig and check for it withIS_ENABLED()which should automatically tell you the right answer depending on whether you're in an SPL build or not.Quoted 2 lines
Then you can set the SPL version to default malloc and the non-SPL version to default .bss.
Marek and Tom rini,
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Are you guys okay with Alex's suggestion?
I'm not a big fan of adding more and more ifdeffery. Is there some other option ?
Is RAM up already at this point? Maybe we could improve the SPL malloc mechanism to move allocations into DRAM once it's available.
Well, the FAT buffers waste some 64kiB of bss, so we can use that area for malloc instead, no ?
Signature · 2 lines
Best regards, Marek Vasut
PATCH v3 1/2 fs: fat: dynamically allocate memory for temporary buffer
Quoted 70 lines · Marek Vasut
Quoted 69 lines · Alexander Graf
Quoted 66 lines · Marek Vasut
Quoted 64 lines · Tien Fong
Quoted 58 lines · Alexander Graf
Quoted 39 lines · Tien Fong
Quoted 18 lines · Michal Simek
Hi Tom,
Quoted 14 lines · Tom Rini
Quoted 8 lines · tien.fong.chee at intel. com
From: Tien Fong Chee <[email protected]>
Drop the statically allocated
get_contents_vfatname_blockand dynamically allocate a buffer only if required. This saves 64KiB of memory.Quoted 6 lines
Quoted 3 lines
Quoted 2 lines
Signed-off-by Stefan Agner <[email protected]>Signed-off-by Tien Fong Chee <[email protected]>Applied to u-boot/master, thanks!
please remove this patch (better both of them because they were in
series)I think patch 2/2 should be safe, because no memory size is changed.
Basically, it just to release the allocated memory immediately when it's not required, so other can re-use it.Quoted 8 lines
because they are breaking at least ZynqMP SPL. It is also too late in cycle to create random fix.
You can't simply move 64KB from code to malloc without reflecting this
by changing MALLOC space size.Quoted 5 lines
Quoted 3 lines
Other boards with SPL fat could be also affected by this if they don't
allocate big malloc space.So, any suggestion to get the patch 1/2 accepted? inform all board maintainers to test it out?
You already received feedback that it does break ZynqMP, so the current
approach won't work.How about you create a new kconfig option that allows you to say whether
you want to use malloc or .bss for temporary data in the FAT driver. You
can then have an _SPL_ version of that kconfig and check for it withIS_ENABLED()which should automatically tell you the right answer depending on whether you're in an SPL build or not.Quoted 2 lines
Then you can set the SPL version to default malloc and the non-SPL version to default .bss.
Marek and Tom rini,
Are you guys okay with Alex's suggestion?
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
I'm not a big fan of adding more and more ifdeffery. Is there some other option ?
Is RAM up already at this point? Maybe we could improve the SPL malloc mechanism to move allocations into DRAM once it's available.
Well, the FAT buffers waste some 64kiB of bss, so we can use that area for malloc instead, no ?
Yes, but that means you need to review every single board that uses FAT in SPL today and adjust its malloc region size.
Alex
PATCH v3 1/2 fs: fat: dynamically allocate memory for temporary buffer
Quoted 76 lines · Alexander Graf
Quoted 73 lines · Marek Vasut
Quoted 72 lines · Alexander Graf
Quoted 68 lines · Marek Vasut
Quoted 59 lines · Tien Fong
Quoted 58 lines · Alexander Graf
Quoted 39 lines · Tien Fong
Quoted 18 lines · Michal Simek
Hi Tom,
Quoted 14 lines · Tom Rini
Quoted 8 lines · tien.fong.chee at intel. com
From: Tien Fong Chee <[email protected]>
Drop the statically allocated
get_contents_vfatname_blockand dynamically allocate a buffer only if required. This saves 64KiB of memory.Quoted 6 lines
Quoted 3 lines
Quoted 2 lines
Signed-off-by Stefan Agner <[email protected]>Signed-off-by Tien Fong Chee <[email protected]>Applied to u-boot/master, thanks!
please remove this patch (better both of them because they were in
series)I think patch 2/2 should be safe, because no memory size is changed.
Basically, it just to release the allocated memory immediately when it's not required, so other can re-use it.Quoted 8 lines
because they are breaking at least ZynqMP SPL. It is also too late in cycle to create random fix.
You can't simply move 64KB from code to malloc without reflecting this
by changing MALLOC space size.Quoted 5 lines
Quoted 3 lines
Other boards with SPL fat could be also affected by this if they don't
allocate big malloc space.So, any suggestion to get the patch 1/2 accepted? inform all board maintainers to test it out?
You already received feedback that it does break ZynqMP, so the current
approach won't work.How about you create a new kconfig option that allows you to say whether
you want to use malloc or .bss for temporary data in the FAT driver. You
can then have an _SPL_ version of that kconfig and check for it withIS_ENABLED()which should automatically tell you the right answer depending on whether you're in an SPL build or not.Quoted 5 lines
Quoted 2 lines
Then you can set the SPL version to default malloc and the non-SPL version to default .bss.
Marek and Tom rini,
Are you guys okay with Alex's suggestion?
I'm not a big fan of adding more and more ifdeffery. Is there some other option ?
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Is RAM up already at this point? Maybe we could improve the SPL malloc mechanism to move allocations into DRAM once it's available.
Well, the FAT buffers waste some 64kiB of bss, so we can use that area for malloc instead, no ?
Yes, but that means you need to review every single board that uses FAT in SPL today and adjust its malloc region size.
That's quite likely ... I still think this patch is beneficial, it's much better to dynamically allocate the cluster size than have this 64kiB chunk of BSS carved out.
Signature · 2 lines
Best regards, Marek Vasut
PATCH v3 1/2 fs: fat: dynamically allocate memory for temporary buffer
Quoted 83 lines · Marek Vasut
Quoted 82 lines · Alexander Graf
Quoted 78 lines · Marek Vasut
Quoted 67 lines · Alexander Graf
Quoted 64 lines · Marek Vasut
Quoted 63 lines · Tien Fong
Quoted 58 lines · Alexander Graf
Quoted 39 lines · Tien Fong
Quoted 18 lines · Michal Simek
Hi Tom,
Quoted 14 lines · Tom Rini
Quoted 8 lines · tien.fong.chee at intel. com
From: Tien Fong Chee <[email protected]>
Drop the statically allocated
get_contents_vfatname_blockand dynamically allocate a buffer only if required. This saves 64KiB of memory.Quoted 6 lines
Quoted 3 lines
Quoted 2 lines
Signed-off-by Stefan Agner <[email protected]>Signed-off-by Tien Fong Chee <[email protected]>Applied to u-boot/master, thanks!
please remove this patch (better both of them because they were in
series)I think patch 2/2 should be safe, because no memory size is changed.
Basically, it just to release the allocated memory immediately when it's not required, so other can re-use it.Quoted 8 lines
because they are breaking at least ZynqMP SPL. It is also too late in cycle to create random fix.
You can't simply move 64KB from code to malloc without reflecting this
by changing MALLOC space size.Quoted 5 lines
Quoted 3 lines
Other boards with SPL fat could be also affected by this if they don't
allocate big malloc space.So, any suggestion to get the patch 1/2 accepted? inform all board maintainers to test it out?
You already received feedback that it does break ZynqMP, so the current
approach won't work.How about you create a new kconfig option that allows you to say whether
you want to use malloc or .bss for temporary data in the FAT driver. You
can then have an _SPL_ version of that kconfig and check for it withIS_ENABLED()which should automatically tell you the right answer depending on whether you're in an SPL build or not.Quoted 2 lines
Then you can set the SPL version to default malloc and the non-SPL version to default .bss.
Marek and Tom rini,
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Are you guys okay with Alex's suggestion?
I'm not a big fan of adding more and more ifdeffery. Is there some other option ?
Is RAM up already at this point? Maybe we could improve the SPL malloc mechanism to move allocations into DRAM once it's available.
Well, the FAT buffers waste some 64kiB of bss, so we can use that area for malloc instead, no ?
Quoted 7 lines
Quoted 2 lines
Yes, but that means you need to review every single board that uses FAT in SPL today and adjust its malloc region size.
That's quite likely ... I still think this patch is beneficial, it's much better to dynamically allocate the cluster size than have this 64kiB chunk of BSS carved out.
ok. I have played with it a little bit and the patch exposed different problem with one of my out of tree patch I am working on.
Anyway that's being said I still think that patches like this shouldn't come to the tree at this stage because it requires checking on other boards. IIRC similar patch was around in past and there was also any issue with it.
Tom: up2you if you want to keep it in the tree or not.
Thanks,
Michal
Signature · 5 lines
Michal Simek, Ing. (M.Eng), OpenPGP -> KeyID: FE3D1F91 w: www.monstr.eu p: +42-0-721842854 Maintainer of Linux kernel - Xilinx Microblaze Maintainer of Linux kernel - Xilinx Zynq ARM and ZynqMP ARM64 SoCs U-Boot custodian - Xilinx Microblaze/Zynq/ZynqMP/Versal SoCs
PATCH v3 1/2 fs: fat: dynamically allocate memory for temporary buffer
Quoted 95 lines · Michal Simek
Quoted 83 lines · Marek Vasut
Quoted 82 lines · Alexander Graf
Quoted 78 lines · Marek Vasut
Quoted 67 lines · Alexander Graf
Quoted 64 lines · Marek Vasut
Quoted 63 lines · Tien Fong
Quoted 58 lines · Alexander Graf
Quoted 39 lines · Tien Fong
Quoted 18 lines · Michal Simek
Hi Tom,
Quoted 14 lines · Tom Rini
Quoted 8 lines · tien.fong.chee at intel. com
From: Tien Fong Chee <[email protected]>
Drop the statically allocated
get_contents_vfatname_blockand dynamically allocate a buffer only if required. This saves 64KiB of memory.Quoted 6 lines
Quoted 3 lines
Quoted 2 lines
Signed-off-by Stefan Agner <[email protected]>Signed-off-by Tien Fong Chee <[email protected]>Applied to u-boot/master, thanks!
please remove this patch (better both of them because they were in
series)I think patch 2/2 should be safe, because no memory size is changed.
Basically, it just to release the allocated memory immediately when it's not required, so other can re-use it.Quoted 8 lines
because they are breaking at least ZynqMP SPL. It is also too late in cycle to create random fix.
You can't simply move 64KB from code to malloc without reflecting this
by changing MALLOC space size.Quoted 5 lines
Quoted 3 lines
Other boards with SPL fat could be also affected by this if they don't
allocate big malloc space.So, any suggestion to get the patch 1/2 accepted? inform all board maintainers to test it out?
You already received feedback that it does break ZynqMP, so the current
approach won't work.How about you create a new kconfig option that allows you to say whether
you want to use malloc or .bss for temporary data in the FAT driver. You
can then have an _SPL_ version of that kconfig and check for it withIS_ENABLED()which should automatically tell you the right answer depending on whether you're in an SPL build or not.Quoted 2 lines
Then you can set the SPL version to default malloc and the non-SPL version to default .bss.
Marek and Tom rini,
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Are you guys okay with Alex's suggestion?
I'm not a big fan of adding more and more ifdeffery. Is there some other option ?
Is RAM up already at this point? Maybe we could improve the SPL malloc mechanism to move allocations into DRAM once it's available.
Well, the FAT buffers waste some 64kiB of bss, so we can use that area for malloc instead, no ?
Quoted 7 lines
Quoted 2 lines
Yes, but that means you need to review every single board that uses FAT in SPL today and adjust its malloc region size.
That's quite likely ... I still think this patch is beneficial, it's much better to dynamically allocate the cluster size than have this 64kiB chunk of BSS carved out.
ok. I have played with it a little bit and the patch exposed different problem with one of my out of tree patch I am working on.
Quoted 6 lines
Anyway that's being said I still think that patches like this shouldn't come to the tree at this stage because it requires checking on other boards. IIRC similar patch was around in past and there was also any issue with it.
Tom: up2you if you want to keep it in the tree or not.
This shouldn't have come in after RC2, so revert and let's fix it for next release.
Signature · 2 lines
Best regards, Marek Vasut
PATCH v3 1/2 fs: fat: dynamically allocate memory for temporary buffer
Quoted 80 lines · Marek Vasut
Quoted 78 lines · Michal Simek
Quoted 77 lines · Marek Vasut
Quoted 76 lines · Alexander Graf
Quoted 73 lines · Marek Vasut
Quoted 72 lines · Alexander Graf
Quoted 68 lines · Marek Vasut
Quoted 59 lines · Tien Fong
Quoted 58 lines · Alexander Graf
Quoted 39 lines · Tien Fong
Quoted 18 lines · Michal Simek
Hi Tom,
Quoted 14 lines · Tom Rini
Quoted 8 lines · tien.fong.chee at intel. com
From: Tien Fong Chee <[email protected]>
Drop the statically allocated get_contents_vfatname_block and
dynamically allocate a buffer only if required. This saves
64KiB of memory.Quoted 6 lines
Quoted 3 lines
Quoted 2 lines
Signed-off-by: Stefan Agner <[email protected]>
Signed-off-by: Tien Fong Chee <[email protected]>Applied to u-boot/master, thanks!
please remove this patch (better both of them because they were in
series)I think patch 2/2 should be safe, because no memory size is changed.
Basically, it just to release the allocated memory immediately when it's not required, so other can re-use it.Quoted 8 lines
because they are breaking at least ZynqMP SPL. It is also too late in cycle to create random fix.
You can't simply move 64KB from code to malloc without reflecting this
by changing MALLOC space size.Quoted 5 lines
Quoted 3 lines
Other boards with SPL fat could be also affected by this if they don't
allocate big malloc space.So, any suggestion to get the patch 1/2 accepted? inform all board maintainers to test it out?
You already received feedback that it does break ZynqMP, so the current
approach won't work.How about you create a new kconfig option that allows you to say whether
you want to use malloc or .bss for temporary data in the FAT driver. You
can then have an _SPL_ version of that kconfig and check for it withIS_ENABLED()which should automatically tell you the right answer depending on whether you're in an SPL build or not.Quoted 5 lines
Quoted 2 lines
Then you can set the SPL version to default malloc and the non-SPL version to default .bss.
Marek and Tom rini,
Are you guys okay with Alex's suggestion?
I'm not a big fan of adding more and more ifdeffery. Is there some other option ?
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Is RAM up already at this point? Maybe we could improve the SPL malloc mechanism to move allocations into DRAM once it's available.
Well, the FAT buffers waste some 64kiB of bss, so we can use that area for malloc instead, no ?
Yes, but that means you need to review every single board that uses FAT in SPL today and adjust its malloc region size.
This happened to me for a different board where the SPI-NOR flash driver was updated, but it required me to increase my malloc size. I don't think it would have been appropriate for me to ask the maintainer to reject a patch when where was a simple solution, and the author helped me identify how to make the updated infrastructure work.
I think the benefits outweigh the cons if (and only if) the
solution is only to increase the malloc size. Many of us have very restricted SPL space.
Quoted 26 lines
Quoted 9 lines
Quoted 5 lines
That's quite likely ... I still think this patch is beneficial, it's much better to dynamically allocate the cluster size than have this 64kiB chunk of BSS carved out.
ok. I have played with it a little bit and the patch exposed different problem with one of my out of tree patch I am working on.
Quoted 6 lines
Anyway that's being said I still think that patches like this shouldn't come to the tree at this stage because it requires checking on other boards. IIRC similar patch was around in past and there was also any issue with it.
Tom: up2you if you want to keep it in the tree or not.
This shouldn't have come in after RC2, so revert and let's fix it for next release.
--
Best regards,
Marek Vasut
_______________________________________________
U-Boot mailing list
U-Boot at lists.denx.de
https://lists.denx.de/listinfo/u-boot
PATCH v3 1/2 fs: fat: dynamically allocate memory for temporary buffer
In reply to Michal SimekQuoted 20 lines · Michal Simek
Hi Tom,
Quoted 4 lines · Tom Rini
Quoted 2 lines · tien.fong.chee at intel.com
From: Tien Fong Chee <[email protected]>
Quoted 8 lines
Quoted 6 lines
Drop the statically allocated
get_contents_vfatname_blockand dynamically allocate a buffer only if required. This saves 64KiB of memory.Signed-off-by Stefan Agner <[email protected]>Signed-off-by Tien Fong Chee <[email protected]>Applied to u-boot/master, thanks!
please remove this patch (better both of them because they were in series) because they are breaking at least ZynqMP SPL. It is also too late in cycle to create random fix.
Quoted 5 lines
You can't simply move 64KB from code to malloc without reflecting this by changing MALLOC space size.
Other boards with SPL fat could be also affected by this if they don't allocate big malloc space.
I see from later in on the thread your specific problem is elsewhere. But to address the root question, we have fairly large malloc requirements in SPL when FAT is involved as there's a lot of other mallocs going on there. It's indeed not impossible there's a board that was on-edge of it's 1MB pool, and now goes over, but that seems unlikely. Thanks all!
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: not available URL: <http://lists.denx.de/pipermail/u-boot/attachments/20190221/4a914ae2/attachment.sig>
Chee
PATCH v3 1/2 fs: fat: dynamically allocate memory for temporary buffer
Quoted 31 lines · Tom Rini
Quoted 23 lines · Michal Simek
Hi Tom,
Quoted 7 lines · Tom Rini
Quoted 3 lines · tien.fong.chee at intel.co m
From: Tien Fong Chee <[email protected]>
Quoted 7 lines
Quoted 6 lines
Drop the statically allocated
get_contents_vfatname_blockand dynamically allocate a buffer only if required. This saves 64KiB of memory.Signed-off-by Stefan Agner <[email protected]>Signed-off-by Tien Fong Chee <[email protected]>Applied to u-boot/master, thanks!
please remove this patch (better both of them because they were in series) because they are breaking at least ZynqMP SPL. It is also too
late in cycle to create random fix.Quoted 7 lines
You can't simply move 64KB from code to malloc without reflecting this
by changing MALLOC space size.Other boards with SPL fat could be also affected by this if they don't
allocate big malloc space.
Quoted 7 lines
I see from later in on the thread your specific problem is elsewhere. But to address the root question, we have fairly large malloc requirements in SPL when FAT is involved as there's a lot of other mallocs going on there. It's indeed not impossible there's a board that
was on-edge of it's 1MB pool, and now goes over, but that seems unlikely. Thanks all!
I'm curious what's the actual problems you found? running out of memory, no?
Increasing Malloc space size is definitely required for The patch[1/2], but patch[2/2] would maximize the re-usable of memory, so in other words, no change on Malloc space size required. Both patches would share the same memory.
You might need to take a look these patches, they are part of vfat optimization.
https://patchwork.ozlabs.org/patch/1029679/
https://patchwork.ozlabs.org/patch/1029681/
https://patchwork.ozlabs.org/patch/1029682/
https://patchwork.ozlabs.org/patch/1029683/
Quoted 1 line
Thanks,
TF
PATCH v3 1/2 fs: fat: dynamically allocate memory for temporary buffer
Quoted 39 lines · Tien Fong
Quoted 31 lines · Tom Rini
Quoted 23 lines · Michal Simek
Hi Tom,
Quoted 7 lines · Tom Rini
Quoted 3 lines · tien.fong.chee at intel.co m
From: Tien Fong Chee <[email protected]>
Quoted 7 lines
Quoted 6 lines
Drop the statically allocated
get_contents_vfatname_blockand dynamically allocate a buffer only if required. This saves 64KiB of memory.Signed-off-by Stefan Agner <[email protected]>Signed-off-by Tien Fong Chee <[email protected]>Applied to u-boot/master, thanks!
please remove this patch (better both of them because they were in series) because they are breaking at least ZynqMP SPL. It is also too
late in cycle to create random fix.Quoted 7 lines
You can't simply move 64KB from code to malloc without reflecting this
by changing MALLOC space size.Other boards with SPL fat could be also affected by this if they don't
allocate big malloc space.Quoted 7 lines
I see from later in on the thread your specific problem is elsewhere. But to address the root question, we have fairly large malloc requirements in SPL when FAT is involved as there's a lot of other mallocs going on there. It's indeed not impossible there's a board that
was on-edge of it's 1MB pool, and now goes over, but that seems unlikely. Thanks all!
Quoted 2 lines
I'm curious what's the actual problems you found? running out of memory, no?
TBH I don't know now. I tried to debug that. I want to support option to use DT from fixed location to support qemu DT generation which we have enabled for next generation. And I use 0x1000 location which is in conflict with bss section. That's why I have moved BSS section out of this space to make sure that I am not breaking dtb out there. And it is working quite well but when this patch is applied DTB is being corrupted by SPL(don't know which code) and on reboot dtb magic is correct but content is broken.
I don't know the first address and I need to find some time to run this on Qemu and see activities in that area to find out which code is breaking this.
Anyway I don't have a time for debugging this more now but bisect pointed to this patch but as I said it just exposed likely problem somewhere else.
Thanks,
Michal
Signature · 5 lines
Michal Simek, Ing. (M.Eng), OpenPGP -> KeyID: FE3D1F91 w: www.monstr.eu p: +42-0-721842854 Maintainer of Linux kernel - Xilinx Microblaze Maintainer of Linux kernel - Xilinx Zynq ARM and ZynqMP ARM64 SoCs U-Boot custodian - Xilinx Microblaze/Zynq/ZynqMP/Versal SoCs
Chee
PATCH v3 1/2 fs: fat: dynamically allocate memory for temporary buffer
Quoted 66 lines · Michal Simek
Quoted 53 lines · Tien Fong
Quoted 41 lines · Tom Rini
Quoted 35 lines · Michal Simek
Hi Tom,
Quoted 10 lines · Tom Rini
Quoted 4 lines · tien.fong.chee at inte l.co m
From: Tien Fong Chee <[email protected]>
Quoted 8 lines
Quoted 7 lines
Drop the statically allocated
get_contents_vfatname_blockand
dynamically allocate a buffer only if required. This saves 64KiB of memory.Signed-off-by Stefan Agner <[email protected]>Signed-off-by Tien Fong Chee <[email protected]>Applied to u-boot/master, thanks!
please remove this patch (better both of them because they were in
series) because they are breaking at least ZynqMP SPL. It is also
too
late in cycle to create random fix.You can't simply move 64KB from code to malloc without reflecting
this
by changing MALLOC space size.Quoted 4 lines
Other boards with SPL fat could be also affected by this if they
don't
allocate big malloc space.Quoted 10 lines
I see from later in on the thread your specific problem is elsewhere.
But to address the root question, we have fairly large malloc requirements in SPL when FAT is involved as there's a lot of other
mallocs going on there. It's indeed not impossible there's a board
that
was on-edge of it's 1MB pool, and now goes over, but that seems unlikely. Thanks all!Quoted 2 lines
I'm curious what's the actual problems you found? running out of memory, no?
TBH I don't know now. I tried to debug that. I want to support option to
use DT from fixed location to support qemu DT generation which we have
enabled for next generation. And I use0x1000location which is in conflict with bss section. That's why I have moved BSS section out of this space to make sure that I am not breaking dtb out there. And it is working quite well but when this patch is applied DTB is being
corrupted by SPL(don't know which code) and on reboot dtb magic is
Quoted 9 lines
correct but content is broken.
I don't know the first address and I need to find some time to run this
on Qemu and see activities in that area to find out which code is breaking this.
Anyway I don't have a time for debugging this more now but bisect pointed to this patch but as I said it just exposed likely problem somewhere else.
Okay. You might need to consider Tom's suggestion, 1MB is too large for SPL malloc pool, some other places might overflow and corruption.