fs/squashfs: fix reading of fragmented files
PATCH 1/1 fs/squashfs: fix reading of fragmented files
The fragmented files were not correctly read because of two issues:
- The squashfs_file_info struct has a field named 'comp', which tells if the file's fragment is compressed or not. This field was always set to 'true' in sqfs_get_regfile_info and sqfs_get_lregfile_info. It should actually take sqfs_frag_lookup's return value. This patch addresses these two assignments.
- In sqfs_read, the fragments (compressed or not) were copied to the output buffer through a for loop which was reading data at the wrong offset. Replace these loops by equivalent calls to memcpy, with the right parameters.
I tested this patch by comparing the MD5 checksum of a few fragmented files with the respective md5sum output in sandbox, considering both compressed and uncompressed fragments.
--- fs/squashfs/sqfs.c | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-)
fs/squashfs/sqfs.c +6 -10
@@ -1253,7 +1253,7 @@ static int sqfs_get_regfile_info(struct squashfs_reg_inode *reg, fentry);if (ret <0 )return -EINVAL ; - finfo->comp = true; + finfo->comp = ret;if (fentry->size <1 || fentry->start ==0x7FFFFFFF )return -EINVAL ; }else {
@@ -1291,7 +1291,7 @@ static int sqfs_get_lregfile_info(struct squashfs_lreg_inode *lreg, fentry);if (ret <0 )return -EINVAL ; - finfo->comp = true; + finfo->comp = ret;if (fentry->size <1 || fentry->start ==0x7FFFFFFF )return -EINVAL ; }else {
@@ -1547,20 +1547,16 @@ int sqfs_read(const char *filename, void *buf, loff_t offset, loff_t len, goto out; } -for (j = *actread; j < finfo.size; j++) { -memcpy (buf + j, &fragment_block[finfo.offset + j],1 ); - (*actread)++; - } +memcpy (buf + *actread, &fragment_block[finfo.offset], finfo.size - *actread); + *actread = finfo.size;free (fragment_block); }else if (finfo.frag && !finfo.comp) { fragment_block = (void *)fragment + table_offset; -for (j = *actread; j < finfo.size; j++) { -memcpy (buf + j, &fragment_block[finfo.offset + j],1 ); - (*actread)++; - } +memcpy (buf + *actread, &fragment_block[finfo.offset], finfo.size - *actread); + *actread = finfo.size; } out:
Signature · 1 line
2.25.1
PATCH 1/1 fs/squashfs: fix reading of fragmented files
Le 17/05/2021 ? 23:20, Joao Marcos Costa a ?crit?:
Quoted 13 lines
The fragmented files were not correctly read because of two issues:
- The
squashfs_file_infostruct has a field named 'comp', which tells if the file's fragment is compressed or not. This field was always set to 'true' insqfs_get_regfile_infoandsqfs_get_lregfile_info. It should actually take sqfs_frag_lookup's return value. This patch addresses these two assignments.- In
sqfs_read, the fragments (compressed or not) were copied to the output buffer through a for loop which was reading data at the wrong offset. Replace these loops by equivalent calls to memcpy, with the right parameters.
Quoted 5 lines
I tested this patch by comparing the MD5 checksum of a few fragmented files with the respective md5sum output in sandbox, considering both compressed and uncompressed fragments.
Signed-off-by Joao Marcos Costa <[email protected]>
Tested it on real hardware, with mksquashfs -always-use-fragments, loading a kernel (fit) from squashfs. It wasn't working before the patch, and works after.
Thanks !
Quoted 52 lines
--- fs/squashfs/sqfs.c | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-)fs/squashfs/sqfs.c +6
-10@@ -1253,7 +1253,7 @@ static int sqfs_get_regfile_info(struct squashfs_reg_inode *reg, fentry);if (ret <0 )return -EINVAL ; - finfo->comp = true; + finfo->comp = ret;if (fentry->size <1 || fentry->start ==0x7FFFFFFF )return -EINVAL ; }else {@@ -1291,7 +1291,7 @@ static int sqfs_get_lregfile_info(struct squashfs_lreg_inode *lreg, fentry);if (ret <0 )return -EINVAL ; - finfo->comp = true; + finfo->comp = ret;if (fentry->size <1 || fentry->start ==0x7FFFFFFF )return -EINVAL ; }else {@@ -1547,20 +1547,16 @@ int sqfs_read(const char *filename, void *buf, loff_t offset, loff_t len, goto out; } -for (j = *actread; j < finfo.size; j++) { -memcpy (buf + j, &fragment_block[finfo.offset + j],1 ); - (*actread)++; - } +memcpy (buf + *actread, &fragment_block[finfo.offset], finfo.size - *actread); + *actread = finfo.size;free (fragment_block); }else if (finfo.frag && !finfo.comp) { fragment_block = (void *)fragment + table_offset; -for (j = *actread; j < finfo.size; j++) { -memcpy (buf + j, &fragment_block[finfo.offset + j],1 ); - (*actread)++; - } +memcpy (buf + *actread, &fragment_block[finfo.offset], finfo.size - *actread); + *actread = finfo.size; } out:
PATCH 1/1 fs/squashfs: fix reading of fragmented files
Hello,
Em qui., 20 de mai. de 2021 ?s 06:54, Richard Genoud < [email protected]> escreveu:
Quoted 18 lines
Le 17/05/2021 ? 23:20, Joao Marcos Costa a ?crit :
Quoted 17 lines
The fragmented files were not correctly read because of two issues:
- The
squashfs_file_infostruct has a field named 'comp', which tells if the file's fragment is compressed or not. This field was always set to 'true' insqfs_get_regfile_infoandsqfs_get_lregfile_info. It should actually take sqfs_frag_lookup's return value. This patch addresses these two assignments.- In
sqfs_read, the fragments (compressed or not) were copied to the output buffer through a for loop which was reading data at the wrong offset. Replace these loops by equivalent calls to memcpy, with the right parameters.I tested this patch by comparing the MD5 checksum of a few fragmented files with the respective md5sum output in sandbox, considering both compressed and uncompressed fragments.
Quoted 9 lines
Quoted 1 line
Signed-off-by Joao Marcos Costa <[email protected]>Tested-by Richard Genoud <[email protected]>Tested it on real hardware, with mksquashfs -always-use-fragments, loading a kernel (fit) from squashfs.
It wasn't working before the patch, and works after.Thanks !
Great! Thanks for testing!
Signature · 5 lines
Atenciosamente, Jo?o Marcos Costa www.linkedin.com/in/jmarcoscosta/ https://github.com/jmarcoscosta
Hi Joao,
Joao Marcos Costa <[email protected]> wrote on Mon, 17 May 2021 18:20:38 -0300:
Quoted 8 lines
The fragmented files were not correctly read because of two issues:
- The
squashfs_file_infostruct has a field named 'comp', which tells if the file's fragment is compressed or not. This field was always set to 'true' insqfs_get_regfile_infoandsqfs_get_lregfile_info. It should actually take sqfs_frag_lookup's return value. This patch addresses these two assignments.
Quoted 4 lines
- In
sqfs_read, the fragments (compressed or not) were copied to the output buffer through a for loop which was reading data at the wrong offset. Replace these loops by equivalent calls to memcpy, with the right parameters.
Good idea to get rid of these memcpy of 1 byte :)
Quoted 5 lines
I tested this patch by comparing the MD5 checksum of a few fragmented files with the respective md5sum output in sandbox, considering both compressed and uncompressed fragments.
Signed-off-by Joao Marcos Costa <[email protected]>
But next time, when you fix two issues (even if they fix the same feature) please provide two patches ;)
Thanks,
Miquèl
Hello, Miquèl
Em qua., 26 de mai. de 2021 às 04:52, Miquel Raynal < [email protected]> escreveu:
Quoted 26 lines
Hi Joao,
Joao Marcos Costa <[email protected]> wrote on Mon, 17 May 2021 18:20:38 -0300:
Quoted 8 lines
The fragmented files were not correctly read because of two issues:
- The
squashfs_file_infostruct has a field named 'comp', which tells if the file's fragment is compressed or not. This field was always set to 'true' insqfs_get_regfile_infoandsqfs_get_lregfile_info. It should actually take sqfs_frag_lookup's return value. This patch addresses these two assignments.Quoted 4 lines
- In
sqfs_read, the fragments (compressed or not) were copied to the output buffer through a for loop which was reading data at the wrong offset. Replace these loops by equivalent calls to memcpy, with the right parameters.Good idea to get rid of these memcpy of 1 byte :)
Quoted 5 lines
I tested this patch by comparing the MD5 checksum of a few fragmented files with the respective md5sum output in sandbox, considering both compressed and uncompressed fragments.
Signed-off-by Joao Marcos Costa <[email protected]>
Quoted 8 lines
Reviewed-by Miquel Raynal <[email protected]>But next time, when you fix two issues (even if they fix the same feature) please provide two patches ;)
Thanks,
Miquèl
Will do! Thanks for the review!
Best regards,
Joao
Signature · 5 lines
Atenciosamente, João Marcos Costa www.linkedin.com/in/jmarcoscosta/ https://github.com/jmarcoscosta
Hello, everyone
Em qua., 26 de mai. de 2021 às 09:35, João Marcos Costa < [email protected]> escreveu:
Quoted 41 lines
Hello, Miquèl
Em qua., 26 de mai. de 2021 às 04:52, Miquel Raynal < [email protected]> escreveu:
Quoted 34 lines
Hi Joao,
Joao Marcos Costa <[email protected]> wrote on Mon, 17 May 2021 18:20:38 -0300:
Quoted 8 lines
The fragmented files were not correctly read because of two issues:
- The
squashfs_file_infostruct has a field named 'comp', which tells if the file's fragment is compressed or not. This field was always set to 'true' insqfs_get_regfile_infoandsqfs_get_lregfile_info. It should actually take sqfs_frag_lookup's return value. This patch addresses these two assignments.Quoted 4 lines
- In
sqfs_read, the fragments (compressed or not) were copied to the output buffer through a for loop which was reading data at the wrong offset. Replace these loops by equivalent calls to memcpy, with the right parameters.Good idea to get rid of these memcpy of 1 byte :)
Quoted 5 lines
I tested this patch by comparing the MD5 checksum of a few fragmented files with the respective md5sum output in sandbox, considering both compressed and uncompressed fragments.
Signed-off-by Joao Marcos Costa <[email protected]>Reviewed-by Miquel Raynal <[email protected]>But next time, when you fix two issues (even if they fix the same feature) please provide two patches ;)
Thanks,
Miquèl
Any updates on this patch review?
Thanks!
Signature · 5 lines
Atenciosamente, João Marcos Costa www.linkedin.com/in/jmarcoscosta/ https://github.com/jmarcoscosta
Quoted 41 lines · João Marcos Costa
Hello, everyone
Em qua., 26 de mai. de 2021 às 09:35, João Marcos Costa < [email protected]> escreveu:
Quoted 36 lines
Hello, Miquèl
Em qua., 26 de mai. de 2021 às 04:52, Miquel Raynal < [email protected]> escreveu:
Quoted 31 lines
Hi Joao,
Joao Marcos Costa <[email protected]> wrote on Mon, 17 May 2021 18:20:38 -0300:
Quoted 8 lines
The fragmented files were not correctly read because of two issues:
- The
squashfs_file_infostruct has a field named 'comp', which tells if the file's fragment is compressed or not. This field was always set to 'true' insqfs_get_regfile_infoandsqfs_get_lregfile_info. It should actually take sqfs_frag_lookup's return value. This patch addresses these two assignments.Quoted 4 lines
- In
sqfs_read, the fragments (compressed or not) were copied to the output buffer through a for loop which was reading data at the wrong offset. Replace these loops by equivalent calls to memcpy, with the right parameters.Good idea to get rid of these memcpy of 1 byte :)
Quoted 5 lines
I tested this patch by comparing the MD5 checksum of a few fragmented files with the respective md5sum output in sandbox, considering both compressed and uncompressed fragments.
Signed-off-by Joao Marcos Costa <[email protected]>Reviewed-by Miquel Raynal <[email protected]>But next time, when you fix two issues (even if they fix the same feature) please provide two patches ;)
Quoted 6 lines
Quoted 5 lines
Quoted 3 lines
Thanks,
MiquèlAny updates on this patch review?
Seems fine, but I'm also leaning on grabbing all of the squashfs patches for -next at this point, unless people have strong feelings about it being safe at this point for master, thanks.
Signature · 1 line
Tom
Hello,
In fact, I really think this patch should be applied to master as soon as possible, since the actual unsafety comes from the current code, which may read past the fragment_block buffer size. Besides, the patch series I sent to rewrite the test suite needs this fix, and the current test suite is error-prone, as it was already reported.
Best regards,
Em qua., 9 de jun. de 2021 às 14:40, Tom Rini <[email protected]> escreveu:
Quoted 56 lines · João Marcos Costa
Quoted 41 lines
Hello, everyone
Em qua., 26 de mai. de 2021 às 09:35, João Marcos Costa < [email protected]> escreveu:
Quoted 36 lines
Hello, Miquèl
Em qua., 26 de mai. de 2021 às 04:52, Miquel Raynal < [email protected]> escreveu:
Quoted 31 lines
Hi Joao,
Joao Marcos Costa <[email protected]> wrote on Mon, 17 May 2021 18:20:38 -0300:
Quoted 8 lines
The fragmented files were not correctly read because of two issues:
- The
squashfs_file_infostruct has a field named 'comp', which tells if the file's fragment is compressed or not. This field was always set to 'true' insqfs_get_regfile_infoandsqfs_get_lregfile_info. It should actually take sqfs_frag_lookup's return value. This patch addresses these two assignments.Quoted 4 lines
- In
sqfs_read, the fragments (compressed or not) were copied to the output buffer through a for loop which was reading data at the wrong offset. Replace these loops by equivalent calls to memcpy, with the right parameters.Good idea to get rid of these memcpy of 1 byte :)
Quoted 5 lines
I tested this patch by comparing the MD5 checksum of a few fragmented files with the respective md5sum output in sandbox, considering both compressed and uncompressed fragments.
Signed-off-by Joao Marcos Costa <[email protected]>Reviewed-by Miquel Raynal <[email protected]>But next time, when you fix two issues (even if they fix the same feature) please provide two patches ;)
Quoted 6 lines
Quoted 5 lines
Quoted 3 lines
Thanks,
MiquèlAny updates on this patch review?
Seems fine, but I'm also leaning on grabbing all of the squashfs patches for -next at this point, unless people have strong feelings about it being safe at this point for master, thanks.
--
Tom
Signature · 5 lines
Atenciosamente, João Marcos Costa www.linkedin.com/in/jmarcoscosta/ https://github.com/jmarcoscosta
By the way, I apologize for the top-posting on this previous email I sent.
Quoted 8 lines · João Marcos Costa
Hello,
In fact, I really think this patch should be applied to master as soon as possible, since the actual unsafety comes from the current code, which may read past the
fragment_blockbuffer size. Besides, the patch series I sent to rewrite the test suite needs this fix, and the current test suite is error-prone, as it was already reported.
OK. So, some local testing shows that we'll need to move the CI images up to a newer base in order to have mksquashfs new enough to support zstd. Today I guess the failure is just ignored (not great!). I'll take the bugfix now and we'll do the tests soon.
Signature · 1 line
Tom
Quoted 13 lines · Joao Marcos Costa
The fragmented files were not correctly read because of two issues:
- The
squashfs_file_infostruct has a field named 'comp', which tells if the file's fragment is compressed or not. This field was always set to 'true' insqfs_get_regfile_infoandsqfs_get_lregfile_info. It should actually take sqfs_frag_lookup's return value. This patch addresses these two assignments.- In
sqfs_read, the fragments (compressed or not) were copied to the output buffer through a for loop which was reading data at the wrong offset. Replace these loops by equivalent calls to memcpy, with the right parameters.
Quoted 7 lines
I tested this patch by comparing the MD5 checksum of a few fragmented files with the respective md5sum output in sandbox, considering both compressed and uncompressed fragments.
Signed-off-by Joao Marcos Costa <[email protected]>Tested-by Richard Genoud <[email protected]>Reviewed-by Miquel Raynal <[email protected]>
Applied to u-boot/master, thanks!
Signature · 1 line
Tom