Skip to contentSkip to conversation

fs/squashfs: fix reading of fragmented files

PATCH In review Joao Marcos Costa 4 participants 11 messages May 17, 2021 – Jun 10, 2021 SquashFS file system

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 fs/squashfs: fix reading of fragmented files

The fragmented files were not correctly read because of two issues:

- The squashfs_file_info struct has a field named 'comp', which tells if the file's fragment is compressed or not. This field was always set to 'true' in sqfs_get_regfile_info and sqfs_get_lregfile_info. It should actually take sqfs_frag_lookup's return value. This patch addresses these two assignments.

- In sqfs_read, the fragments (compressed or not) were copied to the output buffer through a for loop which was reading data at the wrong offset. Replace these loops by equivalent calls to memcpy, with the right parameters.

I tested this patch by comparing the MD5 checksum of a few fragmented files with the respective md5sum output in sandbox, considering both compressed and uncompressed fragments.

---
 fs/squashfs/sqfs.c | 16 ++++++----------
 1 file changed, 6 insertions(+), 10 deletions(-)
fs/squashfs/sqfs.c +6 -10
index 29805c3c6f..22ef4f2691 100644
@@ -1253,7 +1253,7 @@ static int sqfs_get_regfile_info(struct squashfs_reg_inode *reg,
 				       fentry);
 		if (ret < 0)
 			return -EINVAL;
-		finfo->comp = true;
+		finfo->comp = ret;
 		if (fentry->size < 1 || fentry->start == 0x7FFFFFFF)
 			return -EINVAL;
 	} else {
@@ -1291,7 +1291,7 @@ static int sqfs_get_lregfile_info(struct squashfs_lreg_inode *lreg,
 				       fentry);
 		if (ret < 0)
 			return -EINVAL;
-		finfo->comp = true;
+		finfo->comp = ret;
 		if (fentry->size < 1 || fentry->start == 0x7FFFFFFF)
 			return -EINVAL;
 	} else {
@@ -1547,20 +1547,16 @@ int sqfs_read(const char *filename, void *buf, loff_t offset, loff_t len,
 			goto out;
 		}
 
-		for (j = *actread; j < finfo.size; j++) {
-			memcpy(buf + j, &fragment_block[finfo.offset + j], 1);
-			(*actread)++;
-		}
+		memcpy(buf + *actread, &fragment_block[finfo.offset], finfo.size - *actread);
+		*actread = finfo.size;
 
 		free(fragment_block);
 
 	} else if (finfo.frag && !finfo.comp) {
 		fragment_block = (void *)fragment + table_offset;
 
-		for (j = *actread; j < finfo.size; j++) {
-			memcpy(buf + j, &fragment_block[finfo.offset + j], 1);
-			(*actread)++;
-		}
+		memcpy(buf + *actread, &fragment_block[finfo.offset], finfo.size - *actread);
+		*actread = finfo.size;
 	}
 
 out:
Signature · 1 line
2.25.1
#1 lore raw

PATCH 1/1 fs/squashfs: fix reading of fragmented files

Le 17/05/2021 ? 23:20, Joao Marcos Costa a ?crit?:

Quoted 13 lines

The fragmented files were not correctly read because of two issues:

- The squashfs_file_info struct has a field named 'comp', which tells if the file's fragment is compressed or not. This field was always set to 'true' in sqfs_get_regfile_info and sqfs_get_lregfile_info. It should actually take sqfs_frag_lookup's return value. This patch addresses these two assignments.

- In sqfs_read, the fragments (compressed or not) were copied to the output buffer through a for loop which was reading data at the wrong offset. Replace these loops by equivalent calls to memcpy, with the right parameters.

Quoted 5 lines

I tested this patch by comparing the MD5 checksum of a few fragmented files with the respective md5sum output in sandbox, considering both compressed and uncompressed fragments.

Tested it on real hardware, with mksquashfs -always-use-fragments, loading a kernel (fit) from squashfs. It wasn't working before the patch, and works after.

Thanks !

Quoted 52 lines
---
  fs/squashfs/sqfs.c | 16 ++++++----------
  1 file changed, 6 insertions(+), 10 deletions(-)
fs/squashfs/sqfs.c +6 -10
index 29805c3c6f..22ef4f2691 100644
@@ -1253,7 +1253,7 @@ static int sqfs_get_regfile_info(struct squashfs_reg_inode *reg,
  				       fentry);
  		if (ret < 0)
  			return -EINVAL;
-		finfo->comp = true;
+		finfo->comp = ret;
  		if (fentry->size < 1 || fentry->start == 0x7FFFFFFF)
  			return -EINVAL;
  	} else {
@@ -1291,7 +1291,7 @@ static int sqfs_get_lregfile_info(struct squashfs_lreg_inode *lreg,
  				       fentry);
  		if (ret < 0)
  			return -EINVAL;
-		finfo->comp = true;
+		finfo->comp = ret;
  		if (fentry->size < 1 || fentry->start == 0x7FFFFFFF)
  			return -EINVAL;
  	} else {
@@ -1547,20 +1547,16 @@ int sqfs_read(const char *filename, void *buf, loff_t offset, loff_t len,
  			goto out;
  		}
 
-		for (j = *actread; j < finfo.size; j++) {
-			memcpy(buf + j, &fragment_block[finfo.offset + j], 1);
-			(*actread)++;
-		}
+		memcpy(buf + *actread, &fragment_block[finfo.offset], finfo.size - *actread);
+		*actread = finfo.size;
 
  		free(fragment_block);
 
  	} else if (finfo.frag && !finfo.comp) {
  		fragment_block = (void *)fragment + table_offset;
 
-		for (j = *actread; j < finfo.size; j++) {
-			memcpy(buf + j, &fragment_block[finfo.offset + j], 1);
-			(*actread)++;
-		}
+		memcpy(buf + *actread, &fragment_block[finfo.offset], finfo.size - *actread);
+		*actread = finfo.size;
  	}
 
  out:
#2 lore raw

PATCH 1/1 fs/squashfs: fix reading of fragmented files

Hello,

Em qui., 20 de mai. de 2021 ?s 06:54, Richard Genoud < [email protected]> escreveu:

Quoted 18 lines

Le 17/05/2021 ? 23:20, Joao Marcos Costa a ?crit :

Quoted 17 lines

The fragmented files were not correctly read because of two issues:

- The squashfs_file_info struct has a field named 'comp', which tells if the file's fragment is compressed or not. This field was always set to 'true' in sqfs_get_regfile_info and sqfs_get_lregfile_info. It should actually take sqfs_frag_lookup's return value. This patch addresses these two assignments.

- In sqfs_read, the fragments (compressed or not) were copied to the output buffer through a for loop which was reading data at the wrong offset. Replace these loops by equivalent calls to memcpy, with the right parameters.

I tested this patch by comparing the MD5 checksum of a few fragmented files with the respective md5sum output in sandbox, considering both compressed and uncompressed fragments.

Quoted 9 lines
Quoted 1 line

Tested it on real hardware, with mksquashfs -always-use-fragments, loading a kernel (fit) from squashfs.
It wasn't working before the patch, and works after.

Thanks !

Great! Thanks for testing!

Signature · 5 lines
Atenciosamente,
Jo?o Marcos Costa

www.linkedin.com/in/jmarcoscosta/
https://github.com/jmarcoscosta
Reviewed-by on v1 1/1
#3 lore raw
In reply to Joao Marcos Costa

Hi Joao,

Joao Marcos Costa <[email protected]> wrote on Mon, 17 May 2021 18:20:38 -0300:

Quoted 8 lines

The fragmented files were not correctly read because of two issues:

- The squashfs_file_info struct has a field named 'comp', which tells if the file's fragment is compressed or not. This field was always set to 'true' in sqfs_get_regfile_info and sqfs_get_lregfile_info. It should actually take sqfs_frag_lookup's return value. This patch addresses these two assignments.

Quoted 4 lines

- In sqfs_read, the fragments (compressed or not) were copied to the output buffer through a for loop which was reading data at the wrong offset. Replace these loops by equivalent calls to memcpy, with the right parameters.

Good idea to get rid of these memcpy of 1 byte :)

Quoted 5 lines

I tested this patch by comparing the MD5 checksum of a few fragmented files with the respective md5sum output in sandbox, considering both compressed and uncompressed fragments.

But next time, when you fix two issues (even if they fix the same feature) please provide two patches ;)

Thanks,
Miquèl

#4 lore raw

Hello, Miquèl

Em qua., 26 de mai. de 2021 às 04:52, Miquel Raynal < [email protected]> escreveu:

Quoted 26 lines

Hi Joao,

Joao Marcos Costa <[email protected]> wrote on Mon, 17 May 2021 18:20:38 -0300:

Quoted 8 lines

The fragmented files were not correctly read because of two issues:

- The squashfs_file_info struct has a field named 'comp', which tells if the file's fragment is compressed or not. This field was always set to 'true' in sqfs_get_regfile_info and sqfs_get_lregfile_info. It should actually take sqfs_frag_lookup's return value. This patch addresses these two assignments.

Quoted 4 lines

- In sqfs_read, the fragments (compressed or not) were copied to the output buffer through a for loop which was reading data at the wrong offset. Replace these loops by equivalent calls to memcpy, with the right parameters.

Good idea to get rid of these memcpy of 1 byte :)

Quoted 5 lines

I tested this patch by comparing the MD5 checksum of a few fragmented files with the respective md5sum output in sandbox, considering both compressed and uncompressed fragments.

Quoted 8 lines

But next time, when you fix two issues (even if they fix the same feature) please provide two patches ;)

Thanks,
Miquèl

Will do! Thanks for the review!

Best regards,
Joao

Signature · 5 lines
Atenciosamente,
João Marcos Costa

www.linkedin.com/in/jmarcoscosta/
https://github.com/jmarcoscosta
#5 lore raw

Hello, everyone

Em qua., 26 de mai. de 2021 às 09:35, João Marcos Costa < [email protected]> escreveu:

Quoted 41 lines

Hello, Miquèl

Em qua., 26 de mai. de 2021 às 04:52, Miquel Raynal < [email protected]> escreveu:

Quoted 34 lines

Hi Joao,

Joao Marcos Costa <[email protected]> wrote on Mon, 17 May 2021 18:20:38 -0300:

Quoted 8 lines

The fragmented files were not correctly read because of two issues:

- The squashfs_file_info struct has a field named 'comp', which tells if the file's fragment is compressed or not. This field was always set to 'true' in sqfs_get_regfile_info and sqfs_get_lregfile_info. It should actually take sqfs_frag_lookup's return value. This patch addresses these two assignments.

Quoted 4 lines

- In sqfs_read, the fragments (compressed or not) were copied to the output buffer through a for loop which was reading data at the wrong offset. Replace these loops by equivalent calls to memcpy, with the right parameters.

Good idea to get rid of these memcpy of 1 byte :)

Quoted 5 lines

I tested this patch by comparing the MD5 checksum of a few fragmented files with the respective md5sum output in sandbox, considering both compressed and uncompressed fragments.

But next time, when you fix two issues (even if they fix the same feature) please provide two patches ;)

Thanks,
Miquèl

Any updates on this patch review?

Thanks!

Signature · 5 lines
Atenciosamente,
João Marcos Costa

www.linkedin.com/in/jmarcoscosta/
https://github.com/jmarcoscosta
#6 lore raw
Quoted 41 lines · João Marcos Costa

Hello, everyone

Em qua., 26 de mai. de 2021 às 09:35, João Marcos Costa < [email protected]> escreveu:

Quoted 36 lines

Hello, Miquèl

Em qua., 26 de mai. de 2021 às 04:52, Miquel Raynal < [email protected]> escreveu:

Quoted 31 lines

Hi Joao,

Joao Marcos Costa <[email protected]> wrote on Mon, 17 May 2021 18:20:38 -0300:

Quoted 8 lines

The fragmented files were not correctly read because of two issues:

- The squashfs_file_info struct has a field named 'comp', which tells if the file's fragment is compressed or not. This field was always set to 'true' in sqfs_get_regfile_info and sqfs_get_lregfile_info. It should actually take sqfs_frag_lookup's return value. This patch addresses these two assignments.

Quoted 4 lines

- In sqfs_read, the fragments (compressed or not) were copied to the output buffer through a for loop which was reading data at the wrong offset. Replace these loops by equivalent calls to memcpy, with the right parameters.

Good idea to get rid of these memcpy of 1 byte :)

Quoted 5 lines

I tested this patch by comparing the MD5 checksum of a few fragmented files with the respective md5sum output in sandbox, considering both compressed and uncompressed fragments.

But next time, when you fix two issues (even if they fix the same feature) please provide two patches ;)

Quoted 6 lines
Quoted 5 lines
Quoted 3 lines

Thanks,
Miquèl

Any updates on this patch review?

Seems fine, but I'm also leaning on grabbing all of the squashfs patches for -next at this point, unless people have strong feelings about it being safe at this point for master, thanks.

Signature · 1 line
Tom
#7 lore raw

Hello,

In fact, I really think this patch should be applied to master as soon as possible, since the actual unsafety comes from the current code, which may read past the fragment_block buffer size. Besides, the patch series I sent to rewrite the test suite needs this fix, and the current test suite is error-prone, as it was already reported.

Best regards,

Em qua., 9 de jun. de 2021 às 14:40, Tom Rini <[email protected]> escreveu:

Quoted 56 lines · João Marcos Costa
Quoted 41 lines

Hello, everyone

Em qua., 26 de mai. de 2021 às 09:35, João Marcos Costa < [email protected]> escreveu:

Quoted 36 lines

Hello, Miquèl

Em qua., 26 de mai. de 2021 às 04:52, Miquel Raynal < [email protected]> escreveu:

Quoted 31 lines

Hi Joao,

Joao Marcos Costa <[email protected]> wrote on Mon, 17 May 2021 18:20:38 -0300:

Quoted 8 lines

The fragmented files were not correctly read because of two issues:

- The squashfs_file_info struct has a field named 'comp', which tells if the file's fragment is compressed or not. This field was always set to 'true' in sqfs_get_regfile_info and sqfs_get_lregfile_info. It should actually take sqfs_frag_lookup's return value. This patch addresses these two assignments.

Quoted 4 lines

- In sqfs_read, the fragments (compressed or not) were copied to the output buffer through a for loop which was reading data at the wrong offset. Replace these loops by equivalent calls to memcpy, with the right parameters.

Good idea to get rid of these memcpy of 1 byte :)

Quoted 5 lines

I tested this patch by comparing the MD5 checksum of a few fragmented files with the respective md5sum output in sandbox, considering both compressed and uncompressed fragments.

But next time, when you fix two issues (even if they fix the same feature) please provide two patches ;)

Quoted 6 lines
Quoted 5 lines
Quoted 3 lines

Thanks,
Miquèl

Any updates on this patch review?

Seems fine, but I'm also leaning on grabbing all of the squashfs patches for -next at this point, unless people have strong feelings about it being safe at this point for master, thanks.

--
Tom

Signature · 5 lines
Atenciosamente,
João Marcos Costa

www.linkedin.com/in/jmarcoscosta/
https://github.com/jmarcoscosta
#8 lore raw

By the way, I apologize for the top-posting on this previous email I sent.

#9 lore raw
In reply to João Marcos Costa
Quoted 8 lines · João Marcos Costa

Hello,

In fact, I really think this patch should be applied to master as soon as possible, since the actual unsafety comes from the current code, which may read past the fragment_block buffer size. Besides, the patch series I sent to rewrite the test suite needs this fix, and the current test suite is error-prone, as it was already reported.

OK. So, some local testing shows that we'll need to move the CI images up to a newer base in order to have mksquashfs new enough to support zstd. Today I guess the failure is just ignored (not great!). I'll take the bugfix now and we'll do the tests soon.

Signature · 1 line
Tom
#10 lore raw
In reply to Joao Marcos Costa
Quoted 13 lines · Joao Marcos Costa

The fragmented files were not correctly read because of two issues:

- The squashfs_file_info struct has a field named 'comp', which tells if the file's fragment is compressed or not. This field was always set to 'true' in sqfs_get_regfile_info and sqfs_get_lregfile_info. It should actually take sqfs_frag_lookup's return value. This patch addresses these two assignments.

- In sqfs_read, the fragments (compressed or not) were copied to the output buffer through a for loop which was reading data at the wrong offset. Replace these loops by equivalent calls to memcpy, with the right parameters.

Quoted 7 lines

I tested this patch by comparing the MD5 checksum of a few fragmented files with the respective md5sum output in sandbox, considering both compressed and uncompressed fragments.

Applied to u-boot/master, thanks!

Signature · 1 line
Tom