Skip to contentSkip to conversation

erofs: fix wrong primary bvec selection on deduplicated extents

PATCH In review Gao Xiang 4 participants 5 messages Jul 19, 2023 – Jul 30, 2023 EROFS file system

lore
Share & tools
lore

Gao Xiang

author
#0 lore raw

PATCH 1/1 erofs: fix wrong primary bvec selection on deduplicated extents

When handling deduplicated compressed data, there can be multiple decompressed extents pointing to the same compressed data in one shot.

In such cases, the bvecs which belong to the longest extent will be selected as the primary bvecs for real decompressors to decode and the other duplicated bvecs will be directly copied from the primary bvecs.

Previously, only relative offsets of the longest extent was checked to decompress the primary bvecs. On rare occasions, it can be incorrect if there are several extents with the same start relative offset. As a result, some short bvecs could be selected for decompression and then cause data corruption.

For example, as Shijie Sun reported off-list, considering the following extents of a file:

 117:   903345..  915250 |   11905 :     385024..    389120 |    4096
...
 119:   919729..  930323 |   10594 :     385024..    389120 |    4096
...
 124:   968881..  980786 |   11905 :     385024..    389120 |    4096

The start relative offset is the same: 2225, but extent 119 (919729.. 930323) is shorter than the others.

Let's restrict the bvec length in addition to the start offset if bvecs are not full.

Fixes 5c2a64252c5d ("erofs: introduce partial-referenced pclusters")
---
 fs/erofs/zdata.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
fs/erofs/zdata.c +4 -3
index b69d89a11dd0..de4f12152b62 100644
@@ -1144,10 +1144,11 @@ static void z_erofs_do_decompressed_bvec(struct z_erofs_decompress_backend *be,
 					 struct z_erofs_bvec *bvec)
 {
 	struct z_erofs_bvec_item *item;
+	unsigned int pgnr;
 
-	if (!((bvec->offset + be->pcl->pageofs_out) & ~PAGE_MASK)) {
-		unsigned int pgnr;
-
+	if (!((bvec->offset + be->pcl->pageofs_out) & ~PAGE_MASK) &&
+	    (bvec->end == PAGE_SIZE ||
+	     bvec->offset + bvec->end == be->pcl->length)) {
 		pgnr = (bvec->offset + be->pcl->pageofs_out) >> PAGE_SHIFT;
 		DBG_BUGON(pgnr >= be->nr_pages);
 		if (!be->decompressed_pages[pgnr]) {
Signature · 1 line
2.24.4
#1 lore raw

When handling deduplicated compressed data, there can be multiple decompressed extents pointing to the same compressed data in one shot.

In such cases, the bvecs which belong to the longest extent will be selected as the primary bvecs for real decompressors to decode and the other duplicated bvecs will be directly copied from the primary bvecs.

Previously, only relative offsets of the longest extent was checked to decompress the primary bvecs. On rare occasions, it can be incorrect if there are several extents with the same start relative offset. As a result, some short bvecs could be selected for decompression and then cause data corruption.

For example, as Shijie Sun reported off-list, considering the following extents of a file:

 117:   903345..  915250 |   11905 :     385024..    389120 |    4096
...
 119:   919729..  930323 |   10594 :     385024..    389120 |    4096
...
 124:   968881..  980786 |   11905 :     385024..    389120 |    4096

The start relative offset is the same: 2225, but extent 119 (919729.. 930323) is shorter than the others.

Let's restrict the bvec length in addition to the start offset if bvecs are not full.

Fixes 5c2a64252c5d ("erofs: introduce partial-referenced pclusters")
---
 fs/erofs/zdata.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
fs/erofs/zdata.c +4 -3
index b69d89a11dd0..de4f12152b62 100644
@@ -1144,10 +1144,11 @@ static void z_erofs_do_decompressed_bvec(struct z_erofs_decompress_backend *be,
                                         struct z_erofs_bvec *bvec)
 {
        struct z_erofs_bvec_item *item;
+       unsigned int pgnr;
 
-       if (!((bvec->offset + be->pcl->pageofs_out) & ~PAGE_MASK)) {
-               unsigned int pgnr;
-
+       if (!((bvec->offset + be->pcl->pageofs_out) & ~PAGE_MASK) &&
+           (bvec->end == PAGE_SIZE ||
+            bvec->offset + bvec->end == be->pcl->length)) {
                pgnr = (bvec->offset + be->pcl->pageofs_out) >> PAGE_SHIFT;
                DBG_BUGON(pgnr >= be->nr_pages);
                if (!be->decompressed_pages[pgnr]) {

--
2.24.4
Tested-by Shijie Sun <[email protected]>

Thanks,

Sun Shijie

________________________________
发件人: Gao Xiang <[email protected]>
发送时间: 2023年7月19日 14:54:59
收件人: [email protected]
抄送: LKML; Gao Xiang; 孙士杰
主题: [External Mail][PATCH] erofs: fix wrong primary bvec selection on deduplicated extents

[外部邮件] 此邮件来源于小米公司外部,请谨慎处理。若对邮件安全性存疑,请将邮件转发给[email protected]进行反馈

When handling deduplicated compressed data, there can be multiple decompressed extents pointing to the same compressed data in one shot.

In such cases, the bvecs which belong to the longest extent will be selected as the primary bvecs for real decompressors to decode and the other duplicated bvecs will be directly copied from the primary bvecs.

Previously, only relative offsets of the longest extent was checked to decompress the primary bvecs. On rare occasions, it can be incorrect if there are several extents with the same start relative offset. As a result, some short bvecs could be selected for decompression and then cause data corruption.

For example, as Shijie Sun reported off-list, considering the following extents of a file:

 117:   903345..  915250 |   11905 :     385024..    389120 |    4096
...
 119:   919729..  930323 |   10594 :     385024..    389120 |    4096
...
 124:   968881..  980786 |   11905 :     385024..    389120 |    4096

The start relative offset is the same: 2225, but extent 119 (919729.. 930323) is shorter than the others.

Let's restrict the bvec length in addition to the start offset if bvecs are not full.

Fixes 5c2a64252c5d ("erofs: introduce partial-referenced pclusters")
---
 fs/erofs/zdata.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
fs/erofs/zdata.c +4 -3
index b69d89a11dd0..de4f12152b62 100644
@@ -1144,10 +1144,11 @@ static void z_erofs_do_decompressed_bvec(struct z_erofs_decompress_backend *be,
                                         struct z_erofs_bvec *bvec)
 {
        struct z_erofs_bvec_item *item;
+       unsigned int pgnr;
 
-       if (!((bvec->offset + be->pcl->pageofs_out) & ~PAGE_MASK)) {
-               unsigned int pgnr;
-
+       if (!((bvec->offset + be->pcl->pageofs_out) & ~PAGE_MASK) &&
+           (bvec->end == PAGE_SIZE ||
+            bvec->offset + bvec->end == be->pcl->length)) {
                pgnr = (bvec->offset + be->pcl->pageofs_out) >> PAGE_SHIFT;
                DBG_BUGON(pgnr >= be->nr_pages);
                if (!be->decompressed_pages[pgnr]) {

--
2.24.4

#/******本邮件及其附件含有小米公司的保密信息,仅限于发送给上面地址中列出的个人或群组。禁止任何其他人以任何形式使用(包括但不限于全部或部分地泄露、复制、或散发)本邮件中的信息。如果您错收了本邮件,请您立即电话或邮件通知发件人并删除本邮件! This e-mail and its attachments contain confidential information from XIAOMI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!******/#

Tested-by Shijie Sun
#2 lore raw

Gao Xiang <[email protected]>; [email protected];
LKML <[email protected]>
When handling deduplicated compressed data, there can be multiple decompressed extents pointing to the same compressed data in one shot.

In such cases, the bvecs which belong to the longest extent will be selected as the primary bvecs for real decompressors to decode and the other duplicated bvecs will be directly copied from the primary bvecs.

Previously, only relative offsets of the longest extent was checked to decompress the primary bvecs. On rare occasions, it can be incorrect if there are several extents with the same start relative offset. As a result, some short bvecs could be selected for decompression and then cause data corruption.

For example, as Shijie Sun reported off-list, considering the following extents of a file:

 117:   903345..  915250 |   11905 :     385024..    389120 |    4096
...
 119:   919729..  930323 |   10594 :     385024..    389120 |    4096
...
 124:   968881..  980786 |   11905 :     385024..    389120 |    4096

The start relative offset is the same: 2225, but extent 119 (919729.. 930323) is shorter than the others.

Let's restrict the bvec length in addition to the start offset if bvecs are not full.

Fixes 5c2a64252c5d ("erofs: introduce partial-referenced pclusters")

Thanks,
Sun Shijie

________________________________
发件人: 孙士杰
发送时间: 2023年7月19日 18:17:40
收件人: Gao Xiang; [email protected]
抄送: LKML
主题: 答复: [External Mail][PATCH] erofs: fix wrong primary bvec selection on deduplicated extents

When handling deduplicated compressed data, there can be multiple decompressed extents pointing to the same compressed data in one shot.

In such cases, the bvecs which belong to the longest extent will be selected as the primary bvecs for real decompressors to decode and the other duplicated bvecs will be directly copied from the primary bvecs.

Previously, only relative offsets of the longest extent was checked to decompress the primary bvecs. On rare occasions, it can be incorrect if there are several extents with the same start relative offset. As a result, some short bvecs could be selected for decompression and then cause data corruption.

For example, as Shijie Sun reported off-list, considering the following extents of a file:

 117:   903345..  915250 |   11905 :     385024..    389120 |    4096
...
 119:   919729..  930323 |   10594 :     385024..    389120 |    4096
...
 124:   968881..  980786 |   11905 :     385024..    389120 |    4096

The start relative offset is the same: 2225, but extent 119 (919729.. 930323) is shorter than the others.

Let's restrict the bvec length in addition to the start offset if bvecs are not full.

Fixes 5c2a64252c5d ("erofs: introduce partial-referenced pclusters")
---
 fs/erofs/zdata.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
fs/erofs/zdata.c +4 -3
index b69d89a11dd0..de4f12152b62 100644
@@ -1144,10 +1144,11 @@ static void z_erofs_do_decompressed_bvec(struct z_erofs_decompress_backend *be,
                                         struct z_erofs_bvec *bvec)
 {
        struct z_erofs_bvec_item *item;
+       unsigned int pgnr;
 
-       if (!((bvec->offset + be->pcl->pageofs_out) & ~PAGE_MASK)) {
-               unsigned int pgnr;
-
+       if (!((bvec->offset + be->pcl->pageofs_out) & ~PAGE_MASK) &&
+           (bvec->end == PAGE_SIZE ||
+            bvec->offset + bvec->end == be->pcl->length)) {
                pgnr = (bvec->offset + be->pcl->pageofs_out) >> PAGE_SHIFT;
                DBG_BUGON(pgnr >= be->nr_pages);
                if (!be->decompressed_pages[pgnr]) {

--
2.24.4
Tested-by Shijie Sun <[email protected]>

Thanks,

Sun Shijie

________________________________
发件人: Gao Xiang <[email protected]>
发送时间: 2023年7月19日 14:54:59
收件人: [email protected]
抄送: LKML; Gao Xiang; 孙士杰
主题: [External Mail][PATCH] erofs: fix wrong primary bvec selection on deduplicated extents

[外部邮件] 此邮件来源于小米公司外部,请谨慎处理。若对邮件安全性存疑,请将邮件转发给[email protected]进行反馈

When handling deduplicated compressed data, there can be multiple decompressed extents pointing to the same compressed data in one shot.

In such cases, the bvecs which belong to the longest extent will be selected as the primary bvecs for real decompressors to decode and the other duplicated bvecs will be directly copied from the primary bvecs.

Previously, only relative offsets of the longest extent was checked to decompress the primary bvecs. On rare occasions, it can be incorrect if there are several extents with the same start relative offset. As a result, some short bvecs could be selected for decompression and then cause data corruption.

For example, as Shijie Sun reported off-list, considering the following extents of a file:

 117:   903345..  915250 |   11905 :     385024..    389120 |    4096
...
 119:   919729..  930323 |   10594 :     385024..    389120 |    4096
...
 124:   968881..  980786 |   11905 :     385024..    389120 |    4096

The start relative offset is the same: 2225, but extent 119 (919729.. 930323) is shorter than the others.

Let's restrict the bvec length in addition to the start offset if bvecs are not full.

Fixes 5c2a64252c5d ("erofs: introduce partial-referenced pclusters")
---
 fs/erofs/zdata.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
fs/erofs/zdata.c +4 -3
index b69d89a11dd0..de4f12152b62 100644
@@ -1144,10 +1144,11 @@ static void z_erofs_do_decompressed_bvec(struct z_erofs_decompress_backend *be,
                                         struct z_erofs_bvec *bvec)
 {
        struct z_erofs_bvec_item *item;
+       unsigned int pgnr;
 
-       if (!((bvec->offset + be->pcl->pageofs_out) & ~PAGE_MASK)) {
-               unsigned int pgnr;
-
+       if (!((bvec->offset + be->pcl->pageofs_out) & ~PAGE_MASK) &&
+           (bvec->end == PAGE_SIZE ||
+            bvec->offset + bvec->end == be->pcl->length)) {
                pgnr = (bvec->offset + be->pcl->pageofs_out) >> PAGE_SHIFT;
                DBG_BUGON(pgnr >= be->nr_pages);
                if (!be->decompressed_pages[pgnr]) {

--
2.24.4

#/******本邮件及其附件含有小米公司的保密信息,仅限于发送给上面地址中列出的个人或群组。禁止任何其他人以任何形式使用(包括但不限于全部或部分地泄露、复制、或散发)本邮件中的信息。如果您错收了本邮件,请您立即电话或邮件通知发件人并删除本邮件! This e-mail and its attachments contain confidential information from XIAOMI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!******/#

reviewer Reviewed-by on v1 1/1
#3 lore raw
In reply to Gao Xiang
Quoted 24 lines · On Wed, 19 Jul 2023 14:54:59 +0800 Gao Xiang <[email protected]>…

When handling deduplicated compressed data, there can be multiple decompressed extents pointing to the same compressed data in one shot.

In such cases, the bvecs which belong to the longest extent will be selected as the primary bvecs for real decompressors to decode and the other duplicated bvecs will be directly copied from the primary bvecs.

Previously, only relative offsets of the longest extent was checked to decompress the primary bvecs. On rare occasions, it can be incorrect if there are several extents with the same start relative offset. As a result, some short bvecs could be selected for decompression and then cause data corruption.

For example, as Shijie Sun reported off-list, considering the following extents of a file:

 117:   903345..  915250 |   11905 :     385024..    389120 |    4096
...
 119:   919729..  930323 |   10594 :     385024..    389120 |    4096
...
 124:   968881..  980786 |   11905 :     385024..    389120 |    4096

The start relative offset is the same: 2225, but extent 119 (919729.. 930323) is shorter than the others.

Quoted 6 lines

Let's restrict the bvec length in addition to the start offset if bvecs are not full.

Fixes 5c2a64252c5d ("erofs: introduce partial-referenced pclusters")
maintainer Reviewed-by on v1 1/1
#4 lore raw
In reply to Gao Xiang
Quoted 24 lines · Gao Xiang

When handling deduplicated compressed data, there can be multiple decompressed extents pointing to the same compressed data in one shot.

In such cases, the bvecs which belong to the longest extent will be selected as the primary bvecs for real decompressors to decode and the other duplicated bvecs will be directly copied from the primary bvecs.

Previously, only relative offsets of the longest extent was checked to decompress the primary bvecs. On rare occasions, it can be incorrect if there are several extents with the same start relative offset. As a result, some short bvecs could be selected for decompression and then cause data corruption.

For example, as Shijie Sun reported off-list, considering the following extents of a file:

  117:   903345..  915250 |   11905 :     385024..    389120 |    4096
...
  119:   919729..  930323 |   10594 :     385024..    389120 |    4096
...
  124:   968881..  980786 |   11905 :     385024..    389120 |    4096

The start relative offset is the same: 2225, but extent 119 (919729.. 930323) is shorter than the others.

Quoted 6 lines

Let's restrict the bvec length in addition to the start offset if bvecs are not full.

Fixes 5c2a64252c5d ("erofs: introduce partial-referenced pclusters")

Thanks,