Skip to contentSkip to conversation

erofs-utils: lib: fix missing CBLKCNT for big pcluster dedupe

PATCH Superseded Gao Xiang 2 participants 3 messages Nov 22, 2022 – Nov 23, 2022 Library code

lore
Share & tools
lore

Gao Xiang

author
#0 lore raw

PATCH 1/1 erofs-utils: lib: fix missing CBLKCNT for big pcluster dedupe

CBLKCNT needs to be stored for big pcluster dedupe. Otherwise, the decompression could fail due to incomplete compressed data.

---
 lib/compress.c | 21 +++++++++++++++------
 1 file changed, 15 insertions(+), 6 deletions(-)
lib/compress.c +15 -6
index 17b3213..50fea0c 100644
@@ -186,11 +186,6 @@ static int z_erofs_compress_dedupe(struct erofs_inode *inode,
 		if (z_erofs_dedupe_match(&dctx))
 			break;
 
-		/* fall back to noncompact indexes for deduplication */
-		inode->z_advise &= ~Z_EROFS_ADVISE_COMPACTED_2B;
-		inode->datalayout = EROFS_INODE_FLAT_COMPRESSION_LEGACY;
-		erofs_sb_set_dedupe();
-
 		delta = ctx->queue + ctx->head - dctx.cur;
 		if (delta) {
 			DBG_BUGON(delta < 0);
@@ -199,10 +194,24 @@ static int z_erofs_compress_dedupe(struct erofs_inode *inode,
 			ctx->e.length -= delta;
 		}
 
+		/*
+		 * For big pcluster dedupe, leave two indices at least to store
+		 * CBLKCNT as the first step.  Even laterly, an one-block
+		 * decompresssion could be done as another try in practice.
+		 */
+		z_erofs_write_indexes(ctx);
+		if (dctx.e.compressedblks > 1 &&
+		    ctx->clusterofs + dctx.e.length < 2*EROFS_BLKSIZ)
+			return ret;
+
+		/* fall back to noncompact indexes for deduplication */
+		inode->z_advise &= ~Z_EROFS_ADVISE_COMPACTED_2B;
+		inode->datalayout = EROFS_INODE_FLAT_COMPRESSION_LEGACY;
+		erofs_sb_set_dedupe();
+
 		erofs_dbg("Dedupe %u %scompressed data (delta %d) to %u of %u blocks",
 			  dctx.e.length, dctx.e.raw ? "un" : "",
 			  delta, dctx.e.blkaddr, dctx.e.compressedblks);
-		z_erofs_write_indexes(ctx);
 		ctx->e = dctx.e;
 		ctx->head += dctx.e.length - delta;
 		DBG_BUGON(*len < dctx.e.length - delta);

--
2.30.2

author
#1 lore raw

PATCH v2 1/1 erofs-utils: lib: fix missing CBLKCNT for big pcluster dedupe

CBLKCNT needs to be stored for big pcluster dedupe. Otherwise, the decompression could fail due to incomplete compressed data.

---
changes since v1:
- fix potential data corruption of v1

 lib/compress.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
lib/compress.c +11 -1
index 17b3213..8f4c63a 100644
@@ -186,12 +186,22 @@ static int z_erofs_compress_dedupe(struct erofs_inode *inode,
 		if (z_erofs_dedupe_match(&dctx))
 			break;
 
+		delta = ctx->queue + ctx->head - dctx.cur;
+		/*
+		 * For big pcluster dedupe, leave two indices at least to store
+		 * CBLKCNT as the first step.  Even laterly, an one-block
+		 * decompresssion could be done as another try in practice.
+		 */
+		if (dctx.e.compressedblks > 1 &&
+		    (ctx->clusterofs + ctx->e.length - delta) % EROFS_BLKSIZ +
+			dctx.e.length < 2 * EROFS_BLKSIZ)
+			break;
+
 		/* fall back to noncompact indexes for deduplication */
 		inode->z_advise &= ~Z_EROFS_ADVISE_COMPACTED_2B;
 		inode->datalayout = EROFS_INODE_FLAT_COMPRESSION_LEGACY;
 		erofs_sb_set_dedupe();
 
-		delta = ctx->queue + ctx->head - dctx.cur;
 		if (delta) {
 			DBG_BUGON(delta < 0);
 			DBG_BUGON(!ctx->e.length);
Signature · 1 line
2.24.4
Tested-by on v2 1/1
#2 lore raw
Quoted 5 lines · On Wed, 23 Nov 2022 10:30:34 +0800 Gao Xiang <[email protected]>…

CBLKCNT needs to be stored for big pcluster dedupe. Otherwise, the decompression could fail due to incomplete compressed data.

Quoted 35 lines

---
changes since v1:
- fix potential data corruption of v1

 lib/compress.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
lib/compress.c +11 -1
index 17b3213..8f4c63a 100644
@@ -186,12 +186,22 @@ static int z_erofs_compress_dedupe(struct erofs_inode *inode,
 		if (z_erofs_dedupe_match(&dctx))
 			break;
 
+		delta = ctx->queue + ctx->head - dctx.cur;
+		/*
+		 * For big pcluster dedupe, leave two indices at least to store
+		 * CBLKCNT as the first step.  Even laterly, an one-block
+		 * decompresssion could be done as another try in practice.
+		 */
+		if (dctx.e.compressedblks > 1 &&
+		    (ctx->clusterofs + ctx->e.length - delta) % EROFS_BLKSIZ +
+			dctx.e.length < 2 * EROFS_BLKSIZ)
+			break;
+
 		/* fall back to noncompact indexes for deduplication */
 		inode->z_advise &= ~Z_EROFS_ADVISE_COMPACTED_2B;
 		inode->datalayout = EROFS_INODE_FLAT_COMPRESSION_LEGACY;
 		erofs_sb_set_dedupe();
 
-		delta = ctx->queue + ctx->head - dctx.cur;
 		if (delta) {
 			DBG_BUGON(delta < 0);
 			DBG_BUGON(!ctx->e.length);