Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 73B70C433FE for ; Wed, 23 Nov 2022 04:04:17 +0000 (UTC) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4NH6vM4c6sz3cMx for ; Wed, 23 Nov 2022 15:04:15 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=coolpad-com.20200927.dkim.feishu.cn header.i=@coolpad-com.20200927.dkim.feishu.cn header.a=rsa-sha256 header.s=s1 header.b=Ti/T9eYl; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=coolpad.com (client-ip=101.36.218.34; helo=v03.bc.feishu.cn; envelope-from=huyue2@coolpad.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=coolpad-com.20200927.dkim.feishu.cn header.i=@coolpad-com.20200927.dkim.feishu.cn header.a=rsa-sha256 header.s=s1 header.b=Ti/T9eYl; dkim-atps=neutral X-Greylist: delayed 612 seconds by postgrey-1.36 at boromir; Wed, 23 Nov 2022 15:04:04 AEDT Received: from v03.bc.feishu.cn (v03.bc.feishu.cn [101.36.218.34]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with UTF8SMTPS id 4NH6v82Nsgz3bjW for ; Wed, 23 Nov 2022 15:04:02 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=s1; d=coolpad-com.20200927.dkim.feishu.cn; t=1669175509; h=mime-version:from:date:message-id:subject:to; bh=qB4sMM9OXlw0apugrt9C8ZUC1KdXZ7g4kbh2yZYX8P8=; b=Ti/T9eYlJ20+xpMVWbnErYJKusNGqh+SLJYEOjNTJrCvEzg7QFECOqA1AOJNHMOfRevDEk LY3VBW8+TPs2UHe6kVkrILRANDo0gnOFOeLUM42tOa1llR2KsBRTTe0ptMPoAVEguAgWB3 efXElDWX+woCrNJpMECZE86uP6fCb+KiyhsXC1zNiQCk+LKSDwYbT0yd4RYWrEDFpJN2sk 5oTWtT8ujxDE8PugkqXzfYDEm0JgQ22YI62kpXDIDi8RorZ+FdtoBRFwtW6U32/z4UiWqT QU8D0eWSlO3L3NkpRR6p7PyMAYnwsTBv7xMhyoPyoQ1Y9Ipqbu/ayTr6iOqJ4Q== Date: Wed, 23 Nov 2022 11:51:38 +0800 Message-Id: <20221123115540.000033b3.huyue2@coolpad.com> X-Mailer: Claws Mail 3.17.3 (GTK+ 2.24.32; i686-w64-mingw32) In-Reply-To: <20221123023034.123095-1-hsiangkao@linux.alibaba.com> References: <20221122064527.26563-1-hsiangkao@linux.alibaba.com> <20221123023034.123095-1-hsiangkao@linux.alibaba.com> From: "Yue Hu" Subject: Re: [PATCH v2] erofs-utils: lib: fix missing CBLKCNT for big pcluster dedupe Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Lms-Return-Path: Content-Type: multipart/alternative; boundary=ce9377aee8369dfccf3dad66ab53b7343f7cc28c78b5b01d23b1c5264ac9 To: "Gao Xiang" X-BeenThere: linux-erofs@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development of Linux EROFS file system List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-erofs@lists.ozlabs.org, zhangwen@coolpad.com Errors-To: linux-erofs-bounces+linux-erofs=archiver.kernel.org@lists.ozlabs.org Sender: "Linux-erofs" --ce9377aee8369dfccf3dad66ab53b7343f7cc28c78b5b01d23b1c5264ac9 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=UTF-8 On Wed, 23 Nov 2022 10:30:34 +0800 Gao Xiang wrote: > CBLKCNT needs to be stored for big pcluster dedupe. Otherwise, > the decompression could fail due to incomplete compressed data. >=20 > Reported-by: Yue Hu > Signed-off-by: Gao Xiang Tested-by: Yue Hu > --- > changes since v1: > - fix potential data corruption of v1 >=20 > lib/compress.c | 12 +++++++++++- > 1 file changed, 11 insertions(+), 1 deletion(-) >=20 > diff --git a/lib/compress.c b/lib/compress.c > index 17b3213..8f4c63a 100644 > --- a/lib/compress.c > +++ b/lib/compress.c > @@ -186,12 +186,22 @@ static int z_erofs_compress_dedupe(struct erofs_ino= de *inode, > if (z_erofs_dedupe_match(&dctx)) > break; > =20 > + delta =3D ctx->queue + ctx->head - dctx.cur; > + /* > + * For big pcluster dedupe, leave two indices at least to store > + * CBLKCNT as the first step. Even laterly, an one-block > + * decompresssion could be done as another try in practice. > + */ > + if (dctx.e.compressedblks > 1 && > + (ctx->clusterofs + ctx->e.length - delta) % EROFS_BLKSIZ + > + dctx.e.length < 2 * EROFS_BLKSIZ) > + break; > + > /* fall back to noncompact indexes for deduplication */ > inode->z_advise &=3D ~Z_EROFS_ADVISE_COMPACTED_2B; > inode->datalayout =3D EROFS_INODE_FLAT_COMPRESSION_LEGACY; > erofs_sb_set_dedupe(); > =20 > - delta =3D ctx->queue + ctx->head - dctx.cur; > if (delta) { > DBG_BUGON(delta < 0); > DBG_BUGON(!ctx->e.length); --ce9377aee8369dfccf3dad66ab53b7343f7cc28c78b5b01d23b1c5264ac9 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=UTF-8

On Wed, 23 Nov 2022 10:30:34 +0800
Gao Xiang wrote:

> CBLKCNT needs to be stored for big pcluster dedupe. Otherwise,
> the decompression could fail due to incomplete compressed data.
>=20
> Reported-by: Yue Hu
> Signed-off-by: Gao Xiang

Tested-by: Yue Hu

> ---
> changes since v1:
> - fix potential data corruption of v1
>=20
> lib/compress.c | 12 +++++++++++-
> 1 file changed, 11 insertions(+), 1 deletion(-)
>=20
> diff --git a/lib/compress.c b/lib/compress.c
> index 17b3213..8f4c63a 100644
> --- a/lib/compress.c
> +++ b/lib/compress.c
> @@ -186,12 +186,22 @@ static int z_erofs_compress_dedupe(struct erofs= _inode *inode,
> if (z_erofs_dedupe_match(&dctx))
> break;
> =20
> + delta =3D ctx->queue + ctx->head - dctx.cur;
> + /*
> + * For big pcluster dedupe, leave two indices at least to store
> + * CBLKCNT as the first step. Even laterly, an one-block
> + * decompresssion could be done as another try in practice.
> + */
> + if (dctx.e.compressedblks > 1 &&
> + (ctx->clusterofs + ctx->e.length - delta) % EROFS_BLKSIZ +
> + dctx.e.length < 2 * EROFS_BLKSIZ)
> + break;
> +
> /* fall back to noncompact indexes for deduplication */
> inode->z_advise &=3D ~Z_EROFS_ADVISE_COMPACTED_2B;
> inode->datalayout =3D EROFS_INODE_FLAT_COMPRESSION_LEGACY;
> erofs_sb_set_dedupe();
> =20
> - delta =3D ctx->queue + ctx->head - dctx.cur;
> if (delta) {
> DBG_BUGON(delta < 0);
> DBG_BUGON(!ctx->e.length);

------=E6=9C=BA=E5=AF=86=EF=BC=9A=E6=AD=A4=E7=94=B5=E5=AD=90= =E9=82=AE=E4=BB=B6=E6=89=80=E5=8C=85=E5=90=AB=E5=86=85=E5=AE=B9=E4=B8=BA=E9= =85=B7=E6=B4=BE=E6=9C=BA=E5=AF=86=E5=86=85=E5=AE=B9,=E5=B9=B6=E4=B8=94=E5= =8F=97=E5=88=B0=E6=B3=95=E5=BE=8B=E7=9A=84=E4=BF=9D=E6=8A=A4=E3=80=82=E5=A6= =82=E6=9E=9C=E6=82=A8=E4=B8=8D=E5=B1=9E=E4=BA=8E=E4=BB=A5=E4=B8=8A=E7=94=B5= =E5=AD=90=E9=82=AE=E4=BB=B6=E7=9A=84=E7=9B=AE=E6=A0=87=E6=8E=A5=E6=94=B6=E8= =80=85=EF=BC=8C=E6=82=A8=E4=B8=8D=E5=BE=97=E7=BB=86=E8=AF=BB=EF=BC=8C=E4=BD= =BF=E7=94=A8=EF=BC=8C=E4=BC=A0=E6=92=AD=EF=BC=8C=E6=95=A3=E5=B8=83=E6=88=96= =E5=A4=8D=E5=88=B6=E8=AF=A5=E7=94=B5=E5=AD=90=E9=82=AE=E4=BB=B6=E4=B8=AD=E7= =9A=84=E4=BB=BB=E4=BD=95=E4=BF=A1=E6=81=AF=E3=80=82=E5=A6=82=E6=9E=9C=E6=82= =A8=E5=B7=B2=E7=BB=8F=E8=AF=AF=E6=94=B6=E6=AD=A4=E7=94=B5=E5=AD=90=E9=82=AE= =E4=BB=B6=EF=BC=8C=E8=AF=B7=E6=82=A8=E7=AB=8B=E5=8D=B3=E9=80=9A=E7=9F=A5=E6= =88=91=E4=BB=AC=E5=B9=B6=E5=88=A0=E9=99=A4=E5=8E=9F=E7=94=B5=E5=AD=90=E9=82= =AE=E4=BB=B6=E3=80=82 CONFIDENTIAL: This e-mail message contains informatio= n of Coolpad that is confidential and which is subject to legal privilege.I= f you are not the intended recipient as indicated above,you must not peruse= ,use, disseminate,distribute or copy any information contained in this mess= age.If you have received this message in error, please notify us and delete= the original message immediately.
------ =E7=94=B3=E6=98=8E=EF=BC=9A=E6=9C=AC=E9=82=AE=E4=BB=B6=E6= =89=80=E7=8E=B0=E5=86=85=E5=AE=B9=EF=BC=8C=E4=BB=85=E4=BD=9C=E4=B8=BA=E6=88= =91=E4=BB=AC=E4=B9=8B=E9=97=B4=E5=B0=B1=E5=90=88=E4=BD=9C=E7=9A=84=E4=BA=8B= =E5=AE=9C=E8=BF=9B=E8=A1=8C=E7=9A=84=E4=BA=A4=E6=B5=81=E3=80=81=E6=B2=9F=E9= =80=9A=E3=80=81=E6=B4=BD=E8=B0=88=E3=80=81=E5=95=86=E8=AE=AE=EF=BC=8C=E4=B8= =8D=E4=BD=9C=E4=B8=BA=E5=8D=8F=E8=AE=AE=E6=88=96=E6=89=BF=E8=AF=BA=EF=BC=8C= =E4=B8=80=E5=88=87=E5=8D=8F=E8=AE=AE=E5=8F=8A=E6=89=BF=E8=AF=BA=E5=BF=85=E9= =A1=BB=E4=BB=A5=E4=B9=A6=E9=9D=A2=E6=96=87=E6=9C=AC=E7=9B=96=E7=AB=A0=E4=B8= =BA=E5=87=86=E3=80=82 DECLARATION=EF=BC=9AAll contents of this E-mail ,are = only regarded as the cooperation we have had between the exchanges, communi= cation, negotiation and deliberation, not as a agreement or promise. All co= ntracts and commitments must be sealed shall prevail.
--ce9377aee8369dfccf3dad66ab53b7343f7cc28c78b5b01d23b1c5264ac9--