Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: ** X-Spam-Status: No, score=2.5 required=3.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,MAILING_LIST_MULTI, MIME_QP_LONG_LINE,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA764C48BD1 for ; Fri, 11 Jun 2021 09:06:46 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3D226613C9 for ; Fri, 11 Jun 2021 09:06:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3D226613C9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:53246 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lrd7l-0003yS-66 for qemu-devel@archiver.kernel.org; Fri, 11 Jun 2021 05:06:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40822) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lrd6Y-0002YE-SD; Fri, 11 Jun 2021 05:05:31 -0400 Received: from mail-wm1-x32c.google.com ([2a00:1450:4864:20::32c]:46008) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lrd6V-000649-H3; Fri, 11 Jun 2021 05:05:30 -0400 Received: by mail-wm1-x32c.google.com with SMTP id v206-20020a1cded70000b02901a586d3fa23so8033334wmg.4; Fri, 11 Jun 2021 02:05:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=content-transfer-encoding:from:mime-version:subject:date:message-id :references:cc:in-reply-to:to; bh=I+k7JTHnOxfyYhmPdgo4wnF/IlfdOPoPDBQlcb25bUk=; b=lmH+wJsZHVbPGucvqizclehzAW0P9g7SQGhhdgFKzPMNkj8MZGLDUqVrGVL70/8GIc BLO6oKNVO6sx2RZ7hf5Ui1TRiZCAxObJBJ2Ju2HCHeVVCRMRFz0ZSWp5hoLMH8oF0vsX 2iBxBnvfOAuEDY7F7YJtA83bxca20Aw94maH9UUrx32LbJjZjInvziuqDyhdteszHqm2 9nYQvhptsRBUfnpIcxV5U6DUn9SBUR0tNYPmRx/nXDuJ6eomcwuftvks5Epe2+vhySRj /dUO/796abQygvDYHPlDkmjUUPqkVPMaQg4iAZORAzdFbf6PiA9+kfRvhBJLHan9FINZ eXPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:content-transfer-encoding:from:mime-version :subject:date:message-id:references:cc:in-reply-to:to; bh=I+k7JTHnOxfyYhmPdgo4wnF/IlfdOPoPDBQlcb25bUk=; b=LgDA7ExxbtJfPpNrRDp8QkG/+w23eN2EiIYGWFFnLIfD3M48ByJr67Mxyv0KuTDH84 14mIGQ0KrGVMKe8RD36PMwaZPeW1SG+7u9llWM8ATlq2pkA1gGr3ZtSgpj0ShZWpkSmg 8C+mwehmh7yWdDCPIkg/bUKYDhQOV2m1j8AVFGnEUbE+2nq2quRIFMHuI0+VZ3xuEHih fU6Y4yyDCR7Yvo7o6mB0g0603FQ32dbEw6WNs8rx9vT0XaRgipEs6mV8/oiiR+aWukS3 5tccFnh0o+T9DLk0YMngX45/NkNOH+DSnW2G1jyn9SmIAcEHcOsY4PvQt+bhxBCEZKY2 hCFw== X-Gm-Message-State: AOAM533SqZ4M7O2AIvNRIAq4rqJEExuRzGPFFBraDMtBp8y1i4IOxE/o GEPMVV4yMAZNgAbtyffFeSg= X-Google-Smtp-Source: ABdhPJz8pl1ei24A4k1ngJFzj2h945aHAcYED3NRVfgtNPngKdXqK7C5bvqjZIi0dJ0Uzq4ih+Mc6A== X-Received: by 2002:a1c:a785:: with SMTP id q127mr2902569wme.152.1623402321518; Fri, 11 Jun 2021 02:05:21 -0700 (PDT) Received: from smtpclient.apple (89-138-223-238.bb.netvision.net.il. [89.138.223.238]) by smtp.gmail.com with ESMTPSA id p5sm6582061wrd.25.2021.06.11.02.05.20 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 11 Jun 2021 02:05:20 -0700 (PDT) Content-Type: multipart/alternative; boundary=Apple-Mail-1E37F757-26E9-43F9-9073-1DED126039A6 Content-Transfer-Encoding: 7bit From: Nir Soffer Mime-Version: 1.0 (1.0) Subject: Re: [PATCH] qemu-{img,nbd}: Don't report zeroed cluster as a hole Date: Fri, 11 Jun 2021 12:05:19 +0300 Message-Id: References: <49ed9c03-eb24-4e8a-1b5f-8a291466a7b9@virtuozzo.com> In-Reply-To: <49ed9c03-eb24-4e8a-1b5f-8a291466a7b9@virtuozzo.com> To: Vladimir Sementsov-Ogievskiy X-Mailer: iPhone Mail (18F72) Received-SPF: pass client-ip=2a00:1450:4864:20::32c; envelope-from=nirsof@gmail.com; helo=mail-wm1-x32c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, MIME_QP_LONG_LINE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-block , QEMU Developers , Max Reitz , Nir Soffer , Eric Blake Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" --Apple-Mail-1E37F757-26E9-43F9-9073-1DED126039A6 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable > =E2=80=AB=D7=91-11 =D7=91=D7=99=D7=95=D7=A0=D7=99 2021, =D7=91=D7=A9=D7=A2= =D7=94 11:14, =E2=80=8F=E2=80=8FVladimir Sementsov-Ogievskiy =E2=80=8F =D7=9B=D7=AA=D7=91/=D7=94:=E2=80=AC >=20 > =EF=BB=BF11.06.2021 11:09, Kevin Wolf wrote: >> Am 10.06.2021 um 22:46 hat Eric Blake geschrieben: >>>> On Thu, Jun 10, 2021 at 11:09:05PM +0300, Nir Soffer wrote: >>>>>> But: >>>>>>=20 >>>>>> $ qemu-img map --output=3Djson -f qcow2 json:'{"driver":"qcow2","back= ing":null, \ >>>>>> "file":{"driver":"file","filename":"top.qcow2"}}' >>>>>> [{ "start": 0, "length": 65536, "depth": 0, "zero": true, "data": fal= se}, >>>>>> { "start": 65536, "length": 65536, "depth": 0, "zero": false, "data":= true, "offset": 327680}, >>>>>> { "start": 131072, "length": 131072, "depth": 0, "zero": true, "data"= : false}] >>>>>>=20 >>>>>> also reports the entire file at "depth":0, which is misleading, since= >>>>>> we have just been arguing from the qemu:allocation-depth perspective >>>>>> (and also from bdrv_block_status) that the qcow2 image is NOT 100% >>>>>> allocated (in the sense where allocation =3D=3D data comes locally). >>>>>> Perhaps it might be better if we tweaked the above qemu-img map to >>>>>> produce: >>>>>>=20 >>>>>> [{ "start": 0, "length": 65536, "depth": -1, "zero": true, "data": fa= lse}, >>>>>> { "start": 65536, "length": 65536, "depth": 0, "zero": false, "data":= true, "offset": 327680}, >>>>>> { "start": 131072, "length": 65536, "depth": 0, "zero": true, "data":= false}, >>>>>> { "start": 196608, "length": 65536, "depth": -1, "zero": true, "data"= : false}] >>>>>=20 >>>>> It will be more consistent with "offset" to drop "depth" from output >>>>> if we don't have it: >>>>>=20 >>>>> [{ "start": 0, "length": 65536, "zero": true, "data": false}, >>>>> { "start": 65536, "length": 65536, "depth": 0, "zero": false, >>>>> "data": true, "offset": 327680}, >>>>> { "start": 131072, "length": 65536, "depth": 0, "zero": true, >>>>> "data": false}, >>>>> { "start": 196608, "length": 65536, "zero": true, "data": false}]= >>>=20 >>> Yes, that might work as well. But we didn't previously document >>> depth to be optional. Removing something from output risks breaking >>> more downstream tools that expect it to be non-optional, compared to >>> providing a new value. >> A negative value isn't any less unexpected than a missing key. I don't >> think any existing tool would be able to handle it. Encoding different >> meanings in a single value isn't very QAPI-like either. Usually strings >> that are parsed are the problem, but negative integers really isn't that >> much different. I don't really like this solution. >> Leaving out the depth feels like a better suggestion to me. >> But anyway, this seems to only happen at the end of the backing chain. >> So if the backing chain consistents of n images, why not report 'depth': >> n + 1? So, in the above example, you would get 1. I think this has the >> best chances of tools actually working correctly with the new output, >> even though it's still not unlikely to break something. >=20 > Did you consider just add a new field? >=20 > So, "depth" keeps its meaning "which level provides data". >=20 > And we add additional optional field like >=20 > absolutely-completely-absent: bool hole: bool? >=20 > Which is true if data is nowhere in the backing chain. >=20 >=20 > --=20 > Best regards, > Vladimir --Apple-Mail-1E37F757-26E9-43F9-9073-1DED126039A6 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable


=E2=80=AB=D7=91-11 =D7=91=D7=99= =D7=95=D7=A0=D7=99 2021, =D7=91=D7=A9=D7=A2=D7=94 11:14, =E2=80=8F=E2=80=8FV= ladimir Sementsov-Ogievskiy =E2=80=8F<vsementsov@virtuozzo.com> =D7=9B= =D7=AA=D7=91/=D7=94:=E2=80=AC

=EF=BB=BF11.06.2021 11:09, Kevin Wolf wrote:
Am 10.06.2021 um 22:46 hat Eric Blak= e geschrieben:
On Thu, Jun 10, 2021 at 11:09:05PM +0300, Nir Soffer wrot= e:
But= :

$ qemu-img map --output=3Djson -f q= cow2 json:'{"driver":"qcow2","backing":null, \
 &nbs= p;"file":{"driver":"file","filename":"top.qcow2"}}'
<= /blockquote>
[{ "s= tart": 0, "length": 65536, "depth": 0, "zero": true, "data": false},<= br>
{ "start": 65536, "length": 65536, "depth": 0, "zero": false, "= data": true, "offset": 327680},
{ "start": 131072, "leng= th": 131072, "depth": 0, "zero": true, "data": false}]
<= /span>
also reports the entire file at "depth":0, which is misle= ading, since
<= blockquote type=3D"cite">
we have just been arguing from the qemu:all= ocation-depth perspective
<= /blockquote>
(and also from bdrv_block_statu= s) that the qcow2 image is NOT 100%
allocated (in the se= nse where allocation =3D=3D data comes locally).
Perhaps= it might be better if we tweaked the above qemu-img map to
produce:
<= blockquote type=3D"cite">

[{ "start": 0, "length= ": 65536, "depth": -1, "zero": true, "data": false},
=
{ "= start": 65536, "length": 65536, "depth": 0, "zero": false, "data": true, "of= fset": 327680},
{ "start": 131072, "length": 65536, "dep= th": 0, "zero": true, "data": false},
=
{ "start": 196608,= "length": 65536, "depth": -1, "zero": true, "data": false}]

It will be more consistent with "offset= " to drop "depth" from output
if we don't have it:

&nbs= p;   [{ "start": 0, "length": 65536, "zero": true, "data": fa= lse},
  =    { "start": 65536, "length": 65536, "depth": 0, "zero": fal= se,
"data": true, "= offset": 327680},
&= nbsp;    { "start": 131072, "length": 65536, "depth": 0,= "zero": true,
"data= ": false},
 &= nbsp;   { "start": 196608, "length": 65536, "zero": true, "da= ta": false}]

Yes, that mig= ht work as well.  But we didn't previously document
d= epth to be optional.  Removing something from output risks breaking
more downstream tools that expect it to be non-optional, compar= ed to
providing a new value.
A negative value isn't any less unexp= ected than a missing key. I don't
think any existing tool would be able to handle it. Encoding di= fferent
meanings in a= single value isn't very QAPI-like either. Usually strings
that are parsed are the problem, but n= egative integers really isn't that
much different. I don't really like this solution.
Leaving out the depth feels like a= better suggestion to me.
<= span>But anyway, this seems to only happen at the end of the backing chain.<= /span>
So if the backing cha= in consistents of n images, why not report 'depth':
<= blockquote type=3D"cite">n + 1? So, in the above example, you would ge= t 1. I think this has the
<= span>best chances of tools actually working correctly with the new output,
even though it's still= not unlikely to break something.

Did you consider just add a new field?

= So, "depth" keeps its meaning "which level provides data".
<= /span>
And we add additional optional field like

absolutely-completely-absent: bool

hole: bool?


Which is true if data is nowhere in the backing chain.<= /span>


--
Best r= egards,
Vladimir
= --Apple-Mail-1E37F757-26E9-43F9-9073-1DED126039A6--