Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9ADA613C801 for ; Tue, 28 May 2024 07:58:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716883087; cv=none; b=Mhwt9TjsqaWMRQAXPU8CxMtA5DK9m6kJzH693Smqut6As8aOhwxTLZmomm+blBXNUetmvPSnfodlhVggcaKzv3uEs8I7Cmy8xxr4TscDPAi/3Vr4SoIYRS9W6sXKsAFE0N4TURVwhw6QpVKiax5oNq5wKlfzaPGunbYuBFjFNFA= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716883087; c=relaxed/simple; bh=rDiboxDdzeGhERGhVAFrCsaWNO2x/FdCE95ENbIFRCg=; h=Date:Message-ID:From:To:Cc:Subject:In-Reply-To:References: MIME-Version:Content-Type; b=f/T3E/tBnXfNQ9R4Hwl70xug4NrXzF8t5IjFM32C3oRBqbDZWIfRDOQQU+ZS/4zT5fCcTtkdiYMuvef1PqicnvdhTFuTzNn33zhLhUzCYv5jamsK5YDrOXS/lJO+TSEVmRtsberwK4AOh42en78TzcVMgR+WeaRdy7fkC7gEsQM= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=TWfvRLPn; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="TWfvRLPn" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1C113C3277B; Tue, 28 May 2024 07:58:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1716883087; bh=rDiboxDdzeGhERGhVAFrCsaWNO2x/FdCE95ENbIFRCg=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=TWfvRLPnx2KW+hJs9iigUvWAfGeHRMKNsaXkKPp/COJU6FlXfJ0WZusQlCd0cRIo/ qgHIi/ZI6vuYaezt/BeubrjZA67UA0JmTTWj69MR6W37dMt+KkoO9b++AkVD3vqatd Lm4r+QuMnHrMc/l/txrTJc4sYFMGD+frG+fiTVCAFUYwlsXLZneYp0ueZgBLJbLCSD vr8W83VsgcoHP9Oq90cj3XNbz0TuoCdLEyWkHa+jvAYqiHHhDCrS9R4MUbl7+pv2VS mIoqubGcaeLpyLM54f9MQo+zvHGO2gOkSqj027LElth6rBiqlpwIExnnGxicHsuwlf dfxw9/9JJpXNQ== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1sBrie-00GDQz-NJ; Tue, 28 May 2024 08:58:04 +0100 Date: Tue, 28 May 2024 08:58:04 +0100 Message-ID: <86ikyymkqb.wl-maz@kernel.org> From: Marc Zyngier To: Fuad Tabba Cc: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, will@kernel.org, qperret@google.com, seanjc@google.com, alexandru.elisei@arm.com, catalin.marinas@arm.com, philmd@linaro.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, mark.rutland@arm.com, broonie@kernel.org, joey.gouly@arm.com, rananta@google.com, yuzenghui@huawei.com Subject: Re: [PATCH v2 2/7] KVM: arm64: Abstract set/clear of CPTR_EL2 bits behind helper In-Reply-To: References: <20240521163720.3812851-1-tabba@google.com> <20240521163720.3812851-3-tabba@google.com> <86seyana8x.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/29.2 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) Precedence: bulk X-Mailing-List: kvmarm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: tabba@google.com, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, will@kernel.org, qperret@google.com, seanjc@google.com, alexandru.elisei@arm.com, catalin.marinas@arm.com, philmd@linaro.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, mark.rutland@arm.com, broonie@kernel.org, joey.gouly@arm.com, rananta@google.com, yuzenghui@huawei.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false On Wed, 22 May 2024 14:48:19 +0100, Fuad Tabba wrote: >=20 > Hi Marc, >=20 > On Tue, May 21, 2024 at 10:08=E2=80=AFPM Marc Zyngier wr= ote: > > > > On Tue, 21 May 2024 17:37:15 +0100, > > Fuad Tabba wrote: > > > > > > The same traps controlled by CPTR_EL2 or CPACR_EL1 need to be > > > toggled in different parts of the code, but the exact bits and > > > their polarity differ between these two formats and the mode > > > (vhe/nvhe/hvhe). > > > > > > To reduce the amount of duplicated code and the chance of getting > > > the wrong bit/polarity or missing a field, abstract the set/clear > > > of CPTR_EL2 bits behind a helper. > > > > > > Since (h)VHE is the way of the future, use the CPACR_EL1 format, > > > which is a subset of the VHE CPTR_EL2, as a reference. > > > > > > No functional change intended. > > > > > > Suggested-by: Oliver Upton > > > Signed-off-by: Fuad Tabba > > > --- > > > arch/arm64/include/asm/kvm_emulate.h | 34 +++++++++++++++++++++++= ++ > > > arch/arm64/kvm/hyp/include/hyp/switch.h | 17 +++---------- > > > arch/arm64/kvm/hyp/nvhe/hyp-main.c | 6 +---- > > > 3 files changed, 39 insertions(+), 18 deletions(-) > > > > > > diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/includ= e/asm/kvm_emulate.h > > > index 501e3e019c93..74837d1762e5 100644 > > > --- a/arch/arm64/include/asm/kvm_emulate.h > > > +++ b/arch/arm64/include/asm/kvm_emulate.h > > > @@ -557,6 +557,40 @@ static __always_inline void kvm_incr_pc(struct k= vm_vcpu *vcpu) > > > vcpu_set_flag((v), e); = \ > > > } while (0) > > > > > > + > > > +static inline void __cptr_clear_set_nvhe(u64 cpacr_clr, u64 cpacr_se= t) > > > +{ > > > + u64 clr =3D 0, set =3D 0; > > > + > > > + if (cpacr_clr & CPACR_ELx_FPEN) > > > + set |=3D CPTR_EL2_TFP; > > > + if (cpacr_clr & CPACR_ELx_ZEN) > > > + set |=3D CPTR_EL2_TZ; > > > + if (cpacr_clr & CPACR_ELx_SMEN) > > > > These 3 fields are actually pairs of bits. Can we have a compile-time > > check that both bits are set? >=20 > Yes. >=20 > > > + set |=3D CPTR_EL2_TSM; > > > + if (cpacr_clr & CPACR_ELx_TTA) > > > + clr |=3D CPTR_EL2_TTA; > > > > How about TCPAC, TAM, and E0POE? >=20 > CPTR_EL2 (nvhe) doesn't have an equivalent to E0POE. Should I have a > compile time check that it's not cleared at all (since it has negative > polarity), or a runtime check for that only for nvhe? You're right, and I ended figuring that out while reworking the NV side of CPTR_EL2 handling. >=20 > As for TCPAC, TAM, they're not defined under the CPACR bits, but they > are in CPTR_EL2, which is what CPACR refers to in VHE. I will handle > them, but should I rename cpacr_to_cptr() to cptr_vhe_to_nvhe()? I think this is clear enough in context, but I don't mind either way. >=20 > > > > > + > > > + if (cpacr_set & CPACR_ELx_FPEN) > > > + clr |=3D CPTR_EL2_TFP; > > > + if (cpacr_set & CPACR_ELx_ZEN) > > > + clr |=3D CPTR_EL2_TZ; > > > + if (cpacr_set & CPACR_ELx_SMEN) > > > + clr |=3D CPTR_EL2_TSM; > > > + if (cpacr_set & CPACR_ELx_TTA) > > > + set |=3D CPTR_EL2_TTA; > > > > The duplication is pretty unfortunate. Having a single helper that > > translate a register layout into another would be better. > > > + > > > + sysreg_clear_set(cptr_el2, clr, set); > > > > And omit this... > > > > > +} > > > + > > > +static inline void cpacr_clear_set(u64 clr, u64 set) > > > +{ > > > + if (has_vhe() || has_hvhe()) > > > + sysreg_clear_set(cpacr_el1, clr, set); > > > + else > > > + __cptr_clear_set_nvhe(clr, set); > > > > So that this could read as: > > > > sysreg_clear_set(cptr_el2, cpacr_to_cptr(clr), cpacr_to_cptr(se= t)); >=20 > I don't know how this could work with only one function/macro, while > covering all the fields. The problem is that the polarity of these > bits vary: some have the same polarity between CPACR/CPTR. and others > have the opposite polarity. Therefore, the conversion to `clr` is > different from the conversion to `set`. Ah, you're absolutely right. I totally glanced over the fact that the polarities are inverted, totally ruining the fun. >=20 > Say we need to clear both CPACR_ELx_FPEN and CPACR_ELx_TTA. The > problem is that CPACR_ELx_FPEN has the opposite polarity as > CPTR_EL2_TFP, whereas CPACR_ELx_TTA has the same polarity as > CPTR_EL2_TTA (different position though). >=20 > What would cpacr_to_cptr() return that would work in the code you propose: > sysreg_clear_set(cptr_el2, > cpacr_to_cptr(CPACR_ELx_FPEN|CPACR_ELx_TTA), cpacr_to_cptr(0)); >=20 > Desired result: > sysreg_clear_set(cptr_el2, CPTR_EL2_TTA, CPTR_EL2_TFP); >=20 > I can make it work with two functions/macros: > __cpacr_to_cptr_clear(clr, set) and __cpacr_to_cptr_set(clr, set) --> > sysreg_clear_set(cptr_el2, __cpacr_to_cptr_clear(clr, set), > __cpacr_to_cptr_set(clr, set)); >=20 > Unfortunately this doesn't get rid of the duplication, but it might be > tidier (I converted them to macros to add compile-time checks later): >=20 > +#define __cpacr_to_cptr_clr(clr, set) \ > + ({ \ > + u64 cptr =3D 0; = \ > + \ > + if ((set) & CPACR_ELx_FPEN) \ > + cptr |=3D CPTR_EL2_TFP; = \ > + if ((set) & CPACR_ELx_ZEN) \ > + cptr |=3D CPTR_EL2_TZ; = \ > + if ((set) & CPACR_ELx_SMEN) \ > + cptr |=3D CPTR_EL2_TSM; = \ > + if ((clr) & CPACR_ELx_TTA) \ > + cptr |=3D CPTR_EL2_TTA; = \ > + if ((clr) & CPTR_EL2_TAM) \ > + cptr |=3D CPTR_EL2_TAM; = \ > + if ((clr) & CPTR_EL2_TCPAC) \ > + cptr |=3D CPTR_EL2_TCPAC; = \ > + \ > + cptr; \ > + }) > + > +#define __cpacr_to_cptr_set(clr, set) \ > + ({ \ > + u64 cptr =3D 0; = \ > + \ > + if ((clr) & CPACR_ELx_FPEN) \ > + cptr |=3D CPTR_EL2_TFP; = \ > + if ((clr) & CPACR_ELx_ZEN) \ > + cptr |=3D CPTR_EL2_TZ; = \ > + if ((clr) & CPACR_ELx_SMEN) \ > + cptr |=3D CPTR_EL2_TSM; = \ > + if ((set) & CPACR_ELx_TTA) \ > + cptr |=3D CPTR_EL2_TTA; = \ > + if ((set) & CPTR_EL2_TAM) \ > + cptr |=3D CPTR_EL2_TAM; = \ > + if ((set) & CPTR_EL2_TCPAC) \ > + cptr |=3D CPTR_EL2_TCPAC; = \ > + \ > + cptr; \ > + }) > + > +#define cpacr_clear_set(clr, set) \ > + do { \ > + if (has_vhe() || has_hvhe()) \ > + sysreg_clear_set(cpacr_el1, clr, set); \ > + else \ > + sysreg_clear_set(cptr_el2, \ > + __cpacr_to_cptr_clr(clr, set), \ > + __cpacr_to_cptr_set(clr, set));\ > + } while (0) >=20 > What do you think? I quite like this. It is a large body of code, but it is easy to read and symmetric. Thanks, M. --=20 Without deviation from the norm, progress is not possible.