KVM: arm64: ... and FWB for all
Marc Zyngier
maintainercover KVM: arm64: ... and FWB for all
[Yes, you can tell what I was listening to]
Today, running in protected mode puts the host under it's own S2 with FWB=0, even if the rest of the guests are running with FWB=1. The rationale for this special-casing of the host is that we want the combined attributes to actually reflect the ones exposed by S1.
We therefore use NormalCacheable (which is the weakest memory type) as the default attributes at S2 so that S1 can only strengthen the final memory type.
But there is no reason why we cannot achieve the same effect with FWB. We normally use FWB to enforce cacheable memory from S2, irrespective of S1. But it is possible to configure the S2 attributes as "pass-through", so that the S1 attributes are always used.
This small series implements that change, adding a flag that actually describes what we are trying to do (instead of the NOFWB flag which is pretty obscure), and fixes an interesting gotcha with CMOs. Interestingly, CMOs never actually worked with NOFWB on an FWB-aware platform...
Patches on top of 6.19-rc5.
Marc Zyngier (6):
arm64: Add MT_S2{,_FWB}_AS_S1 encodings
KVM: arm64: Add KVM_PGTABLE_S2_AS_S1 flag
KVM: arm64: Make stage2_pte_cacheable() return false when S2_AS_S1 is
set
KVM: arm64: Switch pKVM host S2 over to KVM_PGTABLE_S2_AS_S1
KVM: arm64: Kill KVM_PGTABLE_S2_NOFWB
KVM: arm64: Simplify PAGE_S2_MEMATTR
arch/arm64/include/asm/kvm_pgtable.h | 7 +++--- arch/arm64/include/asm/memory.h | 11 ++++++--- arch/arm64/include/asm/pgtable-prot.h | 4 ++-- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 4 +++- arch/arm64/kvm/hyp/pgtable.c | 32 +++++++++++++++++++-------- 5 files changed, 39 insertions(+), 19 deletions(-)
Signature · 1 line
2.47.3
Marc Zyngier
maintainerPATCH 1/6 arm64: Add MT_S2{,_FWB}_AS_S1 encodings
pKVM usage of S2 translation on the host is purely for isolation purposes, not translation. To that effect, the memory attributes being used must be that of S1.
With FWB=0, this is easily achieved by using the Normal Cacheable type (which is the weakest possible memory type) at S2, and let S1 pick something stronger as required.
With FWB=1, the attributes are combined in a different way, and we cannot arbitrarily use Normal Cacheable. We can, however, use a memattr encoding that indicates that the final attributes are that of Stage-1.
Add these encoding and a few pointers to the relevant parts of the specification. It might come handy some day.
--- arch/arm64/include/asm/memory.h | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
arch/arm64/include/asm/memory.h +8 -3
@@ -175,19 +175,24 @@#define MT_DEVICE_nGnRE 4/* -* Memory types for Stage-2 translation +* Memory types for Stage-2 translation when HCR_EL2.FWB=0. See R_HMNDG, +* R_TNHFM, R_GQFSF and I_MCQKW for the details on how these attributes get +* combined with Stage-1. */ #define MT_S2_NORMAL 0xf#define MT_S2_NORMAL_NC 0x5#define MT_S2_DEVICE_nGnRE 0x1 +#define MT_S2_AS_S1 MT_S2_NORMAL/* -* Memory types for Stage-2 translation when ID_AA64MMFR2_EL1.FWB is 0001 -* Stage-2 enforces Normal-WB and Device-nGnRE +* Memory types for Stage-2 translation when HCR_EL2.FWB=1. Stage-2 enforces +* Normal-WB and Device-nGnRE, unless we actively say that S1 wins. See +* R_VRJSW and R_RHWZM for details. */ #define MT_S2_FWB_NORMAL 6#define MT_S2_FWB_NORMAL_NC 5#define MT_S2_FWB_DEVICE_nGnRE 1 +#define MT_S2_FWB_AS_S1 7#ifdef CONFIG_ARM64_4K_PAGES #define IOREMAP_MAX_ORDER (PUD_SHIFT)
Signature · 1 line
2.47.3
Re: [PATCH 1/6] arm64: Add MT_S2{,_FWB}_AS_S1 encodings
Hi Marc,
Quoted 9 lines · Marc Zyngier
pKVM usage of S2 translation on the host is purely for isolation purposes, not translation. To that effect, the memory attributes being used must be that of S1.
With FWB=0, this is easily achieved by using the Normal Cacheable type (which is the weakest possible memory type) at S2, and let S1 pick something stronger as required.
Quoted 7 lines
With FWB=1, the attributes are combined in a different way, and we cannot arbitrarily use Normal Cacheable. We can, however, use a memattr encoding that indicates that the final attributes are that of Stage-1.
Add these encoding and a few pointers to the relevant parts of the specification. It might come handy some day.
The Day that Never Comes?
Quoted 25 lines
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/include/asm/memory.h | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)arch/arm64/include/asm/memory.h +4
-1@@ -175,19 +175,24 @@#define MT_DEVICE_nGnRE 4/* -* Memory types for Stage-2 translation +* Memory types for Stage-2 translation when HCR_EL2.FWB=0. See R_HMNDG, +* R_TNHFM, R_GQFSF and I_MCQKW for the details on how these attributes get +* combined with Stage-1. */ #define MT_S2_NORMAL 0xf#define MT_S2_NORMAL_NC 0x5#define MT_S2_DEVICE_nGnRE 0x1 +#define MT_S2_AS_S1 MT_S2_NORMAL/*
arch/arm64/include/asm/memory.h line 195
Quoted 10 lines
arch/arm64/include/asm/memory.h +4
-2-* Memory types for Stage-2 translation when ID_AA64MMFR2_EL1.FWB is 0001 -* Stage-2 enforces Normal-WB and Device-nGnRE +* Memory types for Stage-2 translation when HCR_EL2.FWB=1. Stage-2 enforces +* Normal-WB and Device-nGnRE, unless we actively say that S1 wins. See +* R_VRJSW and R_RHWZM for details. */ #define MT_S2_FWB_NORMAL 6#define MT_S2_FWB_NORMAL_NC 5#define MT_S2_FWB_DEVICE_nGnRE 1 +#define MT_S2_FWB_AS_S1 7
Cheers,
/fuad
Quoted 6 lines
arch/arm64/include/asm/memory.h +0
-1#ifdef CONFIG_ARM64_4K_PAGES #define IOREMAP_MAX_ORDER (PUD_SHIFT) --2.47.3
Marc Zyngier
maintainerPATCH 2/6 KVM: arm64: Add KVM_PGTABLE_S2_AS_S1 flag
In reply to Marc ZyngierPlumb the MT_S2{,_FWB}_AS_S1 memory types into the KVM_S2_MEMATTR() macro with a new KVM_PGTABLE_S2_AS_S1 flag.
Nobody selects it yet.
--- arch/arm64/include/asm/kvm_pgtable.h | 2 ++ arch/arm64/kvm/hyp/pgtable.c | 13 ++++++++++++- 2 files changed, 14 insertions(+), 1 deletion(-)
arch/arm64/include/asm/kvm_pgtable.h +2 -0
@@ -232,10 +232,12 @@ struct kvm_pgtable_mm_ops {* @KVM_PGTABLE_S2_NOFWB: Don't enforce Normal-WB even if the CPUs have * ARM64_HAS_STAGE2_FWB. * @KVM_PGTABLE_S2_IDMAP: Only use identity mappings. +* @KVM_PGTABLE_S2_AS_S1: Final memory attributes are that of Stage-1. */ enum kvm_pgtable_stage2_flags {KVM_PGTABLE_S2_NOFWB =BIT (0 ),KVM_PGTABLE_S2_IDMAP =BIT (1 ), +KVM_PGTABLE_S2_AS_S1 =BIT (2 ), };/**
arch/arm64/kvm/hyp/pgtable.c +12 -1
@@ -659,7 +659,18 @@ void kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu, } } -#define KVM_S2_MEMATTR (pgt, attr) PAGE_S2_MEMATTR(attr, stage2_has_fwb(pgt)) +#define KVM_S2_MEMATTR (pgt, attr) \ + ({ \ + kvm_pte_t __attr; \ + if((pgt)->flags & KVM_PGTABLE_S2_AS_S1) \ + __attr = PAGE_S2_MEMATTR(AS_S1, \ + stage2_has_fwb(pgt)); \ + else \ + __attr = PAGE_S2_MEMATTR(attr, \ + stage2_has_fwb(pgt)); \ + \ + __attr; \ + })static int stage2_set_xn_attr (enum kvm_pgtable_prot prot,kvm_pte_t *attr) {
Signature · 1 line
2.47.3
Re: [PATCH 2/6] KVM: arm64: Add KVM_PGTABLE_S2_AS_S1 flag
Quoted 34 lines · Marc Zyngier
Plumb the MT_S2{,_FWB}_AS_S1 memory types into the
KVM_S2_MEMATTR()macro with a newKVM_PGTABLE_S2_AS_S1flag.Nobody selects it yet.
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/include/asm/kvm_pgtable.h | 2 ++ arch/arm64/kvm/hyp/pgtable.c | 13 ++++++++++++- 2 files changed, 14 insertions(+), 1 deletion(-)arch/arm64/include/asm/kvm_pgtable.h +2
-0@@ -232,10 +232,12 @@ struct kvm_pgtable_mm_ops {* @KVM_PGTABLE_S2_NOFWB: Don't enforce Normal-WB even if the CPUs have * ARM64_HAS_STAGE2_FWB. * @KVM_PGTABLE_S2_IDMAP: Only use identity mappings. +* @KVM_PGTABLE_S2_AS_S1: Final memory attributes are that of Stage-1. */ enum kvm_pgtable_stage2_flags {KVM_PGTABLE_S2_NOFWB =BIT (0 ),KVM_PGTABLE_S2_IDMAP =BIT (1 ), +KVM_PGTABLE_S2_AS_S1 =BIT (2 ), };/** arch/arm64/kvm/hyp/pgtable.c +0
-0
arch/arm64/kvm/hyp/pgtable.c void kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu, · line 664
Quoted 7 lines
arch/arm64/kvm/hyp/pgtable.c +3
-1@@ -659,7 +659,18 @@ void kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu, } } -#define KVM_S2_MEMATTR (pgt, attr) PAGE_S2_MEMATTR(attr, stage2_has_fwb(pgt)) +#define KVM_S2_MEMATTR (pgt, attr) \ + ({ \ + kvm_pte_t __attr; \
nit: new line?
Cheers,
/fuad
Quoted 15 lines
arch/arm64/kvm/hyp/pgtable.c +9
-1+if ((pgt)->flags &KVM_PGTABLE_S2_AS_S1 ) \ + __attr =PAGE_S2_MEMATTR (AS_S1 , \ +stage2_has_fwb (pgt)); \ +else \ + __attr =PAGE_S2_MEMATTR (attr, \ +stage2_has_fwb (pgt)); \ + \ + __attr; \ + })static int stage2_set_xn_attr (enum kvm_pgtable_prot prot,kvm_pte_t *attr) { --2.47.3
Will Deacon
maintainerRe: [PATCH 2/6] KVM: arm64: Add KVM_PGTABLE_S2_AS_S1 flag
Quoted 38 lines · Fuad Tabba
Quoted 37 lines · Marc Zyngier
Plumb the MT_S2{,_FWB}_AS_S1 memory types into the
KVM_S2_MEMATTR()macro with a newKVM_PGTABLE_S2_AS_S1flag.Nobody selects it yet.
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/include/asm/kvm_pgtable.h | 2 ++ arch/arm64/kvm/hyp/pgtable.c | 13 ++++++++++++- 2 files changed, 14 insertions(+), 1 deletion(-)arch/arm64/include/asm/kvm_pgtable.h +2
-0@@ -232,10 +232,12 @@ struct kvm_pgtable_mm_ops {* @KVM_PGTABLE_S2_NOFWB: Don't enforce Normal-WB even if the CPUs have * ARM64_HAS_STAGE2_FWB. * @KVM_PGTABLE_S2_IDMAP: Only use identity mappings. +* @KVM_PGTABLE_S2_AS_S1: Final memory attributes are that of Stage-1. */ enum kvm_pgtable_stage2_flags {KVM_PGTABLE_S2_NOFWB =BIT (0 ),KVM_PGTABLE_S2_IDMAP =BIT (1 ), +KVM_PGTABLE_S2_AS_S1 =BIT (2 ), };/** arch/arm64/kvm/hyp/pgtable.c +0
-0@@ -659,7 +659,18 @@ void kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu, } }
Quoted 8 lines
arch/arm64/kvm/hyp/pgtable.c void kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu, · line 664
Quoted 4 lines
arch/arm64/kvm/hyp/pgtable.c +3
-1-#define KVM_S2_MEMATTR (pgt, attr) PAGE_S2_MEMATTR(attr, stage2_has_fwb(pgt)) +#define KVM_S2_MEMATTR (pgt, attr) \ + ({ \ + kvm_pte_t __attr; \nit: new line?
Reviewed-by Fuad Tabba <[email protected]>
If you're going back to fix nits then:
Quoted 1 line
arch/arm64/kvm/hyp/pgtable.c void kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu, · line 665
Quoted 1 line
arch/arm64/kvm/hyp/pgtable.c +1
-0+if ((pgt)->flags &KVM_PGTABLE_S2_AS_S1 ) \
please stick a space after the 'if' as well :)
Will
Marc Zyngier
maintainerPATCH 3/6 KVM: arm64: Make stage2_pte_cacheable() return false when S2_AS_S1 is set
In reply to Marc ZyngierWe cannot tell from the S2 attributes whether what we map is memory or not when S2_AS_S1 is set, and issuing a CMO on device memory may not be the best idea.
In this particular case, pretend that the mapping isn't cacheable.
--- arch/arm64/kvm/hyp/pgtable.c | 8 ++++++++ 1 file changed, 8 insertions(+)
arch/arm64/kvm/hyp/pgtable.c +8 -0
@@ -911,6 +911,14 @@ static void stage2_unmap_put_pte(const struct kvm_pgtable_visit_ctx *ctx,static bool stage2_pte_cacheable (struct kvm_pgtable *pgt,kvm_pte_t pte) {u64 memattr = pte &KVM_PTE_LEAF_ATTR_LO_S2_MEMATTR ; + +/* +* With S2_AS_S1, we have no idea whether the OA is actual memory or +* a device. Don't even try a CMO on that. +*/ +if (pgt->flags &KVM_PGTABLE_S2_AS_S1 ) +return false; +return kvm_pte_valid (pte) && memattr ==KVM_S2_MEMATTR (pgt,NORMAL ); }
Signature · 1 line
2.47.3
Will Deacon
maintainerRe: [PATCH 3/6] KVM: arm64: Make stage2_pte_cacheable() return false when S2_AS_S1 is set
Quoted 18 lines · Marc Zyngier
We cannot tell from the S2 attributes whether what we map is memory or not when
S2_AS_S1is set, and issuing a CMO on device memory may not be the best idea.In this particular case, pretend that the mapping isn't cacheable.
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/kvm/hyp/pgtable.c | 8 ++++++++ 1 file changed, 8 insertions(+)arch/arm64/kvm/hyp/pgtable.c +0
-0@@ -911,6 +911,14 @@ static void stage2_unmap_put_pte(const struct kvm_pgtable_visit_ctx *ctx,static bool stage2_pte_cacheable (struct kvm_pgtable *pgt,kvm_pte_t pte) {
arch/arm64/kvm/hyp/pgtable.c static void stage2_unmap_put_pte(const struct kvm_pgtable_visit_ctx *ctx, · line 922
Quoted 10 lines
arch/arm64/kvm/hyp/pgtable.c +8
-0u64 memattr = pte &KVM_PTE_LEAF_ATTR_LO_S2_MEMATTR ; + +/* +* With S2_AS_S1, we have no idea whether the OA is actual memory or +* a device. Don't even try a CMO on that. +*/ +if (pgt->flags &KVM_PGTABLE_S2_AS_S1 ) +return false; +return kvm_pte_valid (pte) && memattr ==KVM_S2_MEMATTR (pgt,NORMAL );
Since KVM_PGTABLE_S2_AS_S1 is only set for the pKVM host stage-2 and that doesn't provide any cache maintenance callbacks in its kvm_pgtable_mm_ops, I don't think the current code is actually broken, although it's harmless to add the check (and the comment might benefit from some additional rewording?).
Will
Marc Zyngier
maintainerRe: [PATCH 3/6] KVM: arm64: Make stage2_pte_cacheable() return false when S2_AS_S1 is set
Quoted 26 lines · Will Deacon
Quoted 24 lines · Marc Zyngier
We cannot tell from the S2 attributes whether what we map is memory or not when
S2_AS_S1is set, and issuing a CMO on device memory may not be the best idea.In this particular case, pretend that the mapping isn't cacheable.
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/kvm/hyp/pgtable.c | 8 ++++++++ 1 file changed, 8 insertions(+)arch/arm64/kvm/hyp/pgtable.c +5
-0@@ -911,6 +911,14 @@ static void stage2_unmap_put_pte(const struct kvm_pgtable_visit_ctx *ctx,static bool stage2_pte_cacheable (struct kvm_pgtable *pgt,kvm_pte_t pte) {u64 memattr = pte &KVM_PTE_LEAF_ATTR_LO_S2_MEMATTR ; + +/* +* With S2_AS_S1, we have no idea whether the OA is actual memory or +* a device. Don't even try a CMO on that. +*/
Quoted 8 lines
arch/arm64/kvm/hyp/pgtable.c static void stage2_unmap_put_pte(const struct kvm_pgtable_visit_ctx *ctx, · line 922
Quoted 4 lines
arch/arm64/kvm/hyp/pgtable.c +3
-0+if (pgt->flags &KVM_PGTABLE_S2_AS_S1 ) +return false; +return kvm_pte_valid (pte) && memattr ==KVM_S2_MEMATTR (pgt,NORMAL );Since
KVM_PGTABLE_S2_AS_S1is only set for the pKVM host stage-2 and that doesn't provide any cache maintenance callbacks in itskvm_pgtable_mm_ops, I don't think the current code is actually broken,
Frankly, I have no idea how I came to such a stupid conclusion. My only way out is to blame Metallica (don't listen to that shit, kids!).
Quoted 2 lines
although it's harmless to add the check (and the comment might benefit from some additional rewording?).
I don't think there is much value in adding something that is not useful. I'll drop that patch altogether.
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Marc Zyngier
maintainerPATCH 4/6 KVM: arm64: Switch pKVM host S2 over to KVM_PGTABLE_S2_AS_S1
In reply to Marc ZyngierSince we have the basics to use the S1 memory attributes as the final ones with FWB, flip the host over to that when FWB is present.
--- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
arch/arm64/kvm/hyp/nvhe/mem_protect.c +3 -1
@@ -19,7 +19,7 @@#include <nvhe/mem_protect.h> #include <nvhe/mm.h> -#define KVM_HOST_S2_FLAGS (KVM_PGTABLE_S2_NOFWB | KVM_PGTABLE_S2_IDMAP) +#define KVM_HOST_S2_FLAGS (KVM_PGTABLE_S2_AS_S1 | KVM_PGTABLE_S2_IDMAP)struct host_mmu host_mmu;
@@ -324,6 +324,8 @@ int __pkvm_prot_finalize(void) params->vttbr =kvm_get_vttbr (mmu); params->vtcr = mmu->vtcr; params->hcr_el2 |=HCR_VM ; +if (cpus_have_final_cap (ARM64_HAS_STAGE2_FWB )) + params->hcr_el2 |=HCR_FWB ;/* * The CMO below not only cleans the updated params to the
Signature · 1 line
2.47.3
Re: [PATCH 4/6] KVM: arm64: Switch pKVM host S2 over to KVM_PGTABLE_S2_AS_S1
Quoted 23 lines · Marc Zyngier
Since we have the basics to use the S1 memory attributes as the final ones with FWB, flip the host over to that when FWB is present.
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)arch/arm64/kvm/hyp/nvhe/mem_protect.c +1
-1@@ -19,7 +19,7 @@#include <nvhe/mem_protect.h> #include <nvhe/mm.h> -#define KVM_HOST_S2_FLAGS (KVM_PGTABLE_S2_NOFWB | KVM_PGTABLE_S2_IDMAP) +#define KVM_HOST_S2_FLAGS (KVM_PGTABLE_S2_AS_S1 | KVM_PGTABLE_S2_IDMAP)struct host_mmu host_mmu;
arch/arm64/kvm/hyp/nvhe/mem_protect.c int __pkvm_prot_finalize(void) · line 328
Quoted 5 lines
arch/arm64/kvm/hyp/nvhe/mem_protect.c +2
-0@@ -324,6 +324,8 @@ int __pkvm_prot_finalize(void) params->vttbr =kvm_get_vttbr (mmu); params->vtcr = mmu->vtcr; params->hcr_el2 |=HCR_VM ; +if (cpus_have_final_cap (ARM64_HAS_STAGE2_FWB )) + params->hcr_el2 |=HCR_FWB ;
Cheers,
/fuad
Quoted 5 lines
arch/arm64/kvm/hyp/nvhe/mem_protect.c +0
-1/* * The CMO below not only cleans the updated params to the -- 2.47.3
Marc Zyngier
maintainerPATCH 5/6 KVM: arm64: Kill KVM_PGTABLE_S2_NOFWB
In reply to Marc ZyngierNobody is using this flag anymore, so remove it. This allows some cleanup for stage2_has_fwb(), whic doesn't need to take the pgt as a parameter anymore.
--- arch/arm64/include/asm/kvm_pgtable.h | 7 ++----- arch/arm64/kvm/hyp/pgtable.c | 17 +++++++---------- 2 files changed, 9 insertions(+), 15 deletions(-)
arch/arm64/include/asm/kvm_pgtable.h +2 -5
@@ -229,15 +229,12 @@ struct kvm_pgtable_mm_ops {/** * enum kvm_pgtable_stage2_flags - Stage-2 page-table flags. -* @KVM_PGTABLE_S2_NOFWB: Don't enforce Normal-WB even if the CPUs have -* ARM64_HAS_STAGE2_FWB. * @KVM_PGTABLE_S2_IDMAP: Only use identity mappings. * @KVM_PGTABLE_S2_AS_S1: Final memory attributes are that of Stage-1. */ enum kvm_pgtable_stage2_flags { -KVM_PGTABLE_S2_NOFWB =BIT (0 ), -KVM_PGTABLE_S2_IDMAP =BIT (1 ), -KVM_PGTABLE_S2_AS_S1 =BIT (2 ), +KVM_PGTABLE_S2_IDMAP =BIT (0 ), +KVM_PGTABLE_S2_AS_S1 =BIT (1 ), };/**
arch/arm64/kvm/hyp/pgtable.c +7 -10
@@ -631,12 +631,9 @@ u64 kvm_get_vtcr(u64 mmfr0, u64 mmfr1, u32 phys_shift)return vtcr; } -static bool stage2_has_fwb (struct kvm_pgtable *pgt) +static bool stage2_has_fwb (void ) { -if (!cpus_have_final_cap (ARM64_HAS_STAGE2_FWB )) -return false; - -return !(pgt->flags &KVM_PGTABLE_S2_NOFWB ); +return cpus_have_final_cap (ARM64_HAS_STAGE2_FWB ); } void kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu,
@@ -664,10 +661,10 @@ void kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu,kvm_pte_t __attr; \if ((pgt)->flags &KVM_PGTABLE_S2_AS_S1 ) \ __attr =PAGE_S2_MEMATTR (AS_S1 , \ -stage2_has_fwb (pgt)); \ +stage2_has_fwb ()); \else \ __attr =PAGE_S2_MEMATTR (attr, \ -stage2_has_fwb (pgt)); \ +stage2_has_fwb ()); \ \ __attr; \ })
@@ -879,7 +876,7 @@ static bool stage2_unmap_defer_tlb_flush(struct kvm_pgtable *pgt)* system supporting FWB as the optimization is entirely * pointless when the unmap walker needs to perform CMOs. */ -return system_supports_tlb_range () &&stage2_has_fwb (pgt); +return system_supports_tlb_range () &&stage2_has_fwb (); } static void stage2_unmap_put_pte(const struct kvm_pgtable_visit_ctx *ctx,
@@ -1167,7 +1164,7 @@ static int stage2_unmap_walker(const struct kvm_pgtable_visit_ctx *ctx,if (mm_ops->page_count (childp) !=1 )return 0 ; }else if (stage2_pte_cacheable (pgt, ctx->old)) { - need_flush = !stage2_has_fwb (pgt); + need_flush = !stage2_has_fwb (); }/*
@@ -1397,7 +1394,7 @@ int kvm_pgtable_stage2_flush(struct kvm_pgtable *pgt, u64 addr, u64 size) .arg = pgt, }; -if (stage2_has_fwb (pgt)) +if (stage2_has_fwb ())return 0 ;return kvm_pgtable_walk (pgt, addr, size, &walker);
Signature · 1 line
2.47.3
Re: [PATCH 5/6] KVM: arm64: Kill KVM_PGTABLE_S2_NOFWB
Quoted 3 lines · Marc Zyngier
Nobody is using this flag anymore, so remove it. This allows some cleanup for
stage2_has_fwb(), whic doesn't need to take the
nit: s/whic/which
Quoted 3 lines
pgt as a parameter anymore.
Signed-off-by Marc Zyngier <[email protected]>
Cheers,
/fuad
Quoted 89 lines
--- arch/arm64/include/asm/kvm_pgtable.h | 7 ++----- arch/arm64/kvm/hyp/pgtable.c | 17 +++++++---------- 2 files changed, 9 insertions(+), 15 deletions(-)arch/arm64/include/asm/kvm_pgtable.h +2
-5@@ -229,15 +229,12 @@ struct kvm_pgtable_mm_ops {/** * enum kvm_pgtable_stage2_flags - Stage-2 page-table flags. -* @KVM_PGTABLE_S2_NOFWB: Don't enforce Normal-WB even if the CPUs have -* ARM64_HAS_STAGE2_FWB. * @KVM_PGTABLE_S2_IDMAP: Only use identity mappings. * @KVM_PGTABLE_S2_AS_S1: Final memory attributes are that of Stage-1. */ enum kvm_pgtable_stage2_flags { -KVM_PGTABLE_S2_NOFWB =BIT (0 ), -KVM_PGTABLE_S2_IDMAP =BIT (1 ), -KVM_PGTABLE_S2_AS_S1 =BIT (2 ), +KVM_PGTABLE_S2_IDMAP =BIT (0 ), +KVM_PGTABLE_S2_AS_S1 =BIT (1 ), };/** arch/arm64/kvm/hyp/pgtable.c +7
-11@@ -631,12 +631,9 @@ u64 kvm_get_vtcr(u64 mmfr0, u64 mmfr1, u32 phys_shift)return vtcr; } -static bool stage2_has_fwb (struct kvm_pgtable *pgt) +static bool stage2_has_fwb (void ) { -if (!cpus_have_final_cap (ARM64_HAS_STAGE2_FWB )) -return false; - -return !(pgt->flags &KVM_PGTABLE_S2_NOFWB ); +return cpus_have_final_cap (ARM64_HAS_STAGE2_FWB ); } void kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu,@@ -664,10 +661,10 @@ void kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu,kvm_pte_t __attr; \if ((pgt)->flags &KVM_PGTABLE_S2_AS_S1 ) \ __attr =PAGE_S2_MEMATTR (AS_S1 , \ -stage2_has_fwb (pgt)); \ +stage2_has_fwb ()); \else \ __attr =PAGE_S2_MEMATTR (attr, \ -stage2_has_fwb (pgt)); \ +stage2_has_fwb ()); \ \ __attr; \ })@@ -879,7 +876,7 @@ static bool stage2_unmap_defer_tlb_flush(struct kvm_pgtable *pgt)* system supporting FWB as the optimization is entirely * pointless when the unmap walker needs to perform CMOs. */ -return system_supports_tlb_range () &&stage2_has_fwb (pgt); +return system_supports_tlb_range () &&stage2_has_fwb (); } static void stage2_unmap_put_pte(const struct kvm_pgtable_visit_ctx *ctx,@@ -1167,7 +1164,7 @@ static int stage2_unmap_walker(const struct kvm_pgtable_visit_ctx *ctx,if (mm_ops->page_count (childp) !=1 )return 0 ; }else if (stage2_pte_cacheable (pgt, ctx->old)) { - need_flush = !stage2_has_fwb (pgt); + need_flush = !stage2_has_fwb (); }/* @@ -1397,7 +1394,7 @@ int kvm_pgtable_stage2_flush(struct kvm_pgtable *pgt, u64 addr, u64 size) .arg = pgt, }; -if (stage2_has_fwb (pgt)) +if (stage2_has_fwb ())return 0 ;return kvm_pgtable_walk (pgt, addr, size, &walker); --2.47.3
Marc Zyngier
maintainerPATCH 6/6 KVM: arm64: Simplify PAGE_S2_MEMATTR
In reply to Marc ZyngierRestore PAGE_S2_MEMATTR() to its former glory, keeping the use of FWB as an implementation detail.
--- arch/arm64/include/asm/pgtable-prot.h | 4 ++-- arch/arm64/kvm/hyp/pgtable.c | 6 ++---- 2 files changed, 4 insertions(+), 6 deletions(-)
arch/arm64/include/asm/pgtable-prot.h +2 -2
@@ -109,10 +109,10 @@ static inline bool __pure lpa2_is_enabled(void)#define PAGE_KERNEL_EXEC __pgprot(_PAGE_KERNEL_EXEC)#define PAGE_KERNEL_EXEC_CONT __pgprot(_PAGE_KERNEL_EXEC_CONT) -#define PAGE_S2_MEMATTR (attr, has_fwb) \ +#define PAGE_S2_MEMATTR (attr) \ ({ \ u64 __val; \ - if (has_fwb) \ + if (cpus_have_final_cap(ARM64_HAS_STAGE2_FWB)) \ __val = PTE_S2_MEMATTR(MT_S2_FWB_ ## attr); \ else \ __val = PTE_S2_MEMATTR(MT_S2_ ## attr); \
arch/arm64/kvm/hyp/pgtable.c +2 -4
@@ -660,11 +660,9 @@ void kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu, ({ \kvm_pte_t __attr; \if ((pgt)->flags &KVM_PGTABLE_S2_AS_S1 ) \ - __attr =PAGE_S2_MEMATTR (AS_S1 , \ -stage2_has_fwb ()); \ + __attr =PAGE_S2_MEMATTR (AS_S1 ); \else \ - __attr =PAGE_S2_MEMATTR (attr, \ -stage2_has_fwb ()); \ + __attr =PAGE_S2_MEMATTR (attr); \ \ __attr; \ })
Signature · 1 line
2.47.3
Will Deacon
maintainerRe: [PATCH 6/6] KVM: arm64: Simplify PAGE_S2_MEMATTR
Quoted 32 lines · Marc Zyngier
Restore
PAGE_S2_MEMATTR()to its former glory, keeping the use of FWB as an implementation detail.Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/include/asm/pgtable-prot.h | 4 ++-- arch/arm64/kvm/hyp/pgtable.c | 6 ++---- 2 files changed, 4 insertions(+), 6 deletions(-)arch/arm64/include/asm/pgtable-prot.h +2
-2@@ -109,10 +109,10 @@ static inline bool __pure lpa2_is_enabled(void)#define PAGE_KERNEL_EXEC __pgprot(_PAGE_KERNEL_EXEC)#define PAGE_KERNEL_EXEC_CONT __pgprot(_PAGE_KERNEL_EXEC_CONT) -#define PAGE_S2_MEMATTR (attr, has_fwb) \ +#define PAGE_S2_MEMATTR (attr) \ ({ \ u64 __val; \ - if (has_fwb) \ + if (cpus_have_final_cap(ARM64_HAS_STAGE2_FWB)) \ __val = PTE_S2_MEMATTR(MT_S2_FWB_ ## attr); \ else \ __val = PTE_S2_MEMATTR(MT_S2_ ## attr); \arch/arm64/kvm/hyp/pgtable.c +0
-0@@ -660,11 +660,9 @@ void kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu, ({ \
arch/arm64/kvm/hyp/pgtable.c void kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu, · line 666
Quoted 10 lines
arch/arm64/kvm/hyp/pgtable.c +2
-4kvm_pte_t __attr; \if ((pgt)->flags &KVM_PGTABLE_S2_AS_S1 ) \ - __attr =PAGE_S2_MEMATTR (AS_S1 , \ -stage2_has_fwb ()); \ + __attr =PAGE_S2_MEMATTR (AS_S1 ); \else \ - __attr =PAGE_S2_MEMATTR (attr, \ -stage2_has_fwb ()); \ + __attr =PAGE_S2_MEMATTR (attr); \ \
tbh, I'd probably just kill stage2_has_fwb() entirely and use the cap everywhere...
Up to you.
Will
After a crash course in FWB,
Quoted 16 lines · Marc Zyngier
[Yes, you can tell what I was listening to]
Today, running in protected mode puts the host under it's own S2 with FWB=0, even if the rest of the guests are running with FWB=1. The rationale for this special-casing of the host is that we want the combined attributes to actually reflect the ones exposed by S1.
We therefore use NormalCacheable (which is the weakest memory type) as the default attributes at S2 so that S1 can only strengthen the final memory type.
But there is no reason why we cannot achieve the same effect with FWB. We normally use FWB to enforce cacheable memory from S2, irrespective of S1. But it is possible to configure the S2 attributes as "pass-through", so that the S1 attributes are always used.
Quoted 5 lines
This small series implements that change, adding a flag that actually describes what we are trying to do (instead of the NOFWB flag which is pretty obscure), and fixes an interesting gotcha with CMOs. Interestingly, CMOs never actually worked with NOFWB on an FWB-aware platform...
Thanks,
Joey
Quoted 22 lines
Patches on top of 6.19-rc5.
Marc Zyngier (6):
arm64: Add MT_S2{,_FWB}_AS_S1 encodings
KVM: arm64: AddKVM_PGTABLE_S2_AS_S1flag
KVM: arm64: Makestage2_pte_cacheable()return false whenS2_AS_S1is
set
KVM: arm64: Switch pKVM host S2 over toKVM_PGTABLE_S2_AS_S1
KVM: arm64: KillKVM_PGTABLE_S2_NOFWB
KVM: arm64: SimplifyPAGE_S2_MEMATTRarch/arm64/include/asm/kvm_pgtable.h | 7 +++--- arch/arm64/include/asm/memory.h | 11 ++++++--- arch/arm64/include/asm/pgtable-prot.h | 4 ++-- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 4 +++- arch/arm64/kvm/hyp/pgtable.c | 32 +++++++++++++++++++-------- 5 files changed, 39 insertions(+), 19 deletions(-)Signature · 1 line
2.47.3
Hi Marc,
Quoted 25 lines · Marc Zyngier
[Yes, you can tell what I was listening to]
Today, running in protected mode puts the host under it's own S2 with FWB=0, even if the rest of the guests are running with FWB=1. The rationale for this special-casing of the host is that we want the combined attributes to actually reflect the ones exposed by S1.
We therefore use NormalCacheable (which is the weakest memory type) as the default attributes at S2 so that S1 can only strengthen the final memory type.
But there is no reason why we cannot achieve the same effect with FWB. We normally use FWB to enforce cacheable memory from S2, irrespective of S1. But it is possible to configure the S2 attributes as "pass-through", so that the S1 attributes are always used.
This small series implements that change, adding a flag that actually describes what we are trying to do (instead of the NOFWB flag which is pretty obscure), and fixes an interesting gotcha with CMOs. Interestingly, CMOs never actually worked with NOFWB on an FWB-aware platform...
Patches on top of 6.19-rc5.
Quoted 8 lines
Marc Zyngier (6):
arm64: Add MT_S2{,_FWB}_AS_S1 encodings
KVM: arm64: AddKVM_PGTABLE_S2_AS_S1flag
KVM: arm64: Makestage2_pte_cacheable()return false whenS2_AS_S1is
set
KVM: arm64: Switch pKVM host S2 over toKVM_PGTABLE_S2_AS_S1
KVM: arm64: KillKVM_PGTABLE_S2_NOFWB
KVM: arm64: SimplifyPAGE_S2_MEMATTR
Glad to see the host is no longer pulling the shortest straw on memory attributes. ;)
I've applied these patches on top of the pKVM Android branch (with one small fix to the Android pKVM code), and tested it with qemu on cpu max (fwb) neoverse-n1 and cortex-a72 (no fwb).
Stay tuned for the reviews...
Cheers,
/fuad
Quoted 10 lines
arch/arm64/include/asm/kvm_pgtable.h | 7 +++--- arch/arm64/include/asm/memory.h | 11 ++++++--- arch/arm64/include/asm/pgtable-prot.h | 4 ++-- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 4 +++- arch/arm64/kvm/hyp/pgtable.c | 32 +++++++++++++++++++-------- 5 files changed, 39 insertions(+), 19 deletions(-)--
2.47.3
Hi Marc,
Quoted 7 lines · Marc Zyngier
[Yes, you can tell what I was listening to]
Today, running in protected mode puts the host under it's own S2 with FWB=0, even if the rest of the guests are running with FWB=1. The rationale for this special-casing of the host is that we want the combined attributes to actually reflect the ones exposed by S1.
Quoted 8 lines
We therefore use NormalCacheable (which is the weakest memory type) as the default attributes at S2 so that S1 can only strengthen the final memory type.
But there is no reason why we cannot achieve the same effect with FWB. We normally use FWB to enforce cacheable memory from S2, irrespective of S1. But it is possible to configure the S2 attributes as "pass-through", so that the S1 attributes are always used.
Would you mind clarifying why FWB is preferable? Is it so that pkvm uses the same approach for configuring stage 2 for the host as for a guest? Or is it something else?
Quoted 6 lines
This small series implements that change, adding a flag that actually describes what we are trying to do (instead of the NOFWB flag which is pretty obscure), and fixes an interesting gotcha with CMOs. Interestingly, CMOs never actually worked with NOFWB on an FWB-aware platform...
Hmm... as far I can tell pkvm doesn't populate mm_ops->dcache_clean_inval_poc for host_mmu, so pKVM doesn't issue CMOs for the host s2, with or without these changes.
Thanks,
Alex
Quoted 23 lines
Patches on top of 6.19-rc5.
Marc Zyngier (6):
arm64: Add MT_S2{,_FWB}_AS_S1 encodings
KVM: arm64: AddKVM_PGTABLE_S2_AS_S1flag
KVM: arm64: Makestage2_pte_cacheable()return false whenS2_AS_S1is
set
KVM: arm64: Switch pKVM host S2 over toKVM_PGTABLE_S2_AS_S1
KVM: arm64: KillKVM_PGTABLE_S2_NOFWB
KVM: arm64: SimplifyPAGE_S2_MEMATTRarch/arm64/include/asm/kvm_pgtable.h | 7 +++--- arch/arm64/include/asm/memory.h | 11 ++++++--- arch/arm64/include/asm/pgtable-prot.h | 4 ++-- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 4 +++- arch/arm64/kvm/hyp/pgtable.c | 32 +++++++++++++++++++-------- 5 files changed, 39 insertions(+), 19 deletions(-)Signature · 1 line
2.47.3
Marc Zyngier
maintainerQuoted 15 lines · Alexandru Elisei
Hi Marc,
Quoted 11 lines · Marc Zyngier
[Yes, you can tell what I was listening to]
Today, running in protected mode puts the host under it's own S2 with FWB=0, even if the rest of the guests are running with FWB=1. The rationale for this special-casing of the host is that we want the combined attributes to actually reflect the ones exposed by S1.
We therefore use NormalCacheable (which is the weakest memory type) as the default attributes at S2 so that S1 can only strengthen the final memory type.
Quoted 8 lines
Quoted 4 lines
But there is no reason why we cannot achieve the same effect with FWB. We normally use FWB to enforce cacheable memory from S2, irrespective of S1. But it is possible to configure the S2 attributes as "pass-through", so that the S1 attributes are always used.
Would you mind clarifying why FWB is preferable? Is it so that pkvm uses the same approach for configuring stage 2 for the host as for a guest? Or is it something else?
Having only one attribute encoding scheme to think about on a given machine is vastly preferable, specially when you are debugging. Additionally, FWB==0 may not be with us forever.
Quoted 9 lines
Quoted 5 lines
This small series implements that change, adding a flag that actually describes what we are trying to do (instead of the NOFWB flag which is pretty obscure), and fixes an interesting gotcha with CMOs. Interestingly, CMOs never actually worked with NOFWB on an FWB-aware platform...
Hmm... as far I can tell pkvm doesn't populate
mm_ops->dcache_clean_inval_pocforhost_mmu, so pKVM doesn't issue CMOs for the host s2, with or without these changes.
Yeah, I clearly was talking nonsense. I don't know how I came to this silly conclusion.
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Hi Marc,
Quoted 22 lines · Marc Zyngier
Quoted 20 lines · Alexandru Elisei
Hi Marc,
Quoted 15 lines · Marc Zyngier
[Yes, you can tell what I was listening to]
Today, running in protected mode puts the host under it's own S2 with FWB=0, even if the rest of the guests are running with FWB=1. The rationale for this special-casing of the host is that we want the combined attributes to actually reflect the ones exposed by S1.
We therefore use NormalCacheable (which is the weakest memory type) as the default attributes at S2 so that S1 can only strengthen the final memory type.
But there is no reason why we cannot achieve the same effect with FWB. We normally use FWB to enforce cacheable memory from S2, irrespective of S1. But it is possible to configure the S2 attributes as "pass-through", so that the S1 attributes are always used.
Quoted 7 lines
Quoted 3 lines
Would you mind clarifying why FWB is preferable? Is it so that pkvm uses the same approach for configuring stage 2 for the host as for a guest? Or is it something else?
Having only one attribute encoding scheme to think about on a given machine is vastly preferable, specially when you are debugging. Additionally, FWB==0 may not be with us forever.
Got it, thanks for the explanation!
Thanks,
Alex