Skip to contentSkip to conversation

KVM: arm64: ... and FWB for all

PATCH Superseded Marc Zyngier 5 participants 20 messages Jan 19 – Jan 23

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Marc Zyngier

author maintainer
#0 lore raw

cover KVM: arm64: ... and FWB for all

[Yes, you can tell what I was listening to]

Today, running in protected mode puts the host under it's own S2 with FWB=0, even if the rest of the guests are running with FWB=1. The rationale for this special-casing of the host is that we want the combined attributes to actually reflect the ones exposed by S1.

We therefore use NormalCacheable (which is the weakest memory type) as the default attributes at S2 so that S1 can only strengthen the final memory type.

But there is no reason why we cannot achieve the same effect with FWB. We normally use FWB to enforce cacheable memory from S2, irrespective of S1. But it is possible to configure the S2 attributes as "pass-through", so that the S1 attributes are always used.

This small series implements that change, adding a flag that actually describes what we are trying to do (instead of the NOFWB flag which is pretty obscure), and fixes an interesting gotcha with CMOs. Interestingly, CMOs never actually worked with NOFWB on an FWB-aware platform...

Patches on top of 6.19-rc5.

Marc Zyngier (6):
arm64: Add MT_S2{,_FWB}_AS_S1 encodings
KVM: arm64: Add KVM_PGTABLE_S2_AS_S1 flag
KVM: arm64: Make stage2_pte_cacheable() return false when S2_AS_S1 is
set
KVM: arm64: Switch pKVM host S2 over to KVM_PGTABLE_S2_AS_S1
KVM: arm64: Kill KVM_PGTABLE_S2_NOFWB
KVM: arm64: Simplify PAGE_S2_MEMATTR

 arch/arm64/include/asm/kvm_pgtable.h  |  7 +++---
 arch/arm64/include/asm/memory.h       | 11 ++++++---
 arch/arm64/include/asm/pgtable-prot.h |  4 ++--
 arch/arm64/kvm/hyp/nvhe/mem_protect.c |  4 +++-
 arch/arm64/kvm/hyp/pgtable.c          | 32 +++++++++++++++++++--------
 5 files changed, 39 insertions(+), 19 deletions(-)
Signature · 1 line
2.47.3
author maintainer
#1 lore raw

PATCH 1/6 arm64: Add MT_S2{,_FWB}_AS_S1 encodings

pKVM usage of S2 translation on the host is purely for isolation purposes, not translation. To that effect, the memory attributes being used must be that of S1.

With FWB=0, this is easily achieved by using the Normal Cacheable type (which is the weakest possible memory type) at S2, and let S1 pick something stronger as required.

With FWB=1, the attributes are combined in a different way, and we cannot arbitrarily use Normal Cacheable. We can, however, use a memattr encoding that indicates that the final attributes are that of Stage-1.

Add these encoding and a few pointers to the relevant parts of the specification. It might come handy some day.

---
 arch/arm64/include/asm/memory.h | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)
arch/arm64/include/asm/memory.h +8 -3
index 9d54b2ea49d66..a2b7a33966ff1 100644
@@ -175,19 +175,24 @@
 #define MT_DEVICE_nGnRE		4
 
 /*
- * Memory types for Stage-2 translation
+ * Memory types for Stage-2 translation when HCR_EL2.FWB=0. See R_HMNDG,
+ * R_TNHFM, R_GQFSF and I_MCQKW for the details on how these attributes get
+ * combined with Stage-1.
  */
 #define MT_S2_NORMAL		0xf
 #define MT_S2_NORMAL_NC		0x5
 #define MT_S2_DEVICE_nGnRE	0x1
+#define MT_S2_AS_S1		MT_S2_NORMAL
 
 /*
- * Memory types for Stage-2 translation when ID_AA64MMFR2_EL1.FWB is 0001
- * Stage-2 enforces Normal-WB and Device-nGnRE
+ * Memory types for Stage-2 translation when HCR_EL2.FWB=1. Stage-2 enforces
+ * Normal-WB and Device-nGnRE, unless we actively say that S1 wins. See
+ * R_VRJSW and R_RHWZM for details.
  */
 #define MT_S2_FWB_NORMAL	6
 #define MT_S2_FWB_NORMAL_NC	5
 #define MT_S2_FWB_DEVICE_nGnRE	1
+#define MT_S2_FWB_AS_S1		7
 
 #ifdef CONFIG_ARM64_4K_PAGES
 #define IOREMAP_MAX_ORDER	(PUD_SHIFT)
Signature · 1 line
2.47.3
Reviewed-by on v1 1/6
#2 lore raw

Re: [PATCH 1/6] arm64: Add MT_S2{,_FWB}_AS_S1 encodings

Hi Marc,

Quoted 9 lines · Marc Zyngier

pKVM usage of S2 translation on the host is purely for isolation purposes, not translation. To that effect, the memory attributes being used must be that of S1.

With FWB=0, this is easily achieved by using the Normal Cacheable type (which is the weakest possible memory type) at S2, and let S1 pick something stronger as required.

Quoted 7 lines

With FWB=1, the attributes are combined in a different way, and we cannot arbitrarily use Normal Cacheable. We can, however, use a memattr encoding that indicates that the final attributes are that of Stage-1.

Add these encoding and a few pointers to the relevant parts of the specification. It might come handy some day.

The Day that Never Comes?

Quoted 25 lines
---
 arch/arm64/include/asm/memory.h | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)
arch/arm64/include/asm/memory.h +4 -1
index 9d54b2ea49d66..a2b7a33966ff1 100644
@@ -175,19 +175,24 @@
 #define MT_DEVICE_nGnRE                4
 
 /*
- * Memory types for Stage-2 translation
+ * Memory types for Stage-2 translation when HCR_EL2.FWB=0. See R_HMNDG,
+ * R_TNHFM, R_GQFSF and I_MCQKW for the details on how these attributes get
+ * combined with Stage-1.
  */
 #define MT_S2_NORMAL           0xf
 #define MT_S2_NORMAL_NC                0x5
 #define MT_S2_DEVICE_nGnRE     0x1
+#define MT_S2_AS_S1            MT_S2_NORMAL
 
 /*

arch/arm64/include/asm/memory.h line 195

Quoted 10 lines
arch/arm64/include/asm/memory.h +4 -2
- * Memory types for Stage-2 translation when ID_AA64MMFR2_EL1.FWB is 0001
- * Stage-2 enforces Normal-WB and Device-nGnRE
+ * Memory types for Stage-2 translation when HCR_EL2.FWB=1. Stage-2 enforces
+ * Normal-WB and Device-nGnRE, unless we actively say that S1 wins. See
+ * R_VRJSW and R_RHWZM for details.
  */
 #define MT_S2_FWB_NORMAL       6
 #define MT_S2_FWB_NORMAL_NC    5
 #define MT_S2_FWB_DEVICE_nGnRE 1
+#define MT_S2_FWB_AS_S1                7

Cheers,
/fuad

Quoted 6 lines
arch/arm64/include/asm/memory.h +0 -1
 #ifdef CONFIG_ARM64_4K_PAGES
 #define IOREMAP_MAX_ORDER      (PUD_SHIFT)
--

2.47.3

author maintainer
#3 lore raw

PATCH 2/6 KVM: arm64: Add KVM_PGTABLE_S2_AS_S1 flag

In reply to Marc Zyngier

Plumb the MT_S2{,_FWB}_AS_S1 memory types into the KVM_S2_MEMATTR() macro with a new KVM_PGTABLE_S2_AS_S1 flag.

Nobody selects it yet.

---
 arch/arm64/include/asm/kvm_pgtable.h |  2 ++
 arch/arm64/kvm/hyp/pgtable.c         | 13 ++++++++++++-
 2 files changed, 14 insertions(+), 1 deletion(-)
arch/arm64/include/asm/kvm_pgtable.h +2 -0
index fc02de43c68dd..9ce51a637da0a 100644
@@ -232,10 +232,12 @@ struct kvm_pgtable_mm_ops {
  * @KVM_PGTABLE_S2_NOFWB:	Don't enforce Normal-WB even if the CPUs have
  *				ARM64_HAS_STAGE2_FWB.
  * @KVM_PGTABLE_S2_IDMAP:	Only use identity mappings.
+ * @KVM_PGTABLE_S2_AS_S1:	Final memory attributes are that of Stage-1.
  */
 enum kvm_pgtable_stage2_flags {
 	KVM_PGTABLE_S2_NOFWB			= BIT(0),
 	KVM_PGTABLE_S2_IDMAP			= BIT(1),
+	KVM_PGTABLE_S2_AS_S1			= BIT(2),
 };
 
 /**
arch/arm64/kvm/hyp/pgtable.c +12 -1
index 947ac1a951a5b..3c5f399b1b986 100644
@@ -659,7 +659,18 @@ void kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu,
 	}
 }
 
-#define KVM_S2_MEMATTR(pgt, attr) PAGE_S2_MEMATTR(attr, stage2_has_fwb(pgt))
+#define KVM_S2_MEMATTR(pgt, attr)					\
+	({								\
+		kvm_pte_t __attr;					\
+		if((pgt)->flags & KVM_PGTABLE_S2_AS_S1)			\
+			__attr = PAGE_S2_MEMATTR(AS_S1,			\
+						 stage2_has_fwb(pgt));	\
+		else							\
+			__attr = PAGE_S2_MEMATTR(attr,			\
+						 stage2_has_fwb(pgt));	\
+									\
+		__attr;							\
+	})
 
 static int stage2_set_xn_attr(enum kvm_pgtable_prot prot, kvm_pte_t *attr)
 {
Signature · 1 line
2.47.3
Reviewed-by on v1 2/6
#4 lore raw

Re: [PATCH 2/6] KVM: arm64: Add KVM_PGTABLE_S2_AS_S1 flag

Quoted 34 lines · Marc Zyngier

Plumb the MT_S2{,_FWB}_AS_S1 memory types into the KVM_S2_MEMATTR() macro with a new KVM_PGTABLE_S2_AS_S1 flag.

Nobody selects it yet.

---
 arch/arm64/include/asm/kvm_pgtable.h |  2 ++
 arch/arm64/kvm/hyp/pgtable.c         | 13 ++++++++++++-
 2 files changed, 14 insertions(+), 1 deletion(-)
arch/arm64/include/asm/kvm_pgtable.h +2 -0
index fc02de43c68dd..9ce51a637da0a 100644
@@ -232,10 +232,12 @@ struct kvm_pgtable_mm_ops {
  * @KVM_PGTABLE_S2_NOFWB:      Don't enforce Normal-WB even if the CPUs have
  *                             ARM64_HAS_STAGE2_FWB.
  * @KVM_PGTABLE_S2_IDMAP:      Only use identity mappings.
+ * @KVM_PGTABLE_S2_AS_S1:      Final memory attributes are that of Stage-1.
  */
 enum kvm_pgtable_stage2_flags {
        KVM_PGTABLE_S2_NOFWB                    = BIT(0),
        KVM_PGTABLE_S2_IDMAP                    = BIT(1),
+       KVM_PGTABLE_S2_AS_S1                    = BIT(2),
 };
 
 /**
arch/arm64/kvm/hyp/pgtable.c +0 -0
index 947ac1a951a5b..3c5f399b1b986 100644

arch/arm64/kvm/hyp/pgtable.c void kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu, · line 664

Quoted 7 lines
arch/arm64/kvm/hyp/pgtable.c +3 -1
@@ -659,7 +659,18 @@ void kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu,
        }
 }
 
-#define KVM_S2_MEMATTR(pgt, attr) PAGE_S2_MEMATTR(attr, stage2_has_fwb(pgt))
+#define KVM_S2_MEMATTR(pgt, attr)                                      \
+       ({                                                              \
+               kvm_pte_t __attr;                                       \

nit: new line?

Cheers,
/fuad

Quoted 15 lines
arch/arm64/kvm/hyp/pgtable.c +9 -1
+               if((pgt)->flags & KVM_PGTABLE_S2_AS_S1)                 \
+                       __attr = PAGE_S2_MEMATTR(AS_S1,                 \
+                                                stage2_has_fwb(pgt));  \
+               else                                                    \
+                       __attr = PAGE_S2_MEMATTR(attr,                  \
+                                                stage2_has_fwb(pgt));  \
+                                                                       \
+               __attr;                                                 \
+       })
 
 static int stage2_set_xn_attr(enum kvm_pgtable_prot prot, kvm_pte_t *attr)
 {
--

2.47.3

maintainer
#5 lore raw

Re: [PATCH 2/6] KVM: arm64: Add KVM_PGTABLE_S2_AS_S1 flag

Quoted 38 lines · Fuad Tabba
Quoted 37 lines · Marc Zyngier

Plumb the MT_S2{,_FWB}_AS_S1 memory types into the KVM_S2_MEMATTR() macro with a new KVM_PGTABLE_S2_AS_S1 flag.

Nobody selects it yet.

---
 arch/arm64/include/asm/kvm_pgtable.h |  2 ++
 arch/arm64/kvm/hyp/pgtable.c         | 13 ++++++++++++-
 2 files changed, 14 insertions(+), 1 deletion(-)
arch/arm64/include/asm/kvm_pgtable.h +2 -0
index fc02de43c68dd..9ce51a637da0a 100644
@@ -232,10 +232,12 @@ struct kvm_pgtable_mm_ops {
  * @KVM_PGTABLE_S2_NOFWB:      Don't enforce Normal-WB even if the CPUs have
  *                             ARM64_HAS_STAGE2_FWB.
  * @KVM_PGTABLE_S2_IDMAP:      Only use identity mappings.
+ * @KVM_PGTABLE_S2_AS_S1:      Final memory attributes are that of Stage-1.
  */
 enum kvm_pgtable_stage2_flags {
        KVM_PGTABLE_S2_NOFWB                    = BIT(0),
        KVM_PGTABLE_S2_IDMAP                    = BIT(1),
+       KVM_PGTABLE_S2_AS_S1                    = BIT(2),
 };
 
 /**
arch/arm64/kvm/hyp/pgtable.c +0 -0
index 947ac1a951a5b..3c5f399b1b986 100644
@@ -659,7 +659,18 @@ void kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu,
        }
 }
 
Quoted 8 lines

arch/arm64/kvm/hyp/pgtable.c void kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu, · line 664

Quoted 4 lines
arch/arm64/kvm/hyp/pgtable.c +3 -1
-#define KVM_S2_MEMATTR(pgt, attr) PAGE_S2_MEMATTR(attr, stage2_has_fwb(pgt))
+#define KVM_S2_MEMATTR(pgt, attr)                                      \
+       ({                                                              \
+               kvm_pte_t __attr;                                       \

nit: new line?

If you're going back to fix nits then:

Quoted 1 line

arch/arm64/kvm/hyp/pgtable.c void kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu, · line 665

Quoted 1 line
arch/arm64/kvm/hyp/pgtable.c +1 -0
+               if((pgt)->flags & KVM_PGTABLE_S2_AS_S1)                 \

please stick a space after the 'if' as well :)

Will

author maintainer
#6 lore raw

PATCH 3/6 KVM: arm64: Make stage2_pte_cacheable() return false when S2_AS_S1 is set

In reply to Marc Zyngier

We cannot tell from the S2 attributes whether what we map is memory or not when S2_AS_S1 is set, and issuing a CMO on device memory may not be the best idea.

In this particular case, pretend that the mapping isn't cacheable.

---
 arch/arm64/kvm/hyp/pgtable.c | 8 ++++++++
 1 file changed, 8 insertions(+)
arch/arm64/kvm/hyp/pgtable.c +8 -0
index 3c5f399b1b986..07561a227f75e 100644
@@ -911,6 +911,14 @@ static void stage2_unmap_put_pte(const struct kvm_pgtable_visit_ctx *ctx,
 static bool stage2_pte_cacheable(struct kvm_pgtable *pgt, kvm_pte_t pte)
 {
 	u64 memattr = pte & KVM_PTE_LEAF_ATTR_LO_S2_MEMATTR;
+
+	/*
+	 * With S2_AS_S1, we have no idea whether the OA is actual memory or
+	 * a device. Don't even try a CMO on that.
+	 */
+	if (pgt->flags & KVM_PGTABLE_S2_AS_S1)
+		return false;
+
 	return kvm_pte_valid(pte) && memattr == KVM_S2_MEMATTR(pgt, NORMAL);
 }
 
Signature · 1 line
2.47.3
maintainer
#7 lore raw

Re: [PATCH 3/6] KVM: arm64: Make stage2_pte_cacheable() return false when S2_AS_S1 is set

Quoted 18 lines · Marc Zyngier

We cannot tell from the S2 attributes whether what we map is memory or not when S2_AS_S1 is set, and issuing a CMO on device memory may not be the best idea.

In this particular case, pretend that the mapping isn't cacheable.

---
 arch/arm64/kvm/hyp/pgtable.c | 8 ++++++++
 1 file changed, 8 insertions(+)
arch/arm64/kvm/hyp/pgtable.c +0 -0
index 3c5f399b1b986..07561a227f75e 100644
@@ -911,6 +911,14 @@ static void stage2_unmap_put_pte(const struct kvm_pgtable_visit_ctx *ctx,
 static bool stage2_pte_cacheable(struct kvm_pgtable *pgt, kvm_pte_t pte)
 {

arch/arm64/kvm/hyp/pgtable.c static void stage2_unmap_put_pte(const struct kvm_pgtable_visit_ctx *ctx, · line 922

Quoted 10 lines
arch/arm64/kvm/hyp/pgtable.c +8 -0
 	u64 memattr = pte & KVM_PTE_LEAF_ATTR_LO_S2_MEMATTR;
+
+	/*
+	 * With S2_AS_S1, we have no idea whether the OA is actual memory or
+	 * a device. Don't even try a CMO on that.
+	 */
+	if (pgt->flags & KVM_PGTABLE_S2_AS_S1)
+		return false;
+
 	return kvm_pte_valid(pte) && memattr == KVM_S2_MEMATTR(pgt, NORMAL);

Since KVM_PGTABLE_S2_AS_S1 is only set for the pKVM host stage-2 and that doesn't provide any cache maintenance callbacks in its kvm_pgtable_mm_ops, I don't think the current code is actually broken, although it's harmless to add the check (and the comment might benefit from some additional rewording?).

Will

author maintainer
#8 lore raw

Re: [PATCH 3/6] KVM: arm64: Make stage2_pte_cacheable() return false when S2_AS_S1 is set

Quoted 26 lines · Will Deacon
Quoted 24 lines · Marc Zyngier

We cannot tell from the S2 attributes whether what we map is memory or not when S2_AS_S1 is set, and issuing a CMO on device memory may not be the best idea.

In this particular case, pretend that the mapping isn't cacheable.

---
 arch/arm64/kvm/hyp/pgtable.c | 8 ++++++++
 1 file changed, 8 insertions(+)
arch/arm64/kvm/hyp/pgtable.c +5 -0
index 3c5f399b1b986..07561a227f75e 100644
@@ -911,6 +911,14 @@ static void stage2_unmap_put_pte(const struct kvm_pgtable_visit_ctx *ctx,
 static bool stage2_pte_cacheable(struct kvm_pgtable *pgt, kvm_pte_t pte)
 {
 	u64 memattr = pte & KVM_PTE_LEAF_ATTR_LO_S2_MEMATTR;
+
+	/*
+	 * With S2_AS_S1, we have no idea whether the OA is actual memory or
+	 * a device. Don't even try a CMO on that.
+	 */
Quoted 8 lines

arch/arm64/kvm/hyp/pgtable.c static void stage2_unmap_put_pte(const struct kvm_pgtable_visit_ctx *ctx, · line 922

Quoted 4 lines
arch/arm64/kvm/hyp/pgtable.c +3 -0
+	if (pgt->flags & KVM_PGTABLE_S2_AS_S1)
+		return false;
+
 	return kvm_pte_valid(pte) && memattr == KVM_S2_MEMATTR(pgt, NORMAL);

Since KVM_PGTABLE_S2_AS_S1 is only set for the pKVM host stage-2 and that doesn't provide any cache maintenance callbacks in its kvm_pgtable_mm_ops, I don't think the current code is actually broken,

Frankly, I have no idea how I came to such a stupid conclusion. My only way out is to blame Metallica (don't listen to that shit, kids!).

Quoted 2 lines

although it's harmless to add the check (and the comment might benefit from some additional rewording?).

I don't think there is much value in adding something that is not useful. I'll drop that patch altogether.

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
author maintainer
#9 lore raw

PATCH 4/6 KVM: arm64: Switch pKVM host S2 over to KVM_PGTABLE_S2_AS_S1

In reply to Marc Zyngier

Since we have the basics to use the S1 memory attributes as the final ones with FWB, flip the host over to that when FWB is present.

---
 arch/arm64/kvm/hyp/nvhe/mem_protect.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
arch/arm64/kvm/hyp/nvhe/mem_protect.c +3 -1
index 49db32f3ddf71..38f66a56a7665 100644
@@ -19,7 +19,7 @@
 #include <nvhe/mem_protect.h>
 #include <nvhe/mm.h>
 
-#define KVM_HOST_S2_FLAGS (KVM_PGTABLE_S2_NOFWB | KVM_PGTABLE_S2_IDMAP)
+#define KVM_HOST_S2_FLAGS (KVM_PGTABLE_S2_AS_S1 | KVM_PGTABLE_S2_IDMAP)
 
 struct host_mmu host_mmu;
 
@@ -324,6 +324,8 @@ int __pkvm_prot_finalize(void)
 	params->vttbr = kvm_get_vttbr(mmu);
 	params->vtcr = mmu->vtcr;
 	params->hcr_el2 |= HCR_VM;
+	if (cpus_have_final_cap(ARM64_HAS_STAGE2_FWB))
+		params->hcr_el2 |= HCR_FWB;
 
 	/*
 	 * The CMO below not only cleans the updated params to the
Signature · 1 line
2.47.3
Reviewed-by on v1 4/6
#10 lore raw

Re: [PATCH 4/6] KVM: arm64: Switch pKVM host S2 over to KVM_PGTABLE_S2_AS_S1

Quoted 23 lines · Marc Zyngier

Since we have the basics to use the S1 memory attributes as the final ones with FWB, flip the host over to that when FWB is present.

---
 arch/arm64/kvm/hyp/nvhe/mem_protect.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
arch/arm64/kvm/hyp/nvhe/mem_protect.c +1 -1
index 49db32f3ddf71..38f66a56a7665 100644
@@ -19,7 +19,7 @@
 #include <nvhe/mem_protect.h>
 #include <nvhe/mm.h>
 
-#define KVM_HOST_S2_FLAGS (KVM_PGTABLE_S2_NOFWB | KVM_PGTABLE_S2_IDMAP)
+#define KVM_HOST_S2_FLAGS (KVM_PGTABLE_S2_AS_S1 | KVM_PGTABLE_S2_IDMAP)
 
 struct host_mmu host_mmu;
 

arch/arm64/kvm/hyp/nvhe/mem_protect.c int __pkvm_prot_finalize(void) · line 328

Quoted 5 lines
arch/arm64/kvm/hyp/nvhe/mem_protect.c +2 -0
@@ -324,6 +324,8 @@ int __pkvm_prot_finalize(void)
        params->vttbr = kvm_get_vttbr(mmu);
        params->vtcr = mmu->vtcr;
        params->hcr_el2 |= HCR_VM;
+       if (cpus_have_final_cap(ARM64_HAS_STAGE2_FWB))
+               params->hcr_el2 |= HCR_FWB;

Cheers,
/fuad

Quoted 5 lines
arch/arm64/kvm/hyp/nvhe/mem_protect.c +0 -1
        /*
         * The CMO below not only cleans the updated params to the
--

2.47.3

author maintainer
#11 lore raw

PATCH 5/6 KVM: arm64: Kill KVM_PGTABLE_S2_NOFWB

In reply to Marc Zyngier

Nobody is using this flag anymore, so remove it. This allows some cleanup for stage2_has_fwb(), whic doesn't need to take the pgt as a parameter anymore.

---
 arch/arm64/include/asm/kvm_pgtable.h |  7 ++-----
 arch/arm64/kvm/hyp/pgtable.c         | 17 +++++++----------
 2 files changed, 9 insertions(+), 15 deletions(-)
arch/arm64/include/asm/kvm_pgtable.h +2 -5
index 9ce51a637da0a..2198b62428832 100644
@@ -229,15 +229,12 @@ struct kvm_pgtable_mm_ops {
 
 /**
  * enum kvm_pgtable_stage2_flags - Stage-2 page-table flags.
- * @KVM_PGTABLE_S2_NOFWB:	Don't enforce Normal-WB even if the CPUs have
- *				ARM64_HAS_STAGE2_FWB.
  * @KVM_PGTABLE_S2_IDMAP:	Only use identity mappings.
  * @KVM_PGTABLE_S2_AS_S1:	Final memory attributes are that of Stage-1.
  */
 enum kvm_pgtable_stage2_flags {
-	KVM_PGTABLE_S2_NOFWB			= BIT(0),
-	KVM_PGTABLE_S2_IDMAP			= BIT(1),
-	KVM_PGTABLE_S2_AS_S1			= BIT(2),
+	KVM_PGTABLE_S2_IDMAP			= BIT(0),
+	KVM_PGTABLE_S2_AS_S1			= BIT(1),
 };
 
 /**
arch/arm64/kvm/hyp/pgtable.c +7 -10
index 07561a227f75e..3c2f7967fdc03 100644
@@ -631,12 +631,9 @@ u64 kvm_get_vtcr(u64 mmfr0, u64 mmfr1, u32 phys_shift)
 	return vtcr;
 }
 
-static bool stage2_has_fwb(struct kvm_pgtable *pgt)
+static bool stage2_has_fwb(void)
 {
-	if (!cpus_have_final_cap(ARM64_HAS_STAGE2_FWB))
-		return false;
-
-	return !(pgt->flags & KVM_PGTABLE_S2_NOFWB);
+	return cpus_have_final_cap(ARM64_HAS_STAGE2_FWB);
 }
 
 void kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu,
@@ -664,10 +661,10 @@ void kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu,
 		kvm_pte_t __attr;					\
 		if((pgt)->flags & KVM_PGTABLE_S2_AS_S1)			\
 			__attr = PAGE_S2_MEMATTR(AS_S1,			\
-						 stage2_has_fwb(pgt));	\
+						 stage2_has_fwb());	\
 		else							\
 			__attr = PAGE_S2_MEMATTR(attr,			\
-						 stage2_has_fwb(pgt));	\
+						 stage2_has_fwb());	\
 									\
 		__attr;							\
 	})
@@ -879,7 +876,7 @@ static bool stage2_unmap_defer_tlb_flush(struct kvm_pgtable *pgt)
 	 * system supporting FWB as the optimization is entirely
 	 * pointless when the unmap walker needs to perform CMOs.
 	 */
-	return system_supports_tlb_range() && stage2_has_fwb(pgt);
+	return system_supports_tlb_range() && stage2_has_fwb();
 }
 
 static void stage2_unmap_put_pte(const struct kvm_pgtable_visit_ctx *ctx,
@@ -1167,7 +1164,7 @@ static int stage2_unmap_walker(const struct kvm_pgtable_visit_ctx *ctx,
 		if (mm_ops->page_count(childp) != 1)
 			return 0;
 	} else if (stage2_pte_cacheable(pgt, ctx->old)) {
-		need_flush = !stage2_has_fwb(pgt);
+		need_flush = !stage2_has_fwb();
 	}
 
 	/*
@@ -1397,7 +1394,7 @@ int kvm_pgtable_stage2_flush(struct kvm_pgtable *pgt, u64 addr, u64 size)
 		.arg	= pgt,
 	};
 
-	if (stage2_has_fwb(pgt))
+	if (stage2_has_fwb())
 		return 0;
 
 	return kvm_pgtable_walk(pgt, addr, size, &walker);
Signature · 1 line
2.47.3
Reviewed-by on v1 5/6
#12 lore raw

Re: [PATCH 5/6] KVM: arm64: Kill KVM_PGTABLE_S2_NOFWB

Quoted 3 lines · Marc Zyngier

Nobody is using this flag anymore, so remove it. This allows some cleanup for stage2_has_fwb(), whic doesn't need to take the

nit: s/whic/which

Quoted 3 lines

pgt as a parameter anymore.

Cheers,
/fuad

Quoted 89 lines
---
 arch/arm64/include/asm/kvm_pgtable.h |  7 ++-----
 arch/arm64/kvm/hyp/pgtable.c         | 17 +++++++----------
 2 files changed, 9 insertions(+), 15 deletions(-)
arch/arm64/include/asm/kvm_pgtable.h +2 -5
index 9ce51a637da0a..2198b62428832 100644
@@ -229,15 +229,12 @@ struct kvm_pgtable_mm_ops {
 
 /**
  * enum kvm_pgtable_stage2_flags - Stage-2 page-table flags.
- * @KVM_PGTABLE_S2_NOFWB:      Don't enforce Normal-WB even if the CPUs have
- *                             ARM64_HAS_STAGE2_FWB.
  * @KVM_PGTABLE_S2_IDMAP:      Only use identity mappings.
  * @KVM_PGTABLE_S2_AS_S1:      Final memory attributes are that of Stage-1.
  */
 enum kvm_pgtable_stage2_flags {
-       KVM_PGTABLE_S2_NOFWB                    = BIT(0),
-       KVM_PGTABLE_S2_IDMAP                    = BIT(1),
-       KVM_PGTABLE_S2_AS_S1                    = BIT(2),
+       KVM_PGTABLE_S2_IDMAP                    = BIT(0),
+       KVM_PGTABLE_S2_AS_S1                    = BIT(1),
 };
 
 /**
arch/arm64/kvm/hyp/pgtable.c +7 -11
index 07561a227f75e..3c2f7967fdc03 100644
@@ -631,12 +631,9 @@ u64 kvm_get_vtcr(u64 mmfr0, u64 mmfr1, u32 phys_shift)
        return vtcr;
 }
 
-static bool stage2_has_fwb(struct kvm_pgtable *pgt)
+static bool stage2_has_fwb(void)
 {
-       if (!cpus_have_final_cap(ARM64_HAS_STAGE2_FWB))
-               return false;
-
-       return !(pgt->flags & KVM_PGTABLE_S2_NOFWB);
+       return cpus_have_final_cap(ARM64_HAS_STAGE2_FWB);
 }
 
 void kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu,
@@ -664,10 +661,10 @@ void kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu,
                kvm_pte_t __attr;                                       \
                if((pgt)->flags & KVM_PGTABLE_S2_AS_S1)                 \
                        __attr = PAGE_S2_MEMATTR(AS_S1,                 \
-                                                stage2_has_fwb(pgt));  \
+                                                stage2_has_fwb());     \
                else                                                    \
                        __attr = PAGE_S2_MEMATTR(attr,                  \
-                                                stage2_has_fwb(pgt));  \
+                                                stage2_has_fwb());     \
                                                                        \
                __attr;                                                 \
        })
@@ -879,7 +876,7 @@ static bool stage2_unmap_defer_tlb_flush(struct kvm_pgtable *pgt)
         * system supporting FWB as the optimization is entirely
         * pointless when the unmap walker needs to perform CMOs.
         */
-       return system_supports_tlb_range() && stage2_has_fwb(pgt);
+       return system_supports_tlb_range() && stage2_has_fwb();
 }
 
 static void stage2_unmap_put_pte(const struct kvm_pgtable_visit_ctx *ctx,
@@ -1167,7 +1164,7 @@ static int stage2_unmap_walker(const struct kvm_pgtable_visit_ctx *ctx,
                if (mm_ops->page_count(childp) != 1)
                        return 0;
        } else if (stage2_pte_cacheable(pgt, ctx->old)) {
-               need_flush = !stage2_has_fwb(pgt);
+               need_flush = !stage2_has_fwb();
        }
 
        /*
@@ -1397,7 +1394,7 @@ int kvm_pgtable_stage2_flush(struct kvm_pgtable *pgt, u64 addr, u64 size)
                .arg    = pgt,
        };
 
-       if (stage2_has_fwb(pgt))
+       if (stage2_has_fwb())
                return 0;
 
        return kvm_pgtable_walk(pgt, addr, size, &walker);
--

2.47.3

author maintainer
#13 lore raw

PATCH 6/6 KVM: arm64: Simplify PAGE_S2_MEMATTR

In reply to Marc Zyngier

Restore PAGE_S2_MEMATTR() to its former glory, keeping the use of FWB as an implementation detail.

---
 arch/arm64/include/asm/pgtable-prot.h | 4 ++--
 arch/arm64/kvm/hyp/pgtable.c          | 6 ++----
 2 files changed, 4 insertions(+), 6 deletions(-)
arch/arm64/include/asm/pgtable-prot.h +2 -2
index 161e8660edddc..d27e8872fe3c8 100644
@@ -109,10 +109,10 @@ static inline bool __pure lpa2_is_enabled(void)
 #define PAGE_KERNEL_EXEC	__pgprot(_PAGE_KERNEL_EXEC)
 #define PAGE_KERNEL_EXEC_CONT	__pgprot(_PAGE_KERNEL_EXEC_CONT)
 
-#define PAGE_S2_MEMATTR(attr, has_fwb)					\
+#define PAGE_S2_MEMATTR(attr)						\
 	({								\
 		u64 __val;						\
-		if (has_fwb)						\
+		if (cpus_have_final_cap(ARM64_HAS_STAGE2_FWB))		\
 			__val = PTE_S2_MEMATTR(MT_S2_FWB_ ## attr);	\
 		else							\
 			__val = PTE_S2_MEMATTR(MT_S2_ ## attr);		\
arch/arm64/kvm/hyp/pgtable.c +2 -4
index 3c2f7967fdc03..34c9afa442932 100644
@@ -660,11 +660,9 @@ void kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu,
 	({								\
 		kvm_pte_t __attr;					\
 		if((pgt)->flags & KVM_PGTABLE_S2_AS_S1)			\
-			__attr = PAGE_S2_MEMATTR(AS_S1,			\
-						 stage2_has_fwb());	\
+			__attr = PAGE_S2_MEMATTR(AS_S1);		\
 		else							\
-			__attr = PAGE_S2_MEMATTR(attr,			\
-						 stage2_has_fwb());	\
+			__attr = PAGE_S2_MEMATTR(attr);			\
 									\
 		__attr;							\
 	})
Signature · 1 line
2.47.3
maintainer
#14 lore raw

Re: [PATCH 6/6] KVM: arm64: Simplify PAGE_S2_MEMATTR

Quoted 32 lines · Marc Zyngier

Restore PAGE_S2_MEMATTR() to its former glory, keeping the use of FWB as an implementation detail.

---
 arch/arm64/include/asm/pgtable-prot.h | 4 ++--
 arch/arm64/kvm/hyp/pgtable.c          | 6 ++----
 2 files changed, 4 insertions(+), 6 deletions(-)
arch/arm64/include/asm/pgtable-prot.h +2 -2
index 161e8660edddc..d27e8872fe3c8 100644
@@ -109,10 +109,10 @@ static inline bool __pure lpa2_is_enabled(void)
 #define PAGE_KERNEL_EXEC	__pgprot(_PAGE_KERNEL_EXEC)
 #define PAGE_KERNEL_EXEC_CONT	__pgprot(_PAGE_KERNEL_EXEC_CONT)
 
-#define PAGE_S2_MEMATTR(attr, has_fwb)					\
+#define PAGE_S2_MEMATTR(attr)						\
 	({								\
 		u64 __val;						\
-		if (has_fwb)						\
+		if (cpus_have_final_cap(ARM64_HAS_STAGE2_FWB))		\
 			__val = PTE_S2_MEMATTR(MT_S2_FWB_ ## attr);	\
 		else							\
 			__val = PTE_S2_MEMATTR(MT_S2_ ## attr);		\
arch/arm64/kvm/hyp/pgtable.c +0 -0
index 3c2f7967fdc03..34c9afa442932 100644
@@ -660,11 +660,9 @@ void kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu,
 	({								\

arch/arm64/kvm/hyp/pgtable.c void kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu, · line 666

Quoted 10 lines
arch/arm64/kvm/hyp/pgtable.c +2 -4
 		kvm_pte_t __attr;					\
 		if((pgt)->flags & KVM_PGTABLE_S2_AS_S1)			\
-			__attr = PAGE_S2_MEMATTR(AS_S1,			\
-						 stage2_has_fwb());	\
+			__attr = PAGE_S2_MEMATTR(AS_S1);		\
 		else							\
-			__attr = PAGE_S2_MEMATTR(attr,			\
-						 stage2_has_fwb());	\
+			__attr = PAGE_S2_MEMATTR(attr);			\
 									\

tbh, I'd probably just kill stage2_has_fwb() entirely and use the cap everywhere...

Up to you.

Will

reviewer Reviewed-by on the v1 series
#15 lore raw
In reply to Marc Zyngier

After a crash course in FWB,

Quoted 16 lines · Marc Zyngier

[Yes, you can tell what I was listening to]

Today, running in protected mode puts the host under it's own S2 with FWB=0, even if the rest of the guests are running with FWB=1. The rationale for this special-casing of the host is that we want the combined attributes to actually reflect the ones exposed by S1.

We therefore use NormalCacheable (which is the weakest memory type) as the default attributes at S2 so that S1 can only strengthen the final memory type.

But there is no reason why we cannot achieve the same effect with FWB. We normally use FWB to enforce cacheable memory from S2, irrespective of S1. But it is possible to configure the S2 attributes as "pass-through", so that the S1 attributes are always used.

Quoted 5 lines

This small series implements that change, adding a flag that actually describes what we are trying to do (instead of the NOFWB flag which is pretty obscure), and fixes an interesting gotcha with CMOs. Interestingly, CMOs never actually worked with NOFWB on an FWB-aware platform...

Thanks,
Joey

Quoted 22 lines

Patches on top of 6.19-rc5.

Marc Zyngier (6):
arm64: Add MT_S2{,_FWB}_AS_S1 encodings
KVM: arm64: Add KVM_PGTABLE_S2_AS_S1 flag
KVM: arm64: Make stage2_pte_cacheable() return false when S2_AS_S1 is
set
KVM: arm64: Switch pKVM host S2 over to KVM_PGTABLE_S2_AS_S1
KVM: arm64: Kill KVM_PGTABLE_S2_NOFWB
KVM: arm64: Simplify PAGE_S2_MEMATTR

 arch/arm64/include/asm/kvm_pgtable.h  |  7 +++---
 arch/arm64/include/asm/memory.h       | 11 ++++++---
 arch/arm64/include/asm/pgtable-prot.h |  4 ++--
 arch/arm64/kvm/hyp/nvhe/mem_protect.c |  4 +++-
 arch/arm64/kvm/hyp/pgtable.c          | 32 +++++++++++++++++++--------
 5 files changed, 39 insertions(+), 19 deletions(-)
Signature · 1 line
2.47.3
Tested-by on the v1 series
#16 lore raw
In reply to Marc Zyngier

Hi Marc,

Quoted 25 lines · Marc Zyngier

[Yes, you can tell what I was listening to]

Today, running in protected mode puts the host under it's own S2 with FWB=0, even if the rest of the guests are running with FWB=1. The rationale for this special-casing of the host is that we want the combined attributes to actually reflect the ones exposed by S1.

We therefore use NormalCacheable (which is the weakest memory type) as the default attributes at S2 so that S1 can only strengthen the final memory type.

But there is no reason why we cannot achieve the same effect with FWB. We normally use FWB to enforce cacheable memory from S2, irrespective of S1. But it is possible to configure the S2 attributes as "pass-through", so that the S1 attributes are always used.

This small series implements that change, adding a flag that actually describes what we are trying to do (instead of the NOFWB flag which is pretty obscure), and fixes an interesting gotcha with CMOs. Interestingly, CMOs never actually worked with NOFWB on an FWB-aware platform...

Patches on top of 6.19-rc5.

Quoted 8 lines

Marc Zyngier (6):
arm64: Add MT_S2{,_FWB}_AS_S1 encodings
KVM: arm64: Add KVM_PGTABLE_S2_AS_S1 flag
KVM: arm64: Make stage2_pte_cacheable() return false when S2_AS_S1 is
set
KVM: arm64: Switch pKVM host S2 over to KVM_PGTABLE_S2_AS_S1
KVM: arm64: Kill KVM_PGTABLE_S2_NOFWB
KVM: arm64: Simplify PAGE_S2_MEMATTR

Glad to see the host is no longer pulling the shortest straw on memory attributes. ;)

I've applied these patches on top of the pKVM Android branch (with one small fix to the Android pKVM code), and tested it with qemu on cpu max (fwb) neoverse-n1 and cortex-a72 (no fwb).

Stay tuned for the reviews...

Cheers,
/fuad

Quoted 10 lines
 arch/arm64/include/asm/kvm_pgtable.h  |  7 +++---
 arch/arm64/include/asm/memory.h       | 11 ++++++---
 arch/arm64/include/asm/pgtable-prot.h |  4 ++--
 arch/arm64/kvm/hyp/nvhe/mem_protect.c |  4 +++-
 arch/arm64/kvm/hyp/pgtable.c          | 32 +++++++++++++++++++--------
 5 files changed, 39 insertions(+), 19 deletions(-)

--
2.47.3

#17 lore raw
In reply to Marc Zyngier

Hi Marc,

Quoted 7 lines · Marc Zyngier

[Yes, you can tell what I was listening to]

Today, running in protected mode puts the host under it's own S2 with FWB=0, even if the rest of the guests are running with FWB=1. The rationale for this special-casing of the host is that we want the combined attributes to actually reflect the ones exposed by S1.

Quoted 8 lines

We therefore use NormalCacheable (which is the weakest memory type) as the default attributes at S2 so that S1 can only strengthen the final memory type.

But there is no reason why we cannot achieve the same effect with FWB. We normally use FWB to enforce cacheable memory from S2, irrespective of S1. But it is possible to configure the S2 attributes as "pass-through", so that the S1 attributes are always used.

Would you mind clarifying why FWB is preferable? Is it so that pkvm uses the same approach for configuring stage 2 for the host as for a guest? Or is it something else?

Quoted 6 lines

This small series implements that change, adding a flag that actually describes what we are trying to do (instead of the NOFWB flag which is pretty obscure), and fixes an interesting gotcha with CMOs. Interestingly, CMOs never actually worked with NOFWB on an FWB-aware platform...

Hmm... as far I can tell pkvm doesn't populate mm_ops->dcache_clean_inval_poc for host_mmu, so pKVM doesn't issue CMOs for the host s2, with or without these changes.

Thanks,
Alex

Quoted 23 lines

Patches on top of 6.19-rc5.

Marc Zyngier (6):
arm64: Add MT_S2{,_FWB}_AS_S1 encodings
KVM: arm64: Add KVM_PGTABLE_S2_AS_S1 flag
KVM: arm64: Make stage2_pte_cacheable() return false when S2_AS_S1 is
set
KVM: arm64: Switch pKVM host S2 over to KVM_PGTABLE_S2_AS_S1
KVM: arm64: Kill KVM_PGTABLE_S2_NOFWB
KVM: arm64: Simplify PAGE_S2_MEMATTR

 arch/arm64/include/asm/kvm_pgtable.h  |  7 +++---
 arch/arm64/include/asm/memory.h       | 11 ++++++---
 arch/arm64/include/asm/pgtable-prot.h |  4 ++--
 arch/arm64/kvm/hyp/nvhe/mem_protect.c |  4 +++-
 arch/arm64/kvm/hyp/pgtable.c          | 32 +++++++++++++++++++--------
 5 files changed, 39 insertions(+), 19 deletions(-)
Signature · 1 line
2.47.3
author maintainer
#18 lore raw
Quoted 15 lines · Alexandru Elisei

Hi Marc,

Quoted 11 lines · Marc Zyngier

[Yes, you can tell what I was listening to]

Today, running in protected mode puts the host under it's own S2 with FWB=0, even if the rest of the guests are running with FWB=1. The rationale for this special-casing of the host is that we want the combined attributes to actually reflect the ones exposed by S1.

We therefore use NormalCacheable (which is the weakest memory type) as the default attributes at S2 so that S1 can only strengthen the final memory type.

Quoted 8 lines
Quoted 4 lines

But there is no reason why we cannot achieve the same effect with FWB. We normally use FWB to enforce cacheable memory from S2, irrespective of S1. But it is possible to configure the S2 attributes as "pass-through", so that the S1 attributes are always used.

Would you mind clarifying why FWB is preferable? Is it so that pkvm uses the same approach for configuring stage 2 for the host as for a guest? Or is it something else?

Having only one attribute encoding scheme to think about on a given machine is vastly preferable, specially when you are debugging. Additionally, FWB==0 may not be with us forever.

Quoted 9 lines
Quoted 5 lines

This small series implements that change, adding a flag that actually describes what we are trying to do (instead of the NOFWB flag which is pretty obscure), and fixes an interesting gotcha with CMOs. Interestingly, CMOs never actually worked with NOFWB on an FWB-aware platform...

Hmm... as far I can tell pkvm doesn't populate mm_ops->dcache_clean_inval_poc for host_mmu, so pKVM doesn't issue CMOs for the host s2, with or without these changes.

Yeah, I clearly was talking nonsense. I don't know how I came to this silly conclusion.

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
#19 lore raw

Hi Marc,

Quoted 22 lines · Marc Zyngier
Quoted 20 lines · Alexandru Elisei

Hi Marc,

Quoted 15 lines · Marc Zyngier

[Yes, you can tell what I was listening to]

Today, running in protected mode puts the host under it's own S2 with FWB=0, even if the rest of the guests are running with FWB=1. The rationale for this special-casing of the host is that we want the combined attributes to actually reflect the ones exposed by S1.

We therefore use NormalCacheable (which is the weakest memory type) as the default attributes at S2 so that S1 can only strengthen the final memory type.

But there is no reason why we cannot achieve the same effect with FWB. We normally use FWB to enforce cacheable memory from S2, irrespective of S1. But it is possible to configure the S2 attributes as "pass-through", so that the S1 attributes are always used.

Quoted 7 lines
Quoted 3 lines

Would you mind clarifying why FWB is preferable? Is it so that pkvm uses the same approach for configuring stage 2 for the host as for a guest? Or is it something else?

Having only one attribute encoding scheme to think about on a given machine is vastly preferable, specially when you are debugging. Additionally, FWB==0 may not be with us forever.

Got it, thanks for the explanation!

Thanks,
Alex