Skip to contentSkip to conversation

KVM: arm64: Optimize protected mode and FWB

PATCH Superseded Mostafa Saleh 3 participants 5 messages Jul 21 – Jul 23

Subsystems (2)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 KVM: arm64: Optimize protected mode and FWB

KVM opportunistically enables FWB if supported by the system for guest VMs, which allows it to elude cache maintenance for data as they are forced to be cacheable from stage-2.
In that case, __clean_dcache_guest_page() will immediately return. However in protected mode, before calling __clean_dcache_guest_page() it loops over the range and fix_map/unmap it, issuing TLB invalidations, dsb() and isb() unnecessarily.

This can be optimized by returning early if FWB is supported, kvm_pgtable_stage2_map() already issues dsb() and tlb invalidation functions issue dsb() for the unmap path.

---
 arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 +++++++
 1 file changed, 7 insertions(+)
arch/arm64/kvm/hyp/nvhe/mem_protect.c +7 -0
index 4e329e39a695..049a09be2bbf 100644
@@ -261,6 +261,13 @@ static void __apply_guest_page(void *va, size_t size,
 
 static void clean_dcache_guest_page(void *va, size_t size)
 {
+	/*
+	 * Guest stage-2 uses FWB if available, making it safe to elide CMOs.
+	 * In contrast, the host stage-2 never has FWB enabled.
+	 */
+	if (cpus_have_final_cap(ARM64_HAS_STAGE2_FWB))
+		return;
+
 	__apply_guest_page(va, size, __clean_dcache_guest_page);
 }
 
Signature · 1 line
2.55.0.229.g6434b31f56-goog
maintainer
#1 lore raw
Quoted 24 lines · Mostafa Saleh

KVM opportunistically enables FWB if supported by the system for guest VMs, which allows it to elude cache maintenance for data as they are forced to be cacheable from stage-2.
In that case, __clean_dcache_guest_page() will immediately return. However in protected mode, before calling __clean_dcache_guest_page() it loops over the range and fix_map/unmap it, issuing TLB invalidations, dsb() and isb() unnecessarily.

This can be optimized by returning early if FWB is supported, kvm_pgtable_stage2_map() already issues dsb() and tlb invalidation functions issue dsb() for the unmap path.

---
 arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 +++++++
 1 file changed, 7 insertions(+)
arch/arm64/kvm/hyp/nvhe/mem_protect.c +0 -0
index 4e329e39a695..049a09be2bbf 100644

arch/arm64/kvm/hyp/nvhe/mem_protect.c static void __apply_guest_page(void *va, size_t size, · line 266

Quoted 6 lines
arch/arm64/kvm/hyp/nvhe/mem_protect.c +3 -0
@@ -261,6 +261,13 @@ static void __apply_guest_page(void *va, size_t size,
 
 static void clean_dcache_guest_page(void *va, size_t size)
 {
+	/*
+	 * Guest stage-2 uses FWB if available, making it safe to elide CMOs.
+	 * In contrast, the host stage-2 never has FWB enabled.

Beeeeeeep!!! Since a373930ec9406 ("KVM: arm64: Switch pKVM host S2 over to KVM_PGTABLE_S2_AS_S1"), FWB is enabled everywhere when available.

Not sure that changes anything, but the comment is factually wrong.

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
#2 lore raw
Quoted 37 lines · Marc Zyngier
Quoted 24 lines · Mostafa Saleh

KVM opportunistically enables FWB if supported by the system for guest VMs, which allows it to elude cache maintenance for data as they are forced to be cacheable from stage-2.
In that case, __clean_dcache_guest_page() will immediately return. However in protected mode, before calling __clean_dcache_guest_page() it loops over the range and fix_map/unmap it, issuing TLB invalidations, dsb() and isb() unnecessarily.

This can be optimized by returning early if FWB is supported, kvm_pgtable_stage2_map() already issues dsb() and tlb invalidation functions issue dsb() for the unmap path.

---
 arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 +++++++
 1 file changed, 7 insertions(+)
arch/arm64/kvm/hyp/nvhe/mem_protect.c +0 -0
index 4e329e39a695..049a09be2bbf 100644

arch/arm64/kvm/hyp/nvhe/mem_protect.c static void __apply_guest_page(void *va, size_t size, · line 266

Quoted 6 lines
arch/arm64/kvm/hyp/nvhe/mem_protect.c +3 -0
@@ -261,6 +261,13 @@ static void __apply_guest_page(void *va, size_t size,
 
 static void clean_dcache_guest_page(void *va, size_t size)
 {
+	/*
+	 * Guest stage-2 uses FWB if available, making it safe to elide CMOs.
+	 * In contrast, the host stage-2 never has FWB enabled.

Beeeeeeep!!! Since a373930ec9406 ("KVM: arm64: Switch pKVM host S2 over to KVM_PGTABLE_S2_AS_S1"), FWB is enabled everywhere when available.

Quoted 1 line

Not sure that changes anything, but the comment is factually wrong.

Ah, I see it uses KVM_PGTABLE_S2_AS_S1 in that case, that should not change the patch, I will need to update the comment.

Thanks,
Mostafa

Quoted 7 lines

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
reviewer
#3 lore raw
Quoted 39 lines · Mostafa Saleh
Quoted 37 lines · Marc Zyngier
Quoted 24 lines · Mostafa Saleh

KVM opportunistically enables FWB if supported by the system for guest VMs, which allows it to elude cache maintenance for data as they are forced to be cacheable from stage-2.
In that case, __clean_dcache_guest_page() will immediately return. However in protected mode, before calling __clean_dcache_guest_page() it loops over the range and fix_map/unmap it, issuing TLB invalidations, dsb() and isb() unnecessarily.

This can be optimized by returning early if FWB is supported, kvm_pgtable_stage2_map() already issues dsb() and tlb invalidation functions issue dsb() for the unmap path.

---
 arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 +++++++
 1 file changed, 7 insertions(+)
arch/arm64/kvm/hyp/nvhe/mem_protect.c +0 -0
index 4e329e39a695..049a09be2bbf 100644

arch/arm64/kvm/hyp/nvhe/mem_protect.c static void __apply_guest_page(void *va, size_t size, · line 266

Quoted 6 lines
arch/arm64/kvm/hyp/nvhe/mem_protect.c +3 -0
@@ -261,6 +261,13 @@ static void __apply_guest_page(void *va, size_t size,
 
 static void clean_dcache_guest_page(void *va, size_t size)
 {
+   /*
+    * Guest stage-2 uses FWB if available, making it safe to elide CMOs.
+    * In contrast, the host stage-2 never has FWB enabled.

Beeeeeeep!!! Since a373930ec9406 ("KVM: arm64: Switch pKVM host S2 over to KVM_PGTABLE_S2_AS_S1"), FWB is enabled everywhere when available.

Quoted 4 lines
Quoted 1 line

Not sure that changes anything, but the comment is factually wrong.

Ah, I see it uses KVM_PGTABLE_S2_AS_S1 in that case, that should not change the patch, I will need to update the comment.

Sorry, this confusing is partially my fault. The original comment in v1

Quoted 1 line

/* See __clean_dcache_guest_page() */

wasn't very clear to me, since I wasn't sure what I was looking for in __clean_dcache_guest_page(). Talking to Mostafa in person, he's referring to the comment _inside_ __clean_dcache_guest_page. I think the best fix is to rephase as

+ /* See comment in __clean_dcache_guest_page() */

This would save duplication and having to repeat the comment.

Cheers,
/fuad

Quoted 10 lines

Thanks,
Mostafa

Quoted 7 lines

Thanks,

      M.

--
Without deviation from the norm, progress is not possible.

#4 lore raw
Quoted 49 lines · Fuad Tabba
Quoted 39 lines · Mostafa Saleh
Quoted 37 lines · Marc Zyngier
Quoted 24 lines · Mostafa Saleh

KVM opportunistically enables FWB if supported by the system for guest VMs, which allows it to elude cache maintenance for data as they are forced to be cacheable from stage-2.
In that case, __clean_dcache_guest_page() will immediately return. However in protected mode, before calling __clean_dcache_guest_page() it loops over the range and fix_map/unmap it, issuing TLB invalidations, dsb() and isb() unnecessarily.

This can be optimized by returning early if FWB is supported, kvm_pgtable_stage2_map() already issues dsb() and tlb invalidation functions issue dsb() for the unmap path.

---
 arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 +++++++
 1 file changed, 7 insertions(+)
arch/arm64/kvm/hyp/nvhe/mem_protect.c +0 -0
index 4e329e39a695..049a09be2bbf 100644

arch/arm64/kvm/hyp/nvhe/mem_protect.c static void __apply_guest_page(void *va, size_t size, · line 266

Quoted 6 lines
arch/arm64/kvm/hyp/nvhe/mem_protect.c +3 -0
@@ -261,6 +261,13 @@ static void __apply_guest_page(void *va, size_t size,
 
 static void clean_dcache_guest_page(void *va, size_t size)
 {
+   /*
+    * Guest stage-2 uses FWB if available, making it safe to elide CMOs.
+    * In contrast, the host stage-2 never has FWB enabled.

Beeeeeeep!!! Since a373930ec9406 ("KVM: arm64: Switch pKVM host S2 over to KVM_PGTABLE_S2_AS_S1"), FWB is enabled everywhere when available.

Quoted 4 lines
Quoted 1 line

Not sure that changes anything, but the comment is factually wrong.

Ah, I see it uses KVM_PGTABLE_S2_AS_S1 in that case, that should not change the patch, I will need to update the comment.

Sorry, this confusing is partially my fault. The original comment in v1

Quoted 1 line

/* See __clean_dcache_guest_page() */

Quoted 9 lines

wasn't very clear to me, since I wasn't sure what I was looking for in __clean_dcache_guest_page(). Talking to Mostafa in person, he's referring to the comment _inside_ __clean_dcache_guest_page. I think the best fix is to rephase as

+ /* See comment in __clean_dcache_guest_page() */

This would save duplication and having to repeat the comment.

Makes sense, also thinking about this, I think we should do the same for invalidate_icache_guest_page() and ARM64_HAS_CACHE_DIC.

I can include that in the same patch.

Thanks,
Mostafa