Skip to contentSkip to conversation

KVM: arm64: Optimize protected mode with FWB and DIC

PATCH Applied Mostafa Saleh 4 participants 4 messages Jul 23 – Jul 31

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 KVM: arm64: Optimize protected mode with FWB and DIC

KVM opportunistically enables FWB if supported by the system for guest VMs, which allows it to elude cache maintenance for data as they are forced to be cacheable from stage-2.
In that case, __clean_dcache_guest_page() will immediately return. However in protected mode, before calling __clean_dcache_guest_page() it loops over the range and fix_map/unmap it, issuing TLB invalidations, dsb() and isb() unnecessarily.

This can be optimized by returning early if FWB is supported, kvm_pgtable_stage2_map() already issues dsb() and tlb invalidation functions issue dsb() for the unmap path.

Similarly for DIC, we can omit the call in
invalidate_icache_guest_page()

No isb() needed as ERET is context synchronization event.

---
 arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 +++++++
 1 file changed, 7 insertions(+)
arch/arm64/kvm/hyp/nvhe/mem_protect.c +7 -0
index 4e329e39a695..39aa8911f62c 100644
@@ -261,11 +261,18 @@ static void __apply_guest_page(void *va, size_t size,
 
 static void clean_dcache_guest_page(void *va, size_t size)
 {
+	/* See comment in __clean_dcache_guest_page() */
+	if (cpus_have_final_cap(ARM64_HAS_STAGE2_FWB))
+		return;
+
 	__apply_guest_page(va, size, __clean_dcache_guest_page);
 }
 
 static void invalidate_icache_guest_page(void *va, size_t size)
 {
+	if (alternative_has_cap_unlikely(ARM64_HAS_CACHE_DIC))
+		return;
+
 	__apply_guest_page(va, size, __invalidate_icache_guest_page);
 }
 
Signature · 1 line
2.55.0.229.g6434b31f56-goog
reviewer Reviewed-by on v3 1/1
#1 lore raw
Quoted 13 lines · Mostafa Saleh

KVM opportunistically enables FWB if supported by the system for guest VMs, which allows it to elude cache maintenance for data as they are forced to be cacheable from stage-2.
In that case, __clean_dcache_guest_page() will immediately return. However in protected mode, before calling __clean_dcache_guest_page() it loops over the range and fix_map/unmap it, issuing TLB invalidations, dsb() and isb() unnecessarily.

This can be optimized by returning early if FWB is supported, kvm_pgtable_stage2_map() already issues dsb() and tlb invalidation functions issue dsb() for the unmap path.

Quoted 6 lines

Similarly for DIC, we can omit the call in
invalidate_icache_guest_page()

No isb() needed as ERET is context synchronization event.

Cheers,
/fuad

Quoted 30 lines
---
 arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 +++++++
 1 file changed, 7 insertions(+)
arch/arm64/kvm/hyp/nvhe/mem_protect.c +7 -1
index 4e329e39a695..39aa8911f62c 100644
@@ -261,11 +261,18 @@ static void __apply_guest_page(void *va, size_t size,
 
 static void clean_dcache_guest_page(void *va, size_t size)
 {
+       /* See comment in __clean_dcache_guest_page() */
+       if (cpus_have_final_cap(ARM64_HAS_STAGE2_FWB))
+               return;
+
        __apply_guest_page(va, size, __clean_dcache_guest_page);
 }
 
 static void invalidate_icache_guest_page(void *va, size_t size)
 {
+       if (alternative_has_cap_unlikely(ARM64_HAS_CACHE_DIC))
+               return;
+
        __apply_guest_page(va, size, __invalidate_icache_guest_page);
 }
 
--

2.55.0.229.g6434b31f56-goog

maintainer Acked-by on v3 1/1
#2 lore raw
In reply to Mostafa Saleh
Quoted 13 lines · Mostafa Saleh

KVM opportunistically enables FWB if supported by the system for guest VMs, which allows it to elude cache maintenance for data as they are forced to be cacheable from stage-2.
In that case, __clean_dcache_guest_page() will immediately return. However in protected mode, before calling __clean_dcache_guest_page() it loops over the range and fix_map/unmap it, issuing TLB invalidations, dsb() and isb() unnecessarily.

This can be optimized by returning early if FWB is supported, kvm_pgtable_stage2_map() already issues dsb() and tlb invalidation functions issue dsb() for the unmap path.

Quoted 6 lines

Similarly for DIC, we can omit the call in
invalidate_icache_guest_page()

No isb() needed as ERET is context synchronization event.

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
maintainer Applied
#3 lore raw
In reply to Mostafa Saleh
Quoted 9 lines · Mostafa Saleh

KVM opportunistically enables FWB if supported by the system for guest VMs, which allows it to elude cache maintenance for data as they are forced to be cacheable from stage-2.
In that case, __clean_dcache_guest_page() will immediately return. However in protected mode, before calling __clean_dcache_guest_page() it loops over the range and fix_map/unmap it, issuing TLB invalidations, dsb() and isb() unnecessarily.

[...]

Applied to next, thanks!

[1/1] KVM: arm64: Optimize protected mode with FWB and DIC

      https://git.kernel.org/kvmarm/kvmarm/c/81b2d03f8c86

--
Best,
Oliver