KVM: arm64: fix pKVM mapping cache corner cases
Subsystems (3)
v3 cover KVM: arm64: fix pKVM mapping cache corner cases
This is a standalone v3.
Patch 1 fixes pKVM cache maintenance for non cacheable mappings without growing struct pkvm_mapping.
Patch 2 fixes a pKVM mapping cache topup bug on permission faults that replace page mappings with a PMD mapping.
Patch 3 fixes the generic dirty logging case where a permission fault can still need a page table allocation to split a block mapping.
Changes in v3:
- Send as a standalone series with a cover letter.
- Store the pKVM cacheable bit in nr_pages instead of adding a bool.
- Drop stable from patch 1.
- Add patch 3 for dirty logging permission faults.
Changes in v2:
- Add patch 2 for the pKVM permission fault mapping cache bug.
Bradley Morgan (3):
KVM: arm64: skip pKVM cache flushes for non cacheable mappings
KVM: arm64: top up pKVM mapping cache for permission faults
KVM: arm64: top up stage 2 memcache for dirty logging faults
arch/arm64/kvm/mmu.c | 32 +++++++++++++++++++-------- arch/arm64/kvm/pkvm.c | 51 ++++++++++++++++++++++++++++++++++--------- 2 files changed, 64 insertions(+), 19 deletions(-)
Signature · 1 line
2.53.0
PATCH v3 1/3 KVM: arm64: skip pKVM cache flushes for non cacheable mappings
pKVM keeps its own mapping list for stage 2 operations. Its flush path uses that list directly, so it lost the PTE attribute check done by the generic stage 2 walker.
Record whether a mapping is cacheable and skip cache maintenance for mappings that are not cacheable.
--- arch/arm64/kvm/pkvm.c | 51 ++++++++++++++++++++++++++++++++++--------- 1 file changed, 41 insertions(+), 10 deletions(-)
arch/arm64/kvm/pkvm.c +41 -10
@@ -302,9 +302,32 @@ static u64 __pkvm_mapping_start(struct pkvm_mapping *m)return m->gfn *PAGE_SIZE ; } +#define PKVM_MAPPING_NR_PAGES_MASK GENMASK_ULL(47, 0) +#define PKVM_MAPPING_CACHEABLE BIT_ULL(48) + +static u64 pkvm_mapping_nr_pages (struct pkvm_mapping *m) +{ +return m->nr_pages &PKVM_MAPPING_NR_PAGES_MASK ; +} + +static bool pkvm_mapping_is_cacheable (struct pkvm_mapping *m) +{ +return m->nr_pages &PKVM_MAPPING_CACHEABLE ; +} + +static void pkvm_mapping_set_nr_pages (struct pkvm_mapping *m,u64 nr_pages, +bool cacheable) +{ +WARN_ON_ONCE (nr_pages & ~PKVM_MAPPING_NR_PAGES_MASK ); + + m->nr_pages = nr_pages &PKVM_MAPPING_NR_PAGES_MASK ; +if (cacheable) + m->nr_pages |=PKVM_MAPPING_CACHEABLE ; +} +static u64 __pkvm_mapping_end (struct pkvm_mapping *m) { -return (m->gfn + m->nr_pages) *PAGE_SIZE -1 ; +return (m->gfn +pkvm_mapping_nr_pages (m)) *PAGE_SIZE -1 ; } INTERVAL_TREE_DEFINE(struct pkvm_mapping, node, u64, __subtree_last,
@@ -350,7 +373,7 @@ static int __pkvm_pgtable_stage2_reclaim(struct kvm_pgtable *pgt, u64 start, u64continue ; page =pfn_to_page (mapping->pfn); -WARN_ON_ONCE (mapping->nr_pages !=1 ); +WARN_ON_ONCE (pkvm_mapping_nr_pages (mapping) !=1 );unpin_user_pages_dirty_lock (&page,1 , true);account_locked_vm (kvm->mm,1 , false);pkvm_mapping_remove (mapping, &pgt->pkvm_mappings);
@@ -369,7 +392,7 @@ static int __pkvm_pgtable_stage2_unshare(struct kvm_pgtable *pgt, u64 start, u64for_each_mapping_in_range_safe (pgt, start, end, mapping) { ret =kvm_call_hyp_nvhe (__pkvm_host_unshare_guest, handle, mapping->gfn, - mapping->nr_pages); +pkvm_mapping_nr_pages (mapping));if (WARN_ON (ret))return ret;pkvm_mapping_remove (mapping, &pgt->pkvm_mappings);
@@ -448,7 +471,7 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size,* permission faults are handled in the relax_perms() path. */ if (mapping) { -if (size == (mapping->nr_pages *PAGE_SIZE )) +if (size == (pkvm_mapping_nr_pages (mapping) *PAGE_SIZE ))return -EAGAIN ;/*
@@ -472,7 +495,9 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size,swap (mapping, cache->mapping); mapping->gfn = gfn; mapping->pfn = pfn; - mapping->nr_pages = size /PAGE_SIZE ; +pkvm_mapping_set_nr_pages (mapping, size /PAGE_SIZE , + !(prot & (KVM_PGTABLE_PROT_DEVICE | +KVM_PGTABLE_PROT_NORMAL_NC )));pkvm_mapping_insert (mapping, &pgt->pkvm_mappings);return ret;
@@ -503,7 +528,7 @@ int pkvm_pgtable_stage2_wrprotect(struct kvm_pgtable *pgt, u64 addr, u64 size)lockdep_assert_held (&kvm->mmu_lock);for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) { ret =kvm_call_hyp_nvhe (__pkvm_host_wrprotect_guest, handle, mapping->gfn, - mapping->nr_pages); +pkvm_mapping_nr_pages (mapping));if (WARN_ON (ret))break ; }
@@ -517,9 +542,13 @@ int pkvm_pgtable_stage2_flush(struct kvm_pgtable *pgt, u64 addr, u64 size)struct pkvm_mapping *mapping;lockdep_assert_held (&kvm->mmu_lock); -for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) +for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) { +if (!pkvm_mapping_is_cacheable (mapping)) +continue ; +__clean_dcache_guest_page (pfn_to_kaddr (mapping->pfn), -PAGE_SIZE * mapping->nr_pages); +PAGE_SIZE *pkvm_mapping_nr_pages (mapping)); + }return 0 ; }
@@ -536,8 +565,10 @@ bool pkvm_pgtable_stage2_test_clear_young(struct kvm_pgtable *pgt, u64 addr, u64lockdep_assert_held (&kvm->mmu_lock);for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) - young |=kvm_call_hyp_nvhe (__pkvm_host_test_clear_young_guest, handle, mapping->gfn, - mapping->nr_pages, mkold); + young |=kvm_call_hyp_nvhe (__pkvm_host_test_clear_young_guest, + handle, mapping->gfn, +pkvm_mapping_nr_pages (mapping), + mkold);return young; }
Signature · 1 line
2.53.0
Re: [PATCH v3 1/3] KVM: arm64: skip pKVM cache flushes for non cacheable mappings
Quoted 18 lines · Bradley Morgan
pKVM keeps its own mapping list for stage 2 operations. Its flush path uses that list directly, so it lost the PTE attribute check done by the generic stage 2 walker.
Record whether a mapping is cacheable and skip cache maintenance for mappings that are not cacheable.
Fixes e912efed485a ("KVM: arm64: Introduce the EL1 pKVM MMU")Signed-off-by Bradley Morgan <[email protected]>--- arch/arm64/kvm/pkvm.c | 51 ++++++++++++++++++++++++++++++++++--------- 1 file changed, 41 insertions(+), 10 deletions(-)arch/arm64/kvm/pkvm.c +0
-0
arch/arm64/kvm/pkvm.c static u64 __pkvm_mapping_start(struct pkvm_mapping *m) · line 306
Quoted 5 lines
arch/arm64/kvm/pkvm.c +2
-0@@ -302,9 +302,32 @@ static u64 __pkvm_mapping_start(struct pkvm_mapping *m)return m->gfn *PAGE_SIZE ; } +#define PKVM_MAPPING_NR_PAGES_MASK GENMASK_ULL(47, 0) +#define PKVM_MAPPING_CACHEABLE BIT_ULL(48)
Probably better to make it "_NC". Protected VMs only support cacheable and they also use struct pkvm_mapping.
Quoted 16 lines
arch/arm64/kvm/pkvm.c +16
-0+ +static u64 pkvm_mapping_nr_pages (struct pkvm_mapping *m) +{ +return m->nr_pages &PKVM_MAPPING_NR_PAGES_MASK ; +} + +static bool pkvm_mapping_is_cacheable (struct pkvm_mapping *m) +{ +return m->nr_pages &PKVM_MAPPING_CACHEABLE ; +} + +static void pkvm_mapping_set_nr_pages (struct pkvm_mapping *m,u64 nr_pages, +bool cacheable) +{ +WARN_ON_ONCE (nr_pages & ~PKVM_MAPPING_NR_PAGES_MASK ); +
arch/arm64/kvm/pkvm.c static u64 __pkvm_mapping_start(struct pkvm_mapping *m) · line 331
Quoted 10 lines
arch/arm64/kvm/pkvm.c +6
-1+ m->nr_pages = nr_pages &PKVM_MAPPING_NR_PAGES_MASK ; +if (cacheable) + m->nr_pages |=PKVM_MAPPING_CACHEABLE ; +} +static u64 __pkvm_mapping_end (struct pkvm_mapping *m) { -return (m->gfn + m->nr_pages) *PAGE_SIZE -1 ; +return (m->gfn +pkvm_mapping_nr_pages (m)) *PAGE_SIZE -1 ; }
Perhaps using a bitfield would heavily simplify this code?
struct pkvm_mapping {
...
u64 nr_pages : 63
u64 flags : 1
}
Or alternatively, could just make nr_pages u32 and flags u32. nr_pages will not exceed PMD_SIZE / PAGE_SIZE, which is at worst 8192 on 64K systems.
Quoted 81 lines
arch/arm64/kvm/pkvm.c +17
-9INTERVAL_TREE_DEFINE(struct pkvm_mapping, node, u64, __subtree_last,@@ -350,7 +373,7 @@ static int __pkvm_pgtable_stage2_reclaim(struct kvm_pgtable *pgt, u64 start, u64continue ; page =pfn_to_page (mapping->pfn); -WARN_ON_ONCE (mapping->nr_pages !=1 ); +WARN_ON_ONCE (pkvm_mapping_nr_pages (mapping) !=1 );unpin_user_pages_dirty_lock (&page,1 , true);account_locked_vm (kvm->mm,1 , false);pkvm_mapping_remove (mapping, &pgt->pkvm_mappings);@@ -369,7 +392,7 @@ static int __pkvm_pgtable_stage2_unshare(struct kvm_pgtable *pgt, u64 start, u64for_each_mapping_in_range_safe (pgt, start, end, mapping) { ret =kvm_call_hyp_nvhe (__pkvm_host_unshare_guest, handle, mapping->gfn, - mapping->nr_pages); +pkvm_mapping_nr_pages (mapping));if (WARN_ON (ret))return ret;pkvm_mapping_remove (mapping, &pgt->pkvm_mappings);@@ -448,7 +471,7 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size,* permission faults are handled in the relax_perms() path. */ if (mapping) { -if (size == (mapping->nr_pages *PAGE_SIZE )) +if (size == (pkvm_mapping_nr_pages (mapping) *PAGE_SIZE ))return -EAGAIN ;/* @@ -472,7 +495,9 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size,swap (mapping, cache->mapping); mapping->gfn = gfn; mapping->pfn = pfn; - mapping->nr_pages = size /PAGE_SIZE ; +pkvm_mapping_set_nr_pages (mapping, size /PAGE_SIZE , + !(prot & (KVM_PGTABLE_PROT_DEVICE | +KVM_PGTABLE_PROT_NORMAL_NC )));pkvm_mapping_insert (mapping, &pgt->pkvm_mappings);return ret;@@ -503,7 +528,7 @@ int pkvm_pgtable_stage2_wrprotect(struct kvm_pgtable *pgt, u64 addr, u64 size)lockdep_assert_held (&kvm->mmu_lock);for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) { ret =kvm_call_hyp_nvhe (__pkvm_host_wrprotect_guest, handle, mapping->gfn, - mapping->nr_pages); +pkvm_mapping_nr_pages (mapping));if (WARN_ON (ret))break ; }@@ -517,9 +542,13 @@ int pkvm_pgtable_stage2_flush(struct kvm_pgtable *pgt, u64 addr, u64 size)struct pkvm_mapping *mapping;lockdep_assert_held (&kvm->mmu_lock); -for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) +for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) { +if (!pkvm_mapping_is_cacheable (mapping)) +continue ; +__clean_dcache_guest_page (pfn_to_kaddr (mapping->pfn), -PAGE_SIZE * mapping->nr_pages); +PAGE_SIZE *pkvm_mapping_nr_pages (mapping)); + }return 0 ; }@@ -536,8 +565,10 @@ bool pkvm_pgtable_stage2_test_clear_young(struct kvm_pgtable *pgt, u64 addr, u64lockdep_assert_held (&kvm->mmu_lock);for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) - young |=kvm_call_hyp_nvhe (__pkvm_host_test_clear_young_guest, handle, mapping->gfn, - mapping->nr_pages, mkold); + young |=kvm_call_hyp_nvhe (__pkvm_host_test_clear_young_guest, + handle, mapping->gfn, +pkvm_mapping_nr_pages (mapping), + mkold);return young; }Signature · 1 line
2.53.0
Re: [PATCH v3 1/3] KVM: arm64: skip pKVM cache flushes for non cacheable mappings
In reply to Bradley MorganQuoted 18 lines · Bradley Morgan
pKVM keeps its own mapping list for stage 2 operations. Its flush path uses that list directly, so it lost the PTE attribute check done by the generic stage 2 walker.
Record whether a mapping is cacheable and skip cache maintenance for mappings that are not cacheable.
Fixes e912efed485a ("KVM: arm64: Introduce the EL1 pKVM MMU")Signed-off-by Bradley Morgan <[email protected]>--- arch/arm64/kvm/pkvm.c | 51 ++++++++++++++++++++++++++++++++++--------- 1 file changed, 41 insertions(+), 10 deletions(-)arch/arm64/kvm/pkvm.c +0
-0
arch/arm64/kvm/pkvm.c static u64 __pkvm_mapping_start(struct pkvm_mapping *m) · line 306
Quoted 5 lines
arch/arm64/kvm/pkvm.c +2
-0@@ -302,9 +302,32 @@ static u64 __pkvm_mapping_start(struct pkvm_mapping *m)return m->gfn *PAGE_SIZE ; } +#define PKVM_MAPPING_NR_PAGES_MASK GENMASK_ULL(47, 0) +#define PKVM_MAPPING_CACHEABLE BIT_ULL(48)
Out of curiosity here, why do you choose to use bit 48 here instead of, let's say, bit 63?
(I know it makes absolutely no difference to inner working here, as there should probably not be 2^48 pages in one mapping.)
Thanks!
Leo
Quoted 107 lines
arch/arm64/kvm/pkvm.c +39
-10+ +static u64 pkvm_mapping_nr_pages (struct pkvm_mapping *m) +{ +return m->nr_pages &PKVM_MAPPING_NR_PAGES_MASK ; +} + +static bool pkvm_mapping_is_cacheable (struct pkvm_mapping *m) +{ +return m->nr_pages &PKVM_MAPPING_CACHEABLE ; +} + +static void pkvm_mapping_set_nr_pages (struct pkvm_mapping *m,u64 nr_pages, +bool cacheable) +{ +WARN_ON_ONCE (nr_pages & ~PKVM_MAPPING_NR_PAGES_MASK ); + + m->nr_pages = nr_pages &PKVM_MAPPING_NR_PAGES_MASK ; +if (cacheable) + m->nr_pages |=PKVM_MAPPING_CACHEABLE ; +} +static u64 __pkvm_mapping_end (struct pkvm_mapping *m) { -return (m->gfn + m->nr_pages) *PAGE_SIZE -1 ; +return (m->gfn +pkvm_mapping_nr_pages (m)) *PAGE_SIZE -1 ; } INTERVAL_TREE_DEFINE(struct pkvm_mapping, node, u64, __subtree_last,@@ -350,7 +373,7 @@ static int __pkvm_pgtable_stage2_reclaim(struct kvm_pgtable *pgt, u64 start, u64continue ; page =pfn_to_page (mapping->pfn); -WARN_ON_ONCE (mapping->nr_pages !=1 ); +WARN_ON_ONCE (pkvm_mapping_nr_pages (mapping) !=1 );unpin_user_pages_dirty_lock (&page,1 , true);account_locked_vm (kvm->mm,1 , false);pkvm_mapping_remove (mapping, &pgt->pkvm_mappings);@@ -369,7 +392,7 @@ static int __pkvm_pgtable_stage2_unshare(struct kvm_pgtable *pgt, u64 start, u64for_each_mapping_in_range_safe (pgt, start, end, mapping) { ret =kvm_call_hyp_nvhe (__pkvm_host_unshare_guest, handle, mapping->gfn, - mapping->nr_pages); +pkvm_mapping_nr_pages (mapping));if (WARN_ON (ret))return ret;pkvm_mapping_remove (mapping, &pgt->pkvm_mappings);@@ -448,7 +471,7 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size,* permission faults are handled in the relax_perms() path. */ if (mapping) { -if (size == (mapping->nr_pages *PAGE_SIZE )) +if (size == (pkvm_mapping_nr_pages (mapping) *PAGE_SIZE ))return -EAGAIN ;/* @@ -472,7 +495,9 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size,swap (mapping, cache->mapping); mapping->gfn = gfn; mapping->pfn = pfn; - mapping->nr_pages = size /PAGE_SIZE ; +pkvm_mapping_set_nr_pages (mapping, size /PAGE_SIZE , + !(prot & (KVM_PGTABLE_PROT_DEVICE | +KVM_PGTABLE_PROT_NORMAL_NC )));pkvm_mapping_insert (mapping, &pgt->pkvm_mappings);return ret;@@ -503,7 +528,7 @@ int pkvm_pgtable_stage2_wrprotect(struct kvm_pgtable *pgt, u64 addr, u64 size)lockdep_assert_held (&kvm->mmu_lock);for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) { ret =kvm_call_hyp_nvhe (__pkvm_host_wrprotect_guest, handle, mapping->gfn, - mapping->nr_pages); +pkvm_mapping_nr_pages (mapping));if (WARN_ON (ret))break ; }@@ -517,9 +542,13 @@ int pkvm_pgtable_stage2_flush(struct kvm_pgtable *pgt, u64 addr, u64 size)struct pkvm_mapping *mapping;lockdep_assert_held (&kvm->mmu_lock); -for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) +for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) { +if (!pkvm_mapping_is_cacheable (mapping)) +continue ; +__clean_dcache_guest_page (pfn_to_kaddr (mapping->pfn), -PAGE_SIZE * mapping->nr_pages); +PAGE_SIZE *pkvm_mapping_nr_pages (mapping)); + }return 0 ; }@@ -536,8 +565,10 @@ bool pkvm_pgtable_stage2_test_clear_young(struct kvm_pgtable *pgt, u64 addr, u64lockdep_assert_held (&kvm->mmu_lock);for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) - young |=kvm_call_hyp_nvhe (__pkvm_host_test_clear_young_guest, handle, mapping->gfn, - mapping->nr_pages, mkold); + young |=kvm_call_hyp_nvhe (__pkvm_host_test_clear_young_guest, + handle, mapping->gfn, +pkvm_mapping_nr_pages (mapping), + mkold);return young; }Signature · 1 line
2.53.0
Re: [PATCH v3 1/3] KVM: arm64: skip pKVM cache flushes for non cacheable mappings
Quoted 29 lines · Leonardo Bras
arch/arm64/kvm/pkvm.c static u64 __pkvm_mapping_start(struct pkvm_mapping
Quoted 18 lines · Bradley Morgan
pKVM keeps its own mapping list for stage 2 operations. Its flush path uses that list directly, so it lost the PTE attribute check done by the generic stage 2 walker.
Record whether a mapping is cacheable and skip cache maintenance for mappings that are not cacheable.
Fixes e912efed485a ("KVM: arm64: Introduce the EL1 pKVM MMU")Signed-off-by Bradley Morgan <[email protected]>--- arch/arm64/kvm/pkvm.c | 51 ++++++++++++++++++++++++++++++++++--------- 1 file changed, 41 insertions(+), 10 deletions(-)arch/arm64/kvm/pkvm.c +0
-0@@ -302,9 +302,32 @@ static u64 __pkvm_mapping_start(struct pkvm_mapping*m)
arch/arm64/kvm/pkvm.c static u64 __pkvm_mapping_start(struct pkvm_mapping *m) · line 306
Quoted 5 lines
arch/arm64/kvm/pkvm.c +2
-0return m->gfn *PAGE_SIZE ; } +#define PKVM_MAPPING_NR_PAGES_MASK GENMASK_ULL(47, 0) +#define PKVM_MAPPING_CACHEABLE BIT_ULL(48)Out of curiosity here, why do you choose to use bit 48 here instead of, let's say, bit 63?
Quoted 5 lines
(I know it makes absolutely no difference to inner working here, as there should probably not be 2^48 pages in one mapping.)
Thanks!
Leo
sup Leo, here's a quote from maz
"This thing is already big enough, let's not add a bool right in the middle (use pahole to find out why this is bad). Given that nr_pages is for a range, and that the minimum page size uses 12 bits, the largest number of pages you can have here is 56-12=48 bit wide. That's another 16 bits worth of flags you can use."
this should just clarify things, any questions, feel more than free to ask!
(btw V4 is coming soon)
Quoted 110 lines
arch/arm64/kvm/pkvm.c static u64 __pkvm_mapping_start(struct pkvm_mapping *m)
Quoted 12 lines
arch/arm64/kvm/pkvm.c +12
-0+ +static u64 pkvm_mapping_nr_pages (struct pkvm_mapping *m) +{ +return m->nr_pages &PKVM_MAPPING_NR_PAGES_MASK ; +} + +static bool pkvm_mapping_is_cacheable (struct pkvm_mapping *m) +{ +return m->nr_pages &PKVM_MAPPING_CACHEABLE ; +} + +static void pkvm_mapping_set_nr_pages(struct pkvm_mapping *m, u64
nr_pages,arch/arm64/kvm/pkvm.c static int __pkvm_pgtable_stage2_reclaim(struct
Quoted 17 lines
arch/arm64/kvm/pkvm.c +10
-1+ bool cacheable) +{ +WARN_ON_ONCE (nr_pages & ~PKVM_MAPPING_NR_PAGES_MASK ); + + m->nr_pages = nr_pages &PKVM_MAPPING_NR_PAGES_MASK ; +if (cacheable) + m->nr_pages |=PKVM_MAPPING_CACHEABLE ; +} +static u64 __pkvm_mapping_end (struct pkvm_mapping *m) { -return (m->gfn + m->nr_pages) *PAGE_SIZE -1 ; +return (m->gfn +pkvm_mapping_nr_pages (m)) *PAGE_SIZE -1 ; } INTERVAL_TREE_DEFINE(struct pkvm_mapping, node, u64, __subtree_last,@@ -350,7 +373,7 @@ static int __pkvm_pgtable_stage2_reclaim(struct
kvm_pgtable*pgt, u64 start, u64arch/arm64/kvm/pkvm.c static int __pkvm_pgtable_stage2_unshare(struct
Quoted 9 lines
arch/arm64/kvm/pkvm.c +1
-1continue ; page =pfn_to_page (mapping->pfn); -WARN_ON_ONCE (mapping->nr_pages !=1 ); +WARN_ON_ONCE (pkvm_mapping_nr_pages (mapping) !=1 );unpin_user_pages_dirty_lock (&page,1 , true);account_locked_vm (kvm->mm,1 , false);pkvm_mapping_remove (mapping, &pgt->pkvm_mappings);@@ -369,7 +392,7 @@ static int __pkvm_pgtable_stage2_unshare(struct
kvm_pgtable*pgt, u64 start, u64arch/arm64/kvm/pkvm.c int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt,
Quoted 9 lines
arch/arm64/kvm/pkvm.c +1
-1for_each_mapping_in_range_safe (pgt, start, end, mapping) { ret =kvm_call_hyp_nvhe (__pkvm_host_unshare_guest, handle, mapping->gfn, - mapping->nr_pages); +pkvm_mapping_nr_pages (mapping));if (WARN_ON (ret))return ret;pkvm_mapping_remove (mapping, &pgt->pkvm_mappings);@@ -448,7 +471,7 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt,u64 addr, u64 size,
arch/arm64/kvm/pkvm.c int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt,
Quoted 9 lines
arch/arm64/kvm/pkvm.c +1
-1* permission faults are handled in the relax_perms() path. */ if (mapping) { -if (size == (mapping->nr_pages *PAGE_SIZE )) +if (size == (pkvm_mapping_nr_pages (mapping) *PAGE_SIZE ))return -EAGAIN ;/* @@ -472,7 +495,9 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt,u64 addr, u64 size,
arch/arm64/kvm/pkvm.c int pkvm_pgtable_stage2_wrprotect(struct kvm_pgtable
Quoted 11 lines
arch/arm64/kvm/pkvm.c +3
-1swap (mapping, cache->mapping); mapping->gfn = gfn; mapping->pfn = pfn; - mapping->nr_pages = size /PAGE_SIZE ; +pkvm_mapping_set_nr_pages (mapping, size /PAGE_SIZE , + !(prot & (KVM_PGTABLE_PROT_DEVICE | +KVM_PGTABLE_PROT_NORMAL_NC )));pkvm_mapping_insert (mapping, &pgt->pkvm_mappings);return ret;@@ -503,7 +528,7 @@ int pkvm_pgtable_stage2_wrprotect(struct kvm_pgtable*pgt, u64 addr, u64 size)
arch/arm64/kvm/pkvm.c int pkvm_pgtable_stage2_flush(struct kvm_pgtable
Quoted 9 lines
arch/arm64/kvm/pkvm.c +1
-1lockdep_assert_held (&kvm->mmu_lock);for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) { ret =kvm_call_hyp_nvhe (__pkvm_host_wrprotect_guest, handle, mapping->gfn, - mapping->nr_pages); +pkvm_mapping_nr_pages (mapping));if (WARN_ON (ret))break ; }@@ -517,9 +542,13 @@ int pkvm_pgtable_stage2_flush(struct kvm_pgtable*pgt, u64 addr, u64 size)
arch/arm64/kvm/pkvm.c bool pkvm_pgtable_stage2_test_clear_young(struct
Quoted 16 lines
arch/arm64/kvm/pkvm.c +6
-2struct pkvm_mapping *mapping;lockdep_assert_held (&kvm->mmu_lock); -for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) +for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) { +if (!pkvm_mapping_is_cacheable (mapping)) +continue ; +__clean_dcache_guest_page (pfn_to_kaddr (mapping->pfn), -PAGE_SIZE * mapping->nr_pages); +PAGE_SIZE *pkvm_mapping_nr_pages (mapping)); + }return 0 ; }@@ -536,8 +565,10 @@ bool pkvm_pgtable_stage2_test_clear_young(struct
kvm_pgtable*pgt, u64 addr, u64Quoted 10 lines
arch/arm64/kvm/pkvm.c +4
-2lockdep_assert_held (&kvm->mmu_lock);for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) - young |=kvm_call_hyp_nvhe (__pkvm_host_test_clear_young_guest, handle, mapping->gfn, - mapping->nr_pages, mkold); + young |=kvm_call_hyp_nvhe (__pkvm_host_test_clear_young_guest, + handle, mapping->gfn, +pkvm_mapping_nr_pages (mapping), + mkold);
Quoted 6 lines
Quoted 5 lines
arch/arm64/kvm/pkvm.c +0
-0return young; }Signature · 1 line
2.53.0
Thanks!
Re: [PATCH v3 1/3] KVM: arm64: skip pKVM cache flushes for non cacheable mappings
Quoted 31 lines · Bradley Morgan
Quoted 29 lines · Leonardo Bras
arch/arm64/kvm/pkvm.c static u64 __pkvm_mapping_start(struct pkvm_mapping
Quoted 18 lines · Bradley Morgan
pKVM keeps its own mapping list for stage 2 operations. Its flush path uses that list directly, so it lost the PTE attribute check done by the generic stage 2 walker.
Record whether a mapping is cacheable and skip cache maintenance for mappings that are not cacheable.
Fixes e912efed485a ("KVM: arm64: Introduce the EL1 pKVM MMU")Signed-off-by Bradley Morgan <[email protected]>--- arch/arm64/kvm/pkvm.c | 51 ++++++++++++++++++++++++++++++++++--------- 1 file changed, 41 insertions(+), 10 deletions(-)arch/arm64/kvm/pkvm.c +0
-0@@ -302,9 +302,32 @@ static u64 __pkvm_mapping_start(struct pkvm_mapping*m)
arch/arm64/kvm/pkvm.c static u64 __pkvm_mapping_start(struct pkvm_mapping *m) · line 306
Quoted 5 lines
arch/arm64/kvm/pkvm.c +2
-0return m->gfn *PAGE_SIZE ; } +#define PKVM_MAPPING_NR_PAGES_MASK GENMASK_ULL(47, 0) +#define PKVM_MAPPING_CACHEABLE BIT_ULL(48)Out of curiosity here, why do you choose to use bit 48 here instead of, let's say, bit 63?
Quoted 8 lines
Quoted 5 lines
(I know it makes absolutely no difference to inner working here, as there should probably not be 2^48 pages in one mapping.)
Thanks!
Leosup Leo, here's a quote from maz
Hi Bradley,
Quoted 3 lines
"This thing is already big enough, let's not add a bool right in the middle (use pahole to find out why this is bad).
I suppose you proposed to add a bool into a struct, maybe? It would screw the struct alignment.
Quoted 4 lines
Given that
nr_pages
is for a range, and that the minimum page size uses 12 bits, the largest number of pages you can have here is 56-12=48 bit wide. That's another 16 bits worth of flags you can use."
Humm, makes sense.
And since he mentions 16 bits worth of flags, you start by using the 48th bit. Ok, got your rationale.
(I would possibly start with the 63, though, but that's more on personal taste)
Quoted 4 lines
this should just clarify things, any questions, feel more than free to ask!
(btw V4 is coming soon)
Thanks!
Leo
Quoted 119 lines
Quoted 111 lines
arch/arm64/kvm/pkvm.c static u64 __pkvm_mapping_start(struct pkvm_mapping *m)
Quoted 13 lines
arch/arm64/kvm/pkvm.c +12
-0+ +static u64 pkvm_mapping_nr_pages (struct pkvm_mapping *m) +{ +return m->nr_pages &PKVM_MAPPING_NR_PAGES_MASK ; +} + +static bool pkvm_mapping_is_cacheable (struct pkvm_mapping *m) +{ +return m->nr_pages &PKVM_MAPPING_CACHEABLE ; +} + +static void pkvm_mapping_set_nr_pages(struct pkvm_mapping *m, u64
nr_pages,arch/arm64/kvm/pkvm.c static int __pkvm_pgtable_stage2_reclaim(struct
Quoted 17 lines
arch/arm64/kvm/pkvm.c +10
-1+ bool cacheable) +{ +WARN_ON_ONCE (nr_pages & ~PKVM_MAPPING_NR_PAGES_MASK ); + + m->nr_pages = nr_pages &PKVM_MAPPING_NR_PAGES_MASK ; +if (cacheable) + m->nr_pages |=PKVM_MAPPING_CACHEABLE ; +} +static u64 __pkvm_mapping_end (struct pkvm_mapping *m) { -return (m->gfn + m->nr_pages) *PAGE_SIZE -1 ; +return (m->gfn +pkvm_mapping_nr_pages (m)) *PAGE_SIZE -1 ; } INTERVAL_TREE_DEFINE(struct pkvm_mapping, node, u64, __subtree_last,@@ -350,7 +373,7 @@ static int __pkvm_pgtable_stage2_reclaim(struct
kvm_pgtable*pgt, u64 start, u64arch/arm64/kvm/pkvm.c static int __pkvm_pgtable_stage2_unshare(struct
Quoted 9 lines
arch/arm64/kvm/pkvm.c +1
-1continue ; page =pfn_to_page (mapping->pfn); -WARN_ON_ONCE (mapping->nr_pages !=1 ); +WARN_ON_ONCE (pkvm_mapping_nr_pages (mapping) !=1 );unpin_user_pages_dirty_lock (&page,1 , true);account_locked_vm (kvm->mm,1 , false);pkvm_mapping_remove (mapping, &pgt->pkvm_mappings);@@ -369,7 +392,7 @@ static int __pkvm_pgtable_stage2_unshare(struct
kvm_pgtable*pgt, u64 start, u64arch/arm64/kvm/pkvm.c int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt,
Quoted 9 lines
arch/arm64/kvm/pkvm.c +1
-1for_each_mapping_in_range_safe (pgt, start, end, mapping) { ret =kvm_call_hyp_nvhe (__pkvm_host_unshare_guest, handle, mapping->gfn, - mapping->nr_pages); +pkvm_mapping_nr_pages (mapping));if (WARN_ON (ret))return ret;pkvm_mapping_remove (mapping, &pgt->pkvm_mappings);@@ -448,7 +471,7 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt,u64 addr, u64 size,
arch/arm64/kvm/pkvm.c int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt,
Quoted 9 lines
arch/arm64/kvm/pkvm.c +1
-1* permission faults are handled in the relax_perms() path. */ if (mapping) { -if (size == (mapping->nr_pages *PAGE_SIZE )) +if (size == (pkvm_mapping_nr_pages (mapping) *PAGE_SIZE ))return -EAGAIN ;/* @@ -472,7 +495,9 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt,u64 addr, u64 size,
arch/arm64/kvm/pkvm.c int pkvm_pgtable_stage2_wrprotect(struct kvm_pgtable
Quoted 11 lines
arch/arm64/kvm/pkvm.c +3
-1swap (mapping, cache->mapping); mapping->gfn = gfn; mapping->pfn = pfn; - mapping->nr_pages = size /PAGE_SIZE ; +pkvm_mapping_set_nr_pages (mapping, size /PAGE_SIZE , + !(prot & (KVM_PGTABLE_PROT_DEVICE | +KVM_PGTABLE_PROT_NORMAL_NC )));pkvm_mapping_insert (mapping, &pgt->pkvm_mappings);return ret;@@ -503,7 +528,7 @@ int pkvm_pgtable_stage2_wrprotect(struct kvm_pgtable*pgt, u64 addr, u64 size)
arch/arm64/kvm/pkvm.c int pkvm_pgtable_stage2_flush(struct kvm_pgtable
Quoted 9 lines
arch/arm64/kvm/pkvm.c +1
-1lockdep_assert_held (&kvm->mmu_lock);for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) { ret =kvm_call_hyp_nvhe (__pkvm_host_wrprotect_guest, handle, mapping->gfn, - mapping->nr_pages); +pkvm_mapping_nr_pages (mapping));if (WARN_ON (ret))break ; }@@ -517,9 +542,13 @@ int pkvm_pgtable_stage2_flush(struct kvm_pgtable*pgt, u64 addr, u64 size)
arch/arm64/kvm/pkvm.c bool pkvm_pgtable_stage2_test_clear_young(struct
Quoted 16 lines
arch/arm64/kvm/pkvm.c +6
-2struct pkvm_mapping *mapping;lockdep_assert_held (&kvm->mmu_lock); -for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) +for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) { +if (!pkvm_mapping_is_cacheable (mapping)) +continue ; +__clean_dcache_guest_page (pfn_to_kaddr (mapping->pfn), -PAGE_SIZE * mapping->nr_pages); +PAGE_SIZE *pkvm_mapping_nr_pages (mapping)); + }return 0 ; }@@ -536,8 +565,10 @@ bool pkvm_pgtable_stage2_test_clear_young(struct
kvm_pgtable*pgt, u64 addr, u64Quoted 10 lines
arch/arm64/kvm/pkvm.c +4
-2lockdep_assert_held (&kvm->mmu_lock);for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) - young |=kvm_call_hyp_nvhe (__pkvm_host_test_clear_young_guest, handle, mapping->gfn, - mapping->nr_pages, mkold); + young |=kvm_call_hyp_nvhe (__pkvm_host_test_clear_young_guest, + handle, mapping->gfn, +pkvm_mapping_nr_pages (mapping), + mkold);Quoted 6 lines
Quoted 5 lines
arch/arm64/kvm/pkvm.c +0
-0return young; }Signature · 1 line
2.53.0Thanks!
Re: [PATCH v3 1/3] KVM: arm64: skip pKVM cache flushes for non cacheable mappings
Quoted 46 lines · Leonardo Bras
Quoted 4 lines · Bradley Morgan
Quoted 2 lines · Leonardo Bras
Quoted 1 line · Bradley Morgan
pKVM keeps its own mapping list for stage 2 operations. Its flush
path
Quoted 1 line
Quoted 1 line
Quoted 1 line
uses that list directly, so it lost the PTE attribute check done by
the
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
generic stage 2 walker.
Record whether a mapping is cacheable and skip cache maintenance for mappings that are not cacheable.
Fixes e912efed485a ("KVM: arm64: Introduce the EL1 pKVM MMU")Signed-off-by Bradley Morgan <[email protected]>--- arch/arm64/kvm/pkvm.c | 51++++++++++++++++++++++++++++++++++---------
Quoted 7 lines
Quoted 7 lines
arch/arm64/kvm/pkvm.c static u64 __pkvm_mapping_start(struct
Quoted 7 lines
1 file changed, 41 insertions(+), 10 deletions(-)arch/arm64/kvm/pkvm.c +0
-0@@ -302,9 +302,32 @@ static u64 __pkvm_mapping_start(struct
pkvm_mappingQuoted 11 lines
Quoted 11 lines
*m)
arch/arm64/kvm/pkvm.c static u64 __pkvm_mapping_start(struct pkvm_mapping *m) · line 306
Quoted 5 lines
arch/arm64/kvm/pkvm.c +2
-0return m->gfn *PAGE_SIZE ; } +#define PKVM_MAPPING_NR_PAGES_MASK GENMASK_ULL(47, 0) +#define PKVM_MAPPING_CACHEABLE BIT_ULL(48)Out of curiosity here, why do you choose to use bit 48 here instead of, let's say, bit 63?
(I know it makes absolutely no difference to inner working here, as
there
Quoted 7 lines
Quoted 4 lines
should probably not be 2^48 pages in one mapping.)
Thanks!
Leosup Leo, here's a quote from maz
Quoted 8 lines
Hi Bradley,
Quoted 3 lines
"This thing is already big enough, let's not add a bool right in the middle (use pahole to find out why this is bad).
I suppose you proposed to add a bool into a struct, maybe? It would screw the struct alignment.
yep, crappy old me
Quoted 11 lines
Quoted 4 lines
Given that
nr_pages
is for a range, and that the minimum page size uses 12 bits, the largest number of pages you can have here is 56-12=48 bit wide. That's another 16 bits worth of flags you can use."Humm, makes sense.
And since he mentions 16 bits worth of flags, you start by using the 48th bit. Ok, got your rationale.(I would possibly start with the 63, though, but that's more on personal taste)
48 won't make the world blow up :)
Quoted 82 lines
Quoted 2 lines
this should just clarify things, any questions, feel more than free to
ask!
Quoted 2 lines
(btw V4 is coming soon)
Thanks!
LeoQuoted 43 lines
Quoted 43 lines
arch/arm64/kvm/pkvm.c static u64 __pkvm_mapping_start(struct pkvm_mapping *m)
Quoted 13 lines
arch/arm64/kvm/pkvm.c +12
-0+ +static u64 pkvm_mapping_nr_pages (struct pkvm_mapping *m) +{ +return m->nr_pages &PKVM_MAPPING_NR_PAGES_MASK ; +} + +static bool pkvm_mapping_is_cacheable (struct pkvm_mapping *m) +{ +return m->nr_pages &PKVM_MAPPING_CACHEABLE ; +} + +static void pkvm_mapping_set_nr_pages(struct pkvm_mapping *m, u64
nr_pages,arch/arm64/kvm/pkvm.c static int __pkvm_pgtable_stage2_reclaim(struct
Quoted 17 lines
arch/arm64/kvm/pkvm.c +10
-1+ bool cacheable) +{ +WARN_ON_ONCE (nr_pages & ~PKVM_MAPPING_NR_PAGES_MASK ); + + m->nr_pages = nr_pages &PKVM_MAPPING_NR_PAGES_MASK ; +if (cacheable) + m->nr_pages |=PKVM_MAPPING_CACHEABLE ; +} +static u64 __pkvm_mapping_end (struct pkvm_mapping *m) { -return (m->gfn + m->nr_pages) *PAGE_SIZE -1 ; +return (m->gfn +pkvm_mapping_nr_pages (m)) *PAGE_SIZE -1 ; } INTERVAL_TREE_DEFINE(struct pkvm_mapping, node, u64, __subtree_last,@@ -350,7 +373,7 @@ static int __pkvm_pgtable_stage2_reclaim(struct
kvm_pgtable*pgt, u64 start, u64arch/arm64/kvm/pkvm.c static int __pkvm_pgtable_stage2_unshare(struct
Quoted 9 lines
arch/arm64/kvm/pkvm.c +1
-1continue ; page =pfn_to_page (mapping->pfn); -WARN_ON_ONCE (mapping->nr_pages !=1 ); +WARN_ON_ONCE (pkvm_mapping_nr_pages (mapping) !=1 );unpin_user_pages_dirty_lock (&page,1 , true);account_locked_vm (kvm->mm,1 , false);pkvm_mapping_remove (mapping, &pgt->pkvm_mappings);@@ -369,7 +392,7 @@ static int __pkvm_pgtable_stage2_unshare(struct
kvm_pgtable*pgt, u64 start, u64Quoted 1 line
Quoted 8 lines
Quoted 8 lines
arch/arm64/kvm/pkvm.c int pkvm_pgtable_stage2_map(struct kvm_pgtable
Quoted 8 lines
arch/arm64/kvm/pkvm.c +1
-1for_each_mapping_in_range_safe (pgt, start, end, mapping) { ret =kvm_call_hyp_nvhe (__pkvm_host_unshare_guest, handle, mapping->gfn, - mapping->nr_pages); +pkvm_mapping_nr_pages (mapping));if (WARN_ON (ret))return ret;pkvm_mapping_remove (mapping, &pgt->pkvm_mappings);@@ -448,7 +471,7 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable*pgt,
Quoted 10 lines
Quoted 10 lines
u64 addr, u64 size,
arch/arm64/kvm/pkvm.c int pkvm_pgtable_stage2_map(struct kvm_pgtable
Quoted 9 lines
arch/arm64/kvm/pkvm.c +1
-1* permission faults are handled in the relax_perms() path. */ if (mapping) { -if (size == (mapping->nr_pages *PAGE_SIZE )) +if (size == (pkvm_mapping_nr_pages (mapping) *PAGE_SIZE ))return -EAGAIN ;/* @@ -472,7 +495,9 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable*pgt,
Quoted 10 lines
Quoted 10 lines
u64 addr, u64 size,
Quoted 9 lines
arch/arm64/kvm/pkvm.c +3
-1swap (mapping, cache->mapping); mapping->gfn = gfn; mapping->pfn = pfn; - mapping->nr_pages = size /PAGE_SIZE ; +pkvm_mapping_set_nr_pages (mapping, size /PAGE_SIZE , + !(prot & (KVM_PGTABLE_PROT_DEVICE | +KVM_PGTABLE_PROT_NORMAL_NC )));pkvm_mapping_insert (mapping, &pgt->pkvm_mappings);
Quoted 50 lines
Quoted 2 lines
Quoted 2 lines
arch/arm64/kvm/pkvm.c int pkvm_pgtable_stage2_wrprotect(struct
Quoted 2 lines
arch/arm64/kvm/pkvm.c +0
-0return ret;@@ -503,7 +528,7 @@ int pkvm_pgtable_stage2_wrprotect(struct
kvm_pgtableQuoted 46 lines
Quoted 38 lines
*pgt, u64 addr, u64 size)
arch/arm64/kvm/pkvm.c int pkvm_pgtable_stage2_flush(struct kvm_pgtable
Quoted 9 lines
arch/arm64/kvm/pkvm.c +1
-1lockdep_assert_held (&kvm->mmu_lock);for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) { ret =kvm_call_hyp_nvhe (__pkvm_host_wrprotect_guest, handle, mapping->gfn, - mapping->nr_pages); +pkvm_mapping_nr_pages (mapping));if (WARN_ON (ret))break ; }@@ -517,9 +542,13 @@ int pkvm_pgtable_stage2_flush(struct kvm_pgtable*pgt, u64 addr, u64 size)
arch/arm64/kvm/pkvm.c bool pkvm_pgtable_stage2_test_clear_young(struct
Quoted 16 lines
arch/arm64/kvm/pkvm.c +6
-2struct pkvm_mapping *mapping;lockdep_assert_held (&kvm->mmu_lock); -for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) +for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) { +if (!pkvm_mapping_is_cacheable (mapping)) +continue ; +__clean_dcache_guest_page (pfn_to_kaddr (mapping->pfn), -PAGE_SIZE * mapping->nr_pages); +PAGE_SIZE *pkvm_mapping_nr_pages (mapping)); + }return 0 ; }@@ -536,8 +565,10 @@ bool pkvm_pgtable_stage2_test_clear_young(struct
kvm_pgtable*pgt, u64 addr, u64Quoted 10 lines
arch/arm64/kvm/pkvm.c +4
-2lockdep_assert_held (&kvm->mmu_lock);for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) - young |=kvm_call_hyp_nvhe (__pkvm_host_test_clear_young_guest, handle, mapping->gfn, - mapping->nr_pages, mkold); + young |=kvm_call_hyp_nvhe (__pkvm_host_test_clear_young_guest, + handle, mapping->gfn, +pkvm_mapping_nr_pages (mapping), + mkold);Quoted 6 lines
Quoted 5 lines
arch/arm64/kvm/pkvm.c +0
-0return young; }Signature · 1 line
2.53.0Thanks!
Thanks!
Re: [PATCH v3 1/3] KVM: arm64: skip pKVM cache flushes for non cacheable mappings
Quoted 51 lines · Bradley Morgan
Quoted 49 lines · Leonardo Bras
Quoted 4 lines · Bradley Morgan
Quoted 2 lines · Leonardo Bras
Quoted 1 line · Bradley Morgan
pKVM keeps its own mapping list for stage 2 operations. Its flush
path
Quoted 1 line
Quoted 1 line
Quoted 1 line
uses that list directly, so it lost the PTE attribute check done by
the
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
generic stage 2 walker.
Record whether a mapping is cacheable and skip cache maintenance for mappings that are not cacheable.
Fixes e912efed485a ("KVM: arm64: Introduce the EL1 pKVM MMU")Signed-off-by Bradley Morgan <[email protected]>--- arch/arm64/kvm/pkvm.c | 51++++++++++++++++++++++++++++++++++---------
Quoted 7 lines
Quoted 7 lines
arch/arm64/kvm/pkvm.c static u64 __pkvm_mapping_start(struct
Quoted 7 lines
1 file changed, 41 insertions(+), 10 deletions(-)arch/arm64/kvm/pkvm.c +0
-0@@ -302,9 +302,32 @@ static u64 __pkvm_mapping_start(struct
pkvm_mappingQuoted 11 lines
Quoted 11 lines
*m)
arch/arm64/kvm/pkvm.c static u64 __pkvm_mapping_start(struct pkvm_mapping *m) · line 306
Quoted 5 lines
arch/arm64/kvm/pkvm.c +2
-0return m->gfn *PAGE_SIZE ; } +#define PKVM_MAPPING_NR_PAGES_MASK GENMASK_ULL(47, 0) +#define PKVM_MAPPING_CACHEABLE BIT_ULL(48)Out of curiosity here, why do you choose to use bit 48 here instead of, let's say, bit 63?
(I know it makes absolutely no difference to inner working here, as
there
Quoted 7 lines
Quoted 4 lines
should probably not be 2^48 pages in one mapping.)
Thanks!
Leosup Leo, here's a quote from maz
Hi Bradley,
Quoted 1 line
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
"This thing is already big enough, let's not add a bool right in the middle (use pahole to find out why this is bad).
I suppose you proposed to add a bool into a struct, maybe? It would screw the struct alignment.
yep, crappy old me
Hah, you were probably focused on the big picture.
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Given that
nr_pages
is for a range, and that the minimum page size uses 12 bits, the largest number of pages you can have here is 56-12=48 bit wide. That's another 16 bits worth of flags you can use."
Quoted 8 lines
Quoted 6 lines
Humm, makes sense.
And since he mentions 16 bits worth of flags, you start by using the 48th bit. Ok, got your rationale.(I would possibly start with the 63, though, but that's more on personal taste)
48 won't make the world blow up :)
yeap,
Quoted 131 lines
Quoted 131 lines
Quoted 3 lines
this should just clarify things, any questions, feel more than free to
ask!
Quoted 2 lines
(btw V4 is coming soon)
Thanks!
LeoQuoted 43 lines
Quoted 43 lines
arch/arm64/kvm/pkvm.c static u64 __pkvm_mapping_start(struct pkvm_mapping *m)
Quoted 13 lines
arch/arm64/kvm/pkvm.c +12
-0+ +static u64 pkvm_mapping_nr_pages (struct pkvm_mapping *m) +{ +return m->nr_pages &PKVM_MAPPING_NR_PAGES_MASK ; +} + +static bool pkvm_mapping_is_cacheable (struct pkvm_mapping *m) +{ +return m->nr_pages &PKVM_MAPPING_CACHEABLE ; +} + +static void pkvm_mapping_set_nr_pages(struct pkvm_mapping *m, u64
nr_pages,arch/arm64/kvm/pkvm.c static int __pkvm_pgtable_stage2_reclaim(struct
Quoted 17 lines
arch/arm64/kvm/pkvm.c +10
-1+ bool cacheable) +{ +WARN_ON_ONCE (nr_pages & ~PKVM_MAPPING_NR_PAGES_MASK ); + + m->nr_pages = nr_pages &PKVM_MAPPING_NR_PAGES_MASK ; +if (cacheable) + m->nr_pages |=PKVM_MAPPING_CACHEABLE ; +} +static u64 __pkvm_mapping_end (struct pkvm_mapping *m) { -return (m->gfn + m->nr_pages) *PAGE_SIZE -1 ; +return (m->gfn +pkvm_mapping_nr_pages (m)) *PAGE_SIZE -1 ; } INTERVAL_TREE_DEFINE(struct pkvm_mapping, node, u64, __subtree_last,@@ -350,7 +373,7 @@ static int __pkvm_pgtable_stage2_reclaim(struct
kvm_pgtable*pgt, u64 start, u64arch/arm64/kvm/pkvm.c static int __pkvm_pgtable_stage2_unshare(struct
Quoted 9 lines
arch/arm64/kvm/pkvm.c +1
-1continue ; page =pfn_to_page (mapping->pfn); -WARN_ON_ONCE (mapping->nr_pages !=1 ); +WARN_ON_ONCE (pkvm_mapping_nr_pages (mapping) !=1 );unpin_user_pages_dirty_lock (&page,1 , true);account_locked_vm (kvm->mm,1 , false);pkvm_mapping_remove (mapping, &pgt->pkvm_mappings);@@ -369,7 +392,7 @@ static int __pkvm_pgtable_stage2_unshare(struct
kvm_pgtable*pgt, u64 start, u64Quoted 1 line
Quoted 8 lines
Quoted 8 lines
arch/arm64/kvm/pkvm.c int pkvm_pgtable_stage2_map(struct kvm_pgtable
Quoted 8 lines
arch/arm64/kvm/pkvm.c +1
-1for_each_mapping_in_range_safe (pgt, start, end, mapping) { ret =kvm_call_hyp_nvhe (__pkvm_host_unshare_guest, handle, mapping->gfn, - mapping->nr_pages); +pkvm_mapping_nr_pages (mapping));if (WARN_ON (ret))return ret;pkvm_mapping_remove (mapping, &pgt->pkvm_mappings);@@ -448,7 +471,7 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable*pgt,
Quoted 10 lines
Quoted 10 lines
u64 addr, u64 size,
arch/arm64/kvm/pkvm.c int pkvm_pgtable_stage2_map(struct kvm_pgtable
Quoted 9 lines
arch/arm64/kvm/pkvm.c +1
-1* permission faults are handled in the relax_perms() path. */ if (mapping) { -if (size == (mapping->nr_pages *PAGE_SIZE )) +if (size == (pkvm_mapping_nr_pages (mapping) *PAGE_SIZE ))return -EAGAIN ;/* @@ -472,7 +495,9 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable*pgt,
Quoted 10 lines
Quoted 10 lines
u64 addr, u64 size,
Quoted 9 lines
arch/arm64/kvm/pkvm.c +3
-1swap (mapping, cache->mapping); mapping->gfn = gfn; mapping->pfn = pfn; - mapping->nr_pages = size /PAGE_SIZE ; +pkvm_mapping_set_nr_pages (mapping, size /PAGE_SIZE , + !(prot & (KVM_PGTABLE_PROT_DEVICE | +KVM_PGTABLE_PROT_NORMAL_NC )));pkvm_mapping_insert (mapping, &pgt->pkvm_mappings);Quoted 2 lines
Quoted 2 lines
arch/arm64/kvm/pkvm.c int pkvm_pgtable_stage2_wrprotect(struct
Quoted 2 lines
arch/arm64/kvm/pkvm.c +0
-0return ret;@@ -503,7 +528,7 @@ int pkvm_pgtable_stage2_wrprotect(struct
kvm_pgtableQuoted 45 lines
Quoted 44 lines
*pgt, u64 addr, u64 size)
arch/arm64/kvm/pkvm.c int pkvm_pgtable_stage2_flush(struct kvm_pgtable
Quoted 9 lines
arch/arm64/kvm/pkvm.c +1
-1lockdep_assert_held (&kvm->mmu_lock);for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) { ret =kvm_call_hyp_nvhe (__pkvm_host_wrprotect_guest, handle, mapping->gfn, - mapping->nr_pages); +pkvm_mapping_nr_pages (mapping));if (WARN_ON (ret))break ; }@@ -517,9 +542,13 @@ int pkvm_pgtable_stage2_flush(struct kvm_pgtable*pgt, u64 addr, u64 size)
arch/arm64/kvm/pkvm.c bool pkvm_pgtable_stage2_test_clear_young(struct
Quoted 16 lines
arch/arm64/kvm/pkvm.c +6
-2struct pkvm_mapping *mapping;lockdep_assert_held (&kvm->mmu_lock); -for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) +for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) { +if (!pkvm_mapping_is_cacheable (mapping)) +continue ; +__clean_dcache_guest_page (pfn_to_kaddr (mapping->pfn), -PAGE_SIZE * mapping->nr_pages); +PAGE_SIZE *pkvm_mapping_nr_pages (mapping)); + }return 0 ; }@@ -536,8 +565,10 @@ bool pkvm_pgtable_stage2_test_clear_young(struct
kvm_pgtable*pgt, u64 addr, u64Quoted 15 lines
arch/arm64/kvm/pkvm.c +4
-2lockdep_assert_held (&kvm->mmu_lock);for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) - young |=kvm_call_hyp_nvhe (__pkvm_host_test_clear_young_guest, handle, mapping->gfn, - mapping->nr_pages, mkold); + young |=kvm_call_hyp_nvhe (__pkvm_host_test_clear_young_guest, + handle, mapping->gfn, +pkvm_mapping_nr_pages (mapping), + mkold);return young; }Signature · 1 line
2.53.0
Quoted 4 lines
Quoted 2 lines
Quoted 1 line
Thanks!
Thanks!
Thanks!
Re: [PATCH v3 1/3] KVM: arm64: skip pKVM cache flushes for non cacheable mappings
Quoted 79 lines · Leonardo Bras
Quoted 4 lines · Bradley Morgan
Quoted 2 lines · Leonardo Bras
Quoted 1 line · Bradley Morgan
On July 1, 2026 5:05:53 PM GMT+01:00, Leonardo Bras
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Quoted 2 lines · wrote:
Quoted 1 line · Bradley Morgan
pKVM keeps its own mapping list for stage 2 operations. Its flush
path
Quoted 1 line
Quoted 1 line
Quoted 1 line
uses that list directly, so it lost the PTE attribute check done
by
Quoted 4 lines
Quoted 4 lines
the
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
generic stage 2 walker.
Record whether a mapping is cacheable and skip cache maintenance
for
Quoted 15 lines
Quoted 15 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
mappings that are not cacheable.
Fixes e912efed485a ("KVM: arm64: Introduce the EL1 pKVM MMU")Signed-off-by Bradley Morgan <[email protected]>--- arch/arm64/kvm/pkvm.c | 51++++++++++++++++++++++++++++++++++---------
Quoted 7 lines
Quoted 7 lines
arch/arm64/kvm/pkvm.c static u64 __pkvm_mapping_start(struct
Quoted 7 lines
1 file changed, 41 insertions(+), 10 deletions(-)arch/arm64/kvm/pkvm.c +0
-0@@ -302,9 +302,32 @@ static u64 __pkvm_mapping_start(struct
pkvm_mappingQuoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
*m)
arch/arm64/kvm/pkvm.c static u64 __pkvm_mapping_start(struct pkvm_mapping *m) · line 306
Quoted 5 lines
arch/arm64/kvm/pkvm.c +2
-0return m->gfn *PAGE_SIZE ; } +#define PKVM_MAPPING_NR_PAGES_MASK GENMASK_ULL(47, 0) +#define PKVM_MAPPING_CACHEABLE BIT_ULL(48)Out of curiosity here, why do you choose to use bit 48 here instead
of,
Quoted 15 lines
Quoted 15 lines
Quoted 3 lines
Quoted 3 lines
let's say, bit 63?
(I know it makes absolutely no difference to inner working here, as
there
Quoted 7 lines
Quoted 4 lines
should probably not be 2^48 pages in one mapping.)
Thanks!
Leosup Leo, here's a quote from maz
Hi Bradley,
Quoted 1 line
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
"This thing is already big enough, let's not add a bool right in the middle (use pahole to find out why this is bad).
I suppose you proposed to add a bool into a struct, maybe? It would screw the struct alignment.
yep, crappy old me
Hah, you were probably focused on the big picture.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Given that
nr_pages
is for a range, and that the minimum page size uses 12 bits, the largest number of pages you can have here is 56-12=48 bit wide.That's
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
another 16 bits worth of flags you can use."
Humm, makes sense.
And since he mentions 16 bits worth of flags, you start by using the48th
Quoted 2 lines
Quoted 2 lines
bit. Ok, got your rationale.
Quoted 7 lines
Quoted 1 line
Quoted 1 line
(I would possibly start with the 63, though, but that's more on
personal
Quoted 3 lines
Quoted 1 line
taste)
48 won't make the world blow up :)
yeap,
Would you like to be CCed on v4 or nahhhh?
Quoted 137 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
this should just clarify things, any questions, feel more than free
to
Quoted 36 lines
Quoted 36 lines
ask!
Quoted 2 lines
(btw V4 is coming soon)
Thanks!
LeoQuoted 29 lines
Quoted 29 lines
arch/arm64/kvm/pkvm.c static u64 __pkvm_mapping_start(struct pkvm_mapping *m)
Quoted 13 lines
arch/arm64/kvm/pkvm.c +12
-0+ +static u64 pkvm_mapping_nr_pages (struct pkvm_mapping *m) +{ +return m->nr_pages &PKVM_MAPPING_NR_PAGES_MASK ; +} + +static bool pkvm_mapping_is_cacheable (struct pkvm_mapping *m) +{ +return m->nr_pages &PKVM_MAPPING_CACHEABLE ; +} + +static void pkvm_mapping_set_nr_pages(struct pkvm_mapping *m, u64
nr_pages,Quoted 15 lines
arch/arm64/kvm/pkvm.c +10
-1+ bool cacheable) +{ +WARN_ON_ONCE (nr_pages & ~PKVM_MAPPING_NR_PAGES_MASK ); + + m->nr_pages = nr_pages &PKVM_MAPPING_NR_PAGES_MASK ; +if (cacheable) + m->nr_pages |=PKVM_MAPPING_CACHEABLE ; +} +static u64 __pkvm_mapping_end (struct pkvm_mapping *m) { -return (m->gfn + m->nr_pages) *PAGE_SIZE -1 ; +return (m->gfn +pkvm_mapping_nr_pages (m)) *PAGE_SIZE -1 ; }Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
arch/arm64/kvm/pkvm.c static u64 __pkvm_mapping_start(struct · line 333
Quoted 1 line
arch/arm64/kvm/pkvm.c +0
-0INTERVAL_TREE_DEFINE(struct pkvm_mapping, node, u64,
__subtree_last,Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
arch/arm64/kvm/pkvm.c static int
Quoted 1 line
arch/arm64/kvm/pkvm.c +0
-0@@ -350,7 +373,7 @@ static int__pkvm_pgtable_stage2_reclaim(struct
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
kvm_pgtable*pgt, u64 start, u64arch/arm64/kvm/pkvm.c static int
Quoted 9 lines
arch/arm64/kvm/pkvm.c +1
-1continue ; page =pfn_to_page (mapping->pfn); -WARN_ON_ONCE (mapping->nr_pages !=1 ); +WARN_ON_ONCE (pkvm_mapping_nr_pages (mapping) !=1 );unpin_user_pages_dirty_lock (&page,1 , true);account_locked_vm (kvm->mm,1 , false);pkvm_mapping_remove (mapping, &pgt->pkvm_mappings);@@ -369,7 +392,7 @@ static int__pkvm_pgtable_stage2_unshare(struct
Quoted 36 lines
Quoted 36 lines
Quoted 10 lines
Quoted 10 lines
kvm_pgtable*pgt, u64 start, u64arch/arm64/kvm/pkvm.c int pkvm_pgtable_stage2_map(struct kvm_pgtable
Quoted 9 lines
arch/arm64/kvm/pkvm.c +1
-1for_each_mapping_in_range_safe (pgt, start, end, mapping) { ret =kvm_call_hyp_nvhe (__pkvm_host_unshare_guest, handle, mapping->gfn, - mapping->nr_pages); +pkvm_mapping_nr_pages (mapping));if (WARN_ON (ret))return ret;pkvm_mapping_remove (mapping, &pgt->pkvm_mappings);@@ -448,7 +471,7 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable*pgt,
Quoted 10 lines
Quoted 10 lines
u64 addr, u64 size,
arch/arm64/kvm/pkvm.c int pkvm_pgtable_stage2_map(struct kvm_pgtable
Quoted 9 lines
arch/arm64/kvm/pkvm.c +1
-1* permission faults are handled in the relax_perms() path. */ if (mapping) { -if (size == (mapping->nr_pages *PAGE_SIZE )) +if (size == (pkvm_mapping_nr_pages (mapping) *PAGE_SIZE ))return -EAGAIN ;/* @@ -472,7 +495,9 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable*pgt,
Quoted 10 lines
Quoted 10 lines
u64 addr, u64 size,
Quoted 9 lines
arch/arm64/kvm/pkvm.c +3
-1swap (mapping, cache->mapping); mapping->gfn = gfn; mapping->pfn = pfn; - mapping->nr_pages = size /PAGE_SIZE ; +pkvm_mapping_set_nr_pages (mapping, size /PAGE_SIZE , + !(prot & (KVM_PGTABLE_PROT_DEVICE | +KVM_PGTABLE_PROT_NORMAL_NC )));pkvm_mapping_insert (mapping, &pgt->pkvm_mappings);Quoted 2 lines
Quoted 2 lines
arch/arm64/kvm/pkvm.c int pkvm_pgtable_stage2_wrprotect(struct
Quoted 2 lines
arch/arm64/kvm/pkvm.c +0
-0return ret;@@ -503,7 +528,7 @@ int pkvm_pgtable_stage2_wrprotect(struct
kvm_pgtableQuoted 1 line
Quoted 1 line
*pgt, u64 addr, u64 size)
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
arch/arm64/kvm/pkvm.c int pkvm_pgtable_stage2_flush(struct
Quoted 9 lines
arch/arm64/kvm/pkvm.c +1
-1lockdep_assert_held (&kvm->mmu_lock);for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) { ret =kvm_call_hyp_nvhe (__pkvm_host_wrprotect_guest, handle, mapping->gfn, - mapping->nr_pages); +pkvm_mapping_nr_pages (mapping));if (WARN_ON (ret))break ; }@@ -517,9 +542,13 @@ int pkvm_pgtable_stage2_flush(struct
kvm_pgtableQuoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
*pgt, u64 addr, u64 size)
Quoted 13 lines
arch/arm64/kvm/pkvm.c +6
-2struct pkvm_mapping *mapping;lockdep_assert_held (&kvm->mmu_lock); -for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) +for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) { +if (!pkvm_mapping_is_cacheable (mapping)) +continue ; +__clean_dcache_guest_page (pfn_to_kaddr (mapping->pfn), -PAGE_SIZE * mapping->nr_pages); +PAGE_SIZE *pkvm_mapping_nr_pages (mapping)); + }Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
arch/arm64/kvm/pkvm.c bool
Quoted 3 lines
arch/arm64/kvm/pkvm.c +0
-0return 0 ; }@@ -536,8 +565,10 @@ boolpkvm_pgtable_stage2_test_clear_young(struct
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 17 lines
kvm_pgtable*pgt, u64 addr, u64Quoted 15 lines
arch/arm64/kvm/pkvm.c +4
-2lockdep_assert_held (&kvm->mmu_lock);for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) - young |=kvm_call_hyp_nvhe (__pkvm_host_test_clear_young_guest, handle, mapping->gfn, - mapping->nr_pages, mkold); + young |=kvm_call_hyp_nvhe (__pkvm_host_test_clear_young_guest, + handle, mapping->gfn, +pkvm_mapping_nr_pages (mapping), + mkold);return young; }Signature · 1 line
2.53.0
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Quoted 1 line
Thanks!
Thanks!
Thanks!
Thanks!
Re: [PATCH v3 1/3] KVM: arm64: skip pKVM cache flushes for non cacheable mappings
Quoted 83 lines · Bradley Morgan
Quoted 81 lines · Leonardo Bras
Quoted 4 lines · Bradley Morgan
Quoted 2 lines · Leonardo Bras
Quoted 1 line · Bradley Morgan
On July 1, 2026 5:05:53 PM GMT+01:00, Leonardo Bras
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Quoted 2 lines · wrote:
Quoted 1 line · Bradley Morgan
pKVM keeps its own mapping list for stage 2 operations. Its flush
path
Quoted 1 line
Quoted 1 line
Quoted 1 line
uses that list directly, so it lost the PTE attribute check done
by
Quoted 4 lines
Quoted 4 lines
the
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
generic stage 2 walker.
Record whether a mapping is cacheable and skip cache maintenance
for
Quoted 15 lines
Quoted 15 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
mappings that are not cacheable.
Fixes e912efed485a ("KVM: arm64: Introduce the EL1 pKVM MMU")Signed-off-by Bradley Morgan <[email protected]>--- arch/arm64/kvm/pkvm.c | 51++++++++++++++++++++++++++++++++++---------
Quoted 7 lines
Quoted 7 lines
arch/arm64/kvm/pkvm.c static u64 __pkvm_mapping_start(struct
Quoted 7 lines
1 file changed, 41 insertions(+), 10 deletions(-)arch/arm64/kvm/pkvm.c +0
-0@@ -302,9 +302,32 @@ static u64 __pkvm_mapping_start(struct
pkvm_mappingQuoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
*m)
arch/arm64/kvm/pkvm.c static u64 __pkvm_mapping_start(struct · line 306
Quoted 5 lines
arch/arm64/kvm/pkvm.c +2
-0return m->gfn *PAGE_SIZE ; } +#define PKVM_MAPPING_NR_PAGES_MASK GENMASK_ULL(47, 0) +#define PKVM_MAPPING_CACHEABLE BIT_ULL(48)Out of curiosity here, why do you choose to use bit 48 here instead
of,
Quoted 15 lines
Quoted 15 lines
Quoted 3 lines
Quoted 3 lines
let's say, bit 63?
(I know it makes absolutely no difference to inner working here, as
there
Quoted 7 lines
Quoted 4 lines
should probably not be 2^48 pages in one mapping.)
Thanks!
Leosup Leo, here's a quote from maz
Hi Bradley,
Quoted 1 line
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
"This thing is already big enough, let's not add a bool right in the middle (use pahole to find out why this is bad).
I suppose you proposed to add a bool into a struct, maybe? It would screw the struct alignment.
yep, crappy old me
Hah, you were probably focused on the big picture.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Given that
nr_pages
is for a range, and that the minimum page size uses 12 bits, the largest number of pages you can have here is 56-12=48 bit wide.That's
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
another 16 bits worth of flags you can use."
Humm, makes sense.
And since he mentions 16 bits worth of flags, you start by using the48th
Quoted 3 lines
Quoted 3 lines
bit. Ok, got your rationale.
(I would possibly start with the 63, though, but that's more on
personal
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
taste)
48 won't make the world blow up :)
yeap,
Would you like to be CCed on v4 or nahhhh?
Whatever you feel like :)
Thanks!
Leo
Quoted 147 lines
Quoted 138 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
this should just clarify things, any questions, feel more than free
to
Quoted 36 lines
Quoted 36 lines
ask!
Quoted 2 lines
(btw V4 is coming soon)
Thanks!
LeoQuoted 29 lines
Quoted 29 lines
arch/arm64/kvm/pkvm.c static u64 __pkvm_mapping_start(struct · line 318
Quoted 13 lines
arch/arm64/kvm/pkvm.c +12
-0+ +static u64 pkvm_mapping_nr_pages (struct pkvm_mapping *m) +{ +return m->nr_pages &PKVM_MAPPING_NR_PAGES_MASK ; +} + +static bool pkvm_mapping_is_cacheable (struct pkvm_mapping *m) +{ +return m->nr_pages &PKVM_MAPPING_CACHEABLE ; +} + +static void pkvm_mapping_set_nr_pages(struct pkvm_mapping *m, u64
nr_pages,Quoted 15 lines
arch/arm64/kvm/pkvm.c +10
-1+ bool cacheable) +{ +WARN_ON_ONCE (nr_pages & ~PKVM_MAPPING_NR_PAGES_MASK ); + + m->nr_pages = nr_pages &PKVM_MAPPING_NR_PAGES_MASK ; +if (cacheable) + m->nr_pages |=PKVM_MAPPING_CACHEABLE ; +} +static u64 __pkvm_mapping_end (struct pkvm_mapping *m) { -return (m->gfn + m->nr_pages) *PAGE_SIZE -1 ; +return (m->gfn +pkvm_mapping_nr_pages (m)) *PAGE_SIZE -1 ; }Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
arch/arm64/kvm/pkvm.c static u64 __pkvm_mapping_start(struct · line 333
Quoted 1 line
arch/arm64/kvm/pkvm.c +0
-0INTERVAL_TREE_DEFINE(struct pkvm_mapping, node, u64,
__subtree_last,Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
arch/arm64/kvm/pkvm.c static int
Quoted 1 line
arch/arm64/kvm/pkvm.c +0
-0@@ -350,7 +373,7 @@ static int__pkvm_pgtable_stage2_reclaim(struct
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
kvm_pgtable*pgt, u64 start, u64arch/arm64/kvm/pkvm.c static int
Quoted 9 lines
arch/arm64/kvm/pkvm.c +1
-1continue ; page =pfn_to_page (mapping->pfn); -WARN_ON_ONCE (mapping->nr_pages !=1 ); +WARN_ON_ONCE (pkvm_mapping_nr_pages (mapping) !=1 );unpin_user_pages_dirty_lock (&page,1 , true);account_locked_vm (kvm->mm,1 , false);pkvm_mapping_remove (mapping, &pgt->pkvm_mappings);@@ -369,7 +392,7 @@ static int__pkvm_pgtable_stage2_unshare(struct
Quoted 36 lines
Quoted 36 lines
Quoted 10 lines
Quoted 10 lines
kvm_pgtable*pgt, u64 start, u64arch/arm64/kvm/pkvm.c int pkvm_pgtable_stage2_map(struct kvm_pgtable
Quoted 9 lines
arch/arm64/kvm/pkvm.c +1
-1for_each_mapping_in_range_safe (pgt, start, end, mapping) { ret =kvm_call_hyp_nvhe (__pkvm_host_unshare_guest, handle, mapping->gfn, - mapping->nr_pages); +pkvm_mapping_nr_pages (mapping));if (WARN_ON (ret))return ret;pkvm_mapping_remove (mapping, &pgt->pkvm_mappings);@@ -448,7 +471,7 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable*pgt,
Quoted 10 lines
Quoted 10 lines
u64 addr, u64 size,
arch/arm64/kvm/pkvm.c int pkvm_pgtable_stage2_map(struct kvm_pgtable
Quoted 9 lines
arch/arm64/kvm/pkvm.c +1
-1* permission faults are handled in the relax_perms() path. */ if (mapping) { -if (size == (mapping->nr_pages *PAGE_SIZE )) +if (size == (pkvm_mapping_nr_pages (mapping) *PAGE_SIZE ))return -EAGAIN ;/* @@ -472,7 +495,9 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable*pgt,
Quoted 10 lines
Quoted 10 lines
u64 addr, u64 size,
Quoted 9 lines
arch/arm64/kvm/pkvm.c +3
-1swap (mapping, cache->mapping); mapping->gfn = gfn; mapping->pfn = pfn; - mapping->nr_pages = size /PAGE_SIZE ; +pkvm_mapping_set_nr_pages (mapping, size /PAGE_SIZE , + !(prot & (KVM_PGTABLE_PROT_DEVICE | +KVM_PGTABLE_PROT_NORMAL_NC )));pkvm_mapping_insert (mapping, &pgt->pkvm_mappings);Quoted 2 lines
Quoted 2 lines
arch/arm64/kvm/pkvm.c int pkvm_pgtable_stage2_wrprotect(struct
Quoted 2 lines
arch/arm64/kvm/pkvm.c +0
-0return ret;@@ -503,7 +528,7 @@ int pkvm_pgtable_stage2_wrprotect(struct
kvm_pgtableQuoted 1 line
Quoted 1 line
*pgt, u64 addr, u64 size)
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
arch/arm64/kvm/pkvm.c int pkvm_pgtable_stage2_flush(struct
Quoted 9 lines
arch/arm64/kvm/pkvm.c +1
-1lockdep_assert_held (&kvm->mmu_lock);for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) { ret =kvm_call_hyp_nvhe (__pkvm_host_wrprotect_guest, handle, mapping->gfn, - mapping->nr_pages); +pkvm_mapping_nr_pages (mapping));if (WARN_ON (ret))break ; }@@ -517,9 +542,13 @@ int pkvm_pgtable_stage2_flush(struct
kvm_pgtableQuoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
*pgt, u64 addr, u64 size)
Quoted 13 lines
arch/arm64/kvm/pkvm.c +6
-2struct pkvm_mapping *mapping;lockdep_assert_held (&kvm->mmu_lock); -for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) +for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) { +if (!pkvm_mapping_is_cacheable (mapping)) +continue ; +__clean_dcache_guest_page (pfn_to_kaddr (mapping->pfn), -PAGE_SIZE * mapping->nr_pages); +PAGE_SIZE *pkvm_mapping_nr_pages (mapping)); + }Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
arch/arm64/kvm/pkvm.c bool
Quoted 3 lines
arch/arm64/kvm/pkvm.c +0
-0return 0 ; }@@ -536,8 +565,10 @@ boolpkvm_pgtable_stage2_test_clear_young(struct
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 17 lines
kvm_pgtable*pgt, u64 addr, u64Quoted 15 lines
arch/arm64/kvm/pkvm.c +4
-2lockdep_assert_held (&kvm->mmu_lock);for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) - young |=kvm_call_hyp_nvhe (__pkvm_host_test_clear_young_guest, handle, mapping->gfn, - mapping->nr_pages, mkold); + young |=kvm_call_hyp_nvhe (__pkvm_host_test_clear_young_guest, + handle, mapping->gfn, +pkvm_mapping_nr_pages (mapping), + mkold);return young; }Signature · 1 line
2.53.0Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Quoted 1 line
Thanks!
Thanks!
Thanks!
Thanks!
PATCH v3 2/3 KVM: arm64: top up pKVM mapping cache for permission faults
In reply to Bradley MorganPermission faults normally only relax an existing leaf, so the fault path does not top up the memcache.
With pKVM, a permission fault can also replace page mappings with a PMD mapping. That path needs a fresh pkvm_mapping object, and can dereference a NULL cache->mapping if the cache was not topped up.
Allocate just that object for pKVM permission faults.
The issue was discovered [1] by Sashiko.
--- arch/arm64/kvm/mmu.c | 29 ++++++++++++++++++++++------- 1 file changed, 22 insertions(+), 7 deletions(-)
arch/arm64/kvm/mmu.c +22 -7
@@ -1177,17 +1177,26 @@ void free_hyp_memcache(struct kvm_hyp_memcache *mc)__free_hyp_memcache (mc, hyp_mc_free_fn, kvm_host_va, mc); } +static int topup_hyp_memcache_mapping (struct kvm_hyp_memcache *mc) +{ +if (mc->mapping) +return 0 ; + + mc->mapping = kzalloc_obj(struct pkvm_mapping, +GFP_KERNEL_ACCOUNT ); +return mc->mapping ?0 : -ENOMEM ; +} +int topup_hyp_memcache (struct kvm_hyp_memcache *mc,unsigned long min_pages) { +int ret; +if (!is_protected_kvm_enabled ())return 0 ; -if (!mc->mapping) { - mc->mapping = kzalloc_obj(struct pkvm_mapping, -GFP_KERNEL_ACCOUNT ); -if (!mc->mapping) -return -ENOMEM ; - } + ret =topup_hyp_memcache_mapping (mc); +if (ret) +return ret;return __topup_hyp_memcache (mc, min_pages, hyp_mc_alloc_fn, kvm_host_pa, mc);
@@ -2113,7 +2122,9 @@ static int user_mem_abort(const struct kvm_s2_fault_desc *s2fd)* Permission faults just need to update the existing leaf entry, * and so normally don't require allocations from the memcache. The * only exception to this is when dirty logging is enabled at runtime -* and a write fault needs to collapse a block entry into a table. +* and a write fault needs to collapse a block entry into a table. With +* pKVM, they may still need a fresh mapping object if the fault turns +* page entries into a block entry. */ memcache =get_mmu_memcache (s2fd->vcpu); if (!perm_fault || (memslot_is_logging(s2fd->memslot) &&
@@ -2121,6 +2132,10 @@ static int user_mem_abort(const struct kvm_s2_fault_desc *s2fd) ret =topup_mmu_memcache (s2fd->vcpu, memcache);if (ret)return ret; + }else if (is_protected_kvm_enabled ()) { + ret =topup_hyp_memcache_mapping (memcache); +if (ret) +return ret; }/*
Signature · 1 line
2.53.0
PATCH v3 3/3 KVM: arm64: top up stage 2 memcache for dirty logging faults
In reply to Bradley MorganDirty logging forces new stage 2 mappings down to page size, but it does not always remove an existing block mapping before the next fault. Eager splitting is best effort and is disabled by default.
A permission fault on such a block can still need a page table page to install the smaller mapping. Top up the memcache for any permission fault while dirty logging is active, not only for write faults.
The issue was discovered [1] by Sashiko.
--- arch/arm64/kvm/mmu.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-)
arch/arm64/kvm/mmu.c +4 -5
@@ -2122,13 +2122,12 @@ static int user_mem_abort(const struct kvm_s2_fault_desc *s2fd)* Permission faults just need to update the existing leaf entry, * and so normally don't require allocations from the memcache. The * only exception to this is when dirty logging is enabled at runtime -* and a write fault needs to collapse a block entry into a table. With -* pKVM, they may still need a fresh mapping object if the fault turns -* page entries into a block entry. +* and a fault needs to collapse a block entry into a table. With pKVM, +* they may still need a fresh mapping object if the fault turns page +* entries into a block entry. */ memcache =get_mmu_memcache (s2fd->vcpu); -if (!perm_fault || (memslot_is_logging (s2fd->memslot) && -kvm_is_write_fault (s2fd->vcpu))) { +if (!perm_fault ||memslot_is_logging (s2fd->memslot)) { ret =topup_mmu_memcache (s2fd->vcpu, memcache);if (ret)return ret;
Signature · 1 line
2.53.0
Re: [PATCH v3 3/3] KVM: arm64: top up stage 2 memcache for dirty logging faults
Quoted 32 lines · Bradley Morgan
Dirty logging forces new stage 2 mappings down to page size, but it does not always remove an existing block mapping before the next fault. Eager splitting is best effort and is disabled by default.
A permission fault on such a block can still need a page table page to install the smaller mapping. Top up the memcache for any permission fault while dirty logging is active, not only for write faults.
The issue was discovered [1] by Sashiko.
Fixes 6f745f1bb5bf ("KVM: arm64: Convert user_mem_abort() to generic page-table API")Signed-off-by Bradley Morgan <[email protected]>---
arch/arm64/kvm/mmu.c| 9 ++++-----1 file changed, 4 insertions(+), 5 deletions(-)arch/arm64/kvm/mmu.c +1
-3@@ -2122,13 +2122,12 @@ static int user_mem_abort(const struct kvm_s2_fault_desc *s2fd)* Permission faults just need to update the existing leaf entry, * and so normally don't require allocations from the memcache. The * only exception to this is when dirty logging is enabled at runtime -* and a write fault needs to collapse a block entry into a table. With -* pKVM, they may still need a fresh mapping object if the fault turns -* page entries into a block entry. +* and a fault needs to collapse a block entry into a table. With pKVM,
arch/arm64/kvm/mmu.c static int user_mem_abort(const struct kvm_s2_fault_desc *s2fd) · line 2133
Quoted 10 lines
arch/arm64/kvm/mmu.c +3
-2+* they may still need a fresh mapping object if the fault turns page +* entries into a block entry. */ memcache =get_mmu_memcache (s2fd->vcpu); -if (!perm_fault || (memslot_is_logging (s2fd->memslot) && -kvm_is_write_fault (s2fd->vcpu))) { +if (!perm_fault ||memslot_is_logging (s2fd->memslot)) { ret =topup_mmu_memcache (s2fd->vcpu, memcache);if (ret)return ret;
Note: Patch 3 seems to conflict because of patch 2 (the comments)
Oops! :(
V4 (after people have their review go), will contain one commit (patch
3) with the updated comments.
Patch 1 and 2 applies as usual.
Apologies for my messup.
Thanks!
Re: [PATCH v3 3/3] KVM: arm64: top up stage 2 memcache for dirty logging faults
Quoted 54 lines · Bradley Morgan
Quoted 11 lines · Bradley Morgan
Dirty logging forces new stage 2 mappings down to page size, but it does not always remove an existing block mapping before the next fault. Eager splitting is best effort and is disabled by default.
A permission fault on such a block can still need a page table page to install the smaller mapping. Top up the memcache for any permission fault while dirty logging is active, not only for write faults.
The issue was discovered [1] by Sashiko.
Link:
https://lore.kernel.org/all/[email protected]/T/#t
[1]Quoted 2 lines
Fixes 6f745f1bb5bf ("KVM: arm64: Convert user_mem_abort() to genericpage-table API")
arch/arm64/kvm/mmu.c static int user_mem_abort(const struct
Quoted 11 lines
Signed-off-by Bradley Morgan <[email protected]>---
arch/arm64/kvm/mmu.c| 9 ++++-----1 file changed, 4 insertions(+), 5 deletions(-)arch/arm64/kvm/mmu.c +0
-0@@ -2122,13 +2122,12 @@ static int user_mem_abort(const struct
kvm_s2_fault_desc*s2fd)Quoted 8 lines
arch/arm64/kvm/mmu.c +1
-3* Permission faults just need to update the existing leaf entry, * and so normally don't require allocations from the memcache. The * only exception to this is when dirty logging is enabled at runtime -* and a write fault needs to collapse a block entry into a table. With -* pKVM, they may still need a fresh mapping object if the fault turns -* page entries into a block entry. +* and a fault needs to collapse a block entry into a table. With pKVM, arch/arm64/kvm/mmu.c static int user_mem_abort(const struct kvm_s2_fault_desc *s2fd) · line 2133
Quoted 10 lines
arch/arm64/kvm/mmu.c +3
-2+* they may still need a fresh mapping object if the fault turns page +* entries into a block entry. */ memcache =get_mmu_memcache (s2fd->vcpu); -if (!perm_fault || (memslot_is_logging (s2fd->memslot) && -kvm_is_write_fault (s2fd->vcpu))) { +if (!perm_fault ||memslot_is_logging (s2fd->memslot)) { ret =topup_mmu_memcache (s2fd->vcpu, memcache);if (ret)return ret;Note: Patch 3 seems to conflict because of patch 2 (the comments)
Oops! :(
Quoted 8 lines
V4 (after people have their review go), will contain one commit (patch
3) with the updated comments.Patch 1 and 2 applies as usual.
Apologies for my messup.
Thanks!
Actually. Hmm.
I'll just drop patches 2 and 3, I'll do them at a later date, please disregard patches 2 and 3, patch 1 doesn't rely on 2 and 3..
If you guys wanna have a look feel free! :)
Thanks!
Marc Zyngier
maintainerRe: [PATCH v3 3/3] KVM: arm64: top up stage 2 memcache for dirty logging faults
Quoted 19 lines · Bradley Morgan
Quoted 6 lines · Bradley Morgan
Note: Patch 3 seems to conflict because of patch 2 (the comments)
Oops! :(
Quoted 8 lines
V4 (after people have their review go), will contain one commit (patch
3) with the updated comments.Patch 1 and 2 applies as usual.
Apologies for my messup.
Thanks!
Quoted 6 lines
Actually. Hmm.
I'll just drop patches 2 and 3, I'll do them at a later date, please disregard patches 2 and 3, patch 1 doesn't rely on 2 and 3..
If you guys wanna have a look feel free! :)
As I suggested in my reply to your hasty v2, taking a few *days* between versions is generally a good thing. it gives the reviewers time to chime in, and gives you the opportunity to reflect on what you've just written (reading your own patches after a few days is a sure way to go and rewrite them).
Actually, by posting more often, you are guaranteeing that people *avoid* reviewing your stuff, since odds are that there is a new version coming in the next 10 minutes, so why bother...
But hey, that's free advice, so it's probably worthless.
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Re: [PATCH v3 3/3] KVM: arm64: top up stage 2 memcache for dirty logging faults
Quoted 27 lines · Marc Zyngier
Quoted 2 lines · Bradley Morgan
On June 24, 2026 6:39:16 PM GMT+01:00, Bradley Morgan
Quoted 22 lines
Quoted 6 lines · wrote:
Note: Patch 3 seems to conflict because of patch 2 (the comments)
Oops! :(
Quoted 8 lines
V4 (after people have their review go), will contain one commit (patch
3) with the updated comments.Patch 1 and 2 applies as usual.
Apologies for my messup.
Thanks!
Actually. Hmm.
I'll just drop patches 2 and 3, I'll do them at a later date, please disregard patches 2 and 3, patch 1 doesn't rely on 2 and 3..
Quoted 7 lines
Quoted 1 line
If you guys wanna have a look feel free! :)
As I suggested in my reply to your hasty v2, taking a few *days* between versions is generally a good thing. it gives the reviewers time to chime in, and gives you the opportunity to reflect on what you've just written (reading your own patches after a few days is a sure way to go and rewrite them).
welp, I guess I learned the hard way with the hastiness of my V3, (that's kinda why I dropped p2 and p3)
Quoted 3 lines
Actually, by posting more often, you are guaranteeing that people *avoid* reviewing your stuff, since odds are that there is a new version coming in the next 10 minutes, so why bother...
I'm not planning on posting another version, until fuller review concludes
Quoted 1 line
But hey, that's free advice, so it's probably worthless.
advice from someone with experience isn't worthless.
just stupid me decided to unpark my V3 and send it :(
Sorry about that..
Quoted 3 lines
M.
Thanks!