KVM: arm64: skip pKVM cache flushes for non cacheable mappings
Subsystems (3)
PATCH 1/1 KVM: arm64: skip pKVM cache flushes for non cacheable mappings
pKVM keeps its own mapping list for stage 2 operations. Its flush path uses that list directly, so it lost the PTE attribute check done by the generic stage 2 walker.
Record whether a mapping is cacheable and skip cache maintenance for mappings that are not cacheable.
--- arch/arm64/include/asm/kvm_pkvm.h | 1 + arch/arm64/kvm/pkvm.c | 8 +++++++- 2 files changed, 8 insertions(+), 1 deletion(-)
arch/arm64/include/asm/kvm_pkvm.h +1 -0
@@ -196,6 +196,7 @@ struct pkvm_mapping {u64 gfn;u64 pfn;u64 nr_pages; +bool cacheable;u64 __subtree_last;/* Internal member for interval tree */ };
arch/arm64/kvm/pkvm.c +7 -1
@@ -473,6 +473,8 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size, mapping->gfn = gfn; mapping->pfn = pfn; mapping->nr_pages = size /PAGE_SIZE ; + mapping->cacheable = !(prot & (KVM_PGTABLE_PROT_DEVICE | +KVM_PGTABLE_PROT_NORMAL_NC ));pkvm_mapping_insert (mapping, &pgt->pkvm_mappings);return ret;
@@ -517,9 +519,13 @@ int pkvm_pgtable_stage2_flush(struct kvm_pgtable *pgt, u64 addr, u64 size)struct pkvm_mapping *mapping;lockdep_assert_held (&kvm->mmu_lock); -for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) +for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) { +if (!mapping->cacheable) +continue ; +__clean_dcache_guest_page (pfn_to_kaddr (mapping->pfn),PAGE_SIZE * mapping->nr_pages); + }return 0 ; }
Signature · 1 line
2.53.0
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [Critical] Handling a permission fault that upgrades a mapping to a huge page skips memcache top-up, leading to a NULL pointer dereference in pkvm_pgtable_stage2_map(). --
commit d1dad1f606a114503ffa8d910f3c79c232636e5c
Author: Bradley Morgan <[email protected]>
KVM: arm64: skip pKVM cache flushes for non cacheable mappings
This patch adds a cacheable flag to pKVM mappings to avoid flushing non-cacheable stage 2 mappings. It updates pkvm_pgtable_stage2_map to record this attribute and pkvm_pgtable_stage2_flush to skip flushing.
[ ... ]
arch/arm64/kvm/pkvm.c int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size, · line 475
Quoted 8 lines
arch/arm64/kvm/pkvm.c +0
-0@@ -473,6 +473,8 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size, mapping->gfn = gfn; mapping->pfn = pfn; mapping->nr_pages = size /PAGE_SIZE ;
[Severity: Critical]
This is a pre-existing issue, but could this lead to a NULL pointer dereference if the memcache wasn't topped up?
In arch/arm64/kvm/mmu.c:user_mem_abort(), topup_mmu_memcache() is skipped for permission faults when dirty logging is not active:
memcache = get_mmu_memcache(s2fd->vcpu);
if (!perm_fault || (memslot_is_logging(s2fd->memslot) &&
kvm_is_write_fault(s2fd->vcpu))) {
ret = topup_mmu_memcache(s2fd->vcpu, memcache);
if (ret)
return ret;
}If a guest triggers a permission fault on an address backed by a transparent huge page (where the existing stage-2 mapping is a page-level mapping), kvm_s2_fault_map() will upgrade the mapping to a block and call pkvm_pgtable_stage2_map().
For non-protected VMs on a pKVM host, pkvm_pgtable_stage2_map() then unshares the old mapping, sets mapping = NULL, and swaps it with cache->mapping:
swap(mapping, cache->mapping); mapping->gfn = gfn;
Because the top-up was skipped, cache->mapping is NULL, meaning mapping->gfn dereferences NULL, allowing an untrusted guest to crash the host.
Quoted 5 lines
arch/arm64/kvm/pkvm.c +2
-0+ mapping->cacheable = !(prot & (KVM_PGTABLE_PROT_DEVICE | +KVM_PGTABLE_PROT_NORMAL_NC ));pkvm_mapping_insert (mapping, &pgt->pkvm_mappings);return ret;
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Quoted 60 lines · On June 23, 2026 5:15:45 PM GMT+01:00, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [Critical] Handling a permission fault that upgrades a mapping to a huge page skips memcache top-up, leading to a NULL pointer dereference inpkvm_pgtable_stage2_map(). --commit
d1dad1f606a114503ffa8d910f3c79c232636e5c
Author: Bradley Morgan <[email protected]>KVM: arm64: skip pKVM cache flushes for non cacheable mappings
This patch adds a cacheable flag to pKVM mappings to avoid flushing non-cacheable stage 2 mappings. It updates
pkvm_pgtable_stage2_mapto record this attribute andpkvm_pgtable_stage2_flushto skip flushing.[ ... ]
arch/arm64/kvm/pkvm.c int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt,
Quoted 5 lines
arch/arm64/kvm/pkvm.c +0
-0@@ -473,6 +473,8 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt,u64 addr, u64 size,
arch/arm64/kvm/pkvm.c int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size, · line 475
Quoted 3 lines
arch/arm64/kvm/pkvm.c +0
-0mapping->gfn = gfn; mapping->pfn = pfn; mapping->nr_pages = size /PAGE_SIZE ;[Severity: Critical]
This is a pre-existing issue, but could this lead to a NULL pointer dereference if the memcache wasn't topped up?In arch/arm64/kvm/mmu.c:user_mem_abort(),
topup_mmu_memcache()is skipped for permission faults when dirty logging is not active:memcache = get_mmu_memcache(s2fd->vcpu); if (!perm_fault || (memslot_is_logging(s2fd->memslot) && kvm_is_write_fault(s2fd->vcpu))) { ret = topup_mmu_memcache(s2fd->vcpu, memcache); if (ret) return ret; }If a guest triggers a permission fault on an address backed by a transparent
huge page (where the existing stage-2 mapping is a page-level mapping),kvm_s2_fault_map()will upgrade the mapping to a block and callpkvm_pgtable_stage2_map().For non-protected VMs on a pKVM host,
pkvm_pgtable_stage2_map()then unshares
the old mapping, sets mapping = NULL, and swaps it withcache->mapping:swap(mapping, cache->mapping); mapping->gfn = gfn;Because the top-up was skipped,
cache->mappingis NULL, meaningmapping->gfn
dereferences NULL, allowing an untrusted guest to crash the host.
Quoted 5 lines
arch/arm64/kvm/pkvm.c int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size, · line 480
Quoted 5 lines
arch/arm64/kvm/pkvm.c +2
-0+ mapping->cacheable = !(prot & (KVM_PGTABLE_PROT_DEVICE | +KVM_PGTABLE_PROT_NORMAL_NC ));pkvm_mapping_insert (mapping, &pgt->pkvm_mappings);return ret;
Nice catch!
I'll add the lore link
The issue was discovered [1] by Sashiko.
[1]
https://lore.kernel.org/all/[email protected]/
To my second (fixes this) patches desc.
Quoted 1 line
Thanks!
PATCH v2 1/2 KVM: arm64: skip pKVM cache flushes for non cacheable mappings
In reply to Bradley MorganpKVM keeps its own mapping list for stage 2 operations. Its flush path uses that list directly, so it lost the PTE attribute check done by the generic stage 2 walker.
Record whether a mapping is cacheable and skip cache maintenance for mappings that are not cacheable.
---
Changes in v2:
- Add patch 2 for the pKVM permission fault mapping cache bug.
arch/arm64/include/asm/kvm_pkvm.h | 1 + arch/arm64/kvm/pkvm.c | 8 +++++++- 2 files changed, 8 insertions(+), 1 deletion(-)
arch/arm64/include/asm/kvm_pkvm.h +1 -0
@@ -196,6 +196,7 @@ struct pkvm_mapping {u64 gfn;u64 pfn;u64 nr_pages; +bool cacheable;u64 __subtree_last;/* Internal member for interval tree */ };
arch/arm64/kvm/pkvm.c +7 -1
@@ -473,6 +473,8 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size, mapping->gfn = gfn; mapping->pfn = pfn; mapping->nr_pages = size /PAGE_SIZE ; + mapping->cacheable = !(prot & (KVM_PGTABLE_PROT_DEVICE | +KVM_PGTABLE_PROT_NORMAL_NC ));pkvm_mapping_insert (mapping, &pgt->pkvm_mappings);return ret;
@@ -517,9 +519,13 @@ int pkvm_pgtable_stage2_flush(struct kvm_pgtable *pgt, u64 addr, u64 size)struct pkvm_mapping *mapping;lockdep_assert_held (&kvm->mmu_lock); -for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) +for_each_mapping_in_range_safe (pgt, addr, addr + size, mapping) { +if (!mapping->cacheable) +continue ; +__clean_dcache_guest_page (pfn_to_kaddr (mapping->pfn),PAGE_SIZE * mapping->nr_pages); + }return 0 ; }
Signature · 1 line
2.53.0
PATCH v2 2/2 KVM: arm64: top up pKVM mapping cache for permission faults
Permission faults normally only relax an existing leaf, so the fault path does not top up the memcache.
With pKVM, a permission fault can also replace page mappings with a PMD mapping. That path needs a fresh pkvm_mapping object, and can dereference a NULL cache->mapping if the cache was not topped up.
Allocate just that object for pKVM permission faults.
The issue was discovered [1] by Sashiko.
---
Changes in v2:
- New patch.
arch/arm64/kvm/mmu.c | 29 ++++++++++++++++++++++------- 1 file changed, 22 insertions(+), 7 deletions(-)
arch/arm64/kvm/mmu.c +22 -7
@@ -1177,17 +1177,26 @@ void free_hyp_memcache(struct kvm_hyp_memcache *mc)__free_hyp_memcache (mc, hyp_mc_free_fn, kvm_host_va, mc); } +static int topup_hyp_memcache_mapping (struct kvm_hyp_memcache *mc) +{ +if (mc->mapping) +return 0 ; + + mc->mapping = kzalloc_obj(struct pkvm_mapping, +GFP_KERNEL_ACCOUNT ); +return mc->mapping ?0 : -ENOMEM ; +} +int topup_hyp_memcache (struct kvm_hyp_memcache *mc,unsigned long min_pages) { +int ret; +if (!is_protected_kvm_enabled ())return 0 ; -if (!mc->mapping) { - mc->mapping = kzalloc_obj(struct pkvm_mapping, -GFP_KERNEL_ACCOUNT ); -if (!mc->mapping) -return -ENOMEM ; - } + ret =topup_hyp_memcache_mapping (mc); +if (ret) +return ret;return __topup_hyp_memcache (mc, min_pages, hyp_mc_alloc_fn, kvm_host_pa, mc);
@@ -2113,7 +2122,9 @@ static int user_mem_abort(const struct kvm_s2_fault_desc *s2fd)* Permission faults just need to update the existing leaf entry, * and so normally don't require allocations from the memcache. The * only exception to this is when dirty logging is enabled at runtime -* and a write fault needs to collapse a block entry into a table. +* and a write fault needs to collapse a block entry into a table. With +* pKVM, they may still need a fresh mapping object if the fault turns +* page entries into a block entry. */ memcache =get_mmu_memcache (s2fd->vcpu); if (!perm_fault || (memslot_is_logging(s2fd->memslot) &&
@@ -2121,6 +2132,10 @@ static int user_mem_abort(const struct kvm_s2_fault_desc *s2fd) ret =topup_mmu_memcache (s2fd->vcpu, memcache);if (ret)return ret; + }else if (is_protected_kvm_enabled ()) { + ret =topup_hyp_memcache_mapping (memcache); +if (ret) +return ret; }/*
Signature · 1 line
2.53.0
[email protected]
AI reviewRe: [PATCH v2 2/2] KVM: arm64: top up pKVM mapping cache for permission faults
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] Permission faults on a PMD during dirty logging skip page table memcache topup but still trigger block-to-page splits, causing an -ENOMEM guest abort. --
commit 26c4caa3c15677862ff91c344811bed70a7dead2
Author: Bradley Morgan <[email protected]>
KVM: arm64: top up pKVM mapping cache for permission faults
This commit allocates a fresh pkvm_mapping object for pKVM permission faults, as they may replace page entries with a block entry. It adds a topup_hyp_memcache_mapping() call in user_mem_abort() for this case.
Quoted 4 lines
arch/arm64/kvm/mmu.c +0
-0
[ ... ]
arch/arm64/kvm/mmu.c static int user_mem_abort(const struct kvm_s2_fault_desc *s2fd)
Quoted 13 lines
arch/arm64/kvm/mmu.c +4
-2@@ -2073,7 +2082,9 @@ static int user_mem_abort(const struct kvm_s2_fault_desc *s2fd)* Permission faults just need to update the existing leaf entry, * and so normally don't require allocations from the memcache. The * only exception to this is when dirty logging is enabled at runtime -* and a write fault needs to collapse a block entry into a table. +* and a write fault needs to collapse a block entry into a table. With +* pKVM, they may still need a fresh mapping object if the fault turns +* page entries into a block entry. */ memcache =get_mmu_memcache (s2fd->vcpu);if (!perm_fault || (memslot_is_logging (s2fd->memslot) && -kvm_is_write_fault (s2fd->vcpu))) { +kvm_is_write_fault (s2fd->vcpu))) {
[Severity: High]
This is a pre-existing issue, but can this condition lead to an -ENOMEM guest abort during dirty logging?
If a guest triggers an instruction permission fault (IABT) on a PMD while dirty logging is enabled, kvm_is_write_fault() evaluates to false. This skips topup_mmu_memcache().
Later in kvm_s2_fault_map(), dirty logging forces mapping_size to PAGE_SIZE, which mismatches the PMD_SIZE perm_fault_granule. This forces a call to kvm_pgtable_stage2_map() to split the PMD:
arch/arm64/kvm/mmu.c:kvm_s2_fault_map() {
...
} else {
ret = KVM_PGT_FN(kvm_pgtable_stage2_map)(pgt, gfn_to_gpa(gfn),
mapping_size,
__pfn_to_phys(pfn), prot,
memcache, flags);
}
}The split requires a page table allocation from the empty memcache, which fails with -ENOMEM and fatally aborts the guest.
Quoted 10 lines
arch/arm64/kvm/mmu.c +4
-0ret =topup_mmu_memcache (s2fd->vcpu, memcache);if (ret)return ret; + }else if (is_protected_kvm_enabled ()) { + ret =topup_hyp_memcache_mapping (memcache); +if (ret) +return ret; }/*
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Marc Zyngier
maintainerBradley,
Just a few things to keep in mind for your next contributions:
- If you are sending more than a single patch, add a cover letter.
- Don't send a v2 in reply to a v1. It messes the threading we are relying on, and makes it hard to ignore replies to an older version. Always send new series standalone.
- Don't immediately send a V2, even if (especially if!) a bot is pestering you. 34 minutes between versions is way too short (at least a few days is the norm).
Quoted 10 lines · Bradley Morgan
pKVM keeps its own mapping list for stage 2 operations. Its flush path uses that list directly, so it lost the PTE attribute check done by the generic stage 2 walker.
Record whether a mapping is cacheable and skip cache maintenance for mappings that are not cacheable.
Fixes e912efed485a ("KVM: arm64: Introduce the EL1 pKVM MMU")
What device memory gets mapped in an upstream pKVM guest that would require a backport to stable?
Quoted 4 lines
Signed-off-by Bradley Morgan <[email protected]>---
Changes in v2:
- Add patch 2 for the pKVM permission fault mapping cache bug.
This is the sort of information that goes in the cover letter.
Quoted 10 lines
arch/arm64/include/asm/kvm_pkvm.h | 1 + arch/arm64/kvm/pkvm.c | 8 +++++++- 2 files changed, 8 insertions(+), 1 deletion(-)arch/arm64/include/asm/kvm_pkvm.h +0
-0
arch/arm64/include/asm/kvm_pkvm.h struct pkvm_mapping { · line 199
Quoted 4 lines
arch/arm64/include/asm/kvm_pkvm.h +1
-0@@ -196,6 +196,7 @@ struct pkvm_mapping {u64 gfn;u64 pfn;u64 nr_pages; +bool cacheable;
Errr, no. That's a terrible idea.
This thing is already big enough, let's not add a bool right in the middle (use pahole to find out why this is bad). Given that nr_pages is for a range, and that the minimum page size uses 12 bits, the largest number of pages you can have here is 56-12=48 bit wide. That's another 16 bits worth of flags you can use.
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Quoted 61 lines · Marc Zyngier
Bradley,
Just a few things to keep in mind for your next contributions:
- If you are sending more than a single patch, add a cover letter.
- Don't send a v2 in reply to a v1. It messes the threading we are relying on, and makes it hard to ignore replies to an older version. Always send new series standalone.
- Don't immediately send a V2, even if (especially if!) a bot is pestering you. 34 minutes between versions is way too short (at least a few days is the norm).
Quoted 10 lines · Bradley Morgan
pKVM keeps its own mapping list for stage 2 operations. Its flush path uses that list directly, so it lost the PTE attribute check done by the generic stage 2 walker.
Record whether a mapping is cacheable and skip cache maintenance for mappings that are not cacheable.
Fixes e912efed485a ("KVM: arm64: Introduce the EL1 pKVM MMU")What device memory gets mapped in an upstream pKVM guest that would require a backport to stable?
Quoted 4 lines
Signed-off-by Bradley Morgan <[email protected]>---
Changes in v2:
- Add patch 2 for the pKVM permission fault mapping cache bug.This is the sort of information that goes in the cover letter.
Review
Quoted 6 lines
arch/arm64/include/asm/kvm_pkvm.h | 1 + arch/arm64/kvm/pkvm.c | 8 +++++++- 2 files changed, 8 insertions(+), 1 deletion(-)(quoted diff) +0
-0
b/arch/arm64/include/asm/kvm_pkvm.harch/arm64/include/asm/kvm_pkvm.h struct pkvm_mapping { · line 199
Quoted 8 lines
index
74fedd9c5ff0..d9dd8239910d100644arch/arm64/include/asm/kvm_pkvm.h +1
-0@@ -196,6 +196,7 @@ struct pkvm_mapping {u64 gfn;u64 pfn;u64 nr_pages; +bool cacheable;Errr, no. That's a terrible idea.
This thing is already big enough, let's not add a bool right in the middle (use pahole to find out why this is bad). Given that
nr_pagesis for a range, and that the minimum page size uses 12 bits, the largest number of pages you can have here is 56-12=48 bit wide. That's another 16 bits worth of flags you can use.
Quoted 5 lines
Thanks,
M.
thanks.
I'll go and do V3 with another sashiko suggestion. I'll fix your path too. I'll park V3 for a bit.
Thanks!
Marc Zyngier
maintainerQuoted 2 lines · Bradley Morgan
I'll go and do V3 with another sashiko suggestion. I'll fix your path too.
Before you do that, please verify that whatever Sashiko spits out makes any sense. I'm not convinced by its reply on v1 at all.
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Quoted 12 lines · Marc Zyngier
Quoted 2 lines · Bradley Morgan
I'll go and do V3 with another sashiko suggestion. I'll fix your path
too.
Before you do that, please verify that whatever Sashiko spits out makes any sense. I'm not convinced by its reply on v1 at all.
M.
Marc,
hi, I have verified sashikos concern.
I am out right now. So I will give a very short result.
Sashiko is being bit dramatic with the whole "Critical" rating, but it is real in another way
I'll explain it in the code in about 30-40 mins.
Thanks!
Quoted 22 lines · Bradley Morgan
Quoted 12 lines · Marc Zyngier
Quoted 2 lines · Bradley Morgan
I'll go and do V3 with another sashiko suggestion. I'll fix your path
too.
Before you do that, please verify that whatever Sashiko spits out makes any sense. I'm not convinced by its reply on v1 at all.
M.Marc,
hi, I have verified sashikos concern.
I am out right now. So I will give a very short result.
Quoted 7 lines
Sashiko is being bit dramatic with the whole "Critical" rating, but it is real in another way
I'll explain it in the code in about 30-40 mins.
Thanks!
hey, I'm back now. Let me explain
For the dirty logging report, I agree the bot's wording was not the greatest..
The issue is not best described as a *guaranteed* -ENOMEM guest abort.
The actual failure path seems to be:
user_mem_abort() skips topup_mmu_memcache() for permission faults, unless dirty logging is active andkvm_is_write_fault() is true.
For an instruction permission fault, kvm_is_write_fault() is false.
Dirty logging makes kvm_s2_resolve_vma_size() force max_map_size to PAGE_SIZE.
If the existing stage 2 entry is *still* a PMD, then kvm_s2_fault_map() has mapping_size == PAGE_SIZE but perm_fault_granule == PMD_SIZE, so it calls kvm_pgtable_stage2_map() instead of relax_perms().
The generic stage 2 walker then needs a page table page to split the PMD in stage2_map_walk_leaf().
So the real issue is that this path can enter the split path without having topped up the memcache. Eager splitting does not make it impossible....
since it is best effort and the default split chunk size is 0.
About the V3, I won't send the V3 yet. Until more people review / or it's been a few days
Thanks!