KVM: SEV: Drop user-triggerable WARN clean up REG_REGION
Sean Christopherson
maintainercover KVM: SEV: Drop user-triggerable WARN clean up REG_REGION
Drop a WARN in sev_pin_memory() that is comically easy to trigger, and then clean up the code a bit.
*** WARNING ***
The last patch impacts KVM's ABI due to a size restriction that's applied to kvmalloc() but not vmalloc(). If my math is correct (definitely a big "if), it will only impact VMMs that attempt to do KVM_MEMORY_ENCRYPT_REG_REGION with a single region of ~1TiB+.
Liam, I Cc'd you specifically because it looks like Oracle supports SEV VMs with more than 1TiB of _total_ memory. I assume that's spread across multiple memslots and thus multiple KVM_MEMORY_ENCRYPT_REG_REGION calls, but I wanted to double check before potentially breaking userspace.
Sean Christopherson (5):
KVM: SEV: Drop WARN on large size for KVM_MEMORY_ENCRYPT_REG_REGION
KVM: SEV: Drop useless sanity checks in sev_mem_enc_register_region()
KVM: SEV: Disallow pinning more pages than exist in the system
KVM: SEV: Use PFN_DOWN() to simplify "number of pages" math when
pinning memory
KVM: SEV: Use kvzalloc_objs() when pinning userpages
arch/x86/kvm/svm/sev.c | 51 +++++++++++++++++++++--------------------- 1 file changed, 25 insertions(+), 26 deletions(-)
base-commit: bfd7f4adc1230373c25e1b787a6f1ee407eb0656
Signature · 1 line
2.53.0.851.ga537e3e6e9-goog
Sean Christopherson
maintainerPATCH 1/5 KVM: SEV: Drop WARN on large size for KVM_MEMORY_ENCRYPT_REG_REGION
Drop the WARN in sev_pin_memory() on npages overflowing an int, as the WARN is comically trivially to trigger from userspace, e.g. by doing:
struct kvm_enc_region range = {
.addr = 0,
.size = -1ul,
};
__vm_ioctl(vm, KVM_MEMORY_ENCRYPT_REG_REGION, &range);Note, the checks in sev_mem_enc_register_region() that presumably exist to verify the incoming address+size are completely worthless, as both "addr" and "size" are u64s and SEV is 64-bit only, i.e. they _can't_ be greater than ULONG_MAX. That wart will be cleaned up in the near future.
if (range->addr > ULONG_MAX || range->size > ULONG_MAX) return -EINVAL;
Opportunistically add a comment to explain why the code calculates the number of pages the "hard" way, e.g. instead of just shifting @ulen.
--- arch/x86/kvm/svm/sev.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)
arch/x86/kvm/svm/sev.c +7 -4
@@ -690,10 +690,16 @@ static struct page **sev_pin_memory(struct kvm *kvm, unsigned long uaddr,if (ulen ==0 || uaddr + ulen < uaddr)return ERR_PTR (-EINVAL ); -/* Calculate number of pages. */ +/* +* Calculate the number of pages that need to be pinned to cover the +* entire range. Note! This isn't simply ulen >> PAGE_SHIFT, as KVM +* doesn't require the incoming address+size to be page aligned! +*/ first = (uaddr &PAGE_MASK ) >>PAGE_SHIFT ; last = ((uaddr + ulen -1 ) &PAGE_MASK ) >>PAGE_SHIFT ; npages = (last - first +1 ); +if (npages >INT_MAX ) +return ERR_PTR (-EINVAL ); locked = sev->pages_locked + npages; lock_limit =rlimit (RLIMIT_MEMLOCK ) >>PAGE_SHIFT ;
@@ -702,9 +708,6 @@ static struct page **sev_pin_memory(struct kvm *kvm, unsigned long uaddr,return ERR_PTR (-ENOMEM ); } -if (WARN_ON_ONCE (npages >INT_MAX )) -return ERR_PTR (-EINVAL ); -/* Avoid using vmalloc for smaller buffers. */ size = npages *sizeof (struct page *);if (size >PAGE_SIZE )
Signature · 1 line
2.53.0.851.ga537e3e6e9-goog
Sean Christopherson
maintainerPATCH 2/5 KVM: SEV: Drop useless sanity checks in sev_mem_enc_register_region()
In reply to Sean ChristophersonDrop sev_mem_enc_register_region()'s sanity checks on the incoming address and size, as SEV is 64-bit only, making ULONG_MAX a 64-bit, all-ones value, and thus making it impossible for kvm_enc_region.{addr,size} to be greater than ULONG_MAX.
Note, sev_pin_memory() verifies the incoming address is non-NULL (which isn't strictly required, but whatever), and that addr+size don't wrap to zero (which _is_ needed and what really needs to be guarded against).
Note #2, pin_user_pages_fast() guards against the end address walking into kernel address space, so lack of an access_ok() check is also safe (maybe not ideal, but safe).
No functional change intended (the generated code is literally the same, i.e. the compiler was smart enough to know the checks were useless).
--- arch/x86/kvm/svm/sev.c | 3 --- 1 file changed, 3 deletions(-)
arch/x86/kvm/svm/sev.c +0 -3
@@ -2711,9 +2711,6 @@ int sev_mem_enc_register_region(struct kvm *kvm,if (is_mirroring_enc_context (kvm))return -EINVAL ; -if (range->addr >ULONG_MAX || range->size >ULONG_MAX ) -return -EINVAL ; - region =kzalloc_obj (*region,GFP_KERNEL_ACCOUNT );if (!region)return -ENOMEM ;
Signature · 1 line
2.53.0.851.ga537e3e6e9-goog
Sean Christopherson
maintainerPATCH 3/5 KVM: SEV: Disallow pinning more pages than exist in the system
In reply to Sean ChristophersonExplicitly disallow pinning more pages for an SEV VM than exist in the system to defend against absurd userspace requests without relying on somewhat arbitrary kernel functionality to prevent truly stupid KVM behavior. E.g. even with the INT_MAX check, userspace can request that KVM pin nearly 8TiB of memory, regardless of how much RAM exists in the system.
Opportunistically rename "locked" to a more descriptive "total_npages".
--- arch/x86/kvm/svm/sev.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-)
arch/x86/kvm/svm/sev.c +9 -5
@@ -680,7 +680,7 @@ static struct page **sev_pin_memory(struct kvm *kvm, unsigned long uaddr,struct kvm_sev_info *sev =to_kvm_sev_info (kvm);unsigned long npages, size;int npinned; -unsigned long locked, lock_limit; +unsigned long total_npages, lock_limit;struct page **pages;unsigned long first, last;int ret;
@@ -701,10 +701,14 @@ static struct page **sev_pin_memory(struct kvm *kvm, unsigned long uaddr,if (npages >INT_MAX )return ERR_PTR (-EINVAL ); - locked = sev->pages_locked + npages; + total_npages = sev->pages_locked + npages; +if (total_npages >totalram_pages ()) +return ERR_PTR (-EINVAL ); + lock_limit =rlimit (RLIMIT_MEMLOCK ) >>PAGE_SHIFT ; -if (locked > lock_limit && !capable (CAP_IPC_LOCK )) { -pr_err ("SEV: %lu locked pages exceed the lock limit of %lu.\n" , locked, lock_limit); +if (total_npages > lock_limit && !capable (CAP_IPC_LOCK )) { +pr_err ("SEV: %lu total pages would exceed the lock limit of %lu.\n" , + total_npages, lock_limit);return ERR_PTR (-ENOMEM ); }
@@ -727,7 +731,7 @@ static struct page **sev_pin_memory(struct kvm *kvm, unsigned long uaddr, } *n = npages; - sev->pages_locked = locked; + sev->pages_locked = total_npages;return pages;
Signature · 1 line
2.53.0.851.ga537e3e6e9-goog
Sean Christopherson
maintainerPATCH 4/5 KVM: SEV: Use PFN_DOWN() to simplify "number of pages" math when pinning memory
In reply to Sean ChristophersonUse PFN_DOWN() instead of open coded equivalents in sev_pin_memory() to simplify the code and make it easier to read.
No functional change intended (verified before and after versions of the generated code are identical).
--- arch/x86/kvm/svm/sev.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)
arch/x86/kvm/svm/sev.c +2 -5
@@ -682,7 +682,6 @@ static struct page **sev_pin_memory(struct kvm *kvm, unsigned long uaddr,int npinned;unsigned long total_npages, lock_limit;struct page **pages; -unsigned long first, last;int ret;lockdep_assert_held (&kvm->lock);
@@ -692,12 +691,10 @@ static struct page **sev_pin_memory(struct kvm *kvm, unsigned long uaddr,/* * Calculate the number of pages that need to be pinned to cover the -* entire range. Note! This isn't simply ulen >> PAGE_SHIFT, as KVM +* entire range. Note! This isn't simply PFN_DOWN(ulen), as KVM * doesn't require the incoming address+size to be page aligned! */ - first = (uaddr &PAGE_MASK ) >>PAGE_SHIFT ; - last = ((uaddr + ulen -1 ) &PAGE_MASK ) >>PAGE_SHIFT ; - npages = (last - first +1 ); + npages =PFN_DOWN (uaddr + ulen -1 ) -PFN_DOWN (uaddr) +1 ;if (npages >INT_MAX )return ERR_PTR (-EINVAL );
Signature · 1 line
2.53.0.851.ga537e3e6e9-goog
Sean Christopherson
maintainerPATCH 5/5 KVM: SEV: Use kvzalloc_objs() when pinning userpages
In reply to Sean ChristophersonUse kvzalloc_objs() instead of sev_pin_memory()'s open coded (rough) equivalent to harden the code and
Note! This sanity check in __kvmalloc_node_noprof()
/* Don't even allow crazy sizes */
if (unlikely(size > INT_MAX)) {
WARN_ON_ONCE(!(flags & __GFP_NOWARN));
return NULL;
}will artificially limit the maximum size of any single pinned region to just under 1TiB. While there do appear to be providers that support SEV VMs with more than 1TiB of _total_ memory, it's unlikely any KVM-based providers pin 1TiB in a single request.
Allocate with NOWARN so that fuzzers can't trip the WARN_ON_ONCE() when they inevitably run on systems with copious amounts of RAM, i.e. when they can get by KVM's "total_npages > totalram_pages()" restriction.
Note #2, KVM's usage of vmalloc()+kmalloc() instead of kvmalloc() predates commit 7661809d493b ("mm: don't allow oversized kvmalloc() calls") by 4+ years (see commit 89c505809052 ("KVM: SVM: Add support for KVM_SEV_LAUNCH_UPDATE_DATA command"). I.e. the open coded behavior wasn't intended to avoid the aforementioned sanity check. The implementation appears to be pure oversight at the time the code was written, as it showed up in v3[1] of the early RFCs, whereas as v2[2] simply used kmalloc().
--- arch/x86/kvm/svm/sev.c | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-)
arch/x86/kvm/svm/sev.c +9 -11
@@ -678,11 +678,9 @@ static struct page **sev_pin_memory(struct kvm *kvm, unsigned long uaddr, unsigned int flags) {struct kvm_sev_info *sev =to_kvm_sev_info (kvm); -unsigned long npages, size; -int npinned; -unsigned long total_npages, lock_limit; +unsigned long npages, total_npages, lock_limit;struct page **pages; -int ret; +int npinned, ret;lockdep_assert_held (&kvm->lock);
@@ -709,13 +707,13 @@ static struct page **sev_pin_memory(struct kvm *kvm, unsigned long uaddr,return ERR_PTR (-ENOMEM ); } -/* Avoid using vmalloc for smaller buffers. */ - size = npages *sizeof (struct page *); -if (size >PAGE_SIZE ) - pages =__vmalloc (size,GFP_KERNEL_ACCOUNT ); -else - pages =kmalloc (size,GFP_KERNEL_ACCOUNT ); - +/* +* Don't WARN if the kernel (rightly) thinks the total size is absurd, +* i.e. rely on the kernel to reject outrageous range sizes. The above +* check on the number of pages is purely to avoid truncation as +* pin_user_pages_fast() takes the number of pages as a 32-bit int. +*/ + pages =kvzalloc_objs (*pages, npages,GFP_KERNEL_ACCOUNT | __GFP_NOWARN);if (!pages)return ERR_PTR (-ENOMEM );
Signature · 1 line
2.53.0.851.ga537e3e6e9-goog
Quoted 10 lines · Sean Christopherson
Drop a WARN in
sev_pin_memory()that is comically easy to trigger, and then clean up the code a bit.*** WARNING ***
The last patch impacts KVM's ABI due to a size restriction that's applied to
kvmalloc()but notvmalloc(). If my math is correct (definitely a big "if), it will only impact VMMs that attempt to doKVM_MEMORY_ENCRYPT_REG_REGIONwith a single region of ~1TiB+.
Quoted 4 lines
Liam, I Cc'd you specifically because it looks like Oracle supports SEV VMs with more than 1TiB of _total_ memory. I assume that's spread across multiple memslots and thus multiple
KVM_MEMORY_ENCRYPT_REG_REGIONcalls, but I wanted to double check before potentially breaking userspace.
I tested the series with various sized large memory SEV VMs and all seemed good to me. For the series:
Quoted 14 lines
Sean Christopherson (5):
KVM: SEV: Drop WARN on large size forKVM_MEMORY_ENCRYPT_REG_REGION
KVM: SEV: Drop useless sanity checks insev_mem_enc_register_region()
KVM: SEV: Disallow pinning more pages than exist in the system
KVM: SEV: UsePFN_DOWN()to simplify "number of pages" math when
pinning memory
KVM: SEV: Usekvzalloc_objs()when pinning userpagesarch/x86/kvm/svm/sev.c | 51 +++++++++++++++++++++--------------------- 1 file changed, 25 insertions(+), 26 deletions(-)base-commit:
bfd7f4adc1230373c25e1b787a6f1ee407eb0656
Quoted 11 lines · Sean Christopherson
Drop a WARN in
sev_pin_memory()that is comically easy to trigger, and then clean up the code a bit.*** WARNING ***
The last patch impacts KVM's ABI due to a size restriction that's applied to
kvmalloc()but notvmalloc(). If my math is correct (definitely a big "if), it will only impact VMMs that attempt to doKVM_MEMORY_ENCRYPT_REG_REGIONwith a single region of ~1TiB+.[...]
Applied to kvm-x86 svm. Like the other large set of SEV fixes, I'll omit this from the pull request if there are any problems.
[1/5] KVM: SEV: Drop WARN on large size for KVM_MEMORY_ENCRYPT_REG_REGION
https://github.com/kvm-x86/linux/commit/8acffeef5ef7 [2/5] KVM: SEV: Drop useless sanity checks in sev_mem_enc_register_region() https://github.com/kvm-x86/linux/commit/12a8ff869ddc [3/5] KVM: SEV: Disallow pinning more pages than exist in the system https://github.com/kvm-x86/linux/commit/6d71f9349d9b [4/5] KVM: SEV: Use PFN_DOWN() to simplify "number of pages" math when pinning memory https://github.com/kvm-x86/linux/commit/7ad02ff1e4a4 [5/5] KVM: SEV: Use kvzalloc_objs() when pinning userpages https://github.com/kvm-x86/linux/commit/a7f53694d591