Skip to contentSkip to conversation

KVM: Fix for a mostly benign gpc WARN

PATCH Applied Sean Christopherson 3 participants 15 messages Mar 20, 2024 – Apr 9, 2024 KVM

lore
Share & tools
lore

Sean Christopherson

author reviewer
#0 lore raw

cover KVM: Fix for a mostly benign gpc WARN

Fix a bug found by syzkaller, thanks to a new WARN sanity check, where KVM marks a gfn_to_pfn_cache as active without actually setting gpc->gpa or any other metadata. On top, harden against _directly_ setting gpc->gpa to KVM's magic INVALID_GPA, which would also fail the sanity check.

Sean Christopherson (3):
KVM: Add helpers to consolidate gfn_to_pfn_cache's page split check
KVM: Check validity of offset+length of gfn_to_pfn_cache prior to
activation
KVM: Explicitly disallow activatating a gfn_to_pfn_cache with
INVALID_GPA

Sean Christopherson (3):
KVM: Add helpers to consolidate gfn_to_pfn_cache's page split check
KVM: Check validity of offset+length of gfn_to_pfn_cache prior to
activation
KVM: Explicitly disallow activatating a gfn_to_pfn_cache with
INVALID_GPA

 virt/kvm/pfncache.c | 48 ++++++++++++++++++++++++++++++++-------------
 1 file changed, 34 insertions(+), 14 deletions(-)

base-commit: 964d0c614c7f71917305a5afdca9178fe8231434

Signature · 1 line
2.44.0.291.gc1ea87d7ee-goog
author reviewer
#1 lore raw

PATCH 1/3 KVM: Add helpers to consolidate gfn_to_pfn_cache's page split check

Add a helper to check that the incoming length for a gfn_to_pfn_cache is valid with respect to the cache's GPA and/or HVA. To avoid activating a cache with a bogus GPA, a future fix will fork the page split check in the inner refresh path into activate() and the public rerfresh() APIs, at which point KVM will check the length in three separate places.

Deliberately keep the "page offset" logic open coded, as the only other path that consumes the offset, __kvm_gpc_refresh(), already needs to differentiate between GPA-based and HVA-based caches, and it's not obvious that using a helper is a net positive in overall code readability.

Note, for GPA-based caches, this has a subtle side effect of using the GPA instead of the resolved HVA in the check() path, but that should be a nop as the HVA offset is derived from the GPA, i.e. the two offsets are identical, barring a KVM bug.

---
 virt/kvm/pfncache.c | 27 +++++++++++++++++++--------
 1 file changed, 19 insertions(+), 8 deletions(-)
virt/kvm/pfncache.c +19 -8
index 4e07112a24c2..8f2121b5f2a0 100644
@@ -57,6 +57,19 @@ void gfn_to_pfn_cache_invalidate_start(struct kvm *kvm, unsigned long start,
 	spin_unlock(&kvm->gpc_lock);
 }
 
+static bool kvm_gpc_is_valid_len(gpa_t gpa, unsigned long uhva,
+				 unsigned long len)
+{
+	unsigned long offset = kvm_is_error_gpa(gpa) ? offset_in_page(uhva) :
+						       offset_in_page(gpa);
+
+	/*
+	 * The cached access must fit within a single page. The 'len' argument
+	 * to activate() and refresh() exists only to enforce that.
+	 */
+	return offset + len <= PAGE_SIZE;
+}
+
 bool kvm_gpc_check(struct gfn_to_pfn_cache *gpc, unsigned long len)
 {
 	struct kvm_memslots *slots = kvm_memslots(gpc->kvm);
@@ -74,7 +87,7 @@ bool kvm_gpc_check(struct gfn_to_pfn_cache *gpc, unsigned long len)
 	if (kvm_is_error_hva(gpc->uhva))
 		return false;
 
-	if (offset_in_page(gpc->uhva) + len > PAGE_SIZE)
+	if (!kvm_gpc_is_valid_len(gpc->gpa, gpc->uhva, len))
 		return false;
 
 	if (!gpc->valid)
@@ -247,13 +260,7 @@ static int __kvm_gpc_refresh(struct gfn_to_pfn_cache *gpc, gpa_t gpa, unsigned l
 	if (WARN_ON_ONCE(kvm_is_error_gpa(gpa) == kvm_is_error_hva(uhva)))
 		return -EINVAL;
 
-	/*
-	 * The cached acces must fit within a single page. The 'len' argument
-	 * exists only to enforce that.
-	 */
-	page_offset = kvm_is_error_gpa(gpa) ? offset_in_page(uhva) :
-					      offset_in_page(gpa);
-	if (page_offset + len > PAGE_SIZE)
+	if (!kvm_gpc_is_valid_len(gpa, uhva, len))
 		return -EINVAL;
 
 	lockdep_assert_held(&gpc->refresh_lock);
@@ -270,6 +277,8 @@ static int __kvm_gpc_refresh(struct gfn_to_pfn_cache *gpc, gpa_t gpa, unsigned l
 	old_uhva = PAGE_ALIGN_DOWN(gpc->uhva);
 
 	if (kvm_is_error_gpa(gpa)) {
+		page_offset = offset_in_page(uhva);
+
 		gpc->gpa = INVALID_GPA;
 		gpc->memslot = NULL;
 		gpc->uhva = PAGE_ALIGN_DOWN(uhva);
@@ -279,6 +288,8 @@ static int __kvm_gpc_refresh(struct gfn_to_pfn_cache *gpc, gpa_t gpa, unsigned l
 	} else {
 		struct kvm_memslots *slots = kvm_memslots(gpc->kvm);
 
+		page_offset = offset_in_page(gpa);
+
 		if (gpc->gpa != gpa || gpc->generation != slots->generation ||
 		    kvm_is_error_hva(gpc->uhva)) {
 			gfn_t gfn = gpa_to_gfn(gpa);
Signature · 1 line
2.44.0.291.gc1ea87d7ee-goog
Reviewed-by on v1 1/3
#2 lore raw

Re: [PATCH 1/3] KVM: Add helpers to consolidate gfn_to_pfn_cache's page split check

Quoted 11 lines · Sean Christopherson

Add a helper to check that the incoming length for a gfn_to_pfn_cache is valid with respect to the cache's GPA and/or HVA.  To avoid activating a cache with a bogus GPA, a future fix will fork the page split check in the inner refresh path into activate() and the public rerfresh() APIs, at which point KVM will check the length in three separate places.

Deliberately keep the "page offset" logic open coded, as the only other path that consumes the offset, __kvm_gpc_refresh(), already needs to differentiate between GPA-based and HVA-based caches, and it's not obvious that using a helper is a net positive in overall code readability.

Quoted 6 lines

Note, for GPA-based caches, this has a subtle side effect of using the GPA instead of the resolved HVA in the check() path, but that should be a nop as the HVA offset is derived from the GPA, i.e. the two offsets are identical, barring a KVM bug.

Reviewed-by on v1 1/3
#3 lore raw

Re: [PATCH 1/3] KVM: Add helpers to consolidate gfn_to_pfn_cache's page split check

In reply to Sean Christopherson
Quoted 4 lines · Sean Christopherson

Add a helper to check that the incoming length for a gfn_to_pfn_cache is valid with respect to the cache's GPA and/or HVA. To avoid activating a cache with a bogus GPA, a future fix will fork the page split check in the inner refresh path into activate() and the public rerfresh() APIs, at

nit: typo

Quoted 12 lines

which point KVM will check the length in three separate places.

Deliberately keep the "page offset" logic open coded, as the only other path that consumes the offset, __kvm_gpc_refresh(), already needs to differentiate between GPA-based and HVA-based caches, and it's not obvious that using a helper is a net positive in overall code readability.

Note, for GPA-based caches, this has a subtle side effect of using the GPA instead of the resolved HVA in the check() path, but that should be a nop as the HVA offset is derived from the GPA, i.e. the two offsets are identical, barring a KVM bug.

Quoted 5 lines
---
  virt/kvm/pfncache.c | 27 +++++++++++++++++++--------
  1 file changed, 19 insertions(+), 8 deletions(-)
author reviewer
#4 lore raw

PATCH 2/3 KVM: Check validity of offset+length of gfn_to_pfn_cache prior to activation

In reply to Sean Christopherson

When activating a gfn_to_pfn_cache, verify that the offset+length is sane and usable before marking the cache active. Letting __kvm_gpc_refresh() detect the problem results in a cache being marked active without setting the GPA (or any other fields), which in turn results in KVM trying to refresh a cache with INVALID_GPA.

Attempting to refresh a cache with INVALID_GPA isn't functionally problematic, but it runs afoul of the sanity check that exactly one of GPA or userspace HVA is valid, i.e. that a cache is either GPA-based or HVA-based.

Reported-by [email protected]
Fixes 721f5b0dda78 ("KVM: pfncache: allow a cache to be activated with a fixed (userspace) HVA")
Cc David Woodhouse <[email protected]>
Cc Paul Durrant <[email protected]>
---
 virt/kvm/pfncache.c | 16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)
virt/kvm/pfncache.c +9 -7
index 8f2121b5f2a0..91b0e329006b 100644
@@ -245,8 +245,7 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc)
 	return -EFAULT;
 }
 
-static int __kvm_gpc_refresh(struct gfn_to_pfn_cache *gpc, gpa_t gpa, unsigned long uhva,
-			     unsigned long len)
+static int __kvm_gpc_refresh(struct gfn_to_pfn_cache *gpc, gpa_t gpa, unsigned long uhva)
 {
 	unsigned long page_offset;
 	bool unmap_old = false;
@@ -260,9 +259,6 @@ static int __kvm_gpc_refresh(struct gfn_to_pfn_cache *gpc, gpa_t gpa, unsigned l
 	if (WARN_ON_ONCE(kvm_is_error_gpa(gpa) == kvm_is_error_hva(uhva)))
 		return -EINVAL;
 
-	if (!kvm_gpc_is_valid_len(gpa, uhva, len))
-		return -EINVAL;
-
 	lockdep_assert_held(&gpc->refresh_lock);
 
 	write_lock_irq(&gpc->lock);
@@ -365,6 +361,9 @@ int kvm_gpc_refresh(struct gfn_to_pfn_cache *gpc, unsigned long len)
 
 	guard(mutex)(&gpc->refresh_lock);
 
+	if (!kvm_gpc_is_valid_len(gpc->gpa, gpc->uhva, len))
+		return -EINVAL;
+
 	/*
 	 * If the GPA is valid then ignore the HVA, as a cache can be GPA-based
 	 * or HVA-based, not both.  For GPA-based caches, the HVA will be
@@ -372,7 +371,7 @@ int kvm_gpc_refresh(struct gfn_to_pfn_cache *gpc, unsigned long len)
 	 */
 	uhva = kvm_is_error_gpa(gpc->gpa) ? gpc->uhva : KVM_HVA_ERR_BAD;
 
-	return __kvm_gpc_refresh(gpc, gpc->gpa, uhva, len);
+	return __kvm_gpc_refresh(gpc, gpc->gpa, uhva);
 }
 
 void kvm_gpc_init(struct gfn_to_pfn_cache *gpc, struct kvm *kvm)
@@ -392,6 +391,9 @@ static int __kvm_gpc_activate(struct gfn_to_pfn_cache *gpc, gpa_t gpa, unsigned
 {
 	struct kvm *kvm = gpc->kvm;
 
+	if (!kvm_gpc_is_valid_len(gpa, uhva, len))
+		return -EINVAL;
+
 	guard(mutex)(&gpc->refresh_lock);
 
 	if (!gpc->active) {
@@ -411,7 +413,7 @@ static int __kvm_gpc_activate(struct gfn_to_pfn_cache *gpc, gpa_t gpa, unsigned
 		gpc->active = true;
 		write_unlock_irq(&gpc->lock);
 	}
-	return __kvm_gpc_refresh(gpc, gpa, uhva, len);
+	return __kvm_gpc_refresh(gpc, gpa, uhva);
 }
 
 int kvm_gpc_activate(struct gfn_to_pfn_cache *gpc, gpa_t gpa, unsigned long len)
Signature · 1 line
2.44.0.291.gc1ea87d7ee-goog
Reviewed-by on v1 2/3
#5 lore raw

Re: [PATCH 2/3] KVM: Check validity of offset+length of gfn_to_pfn_cache prior to activation

Quoted 6 lines · Sean Christopherson

When activating a gfn_to_pfn_cache, verify that the offset+length is sane and usable before marking the cache active.  Letting __kvm_gpc_refresh() detect the problem results in a cache being marked active without setting the GPA (or any other fields), which in turn results in KVM trying to refresh a cache with INVALID_GPA.

Quoted 8 lines

Attempting to refresh a cache with INVALID_GPA isn't functionally problematic, but it runs afoul of the sanity check that exactly one of GPA or userspace HVA is valid, i.e. that a cache is either GPA-based or HVA-based.

Reported-by [email protected]
Fixes 721f5b0dda78 ("KVM: pfncache: allow a cache to be activated with a fixed (userspace) HVA")
Reviewed-by on v1 2/3
#6 lore raw

Re: [PATCH 2/3] KVM: Check validity of offset+length of gfn_to_pfn_cache prior to activation

In reply to Sean Christopherson
Quoted 17 lines · Sean Christopherson

When activating a gfn_to_pfn_cache, verify that the offset+length is sane and usable before marking the cache active. Letting __kvm_gpc_refresh() detect the problem results in a cache being marked active without setting the GPA (or any other fields), which in turn results in KVM trying to refresh a cache with INVALID_GPA.

Attempting to refresh a cache with INVALID_GPA isn't functionally problematic, but it runs afoul of the sanity check that exactly one of GPA or userspace HVA is valid, i.e. that a cache is either GPA-based or HVA-based.

Reported-by [email protected]
Fixes 721f5b0dda78 ("KVM: pfncache: allow a cache to be activated with a fixed (userspace) HVA")
Cc David Woodhouse <[email protected]>
Cc Paul Durrant <[email protected]>
Quoted 4 lines
---
  virt/kvm/pfncache.c | 16 +++++++++-------
  1 file changed, 9 insertions(+), 7 deletions(-)
author reviewer
#7 lore raw

PATCH 3/3 KVM: Explicitly disallow activatating a gfn_to_pfn_cache with INVALID_GPA

In reply to Sean Christopherson

Explicit disallow activating a gfn_to_pfn_cache with an error gpa, i.e. INVALID_GPA, to ensure that KVM doesn't mistake a GPA-based cache for an HVA-based cache (KVM uses INVALID_GPA as a magic value to differentiate between GPA-based and HVA-based caches).

WARN if KVM attempts to activate a cache with INVALID_GPA, purely so that new caches need to at least consider what to do with a "bad" GPA, as all existing usage of kvm_gpc_activate() guarantees gpa != INVALID_GPA. I.e. removing the WARN in the future is completely reasonable if doing so would yield cleaner/better code overall.

---
 virt/kvm/pfncache.c | 7 +++++++
 1 file changed, 7 insertions(+)
virt/kvm/pfncache.c +7 -0
index 91b0e329006b..f618719644e0 100644
@@ -418,6 +418,13 @@ static int __kvm_gpc_activate(struct gfn_to_pfn_cache *gpc, gpa_t gpa, unsigned
 
 int kvm_gpc_activate(struct gfn_to_pfn_cache *gpc, gpa_t gpa, unsigned long len)
 {
+	/*
+	 * Explicitly disallow INVALID_GPA so that the magic value can be used
+	 * by KVM to differentiate between GPA-based and HVA-based caches.
+	 */
+	if (WARN_ON_ONCE(kvm_is_error_gpa(gpa)))
+		return -EINVAL;
+
 	return __kvm_gpc_activate(gpc, gpa, KVM_HVA_ERR_BAD, len);
 }
 
Signature · 1 line
2.44.0.291.gc1ea87d7ee-goog
Reviewed-by on v1 3/3
#8 lore raw

Re: [PATCH 3/3] KVM: Explicitly disallow activatating a gfn_to_pfn_cache with INVALID_GPA

Quoted 5 lines · Sean Christopherson

Explicit disallow activating a gfn_to_pfn_cache with an error gpa, i.e. INVALID_GPA, to ensure that KVM doesn't mistake a GPA-based cache for an HVA-based cache (KVM uses INVALID_GPA as a magic value to differentiate between GPA-based and HVA-based caches).

Quoted 7 lines

WARN if KVM attempts to activate a cache with INVALID_GPA, purely so that new caches need to at least consider what to do with a "bad" GPA, as all existing usage of kvm_gpc_activate() guarantees gpa != INVALID_GPA.  I.e. removing the WARN in the future is completely reasonable if doing so would yield cleaner/better code overall.

Reviewed-by on v1 3/3
#9 lore raw

Re: [PATCH 3/3] KVM: Explicitly disallow activatating a gfn_to_pfn_cache with INVALID_GPA

In reply to Sean Christopherson
Quoted 11 lines · Sean Christopherson

Explicit disallow activating a gfn_to_pfn_cache with an error gpa, i.e. INVALID_GPA, to ensure that KVM doesn't mistake a GPA-based cache for an HVA-based cache (KVM uses INVALID_GPA as a magic value to differentiate between GPA-based and HVA-based caches).

WARN if KVM attempts to activate a cache with INVALID_GPA, purely so that new caches need to at least consider what to do with a "bad" GPA, as all existing usage of kvm_gpc_activate() guarantees gpa != INVALID_GPA. I.e. removing the WARN in the future is completely reasonable if doing so would yield cleaner/better code overall.

Quoted 5 lines
---
  virt/kvm/pfncache.c | 7 +++++++
  1 file changed, 7 insertions(+)
#10 lore raw
In reply to Sean Christopherson
Quoted 11 lines · Sean Christopherson

Fix a bug found by syzkaller, thanks to a new WARN sanity check, where KVM marks a gfn_to_pfn_cache as active without actually setting gpc->gpa or any other metadata.  On top, harden against _directly_ setting gpc->gpa to KVM's magic INVALID_GPA, which would also fail the sanity check.

Sean Christopherson (3):
  KVM: Add helpers to consolidate gfn_to_pfn_cache's page split check
  KVM: Check validity of offset+length of gfn_to_pfn_cache prior to activation
  KVM: Explicitly disallow activatating a gfn_to_pfn_cache with INVALID_GPA

It looks like these conflict with
https://lore.kernel.org/kvm/[email protected]/

Want to arrange them to come after it?

author reviewer
#11 lore raw
Quoted 13 lines · David Woodhouse
Quoted 11 lines · Sean Christopherson

Fix a bug found by syzkaller, thanks to a new WARN sanity check, where KVM marks a gfn_to_pfn_cache as active without actually setting gpc->gpa or any other metadata.  On top, harden against _directly_ setting gpc->gpa to KVM's magic INVALID_GPA, which would also fail the sanity check.

Sean Christopherson (3):
  KVM: Add helpers to consolidate gfn_to_pfn_cache's page split check
  KVM: Check validity of offset+length of gfn_to_pfn_cache prior to activation
  KVM: Explicitly disallow activatating a gfn_to_pfn_cache with INVALID_GPA

Quoted 4 lines

It looks like these conflict with
https://lore.kernel.org/kvm/[email protected]/

Want to arrange them to come after it?

Very belated, yes. Though by the time you read this, they should be in kvm-x86/next.

#12 lore raw
Quoted 14 lines · Sean Christopherson
Quoted 13 lines · David Woodhouse
Quoted 11 lines · Sean Christopherson

Fix a bug found by syzkaller, thanks to a new WARN sanity check, where KVM marks a gfn_to_pfn_cache as active without actually setting gpc->gpa or any other metadata.  On top, harden against _directly_ setting gpc->gpa to KVM's magic INVALID_GPA, which would also fail the sanity check.

Sean Christopherson (3):
  KVM: Add helpers to consolidate gfn_to_pfn_cache's page split check
  KVM: Check validity of offset+length of gfn_to_pfn_cache prior to activation
  KVM: Explicitly disallow activatating a gfn_to_pfn_cache with INVALID_GPA

Quoted 7 lines
Quoted 4 lines

It looks like these conflict with
https://lore.kernel.org/kvm/[email protected]/

Want to arrange them to come after it?

Very belated, yes.  Though by the time you read this, they should be in kvm-x86/next.

Did that 'yes' mean 'no'? Because your three patches are in, but you didn't arrange them to come after my 'clean up rwlock abuse' patch, as you seemed to be saying 'yes' to...

author reviewer
#13 lore raw
Quoted 18 lines · David Woodhouse
Quoted 17 lines · Sean Christopherson
Quoted 16 lines · David Woodhouse
Quoted 11 lines · Sean Christopherson

Fix a bug found by syzkaller, thanks to a new WARN sanity check, where KVM marks a gfn_to_pfn_cache as active without actually setting gpc->gpa or any other metadata.  On top, harden against _directly_ setting gpc->gpa to KVM's magic INVALID_GPA, which would also fail the sanity check.

Sean Christopherson (3):
  KVM: Add helpers to consolidate gfn_to_pfn_cache's page split check
  KVM: Check validity of offset+length of gfn_to_pfn_cache prior to activation
  KVM: Explicitly disallow activatating a gfn_to_pfn_cache with INVALID_GPA

It looks like these conflict with
https://lore.kernel.org/kvm/[email protected]/

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

Want to arrange them to come after it?

Very belated, yes.  Though by the time you read this, they should be in kvm-x86/next.

Did that 'yes' mean 'no'? Because your three patches are in, but you didn't arrange them to come after my 'clean up rwlock abuse' patch, as you seemed to be saying 'yes' to...

Doh, I misread your question, multiple times. I thought you were asking if I wanted you to arrange your patches after this series.

Your series goes on top because I want to land this series in 6.9 to fix the syzkaller splat (which was effectively introduced in 6.9), whereas your patch is 6.10 material.

author reviewer Applied
#14 lore raw
In reply to Sean Christopherson
Quoted 5 lines · Sean Christopherson

Fix a bug found by syzkaller, thanks to a new WARN sanity check, where KVM marks a gfn_to_pfn_cache as active without actually setting gpc->gpa or any other metadata. On top, harden against _directly_ setting gpc->gpa to KVM's magic INVALID_GPA, which would also fail the sanity check.

Quoted 8 lines

Sean Christopherson (3):
KVM: Add helpers to consolidate gfn_to_pfn_cache's page split check
KVM: Check validity of offset+length of gfn_to_pfn_cache prior to
activation
KVM: Explicitly disallow activatating a gfn_to_pfn_cache with
INVALID_GPA

[...]

Applied to kvm-x86 fixes, thanks!

[1/3] KVM: Add helpers to consolidate gfn_to_pfn_cache's page split check

      https://github.com/kvm-x86/linux/commit/18f06e976925
[2/3] KVM: Check validity of offset+length of gfn_to_pfn_cache prior to activation
      https://github.com/kvm-x86/linux/commit/5c9ca4ed8908
[3/3] KVM: Explicitly disallow activatating a gfn_to_pfn_cache with INVALID_GPA
      https://github.com/kvm-x86/linux/commit/fc62a4e8dee2

--
https://github.com/kvm-x86/linux/tree/next