Skip to contentSkip to conversation

drm/gpusvm, pagemap: Do not assume DRM pagemap owns device pages

PATCH In review Matthew Brost 3 participants 5 messages Apr 9 – Apr 17

Subsystems (3)
lore
Share & tools
lore

Matthew Brost

author maintainer
#0 lore raw

PATCH 1/1 drm/gpusvm, pagemap: Do not assume DRM pagemap owns device pages

Update drm_pagemap_page_zone_device_data() to derive the pgmap ops from the page and compare them against the DRM pagemap ops. If the ops do not match, return NULL.

Also harden two risky call sites by checking for NULL after hmm_range_fault() or migrate_vma_setup() when migrating to device memory, as it is possible to encounter device pages that are not owned by DRM pagemap.

Suggested-by sashiko.dev
---
 drivers/gpu/drm/drm_gpusvm.c  |  5 +++++
 drivers/gpu/drm/drm_pagemap.c | 14 ++++++++++----
 include/drm/drm_pagemap.h     |  5 ++++-
 3 files changed, 19 insertions(+), 5 deletions(-)
drivers/gpu/drm/drm_gpusvm.c +5 -0
index 365a9c0b522a..b3cccd047a21 100644
@@ -1506,6 +1506,11 @@ int drm_gpusvm_get_pages(struct drm_gpusvm *gpusvm,
 			struct drm_pagemap_zdd *__zdd =
 				drm_pagemap_page_zone_device_data(page);
 
+			if (!__zdd) {
+				err = -EINVAL;
+				goto err_unmap;
+			}
+
 			if (!ctx->allow_mixed &&
 			    zdd != __zdd && i > 0) {
 				err = -EOPNOTSUPP;
drivers/gpu/drm/drm_pagemap.c +10 -4
index d82ea7ccb8da..95c951c5b569 100644
@@ -753,10 +753,16 @@ int drm_pagemap_migrate_to_devmem(struct drm_pagemap_devmem *devmem_allocation,
 				own_pages++;
 				goto next;
 			}
-			cur.dpagemap = src_zdd->dpagemap;
-			cur.ops = src_zdd->devmem_allocation->ops;
-			cur.device = cur.dpagemap->drm->dev;
-			pages[i] = src_page;
+			if (src_zdd) {
+				cur.dpagemap = src_zdd->dpagemap;
+				cur.ops = src_zdd->devmem_allocation->ops;
+				cur.device = cur.dpagemap->drm->dev;
+				pages[i] = src_page;
+			} else {
+				npages = i;
+				err = -EINVAL;
+				goto err_finalize;
+			}
 		}
 		if (!pages[i]) {
 			cur.dpagemap = NULL;
include/drm/drm_pagemap.h +4 -1
index 95eb4b66b057..9b7c50932db5 100644
@@ -367,12 +367,15 @@ int drm_pagemap_reinit(struct drm_pagemap *dpagemap);
  * drm_pagemap_page_zone_device_data() - Page to zone_device_data
  * @page: Pointer to the page
  *
- * Return: Page's zone_device_data
+ * Return: Page's zone_device_data if owned by DRM pagemap, NULL otherwise
  */
 static inline struct drm_pagemap_zdd *drm_pagemap_page_zone_device_data(struct page *page)
 {
 	struct folio *folio = page_folio(page);
 
+	if (WARN_ON_ONCE(page_pgmap(page)->ops != drm_pagemap_pagemap_ops_get()))
+		return NULL;
+
 	return folio_zone_device_data(folio);
 }
 
Signature · 1 line
2.34.1
#1 lore raw

Re: [PATCH] drm/gpusvm,pagemap: Do not assume DRM pagemap owns device pages

Quoted 8 lines · Matthew Brost

Update drm_pagemap_page_zone_device_data() to derive the pgmap ops from the page and compare them against the DRM pagemap ops. If the ops do not match, return NULL.

Also harden two risky call sites by checking for NULL after hmm_range_fault() or migrate_vma_setup() when migrating to device memory, as it is possible to encounter device pages that are not owned by DRM pagemap.

Shouldn't we also harden other calls to drm_pagemap_page_zone_device_data() in

    drm_pagemap_migrate_map_device_private_pages()
    drm_pagemap_migrate_unmap_pages()
    drm_pagemap_migrate_populate_ram_pfn()
    __drm_pagemap_migrate_to_ram()
    drm_pagemap_folio_free()
    drm_pagemap_migrate_to_ram()
    drm_pagemap_page_to_dpagemap()

Francois

Quoted 74 lines
Suggested-by sashiko.dev
---
 drivers/gpu/drm/drm_gpusvm.c  |  5 +++++
 drivers/gpu/drm/drm_pagemap.c | 14 ++++++++++----
 include/drm/drm_pagemap.h     |  5 ++++-
 3 files changed, 19 insertions(+), 5 deletions(-)
drivers/gpu/drm/drm_gpusvm.c +5 -0
index 365a9c0b522a..b3cccd047a21 100644
@@ -1506,6 +1506,11 @@ int drm_gpusvm_get_pages(struct drm_gpusvm *gpusvm,
 			struct drm_pagemap_zdd *__zdd =
 				drm_pagemap_page_zone_device_data(page);
 
+			if (!__zdd) {
+				err = -EINVAL;
+				goto err_unmap;
+			}
+
 			if (!ctx->allow_mixed &&
 			    zdd != __zdd && i > 0) {
 				err = -EOPNOTSUPP;
drivers/gpu/drm/drm_pagemap.c +10 -4
index d82ea7ccb8da..95c951c5b569 100644
@@ -753,10 +753,16 @@ int drm_pagemap_migrate_to_devmem(struct drm_pagemap_devmem *devmem_allocation,
 				own_pages++;
 				goto next;
 			}
-			cur.dpagemap = src_zdd->dpagemap;
-			cur.ops = src_zdd->devmem_allocation->ops;
-			cur.device = cur.dpagemap->drm->dev;
-			pages[i] = src_page;
+			if (src_zdd) {
+				cur.dpagemap = src_zdd->dpagemap;
+				cur.ops = src_zdd->devmem_allocation->ops;
+				cur.device = cur.dpagemap->drm->dev;
+				pages[i] = src_page;
+			} else {
+				npages = i;
+				err = -EINVAL;
+				goto err_finalize;
+			}
 		}
 		if (!pages[i]) {
 			cur.dpagemap = NULL;
include/drm/drm_pagemap.h +4 -1
index 95eb4b66b057..9b7c50932db5 100644
@@ -367,12 +367,15 @@ int drm_pagemap_reinit(struct drm_pagemap *dpagemap);
  * drm_pagemap_page_zone_device_data() - Page to zone_device_data
  * @page: Pointer to the page
  *
- * Return: Page's zone_device_data
+ * Return: Page's zone_device_data if owned by DRM pagemap, NULL otherwise
  */
 static inline struct drm_pagemap_zdd *drm_pagemap_page_zone_device_data(struct page *page)
 {
 	struct folio *folio = page_folio(page);
 
+	if (WARN_ON_ONCE(page_pgmap(page)->ops != drm_pagemap_pagemap_ops_get()))
+		return NULL;
+
 	return folio_zone_device_data(folio);
 }
 
Signature · 1 line
2.34.1
author maintainer
#2 lore raw

Re: [PATCH] drm/gpusvm,pagemap: Do not assume DRM pagemap owns device pages

Quoted 10 lines · Francois Dugast
Quoted 8 lines · Matthew Brost

Update drm_pagemap_page_zone_device_data() to derive the pgmap ops from the page and compare them against the DRM pagemap ops. If the ops do not match, return NULL.

Also harden two risky call sites by checking for NULL after hmm_range_fault() or migrate_vma_setup() when migrating to device memory, as it is possible to encounter device pages that are not owned by DRM pagemap.

Quoted 4 lines

Shouldn't we also harden other calls to drm_pagemap_page_zone_device_data() in

drm_pagemap_migrate_map_device_private_pages() drm_pagemap_migrate_unmap_pages()

We sanitize prior to this in drm_pagemap_migrate_to_devmem or are operating on pages handed back via populate_devmem_pfn.

Quoted 1 line

drm_pagemap_migrate_populate_ram_pfn()

Operating on page handed back via populate_devmem_pfn. Also wouldn't NULL ptr reference.

Quoted 3 lines
    __drm_pagemap_migrate_to_ram()
    drm_pagemap_folio_free()
    drm_pagemap_migrate_to_ram()

These are in the vops path of pagemap we check against.

Quoted 1 line

drm_pagemap_page_to_dpagemap()

We sanitize prior to this in drm_gpusvm_get_pages.

Thus all the above sites I figure a warn is enough as it would indicate a fairly serious bug in drm gpusvm/pagemap/calling driver which this code completely controls.

The case where I do sanitize - after collection via hmm_range_fault, migrate_vma_setup, I think it could be possible an outside driver has moved pages to private (very unlikely) and this driver also tries to move, so abort rather NULL ptr dereference.

So basically mitagated the 2 risky places with sanitization. Ofc we could check this everywhere...

Matt

Quoted 77 lines

Francois

Quoted 74 lines
Suggested-by sashiko.dev
---
 drivers/gpu/drm/drm_gpusvm.c  |  5 +++++
 drivers/gpu/drm/drm_pagemap.c | 14 ++++++++++----
 include/drm/drm_pagemap.h     |  5 ++++-
 3 files changed, 19 insertions(+), 5 deletions(-)
drivers/gpu/drm/drm_gpusvm.c +5 -0
index 365a9c0b522a..b3cccd047a21 100644
@@ -1506,6 +1506,11 @@ int drm_gpusvm_get_pages(struct drm_gpusvm *gpusvm,
 			struct drm_pagemap_zdd *__zdd =
 				drm_pagemap_page_zone_device_data(page);
 
+			if (!__zdd) {
+				err = -EINVAL;
+				goto err_unmap;
+			}
+
 			if (!ctx->allow_mixed &&
 			    zdd != __zdd && i > 0) {
 				err = -EOPNOTSUPP;
drivers/gpu/drm/drm_pagemap.c +10 -4
index d82ea7ccb8da..95c951c5b569 100644
@@ -753,10 +753,16 @@ int drm_pagemap_migrate_to_devmem(struct drm_pagemap_devmem *devmem_allocation,
 				own_pages++;
 				goto next;
 			}
-			cur.dpagemap = src_zdd->dpagemap;
-			cur.ops = src_zdd->devmem_allocation->ops;
-			cur.device = cur.dpagemap->drm->dev;
-			pages[i] = src_page;
+			if (src_zdd) {
+				cur.dpagemap = src_zdd->dpagemap;
+				cur.ops = src_zdd->devmem_allocation->ops;
+				cur.device = cur.dpagemap->drm->dev;
+				pages[i] = src_page;
+			} else {
+				npages = i;
+				err = -EINVAL;
+				goto err_finalize;
+			}
 		}
 		if (!pages[i]) {
 			cur.dpagemap = NULL;
include/drm/drm_pagemap.h +4 -1
index 95eb4b66b057..9b7c50932db5 100644
@@ -367,12 +367,15 @@ int drm_pagemap_reinit(struct drm_pagemap *dpagemap);
  * drm_pagemap_page_zone_device_data() - Page to zone_device_data
  * @page: Pointer to the page
  *
- * Return: Page's zone_device_data
+ * Return: Page's zone_device_data if owned by DRM pagemap, NULL otherwise
  */
 static inline struct drm_pagemap_zdd *drm_pagemap_page_zone_device_data(struct page *page)
 {
 	struct folio *folio = page_folio(page);
 
+	if (WARN_ON_ONCE(page_pgmap(page)->ops != drm_pagemap_pagemap_ops_get()))
+		return NULL;
+
 	return folio_zone_device_data(folio);
 }
 
Signature · 1 line
2.34.1
Reviewed-by
#3 lore raw

Re: [PATCH] drm/gpusvm,pagemap: Do not assume DRM pagemap owns device pages

Quoted 34 lines · Matthew Brost
Quoted 10 lines · Francois Dugast
Quoted 8 lines · Matthew Brost

Update drm_pagemap_page_zone_device_data() to derive the pgmap ops from the page and compare them against the DRM pagemap ops. If the ops do not match, return NULL.

Also harden two risky call sites by checking for NULL after hmm_range_fault() or migrate_vma_setup() when migrating to device memory, as it is possible to encounter device pages that are not owned by DRM pagemap.

Quoted 4 lines

Shouldn't we also harden other calls to drm_pagemap_page_zone_device_data() in

drm_pagemap_migrate_map_device_private_pages() drm_pagemap_migrate_unmap_pages()

We sanitize prior to this in drm_pagemap_migrate_to_devmem or are operating on pages handed back via populate_devmem_pfn.

Quoted 1 line

drm_pagemap_migrate_populate_ram_pfn()

Operating on page handed back via populate_devmem_pfn. Also wouldn't NULL ptr reference.

Quoted 3 lines
    __drm_pagemap_migrate_to_ram()
    drm_pagemap_folio_free()
    drm_pagemap_migrate_to_ram()

These are in the vops path of pagemap we check against.

Quoted 1 line

drm_pagemap_page_to_dpagemap()

We sanitize prior to this in drm_gpusvm_get_pages.

Quoted 8 lines

Thus all the above sites I figure a warn is enough as it would indicate a fairly serious bug in drm gpusvm/pagemap/calling driver which this code completely controls.

The case where I do sanitize - after collection via hmm_range_fault, migrate_vma_setup, I think it could be possible an outside driver has moved pages to private (very unlikely) and this driver also tries to move, so abort rather NULL ptr dereference.

Indeed, thanks for pointing it out and for the explanation.

Quoted 83 lines

So basically mitagated the 2 risky places with sanitization. Ofc we could check this everywhere...

Matt

Quoted 77 lines

Francois

Quoted 74 lines
Suggested-by sashiko.dev
---
 drivers/gpu/drm/drm_gpusvm.c  |  5 +++++
 drivers/gpu/drm/drm_pagemap.c | 14 ++++++++++----
 include/drm/drm_pagemap.h     |  5 ++++-
 3 files changed, 19 insertions(+), 5 deletions(-)
drivers/gpu/drm/drm_gpusvm.c +5 -0
index 365a9c0b522a..b3cccd047a21 100644
@@ -1506,6 +1506,11 @@ int drm_gpusvm_get_pages(struct drm_gpusvm *gpusvm,
 			struct drm_pagemap_zdd *__zdd =
 				drm_pagemap_page_zone_device_data(page);
 
+			if (!__zdd) {
+				err = -EINVAL;
+				goto err_unmap;
+			}
+
 			if (!ctx->allow_mixed &&
 			    zdd != __zdd && i > 0) {
 				err = -EOPNOTSUPP;
drivers/gpu/drm/drm_pagemap.c +10 -4
index d82ea7ccb8da..95c951c5b569 100644
@@ -753,10 +753,16 @@ int drm_pagemap_migrate_to_devmem(struct drm_pagemap_devmem *devmem_allocation,
 				own_pages++;
 				goto next;
 			}
-			cur.dpagemap = src_zdd->dpagemap;
-			cur.ops = src_zdd->devmem_allocation->ops;
-			cur.device = cur.dpagemap->drm->dev;
-			pages[i] = src_page;
+			if (src_zdd) {
+				cur.dpagemap = src_zdd->dpagemap;
+				cur.ops = src_zdd->devmem_allocation->ops;
+				cur.device = cur.dpagemap->drm->dev;
+				pages[i] = src_page;
+			} else {
+				npages = i;
+				err = -EINVAL;
+				goto err_finalize;
+			}
 		}
 		if (!pages[i]) {
 			cur.dpagemap = NULL;
include/drm/drm_pagemap.h +4 -1
index 95eb4b66b057..9b7c50932db5 100644
@@ -367,12 +367,15 @@ int drm_pagemap_reinit(struct drm_pagemap *dpagemap);
  * drm_pagemap_page_zone_device_data() - Page to zone_device_data
  * @page: Pointer to the page
  *
- * Return: Page's zone_device_data
+ * Return: Page's zone_device_data if owned by DRM pagemap, NULL otherwise
  */
 static inline struct drm_pagemap_zdd *drm_pagemap_page_zone_device_data(struct page *page)
 {
 	struct folio *folio = page_folio(page);
 
+	if (WARN_ON_ONCE(page_pgmap(page)->ops != drm_pagemap_pagemap_ops_get()))
+		return NULL;
+
 	return folio_zone_device_data(folio);
 }
 
Signature · 1 line
2.34.1
#4 lore raw
In reply to Matthew Brost
Quoted 6 lines · Matthew Brost

Update drm_pagemap_page_zone_device_data() to derive the pgmap ops from
the page and compare them against the DRM pagemap ops. If the ops do not
match, return NULL.

Quoted 7 lines

Also harden two risky call sites by checking for NULL after hmm_range_fault() or migrate_vma_setup() when migrating to device memory, as it is possible to encounter device pages that are not owned
by DRM pagemap.

Suggested-by sashiko.dev

Is there a proper (fake) email we can use?

Quoted 18 lines

---
 drivers/gpu/drm/drm_gpusvm.c  |  5 +++++ drivers/gpu/drm/drm_pagemap.c | 14 ++++++++++---- include/drm/drm_pagemap.h     |  5 ++++-

 3 files changed, 19 insertions(+), 5 deletions(-)
drivers/gpu/drm/drm_gpusvm.c +0 -0
index 365a9c0b522a..b3cccd047a21 100644
@@ -1506,6 +1506,11 @@ int drm_gpusvm_get_pages(struct drm_gpusvm
 *gpusvm,

  struct drm_pagemap_zdd *__zdd = drm_pagemap_page_zone_device_data(pa ge);

drivers/gpu/drm/drm_gpusvm.c int drm_gpusvm_get_pages(struct drm_gpusvm *gpusvm, · line 1513

Quoted 5 lines
drivers/gpu/drm/drm_gpusvm.c +5 -0
+			if (!__zdd) {
+				err = -EINVAL;
+				goto err_unmap;
+			}
+

Here, __zdd is never dereferenced. As I understand it, the current behaviour (before this patch) is that pages from mixed pagemaps are generating an -EOPNOTSUPP, re-trying migration, which I think is the correct behaviour, save possibly for foreign device-coherent pages.

Also see below reasoning WRT "owner".

Quoted 14 lines

  if (!ctx->allow_mixed && zdd != __zdd && i > 0) { err = -EOPNOTSUPP;

drivers/gpu/drm/drm_pagemap.c +0 -1
index d82ea7ccb8da..95c951c5b569 100644
@@ -753,10 +753,16 @@ int drm_pagemap_migrate_to_devmem(struct
 drm_pagemap_devmem *devmem_allocation,
  				own_pages++;
  				goto next;
  			}
-			cur.dpagemap = src_zdd->dpagemap;

drivers/gpu/drm/drm_pagemap.c int drm_pagemap_migrate_to_devmem(struct · line 763

Quoted 10 lines
drivers/gpu/drm/drm_pagemap.c +6 -3
-			cur.ops = src_zdd->devmem_allocation->ops;
-			cur.device = cur.dpagemap->drm->dev;
-			pages[i] = src_page;
+			if (src_zdd) {
+				cur.dpagemap = src_zdd->dpagemap;
+				cur.ops = src_zdd-
 >devmem_allocation->ops;
+				cur.device = cur.dpagemap->drm->dev;
+				pages[i] = src_page;
+			} else {

This shouldn't happen, because we select device_private pages based on dev_private owner. IMO we shouldn't allow non drm_pagemap device_private pages sharing owner because we wouldn't now how to migrate them.

Quoted 26 lines
drivers/gpu/drm/drm_pagemap.c +4 -0
+				npages = i;
+				err = -EINVAL;
+				goto err_finalize;
+			}
  		}

  if (!pages[i]) {
  cur.dpagemap = NULL;

include/drm/drm_pagemap.h +1 -1
index 95eb4b66b057..9b7c50932db5 100644
@@ -367,12 +367,15 @@ int drm_pagemap_reinit(struct drm_pagemap
 *dpagemap);
   * drm_pagemap_page_zone_device_data() - Page to zone_device_data
   * @page: Pointer to the page
   *
- * Return: Page's zone_device_data
+ * Return: Page's zone_device_data if owned by DRM pagemap, NULL

otherwise
  */
 static inline struct drm_pagemap_zdd *drm_pagemap_page_zone_device_data(struct page *page) {
  struct folio *folio = page_folio(page);

include/drm/drm_pagemap.h +1 -0
+	if (WARN_ON_ONCE(page_pgmap(page)->ops !=
Quoted 1 line

drm_pagemap_pagemap_ops_get()))

Driver could be handing a separate ops. Like if we wanted to check for wedging / unplug before trying to migrate back to system, so this wouldn't work.

So in general I don't think this patch is correct. Perhaps we should look at updating the docs around owner assignment, though and also perhaps make drm_gpusvm_get_pages() a bit more permissive?

/Thomas

Quoted 5 lines
include/drm/drm_pagemap.h +2 -0
+		return NULL;
+

  return folio_zone_device_data(folio);
 }