drm/gpusvm, pagemap: Do not assume DRM pagemap owns device pages
Subsystems (3)
Matthew Brost
maintainerPATCH 1/1 drm/gpusvm, pagemap: Do not assume DRM pagemap owns device pages
Update drm_pagemap_page_zone_device_data() to derive the pgmap ops from the page and compare them against the DRM pagemap ops. If the ops do not match, return NULL.
Also harden two risky call sites by checking for NULL after hmm_range_fault() or migrate_vma_setup() when migrating to device memory, as it is possible to encounter device pages that are not owned by DRM pagemap.
--- drivers/gpu/drm/drm_gpusvm.c | 5 +++++ drivers/gpu/drm/drm_pagemap.c | 14 ++++++++++---- include/drm/drm_pagemap.h | 5 ++++- 3 files changed, 19 insertions(+), 5 deletions(-)
drivers/gpu/drm/drm_gpusvm.c +5 -0
@@ -1506,6 +1506,11 @@ int drm_gpusvm_get_pages(struct drm_gpusvm *gpusvm,struct drm_pagemap_zdd *__zdd =drm_pagemap_page_zone_device_data (page); +if (!__zdd) { + err = -EINVAL ; + goto err_unmap; + } +if (!ctx->allow_mixed && zdd != __zdd && i >0 ) { err = -EOPNOTSUPP ;
drivers/gpu/drm/drm_pagemap.c +10 -4
@@ -753,10 +753,16 @@ int drm_pagemap_migrate_to_devmem(struct drm_pagemap_devmem *devmem_allocation, own_pages++; goto next; } - cur.dpagemap = src_zdd->dpagemap; - cur.ops = src_zdd->devmem_allocation->ops; - cur.device = cur.dpagemap->drm->dev; - pages[i] = src_page; +if (src_zdd) { + cur.dpagemap = src_zdd->dpagemap; + cur.ops = src_zdd->devmem_allocation->ops; + cur.device = cur.dpagemap->drm->dev; + pages[i] = src_page; + }else { + npages = i; + err = -EINVAL ; + goto err_finalize; + } }if (!pages[i]) { cur.dpagemap =NULL ;
include/drm/drm_pagemap.h +4 -1
@@ -367,12 +367,15 @@ int drm_pagemap_reinit(struct drm_pagemap *dpagemap);* drm_pagemap_page_zone_device_data() - Page to zone_device_data * @page: Pointer to the page * -* Return: Page's zone_device_data +* Return: Page's zone_device_data if owned by DRM pagemap, NULL otherwise */ static inline struct drm_pagemap_zdd *drm_pagemap_page_zone_device_data (struct page *page) {struct folio *folio =page_folio (page); +if (WARN_ON_ONCE (page_pgmap (page)->ops !=drm_pagemap_pagemap_ops_get ())) +return NULL ; +return folio_zone_device_data (folio); }
Signature · 1 line
2.34.1
Re: [PATCH] drm/gpusvm,pagemap: Do not assume DRM pagemap owns device pages
Quoted 8 lines · Matthew Brost
Update
drm_pagemap_page_zone_device_data()to derive the pgmap ops from the page and compare them against the DRM pagemap ops. If the ops do not match, return NULL.Also harden two risky call sites by checking for NULL after
hmm_range_fault()ormigrate_vma_setup()when migrating to device memory, as it is possible to encounter device pages that are not owned by DRM pagemap.
Shouldn't we also harden other calls to drm_pagemap_page_zone_device_data() in
drm_pagemap_migrate_map_device_private_pages()
drm_pagemap_migrate_unmap_pages()
drm_pagemap_migrate_populate_ram_pfn()
__drm_pagemap_migrate_to_ram()
drm_pagemap_folio_free()
drm_pagemap_migrate_to_ram()
drm_pagemap_page_to_dpagemap()Francois
Quoted 74 lines
Suggested-by sashiko.devSigned-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/drm_gpusvm.c | 5 +++++ drivers/gpu/drm/drm_pagemap.c | 14 ++++++++++---- include/drm/drm_pagemap.h | 5 ++++- 3 files changed, 19 insertions(+), 5 deletions(-)drivers/gpu/drm/drm_gpusvm.c +5
-0@@ -1506,6 +1506,11 @@ int drm_gpusvm_get_pages(struct drm_gpusvm *gpusvm,struct drm_pagemap_zdd *__zdd =drm_pagemap_page_zone_device_data (page); +if (!__zdd) { + err = -EINVAL ; + goto err_unmap; + } +if (!ctx->allow_mixed && zdd != __zdd && i >0 ) { err = -EOPNOTSUPP ;drivers/gpu/drm/drm_pagemap.c +10
-4@@ -753,10 +753,16 @@ int drm_pagemap_migrate_to_devmem(struct drm_pagemap_devmem *devmem_allocation, own_pages++; goto next; } - cur.dpagemap = src_zdd->dpagemap; - cur.ops = src_zdd->devmem_allocation->ops; - cur.device = cur.dpagemap->drm->dev; - pages[i] = src_page; +if (src_zdd) { + cur.dpagemap = src_zdd->dpagemap; + cur.ops = src_zdd->devmem_allocation->ops; + cur.device = cur.dpagemap->drm->dev; + pages[i] = src_page; + }else { + npages = i; + err = -EINVAL ; + goto err_finalize; + } }if (!pages[i]) { cur.dpagemap =NULL ;include/drm/drm_pagemap.h +4
-1@@ -367,12 +367,15 @@ int drm_pagemap_reinit(struct drm_pagemap *dpagemap);* drm_pagemap_page_zone_device_data() - Page to zone_device_data * @page: Pointer to the page * -* Return: Page's zone_device_data +* Return: Page's zone_device_data if owned by DRM pagemap, NULL otherwise */ static inline struct drm_pagemap_zdd *drm_pagemap_page_zone_device_data (struct page *page) {struct folio *folio =page_folio (page); +if (WARN_ON_ONCE (page_pgmap (page)->ops !=drm_pagemap_pagemap_ops_get ())) +return NULL ; +return folio_zone_device_data (folio); }Signature · 1 line
2.34.1
Matthew Brost
maintainerRe: [PATCH] drm/gpusvm,pagemap: Do not assume DRM pagemap owns device pages
Quoted 10 lines · Francois Dugast
Quoted 8 lines · Matthew Brost
Update
drm_pagemap_page_zone_device_data()to derive the pgmap ops from the page and compare them against the DRM pagemap ops. If the ops do not match, return NULL.Also harden two risky call sites by checking for NULL after
hmm_range_fault()ormigrate_vma_setup()when migrating to device memory, as it is possible to encounter device pages that are not owned by DRM pagemap.
Quoted 4 lines
Shouldn't we also harden other calls to
drm_pagemap_page_zone_device_data()in
drm_pagemap_migrate_map_device_private_pages()drm_pagemap_migrate_unmap_pages()
We sanitize prior to this in drm_pagemap_migrate_to_devmem or are operating on pages handed back via populate_devmem_pfn.
Quoted 1 line
drm_pagemap_migrate_populate_ram_pfn()
Operating on page handed back via populate_devmem_pfn. Also wouldn't NULL ptr reference.
Quoted 3 lines
__drm_pagemap_migrate_to_ram()
drm_pagemap_folio_free()
drm_pagemap_migrate_to_ram()These are in the vops path of pagemap we check against.
Quoted 1 line
drm_pagemap_page_to_dpagemap()
We sanitize prior to this in drm_gpusvm_get_pages.
Thus all the above sites I figure a warn is enough as it would indicate a fairly serious bug in drm gpusvm/pagemap/calling driver which this code completely controls.
The case where I do sanitize - after collection via hmm_range_fault, migrate_vma_setup, I think it could be possible an outside driver has moved pages to private (very unlikely) and this driver also tries to move, so abort rather NULL ptr dereference.
So basically mitagated the 2 risky places with sanitization. Ofc we could check this everywhere...
Matt
Quoted 77 lines
Francois
Quoted 74 lines
Suggested-by sashiko.devSigned-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/drm_gpusvm.c | 5 +++++ drivers/gpu/drm/drm_pagemap.c | 14 ++++++++++---- include/drm/drm_pagemap.h | 5 ++++- 3 files changed, 19 insertions(+), 5 deletions(-)drivers/gpu/drm/drm_gpusvm.c +5
-0@@ -1506,6 +1506,11 @@ int drm_gpusvm_get_pages(struct drm_gpusvm *gpusvm,struct drm_pagemap_zdd *__zdd =drm_pagemap_page_zone_device_data (page); +if (!__zdd) { + err = -EINVAL ; + goto err_unmap; + } +if (!ctx->allow_mixed && zdd != __zdd && i >0 ) { err = -EOPNOTSUPP ;drivers/gpu/drm/drm_pagemap.c +10
-4@@ -753,10 +753,16 @@ int drm_pagemap_migrate_to_devmem(struct drm_pagemap_devmem *devmem_allocation, own_pages++; goto next; } - cur.dpagemap = src_zdd->dpagemap; - cur.ops = src_zdd->devmem_allocation->ops; - cur.device = cur.dpagemap->drm->dev; - pages[i] = src_page; +if (src_zdd) { + cur.dpagemap = src_zdd->dpagemap; + cur.ops = src_zdd->devmem_allocation->ops; + cur.device = cur.dpagemap->drm->dev; + pages[i] = src_page; + }else { + npages = i; + err = -EINVAL ; + goto err_finalize; + } }if (!pages[i]) { cur.dpagemap =NULL ;include/drm/drm_pagemap.h +4
-1@@ -367,12 +367,15 @@ int drm_pagemap_reinit(struct drm_pagemap *dpagemap);* drm_pagemap_page_zone_device_data() - Page to zone_device_data * @page: Pointer to the page * -* Return: Page's zone_device_data +* Return: Page's zone_device_data if owned by DRM pagemap, NULL otherwise */ static inline struct drm_pagemap_zdd *drm_pagemap_page_zone_device_data (struct page *page) {struct folio *folio =page_folio (page); +if (WARN_ON_ONCE (page_pgmap (page)->ops !=drm_pagemap_pagemap_ops_get ())) +return NULL ; +return folio_zone_device_data (folio); }Signature · 1 line
2.34.1
Francois Dugast
Reviewed-byRe: [PATCH] drm/gpusvm,pagemap: Do not assume DRM pagemap owns device pages
Quoted 34 lines · Matthew Brost
Quoted 10 lines · Francois Dugast
Quoted 8 lines · Matthew Brost
Update
drm_pagemap_page_zone_device_data()to derive the pgmap ops from the page and compare them against the DRM pagemap ops. If the ops do not match, return NULL.Also harden two risky call sites by checking for NULL after
hmm_range_fault()ormigrate_vma_setup()when migrating to device memory, as it is possible to encounter device pages that are not owned by DRM pagemap.Quoted 4 lines
Shouldn't we also harden other calls to
drm_pagemap_page_zone_device_data()in
drm_pagemap_migrate_map_device_private_pages()drm_pagemap_migrate_unmap_pages()We sanitize prior to this in
drm_pagemap_migrate_to_devmemor are operating on pages handed back viapopulate_devmem_pfn.Quoted 1 line
drm_pagemap_migrate_populate_ram_pfn()Operating on page handed back via
populate_devmem_pfn. Also wouldn't NULL ptr reference.Quoted 3 lines
__drm_pagemap_migrate_to_ram() drm_pagemap_folio_free() drm_pagemap_migrate_to_ram()These are in the vops path of pagemap we check against.
Quoted 1 line
drm_pagemap_page_to_dpagemap()We sanitize prior to this in
drm_gpusvm_get_pages.
Quoted 8 lines
Thus all the above sites I figure a warn is enough as it would indicate a fairly serious bug in drm gpusvm/pagemap/calling driver which this code completely controls.
The case where I do sanitize - after collection via
hmm_range_fault,migrate_vma_setup, I think it could be possible an outside driver has moved pages to private (very unlikely) and this driver also tries to move, so abort rather NULL ptr dereference.
Indeed, thanks for pointing it out and for the explanation.
Quoted 83 lines
So basically mitagated the 2 risky places with sanitization. Ofc we could check this everywhere...
Matt
Quoted 77 lines
Francois
Quoted 74 lines
Suggested-by sashiko.devSigned-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/drm_gpusvm.c | 5 +++++ drivers/gpu/drm/drm_pagemap.c | 14 ++++++++++---- include/drm/drm_pagemap.h | 5 ++++- 3 files changed, 19 insertions(+), 5 deletions(-)drivers/gpu/drm/drm_gpusvm.c +5
-0@@ -1506,6 +1506,11 @@ int drm_gpusvm_get_pages(struct drm_gpusvm *gpusvm,struct drm_pagemap_zdd *__zdd =drm_pagemap_page_zone_device_data (page); +if (!__zdd) { + err = -EINVAL ; + goto err_unmap; + } +if (!ctx->allow_mixed && zdd != __zdd && i >0 ) { err = -EOPNOTSUPP ;drivers/gpu/drm/drm_pagemap.c +10
-4@@ -753,10 +753,16 @@ int drm_pagemap_migrate_to_devmem(struct drm_pagemap_devmem *devmem_allocation, own_pages++; goto next; } - cur.dpagemap = src_zdd->dpagemap; - cur.ops = src_zdd->devmem_allocation->ops; - cur.device = cur.dpagemap->drm->dev; - pages[i] = src_page; +if (src_zdd) { + cur.dpagemap = src_zdd->dpagemap; + cur.ops = src_zdd->devmem_allocation->ops; + cur.device = cur.dpagemap->drm->dev; + pages[i] = src_page; + }else { + npages = i; + err = -EINVAL ; + goto err_finalize; + } }if (!pages[i]) { cur.dpagemap =NULL ;include/drm/drm_pagemap.h +4
-1@@ -367,12 +367,15 @@ int drm_pagemap_reinit(struct drm_pagemap *dpagemap);* drm_pagemap_page_zone_device_data() - Page to zone_device_data * @page: Pointer to the page * -* Return: Page's zone_device_data +* Return: Page's zone_device_data if owned by DRM pagemap, NULL otherwise */ static inline struct drm_pagemap_zdd *drm_pagemap_page_zone_device_data (struct page *page) {struct folio *folio =page_folio (page); +if (WARN_ON_ONCE (page_pgmap (page)->ops !=drm_pagemap_pagemap_ops_get ())) +return NULL ; +return folio_zone_device_data (folio); }Signature · 1 line
2.34.1
Thomas Hellström
reviewerQuoted 6 lines · Matthew Brost
Update
drm_pagemap_page_zone_device_data()to derive the pgmap ops from
the page and compare them against the DRM pagemap ops. If the ops do not
match, return NULL.
Quoted 7 lines
Also harden two risky call sites by checking for NULL after
hmm_range_fault()ormigrate_vma_setup()when migrating to device memory, as it is possible to encounter device pages that are not owned
by DRM pagemap.Suggested-by sashiko.dev
Is there a proper (fake) email we can use?
Quoted 18 lines
Signed-off-by Matthew Brost <[email protected]>---
drivers/gpu/drm/drm_gpusvm.c | 5 +++++drivers/gpu/drm/drm_pagemap.c| 14 ++++++++++---- include/drm/drm_pagemap.h | 5 ++++-3 files changed, 19 insertions(+), 5 deletions(-)drivers/gpu/drm/drm_gpusvm.c +0
-0@@ -1506,6 +1506,11 @@ int drm_gpusvm_get_pages(struct drm_gpusvm *gpusvm,struct
drm_pagemap_zdd*__zdd=drm_pagemap_page_zone_device_data(pa ge);
drivers/gpu/drm/drm_gpusvm.c int drm_gpusvm_get_pages(struct drm_gpusvm *gpusvm, · line 1513
Quoted 5 lines
drivers/gpu/drm/drm_gpusvm.c +5
-0+if (!__zdd) { + err = -EINVAL ; + goto err_unmap; + } +
Here, __zdd is never dereferenced. As I understand it, the current behaviour (before this patch) is that pages from mixed pagemaps are generating an -EOPNOTSUPP, re-trying migration, which I think is the correct behaviour, save possibly for foreign device-coherent pages.
Also see below reasoning WRT "owner".
Quoted 14 lines
if (!ctx->allow_mixed && zdd !=
__zdd&& i > 0) { err =-EOPNOTSUPP;drivers/gpu/drm/drm_pagemap.c +0
-1@@ -753,10 +753,16 @@ int drm_pagemap_migrate_to_devmem(structdrm_pagemap_devmem *devmem_allocation, own_pages++; goto next; } - cur.dpagemap = src_zdd->dpagemap;
drivers/gpu/drm/drm_pagemap.c int drm_pagemap_migrate_to_devmem(struct · line 763
Quoted 10 lines
drivers/gpu/drm/drm_pagemap.c +6
-3- cur.ops = src_zdd->devmem_allocation->ops; - cur.device = cur.dpagemap->drm->dev; - pages[i] = src_page; +if (src_zdd) { + cur.dpagemap = src_zdd->dpagemap; + cur.ops = src_zdd- >devmem_allocation->ops; + cur.device = cur.dpagemap->drm->dev; + pages[i] = src_page; + }else {
This shouldn't happen, because we select device_private pages based on dev_private owner. IMO we shouldn't allow non drm_pagemap device_private pages sharing owner because we wouldn't now how to migrate them.
Quoted 26 lines
drivers/gpu/drm/drm_pagemap.c +4
-0+ npages = i; + err = -EINVAL ; + goto err_finalize; + } }if (!pages[i]) {
cur.dpagemap = NULL;include/drm/drm_pagemap.h +1
-1@@ -367,12 +367,15 @@ int drm_pagemap_reinit(struct drm_pagemap *dpagemap); * drm_pagemap_page_zone_device_data() - Page to zone_device_data * @page: Pointer to the page * - * Return: Page's zone_device_data + * Return: Page's zone_device_data if owned by DRM pagemap, NULLotherwise
*/
static inline structdrm_pagemap_zdd*drm_pagemap_page_zone_device_data(struct page *page) {
struct folio *folio =page_folio(page);include/drm/drm_pagemap.h +1
-0+ if (WARN_ON_ONCE(page_pgmap(page)->ops !=
Quoted 1 line
drm_pagemap_pagemap_ops_get()))
Driver could be handing a separate ops. Like if we wanted to check for wedging / unplug before trying to migrate back to system, so this wouldn't work.
So in general I don't think this patch is correct. Perhaps we should look at updating the docs around owner assignment, though and also perhaps make drm_gpusvm_get_pages() a bit more permissive?
/Thomas
Quoted 5 lines
include/drm/drm_pagemap.h +2
-0+return NULL ; +return
folio_zone_device_data(folio);
}