mm: Fix invalid page pointer returned with FOLL_PIN gups
Subsystems (2)
Peter Xu
reviewerPATCH v2 1/1 mm: Fix invalid page pointer returned with FOLL_PIN gups
Alex reported invalid page pointer returned with pin_user_pages_remote() from vfio after upstream commit 4b6c33b32296 ("vfio/type1: Prepare for batched pinning with struct vfio_batch"). This problem breaks NVIDIA vfio mdev.
It turns out that it's not the fault of the vfio commit; however after vfio switches to a full page buffer to store the page pointers it starts to expose the problem easier.
The problem is for VM_PFNMAP vmas we should normally fail with an -EFAULT then vfio will carry on to handle the MMIO regions. However when the bug triggered, follow_page_mask() returned -EEXIST for such a page, which will jump over the current page, leaving that entry in **pages untouched. However the caller is not aware of it, hence the caller will reference the page as usual even if the pointer data can be anything.
We had that -EEXIST logic since commit 1027e4436b6a ("mm: make GUP handle pfn mapping unless FOLL_GET is requested") which seems very reasonable. It could be that when we reworked GUP with FOLL_PIN we could have overlooked that special path in commit 3faa52c03f44 ("mm/gup: track FOLL_PIN pages"), even if that commit rightfully touched up follow_devmap_pud() on checking FOLL_PIN when it needs to return an -EEXIST.
Attaching the Fixes to the FOLL_PIN rework commit, as it happened later than 1027e4436b6a.
---
v2:
- Drop the WARN_ON_ONCE() [Jason, John]
- Add Alex's Tested-by too, as after dropping the WARN_ON_ONCE() then the patch is exactly the one that Alex helped on bug verification, hence very safe to grant the credit alongside.
--- mm/gup.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/gup.c +1 -1
@@ -440,7 +440,7 @@ static int follow_pfn_pte(struct vm_area_struct *vma, unsigned long address, pte_t *pte, unsigned int flags) {/* No page to get reference */ -if (flags &FOLL_GET ) +if (flags & (FOLL_GET |FOLL_PIN ))return -EFAULT ;if (flags &FOLL_TOUCH ) {
Signature · 1 line
2.32.0
Quoted 22 lines · Peter Xu
Alex reported invalid page pointer returned with
pin_user_pages_remote()from vfio after upstream commit4b6c33b32296("vfio/type1: Prepare for batched pinning with structvfio_batch"). This problem breaks NVIDIA vfio mdev.It turns out that it's not the fault of the vfio commit; however after vfio switches to a full page buffer to store the page pointers it starts to expose the problem easier.
The problem is for
VM_PFNMAPvmas we should normally fail with an-EFAULTthen vfio will carry on to handle the MMIO regions. However when the bug triggered,follow_page_mask()returned-EEXISTfor such a page, which will jump over the current page, leaving that entry in **pages untouched. However the caller is not aware of it, hence the caller will reference the page as usual even if the pointer data can be anything.We had that
-EEXISTlogic since commit1027e4436b6a("mm: make GUP handle pfn mapping unlessFOLL_GETis requested") which seems very reasonable. It could be that when we reworked GUP withFOLL_PINwe could have overlooked that special path in commit3faa52c03f44("mm/gup: trackFOLL_PINpages"), even if that commit rightfully touched upfollow_devmap_pud()on checkingFOLL_PINwhen it needs to return an-EEXIST.
Quoted 3 lines
Attaching the Fixes to the
FOLL_PINrework commit, as it happened later than1027e4436b6a.
thanks,
Signature · 2 lines
John Hubbard NVIDIA
Quoted 34 lines
Cc Andrea Arcangeli <[email protected]>Cc Andrew Morton <[email protected]>Cc Jan Kara <[email protected]>Cc Jérôme Glisse <[email protected]>Cc John Hubbard <[email protected]>Cc Kirill A. Shutemov <[email protected]>Fixes 3faa52c03f44 ("mm/gup: track FOLL_PIN pages")Reported-by Alex Williamson <[email protected]>Debugged-by Alex Williamson <[email protected]>Tested-by Alex Williamson <[email protected]>Signed-off-by Peter Xu <[email protected]>---
v2:
- Drop theWARN_ON_ONCE()[Jason, John]
- Add Alex's Tested-by too, as after dropping theWARN_ON_ONCE()then the patch is exactly the one that Alex helped on bug verification, hence very safe to grant the credit alongside.--- mm/gup.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/gup.c +1
-1@@ -440,7 +440,7 @@ static int follow_pfn_pte(struct vm_area_struct *vma, unsigned long address, pte_t *pte, unsigned int flags) {/* No page to get reference */ -if (flags &FOLL_GET ) +if (flags & (FOLL_GET |FOLL_PIN ))return -EFAULT ;if (flags &FOLL_TOUCH ) {