userfaultfd: release page in error path to avoid BUG_ON
Subsystems (2)
PATCH 1/1 userfaultfd: release page in error path to avoid BUG_ON
Consider the following sequence of events (described from the point of view of the commit that introduced the bug - see "Fixes:" below):
1. Userspace issues a UFFD ioctl, which ends up calling into shmem_mcopy_atomic_pte(). We successfully account the blocks, we shmem_alloc_page(), but then the copy_from_user() fails. We return -EFAULT. We don't release the page we allocated.
2. Our caller detects this error code, tries the copy_from_user() after dropping the mmap_sem, and retries, calling back into shmem_mcopy_atomic_pte().
3. Meanwhile, let's say another process filled up the tmpfs being used.
4. So shmem_mcopy_atomic_pte() fails to account blocks this time, and immediately returns - without releasing the page. This triggers a BUG_ON in our caller, which asserts that the page should always be consumed, unless -EFAULT is returned.
(Later on in the commit history, -EFAULT became -ENOENT, mmap_sem became mmap_lock, and shmem_inode_acct_block() was added.)
A malicious user (even an unprivileged one) could trigger this intentionally without too much trouble.
To fix this, detect if we have a "dangling" page when accounting fails, and if so, release it before returning.
--- mm/shmem.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-)
mm/shmem.c +12 -1
@@ -2375,8 +2375,19 @@ static int shmem_mfill_atomic_pte(struct mm_struct *dst_mm,pgoff_t offset, max_off; ret = -ENOMEM ; -if (!shmem_inode_acct_block (inode,1 )) +if (!shmem_inode_acct_block (inode,1 )) { +/* +* We may have got a page, returned -ENOENT triggering a retry, +* and now we find ourselves with -ENOMEM. Release the page, to +* avoid a BUG_ON in our caller. +*/ +if (unlikely (*pagep)) { +unlock_page (*pagep); +put_page (*pagep); + *pagep =NULL ; + } goto out; + }if (!*pagep) { page =shmem_alloc_page (gfp, info, pgoff);
Signature · 1 line
2.31.1.498.g6c1eba8ee3d-goog
Quoted 16 lines · Axel Rasmussen
Consider the following sequence of events (described from the point of view of the commit that introduced the bug - see "Fixes:" below):
1. Userspace issues a UFFD ioctl, which ends up calling into
shmem_mcopy_atomic_pte(). We successfully account the blocks, weshmem_alloc_page(), but then thecopy_from_user()fails. We return-EFAULT. We don't release the page we allocated.
2. Our caller detects this error code, tries thecopy_from_user()after dropping themmap_sem, and retries, calling back intoshmem_mcopy_atomic_pte().
3. Meanwhile, let's say another process filled up the tmpfs being used.
4. Soshmem_mcopy_atomic_pte()fails to account blocks this time, and immediately returns - without releasing the page. This triggers aBUG_ONin our caller, which asserts that the page should always be consumed, unless-EFAULTis returned.
Quoted 2 lines
(Later on in the commit history,
-EFAULTbecame-ENOENT,mmap_sembecamemmap_lock, andshmem_inode_acct_block()was added.)
I suggest you do s/EFAULT/ENOENT/ directly in above.
Quoted 21 lines
A malicious user (even an unprivileged one) could trigger this intentionally without too much trouble.
To fix this, detect if we have a "dangling" page when accounting fails, and if so, release it before returning.
Fixes cb658a453b93 ("userfaultfd: shmem: avoid leaking blocks and used blocks in UFFDIO_COPY")Reported-by Hugh Dickins <[email protected]>Signed-off-by Axel Rasmussen <[email protected]>--- mm/shmem.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-)mm/shmem.c +0
-0@@ -2375,8 +2375,19 @@ static int shmem_mfill_atomic_pte(struct mm_struct *dst_mm,pgoff_t offset, max_off;
mm/shmem.c static int shmem_mfill_atomic_pte(struct mm_struct *dst_mm, · line 2385
Quoted 10 lines
mm/shmem.c +8
-1ret = -ENOMEM ; -if (!shmem_inode_acct_block (inode,1 )) +if (!shmem_inode_acct_block (inode,1 )) { +/* +* We may have got a page, returned -ENOENT triggering a retry, +* and now we find ourselves with -ENOMEM. Release the page, to +* avoid a BUG_ON in our caller. +*/ +if (unlikely (*pagep)) { +unlock_page (*pagep);
Not necessary?
mm/shmem.c static int shmem_mfill_atomic_pte(struct mm_struct *dst_mm, · line 2389
Quoted 4 lines
mm/shmem.c +3
-0+put_page (*pagep); + *pagep =NULL ; + } goto out;
All "goto out" in this functions looks weird as it returns directly... so if you're touching this after all, I suggest we do "return -ENOMEM" directly and drop the "ret = -ENOMEM".
Thanks,
Quoted 7 lines
mm/shmem.c +1
-0+ }if (!*pagep) { page =shmem_alloc_page (gfp, info, pgoff);Signature · 1 line
2.31.1.498.g6c1eba8ee3d-goog
Signature · 1 line
Peter Xu
Hugh Dickins
maintainerQuoted 17 lines · Peter Xu
Quoted 16 lines · Axel Rasmussen
Consider the following sequence of events (described from the point of view of the commit that introduced the bug - see "Fixes:" below):
1. Userspace issues a UFFD ioctl, which ends up calling into
shmem_mcopy_atomic_pte(). We successfully account the blocks, weshmem_alloc_page(), but then thecopy_from_user()fails. We return-EFAULT. We don't release the page we allocated.
2. Our caller detects this error code, tries thecopy_from_user()after dropping themmap_sem, and retries, calling back intoshmem_mcopy_atomic_pte().
3. Meanwhile, let's say another process filled up the tmpfs being used.
4. Soshmem_mcopy_atomic_pte()fails to account blocks this time, and immediately returns - without releasing the page. This triggers aBUG_ONin our caller, which asserts that the page should always be consumed, unless-EFAULTis returned.
Quoted 4 lines
Quoted 2 lines
(Later on in the commit history,
-EFAULTbecame-ENOENT,mmap_sembecamemmap_lock, andshmem_inode_acct_block()was added.)I suggest you do s/EFAULT/ENOENT/ directly in above.
I haven't looked into the history, but it would be best for this to describe the situation in v5.12, never mind the details which were different at the time of the commit tagged with Fixes. But we stay alert that when it's backported to stable, we may need to adjust something to suit those releases (which will depend on how much else has been backported to them meanwhile).
Quoted 4 lines
Quoted 4 lines
A malicious user (even an unprivileged one) could trigger this intentionally without too much trouble.
I regret having suggested that. Maybe. Opinions differ as to whether it's helpful to call attention like that. I'd say delete that paragraph.
Quoted 7 lines
Quoted 7 lines
To fix this, detect if we have a "dangling" page when accounting fails, and if so, release it before returning.
Fixes cb658a453b93 ("userfaultfd: shmem: avoid leaking blocks and used blocks in UFFDIO_COPY")Reported-by Hugh Dickins <[email protected]>Signed-off-by Axel Rasmussen <[email protected]>
Thanks for getting on to this so quickly, Axel. But Peter is right, that unlock_page() needs removing.
Quoted 11 lines
Quoted 11 lines
--- mm/shmem.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-)mm/shmem.c +0
-0@@ -2375,8 +2375,19 @@ static int shmem_mfill_atomic_pte(struct mm_struct *dst_mm,pgoff_t offset, max_off;
Quoted 12 lines
mm/shmem.c static int shmem_mfill_atomic_pte(struct mm_struct *dst_mm, · line 2385
Quoted 10 lines
mm/shmem.c +8
-1ret = -ENOMEM ; -if (!shmem_inode_acct_block (inode,1 )) +if (!shmem_inode_acct_block (inode,1 )) { +/* +* We may have got a page, returned -ENOENT triggering a retry, +* and now we find ourselves with -ENOMEM. Release the page, to +* avoid a BUG_ON in our caller. +*/ +if (unlikely (*pagep)) { +unlock_page (*pagep);Not necessary?
Worse than not necessary: would trigger a VM_BUG_ON_PAGE(). Delete!
Quoted 9 lines
mm/shmem.c static int shmem_mfill_atomic_pte(struct mm_struct *dst_mm, · line 2389
Quoted 5 lines
mm/shmem.c +3
-0+put_page (*pagep); + *pagep =NULL ; + } goto out;All "goto out" in this functions looks weird as it returns directly... so if you're touching this after all, I suggest we do "return
-ENOMEM" directly and drop the "ret =-ENOMEM".
No strong feeling either way from me on that: whichever looks best to you. But I suspect the "ret = -ENOMEM" cannot be dropped, because it's relied on further down too?
Quoted 13 lines
Thanks,
Quoted 7 lines
mm/shmem.c +1
-0+ }if (!*pagep) { page =shmem_alloc_page (gfp, info, pgoff);Signature · 1 line
2.31.1.498.g6c1eba8ee3d-googSignature · 1 line
Peter Xu
On Wed, Apr 28, 2021 at 02:03:05PM -0700, Hugh Dickins wrote:
[...]
Quoted 9 lines
Quoted 8 lines
mm/shmem.c static int shmem_mfill_atomic_pte(struct mm_struct *dst_mm, · line 2389
Quoted 4 lines
mm/shmem.c +3
-0+put_page (*pagep); + *pagep =NULL ; + } goto out;All "goto out" in this functions looks weird as it returns directly... so if you're touching this after all, I suggest we do "return
-ENOMEM" directly and drop the "ret =-ENOMEM".
Quoted 3 lines
No strong feeling either way from me on that: whichever looks best to you. But I suspect the "ret =
-ENOMEM" cannot be dropped, because it's relied on further down too?
Ah sorry I just noticed Axel didn't really touch that line.. :) So yeah please also feel free to keep it as is.
If to drop it, "ret = -ENOMEM" can go as well, I think.. since all later errors should always reset variable "ret".
Thanks,
Signature · 1 line
Peter Xu
Quoted 18 lines · Peter Xu
On Wed, Apr 28, 2021 at 02:03:05PM -0700, Hugh Dickins wrote:
[...]
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
mm/shmem.c +3
-0+put_page (*pagep); + *pagep =NULL ; + } goto out;Quoted 7 lines
Quoted 3 lines
All "goto out" in this functions looks weird as it returns directly... so if you're touching this after all, I suggest we do "return
-ENOMEM" directly and drop the "ret =-ENOMEM".No strong feeling either way from me on that: whichever looks best to you. But I suspect the "ret =
-ENOMEM" cannot be dropped, because it's relied on further down too?
Quoted 5 lines
Ah sorry I just noticed Axel didn't really touch that line.. :) So yeah please also feel free to keep it as is.
If to drop it, "ret =
-ENOMEM" can go as well, I think.. since all later errors should always reset variable "ret".
Although I can see a refactor which simplifies the error handling a bit, my inclination is to leave it alone in this patch, since it's trying to be a simple fix and especially considering we may need to backport it as far back as 4.14.
But, I'll keep this feedback in mind and try to apply it as much as possible to my other series which is significantly refactoring this function already, so the end state ought to be as simple + consistent error handling as possible.
Quoted 6 lines
Thanks,
--
Peter Xu
PATCH v2 1/1 userfaultfd: release page in error path to avoid BUG_ON
In reply to Axel RasmussenConsider the following sequence of events:
1. Userspace issues a UFFD ioctl, which ends up calling into shmem_mfill_atomic_pte(). We successfully account the blocks, we shmem_alloc_page(), but then the copy_from_user() fails. We return -ENOENT. We don't release the page we allocated.
2. Our caller detects this error code, tries the copy_from_user() after dropping the mmap_lock, and retries, calling back into shmem_mfill_atomic_pte().
3. Meanwhile, let's say another process filled up the tmpfs being used.
4. So shmem_mfill_atomic_pte() fails to account blocks this time, and immediately returns - without releasing the page.
This triggers a BUG_ON in our caller, which asserts that the page should always be consumed, unless -ENOENT is returned.
To fix this, detect if we have such a "dangling" page when accounting fails, and if so, release it before returning.
--- mm/shmem.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)
mm/shmem.c +11 -1
@@ -2375,8 +2375,18 @@ static int shmem_mfill_atomic_pte(struct mm_struct *dst_mm,pgoff_t offset, max_off; ret = -ENOMEM ; -if (!shmem_inode_acct_block (inode,1 )) +if (!shmem_inode_acct_block (inode,1 )) { +/* +* We may have got a page, returned -ENOENT triggering a retry, +* and now we find ourselves with -ENOMEM. Release the page, to +* avoid a BUG_ON in our caller. +*/ +if (unlikely (*pagep)) { +put_page (*pagep); + *pagep =NULL ; + } goto out; + }if (!*pagep) { page =shmem_alloc_page (gfp, info, pgoff);
Signature · 1 line
2.31.1.498.g6c1eba8ee3d-goog
Quoted 16 lines · Axel Rasmussen
Consider the following sequence of events:
1. Userspace issues a UFFD ioctl, which ends up calling into
shmem_mfill_atomic_pte(). We successfully account the blocks, weshmem_alloc_page(), but then thecopy_from_user()fails. We return-ENOENT. We don't release the page we allocated.
2. Our caller detects this error code, tries thecopy_from_user()after dropping themmap_lock, and retries, calling back intoshmem_mfill_atomic_pte().
3. Meanwhile, let's say another process filled up the tmpfs being used.
4. Soshmem_mfill_atomic_pte()fails to account blocks this time, and immediately returns - without releasing the page.This triggers a
BUG_ONin our caller, which asserts that the page should always be consumed, unless-ENOENTis returned.
Quoted 6 lines
To fix this, detect if we have such a "dangling" page when accounting fails, and if so, release it before returning.
Fixes cb658a453b93 ("userfaultfd: shmem: avoid leaking blocks and used blocks in UFFDIO_COPY")Reported-by Hugh Dickins <[email protected]>Signed-off-by Axel Rasmussen <[email protected]>
Thanks!
Quoted 30 lines
--- mm/shmem.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)mm/shmem.c +11
-1@@ -2375,8 +2375,18 @@ static int shmem_mfill_atomic_pte(struct mm_struct *dst_mm,pgoff_t offset, max_off; ret = -ENOMEM ; -if (!shmem_inode_acct_block (inode,1 )) +if (!shmem_inode_acct_block (inode,1 )) { +/* +* We may have got a page, returned -ENOENT triggering a retry, +* and now we find ourselves with -ENOMEM. Release the page, to +* avoid a BUG_ON in our caller. +*/ +if (unlikely (*pagep)) { +put_page (*pagep); + *pagep =NULL ; + } goto out; + }if (!*pagep) { page =shmem_alloc_page (gfp, info, pgoff);Signature · 1 line
2.31.1.498.g6c1eba8ee3d-goog
Quoted 16 lines · Axel Rasmussen
Consider the following sequence of events:
1. Userspace issues a UFFD ioctl, which ends up calling into
shmem_mfill_atomic_pte(). We successfully account the blocks, weshmem_alloc_page(), but then thecopy_from_user()fails. We return-ENOENT. We don't release the page we allocated.
2. Our caller detects this error code, tries thecopy_from_user()after dropping themmap_lock, and retries, calling back intoshmem_mfill_atomic_pte().
3. Meanwhile, let's say another process filled up the tmpfs being used.
4. Soshmem_mfill_atomic_pte()fails to account blocks this time, and immediately returns - without releasing the page.This triggers a
BUG_ONin our caller, which asserts that the page should always be consumed, unless-ENOENTis returned.
Quoted 6 lines
To fix this, detect if we have such a "dangling" page when accounting fails, and if so, release it before returning.
Fixes cb658a453b93 ("userfaultfd: shmem: avoid leaking blocks and used blocks in UFFDIO_COPY")Reported-by Hugh Dickins <[email protected]>Signed-off-by Axel Rasmussen <[email protected]>
Thanks,
Signature · 1 line
Peter Xu
Quoted 17 lines · Axel Rasmussen
Consider the following sequence of events:
1. Userspace issues a UFFD ioctl, which ends up calling into
shmem_mfill_atomic_pte(). We successfully account the blocks, weshmem_alloc_page(), but then thecopy_from_user()fails. We return-ENOENT. We don't release the page we allocated.
2. Our caller detects this error code, tries thecopy_from_user()after dropping themmap_lock, and retries, calling back intoshmem_mfill_atomic_pte().
3. Meanwhile, let's say another process filled up the tmpfs being used.
4. Soshmem_mfill_atomic_pte()fails to account blocks this time, and immediately returns - without releasing the page.This triggers a
BUG_ONin our caller, which asserts that the page should always be consumed, unless-ENOENTis returned.
Quoted 6 lines
To fix this, detect if we have such a "dangling" page when accounting fails, and if so, release it before returning.
Fixes cb658a453b93 ("userfaultfd: shmem: avoid leaking blocks and used blocks in UFFDIO_COPY")Reported-by Hugh Dickins <[email protected]>Signed-off-by Axel Rasmussen <[email protected]>
Apologies, I should have added this line:
I believe this fix ought to go into the 4.14 and later stable branches (the commit referenced in Fixes: was introduced in 4.11).
I can resend with this included, if that would be easier.
Quoted 31 lines
--- mm/shmem.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)mm/shmem.c +11
-2@@ -2375,8 +2375,18 @@ static int shmem_mfill_atomic_pte(struct mm_struct *dst_mm,pgoff_t offset, max_off; ret = -ENOMEM ; -if (!shmem_inode_acct_block (inode,1 )) +if (!shmem_inode_acct_block (inode,1 )) { +/* +* We may have got a page, returned -ENOENT triggering a retry, +* and now we find ourselves with -ENOMEM. Release the page, to +* avoid a BUG_ON in our caller. +*/ +if (unlikely (*pagep)) { +put_page (*pagep); + *pagep =NULL ; + } goto out; + }if (!*pagep) { page =shmem_alloc_page (gfp, info, pgoff); --2.31.1.498.g6c1eba8ee3d-goog