mm/gup: fix FOLL_FORCE COW security issue and remove FOLL_COW
Subsystems (5)
PATCH v2 1/1 mm/gup: fix FOLL_FORCE COW security issue and remove FOLL_COW
Ever since the Dirty COW (CVE-2016-5195) security issue happened, we know that FOLL_FORCE can be possibly dangerous, especially if there are races that can be exploited by user space.
Right now, it would be sufficient to have some code that sets a PTE of a R/O-mapped shared page dirty, in order for it to erroneously become writable by FOLL_FORCE. The implications of setting a write-protected PTE dirty might not be immediately obvious to everyone.
And in fact ever since commit 9ae0f87d009c ("mm/shmem: unconditionally set pte dirty in mfill_atomic_install_pte"), we can use UFFDIO_CONTINUE to map a shmem page R/O while marking the pte dirty. This can be used by unprivileged user space to modify tmpfs/shmem file content even if the user does not have write permissions to the file, and to bypass memfd write sealing -- Dirty COW restricted to tmpfs/shmem (CVE-2022-2590).
To fix such security issues for good, the insight is that we really only need that fancy retry logic (FOLL_COW) for COW mappings that are not writable (!VM_WRITE). And in a COW mapping, we really only broke COW if we have an exclusive anonymous page mapped. If we have something else mapped, or the mapped anonymous page might be shared (!PageAnonExclusive), we have to trigger a write fault to break COW. If we don't find an exclusive anonymous page when we retry, we have to trigger COW breaking once again because something intervened.
Let's move away from this mandatory-retry + dirty handling and rely on our PageAnonExclusive() flag for making a similar decision, to use the same COW logic as in other kernel parts here as well. In case we stumble over a PTE in a COW mapping that does not map an exclusive anonymous page, COW was not properly broken and we have to trigger a fake write-fault to break COW.
Just like we do in can_change_pte_writable() added via
commit 64fe24a3e05e ("mm/mprotect: try avoiding write faults for exclusive anonymous pages when changing protection") and commit 76aefad628aa ("mm/mprotect: fix soft-dirty check in can_change_pte_writable()"), take care of softdirty and uffd-wp manually.
For example, a write() via /proc/self/mem to a uffd-wp-protected range has to fail instead of silently granting write access and bypassing the userspace fault handler. Note that FOLL_FORCE is not only used for debug access, but also triggered by applications without debug intentions, for example, when pinning pages via RDMA.
This fixes CVE-2022-2590. Note that only x86_64 and aarch64 are affected, because only those support CONFIG_HAVE_ARCH_USERFAULTFD_MINOR.
Fortunately, FOLL_COW is no longer required to handle FOLL_FORCE. So let's just get rid of it.
Thanks to Nadav Amit for pointing out that the pte_dirty() check in FOLL_FORCE code is problematic and might be exploitable.
Note 1: We don't check for the PTE being dirty because it doesn't matter
for making a "was COWed" decision anymore, and whoever modifies the page has to set the page dirty either way.
Note 2: Kernels before extended uffd-wp support and before
PageAnonExclusive (< 5.19) can simply revert the problematic commit instead and be safe regarding UFFDIO_CONTINUE. A backport to v5.19 requires minor adjustments due to lack of vma_soft_dirty_enabled().
---
v1 -> v2:
- Make the code easier to digest and even more error prone by performing more explicit checks, just failing gracefully and adding better comments.
- Avoid introducing new VM_BUG_ON().
- Mention Nadav's participation in the description
- Mention that we can bypass memfd write sealing
--- include/linux/mm.h | 1 - mm/gup.c | 68 +++++++++++++++++++++++++++++++--------------- mm/huge_memory.c | 64 +++++++++++++++++++++++++++++-------------- 3 files changed, 89 insertions(+), 44 deletions(-)
include/linux/mm.h +0 -1
@@ -2885,7 +2885,6 @@ struct page *follow_page(struct vm_area_struct *vma, unsigned long address,#define FOLL_MIGRATION 0x400/* wait for page to replace migration entry */ #define FOLL_TRIED 0x800/* a retry, previous pass started an IO */ #define FOLL_REMOTE 0x2000/* we are working on non-current tsk/mm */ -#define FOLL_COW 0x4000/* internal GUP flag */ #define FOLL_ANON 0x8000/* don't do file mappings */ #define FOLL_LONGTERM 0x10000/* mapping lifetime is indefinite: see below */ #define FOLL_SPLIT_PMD 0x20000/* split huge pmd before returning */
mm/gup.c +46 -22
89 lines not on this page · raw message
mm/huge_memory.c +43 -21
98 lines not on this page · raw message
base-commit: 1612c382ffbdf1f673caec76502b1c00e6d35363
Signature · 1 line
2.35.3
From: David Hildenbrand
Quoted 1 line
Sent: 09 August 2022 21:57
...
These two functions seem to contain a lot of the same tests. They also seem a bit large for 'inline'.
Quoted 40 lines
mm/gup.c +35
-3-static inline bool can_follow_write_pte (pte_t pte,unsigned int flags) +/* FOLL_FORCE can write to even unwritable PTEs in COW mappings. */ +static inline bool can_follow_write_pte (pte_t pte,struct page *page, +struct vm_area_struct *vma, +unsigned int flags) { -return pte_write (pte) || - ((flags &FOLL_FORCE ) && (flags &FOLL_COW ) &&pte_dirty (pte)); +/* If the pte is writable, we can write to the page. */ +if (pte_write (pte)) +return true; + +/* Maybe FOLL_FORCE is set to override it? */ +if (!(flags &FOLL_FORCE )) +return false; + +/* But FOLL_FORCE has no effect on shared mappings */ +if (vma->vm_flags & (VM_MAYSHARE |VM_SHARED )) +return false; + +/* ... or read-only private ones */ +if (!(vma->vm_flags &VM_MAYWRITE )) +return false; + +/* ... or already writable ones that just need to take a write fault */ +if (vma->vm_flags &VM_WRITE ) +return false; + +/* +* See can_change_pte_writable(): we broke COW and could map the page +* writable if we have an exclusive anonymous page ... +*/ +if (!page || !PageAnon (page) || !PageAnonExclusive (page)) +return false; + +/* ... and a write-fault isn't required for other reasons. */ +if (vma_soft_dirty_enabled (vma) && !pte_soft_dirty (pte)) +return false; +return !userfaultfd_pte_wp (vma, pte); }
...
Quoted 30 lines
mm/huge_memory.c +26
-3-static inline bool can_follow_write_pmd (pmd_t pmd,unsigned int flags) +/* FOLL_FORCE can write to even unwritable PMDs in COW mappings. */ +static inline bool can_follow_write_pmd (pmd_t pmd,struct page *page, +struct vm_area_struct *vma, +unsigned int flags) { -return pmd_write (pmd) || - ((flags &FOLL_FORCE ) && (flags &FOLL_COW ) &&pmd_dirty (pmd)); +/* If the pmd is writable, we can write to the page. */ +if (pmd_write (pmd)) +return true; + +/* Maybe FOLL_FORCE is set to override it? */ +if (!(flags &FOLL_FORCE )) +return false; + +/* But FOLL_FORCE has no effect on shared mappings */ +if (vma->vm_flags & (VM_MAYSHARE |VM_SHARED )) +return false; + +/* ... or read-only private ones */ +if (!(vma->vm_flags &VM_MAYWRITE )) +return false; + +/* ... or already writable ones that just need to take a write fault */ +if (vma->vm_flags &VM_WRITE ) +return false; + +/* +* See can_change_pte_writable(): we broke COW and could map the page
mm/huge_memory.c vm_fault_t do_huge_pmd_wp_page(struct vm_fault *vmf) · line 1428
Quoted 10 lines
mm/huge_memory.c +9
-0+* writable if we have an exclusive anonymous page ... +*/ +if (!page || !PageAnon (page) || !PageAnonExclusive (page)) +return false; + +/* ... and a write-fault isn't required for other reasons. */ +if (vma_soft_dirty_enabled (vma) && !pmd_soft_dirty (pmd)) +return false; +return !userfaultfd_huge_pmd_wp (vma, pmd); }
Perhaps only the initial call (common success path?) should be inlined?
With the flags and vma tests being moved to an inline helper.
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
Quoted 5 lines · David Laight
From: David Hildenbrand
Quoted 1 line
Sent: 09 August 2022 21:57
...
These two functions seem to contain a lot of the same tests.
Yes, but after Linus and I discussed to not even reuse is_cow_mapping() but instead to spell it out, I refrained from factoring common checks out here to harm readability.
[...]
Quoted 4 lines
Perhaps only the initial call (common success path?) should be inlined?
With the flags and vma tests being moved to an inline helper.
Do we really care enough to hurt readability? I mean, most things here are simple bit checks, not expensive function calls.
inline is only a hint to the compiler after all. Please correct me if I'm wrong.
Now, I don't have any strong opinion, but I do want to make progress for this because -stable trees still need fixing and I'll be posting the reproducer on Monday.
Thanks
Signature · 3 lines
Thanks, David / dhildenb