mm/memfd: fix information leak in hugetlb folios
Subsystems (3)
PATCH v2 1/1 mm/memfd: fix information leak in hugetlb folios
When allocating hugetlb folios for memfd, three initialization steps are missing:
1. Folios are not zeroed, leading to kernel memory disclosure to userspace
2. Folios are not marked uptodate before adding to page cache
3. hugetlb_fault_mutex is not taken before hugetlb_add_to_page_cache()
The memfd allocation path bypasses the normal page fault handler (hugetlb_no_page) which would handle all of these initialization steps. This is problematic especially for udmabuf use cases where folios are pinned and directly accessed by userspace via DMA.
Fix by matching the initialization pattern used in hugetlb_no_page():
- Zero the folio using folio_zero_user() which is optimized for huge pages
- Mark it uptodate with folio_mark_uptodate()
- Take hugetlb_fault_mutex before adding to page cache to prevent races
The folio_zero_user() change also fixes a potential security issue where uninitialized kernel memory could be disclosed to userspace through read() or mmap() operations on the memfd.
---
v1 -> v2:
- Use folio_zero_user() instead of folio_zero_range() (optimized for huge pages)
- Add folio_mark_uptodate() before adding to page cache
- Add hugetlb_fault_mutex locking around hugetlb_add_to_page_cache()
- Add Fixes: tag and Cc: stable for backporting
- Add Suggested-by: tags for Oscar and David
--- mm/memfd.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+)
mm/memfd.c +27 -0
@@ -96,9 +96,36 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t idx)NULL , gfp_mask);if (folio) { +u32 hash; + +/* +* Zero the folio to prevent information leaks to userspace. +* Use folio_zero_user() which is optimized for huge/gigantic +* pages. Pass 0 as addr_hint since this is not a faulting path +* and we don't have a user virtual address yet. +*/ +folio_zero_user (folio,0 ); + +/* +* Mark the folio uptodate before adding to page cache, +* as required by filemap.c and other hugetlb paths. +*/ +__folio_mark_uptodate (folio); + +/* +* Serialize hugepage allocation and instantiation to prevent +* races with concurrent allocations, as required by all other +* callers of hugetlb_add_to_page_cache(). +*/ + hash =hugetlb_fault_mutex_hash (memfd->f_mapping, idx); +mutex_lock (&hugetlb_fault_mutex_table[hash]); + err =hugetlb_add_to_page_cache (folio, memfd->f_mapping, idx); + +mutex_unlock (&hugetlb_fault_mutex_table[hash]); +if (err) {folio_put (folio); goto err_unresv;
Signature · 1 line
2.43.0
Quoted 49 lines · Deepanshu Kartikey
When allocating hugetlb folios for memfd, three initialization steps are missing:
1. Folios are not zeroed, leading to kernel memory disclosure to userspace
2. Folios are not marked uptodate before adding to page cache
3.hugetlb_fault_mutexis not taken beforehugetlb_add_to_page_cache()The memfd allocation path bypasses the normal page fault handler (
hugetlb_no_page) which would handle all of these initialization steps. This is problematic especially for udmabuf use cases where folios are pinned and directly accessed by userspace via DMA.Fix by matching the initialization pattern used in
hugetlb_no_page():
- Zero the folio usingfolio_zero_user()which is optimized for huge pages
- Mark it uptodate withfolio_mark_uptodate()
- Takehugetlb_fault_mutexbefore adding to page cache to prevent racesThe
folio_zero_user()change also fixes a potential security issue where uninitialized kernel memory could be disclosed to userspace throughread()ormmap()operations on the memfd.Reported-by [email protected]Fixes 89c1905d9c14 ("mm/gup: introduce memfd_pin_folios() for pinning memfd folios")Suggested-by Oscar Salvador <[email protected]>Suggested-by David Hildenbrand <[email protected]>Tested-by [email protected]Signed-off-by Deepanshu Kartikey <[email protected]>---
v1 -> v2:
- Usefolio_zero_user()instead offolio_zero_range()(optimized for huge pages)
- Addfolio_mark_uptodate()before adding to page cache
- Addhugetlb_fault_mutexlocking aroundhugetlb_add_to_page_cache()
- Add Fixes: tag and Cc: stable for backporting
- Add Suggested-by: tags for Oscar and David--- mm/memfd.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+)mm/memfd.c +0
-0@@ -96,9 +96,36 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t idx)NULL , gfp_mask);
mm/memfd.c struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t idx) · line 107
Quoted 10 lines
mm/memfd.c +9
-0if (folio) { +u32 hash; + +/* +* Zero the folio to prevent information leaks to userspace. +* Use folio_zero_user() which is optimized for huge/gigantic +* pages. Pass 0 as addr_hint since this is not a faulting path +* and we don't have a user virtual address yet. +*/ +folio_zero_user (folio,0 );
Staring at hugetlbfs_fallocate(), we see, to pass the offset within the file.
I think it shouldn't make a difference here (I don't see how the offset in the file would be better than 0: it's in both cases not the user address).
mm/memfd.c struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t idx) · line 113
Quoted 6 lines
mm/memfd.c +6
-0+ +/* +* Mark the folio uptodate before adding to page cache, +* as required by filemap.c and other hugetlb paths. +*/ +__folio_mark_uptodate (folio);
Personally, I'd drop this comment as it is really just doing what we do everywhere else :)
Hoping we can factor that out into hugetlb code properly.
Signature · 3 lines
Cheers David
Quoted 21 lines · Deepanshu Kartikey
When allocating hugetlb folios for memfd, three initialization steps are missing:
1. Folios are not zeroed, leading to kernel memory disclosure to userspace
2. Folios are not marked uptodate before adding to page cache
3.hugetlb_fault_mutexis not taken beforehugetlb_add_to_page_cache()The memfd allocation path bypasses the normal page fault handler (
hugetlb_no_page) which would handle all of these initialization steps. This is problematic especially for udmabuf use cases where folios are pinned and directly accessed by userspace via DMA.Fix by matching the initialization pattern used in
hugetlb_no_page():
- Zero the folio usingfolio_zero_user()which is optimized for huge pages
- Mark it uptodate withfolio_mark_uptodate()
- Takehugetlb_fault_mutexbefore adding to page cache to prevent racesThe
folio_zero_user()change also fixes a potential security issue where uninitialized kernel memory could be disclosed to userspace throughread()ormmap()operations on the memfd.
Quoted 9 lines
Reported-by [email protected]Fixes 89c1905d9c14 ("mm/gup: introduce memfd_pin_folios() for pinning memfd folios")Suggested-by Oscar Salvador <[email protected]>Suggested-by David Hildenbrand <[email protected]>Tested-by [email protected]Signed-off-by Deepanshu Kartikey <[email protected]>
As David mentioned, we can drop the comment wrt. __folio_mark_uptodate. As for the addr_hint in folio_zero_user, I do not think it makes a difference in here.
AFAIK, it serves the purpose that subpages belong to the addr_hint will be zeroed the latest to keep them in cache, but here it does not really apply, so '0' should just work?
Signature · 2 lines
Oscar Salvador SUSE Labs
Quoted 21 lines · Deepanshu Kartikey
When allocating hugetlb folios for memfd, three initialization steps are missing:
1. Folios are not zeroed, leading to kernel memory disclosure to userspace
2. Folios are not marked uptodate before adding to page cache
3.hugetlb_fault_mutexis not taken beforehugetlb_add_to_page_cache()The memfd allocation path bypasses the normal page fault handler (
hugetlb_no_page) which would handle all of these initialization steps. This is problematic especially for udmabuf use cases where folios are pinned and directly accessed by userspace via DMA.Fix by matching the initialization pattern used in
hugetlb_no_page():
- Zero the folio usingfolio_zero_user()which is optimized for huge pages
- Mark it uptodate withfolio_mark_uptodate()
- Takehugetlb_fault_mutexbefore adding to page cache to prevent racesThe
folio_zero_user()change also fixes a potential security issue where uninitialized kernel memory could be disclosed to userspace throughread()ormmap()operations on the memfd.
Quoted 9 lines
Reported-by [email protected]Fixes 89c1905d9c14 ("mm/gup: introduce memfd_pin_folios() for pinning memfd folios")Suggested-by Oscar Salvador <[email protected]>Suggested-by David Hildenbrand <[email protected]>Tested-by [email protected]Signed-off-by Deepanshu Kartikey <[email protected]>
Sorry if you were all waiting on a Ack from me. We're agreed that the comment above __folio_mark_uptodate() could be deleted, and that it would be much better if this code can be moved to a shared home in hugetlb later on; but for now it's more urgent to get this patch into hotfixes and on to Linus - please Andrew.
Thanks!
Hugh
Quoted 55 lines
---
v1 -> v2:
- Usefolio_zero_user()instead offolio_zero_range()(optimized for huge pages)
- Addfolio_mark_uptodate()before adding to page cache
- Addhugetlb_fault_mutexlocking aroundhugetlb_add_to_page_cache()
- Add Fixes: tag and Cc: stable for backporting
- Add Suggested-by: tags for Oscar and David--- mm/memfd.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+)mm/memfd.c +27
-0@@ -96,9 +96,36 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t idx)NULL , gfp_mask);if (folio) { +u32 hash; + +/* +* Zero the folio to prevent information leaks to userspace. +* Use folio_zero_user() which is optimized for huge/gigantic +* pages. Pass 0 as addr_hint since this is not a faulting path +* and we don't have a user virtual address yet. +*/ +folio_zero_user (folio,0 ); + +/* +* Mark the folio uptodate before adding to page cache, +* as required by filemap.c and other hugetlb paths. +*/ +__folio_mark_uptodate (folio); + +/* +* Serialize hugepage allocation and instantiation to prevent +* races with concurrent allocations, as required by all other +* callers of hugetlb_add_to_page_cache(). +*/ + hash =hugetlb_fault_mutex_hash (memfd->f_mapping, idx); +mutex_lock (&hugetlb_fault_mutex_table[hash]); + err =hugetlb_add_to_page_cache (folio, memfd->f_mapping, idx); + +mutex_unlock (&hugetlb_fault_mutex_table[hash]); +if (err) {folio_put (folio); goto err_unresv;Signature · 1 line
2.43.0
Hugh Dickins
maintainerQuoted 8 lines · Hugh Dickins
Acked-by Hugh Dickins <[email protected]>Sorry if you were all waiting on a Ack from me. We're agreed that the comment above
__folio_mark_uptodate()could be deleted, and that it would be much better if this code can be moved to a shared home in hugetlb later on; but for now it's more urgent to get this patch into hotfixes and on to Linus - please Andrew.
Ah, it is already there in mm-hotfixes-unstable, thanks; sorry, I somehow just never saw the usual mm-commits mail.
Hugh