hugetlb: normalize exported interfaces to use base-page indices
Subsystems (6)
cover hugetlb: normalize exported interfaces to use base-page indices
This series stems from a discussion with David. [1]
The series makes a small cleanup to a few hugetlb interfaces used outside the subsystem by standardizing them on base-page indices. Hopefully this makes the interface semantics a bit more coherent with the rest of mm, while the internal hugetlb code continue to use hugepage indices where that remains the more natural fit.
It is based off mm-stable, 3/30/2026, b2c31180b9d6.
[1] https://lore.kernel.org/linux-mm/[email protected]/
Jane Chu (6):
hugetlb: open-code hugetlb folio lookup index conversion
hugetlb: remove the hugetlb_linear_page_index() helper
hugetlb: make hugetlb_fault_mutex_hash() take PAGE_SIZE index
hugetlb: drop vma_hugecache_offset() in favor of linear_page_index()
hugetlb: make hugetlb_add_to_page_cache() use PAGE_SIZE-based index
hugetlb: pass hugetlb reservation ranges in base-page indices
Documentation/mm/hugetlbfs_reserv.rst | 12 +-- fs/hugetlbfs/inode.c | 60 +++++++------- include/linux/hugetlb.h | 31 +------ mm/hugetlb.c | 112 ++++++++++++++------------ mm/memfd.c | 18 +++-- mm/userfaultfd.c | 6 +- 6 files changed, 109 insertions(+), 130 deletions(-)
Signature · 1 line
2.43.5
PATCH 1/6 hugetlb: open-code hugetlb folio lookup index conversion
This patch removes filemap_lock_hugetlb_folio() and open-codes the index conversion at each call site, making it explicit when hugetlb code is translating a hugepage index into the base-page index expected by filemap_lock_folio(). As part of that cleanup, it also uses a base-page index directly in hugetlbfs_zero_partial_page(), where the byte offset is already page-granular. Overall, the change makes the indexing model more obvious at the call sites and avoids hiding the huge-index to base-index conversion inside a helper.
--- fs/hugetlbfs/inode.c | 20 ++++++++++---------- include/linux/hugetlb.h | 12 ------------ mm/hugetlb.c | 4 ++-- 3 files changed, 12 insertions(+), 24 deletions(-)
fs/hugetlbfs/inode.c +10 -10
@@ -242,9 +242,9 @@ static ssize_t hugetlbfs_read_iter(struct kiocb *iocb, struct iov_iter *to)struct hstate *h =hstate_file (file);struct address_space *mapping = file->f_mapping;struct inode *inode = mapping->host; -unsigned long index = iocb->ki_pos >>huge_page_shift (h); +unsigned long idx = iocb->ki_pos >>huge_page_shift (h);unsigned long offset = iocb->ki_pos & ~huge_page_mask (h); -unsigned long end_index; +unsigned long end_idx;loff_t isize;ssize_t retval =0 ;
@@ -257,10 +257,10 @@ static ssize_t hugetlbfs_read_iter(struct kiocb *iocb, struct iov_iter *to) isize =i_size_read (inode);if (!isize)break ; - end_index = (isize -1 ) >>huge_page_shift (h); -if (index > end_index) + end_idx = (isize -1 ) >>huge_page_shift (h); +if (idx > end_idx)break ; -if (index == end_index) { +if (idx == end_idx) { nr = ((isize -1 ) & ~huge_page_mask (h)) +1 ;if (nr <= offset)break ;
@@ -268,7 +268,7 @@ static ssize_t hugetlbfs_read_iter(struct kiocb *iocb, struct iov_iter *to) nr = nr - offset;/* Find the folio */ - folio =filemap_lock_hugetlb_folio (h, mapping, index); + folio =filemap_lock_folio (mapping, idx <<huge_page_order (h));if (IS_ERR (folio)) {/* * We have a HOLE, zero out the user-buffer for the
@@ -307,10 +307,10 @@ static ssize_t hugetlbfs_read_iter(struct kiocb *iocb, struct iov_iter *to) retval = -EFAULT ;break ; } - index += offset >>huge_page_shift (h); + idx += offset >>huge_page_shift (h); offset &= ~huge_page_mask (h); } - iocb->ki_pos = ((loff_t )index <<huge_page_shift (h)) + offset; + iocb->ki_pos = ((loff_t )idx <<huge_page_shift (h)) + offset;return retval; }
@@ -652,10 +652,10 @@ static void hugetlbfs_zero_partial_page(struct hstate *h, loff_t start, loff_t end) { -pgoff_t idx = start >>huge_page_shift (h); +pgoff_t index = start >>PAGE_SHIFT ;struct folio *folio; - folio =filemap_lock_hugetlb_folio (h, mapping, idx); + folio =filemap_lock_folio (mapping, index);if (IS_ERR (folio))return ;
include/linux/hugetlb.h +0 -12
@@ -829,12 +829,6 @@ static inline unsigned int blocks_per_huge_page(struct hstate *h)return huge_page_size (h) /512 ; } -static inline struct folio *filemap_lock_hugetlb_folio (struct hstate *h, -struct address_space *mapping,pgoff_t idx) -{ -return filemap_lock_folio (mapping, idx <<huge_page_order (h)); -} -#include <asm/hugetlb.h> #ifndef is_hugepage_only_range
@@ -1106,12 +1100,6 @@ static inline struct hugepage_subpool *hugetlb_folio_subpool(struct folio *folioreturn NULL ; } -static inline struct folio *filemap_lock_hugetlb_folio (struct hstate *h, -struct address_space *mapping,pgoff_t idx) -{ -return NULL ; -} -static inline int isolate_or_dissolve_huge_folio (struct folio *folio,struct list_head *list) {
mm/hugetlb.c +2 -2
@@ -5724,7 +5724,7 @@ static vm_fault_t hugetlb_no_page(struct address_space *mapping,* before we get page_table_lock. */ new_folio = false; - folio =filemap_lock_hugetlb_folio (h, mapping, vmf->pgoff); + folio =filemap_lock_folio (mapping, vmf->pgoff <<huge_page_order (h));if (IS_ERR (folio)) { size =i_size_read (mapping->host) >>huge_page_shift (h);if (vmf->pgoff >= size)
@@ -6208,7 +6208,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte,if (is_continue) { ret = -EFAULT ; - folio =filemap_lock_hugetlb_folio (h, mapping, idx); + folio =filemap_lock_folio (mapping, idx <<huge_page_order (h));if (IS_ERR (folio)) goto out; folio_in_pagecache = true;
Signature · 1 line
2.43.5
Mike Rapoport
maintainerRe: [PATCH 1/6] hugetlb: open-code hugetlb folio lookup index conversion
Hi,
Quoted 22 lines · Jane Chu
This patch removes
filemap_lock_hugetlb_folio()and open-codes the index conversion at each call site, making it explicit when hugetlb code is translating a hugepage index into the base-page index expected byfilemap_lock_folio(). As part of that cleanup, it also uses a base-page index directly inhugetlbfs_zero_partial_page(), where the byte offset is already page-granular. Overall, the change makes the indexing model more obvious at the call sites and avoids hiding the huge-index to base-index conversion inside a helper.Suggested-by David Hildenbrand <[email protected]>Signed-off-by Jane Chu <[email protected]>--- fs/hugetlbfs/inode.c | 20 ++++++++++---------- include/linux/hugetlb.h | 12 ------------ mm/hugetlb.c | 4 ++-- 3 files changed, 12 insertions(+), 24 deletions(-)fs/hugetlbfs/inode.c +0
-0
fs/hugetlbfs/inode.c static ssize_t hugetlbfs_read_iter(struct kiocb *iocb, struct iov_iter *to) · line 245
Quoted 5 lines
fs/hugetlbfs/inode.c +1
-1@@ -242,9 +242,9 @@ static ssize_t hugetlbfs_read_iter(struct kiocb *iocb, struct iov_iter *to)struct hstate *h =hstate_file (file);struct address_space *mapping = file->f_mapping;struct inode *inode = mapping->host; -unsigned long index = iocb->ki_pos >>huge_page_shift (h); +unsigned long idx = iocb->ki_pos >>huge_page_shift (h);
Is it necessary to rename index to idx?
Quoted 5 lines
fs/hugetlbfs/inode.c +1
-1unsigned long offset = iocb->ki_pos & ~huge_page_mask (h); -unsigned long end_index; +unsigned long end_idx;loff_t isize;ssize_t retval =0 ;
...
fs/hugetlbfs/inode.c static void hugetlbfs_zero_partial_page(struct hstate *h, · line 655
Quoted 6 lines
fs/hugetlbfs/inode.c +1
-1@@ -652,10 +652,10 @@ static void hugetlbfs_zero_partial_page(struct hstate *h, loff_t start, loff_t end) { -pgoff_t idx = start >>huge_page_shift (h); +pgoff_t index = start >>PAGE_SHIFT ;
And idx to index?
Maybe let's pick one and rename the other or just leave them be.
Quoted 2 lines
fs/hugetlbfs/inode.c +0
-0struct folio *folio;
Signature · 2 lines
Sincerely yours, Mike.
Re: [PATCH 1/6] hugetlb: open-code hugetlb folio lookup index conversion
Quoted 50 lines · Mike Rapoport
Hi,
Quoted 22 lines · Jane Chu
This patch removes
filemap_lock_hugetlb_folio()and open-codes the index conversion at each call site, making it explicit when hugetlb code is translating a hugepage index into the base-page index expected byfilemap_lock_folio(). As part of that cleanup, it also uses a base-page index directly inhugetlbfs_zero_partial_page(), where the byte offset is already page-granular. Overall, the change makes the indexing model more obvious at the call sites and avoids hiding the huge-index to base-index conversion inside a helper.Suggested-by David Hildenbrand <[email protected]>Signed-off-by Jane Chu <[email protected]>--- fs/hugetlbfs/inode.c | 20 ++++++++++---------- include/linux/hugetlb.h | 12 ------------ mm/hugetlb.c | 4 ++-- 3 files changed, 12 insertions(+), 24 deletions(-)fs/hugetlbfs/inode.c +0
-0fs/hugetlbfs/inode.c static ssize_t hugetlbfs_read_iter(struct kiocb *iocb, struct iov_iter *to) · line 245
Quoted 5 lines
fs/hugetlbfs/inode.c +1
-1@@ -242,9 +242,9 @@ static ssize_t hugetlbfs_read_iter(struct kiocb *iocb, struct iov_iter *to)struct hstate *h =hstate_file (file);struct address_space *mapping = file->f_mapping;struct inode *inode = mapping->host; -unsigned long index = iocb->ki_pos >>huge_page_shift (h); +unsigned long idx = iocb->ki_pos >>huge_page_shift (h);Is it necessary to rename index to idx?
Quoted 5 lines
fs/hugetlbfs/inode.c +1
-1unsigned long offset = iocb->ki_pos & ~huge_page_mask (h); -unsigned long end_index; +unsigned long end_idx;loff_t isize;ssize_t retval =0 ;...
fs/hugetlbfs/inode.c static void hugetlbfs_zero_partial_page(struct hstate *h, · line 655
Quoted 6 lines
fs/hugetlbfs/inode.c +1
-1@@ -652,10 +652,10 @@ static void hugetlbfs_zero_partial_page(struct hstate *h, loff_t start, loff_t end) { -pgoff_t idx = start >>huge_page_shift (h); +pgoff_t index = start >>PAGE_SHIFT ;And idx to index?
Quoted 1 line
Maybe let's pick one and rename the other or just leave them be.
As I just replied to Oscar, I found the mixture of idx/index both could represent both huge page index as well as base page index creates a bit dizzying situation to the reader. So through out the patches, 'idx' is made to carry the notion of huge page index while 'index' carry the notion of base page index.
thanks,
-jane
Quoted 4 lines
Quoted 3 lines
fs/hugetlbfs/inode.c +0
-0struct folio *folio;
Re: [PATCH 1/6] hugetlb: open-code hugetlb folio lookup index conversion
In reply to Jane ChuQuoted 11 lines · Jane Chu
This patch removes
filemap_lock_hugetlb_folio()and open-codes the index conversion at each call site, making it explicit when hugetlb code is translating a hugepage index into the base-page index expected byfilemap_lock_folio(). As part of that cleanup, it also uses a base-page index directly inhugetlbfs_zero_partial_page(), where the byte offset is already page-granular. Overall, the change makes the indexing model more obvious at the call sites and avoids hiding the huge-index to base-index conversion inside a helper.Suggested-by David Hildenbrand <[email protected]>Signed-off-by Jane Chu <[email protected]>
It kind of funny that most of the patch is s/index/idx noise. Checking mm/hugetlb* and fs/hugetlb/* we do have a mix of index/idx but I would say that idx predominates, so I am ok with going with that one.
Quoted 126 lines
--- fs/hugetlbfs/inode.c | 20 ++++++++++---------- include/linux/hugetlb.h | 12 ------------ mm/hugetlb.c | 4 ++-- 3 files changed, 12 insertions(+), 24 deletions(-)fs/hugetlbfs/inode.c +10
-10@@ -242,9 +242,9 @@ static ssize_t hugetlbfs_read_iter(struct kiocb *iocb, struct iov_iter *to)struct hstate *h =hstate_file (file);struct address_space *mapping = file->f_mapping;struct inode *inode = mapping->host; -unsigned long index = iocb->ki_pos >>huge_page_shift (h); +unsigned long idx = iocb->ki_pos >>huge_page_shift (h);unsigned long offset = iocb->ki_pos & ~huge_page_mask (h); -unsigned long end_index; +unsigned long end_idx;loff_t isize;ssize_t retval =0 ;@@ -257,10 +257,10 @@ static ssize_t hugetlbfs_read_iter(struct kiocb *iocb, struct iov_iter *to) isize =i_size_read (inode);if (!isize)break ; - end_index = (isize -1 ) >>huge_page_shift (h); -if (index > end_index) + end_idx = (isize -1 ) >>huge_page_shift (h); +if (idx > end_idx)break ; -if (index == end_index) { +if (idx == end_idx) { nr = ((isize -1 ) & ~huge_page_mask (h)) +1 ;if (nr <= offset)break ;@@ -268,7 +268,7 @@ static ssize_t hugetlbfs_read_iter(struct kiocb *iocb, struct iov_iter *to) nr = nr - offset;/* Find the folio */ - folio =filemap_lock_hugetlb_folio (h, mapping, index); + folio =filemap_lock_folio (mapping, idx <<huge_page_order (h));if (IS_ERR (folio)) {/* * We have a HOLE, zero out the user-buffer for the @@ -307,10 +307,10 @@ static ssize_t hugetlbfs_read_iter(struct kiocb *iocb, struct iov_iter *to) retval = -EFAULT ;break ; } - index += offset >>huge_page_shift (h); + idx += offset >>huge_page_shift (h); offset &= ~huge_page_mask (h); } - iocb->ki_pos = ((loff_t )index <<huge_page_shift (h)) + offset; + iocb->ki_pos = ((loff_t )idx <<huge_page_shift (h)) + offset;return retval; }@@ -652,10 +652,10 @@ static void hugetlbfs_zero_partial_page(struct hstate *h, loff_t start, loff_t end) { -pgoff_t idx = start >>huge_page_shift (h); +pgoff_t index = start >>PAGE_SHIFT ;struct folio *folio; - folio =filemap_lock_hugetlb_folio (h, mapping, idx); + folio =filemap_lock_folio (mapping, index);if (IS_ERR (folio))return ;include/linux/hugetlb.h +0
-12@@ -829,12 +829,6 @@ static inline unsigned int blocks_per_huge_page(struct hstate *h)return huge_page_size (h) /512 ; } -static inline struct folio *filemap_lock_hugetlb_folio (struct hstate *h, -struct address_space *mapping,pgoff_t idx) -{ -return filemap_lock_folio (mapping, idx <<huge_page_order (h)); -} -#include <asm/hugetlb.h> #ifndef is_hugepage_only_range@@ -1106,12 +1100,6 @@ static inline struct hugepage_subpool *hugetlb_folio_subpool(struct folio *folioreturn NULL ; } -static inline struct folio *filemap_lock_hugetlb_folio (struct hstate *h, -struct address_space *mapping,pgoff_t idx) -{ -return NULL ; -} -static inline int isolate_or_dissolve_huge_folio (struct folio *folio,struct list_head *list) {mm/hugetlb.c +2
-2@@ -5724,7 +5724,7 @@ static vm_fault_t hugetlb_no_page(struct address_space *mapping,* before we get page_table_lock. */ new_folio = false; - folio =filemap_lock_hugetlb_folio (h, mapping, vmf->pgoff); + folio =filemap_lock_folio (mapping, vmf->pgoff <<huge_page_order (h));if (IS_ERR (folio)) { size =i_size_read (mapping->host) >>huge_page_shift (h);if (vmf->pgoff >= size)@@ -6208,7 +6208,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte,if (is_continue) { ret = -EFAULT ; - folio =filemap_lock_hugetlb_folio (h, mapping, idx); + folio =filemap_lock_folio (mapping, idx <<huge_page_order (h));if (IS_ERR (folio)) goto out; folio_in_pagecache = true;Signature · 1 line
2.43.5
Signature · 2 lines
Oscar Salvador SUSE Labs
Re: [PATCH 1/6] hugetlb: open-code hugetlb folio lookup index conversion
Quoted 10 lines · Oscar Salvador
Quoted 9 lines · Jane Chu
This patch removes
filemap_lock_hugetlb_folio()and open-codes the index conversion at each call site, making it explicit when hugetlb code is translating a hugepage index into the base-page index expected byfilemap_lock_folio(). As part of that cleanup, it also uses a base-page index directly inhugetlbfs_zero_partial_page(), where the byte offset is already page-granular. Overall, the change makes the indexing model more obvious at the call sites and avoids hiding the huge-index to base-index conversion inside a helper.
Quoted 6 lines
Quoted 2 lines
Suggested-by David Hildenbrand <[email protected]>Signed-off-by Jane Chu <[email protected]>It kind of funny that most of the patch is s/index/idx noise. Checking
mm/hugetlb*andfs/hugetlb/*we do have a mix of index/idx but I would say that idx predominates, so I am ok with going with that one.
Indeed the situation that both idx/index can represent both huge page index and base page index had led me intentionally memorize which is representing what in a given local context. I thought that to denote 'index' to base page granularity and 'idx' to huge page granularity could relax the readers.
Quoted 2 lines
Acked-by Oscar Salvador <[email protected]>
thanks,
-jane
Quoted 129 lines
Quoted 128 lines
--- fs/hugetlbfs/inode.c | 20 ++++++++++---------- include/linux/hugetlb.h | 12 ------------ mm/hugetlb.c | 4 ++-- 3 files changed, 12 insertions(+), 24 deletions(-)fs/hugetlbfs/inode.c +10
-10@@ -242,9 +242,9 @@ static ssize_t hugetlbfs_read_iter(struct kiocb *iocb, struct iov_iter *to)struct hstate *h =hstate_file (file);struct address_space *mapping = file->f_mapping;struct inode *inode = mapping->host; -unsigned long index = iocb->ki_pos >>huge_page_shift (h); +unsigned long idx = iocb->ki_pos >>huge_page_shift (h);unsigned long offset = iocb->ki_pos & ~huge_page_mask (h); -unsigned long end_index; +unsigned long end_idx;loff_t isize;ssize_t retval =0 ;@@ -257,10 +257,10 @@ static ssize_t hugetlbfs_read_iter(struct kiocb *iocb, struct iov_iter *to) isize =i_size_read (inode);if (!isize)break ; - end_index = (isize -1 ) >>huge_page_shift (h); -if (index > end_index) + end_idx = (isize -1 ) >>huge_page_shift (h); +if (idx > end_idx)break ; -if (index == end_index) { +if (idx == end_idx) { nr = ((isize -1 ) & ~huge_page_mask (h)) +1 ;if (nr <= offset)break ;@@ -268,7 +268,7 @@ static ssize_t hugetlbfs_read_iter(struct kiocb *iocb, struct iov_iter *to) nr = nr - offset;/* Find the folio */ - folio =filemap_lock_hugetlb_folio (h, mapping, index); + folio =filemap_lock_folio (mapping, idx <<huge_page_order (h));if (IS_ERR (folio)) {/* * We have a HOLE, zero out the user-buffer for the @@ -307,10 +307,10 @@ static ssize_t hugetlbfs_read_iter(struct kiocb *iocb, struct iov_iter *to) retval = -EFAULT ;break ; } - index += offset >>huge_page_shift (h); + idx += offset >>huge_page_shift (h); offset &= ~huge_page_mask (h); } - iocb->ki_pos = ((loff_t )index <<huge_page_shift (h)) + offset; + iocb->ki_pos = ((loff_t )idx <<huge_page_shift (h)) + offset;return retval; }@@ -652,10 +652,10 @@ static void hugetlbfs_zero_partial_page(struct hstate *h, loff_t start, loff_t end) { -pgoff_t idx = start >>huge_page_shift (h); +pgoff_t index = start >>PAGE_SHIFT ;struct folio *folio; - folio =filemap_lock_hugetlb_folio (h, mapping, idx); + folio =filemap_lock_folio (mapping, index);if (IS_ERR (folio))return ;include/linux/hugetlb.h +0
-12@@ -829,12 +829,6 @@ static inline unsigned int blocks_per_huge_page(struct hstate *h)return huge_page_size (h) /512 ; } -static inline struct folio *filemap_lock_hugetlb_folio (struct hstate *h, -struct address_space *mapping,pgoff_t idx) -{ -return filemap_lock_folio (mapping, idx <<huge_page_order (h)); -} -#include <asm/hugetlb.h> #ifndef is_hugepage_only_range@@ -1106,12 +1100,6 @@ static inline struct hugepage_subpool *hugetlb_folio_subpool(struct folio *folioreturn NULL ; } -static inline struct folio *filemap_lock_hugetlb_folio (struct hstate *h, -struct address_space *mapping,pgoff_t idx) -{ -return NULL ; -} -static inline int isolate_or_dissolve_huge_folio (struct folio *folio,struct list_head *list) {mm/hugetlb.c +2
-2@@ -5724,7 +5724,7 @@ static vm_fault_t hugetlb_no_page(struct address_space *mapping,* before we get page_table_lock. */ new_folio = false; - folio =filemap_lock_hugetlb_folio (h, mapping, vmf->pgoff); + folio =filemap_lock_folio (mapping, vmf->pgoff <<huge_page_order (h));if (IS_ERR (folio)) { size =i_size_read (mapping->host) >>huge_page_shift (h);if (vmf->pgoff >= size)@@ -6208,7 +6208,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte,if (is_continue) { ret = -EFAULT ; - folio =filemap_lock_hugetlb_folio (h, mapping, idx); + folio =filemap_lock_folio (mapping, idx <<huge_page_order (h));if (IS_ERR (folio)) goto out; folio_in_pagecache = true;Signature · 1 line
2.43.5
Re: [PATCH 1/6] hugetlb: open-code hugetlb folio lookup index conversion
In reply to Jane ChuQuoted 4 lines · Jane Chu
This patch removes
filemap_lock_hugetlb_folio()and open-codes the index conversion at each call site, making it explicit when hugetlb code is translating a hugepage index into the base-page index expected byfilemap_lock_folio()
I think this is too large a piece to break off in a single patch. The first thing I did was look at hugetlbfs_read_iter() and wonder why we're not able to use generic_file_read_iter() here? It used to be necessary because we used to index the page cache in units of hsize, but now we don't, it seems to me that we could use generic_file_read_iter() instead.
Now, what hugetlbfs_read_iter() does have is support for hwpoison handling. I suspect this is something we want in generic_file_read_iter(), it's just nobody's done it yet.
So perhaps that's patch 1 -- add hwpoison support to generic_file_read_iter(). Then patch 2 removes hugetlbfs_read_iter() in favour of using generic_file_read_iter().
Patch 3 is purely this:
(and you can put my Reviewed-by on it).
fs/hugetlbfs/inode.c static void hugetlbfs_zero_partial_page(struct hstate *h, · line 661
Quoted 13 lines
fs/hugetlbfs/inode.c +2
-2@@ -652,10 +652,10 @@ static void hugetlbfs_zero_partial_page(struct hstate *h, loff_t start, loff_t end) { -pgoff_t idx = start >>huge_page_shift (h); +pgoff_t index = start >>PAGE_SHIFT ;struct folio *folio; - folio =filemap_lock_hugetlb_folio (h, mapping, idx); + folio =filemap_lock_folio (mapping, index);if (IS_ERR (folio))return ;
Now for patch 4 ...
mm/hugetlb.c static vm_fault_t hugetlb_no_page(struct address_space *mapping, · line 5730
Quoted 13 lines
mm/hugetlb.c +1
-1@@ -5724,7 +5724,7 @@ static vm_fault_t hugetlb_no_page(struct address_space *mapping,* before we get page_table_lock. */ new_folio = false; - folio =filemap_lock_hugetlb_folio (h, mapping, vmf->pgoff); + folio =filemap_lock_folio (mapping, vmf->pgoff <<huge_page_order (h));if (IS_ERR (folio)) { size =i_size_read (mapping->host) >>huge_page_shift (h);if (vmf->pgoff >= size)
This points to a horrible problem. Everywhere else in the VM has vmf->pgoff in PAGE_SIZE units, and of course hugetlb works in units of hpagesize. So this is an entirely different piece of work where we convert vmf->pgoff to be in units of PAGE_SIZE. That'll be fun!
mm/hugetlb.c int hugetlb_mfill_atomic_pte(pte_t *dst_pte, · line 6214
Quoted 9 lines
mm/hugetlb.c +1
-1@@ -6208,7 +6208,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte,if (is_continue) { ret = -EFAULT ; - folio =filemap_lock_hugetlb_folio (h, mapping, idx); + folio =filemap_lock_folio (mapping, idx <<huge_page_order (h));if (IS_ERR (folio)) goto out; folio_in_pagecache = true;
This is a much smaller (more contained) problem. At least idx is local to this function, so you can calculate it using linear_page_index() and modify the whole function.
Finally, you can delete filemap_lock_hugetlb_folio():
Quoted 30 lines
include/linux/hugetlb.h +0
-12@@ -829,12 +829,6 @@ static inline unsigned int blocks_per_huge_page(struct hstate *h)return huge_page_size (h) /512 ; } -static inline struct folio *filemap_lock_hugetlb_folio (struct hstate *h, -struct address_space *mapping,pgoff_t idx) -{ -return filemap_lock_folio (mapping, idx <<huge_page_order (h)); -} -#include <asm/hugetlb.h> #ifndef is_hugepage_only_range@@ -1106,12 +1100,6 @@ static inline struct hugepage_subpool *hugetlb_folio_subpool(struct folio *folioreturn NULL ; } -static inline struct folio *filemap_lock_hugetlb_folio (struct hstate *h, -struct address_space *mapping,pgoff_t idx) -{ -return NULL ; -} -static inline int isolate_or_dissolve_huge_folio (struct folio *folio,struct list_head *list) {
Re: [PATCH 1/6] hugetlb: open-code hugetlb folio lookup index conversion
Quoted 73 lines · Matthew Wilcox
Quoted 4 lines · Jane Chu
This patch removes
filemap_lock_hugetlb_folio()and open-codes the index conversion at each call site, making it explicit when hugetlb code is translating a hugepage index into the base-page index expected byfilemap_lock_folio()I think this is too large a piece to break off in a single patch. The first thing I did was look at
hugetlbfs_read_iter()and wonder why we're not able to usegeneric_file_read_iter()here? It used to be necessary because we used to index the page cache in units of hsize, but now we don't, it seems to me that we could usegeneric_file_read_iter()instead.Now, what
hugetlbfs_read_iter()does have is support for hwpoison handling. I suspect this is something we want ingeneric_file_read_iter(), it's just nobody's done it yet.So perhaps that's patch 1 -- add hwpoison support to
generic_file_read_iter(). Then patch 2 removeshugetlbfs_read_iter()in favour of usinggeneric_file_read_iter().Patch 3 is purely this:
(and you can put my Reviewed-by on it).fs/hugetlbfs/inode.c static void hugetlbfs_zero_partial_page(struct hstate *h, · line 661
Quoted 13 lines
fs/hugetlbfs/inode.c +2
-2@@ -652,10 +652,10 @@ static void hugetlbfs_zero_partial_page(struct hstate *h, loff_t start, loff_t end) { -pgoff_t idx = start >>huge_page_shift (h); +pgoff_t index = start >>PAGE_SHIFT ;struct folio *folio; - folio =filemap_lock_hugetlb_folio (h, mapping, idx); + folio =filemap_lock_folio (mapping, index);if (IS_ERR (folio))return ;Now for patch 4 ...
mm/hugetlb.c static vm_fault_t hugetlb_no_page(struct address_space *mapping, · line 5730
Quoted 13 lines
mm/hugetlb.c +1
-1@@ -5724,7 +5724,7 @@ static vm_fault_t hugetlb_no_page(struct address_space *mapping,* before we get page_table_lock. */ new_folio = false; - folio =filemap_lock_hugetlb_folio (h, mapping, vmf->pgoff); + folio =filemap_lock_folio (mapping, vmf->pgoff <<huge_page_order (h));if (IS_ERR (folio)) { size =i_size_read (mapping->host) >>huge_page_shift (h);if (vmf->pgoff >= size)This points to a horrible problem. Everywhere else in the VM has
vmf->pgoffinPAGE_SIZEunits, and of course hugetlb works in units of hpagesize. So this is an entirely different piece of work where we convertvmf->pgoffto be in units ofPAGE_SIZE. That'll be fun!mm/hugetlb.c int hugetlb_mfill_atomic_pte(pte_t *dst_pte, · line 6214
Quoted 9 lines
mm/hugetlb.c +1
-1@@ -6208,7 +6208,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte,if (is_continue) { ret = -EFAULT ; - folio =filemap_lock_hugetlb_folio (h, mapping, idx); + folio =filemap_lock_folio (mapping, idx <<huge_page_order (h));if (IS_ERR (folio)) goto out; folio_in_pagecache = true;This is a much smaller (more contained) problem. At least idx is local to this function, so you can calculate it using
linear_page_index()and modify the whole function.
Quoted 1 line
Finally, you can delete
filemap_lock_hugetlb_folio():
Thanks Matthew, these are great suggestions and I am looking into them. Main concern is potential performance impact, will find out.
Thanks a lot!
-jane
Quoted 31 lines
Quoted 31 lines
include/linux/hugetlb.h +0
-12@@ -829,12 +829,6 @@ static inline unsigned int blocks_per_huge_page(struct hstate *h)return huge_page_size (h) /512 ; } -static inline struct folio *filemap_lock_hugetlb_folio (struct hstate *h, -struct address_space *mapping,pgoff_t idx) -{ -return filemap_lock_folio (mapping, idx <<huge_page_order (h)); -} -#include <asm/hugetlb.h> #ifndef is_hugepage_only_range@@ -1106,12 +1100,6 @@ static inline struct hugepage_subpool *hugetlb_folio_subpool(struct folio *folioreturn NULL ; } -static inline struct folio *filemap_lock_hugetlb_folio (struct hstate *h, -struct address_space *mapping,pgoff_t idx) -{ -return NULL ; -} -static inline int isolate_or_dissolve_huge_folio (struct folio *folio,struct list_head *list) {
Re: [PATCH 1/6] hugetlb: open-code hugetlb folio lookup index conversion
Quoted 2 lines · On Wed, Apr 22, 2026 at 09:44:40AM -0700, [email protected] wrote:
Thanks Matthew, these are great suggestions and I am looking into them. Main concern is potential performance impact, will find out.
I doubt we'll see any performance impact to be honest. We're doing a small amount of extra work, but these are not commonly used code paths and I can't imagine we'll notice it on any benchmark. This is about cleanup; a pgoff_t is supposed to be a file position divided by PAGE_SIZE. Hugetlb mostly doesn't do that, it indexes in hpage_size() which it doesn't need to do any more. It's confusing for people coming to hugetlb from other parts of the kernel, so it really needs to be cleaned up.
PATCH 2/6 hugetlb: remove the hugetlb_linear_page_index() helper
In reply to Jane Chuhugetlb_linear_page_index() is just linear_page_index() converted from base-page units to hugetlb page units.
Open-code that conversion at its remaining call site in mfill_atomic_hugetlb() and drop the helper.
No functional change intended.
--- include/linux/hugetlb.h | 17 ----------------- mm/userfaultfd.c | 3 ++- 2 files changed, 2 insertions(+), 18 deletions(-)
include/linux/hugetlb.h +0 -17
@@ -792,23 +792,6 @@ static inline unsigned huge_page_shift(struct hstate *h)return h->order +PAGE_SHIFT ; } -/** -* hugetlb_linear_page_index() - linear_page_index() but in hugetlb -* page size granularity. -* @vma: the hugetlb VMA -* @address: the virtual address within the VMA -* -* Return: the page offset within the mapping in huge page units. -*/ -static inline pgoff_t hugetlb_linear_page_index (struct vm_area_struct *vma, -unsigned long address) -{ -struct hstate *h =hstate_vma (vma); - -return ((address - vma->vm_start) >>huge_page_shift (h)) + - (vma->vm_pgoff >>huge_page_order (h)); -} -static inline bool order_is_gigantic (unsigned int order) {return order >MAX_PAGE_ORDER ;
mm/userfaultfd.c +2 -1
@@ -573,7 +573,8 @@ static __always_inline ssize_t mfill_atomic_hugetlb(* in the case of shared pmds. fault mutex prevents * races with other faulting threads. */ - idx =hugetlb_linear_page_index (dst_vma, dst_addr); + idx =linear_page_index (dst_vma, dst_addr); + idx >>=huge_page_order (hstate_vma (dst_vma)); mapping = dst_vma->vm_file->f_mapping; hash =hugetlb_fault_mutex_hash (mapping, idx);mutex_lock (&hugetlb_fault_mutex_table[hash]);
Signature · 1 line
2.43.5
Re: [PATCH 2/6] hugetlb: remove the hugetlb_linear_page_index() helper
Quoted 9 lines · Jane Chu
hugetlb_linear_page_index()is justlinear_page_index()converted from base-page units to hugetlb page units.Open-code that conversion at its remaining call site in
mfill_atomic_hugetlb()and drop the helper.No functional change intended.
Signed-off-by Jane Chu <[email protected]>
Signature · 2 lines
Oscar Salvador SUSE Labs
PATCH 3/6 hugetlb: make hugetlb_fault_mutex_hash() take PAGE_SIZE index
In reply to Jane Chuhugetlb_fault_mutex_hash() is used to serialize faults and page cache operations on the same hugetlb file offset. The helper currently expects its index argument in hugetlb page granularity, so callers have to open-code conversions from the PAGE_SIZE-based indices commonly used in the rest of MM helpers.
Change hugetlb_fault_mutex_hash() to take a PAGE_SIZE-based index instead, and perform the hugetlb-granularity conversion inside the helper. Update all callers accordingly.
This makes the helper interface consistent with filemap_get_folio(), and linear_page_index(), while preserving the same lock selection for a given hugetlb file offset.
--- fs/hugetlbfs/inode.c | 19 ++++++++++--------- mm/hugetlb.c | 28 +++++++++++++++++++--------- mm/memfd.c | 11 ++++++----- mm/userfaultfd.c | 7 +++---- 4 files changed, 38 insertions(+), 27 deletions(-)
fs/hugetlbfs/inode.c +10 -9
@@ -575,7 +575,7 @@ static void remove_inode_hugepages(struct inode *inode, loff_t lstart,struct address_space *mapping = &inode->i_data;const pgoff_t end = lend >>PAGE_SHIFT ;struct folio_batch fbatch; -pgoff_t next, index; +pgoff_t next, idx;int i, freed =0 ;bool truncate_op = (lend ==LLONG_MAX );
@@ -586,15 +586,15 @@ static void remove_inode_hugepages(struct inode *inode, loff_t lstart,struct folio *folio = fbatch.folios[i];u32 hash =0 ; - index = folio->index >>huge_page_order (h); - hash =hugetlb_fault_mutex_hash (mapping, index); + hash =hugetlb_fault_mutex_hash (mapping, folio->index);mutex_lock (&hugetlb_fault_mutex_table[hash]);/* * Remove folio that was part of folio_batch. */ + idx = folio->index >>huge_page_order (h);remove_inode_single_folio (h, inode, mapping, folio, - index, truncate_op); + idx, truncate_op); freed++;mutex_unlock (&hugetlb_fault_mutex_table[hash]);
@@ -734,7 +734,7 @@ static long hugetlbfs_fallocate(struct file *file, int mode, loff_t offset,struct mm_struct *mm = current->mm;loff_t hpage_size =huge_page_size (h);unsigned long hpage_shift =huge_page_shift (h); -pgoff_t start, index, end; +pgoff_t start, end, idx, index;int error;u32 hash;
@@ -774,7 +774,7 @@ static long hugetlbfs_fallocate(struct file *file, int mode, loff_t offset,vm_flags_init (&pseudo_vma,VM_HUGETLB |VM_MAYSHARE |VM_SHARED ); pseudo_vma.vm_file = file; -for (index = start; index < end; index++) { +for (idx = start; idx < end; idx++) {/* * This is supposed to be the vaddr where the page is being * faulted in, but we have no vaddr here.
@@ -794,14 +794,15 @@ static long hugetlbfs_fallocate(struct file *file, int mode, loff_t offset, }/* addr is the offset within the file (zero based) */ - addr = index * hpage_size; + addr = idx * hpage_size;/* mutex taken here, fault path and hole punch */ + index = idx <<huge_page_order (h); hash =hugetlb_fault_mutex_hash (mapping, index);mutex_lock (&hugetlb_fault_mutex_table[hash]);/* See if already present in mapping to avoid alloc/free */ - folio =filemap_get_folio (mapping, index <<huge_page_order (h)); + folio =filemap_get_folio (mapping, index);if (!IS_ERR (folio)) {folio_put (folio);mutex_unlock (&hugetlb_fault_mutex_table[hash]);
@@ -824,7 +825,7 @@ static long hugetlbfs_fallocate(struct file *file, int mode, loff_t offset, }folio_zero_user (folio, addr);__folio_mark_uptodate (folio); - error =hugetlb_add_to_page_cache (folio, mapping, index); + error =hugetlb_add_to_page_cache (folio, mapping, idx);if (unlikely (error)) {restore_reserve_on_error (h, &pseudo_vma, addr, folio);folio_put (folio);
mm/hugetlb.c +19 -9
@@ -5515,7 +5515,7 @@ static vm_fault_t hugetlb_wp(struct vm_fault *vmf)*/ if (cow_from_owner) {struct address_space *mapping = vma->vm_file->f_mapping; -pgoff_t idx; +pgoff_t index;u32 hash;folio_put (old_folio);
@@ -5528,8 +5528,9 @@ static vm_fault_t hugetlb_wp(struct vm_fault *vmf)* * Reacquire both after unmap operation. */ - idx =vma_hugecache_offset (h, vma, vmf->address); - hash =hugetlb_fault_mutex_hash (mapping, idx); + index =linear_page_index (vma, vmf->address); + hash =hugetlb_fault_mutex_hash (mapping, index); +hugetlb_vma_unlock_read (vma);mutex_unlock (&hugetlb_fault_mutex_table[hash]);
@@ -5664,6 +5665,10 @@ static inline vm_fault_t hugetlb_handle_userfault(struct vm_fault *vmf, unsigned long reason) {u32 hash; +pgoff_t index; + + index =linear_page_index ((const struct vm_area_struct *)vmf, vmf->address); + hash =hugetlb_fault_mutex_hash (mapping, index);/* * vma_lock and hugetlb_fault_mutex must be dropped before handling
@@ -5671,7 +5676,6 @@ static inline vm_fault_t hugetlb_handle_userfault(struct vm_fault *vmf,* userfault, any vma operation should be careful from here. */ hugetlb_vma_unlock_read (vmf->vma); - hash =hugetlb_fault_mutex_hash (mapping, vmf->pgoff);mutex_unlock (&hugetlb_fault_mutex_table[hash]);return handle_userfault (vmf, reason); }
@@ -5696,7 +5700,8 @@ static bool hugetlb_pte_stable(struct hstate *h, struct mm_struct *mm, unsignedstatic vm_fault_t hugetlb_no_page (struct address_space *mapping,struct vm_fault *vmf) { -u32 hash =hugetlb_fault_mutex_hash (mapping, vmf->pgoff); +u32 hash; +pgoff_t index;bool new_folio, new_anon_folio = false;struct vm_area_struct *vma = vmf->vma;struct mm_struct *mm = vma->vm_mm;
@@ -5707,6 +5712,8 @@ static vm_fault_t hugetlb_no_page(struct address_space *mapping,unsigned long size;pte_t new_pte; + index = vmf->pgoff <<huge_page_order (h); + hash =hugetlb_fault_mutex_hash (mapping, index);/* * Currently, we are forced to kill the process in the event the * original mapper has unmapped pages from the child due to a failed
@@ -5920,13 +5927,14 @@ static vm_fault_t hugetlb_no_page(struct address_space *mapping, }#ifdef CONFIG_SMP -u32 hugetlb_fault_mutex_hash (struct address_space *mapping,pgoff_t idx) +/* 'index' is expected to be in PAGE_SIZE granularity */ +u32 hugetlb_fault_mutex_hash (struct address_space *mapping,pgoff_t index) {unsigned long key[2 ];u32 hash; key[0 ] = (unsigned long ) mapping; - key[1 ] = idx; + key[1 ] = index >>huge_page_order (hstate_inode (mapping->host)); hash =jhash2 ((u32 *)&key,sizeof (key)/(sizeof (u32)),0 );
@@ -5937,7 +5945,7 @@ u32 hugetlb_fault_mutex_hash(struct address_space *mapping, pgoff_t idx)* For uniprocessor systems we always use a single mutex, so just * return 0 and avoid the hashing overhead. */ -u32 hugetlb_fault_mutex_hash (struct address_space *mapping,pgoff_t idx) +u32 hugetlb_fault_mutex_hash (struct address_space *mapping,pgoff_t index) {return 0 ; }
@@ -5952,6 +5960,7 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma,struct hstate *h =hstate_vma (vma);struct address_space *mapping;bool need_wait_lock = false; +pgoff_t index;struct vm_fault vmf = { .vma = vma, .address = address &huge_page_mask (h),
@@ -5972,8 +5981,9 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma,* get spurious allocation failures if two CPUs race to instantiate * the same page in the page cache. */ + index =linear_page_index (vma, vmf.address); mapping = vma->vm_file->f_mapping; - hash =hugetlb_fault_mutex_hash (mapping, vmf.pgoff); + hash =hugetlb_fault_mutex_hash (mapping, index);mutex_lock (&hugetlb_fault_mutex_table[hash]);/*
mm/memfd.c +6 -5
@@ -64,7 +64,7 @@ static void memfd_tag_pins(struct xa_state *xas)* (memfd_pin_folios()) cannot find a folio in the page cache at a given * index in the mapping. */ -struct folio *memfd_alloc_folio (struct file *memfd,pgoff_t idx) +struct folio *memfd_alloc_folio (struct file *memfd,pgoff_t index) {#ifdef CONFIG_HUGETLB_PAGE struct folio *folio;
@@ -79,12 +79,13 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t idx)*/ struct inode *inode =file_inode (memfd);struct hstate *h =hstate_file (memfd); -int err = -ENOMEM ;long nr_resv; +pgoff_t idx; +int err = -ENOMEM ; gfp_mask =htlb_alloc_mask (h); gfp_mask &= ~(__GFP_HIGHMEM | __GFP_MOVABLE); - idx >>=huge_page_order (h); + idx = index >>huge_page_order (h); nr_resv =hugetlb_reserve_pages (inode, idx, idx +1 ,NULL ,EMPTY_VMA_FLAGS );if (nr_resv <0 )
@@ -116,7 +117,7 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t idx)* races with concurrent allocations, as required by all other * callers of hugetlb_add_to_page_cache(). */ - hash =hugetlb_fault_mutex_hash (memfd->f_mapping, idx); + hash =hugetlb_fault_mutex_hash (memfd->f_mapping, index);mutex_lock (&hugetlb_fault_mutex_table[hash]); err = hugetlb_add_to_page_cache(folio,
@@ -140,7 +141,7 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t idx) return ERR_PTR(err); }#endif -return shmem_read_folio (memfd->f_mapping, idx); +return shmem_read_folio (memfd->f_mapping, index); }/*
mm/userfaultfd.c +3 -4
@@ -504,7 +504,7 @@ static __always_inline ssize_t mfill_atomic_hugetlb(long copied;struct folio *folio;unsigned long vma_hpagesize; -pgoff_t idx; +pgoff_t index;u32 hash;struct address_space *mapping;
@@ -573,10 +573,9 @@ static __always_inline ssize_t mfill_atomic_hugetlb(* in the case of shared pmds. fault mutex prevents * races with other faulting threads. */ - idx =linear_page_index (dst_vma, dst_addr); - idx >>=huge_page_order (hstate_vma (dst_vma)); + index =linear_page_index (dst_vma, dst_addr); mapping = dst_vma->vm_file->f_mapping; - hash =hugetlb_fault_mutex_hash (mapping, idx); + hash =hugetlb_fault_mutex_hash (mapping, index);mutex_lock (&hugetlb_fault_mutex_table[hash]);hugetlb_vma_lock_read (dst_vma);
Signature · 1 line
2.43.5
Re: [PATCH 3/6] hugetlb: make hugetlb_fault_mutex_hash() take PAGE_SIZE index
Quoted 125 lines · On Thu, 9 Apr 2026 17:41:54 -0600 Jane Chu <[email protected]> wrote:
hugetlb_fault_mutex_hash()is used to serialize faults and page cache operations on the same hugetlb file offset. The helper currently expects its index argument in hugetlb page granularity, so callers have to open-code conversions from the PAGE_SIZE-based indices commonly used in the rest of MM helpers.Change
hugetlb_fault_mutex_hash()to take a PAGE_SIZE-based index instead, and perform the hugetlb-granularity conversion inside the helper. Update all callers accordingly.This makes the helper interface consistent with
filemap_get_folio(), andlinear_page_index(), while preserving the same lock selection for a given hugetlb file offset.Signed-off-by Jane Chu <[email protected]>--- fs/hugetlbfs/inode.c | 19 ++++++++++--------- mm/hugetlb.c | 28 +++++++++++++++++++--------- mm/memfd.c | 11 ++++++----- mm/userfaultfd.c | 7 +++---- 4 files changed, 38 insertions(+), 27 deletions(-)fs/hugetlbfs/inode.c +10
-9@@ -575,7 +575,7 @@ static void remove_inode_hugepages(struct inode *inode, loff_t lstart,struct address_space *mapping = &inode->i_data;const pgoff_t end = lend >>PAGE_SHIFT ;struct folio_batch fbatch; -pgoff_t next, index; +pgoff_t next, idx;int i, freed =0 ;bool truncate_op = (lend ==LLONG_MAX );@@ -586,15 +586,15 @@ static void remove_inode_hugepages(struct inode *inode, loff_t lstart,struct folio *folio = fbatch.folios[i];u32 hash =0 ; - index = folio->index >>huge_page_order (h); - hash =hugetlb_fault_mutex_hash (mapping, index); + hash =hugetlb_fault_mutex_hash (mapping, folio->index);mutex_lock (&hugetlb_fault_mutex_table[hash]);/* * Remove folio that was part of folio_batch. */ + idx = folio->index >>huge_page_order (h);remove_inode_single_folio (h, inode, mapping, folio, - index, truncate_op); + idx, truncate_op); freed++;mutex_unlock (&hugetlb_fault_mutex_table[hash]);@@ -734,7 +734,7 @@ static long hugetlbfs_fallocate(struct file *file, int mode, loff_t offset,struct mm_struct *mm = current->mm;loff_t hpage_size =huge_page_size (h);unsigned long hpage_shift =huge_page_shift (h); -pgoff_t start, index, end; +pgoff_t start, end, idx, index;int error;u32 hash;@@ -774,7 +774,7 @@ static long hugetlbfs_fallocate(struct file *file, int mode, loff_t offset,vm_flags_init (&pseudo_vma,VM_HUGETLB |VM_MAYSHARE |VM_SHARED ); pseudo_vma.vm_file = file; -for (index = start; index < end; index++) { +for (idx = start; idx < end; idx++) {/* * This is supposed to be the vaddr where the page is being * faulted in, but we have no vaddr here. @@ -794,14 +794,15 @@ static long hugetlbfs_fallocate(struct file *file, int mode, loff_t offset, }/* addr is the offset within the file (zero based) */ - addr = index * hpage_size; + addr = idx * hpage_size;/* mutex taken here, fault path and hole punch */ + index = idx <<huge_page_order (h); hash =hugetlb_fault_mutex_hash (mapping, index);mutex_lock (&hugetlb_fault_mutex_table[hash]);/* See if already present in mapping to avoid alloc/free */ - folio =filemap_get_folio (mapping, index <<huge_page_order (h)); + folio =filemap_get_folio (mapping, index);if (!IS_ERR (folio)) {folio_put (folio);mutex_unlock (&hugetlb_fault_mutex_table[hash]);@@ -824,7 +825,7 @@ static long hugetlbfs_fallocate(struct file *file, int mode, loff_t offset, }folio_zero_user (folio, addr);__folio_mark_uptodate (folio); - error =hugetlb_add_to_page_cache (folio, mapping, index); + error =hugetlb_add_to_page_cache (folio, mapping, idx);if (unlikely (error)) {restore_reserve_on_error (h, &pseudo_vma, addr, folio);folio_put (folio);mm/hugetlb.c +4
-3@@ -5515,7 +5515,7 @@ static vm_fault_t hugetlb_wp(struct vm_fault *vmf)*/ if (cow_from_owner) {struct address_space *mapping = vma->vm_file->f_mapping; -pgoff_t idx; +pgoff_t index;u32 hash;folio_put (old_folio);@@ -5528,8 +5528,9 @@ static vm_fault_t hugetlb_wp(struct vm_fault *vmf)* * Reacquire both after unmap operation. */ - idx =vma_hugecache_offset (h, vma, vmf->address); - hash =hugetlb_fault_mutex_hash (mapping, idx); + index =linear_page_index (vma, vmf->address); + hash =hugetlb_fault_mutex_hash (mapping, index); +hugetlb_vma_unlock_read (vma);mutex_unlock (&hugetlb_fault_mutex_table[hash]);
mm/hugetlb.c static inline vm_fault_t hugetlb_handle_userfault(struct vm_fault *vmf, · line 5670
Quoted 6 lines
mm/hugetlb.c +3
-0@@ -5664,6 +5665,10 @@ static inline vm_fault_t hugetlb_handle_userfault(struct vm_fault *vmf, unsigned long reason) {u32 hash; +pgoff_t index; + + index =linear_page_index ((const struct vm_area_struct *)vmf, vmf->address);
This is supposed to be linear_page_index(vmf->vma, vmf->address), right?
Re: [PATCH 3/6] hugetlb: make hugetlb_fault_mutex_hash() take PAGE_SIZE index
Quoted 2 lines · Usama Arif
On Thu, 9 Apr 2026 17:41:54 -0600 Jane Chu <[email protected]> wrote:
[..]
Quoted 11 lines
mm/hugetlb.c static inline vm_fault_t hugetlb_handle_userfault(struct vm_fault *vmf, · line 5670
Quoted 7 lines
mm/hugetlb.c +3
-0@@ -5664,6 +5665,10 @@ static inline vm_fault_t hugetlb_handle_userfault(struct vm_fault *vmf, unsigned long reason) {u32 hash; +pgoff_t index; + + index =linear_page_index ((const struct vm_area_struct *)vmf, vmf->address);This is supposed to be
linear_page_index(vmf->vma, vmf->address), right?
Indeed, sysbot also complained.
Will fix soon.
Thanks a lot!
-jane
Oscar Salvador
maintainerRe: [PATCH 3/6] hugetlb: make hugetlb_fault_mutex_hash() take PAGE_SIZE index
In reply to Jane ChuQuoted 41 lines · Jane Chu
hugetlb_fault_mutex_hash()is used to serialize faults and page cache operations on the same hugetlb file offset. The helper currently expects its index argument in hugetlb page granularity, so callers have to open-code conversions from the PAGE_SIZE-based indices commonly used in the rest of MM helpers.Change
hugetlb_fault_mutex_hash()to take a PAGE_SIZE-based index instead, and perform the hugetlb-granularity conversion inside the helper. Update all callers accordingly.This makes the helper interface consistent with
filemap_get_folio(), andlinear_page_index(), while preserving the same lock selection for a given hugetlb file offset.Signed-off-by Jane Chu <[email protected]>--- fs/hugetlbfs/inode.c | 19 ++++++++++--------- mm/hugetlb.c | 28 +++++++++++++++++++--------- mm/memfd.c | 11 ++++++----- mm/userfaultfd.c | 7 +++---- 4 files changed, 38 insertions(+), 27 deletions(-)fs/hugetlbfs/inode.c +1
-3@@ -575,7 +575,7 @@ static void remove_inode_hugepages(struct inode *inode, loff_t lstart,struct address_space *mapping = &inode->i_data;const pgoff_t end = lend >>PAGE_SHIFT ;struct folio_batch fbatch; -pgoff_t next, index; +pgoff_t next, idx;int i, freed =0 ;bool truncate_op = (lend ==LLONG_MAX );@@ -586,15 +586,15 @@ static void remove_inode_hugepages(struct inode *inode, loff_t lstart,struct folio *folio = fbatch.folios[i];u32 hash =0 ; - index = folio->index >>huge_page_order (h); - hash =hugetlb_fault_mutex_hash (mapping, index);
fs/hugetlbfs/inode.c static void remove_inode_hugepages(struct inode *inode, loff_t lstart, · line 597
Quoted 10 lines
fs/hugetlbfs/inode.c +3
-1+ hash =hugetlb_fault_mutex_hash (mapping, folio->index);mutex_lock (&hugetlb_fault_mutex_table[hash]);/* * Remove folio that was part of folio_batch. */ + idx = folio->index >>huge_page_order (h);remove_inode_single_folio (h, inode, mapping, folio, - index, truncate_op); + idx, truncate_op);
Since this is the only place we call remove_inode_single_folio(), and that we do not the index (at least index >> huge_page_order()) directly in this function, would it not be better to make remove_inode_single_folio do the conversion itself?
Also, I am thinking out loud here but we do have a few places where we
go: idx = index >> huge_page_order() to convert it into hugepage units, but the casual reader might be a bit puzzled about that.
So, would it be worth to have implement an inline helper with an accurate name to do that? It might help whoever reads that?
Signature · 2 lines
Oscar Salvador SUSE Labs
Re: [PATCH 3/6] hugetlb: make hugetlb_fault_mutex_hash() take PAGE_SIZE index
Quoted 45 lines · Oscar Salvador
Quoted 44 lines · Jane Chu
hugetlb_fault_mutex_hash()is used to serialize faults and page cache operations on the same hugetlb file offset. The helper currently expects its index argument in hugetlb page granularity, so callers have to open-code conversions from the PAGE_SIZE-based indices commonly used in the rest of MM helpers.Change
hugetlb_fault_mutex_hash()to take a PAGE_SIZE-based index instead, and perform the hugetlb-granularity conversion inside the helper. Update all callers accordingly.This makes the helper interface consistent with
filemap_get_folio(), andlinear_page_index(), while preserving the same lock selection for a given hugetlb file offset.Signed-off-by Jane Chu <[email protected]>--- fs/hugetlbfs/inode.c | 19 ++++++++++--------- mm/hugetlb.c | 28 +++++++++++++++++++--------- mm/memfd.c | 11 ++++++----- mm/userfaultfd.c | 7 +++---- 4 files changed, 38 insertions(+), 27 deletions(-)fs/hugetlbfs/inode.c +2
-3@@ -575,7 +575,7 @@ static void remove_inode_hugepages(struct inode *inode, loff_t lstart,struct address_space *mapping = &inode->i_data;const pgoff_t end = lend >>PAGE_SHIFT ;struct folio_batch fbatch; -pgoff_t next, index; +pgoff_t next, idx;int i, freed =0 ;bool truncate_op = (lend ==LLONG_MAX );@@ -586,15 +586,15 @@ static void remove_inode_hugepages(struct inode *inode, loff_t lstart,struct folio *folio = fbatch.folios[i];u32 hash =0 ; - index = folio->index >>huge_page_order (h); - hash =hugetlb_fault_mutex_hash (mapping, index); + hash =hugetlb_fault_mutex_hash (mapping, folio->index);mutex_lock (&hugetlb_fault_mutex_table[hash]);
Quoted 11 lines
fs/hugetlbfs/inode.c static void remove_inode_hugepages(struct inode *inode, loff_t lstart, · line 597
Quoted 7 lines
fs/hugetlbfs/inode.c +2
-1/* * Remove folio that was part of folio_batch. */ + idx = folio->index >>huge_page_order (h);remove_inode_single_folio (h, inode, mapping, folio, - index, truncate_op); + idx, truncate_op);Since this is the only place we call
remove_inode_single_folio(), and that we do not the index (at least index >>huge_page_order()) directly in this function, would it not be better to makeremove_inode_single_foliodo the conversion itself?
In PATCH 6/6, remove_inode_hugepages() is changed to call remove_inode_single_folio() passing "folio->index" directly, thus eliminating the above conversion altogether.
I apologize for dividing up the patches this way, function by function,
for my convenience, introduced some temporary changes. The overall resulted code hopefully is clearer.
Quoted 7 lines
Also, I am thinking out loud here but we do have a few places where we
go: idx = index >>huge_page_order()to convert it into hugepage units, but the casual reader might be a bit puzzled about that.
So, would it be worth to have implement an inline helper with an accurate name to do that? It might help whoever reads that?
Indeed, will add below inline helpers -
pgoff_t huge_to_base(pgoff_t idx);
pgoff_t base_to_huge(pgoff_t index);
thanks!
-jane
PATCH 4/6 hugetlb: drop vma_hugecache_offset() in favor of linear_page_index()
In reply to Jane Chuvma_hugecache_offset() converts a hugetlb VMA address into a mapping offset in hugepage units. While the helper is small, its name is not very clear, and the resulting code is harder to follow than using the common MM helper directly.
Use linear_page_index() instead, with an explicit conversion from PAGE_SIZE units to hugepage units at each call site, and remove vma_hugecache_offset().
This makes the code a bit more direct and avoids a hugetlb-specific helper whose behavior is already expressible with existing MM primitives.
--- mm/hugetlb.c | 49 +++++++++++++++++++++---------------------------- 1 file changed, 21 insertions(+), 28 deletions(-)
mm/hugetlb.c +21 -28
@@ -1006,17 +1006,6 @@ static long region_count(struct resv_map *resv, long f, long t)return chg; } -/* -* Convert the address within this vma to the page offset within -* the mapping, huge page units here. -*/ -static pgoff_t vma_hugecache_offset (struct hstate *h, -struct vm_area_struct *vma,unsigned long address) -{ -return ((address - vma->vm_start) >>huge_page_shift (h)) + - (vma->vm_pgoff >>huge_page_order (h)); -} -/* * Flags for MAP_PRIVATE reservations. These are stored in the bottom * bits of the reservation map pointer, which are always clear due to
@@ -2465,7 +2454,9 @@ static long __vma_reservation_common(struct hstate *h,if (!resv)return 1 ; - idx =vma_hugecache_offset (h, vma, addr); + idx =linear_page_index (vma, addr); + idx >>=huge_page_order (h); +switch (mode) {case VMA_NEEDS_RESV : ret =region_chg (resv, idx, idx +1 , &dummy_out_regions_needed);
@@ -4718,8 +4709,10 @@ static void hugetlb_vm_op_close(struct vm_area_struct *vma)if (!resv || !is_vma_resv_set (vma,HPAGE_RESV_OWNER ))return ; - start =vma_hugecache_offset (h, vma, vma->vm_start); - end =vma_hugecache_offset (h, vma, vma->vm_end); + start =linear_page_index (vma, vma->vm_start); + start >>=huge_page_order (h); + end =linear_page_index (vma, vma->vm_end); + end >>=huge_page_order (h); reserve = (end - start) -region_count (resv, start, end);hugetlb_cgroup_uncharge_counter (resv, start, end);
@@ -5960,14 +5953,13 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma,struct hstate *h =hstate_vma (vma);struct address_space *mapping;bool need_wait_lock = false; -pgoff_t index; +pgoff_t index =linear_page_index (vma, address &huge_page_mask (h));struct vm_fault vmf = { .vma = vma, .address = address &huge_page_mask (h), .real_address = address, .flags = flags, - .pgoff =vma_hugecache_offset (h, vma, - address &huge_page_mask (h)), + .pgoff = index >>huge_page_order (h),/* TODO: Track hugetlb faults using vm_fault */ /*
@@ -5981,7 +5973,6 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma,* get spurious allocation failures if two CPUs race to instantiate * the same page in the page cache. */ - index =linear_page_index (vma, vmf.address); mapping = vma->vm_file->f_mapping; hash =hugetlb_fault_mutex_hash (mapping, index);mutex_lock (&hugetlb_fault_mutex_table[hash]);
@@ -6182,20 +6173,22 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte,uffd_flags_t flags, struct folio **foliop) { - struct mm_struct *dst_mm = dst_vma->vm_mm; -bool is_continue =uffd_flags_mode_is (flags,MFILL_ATOMIC_CONTINUE ); -bool wp_enabled = (flags &MFILL_ATOMIC_WP ); + pgoff_t idx; +spinlock_t *ptl; +struct folio *folio; +pte_t _dst_pte, dst_ptep; +bool folio_in_pagecache = false;struct hstate *h =hstate_vma (dst_vma); -struct address_space *mapping = dst_vma->vm_file->f_mapping; -pgoff_t idx =vma_hugecache_offset (h, dst_vma, dst_addr);unsigned long size =huge_page_size (h); +struct mm_struct *dst_mm = dst_vma->vm_mm; +bool wp_enabled = (flags &MFILL_ATOMIC_WP );int vm_shared = dst_vma->vm_flags &VM_SHARED ; -pte_t _dst_pte; -spinlock_t *ptl; +struct address_space *mapping = dst_vma->vm_file->f_mapping; +bool is_continue =uffd_flags_mode_is (flags,MFILL_ATOMIC_CONTINUE );int ret = -ENOMEM ; -struct folio *folio; -bool folio_in_pagecache = false; -pte_t dst_ptep; + + idx =linear_page_index (dst_vma, dst_addr); + idx >>=huge_page_order (h);if (uffd_flags_mode_is (flags,MFILL_ATOMIC_POISON )) { ptl =huge_pte_lock (h, dst_mm, dst_pte);
Signature · 1 line
2.43.5
Oscar Salvador
maintainerRe: [PATCH 4/6] hugetlb: drop vma_hugecache_offset() in favor of linear_page_index()
Quoted 5 lines · Jane Chu
vma_hugecache_offset()converts a hugetlb VMA address into a mapping offset in hugepage units. While the helper is small, its name is not very clear, and the resulting code is harder to follow than using the common MM helper directly.
Quoted 8 lines
Use
linear_page_index()instead, with an explicit conversion fromPAGE_SIZEunits to hugepage units at each call site, and removevma_hugecache_offset().This makes the code a bit more direct and avoids a hugetlb-specific helper whose behavior is already expressible with existing MM primitives.
Signed-off-by Jane Chu <[email protected]>
Looks good to me, the only thing is the conversion to hugepage units which may not be very clear to the casual reader, but you already mentioned that you will add a helper, so all good.
Signature · 2 lines
Oscar Salvador SUSE Labs
Re: [PATCH 4/6] hugetlb: drop vma_hugecache_offset() in favor of linear_page_index()
Quoted 13 lines · Oscar Salvador
Quoted 12 lines · Jane Chu
vma_hugecache_offset()converts a hugetlb VMA address into a mapping offset in hugepage units. While the helper is small, its name is not very clear, and the resulting code is harder to follow than using the common MM helper directly.Use
linear_page_index()instead, with an explicit conversion fromPAGE_SIZEunits to hugepage units at each call site, and removevma_hugecache_offset().This makes the code a bit more direct and avoids a hugetlb-specific helper whose behavior is already expressible with existing MM primitives.
Quoted 8 lines
Quoted 1 line
Signed-off-by Jane Chu <[email protected]>Looks good to me, the only thing is the conversion to hugepage units which may not be very clear to the casual reader, but you already mentioned that you will add a helper, so all good.
Yes, will do.
thanks!
-jane
Quoted 1 line
PATCH 5/6 hugetlb: make hugetlb_add_to_page_cache() use PAGE_SIZE-based index
In reply to Jane Chuhugetlb_add_to_page_cache() currently takes a parameter named 'idx', but internally converts it from hugetlb page units into PAGE_SIZE-based page-cache index units before calling __filemap_add_folio().
Make hugetlb_add_to_page_cache() take a PAGE_SIZE-based index directly and update its callers accordingly. This removes the internal shift, keeps the index units consistent with filemap_lock_folio() and __filemap_add_folio(), and simplifies the surrounding code.
--- fs/hugetlbfs/inode.c | 2 +- include/linux/hugetlb.h | 2 +- mm/hugetlb.c | 21 ++++++++------------- mm/memfd.c | 2 +- 4 files changed, 11 insertions(+), 16 deletions(-)
fs/hugetlbfs/inode.c +1 -1
@@ -825,7 +825,7 @@ static long hugetlbfs_fallocate(struct file *file, int mode, loff_t offset, }folio_zero_user (folio, addr);__folio_mark_uptodate (folio); - error =hugetlb_add_to_page_cache (folio, mapping, idx); + error =hugetlb_add_to_page_cache (folio, mapping, index);if (unlikely (error)) {restore_reserve_on_error (h, &pseudo_vma, addr, folio);folio_put (folio);
include/linux/hugetlb.h +1 -1
@@ -713,7 +713,7 @@ struct folio *alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_nid, nodemask_t *nmask, gfp_t gfp_mask);int hugetlb_add_to_page_cache (struct folio *folio,struct address_space *mapping, -pgoff_t idx); +pgoff_t index);void restore_reserve_on_error (struct hstate *h,struct vm_area_struct *vma,unsigned long address,struct folio *folio);
mm/hugetlb.c +8 -13
@@ -5625,15 +5625,14 @@ bool hugetlbfs_pagecache_present(struct hstate *h, }int hugetlb_add_to_page_cache (struct folio *folio,struct address_space *mapping, -pgoff_t idx) +pgoff_t index) {struct inode *inode = mapping->host;struct hstate *h =hstate_inode (inode);int err; - idx <<=huge_page_order (h);__folio_set_locked (folio); - err =__filemap_add_folio (mapping, folio, idx,GFP_KERNEL ,NULL ); + err =__filemap_add_folio (mapping, folio, index,GFP_KERNEL ,NULL );if (unlikely (err)) {__folio_clear_locked (folio);
@@ -5724,7 +5723,7 @@ static vm_fault_t hugetlb_no_page(struct address_space *mapping,* before we get page_table_lock. */ new_folio = false; - folio =filemap_lock_folio (mapping, vmf->pgoff <<huge_page_order (h)); + folio =filemap_lock_folio (mapping, index);if (IS_ERR (folio)) { size =i_size_read (mapping->host) >>huge_page_shift (h);if (vmf->pgoff >= size)
@@ -5788,8 +5787,7 @@ static vm_fault_t hugetlb_no_page(struct address_space *mapping, new_folio = true;if (vma->vm_flags &VM_MAYSHARE ) { -int err =hugetlb_add_to_page_cache (folio, mapping, - vmf->pgoff); +int err =hugetlb_add_to_page_cache (folio, mapping, index);if (err) {/* * err can't be -EEXIST which implies someone
@@ -6173,7 +6171,6 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte,uffd_flags_t flags, struct folio **foliop) { - pgoff_t idx; spinlock_t *ptl;struct folio *folio;pte_t _dst_pte, dst_ptep;
@@ -6183,13 +6180,11 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte,struct mm_struct *dst_mm = dst_vma->vm_mm;bool wp_enabled = (flags &MFILL_ATOMIC_WP );int vm_shared = dst_vma->vm_flags &VM_SHARED ; +pgoff_t index =linear_page_index (dst_vma, dst_addr);struct address_space *mapping = dst_vma->vm_file->f_mapping;bool is_continue =uffd_flags_mode_is (flags,MFILL_ATOMIC_CONTINUE );int ret = -ENOMEM ; - idx =linear_page_index (dst_vma, dst_addr); - idx >>=huge_page_order (h); -if (uffd_flags_mode_is (flags,MFILL_ATOMIC_POISON )) { ptl =huge_pte_lock (h, dst_mm, dst_pte);
@@ -6211,7 +6206,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte,if (is_continue) { ret = -EFAULT ; - folio =filemap_lock_folio (mapping, idx <<huge_page_order (h)); + folio =filemap_lock_folio (mapping, index);if (IS_ERR (folio)) goto out; folio_in_pagecache = true;
@@ -6307,7 +6302,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte,/* Add shared, newly allocated pages to the page cache. */ if (vm_shared && !is_continue) { ret = -EFAULT ; -if (idx >= (i_size_read (mapping->host) >>huge_page_shift (h))) +if (index >= (i_size_read (mapping->host) >>PAGE_SHIFT )) goto out_release_nounlock;/*
@@ -6316,7 +6311,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte,* hugetlb_fault_mutex_table that here must be hold by * the caller. */ - ret =hugetlb_add_to_page_cache (folio, mapping, idx); + ret =hugetlb_add_to_page_cache (folio, mapping, index);if (ret) goto out_release_nounlock; folio_in_pagecache = true;
mm/memfd.c +1 -1
@@ -122,7 +122,7 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t index) err =hugetlb_add_to_page_cache (folio, memfd->f_mapping, - idx); + index);mutex_unlock (&hugetlb_fault_mutex_table[hash]);
Signature · 1 line
2.43.5
Re: [PATCH 5/6] hugetlb: make hugetlb_add_to_page_cache() use PAGE_SIZE-based index
Quoted 10 lines · Jane Chu
hugetlb_add_to_page_cache()currently takes a parameter named 'idx', but internally converts it from hugetlb page units into PAGE_SIZE-based page-cache index units before calling__filemap_add_folio().Make
hugetlb_add_to_page_cache()take a PAGE_SIZE-based index directly and update its callers accordingly. This removes the internal shift, keeps the index units consistent withfilemap_lock_folio()and__filemap_add_folio(), and simplifies the surrounding code.Signed-off-by Jane Chu <[email protected]>
Signature · 2 lines
Oscar Salvador SUSE Labs
PATCH 6/6 hugetlb: pass hugetlb reservation ranges in base-page indices
In reply to Jane Chuhugetlb_reserve_pages() consume indices in hugepage granularity although some callers naturally compute offsets in PAGE_SIZE units.
Teach the reservation helpers to accept base-page index ranges and convert to hugepage indices internally before operating on the reservation map. This keeps the internal representation unchanged while making the API contract more uniform for callers.
Update hugetlbfs and memfd call sites to pass base-page indices, and adjust the documentation to describe the new calling convention. Add alignment warnings in hugetlb_reserve_pages() to catch invalid ranges early.
No functional changes.
--- Documentation/mm/hugetlbfs_reserv.rst | 12 +++++------ fs/hugetlbfs/inode.c | 29 ++++++++++++--------------- mm/hugetlb.c | 26 ++++++++++++++++-------- mm/memfd.c | 9 +++++---- 4 files changed, 42 insertions(+), 34 deletions(-)
Documentation/mm/hugetlbfs_reserv.rst +6 -6
@@ -112,8 +112,8 @@ flag was specified in either the shmget() or mmap() call. If NORESERVE was specified, then this routine returns immediately as no reservations are desired. -The arguments 'from' and 'to' are huge page indices into the mapping or -underlying file. For shmget(), 'from' is always 0 and 'to' corresponds to +The arguments 'from' and 'to' are base page indices into the mapping or +underlying file. For shmget(), 'from' is always 0 and 'to' corresponds to the length of the segment/mapping. For mmap(), the offset argument could be used to specify the offset into the underlying file. In such a case, the 'from' and 'to' arguments have been adjusted by this offset.
@@ -136,10 +136,10 @@ to indicate this VMA owns the reservations. The reservation map is consulted to determine how many huge page reservations are needed for the current mapping/segment. For private mappings, this is -always the value (to - from). However, for shared mappings it is possible that -some reservations may already exist within the range (to - from). See the -section :ref:`Reservation Map Modifications <resv_map_modifications>` -for details on how this is accomplished. +always the number of huge pages covered by the range [from, to). However, +for shared mappings it is possible that some reservations may already exist +within the range [from, to). See the section :ref:`Reservation Map Modifications +<resv_map_modifications>` for details on how this is accomplished. The mapping may be associated with a subpool. If so, the subpool is consulted to ensure there is sufficient space for the mapping. It is possible that the
fs/hugetlbfs/inode.c +13 -16
@@ -157,10 +157,8 @@ static int hugetlbfs_file_mmap_prepare(struct vm_area_desc *desc)if (inode->i_flags &S_PRIVATE )vma_flags_set (&vma_flags,VMA_NORESERVE_BIT ); -if (hugetlb_reserve_pages (inode, - desc->pgoff >>huge_page_order (h), - len >>huge_page_shift (h), desc, - vma_flags) <0 ) +if (hugetlb_reserve_pages (inode, desc->pgoff, len >>PAGE_SHIFT , desc, + vma_flags) <0 ) goto out; ret =0 ;
@@ -408,8 +406,8 @@ static void hugetlb_unmap_file_folio(struct hstate *h,unsigned long v_end;pgoff_t start, end; - start = index *pages_per_huge_page (h); - end = (index +1 ) *pages_per_huge_page (h); + start = index; + end = start +pages_per_huge_page (h);i_mmap_lock_write (mapping); retry:
@@ -518,6 +516,8 @@ static void remove_inode_single_folio(struct hstate *h, struct inode *inode, struct address_space *mapping, struct folio *folio, pgoff_t index, bool truncate_op) { + pgoff_t next_index; +/* * If folio is mapped, it was faulted in after being * unmapped in caller or hugetlb_vmdelete_list() skips
@@ -540,8 +540,9 @@ static void remove_inode_single_folio(struct hstate *h, struct inode *inode,VM_BUG_ON_FOLIO (folio_test_hugetlb_restore_reserve (folio), folio);hugetlb_delete_from_page_cache (folio);if (!truncate_op) { + next_index = index +pages_per_huge_page (h);if (unlikely (hugetlb_unreserve_pages (inode, index, - index +1 ,1 ))) + next_index,1 )))hugetlb_fix_reserve_counts (inode); }
@@ -575,7 +576,7 @@ static void remove_inode_hugepages(struct inode *inode, loff_t lstart,struct address_space *mapping = &inode->i_data;const pgoff_t end = lend >>PAGE_SHIFT ;struct folio_batch fbatch; -pgoff_t next, idx; +pgoff_t next;int i, freed =0 ;bool truncate_op = (lend ==LLONG_MAX );
@@ -592,9 +593,8 @@ static void remove_inode_hugepages(struct inode *inode, loff_t lstart,/* * Remove folio that was part of folio_batch. */ - idx = folio->index >>huge_page_order (h);remove_inode_single_folio (h, inode, mapping, folio, - idx, truncate_op); + folio->index, truncate_op); freed++;mutex_unlock (&hugetlb_fault_mutex_table[hash]);
@@ -604,9 +604,8 @@ static void remove_inode_hugepages(struct inode *inode, loff_t lstart, }if (truncate_op) - (void )hugetlb_unreserve_pages (inode, - lstart >>huge_page_shift (h), -LONG_MAX , freed); + (void )hugetlb_unreserve_pages (inode, lstart >>PAGE_SHIFT , +LONG_MAX , freed); }static void hugetlbfs_evict_inode (struct inode *inode)
@@ -1561,9 +1560,7 @@ struct file *hugetlb_file_setup(const char *name, size_t size, inode->i_size = size;clear_nlink (inode); -if (hugetlb_reserve_pages (inode,0 , - size >>huge_page_shift (hstate_inode (inode)),NULL , - acctflag) <0 ) +if (hugetlb_reserve_pages (inode,0 , size >>PAGE_SHIFT ,NULL , acctflag) <0 ) file =ERR_PTR (-ENOMEM );else file = alloc_file_pseudo(inode, mnt, name, O_RDWR,
mm/hugetlb.c +18 -8
@@ -6532,10 +6532,11 @@ long hugetlb_change_protection(struct vm_area_struct *vma, }/* -* Update the reservation map for the range [from, to]. +* Update the reservation map for the range [from, to) where 'from' and 'to' +* are base-page indices that are expected to be huge page aligned. * -* Returns the number of entries that would be added to the reservation map -* associated with the range [from, to]. This number is greater or equal to +* Returns the number of huge pages that would be added to the reservation map +* associated with the range [from, to). This number is greater or equal to * zero. -EINVAL or -ENOMEM is returned in case of any errors. */
@@ -6550,6 +6551,7 @@ long hugetlb_reserve_pages(struct inode *inode,struct resv_map *resv_map;struct hugetlb_cgroup *h_cg =NULL ;long gbl_reserve, regions_needed =0 ; +long from_idx, to_idx;int err;/* This should never happen */
@@ -6558,6 +6560,12 @@ long hugetlb_reserve_pages(struct inode *inode,return -EINVAL ; } +VM_WARN_ON (!IS_ALIGNED (from,1UL <<huge_page_order (h))); +VM_WARN_ON (!IS_ALIGNED (to,1UL <<huge_page_order (h))); + + from_idx = from >>huge_page_order (h); + to_idx = to >>huge_page_order (h); +/* * Only apply hugepage reservation if asked. At fault time, an * attempt will be made for VM_NORESERVE to allocate a page
@@ -6580,7 +6588,7 @@ long hugetlb_reserve_pages(struct inode *inode,*/ resv_map =inode_resv_map (inode); - chg =region_chg (resv_map, from, to, ®ions_needed); + chg =region_chg (resv_map, from_idx, to_idx, ®ions_needed); }else {/* Private mapping. */ resv_map =resv_map_alloc ();
@@ -6589,7 +6597,7 @@ long hugetlb_reserve_pages(struct inode *inode, goto out_err; } - chg = to - from; + chg = to_idx - from_idx;set_vma_desc_resv_map (desc, resv_map);set_vma_desc_resv_flags (desc,HPAGE_RESV_OWNER );
@@ -6644,7 +6652,7 @@ long hugetlb_reserve_pages(struct inode *inode,* else has to be done for private mappings here */ if (!desc ||vma_desc_test (desc,VMA_MAYSHARE_BIT )) { - add =region_add (resv_map, from, to, regions_needed, h, h_cg); + add =region_add (resv_map, from_idx, to_idx, regions_needed, h, h_cg);if (unlikely (add <0 )) {hugetlb_acct_memory (h, -gbl_reserve);
@@ -6712,7 +6720,7 @@ long hugetlb_reserve_pages(struct inode *inode,* region_add failed or didn't run. */ if (chg >=0 && add <0 ) -region_abort (resv_map, from, to, regions_needed); +region_abort (resv_map, from_idx, to_idx, regions_needed);if (desc &&is_vma_desc_resv_set (desc,HPAGE_RESV_OWNER )) {kref_put (&resv_map->refs, resv_map_release);set_vma_desc_resv_map (desc,NULL );
@@ -6728,13 +6736,15 @@ long hugetlb_unreserve_pages(struct inode *inode, long start, long end,long chg =0 ;struct hugepage_subpool *spool =subpool_inode (inode);long gbl_reserve; +long start_idx = start >>huge_page_order (h); +long end_idx = end >>huge_page_order (h);/* * Since this routine can be called in the evict inode path for all * hugetlbfs inodes, resv_map could be NULL. */ if (resv_map) { - chg =region_del (resv_map, start, end); + chg =region_del (resv_map, start_idx, end_idx);/* * region_del() can fail in the rare case where a region * must be split and another region descriptor can not be
mm/memfd.c +5 -4
@@ -80,14 +80,15 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t index)struct inode *inode =file_inode (memfd);struct hstate *h =hstate_file (memfd);long nr_resv; -pgoff_t idx; +pgoff_t next_index;int err = -ENOMEM ; gfp_mask =htlb_alloc_mask (h); gfp_mask &= ~(__GFP_HIGHMEM | __GFP_MOVABLE); - idx = index >>huge_page_order (h); + next_index = index +pages_per_huge_page (h); - nr_resv =hugetlb_reserve_pages (inode, idx, idx +1 ,NULL ,EMPTY_VMA_FLAGS ); + nr_resv =hugetlb_reserve_pages (inode, index, next_index,NULL , +EMPTY_VMA_FLAGS );if (nr_resv <0 )return ERR_PTR (nr_resv);
@@ -137,7 +138,7 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t index) } err_unresv:if (nr_resv >0 ) -hugetlb_unreserve_pages (inode, idx, idx +1 ,0 ); +hugetlb_unreserve_pages (inode, index, next_index,0 );return ERR_PTR (err); }#endif
Signature · 1 line
2.43.5
Oscar Salvador
maintainerRe: [PATCH 6/6] hugetlb: pass hugetlb reservation ranges in base-page indices
Quoted 58 lines · Jane Chu
hugetlb_reserve_pages()consume indices in hugepage granularity although some callers naturally compute offsets inPAGE_SIZEunits.Teach the reservation helpers to accept base-page index ranges and convert to hugepage indices internally before operating on the reservation map. This keeps the internal representation unchanged while making the API contract more uniform for callers.
Update hugetlbfs and memfd call sites to pass base-page indices, and adjust the documentation to describe the new calling convention. Add alignment warnings in
hugetlb_reserve_pages()to catch invalid ranges early.No functional changes.
Signed-off-by Jane Chu <[email protected]>--- Documentation/mm/hugetlbfs_reserv.rst | 12 +++++------ fs/hugetlbfs/inode.c | 29 ++++++++++++--------------- mm/hugetlb.c | 26 ++++++++++++++++-------- mm/memfd.c | 9 +++++---- 4 files changed, 42 insertions(+), 34 deletions(-)Documentation/mm/hugetlbfs_reserv.rst +6
-6@@ -112,8 +112,8 @@ flag was specified in either the shmget() or mmap() call. If NORESERVE was specified, then this routine returns immediately as no reservations are desired. -The arguments 'from' and 'to' are huge page indices into the mapping or -underlying file. For shmget(), 'from' is always 0 and 'to' corresponds to +The arguments 'from' and 'to' are base page indices into the mapping or +underlying file. For shmget(), 'from' is always 0 and 'to' corresponds to the length of the segment/mapping. For mmap(), the offset argument could be used to specify the offset into the underlying file. In such a case, the 'from' and 'to' arguments have been adjusted by this offset.@@ -136,10 +136,10 @@ to indicate this VMA owns the reservations. The reservation map is consulted to determine how many huge page reservations are needed for the current mapping/segment. For private mappings, this is -always the value (to - from). However, for shared mappings it is possible that -some reservations may already exist within the range (to - from). See the -section :ref:`Reservation Map Modifications <resv_map_modifications>` -for details on how this is accomplished. +always the number of huge pages covered by the range [from, to). However, +for shared mappings it is possible that some reservations may already exist +within the range [from, to). See the section :ref:`Reservation Map Modifications +<resv_map_modifications>` for details on how this is accomplished. The mapping may be associated with a subpool. If so, the subpool is consulted to ensure there is sufficient space for the mapping. It is possible that thefs/hugetlbfs/inode.c +0
-0
fs/hugetlbfs/inode.c static int hugetlbfs_file_mmap_prepare(struct vm_area_desc *desc) · line 161
Quoted 9 lines
fs/hugetlbfs/inode.c +2
-4@@ -157,10 +157,8 @@ static int hugetlbfs_file_mmap_prepare(struct vm_area_desc *desc)if (inode->i_flags &S_PRIVATE )vma_flags_set (&vma_flags,VMA_NORESERVE_BIT ); -if (hugetlb_reserve_pages (inode, - desc->pgoff >>huge_page_order (h), - len >>huge_page_shift (h), desc, - vma_flags) <0 ) +if (hugetlb_reserve_pages (inode, desc->pgoff, len >>PAGE_SHIFT , desc, + vma_flags) <0 )
Ok, this is something that I have been thinking every time I looked into hugetlb reserve code, but I think we should be really starting to put some meaningful names for from and to, and pass that to hugetlb_reserve_pages.
Because "desc->pgoff" and "len >> PAGE_SHIFT", meh, and it is not that many places we need to touch, but we might want in clarity. The same goes for hugetlb_unreserve_pages() of course.
Quoted 6 lines
mm/hugetlb.c +0
-0@@ -6532,10 +6532,11 @@ long hugetlb_change_protection(struct vm_area_struct *vma, }
[...]
mm/hugetlb.c long hugetlb_reserve_pages(struct inode *inode, · line 6564
Quoted 6 lines
mm/hugetlb.c +2
-0@@ -6558,6 +6560,12 @@ long hugetlb_reserve_pages(struct inode *inode,return -EINVAL ; } +VM_WARN_ON (!IS_ALIGNED (from,1UL <<huge_page_order (h))); +VM_WARN_ON (!IS_ALIGNED (to,1UL <<huge_page_order (h)));
If we want to scream if someone passes us unaligned indices, we might want to do the same in hugetlb_unreserve_pages() ?
Quoted 6 lines
mm/memfd.c +0
-0@@ -80,14 +80,15 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t index)struct inode *inode =file_inode (memfd);
mm/memfd.c struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t index) · line 88
Quoted 10 lines
mm/memfd.c +2
-2struct hstate *h =hstate_file (memfd);long nr_resv; -pgoff_t idx; +pgoff_t next_index;int err = -ENOMEM ; gfp_mask =htlb_alloc_mask (h); gfp_mask &= ~(__GFP_HIGHMEM | __GFP_MOVABLE); - idx = index >>huge_page_order (h); + next_index = index +pages_per_huge_page (h);
Trailing white space.
Signature · 2 lines
Oscar Salvador SUSE Labs
Re: [PATCH 6/6] hugetlb: pass hugetlb reservation ranges in base-page indices
Quoted 62 lines · Oscar Salvador
Quoted 61 lines · Jane Chu
hugetlb_reserve_pages()consume indices in hugepage granularity although some callers naturally compute offsets inPAGE_SIZEunits.Teach the reservation helpers to accept base-page index ranges and convert to hugepage indices internally before operating on the reservation map. This keeps the internal representation unchanged while making the API contract more uniform for callers.
Update hugetlbfs and memfd call sites to pass base-page indices, and adjust the documentation to describe the new calling convention. Add alignment warnings in
hugetlb_reserve_pages()to catch invalid ranges early.No functional changes.
Signed-off-by Jane Chu <[email protected]>--- Documentation/mm/hugetlbfs_reserv.rst | 12 +++++------ fs/hugetlbfs/inode.c | 29 ++++++++++++--------------- mm/hugetlb.c | 26 ++++++++++++++++-------- mm/memfd.c | 9 +++++---- 4 files changed, 42 insertions(+), 34 deletions(-)Documentation/mm/hugetlbfs_reserv.rst +6
-6@@ -112,8 +112,8 @@ flag was specified in either the shmget() or mmap() call. If NORESERVE was specified, then this routine returns immediately as no reservations are desired. -The arguments 'from' and 'to' are huge page indices into the mapping or -underlying file. For shmget(), 'from' is always 0 and 'to' corresponds to +The arguments 'from' and 'to' are base page indices into the mapping or +underlying file. For shmget(), 'from' is always 0 and 'to' corresponds to the length of the segment/mapping. For mmap(), the offset argument could be used to specify the offset into the underlying file. In such a case, the 'from' and 'to' arguments have been adjusted by this offset.@@ -136,10 +136,10 @@ to indicate this VMA owns the reservations. The reservation map is consulted to determine how many huge page reservations are needed for the current mapping/segment. For private mappings, this is -always the value (to - from). However, for shared mappings it is possible that -some reservations may already exist within the range (to - from). See the -section :ref:`Reservation Map Modifications <resv_map_modifications>` -for details on how this is accomplished. +always the number of huge pages covered by the range [from, to). However, +for shared mappings it is possible that some reservations may already exist +within the range [from, to). See the section :ref:`Reservation Map Modifications +<resv_map_modifications>` for details on how this is accomplished. The mapping may be associated with a subpool. If so, the subpool is consulted to ensure there is sufficient space for the mapping. It is possible that thefs/hugetlbfs/inode.c +0
-0@@ -157,10 +157,8 @@ static int hugetlbfs_file_mmap_prepare(struct vm_area_desc *desc)if (inode->i_flags &S_PRIVATE )vma_flags_set (&vma_flags,VMA_NORESERVE_BIT );
Quoted 14 lines
fs/hugetlbfs/inode.c static int hugetlbfs_file_mmap_prepare(struct vm_area_desc *desc) · line 161
Quoted 6 lines
fs/hugetlbfs/inode.c +2
-4-if (hugetlb_reserve_pages (inode, - desc->pgoff >>huge_page_order (h), - len >>huge_page_shift (h), desc, - vma_flags) <0 ) +if (hugetlb_reserve_pages (inode, desc->pgoff, len >>PAGE_SHIFT , desc, + vma_flags) <0 )Ok, this is something that I have been thinking every time I looked into hugetlb reserve code, but I think we should be really starting to put some meaningful names for from and to, and pass that to
hugetlb_reserve_pages.
Because "desc->pgoff" and "len >>PAGE_SHIFT", meh, and it is not that many places we need to touch, but we might want in clarity. The same goes forhugetlb_unreserve_pages()of course.
indeed, will try to work on that in v2.
Quoted 12 lines
Quoted 7 lines
mm/hugetlb.c +0
-0@@ -6532,10 +6532,11 @@ long hugetlb_change_protection(struct vm_area_struct *vma, }[...]
Quoted 4 lines
mm/hugetlb.c +0
-0@@ -6558,6 +6560,12 @@ long hugetlb_reserve_pages(struct inode *inode,return -EINVAL ; }
Quoted 5 lines
mm/hugetlb.c long hugetlb_reserve_pages(struct inode *inode, · line 6564
Quoted 2 lines
mm/hugetlb.c +2
-0+VM_WARN_ON (!IS_ALIGNED (from,1UL <<huge_page_order (h))); +VM_WARN_ON (!IS_ALIGNED (to,1UL <<huge_page_order (h)));If we want to scream if someone passes us unaligned indices, we might want to do the same in
hugetlb_unreserve_pages()?
Sure.
Quoted 13 lines
Quoted 13 lines
mm/memfd.c +1
-1@@ -80,14 +80,15 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t index)struct inode *inode =file_inode (memfd);struct hstate *h =hstate_file (memfd);long nr_resv; -pgoff_t idx; +pgoff_t next_index;int err = -ENOMEM ;
Quoted 6 lines
mm/memfd.c struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t index) · line 88
Quoted 4 lines
mm/memfd.c +1
-1gfp_mask =htlb_alloc_mask (h); gfp_mask &= ~(__GFP_HIGHMEM | __GFP_MOVABLE); - idx = index >>huge_page_order (h); + next_index = index +pages_per_huge_page (h);Trailing white space.
My bad, should have checked.
Thanks!
-jane
Quoted 2 lines
syzbot ci has tested the following series
syzbot ci has tested the following series
[v1] hugetlb: normalize exported interfaces to use base-page indices
https://lore.kernel.org/all/[email protected]
* [PATCH 1/6] hugetlb: open-code hugetlb folio lookup index conversion
* [PATCH 2/6] hugetlb: remove the hugetlb_linear_page_index() helper
* [PATCH 3/6] hugetlb: make hugetlb_fault_mutex_hash() take PAGE_SIZE index
* [PATCH 4/6] hugetlb: drop vma_hugecache_offset() in favor of linear_page_index()
* [PATCH 5/6] hugetlb: make hugetlb_add_to_page_cache() use PAGE_SIZE-based index
* [PATCH 6/6] hugetlb: pass hugetlb reservation ranges in base-page indices
and found the following issue:
WARNING: bad unlock balance in hugetlb_no_page
Full report is available here:
https://ci.syzbot.org/series/95c5ba82-0135-4026-b7c7-b0819e1ca4d6
***
WARNING: bad unlock balance in hugetlb_no_page
tree: mm-new URL: https://kernel.googlesource.com/pub/scm/linux/kernel/git/akpm/mm.git base: 06a6cfb92448a97ef429a7fbd395a20a9d388acc arch: amd64 compiler: Debian clang version 21.1.8 (++20251221033036+2078da43e25a-1~exp1~20251221153213.50), Debian LLD 21.1.8 config: https://ci.syzbot.org/builds/cefe8576-3c99-42d3-9b51-1e70d62a64a7/config syz repro: https://ci.syzbot.org/findings/3a14cc12-14a8-4fac-9614-ae7ae2555e58/syz_repro
=====================================
WARNING: bad unlock balance detected!
syzkaller #0 Not tainted
-------------------------------------
syz.0.17/5971 is trying to release lock (&hugetlb_fault_mutex_table[i]) at:
[<ffffffff8229b876>] hugetlb_handle_userfault mm/hugetlb.c:5686 [inline] [<ffffffff8229b876>] hugetlb_no_page+0x1986/0x1da0 mm/hugetlb.c:5770
but there are no more locks to release!
other info that might help us debug this:
2 locks held by syz.0.17/5971:
#0: ffff88816b85fb88 (vm_lock){++++}-{0:0}, at: lock_vma_under_rcu+0x1d1/0x500 mm/mmap_lock.c:310 #1: ffff88816079e338 (&hugetlb_fault_mutex_table[i]){+.+.}-{4:4}, at: hugetlb_fault+0x317/0x1440 mm/hugetlb.c:5991
stack backtrace: CPU: 0 UID: 0 PID: 5971 Comm: syz.0.17 Not tainted syzkaller #0 PREEMPT(full) Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0xe8/0x150 lib/dump_stack.c:120 print_unlock_imbalance_bug+0xdc/0xf0 kernel/locking/lockdep.c:5298 __lock_release kernel/locking/lockdep.c:5537 [inline] lock_release+0x248/0x3d0 kernel/locking/lockdep.c:5889 __mutex_unlock_slowpath+0xd3/0x7d0 kernel/locking/mutex.c:938 hugetlb_handle_userfault mm/hugetlb.c:5686 [inline] hugetlb_no_page+0x1986/0x1da0 mm/hugetlb.c:5770 hugetlb_fault+0x67f/0x1440 mm/hugetlb.c:-1 handle_mm_fault+0x2007/0x3170 mm/memory.c:6716 do_user_addr_fault+0xa73/0x1340 arch/x86/mm/fault.c:1334 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 RIP: 0033:0x7fa742251964 Code: 41 89 00 31 c0 c3 b9 40 00 00 00 bf 40 00 00 00 eb bc 0f 1f 40 00 48 89 7c 24 f8 48 89 74 24 f0 48 8b 7c 24 f8 4c 8b 44 24 f0 <8b> 4f 50 8b 47 58 4c 01 c1 41 8b 34 00 8b 11 21 d6 89 f0 8d 72 01 RSP: 002b:00007fa7431fd018 EFLAGS: 00010212 RAX: 00007fa742251950 RBX: 00007fa742615fa0 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000200000400000 RBP: 00007fa742432c91 R08: 0000000000000000 R09: 0000000000000000 R10: 0000200000400000 R11: 0000000000000000 R12: 0000000000000000 R13: 00007fa742616038 R14: 00007fa742615fa0 R15: 00007ffe952c6908 </TASK>
***
If these findings have caused you to resend the series or submit a separate fix, please add the following tag to your commit message:
---
This report is generated by a bot. It may contain errors. syzbot ci engineers can be reached at [email protected].
To test a patch for this bug, please reply with #syz test (should be on a separate line).
The patch should be attached to the email.
Note: arguments like custom git repos and branches are not supported.
Quoted 11 lines · syzbot ci
syzbot ci has tested the following series
[v1] hugetlb: normalize exported interfaces to use base-page indices
https://lore.kernel.org/all/[email protected]
* [PATCH 1/6] hugetlb: open-code hugetlb folio lookup index conversion
* [PATCH 2/6] hugetlb: remove thehugetlb_linear_page_index()helper
* [PATCH 3/6] hugetlb: makehugetlb_fault_mutex_hash()takePAGE_SIZEindex
* [PATCH 4/6] hugetlb: dropvma_hugecache_offset()in favor oflinear_page_index()
* [PATCH 5/6] hugetlb: makehugetlb_add_to_page_cache()use PAGE_SIZE-based index
* [PATCH 6/6] hugetlb: pass hugetlb reservation ranges in base-page indices
Quoted 2 lines
and found the following issue:
WARNING: bad unlock balance inhugetlb_no_page
Thanks for catching the bug. I was able to reproduce by turning on a few configs. It appears below change fixed the issue, please confirm.
$ diff -c mm/hugetlb.c-BAD mm/hugetlb.c
*** mm/hugetlb.c-BAD 2026-04-10 13:36:52.417044993 -0600
--- mm/hugetlb.c 2026-04-10 14:33:31.637033381 -0600
***************
*** 5659,5665 ****
u32 hash;
pgoff_t index;! index = linear_page_index((const struct vm_area_struct *)vmf, vmf->address);
hash = hugetlb_fault_mutex_hash(mapping, index);
/*
--- 5659,5665 ----
u32 hash;
pgoff_t index;! index = linear_page_index(vmf->vma, vmf->address);
hash = hugetlb_fault_mutex_hash(mapping, index);thanks,
-jane
Quoted 74 lines
Full report is available here:
https://ci.syzbot.org/series/95c5ba82-0135-4026-b7c7-b0819e1ca4d6***
WARNING: bad unlock balance in
hugetlb_no_pagetree: mm-new URL: https://kernel.googlesource.com/pub/scm/linux/kernel/git/akpm/mm.git base: 06a6cfb92448a97ef429a7fbd395a20a9d388acc arch: amd64 compiler: Debian clang version 21.1.8 (++20251221033036+2078da43e25a-1~exp1~20251221153213.50), Debian LLD 21.1.8 config: https://ci.syzbot.org/builds/cefe8576-3c99-42d3-9b51-1e70d62a64a7/config syz repro: https://ci.syzbot.org/findings/3a14cc12-14a8-4fac-9614-ae7ae2555e58/syz_repro=====================================
WARNING: bad unlock balance detected!
syzkaller #0 Not tainted
-------------------------------------
syz.0.17/5971 is trying to release lock (&hugetlb_fault_mutex_table[i]) at:[<ffffffff8229b876>] hugetlb_handle_userfault mm/hugetlb.c:5686 [inline] [<ffffffff8229b876>] hugetlb_no_page+0x1986/0x1da0 mm/hugetlb.c:5770but there are no more locks to release!
other info that might help us debug this:
2 locks held by syz.0.17/5971:
#0:ffff88816b85fb88(vm_lock){++++}-{0:0}, at:lock_vma_under_rcu+0x1d1/0x500mm/mmap_lock.c:310#1:ffff88816079e338(&hugetlb_fault_mutex_table[i]){+.+.}-{4:4}, at:hugetlb_fault+0x317/0x1440mm/hugetlb.c:5991stack backtrace: CPU: 0 UID: 0 PID: 5971 Comm: syz.0.17 Not tainted syzkaller #0 PREEMPT(full) Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0xe8/0x150 lib/dump_stack.c:120 print_unlock_imbalance_bug+0xdc/0xf0 kernel/locking/lockdep.c:5298 __lock_release kernel/locking/lockdep.c:5537 [inline] lock_release+0x248/0x3d0 kernel/locking/lockdep.c:5889 __mutex_unlock_slowpath+0xd3/0x7d0 kernel/locking/mutex.c:938 hugetlb_handle_userfault mm/hugetlb.c:5686 [inline] hugetlb_no_page+0x1986/0x1da0 mm/hugetlb.c:5770 hugetlb_fault+0x67f/0x1440 mm/hugetlb.c:-1 handle_mm_fault+0x2007/0x3170 mm/memory.c:6716 do_user_addr_fault+0xa73/0x1340 arch/x86/mm/fault.c:1334 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 RIP: 0033:0x7fa742251964 Code: 41 89 00 31 c0 c3 b9 40 00 00 00 bf 40 00 00 00 eb bc 0f 1f 40 00 48 89 7c 24 f8 48 89 74 24 f0 48 8b 7c 24 f8 4c 8b 44 24 f0 <8b> 4f 50 8b 47 58 4c 01 c1 41 8b 34 00 8b 11 21 d6 89 f0 8d 72 01 RSP: 002b:00007fa7431fd018 EFLAGS: 00010212 RAX: 00007fa742251950 RBX: 00007fa742615fa0 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000200000400000 RBP: 00007fa742432c91 R08: 0000000000000000 R09: 0000000000000000 R10: 0000200000400000 R11: 0000000000000000 R12: 0000000000000000 R13: 00007fa742616038 R14: 00007fa742615fa0 R15: 00007ffe952c6908 </TASK>***
If these findings have caused you to resend the series or submit a separate fix, please add the following tag to your commit message:
Tested-by [email protected]---
This report is generated by a bot. It may contain errors. syzbot ci engineers can be reached at [email protected].To test a patch for this bug, please reply with
#syz test(should be on a separate line).The patch should be attached to the email.
Note: arguments like custom git repos and branches are not supported.
Oscar Salvador
maintainerQuoted 10 lines · Jane Chu
This series stems from a discussion with David. [1]
The series makes a small cleanup to a few hugetlb interfaces used outside the subsystem by standardizing them on base-page indices. Hopefully this makes the interface semantics a bit more coherent with the rest of mm, while the internal hugetlb code continue to use hugepage indices where that remains the more natural fit.It is based off mm-stable, 3/30/2026,
b2c31180b9d6.
It seems you got some trailing spaces issues:
Applying: hugetlb: open-code hugetlb folio lookup index conversion .git/rebase-apply/patch:64: trailing whitespace.
pgoff_t index = start >> PAGE_SHIFT;
Applying: hugetlb: make hugetlb_fault_mutex_hash() take PAGE_SIZE index .git/rebase-apply/patch:161: trailing whitespace.
key[1] = index >> huge_page_order(hstate_inode(mapping->host));
Applying: hugetlb: drop vma_hugecache_offset() in favor of linear_page_index() .git/rebase-apply/patch:44: trailing whitespace.
start = linear_page_index(vma, vma->vm_start); .git/rebase-apply/patch:46: trailing whitespace. end = linear_page_index(vma, vma->vm_end);
Applying: hugetlb: pass hugetlb reservation ranges in base-page indices .git/rebase-apply/patch:237: trailing whitespace.
next_index = index + pages_per_huge_page(h)
Signature · 2 lines
Oscar Salvador SUSE Labs
Quoted 24 lines · Oscar Salvador
Quoted 10 lines · Jane Chu
This series stems from a discussion with David. [1]
The series makes a small cleanup to a few hugetlb interfaces used outside the subsystem by standardizing them on base-page indices. Hopefully this makes the interface semantics a bit more coherent with the rest of mm, while the internal hugetlb code continue to use hugepage indices where that remains the more natural fit.It is based off mm-stable, 3/30/2026,
b2c31180b9d6.It seems you got some trailing spaces issues:
Applying: hugetlb: open-code hugetlb folio lookup index conversion .git/rebase-apply/patch:64: trailing whitespace.
pgoff_t index = start >> PAGE_SHIFT;Applying: hugetlb: make
hugetlb_fault_mutex_hash()takePAGE_SIZEindex .git/rebase-apply/patch:161: trailing whitespace.key[1] = index >> huge_page_order(hstate_inode(mapping->host));Applying: hugetlb: drop
vma_hugecache_offset()in favor oflinear_page_index().git/rebase-apply/patch:44: trailing whitespace.
Quoted 9 lines
start = linear_page_index(vma, vma->vm_start); .git/rebase-apply/patch:46: trailing whitespace. end = linear_page_index(vma, vma->vm_end);Applying: hugetlb: pass hugetlb reservation ranges in base-page indices .git/rebase-apply/patch:237: trailing whitespace.
next_index= index +pages_per_huge_page(h)
Sorry, will remove them in v2.
thanks!
-jane