Split huge pages to any lower order pages and selftests.
Subsystems (10)
cover Split huge pages to any lower order pages and selftests.
From: Zi Yan <[email protected]>
Hi all,
With Matthew's THP in pagecache patches[1], we will be able to handle any size pagecache THPs, but currently split_huge_page can only split a THP to order-0 pages. This can easily erase the benefit of having pagecache THPs, when operations like truncate might want to keep pages larger than order-0. In response, here is the patches to add support for splitting a THP to any lower order pages. In addition, this patchset prepares for my PUD THP patchset[2], since splitting a PUD THP to multiple PMD THPs can be handled by split_huge_page_to_list_to_order function added by this patchset, which reduces a lot of redundant code without just replicating split_huge_page for PUD THP.
To help the tests of splitting huge pages, I added a new debugfs interface at <debugfs>/split_huge_pages_in_range_pid, so developers can split THPs in a given range from a process with the given pid by writing "<pid>,<vaddr_start>,<vaddr_end>,<to_order>" to the interface. I also added a new test program to test 1) splitting PMD THPs, 2) splitting PTE-mapped THPs,
3) splitting pagecache THPs to any lower order, 4) truncating a pagecache THP to a page with a lower order, and 5) punching holes in a pagecache THP to cause splitting THPs to lower order THPs.
The patchset is on top of Matthew's pagecache/next tree[3].
* Patch 1 is cherry-picked from Matthew's recent xarray fix [4] just to make sure Patch 3 to 7 can run without problem. I let Matthew decide how it should get picked up.
* Patch 2 is self-contained and can be merged if it looks OK.
Comments and/or suggestions are welcome.
ChangeLog
===
From RFC:
1. Fixed debugfs to handle splitting PTE-mapped THPs properly and added stats for split THPs.
2. Added a new test case for splitting PTE-mapped THPs. Each of the four PTEs points to a different subpage from four THPs and used kpageflags to check whether a PTE points to a THP or not (AnonHugePages from smap does not show PTE-mapped THPs).
3. mem_cgroup_split_huge_fixup() takes order instead of nr.
4. split_page_owner takes old_order and new_order instead of nr and new_order.
5. Corrected __split_page_owner declaration and fixed its implementation when splitting a THP to a new order.
6. Renamed left to remaining in truncate_inode_partial_page().
7. Use VM_BUG_ON instead of WARN_ONCE when splitting a THP to the unsupported order-0 and splitting anonymous THPs to non-zero orders.
8. Added punching holes in a file as a new pagecache THP split test case, which uncovered an xarray bug.
[1] https://lore.kernel.org/linux-mm/[email protected]/
[2] https://lore.kernel.org/linux-mm/[email protected]/
[3] https://git.infradead.org/users/willy/pagecache.git/shortlog/refs/heads/next
[4] https://git.infradead.org/users/willy/xarray.git
Matthew Wilcox (Oracle) (1):
XArray: Fix splitting to non-zero orders
Zi Yan (6):
mm: huge_memory: add new debugfs interface to trigger split huge page
on any page range.
mm: memcg: make memcg huge page split support any order split.
mm: page_owner: add support for splitting to any order in split
page_owner.
mm: thp: split huge page to any lower order pages.
mm: truncate: split thp to a non-zero order if possible.
mm: huge_memory: enable debugfs to split huge pages to any order.
include/linux/huge_mm.h | 8 + include/linux/memcontrol.h | 5 +- include/linux/page_owner.h | 10 +- lib/test_xarray.c | 26 +- lib/xarray.c | 4 +- mm/huge_memory.c | 219 ++++++-- mm/internal.h | 1 + mm/memcontrol.c | 6 +- mm/migrate.c | 2 +- mm/page_alloc.c | 2 +- mm/page_owner.c | 13 +- mm/swap.c | 1 - mm/truncate.c | 29 +- tools/testing/selftests/vm/.gitignore | 1 + tools/testing/selftests/vm/Makefile | 1 + .../selftests/vm/split_huge_page_test.c | 479 ++++++++++++++++++ 16 files changed, 742 insertions(+), 65 deletions(-) create mode 100644 tools/testing/selftests/vm/split_huge_page_test.c
--
2.28.0
PATCH 1/7 XArray: Fix splitting to non-zero orders
From: "Matthew Wilcox (Oracle)" <[email protected]>
Splitting an order-4 entry into order-2 entries would leave the array containing pointers to 000040008000c000 instead of 000044448888cccc. This is a one-character fix, but enhance the test suite to check this case.
--- lib/test_xarray.c | 26 ++++++++++++++------------ lib/xarray.c | 4 ++-- 2 files changed, 16 insertions(+), 14 deletions(-)
lib/test_xarray.c +14 -12
@@ -1530,24 +1530,24 @@ static noinline void check_store_range(struct xarray *xa)#ifdef CONFIG_XARRAY_MULTI static void check_split_1 (struct xarray *xa,unsigned long index, -unsigned int order) +unsigned int order,unsigned int new_order) { -XA_STATE (xas, xa, index); -void *entry; -unsigned int i =0 ; +XA_STATE_ORDER (xas, xa, index, new_order); +unsigned int i;xa_store_order (xa, index, order, xa,GFP_KERNEL );xas_split_alloc (&xas, xa, order,GFP_KERNEL );xas_lock (&xas);xas_split (&xas, xa, order); +for (i =0 ; i < (1 << order); i += (1 << new_order)) +__xa_store (xa, index + i,xa_mk_index (index + i),0 );xas_unlock (&xas); -xa_for_each (xa, index, entry) { -XA_BUG_ON (xa, entry != xa); - i++; +for (i =0 ; i < (1 << order); i++) { +unsigned int val = index + (i & ~((1 << new_order) -1 )); +XA_BUG_ON (xa,xa_load (xa, index + i) !=xa_mk_index (val)); } -XA_BUG_ON (xa, i !=1 << order);xa_set_mark (xa, index,XA_MARK_0 );XA_BUG_ON (xa, !xa_get_mark (xa, index,XA_MARK_0 ));
@@ -1557,14 +1557,16 @@ static void check_split_1(struct xarray *xa, unsigned long index, static noinline void check_split(struct xarray *xa) { -unsigned int order; +unsigned int order, new_order;XA_BUG_ON (xa, !xa_empty (xa));for (order =1 ; order <2 *XA_CHUNK_SHIFT ; order++) { -check_split_1 (xa,0 , order); -check_split_1 (xa,1UL << order, order); -check_split_1 (xa,3UL << order, order); +for (new_order =0 ; new_order < order; new_order++) { +check_split_1 (xa,0 , order, new_order); +check_split_1 (xa,1UL << order, order, new_order); +check_split_1 (xa,3UL << order, order, new_order); + } } }#else
lib/xarray.c +2 -2
@@ -1012,7 +1012,7 @@ void xas_split_alloc(struct xa_state *xas, void *entry, unsigned int order, do { unsigned int i; - void *sibling; + void *sibling = NULL; struct xa_node *node; node = kmem_cache_alloc(radix_tree_node_cachep, gfp);
@@ -1022,7 +1022,7 @@ void xas_split_alloc(struct xa_state *xas, void *entry, unsigned int order,for (i =0 ; i <XA_CHUNK_SIZE ; i++) {if ((i & mask) ==0 ) {RCU_INIT_POINTER (node->slots[i], entry); - sibling =xa_mk_sibling (0 ); + sibling =xa_mk_sibling (i); }else {RCU_INIT_POINTER (node->slots[i], sibling); }
Signature · 1 line
2.28.0
PATCH 2/7 mm: huge_memory: add new debugfs interface to trigger split huge page on any page range.
In reply to Zi YanFrom: Zi Yan <[email protected]>
Huge pages in the process with the given pid and virtual address range are split. It is used to test split huge page function. In addition, a testing program is added to tools/testing/selftests/vm to utilize the interface by splitting PMD THPs and PTE-mapped THPs.
--- mm/huge_memory.c | 98 ++++++ mm/internal.h | 1 + mm/migrate.c | 2 +- tools/testing/selftests/vm/.gitignore | 1 + tools/testing/selftests/vm/Makefile | 1 + .../selftests/vm/split_huge_page_test.c | 313 ++++++++++++++++++ 6 files changed, 415 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/vm/split_huge_page_test.c
mm/huge_memory.c +98 -0
@@ -7,6 +7,7 @@#include <linux/mm.h> #include <linux/sched.h> +#include <linux/sched/mm.h> #include <linux/sched/coredump.h> #include <linux/sched/numa_balancing.h> #include <linux/highmem.h>
@@ -2934,10 +2935,107 @@ static int split_huge_pages_set(void *data, u64 val)DEFINE_DEBUGFS_ATTRIBUTE (split_huge_pages_fops,NULL , split_huge_pages_set,"%llu\n" ); +static ssize_t split_huge_pages_in_range_pid_write (struct file *file, +const char __user *buf,size_t count,loff_t *ppops) +{ +static DEFINE_MUTEX (mutex); +ssize_t ret; +char input_buf[80 ];/* hold pid, start_vaddr, end_vaddr */ +int pid; +unsigned long vaddr_start, vaddr_end, addr; +nodemask_t task_nodes; +struct mm_struct *mm; +unsigned long total =0 , split =0 ; + + ret =mutex_lock_interruptible (&mutex); +if (ret) +return ret; + + ret = -EFAULT ; + +memset (input_buf,0 ,80 ); +if (copy_from_user (input_buf, buf,min_t (size_t, count,80 ))) + goto out; + + input_buf[79 ] ='\0' ; + ret =sscanf (input_buf,"%d,0x%lx,0x%lx" , &pid, &vaddr_start, &vaddr_end); +if (ret !=3 ) { + ret = -EINVAL ; + goto out; + } + vaddr_start &=PAGE_MASK ; + vaddr_end &=PAGE_MASK ; + + ret =strlen (input_buf); +pr_debug ("split huge pages in pid: %d, vaddr: [%lx - %lx]\n" , + pid, vaddr_start, vaddr_end); + + mm =find_mm_struct (pid, &task_nodes); +if (IS_ERR (mm)) { + ret = -EINVAL ; + goto out; + } + +mmap_read_lock (mm); +/* +* always increase addr by PAGE_SIZE, since we could have a PTE page +* table filled with PTE-mapped THPs, each of which is distinct. +*/ +for (addr = vaddr_start; addr < vaddr_end; addr +=PAGE_SIZE ) { +struct vm_area_struct *vma =find_vma (mm, addr); +unsigned int follflags; +struct page *page; + +if (!vma || addr < vma->vm_start || !vma_migratable (vma)) +break ; + +/* FOLL_DUMP to ignore special (like zero) pages */ + follflags =FOLL_GET |FOLL_DUMP ; + page =follow_page (vma, addr, follflags); + +if (IS_ERR (page)) +break ; +if (!page) +break ; + +if (!is_transparent_hugepage (page)) +continue ; + + total++; +if (!can_split_huge_page (compound_head (page),NULL )) +continue ; + +if (!trylock_page (page)) +continue ; + +if (!split_huge_page (page)) + split++; + +unlock_page (page); +put_page (page); + } +mmap_read_unlock (mm); +mmput (mm); + +pr_debug ("%lu of %lu THP split\n" , split, total); +out: +mutex_unlock (&mutex); +return ret; + +} + +static const struct file_operations split_huge_pages_in_range_pid_fops = { + .owner =THIS_MODULE , + .write = split_huge_pages_in_range_pid_write, + .llseek = no_llseek, +}; +static int __init split_huge_pages_debugfs (void ) {debugfs_create_file ("split_huge_pages" ,0200 ,NULL ,NULL , &split_huge_pages_fops); +debugfs_create_file ("split_huge_pages_in_range_pid" ,0200 ,NULL ,NULL , + &split_huge_pages_in_range_pid_fops);return 0 ; }late_initcall (split_huge_pages_debugfs);
mm/internal.h +1 -0
@@ -627,4 +627,5 @@ struct migration_target_control {bool truncate_inode_partial_page (struct page *page,loff_t start,loff_t end);void page_cache_free_page (struct address_space *mapping,struct page *page); +struct mm_struct *find_mm_struct (pid_t pid,nodemask_t *mem_nodes);#endif /* __MM_INTERNAL_H */
mm/migrate.c +1 -1
@@ -1853,7 +1853,7 @@ static int do_pages_stat(struct mm_struct *mm, unsigned long nr_pages,return nr_pages ? -EFAULT :0 ; } -static struct mm_struct *find_mm_struct (pid_t pid,nodemask_t *mem_nodes) +struct mm_struct *find_mm_struct (pid_t pid,nodemask_t *mem_nodes) {struct task_struct *task;struct mm_struct *mm;
tools/testing/selftests/vm/.gitignore +1 -0
@@ -23,3 +23,4 @@ write_to_hugetlbfs hmm-tests memfd_secret local_config.* +split_huge_page_test
tools/testing/selftests/vm/Makefile +1 -0
@@ -43,6 +43,7 @@ TEST_GEN_FILES += on-fault-limit TEST_GEN_FILES += thuge-gen TEST_GEN_FILES += transhuge-stress TEST_GEN_FILES += userfaultfd +TEST_GEN_FILES += split_huge_page_test ifeq ($(ARCH),x86_64) CAN_BUILD_I386 := $(shell ./../x86/check_cc.sh $(CC) ../x86/trivial_32bit_program.c -m32)
tools/testing/selftests/vm/split_huge_page_test.c new file +313 -0
313 lines not on this page · raw message
Signature · 1 line
2.28.0
PATCH 3/7 mm: memcg: make memcg huge page split support any order split.
In reply to Zi YanFrom: Zi Yan <[email protected]>
It sets memcg information for the pages after the split. A new parameter new_order is added to tell the new page order, always 0 for now. It prepares for upcoming changes to support split huge page to any lower order.
--- include/linux/memcontrol.h | 5 +++-- mm/huge_memory.c | 2 +- mm/memcontrol.c | 6 +++--- 3 files changed, 7 insertions(+), 6 deletions(-)
include/linux/memcontrol.h +3 -2
@@ -1062,7 +1062,7 @@ static inline void memcg_memory_event_mm(struct mm_struct *mm, }#ifdef CONFIG_TRANSPARENT_HUGEPAGE -void mem_cgroup_split_huge_fixup (struct page *head); +void mem_cgroup_split_huge_fixup (struct page *head,unsigned int new_order);#endif #else /* CONFIG_MEMCG */
@@ -1396,7 +1396,8 @@ unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order,return 0 ; } -static inline void mem_cgroup_split_huge_fixup (struct page *head) +static inline void mem_cgroup_split_huge_fixup (struct page *head, +unsigned int new_order) { }
mm/huge_memory.c +1 -1
@@ -2428,7 +2428,7 @@ static void __split_huge_page(struct page *page, struct list_head *list, lruvec =mem_cgroup_page_lruvec (head, pgdat);/* complete memcg works before add pages to LRU */ -mem_cgroup_split_huge_fixup (head); +mem_cgroup_split_huge_fixup (head,0 );if (PageAnon (head) &&PageSwapCache (head)) {swp_entry_t entry = { .val =page_private (head) };
mm/memcontrol.c +3 -3
@@ -3223,15 +3223,15 @@ void obj_cgroup_uncharge(struct obj_cgroup *objcg, size_t size)* Because tail pages are not marked as "used", set it. We're under * pgdat->lru_lock and migration entries setup in all page mappings. */ -void mem_cgroup_split_huge_fixup (struct page *head) +void mem_cgroup_split_huge_fixup (struct page *head,unsigned int new_order) {struct mem_cgroup *memcg =page_memcg (head); -int i; +int i, new_nr =1 << new_order;if (mem_cgroup_disabled ())return ; -for (i =1 ; i <thp_nr_pages (head); i++) { +for (i = new_nr; i <thp_nr_pages (head); i += new_nr) {css_get (&memcg->css); head[i].memcg_data = (unsigned long )memcg; }
Signature · 1 line
2.28.0
PATCH 4/7 mm: page_owner: add support for splitting to any order in split page_owner.
In reply to Zi YanFrom: Zi Yan <[email protected]>
It adds a new_order parameter to set new page order in page owner and uses old_order instead of nr to make the parameters look consistent. It prepares for upcoming changes to support split huge page to any lower order.
--- include/linux/page_owner.h | 10 ++++++---- mm/huge_memory.c | 3 ++- mm/page_alloc.c | 2 +- mm/page_owner.c | 13 +++++++------ 4 files changed, 16 insertions(+), 12 deletions(-)
include/linux/page_owner.h +6 -4
@@ -11,7 +11,8 @@ extern struct page_ext_operations page_owner_ops;extern void __reset_page_owner (struct page *page,unsigned int order);extern void __set_page_owner (struct page *page,unsigned int order,gfp_t gfp_mask); -extern void __split_page_owner (struct page *page,unsigned int nr); +extern void __split_page_owner (struct page *page,unsigned int old_order, +unsigned int new_order);extern void __copy_page_owner (struct page *oldpage,struct page *newpage);extern void __set_page_owner_migrate_reason (struct page *page,int reason);extern void __dump_page_owner (struct page *page);
@@ -31,10 +32,11 @@ static inline void set_page_owner(struct page *page,__set_page_owner (page, order, gfp_mask); } -static inline void split_page_owner (struct page *page,unsigned int nr) +static inline void split_page_owner (struct page *page,unsigned int old_order, +unsigned int new_order) {if (static_branch_unlikely (&page_owner_inited)) -__split_page_owner (page, nr); +__split_page_owner (page, old_order, new_order); }static inline void copy_page_owner (struct page *oldpage,struct page *newpage) {
@@ -60,7 +62,7 @@ static inline void set_page_owner(struct page *page, { }static inline void split_page_owner (struct page *page, -unsigned int order) +unsigned int old_order,unsigned int new_order) { }static inline void copy_page_owner (struct page *oldpage,struct page *newpage)
mm/huge_memory.c +2 -1
@@ -2422,6 +2422,7 @@ static void __split_huge_page(struct page *page, struct list_head *list,struct lruvec *lruvec;struct address_space *swap_cache =NULL ;unsigned long offset =0 ; +unsigned int order =thp_order (head);unsigned int nr =thp_nr_pages (head);int i;
@@ -2458,7 +2459,7 @@ static void __split_huge_page(struct page *page, struct list_head *list,ClearPageCompound (head); -split_page_owner (head, nr); +split_page_owner (head, order,0 );/* See comment in __split_huge_page_tail() */ if (PageAnon (head)) {
mm/page_alloc.c +1 -1
@@ -3297,7 +3297,7 @@ void split_page(struct page *page, unsigned int order)for (i =1 ; i < (1 << order); i++)set_page_refcounted (page + i); -split_page_owner (page,1 << order); +split_page_owner (page, order,0 ); }EXPORT_SYMBOL_GPL (split_page);
mm/page_owner.c +7 -6
@@ -204,19 +204,20 @@ void __set_page_owner_migrate_reason(struct page *page, int reason) page_owner->last_migrate_reason = reason; } -void __split_page_owner (struct page *page,unsigned int nr) +void __split_page_owner (struct page *page,unsigned int old_order, +unsigned int new_order) { -int i; -struct page_ext *page_ext =lookup_page_ext (page); +int i, old_nr =1 << old_order, new_nr =1 << new_order; +struct page_ext *page_ext;struct page_owner *page_owner;if (unlikely (!page_ext))return ; -for (i =0 ; i < nr; i++) { +for (i =0 ; i < old_nr; i += new_nr) { + page_ext =lookup_page_ext (page + i); page_owner =get_page_owner (page_ext); - page_owner->order =0 ; - page_ext =page_ext_next (page_ext); + page_owner->order = new_order; } }
Signature · 1 line
2.28.0
PATCH 5/7 mm: thp: split huge page to any lower order pages.
In reply to Zi YanFrom: Zi Yan <[email protected]>
To split a THP to any lower order pages, we need to reform THPs on subpages at given order and add page refcount based on the new page order. Also we need to reinitialize page_deferred_list after removing the page from the split_queue, otherwise a subsequent split will see list corruption when checking the page_deferred_list again.
It has many uses, like minimizing the number of pages after truncating a pagecache THP. For anonymous THPs, we can only split them to order-0 like before until we add support for any size anonymous THPs.
--- include/linux/huge_mm.h | 8 +++ mm/huge_memory.c | 119 +++++++++++++++++++++++++++++----------- mm/swap.c | 1 - 3 files changed, 96 insertions(+), 32 deletions(-)
include/linux/huge_mm.h +8 -0
@@ -182,6 +182,8 @@ bool is_transparent_hugepage(struct page *page);bool can_split_huge_page (struct page *page,int *pextra_pins);int split_huge_page_to_list (struct page *page,struct list_head *list); +int split_huge_page_to_list_to_order (struct page *page,struct list_head *list, +unsigned int new_order);static inline int split_huge_page (struct page *page) {return split_huge_page_to_list (page,NULL );
@@ -385,6 +387,12 @@ split_huge_page_to_list(struct page *page, struct list_head *list) {return 0 ; } +static inline int +split_huge_page_to_list_to_order (struct page *page,struct list_head *list, +unsigned int new_order) +{ +return 0 ; +}static inline int split_huge_page (struct page *page) {return 0 ;
mm/huge_memory.c +88 -31
278 lines not on this page · raw message
mm/swap.c +0 -1
@@ -983,7 +983,6 @@ void lru_add_page_tail(struct page *page, struct page *page_tail, struct lruvec *lruvec, struct list_head *list) { VM_BUG_ON_PAGE(!PageHead(page), page); -VM_BUG_ON_PAGE (PageCompound (page_tail), page);VM_BUG_ON_PAGE (PageLRU (page_tail), page);lockdep_assert_held (&lruvec_pgdat (lruvec)->lru_lock);
Signature · 1 line
2.28.0
PATCH 6/7 mm: truncate: split thp to a non-zero order if possible.
In reply to Zi YanFrom: Zi Yan <[email protected]>
To minimize the number of pages after a truncation, when truncating a THP, we do not need to split it all the way down to order-0. The THP has at most three parts, the part before offset, the part to be truncated, the part left at the end. Use the non-zero minimum of them to decide what order we split the THP to.
--- mm/truncate.c | 29 +++++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-)
mm/truncate.c +27 -2
@@ -237,7 +237,9 @@ int truncate_inode_page(struct address_space *mapping, struct page *page)bool truncate_inode_partial_page (struct page *page,loff_t start,loff_t end) {loff_t pos =page_offset (page); -unsigned int offset, length; +unsigned int offset, length, remaining, min_subpage_size =PAGE_SIZE ; +unsigned int new_order; +if (pos < start) offset = start - pos;
@@ -248,6 +250,7 @@ bool truncate_inode_partial_page(struct page *page, loff_t start, loff_t end) length = length - offset; else length = end +1 - pos - offset; + remaining =thp_size (page) - offset - length;wait_on_page_writeback (page);if (length ==thp_size (page)) {
@@ -267,7 +270,29 @@ bool truncate_inode_partial_page(struct page *page, loff_t start, loff_t end)do_invalidatepage (page, offset, length);if (!PageTransHuge (page))return true; -return split_huge_page (page) ==0 ; + +/* +* find the non-zero minimum of offset, length, and remaining and use it +* to decide the new order of the page after split +*/ +if (offset && remaining) + min_subpage_size = min_t(unsigned int, + min_t(unsigned int, offset, length), + remaining); +else if (!offset) + min_subpage_size = min_t(unsigned int, length, remaining); +else /* remaining == 0 */ + min_subpage_size = min_t(unsigned int, length, offset); + + min_subpage_size = max_t(unsigned int, PAGE_SIZE, min_subpage_size); + + new_order =ilog2 (min_subpage_size/PAGE_SIZE ); + +/* order-1 THP not supported, downgrade to order-0 */ +if (new_order ==1 ) + new_order =0 ; + +return split_huge_page_to_list_to_order (page,NULL , new_order) ==0 ; }/*
Signature · 1 line
2.28.0
PATCH 7/7 mm: huge_memory: enable debugfs to split huge pages to any order.
In reply to Zi YanFrom: Zi Yan <[email protected]>
It is used to test split_huge_page_to_list_to_order for pagecache THPs. Also add test cases for split_huge_page_to_list_to_order via both debugfs, truncating a file, and punching holes in a file.
--- mm/huge_memory.c | 13 +- .../selftests/vm/split_huge_page_test.c | 192 ++++++++++++++++-- 2 files changed, 186 insertions(+), 19 deletions(-)
mm/huge_memory.c +7 -6
@@ -2999,7 +2999,7 @@ static ssize_t split_huge_pages_in_range_pid_write(struct file *file,static DEFINE_MUTEX (mutex);ssize_t ret;char input_buf[80 ];/* hold pid, start_vaddr, end_vaddr */ -int pid; +int pid, to_order =0 ;unsigned long vaddr_start, vaddr_end, addr;nodemask_t task_nodes;struct mm_struct *mm;
@@ -3016,8 +3016,9 @@ static ssize_t split_huge_pages_in_range_pid_write(struct file *file, goto out; input_buf[79 ] ='\0' ; - ret =sscanf (input_buf,"%d,0x%lx,0x%lx" , &pid, &vaddr_start, &vaddr_end); -if (ret !=3 ) { + ret =sscanf (input_buf,"%d,0x%lx,0x%lx,%d" , &pid, &vaddr_start, &vaddr_end, &to_order); +/* cannot split to order-1 THP, which is not possible */ +if ((ret !=3 && ret !=4 ) || to_order ==1 ) { ret = -EINVAL ; goto out; }
@@ -3025,8 +3026,8 @@ static ssize_t split_huge_pages_in_range_pid_write(struct file *file, vaddr_end &=PAGE_MASK ; ret =strlen (input_buf); -pr_debug ("split huge pages in pid: %d, vaddr: [%lx - %lx]\n" , - pid, vaddr_start, vaddr_end); +pr_debug ("split huge pages in pid: %d, vaddr: [%lx - %lx], to order: %d\n" , + pid, vaddr_start, vaddr_end, to_order); mm =find_mm_struct (pid, &task_nodes);if (IS_ERR (mm)) {
@@ -3066,7 +3067,7 @@ static ssize_t split_huge_pages_in_range_pid_write(struct file *file,if (!trylock_page (page))continue ; -if (!split_huge_page (page)) +if (!split_huge_page_to_list_to_order (page,NULL , to_order)) split++;unlock_page (page);
tools/testing/selftests/vm/split_huge_page_test.c +179 -13
284 lines not on this page · raw message
Signature · 1 line
2.28.0