Memory management -- THP, hugetlb, scalability
Subsystems (2)
Hi,
I would like to attend LSF/MM summit. I'm interested in discussion about huge pages, scalability of memory management subsystem and persistent memory.
Last year I did some work to fix THP-related regressions and improve scalability. I also work on THP for file-backed pages.
Depending on project status, I probably want to bring transparent huge pagecache as a topic.
Signature · 1 line
Kirill A. Shutemov
Quoted 6 lines · Kirill A. Shutemov
Hi,
I would like to attend LSF/MM summit. I'm interested in discussion about huge pages, scalability of memory management subsystem and persistent memory.
Quoted 6 lines
Last year I did some work to fix THP-related regressions and improve scalability. I also work on THP for file-backed pages.
Depending on project status, I probably want to bring transparent huge pagecache as a topic.
I think transparent huge pagecache is likely to crop up for more than one reason. There is the TLB issue and the motivation that i-TLB pressure is a problem in some specialised cases. Whatever the merits of that case, transparent hugepage cache has been raised as a potential solution for some VM scalability problems. I recognise that dealing with large numbers of struct pages is now a problem on larger machines (although I have not seen quantified data on the problem nor do I have access to a machine large enough to measure it myself) but I'm wary of transparent hugepage cache being treated as a primary solution for VM scalability problems. Lacking performance data I have no suggestions on what these alternative solutions might look like.
Signature · 8 lines
Mel Gorman SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 14 lines · Mel Gorman
Quoted 6 lines · Kirill A. Shutemov
Hi,
I would like to attend LSF/MM summit. I'm interested in discussion about huge pages, scalability of memory management subsystem and persistent memory.
Quoted 6 lines
Last year I did some work to fix THP-related regressions and improve scalability. I also work on THP for file-backed pages.
Depending on project status, I probably want to bring transparent huge pagecache as a topic.
Quoted 11 lines
I think transparent huge pagecache is likely to crop up for more than one reason. There is the TLB issue and the motivation that i-TLB pressure is a problem in some specialised cases. Whatever the merits of that case, transparent hugepage cache has been raised as a potential solution for some VM scalability problems. I recognise that dealing with large numbers of struct pages is now a problem on larger machines (although I have not seen quantified data on the problem nor do I have access to a machine large enough to measure it myself) but I'm wary of transparent hugepage cache being treated as a primary solution for VM scalability problems. Lacking performance data I have no suggestions on what these alternative solutions might look like.
Yes, performance data is critical. I'll try bring some.
The only alternative I see is some kind of THP, implemented on filesystem level. It can work for tmpfs/shm reasonably well. But it looks ad-hoc and in long term transparent huge pagecache is the way to go, I believe.
Sibling topic is THP for XIP (see Matthew's patchset). Guys want to manage persistent memory in 2M chunks where it's possible. And THP (but without struct page in this case) is the obvious solution.
Signature · 7 lines
Kirill A. Shutemov -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 12 lines · Kirill A. Shutemov
Quoted 11 lines · Mel Gorman
I think transparent huge pagecache is likely to crop up for more than one reason. There is the TLB issue and the motivation that i-TLB pressure is a problem in some specialised cases. Whatever the merits of that case, transparent hugepage cache has been raised as a potential solution for some VM scalability problems. I recognise that dealing with large numbers of struct pages is now a problem on larger machines (although I have not seen quantified data on the problem nor do I have access to a machine large enough to measure it myself) but I'm wary of transparent hugepage cache being treated as a primary solution for VM scalability problems. Lacking performance data I have no suggestions on what these alternative solutions might look like.
Something I'd like to see discussed (but don't have the MM chops to lead a discussion on myself) is the PAGE_CACHE_SIZE vs PAGE_SIZE split. This needs to be either fixed or removed, IMO. It's been in the tree since before git history began (ie before 2005), it imposes a reasonably large cognitive burden on programmers ("what kind of page size do I want here?"), it's not intuitively obvious (to a non-mm person) which page size is which, and it's never actually bought us anything because it's always been the same!
Also, it bitrots. Look at this:
pgoff_t pgoff = (((address & PAGE_MASK)
- vma->vm_start) >> PAGE_SHIFT) + vma->vm_pgoff;
vmf.pgoff = pgoff;
pgoff_t offset = vmf->pgoff;
size = (i_size_read(inode) + PAGE_CACHE_SIZE - 1) >> PAGE_CACHE_SHIFT;
if (offset >= size)
return VM_FAULT_SIGBUS;That's spread over three functions, but that goes to illustrate my point; getting this stuff right is Hard; core mm developers get it wrong, we don't have the right types to document whether a variable is in PAGE_SIZE or PAGE_CACHE_SIZE units, and we're not getting any benefit from it today.
Quoted 3 lines
Sibling topic is THP for XIP (see Matthew's patchset). Guys want to manage persistent memory in 2M chunks where it's possible. And THP (but without struct page in this case) is the obvious solution.
Not just 2MB, we also want 1GB pages for some special cases. It looks doable (XFS can allocate aligned 1GB blocks). I've written some supporting code that will at least get us to the point where we can insert a 1GB page. I haven't been able to test anything yet.
Quoted 33 lines · Matthew Wilcox
Quoted 12 lines · Kirill A. Shutemov
Quoted 11 lines · Mel Gorman
I think transparent huge pagecache is likely to crop up for more than one reason. There is the TLB issue and the motivation that i-TLB pressure is a problem in some specialised cases. Whatever the merits of that case, transparent hugepage cache has been raised as a potential solution for some VM scalability problems. I recognise that dealing with large numbers of struct pages is now a problem on larger machines (although I have not seen quantified data on the problem nor do I have access to a machine large enough to measure it myself) but I'm wary of transparent hugepage cache being treated as a primary solution for VM scalability problems. Lacking performance data I have no suggestions on what these alternative solutions might look like.
Something I'd like to see discussed (but don't have the MM chops to lead a discussion on myself) is the
PAGE_CACHE_SIZEvsPAGE_SIZEsplit. This needs to be either fixed or removed, IMO. It's been in the tree since before git history began (ie before 2005), it imposes a reasonably large cognitive burden on programmers ("what kind of page size do I want here?"), it's not intuitively obvious (to a non-mm person) which page size is which, and it's never actually bought us anything because it's always been the same!Also, it bitrots. Look at this:
pgoff_t pgoff = (((address & PAGE_MASK) - vma->vm_start) >> PAGE_SHIFT) + vma->vm_pgoff; vmf.pgoff = pgoff; pgoff_t offset = vmf->pgoff; size = (i_size_read(inode) + PAGE_CACHE_SIZE - 1) >> PAGE_CACHE_SHIFT; if (offset >= size) return VM_FAULT_SIGBUS;
Quoted 4 lines
That's spread over three functions, but that goes to illustrate my point; getting this stuff right is Hard; core mm developers get it wrong, we don't have the right types to document whether a variable is in
PAGE_SIZEorPAGE_CACHE_SIZEunits, and we're not getting any benefit from it today.
I also want to drop PAGE_CACHE_*. It's on my todo list almost a year now ;)
Quoted 8 lines
Quoted 3 lines
Sibling topic is THP for XIP (see Matthew's patchset). Guys want to manage persistent memory in 2M chunks where it's possible. And THP (but without struct page in this case) is the obvious solution.
Not just 2MB, we also want 1GB pages for some special cases. It looks doable (XFS can allocate aligned 1GB blocks). I've written some supporting code that will at least get us to the point where we can insert a 1GB page. I haven't been able to test anything yet.
It's probably doable from fs point of view, but adding PUD-level THP page is not trivial at all. I think it's more productive better to concentrate on 2M for now.
Signature · 7 lines
Kirill A. Shutemov -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 24 lines · Kirill A. Shutemov
Quoted 14 lines · Matthew Wilcox
Quoted 12 lines · Kirill A. Shutemov
Quoted 11 lines · Mel Gorman
I think transparent huge pagecache is likely to crop up for more than one reason. There is the TLB issue and the motivation that i-TLB pressure is a problem in some specialised cases. Whatever the merits of that case, transparent hugepage cache has been raised as a potential solution for some VM scalability problems. I recognise that dealing with large numbers of struct pages is now a problem on larger machines (although I have not seen quantified data on the problem nor do I have access to a machine large enough to measure it myself) but I'm wary of transparent hugepage cache being treated as a primary solution for VM scalability problems. Lacking performance data I have no suggestions on what these alternative solutions might look like.
Quoted 8 lines
Something I'd like to see discussed (but don't have the MM chops to lead a discussion on myself) is the
PAGE_CACHE_SIZEvsPAGE_SIZEsplit. This needs to be either fixed or removed, IMO. It's been in the tree since before git history began (ie before 2005), it imposes a reasonably large cognitive burden on programmers ("what kind of page size do I want here?"), it's not intuitively obvious (to a non-mm person) which page size is which, and it's never actually bought us anything because it's always been the same!
Quoted 1 line
I also want to drop PAGE_CACHE_*. It's on my todo list almost a year now ;)
I dno't necessarily want to drop the concept of having 'the size of memory referenced by struct page' != 'the size of memory pointed at by a single PTE'. I just want to see it *implemented* for at least one architecture if we're going to have the distinction. It's one way of solving the problem that Mel mentioned (dealing with a large number of struct pages).
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Sibling topic is THP for XIP (see Matthew's patchset). Guys want to manage persistent memory in 2M chunks where it's possible. And THP (but without struct page in this case) is the obvious solution.
Quoted 8 lines
Quoted 4 lines
Not just 2MB, we also want 1GB pages for some special cases. It looks doable (XFS can allocate aligned 1GB blocks). I've written some supporting code that will at least get us to the point where we can insert a 1GB page. I haven't been able to test anything yet.
It's probably doable from fs point of view, but adding PUD-level THP page is not trivial at all. I think it's more productive better to concentrate on 2M for now.
It's clearly Hard to get to a point where we're inserting PUD entries for anonymous pages. While I don't think it's trivial to get to PUD entries for PFNMAP, I think it is doable.
Last time we discussed this, your concern was around splitting a PUD entry down into PTEs and having to preallocate all the memory required to do that.
We can't possibly need to call split_huge_page() for the PFNMAP case because we don't have a struct page, so none of those code paths can be run. I think that leaves split_huge_page_pmd() as the only place where we can try to split a huge PFNMAP PMD. That's called from:
mem_cgroup_count_precharge_pte_range() mem_cgroup_move_charge_pte_range()
These two look like they need to be converted to work on unsplit PMDs anyway, for efficiency reasons. Perhaps someone who's hacked on this file as recently as 2009 would care to do that work? :-)
zap_pmd_range() does this:
if (pmd_trans_huge(*pmd)) {
if (next-addr != HPAGE_PMD_SIZE) {
VM_BUG_ON(!rwsem_is_locked(&tlb->mm->mmap_sem));
split_huge_page_pmd(vma->vm_mm, pmd);
} else if (zap_huge_pmd(tlb, vma, pmd))
continue;
/* fall through */
}I don't understand why it bothers to split rather than just zapping the PMD and allowing refaults to populate the PTEs later.
follow_page() calls it, but I think we can give up way earlier in this function, since we know there's no struct page to return. We can put in something like:
if (IS_XIP(file_inode(vma->vm_file))) return ERR_PTR(-Ewhatever);
check_pmd_range() calls it, but this is NUMA policy for the page cache. We should be skipping this code for XIP files too, if we aren't already.
change_pmd_range() calls split_huge_page_pmd() if an mprotect call lands in the middle of a PMD range. Again, I'd be *fine* with just dropping the PMD entry here and allowing faults to repopulate the PTEs.
Looks like the mremap code may need some work. I'm not sure what that work is right now.
That leaves us with walk_page_range() ... which also looks like it's going to need some work in the callers.
So yeah, not trivial at all, but doable with a few weeks of work, I think. Unless there's some other major concern that I've missed (which is possible since I'm not a MM hacker).
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 8 lines · Matthew Wilcox
Quoted 1 line
I also want to drop PAGE_CACHE_*. It's on my todo list almost a year now ;)
I dno't necessarily want to drop the concept of having 'the size of memory referenced by struct page' != 'the size of memory pointed at by a single PTE'. I just want to see it *implemented* for at least one architecture if we're going to have the distinction. It's one way of solving the problem that Mel mentioned (dealing with a large number of struct pages).
Okay. But I don't think it's going to happen.
Quoted 13 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Sibling topic is THP for XIP (see Matthew's patchset). Guys want to manage persistent memory in 2M chunks where it's possible. And THP (but without struct page in this case) is the obvious solution.
Quoted 8 lines
Quoted 4 lines
Not just 2MB, we also want 1GB pages for some special cases. It looks doable (XFS can allocate aligned 1GB blocks). I've written some supporting code that will at least get us to the point where we can insert a 1GB page. I haven't been able to test anything yet.
It's probably doable from fs point of view, but adding PUD-level THP page is not trivial at all. I think it's more productive better to concentrate on 2M for now.
Quoted 6 lines
It's clearly Hard to get to a point where we're inserting PUD entries for anonymous pages. While I don't think it's trivial to get to PUD entries for PFNMAP, I think it is doable.
Last time we discussed this, your concern was around splitting a PUD entry down into PTEs and having to preallocate all the memory required to do that.
Other thing is dealing with PMD vs PTE races (due splitting or MADV_DONTNEED or something else). Adding PUD to the picture doesn't make it easier.
Quoted 5 lines
We can't possibly need to call
split_huge_page()for the PFNMAP case because we don't have a struct page, so none of those code paths can be run. I think that leavessplit_huge_page_pmd()as the only place where we can try to split a huge PFNMAP PMD.
Quoted 7 lines
That's called from:
mem_cgroup_count_precharge_pte_range()mem_cgroup_move_charge_pte_range()
These two look like they need to be converted to work on unsplit PMDs anyway, for efficiency reasons. Perhaps someone who's hacked on this file as recently as 2009 would care to do that work? :-)
:) May be. I don't really remember anything there.
Quoted 11 lines
zap_pmd_range()does this:if (pmd_trans_huge(*pmd)) { if (next-addr != HPAGE_PMD_SIZE) { VM_BUG_ON(!rwsem_is_locked(&tlb->mm->mmap_sem)); split_huge_page_pmd(vma->vm_mm, pmd); } else if (zap_huge_pmd(tlb, vma, pmd)) continue; /* fall through */ }
Quoted 2 lines
I don't understand why it bothers to split rather than just zapping the PMD and allowing refaults to populate the PTEs later.
Because with anon-pages you don't have a backing storage to repopulate
from: it will free the memory and you will end up with clear pages after next page fault.
Yeah, it's not relevant for file pages, you can just unmap the pmd.
Quoted 6 lines
follow_page()calls it, but I think we can give up way earlier in this function, since we know there's no struct page to return. We can put in something like:if (IS_XIP(file_inode(vma->vm_file))) return ERR_PTR(-Ewhatever);
Are you sure you will not need a temporary struct page here to show to caller or something like this?
Quoted 2 lines
check_pmd_range()calls it, but this is NUMA policy for the page cache. We should be skipping this code for XIP files too, if we aren't already.
Okay.
Quoted 3 lines
change_pmd_range()callssplit_huge_page_pmd()if an mprotect call lands in the middle of a PMD range. Again, I'd be *fine* with just dropping the PMD entry here and allowing faults to repopulate the PTEs.
Do you have a way to store info that the area should be repopulated with PTEs, not PMD?
Quoted 2 lines
Looks like the mremap code may need some work. I'm not sure what that work is right now.
You probably may unmap there too and handle as !old_pmd.
Quoted 6 lines
That leaves us with
walk_page_range()... which also looks like it's going to need some work in the callers.So yeah, not trivial at all, but doable with a few weeks of work, I think. Unless there's some other major concern that I've missed (which is possible since I'm not a MM hacker).
Okay, doable. I guess.
The general approach is replace split with unmap. unmap_mapping_range() takes ->i_mmap_mutex and we probably will hit locking ordering issues.
But I would suggest to take more conservative approach first: leave 1G pages aside, use 2M pages with page table pre-allocation and implement proper splitting in split_huge_page_pmd() for this case. It should be much easier then fix all split_huge_page_pmd() callers.
After getting this work we can look how to eliminate memory overhead on preallocated page tables and bring 1G pages.
By the time you probably will have some performance data to say that you don't really need 1G pages that much. ;)
Signature · 7 lines
Kirill A. Shutemov -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>