Hugetlb demotion for x86
Subsystems (3)
RFC 1/1 Hugetlb demotion for x86
The following patch enables demotion of MAP_PRIVATE hugetlb memory to normal anonymous memory on the i386 architecture. Below is a short description of the problem from a previous posting.
Quoted 10 lines
Thanks to the latest hugetlb accounting patches, we now have reliable shared mappings. Private mappings are much more difficult because there is no way to know up-front how many huge pages will be required (we may have forking combined with unknown copy-on-write activity). So private mappings currently get full overcommit semantics and when a fault cannot be handled, the apps get SIGBUS.
The problem: Random SIGBUS crashes for applications using large pages are not acceptable. We need a way to handle the fault without giving up and killing the process.
The mechanics of this approach are straightforward. When failing to allocate a huge page, the HPAGE_SIZE area is munmap'ed and mmap'ed as anonymous memory. If a hugepte existed (happens on a COW fault) a series of write-protected normal ptes that point to the sub-pages of the original huge page are installed. When finished, the normal fault handling path finishes the fault via do_anonymous page (or do_wp_page for COW faults).
At this point I am looking for comments on the approach. It has been working reliably on my system so far but I have probably missed something. Also, enabling other architectures will involve a more work (such as demotion regions that span multiple pages and vmas), which I am now looking at.
* Patch not ready for merging
include/linux/mman.h | 22 ++++++++++++ mm/hugetlb.c | 88 ++++++++++++++++++++++++++++++++++++++++++++++++++- mm/rmap.c | 3 + 3 files changed, 112 insertions(+), 1 deletion(-)
include/linux/mman.h +22 -0
@@ -53,6 +53,17 @@ calc_vm_prot_bits(unsigned long prot) }/* +* Combine the vm_flags protection bits into mmap "prot" argument +*/ +static inline unsigned long +calc_mmap_prot_bits (unsigned long vm_flags) +{ +return _calc_vm_trans (vm_flags,VM_READ ,PROT_READ ) | +_calc_vm_trans (vm_flags,VM_WRITE ,PROT_WRITE ) | +_calc_vm_trans (vm_flags,VM_EXEC ,PROT_EXEC ); +} + +/* * Combine the mmap "flags" argument into "vm_flags" used internally. */ static inline unsigned long
@@ -64,4 +75,15 @@ calc_vm_flag_bits(unsigned long flags) _calc_vm_trans(flags, MAP_LOCKED, VM_LOCKED ); } +/* +* Convert vm_flags into the mmap "flags" argument +*/ +static inline unsigned long +calc_mmap_flag_bits (unsigned long vm_flags) +{ +return _calc_vm_trans (vm_flags,VM_GROWSDOWN ,MAP_GROWSDOWN ) | +_calc_vm_trans (vm_flags,VM_DENYWRITE ,MAP_DENYWRITE ) | +_calc_vm_trans (vm_flags,VM_EXECUTABLE ,MAP_EXECUTABLE ) | +_calc_vm_trans (vm_flags,VM_LOCKED ,MAP_LOCKED ); +}#endif /* _LINUX_MMAN_H */
mm/hugetlb.c +87 -1
113 lines not on this page · raw message
mm/rmap.c +3 -0
@@ -548,6 +548,9 @@ void page_add_file_rmap(struct page *pag*/ void page_remove_rmap (struct page *page) { +if (unlikely (PageCompound (page))) +return ; +if (atomic_add_negative (-1 , &page->_mapcount)) {#ifdef CONFIG_DEBUG_VM if (unlikely (page_mapcount (page) <0 )) {
Signature · 2 lines
Adam Litke - (agl at us.ibm.com) IBM Linux Technology Center
Quoted 6 lines · Adam Litke
mm/hugetlb.c +6
-0+install_demotion_ptes(struct mm_struct *mm, struct page *page, + pgprot_t prot, unsigned long address) +{ + pgd_t *pgd; +pud_t *pud; +pmd_t *pmd;
mm/hugetlb.c void unmap_hugepage_range(struct vm_area · line 530
Quoted 10 lines
mm/hugetlb.c +10
-0+pte_t entry, *ptep; +int pfn, i; + + pgd =pgd_offset (mm, address); + pud =pud_alloc (mm, pgd, address); +if (!pud) +return -ENOMEM ; + pmd =pmd_alloc (mm, pud, address); +if (!pmd) +return -ENOMEM ;
That looks to be a pretty direct copy of what is already in __handle_mm_fault(). Can they be consolidated?
-- Dave
mm/hugetlb.c void unmap_hugepage_range(struct vm_area · line 575
Quoted 6 lines · Adam Litke
mm/hugetlb.c +5
-0+do_munmap (mm, start,HPAGE_SIZE ); + start =do_mmap_pgoff (0 , start,HPAGE_SIZE , prot, flags,0 ); +if (start <0 ) { +return VM_FAULT_OOM ; + }
Hmm.. These are being done in this path, right?
do_page_fault() handle_mm_fault() __handle_mm_fault() hugetlb_fault() hugetlb_demote_page()
I believe do_munmap() requires a write on mmap_sem(), but do_page_fault() only takes a read.
-- Dave
Quoted 2 lines · Adam Litke
The following patch enables demotion of
MAP_PRIVATEhugetlb memory to normal anonymous memory on the i386 architecture.
This is an awfully bad idea. Applications should do smart fallback instead. For the same reason we for example fail O_DIRECT requests we can fullfill instead of doing the half buffered I/O braindamage solaris does.
Quoted 8 lines · Christoph Hellwig
Quoted 2 lines · Adam Litke
The following patch enables demotion of
MAP_PRIVATEhugetlb memory to normal anonymous memory on the i386 architecture.This is an awfully bad idea. Applications should do smart fallback instead. For the same reason we for example fail
O_DIRECTrequests we can fullfill instead of doing the half buffered I/O braindamage solaris does.
By smart fallback do you mean we should convert the hugetlb fault code back to using VM_FAULT_SIGBUS and writing userspace sighandlers to do the same thing I am, but in userspace? FWIW I did implement that in libhugetlbfs to try it out, but that seems much dirtier to me than handling faults in the kernel.
Signature · 2 lines
Adam Litke - (agl at us.ibm.com) IBM Linux Technology Center
Quoted 5 lines · Adam Litke
By smart fallback do you mean we should convert the hugetlb fault code back to using
VM_FAULT_SIGBUSand writing userspace sighandlers to do the same thing I am, but in userspace? FWIW I did implement that in libhugetlbfs to try it out, but that seems much dirtier to me than handling faults in the kernel.
Umm, why do these faults happen at all? When all the hugetlb code went in it we allocated at mmap time. Later it was converted to demand faulting but under the premise that we keep the strict overcommit accounting. When did that part go away aswell? With strict overcommit handling for huge pages no fault should happen when the pool is exausted.
Quoted 7 lines · Christoph Hellwig
Quoted 5 lines · Adam Litke
By smart fallback do you mean we should convert the hugetlb fault code back to using
VM_FAULT_SIGBUSand writing userspace sighandlers to do the same thing I am, but in userspace? FWIW I did implement that in libhugetlbfs to try it out, but that seems much dirtier to me than handling faults in the kernel.
Quoted 5 lines
Umm, why do these faults happen at all? When all the hugetlb code went in it we allocated at mmap time. Later it was converted to demand faulting but under the premise that we keep the strict overcommit accounting. When did that part go away aswell? With strict overcommit handling for huge pages no fault should happen when the pool is exausted.
Strict overcommit is there for shared mappings. When private mapping support was added, people agreed that full overcommit should apply to private mappings for the same reasons normal page overcommit is desired. For one: an application using lots of private huge pages should not be prohibited from forking if it's likely to just exec a small helper program.
"These faults" are happening in two cases when MAP_PRIVATE huge pages are being used:
1) Fault on an uninstantiated huge page: This can happen when numerous users of huge pages in the system are competing for a finite number of huge pages. Even if the process checks for free huge pages before mmaping the area, another process is free to "steal" those pages out from under the careful process.
2) COW fault on an instantiated huge page: Happens in child processes who inherit a private hugetlb region and write to it.
Both of these cases are non-deterministic and should be handled in some way. Just killing the process doesn't seem like a permanent solution to me.
Signature · 2 lines
Adam Litke - (agl at us.ibm.com) IBM Linux Technology Center
Quoted 2 lines · Adam Litke
Strict overcommit is there for shared mappings. When private mapping
I presume that by "strict overcommit" you mean "strict no overcommit".
Quoted 2 lines
support was added, people agreed that full overcommit should apply to private mappings for the same reasons normal page overcommit is desired.
I'm not sure how wide that agreement was. But what I wanted to say is...
Quoted 3 lines
For one: an application using lots of private huge pages should not be prohibited from forking if it's likely to just exec a small helper program.
This is an excellent use for madvise(start, length, MADV_DONTFORK). Though it was added mainly for RDMA issues, it's a great way for a program with a huge commitment to exclude areas of its address space from the fork, so making that fork much more likely to succeed.
Hugh
Quoted 8 lines · Hugh Dickins
Quoted 3 lines
For one: an application using lots of private huge pages should not be prohibited from forking if it's likely to just exec a small helper program.
This is an excellent use for
madvise(start, length, MADV_DONTFORK). Though it was added mainly for RDMA issues, it's a great way for a program with a huge commitment to exclude areas of its address space from the fork, so making that fork much more likely to succeed.
Or fork using vfork() in that case which has even more wins and is a more efficient if more hair-raising way of doing it
Quoted 11 lines · Hugh Dickins
Quoted 2 lines · Adam Litke
Strict overcommit is there for shared mappings. When private mapping
I presume that by "strict overcommit" you mean "strict no overcommit".
Quoted 2 lines
support was added, people agreed that full overcommit should apply to private mappings for the same reasons normal page overcommit is desired.
I'm not sure how wide that agreement was. But what I wanted to say is...
Quoted 8 lines
Quoted 3 lines
For one: an application using lots of private huge pages should not be prohibited from forking if it's likely to just exec a small helper program.
This is an excellent use for
madvise(start, length, MADV_DONTFORK). Though it was added mainly for RDMA issues, it's a great way for a program with a huge commitment to exclude areas of its address space from the fork, so making that fork much more likely to succeed.
I guess it's time for me to take a step back and explain why I am doing this. libhugetlbfs (announced here recently) has the ability to remap an executable's ELF segments into huge pages. So madvise(MADV_DONTFORK) would be pretty bad ;)
Signature · 2 lines
Adam Litke - (agl at us.ibm.com) IBM Linux Technology Center
Quoted 3 lines · Adam Litke
The following patch enables demotion of
MAP_PRIVATEhugetlb memory to normal anonymous memory on the i386 architecture. Below is a short description of the problem from a previous posting.
I'm not sure it's a good idea really. I think people have the reasonable expection that if you use hugetlb you really get huge pages.
If you really implement you should at least printk clearly. But it's probably better to not implement it.
If there was generic transparent hugepages support in the VM the case would be probably different. Then it would make sense. But not as part of hugetlbfs.
-Andi
Seems that the code is not modifying x86 code but all code.
An app should be getting an out of memory error and not a SIGBUS when running out of memory.
I thought we fixed the SIGBUS problems and were now reporting out of memory? If there still is an issue then we better fix out of memory handling. Provide a way for the app to trap OOM conditions?
Quoted 1 line · Christoph Lameter
Seems that the code is not modifying x86 code but all code.
Right. It's definitely broken in that regard. I sent it out in this condition so the patch was small, easy to review, and my approach would be easy to see.
Quoted 6 lines
An app should be getting an out of memory error and not a SIGBUS when running out of memory.
I thought we fixed the SIGBUS problems and were now reporting out of memory? If there still is an issue then we better fix out of memory handling. Provide a way for the app to trap OOM conditions?
Yes, the SIGBUS issues are "fixed". Now the application is killed directly via VM_FAULT_OOM so it is not possible to handle the fault from userspace. For my libhugetlbfs-based fallback approach, I needed to patch the kernel so that SIGBUS was delivered to the process like in the days of old.
Signature · 2 lines
Adam Litke - (agl at us.ibm.com) IBM Linux Technology Center
Quoted 5 lines · Adam Litke
Yes, the SIGBUS issues are "fixed". Now the application is killed directly via
VM_FAULT_OOMso it is not possible to handle the fault from userspace. For my libhugetlbfs-based fallback approach, I needed to patch the kernel so that SIGBUS was delivered to the process like in the days of old.
Maybe this could be off-by-default behavior that can be enabled with a special mmap flag or madvise, or something similar. It seems that apps don't want to get SIGBUS for low memory. But, if they have _asked_ for it, perhaps they'd be a bit more willing.
(BTW, I fixed the bogus linux-mm cc, finally ;)
-- Dave
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>