hugetlbfs: fix hugetlb_get_unmapped_area
Subsystems (5)
PATCH 1/5 hugetlbfs: fix hugetlb_get_unmapped_area
Using/updating cached_hole_size and free_area_cache properly to speedup find free region
--- fs/hugetlbfs/inode.c | 15 +++++++++++---- 1 files changed, 11 insertions(+), 4 deletions(-)
fs/hugetlbfs/inode.c +11 -4
@@ -154,10 +154,12 @@ hugetlb_get_unmapped_area(struct file *file, unsigned long addr,return addr; } - start_addr = mm->free_area_cache; - -if (len <= mm->cached_hole_size) +if (len > mm->cached_hole_size) + start_addr = mm->free_area_cache; +else { start_addr =TASK_UNMAPPED_BASE ; + mm->cached_hole_size =0 ; + } full_search: addr =ALIGN (start_addr,huge_page_size (h));
@@ -171,13 +173,18 @@ full_search:*/ if (start_addr !=TASK_UNMAPPED_BASE ) { start_addr =TASK_UNMAPPED_BASE ; + mm->cached_hole_size =0 ; goto full_search; }return -ENOMEM ; } -if (!vma || addr + len <= vma->vm_start) +if (!vma || addr + len <= vma->vm_start) { + mm->free_area_cache = addr + len;return addr; + } +if (addr + mm->cached_hole_size < vma->vm_start) + mm->cached_hole_size = vma->vm_start - addr; addr =ALIGN (vma->vm_end,huge_page_size (h)); } }
Signature · 1 line
1.7.7.5
PATCH 2/5 hugetlb: drop prev_vma in hugetlb_get_unmapped_area_topdown
Afte call find_vma_prev(mm, addr, &prev_vma), following condition is always true:
!prev_vma || (addr >= prev_vma->vm_end)
it can be happily drop prev_vma and use find_vma instead of find_vma_prev
--- arch/x86/mm/hugetlbpage.c | 21 ++++++--------------- 1 files changed, 6 insertions(+), 15 deletions(-)
arch/x86/mm/hugetlbpage.c +6 -15
@@ -308,7 +308,7 @@ static unsigned long hugetlb_get_unmapped_area_topdown(struct file *file, {struct hstate *h =hstate_file (file);struct mm_struct *mm = current->mm; -struct vm_area_struct *vma, *prev_vma; +struct vm_area_struct *vma;unsigned long base = mm->mmap_base, addr = addr0;unsigned long largest_hole = mm->cached_hole_size;int first_time =1 ;
@@ -333,24 +333,15 @@ try_again:* Lookup failure means no vma is above this address, * i.e. return with success: */ -if (!(vma =find_vma_prev (mm, addr, &prev_vma))) -return addr; - -/* -* new region fits between prev_vma->vm_end and -* vma->vm_start, use it: -*/ -if (addr + len <= vma->vm_start && - (!prev_vma || (addr >= prev_vma->vm_end))) { + vma =find_vma (mm, addr); +if (!vma || addr + len <= vma->vm_start) {/* remember the address as a hint for next time */ mm->cached_hole_size = largest_hole;return (mm->free_area_cache = addr); - }else { + }else if (mm->free_area_cache == vma->vm_end) {/* pull free_area_cache down to the first hole */ -if (mm->free_area_cache == vma->vm_end) { - mm->free_area_cache = vma->vm_start; - mm->cached_hole_size = largest_hole; - } + mm->free_area_cache = vma->vm_start; + mm->cached_hole_size = largest_hole; }/* remember the largest hole we saw so far */
Signature · 8 lines
1.7.7.5 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Andrew Morton
maintainerRe: [PATCH 2/5] hugetlb: drop prev_vma in hugetlb_get_unmapped_area_topdown
Quoted 4 lines · On Fri, 13 Jan 2012 19:44:53 +0800 Xiao Guangrong…
Afte call
find_vma_prev(mm, addr, &prev_vma), following condition is always true:!prev_vma || (addr >= prev_vma->vm_end)it can be happily drop
prev_vmaand usefind_vmainstead offind_vma_prev
I had to rework this patch due to 097d59106a8e4b ("vm: avoid using find_vma_prev() unnecessarily") in mainline. Can you please check my handiwork?
From: Xiao Guangrong <[email protected]>
Subject: hugetlb: drop prev_vma in hugetlb_get_unmapped_area_topdown()
After the call find_vma_prev(mm, addr, &prev_vma), the following condition is always true:
!prev_vma || (addr >= prev_vma->vm_end)
it can happily drop prev_vma and use find_vma() instead of find_vma_prev().
---
arch/x86/mm/hugetlbpage.c | 23 +++++++---------------- 1 file changed, 7 insertions(+), 16 deletions(-)
arch/x86/mm/hugetlbpage.c renamed from arch/x86/mm/hugetlbpage.c~hugetlb-drop-prev_vma-in-hugetlb_get_unmapped_area_topdown +7 -16
@@ -308,7 +308,7 @@ static unsigned long hugetlb_get_unmappe {struct hstate *h =hstate_file (file);struct mm_struct *mm = current->mm; -struct vm_area_struct *vma, *prev_vma; +struct vm_area_struct *vma;unsigned long base = mm->mmap_base, addr = addr0;unsigned long largest_hole = mm->cached_hole_size;int first_time =1 ;
@@ -334,25 +334,16 @@ try_again:* i.e. return with success: */ vma =find_vma (mm, addr); -if (!vma) -return addr; - -/* -* new region fits between prev_vma->vm_end and -* vma->vm_start, use it: -*/ - prev_vma = vma->vm_prev; -if (addr + len <= vma->vm_start && - (!prev_vma || (addr >= prev_vma->vm_end))) { +if (vma) + prev_vma = vma->vm_prev; +if (!vma || addr + len <= vma->vm_start) {/* remember the address as a hint for next time */ mm->cached_hole_size = largest_hole;return (mm->free_area_cache = addr); - }else { + }else if (mm->free_area_cache == vma->vm_end) {/* pull free_area_cache down to the first hole */ -if (mm->free_area_cache == vma->vm_end) { - mm->free_area_cache = vma->vm_start; - mm->cached_hole_size = largest_hole; - } + mm->free_area_cache = vma->vm_start; + mm->cached_hole_size = largest_hole; }/* remember the largest hole we saw so far */
_
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH 2/5] hugetlb: drop prev_vma in hugetlb_get_unmapped_area_topdown
Quoted 12 lines · Andrew Morton
Quoted 4 lines · On Fri, 13 Jan 2012 19:44:53 +0800 Xiao Guangrong…
Afte call
find_vma_prev(mm, addr, &prev_vma), following condition is always true:!prev_vma || (addr >= prev_vma->vm_end)it can be happily drop
prev_vmaand usefind_vmainstead offind_vma_prevI had to rework this patch due to
097d59106a8e4b("vm: avoid usingfind_vma_prev()unnecessarily") in mainline. Can you please check my handiwork?
It looks good to me, thanks Andrew!
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 3/5 hugetlb: try to search again if it is really needed
In reply to Xiao GuangrongSearch again only if some holes may be skipped in the first time
--- arch/x86/mm/hugetlbpage.c | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-)
arch/x86/mm/hugetlbpage.c +4 -4
@@ -309,9 +309,8 @@ static unsigned long hugetlb_get_unmapped_area_topdown(struct file *file,struct hstate *h =hstate_file (file);struct mm_struct *mm = current->mm;struct vm_area_struct *vma; -unsigned long base = mm->mmap_base, addr = addr0; +unsigned long base = mm->mmap_base, addr = addr0, start_addr;unsigned long largest_hole = mm->cached_hole_size; -int first_time =1 ;/* don't allow allocations above current base */ if (mm->free_area_cache > base)
@@ -322,6 +321,8 @@ static unsigned long hugetlb_get_unmapped_area_topdown(struct file *file, mm->free_area_cache = base; } try_again: + start_addr = mm->free_area_cache; +/* make sure it can fit in the remaining address space */ if (mm->free_area_cache < len) goto fail;
@@ -357,10 +358,9 @@ fail:* if hint left us with no space for the requested * mapping then try again: */ -if (first_time) { +if (start_addr != base) { mm->free_area_cache = base; largest_hole =0 ; - first_time =0 ; goto try_again; }/*
Signature · 8 lines
1.7.7.5 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Andrew Morton
maintainerRe: [PATCH 3/5] hugetlb: try to search again if it is really needed
Quoted 12 lines · On Fri, 13 Jan 2012 19:45:45 +0800 Xiao Guangrong…
Search again only if some holes may be skipped in the first time
Signed-off-by Xiao Guangrong <[email protected]>--- arch/x86/mm/hugetlbpage.c | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-)arch/x86/mm/hugetlbpage.c +0
-0
arch/x86/mm/hugetlbpage.c static unsigned long hugetlb_get_unmapped_area_topdown(struct file *file, · line 312
Quoted 5 lines
arch/x86/mm/hugetlbpage.c +1
-1@@ -309,9 +309,8 @@ static unsigned long hugetlb_get_unmapped_area_topdown(struct file *file,struct hstate *h =hstate_file (file);struct mm_struct *mm = current->mm;struct vm_area_struct *vma; -unsigned long base = mm->mmap_base, addr = addr0; +unsigned long base = mm->mmap_base, addr = addr0, start_addr;
grr. The multiple-definitions-per-line thing is ugly, makes for more patch conflicts and reduces opportunities to add useful comments.
arch/x86/mm/hugetlbpage.c renamed from arch/x86/mm/hugetlbpage.c~hugetlb-try-to-search-again-if-it-is-really-needed-fix +3 -1
@@ -309,7 +309,9 @@ static unsigned long hugetlb_get_unmappestruct hstate *h =hstate_file (file);struct mm_struct *mm = current->mm;struct vm_area_struct *vma; -unsigned long base = mm->mmap_base, addr = addr0, start_addr; +unsigned long base = mm->mmap_base; +unsigned long addr = addr0; +unsigned long start_addr;unsigned long largest_hole = mm->cached_hole_size;/* don't allow allocations above current base */
_
Quoted 15 lines
arch/x86/mm/hugetlbpage.c +2
-1unsigned long largest_hole = mm->cached_hole_size; -int first_time =1 ;/* don't allow allocations above current base */ if (mm->free_area_cache > base)@@ -322,6 +321,8 @@ static unsigned long hugetlb_get_unmapped_area_topdown(struct file *file, mm->free_area_cache = base; } try_again: + start_addr = mm->free_area_cache; +/* make sure it can fit in the remaining address space */ if (mm->free_area_cache < len) goto fail;
arch/x86/mm/hugetlbpage.c fail: · line 364
Quoted 9 lines
arch/x86/mm/hugetlbpage.c +1
-2@@ -357,10 +358,9 @@ fail:* if hint left us with no space for the requested * mapping then try again: */ -if (first_time) { +if (start_addr != base) { mm->free_area_cache = base; largest_hole =0 ; - first_time =0 ; goto try_again;
The code used to retry a single time. With this change the retrying is potentially infinite. What is the reason for this change? What is the potential for causing a lockup?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH 3/5] hugetlb: try to search again if it is really needed
Hi Andrew,
Thanks for your review!
Quoted 10 lines · Andrew Morton
Quoted 7 lines · On Fri, 13 Jan 2012 19:45:45 +0800 Xiao Guangrong…
Search again only if some holes may be skipped in the first time
Signed-off-by Xiao Guangrong <[email protected]>--- arch/x86/mm/hugetlbpage.c | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-)
Quoted 13 lines
arch/x86/mm/hugetlbpage.c static unsigned long hugetlb_get_unmapped_area_topdown(struct file *file, · line 312
Quoted 10 lines
arch/x86/mm/hugetlbpage.c +1
-1@@ -309,9 +309,8 @@ static unsigned long hugetlb_get_unmapped_area_topdown(struct file *file,struct hstate *h =hstate_file (file);struct mm_struct *mm = current->mm;struct vm_area_struct *vma; -unsigned long base = mm->mmap_base, addr = addr0; +unsigned long base = mm->mmap_base, addr = addr0, start_addr;grr. The multiple-definitions-per-line thing is ugly, makes for more patch conflicts and reduces opportunities to add useful comments.
Yes, thanks.
Quoted 12 lines
Quoted 12 lines
arch/x86/mm/hugetlbpage.c +3
-1try_again: + start_addr = mm->free_area_cache; +/* make sure it can fit in the remaining address space */ if (mm->free_area_cache < len) goto fail;@@ -357,10 +358,9 @@ fail:* if hint left us with no space for the requested * mapping then try again: */ -if (first_time) { +if (start_addr != base) {
Quoted 9 lines
arch/x86/mm/hugetlbpage.c fail: · line 364
Quoted 4 lines
arch/x86/mm/hugetlbpage.c +0
-1mm->free_area_cache = base; largest_hole =0 ; - first_time =0 ; goto try_again;The code used to retry a single time. With this change the retrying is potentially infinite. What is the reason for this change? What is the potential for causing a lockup?
Actually, it is not infinite, after retry, we set "mm->free_area_cache = base", "start_addr" will set to "base" in the second search, so the condition of "goto try_again" will be unsatisfied.
PATCH 4/5 mm: do not reset cached_hole_size when vma is unmapped
In reply to Xiao GuangrongIn current code, cached_hole_size is set to the maximal value if the unmapped vma is under free_area_cache, next search will search from the base addr
Actually, we can keep cached_hole_size so that if next required size is more that cached_hole_size, it can search from free_area_cache
--- mm/mmap.c | 4 +--- 1 files changed, 1 insertions(+), 3 deletions(-)
mm/mmap.c +1 -3
@@ -1423,10 +1423,8 @@ void arch_unmap_area(struct mm_struct *mm, unsigned long addr)/* * Is this a new hole at the lowest possible address? */ -if (addr >=TASK_UNMAPPED_BASE && addr < mm->free_area_cache) { +if (addr >=TASK_UNMAPPED_BASE && addr < mm->free_area_cache) mm->free_area_cache = addr; - mm->cached_hole_size = ~0UL ; - } }/*
Signature · 8 lines
1.7.7.5 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 5/5 mm: search from free_area_cache for the bigger size
In reply to Xiao GuangrongIf the required size is bigger than cached_hole_size, we would better search from free_area_cache, it is more easier to get free region, specifically for the 64 bit process whose address space is large enough
Do it just as hugetlb_get_unmapped_area_topdown() in arch/x86/mm/hugetlbpage.c
--- arch/x86/kernel/sys_x86_64.c | 34 +++++++++++++++++----------------- mm/mmap.c | 36 +++++++++++++++++++++--------------- 2 files changed, 38 insertions(+), 32 deletions(-)
arch/x86/kernel/sys_x86_64.c +17 -17
@@ -195,7 +195,7 @@ arch_get_unmapped_area_topdown(struct file *filp, const unsigned long addr0, {struct vm_area_struct *vma;struct mm_struct *mm = current->mm; -unsigned long addr = addr0; +unsigned long addr = addr0, start_addr;/* requested length too big for entire address space */ if (len >TASK_SIZE )
@@ -223,25 +223,14 @@ arch_get_unmapped_area_topdown(struct file *filp, const unsigned long addr0, mm->free_area_cache = mm->mmap_base; } +try_again:/* either no address requested or can't fit in requested address hole */ - addr = mm->free_area_cache; - -/* make sure it can fit in the remaining address space */ -if (addr > len) { -unsigned long tmp_addr =align_addr (addr - len, filp, -ALIGN_TOPDOWN ); - - vma =find_vma (mm, tmp_addr); -if (!vma || tmp_addr + len <= vma->vm_start) -/* remember the address as a hint for next time */ -return mm->free_area_cache = tmp_addr; - } - -if (mm->mmap_base < len) - goto bottomup; + start_addr = addr = mm->free_area_cache; - addr = mm->mmap_base-len; +if (addr < len) + goto fail; + addr -= len; do { addr = align_addr(addr, filp, ALIGN_TOPDOWN);
@@ -263,6 +252,17 @@ arch_get_unmapped_area_topdown(struct file *filp, const unsigned long addr0, addr = vma->vm_start-len; }while (len < vma->vm_start); +fail: +/* +* if hint left us with no space for the requested +* mapping then try again: +*/ +if (start_addr != mm->mmap_base) { + mm->free_area_cache = mm->mmap_base; + mm->cached_hole_size =0 ; + goto try_again; + } + bottomup:/* * A failed mmap() very likely causes application failure,
mm/mmap.c +21 -15
@@ -1439,7 +1439,7 @@ arch_get_unmapped_area_topdown(struct file *filp, const unsigned long addr0, {struct vm_area_struct *vma;struct mm_struct *mm = current->mm; -unsigned long addr = addr0; +unsigned long addr = addr0, start_addr;/* requested length too big for entire address space */ if (len >TASK_SIZE )
@@ -1463,22 +1463,14 @@ arch_get_unmapped_area_topdown(struct file *filp, const unsigned long addr0, mm->free_area_cache = mm->mmap_base; } +try_again:/* either no address requested or can't fit in requested address hole */ - addr = mm->free_area_cache; + start_addr = addr = mm->free_area_cache; -/* make sure it can fit in the remaining address space */ -if (addr > len) { - vma =find_vma (mm, addr-len); -if (!vma || addr <= vma->vm_start) -/* remember the address as a hint for next time */ -return (mm->free_area_cache = addr-len); - } - -if (mm->mmap_base < len) - goto bottomup; - - addr = mm->mmap_base-len; +if (addr < len) + goto fail; + addr -= len; do { /* * Lookup failure means no vma is above this address,
@@ -1498,7 +1490,21 @@ arch_get_unmapped_area_topdown(struct file *filp, const unsigned long addr0, addr = vma->vm_start-len; }while (len < vma->vm_start); -bottomup: +fail: +/* +* if hint left us with no space for the requested +* mapping then try again: +* +* Note: this is different with the case of bottomup +* which does the fully line-search, but we use find_vma +* here that causes some holes skipped. +*/ +if (start_addr != mm->mmap_base) { + mm->free_area_cache = mm->mmap_base; + mm->cached_hole_size =0 ; + goto try_again; + } +/* * A failed mmap() very likely causes application failure, * so fall back to the bottom-up function here. This scenario
Signature · 8 lines
1.7.7.5 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Andrew Morton
maintainerRe: [PATCH 5/5] mm: search from free_area_cache for the bigger size
Quoted 5 lines · On Fri, 13 Jan 2012 19:47:31 +0800 Xiao Guangrong…
If the required size is bigger than
cached_hole_size, we would better search fromfree_area_cache, it is more easier to get free region, specifically for the 64 bit process whose address space is large enoughDo it just as
hugetlb_get_unmapped_area_topdown()inarch/x86/mm/hugetlbpage.c
Can this cause additional fragmentation of the virtual address region? If so, what might be the implications of this?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH 5/5] mm: search from free_area_cache for the bigger size
Quoted 11 lines · Andrew Morton
Quoted 5 lines · On Fri, 13 Jan 2012 19:47:31 +0800 Xiao Guangrong…
If the required size is bigger than
cached_hole_size, we would better search fromfree_area_cache, it is more easier to get free region, specifically for the 64 bit process whose address space is large enoughDo it just as
hugetlb_get_unmapped_area_topdown()inarch/x86/mm/hugetlbpage.cCan this cause additional fragmentation of the virtual address region? If so, what might be the implications of this?
Hmm, i think it is not bad since we have cached_hole_size, and, this way is also used in other functions and architectures(arch_get_unmapped_area, hugetlb_get_unmapped_area_bottomup, hugetlb_get_unmapped_area_topdown......).
Quoted 3 lines · Xiao Guangrong
Using/updating
cached_hole_sizeandfree_area_cacheproperly to speedup find free region
Ping...???
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Andrew Morton
maintainerQuoted 8 lines · On Tue, 31 Jan 2012 14:05:50 +0800 Xiao Guangrong…
Quoted 3 lines · Xiao Guangrong
Using/updating
cached_hole_sizeandfree_area_cacheproperly to speedup find free regionPing...???
I'd saved the patches away, hoping that someone who works on hugetlb would comment. They didn't. We haven't heard from wli in a long time.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>