mm/hugetlb: Fix calculation of adjust_range_if_pmd_sharing_possible
Subsystems (2)
PATCH 1/1 mm/hugetlb: Fix calculation of adjust_range_if_pmd_sharing_possible
This is found by code observation only.
Firstly, the worst case scenario should assume the whole range was covered by pmd sharing. The old algorithm might not work as expected for ranges like (1g-2m, 1g+2m), where the adjusted range should be (0, 1g+2m) but the expected range should be (0, 2g).
Since at it, remove the loop since it should not be required. With that, the new code should be faster too when the invalidating range is huge.
CC Andrea Arcangeli <[email protected]>
CC Mike Kravetz <[email protected]>
CC Andrew Morton <[email protected]>
Signed-off-by Peter Xu <[email protected]>
--- mm/hugetlb.c | 27 +++++++++++++-------------- 1 file changed, 13 insertions(+), 14 deletions(-)
mm/hugetlb.c +13 -14
@@ -43,6 +43,9 @@#include <linux/page_owner.h> #include "internal.h" +#define MAX (a,b) (((a)>(b))?(a):(b)) +#define MIN (a,b) (((a)<(b))?(a):(b)) +int hugetlb_max_hstate__read_mostly ;unsigned int default_hstate_idx;struct hstate hstates[HUGE_MAX_HSTATE ];
@@ -5321,25 +5324,21 @@ static bool vma_shareable(struct vm_area_struct *vma, unsigned long addr)void adjust_range_if_pmd_sharing_possible (struct vm_area_struct *vma,unsigned long *start,unsigned long *end) { -unsigned long check_addr; +unsigned long a_start, a_end;if (!(vma->vm_flags &VM_MAYSHARE ))return ; -for (check_addr = *start; check_addr < *end; check_addr +=PUD_SIZE ) { -unsigned long a_start = check_addr &PUD_MASK ; -unsigned long a_end = a_start +PUD_SIZE ; +/* Extend the range to be PUD aligned for a worst case scenario */ + a_start =ALIGN_DOWN (*start,PUD_SIZE ); + a_end =ALIGN (*end,PUD_SIZE ); -/* -* If sharing is possible, adjust start/end if necessary. -*/ -if (range_in_vma (vma, a_start, a_end)) { -if (a_start < *start) - *start = a_start; -if (a_end > *end) - *end = a_end; - } - } +/* +* Intersect the range with the vma range, since pmd sharing won't be +* across vma after all +*/ + *start =MAX (vma->vm_start, a_start); + *end =MIN (vma->vm_end, a_end); }/*
Signature · 1 line
2.26.2
mm/hugetlb.c line 47
Quoted 2 lines · Peter Xu
mm/hugetlb.c +2
-0+#define MAX (a,b) (((a)>(b))?(a):(b)) +#define MIN (a,b) (((a)<(b))?(a):(b))
What's wrong with max() and min() defined in kernel.h?
Quoted 5 lines · Matthew Wilcox
mm/hugetlb.c line 47
Quoted 2 lines · Peter Xu
mm/hugetlb.c +2
-0+#define MAX (a,b) (((a)>(b))?(a):(b)) +#define MIN (a,b) (((a)<(b))?(a):(b))What's wrong with
max()andmin()defined inkernel.h?
Oops.. Posting another one... Thanks.
Signature · 1 line
Peter Xu