mm/vmalloc: Move resched point into alloc_vmap_area()
Subsystems (2)
Uladzislau Rezki (Sony)
maintainerPATCH 1/1 mm/vmalloc: Move resched point into alloc_vmap_area()
Currently vm_area_alloc_pages() contains two cond_resched() points. However, the page allocator already has its own in slow path so an extra resched is not optimal because it delays the loops.
The place where a CPU time can be stolen is a VA-space search in the alloc_vmap_area(), especially if the space is really fragmented using synthetic stress tests, after a fast path falls back to a slow one.
Move a single cond_resched() there, after dropping free_vmap_area_lock in a slow path. This keeps fairness where it matters while removing redundant yields from the page-allocation path.
Signed-off-by Uladzislau Rezki (Sony) <[email protected]>
--- mm/vmalloc.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
mm/vmalloc.c +7 -2
@@ -2057,6 +2057,13 @@ static struct vmap_area *alloc_vmap_area(unsigned long size, addr =__alloc_vmap_area (&free_vmap_area_root, &free_vmap_area_list, size, align, vstart, vend);spin_unlock (&free_vmap_area_lock); + +/* +* This is not a fast path. Check if yielding is +* needed. This is the only one reschedule point +* in vmalloc() path. +*/ +cond_resched (); }trace_alloc_vmap_area (addr, size, align, vstart, vend,IS_ERR_VALUE (addr));
@@ -3622,7 +3629,6 @@ vm_area_alloc_pages(gfp_t gfp, int nid, pages + nr_allocated); nr_allocated += nr; -cond_resched ();/* * If zero or pages were obtained partly,
@@ -3664,7 +3670,6 @@ vm_area_alloc_pages(gfp_t gfp, int nid,for (i =0 ; i < (1U << order); i++) pages[nr_allocated + i] = page + i; -cond_resched (); nr_allocated +=1U << order; }
Signature · 1 line
2.47.3