high preempt off latency in vfree path

RFC In review Joel Fernandes 2 participants 3 messages Mar 23, 2016

Subsystems (2)

Memory management vmalloc

lore

Share & tools

lore

Joel Fernandes

author

2016-03-23 01:43 UTC #0 lore raw

RFC 1/1 high preempt off latency in vfree path

Hi,

I'm seeing on my system with some real time audio requirements, I'm seeing the preemptirqsoff tracer complaining that preempt was off for 17ms in the vfree path. Since we have requirements of 8ms scheduling this seems awfully bad.

The tracer output showed __free_vmap_area was about 7300 times. Can we do better here? I have proposed 2 potential fixes here, any thoughts on which one's better?

Here's the path that blocks preempt (full latency ftrace output uploaded to http://raw.codepile.net/pile/OWNpvKkB.js)

=> preempt_count_sub
=> _raw_spin_unlock
=> __purge_vmap_area_lazy
=> free_vmap_area_noflush
=> remove_vm_area
=> __vunmap
=> vfree
=> n_tty_close
=> tty_ldisc_close.isra.1
=> tty_ldisc_kill
=> tty_ldisc_release
=> tty_release

Here are the approaches:
(1)
One is we reduce the number of lazy_max_pages (right now its around 32MB per core worth of pages).

mm/vmalloc.c +1 -1

index aa3891e..2720f4f 100644
@@ -564,7 +564,7 @@ static unsigned long lazy_max_pages(void)
 
        log = fls(num_online_cpus());
 
-       return log * (32UL * 1024 * 1024 / PAGE_SIZE);
+       return log * (8UL * 1024 * 1024 / PAGE_SIZE);
 }

(2) Second alternative approach I am thinking is to change purge_lock into a mutex and then move the vmap_area spinlock around the free_vmap_area call. Thus giving the scheduler a chance to put something else on the CPU in between free_vmap_area calls. That would look like:

mm/vmalloc.c +8 -7

index aa3891e..9565d72 100644
@@ -594,7 +594,7 @@ void set_iounmap_nonlazy(void)
 static void __purge_vmap_area_lazy(unsigned long *start, unsigned long *end,
                                        int sync, int force_flush)
 {
-       static DEFINE_SPINLOCK(purge_lock);
+       static DEFINE_MUTEX(purge_lock);
        LIST_HEAD(valist);
        struct vmap_area *va;
        struct vmap_area *n_va;

@@ -606,10 +606,10 @@ static void __purge_vmap_area_lazy(unsigned long *start, unsigned long *end,
         * the case that isn't actually used at the moment anyway.
         */
        if (!sync && !force_flush) {
-               if (!spin_trylock(&purge_lock))
+               if (!mutex_trylock(&purge_lock))
                        return;
        } else
-               spin_lock(&purge_lock);
+               mutex_lock(&purge_lock);
 
        if (sync)
                purge_fragmented_blocks_allcpus();

@@ -636,12 +636,13 @@ static void __purge_vmap_area_lazy(unsigned long *start, unsigned long *end,
                flush_tlb_kernel_range(*start, *end);
 
        if (nr) {
-               spin_lock(&vmap_area_lock);
-               list_for_each_entry_safe(va, n_va, &valist, purge_list)
+               list_for_each_entry_safe(va, n_va, &valist, purge_list) {
+                       spin_lock(&vmap_area_lock);
                        __free_vmap_area(va);
+                       spin_unlock(&vmap_area_lock);
+               }
-               spin_unlock(&vmap_area_lock);
 
        }
-       spin_unlock(&purge_lock);
+       mutex_unlock(&purge_lock);
 }

Thanks!
Joel

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Andi Kleen

2016-03-23 02:44 UTC #1 lore raw

Quoted 8 lines

(1)
One is we reduce the number of lazy_max_pages (right now its around 32MB per core worth of pages).
mm/vmalloc.c +0 -0
index aa3891e..2720f4f 100644

mm/vmalloc.c static unsigned long lazy_max_pages(void) · line 568

Quoted 6 lines

mm/vmalloc.c +1 -1

@@ -564,7 +564,7 @@ static unsigned long lazy_max_pages(void)
 
        log = fls(num_online_cpus());
 
-       return log * (32UL * 1024 * 1024 / PAGE_SIZE);
+       return log * (8UL * 1024 * 1024 / PAGE_SIZE);
 }

This seems like the right fix to me. Perhaps even make it somewhat smaller.

Even on larger systems it's probably fine because they have a lot more cores/threads these days, so it will be still sufficiently large.

-Andi

Joel Fernandes

author

2016-03-23 19:03 UTC #2 lore raw

Quoted 11 lines · Andi Kleen

Quoted 11 lines
(1)
One is we reduce the number of lazy_max_pages (right now its around 32MB per core worth of pages).
mm/vmalloc.c +0 -0
index aa3891e..2720f4f 100644
@@ -564,7 +564,7 @@ static unsigned long lazy_max_pages(void)
 
         log = fls(num_online_cpus());
 

Quoted 9 lines

mm/vmalloc.c static unsigned long lazy_max_pages(void) · line 568
Quoted 3 lines
mm/vmalloc.c +1 -1
-       return log * (32UL * 1024 * 1024 / PAGE_SIZE);
+       return log * (8UL * 1024 * 1024 / PAGE_SIZE);
  }
This seems like the right fix to me. Perhaps even make it somewhat smaller.
Even on larger systems it's probably fine because they have a lot more cores/threads these days, so it will be still sufficiently large.

Thanks Andi. I'll post a patch then.

Regards,
Joel

Quoted 2 lines

-Andi