Skip to contentSkip to conversation

fork: free vmapped stacks in cache when cpus are offline

PATCH Superseded Hoeun Ryu 3 participants 5 messages Feb 11, 2017 – Feb 12, 2017

Subsystems (3)

v5 of 7 Superseded Latest is v7 →

lore
Share & tools
lore

Hoeun Ryu

author
#0 lore raw

PATCH v5 1/1 fork: free vmapped stacks in cache when cpus are offline

Using virtually mapped stack, kernel stacks are allocated via vmalloc.
In the current implementation, two stacks per cpu can be cached when tasks are freed and the cached stacks are used again in task duplications. but the cached stacks may remain unfreed even when cpus are offline.
By adding a cpu hotplug callback to free the cached stacks when a cpu
goes offline, the pages of the cached stacks are not wasted.

---
v5:
- wrap cpuhp_setup_state() in a new function, vm_stack_cache_init() which actually do nothing when !CONFIG_VMAP_STACK
- add __may_unused to free_vm_stack_cache()
v4:
- use CPUHP_BP_PREPARE_DYN state for cpuhp setup
- fix minor coding style
v3:
- fix misuse of per-cpu api
- fix location of function definition within CONFIG_VMAP_STACK
v2:
- remove cpuhp callback for startup, only teardown callback is installed.

 kernel/fork.c | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)
kernel/fork.c +28 -0
index 937ba59..86a1110 100644
@@ -168,6 +168,32 @@ void __weak arch_release_thread_stack(unsigned long *stack)
  */
 #define NR_CACHED_STACKS 2
 static DEFINE_PER_CPU(struct vm_struct *, cached_stacks[NR_CACHED_STACKS]);
+
+static int __maybe_unused free_vm_stack_cache(unsigned int cpu)
+{
+	struct vm_struct **cached_vm_stacks = per_cpu_ptr(cached_stacks, cpu);
+	int i;
+
+	for (i = 0; i < NR_CACHED_STACKS; i++) {
+		struct vm_struct *vm_stack = cached_vm_stacks[i];
+
+		if (!vm_stack)
+			continue;
+
+		vfree(vm_stack->addr);
+		cached_vm_stacks[i] = NULL;
+	}
+
+	return 0;
+}
+
+static inline int vm_stack_cache_init(void)
+{
+	return cpuhp_setup_state(CPUHP_BP_PREPARE_DYN, "fork:vm_stack_cache",
+				 NULL, free_vm_stack_cache);
+}
+#else
+static __always_inline int vm_stack_cache_init(void) { return 0; }
 #endif
 
 static unsigned long *alloc_thread_stack_node(struct task_struct *tsk, int node)
@@ -456,6 +482,8 @@ void __init fork_init(void)
 	for (i = 0; i < UCOUNT_COUNTS; i++) {
 		init_user_ns.ucount_max[i] = max_threads/2;
 	}
+
+	vm_stack_cache_init();
 }
 
 int __weak arch_dup_task_struct(struct task_struct *dst,
Signature · 1 line
2.7.4
#1 lore raw
url: https://github.com/0day-ci/linux/commits/Hoeun-Ryu/fork-free-vmapped-stacks-in-cache-when-cpus-are-offline/20170211-183401 config: ia64-allmodconfig (attached as .config) compiler…

Hi Hoeun,

[auto build test ERROR on linus/master]
[also build test ERROR on v4.10-rc7 next-20170210]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url: https://github.com/0day-ci/linux/commits/Hoeun-Ryu/fork-free-vmapped-stacks-in-cache-when-cpus-are-offline/20170211-183401
config: ia64-allmodconfig (attached as .config)
compiler: ia64-linux-gcc (GCC) 6.2.0
reproduce:
wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
make.cross ARCH=ia64

All errors (new ones prefixed by >>):

kernel/fork.c: In function 'fork_init':

>> kernel/fork.c:483:2: error: implicit declaration of function 'vm_stack_cache_init' [-Werror=implicit-function-declaration]
     vm_stack_cache_init();
     ^~~~~~~~~~~~~~~~~~~

cc1: some warnings being treated as errors

vim +/vm_stack_cache_init +483 kernel/fork.c

477 init_task.signal->rlim[RLIMIT_NPROC];
478
479 for (i = 0; i < UCOUNT_COUNTS; i++) {
480 init_user_ns.ucount_max[i] = max_threads/2;
481 }
482

 > 483		vm_stack_cache_init();

484 }
485
486 int __weak arch_dup_task_struct(struct task_struct *dst,

---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation

#2 lore raw
Quoted 15 lines · kbuild test robot

Hi Hoeun,

[auto build test ERROR on linus/master]
[also build test ERROR on v4.10-rc7 next-20170210]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url: https://github.com/0day-ci/linux/commits/Hoeun-Ryu/fork-free-vmapped-stacks-in-cache-when-cpus-are-offline/20170211-183401
config: ia64-allmodconfig (attached as .config)
compiler: ia64-linux-gcc (GCC) 6.2.0
reproduce:
wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
make.cross ARCH=ia64

Quoted 6 lines

All errors (new ones prefixed by >>):

kernel/fork.c: In function 'fork_init':

Quoted 1 line
Quoted 1 line

kernel/fork.c:483:2: error: implicit declaration of function 'vm_stack_cache_init' [-Werror=implicit-function-declaration]

     vm_stack_cache_init();
     ^~~~~~~~~~~~~~~~~~~

Right. This needs to move to a different place as that part is nested inside

#ifndef CONFIG_ARCH_THREAD_STACK_ALLOCATOR

That's an horrible ifdef maze in that file.

Thanks,

tglx

author
#3 lore raw
Quoted 24 lines · Thomas Gleixner
Quoted 22 lines · kbuild test robot

Hi Hoeun,

[auto build test ERROR on linus/master]
[also build test ERROR on v4.10-rc7 next-20170210]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url: https://github.com/0day-ci/linux/commits/Hoeun-Ryu/fork-free-vmapped-stacks-in-cache-when-cpus-are-offline/20170211-183401
config: ia64-allmodconfig (attached as .config)
compiler: ia64-linux-gcc (GCC) 6.2.0
reproduce:
wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
make.cross ARCH=ia64

All errors (new ones prefixed by >>):

kernel/fork.c: In function 'fork_init':

Quoted 1 line
Quoted 1 line

kernel/fork.c:483:2: error: implicit declaration of function 'vm_stack_cache_init' [-Werror=implicit-function-declaration]

Quoted 9 lines
Quoted 2 lines
    vm_stack_cache_init();
    ^~~~~~~~~~~~~~~~~~~

Right. This needs to move to a different place as that part is nested inside

#ifndef CONFIG_ARCH_THREAD_STACK_ALLOCATOR

That's an horrible ifdef maze in that file.

I think we should rollback to v4. Do you agree?

Quoted 3 lines

Thanks,

tglx

#4 lore raw
Quoted 27 lines · Hoeun Ryu
Quoted 27 lines · Thomas Gleixner
Quoted 24 lines · kbuild test robot

Hi Hoeun,

[auto build test ERROR on linus/master]
[also build test ERROR on v4.10-rc7 next-20170210]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url: https://github.com/0day-ci/linux/commits/Hoeun-Ryu/fork-free-vmapped-stacks-in-cache-when-cpus-are-offline/20170211-183401
config: ia64-allmodconfig (attached as .config)
compiler: ia64-linux-gcc (GCC) 6.2.0
reproduce:
wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
make.cross ARCH=ia64

All errors (new ones prefixed by >>):

kernel/fork.c: In function 'fork_init':

Quoted 1 line
Quoted 1 line

kernel/fork.c:483:2: error: implicit declaration of function 'vm_stack_cache_init' [-Werror=implicit-function-declaration]

    vm_stack_cache_init();
    ^~~~~~~~~~~~~~~~~~~
Quoted 8 lines
Quoted 6 lines

Right. This needs to move to a different place as that part is nested inside

#ifndef CONFIG_ARCH_THREAD_STACK_ALLOCATOR

That's an horrible ifdef maze in that file.

I think we should rollback to v4. Do you agree?

Sigh. Yes.