debugobjects: install cpu hotplug callback
Subsystems (3)
PATCH 1/1 debugobjects: install cpu hotplug callback
From: Zqiang <[email protected]>
When a cpu going offline, we should free objects in "percpu_obj_pool" free_objs list which corresponding to this cpu.
--- include/linux/cpuhotplug.h | 1 + lib/debugobjects.c | 23 +++++++++++++++++++++++ 2 files changed, 24 insertions(+)
include/linux/cpuhotplug.h +1 -0
@@ -36,6 +36,7 @@ enum cpuhp_state {CPUHP_X86_MCE_DEAD ,CPUHP_VIRT_NET_DEAD ,CPUHP_SLUB_DEAD , +CPUHP_DEBUG_OBJ_DEAD ,CPUHP_MM_WRITEBACK_DEAD ,CPUHP_MM_VMSTAT_DEAD , CPUHP_SOFTIRQ_DEAD,
lib/debugobjects.c +23 -0
@@ -19,6 +19,7 @@#include <linux/slab.h> #include <linux/hash.h> #include <linux/kmemleak.h> +#include <linux/cpu.h> #define ODEBUG_HASH_BITS 14#define ODEBUG_HASH_SIZE (1 << ODEBUG_HASH_BITS)
@@ -433,6 +434,23 @@ static void free_object(struct debug_obj *obj) } } +#if defined(CONFIG_HOTPLUG_CPU ) +static int object_cpu_offline (unsigned int cpu) +{ +struct debug_percpu_free *percpu_pool; +struct hlist_node *tmp; +struct debug_obj *obj; + + percpu_pool =per_cpu_ptr (&percpu_obj_pool, cpu); +hlist_for_each_entry_safe (obj, tmp, &percpu_pool->free_objs, node) { +hlist_del (&obj->node); +kmem_cache_free (obj_cache, obj); + } + +return 0 ; +} +#endif +/* * We run out of memory. That means we probably have tons of objects * allocated.
@@ -1367,6 +1385,11 @@ void __init debug_objects_mem_init(void) }else debug_objects_selftest (); +#if defined(CONFIG_HOTPLUG_CPU ) +cpuhp_setup_state_nocalls (CPUHP_DEBUG_OBJ_DEAD ,"object:offline" ,NULL , + object_cpu_offline); +#endif +/* * Increase the thresholds for allocating and freeing objects * according to the number of possible CPUs available in the system.
Signature · 1 line
2.17.1
________________________________________
发件人: [email protected] <[email protected]> 代表 [email protected] <[email protected]> 发送时间: 2020年8月20日 11:24
收件人: [email protected]; [email protected]; [email protected]
抄送: [email protected]
主题: [PATCH] debugobjects: install cpu hotplug callback
From: Zqiang <[email protected]>
When a cpu going offline, we should free objects in "percpu_obj_pool" free_objs list which corresponding to this cpu.
--- include/linux/cpuhotplug.h | 1 + lib/debugobjects.c | 23 +++++++++++++++++++++++ 2 files changed, 24 insertions(+)
include/linux/cpuhotplug.h +1 -0
@@ -36,6 +36,7 @@ enum cpuhp_state {CPUHP_X86_MCE_DEAD ,CPUHP_VIRT_NET_DEAD ,CPUHP_SLUB_DEAD , +CPUHP_DEBUG_OBJ_DEAD ,CPUHP_MM_WRITEBACK_DEAD ,CPUHP_MM_VMSTAT_DEAD , CPUHP_SOFTIRQ_DEAD,
lib/debugobjects.c +23 -0
@@ -19,6 +19,7 @@#include <linux/slab.h> #include <linux/hash.h> #include <linux/kmemleak.h> +#include <linux/cpu.h> #define ODEBUG_HASH_BITS 14#define ODEBUG_HASH_SIZE (1 << ODEBUG_HASH_BITS)
@@ -433,6 +434,23 @@ static void free_object(struct debug_obj *obj) } } +#if defined(CONFIG_HOTPLUG_CPU ) +static int object_cpu_offline (unsigned int cpu) +{ +struct debug_percpu_free *percpu_pool; +struct hlist_node *tmp; +struct debug_obj *obj; + + percpu_pool =per_cpu_ptr (&percpu_obj_pool, cpu); +hlist_for_each_entry_safe (obj, tmp, &percpu_pool->free_objs, node) { +hlist_del (&obj->node); +kmem_cache_free (obj_cache, obj); + } + +return 0 ; +} +#endif +/* * We run out of memory. That means we probably have tons of objects * allocated.
@@ -1367,6 +1385,11 @@ void __init debug_objects_mem_init(void) }else debug_objects_selftest (); +#if defined(CONFIG_HOTPLUG_CPU ) +cpuhp_setup_state_nocalls (CPUHP_DEBUG_OBJ_DEAD ,"object:offline" ,NULL , + object_cpu_offline); +#endif +/* * Increase the thresholds for allocating and freeing objects * according to the number of possible CPUs available in the system.
--
2.17.1
Quoted 11 lines · Qiang
________________________________________
发件人: [email protected] <[email protected]> 代表 [email protected] <[email protected]> 发送时间: 2020年8月20日 11:24
收件人: [email protected]; [email protected]; [email protected]
抄送: [email protected]
主题: [PATCH] debugobjects: install cpu hotplug callbackFrom: Zqiang <[email protected]>
When a cpu going offline, we should free objects in "
percpu_obj_pool"free_objslist which corresponding to this cpu.
The percpu free object pool is supposed to be accessed only by that particular cpu without any lock. Trying to access it from another cpu can cause a race condition unless one can make sure that the offline cpu won't become online in the mean time. There shouldn't be too many free objects in the percpu pool. Is it worth the effort to free them?
Cheers,
Longman
Quoted 8 lines · Waiman Long
Quoted 7 lines · Qiang
________________________________________
发件人: [email protected] <[email protected]> 代表 [email protected] <[email protected]> 发送时间: 2020年8月20日 11:24
收件人: [email protected]; [email protected]; [email protected]
抄送: [email protected]
主题: [PATCH] debugobjects: install cpu hotplug callback
Quoted 6 lines
Quoted 4 lines
From: Zqiang <[email protected]>
When a cpu going offline, we should free objects in "
percpu_obj_pool"free_objslist which corresponding to this cpu.The percpu free object pool is supposed to be accessed only by that particular cpu without any lock. Trying to access it from another cpu can cause a race condition unless one can make sure that the offline cpu won't become online in the mean time. There shouldn't be too many free objects in the percpu pool. Is it worth the effort to free them?
Or if you can make the to-be-offlined cpu free the debugobjs before it is offlined. That will work too.
Cheers,
Longman
Quoted 9 lines · Waiman Long
Quoted 3 lines · Qiang
When a cpu going offline, we should free objects in "
percpu_obj_pool"free_objslist which corresponding to this cpu.The percpu free object pool is supposed to be accessed only by that particular cpu without any lock. Trying to access it from another cpu can cause a race condition unless one can make sure that the offline cpu won't become online in the mean time.
It is actually safe because CPU hotplug is globally serialized and there is no way that an offline CPU will come back from death valley magically. If such a zombie ever surfaces then we have surely more serious problems than accessing that pool :)
Quoted 2 lines
There shouldn't be too many free objects in the percpu pool. Is it worth the effort to free them?
That's a really good question nevertheless. The only case where this ever matters is physical hotplug. All other CPU hotplug stuff is temporarily or in case of a late (post boottime) SMT disable it's going to be a handful of free objects on that pool. As debugobjects is as the name says a debug facility the benefit is questionable unless there is a good reason to do so.
Thanks,
tglx
Quoted 4 lines · Thomas Gleixner
Quoted 3 lines · Waiman Long
Quoted 2 lines · Qiang
When a cpu going offline, we should free objects in "
percpu_obj_pool"free_objslist which corresponding to this cpu.
Quoted 8 lines
Quoted 4 lines
The percpu free object pool is supposed to be accessed only by that particular cpu without any lock. Trying to access it from another cpu can cause a race condition unless one can make sure that the offline cpu won't become online in the mean time.
It is actually safe because CPU hotplug is globally serialized and there is no way that an offline CPU will come back from death valley magically. If such a zombie ever surfaces then we have surely more serious problems than accessing that pool :)
Thanks for the clarification. I haven't looked into where the callback functions are being called so I am not 100% sure.
Cheers,
Longman
________________________________________
发件人: [email protected] <[email protected]> 代表 Thomas Gleixner <[email protected]> 发送时间: 2020年8月26日 7:53
收件人: Waiman Long; Zhang, Qiang; [email protected]
抄送: [email protected]; [email protected]
主题: Re: 回复: [PATCH] debugobjects: install cpu hotplug callback
Quoted 18 lines · Waiman Long
Quoted 3 lines · Qiang
When a cpu going offline, we should free objects in "
percpu_obj_pool"free_objslist which corresponding to this cpu.The percpu free object pool is supposed to be accessed only by that particular cpu without any lock. Trying to access it from another cpu can cause a race condition unless one can make sure that the offline cpu won't become online in the mean time.
It is actually safe because CPU hotplug is globally serialized and there is no way that an offline CPU will come back from death valley magically. If such a zombie ever surfaces then we have surely more serious problems than accessing that pool :)
There shouldn't be too many free objects in the percpu pool. Is it worth the effort to free them?
Quoted 6 lines
That's a really good question nevertheless. The only case where this ever matters is physical hotplug. All other CPU hotplug stuff is temporarily or in case of a late (post boottime) SMT disable it's going to be a handful of free objects on that pool. As debugobjects is as the name says a debug facility the benefit is questionable unless there is a good reason to do so.
I don't know there may not be too many objects in the percpu pool,
but that doesn't mean they no need to be free, a CPU may never be online after it is offline. some objects in percpu pool is never free.
Quoted 3 lines
Thanks,
tglx
Quoted 6 lines · Qiang Zhang
________________________________________
发件人: [email protected] <[email protected]> 代表 Thomas Gleixner <[email protected]> 发送时间: 2020年8月26日 7:53
收件人: Waiman Long; Zhang, Qiang; [email protected]
抄送: [email protected]; [email protected]
主题: Re: 回复: [PATCH] debugobjects: install cpu hotplug callback
Can you please fix your mail client not to copy the headers into the mail body? The headers are already in the mail itself.
Quoted 1 line · Waiman Long
Something like this is completely sufficient.
Quoted 11 lines
Quoted 6 lines
That's a really good question nevertheless. The only case where this ever matters is physical hotplug. All other CPU hotplug stuff is temporarily or in case of a late (post boottime) SMT disable it's going to be a handful of free objects on that pool. As debugobjects is as the name says a debug facility the benefit is questionable unless there is a good reason to do so.
I don't know there may not be too many objects in the percpu pool, but that doesn't mean they no need to be free, a CPU may never be online after it is offline. some objects in percpu pool is never free.
And this matters because? Because your fully debug enabled kernel will have an uptime of years after disabling the CPU?
That said, I'm not opposed against this patch, but
'we should free objects'
is not a convincing technical argument for doing this. If we want to have that then please add proper technical arguments to the changelog.
Thanks,
tglx