fastboot: keep at least one thread per cpu during boot
PATCH 1/1 fastboot: keep at least one thread per cpu during boot
Async threads are created and destroyed depending on the number of jobs in queue. It means that several async threads can be created for a specific batch of work, then the threads will die after the completion of this batch, but they could be needed just after this completion for another batch of work.
During the boot, such repetitive thread creations can be wasteful, that's why this patch proposes to keep at least one thread per cpu (if they already have been created once). Such a threshold of threads kept alive will prevent from a part of the thread creation overhead.
This threshold will be dropped one the system_state switches from SYSTEM_BOOTING to SYSTEM_RUNNING.
Note:
_ If this patch is accepted, I will try to extend it to modules loading on boot _ One thread per cpu could sound a bit arbitrary here. Actually this is compromize between memory saving (if we just created lots of async thread for a large batch of jobs) and task creation overhead.
--- include/linux/async.h | 2 + init/main.c | 1 + kernel/async.c | 52 +++++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 53 insertions(+), 2 deletions(-)
include/linux/async.h +2 -0
@@ -25,3 +25,5 @@ extern void async_synchronize_cookie(async_cookie_t cookie);extern void async_synchronize_cookie_domain (async_cookie_t cookie,struct list_head *list); +extern void async_finish_boot (void ); +
init/main.c +1 -0
@@ -806,6 +806,7 @@ static noinline int init_post(void)unlock_kernel ();mark_rodata_ro (); system_state =SYSTEM_RUNNING ; +async_finish_boot ();numa_default_policy ();if (sys_open ((const char __user *)"/dev/console" ,O_RDWR ,0 ) <0 )
kernel/async.c +50 -2
70 lines not on this page · raw message
Quoted 11 lines · On Mon, 9 Feb 2009 04:48:27 +0100 Frederic Weisbecker <[email protected]>…
Async threads are created and destroyed depending on the number of jobs in queue. It means that several async threads can be created for a specific batch of work, then the threads will die after the completion of this batch, but they could be needed just after this completion for another batch of work. During the boot, such repetitive thread creations can be wasteful, that's why this patch proposes to keep at least one thread per cpu (if they already have been created once). Such a threshold of threads kept alive will prevent from a part of the thread creation overhead. This threshold will be dropped one the
system_stateswitches fromSYSTEM_BOOTINGtoSYSTEM_RUNNING.
I'm not very fond of this to be honest;
at least during boot there's enough activity, and the time is so short (that's the point of the parallel stuff!) that this will not kick in to make a difference; specifically, every boot I've seen the number of threads is highest near the end, and also the total kernel boot time is below 1.5 seconds or so, not long enough for the threads to die.
Creating a thread is *CHEAP*. Really really cheap. You can do 100 thousands/second on even a modest CPU. If you have a high frequency of events, you don't want this, sure, and that is why there is a one second delay to give opportunity for reuse... but really....
Now, if async function calls get used more, I can see the point of always keeping one thread alive, just for both performance and VM low memory issues; but that's not what your patch is doing.
Signature · 3 lines
Arjan van de Ven Intel Open Source Technology Centre For development, discussion and tips for power savings, visit http://www.lesswatts.org
Quoted 22 lines · Arjan van de Ven
Quoted 11 lines · On Mon, 9 Feb 2009 04:48:27 +0100 Frederic Weisbecker <[email protected]>…
Async threads are created and destroyed depending on the number of jobs in queue. It means that several async threads can be created for a specific batch of work, then the threads will die after the completion of this batch, but they could be needed just after this completion for another batch of work. During the boot, such repetitive thread creations can be wasteful, that's why this patch proposes to keep at least one thread per cpu (if they already have been created once). Such a threshold of threads kept alive will prevent from a part of the thread creation overhead. This threshold will be dropped one the
system_stateswitches fromSYSTEM_BOOTINGtoSYSTEM_RUNNING.I'm not very fond of this to be honest;
at least during boot there's enough activity, and the time is so short (that's the point of the parallel stuff!) that this will not kick in to make a difference; specifically, every boot I've seen the number of threads is highest near the end, and also the total kernel boot time is below 1.5 seconds or so, not long enough for the threads to die.
Quoted 4 lines
Creating a thread is *CHEAP*. Really really cheap. You can do 100 thousands/second on even a modest CPU. If you have a high frequency of events, you don't want this, sure, and that is why there is a one second delay to give opportunity for reuse... but really....
Agreed.
Quoted 5 lines
Now, if async function calls get used more, I can see the point of always keeping one thread alive, just for both performance and VM low memory issues; but that's not what your patch is doing.
I'd argue that the ability to _schedule_ async stuff without memory allocation would help more with low memory situations - after all, the work has been scheduled for later.
(For that, I have hacked up the following completely untested patch, but I'm not yet completely happy with it.)
--- include/linux/async.h | 19 +++++++++ kernel/async.c | 105 +++++++++++++++++++++++++++++++++----------------- 2 files changed, 89 insertions(+), 35 deletions(-)
include/linux/async.h +19 -0
@@ -16,9 +16,28 @@typedef u64 async_cookie_t ;typedef void (async_func_ptr ) (void *data,async_cookie_t cookie); +/** +* struct async_entry - entry for asynchronous scheduling +* @list: anchor for internal lists +* @cookie: cookie for checkpointing +* @func: asynchronous function to execute +* @data: data to pass to the function +* @running: synchronization domain to use +* @persistent: 1 if the entry must not be deleted by the core +*/ +struct async_entry { +struct list_head list; +async_cookie_t cookie; +async_func_ptr *func; +void *data; +struct list_head *running; +int *persistent; +}; +extern async_cookie_t async_schedule (async_func_ptr *ptr,void *data);extern async_cookie_t async_schedule_domain (async_func_ptr *ptr,void *data,struct list_head *list); +extern async_cookie_t async_schedule_prealloc (struct async_entry *entry);extern void async_synchronize_full (void );extern void async_synchronize_full_domain (struct list_head *list);extern void async_synchronize_cookie (async_cookie_t cookie);
kernel/async.c +70 -35
149 lines not on this page · raw message
Quoted 15 lines · Arjan van de Ven
Quoted 11 lines · On Mon, 9 Feb 2009 04:48:27 +0100 Frederic Weisbecker <[email protected]>…
Async threads are created and destroyed depending on the number of jobs in queue. It means that several async threads can be created for a specific batch of work, then the threads will die after the completion of this batch, but they could be needed just after this completion for another batch of work. During the boot, such repetitive thread creations can be wasteful, that's why this patch proposes to keep at least one thread per cpu (if they already have been created once). Such a threshold of threads kept alive will prevent from a part of the thread creation overhead. This threshold will be dropped one the
system_stateswitches fromSYSTEM_BOOTINGtoSYSTEM_RUNNING.
Quoted 6 lines
I'm not very fond of this to be honest;
at least during boot there's enough activity, and the time is so short (that's the point of the parallel stuff!) that this will not kick in to make a difference; specifically, every boot I've seen the number of threads is highest near the end, and also the total kernel boot time is below 1.5 seconds or so, not long enough for the threads to die.
My boot takes more time (about 5 seconds before modules loading).
Quoted 4 lines
Creating a thread is *CHEAP*. Really really cheap. You can do 100 thousands/second on even a modest CPU. If you have a high frequency of events, you don't want this, sure, and that is why there is a one second delay to give opportunity for reuse... but really....
Ok. And that's a problem with my patch. I did not have a suitable testcase to produce a relevant benchmark: the aync insertions were to close in time to capture something interesting.
If I saw the result of a testcase, I would have seen probably no different :-)
I guess you're right, this would have added new code to maintain for only micro optimizations...
Quoted 4 lines
Now, if async function calls get used more, I can see the point of always keeping one thread alive, just for both performance and VM low memory issues; but that's not what your patch is doing.
Ok.
Perhaps the testcase would be suitable under embedeed systems. I will perhaps test one day :-)
Thanks!
Quoted 4 lines
Signature · 3 lines
Arjan van de Ven Intel Open Source Technology Centre For development, discussion and tips for power savings, visit http://www.lesswatts.org