Revised locking for taskstats interface
Subsystems (3)
PATCH 1/1 Revised locking for taskstats interface
Convert locking used within taskstats interface and delay accounting code to be more fine-grained.
Dynamic allocation of the delays data structure led to exit race conditions that were being fixed through use of a global mutex at the taskstats interface level. The same mutex was also being used for protecting per-task delay structure allocation. Together, these were causing higher contention and unnecessary serialization.
This patch switches to a per-task locking for protecting tsk->delays and eliminates global locking within the taskstats interface.
Results collected by Jay Lan ([email protected]) from running a test with rapid forks/exits shows substantial improvement in system time. For a benchmark that only forks+exits 1000 threads and runs for 5000 iterations, the reduction seen are as follows:
base +patch %improvement
user 0.06 0.07 -16%
system 1.34 0.86 35%
elapsed 622 470 24%
---
include/linux/sched.h | 1 + kernel/delayacct.c | 18 ++++++++++++++++-- kernel/taskstats.c | 7 ------- 3 files changed, 17 insertions(+), 9 deletions(-)
include/linux/sched.h +1 -0
@@ -933,6 +933,7 @@ struct task_struct {*/ struct pipe_inode_info *splice_pipe;#ifdef CONFIG_TASK_DELAY_ACCT +spinlock_t delays_lock;struct task_delay_info *delays;#endif };
kernel/delayacct.c +16 -2
@@ -41,6 +41,10 @@ void delayacct_init(void)void __delayacct_tsk_init (struct task_struct *tsk) { +spin_lock_init (&tsk->delays_lock); +/* No need to acquire tsk->delays_lock for allocation here unless +__delayacct_tsk_init called after tsk is attached to tasklist +*/ tsk->delays =kmem_cache_zalloc (delayacct_cache,SLAB_KERNEL );if (tsk->delays)spin_lock_init (&tsk->delays->lock);
@@ -49,9 +53,9 @@ void __delayacct_tsk_init(struct task_stvoid __delayacct_tsk_exit (struct task_struct *tsk) {struct task_delay_info *delays = tsk->delays; -mutex_lock (&taskstats_exit_mutex); +spin_lock (&tsk->delays_lock); tsk->delays =NULL ; -mutex_unlock (&taskstats_exit_mutex); +spin_unlock (&tsk->delays_lock);kmem_cache_free (delayacct_cache, delays); }
@@ -114,6 +118,14 @@ int __delayacct_add_tsk(struct taskstatsstruct timespec ts;unsigned long t1,t2,t3; +spin_lock (&tsk->delays_lock); + +/* Though tsk->delays accessed later, early exit avoids +* unnecessary returning of other data +*/ +if (!tsk->delays) + goto done; + tmp = (s64 )d->cpu_run_real_total;cputime_to_timespec (tsk->utime + tsk->stime, &ts); tmp +=timespec_to_ns (&ts);
@@ -148,6 +160,8 @@ int __delayacct_add_tsk(struct taskstats d->swapin_count += tsk->delays->swapin_count;spin_unlock (&tsk->delays->lock); +done: +spin_unlock (&tsk->delays_lock);return 0 ; }
kernel/taskstats.c +0 -7
@@ -25,7 +25,6 @@ static DEFINE_PER_CPU(__u32, taskstats_seqnum) = { 0 };static int family_registered =0 ;kmem_cache_t *taskstats_cache; -DEFINE_MUTEX (taskstats_exit_mutex);static struct genl_family family = { .id =GENL_ID_GENERATE ,
@@ -193,7 +192,6 @@ static int taskstats_send_stats(struct sif (rc <0 )return rc; -mutex_lock (&taskstats_exit_mutex);if (info->attrs[TASKSTATS_CMD_ATTR_PID ]) {u32 pid =nla_get_u32 (info->attrs[TASKSTATS_CMD_ATTR_PID ]); rc =fill_pid (pid,NULL , &stats);
@@ -219,7 +217,6 @@ static int taskstats_send_stats(struct s goto err; } -mutex_unlock (&taskstats_exit_mutex);nla_nest_end (rep_skb, na);return send_reply (rep_skb, info->snd_pid,TASKSTATS_MSG_UNICAST );
@@ -228,7 +225,6 @@ nla_put_failure:return genlmsg_cancel (rep_skb, reply); err:nlmsg_free (rep_skb); -mutex_unlock (&taskstats_exit_mutex);return rc; }
@@ -246,8 +242,6 @@ void taskstats_exit_send(struct task_strif (!family_registered || !tidstats)return ; -mutex_lock (&taskstats_exit_mutex); - is_thread_group = !thread_group_empty (tsk); rc =0 ;
@@ -305,7 +299,6 @@ nla_put_failure: err_skb:nlmsg_free (rep_skb); ret: -mutex_unlock (&taskstats_exit_mutex);return ; }
Quoted 3 lines · Shailabh Nagar
Convert locking used within taskstats interface and delay accounting code to be more fine-grained.
I ran tests without locking and with per-TG processing disabled.
The tests completed without a panic without the lock. So, the protection we need is not on per-task delayacct, but when thread group processing needs to access other tasks.
A lock is needed for TG, not for delayacct.
However, the tests data showed no impact on performance due to the lock itself though. I would not argue for a cleaner design on this if 'locking only on TG' is difficult to implement.
Regards,
- jay
Quoted 145 lines
Dynamic allocation of the delays data structure led to exit race conditions that were being fixed through use of a global mutex at the taskstats interface level. The same mutex was also being used for protecting per-task delay structure allocation. Together, these were causing higher contention and unnecessary serialization.
This patch switches to a per-task locking for protecting
tsk->delaysand eliminates global locking within the taskstats interface.Results collected by Jay Lan ([email protected]) from running a test with rapid forks/exits shows substantial improvement in system time. For a benchmark that only forks+exits 1000 threads and runs for 5000 iterations, the reduction seen are as follows:
base +patch %improvement
user 0.06 0.07 -16%
system 1.34 0.86 35%
elapsed 622 470 24%Signed-off-by Shailabh Nagar <[email protected]>---
include/linux/sched.h | 1 + kernel/delayacct.c | 18 ++++++++++++++++-- kernel/taskstats.c | 7 ------- 3 files changed, 17 insertions(+), 9 deletions(-)Index:
linux-2.6.17/include/linux/sched.h
===================================================================include/linux/sched.h +1
-0@@ -933,6 +933,7 @@ struct task_struct {*/ struct pipe_inode_info *splice_pipe;#ifdef CONFIG_TASK_DELAY_ACCT +spinlock_t delays_lock;struct task_delay_info *delays;#endif };Index:
linux-2.6.17/kernel/delayacct.c
===================================================================kernel/delayacct.c +16
-2@@ -41,6 +41,10 @@ void delayacct_init(void)void __delayacct_tsk_init (struct task_struct *tsk) { +spin_lock_init (&tsk->delays_lock); +/* No need to acquire tsk->delays_lock for allocation here unless +__delayacct_tsk_init called after tsk is attached to tasklist +*/ tsk->delays =kmem_cache_zalloc (delayacct_cache,SLAB_KERNEL );if (tsk->delays)spin_lock_init (&tsk->delays->lock);@@ -49,9 +53,9 @@ void __delayacct_tsk_init(struct task_stvoid __delayacct_tsk_exit (struct task_struct *tsk) {struct task_delay_info *delays = tsk->delays; -mutex_lock (&taskstats_exit_mutex); +spin_lock (&tsk->delays_lock); tsk->delays =NULL ; -mutex_unlock (&taskstats_exit_mutex); +spin_unlock (&tsk->delays_lock);kmem_cache_free (delayacct_cache, delays); }@@ -114,6 +118,14 @@ int __delayacct_add_tsk(struct taskstatsstruct timespec ts;unsigned long t1,t2,t3; +spin_lock (&tsk->delays_lock); + +/* Though tsk->delays accessed later, early exit avoids +* unnecessary returning of other data +*/ +if (!tsk->delays) + goto done; + tmp = (s64 )d->cpu_run_real_total;cputime_to_timespec (tsk->utime + tsk->stime, &ts); tmp +=timespec_to_ns (&ts);@@ -148,6 +160,8 @@ int __delayacct_add_tsk(struct taskstats d->swapin_count += tsk->delays->swapin_count;spin_unlock (&tsk->delays->lock); +done: +spin_unlock (&tsk->delays_lock);return 0 ; }Index:
linux-2.6.17/kernel/taskstats.c
===================================================================kernel/taskstats.c +0
-3@@ -25,7 +25,6 @@ static DEFINE_PER_CPU(__u32, taskstats_seqnum) = { 0 };static int family_registered =0 ;kmem_cache_t *taskstats_cache; -DEFINE_MUTEX (taskstats_exit_mutex);static struct genl_family family = { .id =GENL_ID_GENERATE ,@@ -193,7 +192,6 @@ static int taskstats_send_stats(struct sif (rc <0 )return rc; -mutex_lock (&taskstats_exit_mutex);if (info->attrs[TASKSTATS_CMD_ATTR_PID ]) {u32 pid =nla_get_u32 (info->attrs[TASKSTATS_CMD_ATTR_PID ]); rc =fill_pid (pid,NULL , &stats);@@ -219,7 +217,6 @@ static int taskstats_send_stats(struct s goto err; } -mutex_unlock (&taskstats_exit_mutex);nla_nest_end (rep_skb, na);return send_reply (rep_skb, info->snd_pid,TASKSTATS_MSG_UNICAST );@@ -228,7 +225,6 @@ nla_put_failure:return genlmsg_cancel (rep_skb, reply);err:
nlmsg_free(rep_skb);kernel/taskstats.c +0
-1-mutex_unlock (&taskstats_exit_mutex);return rc; }kernel/taskstats.c +0
-2@@ -246,8 +242,6 @@ void taskstats_exit_send(struct task_strif (!family_registered || !tidstats)return ; -mutex_lock (&taskstats_exit_mutex); - is_thread_group = !thread_group_empty (tsk); rc =0 ;@@ -305,7 +299,6 @@ nla_put_failure:
err_skb:nlmsg_free(rep_skb);ret:
kernel/taskstats.c +0
-1-mutex_unlock (&taskstats_exit_mutex);return; }
Quoted 2 lines · On Fri, 23 Jun 2006 13:52:04 -0400 Shailabh Nagar <[email protected]> wrote:
Convert locking used within taskstats interface and delay accounting code to be more fine-grained.
This patch is based on per-task-delay-accounting-taskstats-interface-fix-exit-race-in-per-task-delay-accounting.patch, which I've noted as `nacked' but didn't drop.
So I guess that's now un-nacked?
Quoted 10 lines · Andrew Morton
Quoted 4 lines · On Fri, 23 Jun 2006 13:52:04 -0400 Shailabh Nagar <[email protected]> wrote:
Convert locking used within taskstats interface and delay accounting code to be more fine-grained.
Quoted 7 lines
This patch is based on per-task-delay-accounting-taskstats-interface-fix-exit-race-in-per-task-delay-accounting.patch, which I've noted as `nacked' but didn't drop.
So I guess that's now un-nacked?
Not in intent. This patch reverses all the changes made by that patch. So effectively the previous patch is still nacked. However, I based this patch on the previous one because you hadn't dropped the latter (so we're going round in circles !)
How about I just send one patch that covers the whole locking thing ?
--Shailabh
Quoted 19 lines · On Sat, 24 Jun 2006 03:36:28 -0400 Shailabh Nagar <[email protected]> wrote:
Quoted 10 lines · Andrew Morton
Quoted 4 lines · On Fri, 23 Jun 2006 13:52:04 -0400 Shailabh Nagar <[email protected]> wrote:
Convert locking used within taskstats interface and delay accounting code to be more fine-grained.
Quoted 7 lines
This patch is based on per-task-delay-accounting-taskstats-interface-fix-exit-race-in-per-task-delay-accounting.patch, which I've noted as `nacked' but didn't drop.
So I guess that's now un-nacked?
Quoted 7 lines
Not in intent. This patch reverses all the changes made by that patch. So effectively the previous patch is still nacked.
However, I based this patch on the previous one because you hadn't dropped the latter (so we're going round in circles !)How about I just send one patch that covers the whole locking thing ?
Is OK - I'll concatenate these:
per-task-delay-accounting-taskstats-interface-fix-exit-race-in-per-task-delay-accounting.patch per-task-delay-accounting-delay-accounting-usage-of-taskstats-interface.patch per-task-delay-accounting-delay-accounting-usage-of-taskstats-interface-use-portable-cputime-api-in-__delayacct_add_tsk.patch per-task-delay-accounting-delay-accounting-usage-of-taskstats-interface-fix-return-value-of-delayacct_add_tsk.patch revised-locking-for-taskstats-interface.patch
into a single
per-task-delay-accounting-delay-accounting-usage-of-taskstats-interface.patch