Performance of del_timer_sync
This is with reference to the del_timer_sync patch submitted by Geoff Gustafson http://lwn.net/Articles/84839/.
I profiled a postgress load on a 8p( 4 nodes) NUMA machine. This profiler logs functions that access memory on remote nodes, and del_timer_sync was one of the top few functions doing this. The culprit is the for loop in del_timer_sync that reads the running_timer field of the per cpu tvec_bases_t structure on all nodes in the system. It gets invoked most times from del_singleshot_timer_sync.
Andrew Morton had suggested a patch for this, when the top callers to del_timer sync were schedule_timeout and clear_timeout, the fix being the del_single_shot_timer_sync patch, with the knowledge that schedule_timeout and clear_timeout do not create timers that re-add themselves.
This may not be enough, the problem is that schedule_timeout at most times will call del_single_shot_timer_sync after the timer has expired. This will cause del_timer to asume that the handler is running on some cpu and del_timer_sync will still get invoked.
Ofcourse, it also means that del_timer_sync will continue to be a hotspot as far as taking up cpu time is concerned.
I tried Geoff's patch and it does seem to improve performance.
Has anyone else seen del_timer_sync as a hotspot after the del_singleshot_timer_sync patch?
regards
Shobhit
I profiled the 2.6.7 kernel, running a postgres load, on a 8p 4 node NUMA machine, logging occurances of timer deletions before they expired and timer deletions after they expired.
The results show that the ratio of timers being deleted after expiry to timers being deleted before expiry was 10:1.
This profiling was only done for schedule_timeout since it is the most frequent caller to del_timer_singleshot_sync.
There were 10000 occurance of timers being deleted after expiry from schedule_timeout and 997 occurances of timers being deleted before expiry from schedule_timout while the postgress workload ran. The ratio was always 10:1 from the start of the workload to end as the number of deletions climbed.
The del_timer_singleshot_sync improves performance for the case where timers are deleted before expiry but not for the case of timers being deleted after expiry.
The profiling data shows that the most frequesnt occurance, is that of timers being deleted after expiry, to the ratio of 10:1. In such a case del_timer_singleshot_sync may not provide the needed performance gain.
regards
shobhit
Quoted 42 lines · shobhit dayal
This is with reference to the
del_timer_syncpatch submitted by Geoff Gustafson http://lwn.net/Articles/84839/.I profiled a postgress load on a 8p( 4 nodes) NUMA machine. This profiler logs functions that access memory on remote nodes, and
del_timer_syncwas one of the top few functions doing this. The culprit is the for loop indel_timer_syncthat reads therunning_timerfield of the per cputvec_bases_tstructure on all nodes in the system. It gets invoked most times fromdel_singleshot_timer_sync.Andrew Morton had suggested a patch for this, when the top callers to
del_timersync wereschedule_timeoutandclear_timeout, the fix being thedel_single_shot_timer_syncpatch, with the knowledge thatschedule_timeoutandclear_timeoutdo not create timers that re-add themselves.This may not be enough, the problem is that
schedule_timeoutat most times will calldel_single_shot_timer_syncafter the timer has expired. This will causedel_timerto asume that the handler is running on some cpu anddel_timer_syncwill still get invoked.Ofcourse, it also means that
del_timer_syncwill continue to be a hotspot as far as taking up cpu time is concerned.I tried Geoff's patch and it does seem to improve performance.
Has anyone else seen
del_timer_syncas a hotspot after thedel_singleshot_timer_syncpatch?regards
Shobhit-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/