timer: document TIMER_PINNED
PATCH v2 1/1 timer: document TIMER_PINNED
The flag hints the user that the pinned timers will always be run on a static CPU (because that should be what "pinned" means...) but that's not the truth, at least with current implementation.
For example, currently if we setup a pinned timer but later on we call mod_timer() upon the pinned timer, the mod_timer() will still try to run the timer on the current processor and migrate the timer if necessary. In other words, the suggested way to arm a pinned timer should be add_timer_on() always. mod_timer() can be used in this case only if current processor is the one that we want to pin the timer on.
Document it a bit with the definition of TIMER_PINNED so that all future users will use it correctly.
--- include/linux/timer.h | 28 ++++++++++++++++++++-------- 1 file changed, 20 insertions(+), 8 deletions(-)
include/linux/timer.h +20 -8
@@ -36,19 +36,31 @@ struct timer_list {#define __TIMER_LOCKDEP_MAP_INITIALIZER (_kn)#endif -/* -* A deferrable timer will work normally when the system is busy, but -* will not cause a CPU to come out of idle just to service it; instead, -* the timer will be serviced when the CPU eventually wakes up with a -* subsequent non-deferrable timer. +/** +* @TIMER_DEFERRABLE: A deferrable timer will work normally when the +* system is busy, but will not cause a CPU to come out of idle just +* to service it; instead, the timer will be serviced when the CPU +* eventually wakes up with a subsequent non-deferrable timer. * -* An irqsafe timer is executed with IRQ disabled and it's safe to wait for -* the completion of the running instance from IRQ handlers, for example, -* by calling del_timer_sync(). +* @TIMER_IRQSAFE: An irqsafe timer is executed with IRQ disabled and +* it's safe to wait for the completion of the running instance from +* IRQ handlers, for example, by calling del_timer_sync(). * * Note: The irq disabled callback execution is a special case for * workqueue locking issues. It's not meant for executing random crap * with interrupts disabled. Abuse is monitored! +* +* @TIMER_PINNED: A pinned timer will not be affected by any timer +* placement heuristics (like, NOHZ) and will always be run on the CPU +* when the timer was enqueued. +* +* Note: Because enqueuing of timers can actually migrate the timer +* from one CPU to another, pinned timers are not guaranteed to stay +* on the initialy selected CPU. They move to the CPU on which the +* enqueue function is invoked via mod_timer() or add_timer(). If the +* timer should be placed on a particular CPU, then add_timer_on() has +* to be used. It is also suggested that the user should always use +* add_timer_on() explicitly for pinned timers. */ #define TIMER_CPUMASK 0x0003FFFF#define TIMER_MIGRATING 0x00040000
Signature · 1 line
2.21.0
Quoted 3 lines · Peter Xu
+ * @TIMER_PINNED: A pinned timer will not be affected by any timer
+ * placement heuristics (like, NOHZ) and will always be run on the CPU
+ * when the timer was enqueued.
s/when/on which/
Quoted 8 lines
+ *
+ * Note: Because enqueuing of timers can actually migrate the timer
+ * from one CPU to another, pinned timers are not guaranteed to stay
+ * on the initialy selected CPU. They move to the CPU on which the
+ * enqueue function is invoked viamod_timer()oradd_timer(). If the
+ * timer should be placed on a particular CPU, thenadd_timer_on()has
+ * to be used. It is also suggested that the user should always use
+ *add_timer_on()explicitly for pinned timers.
That last sentence is not correct. add_timer_on() has limitations over mod_timer(). As pinned prevents the timer from being queued on a remote CPU mod timer is perfectly fine for many cases.
add_timer_on() is really about queueing a timer on a dedicated CPU, which is often enough a remote CPU.
Thanks,
tglx
Quoted 6 lines · Thomas Gleixner
Quoted 3 lines · Peter Xu
+ * @TIMER_PINNED: A pinned timer will not be affected by any timer
+ * placement heuristics (like, NOHZ) and will always be run on the CPU
+ * when the timer was enqueued.s/when/on which/
Fixed.
Quoted 10 lines
Quoted 9 lines
+ *
+ * Note: Because enqueuing of timers can actually migrate the timer
+ * from one CPU to another, pinned timers are not guaranteed to stay
+ * on the initialy selected CPU. They move to the CPU on which the
+ * enqueue function is invoked viamod_timer()oradd_timer(). If the
+ * timer should be placed on a particular CPU, thenadd_timer_on()has
+ * to be used. It is also suggested that the user should always use
+ *add_timer_on()explicitly for pinned timers.
Quoted 6 lines
That last sentence is not correct.
add_timer_on()has limitations overmod_timer(). As pinned prevents the timer from being queued on a remote CPU mod timer is perfectly fine for many cases.
add_timer_on()is really about queueing a timer on a dedicated CPU, which is often enough a remote CPU.
Frankly speaking I still think add_timer_on() is preferred here because mod_timer() users will really need to be careful to make sure they'll pin the timers correctly all the time, and I assume that's why we've tried to find all the TIMER_PINNED users and tried to make sure there's nothing wrong on using them during previous discussion (and more than half of them do use add_timer_on() which seems to be good). In all cases, I'll take your suggestion to drop the last sentence.
Thanks for reviewing this document patch. I'll repost.
Signature · 1 line
Peter Xu