Skip to contentSkip to conversation

timer: document TIMER_PINNED

PATCH Superseded Peter Xu 2 participants 3 messages Jun 27, 2019 – Jun 28, 2019 High-resolution timers, timer wheel, clockevents

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Peter Xu

author
#0 lore raw

PATCH v2 1/1 timer: document TIMER_PINNED

The flag hints the user that the pinned timers will always be run on a static CPU (because that should be what "pinned" means...) but that's not the truth, at least with current implementation.

For example, currently if we setup a pinned timer but later on we call mod_timer() upon the pinned timer, the mod_timer() will still try to run the timer on the current processor and migrate the timer if necessary. In other words, the suggested way to arm a pinned timer should be add_timer_on() always. mod_timer() can be used in this case only if current processor is the one that we want to pin the timer on.

Document it a bit with the definition of TIMER_PINNED so that all future users will use it correctly.

---
 include/linux/timer.h | 28 ++++++++++++++++++++--------
 1 file changed, 20 insertions(+), 8 deletions(-)
include/linux/timer.h +20 -8
index 7b066fd38248..1b96cc623a12 100644
@@ -36,19 +36,31 @@ struct timer_list {
 #define __TIMER_LOCKDEP_MAP_INITIALIZER(_kn)
 #endif
 
-/*
- * A deferrable timer will work normally when the system is busy, but
- * will not cause a CPU to come out of idle just to service it; instead,
- * the timer will be serviced when the CPU eventually wakes up with a
- * subsequent non-deferrable timer.
+/**
+ * @TIMER_DEFERRABLE: A deferrable timer will work normally when the
+ * system is busy, but will not cause a CPU to come out of idle just
+ * to service it; instead, the timer will be serviced when the CPU
+ * eventually wakes up with a subsequent non-deferrable timer.
  *
- * An irqsafe timer is executed with IRQ disabled and it's safe to wait for
- * the completion of the running instance from IRQ handlers, for example,
- * by calling del_timer_sync().
+ * @TIMER_IRQSAFE: An irqsafe timer is executed with IRQ disabled and
+ * it's safe to wait for the completion of the running instance from
+ * IRQ handlers, for example, by calling del_timer_sync().
  *
  * Note: The irq disabled callback execution is a special case for
  * workqueue locking issues. It's not meant for executing random crap
  * with interrupts disabled. Abuse is monitored!
+ *
+ * @TIMER_PINNED: A pinned timer will not be affected by any timer
+ * placement heuristics (like, NOHZ) and will always be run on the CPU
+ * when the timer was enqueued.
+ *
+ * Note: Because enqueuing of timers can actually migrate the timer
+ * from one CPU to another, pinned timers are not guaranteed to stay
+ * on the initialy selected CPU.  They move to the CPU on which the
+ * enqueue function is invoked via mod_timer() or add_timer().  If the
+ * timer should be placed on a particular CPU, then add_timer_on() has
+ * to be used.  It is also suggested that the user should always use
+ * add_timer_on() explicitly for pinned timers.
  */
 #define TIMER_CPUMASK		0x0003FFFF
 #define TIMER_MIGRATING		0x00040000
Signature · 1 line
2.21.0
#1 lore raw
Quoted 3 lines · Peter Xu

+ * @TIMER_PINNED: A pinned timer will not be affected by any timer
+ * placement heuristics (like, NOHZ) and will always be run on the CPU
+ * when the timer was enqueued.

s/when/on which/

Quoted 8 lines

+ *
+ * Note: Because enqueuing of timers can actually migrate the timer
+ * from one CPU to another, pinned timers are not guaranteed to stay
+ * on the initialy selected CPU. They move to the CPU on which the
+ * enqueue function is invoked via mod_timer() or add_timer(). If the
+ * timer should be placed on a particular CPU, then add_timer_on() has
+ * to be used. It is also suggested that the user should always use
+ * add_timer_on() explicitly for pinned timers.

That last sentence is not correct. add_timer_on() has limitations over mod_timer(). As pinned prevents the timer from being queued on a remote CPU mod timer is perfectly fine for many cases.

add_timer_on() is really about queueing a timer on a dedicated CPU, which is often enough a remote CPU.

Thanks,

tglx

author
#2 lore raw
Quoted 6 lines · Thomas Gleixner
Quoted 3 lines · Peter Xu

+ * @TIMER_PINNED: A pinned timer will not be affected by any timer
+ * placement heuristics (like, NOHZ) and will always be run on the CPU
+ * when the timer was enqueued.

s/when/on which/

Fixed.

Quoted 10 lines
Quoted 9 lines

+ *
+ * Note: Because enqueuing of timers can actually migrate the timer
+ * from one CPU to another, pinned timers are not guaranteed to stay
+ * on the initialy selected CPU. They move to the CPU on which the
+ * enqueue function is invoked via mod_timer() or add_timer(). If the
+ * timer should be placed on a particular CPU, then add_timer_on() has
+ * to be used. It is also suggested that the user should always use
+ * add_timer_on() explicitly for pinned timers.

Quoted 6 lines

That last sentence is not correct. add_timer_on() has limitations over mod_timer(). As pinned prevents the timer from being queued on a remote CPU mod timer is perfectly fine for many cases.

add_timer_on() is really about queueing a timer on a dedicated CPU, which is often enough a remote CPU.

Frankly speaking I still think add_timer_on() is preferred here because mod_timer() users will really need to be careful to make sure they'll pin the timers correctly all the time, and I assume that's why we've tried to find all the TIMER_PINNED users and tried to make sure there's nothing wrong on using them during previous discussion (and more than half of them do use add_timer_on() which seems to be good). In all cases, I'll take your suggestion to drop the last sentence.

Thanks for reviewing this document patch. I'll repost.

Signature · 1 line
Peter Xu