hrtimer/sched: Improve hrtick
Subsystems (4)
Peter Zijlstra
maintainerv2 cover hrtimer/sched: Improve hrtick
Hi!
At long last a new version of the hrtick rework! The previous version had a mysterious deadlock which has been resolved. So far (weeks) the 0-day robot has not found more issues with these patches.
These patches aim to reduce the hrtick overhead to such an extent that it can be default enabled. Decoupling the preemption behaviour from CONFIG_HZ, leaving only load-balancing and time keeping dependent on HZ.
Some (limited) performance runs from 0-day have also not found any regressions from enabling HRTICK, but it has not run the full suite yet (AFAIU).
Patches also at:
git://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git sched/hrtick
Peter Zijlstra
maintainerPATCH v2 1/6 sched/eevdf: Fix HRTICK duration
The nominal duration for an EEVDF task to run is until its deadline. At which point the deadline is moved ahead and a new task selection is done.
Try and predict the time 'lost' to higher scheduling classes. Since this is an estimate, the timer can be both early or late. In case it is early task_tick_fair() will take the !need_resched() path and restarts the timer.
--- kernel/sched/fair.c | 55 +++++++++++++++++++++++++++++----------------------- 1 file changed, 31 insertions(+), 24 deletions(-)
kernel/sched/fair.c +31 -24
@@ -5511,7 +5511,7 @@ static void put_prev_entity(struct cfs_r }static void -entity_tick (struct cfs_rq *cfs_rq,struct sched_entity *curr,int queued) +entity_tick (struct cfs_rq *cfs_rq,struct sched_entity *curr) {/* * Update run-time statistics of the 'current'.
@@ -5523,17 +5523,6 @@ entity_tick(struct cfs_rq *cfs_rq, struc*/ update_load_avg (cfs_rq, curr,UPDATE_TG );update_cfs_group (curr); - -#ifdef CONFIG_SCHED_HRTICK -/* -* queued ticks are scheduled to match the slice, so don't bother -* validating it and just reschedule. -*/ -if (queued) { -resched_curr_lazy (rq_of (cfs_rq)); -return ; - } -#endif }
@@ -6735,21 +6724,39 @@ static inline void sched_fair_update_stostatic void hrtick_start_fair (struct rq *rq,struct task_struct *p) {struct sched_entity *se = &p->se; +unsigned long scale =1024 ; +unsigned long util =0 ; +u64 vdelta; +u64 delta;WARN_ON_ONCE (task_rq (p) != rq); -if (rq->cfs.h_nr_queued >1 ) { -u64 ran = se->sum_exec_runtime - se->prev_sum_exec_runtime; -u64 slice = se->slice; -s64 delta = slice - ran; - -if (delta <0 ) { -if (task_current_donor (rq, p)) -resched_curr (rq); -return ; - } -hrtick_start (rq, delta); +if (rq->cfs.h_nr_queued <=1 ) +return ; + +/* +* Compute time until virtual deadline +*/ + vdelta = se->deadline - se->vruntime; +if ((s64 )vdelta <0 ) { +if (task_current_donor (rq, p)) +resched_curr (rq); +return ; + } + delta = (se->load.weight * vdelta) /NICE_0_LOAD ; + +/* +* Correct for instantaneous load of other classes. +*/ + util +=cpu_util_dl (rq); + util +=cpu_util_rt (rq); + util +=cpu_util_irq (rq); +if (util && util <1024 ) { + scale *=1024 ; + scale /= (1024 - util); } + +hrtick_start (rq, (scale * delta) /1024 ); }/*
@@ -13373,7 +13380,7 @@ static void task_tick_fair(struct rq *rqfor_each_sched_entity (se) { cfs_rq =cfs_rq_of (se); -entity_tick (cfs_rq, se, queued); +entity_tick (cfs_rq, se); }if (queued) {
Re: [PATCH v2 1/6] sched/eevdf: Fix HRTICK duration
Hello,
Quoted 11 lines · Peter Zijlstra
The nominal duration for an EEVDF task to run is until its deadline. At which point the deadline is moved ahead and a new task selection is done.
Try and predict the time 'lost' to higher scheduling classes. Since this is an estimate, the timer can be both early or late. In case it is early
task_tick_fair()will take the !need_resched() path and restarts the timer.Signed-off-by Peter Zijlstra (Intel) <[email protected]>---
...
Quoted 25 lines
kernel/sched/fair.c +7
-11@@ -6735,21 +6724,39 @@ static inline void sched_fair_update_stostatic void hrtick_start_fair (struct rq *rq,struct task_struct *p) {struct sched_entity *se = &p->se; +unsigned long scale =1024 ; +unsigned long util =0 ; +u64 vdelta; +u64 delta;WARN_ON_ONCE (task_rq (p) != rq); -if (rq->cfs.h_nr_queued >1 ) { -u64 ran = se->sum_exec_runtime - se->prev_sum_exec_runtime; -u64 slice = se->slice; -s64 delta = slice - ran; - -if (delta <0 ) { -if (task_current_donor (rq, p)) -resched_curr (rq); -return ; - } -hrtick_start (rq, delta); +if (rq->cfs.h_nr_queued <=1 ) +return ; +
kernel/sched/fair.c static inline void sched_fair_update_sto · line 6746
Quoted 10 lines
kernel/sched/fair.c +10
-0+/* +* Compute time until virtual deadline +*/ + vdelta = se->deadline - se->vruntime; +if ((s64 )vdelta <0 ) { +if (task_current_donor (rq, p)) +resched_curr (rq); +return ; + } + delta = (se->load.weight * vdelta) /NICE_0_LOAD ;
Nit.. guess we don't fear overflow since vdelta should be bounded anyway.
Thanks,
Juri
Peter Zijlstra
maintainerRe: [PATCH v2 1/6] sched/eevdf: Fix HRTICK duration
In reply to Peter ZijlstraQuoted 22 lines · Peter Zijlstra
kernel/sched/fair.c +13
-0@@ -6735,21 +6724,39 @@ static inline void sched_fair_update_stostatic void hrtick_start_fair (struct rq *rq,struct task_struct *p) {struct sched_entity *se = &p->se; +unsigned long scale =1024 ; +unsigned long util =0 ; +u64 vdelta; +u64 delta;WARN_ON_ONCE (task_rq (p) != rq); +if (rq->cfs.h_nr_queued <=1 ) +return ; + +/* +* Compute time until virtual deadline +*/ + vdelta = se->deadline - se->vruntime; +if ((s64 )vdelta <0 ) { +if (task_current_donor (rq, p))
kernel/sched/fair.c static inline void sched_fair_update_sto · line 6752
Quoted 10 lines
kernel/sched/fair.c +10
-0+resched_curr (rq); +return ; + } + delta = (se->load.weight * vdelta) /NICE_0_LOAD ; + +/* +* Correct for instantaneous load of other classes. +*/ + util +=cpu_util_dl (rq); + util +=cpu_util_rt (rq);
Since this is all about current, other scheduling classes are irrelevant, they cannot run without causing schedule() which will cause the hrtick to be reprogrammed anyway.
So I'm thinking those two lines above ought to go.
Quoted 40 lines
kernel/sched/fair.c +7
-12+ util +=cpu_util_irq (rq); +if (util && util <1024 ) { + scale *=1024 ; + scale /= (1024 - util); } + +hrtick_start (rq, (scale * delta) /1024 ); }/* @@ -5511,7 +5511,7 @@ static void put_prev_entity(struct cfs_r }static void -entity_tick (struct cfs_rq *cfs_rq,struct sched_entity *curr,int queued) +entity_tick (struct cfs_rq *cfs_rq,struct sched_entity *curr) {/* * Update run-time statistics of the 'current'. @@ -5523,17 +5523,6 @@ entity_tick(struct cfs_rq *cfs_rq, struc*/ update_load_avg (cfs_rq, curr,UPDATE_TG );update_cfs_group (curr); - -#ifdef CONFIG_SCHED_HRTICK -/* -* queued ticks are scheduled to match the slice, so don't bother -* validating it and just reschedule. -*/ -if (queued) { -resched_curr_lazy (rq_of (cfs_rq)); -return ; - } -#endif }
kernel/sched/fair.c static void task_tick_fair(struct rq *rq · line 13386
Quoted 8 lines
kernel/sched/fair.c +1
-1@@ -13373,7 +13380,7 @@ static void task_tick_fair(struct rq *rqfor_each_sched_entity (se) { cfs_rq =cfs_rq_of (se); -entity_tick (cfs_rq, se, queued); +entity_tick (cfs_rq, se); }if (queued) {
So Thomas did observe some really small hrtimer reprogramming because of this. If we just miss the normal deadline, it will re-try with a stupid sliver of time.
Perhaps it makes sense to leave these two hunks, and simply hard preempt when the hrtick goes, irrespective of slightly missing the vruntime due to the approximation on task-clock.
Peter Zijlstra
maintainerPATCH v2 2/6 hrtimer: Optimize __hrtimer_start_range_ns()
In reply to Peter ZijlstraMuch like hrtimer_reprogram(), skip programming if the cpu_base is running the hrtimer interrupt.
--- kernel/time/hrtimer.c | 8 ++++++++ 1 file changed, 8 insertions(+)
kernel/time/hrtimer.c +8 -0
@@ -1261,6 +1261,14 @@ static int __hrtimer_start_range_ns(stru } first =enqueue_hrtimer (timer, new_base, mode); + +/* +* If the hrtimer interrupt is running, then it will reevaluate the +* clock bases and reprogram the clock event device. +*/ +if (new_base->cpu_base->in_hrtirq) +return 0 ; +if (!force_local) {/* * If the current CPU base is online, then the timer is
Re: [PATCH v2 2/6] hrtimer: Optimize __hrtimer_start_range_ns()
Hello,
Quoted 4 lines · Peter Zijlstra
Much like
hrtimer_reprogram(), skip programming if thecpu_baseis running the hrtimer interrupt.Signed-off-by Peter Zijlstra (Intel) <[email protected]>
Thanks,
Juri
Re: [PATCH v2 2/6] hrtimer: Optimize __hrtimer_start_range_ns()
In reply to Peter ZijlstraQuoted 4 lines · Peter Zijlstra
Much like
hrtimer_reprogram(), skip programming if thecpu_baseis running the hrtimer interrupt.Signed-off-by Peter Zijlstra (Intel) <[email protected]>
Peter Zijlstra
maintainerPATCH v2 3/6 hrtimer,sched: Add fuzzy hrtimer mode for HRTICK
In reply to Peter ZijlstraUpon schedule() HRTICK will cancel the current timer, pick the next task and reprogram the timer. When schedule() consistently triggers due to blocking conditions instead of the timer, this leads to endless reprogramming without ever firing.
Mitigate this with a new hrtimer mode: fuzzy (not really happy with that name); this mode does two things:
- skip reprogramming the hardware on timer remove;
- skip reprogramming the hardware when the new timer is after cpu_base->expires_next
Both things are already possible;
- removing a remote timer will leave the hardware programmed and cause a spurious interrupt.
- this remote CPU adding a timer can skip the reprogramming when the timer's expiration is after the (spurious) expiration.
This new timer mode simply causes more of this 'fuzzy' behaviour; it causes a few spurious interrupts, but similarly avoids endlessly reprogramming the timer.
This makes the HRTICK match the NO_HRTICK hackbench runs -- the case where a task never runs until its slice is complete but always goes sleep early.
--- include/linux/hrtimer.h | 1 + include/linux/hrtimer_types.h | 1 + kernel/sched/core.c | 3 ++- kernel/time/hrtimer.c | 16 +++++++++++++++- 4 files changed, 19 insertions(+), 2 deletions(-)
include/linux/hrtimer.h +1 -0
@@ -38,6 +38,7 @@ enum hrtimer_mode {HRTIMER_MODE_PINNED =0x02 ,HRTIMER_MODE_SOFT =0x04 ,HRTIMER_MODE_HARD =0x08 , +HRTIMER_MODE_FUZZY =0x10 ,HRTIMER_MODE_ABS_PINNED =HRTIMER_MODE_ABS |HRTIMER_MODE_PINNED , HRTIMER_MODE_REL_PINNED = HRTIMER_MODE_REL | HRTIMER_MODE_PINNED,
include/linux/hrtimer_types.h +1 -0
@@ -45,6 +45,7 @@ struct hrtimer {u8 is_rel;u8 is_soft;u8 is_hard; + u8 is_fuzzy; };#endif /* _LINUX_HRTIMER_TYPES_H */
kernel/sched/core.c +2 -1
@@ -928,7 +928,8 @@ void hrtick_start(struct rq *rq, u64 delstatic void hrtick_rq_init (struct rq *rq) {INIT_CSD (&rq->hrtick_csd, __hrtick_start, rq); -hrtimer_setup (&rq->hrtick_timer, hrtick,CLOCK_MONOTONIC ,HRTIMER_MODE_REL_HARD ); +hrtimer_setup (&rq->hrtick_timer, hrtick,CLOCK_MONOTONIC , +HRTIMER_MODE_REL_HARD |HRTIMER_MODE_FUZZY ); }#else /* !CONFIG_SCHED_HRTICK: */ static inline void hrtick_clear (struct rq *rq)
kernel/time/hrtimer.c +15 -1
@@ -1122,7 +1122,7 @@ static void __remove_hrtimer(struct hrti* an superfluous call to hrtimer_force_reprogram() on the * remote cpu later on if the same timer gets enqueued again. */ -if (reprogram && timer == cpu_base->next_timer) +if (!timer->is_fuzzy && reprogram && timer == cpu_base->next_timer)hrtimer_force_reprogram (cpu_base,1 ); }
@@ -1269,6 +1269,19 @@ static int __hrtimer_start_range_ns(struif (new_base->cpu_base->in_hrtirq)return 0 ; +if (timer->is_fuzzy) { +/* +* XXX fuzzy implies pinned! not sure how to deal with +* retrigger_next_event() for the !local case. +*/ +WARN_ON_ONCE (!(mode &HRTIMER_MODE_PINNED )); +/* +* Notably, by going into hrtimer_reprogram(), it will +* not reprogram if cpu_base->expires_next is earlier. +*/ +return first; + } +if (!force_local) {/* * If the current CPU base is online, then the timer is
@@ -1645,6 +1658,7 @@ static void __hrtimer_setup(struct hrtim base +=hrtimer_clockid_to_base (clock_id); timer->is_soft = softtimer; timer->is_hard = !!(mode &HRTIMER_MODE_HARD ); + timer->is_fuzzy = !!(mode &HRTIMER_MODE_FUZZY ); timer->base = &cpu_base->clock_base[base];timerqueue_init (&timer->node);
Re: [PATCH v2 3/6] hrtimer,sched: Add fuzzy hrtimer mode for HRTICK
Hello,
Quoted 7 lines · Peter Zijlstra
Upon
schedule()HRTICK will cancel the current timer, pick the next task and reprogram the timer. Whenschedule()consistently triggers due to blocking conditions instead of the timer, this leads to endless reprogramming without ever firing.Mitigate this with a new hrtimer mode: fuzzy (not really happy with that name); this mode does two things:
Does the more common (lazier :) 'lazy' work better?
Quoted 22 lines
- skip reprogramming the hardware on timer remove;
- skip reprogramming the hardware when the new timer is aftercpu_base->expires_nextBoth things are already possible;
- removing a remote timer will leave the hardware programmed and cause a spurious interrupt.
- this remote CPU adding a timer can skip the reprogramming when the timer's expiration is after the (spurious) expiration.This new timer mode simply causes more of this 'fuzzy' behaviour; it causes a few spurious interrupts, but similarly avoids endlessly reprogramming the timer.
This makes the HRTICK match the
NO_HRTICKhackbench runs -- the case where a task never runs until its slice is complete but always goes sleep early.Signed-off-by Peter Zijlstra (Intel) <[email protected]>---
...
kernel/time/hrtimer.c static int __hrtimer_start_range_ns(stru · line 1277
Quoted 10 lines
kernel/time/hrtimer.c +6
-0@@ -1269,6 +1269,19 @@ static int __hrtimer_start_range_ns(struif (new_base->cpu_base->in_hrtirq)return 0 ; +if (timer->is_fuzzy) { +/* +* XXX fuzzy implies pinned! not sure how to deal with +* retrigger_next_event() for the !local case. +*/ +WARN_ON_ONCE (!(mode &HRTIMER_MODE_PINNED ));
Not sure either, but since it's improving things for local already, maybe it's an acceptable first step?
Thanks,
Juri
Steven Rostedt
reviewerRe: [PATCH v2 3/6] hrtimer,sched: Add fuzzy hrtimer mode for HRTICK
Quoted 8 lines · On Thu, 22 Jan 2026 14:12:28 +0100 Juri Lelli <[email protected]> wrote:
Hello,
Quoted 5 lines · Peter Zijlstra
Upon
schedule()HRTICK will cancel the current timer, pick the next task and reprogram the timer. Whenschedule()consistently triggers due to blocking conditions instead of the timer, this leads to endless reprogramming without ever firing.
Quoted 4 lines
Quoted 2 lines
Mitigate this with a new hrtimer mode: fuzzy (not really happy with that name); this mode does two things:
Does the more common (lazier :) 'lazy' work better?
I don't like either fuzzy or lazy.
Fuzzy makes me think of just random entries (for fuzz testing and such).
Lazy is to postpone things to do things less often.
What about "speculative"? Like branch prediction and such. Where a timer is expected to be used at a certain time but it may not be?
-- Steve
Re: [PATCH v2 3/6] hrtimer,sched: Add fuzzy hrtimer mode for HRTICK
In reply to Peter Zijlstrakernel/sched/core.c void hrtick_start(struct rq *rq, u64 del · line 932
Quoted 9 lines · Peter Zijlstra
kernel/sched/core.c +2
-1@@ -928,7 +928,8 @@ void hrtick_start(struct rq *rq, u64 delstatic void hrtick_rq_init (struct rq *rq) {INIT_CSD (&rq->hrtick_csd, __hrtick_start, rq); -hrtimer_setup (&rq->hrtick_timer, hrtick,CLOCK_MONOTONIC ,HRTIMER_MODE_REL_HARD ); +hrtimer_setup (&rq->hrtick_timer, hrtick,CLOCK_MONOTONIC , +HRTIMER_MODE_REL_HARD |HRTIMER_MODE_FUZZY );
SHouldn't this be HRTIMER_MODE_REL_PINNED_HARD? I know it's set when starting the timer, but I had to double check it.
Quoted 15 lines
kernel/sched/core.c +0
-0}#else /* !CONFIG_SCHED_HRTICK: */ static inline void hrtick_clear (struct rq *rq)kernel/time/hrtimer.c +1
-1@@ -1122,7 +1122,7 @@ static void __remove_hrtimer(struct hrti* an superfluous call to hrtimer_force_reprogram() on the * remote cpu later on if the same timer gets enqueued again. */ -if (reprogram && timer == cpu_base->next_timer) +if (!timer->is_fuzzy && reprogram && timer == cpu_base->next_timer)hrtimer_force_reprogram (cpu_base,1 ); }
kernel/time/hrtimer.c static int __hrtimer_start_range_ns(stru · line 1275
Quoted 7 lines
kernel/time/hrtimer.c +4
-0@@ -1269,6 +1269,19 @@ static int __hrtimer_start_range_ns(struif (new_base->cpu_base->in_hrtirq)return 0 ; +if (timer->is_fuzzy) { +/* +* XXX fuzzy implies pinned! not sure how to deal with +* retrigger_next_event() for the !local case.
I'd rather say:
Fuzzy requires pinned as the lazy reprogramming only works for CPU local timers.
kernel/time/hrtimer.c static int __hrtimer_start_range_ns(stru · line 1277
Quoted 2 lines
kernel/time/hrtimer.c +2
-0+*/ +WARN_ON_ONCE (!(mode &HRTIMER_MODE_PINNED ));
Other than that:
Thanks,
tglx
Peter Zijlstra
maintainerPATCH v2 4/6 hrtimer: Re-arrange hrtimer_interrupt()
In reply to Peter ZijlstraRework hrtimer_interrupt() such that reprogramming is split out into an independent function at the end of the interrupt.
This prepares for reprogramming getting delayed beyond the end of hrtimer_interrupt().
Notably, this changes the hang handling to always wait 100ms instead of trying to keep it proportional to the actual delay. This simplifies the state, also this really shouldn't be happening.
--- kernel/time/hrtimer.c | 87 ++++++++++++++++++++++---------------------------- 1 file changed, 39 insertions(+), 48 deletions(-)
kernel/time/hrtimer.c +39 -48
@@ -1889,6 +1889,29 @@ static __latent_entropy void hrtimer_run#ifdef CONFIG_HIGH_RES_TIMERS /* +* Very similar to hrtimer_force_reprogram(), except it deals with +* in_hrirq and hang_detected. +*/ +static void __hrtimer_rearm (struct hrtimer_cpu_base *cpu_base,ktime_t now) +{ +ktime_t expires_next =hrtimer_update_next_event (cpu_base); + + cpu_base->expires_next = expires_next; + cpu_base->in_hrtirq =0 ; + +if (unlikely (cpu_base->hang_detected)) { +/* +* Give the system a chance to do something else than looping +* on hrtimer interrupts. +*/ + expires_next =ktime_add_ns (now,100 *NSEC_PER_MSEC ); + cpu_base->hang_detected =0 ; + } + +tick_program_event (expires_next,1 ); +} + +/* * High resolution timer interrupt * Called with interrupts disabled */
@@ -1924,63 +1947,31 @@ void hrtimer_interrupt(struct clock_even__hrtimer_run_queues (cpu_base, now, flags,HRTIMER_ACTIVE_HARD ); -/* Reevaluate the clock bases for the [soft] next expiry */ - expires_next =hrtimer_update_next_event (cpu_base); -/* -* Store the new expiry value so the migration code can verify -* against it. -*/ - cpu_base->expires_next = expires_next; - cpu_base->in_hrtirq =0 ; -raw_spin_unlock_irqrestore (&cpu_base->lock, flags); - -/* Reprogramming necessary ? */ -if (!tick_program_event (expires_next,0 )) { - cpu_base->hang_detected =0 ; -return ; - } -/* * The next timer was already expired due to: * - tracing * - long lasting callbacks * - being scheduled away when running in a VM * -* We need to prevent that we loop forever in the hrtimer -* interrupt routine. We give it 3 attempts to avoid -* overreacting on some spurious event. -* -* Acquire base lock for updating the offsets and retrieving -* the current time. +* We need to prevent that we loop forever in the hrtiner interrupt +* routine. We give it 3 attempts to avoid overreacting on some +* spurious event. */ -raw_spin_lock_irqsave (&cpu_base->lock, flags); + expires_next =hrtimer_update_next_event (cpu_base); now =hrtimer_update_base (cpu_base); - cpu_base->nr_retries++; -if (++retries <3 ) - goto retry; -/* -* Give the system a chance to do something else than looping -* here. We stored the entry time, so we know exactly how long -* we spent here. We schedule the next event this amount of -* time away. -*/ - cpu_base->nr_hangs++; - cpu_base->hang_detected =1 ; -raw_spin_unlock_irqrestore (&cpu_base->lock, flags); +if (expires_next < now) { +if (++retries <3 ) + goto retry; + + delta =ktime_sub (now, entry_time); + cpu_base->max_hang_time = max_t(unsigned int, + cpu_base->max_hang_time, delta); + cpu_base->nr_hangs++; + cpu_base->hang_detected =1 ; + } - delta =ktime_sub (now, entry_time); -if ((unsigned int )delta > cpu_base->max_hang_time) - cpu_base->max_hang_time = (unsigned int ) delta; -/* -* Limit it to a sensible value as we enforce a longer -* delay. Give the CPU at least 100ms to catch up. -*/ -if (delta >100 *NSEC_PER_MSEC ) - expires_next =ktime_add_ns (now,100 *NSEC_PER_MSEC ); -else - expires_next =ktime_add (now, delta); -tick_program_event (expires_next,1 ); - pr_warn_once("hrtimer: interrupt took %llu ns\n", ktime_to_ns(delta)); +__hrtimer_rearm (cpu_base, now); + raw_spin_unlock_irqrestore(&cpu_base->lock, flags); }#endif /* !CONFIG_HIGH_RES_TIMERS */
Re: [PATCH v2 4/6] hrtimer: Re-arrange hrtimer_interrupt()
Quoted 9 lines · Peter Zijlstra
Rework
hrtimer_interrupt()such that reprogramming is split out into an independent function at the end of the interrupt.This prepares for reprogramming getting delayed beyond the end of
hrtimer_interrupt().Notably, this changes the hang handling to always wait 100ms instead of trying to keep it proportional to the actual delay. This simplifies the state, also this really shouldn't be happening.
Indeed.
Quoted 3 lines
/*
+ * Very similar tohrtimer_force_reprogram(), except it deals with
+ *in_hrirqandhang_detected.
in_hrtirq
Peter Zijlstra
maintainerPATCH v2 5/6 entry,hrtimer: Push reprogramming timers into the interrupt return path
In reply to Peter ZijlstraCurrently hrtimer_interrupt() runs expired timers, which can re-arm themselves, after which it computes the next expiration time and re-programs the hardware.
However, things like HRTICK, a highres timer driving preemption, cannot re-arm itself at the point of running, since the next task has not been determined yet. The schedule() in the interrupt return path will switch to the next task, which then causes a new hrtimer to be programmed.
This then results in reprogramming the hardware at least twice, once after running the timers, and once upon selecting the new task.
Notably, *both* events happen in the interrupt.
By pushing the hrtimer reprogram all the way into the interrupt return path, it runs after schedule() and this double reprogram can be avoided.
--- include/asm-generic/thread_info_tif.h | 5 ++++- include/linux/hrtimer.h | 17 +++++++++++++++++ include/linux/irq-entry-common.h | 2 ++ kernel/entry/common.c | 13 +++++++++++++ kernel/sched/core.c | 10 ++++++++++ kernel/time/hrtimer.c | 28 ++++++++++++++++++++++++---- 6 files changed, 70 insertions(+), 5 deletions(-)
include/asm-generic/thread_info_tif.h +4 -1
@@ -41,11 +41,14 @@#define _TIF_PATCH_PENDING BIT(TIF_PATCH_PENDING)#ifdef HAVE_TIF_RESTORE_SIGMASK -# define TIF_RESTORE_SIGMASK 10 // Restore signal mask in do_signal() */ +# define TIF_RESTORE_SIGMASK 10 // Restore signal mask in do_signal()# define _TIF_RESTORE_SIGMASK BIT(TIF_RESTORE_SIGMASK)#endif #define TIF_RSEQ 11 // Run RSEQ fast path#define _TIF_RSEQ BIT(TIF_RSEQ) +#define TIF_HRTIMER_REARM 12 // re-arm the timer +#define _TIF_HRTIMER_REARM BIT(TIF_HRTIMER_REARM) +#endif /* _ASM_GENERIC_THREAD_INFO_TIF_H_ */
include/linux/hrtimer.h +17 -0
@@ -175,10 +175,27 @@ extern void hrtimer_interrupt(struct cloextern unsigned int hrtimer_resolution; +#ifdef TIF_HRTIMER_REARM +extern void _hrtimer_rearm (void ); +/* +* This is to be called on all irqentry_exit() paths that will enable +* interrupts; as well as in the context switch path before switch_to(). +*/ +static inline void hrtimer_rearm (void ) +{ +if (test_thread_flag (TIF_HRTIMER_REARM )) +_hrtimer_rearm (); +} +#else +static inline void hrtimer_rearm (void ) { } +#endif /* TIF_HRTIMER_REARM */ +#else #define hrtimer_resolution (unsigned int)LOW_RES_NSEC +static inline void hrtimer_rearm (void ) { } +#endif static inline ktime_t
include/linux/irq-entry-common.h +2 -0
@@ -224,6 +224,8 @@ static __always_inline void __exit_to_us ti_work =read_thread_flags ();if (unlikely (ti_work &EXIT_TO_USER_MODE_WORK )) ti_work =exit_to_user_mode_loop (regs, ti_work); +else +hrtimer_rearm ();arch_exit_to_user_mode_prepare (regs, ti_work); }
kernel/entry/common.c +13 -0
@@ -7,6 +7,7 @@#include <linux/kmsan.h> #include <linux/livepatch.h> #include <linux/tick.h> +#include <linux/hrtimer.h> /* Workaround to allow gradual conversion of architecture code */ void __weak arch_do_signal_or_restart (struct pt_regs *regs) { }
@@ -26,6 +27,16 @@ static __always_inline unsigned long __e*/ while (ti_work &EXIT_TO_USER_MODE_WORK_LOOP ) { +/* +* If hrtimer need re-arming, do so before enabling IRQs, +* except when a reschedule is needed, in that case schedule() +* will do this. +*/ +if ((ti_work & (_TIF_NEED_RESCHED | + _TIF_NEED_RESCHED_LAZY | + _TIF_HRTIMER_REARM)) == _TIF_HRTIMER_REARM) +hrtimer_rearm (); +local_irq_enable_exit_to_user (ti_work);if (ti_work & (_TIF_NEED_RESCHED | _TIF_NEED_RESCHED_LAZY))
@@ -202,6 +213,7 @@ noinstr void irqentry_exit(struct pt_reg*/ if (state.exit_rcu) {instrumentation_begin (); +hrtimer_rearm ();/* Tell the tracer that IRET will enable interrupts */ trace_hardirqs_on_prepare ();lockdep_hardirqs_on_prepare ();
@@ -215,6 +227,7 @@ noinstr void irqentry_exit(struct pt_regif (IS_ENABLED (CONFIG_PREEMPTION ))irqentry_exit_cond_resched (); +hrtimer_rearm ();/* Covers both tracing and lockdep */ trace_hardirqs_on ();instrumentation_end ();
kernel/sched/core.c +10 -0
@@ -6814,6 +6814,16 @@ static void __sched notrace __schedule(i keep_resched: rq->last_seen_need_resched_ns =0 ; +/* +* Notably, this must be called after pick_next_task() but before +* switch_to(), since the new task need not be on the return from +* interrupt path. Additionally, exit_to_user_mode_loop() relies on +* any schedule() call to imply this call, so do it unconditionally. +* +* We've just cleared TIF_NEED_RESCHED, TIF word should be in cache. +*/ +hrtimer_rearm (); + is_switch = prev != next;if (likely (is_switch)) { rq->nr_switches++;
kernel/time/hrtimer.c +24 -4
@@ -1892,10 +1892,9 @@ static __latent_entropy void hrtimer_run* Very similar to hrtimer_force_reprogram(), except it deals with * in_hrirq and hang_detected. */ -static void __hrtimer_rearm (struct hrtimer_cpu_base *cpu_base,ktime_t now) +static void __hrtimer_rearm (struct hrtimer_cpu_base *cpu_base, +ktime_t now,ktime_t expires_next) { -ktime_t expires_next =hrtimer_update_next_event (cpu_base); - cpu_base->expires_next = expires_next; cpu_base->in_hrtirq =0 ;
@@ -1970,9 +1969,30 @@ void hrtimer_interrupt(struct clock_even cpu_base->hang_detected = 1; } -__hrtimer_rearm (cpu_base, now); +#ifdef TIF_HRTIMER_REARM +set_thread_flag (TIF_HRTIMER_REARM ); +#else +__hrtimer_rearm (cpu_base, now, expires_next); +#endif raw_spin_unlock_irqrestore (&cpu_base->lock, flags); } + +#ifdef TIF_HRTIMER_REARM +void _hrtimer_rearm (void ) +{ +struct hrtimer_cpu_base *cpu_base =this_cpu_ptr (&hrtimer_bases); +ktime_t now, expires_next; + +lockdep_assert_irqs_disabled (); + + scoped_guard (raw_spinlock, &cpu_base->lock) { + now =hrtimer_update_base (cpu_base); + expires_next =hrtimer_update_next_event (cpu_base); +__hrtimer_rearm (cpu_base, now, expires_next); +clear_thread_flag (TIF_HRTIMER_REARM ); + } +} +#endif /* TIF_HRTIMER_REARM */ #endif /* !CONFIG_HIGH_RES_TIMERS */ /*
Steven Rostedt
reviewerRe: [PATCH v2 5/6] entry,hrtimer: Push reprogramming timers into the interrupt return path
Quoted 5 lines · On Wed, 21 Jan 2026 17:20:15 +0100 Peter Zijlstra <[email protected]> wrote:
kernel/time/hrtimer.c +5
-0+#ifdef TIF_HRTIMER_REARM +void _hrtimer_rearm (void ) +{ +struct hrtimer_cpu_base *cpu_base =this_cpu_ptr (&hrtimer_bases); +ktime_t now, expires_next;
kernel/time/hrtimer.c void hrtimer_interrupt(struct clock_even · line 1994
Quoted 10 lines
kernel/time/hrtimer.c +10
-0+ +lockdep_assert_irqs_disabled (); + + scoped_guard (raw_spinlock, &cpu_base->lock) { + now =hrtimer_update_base (cpu_base); + expires_next =hrtimer_update_next_event (cpu_base); +__hrtimer_rearm (cpu_base, now, expires_next); +clear_thread_flag (TIF_HRTIMER_REARM ); + } +}
I'm curious to why you decided to use scoped_guard() here and not just guard() and not add the extra indentation? The function is small enough where everything is expected to be protected by the spinlock.
-- Steve
Peter Zijlstra
maintainerRe: [PATCH v2 5/6] entry,hrtimer: Push reprogramming timers into the interrupt return path
Quoted 14 lines · Steven Rostedt
Quoted 11 lines · On Wed, 21 Jan 2026 17:20:15 +0100 Peter Zijlstra <[email protected]> wrote:
kernel/time/hrtimer.c +11
-0+#ifdef TIF_HRTIMER_REARM +void _hrtimer_rearm (void ) +{ +struct hrtimer_cpu_base *cpu_base =this_cpu_ptr (&hrtimer_bases); +ktime_t now, expires_next; + +lockdep_assert_irqs_disabled (); + + scoped_guard (raw_spinlock, &cpu_base->lock) { + now =hrtimer_update_base (cpu_base); + expires_next =hrtimer_update_next_event (cpu_base);
Quoted 8 lines
kernel/time/hrtimer.c void hrtimer_interrupt(struct clock_even · line 1994
Quoted 4 lines
kernel/time/hrtimer.c +4
-0+__hrtimer_rearm (cpu_base, now, expires_next); +clear_thread_flag (TIF_HRTIMER_REARM ); + } +}I'm curious to why you decided to use
scoped_guard()here and not justguard()and not add the extra indentation? The function is small enough where everything is expected to be protected by the spinlock.
Yeah, I'm not entirely sure... its been over 6 months since I wrote this code :-/
Re: [PATCH v2 5/6] entry,hrtimer: Push reprogramming timers into the interrupt return path
In reply to Peter Zijlstrakernel/entry/common.c static __always_inline unsigned long __e · line 38
Quoted 11 lines · Peter Zijlstra
kernel/entry/common.c +9
-0while (ti_work &EXIT_TO_USER_MODE_WORK_LOOP ) { +/* +* If hrtimer need re-arming, do so before enabling IRQs, +* except when a reschedule is needed, in that case schedule() +* will do this. +*/ +if ((ti_work & (_TIF_NEED_RESCHED | + _TIF_NEED_RESCHED_LAZY | + _TIF_HRTIMER_REARM)) == _TIF_HRTIMER_REARM) +hrtimer_rearm ();
Two things I'm not convinced that they are handled correctly:
1) Interrupts
After reenabling interrupts and before reaching schedule() an interrupt comes in and runs soft interrupt processing for a while on the way back, which delays the update until that processing completes.
2) Time slice extension
When the time slice is granted this will not rearm the clockevent device unless the slice hrtimer becomes the first expiring timer on that CPU, but even then that misses the full reevaluation of the next timer event.
Quoted 10 lines
kernel/time/hrtimer.c +2
-3-static void __hrtimer_rearm (struct hrtimer_cpu_base *cpu_base,ktime_t now) +static void __hrtimer_rearm (struct hrtimer_cpu_base *cpu_base, +ktime_t now,ktime_t expires_next) { -ktime_t expires_next =hrtimer_update_next_event (cpu_base); - cpu_base->expires_next = expires_next; cpu_base->in_hrtirq =0 ;
kernel/time/hrtimer.c void hrtimer_interrupt(struct clock_even · line 1976
Quoted 9 lines
kernel/time/hrtimer.c +5
-1@@ -1970,9 +1969,30 @@ void hrtimer_interrupt(struct clock_even cpu_base->hang_detected =1 ; } -__hrtimer_rearm (cpu_base, now); +#ifdef TIF_HRTIMER_REARM +set_thread_flag (TIF_HRTIMER_REARM ); +#else +__hrtimer_rearm (cpu_base, now, expires_next); +#endif
in hrtimer.h where you already have the #ifdef TIF_HRTIMER_REARM section:
static inline bool hrtimer_set_rearm_delayed()
{
set_thread_flag(TIF_HRTIMER_REARM);
return true;
}and a empty stub returning false for the other case then this becomes:
if (!hrtimer_set_rearm_delayed())
hrtimer_rearm(...);and the ugly ifdef in the code goes away.
kernel/time/hrtimer.c void hrtimer_interrupt(struct clock_even · line 1981
Quoted 5 lines
kernel/time/hrtimer.c +3
-0raw_spin_unlock_irqrestore (&cpu_base->lock, flags); } + +#ifdef TIF_HRTIMER_REARM +void _hrtimer_rearm (void )
Grr. I had to read this five times to figure out that we now have
hrtimer_rearm()
_hrtimer_rearm()
__hrtimer_rearm()
You clearly ran out of characters to make that obvious:
hrtimer_rearm_delayed()
hrtimer_rearm()
hrtimer_do_rearm()
or something like that.
Thanks,
tglx
Peter Zijlstra
maintainerRe: [PATCH v2 5/6] entry,hrtimer: Push reprogramming timers into the interrupt return path
Quoted 15 lines · Thomas Gleixner
kernel/entry/common.c static __always_inline unsigned long __e · line 38
Quoted 11 lines · Peter Zijlstra
kernel/entry/common.c +9
-0while (ti_work &EXIT_TO_USER_MODE_WORK_LOOP ) { +/* +* If hrtimer need re-arming, do so before enabling IRQs, +* except when a reschedule is needed, in that case schedule() +* will do this. +*/ +if ((ti_work & (_TIF_NEED_RESCHED | + _TIF_NEED_RESCHED_LAZY | + _TIF_HRTIMER_REARM)) == _TIF_HRTIMER_REARM) +hrtimer_rearm ();Two things I'm not convinced that they are handled correctly:
Quoted 6 lines
1) Interrupts
After reenabling interrupts and before reaching
schedule()an interrupt comes in and runs soft interrupt processing for a while on the way back, which delays the update until that processing completes.
So the basic thing looks like:
<USER-MODE> irqentry_enter() run_irq_on_irqstack_cond() if (user_mode() || hardirq_stack_inuse) irq_enter_rcu(); func_c(); irq_exit_rcu() __irq_exit_rcu() invoke_softirq() irqentry_exit() irqentry_exit_to_user_mode() irqentry_exit_to_user_mode_prepare() __exit_to_user_mode_prepare() exit_to_user_mode_loop() ...here...
So a nested IRQ at this point will have !user_mode(), but I think it can still end up in softirqs due to that hardirq_stack_inuse. Should we perhaps make sure only user_mode() ends up in softirqs?
Quoted 6 lines
2) Time slice extension
When the time slice is granted this will not rearm the clockevent device unless the slice hrtimer becomes the first expiring timer on that CPU, but even then that misses the full reevaluation of the next timer event.
Oh crud yes, that should be something like:
if (!rseq_grant_slice_extension(ti_work & TIF_SLICE_EXT_DENY)) schedule(); else hrtimer_rearm();
Thomas Gleixner
maintainerRe: [PATCH v2 5/6] entry,hrtimer: Push reprogramming timers into the interrupt return path
Quoted 41 lines · Peter Zijlstra
Quoted 15 lines · Thomas Gleixner
kernel/entry/common.c static __always_inline unsigned long __e · line 38
Quoted 11 lines · Peter Zijlstra
kernel/entry/common.c +9
-0while (ti_work &EXIT_TO_USER_MODE_WORK_LOOP ) { +/* +* If hrtimer need re-arming, do so before enabling IRQs, +* except when a reschedule is needed, in that case schedule() +* will do this. +*/ +if ((ti_work & (_TIF_NEED_RESCHED | + _TIF_NEED_RESCHED_LAZY | + _TIF_HRTIMER_REARM)) == _TIF_HRTIMER_REARM) +hrtimer_rearm ();Two things I'm not convinced that they are handled correctly:
Quoted 6 lines
1) Interrupts
After reenabling interrupts and before reaching
schedule()an interrupt comes in and runs soft interrupt processing for a while on the way back, which delays the update until that processing completes.So the basic thing looks like:
<USER-MODE> irqentry_enter() run_irq_on_irqstack_cond() if (user_mode() || hardirq_stack_inuse) irq_enter_rcu(); func_c(); irq_exit_rcu() __irq_exit_rcu() invoke_softirq() irqentry_exit() irqentry_exit_to_user_mode() irqentry_exit_to_user_mode_prepare() __exit_to_user_mode_prepare() exit_to_user_mode_loop() ...here...
Quoted 3 lines
So a nested IRQ at this point will have !user_mode(), but I think it can still end up in softirqs due to that
hardirq_stack_inuse. Should we perhaps make sure onlyuser_mode()ends up in softirqs?
All interrupts independent of the mode they hit are ending up in irq_exit_rcu() and therefore in __irq_exit_rcu()
run_irq_on_irqstack_cond()
if (user_mode() || hardirq_stack_inuse)
// Stay on user or hardirq stack
irq_enter_rcu();
func_c();
irq_exit_rcu()
else
// MAGIC ASM to switch to hardirq stack
call irq_enter_rcu
call func_c
call irq_exit_rcuThe only reason why invoke_softirq() won't be called is when the interrupt hits into the softirq processing region of the previous interrupt, which means it's already on the hardirq stack.
But looking at this there is already a problem without interrupt nesting:
irq_enter_rcu();
timer_interrupt()
hrtimer_interrupt()
delay_rearm();
irq_exit_rcu()
__irq_exit_rcu()
invoke_softirq() <- HereSoft interrupts can run for quite some time, which means this already can cause timers being delayed for way too long. I think in __irq_exit_rcu() you want to do:
if (!in_interrupt() && local_softirq_pending()) {
hrtimer_rearm();
invoke_softirq();
}Thanks,
tglx
Thomas Gleixner
maintainerRe: [PATCH v2 5/6] entry,hrtimer: Push reprogramming timers into the interrupt return path
Quoted 20 lines · Thomas Gleixner
Quoted 3 lines · Peter Zijlstra
So a nested IRQ at this point will have !user_mode(), but I think it can still end up in softirqs due to that
hardirq_stack_inuse. Should we perhaps make sure onlyuser_mode()ends up in softirqs?All interrupts independent of the mode they hit are ending up in
irq_exit_rcu()and therefore in__irq_exit_rcu()(quoted diff) +0
-0run_irq_on_irqstack_cond() if (user_mode() || hardirq_stack_inuse) // Stay on user or hardirq stack irq_enter_rcu(); func_c(); irq_exit_rcu() else // MAGIC ASM to switch to hardirq stack call irq_enter_rcu call func_c call irq_exit_rcu
Quoted 3 lines
The only reason why
invoke_softirq()won't be called is when the interrupt hits into the softirq processing region of the previous interrupt, which means it's already on the hardirq stack.
In the case I pointed out where the second interrupt hits right after exit to user enabled interupts, there is no nesting and it will happily take the second path which switches to the hardirq stack and then on return processes soft interrupts.
Quoted 11 lines
But looking at this there is already a problem without interrupt nesting:
irq_enter_rcu(); timer_interrupt() hrtimer_interrupt() delay_rearm(); irq_exit_rcu() __irq_exit_rcu() invoke_softirq() <- Here
Quoted 8 lines
Soft interrupts can run for quite some time, which means this already can cause timers being delayed for way too long. I think in
__irq_exit_rcu()you want to do:if (!in_interrupt() && local_softirq_pending()) { hrtimer_rearm(); invoke_softirq(); }
Actually it's worse. Assume the CPU on which this happens has the jiffies duty. As the timer does not fire, jiffies become stale. So anything which relies on jiffies going forward will get stuck until some other condition breaks the tie. That's going to be fun to debug :)
Thanks,
tglx
Peter Zijlstra
maintainerRe: [PATCH v2 5/6] entry,hrtimer: Push reprogramming timers into the interrupt return path
In reply to Thomas GleixnerQuoted 11 lines · Thomas Gleixner
But looking at this there is already a problem without interrupt nesting:
irq_enter_rcu(); timer_interrupt() hrtimer_interrupt() delay_rearm(); irq_exit_rcu() __irq_exit_rcu() invoke_softirq() <- Here
Quoted 8 lines
Soft interrupts can run for quite some time, which means this already can cause timers being delayed for way too long. I think in
__irq_exit_rcu()you want to do:if (!in_interrupt() && local_softirq_pending()) { hrtimer_rearm(); invoke_softirq(); }
Right, and we can do the same on (nested) IRQ entry. Something like so:
---
kernel/entry/common.c +2 -0
@@ -63,6 +63,8 @@ static __always_inline unsigned long __eif (ti_work & (_TIF_NEED_RESCHED | _TIF_NEED_RESCHED_LAZY)) {if (!rseq_grant_slice_extension (ti_work &TIF_SLICE_EXT_DENY ))schedule (); +else +hrtimer_rearm (); }if (ti_work & _TIF_UPROBE)
kernel/softirq.c +14 -1
@@ -663,6 +663,13 @@ void irq_enter_rcu(void) {__irq_enter_raw (); +/* +* If this is a nested IRQ that hits the exit_to_user_mode_loop +* where it has enabled IRQs but before it has hit schedule() +* we could have hrtimers in an undefined state. Fix it up here. +*/ +hrtimer_rearm (); +if (tick_nohz_full_cpu (smp_processor_id ()) || (is_idle_task (current) && (irq_count () ==HARDIRQ_OFFSET )))tick_irq_enter ();
@@ -719,8 +726,14 @@ static inline void __irq_exit_rcu(void)#endif account_hardirq_exit (current);preempt_count_sub (HARDIRQ_OFFSET ); -if (!in_interrupt () &&local_softirq_pending ()) +if (!in_interrupt () &&local_softirq_pending ()) { +/* +* If we left hrtimers unarmed, make sure to arm them now, +* before enabling interrupts to run SoftIRQ. +*/ +hrtimer_rearm ();invoke_softirq (); + }if (IS_ENABLED (CONFIG_IRQ_FORCED_THREADING ) &&force_irqthreads () &&local_timers_pending_force_th () && !(in_nmi () |in_hardirq ()))
kernel/time/hrtimer.c +3 -3
@@ -1279,8 +1279,8 @@ static int __hrtimer_start_range_ns(struif (timer->is_fuzzy) {/* -* XXX fuzzy implies pinned! not sure how to deal with -* retrigger_next_event() for the !local case. +* Fuzzy requires pinned as the lazy programming only works +* for CPU local timers. */ WARN_ON_ONCE (!(mode &HRTIMER_MODE_PINNED ));/*
@@ -1898,7 +1898,7 @@ static __latent_entropy void hrtimer_run/* * Very similar to hrtimer_force_reprogram(), except it deals with -* in_hrirq and hang_detected. +* in_hrtirq and hang_detected. */ static void __hrtimer_rearm (struct hrtimer_cpu_base *cpu_base,ktime_t now,ktime_t expires_next)
Peter Zijlstra
maintainerPATCH v2 6/6 sched: Default enable HRTICK
In reply to Peter Zijlstra... for generic entry architectures. This decouples preemption from CONFIG_HZ, leaving only the periodic load-balancer and various accounting things relying on the tick.
--- kernel/sched/features.h | 5 +++++ 1 file changed, 5 insertions(+)
kernel/sched/features.h +5 -0
@@ -63,8 +63,13 @@ SCHED_FEAT(DELAY_ZERO, true)*/ SCHED_FEAT(WAKEUP_PREEMPTION, true) +#ifdef TIF_HRTIMER_REARM +SCHED_FEAT(HRTICK, true) +SCHED_FEAT(HRTICK_DL, true) +#else SCHED_FEAT(HRTICK, false) SCHED_FEAT(HRTICK_DL, false) +#endif /* * Decrement CPU capacity based on time not spent running tasks
Re: [PATCH v2 6/6] sched: Default enable HRTICK
Hi Peter,
Quoted 12 lines · On Wed, Jan 21, 2026 at 05:20:16PM +0100 Peter Zijlstra wrote:
... for generic entry architectures. This decouples preemption from
CONFIG_HZ, leaving only the periodic load-balancer and various accounting things relying on the tick.Signed-off-by Peter Zijlstra (Intel) <[email protected]>--- kernel/sched/features.h | 5 +++++ 1 file changed, 5 insertions(+)
kernel/sched/features.h SCHED_FEAT(DELAY_ZERO, true) · line 72
Quoted 10 lines
kernel/sched/features.h +5
-0@@ -63,8 +63,13 @@ SCHED_FEAT(DELAY_ZERO, true)*/ SCHED_FEAT(WAKEUP_PREEMPTION, true) +#ifdef TIF_HRTIMER_REARM +SCHED_FEAT(HRTICK, true) +SCHED_FEAT(HRTICK_DL, true) +#else SCHED_FEAT(HRTICK, false) SCHED_FEAT(HRTICK_DL, false) +#endif
I maybe be missing something. But the title of this patch and the above code do not seem to match.
Cheers,
Phil
Quoted 6 lines
kernel/sched/features.h +0
-0/* * Decrement CPU capacity based on time not spent running tasks
Peter Zijlstra
maintainerRe: [PATCH v2 6/6] sched: Default enable HRTICK
Quoted 12 lines · Phil Auld
Hi Peter,
Quoted 9 lines · On Wed, Jan 21, 2026 at 05:20:16PM +0100 Peter Zijlstra wrote:
... for generic entry architectures. This decouples preemption from
CONFIG_HZ, leaving only the periodic load-balancer and various accounting things relying on the tick.Signed-off-by Peter Zijlstra (Intel) <[email protected]>--- kernel/sched/features.h | 5 +++++ 1 file changed, 5 insertions(+)
Quoted 7 lines
kernel/sched/features.h SCHED_FEAT(DELAY_ZERO, true) · line 66
Quoted 7 lines
kernel/sched/features.h +1
-0@@ -63,8 +63,13 @@ SCHED_FEAT(DELAY_ZERO, true)*/ SCHED_FEAT(WAKEUP_PREEMPTION, true) +#ifdef TIF_HRTIMER_REARM
Arguably this should be CONFIG_GENERIC_ENTRY I suppose
Quoted 9 lines
kernel/sched/features.h SCHED_FEAT(DELAY_ZERO, true) · line 72
Quoted 6 lines
kernel/sched/features.h +4
-0+SCHED_FEAT(HRTICK, true) +SCHED_FEAT(HRTICK_DL, true) +#else SCHED_FEAT(HRTICK, false) SCHED_FEAT(HRTICK_DL, false) +#endif I maybe be missing something. But the title of this patch and the above code do not seem to match.
You mean it only default enables it for a subset of architectures?
Re: [PATCH v2 6/6] sched: Default enable HRTICK
Quoted 33 lines · On Thu, Jan 22, 2026 at 12:40:54PM +0100 Peter Zijlstra wrote:
Quoted 12 lines · Phil Auld
Hi Peter,
Quoted 9 lines · On Wed, Jan 21, 2026 at 05:20:16PM +0100 Peter Zijlstra wrote:
... for generic entry architectures. This decouples preemption from
CONFIG_HZ, leaving only the periodic load-balancer and various accounting things relying on the tick.Signed-off-by Peter Zijlstra (Intel) <[email protected]>--- kernel/sched/features.h | 5 +++++ 1 file changed, 5 insertions(+)Quoted 7 lines
kernel/sched/features.h SCHED_FEAT(DELAY_ZERO, true) · line 66
Quoted 7 lines
kernel/sched/features.h +1
-0@@ -63,8 +63,13 @@ SCHED_FEAT(DELAY_ZERO, true)*/ SCHED_FEAT(WAKEUP_PREEMPTION, true) +#ifdef TIF_HRTIMER_REARM Arguably this should be
CONFIG_GENERIC_ENTRYI supposeQuoted 9 lines
kernel/sched/features.h SCHED_FEAT(DELAY_ZERO, true) · line 72
Quoted 6 lines
kernel/sched/features.h +4
-0+SCHED_FEAT(HRTICK, true) +SCHED_FEAT(HRTICK_DL, true) +#else SCHED_FEAT(HRTICK, false) SCHED_FEAT(HRTICK_DL, false) +#endif I maybe be missing something. But the title of this patch and the above code do not seem to match.
Quoted 2 lines
You mean it only default enables it for a subset of architectures?
Nope, I mean I can't read... nevermind.
Cheers,
Phil