posix-cpu-timers: Move expiry into task work context
Subsystems (2)
v2 cover posix-cpu-timers: Move expiry into task work context
This is the second attempt of moving the posix CPU timer heavy lifting out of interrupt context. The initial version can be found here:
https://lore.kernel.org/r/[email protected]
Running posix cpu timers in hard interrupt context has a few downsides:
- For PREEMPT_RT it cannot work as the expiry code needs to take sighand lock, which is a 'sleeping spinlock' in RT
- For fine grained accounting it's just wrong to run this in context of the timer interrupt because that way a process specific cpu time is accounted to the timer interrupt.
- Long running timer interrupts caused by a large amount of expiring timers which can be created and armed by unpriviledged user space.
There is no real hard requirement to run the expiry code in hard interrupt context. The posix CPU timers are an approximation anyway, so having them expired and evaluated in task work context does not really make them worse.
This version is more or less a resend of the posix CPU timer bits of V1. The back then observed issue that KVM does not handle pending task work before going into guest mode has been solved differently. The related patches have been posted here:
https://lore.kernel.org/r/[email protected]
This patch series has no code dependency on the entry/KVM work, but the functional dependency vs. KVM exists.
It applies on mainline and passes all tests - except when KVM is active and timers are armed on the KVM threads. This particular issue is solved when the entry/KVM series is applied as well.
The entry/KVM changes are available from:
git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git x86/entry
and the whole lot (entry/kvm + posix CPU timers) is available from:
git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git timers/posixtimer
Thanks,
tglx
----
b/arch/x86/Kconfig | 1 include/linux/posix-timers.h | 33 ++++-- include/linux/sched/cputime.h | 2 kernel/time/Kconfig | 5 kernel/time/posix-cpu-timers.c | 213 ++++++++++++++++++++++++++++++----------- 5 files changed, 192 insertions(+), 62 deletions(-)
PATCH v2 1/5 posix-cpu-timers: Split run_posix_cpu_timers()
Split it up as a preparatory step to move the heavy lifting out of interrupt context.
--- kernel/time/posix-cpu-timers.c | 38 ++++++++++++++++++++++---------------- 1 file changed, 22 insertions(+), 16 deletions(-)
kernel/time/posix-cpu-timers.c +22 -16
@@ -1080,27 +1080,12 @@ static inline bool fastpath_timer_check(return false; } -/* -* This is called from the timer interrupt handler. The irq handler has -* already updated our counts. We need to check if any timers fire now. -* Interrupts are disabled. -*/ -void run_posix_cpu_timers (void ) +static void __run_posix_cpu_timers (struct task_struct *tsk) { -struct task_struct *tsk = current;struct k_itimer *timer, *next;unsigned long flags;LIST_HEAD (firing); -lockdep_assert_irqs_disabled (); - -/* -* The fast path checks that there are no expired thread or thread -* group timers. If that's so, just return. -*/ -if (!fastpath_timer_check (tsk)) -return ; -lockdep_posixtimer_enter ();if (!lock_task_sighand (tsk, &flags)) {lockdep_posixtimer_exit ();
@@ -1151,6 +1136,27 @@ void run_posix_cpu_timers(void) }/* +* This is called from the timer interrupt handler. The irq handler has +* already updated our counts. We need to check if any timers fire now. +* Interrupts are disabled. +*/ +void run_posix_cpu_timers (void ) +{ +struct task_struct *tsk = current; + +lockdep_assert_irqs_disabled (); + +/* +* The fast path checks that there are no expired thread or thread +* group timers. If that's so, just return. +*/ +if (!fastpath_timer_check (tsk)) +return ; + +__run_posix_cpu_timers (tsk); +} + +/* * Set one of the process-wide special case CPU timers or RLIMIT_CPU. * The tsk->sighand->siglock must be held by the caller. */
PATCH v2 4/5 posix-cpu-timers: Expiry timers directly when in task work context
In reply to Thomas GleixnerIf the expiry happens in task context, there is no point in collecting the expired timers on a list first. Just expire them directly.
--- kernel/time/posix-cpu-timers.c | 92 +++++++++++++++++++++++++++++++++++------ 1 file changed, 79 insertions(+), 13 deletions(-)
kernel/time/posix-cpu-timers.c +79 -13
@@ -753,13 +753,71 @@ static void posix_cpu_timer_get(struct k#define MAX_COLLECTED 20 -static u64 collect_timerqueue (struct timerqueue_head *head, +#ifdef CONFIG_POSIX_CPU_TIMERS_TASK_WORK +/* +* Expiry in task work context. Access to sighand->siglock is safe here. +*/ +static void handle_expired_timer (struct cpu_timer *ctmr, +struct list_head *firing) +{ +struct k_itimer *timer; +int cpu_firing; + +/* +* Unlock sighand lock so the timer can be locked. Keep interrupts +* disabled accross the lock switch. +*/ +spin_unlock (¤t->sighand->siglock); + timer = container_of(ctmr, struct k_itimer, it.cpu); +spin_lock (&timer->it_lock); + cpu_firing = timer->it.cpu.firing; + timer->it.cpu.firing =0 ; +/* +* The firing flag is -1 if this raced with a reset of the timer, +* which already reported this almost-firing as an overrun. So +* don't generate an event. +*/ +if (likely (cpu_firing >=0 )) +cpu_timer_fire (timer); +/* +* Drop timer lock again and reacquire sighand lock. Allow +* interrupts to come in between so this wont block interrupts +* accross the delivery of a gazillion of timers. +*/ +spin_unlock_irq (&timer->it_lock); +spin_lock_irq (¤t->sighand->siglock); +} +#else +/* +* Expiry in interupt context. Just move them to the firing list. +*/ +static void handle_expired_timer (struct cpu_timer *ctmr, +struct list_head *firing) +{ +list_add_tail (&ctmr->elist, firing); +} +#endif + +static u64 collect_timerqueue (struct posix_cputimer_base *base,struct list_head *firing,u64 now) {struct timerqueue_node *next;int i =0 ; -while ((next =timerqueue_getnext (head))) { +/* +* Reset the expiry cache first when expiry context is task work. +* This is required because when sighand lock is dropped new timers +* can be enqueued. That's not a problem for regular posix timers +* as the expiry time would be correct when expire_timerqueue() +* returns, but the expiry cache is also used by itimers which do +* not have a corresponding posix timer and therefore a simple +* update after expire_timerqueue() might overwrite their newly +* written expiry time. +*/ +if (IS_ENABLED (CONFIG_POSIX_CPU_TIMERS_TASK_WORK )) + base->nextevt =U64_MAX ; + +while ((next =timerqueue_getnext (&base->tqhead))) {struct cpu_timer *ctmr;u64 expires;
@@ -771,7 +829,7 @@ static u64 collect_timerqueue(struct tim ctmr->firing =1 ;cpu_timer_dequeue (ctmr); -list_add_tail (&ctmr->elist, firing); +handle_expired_timer (ctmr, firing); }return U64_MAX ;
@@ -783,10 +841,8 @@ static void collect_posix_cputimers(strustruct posix_cputimer_base *base = pct->bases;int i; -for (i =0 ; i <CPUCLOCK_MAX ; i++, base++) { - base->nextevt =collect_timerqueue (&base->tqhead, firing, - samples[i]); - } +for (i =0 ; i <CPUCLOCK_MAX ; i++, base++) + base->nextevt =collect_timerqueue (base, firing, samples[i]); }static inline void check_dl_overrun (struct task_struct *tsk)
@@ -812,9 +868,10 @@ static bool check_rlimit(u64 time, u64 l }/* -* Check for any per-thread CPU timers that have fired and move them off -* the tsk->cpu_timers[N] list onto the firing list. Here we update the -* tsk->it_*_expires values to reflect the remaining thread CPU timers. +* Check for any per-thread CPU timers that have fired and depending on the +* context (task work or interrupt) move them off the tsk->cpu_timers[N] +* list onto the firing list or expire them directly. Update the expiry +* cache as well to reflect the remaining thread CPU timers. */ static void check_thread_timers (struct task_struct *tsk,struct list_head *firing)
@@ -889,9 +946,11 @@ static void check_cpu_itimer(struct task }/* -* Check for any per-thread CPU timers that have fired and move them -* off the tsk->*_timers list onto the firing list. Per-thread timers -* have already been taken off. +* Check for any per-process CPU timers that have fired and depending on +* the context (task work or interrupt) move them off the tsk->signal timer +* list onto the firing list or expire them directly. Update the expiry +* cache to reflect the resulting state. Per-thread timers have already +* been handled. */ static void check_process_timers (struct task_struct *tsk,struct list_head *firing)
@@ -1115,6 +1174,12 @@ static void handle_posix_cpu_timers(struunlock_task_sighand (tsk, &flags);/* +* If task work delivery is enabled, the timers are already +* expired. +*/ +if (IS_ENABLED (CONFIG_POSIX_CPU_TIMERS_TASK_WORK )) + goto out; +/* * Now that all the timers on our list have the firing flag, * no one will touch their list entries but us. We'll take * each timer's lock before clearing its firing flag, so no
@@ -1136,6 +1201,7 @@ static void handle_posix_cpu_timers(strucpu_timer_fire (timer);spin_unlock (&timer->it_lock); } +out:lockdep_posixtimer_exit (); }
PATCH v2 3/5 posix-cpu-timers: Provide mechanisms to defer timer handling to task_work
In reply to Thomas GleixnerRunning posix cpu timers in hard interrupt context has a few downsides:
- For PREEMPT_RT it cannot work as the expiry code needs to take sighand lock, which is a 'sleeping spinlock' in RT. The original RT approach of offloading the posix CPU timer handling into a high priority thread was clumsy and provided no real benefit in general.
- For fine grained accounting it's just wrong to run this in context of the timer interrupt because that way a process specific cpu time is accounted to the timer interrupt.
- Long running timer interrupts caused by a large amount of expiring timers which can be created and armed by unpriviledged user space.
There is no hard requirement to expire them in interrupt context.
Provide infrastructure to schedule task work which allows splitting the posix CPU timer code into a quick check in interrupt context and a thread context expiry and signal delivery function. This has to be enabled by architectures as it requires that the architecture specific KVM implementation handles pending task work before exiting to guest mode.
--- include/linux/posix-timers.h | 17 ++++++++++++++++ kernel/time/Kconfig | 5 ++++ kernel/time/posix-cpu-timers.c | 42 ++++++++++++++++++++++++++++++++++++++++- 3 files changed, 63 insertions(+), 1 deletion(-)
include/linux/posix-timers.h +17 -0
@@ -112,25 +112,42 @@ struct posix_cputimer_base {enum {CPUTIMERS_ACTIVE ,CPUTIMERS_EXPIRING , +CPUTIMERS_WORK_SCHEDULED , };/** * posix_cputimers - Container for posix CPU timer related data * @bases: Base container for posix CPU clocks * @flags: Flags for various CPUTIMERS_* states +* @task_work: Task work to defer timer expiry into task context * Used in task_struct and signal_struct */ struct posix_cputimers {struct posix_cputimer_base bases[CPUCLOCK_MAX ];unsigned long flags; +#ifdef CONFIG_POSIX_CPU_TIMERS_TASK_WORK +struct callback_head task_work; +#endif }; +#ifdef CONFIG_POSIX_CPU_TIMERS_TASK_WORK +void posix_cpu_timers_work (struct callback_head *work); + +static inline void posix_cputimer_init_work (struct posix_cputimers *pct) +{ + pct->task_work.func = posix_cpu_timers_work; +} +#else +static inline void posix_cputimer_init_work (struct posix_cputimers *pct) { } +#endif +static inline void posix_cputimers_init (struct posix_cputimers *pct) {memset (pct,0 ,sizeof (*pct)); pct->bases[0 ].nextevt =U64_MAX ; pct->bases[1 ].nextevt =U64_MAX ; pct->bases[2 ].nextevt =U64_MAX ; +posix_cputimer_init_work (pct); }void posix_cputimers_group_init (struct posix_cputimers *pct,u64 cpu_limit);
kernel/time/Kconfig +5 -0
@@ -52,6 +52,11 @@ config GENERIC_CLOCKEVENTS_MIN_ADJUST config GENERIC_CMOS_UPDATE bool +# Select to handle posix CPU timers from task_work +# and not from the timer interrupt context +config POSIX_CPU_TIMERS_TASK_WORK + bool + if GENERIC_CLOCKEVENTS menu "Timers subsystem"
kernel/time/posix-cpu-timers.c +41 -1
66 lines not on this page · raw message
PATCH v2 5/5 x86: Select POSIX_CPU_TIMERS_TASK_WORK
In reply to Thomas GleixnerMove POSIX CPU timer expiry and signal delivery into task context.
--- arch/x86/Kconfig | 1 + 1 file changed, 1 insertion(+)
arch/x86/Kconfig +1 -0
@@ -224,6 +224,7 @@ config X86 select NEED_SG_DMA_LENGTH select PCI_DOMAINS if PCI select PCI_LOCKLESS_CONFIG if PCI + select POSIX_CPU_TIMERS_TASK_WORK select PERF_EVENTS select RTC_LIB select RTC_MC146818_LIB
PATCH v2 2/5 posix-cpu-timers: Convert the flags to a bitmap
In reply to Thomas GleixnerFor splitting the posix cpu timer code into an interrupt context check and the actual expiry and signal handling it's required to be able to set and test for the flags atomically.
--- include/linux/posix-timers.h | 16 +++++++-------- include/linux/sched/cputime.h | 2 - kernel/time/posix-cpu-timers.c | 43 ++++++++++++++++++----------------------- 3 files changed, 28 insertions(+), 33 deletions(-)
include/linux/posix-timers.h +8 -8
@@ -109,20 +109,20 @@ struct posix_cputimer_base {struct timerqueue_head tqhead; }; +enum { +CPUTIMERS_ACTIVE , +CPUTIMERS_EXPIRING , +}; +/** * posix_cputimers - Container for posix CPU timer related data -* @bases: Base container for posix CPU clocks -* @timers_active: Timers are queued. -* @expiry_active: Timer expiry is active. Used for -* process wide timers to avoid multiple -* task trying to handle expiry concurrently -* +* @bases: Base container for posix CPU clocks +* @flags: Flags for various CPUTIMERS_* states * Used in task_struct and signal_struct */ struct posix_cputimers {struct posix_cputimer_base bases[CPUCLOCK_MAX ]; -unsigned int timers_active; -unsigned int expiry_active; +unsigned long flags; };static inline void posix_cputimers_init (struct posix_cputimers *pct)
include/linux/sched/cputime.h +1 -1
@@ -84,7 +84,7 @@ struct thread_group_cputimer *get_runnin* Check whether posix CPU timers are active. If not the thread * group accounting is not active either. Lockless check. */ -if (!READ_ONCE (tsk->signal->posix_cputimers.timers_active)) +if (!test_bit (CPUTIMERS_ACTIVE , &tsk->signal->posix_cputimers.flags))return NULL ;/*
kernel/time/posix-cpu-timers.c +19 -24
104 lines not on this page · raw message
Re: [patch V2 3/5] posix-cpu-timers: Provide mechanisms to defer timer handling to task_work
In reply to Thomas Gleixnerkernel/time/posix-cpu-timers.c static void __run_posix_cpu_timers(struc · line 1110
Quoted 13 lines · Thomas Gleixner
kernel/time/posix-cpu-timers.c +6
-0@@ -1096,6 +1099,12 @@ static void __run_posix_cpu_timers(struccheck_process_timers (tsk, &firing);/* +* Allow new work to be scheduled. The expiry cache +* is up to date. +*/ +posix_cpu_timers_enable_work (tsk); + +/* * We must release these locks before taking any timer's lock. * There is a potential race with timer deletion here, as the * siglock now protects our private firing list. We have set
I think I would feel more comfortable if this was done at the very beginning of that function, possibly even with:
kernel/time/posix-cpu-timers.c static void __run_posix_cpu_timers(struc · line 1159
Quoted 11 lines
kernel/time/posix-cpu-timers.c +11
-0+static void __run_posix_cpu_timers (struct task_struct *tsk) +{ +struct posix_cputimers *pct = &tsk->posix_cputimers; + +if (!test_and_set_bit (CPUTIMERS_WORK_SCHEDULED , &pct->flags)) +task_work_add (tsk, &pct->task_work, true); +} + +static inline void posix_cpu_timers_enable_work (struct task_struct *tsk) +{ +clear_bit (CPUTIMERS_WORK_SCHEDULED , &tsk->posix_cputimers.flags);
/* * Ensure we observe everything before a failing test_and_set() * in __run_posix_cpu_timers(). */ smp_mb__after_atomic();
Quoted 1 line
+}
Such that when another timer interrupt happens while we run this, we're guaranteed to either see it, or get re-queued and thus re-run the function.
Re: [patch V2 3/5] posix-cpu-timers: Provide mechanisms to defer timer handling to task_work
In reply to Thomas Gleixnerkernel/time/posix-cpu-timers.c static void __run_posix_cpu_timers(struc · line 1154
Quoted 6 lines · Thomas Gleixner
kernel/time/posix-cpu-timers.c +6
-0+static void __run_posix_cpu_timers (struct task_struct *tsk) +{ +struct posix_cputimers *pct = &tsk->posix_cputimers; + +if (!test_and_set_bit (CPUTIMERS_WORK_SCHEDULED , &pct->flags)) +task_work_add (tsk, &pct->task_work, true);
s/true/TWA_RESUME/g
see: e91b48162332 ("task_work: teach task_work_add() to do signal_wake_up()")
Quoted 1 line
+}
Re: [patch V2 3/5] posix-cpu-timers: Provide mechanisms to defer timer handling to task_work
In reply to Thomas GleixnerLooks correct to me, but I forgot everything about posix-timers.c
this obviously means that the expired timer won't fire until the task returns to user-mode but probably we don't care.
One cosmetic nit below,
include/linux/posix-timers.h struct posix_cputimer_base { · line 138
Quoted 7 lines · Thomas Gleixner
include/linux/posix-timers.h +6
-0+#ifdef CONFIG_POSIX_CPU_TIMERS_TASK_WORK +void posix_cpu_timers_work (struct callback_head *work); + +static inline void posix_cputimer_init_work (struct posix_cputimers *pct) +{ + pct->task_work.func = posix_cpu_timers_work;
init_task_work() ?
include/linux/posix-timers.h struct posix_cputimer_base { · line 151
Quoted 13 lines
include/linux/posix-timers.h +6
-0+} +#else +static inline void posix_cputimer_init_work (struct posix_cputimers *pct) { } +#endif +static inline void posix_cputimers_init (struct posix_cputimers *pct) {memset (pct,0 ,sizeof (*pct)); pct->bases[0 ].nextevt =U64_MAX ; pct->bases[1 ].nextevt =U64_MAX ; pct->bases[2 ].nextevt =U64_MAX ; +posix_cputimer_init_work (pct); }
And I can't resist. I know this is a common practice, please ignore, but to me
static inline void posix_cputimers_init(struct posix_cputimers *pct)
{
memset(pct, 0, sizeof(*pct));
pct->bases[0].nextevt = U64_MAX;
pct->bases[1].nextevt = U64_MAX;
pct->bases[2].nextevt = U64_MAX;
#ifdef CONFIG_POSIX_CPU_TIMERS_TASK_WORK
init_task_work(&pct->task_work, posix_cpu_timers_work);
#endif
}looks better than 2 posix_cputimer_init_work() definitions above.
Note also that signal_struct->posix_cputimers.task_work is never used, perhaps it would be better to move this task_work into task_struct? This way we do not even need to change posix_cputimers_init(), we call simply initialize init_task.posix_task_work.
Oleg.
Re: [patch V2 3/5] posix-cpu-timers: Provide mechanisms to defer timer handling to task_work
Oleg,
Quoted 1 line · Oleg Nesterov
Looks correct to me, but I forgot everything about
posix-timers.c
that's not a problem because this is about posix-cpu-timers.c :)
Quoted 2 lines
this obviously means that the expired timer won't fire until the task returns to user-mode but probably we don't care.
If the signal goes to the task itself it does not matter at all because it's going to be delivered when the task goes out to user space.
If the signal goes to a supervisor process, then it will be slightly delayed but I could not find a problem with that at all.
I'll add more reasoning to the changelog on V3.
Quoted 8 lines
include/linux/posix-timers.h struct posix_cputimer_base { · line 138
Quoted 6 lines
include/linux/posix-timers.h +6
-0+#ifdef CONFIG_POSIX_CPU_TIMERS_TASK_WORK +void posix_cpu_timers_work (struct callback_head *work); + +static inline void posix_cputimer_init_work (struct posix_cputimers *pct) +{ + pct->task_work.func = posix_cpu_timers_work;
init_task_work()?
Yeah.
Quoted 27 lines
include/linux/posix-timers.h struct posix_cputimer_base { · line 151
Quoted 13 lines
include/linux/posix-timers.h +6
-0+} +#else +static inline void posix_cputimer_init_work (struct posix_cputimers *pct) { } +#endif +static inline void posix_cputimers_init (struct posix_cputimers *pct) {memset (pct,0 ,sizeof (*pct)); pct->bases[0 ].nextevt =U64_MAX ; pct->bases[1 ].nextevt =U64_MAX ; pct->bases[2 ].nextevt =U64_MAX ; +posix_cputimer_init_work (pct); }And I can't resist. I know this is a common practice, please ignore, but to me
static inline void posix_cputimers_init(struct posix_cputimers *pct) { memset(pct, 0, sizeof(*pct)); pct->bases[0].nextevt = U64_MAX; pct->bases[1].nextevt = U64_MAX; pct->bases[2].nextevt = U64_MAX; #ifdef CONFIG_POSIX_CPU_TIMERS_TASK_WORK init_task_work(&pct->task_work, posix_cpu_timers_work); #endif }
Quoted 1 line
looks better than 2
posix_cputimer_init_work()definitions above.
Gah, I hate ifdefs in the middle of the code :)
Quoted 4 lines
Note also that
signal_struct->posix_cputimers.task_workis never used, perhaps it would be better to move thistask_workintotask_struct? This way we do not even need to changeposix_cputimers_init(), we call simply initialize init_task.posix_task_work.
Let me look into that.
Thanks,
tglx
Re: [patch V2 3/5] posix-cpu-timers: Provide mechanisms to defer timer handling to task_work
In reply to Peter ZijlstraQuoted 35 lines · Peter Zijlstra
kernel/time/posix-cpu-timers.c static void __run_posix_cpu_timers(struc · line 1110
Quoted 13 lines · Thomas Gleixner
kernel/time/posix-cpu-timers.c +6
-0@@ -1096,6 +1099,12 @@ static void __run_posix_cpu_timers(struccheck_process_timers (tsk, &firing);/* +* Allow new work to be scheduled. The expiry cache +* is up to date. +*/ +posix_cpu_timers_enable_work (tsk); + +/* * We must release these locks before taking any timer's lock. * There is a potential race with timer deletion here, as the * siglock now protects our private firing list. We have set I think I would feel more comfortable if this was done at the very beginning of that function, possibly even with:
kernel/time/posix-cpu-timers.c static void __run_posix_cpu_timers(struc · line 1159
Quoted 11 lines
kernel/time/posix-cpu-timers.c +11
-0+static void __run_posix_cpu_timers (struct task_struct *tsk) +{ +struct posix_cputimers *pct = &tsk->posix_cputimers; + +if (!test_and_set_bit (CPUTIMERS_WORK_SCHEDULED , &pct->flags)) +task_work_add (tsk, &pct->task_work, true); +} + +static inline void posix_cpu_timers_enable_work (struct task_struct *tsk) +{ +clear_bit (CPUTIMERS_WORK_SCHEDULED , &tsk->posix_cputimers.flags);/* * Ensure we observe everything before a failing test_and_set() * in __run_posix_cpu_timers(). */ smp_mb__after_atomic();
Quoted 5 lines
Quoted 1 line
+}
Such that when another timer interrupt happens while we run this, we're guaranteed to either see it, or get re-queued and thus re-run the function.
Makes sense.
Thanks,
tglx
Re: [patch V2 3/5] posix-cpu-timers: Provide mechanisms to defer timer handling to task_work
In reply to Peter ZijlstraQuoted 11 lines · Peter Zijlstra
kernel/time/posix-cpu-timers.c static void __run_posix_cpu_timers(struc · line 1154
Quoted 6 lines · Thomas Gleixner
kernel/time/posix-cpu-timers.c +6
-0+static void __run_posix_cpu_timers (struct task_struct *tsk) +{ +struct posix_cputimers *pct = &tsk->posix_cputimers; + +if (!test_and_set_bit (CPUTIMERS_WORK_SCHEDULED , &pct->flags)) +task_work_add (tsk, &pct->task_work, true);s/true/TWA_RESUME/g
see:
e91b48162332("task_work: teachtask_work_add()to dosignal_wake_up()")
Duh, yes.
Re: [patch V2 3/5] posix-cpu-timers: Provide mechanisms to defer timer handling to task_work
Quoted 11 lines · Thomas Gleixner
Quoted 10 lines · Peter Zijlstra
kernel/time/posix-cpu-timers.c static void __run_posix_cpu_timers(struc · line 1154
Quoted 6 lines · Thomas Gleixner
kernel/time/posix-cpu-timers.c +6
-0+static void __run_posix_cpu_timers (struct task_struct *tsk) +{ +struct posix_cputimers *pct = &tsk->posix_cputimers; + +if (!test_and_set_bit (CPUTIMERS_WORK_SCHEDULED , &pct->flags)) +task_work_add (tsk, &pct->task_work, true);s/true/TWA_RESUME/g
Quoted 3 lines
Quoted 1 line
see:
e91b48162332("task_work: teachtask_work_add()to dosignal_wake_up()")Duh, yes.
Bah, that creates a dependency on sched/core ...
Frederic Weisbecker
maintainerRe: [patch V2 2/5] posix-cpu-timers: Convert the flags to a bitmap
In reply to Thomas Gleixnerkernel/time/posix-cpu-timers.c void posix_cputimers_group_init(struct p · line 28
Quoted 8 lines · Thomas Gleixner
kernel/time/posix-cpu-timers.c +1
-1@@ -25,7 +25,7 @@ void posix_cputimers_group_init(struct pposix_cputimers_init (pct);if (cpu_limit !=RLIM_INFINITY ) { pct->bases[CPUCLOCK_PROF ].nextevt = cpu_limit *NSEC_PER_SEC ; - pct->timers_active = true; +set_bit (CPUTIMERS_ACTIVE , &pct->flags);
I guess this one could be __set_bit().
Quoted 2 lines
kernel/time/posix-cpu-timers.c +0
-0} }
Re: [patch V2 2/5] posix-cpu-timers: Convert the flags to a bitmap
Quoted 11 lines · Frederic Weisbecker
kernel/time/posix-cpu-timers.c void posix_cputimers_group_init(struct p · line 28
Quoted 8 lines · Thomas Gleixner
kernel/time/posix-cpu-timers.c +1
-1@@ -25,7 +25,7 @@ void posix_cputimers_group_init(struct pposix_cputimers_init (pct);if (cpu_limit !=RLIM_INFINITY ) { pct->bases[CPUCLOCK_PROF ].nextevt = cpu_limit *NSEC_PER_SEC ; - pct->timers_active = true; +set_bit (CPUTIMERS_ACTIVE , &pct->flags);I guess this one could be
__set_bit().
True :)
RE: [patch V2 2/5] posix-cpu-timers: Convert the flags to a bitmap
From: Thomas Gleixner
Quoted 15 lines
Sent: 21 July 2020 17:11
Quoted 11 lines · Frederic Weisbecker
kernel/time/posix-cpu-timers.c void posix_cputimers_group_init(struct p · line 28
Quoted 8 lines · Thomas Gleixner
kernel/time/posix-cpu-timers.c +1
-1@@ -25,7 +25,7 @@ void posix_cputimers_group_init(struct pposix_cputimers_init (pct);if (cpu_limit !=RLIM_INFINITY ) { pct->bases[CPUCLOCK_PROF ].nextevt = cpu_limit *NSEC_PER_SEC ; - pct->timers_active = true; +set_bit (CPUTIMERS_ACTIVE , &pct->flags);I guess this one could be
__set_bit().
Quoted 1 line
True :)
Hmmm... does this code need the bit operations to be atmomic? If not then an bitmap is completely the wrong thing to be using.
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
RE: [patch V2 2/5] posix-cpu-timers: Convert the flags to a bitmap
Quoted 16 lines · David Laight
From: Thomas Gleixner
Quoted 15 lines
Sent: 21 July 2020 17:11
Quoted 11 lines · Frederic Weisbecker
kernel/time/posix-cpu-timers.c void posix_cputimers_group_init(struct p · line 28
Quoted 8 lines · Thomas Gleixner
kernel/time/posix-cpu-timers.c +1
-1@@ -25,7 +25,7 @@ void posix_cputimers_group_init(struct pposix_cputimers_init (pct);if (cpu_limit !=RLIM_INFINITY ) { pct->bases[CPUCLOCK_PROF ].nextevt = cpu_limit *NSEC_PER_SEC ; - pct->timers_active = true; +set_bit (CPUTIMERS_ACTIVE , &pct->flags);I guess this one could be
__set_bit().
Quoted 4 lines
Quoted 1 line
True :)
Hmmm... does this code need the bit operations to be atmomic? If not then an bitmap is completely the wrong thing to be using.
Some of it does, otherwise the booleans would have stayed, but I'm reworking parts of it so this might change.
Thanks,
tglx
Re: [patch V2 3/5] posix-cpu-timers: Provide mechanisms to defer timer handling to task_work
In reply to Thomas GleixnerQuoted 2 lines · Thomas Gleixner
Bah, that creates a dependency on sched/core ...
Only when looking at the wrong tree :)
Frederic Weisbecker
maintainerRe: [patch V2 3/5] posix-cpu-timers: Provide mechanisms to defer timer handling to task_work
In reply to Peter ZijlstraQuoted 17 lines · Peter Zijlstra
kernel/time/posix-cpu-timers.c static void __run_posix_cpu_timers(struc · line 1159
Quoted 11 lines · Thomas Gleixner
kernel/time/posix-cpu-timers.c +11
-0+static void __run_posix_cpu_timers (struct task_struct *tsk) +{ +struct posix_cputimers *pct = &tsk->posix_cputimers; + +if (!test_and_set_bit (CPUTIMERS_WORK_SCHEDULED , &pct->flags)) +task_work_add (tsk, &pct->task_work, true); +} + +static inline void posix_cpu_timers_enable_work (struct task_struct *tsk) +{ +clear_bit (CPUTIMERS_WORK_SCHEDULED , &tsk->posix_cputimers.flags);/* * Ensure we observe everything before a failing test_and_set() * in __run_posix_cpu_timers(). */ smp_mb__after_atomic();
Quoted 5 lines
Quoted 1 line
+}
Such that when another timer interrupt happens while we run this, we're guaranteed to either see it, or get re-queued and thus re-run the function.
But each thread in the process enqueues its own task work and flips its own flags. So if task A runs the task work and task B runs __run_posix_cpu_timers(), they wouldn't be ordering against the same flags.
Re: [patch V2 3/5] posix-cpu-timers: Provide mechanisms to defer timer handling to task_work
Quoted 20 lines · Frederic Weisbecker
Quoted 19 lines · Peter Zijlstra
kernel/time/posix-cpu-timers.c static void __run_posix_cpu_timers(struc · line 1159
Quoted 11 lines · Thomas Gleixner
kernel/time/posix-cpu-timers.c +11
-0+static void __run_posix_cpu_timers (struct task_struct *tsk) +{ +struct posix_cputimers *pct = &tsk->posix_cputimers; + +if (!test_and_set_bit (CPUTIMERS_WORK_SCHEDULED , &pct->flags)) +task_work_add (tsk, &pct->task_work, true); +} + +static inline void posix_cpu_timers_enable_work (struct task_struct *tsk) +{ +clear_bit (CPUTIMERS_WORK_SCHEDULED , &tsk->posix_cputimers.flags);/* * Ensure we observe everything before a failing test_and_set() * in __run_posix_cpu_timers(). */ smp_mb__after_atomic();Quoted 1 line
+}
Quoted 7 lines
Quoted 3 lines
Such that when another timer interrupt happens while we run this, we're guaranteed to either see it, or get re-queued and thus re-run the function.
But each thread in the process enqueues its own task work and flips its own flags. So if task A runs the task work and task B runs
__run_posix_cpu_timers(), they wouldn't be ordering against the same flags.
If two tasks queue work independent of each other then one of them will find it done already, which is the same as if two tasks of the same process execute run_posix_cpu_timers() in parallel.
I really don't want to go into the rathole of making the work or the synchronization process wide. That's a guarantee for disaster.
Handling task work strictly per task is straight forward and simple. The eventually resulting contention on sighand lock in task work is unavoidable, but that's a reasonable tradeoff vs. the complexity you need to handle task work process wide.
Thanks,
tglx
Frederic Weisbecker
maintainerRe: [patch V2 3/5] posix-cpu-timers: Provide mechanisms to defer timer handling to task_work
Quoted 33 lines · Thomas Gleixner
Quoted 20 lines · Frederic Weisbecker
Quoted 19 lines · Peter Zijlstra
kernel/time/posix-cpu-timers.c static void __run_posix_cpu_timers(struc · line 1159
Quoted 11 lines · Thomas Gleixner
kernel/time/posix-cpu-timers.c +11
-0+static void __run_posix_cpu_timers (struct task_struct *tsk) +{ +struct posix_cputimers *pct = &tsk->posix_cputimers; + +if (!test_and_set_bit (CPUTIMERS_WORK_SCHEDULED , &pct->flags)) +task_work_add (tsk, &pct->task_work, true); +} + +static inline void posix_cpu_timers_enable_work (struct task_struct *tsk) +{ +clear_bit (CPUTIMERS_WORK_SCHEDULED , &tsk->posix_cputimers.flags);/* * Ensure we observe everything before a failing test_and_set() * in __run_posix_cpu_timers(). */ smp_mb__after_atomic();Quoted 1 line
+}
Quoted 7 lines
Quoted 3 lines
Such that when another timer interrupt happens while we run this, we're guaranteed to either see it, or get re-queued and thus re-run the function.
But each thread in the process enqueues its own task work and flips its own flags. So if task A runs the task work and task B runs
__run_posix_cpu_timers(), they wouldn't be ordering against the same flags.If two tasks queue work independent of each other then one of them will find it done already, which is the same as if two tasks of the same process execute
run_posix_cpu_timers()in parallel.
Quoted 7 lines
I really don't want to go into the rathole of making the work or the synchronization process wide. That's a guarantee for disaster.
Handling task work strictly per task is straight forward and simple. The eventually resulting contention on sighand lock in task work is unavoidable, but that's a reasonable tradeoff vs. the complexity you need to handle task work process wide.
Definetly!
I was only commenting on the barrier suggestion. But I believe it shouldn't be needed in the end.
If we were to have a per task work for thread timers and a per process work for process timers, that means we would need to cut down the whole thing, and also take care about timers firing after exit_task_work(), which isn't an issue in the thread case as the work will simply be ignored for an exiting task but it's a big issue in the case of process wide handling.
Anyway, the current layout is simple enough.