Skip to contentSkip to conversation

nohz: Move full nohz kick to its own IPI

tip:timers/nohz Peter Zijlstra 3 participants 16 messages May 5, 2014 – May 11, 2014 1 message not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw
Quoted 17 lines · tip-bot for Frederic Weisbecker
Commit-ID:  72aacf0259bb7d53b7a3b5b2f7bf982acaa52b61
Gitweb:     http://git.kernel.org/tip/72aacf0259bb7d53b7a3b5b2f7bf982acaa52b61
Author:     Frederic Weisbecker <[email protected]>
AuthorDate: Tue, 18 Mar 2014 21:12:53 +0100
Committer:  Frederic Weisbecker <[email protected]>
CommitDate: Thu, 3 Apr 2014 18:05:21 +0200

nohz: Move full nohz kick to its own IPI

Now that we have smp_queue_function_single() which can be used to safely queue IPIs when interrupts are disabled and without worrying about concurrent callers, lets use it for the full dynticks kick to notify a CPU that it's exiting single task mode.

This unbloats a bit the scheduler IPI that the nohz code was abusing for its cool "callable anywhere/anytime" properties.

Quoted 9 lines
Cc Andrew Morton <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Jens Axboe <[email protected]>
Cc Kevin Hilman <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Thomas Gleixner <[email protected]>

So I suspect this is the patch that makes Ingo's machines unhappy, they appear to get stuck thusly:

[10513.382910] RIP: 0010:[<ffffffff8112b7da>]  [<ffffffff8112b7da>] generic_exec_single+0x9a/0x180
[10513.481704]  [<ffffffff8112c092>] smp_queue_function_single+0x42/0xa0
[10513.488251]  [<ffffffff81126ce0>] tick_nohz_full_kick_cpu+0x50/0x80
[10513.494661]  [<ffffffff810f4b0e>] enqueue_task_fair+0x59e/0x6c0
[10513.506469]  [<ffffffff810e3d6a>] enqueue_task+0x3a/0x60
[10513.511836]  [<ffffffff810e8ac3>] __migrate_task+0x123/0x150
[10513.523535]  [<ffffffff810e8b0d>] migration_cpu_stop+0x1d/0x30
[10513.529401]  [<ffffffff81143460>] cpu_stopper_thread+0x70/0x120

I'm not entirely sure how yet, but this is by far the most likely candidate. Ingo, if you still have the vmlinuz matching this trace (your hang2.txt) could you have a peek where that RIP lands?

If that is indeed the csd_lock() function, then this is it and something's buggered.

kernel/sched/sched.h static inline void inc_nr_running(struct rq *rq) · line 1227

Quoted 8 lines
kernel/sched/sched.h +0 -0
index c9007f2..4771063 100644
@@ -1225,7 +1225,7 @@ static inline void inc_nr_running(struct rq *rq)
 		if (tick_nohz_full_cpu(rq->cpu)) {
 			/* Order rq->nr_running write against the IPI */
 			smp_wmb();

FWIW that barrier is complete crap ;-)

Quoted 23 lines
kernel/sched/sched.h +1 -1
-			smp_send_reschedule(rq->cpu);
+			tick_nohz_full_kick_cpu(rq->cpu);
 		}
        }
 #endif
kernel/time/tick-sched.c +10 -0
index 9f8af69..582d3f6 100644
@@ -230,6 +230,27 @@ void tick_nohz_full_kick(void)
 		irq_work_queue(&__get_cpu_var(nohz_full_kick_work));
 }
 
+static void nohz_full_kick_queue(struct queue_single_data *qsd)
+{
+	__tick_nohz_full_check();
+}
+
+static DEFINE_PER_CPU(struct queue_single_data, nohz_full_kick_qsd) = {
+	.func = nohz_full_kick_queue,
+};
+
+void tick_nohz_full_kick_cpu(int cpu)

kernel/time/tick-sched.c void tick_nohz_full_kick(void) · line 252

Quoted 10 lines
kernel/time/tick-sched.c +10 -0
+{
+	if (!tick_nohz_full_cpu(cpu))
+		return;
+
+	if (cpu == smp_processor_id()) {
+		irq_work_queue(&__get_cpu_var(nohz_full_kick_work));
+	} else {
+		smp_queue_function_single(cpu, &per_cpu(nohz_full_kick_qsd, cpu));
+	}
+}

Should we instead do irq_work_queue_on() ?

#2 lore raw
Quoted 41 lines · Peter Zijlstra
Quoted 17 lines · tip-bot for Frederic Weisbecker
Commit-ID:  72aacf0259bb7d53b7a3b5b2f7bf982acaa52b61
Gitweb:     http://git.kernel.org/tip/72aacf0259bb7d53b7a3b5b2f7bf982acaa52b61
Author:     Frederic Weisbecker <[email protected]>
AuthorDate: Tue, 18 Mar 2014 21:12:53 +0100
Committer:  Frederic Weisbecker <[email protected]>
CommitDate: Thu, 3 Apr 2014 18:05:21 +0200

nohz: Move full nohz kick to its own IPI

Now that we have smp_queue_function_single() which can be used to safely queue IPIs when interrupts are disabled and without worrying about concurrent callers, lets use it for the full dynticks kick to notify a CPU that it's exiting single task mode.

This unbloats a bit the scheduler IPI that the nohz code was abusing for its cool "callable anywhere/anytime" properties.

Quoted 9 lines
Cc Andrew Morton <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Jens Axboe <[email protected]>
Cc Kevin Hilman <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Thomas Gleixner <[email protected]>

So I suspect this is the patch that makes Ingo's machines unhappy, they appear to get stuck thusly:

[10513.382910] RIP: 0010:[<ffffffff8112b7da>]  [<ffffffff8112b7da>] generic_exec_single+0x9a/0x180
[10513.481704]  [<ffffffff8112c092>] smp_queue_function_single+0x42/0xa0
[10513.488251]  [<ffffffff81126ce0>] tick_nohz_full_kick_cpu+0x50/0x80
[10513.494661]  [<ffffffff810f4b0e>] enqueue_task_fair+0x59e/0x6c0
[10513.506469]  [<ffffffff810e3d6a>] enqueue_task+0x3a/0x60
[10513.511836]  [<ffffffff810e8ac3>] __migrate_task+0x123/0x150
[10513.523535]  [<ffffffff810e8b0d>] migration_cpu_stop+0x1d/0x30
[10513.529401]  [<ffffffff81143460>] cpu_stopper_thread+0x70/0x120
Quoted 6 lines

I'm not entirely sure how yet, but this is by far the most likely candidate. Ingo, if you still have the vmlinuz matching this trace (your hang2.txt) could you have a peek where that RIP lands?

If that is indeed the csd_lock() function, then this is it and something's buggered.

On a kernel build from your .config the +0x9a is indeed very close to that wait loop; of course 0x9a isn't even an instruction boundary for me so its all a bit of a guess.

#3 lore raw
Quoted 46 lines · Peter Zijlstra
Quoted 45 lines · Peter Zijlstra
Quoted 17 lines · tip-bot for Frederic Weisbecker
Commit-ID:  72aacf0259bb7d53b7a3b5b2f7bf982acaa52b61
Gitweb:     http://git.kernel.org/tip/72aacf0259bb7d53b7a3b5b2f7bf982acaa52b61
Author:     Frederic Weisbecker <[email protected]>
AuthorDate: Tue, 18 Mar 2014 21:12:53 +0100
Committer:  Frederic Weisbecker <[email protected]>
CommitDate: Thu, 3 Apr 2014 18:05:21 +0200

nohz: Move full nohz kick to its own IPI

Now that we have smp_queue_function_single() which can be used to safely queue IPIs when interrupts are disabled and without worrying about concurrent callers, lets use it for the full dynticks kick to notify a CPU that it's exiting single task mode.

This unbloats a bit the scheduler IPI that the nohz code was abusing for its cool "callable anywhere/anytime" properties.

Quoted 9 lines
Cc Andrew Morton <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Jens Axboe <[email protected]>
Cc Kevin Hilman <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Thomas Gleixner <[email protected]>

So I suspect this is the patch that makes Ingo's machines unhappy, they appear to get stuck thusly:

[10513.382910] RIP: 0010:[<ffffffff8112b7da>]  [<ffffffff8112b7da>] generic_exec_single+0x9a/0x180
[10513.481704]  [<ffffffff8112c092>] smp_queue_function_single+0x42/0xa0
[10513.488251]  [<ffffffff81126ce0>] tick_nohz_full_kick_cpu+0x50/0x80
[10513.494661]  [<ffffffff810f4b0e>] enqueue_task_fair+0x59e/0x6c0
[10513.506469]  [<ffffffff810e3d6a>] enqueue_task+0x3a/0x60
[10513.511836]  [<ffffffff810e8ac3>] __migrate_task+0x123/0x150
[10513.523535]  [<ffffffff810e8b0d>] migration_cpu_stop+0x1d/0x30
[10513.529401]  [<ffffffff81143460>] cpu_stopper_thread+0x70/0x120

I'm not entirely sure how yet, but this is by far the most likely candidate. Ingo, if you still have the vmlinuz matching this trace (your hang2.txt) could you have a peek where that RIP lands?

Quoted 6 lines
Quoted 2 lines

If that is indeed the csd_lock() function, then this is it and something's buggered.

On a kernel build from your .config the +0x9a is indeed very close to that wait loop; of course 0x9a isn't even an instruction boundary for me so its all a bit of a guess.

Note the current ordering:

    cmpxchg(&qsd->pending, 0, 1)       get ipi
    csd_lock(qsd->csd)                 xchg(&qsd->pending, 1)
    send ipi                           csd_unlock(qsd->csd)

So there shouldn't be racing updaters. Also ipi sender shouldn't race with ipi receiver, the update shouldn't always eventually see the unlock happening.

OTOH the ordering above is anything but intuitive. In fact csd_lock() is on the way here. smp_queue_function_single() doesn't need it at all. So it's just yet another risk for a deadlock.

One more reason why I would much prefer irq_work_queue_on(). But I'm also not very happy with the possible result since we are going to maintain two different APIs set doing almost the same thing with just slightly different constraints or semantics :-(

#4 lore raw
Quoted 10 lines

Note the current ordering:

    cmpxchg(&qsd->pending, 0, 1)       get ipi
    csd_lock(qsd->csd)                 xchg(&qsd->pending, 1)
    send ipi                           csd_unlock(qsd->csd)

So there shouldn't be racing updaters. Also ipi sender shouldn't race with ipi receiver, the update shouldn't always eventually see the unlock happening.

Yeah, I've not spotted how this particular train wreck happens either.

The problem is reproduction, it took me 9 hours to confirm I could reproduce the problem on my machine. So how long to I run it with this patch reverted to show its gone..

#5 lore raw
Quoted 11 lines · Peter Zijlstra
Quoted 10 lines

Note the current ordering:

    cmpxchg(&qsd->pending, 0, 1)       get ipi
    csd_lock(qsd->csd)                 xchg(&qsd->pending, 1)
    send ipi                           csd_unlock(qsd->csd)

So there shouldn't be racing updaters. Also ipi sender shouldn't race with ipi receiver, the update shouldn't always eventually see the unlock happening.

Quoted 5 lines

Yeah, I've not spotted how this particular train wreck happens either.

The problem is reproduction, it took me 9 hours to confirm I could reproduce the problem on my machine. So how long to I run it with this patch reverted to show its gone..

Maybe it could be favoured cpu hotplug. Anyway converting to irq_work should fix it.

#6 lore raw
Quoted 12 lines · Frederic Weisbecker
Quoted 11 lines · Peter Zijlstra
Quoted 10 lines

Note the current ordering:

    cmpxchg(&qsd->pending, 0, 1)       get ipi
    csd_lock(qsd->csd)                 xchg(&qsd->pending, 1)
    send ipi                           csd_unlock(qsd->csd)

So there shouldn't be racing updaters. Also ipi sender shouldn't race with ipi receiver, the update shouldn't always eventually see the unlock happening.

Quoted 8 lines
Quoted 5 lines

Yeah, I've not spotted how this particular train wreck happens either.

The problem is reproduction, it took me 9 hours to confirm I could reproduce the problem on my machine. So how long to I run it with this patch reverted to show its gone..

Maybe it could be favoured cpu hotplug. Anyway converting to irq_work should fix it.

Ingo needs a commit msg for the revert of this patch; do you think you have time to look into _why_ this patch is broken and write such a thing?

#7 lore raw
Quoted 19 lines · Peter Zijlstra
Quoted 18 lines · Frederic Weisbecker
Quoted 16 lines · Peter Zijlstra
Quoted 10 lines

Note the current ordering:

    cmpxchg(&qsd->pending, 0, 1)       get ipi
    csd_lock(qsd->csd)                 xchg(&qsd->pending, 1)
    send ipi                           csd_unlock(qsd->csd)

So there shouldn't be racing updaters. Also ipi sender shouldn't race with ipi receiver, the update shouldn't always eventually see the unlock happening.

Yeah, I've not spotted how this particular train wreck happens either.

The problem is reproduction, it took me 9 hours to confirm I could reproduce the problem on my machine. So how long to I run it with this patch reverted to show its gone..

Quoted 6 lines
Quoted 2 lines

Maybe it could be favoured cpu hotplug. Anyway converting to irq_work should fix it.

Ingo needs a commit msg for the revert of this patch; do you think you have time to look into _why_ this patch is broken and write such a thing?

I can try but I need to reproduce it. Do you have any clue on how to do so? Also which HEAD were you guys using?

Thanks.

#8 lore raw
Quoted 23 lines · Frederic Weisbecker
Quoted 22 lines · Peter Zijlstra
Quoted 20 lines · Frederic Weisbecker
Quoted 11 lines · Peter Zijlstra
Quoted 10 lines

Note the current ordering:

    cmpxchg(&qsd->pending, 0, 1)       get ipi
    csd_lock(qsd->csd)                 xchg(&qsd->pending, 1)
    send ipi                           csd_unlock(qsd->csd)

So there shouldn't be racing updaters. Also ipi sender shouldn't race with ipi receiver, the update shouldn't always eventually see the unlock happening.

Quoted 5 lines

Yeah, I've not spotted how this particular train wreck happens either.

The problem is reproduction, it took me 9 hours to confirm I could reproduce the problem on my machine. So how long to I run it with this patch reverted to show its gone..

Maybe it could be favoured cpu hotplug. Anyway converting to irq_work should fix it.

Quoted 6 lines
Quoted 3 lines

Ingo needs a commit msg for the revert of this patch; do you think you have time to look into _why_ this patch is broken and write such a thing?

I can try but I need to reproduce it. Do you have any clue on how to do so? Also which HEAD were you guys using?

Ha!, so I was running a tip/master with that commit in -- a few days ago, v3.15-rc4-1644-g5c658b0cdf22 might've been it.

Then I ran it on my dual socket AMD interlagos, with:

while :; make O=allyesconfig-build/ clean; make O=allyesconfig-build/ -j96 -s; done

for 9 hours, and then got empty RCU stall warns and a bricked machine.

I might still have the .config, but I don't think there was anything particularly odd about the config other than having NOHZ_FULL enabled.

The only way I found this patch was by staring at some RCU stall warns Ingo managed to get, sometimes they actually got backtraces in them apparently.

According to Ingo the bigger the machine the faster it reproduces, but reproduction times, even for these 32 cpu machines, are in the many hours range.

#9 lore raw
Quoted 44 lines · Peter Zijlstra
Quoted 23 lines · Frederic Weisbecker
Quoted 22 lines · Peter Zijlstra
Quoted 20 lines · Frederic Weisbecker
Quoted 11 lines · Peter Zijlstra
Quoted 10 lines

Note the current ordering:

    cmpxchg(&qsd->pending, 0, 1)       get ipi
    csd_lock(qsd->csd)                 xchg(&qsd->pending, 1)
    send ipi                           csd_unlock(qsd->csd)

So there shouldn't be racing updaters. Also ipi sender shouldn't race with ipi receiver, the update shouldn't always eventually see the unlock happening.

Quoted 5 lines

Yeah, I've not spotted how this particular train wreck happens either.

The problem is reproduction, it took me 9 hours to confirm I could reproduce the problem on my machine. So how long to I run it with this patch reverted to show its gone..

Maybe it could be favoured cpu hotplug. Anyway converting to irq_work should fix it.

Quoted 6 lines
Quoted 3 lines

Ingo needs a commit msg for the revert of this patch; do you think you have time to look into _why_ this patch is broken and write such a thing?

I can try but I need to reproduce it. Do you have any clue on how to do so? Also which HEAD were you guys using?

Ha!, so I was running a tip/master with that commit in -- a few days ago, v3.15-rc4-1644-g5c658b0cdf22 might've been it.

Then I ran it on my dual socket AMD interlagos, with:

while :; make O=allyesconfig-build/ clean; make O=allyesconfig-build/ -j96 -s; done

for 9 hours, and then got empty RCU stall warns and a bricked machine.

I might still have the .config, but I don't think there was anything particularly odd about the config other than having NOHZ_FULL enabled.

Quoted 7 lines

The only way I found this patch was by staring at some RCU stall warns Ingo managed to get, sometimes they actually got backtraces in them apparently.

According to Ingo the bigger the machine the faster it reproduces, but reproduction times, even for these 32 cpu machines, are in the many hours range.

Ok then, I'll try something.

But note that those commits aren't upstream yet and they are in a seperate branch tip:timers/nohz with no other non-upstream commits.

And I work alone on this branch.

So we can as well zap these commits and replace them with the irq_work_on() conversion (still preparing that).

#10 lore raw
Quoted 2 lines · Frederic Weisbecker

So we can as well zap these commits and replace them with the irq_work_on() conversion (still preparing that).

I hear you, but I think Ingo doesn't want to rebase the tree because its public, but maybe he can make an exception..

#11 lore raw
Quoted 6 lines · Peter Zijlstra
Quoted 2 lines · Frederic Weisbecker

So we can as well zap these commits and replace them with the irq_work_on() conversion (still preparing that).

I hear you, but I think Ingo doesn't want to rebase the tree because its public, but maybe he can make an exception..

Since tip:timers/nohz only contains these commits:

fc1781cc66b1 Merge branch 'timers/nohz-ipi-for-tip-v3' of git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks into timers/nohz 72aacf0259bb nohz: Move full nohz kick to its own IPI
55d77c215c74 smp: Non busy-waiting IPI queue

we can zap them all, or even keep 55d77c215c74.

Frederic, which one would you prefer?

Thanks,

Ingo

#12 lore raw
Quoted 12 lines · Ingo Molnar
Quoted 6 lines · Peter Zijlstra
Quoted 2 lines · Frederic Weisbecker

So we can as well zap these commits and replace them with the irq_work_on() conversion (still preparing that).

I hear you, but I think Ingo doesn't want to rebase the tree because its public, but maybe he can make an exception..

Since tip:timers/nohz only contains these commits:

Quoted 7 lines

fc1781cc66b1 Merge branch 'timers/nohz-ipi-for-tip-v3' of git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks into timers/nohz 72aacf0259bb nohz: Move full nohz kick to its own IPI
55d77c215c74 smp: Non busy-waiting IPI queue

we can zap them all, or even keep 55d77c215c74.

Frederic, which one would you prefer?

I think we can zap them all. But since I'm not yet sure how the new solution, based on irq work, is going to be received, do you mind if we just wait for the new solution to be acked by Peterz?

If that's ok for you, I'll then do a pull request that includes the solution and zaps the old two commits.

Thanks!

#13 lore raw
Quoted 19 lines · Frederic Weisbecker
Quoted 18 lines · Ingo Molnar
Quoted 6 lines · Peter Zijlstra
Quoted 2 lines · Frederic Weisbecker

So we can as well zap these commits and replace them with the irq_work_on() conversion (still preparing that).

I hear you, but I think Ingo doesn't want to rebase the tree because its public, but maybe he can make an exception..

Since tip:timers/nohz only contains these commits:

fc1781cc66b1 Merge branch 'timers/nohz-ipi-for-tip-v3' of git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks into timers/nohz 72aacf0259bb nohz: Move full nohz kick to its own IPI
55d77c215c74 smp: Non busy-waiting IPI queue

we can zap them all, or even keep 55d77c215c74.

Quoted 8 lines
Quoted 1 line

Frederic, which one would you prefer?

I think we can zap them all. But since I'm not yet sure how the new solution, based on irq work, is going to be received, do you mind if we just wait for the new solution to be acked by Peterz?

If that's ok for you, I'll then do a pull request that includes the solution and zaps the old two commits.

It's all isolated to timers/nohz, not mixed up with anything else, so zapping it is easy (I already zapped the second commit) and you can simply use latest -Linus to base your new solution on.

Thanks,

Ingo

#14 lore raw
In reply to Peter Zijlstra
Quoted 41 lines · Peter Zijlstra
Quoted 17 lines · tip-bot for Frederic Weisbecker
Commit-ID:  72aacf0259bb7d53b7a3b5b2f7bf982acaa52b61
Gitweb:     http://git.kernel.org/tip/72aacf0259bb7d53b7a3b5b2f7bf982acaa52b61
Author:     Frederic Weisbecker <[email protected]>
AuthorDate: Tue, 18 Mar 2014 21:12:53 +0100
Committer:  Frederic Weisbecker <[email protected]>
CommitDate: Thu, 3 Apr 2014 18:05:21 +0200

nohz: Move full nohz kick to its own IPI

Now that we have smp_queue_function_single() which can be used to safely queue IPIs when interrupts are disabled and without worrying about concurrent callers, lets use it for the full dynticks kick to notify a CPU that it's exiting single task mode.

This unbloats a bit the scheduler IPI that the nohz code was abusing for its cool "callable anywhere/anytime" properties.

Quoted 9 lines
Cc Andrew Morton <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Jens Axboe <[email protected]>
Cc Kevin Hilman <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Thomas Gleixner <[email protected]>

So I suspect this is the patch that makes Ingo's machines unhappy, they appear to get stuck thusly:

[10513.382910] RIP: 0010:[<ffffffff8112b7da>]  [<ffffffff8112b7da>] generic_exec_single+0x9a/0x180
[10513.481704]  [<ffffffff8112c092>] smp_queue_function_single+0x42/0xa0
[10513.488251]  [<ffffffff81126ce0>] tick_nohz_full_kick_cpu+0x50/0x80
[10513.494661]  [<ffffffff810f4b0e>] enqueue_task_fair+0x59e/0x6c0
[10513.506469]  [<ffffffff810e3d6a>] enqueue_task+0x3a/0x60
[10513.511836]  [<ffffffff810e8ac3>] __migrate_task+0x123/0x150
[10513.523535]  [<ffffffff810e8b0d>] migration_cpu_stop+0x1d/0x30
[10513.529401]  [<ffffffff81143460>] cpu_stopper_thread+0x70/0x120
Quoted 6 lines

I'm not entirely sure how yet, but this is by far the most likely candidate. Ingo, if you still have the vmlinuz matching this trace (your hang2.txt) could you have a peek where that RIP lands?

If that is indeed the csd_lock() function, then this is it and something's buggered.

Aye!

Quoted 11 lines

kernel/sched/sched.h static inline void inc_nr_running(struct rq *rq) · line 1227

Quoted 9 lines
kernel/sched/sched.h +0 -0
index c9007f2..4771063 100644
@@ -1225,7 +1225,7 @@ static inline void inc_nr_running(struct rq *rq)
 		if (tick_nohz_full_cpu(rq->cpu)) {
 			/* Order rq->nr_running write against the IPI */
 			smp_wmb();

FWIW that barrier is complete crap ;-)

Yeah, I'm queing the removal of that now :)

Quoted 28 lines
Quoted 28 lines
kernel/sched/sched.h +1 -1
-			smp_send_reschedule(rq->cpu);
+			tick_nohz_full_kick_cpu(rq->cpu);
 		}
        }
 #endif
kernel/time/tick-sched.c +14 -0
index 9f8af69..582d3f6 100644
@@ -230,6 +230,27 @@ void tick_nohz_full_kick(void)
 		irq_work_queue(&__get_cpu_var(nohz_full_kick_work));
 }
 
+static void nohz_full_kick_queue(struct queue_single_data *qsd)
+{
+	__tick_nohz_full_check();
+}
+
+static DEFINE_PER_CPU(struct queue_single_data, nohz_full_kick_qsd) = {
+	.func = nohz_full_kick_queue,
+};
+
+void tick_nohz_full_kick_cpu(int cpu)
+{
+	if (!tick_nohz_full_cpu(cpu))
+		return;
+
Quoted 8 lines

kernel/time/tick-sched.c void tick_nohz_full_kick(void) · line 252

Quoted 6 lines
kernel/time/tick-sched.c +6 -0
+	if (cpu == smp_processor_id()) {
+		irq_work_queue(&__get_cpu_var(nohz_full_kick_work));
+	} else {
+		smp_queue_function_single(cpu, &per_cpu(nohz_full_kick_qsd, cpu));
+	}
+}

Should we instead do irq_work_queue_on() ?

I would really much prefer that yeah. But if we do that, expect some added overhead on the local irq_work_queue() path though. irq_work_raise can't use local cmpxchg ops anymore.

Or we can have a different pending raise system for remote irq work.

I can try something.

#15 lore raw
Quoted 8 lines · Frederic Weisbecker
Quoted 1 line

Should we instead do irq_work_queue_on() ?

I would really much prefer that yeah. But if we do that, expect some added overhead on the local irq_work_queue() path though. irq_work_raise can't use local cmpxchg ops anymore.

Or we can have a different pending raise system for remote irq work.

I can try something.

Loosing that local cmpxchg shouldn't be a problem, I don't thnk this is a particularly hot path.

#16 lore raw
Quoted 6 lines · Peter Zijlstra
Quoted 5 lines · Frederic Weisbecker
Quoted 1 line

Should we instead do irq_work_queue_on() ?

I would really much prefer that yeah. But if we do that, expect some added overhead on the local irq_work_queue() path though. irq_work_raise can't use local cmpxchg ops anymore.

Quoted 6 lines
Quoted 3 lines

Or we can have a different pending raise system for remote irq work.

I can try something.

Loosing that local cmpxchg shouldn't be a problem, I don't thnk this is a particularly hot path.

Then the conversion is easy since most of the irq work code should already work for remote queuing.

I'll come up with a patch soon.

Thanks.