Skip to contentSkip to conversation

WARN_ON_ONCE(in_nmi()) hit in irq_work_queue_on

DISCUSSION Dave Jones 3 participants 9 messages Aug 6, 2014 – Aug 7, 2014

Subsystems (2)
lore
Share & tools
lore

Dave Jones

author
#0 lore raw
WARNING: CPU: 3 PID: 18062 at kernel/irq_work.c:72 irq_work_queue_on+0x11e/0x140()
CPU: 3 PID: 18062 Comm: trinity-subchil Not tainted 3.16.0+ #34
 0000000000000009 00000000903774d1 ffff880244e06c00 ffffffff9a7f1e37
 0000000000000000 ffff880244e06c38 ffffffff9a0791dd ffff880244fce180
 0000000000000003 ffff880244e06d58 ffff880244e06ef8 0000000000000000
Call Trace:
 <NMI>  [<ffffffff9a7f1e37>] dump_stack+0x4e/0x7a
 [<ffffffff9a0791dd>] warn_slowpath_common+0x7d/0xa0
 [<ffffffff9a07930a>] warn_slowpath_null+0x1a/0x20
 [<ffffffff9a17ca1e>] irq_work_queue_on+0x11e/0x140
 [<ffffffff9a10a2c7>] tick_nohz_full_kick_cpu+0x57/0x90
 [<ffffffff9a186cd5>] __perf_event_overflow+0x275/0x350
 [<ffffffff9a184f80>] ? perf_event_task_disable+0xa0/0xa0
 [<ffffffff9a01a4cf>] ? x86_perf_event_set_period+0xbf/0x150
 [<ffffffff9a187934>] perf_event_overflow+0x14/0x20
 [<ffffffff9a020386>] intel_pmu_handle_irq+0x206/0x410
 [<ffffffff9a0b54d3>] ? arch_vtime_task_switch+0x63/0x130
 [<ffffffff9a01937b>] perf_event_nmi_handler+0x2b/0x50
 [<ffffffff9a007b72>] nmi_handle+0xd2/0x390
 [<ffffffff9a007aa5>] ? nmi_handle+0x5/0x390
 [<ffffffff9a0d131b>] ? lock_release+0xab/0x330
 [<ffffffff9a008062>] default_do_nmi+0x72/0x1c0
 [<ffffffff9a0c925f>] ? cpuacct_account_field+0xcf/0x200
 [<ffffffff9a008268>] do_nmi+0xb8/0x100
 [<ffffffff9a7ff66a>] end_repeat_nmi+0x1e/0x2e
 [<ffffffff9a0c925f>] ? cpuacct_account_field+0xcf/0x200
 [<ffffffff9a0d131b>] ? lock_release+0xab/0x330
 [<ffffffff9a0d131b>] ? lock_release+0xab/0x330
 [<ffffffff9a0d131b>] ? lock_release+0xab/0x330
 <<EOE>>  [<ffffffff9a0c9277>] cpuacct_account_field+0xe7/0x200
 [<ffffffff9a0c9195>] ? cpuacct_account_field+0x5/0x200
 [<ffffffff9a0b4d18>] account_system_time+0x98/0x1a0
 [<ffffffff9a0b4e4e>] __vtime_account_system+0x2e/0x40
 [<ffffffff9a0b5419>] vtime_user_enter+0x59/0x90
 [<ffffffff9a18e183>] ? context_tracking_user_enter+0xd3/0x1f0
 [<ffffffff9a18e183>] context_tracking_user_enter+0xd3/0x1f0
 [<ffffffff9a013815>] syscall_trace_leave+0xa5/0x210
 [<ffffffff9a7fd8bf>] int_check_syscall_exit_work+0x34/0x3d
---[ end trace 73831bdc3ef3ba75 ]---
maintainer
#1 lore raw
Quoted 31 lines · Dave Jones
WARNING: CPU: 3 PID: 18062 at kernel/irq_work.c:72 irq_work_queue_on+0x11e/0x140()
CPU: 3 PID: 18062 Comm: trinity-subchil Not tainted 3.16.0+ #34
 0000000000000009 00000000903774d1 ffff880244e06c00 ffffffff9a7f1e37
 0000000000000000 ffff880244e06c38 ffffffff9a0791dd ffff880244fce180
 0000000000000003 ffff880244e06d58 ffff880244e06ef8 0000000000000000
Call Trace:
 <NMI>  [<ffffffff9a7f1e37>] dump_stack+0x4e/0x7a
 [<ffffffff9a0791dd>] warn_slowpath_common+0x7d/0xa0
 [<ffffffff9a07930a>] warn_slowpath_null+0x1a/0x20
 [<ffffffff9a17ca1e>] irq_work_queue_on+0x11e/0x140
 [<ffffffff9a10a2c7>] tick_nohz_full_kick_cpu+0x57/0x90
 [<ffffffff9a186cd5>] __perf_event_overflow+0x275/0x350
 [<ffffffff9a184f80>] ? perf_event_task_disable+0xa0/0xa0
 [<ffffffff9a01a4cf>] ? x86_perf_event_set_period+0xbf/0x150
 [<ffffffff9a187934>] perf_event_overflow+0x14/0x20
 [<ffffffff9a020386>] intel_pmu_handle_irq+0x206/0x410
 [<ffffffff9a0b54d3>] ? arch_vtime_task_switch+0x63/0x130
 [<ffffffff9a01937b>] perf_event_nmi_handler+0x2b/0x50
 [<ffffffff9a007b72>] nmi_handle+0xd2/0x390
 [<ffffffff9a007aa5>] ? nmi_handle+0x5/0x390
 [<ffffffff9a0d131b>] ? lock_release+0xab/0x330
 [<ffffffff9a008062>] default_do_nmi+0x72/0x1c0
 [<ffffffff9a0c925f>] ? cpuacct_account_field+0xcf/0x200
 [<ffffffff9a008268>] do_nmi+0xb8/0x100
 [<ffffffff9a7ff66a>] end_repeat_nmi+0x1e/0x2e
 [<ffffffff9a0c925f>] ? cpuacct_account_field+0xcf/0x200
 [<ffffffff9a0d131b>] ? lock_release+0xab/0x330
 [<ffffffff9a0d131b>] ? lock_release+0xab/0x330
 [<ffffffff9a0d131b>] ? lock_release+0xab/0x330
 <<EOE>>  [<ffffffff9a0c9277>] cpuacct_account_field+0xe7/0x200
 [<ffffffff9a0c9195>] ? cpuacct_account_field+0x5/0x200
Quoted 8 lines
 [<ffffffff9a0b4d18>] account_system_time+0x98/0x1a0
 [<ffffffff9a0b4e4e>] __vtime_account_system+0x2e/0x40
 [<ffffffff9a0b5419>] vtime_user_enter+0x59/0x90
 [<ffffffff9a18e183>] ? context_tracking_user_enter+0xd3/0x1f0
 [<ffffffff9a18e183>] context_tracking_user_enter+0xd3/0x1f0
 [<ffffffff9a013815>] syscall_trace_leave+0xa5/0x210
 [<ffffffff9a7fd8bf>] int_check_syscall_exit_work+0x34/0x3d
---[ end trace 73831bdc3ef3ba75 ]---

Urgh, Frederic, any idea how that happened?

maintainer
#2 lore raw
Quoted 41 lines · Peter Zijlstra
Quoted 31 lines · Dave Jones
WARNING: CPU: 3 PID: 18062 at kernel/irq_work.c:72 irq_work_queue_on+0x11e/0x140()
CPU: 3 PID: 18062 Comm: trinity-subchil Not tainted 3.16.0+ #34
 0000000000000009 00000000903774d1 ffff880244e06c00 ffffffff9a7f1e37
 0000000000000000 ffff880244e06c38 ffffffff9a0791dd ffff880244fce180
 0000000000000003 ffff880244e06d58 ffff880244e06ef8 0000000000000000
Call Trace:
 <NMI>  [<ffffffff9a7f1e37>] dump_stack+0x4e/0x7a
 [<ffffffff9a0791dd>] warn_slowpath_common+0x7d/0xa0
 [<ffffffff9a07930a>] warn_slowpath_null+0x1a/0x20
 [<ffffffff9a17ca1e>] irq_work_queue_on+0x11e/0x140
 [<ffffffff9a10a2c7>] tick_nohz_full_kick_cpu+0x57/0x90
 [<ffffffff9a186cd5>] __perf_event_overflow+0x275/0x350
 [<ffffffff9a184f80>] ? perf_event_task_disable+0xa0/0xa0
 [<ffffffff9a01a4cf>] ? x86_perf_event_set_period+0xbf/0x150
 [<ffffffff9a187934>] perf_event_overflow+0x14/0x20
 [<ffffffff9a020386>] intel_pmu_handle_irq+0x206/0x410
 [<ffffffff9a0b54d3>] ? arch_vtime_task_switch+0x63/0x130
 [<ffffffff9a01937b>] perf_event_nmi_handler+0x2b/0x50
 [<ffffffff9a007b72>] nmi_handle+0xd2/0x390
 [<ffffffff9a007aa5>] ? nmi_handle+0x5/0x390
 [<ffffffff9a0d131b>] ? lock_release+0xab/0x330
 [<ffffffff9a008062>] default_do_nmi+0x72/0x1c0
 [<ffffffff9a0c925f>] ? cpuacct_account_field+0xcf/0x200
 [<ffffffff9a008268>] do_nmi+0xb8/0x100
 [<ffffffff9a7ff66a>] end_repeat_nmi+0x1e/0x2e
 [<ffffffff9a0c925f>] ? cpuacct_account_field+0xcf/0x200
 [<ffffffff9a0d131b>] ? lock_release+0xab/0x330
 [<ffffffff9a0d131b>] ? lock_release+0xab/0x330
 [<ffffffff9a0d131b>] ? lock_release+0xab/0x330
 <<EOE>>  [<ffffffff9a0c9277>] cpuacct_account_field+0xe7/0x200
 [<ffffffff9a0c9195>] ? cpuacct_account_field+0x5/0x200
Quoted 8 lines
 [<ffffffff9a0b4d18>] account_system_time+0x98/0x1a0
 [<ffffffff9a0b4e4e>] __vtime_account_system+0x2e/0x40
 [<ffffffff9a0b5419>] vtime_user_enter+0x59/0x90
 [<ffffffff9a18e183>] ? context_tracking_user_enter+0xd3/0x1f0
 [<ffffffff9a18e183>] context_tracking_user_enter+0xd3/0x1f0
 [<ffffffff9a013815>] syscall_trace_leave+0xa5/0x210
 [<ffffffff9a7fd8bf>] int_check_syscall_exit_work+0x34/0x3d
---[ end trace 73831bdc3ef3ba75 ]---
Quoted 1 line

Urgh, Frederic, any idea how that happened?

Sigh, that's d84153d6c96f61a so that's been there a while, and been broken equally long.

So this is where we run a low period (!freq) hardware event on a nohz_full cpu or so? And because it throttles, we need to kick the tick into action to unthrottle it.

I suppose there's a good reason I never build with that nohz_full nonsense enabled :/

Not sure how we should go fix that, you can't just issue random IPIs from NMI context.

maintainer
#3 lore raw
Quoted 7 lines

Sigh, that's d84153d6c96f61a so that's been there a while, and been broken equally long.

So this is where we run a low period (!freq) hardware event on a nohz_full cpu or so? And because it throttles, we need to kick the tick into action to unthrottle it.

Quoted 5 lines

I suppose there's a good reason I never build with that nohz_full nonsense enabled :/

Not sure how we should go fix that, you can't just issue random IPIs from NMI context.

OK, thinking one more second would've done it, how about so?

---
 include/linux/perf_event.h | 7 ++++---
 kernel/events/core.c       | 8 +++++++-
 2 files changed, 11 insertions(+), 4 deletions(-)
include/linux/perf_event.h +4 -3
index 707617a8c0f6..177411e3ffc4 100644
@@ -421,9 +421,10 @@ struct perf_event {
 	struct fasync_struct		*fasync;
 
 	/* delayed work for NMIs and such */
-	int				pending_wakeup;
-	int				pending_kill;
-	int				pending_disable;
+	int				pending_kill      : 16;
+	int				pending_wakeup    : 1;
+	int				pending_disable   : 1;
+	int				pending_nohz_kick : 1;
 	struct irq_work			pending;
 
 	atomic_t			event_limit;
kernel/events/core.c +7 -1
index 1cf24b3e42ec..e95fca20e26f 100644
@@ -4258,6 +4258,11 @@ static void perf_pending_event(struct irq_work *entry)
 		event->pending_wakeup = 0;
 		perf_event_wakeup(event);
 	}
+
+	if (event->pending_nohz_kick) {
+		event->pending_nohz_kick = 0;
+		tick_nohz_full_kick();
+	}
 }
 
 /*
@@ -5431,7 +5436,8 @@ static int __perf_event_overflow(struct perf_event *event,
 			__this_cpu_inc(perf_throttled_count);
 			hwc->interrupts = MAX_INTERRUPTS;
 			perf_log_throttle(event, 0);
-			tick_nohz_full_kick();
+			event->pending_nohz_kick = 1;
+			irq_work_queue(&event->pending);
 			ret = 1;
 		}
 	}
#4 lore raw
Quoted 55 lines · Peter Zijlstra
Quoted 7 lines

Sigh, that's d84153d6c96f61a so that's been there a while, and been broken equally long.

So this is where we run a low period (!freq) hardware event on a nohz_full cpu or so? And because it throttles, we need to kick the tick into action to unthrottle it.

Quoted 5 lines

I suppose there's a good reason I never build with that nohz_full nonsense enabled :/

Not sure how we should go fix that, you can't just issue random IPIs from NMI context.

OK, thinking one more second would've done it, how about so?

---
 include/linux/perf_event.h | 7 ++++---
 kernel/events/core.c       | 8 +++++++-
 2 files changed, 11 insertions(+), 4 deletions(-)
include/linux/perf_event.h +4 -3
index 707617a8c0f6..177411e3ffc4 100644
@@ -421,9 +421,10 @@ struct perf_event {
 	struct fasync_struct		*fasync;
 
 	/* delayed work for NMIs and such */
-	int				pending_wakeup;
-	int				pending_kill;
-	int				pending_disable;
+	int				pending_kill      : 16;
+	int				pending_wakeup    : 1;
+	int				pending_disable   : 1;
+	int				pending_nohz_kick : 1;
 	struct irq_work			pending;
 
 	atomic_t			event_limit;
kernel/events/core.c +5 -0
index 1cf24b3e42ec..e95fca20e26f 100644
@@ -4258,6 +4258,11 @@ static void perf_pending_event(struct irq_work *entry)
 		event->pending_wakeup = 0;
 		perf_event_wakeup(event);
 	}
+
+	if (event->pending_nohz_kick) {
+		event->pending_nohz_kick = 0;
+		tick_nohz_full_kick();
+	}
 }
 
 /*

kernel/events/core.c static int __perf_event_overflow(struct perf_event *event, · line 5443

Quoted 9 lines
kernel/events/core.c +2 -1
@@ -5431,7 +5436,8 @@ static int __perf_event_overflow(struct perf_event *event,
 			__this_cpu_inc(perf_throttled_count);
 			hwc->interrupts = MAX_INTERRUPTS;
 			perf_log_throttle(event, 0);
-			tick_nohz_full_kick();
+			event->pending_nohz_kick = 1;
+			irq_work_queue(&event->pending);
 			ret = 1;
 		}
 	}

In fact the problem has arised since the recent irq work patches I did. There I've changed tick_nohz_full_kick() to use irq_work_queue_on() instead of irq_work_queue() so it has become NMI unsafe by accident.

So I'd rather suggest this instead of queuing two levels of irq_work:

include/linux/tick.h +1 -6
index 8a4987f..fed88b5 100644
@@ -181,13 +181,8 @@ static inline bool tick_nohz_full_cpu(int cpu)
 
 extern void tick_nohz_init(void);
 extern void __tick_nohz_full_check(void);
+extern void tick_nohz_full_kick(void);
 extern void tick_nohz_full_kick_cpu(int cpu);
-
-static inline void tick_nohz_full_kick(void)
-{
-	tick_nohz_full_kick_cpu(smp_processor_id());
-}
-
 extern void tick_nohz_full_kick_all(void);
 extern void __tick_nohz_task_switch(struct task_struct *tsk);
 #else
kernel/time/tick-sched.c +6 -0
index d4ccb96..8e0d347 100644
@@ -225,6 +225,12 @@ static DEFINE_PER_CPU(struct irq_work, nohz_full_kick_work) = {
 	.func = nohz_full_kick_work_func,
 };
 
+void tick_nohz_full_kick(void)
+{
+	if (!tick_nohz_full_cpu(smp_processor_id()))
+		irq_work_queue(&__get_cpu_var(nohz_full_kick_work));
+}
+
 /*
  * Kick the CPU if it's full dynticks in order to force it to
  * re-evaluate its dependency on the tick and restart it if necessary.
maintainer
#5 lore raw
Quoted 1 line · Frederic Weisbecker

In fact the problem has arised since the recent irq work patches I did.

No, those just added the WARN, previously we send the resched IPI, and that's equally wrong from NMI context.

kernel/time/tick-sched.c static DEFINE_PER_CPU(struct irq_work, nohz_full_kick_work) = { · line 232

Quoted 10 lines

There I've changed tick_nohz_full_kick() to use irq_work_queue_on() instead of irq_work_queue() so it has become NMI unsafe by accident.

So I'd rather suggest this instead of queuing two levels of irq_work:

kernel/time/tick-sched.c +5 -0
+void tick_nohz_full_kick(void)
+{
+	if (!tick_nohz_full_cpu(smp_processor_id()))
+		irq_work_queue(&__get_cpu_var(nohz_full_kick_work));
+}

Indeed, that's better. Thanks!

#6 lore raw
Quoted 5 lines · Peter Zijlstra
Quoted 1 line · Frederic Weisbecker

In fact the problem has arised since the recent irq work patches I did.

No, those just added the WARN, previously we send the resched IPI, and that's equally wrong from NMI context.

Well the scheduler IPI was rather used for remote kicks before we had remote irq work. This includes local kicks as well as the caller could need to kick anywhere. As in inc_nr_running().

But for strict local kicks, as in perf, we were using tick_nohz_full_kick() which has been using irq work for a while. But it got broken when we replaced it to call irq_work_queue_on instead of irq_work_queue.

Quoted 13 lines

kernel/time/tick-sched.c static DEFINE_PER_CPU(struct irq_work, nohz_full_kick_work) = { · line 232

Quoted 11 lines

There I've changed tick_nohz_full_kick() to use irq_work_queue_on() instead of irq_work_queue() so it has become NMI unsafe by accident.

So I'd rather suggest this instead of queuing two levels of irq_work:

kernel/time/tick-sched.c +5 -0
+void tick_nohz_full_kick(void)
+{
+	if (!tick_nohz_full_cpu(smp_processor_id()))
+		irq_work_queue(&__get_cpu_var(nohz_full_kick_work));
+}

Indeed, that's better. Thanks!

maintainer
#7 lore raw
Quoted 7 lines · Frederic Weisbecker
Quoted 5 lines · Peter Zijlstra
Quoted 1 line · Frederic Weisbecker

In fact the problem has arised since the recent irq work patches I did.

No, those just added the WARN, previously we send the resched IPI, and that's equally wrong from NMI context.

Quoted 7 lines

Well the scheduler IPI was rather used for remote kicks before we had remote irq work. This includes local kicks as well as the caller could need to kick anywhere. As in inc_nr_running().

But for strict local kicks, as in perf, we were using tick_nohz_full_kick() which has been using irq work for a while. But it got broken when we replaced it to call irq_work_queue_on instead of irq_work_queue.

OK, clearly I made a mess of things in my head ;-)

#8 lore raw
Quoted 12 lines · Peter Zijlstra
Quoted 11 lines · Frederic Weisbecker
Quoted 5 lines · Peter Zijlstra
Quoted 1 line · Frederic Weisbecker

In fact the problem has arised since the recent irq work patches I did.

No, those just added the WARN, previously we send the resched IPI, and that's equally wrong from NMI context.

Well the scheduler IPI was rather used for remote kicks before we had remote irq work. This includes local kicks as well as the caller could need to kick anywhere. As in inc_nr_running().

Quoted 5 lines
Quoted 3 lines

But for strict local kicks, as in perf, we were using tick_nohz_full_kick() which has been using irq work for a while. But it got broken when we replaced it to call irq_work_queue_on instead of irq_work_queue.

OK, clearly I made a mess of things in my head ;-)

I'm a bit responsible for that mess since I created quite some flavours of nohz kicks all around :)

But now they should be more unified.