Skip to contentSkip to conversation

preempt_tracer: Fix preempt_disable tracepoint

RFC In review Daniel Bristot de Oliveira 4 participants 27 messages May 28, 2019 – Jun 5, 2019 Scheduler

lore
Share & tools
lore
#0 lore raw

cover preempt_tracer: Fix preempt_disable tracepoint

While playing with the model + working in a fix for the task context & trace recursion, I ended up hitting two cases in which the preempt_disable/enable tracepoint was supposed to happen, but didn't.

There is an explanation for each case in the log message.

This is an RFC exposing the problem, with possible ways to fix it. But I bet there might be better solutions as well, so this is a real RFC.

Daniel Bristot de Oliveira (3):
softirq: Use preempt_latency_stop/start to trace preemption
preempt_tracer: Disable IRQ while starting/stopping due to a
preempt_counter change
preempt_tracer: Use a percpu variable to control traceble calls

 kernel/sched/core.c | 66 ++++++++++++++++++++++++++++++++++-----------
 kernel/softirq.c    | 13 ++++-----
 2 files changed, 55 insertions(+), 24 deletions(-)
Signature · 1 line
2.20.1
#1 lore raw

RFC 1/3 softirq: Use preempt_latency_stop/start to trace preemption

prempt_disable/enable tracepoints occurs only in the preemption enabled <-> disable transition. As preempt_latency_stop() and preempt_latency_start() already do this control, avoid code duplication by using these functions in the softirq code as well.

RFC: Should we move preempt_latency_start/preempt_latency_stop to a trace source file... or even a header?

Cc "Steven Rostedt (VMware)" <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Matthias Kaehlcke <[email protected]>
Cc "Joel Fernandes (Google)" <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Yangtao Li <[email protected]>
Cc Tommaso Cucinotta <[email protected]>
---
 kernel/sched/core.c |  4 ++--
 kernel/softirq.c    | 13 +++++--------
 2 files changed, 7 insertions(+), 10 deletions(-)
kernel/sched/core.c +2 -2
index 874c427742a9..8c0b414e45dc 100644
@@ -3152,7 +3152,7 @@ static inline void sched_tick_stop(int cpu) { }
  * If the value passed in is equal to the current preempt count
  * then we just disabled preemption. Start timing the latency.
  */
-static inline void preempt_latency_start(int val)
+void preempt_latency_start(int val)
 {
 	if (preempt_count() == val) {
 		unsigned long ip = get_lock_parent_ip();
@@ -3189,7 +3189,7 @@ NOKPROBE_SYMBOL(preempt_count_add);
  * If the value passed in equals to the current preempt count
  * then we just enabled preemption. Stop timing the latency.
  */
-static inline void preempt_latency_stop(int val)
+void preempt_latency_stop(int val)
 {
 	if (preempt_count() == val)
 		trace_preempt_on(CALLER_ADDR0, get_lock_parent_ip());
kernel/softirq.c +5 -8
index 2c3382378d94..c9ad89c3dfed 100644
@@ -108,6 +108,8 @@ static bool ksoftirqd_running(unsigned long pending)
  * where hardirqs are disabled legitimately:
  */
 #ifdef CONFIG_TRACE_IRQFLAGS
+extern void preempt_latency_start(int val);
+extern void preempt_latency_stop(int val);
 void __local_bh_disable_ip(unsigned long ip, unsigned int cnt)
 {
 	unsigned long flags;
@@ -130,12 +132,8 @@ void __local_bh_disable_ip(unsigned long ip, unsigned int cnt)
 		trace_softirqs_off(ip);
 	raw_local_irq_restore(flags);
 
-	if (preempt_count() == cnt) {
-#ifdef CONFIG_DEBUG_PREEMPT
-		current->preempt_disable_ip = get_lock_parent_ip();
-#endif
-		trace_preempt_off(CALLER_ADDR0, get_lock_parent_ip());
-	}
+	preempt_latency_start(cnt);
+
 }
 EXPORT_SYMBOL(__local_bh_disable_ip);
 #endif /* CONFIG_TRACE_IRQFLAGS */
@@ -144,8 +142,7 @@ static void __local_bh_enable(unsigned int cnt)
 {
 	lockdep_assert_irqs_disabled();
 
-	if (preempt_count() == cnt)
-		trace_preempt_on(CALLER_ADDR0, get_lock_parent_ip());
+	preempt_latency_stop(cnt);
 
 	if (softirq_count() == (cnt & SOFTIRQ_MASK))
 		trace_softirqs_on(_RET_IP_);
Signature · 1 line
2.20.1
maintainer
#2 lore raw

Re: [RFC 1/3] softirq: Use preempt_latency_stop/start to trace preemption

Quoted 7 lines · Daniel Bristot de Oliveira

prempt_disable/enable tracepoints occurs only in the preemption enabled <-> disable transition. As preempt_latency_stop() and preempt_latency_start() already do this control, avoid code duplication by using these functions in the softirq code as well.

RFC: Should we move preempt_latency_start/preempt_latency_stop to a trace source file... or even a header?

Yeah, a header might not be a bad idea.

kernel/softirq.c void __local_bh_disable_ip(unsigned long ip, unsigned int cnt) · line 136

Quoted 12 lines
kernel/softirq.c +2 -6
@@ -130,12 +132,8 @@ void __local_bh_disable_ip(unsigned long ip, unsigned int cnt)
 		trace_softirqs_off(ip);
 	raw_local_irq_restore(flags);
 
-	if (preempt_count() == cnt) {
-#ifdef CONFIG_DEBUG_PREEMPT
-		current->preempt_disable_ip = get_lock_parent_ip();
-#endif
-		trace_preempt_off(CALLER_ADDR0, get_lock_parent_ip());
-	}
+	preempt_latency_start(cnt);
+

I'm thinking we can do without that empty line.

Quoted 3 lines
kernel/softirq.c +0 -0
 }
 EXPORT_SYMBOL(__local_bh_disable_ip);
 #endif /* CONFIG_TRACE_IRQFLAGS */
#3 lore raw

Re: [RFC 1/3] softirq: Use preempt_latency_stop/start to trace preemption

In reply to Daniel Bristot de Oliveira
Quoted 7 lines · Daniel Bristot de Oliveira

prempt_disable/enable tracepoints occurs only in the preemption enabled <-> disable transition. As preempt_latency_stop() and preempt_latency_start() already do this control, avoid code duplication by using these functions in the softirq code as well.

RFC: Should we move preempt_latency_start/preempt_latency_stop to a trace source file... or even a header?

Yes, I think so. Also this patch changes CALLER_ADDR0 passed to the tracepoint because there's one more level of a non-inlined function call in the call chain right? Very least the changelog should document this change in functional behavior, IMO.

Looks like a nice change otherwise, thanks!

Quoted 80 lines
Cc "Steven Rostedt (VMware)" <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Matthias Kaehlcke <[email protected]>
Cc "Joel Fernandes (Google)" <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Yangtao Li <[email protected]>
Cc Tommaso Cucinotta <[email protected]>
---
 kernel/sched/core.c |  4 ++--
 kernel/softirq.c    | 13 +++++--------
 2 files changed, 7 insertions(+), 10 deletions(-)
kernel/sched/core.c +2 -2
index 874c427742a9..8c0b414e45dc 100644
@@ -3152,7 +3152,7 @@ static inline void sched_tick_stop(int cpu) { }
  * If the value passed in is equal to the current preempt count
  * then we just disabled preemption. Start timing the latency.
  */
-static inline void preempt_latency_start(int val)
+void preempt_latency_start(int val)
 {
 	if (preempt_count() == val) {
 		unsigned long ip = get_lock_parent_ip();
@@ -3189,7 +3189,7 @@ NOKPROBE_SYMBOL(preempt_count_add);
  * If the value passed in equals to the current preempt count
  * then we just enabled preemption. Stop timing the latency.
  */
-static inline void preempt_latency_stop(int val)
+void preempt_latency_stop(int val)
 {
 	if (preempt_count() == val)
 		trace_preempt_on(CALLER_ADDR0, get_lock_parent_ip());
kernel/softirq.c +5 -8
index 2c3382378d94..c9ad89c3dfed 100644
@@ -108,6 +108,8 @@ static bool ksoftirqd_running(unsigned long pending)
  * where hardirqs are disabled legitimately:
  */
 #ifdef CONFIG_TRACE_IRQFLAGS
+extern void preempt_latency_start(int val);
+extern void preempt_latency_stop(int val);
 void __local_bh_disable_ip(unsigned long ip, unsigned int cnt)
 {
 	unsigned long flags;
@@ -130,12 +132,8 @@ void __local_bh_disable_ip(unsigned long ip, unsigned int cnt)
 		trace_softirqs_off(ip);
 	raw_local_irq_restore(flags);
 
-	if (preempt_count() == cnt) {
-#ifdef CONFIG_DEBUG_PREEMPT
-		current->preempt_disable_ip = get_lock_parent_ip();
-#endif
-		trace_preempt_off(CALLER_ADDR0, get_lock_parent_ip());
-	}
+	preempt_latency_start(cnt);
+
 }
 EXPORT_SYMBOL(__local_bh_disable_ip);
 #endif /* CONFIG_TRACE_IRQFLAGS */
@@ -144,8 +142,7 @@ static void __local_bh_enable(unsigned int cnt)
 {
 	lockdep_assert_irqs_disabled();
 
-	if (preempt_count() == cnt)
-		trace_preempt_on(CALLER_ADDR0, get_lock_parent_ip());
+	preempt_latency_stop(cnt);
 
 	if (softirq_count() == (cnt & SOFTIRQ_MASK))
 		trace_softirqs_on(_RET_IP_);
Signature · 1 line
2.20.1
#4 lore raw

Re: [RFC 1/3] softirq: Use preempt_latency_stop/start to trace preemption

Quoted 4 lines · On Wed, 29 May 2019 05:30:56 -0400 Joel Fernandes <[email protected]>…

Yes, I think so. Also this patch changes CALLER_ADDR0 passed to the tracepoint because there's one more level of a non-inlined function call in the call chain right? Very least the changelog should document this change in functional behavior, IMO.

This sounds more like a break in behavior not a functional change. I guess moving it to a header and making it a static __always_inline should be fine though.

-- Steve

#5 lore raw

Re: [RFC 1/3] softirq: Use preempt_latency_stop/start to trace preemption

Quoted 7 lines · Steven Rostedt
Quoted 4 lines · On Wed, 29 May 2019 05:30:56 -0400 Joel Fernandes <[email protected]>…

Yes, I think so. Also this patch changes CALLER_ADDR0 passed to the tracepoint because there's one more level of a non-inlined function call in the call chain right? Very least the changelog should document this change in functional behavior, IMO.

In practice I am seeing no change in the values printed, but there is another problem with this regard: there are cases in which both caller and parent have the same address.

I am quite sure it has to do with the in_lock_function() behavior. Anyway, I was already planing to propose a cleanup in the in_lock_function/in_sched_function. I will investigate it more.

Quoted 3 lines

This sounds more like a break in behavior not a functional change. I guess moving it to a header and making it a static __always_inline should be fine though.

Steve, which header should I use?

Thanks!

-- Daniel

Quoted 2 lines

-- Steve

#6 lore raw

Re: [RFC 1/3] softirq: Use preempt_latency_stop/start to trace preemption

Quoted 5 lines · On Tue, 4 Jun 2019 12:39:03 +0200 Daniel Bristot de Oliveira…
Quoted 3 lines

This sounds more like a break in behavior not a functional change. I guess moving it to a header and making it a static __always_inline should be fine though.

Steve, which header should I use?

I would say include/linux/preempt.h if Peter doesn't have any issues with that.

-- Steve

#7 lore raw

RFC 2/3 preempt_tracer: Disable IRQ while starting/stopping due to a preempt_counter change

In reply to Daniel Bristot de Oliveira

The preempt_disable/enable tracepoint only traces in the disable <-> enable case, which is correct. But think about this case:

---------------------------- %< ------------------------------

	THREAD					IRQ
	   |					 |
preempt_disable() {
    __preempt_count_add(1)
	------->	    smp_apic_timer_interrupt() {
				preempt_disable()
				    do not trace (preempt count >= 1)
				    ....
				preempt_enable()
				    do not trace (preempt count >= 1)
			    }
    trace_preempt_disable();
}

---------------------------- >% ------------------------------

The tracepoint will be skipped.

It is possible to observe this problem using this kernel module:

http://bristot.me/files/efficient_verification/wip.tar.gz [*]

and doing the following trace:

	# cd /sys/kernel/debug/tracing/
	# echo 1 > snapshot
	# cat available_events | grep preempt_ > set_event
	# echo irq_vectors >> /sys/kernel/debug/tracing/set_event
	# insmod wip.ko
	/* wait for a snapshot creation */
	# tail -100 snapshot
		[...]
	    tail-5572  [001] ....1..  2888.401184: preempt_enable: caller=_raw_spin_unlock_irqrestore+0x2a/0x70 parent=          (null)
	    tail-5572  [001] ....1..  2888.401184: preempt_disable: caller=migrate_disable+0x8b/0x1e0 parent=migrate_disable+0x8b/0x1e0
	    tail-5572  [001] ....111  2888.401184: preempt_enable: caller=migrate_disable+0x12f/0x1e0 parent=migrate_disable+0x12f/0x1e0
	    tail-5572  [001] d..h212  2888.401189: local_timer_entry: vector=236
	    tail-5572  [001] dN.h512  2888.401192: process_event: event sched_waking not expected in the state preemptive
	    tail-5572  [001] dN.h512  2888.401200: <stack trace>
	 => process_event
	 => __handle_event
	 => ttwu_do_wakeup
	 => try_to_wake_up
	 => invoke_rcu_core
	 => rcu_check_callbacks
	 => update_process_times
	 => tick_sched_handle
	 => tick_sched_timer
	 => __hrtimer_run_queues
	 => hrtimer_interrupt
	 => smp_apic_timer_interrupt
	 => apic_timer_interrupt
	 => trace_event_raw_event_preemptirq_template
	 => trace_preempt_off
	 => get_page_from_freelist
	 => __alloc_pages_nodemask
	 => __handle_mm_fault
	 => handle_mm_fault
	 => __do_page_fault
	 => do_page_fault
	 => async_page_fault

To avoid skipping the trace, the change in the counter should be "atomic" with the start/stop, w.r.t the interrupts.

Disable interrupts while the adding/starting stopping/subtracting.

Cc "Steven Rostedt (VMware)" <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Matthias Kaehlcke <[email protected]>
Cc "Joel Fernandes (Google)" <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Yangtao Li <[email protected]>
Cc Tommaso Cucinotta <[email protected]>

---
[*] with some luck we will talk about this at the Plumbers.

 kernel/sched/core.c | 48 +++++++++++++++++++++++++++++++++------------
 kernel/softirq.c    |  2 +-
 2 files changed, 37 insertions(+), 13 deletions(-)
kernel/sched/core.c +36 -12
index 8c0b414e45dc..be4117d7384f 100644
@@ -3163,6 +3163,16 @@ void preempt_latency_start(int val)
 	}
 }
 
+static inline void preempt_add_start_latency(int val)
+{
+	unsigned long flags;
+
+	raw_local_irq_save(flags);
+	__preempt_count_add(val);
+	preempt_latency_start(val);
+	raw_local_irq_restore(flags);
+}
+
 void preempt_count_add(int val)
 {
 #ifdef CONFIG_DEBUG_PREEMPT
@@ -3172,7 +3182,7 @@ void preempt_count_add(int val)
 	if (DEBUG_LOCKS_WARN_ON((preempt_count() < 0)))
 		return;
 #endif
-	__preempt_count_add(val);
+	preempt_add_start_latency(val);
 #ifdef CONFIG_DEBUG_PREEMPT
 	/*
 	 * Spinlock count overflowing soon?
@@ -3180,7 +3190,6 @@ void preempt_count_add(int val)
 	DEBUG_LOCKS_WARN_ON((preempt_count() & PREEMPT_MASK) >=
 				PREEMPT_MASK - 10);
 #endif
-	preempt_latency_start(val);
 }
 EXPORT_SYMBOL(preempt_count_add);
 NOKPROBE_SYMBOL(preempt_count_add);
@@ -3195,6 +3204,16 @@ void preempt_latency_stop(int val)
 		trace_preempt_on(CALLER_ADDR0, get_lock_parent_ip());
 }
 
+static inline void preempt_sub_stop_latency(int val)
+{
+	unsigned long flags;
+
+	raw_local_irq_save(flags);
+	preempt_latency_stop(val);
+	__preempt_count_sub(val);
+	raw_local_irq_restore(flags);
+}
+
 void preempt_count_sub(int val)
 {
 #ifdef CONFIG_DEBUG_PREEMPT
@@ -3211,8 +3230,7 @@ void preempt_count_sub(int val)
 		return;
 #endif
 
-	preempt_latency_stop(val);
-	__preempt_count_sub(val);
+	preempt_sub_stop_latency(val);
 }
 EXPORT_SYMBOL(preempt_count_sub);
 NOKPROBE_SYMBOL(preempt_count_sub);
@@ -3220,6 +3238,16 @@ NOKPROBE_SYMBOL(preempt_count_sub);
 #else
 static inline void preempt_latency_start(int val) { }
 static inline void preempt_latency_stop(int val) { }
+
+static inline void preempt_sub_stop_latency(int val)
+{
+	__preempt_count_sub(val);
+}
+
+static inline void preempt_add_start_latency(int val)
+{
+	__preempt_count_add(val);
+}
 #endif
 
 static inline unsigned long get_preempt_disable_ip(struct task_struct *p)
@@ -3585,11 +3613,9 @@ static void __sched notrace preempt_schedule_common(void)
 		 * traced. The other to still record the preemption latency,
 		 * which can also be traced by the function tracer.
 		 */
-		preempt_disable_notrace();
-		preempt_latency_start(1);
+		preempt_add_start_latency(1);
 		__schedule(true);
-		preempt_latency_stop(1);
-		preempt_enable_no_resched_notrace();
+		preempt_sub_stop_latency(1);
 
 		/*
 		 * Check again in case we missed a preemption opportunity
@@ -3653,8 +3679,7 @@ asmlinkage __visible void __sched notrace preempt_schedule_notrace(void)
 		 * traced. The other to still record the preemption latency,
 		 * which can also be traced by the function tracer.
 		 */
-		preempt_disable_notrace();
-		preempt_latency_start(1);
+		preempt_add_start_latency(1);
 		/*
 		 * Needs preempt disabled in case user_exit() is traced
 		 * and the tracer calls preempt_enable_notrace() causing
@@ -3664,8 +3689,7 @@ asmlinkage __visible void __sched notrace preempt_schedule_notrace(void)
 		__schedule(true);
 		exception_exit(prev_ctx);
 
-		preempt_latency_stop(1);
-		preempt_enable_no_resched_notrace();
+		preempt_sub_stop_latency(1);
 	} while (need_resched());
 }
 EXPORT_SYMBOL_GPL(preempt_schedule_notrace);
kernel/softirq.c +1 -1
index c9ad89c3dfed..9c64522ecc76 100644
@@ -130,9 +130,9 @@ void __local_bh_disable_ip(unsigned long ip, unsigned int cnt)
 	 */
 	if (softirq_count() == (cnt & SOFTIRQ_MASK))
 		trace_softirqs_off(ip);
-	raw_local_irq_restore(flags);
 
 	preempt_latency_start(cnt);
+	raw_local_irq_restore(flags);
 
 }
 EXPORT_SYMBOL(__local_bh_disable_ip);
Signature · 1 line
2.20.1
maintainer
#8 lore raw

Re: [RFC 2/3] preempt_tracer: Disable IRQ while starting/stopping due to a preempt_counter change

Quoted 17 lines · Daniel Bristot de Oliveira

The preempt_disable/enable tracepoint only traces in the disable <-> enable case, which is correct. But think about this case:

---------------------------- %< ------------------------------

	THREAD					IRQ
	   |					 |
preempt_disable() {
    __preempt_count_add(1)
	------->	    smp_apic_timer_interrupt() {
				preempt_disable()
				    do not trace (preempt count >= 1)
				    ....
				preempt_enable()
				    do not trace (preempt count >= 1)
			    }
    trace_preempt_disable();
}
Quoted 3 lines

---------------------------- >% ------------------------------

The tracepoint will be skipped.

.... for the IRQ. But IRQs are not preemptible anyway, so what the problem?

Quoted 14 lines

To avoid skipping the trace, the change in the counter should be "atomic" with the start/stop, w.r.t the interrupts.

Disable interrupts while the adding/starting stopping/subtracting.

kernel/sched/core.c +9 -0
+static inline void preempt_add_start_latency(int val)
+{
+	unsigned long flags;
+
+	raw_local_irq_save(flags);
+	__preempt_count_add(val);
+	preempt_latency_start(val);
+	raw_local_irq_restore(flags);
+}

kernel/sched/core.c void preempt_latency_start(int val)

Quoted 10 lines
kernel/sched/core.c +9 -0
 
+static inline void preempt_sub_stop_latency(int val)
+{
+	unsigned long flags;
+
+	raw_local_irq_save(flags);
+	preempt_latency_stop(val);
+	__preempt_count_sub(val);
+	raw_local_irq_restore(flags);
+}

That is hideously expensive :/

#9 lore raw

Re: [RFC 2/3] preempt_tracer: Disable IRQ while starting/stopping due to a preempt_counter change

Quoted 20 lines · Peter Zijlstra
Quoted 19 lines · Daniel Bristot de Oliveira

The preempt_disable/enable tracepoint only traces in the disable <-> enable case, which is correct. But think about this case:

---------------------------- %< ------------------------------

	THREAD					IRQ
	   |					 |
preempt_disable() {
    __preempt_count_add(1)
	------->	    smp_apic_timer_interrupt() {
				preempt_disable()
				    do not trace (preempt count >= 1)
				    ....
				preempt_enable()
				    do not trace (preempt count >= 1)
			    }
    trace_preempt_disable();
}

---------------------------- >% ------------------------------

Quoted 4 lines
Quoted 1 line

The tracepoint will be skipped.

.... for the IRQ. But IRQs are not preemptible anyway, so what the problem?

right, they are.

exposing my problem in a more specific way:

To show in a model that an event always takes place with preemption disabled, but not necessarily with IRQs disabled, it is worth having the preemption disable events separated from IRQ disable ones.

The main reason is that, although IRQs disabled postpone the execution of the scheduler, it is more pessimistic, as it also delays IRQs. So the more precise the model is, the less pessimistic the analysis will be.

But there are other use-cases, for instance:

(Steve, correct me if I am wrong)

The preempt_tracer will not notice a "preempt disabled" section in an IRQ handler if the problem above happens.

(Yeah, I know these problems are very specific... but...)

Quoted 15 lines
Quoted 14 lines

To avoid skipping the trace, the change in the counter should be "atomic" with the start/stop, w.r.t the interrupts.

Disable interrupts while the adding/starting stopping/subtracting.

kernel/sched/core.c +9 -0
+static inline void preempt_add_start_latency(int val)
+{
+	unsigned long flags;
+
+	raw_local_irq_save(flags);
+	__preempt_count_add(val);
+	preempt_latency_start(val);
+	raw_local_irq_restore(flags);
+}
Quoted 11 lines

kernel/sched/core.c void preempt_latency_stop(int val) · line 3215

Quoted 9 lines
kernel/sched/core.c +9 -0
+static inline void preempt_sub_stop_latency(int val)
+{
+	unsigned long flags;
+
+	raw_local_irq_save(flags);
+	preempt_latency_stop(val);
+	__preempt_count_sub(val);
+	raw_local_irq_restore(flags);
+}

That is hideously expensive :/

Yeah... :-( Is there another way to provide such "atomicity"?

Can I use the argument "if one has these tracepoints enabled, they are not considering it as a hot-path?"

-- Daniel

maintainer
#10 lore raw

Re: [RFC 2/3] preempt_tracer: Disable IRQ while starting/stopping due to a preempt_counter change

Quoted 31 lines · Daniel Bristot de Oliveira
Quoted 20 lines · Peter Zijlstra
Quoted 19 lines · Daniel Bristot de Oliveira

The preempt_disable/enable tracepoint only traces in the disable <-> enable case, which is correct. But think about this case:

---------------------------- %< ------------------------------

	THREAD					IRQ
	   |					 |
preempt_disable() {
    __preempt_count_add(1)
	------->	    smp_apic_timer_interrupt() {
				preempt_disable()
				    do not trace (preempt count >= 1)
				    ....
				preempt_enable()
				    do not trace (preempt count >= 1)
			    }
    trace_preempt_disable();
}

---------------------------- >% ------------------------------

Quoted 4 lines
Quoted 1 line

The tracepoint will be skipped.

.... for the IRQ. But IRQs are not preemptible anyway, so what the problem?

right, they are.

exposing my problem in a more specific way:

Quoted 7 lines

To show in a model that an event always takes place with preemption disabled, but not necessarily with IRQs disabled, it is worth having the preemption disable events separated from IRQ disable ones.

The main reason is that, although IRQs disabled postpone the execution of the scheduler, it is more pessimistic, as it also delays IRQs. So the more precise the model is, the less pessimistic the analysis will be.

I'm not sure I follow, IRQs disabled fully implies !preemptible. I don't see how the model would be more pessimistic than reality if it were to use this knowledge.

Any !0 preempt_count(), which very much includes (Hard)IRQ and SoftIRQ counts, means non-preemptible.

#11 lore raw

Re: [RFC 2/3] preempt_tracer: Disable IRQ while starting/stopping due to a preempt_counter change

Quoted 40 lines · On Wed, 29 May 2019 12:20:38 +0200 Peter Zijlstra <[email protected]> wrote:
Quoted 31 lines · Daniel Bristot de Oliveira
Quoted 20 lines · Peter Zijlstra
Quoted 19 lines · Daniel Bristot de Oliveira

The preempt_disable/enable tracepoint only traces in the disable <-> enable case, which is correct. But think about this case:

---------------------------- %< ------------------------------

	THREAD					IRQ
	   |					 |
preempt_disable() {
    __preempt_count_add(1)  
	------->	    smp_apic_timer_interrupt() {  
				preempt_disable()
				    do not trace (preempt count >= 1)
				    ....
				preempt_enable()
				    do not trace (preempt count >= 1)
			    }
    trace_preempt_disable();
}  

---------------------------- >% ------------------------------

Quoted 4 lines
Quoted 1 line

The tracepoint will be skipped.

.... for the IRQ. But IRQs are not preemptible anyway, so what the problem?

right, they are.

exposing my problem in a more specific way:

Quoted 7 lines

To show in a model that an event always takes place with preemption disabled, but not necessarily with IRQs disabled, it is worth having the preemption disable events separated from IRQ disable ones.

The main reason is that, although IRQs disabled postpone the execution of the scheduler, it is more pessimistic, as it also delays IRQs. So the more precise the model is, the less pessimistic the analysis will be.

Quoted 6 lines

I'm not sure I follow, IRQs disabled fully implies !preemptible. I don't see how the model would be more pessimistic than reality if it were to use this knowledge.

Any !0 preempt_count(), which very much includes (Hard)IRQ and SoftIRQ counts, means non-preemptible.

I believe I see what Daniel is talking about, but I hate the proposed solution ;-)

First, if you care about real times that the CPU can't preempt (preempt_count != 0 or interrupts disabled), then you want the preempt_irqsoff tracer. The preempt_tracer is more academic where it just shows you when we disable preemption via the counter. But even with the preempt_irqsoff tracer you may not get the full length of time due to the above explained race.

__preempt_count_add(1) <-- CPU now prevents preemption

<interrupt>
---->
trace_hardirqs_off() <-- Start preempt disable timer handler();
trace_hardirqs_on() <-- Stop preempt disable timer (Diff A)
<----
trace_preempt_disable() <-- Start preempt disable timer

[..]

trace_preempt_enable() <-- Stop preempt disable timer (Diff B)

__preempt_count_sub(1) <-- CPU re-enables preemption

The preempt_irqsoff tracer will break this into two parts: Diff A and Diff B, when in reality, the total time the CPU prevented preemption was A + B.

Note, we can have the same race if an interrupt came in just after calling trace_preempt_enable() and before the __preempt_count_sub().

What I would recommend is adding a flag to the task_struct that gets set before the __preempt_count_add() and cleared by the tracing function. If an interrupt goes off during this time, it will start the total time to record, and not end it on the trace_hardirqs_on() part. Now since we set this flag before disabling preemption, what if we get preempted before calling __preempt_count_add()?. Simple, have a hook in the scheduler (just connect to the sched_switch tracepoint) that checks that flag, and if it is set, it ends the preempt disable recording time. Also on scheduling that task back in, if that flag is set, start the preempt disable timer.

It may get a bit more complex, but we can contain that complexity within the tracing code, and we don't have to do added irq disabling.

-- Steve

maintainer
#12 lore raw

Re: [RFC 2/3] preempt_tracer: Disable IRQ while starting/stopping due to a preempt_counter change

Quoted 9 lines · Steven Rostedt

I believe I see what Daniel is talking about, but I hate the proposed solution ;-)

First, if you care about real times that the CPU can't preempt (preempt_count != 0 or interrupts disabled), then you want the preempt_irqsoff tracer. The preempt_tracer is more academic where it just shows you when we disable preemption via the counter. But even with the preempt_irqsoff tracer you may not get the full length of time due to the above explained race.

IOW, that tracer gives a completely 'make believe' number? What's the point? Just delete the pure preempt tracer.

And the preempt_irqoff tracer had better also consume the IRQ events, and if it does that it can DTRT without extra bits on, even with that race.

Consider:

	preempt_disable()
	  preempt_count += 1;
	  <IRQ>
	    trace_irq_enter();

	    trace_irq_exit();
	  </IRQ>
	  trace_preempt_disable();

	/* does stuff */

	preempt_enable()
	  preempt_count -= 1;
	  trace_preempt_enable();

You're saying preempt_irqoff() fails to connect the two because of the hole between trace_irq_exit() and trace_preempt_disable() ?

But trace_irq_exit() can see the raised preempt_count and set state for trace_preempt_disable() to connect.

Quoted 10 lines

What I would recommend is adding a flag to the task_struct that gets set before the __preempt_count_add() and cleared by the tracing function. If an interrupt goes off during this time, it will start the total time to record, and not end it on the trace_hardirqs_on() part. Now since we set this flag before disabling preemption, what if we get preempted before calling __preempt_count_add()?. Simple, have a hook in the scheduler (just connect to the sched_switch tracepoint) that checks that flag, and if it is set, it ends the preempt disable recording time. Also on scheduling that task back in, if that flag is set, start the preempt disable timer.

I don't think that works, you also have to consider softirq. And yes you can make it more complicated, but I still don't see the point.

And none of this is relevant for Daniels model stuff. He just needs to consider in-IRQ as !preempt.

maintainer
#13 lore raw

Re: [RFC 2/3] preempt_tracer: Disable IRQ while starting/stopping due to a preempt_counter change

Quoted 11 lines · Peter Zijlstra
Quoted 9 lines · Steven Rostedt

I believe I see what Daniel is talking about, but I hate the proposed solution ;-)

First, if you care about real times that the CPU can't preempt (preempt_count != 0 or interrupts disabled), then you want the preempt_irqsoff tracer. The preempt_tracer is more academic where it just shows you when we disable preemption via the counter. But even with the preempt_irqsoff tracer you may not get the full length of time due to the above explained race.

Quoted 2 lines

IOW, that tracer gives a completely 'make believe' number? What's the point? Just delete the pure preempt tracer.

Alternatively, fix the preempt tracer by having it completely disregard IRQs.

#14 lore raw

Re: [RFC 2/3] preempt_tracer: Disable IRQ while starting/stopping due to a preempt_counter change

In reply to Peter Zijlstra
Quoted 11 lines · On Wed, 29 May 2019 15:19:57 +0200 Peter Zijlstra <[email protected]> wrote:
Quoted 9 lines · Steven Rostedt

I believe I see what Daniel is talking about, but I hate the proposed solution ;-)

First, if you care about real times that the CPU can't preempt (preempt_count != 0 or interrupts disabled), then you want the preempt_irqsoff tracer. The preempt_tracer is more academic where it just shows you when we disable preemption via the counter. But even with the preempt_irqsoff tracer you may not get the full length of time due to the above explained race.

Quoted 2 lines

IOW, that tracer gives a completely 'make believe' number? What's the point? Just delete the pure preempt tracer.

The preempt_tracer is there as part of the preempt_irqsoff tracer implementation. By removing it, the only code we would remove is displaying preemptoff as a tracer. I stated this when it was created, that it was more of an academic exercise if you use it, but that code was required to get preempt_irqsoff working.

Quoted 22 lines

And the preempt_irqoff tracer had better also consume the IRQ events, and if it does that it can DTRT without extra bits on, even with that race.

Consider:

	preempt_disable()
	  preempt_count += 1;
	  <IRQ>
	    trace_irq_enter();

	    trace_irq_exit();
	  </IRQ>
	  trace_preempt_disable();

	/* does stuff */

	preempt_enable()
	  preempt_count -= 1;
	  trace_preempt_enable();
Quoted 5 lines

You're saying preempt_irqoff() fails to connect the two because of the hole between trace_irq_exit() and trace_preempt_disable() ?

But trace_irq_exit() can see the raised preempt_count and set state for trace_preempt_disable() to connect.

That's basically what I was suggesting as the solution to this ;-)

Quoted 12 lines
Quoted 11 lines

What I would recommend is adding a flag to the task_struct that gets set before the __preempt_count_add() and cleared by the tracing function. If an interrupt goes off during this time, it will start the total time to record, and not end it on the trace_hardirqs_on() part. Now since we set this flag before disabling preemption, what if we get preempted before calling __preempt_count_add()?. Simple, have a hook in the scheduler (just connect to the sched_switch tracepoint) that checks that flag, and if it is set, it ends the preempt disable recording time. Also on scheduling that task back in, if that flag is set, start the preempt disable timer.

Quoted 2 lines

I don't think that works, you also have to consider softirq. And yes you can make it more complicated, but I still don't see the point.

Note, there's places that disable preemption without being traced. If we trigger only on preemption being disabled and start the "timer", there may not be any code to stop it. That was why I recommended the flag in the code that starts the timing.

Quoted 3 lines

And none of this is relevant for Daniels model stuff. He just needs to consider in-IRQ as !preempt.

But he does bring up an issues that preempt_irqsoff fails.

-- Steve

maintainer
#15 lore raw

Re: [RFC 2/3] preempt_tracer: Disable IRQ while starting/stopping due to a preempt_counter change

Quoted 21 lines · Steven Rostedt
Quoted 21 lines

And the preempt_irqoff tracer had better also consume the IRQ events, and if it does that it can DTRT without extra bits on, even with that race.

Consider:

	preempt_disable()
	  preempt_count += 1;
	  <IRQ>
	    trace_irq_enter();

	    trace_irq_exit();
	  </IRQ>
	  trace_preempt_disable();

	/* does stuff */

	preempt_enable()
	  preempt_count -= 1;
	  trace_preempt_enable();
Quoted 7 lines
Quoted 5 lines

You're saying preempt_irqoff() fails to connect the two because of the hole between trace_irq_exit() and trace_preempt_disable() ?

But trace_irq_exit() can see the raised preempt_count and set state for trace_preempt_disable() to connect.

That's basically what I was suggesting as the solution to this ;-)

You were wanting changes to preempt_disable() and task_struct, neither of which is required. The above only needs some per-cpu storage in the tracer implementation.

#16 lore raw

Re: [RFC 2/3] preempt_tracer: Disable IRQ while starting/stopping due to a preempt_counter change

Quoted 5 lines · On Wed, 29 May 2019 15:49:46 +0200 Peter Zijlstra <[email protected]> wrote:
Quoted 1 line

That's basically what I was suggesting as the solution to this ;-)

You were wanting changes to preempt_disable() and task_struct, neither of which is required. The above only needs some per-cpu storage in the tracer implementation.

Only changes were to the trace preempt_disable() code. Which I still think needs to be done regardless to differentiate between when we are tracing preempt disable and when we are not.

And no modification would need to be done to task_struct as we already have a place to add tracing flags.

-- Steve

#17 lore raw

Re: [RFC 2/3] preempt_tracer: Disable IRQ while starting/stopping due to a preempt_counter change

In reply to Peter Zijlstra
Quoted 36 lines · Peter Zijlstra
Quoted 35 lines · Daniel Bristot de Oliveira
Quoted 20 lines · Peter Zijlstra
Quoted 19 lines · Daniel Bristot de Oliveira

The preempt_disable/enable tracepoint only traces in the disable <-> enable case, which is correct. But think about this case:

---------------------------- %< ------------------------------

	THREAD					IRQ
	   |					 |
preempt_disable() {
    __preempt_count_add(1)
	------->	    smp_apic_timer_interrupt() {
				preempt_disable()
				    do not trace (preempt count >= 1)
				    ....
				preempt_enable()
				    do not trace (preempt count >= 1)
			    }
    trace_preempt_disable();
}

---------------------------- >% ------------------------------

Quoted 4 lines
Quoted 1 line

The tracepoint will be skipped.

.... for the IRQ. But IRQs are not preemptible anyway, so what the problem?

right, they are.

exposing my problem in a more specific way:

To show in a model that an event always takes place with preemption disabled, but not necessarily with IRQs disabled, it is worth having the preemption disable events separated from IRQ disable ones.

Quoted 7 lines
Quoted 3 lines

The main reason is that, although IRQs disabled postpone the execution of the scheduler, it is more pessimistic, as it also delays IRQs. So the more precise the model is, the less pessimistic the analysis will be.

I'm not sure I follow, IRQs disabled fully implies !preemptible. I don't see how the model would be more pessimistic than reality if it were to use this knowledge.

Maybe I did not expressed myself well... and the example was not good either.

"IRQs disabled fully implies !preemptible" is a "to big" step. In modeling (or mathematical reasoning?), a good practice is to break the properties into small piece, and then build more complex reasoning/implications using these "small properties."

Doing "big steps" makes you prone "miss interpretations", creating ambiguity. Then, -RT people are prone to be pessimist, non-RT optimistic, and so on... and that is what models try to avoid.

For instance, explaining this using words is contradictory:>

Quoted 2 lines

Any !0 preempt_count(), which very much includes (Hard)IRQ and SoftIRQ counts, means non-preemptible.

One might argue that, the preemption of a thread always takes place with preempt_count() != 0, because __schedule() is always called with preemption disabled, so the preemption takes place while in non-preemptive.

A more elaborated example:

------------------ %< -------------------------- Thread A is running, and goes to sleep waiting for a timer...

	schedule() {
		preempt_disable();
		__schedule() {
			smp_apic_timer_interrupt() {
				sched_wakeup (Thread A);
				sched_wakeup (Thread B: highest prio) {
					sched_set_need_resched();
				}
			}
			local_irq_disable()
			context switch to B, A leaves in state=R.

------------------ %< --------------------------

In this case, the thread A suffered a "preemption" with "!0 preempt_count()"

The fact is, Linux does not fit straight in the "well known terminology" of academic papers because many of those terminology bases in the fact that operations are atomic. But they are not and Linux has some behaviors that desires new terminology/interpretation...

- WAIT But you (daniel) wants to fake the atomicity between preempt_disable and its tracepoint!

Yes, I do, but this is a very straightforward step/assumption: the atomicity is about the real-event and the tracepoint that notifies it. It is not about two different events.

That is why it is worth letting the modeling rules to clarify the behavior of system, without doing non-obvious implication in the code part, so we can have a model that fits better in the Linux actions/events to avoid ambiguity.

[ note 1: the tracepoint is only enabled if CONFIG_PREEMPTIRQ_TRACEPOINTS=y
which is not enabled by default ]

[ note 2: I just saw that Steven replied while I was writing this email... I will read them now... sorry for some repetitive topic here ]

-- Daniel

maintainer
#18 lore raw

Re: [RFC 2/3] preempt_tracer: Disable IRQ while starting/stopping due to a preempt_counter change

Quoted 8 lines · Daniel Bristot de Oliveira
Quoted 3 lines · Peter Zijlstra

I'm not sure I follow, IRQs disabled fully implies !preemptible. I don't see how the model would be more pessimistic than reality if it were to use this knowledge.

Maybe I did not expressed myself well... and the example was not good either.

Quoted 8 lines

"IRQs disabled fully implies !preemptible" is a "to big" step. In modeling (or mathematical reasoning?), a good practice is to break the properties into small piece, and then build more complex reasoning/implications using these "small properties."

Doing "big steps" makes you prone "miss interpretations", creating ambiguity. Then, -RT people are prone to be pessimist, non-RT optimistic, and so on... and that is what models try to avoid.

You already construct the big model out of small generators, this is just one more of those little generators.

Quoted 7 lines

For instance, explaining this using words is contradictory:>

Quoted 2 lines

Any !0 preempt_count(), which very much includes (Hard)IRQ and SoftIRQ counts, means non-preemptible.

One might argue that, the preemption of a thread always takes place with preempt_count() != 0, because __schedule() is always called with preemption disabled, so the preemption takes place while in non-preemptive.

Yeah, I know about that one; you've used it in your talks. Also, you've modeled the schedule preempt disable as a special state. If you want we can actually make it a special bit in the preempt_count word too, the patch shouldn't be too hard, although it would make no practical difference.

Quoted 10 lines

- WAIT But you (daniel) wants to fake the atomicity between preempt_disable and its tracepoint!

Yes, I do, but this is a very straightforward step/assumption: the atomicity is about the real-event and the tracepoint that notifies it. It is not about two different events.

That is why it is worth letting the modeling rules to clarify the behavior of system, without doing non-obvious implication in the code part, so we can have a model that fits better in the Linux actions/events to avoid ambiguity.

You can easily build a little shim betwen the model and the tracehooks that fix this up if you don't want to stick it in the model proper.

All the information is there. Heck you can even do that 3/3 thing internally I think.

#19 lore raw

Re: [RFC 2/3] preempt_tracer: Disable IRQ while starting/stopping due to a preempt_counter change

Quoted 14 lines · Peter Zijlstra
Quoted 13 lines · Daniel Bristot de Oliveira
Quoted 3 lines · Peter Zijlstra

I'm not sure I follow, IRQs disabled fully implies !preemptible. I don't see how the model would be more pessimistic than reality if it were to use this knowledge.

Maybe I did not expressed myself well... and the example was not good either.

"IRQs disabled fully implies !preemptible" is a "to big" step. In modeling (or mathematical reasoning?), a good practice is to break the properties into small piece, and then build more complex reasoning/implications using these "small properties."

Quoted 6 lines
Quoted 3 lines

Doing "big steps" makes you prone "miss interpretations", creating ambiguity. Then, -RT people are prone to be pessimist, non-RT optimistic, and so on... and that is what models try to avoid.

You already construct the big model out of small generators, this is just one more of those little generators.

Yes, we can take that way too...

Quoted 8 lines
Quoted 7 lines

For instance, explaining this using words is contradictory:>

Quoted 2 lines

Any !0 preempt_count(), which very much includes (Hard)IRQ and SoftIRQ counts, means non-preemptible.

One might argue that, the preemption of a thread always takes place with preempt_count() != 0, because __schedule() is always called with preemption disabled, so the preemption takes place while in non-preemptive.

Quoted 5 lines

Yeah, I know about that one; you've used it in your talks. Also, you've modeled the schedule preempt disable as a special state. If you want we can actually make it a special bit in the preempt_count word too, the patch shouldn't be too hard, although it would make no practical difference.

Good to know! I am trying not to change the code or abstractions. In the user-space version I was using the caller address to figure out if it was a call in the scheduler or not. In the kernel version I am planing to use a is_sched_function() like approach. But if nothing of that works, I will explore this possibility, thanks for the suggestion.

Quoted 11 lines
Quoted 10 lines

- WAIT But you (daniel) wants to fake the atomicity between preempt_disable and its tracepoint!

Yes, I do, but this is a very straightforward step/assumption: the atomicity is about the real-event and the tracepoint that notifies it. It is not about two different events.

That is why it is worth letting the modeling rules to clarify the behavior of system, without doing non-obvious implication in the code part, so we can have a model that fits better in the Linux actions/events to avoid ambiguity.

Quoted 6 lines

You can easily build a little shim betwen the model and the tracehooks that fix this up if you don't want to stick it in the model proper.

All the information is there. Heck you can even do that 3/3 thing internally I think.

Ack, let's see what I can came up.

Thanks!

-- Daniel

#20 lore raw

Re: [RFC 2/3] preempt_tracer: Disable IRQ while starting/stopping due to a preempt_counter change

In reply to Daniel Bristot de Oliveira
Quoted 39 lines · Daniel Bristot de Oliveira
Quoted 20 lines · Peter Zijlstra
Quoted 19 lines · Daniel Bristot de Oliveira

The preempt_disable/enable tracepoint only traces in the disable <-> enable case, which is correct. But think about this case:

---------------------------- %< ------------------------------

	THREAD					IRQ
	   |					 |
preempt_disable() {
    __preempt_count_add(1)
	------->	    smp_apic_timer_interrupt() {
				preempt_disable()
				    do not trace (preempt count >= 1)
				    ....
				preempt_enable()
				    do not trace (preempt count >= 1)
			    }
    trace_preempt_disable();
}

---------------------------- >% ------------------------------

Quoted 4 lines
Quoted 1 line

The tracepoint will be skipped.

.... for the IRQ. But IRQs are not preemptible anyway, so what the problem?

right, they are.

exposing my problem in a more specific way:

To show in a model that an event always takes place with preemption disabled, but not necessarily with IRQs disabled, it is worth having the preemption disable events separated from IRQ disable ones.

The main reason is that, although IRQs disabled postpone the execution of the scheduler, it is more pessimistic, as it also delays IRQs. So the more precise the model is, the less pessimistic the analysis will be.

Quoted 8 lines

But there are other use-cases, for instance:

(Steve, correct me if I am wrong)

The preempt_tracer will not notice a "preempt disabled" section in an IRQ handler if the problem above happens.

(Yeah, I know these problems are very specific... but...)

I agree with the problem. I think Daniel does not want to miss the preemption disabled event caused by the IRQ disabling.

Quoted 27 lines
Quoted 15 lines
Quoted 14 lines

To avoid skipping the trace, the change in the counter should be "atomic" with the start/stop, w.r.t the interrupts.

Disable interrupts while the adding/starting stopping/subtracting.

kernel/sched/core.c +9 -0
+static inline void preempt_add_start_latency(int val)
+{
+	unsigned long flags;
+
+	raw_local_irq_save(flags);
+	__preempt_count_add(val);
+	preempt_latency_start(val);
+	raw_local_irq_restore(flags);
+}
Quoted 11 lines

kernel/sched/core.c void preempt_latency_stop(int val) · line 3215

Quoted 9 lines
kernel/sched/core.c +9 -0
+static inline void preempt_sub_stop_latency(int val)
+{
+	unsigned long flags;
+
+	raw_local_irq_save(flags);
+	preempt_latency_stop(val);
+	__preempt_count_sub(val);
+	raw_local_irq_restore(flags);
+}

That is hideously expensive :/

Quoted 4 lines

Yeah... :-( Is there another way to provide such "atomicity"?

Can I use the argument "if one has these tracepoints enabled, they are not considering it as a hot-path?"

The only addition here seems to the raw_local_irq_{save,restore} around the calls to increment the preempt counter and start the latency tracking.

Is there any performance data with the tracepoint enabled and with/without this patch? Like with hackbench?

Thanks.

#21 lore raw

Re: [RFC 2/3] preempt_tracer: Disable IRQ while starting/stopping due to a preempt_counter change

Quoted 44 lines · Joel Fernandes
Quoted 43 lines · Daniel Bristot de Oliveira
Quoted 20 lines · Peter Zijlstra
Quoted 19 lines · Daniel Bristot de Oliveira

The preempt_disable/enable tracepoint only traces in the disable <-> enable case, which is correct. But think about this case:

---------------------------- %< ------------------------------

	THREAD					IRQ
	   |					 |
preempt_disable() {
    __preempt_count_add(1)
	------->	    smp_apic_timer_interrupt() {
				preempt_disable()
				    do not trace (preempt count >= 1)
				    ....
				preempt_enable()
				    do not trace (preempt count >= 1)
			    }
    trace_preempt_disable();
}

---------------------------- >% ------------------------------

Quoted 4 lines
Quoted 1 line

The tracepoint will be skipped.

.... for the IRQ. But IRQs are not preemptible anyway, so what the problem?

right, they are.

exposing my problem in a more specific way:

To show in a model that an event always takes place with preemption disabled, but not necessarily with IRQs disabled, it is worth having the preemption disable events separated from IRQ disable ones.

The main reason is that, although IRQs disabled postpone the execution of the scheduler, it is more pessimistic, as it also delays IRQs. So the more precise the model is, the less pessimistic the analysis will be.

But there are other use-cases, for instance:

(Steve, correct me if I am wrong)

Quoted 7 lines
Quoted 4 lines

The preempt_tracer will not notice a "preempt disabled" section in an IRQ handler if the problem above happens.

(Yeah, I know these problems are very specific... but...)

I agree with the problem. I think Daniel does not want to miss the preemption disabled event caused by the IRQ disabling.

Hi Joel!

Correct, but ... look bellow.

Quoted 29 lines
Quoted 29 lines
Quoted 15 lines
Quoted 14 lines

To avoid skipping the trace, the change in the counter should be "atomic" with the start/stop, w.r.t the interrupts.

Disable interrupts while the adding/starting stopping/subtracting.

kernel/sched/core.c +9 -0
+static inline void preempt_add_start_latency(int val)
+{
+	unsigned long flags;
+
+	raw_local_irq_save(flags);
+	__preempt_count_add(val);
+	preempt_latency_start(val);
+	raw_local_irq_restore(flags);
+}
Quoted 11 lines

kernel/sched/core.c void preempt_latency_stop(int val) · line 3215

Quoted 9 lines
kernel/sched/core.c +9 -0
+static inline void preempt_sub_stop_latency(int val)
+{
+	unsigned long flags;
+
+	raw_local_irq_save(flags);
+	preempt_latency_stop(val);
+	__preempt_count_sub(val);
+	raw_local_irq_restore(flags);
+}

That is hideously expensive :/

Yeah... :-( Is there another way to provide such "atomicity"?

Quoted 8 lines
Quoted 2 lines

Can I use the argument "if one has these tracepoints enabled, they are not considering it as a hot-path?"

The only addition here seems to the raw_local_irq_{save,restore} around the calls to increment the preempt counter and start the latency tracking.

Is there any performance data with the tracepoint enabled and with/without this patch? Like with hackbench?

I discussed this with Steve at the Summit on the Summit (the reason why I did not reply this email earlier is because I was in the conf/traveling), and he also agrees with peterz, disabling and (mainly) re-enabling IRQs costs too much.

We need to find another way to resolve this problem (or mitigate the cost).... :-(.

Ideas?

Thanks!!

-- Daniel

#22 lore raw

Re: [RFC 2/3] preempt_tracer: Disable IRQ while starting/stopping due to a preempt_counter change

Quoted 7 lines · On Tue, 4 Jun 2019 12:12:36 +0200 Daniel Bristot de Oliveira…

I discussed this with Steve at the Summit on the Summit (the reason why I did not reply this email earlier is because I was in the conf/traveling), and he also agrees with peterz, disabling and (mainly) re-enabling IRQs costs too much.

We need to find another way to resolve this problem (or mitigate the cost).... :-(.

Ideas?

I thought we talked about using flags in the pc to let us know that we are about to trace the preemption off?

If an interrupt comes in, we check the flag:

	irq_enter()
		preempt_count_add(HARDIRQ_OFFSET)

Then we can have something like:

preempt_count_add(val) {
	int pc = preempt_count();
	int trace_val = TRACE_FLAG;

	if (val == HARDIRQ_OFFSET && (pc & TRACE_FLAG)) {
		__preempt_count_sub(TRACE_FLAG);
		trace_val |= TRACE_SET;
	}

	__preempt_count_add(val + trace_val);
	if (!pc)
		trace_preempt_disable();
	__preempt_count_sub(trace_val);

And in trace_preempt_enable() we have:

	if ((preempt_count() & TRACE_SET) && in_irq())
		return;

Thus, we wont call the preempt_enable tracing code if we started it due to an interrupt preempting the process of setting the trace.

Note, I'm writing this while extremely tired (still have yet to get more than 4 hours of sleep) so it may still have bugs, but you should get the idea ;-)

-- Steve

#23 lore raw

Re: [RFC 2/3] preempt_tracer: Disable IRQ while starting/stopping due to a preempt_counter change

Quoted 38 lines · Steven Rostedt
Quoted 7 lines · On Tue, 4 Jun 2019 12:12:36 +0200 Daniel Bristot de Oliveira…

I discussed this with Steve at the Summit on the Summit (the reason why I did not reply this email earlier is because I was in the conf/traveling), and he also agrees with peterz, disabling and (mainly) re-enabling IRQs costs too much.

We need to find another way to resolve this problem (or mitigate the cost).... :-(.

Ideas?

I thought we talked about using flags in the pc to let us know that we are about to trace the preemption off?

If an interrupt comes in, we check the flag:

	irq_enter()
		preempt_count_add(HARDIRQ_OFFSET)

Then we can have something like:

preempt_count_add(val) {
	int pc = preempt_count();
	int trace_val = TRACE_FLAG;

	if (val == HARDIRQ_OFFSET && (pc & TRACE_FLAG)) {
		__preempt_count_sub(TRACE_FLAG);
		trace_val |= TRACE_SET;
	}

	__preempt_count_add(val + trace_val);
	if (!pc)
		trace_preempt_disable();
	__preempt_count_sub(trace_val);
Quoted 7 lines

And in trace_preempt_enable() we have:

	if ((preempt_count() & TRACE_SET) && in_irq())
		return;

Thus, we wont call the preempt_enable tracing code if we started it due to an interrupt preempting the process of setting the trace.

Hmm, the interrupt handler will not be calling preempt_enable anyway since the preempt counter was already set by the interrupted code. The situation Daniel describes in patch 2/3 is:

---------------------------- %< ------------------------------

	THREAD					IRQ
	   |					 |
preempt_disable() {
    __preempt_count_add(1)
	------->	    smp_apic_timer_interrupt() {
				preempt_disable()
				    do not trace (preempt count >= 1)
				    ....
				preempt_enable()
				    do not trace (preempt count >= 1)
			    }
    trace_preempt_disable();
}

---------------------------- >% ------------------------------

Here the preempt_enable in the IRQ will not call tracing. If I understand correctly, he *does* want to call tracing since the IRQ's preempt disable/enable section could be running for some time, and so it would not be nice to miss these events from the traces.

Regarding the preempt count flag idea, I really like your idea. I did not fully follow the code snip you shared above though. My understanding of your idea initially was, we set a flag in pc, then increment the preempt count, call tracing and the reset the pc flag. In between the increment and the call to the tracer, if we get interrupted, then we check the flag from IRQ context, and still call tracing. If we get interrupted after calling tracing, but before resetting the flag, then we may end up with a nested pair of preempt enable/disable calls, but that's Ok. At least we wont miss any events.

At least I think we only need to modify the preempt disable path, something like this in pseudo code in the preempt disable path (of the THREAD context):

set pc flag
inc pc
<--- if interrupted here, still trace from IRQ.
call tracer
<--- if interrupted here, still trace from IRQ.
reset pc flag
<--- if interrupted here, no need to trace from IRQ since
we already recorded at least one preempt disable event.

Did I get your idea right?

I need to think about this some more, but got to run to an appointment. Will take a look once I'm back.

Quoted 3 lines

Note, I'm writing this while extremely tired (still have yet to get more than 4 hours of sleep) so it may still have bugs, but you should get the idea ;-)

No problem ;-)

thanks,

- Joel

#24 lore raw

RFC 3/3 preempt_tracer: Use a percpu variable to control traceble calls

In reply to Daniel Bristot de Oliveira

The preempt_disable tracepoint only traces in the disable <-> enable case. Which is correct, but think about this case:

--------------------------- %< ----------------------

	 THREAD					IRQ
	   |					 |
preempt_disable_notrace() {
    __preempt_count_add(1)
}

/* preemption disabled/IRQs enabled */

	------->		smp_apic_timer_interrupt() {
				    preempt_disable() {
					do not trace (preempt count >= 1)
				    }
				    ....
				    preempt_enable() {
						do not trace (preempt count >= 1)
				    }
	<-------		}
preempt_enable_notrace() {
    __preempt_count_sub(1)
}

--------------------------- >% ----------------------

The non-traceble preempt_disable can hide a legit preempt_disable (which is worth tracing).

It is possible to observe this problem using this kernel module:

http://bristot.me/files/efficient_verification/wip.tar.gz

and doing the following trace:

	# cd /sys/kernel/debug/tracing/
	# echo 1 > snapshot
	# cat available_events | grep preempt_ > set_event
	# echo irq_vectors >> /sys/kernel/debug/tracing/set_event
	# insmod wip.ko
	/* wait for a snapshot creation */
	# tail -100 snapshot
	    sshd-1159  [000] d...1..  2440.866116: preempt_enable: caller=migrate_enable+0x1bb/0x330 parent=migrate_enable+0x1bb/0x330
	    sshd-1159  [000] d..h1..  2440.866122: local_timer_entry: vector=236
	    sshd-1159  [000] d..h1..  2440.866127: local_timer_exit: vector=236
	    sshd-1159  [000] d.L.4..  2440.866129: process_event: event sched_waking not expected in the state preemptive
	    sshd-1159  [000] d.L.4..  2440.866137: <stack trace>
	 => process_event
	 => __handle_event
	 => ttwu_do_wakeup
	 => try_to_wake_up
	 => irq_exit
	 => smp_apic_timer_interrupt
	 => apic_timer_interrupt
	 => kvm_clock_read
	 => ktime_get_with_offset
	 => posix_get_boottime
	 => __x64_sys_clock_gettime
	 => do_syscall_64
	 => entry_SYSCALL_64_after_hwframe

and kvm_clock_read() disables preemption without tracing:

--------------------------- %< ---------------------- static u64 kvm_clock_read(void)

{
	u64 ret;

	preempt_disable_notrace();
	ret = pvclock_clocksource_read(this_cpu_pvti());
	preempt_enable_notrace();
	return ret;
}

--------------------------- >% ----------------------

Use a percpu variable for the traced preempt_disable/enable, and use it to decide whether trace or not.

Cc "Steven Rostedt (VMware)" <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Matthias Kaehlcke <[email protected]>
Cc "Joel Fernandes (Google)" <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Yangtao Li <[email protected]>
Cc Tommaso Cucinotta <[email protected]>
---
 kernel/sched/core.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
kernel/sched/core.c +12 -2
index be4117d7384f..2e07d4174778 100644
@@ -3148,19 +3148,25 @@ static inline void sched_tick_stop(int cpu) { }
 
 #if defined(CONFIG_PREEMPT) && (defined(CONFIG_DEBUG_PREEMPT) || \
 				defined(CONFIG_TRACE_PREEMPT_TOGGLE))
+
+DEFINE_PER_CPU(int, __traced_preempt_count) = 0;
 /*
  * If the value passed in is equal to the current preempt count
  * then we just disabled preemption. Start timing the latency.
  */
 void preempt_latency_start(int val)
 {
-	if (preempt_count() == val) {
+	int curr = this_cpu_read(__traced_preempt_count);
+
+	if (!curr) {
 		unsigned long ip = get_lock_parent_ip();
 #ifdef CONFIG_DEBUG_PREEMPT
 		current->preempt_disable_ip = ip;
 #endif
 		trace_preempt_off(CALLER_ADDR0, ip);
 	}
+
+	this_cpu_write(__traced_preempt_count, curr + val);
 }
 
 static inline void preempt_add_start_latency(int val)
@@ -3200,8 +3206,12 @@ NOKPROBE_SYMBOL(preempt_count_add);
  */
 void preempt_latency_stop(int val)
 {
-	if (preempt_count() == val)
+	int curr = this_cpu_read(__traced_preempt_count) - val;
+
+	if (!curr)
 		trace_preempt_on(CALLER_ADDR0, get_lock_parent_ip());
+
+	this_cpu_write(__traced_preempt_count, curr);
 }
 
 static inline void preempt_sub_stop_latency(int val)
Signature · 1 line
2.20.1
maintainer
#25 lore raw

Re: [RFC 3/3] preempt_tracer: Use a percpu variable to control traceble calls

kernel/sched/core.c static inline void sched_tick_stop(int cpu) { } · line 3159

Quoted 12 lines · Daniel Bristot de Oliveira

#if defined(CONFIG_PREEMPT) && (defined(CONFIG_DEBUG_PREEMPT) || \

kernel/sched/core.c +3 -1
 				defined(CONFIG_TRACE_PREEMPT_TOGGLE))
+
+DEFINE_PER_CPU(int, __traced_preempt_count) = 0;
 /*
  * If the value passed in is equal to the current preempt count
  * then we just disabled preemption. Start timing the latency.
  */
 void preempt_latency_start(int val)
 {
-	if (preempt_count() == val) {
+	int curr = this_cpu_read(__traced_preempt_count);

We actually have this_cpu_add_return();

Quoted 14 lines
kernel/sched/core.c +4 -0
+
+	if (!curr) {
 		unsigned long ip = get_lock_parent_ip();
 #ifdef CONFIG_DEBUG_PREEMPT
 		current->preempt_disable_ip = ip;
 #endif
 		trace_preempt_off(CALLER_ADDR0, ip);
 	}
+
+	this_cpu_write(__traced_preempt_count, curr + val);
 }
 
 static inline void preempt_add_start_latency(int val)

kernel/sched/core.c NOKPROBE_SYMBOL(preempt_count_add); · line 3209

Quoted 5 lines
kernel/sched/core.c +1 -1
@@ -3200,8 +3206,12 @@ NOKPROBE_SYMBOL(preempt_count_add);
  */
 void preempt_latency_stop(int val)
 {
-	if (preempt_count() == val)
+	int curr = this_cpu_read(__traced_preempt_count) - val;

this_cpu_sub_return();

kernel/sched/core.c NOKPROBE_SYMBOL(preempt_count_add); · line 3215

Quoted 6 lines
kernel/sched/core.c +4 -0
+
+	if (!curr)
 		trace_preempt_on(CALLER_ADDR0, get_lock_parent_ip());
+
+	this_cpu_write(__traced_preempt_count, curr);
 }

Can't say I love this, but it is miles better than the last patch.

#26 lore raw

Re: [RFC 3/3] preempt_tracer: Use a percpu variable to control traceble calls

Quoted 38 lines · Peter Zijlstra

kernel/sched/core.c static inline void sched_tick_stop(int cpu) { } · line 3159

Quoted 12 lines · Daniel Bristot de Oliveira

#if defined(CONFIG_PREEMPT) && (defined(CONFIG_DEBUG_PREEMPT) || \

kernel/sched/core.c +3 -1
 				defined(CONFIG_TRACE_PREEMPT_TOGGLE))
+
+DEFINE_PER_CPU(int, __traced_preempt_count) = 0;
 /*
  * If the value passed in is equal to the current preempt count
  * then we just disabled preemption. Start timing the latency.
  */
 void preempt_latency_start(int val)
 {
-	if (preempt_count() == val) {
+	int curr = this_cpu_read(__traced_preempt_count);

We actually have this_cpu_add_return();

Quoted 14 lines
kernel/sched/core.c +4 -0
+
+	if (!curr) {
 		unsigned long ip = get_lock_parent_ip();
 #ifdef CONFIG_DEBUG_PREEMPT
 		current->preempt_disable_ip = ip;
 #endif
 		trace_preempt_off(CALLER_ADDR0, ip);
 	}
+
+	this_cpu_write(__traced_preempt_count, curr + val);
 }
 
 static inline void preempt_add_start_latency(int val)

kernel/sched/core.c NOKPROBE_SYMBOL(preempt_count_add); · line 3209

Quoted 5 lines
kernel/sched/core.c +1 -1
@@ -3200,8 +3206,12 @@ NOKPROBE_SYMBOL(preempt_count_add);
  */
 void preempt_latency_stop(int val)
 {
-	if (preempt_count() == val)
+	int curr = this_cpu_read(__traced_preempt_count) - val;

this_cpu_sub_return();

Quoted 9 lines

kernel/sched/core.c NOKPROBE_SYMBOL(preempt_count_add); · line 3215

Quoted 6 lines
kernel/sched/core.c +4 -0
+
+	if (!curr)
 		trace_preempt_on(CALLER_ADDR0, get_lock_parent_ip());
+
+	this_cpu_write(__traced_preempt_count, curr);
 }

Can't say I love this, but it is miles better than the last patch.

ack! I will change the methods (and remove some blank lines here and there... :-))

Thanks Peter!

-- Daniel