Fix excessive host tick latencies

RFC In review Jan Kiszka 3 participants 6 messages Apr 30, 2010 – May 1, 2010

lore

Share & tools

lore

Jan Kiszka

author

2010-04-30 13:33 UTC #0 lore raw

RFC 1/1 Fix excessive host tick latencies

Hi Philippe,

I'm not 100% sure if this plugs all remaining wholes in the deferred host tick processing, but at least the most easiest reproducible one is cured now for me (Linux latency peak after termination of 'latency'). Please let me know if you see more potential issues, otherwise I would include this in my for-upstream queue.

Found while instrumenting the timer core to find RT timer latencies. Hope they become visible now...

Jan

-------

If a reschedule is pending, we won't allow the host timer to decide about the next hardware tick. But if that next reschedule will simply keep ROOT (which, e.g., happens during thread deletion), this deferral can impose significant unwanted timer latencies on Linux.

ksrc/nucleus/pod.c +8 -1

index 3ffd548..fb63f35 100644
@@ -2184,9 +2184,16 @@ void __xnpod_schedule(struct xnsched *sched)
 	zombie = xnthread_test_state(curr, XNZOMBIE);
 
 	next = xnsched_pick_next(sched);
-	if (next == curr && !xnthread_test_state(curr, XNRESTART))
+	if (next == curr && !xnthread_test_state(curr, XNRESTART)) {
 		/* Note: the root thread never restarts. */
+		if (unlikely(xnthread_test_state(next, XNROOT))) {
+			if (testbits(sched->status, XNHTICK))
+				xnintr_host_tick(sched);
+			if (testbits(sched->status, XNHDEFER))
+				xntimer_next_local_shot(sched);
+		}
 		goto signal_unlock_and_exit;
+	}
 
 	XENO_BUGON(NUCLEUS, need_resched == 0);

Gilles Chanteperdrix

2010-04-30 14:29 UTC #1 lore raw

Quoted 10 lines · Jan Kiszka

Hi Philippe,
I'm not 100% sure if this plugs all remaining wholes in the deferred host tick processing, but at least the most easiest reproducible one is cured now for me (Linux latency peak after termination of 'latency'). Please let me know if you see more potential issues, otherwise I would include this in my for-upstream queue.
Found while instrumenting the timer core to find RT timer latencies. Hope they become visible now...

Note that the behaviour of xnpod_schedule may have changed since the fix of the XENO_OPT_DEBUG_NUCLEUS issue. Notably, some calls to xnpod_schedule should now be skipped if nucleus debug is turned off.

Are you running the tests with or without nucleus debug?

Signature · 1 line

					    Gilles.

Jan Kiszka

author

2010-04-30 14:34 UTC #2 lore raw

Quoted 9 lines · Gilles Chanteperdrix

Quoted 8 lines · Jan Kiszka
Hi Philippe,
I'm not 100% sure if this plugs all remaining wholes in the deferred host tick processing, but at least the most easiest reproducible one is cured now for me (Linux latency peak after termination of 'latency'). Please let me know if you see more potential issues, otherwise I would include this in my for-upstream queue.

Quoted 9 lines

Quoted 2 lines
Found while instrumenting the timer core to find RT timer latencies. Hope they become visible now...
Note that the behaviour of xnpod_schedule may have changed since the fix of the XENO_OPT_DEBUG_NUCLEUS issue. Notably, some calls to xnpod_schedule should now be skipped if nucleus debug is turned off.
Are you running the tests with or without nucleus debug?

Without CONFIG_XENO_OPT_DEBUG_NUCLEUS and with the debug fix applied.

Jan

Signature · 2 lines

Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Philippe Gerum

2010-04-30 20:13 UTC #3 lore raw

In reply to Jan Kiszka

Quoted 7 lines · Jan Kiszka

Hi Philippe,
I'm not 100% sure if this plugs all remaining wholes in the deferred host tick processing, but at least the most easiest reproducible one is cured now for me (Linux latency peak after termination of 'latency'). Please let me know if you see more potential issues, otherwise I would include this in my for-upstream queue.

That patch is correct, please queue it. Anything that breaks the assumption described in the following comment from xntimer_next_local_shot() is wrong wrt tick deferral: "The host tick deferral is cleared whenever Xenomai is about to yield control to the host kernel".

In short, when the code will match the comments and documentation, we will be done with debugging.

Quoted 36 lines

Found while instrumenting the timer core to find RT timer latencies. Hope they become visible now...
Jan
-------
If a reschedule is pending, we won't allow the host timer to decide about the next hardware tick. But if that next reschedule will simply keep ROOT (which, e.g., happens during thread deletion), this deferral can impose significant unwanted timer latencies on Linux.
ksrc/nucleus/pod.c +8 -1
index 3ffd548..fb63f35 100644
@@ -2184,9 +2184,16 @@ void __xnpod_schedule(struct xnsched *sched)
 	zombie = xnthread_test_state(curr, XNZOMBIE);
 
 	next = xnsched_pick_next(sched);
-	if (next == curr && !xnthread_test_state(curr, XNRESTART))
+	if (next == curr && !xnthread_test_state(curr, XNRESTART)) {
 		/* Note: the root thread never restarts. */
+		if (unlikely(xnthread_test_state(next, XNROOT))) {
+			if (testbits(sched->status, XNHTICK))
+				xnintr_host_tick(sched);
+			if (testbits(sched->status, XNHDEFER))
+				xntimer_next_local_shot(sched);
+		}
 		goto signal_unlock_and_exit;
+	}
 
 	XENO_BUGON(NUCLEUS, need_resched == 0);
 

Signature · 1 line

Philippe.

Gilles Chanteperdrix

2010-04-30 21:01 UTC #4 lore raw

Quoted 9 lines · Philippe Gerum

Quoted 7 lines · Jan Kiszka
Hi Philippe,
I'm not 100% sure if this plugs all remaining wholes in the deferred host tick processing, but at least the most easiest reproducible one is cured now for me (Linux latency peak after termination of 'latency'). Please let me know if you see more potential issues, otherwise I would include this in my for-upstream queue.

Quoted 8 lines

That patch is correct, please queue it. Anything that breaks the assumption described in the following comment from xntimer_next_local_shot() is wrong wrt tick deferral: "The host tick deferral is cleared whenever Xenomai is about to yield control to the host kernel".
In short, when the code will match the comments and documentation, we will be done with debugging.

What I meant is that with the nucleus debugging fixed, we may not even enter __xnpod_schedule often enough to handle the host tick propagation. So, maybe we should make xnpod_schedule call __xnpod_schedule if XNHTICK or XNHDEFER are set.

Signature · 1 line

					    Gilles.

Philippe Gerum

2010-05-01 08:24 UTC #5 lore raw

Quoted 16 lines · Gilles Chanteperdrix

Quoted 15 lines · Philippe Gerum
Quoted 7 lines · Jan Kiszka
Hi Philippe,
I'm not 100% sure if this plugs all remaining wholes in the deferred host tick processing, but at least the most easiest reproducible one is cured now for me (Linux latency peak after termination of 'latency'). Please let me know if you see more potential issues, otherwise I would include this in my for-upstream queue.
That patch is correct, please queue it. Anything that breaks the assumption described in the following comment from xntimer_next_local_shot() is wrong wrt tick deferral: "The host tick deferral is cleared whenever Xenomai is about to yield control to the host kernel".

Quoted 8 lines

Quoted 2 lines
In short, when the code will match the comments and documentation, we will be done with debugging.
What I meant is that with the nucleus debugging fixed, we may not even enter __xnpod_schedule often enough to handle the host tick propagation. So, maybe we should make xnpod_schedule call __xnpod_schedule if XNHTICK or XNHDEFER are set.

I got that, but this is ok. Deferral is marked whenever the resched bit is already set, or we don't run over the root thread, which means that we will raise the resched bit to switch back to root at some point, and clear the deferral there. IOW, xnpod_schedule() cannot filter out a valid condition for clearing the host timer deferral, in any of its debug/non-debug forms.

Signature · 1 line

Philippe.