One-shot high-resolution POSIX timer periodically late
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 35 lines · John
I'm using the POSIX timers API. My platform is x86 running Linux 2.6.18.6 patched with the high-resolution timer subsystem.
http://www.tglx.de/hrtimers.html
I've written a small "de-jittering engine" that receives packets in small bursts due to network jitter (typical average rate of 1000 packets per second), and re-sends them at a "smooth" rate.
Just before I re-send a packet, I arm a one-shot timer in order to receive a signal when it is time to send the next packet.
I've noticed a strange phenomenon that I cannot explain.
Sometimes (rarely) the one-shot timer will expire more than 50 µs later than expected. This would seem normal, except that it happens periodically.
For example, my app had been running normally for 2 minutes when it started printing diagnostics (see below).
The first
T_NEXT_POPis the date the timer was supposed to expire,NOW is the date the timer was handled after returning from sigwaitinfo (I am aware that blocking signals, and handling them at a specific point in the code will add some latency)
The second
T_NEXT_POPis the date the next timer is supposed to expire.DIFF is the difference between real and expected dates.
(All dates are
CLOCK_MONOTONICby the way.)As you can see, the first diagnostic came at 472.410501014... Then another diagnostic almost exactly two seconds apart 9 times in a row!
Quoted 6 lines
My process is the only
SCHED_FIFOprocess on the system. There are no user-space processes with a higher priority. AFAICT, only a kernel thread could keep the CPU away from my app.Is there a periodic kernel thread that runs every 2 seconds, cannot be preempted, and runs for over 50 µs??
This sounds like a BIOS SMI issue. Can you reproduce this behavior on different hardware?
thanks
-john
Quoted 10 lines · john stultz
Quoted 9 lines · John
I'm using the POSIX timers API. My platform is x86 running Linux 2.6.18.6 patched with the high-resolution timer subsystem.
http://www.tglx.de/hrtimers.html
My process is the only
SCHED_FIFOprocess on the system. There are no user-space processes with a higher priority. AFAICT, only a kernel thread could keep the CPU away from my app.
Quoted 5 lines
Quoted 2 lines
Is there a periodic kernel thread that runs every 2 seconds, cannot be preempted, and runs for over 50 µs??
This sounds like a BIOS SMI issue. Can you reproduce this behavior on different hardware?
note that only the -hrt patchset is used - not the full -rt patchset - so 50 usecs delays (and more) are quite possible and common.
My question would be: does the same problem occur with the full -rt patchset and PREEMPT_RT? (see http://rt.wiki.kernel.org for details)
Ingo