CLOCK_MONOTONIC datagram timestamps by the kernel
Hello,
It is possible to ask Linux to timestamp incoming datagrams when they are received, then to retrieve this timestamp with an ioctl command or a recvmsg call (which would save one round trip to kernel space).
SIOCGSTAMP
Return a struct timeval with the receive timestamp of the last
packet passed to the user. This is useful for accurate round trip time measurements. See setitimer(2) for a description of struct timeval.
As far as I understand, this timestamp is given by the CLOCK_REALTIME clock. I would like to get the timestamp given by a different clock: the CLOCK_MONOTONIC clock.
In other words, I would like the kernel to do the equivalent of
struct timespec spec;
clock_gettime(CLOCK_MONOTONIC, &spec)
for each datagram the system receives, as soon as it is received.
Is there a way to achieve that?
Is there a different ioctl perhaps? (I don't think so.)
Regards.
Quoted 15 lines · John
It is possible to ask Linux to timestamp incoming datagrams when they are received, then to retrieve this timestamp with an ioctl command or a recvmsg call (which would save one round trip to kernel space).
SIOCGSTAMP
Return a struct timeval with the receive timestamp of the last
packet passed to the user. This is useful for accurate round trip time measurements. See setitimer(2) for a description of struct timeval.As far as I understand, this timestamp is given by the
CLOCK_REALTIMEclock. I would like to get the timestamp given by a different clock:
theCLOCK_MONOTONICclock.In other words, I would like the kernel to do the equivalent of
Quoted 8 lines
struct timespec spec;
clock_gettime(CLOCK_MONOTONIC, &spec)for each datagram the system receives, as soon as it is received.
Is there a way to achieve that?
Is there a different ioctl perhaps? (I don't think so.)
I've been reading the following documents:
Clocks and Timers
http://www.opengroup.org/onlinepubs/009695399/xrat/xsh_chap02.html#tag_03_02_08_18
Rationale for the Monotonic Clock
http://www.opengroup.org/onlinepubs/009695399/xrat/xsh_chap02.html#tag_03_02_08_19
"For those applications that use time services to achieve realtime behavior, changing the value of the clock on which these services rely may cause erroneous timing behavior. For these applications, it is necessary to have a monotonic clock which cannot run backwards, and which has a maximum clock jump that is required to be documented by the implementation. Additionally, it is desirable (but not required by IEEE Std 1003.1-2001) that the monotonic clock increases its value uniformly. This clock should not be affected by changes to the system time; for example, to synchronize the clock with an external source or to account for leap seconds. Such changes would cause errors in the measurement of time intervals for those time services that use the absolute value of the clock."
clock and timer functions
http://www.opengroup.org/onlinepubs/009695399/functions/clock_settime.html
"If the value of the CLOCK_REALTIME clock is set via clock_settime(), the new value of the clock shall be used to determine the time of expiration for absolute time services based upon the CLOCK_REALTIME clock. This applies to the time at which armed absolute timers expire. If the absolute time requested at the invocation of such a time service is before the new value of the clock, the time service shall expire immediately as if the clock had reached the requested time normally."
"The effect of setting a clock via clock_settime() on armed per-process timers associated with a clock other than CLOCK_REALTIME is implementation-defined."
Here's some background on what I'm trying to do:
My app expects a stream of UDP datagrams containing compressed video. These datagrams have been time stamped by the source using a high resolution clock.
The video stream is played out in real-time. I buffer several packets, then repeatedly arm a one-shot timer (with TIMER_ABSTIME set) to wait until it's time to send the oldest packet.
AFAIU, if I use the CLOCK_REALTIME clock in my app, and if the sysadmin changes the date, hell will break loose in my app. Hence my need for CLOCK_MONOTONIC.
On a related note, I see that do_gettimeofday() in kernel/timer.c actually gets a struct timespec from __get_realtime_clock_ts() then throws bits away in the ns to us conversion. The integer division is also a waste of cycles in my case, as I want the result in ns.
I suppose that if I change sock_get_timestamp() in core/sock.c to call __get_realtime_clock_ts() instead of do_gettimeofday() I'll break 50 billion applications (ping? traceroute?) that expect a struct timeval?
Anyway, thanks for reading this far.
Regards.
Mr Anonymous,
Quoted 4 lines
My app expects a stream of UDP datagrams containing compressed video. These datagrams have been time stamped by the source using a high resolution clock.
Why do you need another time stamp them?
Quoted 7 lines
The video stream is played out in real-time. I buffer several packets, then repeatedly arm a one-shot timer (with
TIMER_ABSTIMEset) to wait until it's time to send the oldest packet.AFAIU, if I use the
CLOCK_REALTIMEclock in my app, and if the sysadmin changes the date, hell will break loose in my app.
Only if your app cannot handle time going backwards.
Quoted 3 lines
I suppose that if I change
sock_get_timestamp()incore/sock.cto call__get_realtime_clock_ts()instead ofdo_gettimeofday()I'll break 50 billion applications (ping? traceroute?) that expect a struct timeval?
Not that many, but some probably.
Letting the kernel do the time stamping is usually quite useless anyways. You can as well do it in your application when you receive it. After all you're interested in when you can read the packet in your app, not when the driver processes it.
The kernel time stamping is mainly for sniffing applications that want to know exactly what's going on on the wire. But even then it's fairly arbitary because it can be a long time between the packet arriving on the PHY and being processed by the driver.
-Andi
Quoted 7 lines · Andi Kleen
Quoted 3 lines · John
My app expects a stream of UDP datagrams containing compressed video. These datagrams have been time stamped by the source using a high resolution clock.
Why do you need another time stamp then?
Source and receiver do not share a common clock. Thus, I have to monitor clock skew. If I simply use the time stamps produced by the source, my buffer will eventually underflow or overflow.
As far as I have seen, clock skew is on the order of ±10^-5, i.e. when the source thinks 100000 seconds have elapsed, the receiver thinks 100001 seconds have elapsed.
Quoted 7 lines
Quoted 5 lines
The video stream is played out in real-time. I buffer several packets, then repeatedly arm a one-shot timer (with
TIMER_ABSTIMEset) to wait until it's time to send the oldest packet.AFAIU, if I use the
CLOCK_REALTIMEclock in my app, and if the sysadmin changes the date, hell will break loose in my app.Only if your app cannot handle time going backwards.
My app cannot handle time going backwards.
The algorithm looks like:
/* Start transmission, record start date */NEXT_DEADLINE = now;
while ( 1 )
{
send(oldest_packet); recv_available_packets();
/* compute next deadline */
NEXT_DEADLINE += time_to_wait_until_next_packet(); sleep_until(NEXT_DEADLINE);
}
If the clock changes under me, I'm toast.
Quoted 3 lines
Quoted 1 line
I suppose that if I change
sock_get_timestamp()incore/sock.cto call__get_realtime_clock_ts()instead ofdo_gettimeofday()I'll break 50 billion applications (ping? traceroute?) that expect a struct timeval?Not that many, but some probably.
Another option would be to change sock_get_timestamp() to call ktime_get_ts() instead of do_gettimeofday() and convert ns to us.
i.e. I still return a struct timeval, but using CLOCK_MONOTONIC.
Quoted 2 lines
Letting the kernel do the time stamping is usually quite useless anyways. You can as well do it in your application when you receive it. After all you're interested in when you can read the packet in your app, not when the driver processes it.
I use clock_nanosleep() to sleep until it's time to send the next packet. I check for packets when I wake up. I need the kernel to time stamp the packets because I was sleeping when they reached the system.
I try to do it this way because I read this:
http://rt.wiki.kernel.org/index.php/High_resolution_timers
"Note that (clock_)nanosleep functions do not suffer from this problem as the wakeup function at timer expiry is executed in the context of the high resolution timer interrupt. If an application is not using asynchronous signal handlers, it is recommended to use the clock_nanosleep() function with the TIMER_ABSTIME flag set instead of waiting for the periodic timer signal delivery. The application has to maintain the absolute expiry time for the next interval itself, but this is a lightweight operation of adding and normalizing two struct timespec values. The benefit is significantly lower maximum latencies (~50us) and less OS overhead in general."
Regards.
Quoted 10 lines · John
Quoted 7 lines · Andi Kleen
Quoted 3 lines · John
My app expects a stream of UDP datagrams containing compressed video. These datagrams have been time stamped by the source using a high resolution clock.
Why do you need another time stamp then?
Quoted 7 lines
Source and receiver do not share a common clock. Thus, I have to monitor clock skew. If I simply use the time stamps produced by the source, my buffer will eventually underflow or overflow.
As far as I have seen, clock skew is on the order of ±10^-5, i.e. when the source thinks 100000 seconds have elapsed, the receiver thinks 100001 seconds have elapsed.
I've considered running NTP to synchronize the clocks, but I'm afraid it wouldn't help, for several reasons:
1) The 40.5 MHz clock used to timestamp datagrams at the source is located on a PCI board. I have read access to a 32-bit counter that is incremented every cycle (somewhat like the TSC on x86).
2) Source and receiver are inside a VPN, and do not have access to the Internet. I don't think I can keep NTP happy if it can't talk to any NTP servers.
(I'm not snipping the rest of the message in case someone is willing to comment.)
Quoted 61 lines
Quoted 7 lines
Quoted 5 lines
The video stream is played out in real-time. I buffer several packets, then repeatedly arm a one-shot timer (with
TIMER_ABSTIMEset) to wait until it's time to send the oldest packet.AFAIU, if I use the
CLOCK_REALTIMEclock in my app, and if the sysadmin changes the date, hell will break loose in my app.Only if your app cannot handle time going backwards.
My app cannot handle time going backwards.
The algorithm looks like:
/* Start transmission, record start date */
NEXT_DEADLINE= now;while ( 1 )
{
send(oldest_packet);recv_available_packets();
/* compute next deadline */
NEXT_DEADLINE+=time_to_wait_until_next_packet();sleep_until(NEXT_DEADLINE);
}If the clock changes under me, I'm toast.
Quoted 6 lines
Quoted 4 lines
I suppose that if I change
sock_get_timestamp()incore/sock.cto call__get_realtime_clock_ts()instead ofdo_gettimeofday()I'll break 50 billion applications (ping? traceroute?) that expect a struct timeval?Not that many, but some probably.
Another option would be to change
sock_get_timestamp()to callktime_get_ts()instead ofdo_gettimeofday()and convert ns to us.i.e. I still return a struct timeval, but using
CLOCK_MONOTONIC.Quoted 4 lines
Letting the kernel do the time stamping is usually quite useless anyways. You can as well do it in your application when you receive it. After all you're interested in when you can read the packet in your app, not when the driver processes it.
I use
clock_nanosleep()to sleep until it's time to send the next packet. I check for packets when I wake up. I need the kernel to time stamp the packets because I was sleeping when they reached the system.I try to do it this way because I read this:
http://rt.wiki.kernel.org/index.php/High_resolution_timers
"Note that (clock_)nanosleep functions do not suffer from this problem as the wakeup function at timer expiry is executed in the context of the high resolution timer interrupt. If an application is not using asynchronous signal handlers, it is recommended to use the
clock_nanosleep()function with theTIMER_ABSTIMEflag set instead of waiting for the periodic timer signal delivery. The application has to maintain the absolute expiry time for the next interval itself, but this is a lightweight operation of adding and normalizing two struct timespec values. The benefit is significantly lower maximum latencies (~50us) and less OS overhead in general."