How much we can trust packet timestamping

Hi all

IMHO The problem is quite complicated because

+ common hardware is not designed for real time:

- sends multiple PDUs within one interrupt, and can be delayed
- Host adapter bus & infraestructure is not designed to garantee latency etc...

+ software is also not designed for realtime

- drivers may timestamp in softirq's
- irqs has no deterministic latency either etc...

So even if do_gettimeofday() has 1/CPUfreq resolution by using TSC register packet timestamping meassurement is biased, how much?

anybody has studied this? are there reports/doc about this topic?

of course, this can be avoided by using specialiced hardware, but I'm not interested on that and I would like to know how much I can trust this timestamps

Any comment would be greatly appreciated

Thanks

Ulisses

Debian GNU/Linux: a dream come true
-----------------------------------------------------------------------------
"Computers are useless. They can only give answers." Pablo Picasso

---> Visita http://www.valux.org/ para saber acerca de la <---
---> Asociación Valenciana de Usuarios de Linux <---

The packet can be timestamped by the hardware receiving as well as by the kernel netif_rx code. This is actually intentional and there is hardware that supports doing IRQ raise time sampling which the driver can then use to get very accurate data.

Alan

Thanks Alan

Anybody know about a Linux driver that supports doing IRQ raise time sampling? any doc/pointer/suggestion would be greatly appreciated

Thanks in advance

Ulisses
Debian GNU/Linux: a dream come true
-----------------------------------------------------------------------------
"Computers are useless. They can only give answers." Pablo Picasso

---> Visita http://www.valux.org/ para saber acerca de la <---
---> Asociación Valenciana de Usuarios de Linux <---

Caederus has engineers who did the ISA bus hardware that led to the SIOCGSTAMP facility in the first place (then the tcpdump type folks decided it was really rather cool so it went generic anyway)

www.caederus.co.uk

I don't know if anyone is doing this in the PCI bus world

By the way, the group of Darryl Veitch have done some extremely interesting work with high-resolution timestamps, in particular using the TSC on recent ia32:

http://www.cubinlab.ee.mu.oz.au/probing/
http://www.cubinlab.ee.mu.oz.au/~darryl/tscclock_final.pdf.gz

One general issue in this area is what we can do with time sources that aren't system-wide, e.g. NIC-local timers. The problem is to calibrate them and to synchronize them to wall-clock time. I think there are basically two possible approaches:

1) driver gives time synchronization system (in user-space) access to "raw" running timer value. Timestamps are also "raw" timer values, plus a time source ID, which can then be used to convert the values to wall-clock time.

2) user space pushes exact time to kernel space, which then does all the math. Timestamps are already converted to wall-clock time.

2) is essentially what we can do with today's interfaces (an event notifier would be useful, though). The big drawback is that non-trivial math would have to be done in kernel space.
1) is much easier on the kernel, but has the issue of requiring some API to get time values and time source characteristics (time representation, range, etc.).

I'm leaning towards solution 1), because it keeps things simple for the kernel. But perhaps the best approach is to simply implement both, and then compare ...

- Werner

  _________________________________________________________________________
 / Werner Almesberger, Buenos Aires, Argentina         [email protected] /
/_http://www.almesberger.net/____________________________________________/

You run NTP between the host clock and the nic timer. Its all you really need. In the i2it hardware the NIC clock implemented hardware slewing so it could do NTP stuff

While NTP is a good synchronization source (frequently the only affordable one around), I'm not so sure it's such a good tool for correcting drift. If you have a look at figure 5 in
http://www.cubinlab.ee.mu.oz.au/~darryl/tscclock_final.pdf.gz
you'll see that NTP uses drift to correct for offset errors, so using NTP directly doesn't yield a clock that remains stable unless it's constantly getting corrected by NTP.

What should work better is to use NTP only as a reference for offset, and then calibrate the hardware clock from that. Particularly the TSC is very stable, so there isn't much drift to worry about.

But what I'm after is the interface between kernel and user space, and any kernel-internal interfaces that may be needed. If people really want to use NTP directly on hardware clocks, I guess my approach 1) (export everything to user space, and let user space worry about the details) would then be the appropriately flexible choice ?

- Werner

  _________________________________________________________________________
 / Werner Almesberger, Buenos Aires, Argentina         [email protected] /
/_http://www.almesberger.net/____________________________________________/