SO_TIMESTAMPING fix and design decisions
Subsystems (2)
Hi,
I'm currently working on the SO_TIMESTAMPING feature which is currently pretty much broken. The current status is the following:
-tx software timestamps don't work because of a race condition. See commit cd4d8fdad1f1320.
-rx software timestamps do work. But they are nothing new. SO_TIMESTAMP[NS] has been available for years.
hardware timestamps only work for the Intel igb driver. I have access to two test machines with NICs supported by this driver.
-tx hardware timestamps do work. I still have to check what happens when there is high load of packets requesting timestamping.
-rx hardware timestamps work only for special PTP (Precision Time Protocol) packets. There exists a HWTSTAMP_FILTER_ALL option to timestamp all packets, but it doesn't work and it will not work. This is due to a problem in the hardware design. The Intel hardware is tuned for PTP (and so is the ioctl interface).
Right now I'm trying to fix the software tx timestamps. I see several ways to fix it:
-Do skb_get() before calling ops->ndo_start_xmit(). This breaks the wireless stack, because it is incompatible with pskb_expand_head().
-Do skb_clone() and skb_set_owner_w() before calling ops->ndo_start_xmit(). If the driver promises to do timestamping (shtx->in_progress==1), then this clone will be abandoned. -> Software timestamps only as fallback.
-Do skb_clone() and skb_set_owner_w() before calling ops->ndo_start_xmit(). Use this to send the software timestamp regardless of what the driver is doing. This results in software timestamps being always generated. Not only as fallback. The drawback is that userspace will eventually have to parse two timestamping messages (only if requested). This is not a big deal.
I chose the last option since it is easiest to implement without much interaction with the driver. Patch is attached. Any comments or ideas for a better implementation are welcome.
Currently the tx timestamp is returned with the whole packet, including all transport layer headers. I would like to return only the payload, since this would make the interface easier for userspace. There is nothing like a "payload" pointer in the sk_buff. How can I solve this? Add such a pointer?
To fix the hardware rx timestamps my idea is to change the ioctl interface, so that it allows userspace to fine tune the relevant hardware registers of the intel controler. This would allow hardware timestamps to be used in other scenarios than just for PTP. I don't know any application which already uses this interface right now. But if there is one it would be easy to fix.
In case there appear some other controlers with timestamping support they will either need their own custom interface (if they are similarly limited) or they can just go without the ioctl interface and do hardware timestamping of all received packages if they are not so limited.
Could this be a way to go or what would you suggest?
Regards,
Christopher Zimmermann
Quoted 5 lines · Christopher Zimmermann
Hi,
I'm currently working on the
SO_TIMESTAMPINGfeature which is currently pretty much broken. The current status is the following:
Quoted 8 lines
-tx software timestamps don't work because of a race condition. See commit
cd4d8fdad1f1320.
-rx software timestamps do work. But they are nothing new. SO_TIMESTAMP[NS] has been available for years.hardware timestamps only work for the Intel igb driver. I have access to two test machines with NICs supported by this driver.
Intel's 82599, supported by ixgbe, also has the same IEEE 1588 timestamping support in hardware. We haven't implemented the support yet in ixgbe, but the hardware is there and does work. If you were curious of the interface, the datasheet for the hardware is available on our SourceForge site (e1000.sf.net).
Cheers,
-PJ
Quoted 8 lines · On Sat, 19 Sep 2009 15:09:21 -0700 Peter P Waskiewicz Jr…
Quoted 2 lines
hardware timestamps only work for the Intel igb driver. I have access to two test machines with NICs supported by this driver.
Intel's 82599, supported by ixgbe, also has the same IEEE 1588 timestamping support in hardware. We haven't implemented the support yet in ixgbe, but the hardware is there and does work. If you were curious of the interface, the datasheet for the hardware is available on our SourceForge site (e1000.sf.net).
hi! thanks for the reply.
I already got the documentation for the 82576 cards I have access to. I won't be able to afford another pair.
What do you think about my idea to expose the relevant registers to userspace? I believe it would not be too difficult for userspace to configure the timestamps this way and would allow way more flexibility. Of course I would #DEFINE the constants used to set the registers.
Christopher Zimmermann
Quoted 14 lines · Christopher Zimmermann
Quoted 8 lines · On Sat, 19 Sep 2009 15:09:21 -0700 Peter P Waskiewicz Jr…
Quoted 2 lines
hardware timestamps only work for the Intel igb driver. I have access to two test machines with NICs supported by this driver.
Intel's 82599, supported by ixgbe, also has the same IEEE 1588 timestamping support in hardware. We haven't implemented the support yet in ixgbe, but the hardware is there and does work. If you were curious of the interface, the datasheet for the hardware is available on our SourceForge site (e1000.sf.net).
hi! thanks for the reply.
Quoted 7 lines
I already got the documentation for the 82576 cards I have access to. I won't be able to afford another pair.
What do you think about my idea to expose the relevant registers to userspace? I believe it would not be too difficult for userspace to configure the timestamps this way and would allow way more flexibility. Of course I would #DEFINE the constants used to set the registers.
The patch seems reasonable, but I haven't played with the igb timestamping very much. However, what impact will this have on the existing ptpd userspace daemon?
-PJ
Quoted 18 lines · On Sun, 20 Sep 2009 10:48:13 -0700 Peter P Waskiewicz Jr…
Quoted 17 lines · Christopher Zimmermann
Quoted 8 lines · On Sat, 19 Sep 2009 15:09:21 -0700 Peter P Waskiewicz Jr…
Quoted 2 lines
hardware timestamps only work for the Intel igb driver. I have access to two test machines with NICs supported by this driver.
Intel's 82599, supported by ixgbe, also has the same IEEE 1588 timestamping support in hardware. We haven't implemented the support yet in ixgbe, but the hardware is there and does work. If you were curious of the interface, the datasheet for the hardware is available on our SourceForge site (e1000.sf.net).
hi! thanks for the reply.
I already got the documentation for the 82576 cards I have access to. I won't be able to afford another pair.
Quoted 8 lines
Quoted 4 lines
What do you think about my idea to expose the relevant registers to userspace? I believe it would not be too difficult for userspace to configure the timestamps this way and would allow way more flexibility. Of course I would #DEFINE the constants used to set the registers.
The patch seems reasonable, but I haven't played with the igb timestamping very much. However, what impact will this have on the existing ptpd userspace daemon?
It will need to be modified. If you want to avoid this, one could keep the HWTSTAMP_FILTER_PTP_.... defines and just redifine them to reflect the new interface.
Where can I find this ptpd userspace daemon, which supports hardware timestamps using the ioctl interface? ptpd.sourceforge.net doesn't.
Cheers,
Christopher Zimmermann
Quoted 5 lines · Christopher Zimmermann
It will need to be modified. If you want to avoid this, one could keep the
HWTSTAMP_FILTER_PTP_.... defines and just redifine them to reflect the new interface.
Where can I find this ptpd userspace daemon, which supports hardware timestamps using the ioctl interface? ptpd.sourceforge.net doesn't.
You can find a version of the modified ptpd on the e1000 Sourceforge site http://e1000.sf.net. It has a version of the igb driver that also supports this interface though it's a bit behind the regular igb driver version.
I'm review your comments that start this thread and should have some comments soon.
Signature · 2 lines
Cheers, John