net: fall back to skb_iif for the timestamping pktinfo if_index
Subsystems (3)
PATCH 1/1 net: fall back to skb_iif for the timestamping pktinfo if_index
put_ts_pktinfo() resolves the interface index of a received packet from its napi id. The lookup fails on drivers whose napi instances are not attached to the delivering net_device (e.g. ti cpsw, which keeps them on an internal dummy device) and on kernels built without CONFIG_NET_RX_BUSY_POLL, where skb_napi_id() is always 0. In those cases SCM_TIMESTAMPING_PKTINFO carries if_index 0 and applications cannot tell which interface produced the hardware timestamp. chronyd, for example, then ignores valid hardware timestamps.
Fall back to skb->skb_iif, which is set for every received packet. On aggregated interfaces this reports the aggregating device instead of the physical one, but only in cases where the napi lookup already failed and nothing was reported at all.
---
A userspace fallback for existing kernels was proposed to chrony separately.
net/socket.c | 2 ++ 1 file changed, 2 insertions(+)
net/socket.c +2 -0
@@ -900,6 +900,8 @@ static void put_ts_pktinfo(struct msghdr *msg, struct sk_buff *skb, if_index = orig_dev->ifindex;rcu_read_unlock (); } +if (!if_index) + if_index = skb->skb_iif; ts_pktinfo.if_index = if_index; ts_pktinfo.pkt_length = skb->len -skb_mac_offset (skb);
Quoted 9 lines · JR Lanteigne
put_ts_pktinfo()resolves the interface index of a received packet from its napi id. The lookup fails on drivers whose napi instances are not attached to the deliveringnet_device(e.g. ti cpsw, which keeps them on an internal dummy device) and on kernels built withoutCONFIG_NET_RX_BUSY_POLL, whereskb_napi_id()is always 0. In those casesSCM_TIMESTAMPING_PKTINFOcarriesif_index0 and applications cannot tell which interface produced the hardware timestamp. chronyd, for example, then ignores valid hardware timestamps.
Quoted 4 lines
Fall back to
skb->skb_iif, which is set for every received packet. On aggregated interfaces this reports the aggregating device instead of the physical one, but only in cases where the napi lookup already failed and nothing was reported at all.
Interestingly, the referenced commit explains that it added this method because the skb_iif reported by IP_PKTINFO is insufficient:
"The index is useful with bonding, bridges and other interfaces, where IP_PKTINFO doesn't allow applications to determine which PHC made the timestamp."
I don't mind adding a fallback. As long as possibly passing an aggregate (or tunnel) device cannot cause regressions to existing users, notably chrony and linuxptp.
Quoted 21 lines
Fixes aad9c8c470f2 ("net: add new control message for incoming HW-timestamped packets")Signed-off-by JR Lanteigne <[email protected]>---
A userspace fallback for existing kernels was proposed to chrony separately.net/socket.c | 2 ++ 1 file changed, 2 insertions(+)net/socket.c +2
-0@@ -900,6 +900,8 @@ static void put_ts_pktinfo(struct msghdr *msg, struct sk_buff *skb, if_index = orig_dev->ifindex;rcu_read_unlock (); } +if (!if_index) + if_index = skb->skb_iif; ts_pktinfo.if_index = if_index; ts_pktinfo.pkt_length = skb->len -skb_mac_offset (skb);
Quoted 3 lines · Willem de Bruijn
I don't mind adding a fallback. As long as possibly passing an aggregate (or tunnel) device cannot cause regressions to existing users, notably chrony and linuxptp.
I checked both.
linuxptp does not use SCM_TIMESTAMPING_PKTINFO at all, and its event sockets are opened per port and bound with SO_BINDTODEVICE, so timestamps are attributed to an interface by socket, not by this cmsg.
chrony matches the cmsg if_index only against interfaces listed in its hwtimestamp directive, with the PHC resolved by its own ETHTOOL_GET_TS_INFO on the configured name. An index it was not configured for (an unconfigured aggregate) fails the lookup exactly like the 0 it gets today, where it silently degrades to the kernel software timestamp. If the user did configure the aggregate, the value is nothing new either: on kernels without this cmsg (pre-4.13) chrony already falls back to the IP_PKTINFO/IPV6_PKTINFO index, which is the same aggregate device skb_iif holds, since __netif_receive_skb_core() resets skb_iif after the bond/bridge rx_handler rewrites skb->dev and inet_iif()/IP6CB(skb)->iif report that device.
The fallback also only fires when the napi lookup already failed, so the bonding configurations the referenced commit was added for, where the napi id resolves to the physical slave, are unchanged.
Quoted 8 lines · JR Lanteigne
put_ts_pktinfo()resolves the interface index of a received packet from its napi id. The lookup fails on drivers whose napi instances are not attached to the deliveringnet_device(e.g. ti cpsw, which keeps them on an internal dummy device) and on kernels built withoutCONFIG_NET_RX_BUSY_POLL, whereskb_napi_id()is always 0. In those casesSCM_TIMESTAMPING_PKTINFOcarriesif_index0 and applications cannot tell which interface produced the hardware timestamp. chronyd, for example, then ignores valid hardware timestamps.
That's the intended behavior, from timestamping.rst:
A valid (non-zero) interface index will be returned only if CONFIG_NET_RX_BUSY_POLL is enabled and the driver is using NAPI.
Quoted 4 lines
Fall back to
skb->skb_iif, which is set for every received packet. On aggregated interfaces this reports the aggregating device instead of the physical one, but only in cases where the napi lookup already failed and nothing was reported at all.
That would cause applications to receive a wrong index in some cases, right? If it's possible, it should report the index of the physical interface, but it should not be guessing.
Quoted 2 lines
A userspace fallback for existing kernels was proposed to chrony separately.
That makes sense to me. It's a bug that it doesn't check for zero value.
Signature · 1 line
Miroslav Lichvar
Quoted 3 lines · Miroslav Lichvar
That would cause applications to receive a wrong index in some cases, right? If it's possible, it should report the index of the physical interface, but it should not be guessing.
Right: with the fallback a nonzero if_index is no longer guaranteed to be the physical timestamping device. An application that maps the index to a PHC without validating it could see the aggregating device where it previously saw 0.
Reporting the physical interface does not look implementable in the two failing cases. Without CONFIG_NET_RX_BUSY_POLL there is no napi id recorded at all. For cpsw the napi belongs to a DMA channel shared by both ports, so there is no per-napi net_device to attach; the delivering port is only known per packet, and there skb->dev is set correctly, so skb_iif is exactly the physical interface. The index only becomes an aggregate with bonding or bridging on top, where the napi lookup would have failed anyway.
So the choice for these packets is between 0 and the skb_iif value, which is what IP_PKTINFO already reports. No objection from me if this is dropped; I can respin if there is a form of the fallback worth keeping.
Quoted 19 lines · JR Lanteigne
Quoted 3 lines · Miroslav Lichvar
That would cause applications to receive a wrong index in some cases, right? If it's possible, it should report the index of the physical interface, but it should not be guessing.
Right: with the fallback a nonzero
if_indexis no longer guaranteed to be the physical timestamping device. An application that maps the index to a PHC without validating it could see the aggregating device where it previously saw 0.Reporting the physical interface does not look implementable in the two failing cases. Without
CONFIG_NET_RX_BUSY_POLLthere is no napi id recorded at all. For cpsw the napi belongs to a DMA channel shared by both ports, so there is no per-napinet_deviceto attach; the delivering port is only known per packet, and thereskb->devis set correctly, soskb_iifis exactly the physical interface. The index only becomes an aggregate with bonding or bridging on top, where the napi lookup would have failed anyway.
Quoted 4 lines
So the choice for these packets is between 0 and the
skb_iifvalue, which is whatIP_PKTINFOalready reports. No objection from me if this is dropped; I can respin if there is a form of the fallback worth keeping.
It may be possible to detect whether an skb_iif points to a physical or aggregate device (e.g., through lower_dev)? If so, then the value can still be used in most cases.
Quoted 3 lines · Willem de Bruijn
It may be possible to detect whether an
skb_iifpoints to a physical or aggregate device (e.g., throughlower_dev)? If so, then the value can still be used in most cases.
That only guesses less often (tunnels register no lower adjacency). I agree with Miroslav: 0 when the physical interface is unknown.
The userspace check fixes chrony only; every other consumer of SCM_TIMESTAMPING_PKTINFO has to rediscover that if_index can be 0 and grow its own IP_PKTINFO fallback. The kernel is the one place a fix reaches all of them.
The gap on cpsw: one napi serves both ports, the driver delivers with netif_receive_skb() so the id is never set, and it would name port 1 for every frame if it were. The driver knows the source port per packet (CPDMA_RX_SOURCE_PORT) and needs a way to hand it to put_ts_pktinfo(), e.g. via skb_shared_hwtstamps.
Quoted 6 lines · JR Lanteigne
Quoted 3 lines · Willem de Bruijn
It may be possible to detect whether an
skb_iifpoints to a physical or aggregate device (e.g., throughlower_dev)? If so, then the value can still be used in most cases.That only guesses less often (tunnels register no lower adjacency).
The lower_dev was only an example. There may be other ways to weed out non applicable devices. Like dev->type (ARPHRD_..).
Though it may be hard to prove that all cases have been identified and skb_iif can be trusted to come from a physical NIC.
Or inverse, iff dev->netdev_ops->ndo_hwtstamp_get is non-NULL. Also not foolproof. vlan_hwtstamp_get is one counterexample.
I don't necessarily have an exact answer. But optionally an avenue to explore.
Quoted 12 lines
I agree with Miroslav: 0 when the physical interface is unknown.
The userspace check fixes chrony only; every other consumer of
SCM_TIMESTAMPING_PKTINFOhas to rediscover thatif_indexcan be 0 and grow its ownIP_PKTINFOfallback. The kernel is the one place a fix reaches all of them.The gap on cpsw: one napi serves both ports, the driver delivers with
netif_receive_skb()so the id is never set, and it would name port 1 for every frame if it were. The driver knows the source port per packet (CPDMA_RX_SOURCE_PORT) and needs a way to hand it toput_ts_pktinfo(), e.g. viaskb_shared_hwtstamps.