Skip to contentSkip to conversation

net: fall back to skb_iif for the timestamping pktinfo if_index

PATCH In review JR Lanteigne 3 participants 8 messages Aug 23 – Aug 24

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 net: fall back to skb_iif for the timestamping pktinfo if_index

put_ts_pktinfo() resolves the interface index of a received packet from its napi id. The lookup fails on drivers whose napi instances are not attached to the delivering net_device (e.g. ti cpsw, which keeps them on an internal dummy device) and on kernels built without CONFIG_NET_RX_BUSY_POLL, where skb_napi_id() is always 0. In those cases SCM_TIMESTAMPING_PKTINFO carries if_index 0 and applications cannot tell which interface produced the hardware timestamp. chronyd, for example, then ignores valid hardware timestamps.

Fall back to skb->skb_iif, which is set for every received packet. On aggregated interfaces this reports the aggregating device instead of the physical one, but only in cases where the napi lookup already failed and nothing was reported at all.

Fixes aad9c8c470f2 ("net: add new control message for incoming HW-timestamped packets")

---
A userspace fallback for existing kernels was proposed to chrony separately.

 net/socket.c | 2 ++
 1 file changed, 2 insertions(+)
net/socket.c +2 -0
@@ -900,6 +900,8 @@ static void put_ts_pktinfo(struct msghdr *msg, struct sk_buff *skb,
 			if_index = orig_dev->ifindex;
 		rcu_read_unlock();
 	}
+	if (!if_index)
+		if_index = skb->skb_iif;
 	ts_pktinfo.if_index = if_index;
 
 	ts_pktinfo.pkt_length = skb->len - skb_mac_offset(skb);
#1 lore raw
Quoted 9 lines · JR Lanteigne

put_ts_pktinfo() resolves the interface index of a received packet from its napi id. The lookup fails on drivers whose napi instances are not attached to the delivering net_device (e.g. ti cpsw, which keeps them on an internal dummy device) and on kernels built without CONFIG_NET_RX_BUSY_POLL, where skb_napi_id() is always 0. In those cases SCM_TIMESTAMPING_PKTINFO carries if_index 0 and applications cannot tell which interface produced the hardware timestamp. chronyd, for example, then ignores valid hardware timestamps.

Quoted 4 lines

Fall back to skb->skb_iif, which is set for every received packet. On aggregated interfaces this reports the aggregating device instead of the physical one, but only in cases where the napi lookup already failed and nothing was reported at all.

Interestingly, the referenced commit explains that it added this method because the skb_iif reported by IP_PKTINFO is insufficient:

"The index is useful with bonding, bridges and other interfaces, where IP_PKTINFO doesn't allow applications to determine which PHC made the timestamp."

I don't mind adding a fallback. As long as possibly passing an aggregate (or tunnel) device cannot cause regressions to existing users, notably chrony and linuxptp.

Quoted 21 lines
Fixes aad9c8c470f2 ("net: add new control message for incoming HW-timestamped packets")

---
A userspace fallback for existing kernels was proposed to chrony separately.

 net/socket.c | 2 ++
 1 file changed, 2 insertions(+)
net/socket.c +2 -0
@@ -900,6 +900,8 @@ static void put_ts_pktinfo(struct msghdr *msg, struct sk_buff *skb,
 			if_index = orig_dev->ifindex;
 		rcu_read_unlock();
 	}
+	if (!if_index)
+		if_index = skb->skb_iif;
 	ts_pktinfo.if_index = if_index;
 
 	ts_pktinfo.pkt_length = skb->len - skb_mac_offset(skb);
#2 lore raw
Quoted 3 lines · Willem de Bruijn

I don't mind adding a fallback. As long as possibly passing an aggregate (or tunnel) device cannot cause regressions to existing users, notably chrony and linuxptp.

I checked both.

linuxptp does not use SCM_TIMESTAMPING_PKTINFO at all, and its event sockets are opened per port and bound with SO_BINDTODEVICE, so timestamps are attributed to an interface by socket, not by this cmsg.

chrony matches the cmsg if_index only against interfaces listed in its hwtimestamp directive, with the PHC resolved by its own ETHTOOL_GET_TS_INFO on the configured name. An index it was not configured for (an unconfigured aggregate) fails the lookup exactly like the 0 it gets today, where it silently degrades to the kernel software timestamp. If the user did configure the aggregate, the value is nothing new either: on kernels without this cmsg (pre-4.13) chrony already falls back to the IP_PKTINFO/IPV6_PKTINFO index, which is the same aggregate device skb_iif holds, since __netif_receive_skb_core() resets skb_iif after the bond/bridge rx_handler rewrites skb->dev and inet_iif()/IP6CB(skb)->iif report that device.

The fallback also only fires when the napi lookup already failed, so the bonding configurations the referenced commit was added for, where the napi id resolves to the physical slave, are unchanged.

#3 lore raw
In reply to JR Lanteigne
Quoted 8 lines · JR Lanteigne

put_ts_pktinfo() resolves the interface index of a received packet from its napi id. The lookup fails on drivers whose napi instances are not attached to the delivering net_device (e.g. ti cpsw, which keeps them on an internal dummy device) and on kernels built without CONFIG_NET_RX_BUSY_POLL, where skb_napi_id() is always 0. In those cases SCM_TIMESTAMPING_PKTINFO carries if_index 0 and applications cannot tell which interface produced the hardware timestamp. chronyd, for example, then ignores valid hardware timestamps.

That's the intended behavior, from timestamping.rst:
A valid (non-zero) interface index will be returned only if CONFIG_NET_RX_BUSY_POLL is enabled and the driver is using NAPI.

Quoted 4 lines

Fall back to skb->skb_iif, which is set for every received packet. On aggregated interfaces this reports the aggregating device instead of the physical one, but only in cases where the napi lookup already failed and nothing was reported at all.

That would cause applications to receive a wrong index in some cases, right? If it's possible, it should report the index of the physical interface, but it should not be guessing.

Quoted 2 lines

A userspace fallback for existing kernels was proposed to chrony separately.

That makes sense to me. It's a bug that it doesn't check for zero value.

Signature · 1 line
Miroslav Lichvar
#4 lore raw
Quoted 3 lines · Miroslav Lichvar

That would cause applications to receive a wrong index in some cases, right? If it's possible, it should report the index of the physical interface, but it should not be guessing.

Right: with the fallback a nonzero if_index is no longer guaranteed to be the physical timestamping device. An application that maps the index to a PHC without validating it could see the aggregating device where it previously saw 0.

Reporting the physical interface does not look implementable in the two failing cases. Without CONFIG_NET_RX_BUSY_POLL there is no napi id recorded at all. For cpsw the napi belongs to a DMA channel shared by both ports, so there is no per-napi net_device to attach; the delivering port is only known per packet, and there skb->dev is set correctly, so skb_iif is exactly the physical interface. The index only becomes an aggregate with bonding or bridging on top, where the napi lookup would have failed anyway.

So the choice for these packets is between 0 and the skb_iif value, which is what IP_PKTINFO already reports. No objection from me if this is dropped; I can respin if there is a form of the fallback worth keeping.

#5 lore raw
Quoted 19 lines · JR Lanteigne
Quoted 3 lines · Miroslav Lichvar

That would cause applications to receive a wrong index in some cases, right? If it's possible, it should report the index of the physical interface, but it should not be guessing.

Right: with the fallback a nonzero if_index is no longer guaranteed to be the physical timestamping device. An application that maps the index to a PHC without validating it could see the aggregating device where it previously saw 0.

Reporting the physical interface does not look implementable in the two failing cases. Without CONFIG_NET_RX_BUSY_POLL there is no napi id recorded at all. For cpsw the napi belongs to a DMA channel shared by both ports, so there is no per-napi net_device to attach; the delivering port is only known per packet, and there skb->dev is set correctly, so skb_iif is exactly the physical interface. The index only becomes an aggregate with bonding or bridging on top, where the napi lookup would have failed anyway.

Quoted 4 lines

So the choice for these packets is between 0 and the skb_iif value, which is what IP_PKTINFO already reports. No objection from me if this is dropped; I can respin if there is a form of the fallback worth keeping.

It may be possible to detect whether an skb_iif points to a physical or aggregate device (e.g., through lower_dev)? If so, then the value can still be used in most cases.

#6 lore raw
Quoted 3 lines · Willem de Bruijn

It may be possible to detect whether an skb_iif points to a physical or aggregate device (e.g., through lower_dev)? If so, then the value can still be used in most cases.

That only guesses less often (tunnels register no lower adjacency). I agree with Miroslav: 0 when the physical interface is unknown.

The userspace check fixes chrony only; every other consumer of SCM_TIMESTAMPING_PKTINFO has to rediscover that if_index can be 0 and grow its own IP_PKTINFO fallback. The kernel is the one place a fix reaches all of them.

The gap on cpsw: one napi serves both ports, the driver delivers with netif_receive_skb() so the id is never set, and it would name port 1 for every frame if it were. The driver knows the source port per packet (CPDMA_RX_SOURCE_PORT) and needs a way to hand it to put_ts_pktinfo(), e.g. via skb_shared_hwtstamps.

#7 lore raw
Quoted 6 lines · JR Lanteigne
Quoted 3 lines · Willem de Bruijn

It may be possible to detect whether an skb_iif points to a physical or aggregate device (e.g., through lower_dev)? If so, then the value can still be used in most cases.

That only guesses less often (tunnels register no lower adjacency).

The lower_dev was only an example. There may be other ways to weed out non applicable devices. Like dev->type (ARPHRD_..).

Though it may be hard to prove that all cases have been identified and skb_iif can be trusted to come from a physical NIC.

Or inverse, iff dev->netdev_ops->ndo_hwtstamp_get is non-NULL. Also not foolproof. vlan_hwtstamp_get is one counterexample.

I don't necessarily have an exact answer. But optionally an avenue to explore.

Quoted 12 lines

I agree with Miroslav: 0 when the physical interface is unknown.

The userspace check fixes chrony only; every other consumer of SCM_TIMESTAMPING_PKTINFO has to rediscover that if_index can be 0 and grow its own IP_PKTINFO fallback. The kernel is the one place a fix reaches all of them.

The gap on cpsw: one napi serves both ports, the driver delivers with netif_receive_skb() so the id is never set, and it would name port 1 for every frame if it were. The driver knows the source port per packet (CPDMA_RX_SOURCE_PORT) and needs a way to hand it to put_ts_pktinfo(), e.g. via skb_shared_hwtstamps.