Skip to contentSkip to conversation

l3mdev icmp error route lookup fixes

RFC PATCH In review Mathieu Desnoyers 2 participants 10 messages Sep 25, 2020 – Oct 12, 2020

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover l3mdev icmp error route lookup fixes

Hi,

Here is an updated series of fixes for ipv4 and ipv6 which which ensure the route lookup is performed on the right routing table in VRF configurations when sending TTL expired icmp errors (useful for traceroute).

It includes tests for both ipv4 and ipv6.

These fixes address specifically address the code paths involved in sending TTL expired icmp errors. As detailed in the individual commit messages, those fixes do not address similar icmp errors related to network namespaces and unreachable / fragmentation needed messages, which appear to use different code paths.

The main changes since the last round are updates to the selftests.

Thanks,

Mathieu

Mathieu Desnoyers (2):
ipv4/icmp: l3mdev: Perform icmp error route lookup on source device
routing table (v2)
ipv6/icmp: l3mdev: Perform icmp error route lookup on source device
routing table (v2)

Michael Jeanson (1):
selftests: Add VRF route leaking tests

 net/ipv4/icmp.c                               |  23 +-
 net/ipv6/icmp.c                               |   7 +-
 net/ipv6/ip6_output.c                         |   2 -
 tools/testing/selftests/net/Makefile          |   1 +
 .../selftests/net/vrf_route_leaking.sh        | 626 ++++++++++++++++++
 5 files changed, 653 insertions(+), 6 deletions(-)
 create mode 100755 tools/testing/selftests/net/vrf_route_leaking.sh
Signature · 1 line
2.17.1
#1 lore raw

RFC PATCH 1/3 ipv4/icmp: l3mdev: Perform icmp error route lookup on source device routing table (v2)

As per RFC792, ICMP errors should be sent to the source host.

However, in configurations with Virtual Routing and Forwarding tables, looking up which routing table to use is currently done by using the destination net_device.

commit 9d1a6c4ea43e ("net: icmp_route_lookup should use rt dev to determine L3 domain") changes the interface passed to l3mdev_master_ifindex() and inet_addr_type_dev_table() from skb_in->dev to skb_dst(skb_in)->dev. This effectively uses the destination device rather than the source device for choosing which routing table should be used to lookup where to send the ICMP error.

Therefore, if the source and destination interfaces are within separate VRFs, or one in the global routing table and the other in a VRF, looking up the source host in the destination interface's routing table will fail if the destination interface's routing table contains no route to the source host.

One observable effect of this issue is that traceroute does not work in the following cases:

- Route leaking between global routing table and VRF
- Route leaking between VRFs

Preferably use the source device routing table when sending ICMP error messages. If no source device is set, fall-back on the destination device routing table. Else, use the main routing table (index 0).

[ It has been pointed out that a similar issue may exist with ICMP
errors triggered when forwarding between network namespaces. It would be worthwhile to investigate, but is outside of the scope of this investigation. ]

[ It has also been pointed out that a similar issue exists with
unreachable / fragmentation needed messages, which can be triggered by changing the MTU of eth1 in r1 to 1400 and running:

ip netns exec h1 ping -s 1450 -Mdo -c1 172.16.2.2

Some investigation points to raw_icmp_error() and raw_err() as being involved in this last scenario. The focus of this patch is TTL expired ICMP messages, which go through icmp_route_lookup. Investigation of failure modes related to raw_icmp_error() is beyond this investigation's scope. ]

Fixes 9d1a6c4ea43e ("net: icmp_route_lookup should use rt dev to determine L3 domain")
Cc David Ahern <[email protected]>
Cc David S. Miller <[email protected]>

---
Changes since v1:
- Introduce icmp_get_route_lookup_dev.

---
 net/ipv4/icmp.c | 23 +++++++++++++++++++++--
 1 file changed, 21 insertions(+), 2 deletions(-)
net/ipv4/icmp.c +21 -2
index cf36f955bfe6..9ea66d903c41 100644
@@ -457,6 +457,23 @@ static void icmp_reply(struct icmp_bxm *icmp_param, struct sk_buff *skb)
 	local_bh_enable();
 }
 
+/*
+ * The device used for looking up which routing table to use for sending an ICMP
+ * error is preferably the source whenever it is set, which should ensure the
+ * icmp error can be sent to the source host, else lookup using the routing
+ * table of the destination device, else use the main routing table (index 0).
+ */
+static struct net_device *icmp_get_route_lookup_dev(struct sk_buff *skb)
+{
+	struct net_device *route_lookup_dev = NULL;
+
+	if (skb->dev)
+		route_lookup_dev = skb->dev;
+	else if (skb_dst(skb))
+		route_lookup_dev = skb_dst(skb)->dev;
+	return route_lookup_dev;
+}
+
 static struct rtable *icmp_route_lookup(struct net *net,
 					struct flowi4 *fl4,
 					struct sk_buff *skb_in,
@@ -465,6 +482,7 @@ static struct rtable *icmp_route_lookup(struct net *net,
 					int type, int code,
 					struct icmp_bxm *param)
 {
+	struct net_device *route_lookup_dev;
 	struct rtable *rt, *rt2;
 	struct flowi4 fl4_dec;
 	int err;
@@ -479,7 +497,8 @@ static struct rtable *icmp_route_lookup(struct net *net,
 	fl4->flowi4_proto = IPPROTO_ICMP;
 	fl4->fl4_icmp_type = type;
 	fl4->fl4_icmp_code = code;
-	fl4->flowi4_oif = l3mdev_master_ifindex(skb_dst(skb_in)->dev);
+	route_lookup_dev = icmp_get_route_lookup_dev(skb_in);
+	fl4->flowi4_oif = l3mdev_master_ifindex(route_lookup_dev);
 
 	security_skb_classify_flow(skb_in, flowi4_to_flowi(fl4));
 	rt = ip_route_output_key_hash(net, fl4, skb_in);
@@ -503,7 +522,7 @@ static struct rtable *icmp_route_lookup(struct net *net,
 	if (err)
 		goto relookup_failed;
 
-	if (inet_addr_type_dev_table(net, skb_dst(skb_in)->dev,
+	if (inet_addr_type_dev_table(net, route_lookup_dev,
 				     fl4_dec.saddr) == RTN_LOCAL) {
 		rt2 = __ip_route_output_key(net, &fl4_dec);
 		if (IS_ERR(rt2))
Signature · 1 line
2.17.1
#2 lore raw

RFC PATCH 2/3 ipv6/icmp: l3mdev: Perform icmp error route lookup on source device routing table (v2)

In reply to Mathieu Desnoyers

As per RFC4443, the destination address field for ICMPv6 error messages is copied from the source address field of the invoking packet.

In configurations with Virtual Routing and Forwarding tables, looking up which routing table to use for sending ICMPv6 error messages is currently done by using the destination net_device.

If the source and destination interfaces are within separate VRFs, or one in the global routing table and the other in a VRF, looking up the source address of the invoking packet in the destination interface's routing table will fail if the destination interface's routing table contains no route to the invoking packet's source address.

One observable effect of this issue is that traceroute6 does not work in the following cases:

- Route leaking between global routing table and VRF
- Route leaking between VRFs

Use the source device routing table when sending ICMPv6 error messages.

[ In the context of ipv4, it has been pointed out that a similar issue
may exist with ICMP errors triggered when forwarding between network namespaces. It would be worthwhile to investigate whether ipv6 has similar issues, but is outside of the scope of this investigation. ]

[ Testing shows that similar issues exist with ipv6 unreachable /
fragmentation needed messages. However, investigation of this additional failure mode is beyond this investigation's scope. ]

---
Changes since v1:
- Introduce icmp6_get_route_lookup_dev.
- Use skb->dev for routing table lookup, because it is guaranteed to be non-NULL.

---
 net/ipv6/icmp.c       | 7 +++++--
 net/ipv6/ip6_output.c | 2 --
 2 files changed, 5 insertions(+), 4 deletions(-)
net/ipv6/icmp.c +5 -2
index a4e4912ad607..91209a2760aa 100644
@@ -501,8 +501,11 @@ void icmp6_send(struct sk_buff *skb, u8 type, u8 code, __u32 info,
 	if (__ipv6_addr_needs_scope_id(addr_type)) {
 		iif = icmp6_iif(skb);
 	} else {
-		dst = skb_dst(skb);
-		iif = l3mdev_master_ifindex(dst ? dst->dev : skb->dev);
+		/*
+		 * The source device is used for looking up which routing table
+		 * to use for sending an ICMP error.
+		 */
+		iif = l3mdev_master_ifindex(skb->dev);
 	}
 
 	/*
net/ipv6/ip6_output.c +0 -2
index c78e67d7747f..cd623068de53 100644
@@ -468,8 +468,6 @@ int ip6_forward(struct sk_buff *skb)
 	 *	check and decrement ttl
 	 */
 	if (hdr->hop_limit <= 1) {
-		/* Force OUTPUT device used as source address */
-		skb->dev = dst->dev;
 		icmpv6_send(skb, ICMPV6_TIME_EXCEED, ICMPV6_EXC_HOPLIMIT, 0);
 		__IP6_INC_STATS(net, idev, IPSTATS_MIB_INHDRERRORS);
 
Signature · 1 line
2.17.1
#3 lore raw

RFC PATCH 3/3 selftests: Add VRF route leaking tests

In reply to Mathieu Desnoyers

From: Michael Jeanson <[email protected]>

The objective of the tests is to check that ICMP errors generated while crossing between VRFs are properly routed back to the source host.

The first ttl test sends a ping with a ttl of 1 from h1 to h2 and parses the output of the command to check that a ttl expired error is received.

The second ttl test runs traceroute from h1 to h2 and parses the output to check for a hop on r1.

The mtu test sends a ping with a payload of 1450 from h1 to h2, through r1 which has an interface with a mtu of 1400 and parses the output of the command to check that a fragmentation needed error is received.

[ The IPv6 MTU test still fails with the symmetric routing setup and
I'm not sure why. ]

---
Changes since v2:
- Renamed test script to vrf_route_leaking.sh
- Added a default symmetric routing topology
- Documented the asymmetric routing topology
Changes since v1:
- Formating and indentation fixes
- Added '-t' to getopts
- Reworked verbose output of grep'd commands with a new function
- Expanded ip command names
- Added fragmentation tests

---
 tools/testing/selftests/net/Makefile          |   1 +
 .../selftests/net/vrf_route_leaking.sh        | 626 ++++++++++++++++++
 2 files changed, 627 insertions(+)
 create mode 100755 tools/testing/selftests/net/vrf_route_leaking.sh
tools/testing/selftests/net/Makefile +1 -0
index 9491bbaa0831..3e7fb1e70c77 100644
@@ -19,6 +19,7 @@ TEST_PROGS += txtimestamp.sh
 TEST_PROGS += vrf-xfrm-tests.sh
 TEST_PROGS += rxtimestamp.sh
 TEST_PROGS += devlink_port_split.py
+TEST_PROGS += vrf_route_leaking.sh
 TEST_PROGS_EXTENDED := in_netns.sh
 TEST_GEN_FILES =  socket nettest
 TEST_GEN_FILES += psock_fanout psock_tpacket msg_zerocopy reuseport_addr_any
tools/testing/selftests/net/vrf_route_leaking.sh new file +626 -0

626 lines not on this page · raw message

Signature · 1 line
2.17.1
#4 lore raw
In reply to Mathieu Desnoyers
Quoted 9 lines · Mathieu Desnoyers

Hi,

Here is an updated series of fixes for ipv4 and ipv6 which which ensure the route lookup is performed on the right routing table in VRF configurations when sending TTL expired icmp errors (useful for traceroute).

It includes tests for both ipv4 and ipv6.

Quoted 8 lines

These fixes address specifically address the code paths involved in sending TTL expired icmp errors. As detailed in the individual commit messages, those fixes do not address similar icmp errors related to network namespaces and unreachable / fragmentation needed messages, which appear to use different code paths.

The main changes since the last round are updates to the selftests.

This looks fine to me. I noticed the IPv6 large packet test case is failing; the fib6 tracepoint is showing the loopback as the iif which is wrong:

ping6 8488 [004] 502.015817: fib6:fib6_table_lookup: table 255 oif 0
iif 1 proto 58 ::/0 -> 2001:db8:16:1::1/0 tos 0 scope 0 flags 0 ==> dev lo gw :: err -113

I will dig into it later this week.

#5 lore raw
Quoted 23 lines · David Ahern
Quoted 9 lines · Mathieu Desnoyers

Hi,

Here is an updated series of fixes for ipv4 and ipv6 which which ensure the route lookup is performed on the right routing table in VRF configurations when sending TTL expired icmp errors (useful for traceroute).

It includes tests for both ipv4 and ipv6.

Quoted 8 lines

These fixes address specifically address the code paths involved in sending TTL expired icmp errors. As detailed in the individual commit messages, those fixes do not address similar icmp errors related to network namespaces and unreachable / fragmentation needed messages, which appear to use different code paths.

The main changes since the last round are updates to the selftests.

This looks fine to me. I noticed the IPv6 large packet test case is failing; the fib6 tracepoint is showing the loopback as the iif which is wrong:

Quoted 6 lines

ping6 8488 [004] 502.015817: fib6:fib6_table_lookup: table 255 oif 0
iif 1 proto 58 ::/0 -> 2001:db8:16:1::1/0 tos 0 scope 0 flags 0 ==> dev lo gw :: err -113

I will dig into it later this week.

I see the problem here -- source address selection is picking ::1. I do not have a solution to the problem yet, but its resolution is independent of the change in this set so I think this one is good to go.

#6 lore raw
Quoted 28 lines · ----- On Oct 11, 2020, at 7:56 PM, David Ahern [email protected] wrote:
Quoted 27 lines · David Ahern
Quoted 9 lines · Mathieu Desnoyers

Hi,

Here is an updated series of fixes for ipv4 and ipv6 which which ensure the route lookup is performed on the right routing table in VRF configurations when sending TTL expired icmp errors (useful for traceroute).

It includes tests for both ipv4 and ipv6.

Quoted 8 lines

These fixes address specifically address the code paths involved in sending TTL expired icmp errors. As detailed in the individual commit messages, those fixes do not address similar icmp errors related to network namespaces and unreachable / fragmentation needed messages, which appear to use different code paths.

The main changes since the last round are updates to the selftests.

This looks fine to me. I noticed the IPv6 large packet test case is failing; the fib6 tracepoint is showing the loopback as the iif which is wrong:

ping6 8488 [004] 502.015817: fib6:fib6_table_lookup: table 255 oif 0
iif 1 proto 58 ::/0 -> 2001:db8:16:1::1/0 tos 0 scope 0 flags 0 ==> dev lo gw :: err -113

Quoted 6 lines
Quoted 2 lines

I will dig into it later this week.

I see the problem here -- source address selection is picking ::1. I do not have a solution to the problem yet, but its resolution is independent of the change in this set so I think this one is good to go.

OK, do you want to pick up the RFC patch series, or should I re-send it without RFC tag ?

Thanks,

Mathieu

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#7 lore raw
Quoted 2 lines · Mathieu Desnoyers

OK, do you want to pick up the RFC patch series, or should I re-send it without RFC tag ?

you need to re-send for Dave or Jakub to pick them up via patchworks

#8 lore raw
Quoted 5 lines · ----- On Oct 12, 2020, at 9:45 AM, David Ahern [email protected] wrote:
Quoted 2 lines · Mathieu Desnoyers

OK, do you want to pick up the RFC patch series, or should I re-send it without RFC tag ?

you need to re-send for Dave or Jakub to pick them up via patchworks

OK. Can I have your Acked-by or Reviewed-by for all three patches ?

Thanks,

Mathieu

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
Reviewed-by
#9 lore raw
Quoted 10 lines · Mathieu Desnoyers
Quoted 5 lines · ----- On Oct 12, 2020, at 9:45 AM, David Ahern [email protected] wrote:
Quoted 2 lines · Mathieu Desnoyers

OK, do you want to pick up the RFC patch series, or should I re-send it without RFC tag ?

you need to re-send for Dave or Jakub to pick them up via patchworks

OK. Can I have your Acked-by or Reviewed-by for all three patches ?

sure.