Skip to contentSkip to conversation

l3mdev icmp error route lookup fixes

RFC PATCH In review Mathieu Desnoyers 3 participants 15 messages Sep 18, 2020 – Sep 23, 2020

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

v2 cover l3mdev icmp error route lookup fixes

Hi,

Here is an updated series of fixes for ipv4 and ipv6 which which ensure the route lookup is performed on the right routing table in VRF configurations when sending TTL expired icmp errors (useful for traceroute).

It includes tests for both ipv4 and ipv6.

These fixes address specifically address the code paths involved in sending TTL expired icmp errors. As detailed in the individual commit messages, those fixes do not address similar issues related to network namespaces and unreachable / fragmentation needed messages, which appear to use different code paths.

Thanks,

Mathieu

Mathieu Desnoyers (2):
ipv4/icmp: l3mdev: Perform icmp error route lookup on source device
routing table (v2)
ipv6/icmp: l3mdev: Perform icmp error route lookup on source device
routing table (v2)

Michael Jeanson (1):
selftests: Add VRF icmp error route lookup test

 net/ipv4/icmp.c                               |  23 +-
 net/ipv6/icmp.c                               |   7 +-
 net/ipv6/ip6_output.c                         |   2 -
 tools/testing/selftests/net/Makefile          |   1 +
 .../selftests/net/vrf_icmp_error_route.sh     | 475 ++++++++++++++++++
 5 files changed, 502 insertions(+), 6 deletions(-)
 create mode 100755 tools/testing/selftests/net/vrf_icmp_error_route.sh
Signature · 1 line
2.17.1
#1 lore raw

RFC PATCH v2 1/3 ipv4/icmp: l3mdev: Perform icmp error route lookup on source device routing table (v2)

As per RFC792, ICMP errors should be sent to the source host.

However, in configurations with Virtual Routing and Forwarding tables, looking up which routing table to use is currently done by using the destination net_device.

commit 9d1a6c4ea43e ("net: icmp_route_lookup should use rt dev to determine L3 domain") changes the interface passed to l3mdev_master_ifindex() and inet_addr_type_dev_table() from skb_in->dev to skb_dst(skb_in)->dev. This effectively uses the destination device rather than the source device for choosing which routing table should be used to lookup where to send the ICMP error.

Therefore, if the source and destination interfaces are within separate VRFs, or one in the global routing table and the other in a VRF, looking up the source host in the destination interface's routing table will fail if the destination interface's routing table contains no route to the source host.

One observable effect of this issue is that traceroute does not work in the following cases:

- Route leaking between global routing table and VRF
- Route leaking between VRFs

Preferably use the source device routing table when sending ICMP error messages. If no source device is set, fall-back on the destination device routing table. Else, use the main routing table (index 0).

[ It has been pointed out that a similar issue may exist with ICMP
errors triggered when forwarding between network namespaces. It would be worthwhile to investigate, but is outside of the scope of this investigation. ]

[ It has also been pointed out that a similar issue exists with
unreachable / fragmentation needed messages, which can be triggered by changing the MTU of eth1 in r1 to 1400 and running:

ip netns exec h1 ping -s 1450 -Mdo -c1 172.16.2.2

Some investigation points to raw_icmp_error() and raw_err() as being involved in this last scenario. The focus of this patch is TTL expired ICMP messages, which go through icmp_route_lookup. Investigation of failure modes related to raw_icmp_error() is beyond this investigation's scope. ]

Fixes 9d1a6c4ea43e ("net: icmp_route_lookup should use rt dev to determine L3 domain")
Cc David Ahern <[email protected]>
Cc David S. Miller <[email protected]>

---
Changes since v1:
- Introduce icmp_get_route_lookup_dev.

---
 net/ipv4/icmp.c | 23 +++++++++++++++++++++--
 1 file changed, 21 insertions(+), 2 deletions(-)
net/ipv4/icmp.c +21 -2
index cf36f955bfe6..9ea66d903c41 100644
@@ -457,6 +457,23 @@ static void icmp_reply(struct icmp_bxm *icmp_param, struct sk_buff *skb)
 	local_bh_enable();
 }
 
+/*
+ * The device used for looking up which routing table to use for sending an ICMP
+ * error is preferably the source whenever it is set, which should ensure the
+ * icmp error can be sent to the source host, else lookup using the routing
+ * table of the destination device, else use the main routing table (index 0).
+ */
+static struct net_device *icmp_get_route_lookup_dev(struct sk_buff *skb)
+{
+	struct net_device *route_lookup_dev = NULL;
+
+	if (skb->dev)
+		route_lookup_dev = skb->dev;
+	else if (skb_dst(skb))
+		route_lookup_dev = skb_dst(skb)->dev;
+	return route_lookup_dev;
+}
+
 static struct rtable *icmp_route_lookup(struct net *net,
 					struct flowi4 *fl4,
 					struct sk_buff *skb_in,
@@ -465,6 +482,7 @@ static struct rtable *icmp_route_lookup(struct net *net,
 					int type, int code,
 					struct icmp_bxm *param)
 {
+	struct net_device *route_lookup_dev;
 	struct rtable *rt, *rt2;
 	struct flowi4 fl4_dec;
 	int err;
@@ -479,7 +497,8 @@ static struct rtable *icmp_route_lookup(struct net *net,
 	fl4->flowi4_proto = IPPROTO_ICMP;
 	fl4->fl4_icmp_type = type;
 	fl4->fl4_icmp_code = code;
-	fl4->flowi4_oif = l3mdev_master_ifindex(skb_dst(skb_in)->dev);
+	route_lookup_dev = icmp_get_route_lookup_dev(skb_in);
+	fl4->flowi4_oif = l3mdev_master_ifindex(route_lookup_dev);
 
 	security_skb_classify_flow(skb_in, flowi4_to_flowi(fl4));
 	rt = ip_route_output_key_hash(net, fl4, skb_in);
@@ -503,7 +522,7 @@ static struct rtable *icmp_route_lookup(struct net *net,
 	if (err)
 		goto relookup_failed;
 
-	if (inet_addr_type_dev_table(net, skb_dst(skb_in)->dev,
+	if (inet_addr_type_dev_table(net, route_lookup_dev,
 				     fl4_dec.saddr) == RTN_LOCAL) {
 		rt2 = __ip_route_output_key(net, &fl4_dec);
 		if (IS_ERR(rt2))
Signature · 1 line
2.17.1
#2 lore raw

RFC PATCH v2 2/3 ipv6/icmp: l3mdev: Perform icmp error route lookup on source device routing table (v2)

In reply to Mathieu Desnoyers

As per RFC4443, the destination address field for ICMPv6 error messages is copied from the source address field of the invoking packet.

In configurations with Virtual Routing and Forwarding tables, looking up which routing table to use for sending ICMPv6 error messages is currently done by using the destination net_device.

If the source and destination interfaces are within separate VRFs, or one in the global routing table and the other in a VRF, looking up the source address of the invoking packet in the destination interface's routing table will fail if the destination interface's routing table contains no route to the invoking packet's source address.

One observable effect of this issue is that traceroute6 does not work in the following cases:

- Route leaking between global routing table and VRF
- Route leaking between VRFs

Use the source device routing table when sending ICMPv6 error messages.

[ In the context of ipv4, it has been pointed out that a similar issue
may exist with ICMP errors triggered when forwarding between network namespaces. It would be worthwhile to investigate whether ipv6 has similar issues, but is outside of the scope of this investigation. ]

[ Testing shows that similar issues exist with ipv6 unreachable /
fragmentation needed messages. However, investigation of this additional failure mode is beyond this investigation's scope. ]

---
Changes since v1:
- Introduce icmp6_get_route_lookup_dev.
- Use skb->dev for routing table lookup, because it is guaranteed to be non-NULL.

---
 net/ipv6/icmp.c       | 7 +++++--
 net/ipv6/ip6_output.c | 2 --
 2 files changed, 5 insertions(+), 4 deletions(-)
net/ipv6/icmp.c +5 -2
index a4e4912ad607..91209a2760aa 100644
@@ -501,8 +501,11 @@ void icmp6_send(struct sk_buff *skb, u8 type, u8 code, __u32 info,
 	if (__ipv6_addr_needs_scope_id(addr_type)) {
 		iif = icmp6_iif(skb);
 	} else {
-		dst = skb_dst(skb);
-		iif = l3mdev_master_ifindex(dst ? dst->dev : skb->dev);
+		/*
+		 * The source device is used for looking up which routing table
+		 * to use for sending an ICMP error.
+		 */
+		iif = l3mdev_master_ifindex(skb->dev);
 	}
 
 	/*
net/ipv6/ip6_output.c +0 -2
index c78e67d7747f..cd623068de53 100644
@@ -468,8 +468,6 @@ int ip6_forward(struct sk_buff *skb)
 	 *	check and decrement ttl
 	 */
 	if (hdr->hop_limit <= 1) {
-		/* Force OUTPUT device used as source address */
-		skb->dev = dst->dev;
 		icmpv6_send(skb, ICMPV6_TIME_EXCEED, ICMPV6_EXC_HOPLIMIT, 0);
 		__IP6_INC_STATS(net, idev, IPSTATS_MIB_INHDRERRORS);
 
Signature · 1 line
2.17.1
#3 lore raw

RFC PATCH v2 3/3 selftests: Add VRF icmp error route lookup test

In reply to Mathieu Desnoyers

From: Michael Jeanson <[email protected]>

The objective is to check that the incoming vrf routing table is selected to send an ICMP error back to the source. We test two scenarios: when the ttl of a packet reaches 1 while it is forwarded between different vrfs and when a packet is bigger than the mtu of the second interface is forwarded between different vrfs.

The first ttl test sends a ping with a ttl of 1 from h1 to h2 and parses the output of the command to check that a ttl expired error is received.

The second ttl test runs traceroute from h1 to h2 and parses the output to check for a hop on r1.

The mtu test sends a ping with a payload of 1450 from h1 to h2, through r1 which has an interface with a mtu of 1400 and parses the output of the command to check that a fragmentation needed error is received.

---
Changes since v1:
- Formating and indentation fixes
- Added '-t' to getopts
- Reworked verbose output of grep'd commands with a new function
- Expanded ip command names
- Added fragmentation tests

---
 tools/testing/selftests/net/Makefile          |   1 +
 .../selftests/net/vrf_icmp_error_route.sh     | 475 ++++++++++++++++++
 2 files changed, 476 insertions(+)
 create mode 100755 tools/testing/selftests/net/vrf_icmp_error_route.sh
tools/testing/selftests/net/Makefile +1 -0
index 9491bbaa0831..a716fbf780b3 100644
@@ -19,6 +19,7 @@ TEST_PROGS += txtimestamp.sh
 TEST_PROGS += vrf-xfrm-tests.sh
 TEST_PROGS += rxtimestamp.sh
 TEST_PROGS += devlink_port_split.py
+TEST_PROGS += vrf_icmp_error_route.sh
 TEST_PROGS_EXTENDED := in_netns.sh
 TEST_GEN_FILES =  socket nettest
 TEST_GEN_FILES += psock_fanout psock_tpacket msg_zerocopy reuseport_addr_any
tools/testing/selftests/net/vrf_icmp_error_route.sh new file +475 -0

475 lines not on this page · raw message

Signature · 1 line
2.17.1
#4 lore raw
In reply to Mathieu Desnoyers
Quoted 7 lines · Mathieu Desnoyers

Hi,

Here is an updated series of fixes for ipv4 and ipv6 which which ensure the route lookup is performed on the right routing table in VRF configurations when sending TTL expired icmp errors (useful for traceroute).

Quoted 8 lines

It includes tests for both ipv4 and ipv6.

These fixes address specifically address the code paths involved in sending TTL expired icmp errors. As detailed in the individual commit messages, those fixes do not address similar issues related to network namespaces and unreachable / fragmentation needed messages, which appear to use different code paths.

New selftests are failing:
TEST: Ping received ICMP frag needed [FAIL]

Both IPv4 and IPv6 versions are failing.

#5 lore raw
Quoted 17 lines · ----- On Sep 21, 2020, at 2:36 PM, David Ahern [email protected] wrote:
Quoted 7 lines · Mathieu Desnoyers

Hi,

Here is an updated series of fixes for ipv4 and ipv6 which which ensure the route lookup is performed on the right routing table in VRF configurations when sending TTL expired icmp errors (useful for traceroute).

Quoted 8 lines

It includes tests for both ipv4 and ipv6.

These fixes address specifically address the code paths involved in sending TTL expired icmp errors. As detailed in the individual commit messages, those fixes do not address similar issues related to network namespaces and unreachable / fragmentation needed messages, which appear to use different code paths.

Quoted 4 lines

New selftests are failing:
TEST: Ping received ICMP frag needed [FAIL]

Both IPv4 and IPv6 versions are failing.

Indeed, this situation is discussed in each patch commit message:

ipv4:

[ It has also been pointed out that a similar issue exists with
unreachable / fragmentation needed messages, which can be triggered by changing the MTU of eth1 in r1 to 1400 and running:

ip netns exec h1 ping -s 1450 -Mdo -c1 172.16.2.2

Some investigation points to raw_icmp_error() and raw_err() as being involved in this last scenario. The focus of this patch is TTL expired ICMP messages, which go through icmp_route_lookup. Investigation of failure modes related to raw_icmp_error() is beyond this investigation's scope. ]

ipv6:

[ Testing shows that similar issues exist with ipv6 unreachable /
fragmentation needed messages. However, investigation of this additional failure mode is beyond this investigation's scope. ]

I do not have the time to investigate further unfortunately, so I thought it best to post what I have.

Note that network namespaces also probably have the same problem, but those are not covered by the test cases.

Thanks,

Mathieu

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#6 lore raw
Quoted 40 lines · Mathieu Desnoyers
Quoted 17 lines · ----- On Sep 21, 2020, at 2:36 PM, David Ahern [email protected] wrote:
Quoted 7 lines · Mathieu Desnoyers

Hi,

Here is an updated series of fixes for ipv4 and ipv6 which which ensure the route lookup is performed on the right routing table in VRF configurations when sending TTL expired icmp errors (useful for traceroute).

Quoted 8 lines

It includes tests for both ipv4 and ipv6.

These fixes address specifically address the code paths involved in sending TTL expired icmp errors. As detailed in the individual commit messages, those fixes do not address similar issues related to network namespaces and unreachable / fragmentation needed messages, which appear to use different code paths.

Quoted 4 lines

New selftests are failing:
TEST: Ping received ICMP frag needed [FAIL]

Both IPv4 and IPv6 versions are failing.

Indeed, this situation is discussed in each patch commit message:

ipv4:

[ It has also been pointed out that a similar issue exists with
unreachable / fragmentation needed messages, which can be triggered by changing the MTU of eth1 in r1 to 1400 and running:

ip netns exec h1 ping -s 1450 -Mdo -c1 172.16.2.2

Some investigation points to raw_icmp_error() and raw_err() as being involved in this last scenario. The focus of this patch is TTL expired ICMP messages, which go through icmp_route_lookup. Investigation of failure modes related to raw_icmp_error() is beyond this investigation's scope. ]

Quoted 9 lines

ipv6:

[ Testing shows that similar issues exist with ipv6 unreachable /
fragmentation needed messages. However, investigation of this additional failure mode is beyond this investigation's scope. ]

I do not have the time to investigate further unfortunately, so I thought it best to post what I have.

the test setup is bad. You have r1 dropping the MTU in VRF red, but not telling VRF red how to send back the ICMP. e.g., for IPv4 add:

    ip -netns r1 ro add vrf red 172.16.1.0/24 dev blue

do the same for v6.

Also, I do not see a reason for r2; I suggest dropping it. What you are testing is icmp crossing VRF with route leaking, so there should not be a need for r2 which leads to asymmetrical routing (172.16.1.0 via r1 and the return via r2).

#7 lore raw
Quoted 54 lines · ----- On Sep 21, 2020, at 3:11 PM, David Ahern [email protected] wrote:
Quoted 40 lines · Mathieu Desnoyers
Quoted 17 lines · ----- On Sep 21, 2020, at 2:36 PM, David Ahern [email protected] wrote:
Quoted 7 lines · Mathieu Desnoyers

Hi,

Here is an updated series of fixes for ipv4 and ipv6 which which ensure the route lookup is performed on the right routing table in VRF configurations when sending TTL expired icmp errors (useful for traceroute).

Quoted 8 lines

It includes tests for both ipv4 and ipv6.

These fixes address specifically address the code paths involved in sending TTL expired icmp errors. As detailed in the individual commit messages, those fixes do not address similar issues related to network namespaces and unreachable / fragmentation needed messages, which appear to use different code paths.

Quoted 4 lines

New selftests are failing:
TEST: Ping received ICMP frag needed [FAIL]

Both IPv4 and IPv6 versions are failing.

Indeed, this situation is discussed in each patch commit message:

ipv4:

[ It has also been pointed out that a similar issue exists with
unreachable / fragmentation needed messages, which can be triggered by changing the MTU of eth1 in r1 to 1400 and running:

ip netns exec h1 ping -s 1450 -Mdo -c1 172.16.2.2

Some investigation points to raw_icmp_error() and raw_err() as being involved in this last scenario. The focus of this patch is TTL expired ICMP messages, which go through icmp_route_lookup. Investigation of failure modes related to raw_icmp_error() is beyond this investigation's scope. ]

Quoted 9 lines

ipv6:

[ Testing shows that similar issues exist with ipv6 unreachable /
fragmentation needed messages. However, investigation of this additional failure mode is beyond this investigation's scope. ]

I do not have the time to investigate further unfortunately, so I thought it best to post what I have.

the test setup is bad. You have r1 dropping the MTU in VRF red, but not telling VRF red how to send back the ICMP. e.g., for IPv4 add:

Quoted 8 lines

ip -netns r1 ro add vrf red 172.16.1.0/24 dev blue

do the same for v6.

Also, I do not see a reason for r2; I suggest dropping it. What you are testing is icmp crossing VRF with route leaking, so there should not be a need for r2 which leads to asymmetrical routing (172.16.1.0 via r1 and the return via r2).

CCing Michael Jeanson, author of the selftests patch.

Thanks for your feedback,

Mathieu

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#8 lore raw
Quoted 56 lines · ----- On 21 Sep, 2020, at 15:33, Mathieu Desnoyers…
Quoted 54 lines · ----- On Sep 21, 2020, at 3:11 PM, David Ahern [email protected] wrote:
Quoted 40 lines · Mathieu Desnoyers
Quoted 17 lines · ----- On Sep 21, 2020, at 2:36 PM, David Ahern [email protected] wrote:
Quoted 7 lines · Mathieu Desnoyers

Hi,

Here is an updated series of fixes for ipv4 and ipv6 which which ensure the route lookup is performed on the right routing table in VRF configurations when sending TTL expired icmp errors (useful for traceroute).

Quoted 8 lines

It includes tests for both ipv4 and ipv6.

These fixes address specifically address the code paths involved in sending TTL expired icmp errors. As detailed in the individual commit messages, those fixes do not address similar issues related to network namespaces and unreachable / fragmentation needed messages, which appear to use different code paths.

Quoted 4 lines

New selftests are failing:
TEST: Ping received ICMP frag needed [FAIL]

Both IPv4 and IPv6 versions are failing.

Indeed, this situation is discussed in each patch commit message:

ipv4:

[ It has also been pointed out that a similar issue exists with
unreachable / fragmentation needed messages, which can be triggered by changing the MTU of eth1 in r1 to 1400 and running:

ip netns exec h1 ping -s 1450 -Mdo -c1 172.16.2.2

Some investigation points to raw_icmp_error() and raw_err() as being involved in this last scenario. The focus of this patch is TTL expired ICMP messages, which go through icmp_route_lookup. Investigation of failure modes related to raw_icmp_error() is beyond this investigation's scope. ]

Quoted 9 lines

ipv6:

[ Testing shows that similar issues exist with ipv6 unreachable /
fragmentation needed messages. However, investigation of this additional failure mode is beyond this investigation's scope. ]

I do not have the time to investigate further unfortunately, so I thought it best to post what I have.

the test setup is bad. You have r1 dropping the MTU in VRF red, but not telling VRF red how to send back the ICMP. e.g., for IPv4 add:

Quoted 8 lines
Quoted 8 lines

ip -netns r1 ro add vrf red 172.16.1.0/24 dev blue

do the same for v6.

Also, I do not see a reason for r2; I suggest dropping it. What you are testing is icmp crossing VRF with route leaking, so there should not be a need for r2 which leads to asymmetrical routing (172.16.1.0 via r1 and the return via r2).

The objective of the test was to replicate a clients environment where packets are crossing from a VRF which has a route back to the source to one which doesn't while reaching a ttl of 0. If the route lookup for the icmp error is done on the interface in the first VRF, it can be routed to the source but not on the interface in the second VRF which is the current behaviour for icmp errors generated while crossing between VRFs.

There may be a better test case that doesn't involve asymmetric routing to test this but it's the only way I found to replicate this.

#9 lore raw
Quoted 20 lines · Michael Jeanson
Quoted 4 lines
Quoted 4 lines

the test setup is bad. You have r1 dropping the MTU in VRF red, but not telling VRF red how to send back the ICMP. e.g., for IPv4 add:

Quoted 8 lines
Quoted 8 lines

ip -netns r1 ro add vrf red 172.16.1.0/24 dev blue

do the same for v6.

Also, I do not see a reason for r2; I suggest dropping it. What you are testing is icmp crossing VRF with route leaking, so there should not be a need for r2 which leads to asymmetrical routing (172.16.1.0 via r1 and the return via r2).

The objective of the test was to replicate a clients environment where packets are crossing from a VRF which has a route back to the source to one which doesn't while reaching a ttl of 0. If the route lookup for the icmp error is done on the interface in the first VRF, it can be routed to the source but not on the interface in the second VRF which is the current behaviour for icmp errors generated while crossing between VRFs.

Quoted 3 lines

There may be a better test case that doesn't involve asymmetric routing to test this but it's the only way I found to replicate this.

It should work without asymmetric routing; adding the return route to the second vrf as I mentioned above fixes the FRAG_NEEDED problem. It should work for TTL as well.

Adding a second pass on the tests with the return through r2 is fine, but add a first pass for the more typical case.

#10 lore raw
Quoted 25 lines · David Ahern
Quoted 20 lines · Michael Jeanson
Quoted 4 lines
Quoted 4 lines

the test setup is bad. You have r1 dropping the MTU in VRF red, but not telling VRF red how to send back the ICMP. e.g., for IPv4 add:

Quoted 8 lines
Quoted 8 lines
    ip -netns r1 ro add vrf red 172.16.1.0/24 dev blue

do the same for v6.

Also, I do not see a reason for r2; I suggest dropping it. What you are testing is icmp crossing VRF with route leaking, so there should not be a need for r2 which leads to asymmetrical routing (172.16.1.0 via r1 and the return via r2).

The objective of the test was to replicate a clients environment where packets are crossing from a VRF which has a route back to the source to one which doesn't while reaching a ttl of 0. If the route lookup for the icmp error is done on the interface in the first VRF, it can be routed to the source but not on the interface in the second VRF which is the current behaviour for icmp errors generated while crossing between VRFs.

Quoted 3 lines

There may be a better test case that doesn't involve asymmetric routing to test this but it's the only way I found to replicate this.

Quoted 6 lines

It should work without asymmetric routing; adding the return route to the second vrf as I mentioned above fixes the FRAG_NEEDED problem. It should work for TTL as well.

Adding a second pass on the tests with the return through r2 is fine, but add a first pass for the more typical case.

Hi,

Before writing new tests I just want to make sure we are trying to fix the same issue. If I add a return route to the red VRF then we don't need this patchset because whether the ICMP error are routed using the
table from the source or destination interface they will reach the source host.

The issue for which this patchset was sent only happens when the destination interface's VRF doesn't have a route back to the source host. I guess we might question if this is actually a bug or not.

So the question really is, when a packet is forwarded between VRFs through route leaking and an icmp error is generated, which table should be used for the route lookup? And does it depend on the type of icmp error? (e.g. TTL=1 happens before forwarding, but fragmentation needed happens after when on the destination interface)

Cheers,

Michael

#11 lore raw
Quoted 9 lines · Michael Jeanson
Quoted 6 lines

It should work without asymmetric routing; adding the return route to the second vrf as I mentioned above fixes the FRAG_NEEDED problem. It should work for TTL as well.

Adding a second pass on the tests with the return through r2 is fine, but add a first pass for the more typical case.

Hi,

Quoted 7 lines

Before writing new tests I just want to make sure we are trying to fix the same issue. If I add a return route to the red VRF then we don't need this patchset because whether the ICMP error are routed using the
table from the source or destination interface they will reach the source host.

The issue for which this patchset was sent only happens when the destination interface's VRF doesn't have a route back to the source host. I guess we might question if this is actually a bug or not.

So the question really is, when a packet is forwarded between VRFs through route leaking and an icmp error is generated, which table should be used for the route lookup? And does it depend on the type of icmp error? (e.g. TTL=1 happens before forwarding, but fragmentation needed happens after when on the destination interface)

As a side note, I don't mind reworking the tests as you requested even if the patchset as a whole ends up not being needed and if you think they are still useful. I just wanted to make sure we understood each other.

Cheers,

Michael

#12 lore raw
Quoted 26 lines · Michael Jeanson
Quoted 19 lines · Michael Jeanson
Quoted 6 lines

It should work without asymmetric routing; adding the return route to the second vrf as I mentioned above fixes the FRAG_NEEDED problem. It should work for TTL as well.

Adding a second pass on the tests with the return through r2 is fine, but add a first pass for the more typical case.

Hi,

Before writing new tests I just want to make sure we are trying to fix the same issue. If I add a return route to the red VRF then we don't need this patchset because whether the ICMP error are routed using the table from the source or destination interface they will reach the source host.

The issue for which this patchset was sent only happens when the destination interface's VRF doesn't have a route back to the source host. I guess we might question if this is actually a bug or not.

Quoted 5 lines

So the question really is, when a packet is forwarded between VRFs through route leaking and an icmp error is generated, which table should be used for the route lookup? And does it depend on the type of icmp error? (e.g. TTL=1 happens before forwarding, but fragmentation needed happens after when on the destination interface)

Quoted 4 lines

As a side note, I don't mind reworking the tests as you requested even if the patchset as a whole ends up not being needed and if you think they are still useful. I just wanted to make sure we understood each other.

if you are leaking from VRF 1 to VRF 2 and you do not configure VRF 2 with how to send to errors back to source - MTU or TTL - then I will argue that is a configuration problem, not a bug.

Now the TTL problem is interesting. You need the FIB lookup to know that the packet is forwarded, and by the time of the ttl check in ip_forward skb->dev points to the ingress VRF and dst points to the egress VRF. So I think the change is warranted.

Let's do this for the tests:
1 pass through all of the tests (TTL and MTU, v4 and v6) with symmetric routing configured and make sure they all pass. ie., keep all of them and make sure all tests pass. No sense losing the tests and the thoughts behind them.

Add a second pass with the asymmetric routing per the customer setup since it motivated the investigation.

Rename the test to something like vrf_route_leaking.sh. It can be expanded with more tests related to route leaking as they come up.

#13 lore raw
Quoted 47 lines · David Ahern
Quoted 26 lines · Michael Jeanson
Quoted 19 lines · Michael Jeanson
Quoted 6 lines

It should work without asymmetric routing; adding the return route to the second vrf as I mentioned above fixes the FRAG_NEEDED problem. It should work for TTL as well.

Adding a second pass on the tests with the return through r2 is fine, but add a first pass for the more typical case.

Hi,

Before writing new tests I just want to make sure we are trying to fix the same issue. If I add a return route to the red VRF then we don't need this patchset because whether the ICMP error are routed using the table from the source or destination interface they will reach the source host.

The issue for which this patchset was sent only happens when the destination interface's VRF doesn't have a route back to the source host. I guess we might question if this is actually a bug or not.

Quoted 5 lines

So the question really is, when a packet is forwarded between VRFs through route leaking and an icmp error is generated, which table should be used for the route lookup? And does it depend on the type of icmp error? (e.g. TTL=1 happens before forwarding, but fragmentation needed happens after when on the destination interface)

Quoted 4 lines

As a side note, I don't mind reworking the tests as you requested even if the patchset as a whole ends up not being needed and if you think they are still useful. I just wanted to make sure we understood each other.

if you are leaking from VRF 1 to VRF 2 and you do not configure VRF 2 with how to send to errors back to source - MTU or TTL - then I will argue that is a configuration problem, not a bug.

Now the TTL problem is interesting. You need the FIB lookup to know that the packet is forwarded, and by the time of the ttl check in ip_forward skb->dev points to the ingress VRF and dst points to the egress VRF. So I think the change is warranted.

Let's do this for the tests:
1 pass through all of the tests (TTL and MTU, v4 and v6) with symmetric routing configured and make sure they all pass. ie., keep all of them and make sure all tests pass. No sense losing the tests and the thoughts behind them.

Quoted 6 lines

Add a second pass with the asymmetric routing per the customer setup since it motivated the investigation.

Rename the test to something like vrf_route_leaking.sh. It can be expanded with more tests related to route leaking as they come up.

Just a final clarification, the asymmetric setup would have no return route in VRF 2 and only test the TTL case since the others would fail?

#14 lore raw
Quoted 3 lines · Michael Jeanson

Just a final clarification, the asymmetric setup would have no return route in VRF 2 and only test the TTL case since the others would fail?

correct. add a statement about it representing a customer setup so it is clear such a config is a 1-off