netfilter: nf_nat: fix yet another bizarre early demux corner case
Subsystems (3)
Florian Westphal
maintainercover netfilter: nf_nat: fix yet another bizarre early demux corner case
There is another corner case where early demux can end up assigning the wrong socket to skb->sk.
This extends the existing workaround in nf_nat and the test case to demonstrate the problem.
Florian Westphal (2):
netfilter: nf_nat: undo erroneous tcp edemux lookup after port clash
selftests: netfilter: test nat source port clash resolution
interaction with tcp early demux
net/netfilter/nf_nat_proto.c | 63 ++++++++++++++++++- .../selftests/netfilter/nf_nat_edemux.sh | 46 +++++++++++--- 2 files changed, 97 insertions(+), 12 deletions(-)
Signature · 1 line
2.41.0
Florian Westphal
maintainerPATCH 1/2 netfilter: nf_nat: undo erroneous tcp edemux lookup after port clash
In commit 03a3ca37e4c6 ("netfilter: nf_nat: undo erroneous tcp edemux lookup") I fixed a problem with source port clash resolution and DNAT.
A very similar issue exists with REDIRECT (DNAT to local address) and port rewrites.
Consider two port redirections done at prerouting hook:
-p tcp --port 1111 -j REDIRECT --to-ports 80
-p tcp --port 1112 -j REDIRECT --to-ports 80
Its possible, however unlikely, that we get two connections sharing the same source port, i.e.
saddr:12345 -> daddr:1111
saddr:12345 -> daddr:1112
This works on sender side because destination address is different.
After prerouting, nat will change first syn packet to saddr:12345 -> daddr:80, stack will send a syn-ack back and 3whs completes.
The second syn however will result in a source port clash:
after dnat rewrite, new syn has
saddr:12345 -> daddr:80
This collides with the reply direction of the first connection.
The NAT engine will handle this in the input nat hook by also altering the source port, so we get for example
saddr:13535 -> daddr:80
This allows the stack to send back a syn-ack to that address. Reverse NAT during POSTROUTING will rewrite the packet to daddr:1112 -> saddr:12345 again. Tuple will be unique on-wire and peer can process it normally.
Problem is when ACK packet comes in:
After prerouting, packet payload is mangled to saddr:12345 -> daddr:80. Early demux will assign the 3whs-completing ACK skb to the first connections' established socket.
This will then elicit a challenge ack from the first connections' socket rather than complete the connection of the second. The second connection can never complete.
Detect this condition by checking if the associated sockets port matches the conntrack entries reply tuple.
If it doesn't, then input source address translation mangled payload after early demux and the found sk is incorrect.
Discard this sk and let TCP stack do another lookup.
--- net/netfilter/nf_nat_proto.c | 64 ++++++++++++++++++++++++++++++++++-- 1 file changed, 61 insertions(+), 3 deletions(-)
net/netfilter/nf_nat_proto.c +61 -3
@@ -697,6 +697,31 @@ static int nf_xfrm_me_harder(struct net *net, struct sk_buff *skb, unsigned int }#endif +static bool nf_nat_inet_port_was_mangled (const struct sk_buff *skb,__be16 sport) +{ +enum ip_conntrack_info ctinfo; +enum ip_conntrack_dir dir; +const struct nf_conn *ct; + + ct =nf_ct_get (skb, &ctinfo); +if (!ct) +return false; + +switch (nf_ct_protonum (ct)) { +case IPPROTO_TCP : +case IPPROTO_UDP : +break ; +default : +return false; + } + + dir =CTINFO2DIR (ctinfo); +if (dir !=IP_CT_DIR_ORIGINAL ) +return false; + +return ct->tuplehash[!dir].tuple.dst.u.all != sport; +} +static unsigned int nf_nat_ipv4_local_in (void *priv,struct sk_buff *skb,const struct nf_hook_state *state)
@@ -707,8 +732,20 @@ nf_nat_ipv4_local_in(void *priv, struct sk_buff *skb, ret =nf_nat_ipv4_fn (priv, skb, state); -if (ret ==NF_ACCEPT && sk && saddr !=ip_hdr (skb)->saddr && - !inet_sk_transparent (sk)) +if (ret !=NF_ACCEPT || !sk ||inet_sk_transparent (sk)) +return ret; + +/* skb has a socket assigned via tcp edemux. We need to check +* if nf_nat_ipv4_fn() has mangled the packet in a way that +* edemux would not have found this socket. +* +* This includes both changes to the source address and changes +* to the source port, which are both handled by the +* nf_nat_ipv4_fn() call above -- long after tcp/udp early demux +* might have found a socket for the old (pre-snat) address. +*/ +if (saddr !=ip_hdr (skb)->saddr || +nf_nat_inet_port_was_mangled (skb, sk->sk_dport))skb_orphan (skb);/* TCP edemux obtained wrong socket */ return ret;
@@ -937,6 +974,27 @@ nf_nat_ipv6_fn(void *priv, struct sk_buff *skb,return nf_nat_inet_fn (priv, skb, state); } +static unsigned int +nf_nat_ipv6_local_in (void *priv,struct sk_buff *skb, +const struct nf_hook_state *state) +{ +struct in6_addr saddr =ipv6_hdr (skb)->saddr; +struct sock *sk = skb->sk; +unsigned int ret; + + ret =nf_nat_ipv6_fn (priv, skb, state); + +if (ret !=NF_ACCEPT || !sk ||inet_sk_transparent (sk)) +return ret; + +/* see nf_nat_ipv4_local_in */ +if (ipv6_addr_cmp (&saddr, &ipv6_hdr (skb)->saddr) || +nf_nat_inet_port_was_mangled (skb, sk->sk_dport)) +skb_orphan (skb); + +return ret; +} +static unsigned int nf_nat_ipv6_in (void *priv,struct sk_buff *skb,const struct nf_hook_state *state)
@@ -1051,7 +1109,7 @@ static const struct nf_hook_ops nf_nat_ipv6_ops[] = { },/* After packet filtering, change source */ { - .hook = nf_nat_ipv6_fn, + .hook = nf_nat_ipv6_local_in, .pf = NFPROTO_IPV6, .hooknum = NF_INET_LOCAL_IN, .priority = NF_IP6_PRI_NAT_SRC,
Signature · 1 line
2.41.0
Florian Westphal
maintainerPATCH 2/2 selftests: netfilter: test nat source port clash resolution interaction with tcp early demux
In reply to Florian WestphalTest that nat engine resolves the source port clash and tcp packet is passed to the correct socket.
While at it, get rid of the iperf3 dependency, just use socat for listener side too.
--- .../selftests/netfilter/nf_nat_edemux.sh | 46 +++++++++++++++---- 1 file changed, 37 insertions(+), 9 deletions(-)
tools/testing/selftests/netfilter/nf_nat_edemux.sh +37 -9
@@ -11,16 +11,18 @@ ret=0 sfx=$(mktemp -u "XXXXXXXX") ns1="ns1-$sfx" ns2="ns2-$sfx" +socatpid=0 cleanup() { + [ $socatpid -gt 0 ] && kill $socatpid ip netns del $ns1 ip netns del $ns2 } -iperf3 -v > /dev/null 2>&1 +socat -h > /dev/null 2>&1 if [ $? -ne 0 ];then - echo "SKIP: Could not run test without iperf3" + echo "SKIP: Could not run test without socat" exit $ksft_skip fi
@@ -60,8 +62,8 @@ ip netns exec $ns2 ip link set up dev veth2 ip netns exec $ns2 ip addr add 192.168.1.2/24 dev veth2 # Create a server in one namespace -ip netns exec $ns1 iperf3 -s > /dev/null 2>&1 & -iperfs=$! +ip netns exec $ns1 socat -u TCP-LISTEN:5201,fork OPEN:/dev/null,wronly=1 & +socatpid=$! # Restrict source port to just one so we don't have to exhaust # all others.
@@ -83,17 +85,43 @@ sleep 1 # ip daddr:dport will be rewritten to 192.168.1.1 5201 # NAT must reallocate source port 10000 because # 192.168.1.2:10000 -> 192.168.1.1:5201 is already in use -echo test | ip netns exec $ns2 socat -t 3 -u STDIN TCP:10.96.0.1:443 >/dev/null +echo test | ip netns exec $ns2 socat -t 3 -u STDIN TCP:10.96.0.1:443,connect-timeout=3 >/dev/null ret=$? -kill $iperfs - # Check socat can connect to 10.96.0.1:443 (aka 192.168.1.1:5201). if [ $ret -eq 0 ]; then echo "PASS: socat can connect via NAT'd address" else echo "FAIL: socat cannot connect via NAT'd address" - exit 1 fi -exit 0 +# check sport clashres. +ip netns exec $ns1 iptables -t nat -A PREROUTING -p tcp --dport 5202 -j REDIRECT --to-ports 5201 +ip netns exec $ns1 iptables -t nat -A PREROUTING -p tcp --dport 5203 -j REDIRECT --to-ports 5201 + +sleep 5 | ip netns exec $ns2 socat -t 5 -u STDIN TCP:192.168.1.1:5202,connect-timeout=5 >/dev/null & +cpid1=$! +sleep 1 + +# if connect succeeds, client closes instantly due to EOF on stdin. +# if connect hangs, it will time out after 5s. +echo | ip netns exec $ns2 socat -t 3 -u STDIN TCP:192.168.1.1:5203,connect-timeout=5 >/dev/null & +cpid2=$! + +time_then=$(date +%s) +wait $cpid2 +rv=$? +time_now=$(date +%s) + +# Check how much time has elapsed, expectation is for +# 'cpid2' to connect and then exit (and no connect delay). +delta=$((time_now - time_then)) + +if [ $delta -lt 2 -a $rv -eq 0 ]; then + echo "PASS: could connect to service via redirected ports" +else + echo "FAIL: socat cannot connect to service via redirect ($delta seconds elapsed, returned $rv)" + ret=1 +fi + +exit $ret
Signature · 1 line
2.41.0