Skip to contentSkip to conversation

nfqueue: nf_conntrack_confirm race condition

RFC PATCH In review Ulrich Weber 2 participants 3 messages Nov 19, 2010 – Nov 22, 2010

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 nfqueue: nf_conntrack_confirm race condition

Hi,

glibc 2.9 implement parallel IPv4/IPv6 DNS lookup. This caused lots of trouble
in all kind of implementations, so all major Linux distributions removed _nss_dns_gethostbyname4_r in their glibc version, except for Debian Squeeze, see also "options single-request" for more information.

Normally parallel DNS lookups works fine, first packet is received and forwarded, so conntrack is confirmed before second packet is received.

However in combination with NFQUEUE, the second DNS requests is received while the first one is still in the queue and both DNS requests have an unconfirmed conntrack. So the second one will be dropped in nf_conntrack_confirm, which results in an DNS timeout and retransmit.

Can be reproduced with: adnshost yahoo.com google.com

My first idea was to re-lookup the conntrack in nf_conntrack_confirm, but at that time the seconds request was already NATed. So I moved that code to nf_nat_fn(). Of course this only works if nat is loaded...

Any comments or ideas, how to address this problem?

Cheers
Ulrich

Signature · 3 lines
Ulrich Weber | [email protected] | Software Engineer
Astaro GmbH & Co. KG | www.astaro.com | Phone +49-721-25516-0 | Fax –200
An der RaumFabrik 33a | 76227 Karlsruhe | Germany
Quoted 1 line

From 2f23d860cd34a006cf4a118536340d8cfcc0d8fa Mon Sep 17 00:00:00 2001

From: Ulrich Weber <[email protected]>
Date: Tue, 9 Nov 2010 16:03:08 +0100
Subject: [PATCH] nat: re-lookup unconfirmed UDP conntracks

glibc 2.9 implemented DNS IPv4-IPv6 parallel lookup, where two DNS requests will be send at once with the same tuple information.

Due nfqueue the second request will have another conntrack because the first is not confirmed yet and the second request will be dropped by conntrack_confirm afterwards.

To avoid an DNS timeout, re-lookup the conntrack
in nf_nat_fn and attach the real one...

---
 net/ipv4/netfilter/nf_nat_standalone.c |   11 +++++++++++
 1 files changed, 11 insertions(+), 0 deletions(-)
net/ipv4/netfilter/nf_nat_standalone.c +11 -0
index 5f41d01..06ef2c5 100644
@@ -124,6 +124,17 @@ nf_nat_fn(unsigned int hooknum,
 		}
 		/* Fall thru... (Only ICMPs can be IP_CT_IS_REPLY) */
 	case IP_CT_NEW:
+		/* Nasty asynchronous DNS hack: Avoid NAT and conntrack_confirm race */
+		if (!nf_ct_is_confirmed(ct) && CTINFO2DIR(ctinfo) == IP_CT_DIR_ORIGINAL &&
+		    ct->tuplehash[IP_CT_DIR_ORIGINAL].tuple.dst.protonum == IPPROTO_UDP) {
+			struct nf_conntrack_tuple_hash *h = nf_conntrack_find_get(nf_ct_net(ct),
+					&ct->tuplehash[IP_CT_DIR_ORIGINAL].tuple);
+			if (h) {
+				ct = nf_ct_tuplehash_to_ctrack(h);
+				nf_conntrack_put(skb->nfct);
+				skb->nfct = &ct->ct_general;
+			}
+		}
 
 		/* Seen it before?  This can happen for loopback, retrans,
 		   or local packets.. */
Signature · 1 line
1.7.1
#1 lore raw
Quoted 16 lines · Ulrich Weber

Hi,

glibc 2.9 implement parallel IPv4/IPv6 DNS lookup. This caused lots of trouble
in all kind of implementations, so all major Linux distributions removed _nss_dns_gethostbyname4_r in their glibc version, except for Debian Squeeze, see also "options single-request" for more information.

Normally parallel DNS lookups works fine, first packet is received and forwarded, so conntrack is confirmed before second packet is received.

However in combination with NFQUEUE, the second DNS requests is received while the first one is still in the queue and both DNS requests have an unconfirmed conntrack. So the second one will be dropped in nf_conntrack_confirm, which results in an DNS timeout and retransmit.

Quoted 7 lines

Can be reproduced with: adnshost yahoo.com google.com

My first idea was to re-lookup the conntrack in nf_conntrack_confirm, but at that time the seconds request was already NATed. So I moved that code to nf_nat_fn(). Of course this only works if nat is loaded...

Any comments or ideas, how to address this problem?

It seems that you queue packets in the middle of conntrack. Beside NFQUEUE, IMQ may causes the same problem. I think you'd better queue packets before conntrack, raw table? Or lookup the conntrack again after packets are reinjected.

Signature · 2 lines
Regards,
Changli Gao([email protected])
#2 lore raw

Yes you are right, queuing could be done in raw table PRE_ROUTING and LOCAL_OUT. However I dont like the idea of queuing packets which are going to be dropped anyway, could trigger an DOS attack.

We could add two additional raw hooks for LOCAL_IN and POST_ROUTING, after conntrack is confirmed and queue packets then...

Conntrack lookup after reinjection is racy too with multiple queues.

Cheers
Ulrich

Quoted 30 lines · Changli Gao
Quoted 16 lines · Ulrich Weber

Hi,

glibc 2.9 implement parallel IPv4/IPv6 DNS lookup. This caused lots of trouble
in all kind of implementations, so all major Linux distributions removed _nss_dns_gethostbyname4_r in their glibc version, except for Debian Squeeze, see also "options single-request" for more information.

Normally parallel DNS lookups works fine, first packet is received and forwarded, so conntrack is confirmed before second packet is received.

However in combination with NFQUEUE, the second DNS requests is received while the first one is still in the queue and both DNS requests have an unconfirmed conntrack. So the second one will be dropped in nf_conntrack_confirm, which results in an DNS timeout and retransmit.

Quoted 7 lines

Can be reproduced with: adnshost yahoo.com google.com

My first idea was to re-lookup the conntrack in nf_conntrack_confirm, but at that time the seconds request was already NATed. So I moved that code to nf_nat_fn(). Of course this only works if nat is loaded...

Any comments or ideas, how to address this problem?

It seems that you queue packets in the middle of conntrack. Beside NFQUEUE, IMQ may causes the same problem. I think you'd better queue packets before conntrack, raw table? Or lookup the conntrack again after packets are reinjected.

Signature · 7 lines
Ulrich Weber | [email protected] | Software Engineer
Astaro GmbH & Co. KG | www.astaro.com | Phone +49-721-25516-0 | Fax –200
An der RaumFabrik 33a | 76227 Karlsruhe | Germany
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html