nfqueue: nf_conntrack_confirm race condition
Subsystems (2)
RFC PATCH 1/1 nfqueue: nf_conntrack_confirm race condition
Hi,
glibc 2.9 implement parallel IPv4/IPv6 DNS lookup. This caused lots of trouble
in all kind of implementations, so all major Linux distributions removed _nss_dns_gethostbyname4_r in their glibc version, except for Debian Squeeze, see also "options single-request" for more information.
Normally parallel DNS lookups works fine, first packet is received and forwarded, so conntrack is confirmed before second packet is received.
However in combination with NFQUEUE, the second DNS requests is received while the first one is still in the queue and both DNS requests have an unconfirmed conntrack. So the second one will be dropped in nf_conntrack_confirm, which results in an DNS timeout and retransmit.
Can be reproduced with: adnshost yahoo.com google.com
My first idea was to re-lookup the conntrack in nf_conntrack_confirm, but at that time the seconds request was already NATed. So I moved that code to nf_nat_fn(). Of course this only works if nat is loaded...
Any comments or ideas, how to address this problem?
Cheers
Ulrich
Signature · 3 lines
Ulrich Weber | [email protected] | Software Engineer Astaro GmbH & Co. KG | www.astaro.com | Phone +49-721-25516-0 | Fax –200 An der RaumFabrik 33a | 76227 Karlsruhe | Germany
Quoted 1 line
From
2f23d860cd34a006cf4a118536340d8cfcc0d8faMon Sep 17 00:00:00 2001
From: Ulrich Weber <[email protected]>
Date: Tue, 9 Nov 2010 16:03:08 +0100
Subject: [PATCH] nat: re-lookup unconfirmed UDP conntracks
glibc 2.9 implemented DNS IPv4-IPv6 parallel lookup, where two DNS requests will be send at once with the same tuple information.
Due nfqueue the second request will have another conntrack because the first is not confirmed yet and the second request will be dropped by conntrack_confirm afterwards.
To avoid an DNS timeout, re-lookup the conntrack
in nf_nat_fn and attach the real one...
--- net/ipv4/netfilter/nf_nat_standalone.c | 11 +++++++++++ 1 files changed, 11 insertions(+), 0 deletions(-)
net/ipv4/netfilter/nf_nat_standalone.c +11 -0
@@ -124,6 +124,17 @@ nf_nat_fn(unsigned int hooknum, }/* Fall thru... (Only ICMPs can be IP_CT_IS_REPLY) */ case IP_CT_NEW : +/* Nasty asynchronous DNS hack: Avoid NAT and conntrack_confirm race */ +if (!nf_ct_is_confirmed (ct) &&CTINFO2DIR (ctinfo) ==IP_CT_DIR_ORIGINAL && + ct->tuplehash[IP_CT_DIR_ORIGINAL ].tuple.dst.protonum ==IPPROTO_UDP ) { +struct nf_conntrack_tuple_hash *h =nf_conntrack_find_get (nf_ct_net (ct), + &ct->tuplehash[IP_CT_DIR_ORIGINAL ].tuple); +if (h) { + ct =nf_ct_tuplehash_to_ctrack (h); +nf_conntrack_put (skb->nfct); + skb->nfct = &ct->ct_general; + } + }/* Seen it before? This can happen for loopback, retrans, or local packets.. */
Signature · 1 line
1.7.1
Quoted 16 lines · Ulrich Weber
Hi,
glibc 2.9 implement parallel IPv4/IPv6 DNS lookup. This caused lots of trouble
in all kind of implementations, so all major Linux distributions removed_nss_dns_gethostbyname4_rin their glibc version, except for Debian Squeeze, see also "options single-request" for more information.Normally parallel DNS lookups works fine, first packet is received and forwarded, so conntrack is confirmed before second packet is received.
However in combination with NFQUEUE, the second DNS requests is received while the first one is still in the queue and both DNS requests have an unconfirmed conntrack. So the second one will be dropped in
nf_conntrack_confirm, which results in an DNS timeout and retransmit.
Quoted 7 lines
Can be reproduced with: adnshost yahoo.com google.com
My first idea was to re-lookup the conntrack in
nf_conntrack_confirm, but at that time the seconds request was already NATed. So I moved that code tonf_nat_fn(). Of course this only works if nat is loaded...Any comments or ideas, how to address this problem?
It seems that you queue packets in the middle of conntrack. Beside NFQUEUE, IMQ may causes the same problem. I think you'd better queue packets before conntrack, raw table? Or lookup the conntrack again after packets are reinjected.
Signature · 2 lines
Regards, Changli Gao([email protected])
Yes you are right, queuing could be done in raw table PRE_ROUTING and LOCAL_OUT. However I dont like the idea of queuing packets which are going to be dropped anyway, could trigger an DOS attack.
We could add two additional raw hooks for LOCAL_IN and POST_ROUTING, after conntrack is confirmed and queue packets then...
Conntrack lookup after reinjection is racy too with multiple queues.
Cheers
Ulrich
Quoted 30 lines · Changli Gao
Quoted 16 lines · Ulrich Weber
Hi,
glibc 2.9 implement parallel IPv4/IPv6 DNS lookup. This caused lots of trouble
in all kind of implementations, so all major Linux distributions removed_nss_dns_gethostbyname4_rin their glibc version, except for Debian Squeeze, see also "options single-request" for more information.Normally parallel DNS lookups works fine, first packet is received and forwarded, so conntrack is confirmed before second packet is received.
However in combination with NFQUEUE, the second DNS requests is received while the first one is still in the queue and both DNS requests have an unconfirmed conntrack. So the second one will be dropped in
nf_conntrack_confirm, which results in an DNS timeout and retransmit.Quoted 7 lines
Can be reproduced with: adnshost yahoo.com google.com
My first idea was to re-lookup the conntrack in
nf_conntrack_confirm, but at that time the seconds request was already NATed. So I moved that code tonf_nat_fn(). Of course this only works if nat is loaded...Any comments or ideas, how to address this problem?
It seems that you queue packets in the middle of conntrack. Beside NFQUEUE, IMQ may causes the same problem. I think you'd better queue packets before conntrack, raw table? Or lookup the conntrack again after packets are reinjected.
Signature · 7 lines
Ulrich Weber | [email protected] | Software Engineer Astaro GmbH & Co. KG | www.astaro.com | Phone +49-721-25516-0 | Fax –200 An der RaumFabrik 33a | 76227 Karlsruhe | Germany -- To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html