Skip to contentSkip to conversation

iptables and tcpdump

DISCUSSION Applied Rolf Fokkens 4 participants 8 messages Oct 28, 2001 – Nov 6, 2001

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi!

I've been "tcpdumping" traffic that passes through a NAT box based on netfilter. Everything works wonderful, but tcpdump presents confusing data. With the help of google I found out that tcpdump sees the data right after the NF_IP_PRE_ROUTING and the NF_IP_POST_ROUTING hooks. This explains it all, but results in a new question: why does tcpdump "see" the data after the NF_IP_PRE_ROUTING hook instead of before, which more accurately reflects the data that's on the wire?

I can imagine this has been explained before, but I haven't found the full explanation. Could someone enlighten me?

Another thing is /proc/net/ip_conntrack. It shows also some confusing information like this:

icmp 1 29 src=145.66.17.200 dst=10.13.92.231 ... [UNREPLIED] src=130.130.92.231 dst=145.66.17.200 ...

One half shows an unNATted dst, the second half shows the NATted src. Logically speaking they should match but now they don't.

So everything works fine, but it's presented in a confusing way (tcpdump, ip_conntrack). This may be intentionally but it seems a little accidentally to me.

Rolf

-------------------------------------------------------

#1 lore raw
Quoted 9 lines · On Sun, 28 Oct 2001 17:10:41 -0800 Rolf Fokkens <[email protected]>…

Hi!

I've been "tcpdumping" traffic that passes through a NAT box based on netfilter. Everything works wonderful, but tcpdump presents confusing data. With the help of google I found out that tcpdump sees the data right after the NF_IP_PRE_ROUTING and the NF_IP_POST_ROUTING hooks. This explains it all, but results in a new question: why does tcpdump "see" the data after the NF_IP_PRE_ROUTING hook instead of before, which more accurately reflects the data that's on the wire?

It should see the packets on the wire (they are grabbed by tcpdump before IP processing), but IIRC they are cloned (not copied) for tcpdump's use.

Alexey, should the NAT layer be doing skb_unshare() before altering the packet?

Quoted 5 lines

icmp 1 29 src=145.66.17.200 dst=10.13.92.231 ... [UNREPLIED] src=130.130.92.231 dst=145.66.17.200 ...

One half shows an unNATted dst, the second half shows the NATted src. Logically speaking they should match but now they don't.

No, that's what the connection tracking will actually see. If there is no NAT, they will match.

Hope that clarifies,
Rusty.

#2 lore raw

From: Rusty Russell <[email protected]>
Date: Tue, 30 Oct 2001 15:28:12 +1100

should the NAT layer be doing skb_unshare() before altering the packet?

I think it should.

Look, if you are messing with packets before they go back out, and tcpdump could have sniffed it on the way in, you can't change it's contents blindly.

Franks a lot,
David S. Miller
[email protected]

#3 lore raw
Quoted 6 lines · On Mon, 29 Oct 2001 21:31:57 -0800 (PST) "David S. Miller" <[email protected]>…

From: Rusty Russell <[email protected]>
Date: Tue, 30 Oct 2001 15:28:12 +1100

should the NAT layer be doing skb_unshare() before altering the packet?

I think it should.

Agreed. The 2.2 masq code didn't do this, and hence the "don't tcpdump on masq host" recommendation.

Please try this patch (compiles at least),
Rusty.

net/ipv4/netfilter/ip_fw_compat.c +8 -0
diff -urN -I \$.*\$ --exclude TAGS -X /home/rusty/devel/kernel/kernel-patches/current-dontdiff --minimal linux-2.4.13-official/net/ipv4/netfilter/ip_fw_compat.c working-2.4.13-nfunshare/net/ipv4/netfilter/ip_fw_compat.c
@@ -78,11 +78,19 @@
 {
 	int ret = FW_BLOCK;
 	u_int16_t redirpt;
+	struct sk_buff *nskb;
 
 	/* Assume worse case: any hook could change packet */
 	(*pskb)->nfcache |= NFC_UNKNOWN | NFC_ALTERED;
 	if ((*pskb)->ip_summed == CHECKSUM_HW)
 		(*pskb)->ip_summed = CHECKSUM_NONE;
+
+	/* Firewall rules can alter TOS: raw socket may have clone of
+           skb: don't disturb it --RR */
+	nskb = skb_unshare(*pskb, GFP_ATOMIC);
+	if (!nskb)
+		return NF_DROP;
+	*pskb = nskb;
 
 	switch (hooknum) {
 	case NF_IP_PRE_ROUTING:
net/ipv4/netfilter/ip_nat_core.c +9 -0
diff -urN -I \$.*\$ --exclude TAGS -X /home/rusty/devel/kernel/kernel-patches/current-dontdiff --minimal linux-2.4.13-official/net/ipv4/netfilter/ip_nat_core.c working-2.4.13-nfunshare/net/ipv4/netfilter/ip_nat_core.c
@@ -734,6 +734,15 @@
 	   synchronize_bh()) can vanish. */
 	READ_LOCK(&ip_nat_lock);
 	for (i = 0; i < info->num_manips; i++) {
+		struct sk_buff *nskb;
+		/* raw socket may have clone of skb: don't disturb it --RR */
+		nskb = skb_unshare(*pskb, GFP_ATOMIC);
+		if (!nskb) {
+			READ_UNLOCK(&ip_nat_lock);
+			return NF_DROP;
+		}
+		*pskb = nskb;
+
 		if (info->manips[i].direction == dir
 		    && info->manips[i].hooknum == hooknum) {
 			DEBUGP("Mangling %p: %s to %u.%u.%u.%u %u\n",
net/ipv4/netfilter/ipt_TCPMSS.c +7 -0
diff -urN -I \$.*\$ --exclude TAGS -X /home/rusty/devel/kernel/kernel-patches/current-dontdiff --minimal linux-2.4.13-official/net/ipv4/netfilter/ipt_TCPMSS.c working-2.4.13-nfunshare/net/ipv4/netfilter/ipt_TCPMSS.c
@@ -48,6 +48,13 @@
 	u_int16_t tcplen, newtotlen, oldval, newmss;
 	unsigned int i;
 	u_int8_t *opt;
+	struct sk_buff *nskb;
+
+	/* raw socket may have clone of skb: don't disturb it --RR */
+	nskb = skb_unshare(*pskb, GFP_ATOMIC);
+	if (!nskb)
+		return NF_DROP;
+	*pskb = nskb;
 
 	tcplen = (*pskb)->len - iph->ihl*4;
 
net/ipv4/netfilter/ipt_TOS.c +7 -0
diff -urN -I \$.*\$ --exclude TAGS -X /home/rusty/devel/kernel/kernel-patches/current-dontdiff --minimal linux-2.4.13-official/net/ipv4/netfilter/ipt_TOS.c working-2.4.13-nfunshare/net/ipv4/netfilter/ipt_TOS.c
@@ -19,7 +19,14 @@
 	const struct ipt_tos_target_info *tosinfo = targinfo;
 
 	if ((iph->tos & IPTOS_TOS_MASK) != tosinfo->tos) {
+		struct sk_buff *nskb;
 		u_int16_t diffs[2];
+
+		/* raw socket may have clone of skb: don't disturb it --RR */
+		nskb = skb_unshare(*pskb, GFP_ATOMIC);
+		if (!nskb)
+			return NF_DROP;
+		*pskb = nskb;
 
 		diffs[0] = htons(iph->tos) ^ 0xFFFF;
 		iph->tos = (iph->tos & IPTOS_PREC_MASK) | tosinfo->tos;
#4 lore raw

Hello!

Quoted 2 lines

Agreed. The 2.2 masq code didn't do this, and hence the "don't tcpdump on masq host" recommendation.

Paul, it is very possible that I smoke/drunk something wrong and saw this in dreams, but I really remember that this bug has been fixed in some 2.1.x. :-)

Only function is different: that time skb_unshare() did some unitelligible thing and was used only by AX.25 for an unknown purpose. So, the function which does the work was called skb_cow().

Alexey

#5 lore raw
In reply to Rusty Russell

From: Rusty Russell <[email protected]>
Date: Wed, 31 Oct 2001 17:28:35 +1100

Quoted 6 lines · On Mon, 29 Oct 2001 21:31:57 -0800 (PST) "David S. Miller" <[email protected]>…

From: Rusty Russell <[email protected]>
Date: Tue, 30 Oct 2001 15:28:12 +1100

should the NAT layer be doing skb_unshare() before altering the packet?

I think it should.

Agreed. The 2.2 masq code didn't do this, and hence the "don't tcpdump on masq host" recommendation.

Please try this patch (compiles at least),

Applied to my sources...

Franks a lot,
David S. Miller
[email protected]

#6 lore raw
In reply to Rusty Russell

Hello!

Quoted 1 line

Alexey, should the NAT layer be doing skb_unshare() before altering the packet?

MUST. Cloned skbs are read-only.

I did not expect such question from you. :-)

Alexey

#7 lore raw
In reply to Rolf Fokkens

Hi!

I may have missed something, but I'm not on the maillists which would explain why. And the archives dont contain the email messages (yet) between my initial question and this part of the discussion.

Apparently my question triggered a discussion about some deep NAT details at the skb level. As much as I understand it, something goes wrong with the skb cloning in the NAT layer, NAT changes read-only copies.

Is this the cause of the weird data that shows up with tcpdump?

Or in other words: does tcpdump show something buggy?

Rolf

Quoted 10 lines · you

Hello!

Quoted 2 lines

Alexey, should the NAT layer be doing skb_unshare() before altering the packet?

MUST. Cloned skbs are read-only.

I did not expect such question from you. :-)

Alexey