Skip to contentSkip to conversation

iptables in 2.4.10, 2.4.11pre6 problems

DISCUSSION Trever L. Adams 8 participants 19 messages Oct 9, 2001 – Oct 28, 2001

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

I am seeing messages such as:

Oct  9 12:52:51 smeagol kernel: Firewall:IN=ppp0 OUT= MAC=

SRC=64.152.2.36 DST=MY_IP_ADDRESS LEN=52 TOS=0x00 PREC=0x00 TTL=246 ID=1093 DF PROTO=TCP SPT=80 DPT=33157 WINDOW=34752 RES=0x00 ACK FIN URGP=0

In my firewall logs. I see them for ACK RST as well. These are valid connections. My rules follow for the most part (a few allowed connections to the machine in question have been removed from the list). This often leaves open connections in a half closed state on machines behind this firewall. It also some times kills totally open connections and I see packets rejected that should be allowed through.

Please, help me help. Also, please CC me. The account I am subscribed to the list with is acting up and I may be killing it.

Chain INPUT (policy DROP 3 packets, 999 bytes)
pkts bytes target prot opt in out source
destination
30 2880 DROP udp -- any any anywhere
anywhere udp spt:netbios-ns
33 7607 DROP udp -- any any anywhere
anywhere udp spt:netbios-dgm
37 3680 ICMP_INPUT icmp -- any any anywhere
anywhere
16543 5783K tcprules all -- any any anywhere
anywhere
0 0 firewall all -- any any anywhere
anywhere

Chain FORWARD (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source
destination
1485 280K tcprules all -- any any anywhere
anywhere
0 0 firewall all -- any any anywhere
anywhere

Chain OUTPUT (policy ACCEPT 17692 packets, 6181610 bytes)
pkts bytes target prot opt in out source
destination

Chain ICMP_INPUT (1 references)
pkts bytes target prot opt in out source
destination
26 2184 ACCEPT icmp -- !ppp+ any anywhere
anywhere icmp echo-request
0 0 ACCEPT icmp -- any any anywhere
anywhere icmp echo-reply
11 1496 ACCEPT icmp -- any any anywhere
anywhere icmp destination-unreachable
0 0 LOGACCEPT icmp -- any any anywhere
anywhere icmp source-quench
0 0 LOGACCEPT icmp -- any any anywhere
anywhere icmp time-exceeded
0 0 LOGACCEPT icmp -- any any anywhere
anywhere icmp parameter-problem
0 0 LOGACCEPT icmp -- any any anywhere
anywhere icmp timestamp-request
0 0 LOGACCEPT icmp -- any any anywhere
anywhere icmp timestamp-reply
0 0 firewall icmp -- any any anywhere
anywhere

Chain LOGACCEPT (5 references)
pkts bytes target prot opt in out source
destination
0 0 LOG all -- any any anywhere
anywhere LOG level info prefix `PACKET ACCEPTED:'
0 0 ACCEPT all -- any any anywhere
anywhere

Chain M_DEBUG (0 references)
pkts bytes target prot opt in out source
destination

Chain firewall (4 references)
pkts bytes target prot opt in out source
destination
116 18258 LOG all -- any any anywhere
anywhere LOG level info prefix `Firewall:'
116 18258 DROP all -- any any anywhere
anywhere

Chain tcprules (2 references)
pkts bytes target prot opt in out source
destination
17325 6007K ACCEPT all -- any any anywhere
anywhere state RELATED,ESTABLISHED
587 37774 ACCEPT all -- !ppp+ any anywhere
anywhere state NEW
116 18258 firewall all -- ppp+ any anywhere
anywhere state INVALID,NEW

_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com

#1 lore raw
Quoted 7 lines · Trever L. Adams

I am seeing messages such as:

Oct  9 12:52:51 smeagol kernel: Firewall:IN=ppp0 OUT= MAC=

SRC=64.152.2.36 DST=MY_IP_ADDRESS LEN=52 TOS=0x00 PREC=0x00 TTL=246 ID=1093 DF PROTO=TCP SPT=80 DPT=33157 WINDOW=34752 RES=0x00 ACK FIN URGP=0

Quoted 6 lines

In my firewall logs. I see them for ACK RST as well. These are valid connections. My rules follow for the most part (a few allowed connections to the machine in question have been removed from the list). This often leaves open connections in a half closed state on machines behind this firewall. It also some times kills totally open connections and I see packets rejected that should be allowed through.

I see this too. iptables is refusing packets on locally-initiated TCP connections when the RELATED,ESTABLISHED rule should be letting them through.

I mentioned this problem on the netfilter list but my message fell into a black hole and was apparently beyond the horizon of the developers.

-jwb

#2 lore raw
Quoted 8 lines · Jeffrey W. Baker

I see this too. iptables is refusing packets on locally-initiated TCP connections when the RELATED,ESTABLISHED rule should be letting them through.

I mentioned this problem on the netfilter list but my message fell into a black hole and was apparently beyond the horizon of the developers.

-jwb

Maybe I misunderstand you, define locally-initiated. Do you mean net or do you mean box? Mine happens on connections made by the firewall (proxy for web) and on other connections initiated internally. We currently only allow identd and a few others from external (identd is spoofed more or less).

I am glad I am not the only one seeing this problem. I have the established, related as well. The only thing that should be dropped before it gets handled is certain ICMP messages (if someone thinks I am dropping something I shouldn't, let me know) and windows networking datagrams (used to keep the line up from internal machines, so I killed it).

The big problems I have are a few websites and digitalme.com (all services... web, mail, etc.).

Trever Adams

_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com

#14 lore raw
In reply to Trever L. Adams

----- Original Message -----
From: "Trever L. Adams" <[email protected]>
To: "Linux Kernel Mailing List" <[email protected]>

Quoted 7 lines

I am seeing messages such as:

Oct  9 12:52:51 smeagol kernel: Firewall:IN=ppp0 OUT= MAC=

SRC=64.152.2.36 DST=MY_IP_ADDRESS LEN=52 TOS=0x00 PREC=0x00 TTL=246 ID=1093 DF PROTO=TCP SPT=80 DPT=33157 WINDOW=34752 RES=0x00 ACK FIN URGP=0

Quoted 6 lines

In my firewall logs. I see them for ACK RST as well. These are valid connections. My rules follow for the most part (a few allowed connections to the machine in question have been removed from the list). This often leaves open connections in a half closed state on machines behind this firewall. It also some times kills totally open connections and I see packets rejected that should be allowed through.

This is someone on the outside scanning you for Back Orifice. I don't understand why this is supposed to be a valid connection.

Quoted 1 line

From your iptables output, it looks like your rules are working properly.

(They aren't set up exactly how I would have done it, but it's all a matter of style.)
Can you give a more specific example of something that it being DROPped when it shouldn't be?

--Wilson.

#3 lore raw
In reply to Trever L. Adams
Quoted 8 lines · Trever L. Adams
Quoted 7 lines · Jeffrey W. Baker

I see this too. iptables is refusing packets on locally-initiated TCP connections when the RELATED,ESTABLISHED rule should be letting them through.

I mentioned this problem on the netfilter list but my message fell into a black hole and was apparently beyond the horizon of the developers.

Quoted 7 lines
Quoted 1 line

-jwb

Maybe I misunderstand you, define locally-initiated. Do you mean net or do you mean box? Mine happens on connections made by the firewall (proxy for web) and on other connections initiated internally. We currently only allow identd and a few others from external (identd is spoofed more or less).

I mean connections originating from userland processes running on the machine with iptables configured. This machine does not act as a NAT or router for any other machine.

We make about 200000 outbound connections to web sites in a day. Of these connections, a few thousand get fucked up by iptables (iptables suddenly decides to drop packets on this connection).

-jwb

#4 lore raw
Quoted 9 lines · Jeffrey W. Baker

I mean connections originating from userland processes running on the machine with iptables configured. This machine does not act as a NAT or router for any other machine.

We make about 200000 outbound connections to web sites in a day. Of these connections, a few thousand get fucked up by iptables (iptables suddenly decides to drop packets on this connection).

-jwb

Mine does NAT. So it appears this is not NAT related (though it may be further aggravated by NAT). My connection rate is FAR lower than yours. Our total connections may be 100,000 on a high rate day (just a guess... I really do not know).

Interesting.

Trever Adams

_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com

#15 lore raw
In reply to Wilson
Quoted 10 lines · Wilson

This is someone on the outside scanning you for Back Orifice. I don't understand why this is supposed to be a valid connection.

Quoted 1 line

From your iptables output, it looks like your rules are working properly.

(They aren't set up exactly how I would have done it, but it's all a matter of style.)
Can you give a more specific example of something that it being DROPped when it shouldn't be?

--Wilson.

Sure thing. I suppose I should have chosen a connection better from the logs. These were valid connections to digitalme.com on pop3 (I think or is that smtp...). Happens with web connections to there and other places as well.

Oct  9 13:27:05 smeagol kernel: Firewall:IN=ppp0 OUT= MAC=
SRC=193.97.97.75 DST=209.237.196.198 LEN=40 TOS=0x00 PREC=0x00 TTL=116
ID=6369 DF PROTO=TCP SPT=110 DPT=2213 WINDOW=0 RES=0x00 ACK RST URGP=0 
Oct  9 13:31:44 smeagol kernel: Firewall:IN=ppp0 OUT= MAC=
SRC=193.97.97.75 DST=209.237.196.198 LEN=40 TOS=0x00 PREC=0x00 TTL=116
ID=13344 DF PROTO=TCP SPT=110 DPT=2220 WINDOW=0 RES=0x00 ACK RST URGP=0 
Oct  9 13:39:03 smeagol kernel: Firewall:IN=ppp0 OUT= MAC=
SRC=193.97.97.75 DST=209.237.196.198 LEN=40 TOS=0x00 PREC=0x00 TTL=116
ID=26891 DF PROTO=TCP SPT=110 DPT=2251 WINDOW=0 RES=0x00 ACK RST URGP=0 
Oct  9 13:47:16 smeagol kernel: Firewall:IN=ppp0 OUT= MAC=

SRC=193.97.97.75 DST=209.237.196.198 LEN=40 TOS=0x00 PREC=0x00 TTL=116 ID=41385 DF PROTO=TCP SPT=110 DPT=2254 WINDOW=0 RES=0x00 ACK RST URGP=0

_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com

#5 lore raw
In reply to Trever L. Adams
Quoted 9 lines · Trever L. Adams
Quoted 8 lines · Jeffrey W. Baker

I mean connections originating from userland processes running on the machine with iptables configured. This machine does not act as a NAT or router for any other machine.

We make about 200000 outbound connections to web sites in a day. Of these connections, a few thousand get fucked up by iptables (iptables suddenly decides to drop packets on this connection).

Quoted 6 lines
Quoted 1 line

-jwb

Mine does NAT. So it appears this is not NAT related (though it may be further aggravated by NAT). My connection rate is FAR lower than yours. Our total connections may be 100,000 on a high rate day (just a guess... I really do not know).

My machine has three IP addrs bound to one physical interface and uses policy routing. Do you use any of those?

-jwb

#6 lore raw
Quoted 9 lines · Jeffrey W. Baker
Quoted 4 lines

Mine does NAT. So it appears this is not NAT related (though it may be further aggravated by NAT). My connection rate is FAR lower than yours. Our total connections may be 100,000 on a high rate day (just a guess... I really do not know).

My machine has three IP addrs bound to one physical interface and uses policy routing. Do you use any of those?

-jwb

Two IP addresses. One to the internal ethernet, one to the external (dynamic ip) ppp. I do not believe I am doing policy routing. I have the static route for ethernet and ppp sets up its own.

Trever

_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com

#7 lore raw
In reply to Jeffrey W. Baker

stupid question...
have you got a rule like

iptables -A INPUT -m unclean -j DROP

enabled?

Quoted 34 lines · Jeffrey W. Baker
Quoted 8 lines · Trever L. Adams
Quoted 7 lines · Jeffrey W. Baker

I see this too. iptables is refusing packets on locally-initiated TCP connections when the RELATED,ESTABLISHED rule should be letting them through.

I mentioned this problem on the netfilter list but my message fell into a black hole and was apparently beyond the horizon of the developers.

Quoted 7 lines
Quoted 1 line

-jwb

Maybe I misunderstand you, define locally-initiated. Do you mean net or do you mean box? Mine happens on connections made by the firewall (proxy for web) and on other connections initiated internally. We currently only allow identd and a few others from external (identd is spoofed more or less).

I mean connections originating from userland processes running on the machine with iptables configured. This machine does not act as a NAT or router for any other machine.

We make about 200000 outbound connections to web sites in a day. Of these connections, a few thousand get fucked up by iptables (iptables suddenly decides to drop packets on this connection).

(quoted diff) +0 -2
-jwb
 
-

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#8 lore raw
In reply to Jeffrey W. Baker
Quoted 6 lines · Luigi Genoni

stupid question...
have you got a rule like

iptables -A INPUT -m unclean -j DROP

enabled?

I do not know what unclean means, so I don't know.

I have one that only accepts NEW from the inside. I have one that accepts all ESTABLISHED,RELATED.
I have one that drops NEW,INVALID from ppp0 (outside world).

Trever

_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com

#9 lore raw

and what is the content of
/proc/net/ip_conntrack
file?

how big is the buffer you are using for packet tracing? (/proc/sys/net/ipv4/ip_conntrack_max)

Quoted 21 lines · Trever L. Adams
Quoted 6 lines · Luigi Genoni

stupid question...
have you got a rule like

iptables -A INPUT -m unclean -j DROP

enabled?

I do not know what unclean means, so I don't know.

I have one that only accepts NEW from the inside. I have one that accepts all ESTABLISHED,RELATED.
I have one that drops NEW,INVALID from ppp0 (outside world).

Trever

_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com

#10 lore raw
Quoted 6 lines · Luigi Genoni

and what is the content of
/proc/net/ip_conntrack
file?

how big is the buffer you are using for packet tracing? (/proc/sys/net/ipv4/ip_conntrack_max)

Well, I'm not going to send that file to the Internet at large, but ip_conntrack current has about 2100 lines and the max is 65535.

-jwb

#11 lore raw

So the buffer is big enought for your traffic, I suppose.

I saw something similar to your report, but indeed the buffer was too small.

A good thing would be to see a packet dropped and the entries in /proc/net/ip_conntrack, so that it will be possible to infer something.

with 2.4.10 i see packet tracing working very well also under eavy network loads if the buffer is big enought...

and you are not using the controvers unclean module...

Quoted 16 lines · Jeffrey W. Baker
Quoted 6 lines · Luigi Genoni

and what is the content of
/proc/net/ip_conntrack
file?

how big is the buffer you are using for packet tracing? (/proc/sys/net/ipv4/ip_conntrack_max)

Well, I'm not going to send that file to the Internet at large, but ip_conntrack current has about 2100 lines and the max is 65535.

-jwb

#12 lore raw
In reply to Jeffrey W. Baker
Quoted 16 lines · On Tue, 9 Oct 2001 10:07:05 -0700 (PDT) "Jeffrey W. Baker" <[email protected]>…
Quoted 7 lines · Trever L. Adams

I am seeing messages such as:

Oct  9 12:52:51 smeagol kernel: Firewall:IN=ppp0 OUT= MAC=

SRC=64.152.2.36 DST=MY_IP_ADDRESS LEN=52 TOS=0x00 PREC=0x00 TTL=246 ID=1093 DF PROTO=TCP SPT=80 DPT=33157 WINDOW=34752 RES=0x00 ACK FIN URGP=0

Quoted 6 lines

In my firewall logs. I see them for ACK RST as well. These are valid connections. My rules follow for the most part (a few allowed connections to the machine in question have been removed from the list). This often leaves open connections in a half closed state on machines behind this firewall. It also some times kills totally open connections and I see packets rejected that should be allowed through.

Quoted 3 lines

I see this too. iptables is refusing packets on locally-initiated TCP connections when the RELATED,ESTABLISHED rule should be letting them through.

Yes, but it has forgotten them. Play with the TCP timeout numbers in net/ipv4/netfilter/ip_conntrack_proto_tcp.c. Especially the 60 seconds for TCP_CONNTRACK_CLOSE_WAIT for the ACK FIN case. These numbers were stolen from the 2.0 and 2.2 masq code, which had real world testing (but didn't report failures, so...)

Given some actual feedback on appropriate numbers, this can be fed as a patch to Linus...

Hope that helps,
Rusty.

#13 lore raw
In reply to Jeffrey W. Baker

Trever L. Adams([email protected])@2001.10.09 13:30:14 +0000:

Quoted 2 lines

The big problems I have are a few websites and digitalme.com (all services... web, mail, etc.).

Whats the content of /proc/sys/net/ipv4/tcp_ecn ?

/Benno

Signature · 5 lines
Sebastian Benoit <[email protected]>
OpenPGP-Key ID 0x82AE75E4                             
fingerprint 0BDA 0CB7 9BCA AF77 28EE  D91A 396D 93BC 82AE 75E

The trouble with political jokes is that they get elected.
#16 lore raw
In reply to Trever L. Adams

I have been running 2.4.10-ac11 for 7 days now with TCP_CONNTRACK_CLOSE_WAIT set to 120 seconds - this has stopped nearly all firewall activity on established connections.

I use "mc" extensively for ftp, and previously would get numerous blocks after closing a session or starting another ftp session with the same instance of "mc".

What would we need to do to make these values tuneable either via insmod/modprobe or /proc?

Darrell

Quoted 33 lines · Rusty Russell
Quoted 15 lines · On Tue, 9 Oct 2001 10:07:05 -0700 (PDT) "Jeffrey W. Baker" <[email protected]>…
Quoted 6 lines · Trever L. Adams

I am seeing messages such as:

Oct  9 12:52:51 smeagol kernel: Firewall:IN=ppp0 OUT= MAC=

SRC=64.152.2.36 DST=MY_IP_ADDRESS LEN=52 TOS=0x00 PREC=0x00 TTL=246 ID=1093 DF PROTO=TCP SPT=80 DPT=33157 WINDOW=34752 RES=0x00 ACK FIN URGP=0

Quoted 6 lines

In my firewall logs. I see them for ACK RST as well. These are valid connections. My rules follow for the most part (a few allowed connections to the machine in question have been removed from the list). This often leaves open connections in a half closed state on machines behind this firewall. It also some times kills totally open connections and I see packets rejected that should be allowed through.

Quoted 3 lines

I see this too. iptables is refusing packets on locally-initiated TCP connections when the RELATED,ESTABLISHED rule should be letting them through.

Yes, but it has forgotten them. Play with the TCP timeout numbers in net/ipv4/netfilter/ip_conntrack_proto_tcp.c. Especially the 60 seconds for TCP_CONNTRACK_CLOSE_WAIT for the ACK FIN case. These numbers were stolen from the 2.0 and 2.2 masq code, which had real world testing (but didn't report failures, so...)

Given some actual feedback on appropriate numbers, this can be fed as a patch to Linus...

Hope that helps,
Rusty.

#17 lore raw
Quoted 3 lines · On Fri, 19 Oct 2001 06:18:30 -0700 Darrell A Escola <[email protected]>…

I have been running 2.4.10-ac11 for 7 days now with TCP_CONNTRACK_CLOSE_WAIT set to 120 seconds - this has stopped nearly all firewall activity on established connections.

OK... I think this needs changing then. Can everyone please try the following trivial patch and report any changes?

Thanks!
Rusty.

net/ipv4/netfilter/ip_conntrack_proto_tcp.c +1 -1
diff -urN -I \$.*\$ --exclude TAGS -X /home/rusty/devel/kernel/kernel-patches/current-dontdiff --minimal linux-2.4.12-official/net/ipv4/netfilter/ip_conntrack_proto_tcp.c working-2.4.12-tcptime/net/ipv4/netfilter/ip_conntrack_proto_tcp.c
@@ -55,7 +55,7 @@
     2 MINS,	/*	TCP_CONNTRACK_FIN_WAIT,	*/
     2 MINS,	/*	TCP_CONNTRACK_TIME_WAIT,	*/
     10 SECS,	/*	TCP_CONNTRACK_CLOSE,	*/
-    60 SECS,	/*	TCP_CONNTRACK_CLOSE_WAIT,	*/
+    2 MINS,	/*	TCP_CONNTRACK_CLOSE_WAIT,	*/
     30 SECS,	/*	TCP_CONNTRACK_LAST_ACK,	*/
     2 MINS,	/*	TCP_CONNTRACK_LISTEN,	*/
 };
#18 lore raw
In reply to Trever L. Adams
Quoted 9 lines · Rusty Russell
Quoted 3 lines · On Fri, 19 Oct 2001 06:18:30 -0700 Darrell A Escola <[email protected]>…

I have been running 2.4.10-ac11 for 7 days now with TCP_CONNTRACK_CLOSE_WAIT set to 120 seconds - this has stopped nearly all firewall activity on established connections.

OK... I think this needs changing then. Can everyone please try the following trivial patch and report any changes?

Running 2.4.4 with this patch for the past 4 days has reduced the number of inappropriately dropped packets by ip_conntrack to nearly zero. The number of legitimate packets that used to be dropped previous to running this patch would sometimes reach into the low hundreds over the same time frame. (FWIW, I have a cable modem connection to the 'net, and so it gets a bit slow from time to time since my bandwidth is shared...).

--Mike

Quoted 14 lines

diff -urN -I \$.*\$ --exclude TAGS -X /home/rusty/devel/kernel/kernel-patches/current-dontdiff --minimal linux-2.4.12-official/net/ipv4/netfilter/ip_conntrack_proto_tcp.c working-2.4.12-tcptime/net/ipv4/netfilter/ip_conntrack_proto_tcp.c

net/ipv4/netfilter/ip_conntrack_proto_tcp.c +1 -2
@@ -55,7 +55,7 @@
     2 MINS,	/*	TCP_CONNTRACK_FIN_WAIT,	*/
     2 MINS,	/*	TCP_CONNTRACK_TIME_WAIT,	*/
     10 SECS,	/*	TCP_CONNTRACK_CLOSE,	*/
-    60 SECS,	/*	TCP_CONNTRACK_CLOSE_WAIT,	*/
+    2 MINS,	/*	TCP_CONNTRACK_CLOSE_WAIT,	*/
     30 SECS,	/*	TCP_CONNTRACK_LAST_ACK,	*/
     2 MINS,	/*	TCP_CONNTRACK_LISTEN,	*/
 };
 
-
Quoted 4 lines

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Michael B. Rash
http://www.cipherdyne.com
Key fingerprint = 8E40 0826 4BBD 9DAF 4563 695C AC21 A428 70C9 B006