NEW packets with no SYN bit set in OUTPUT

Hi,

I am having an issue related to conntrack and iptables than I am having a hard time fixing. My policy for the OUTPUT chain looks basically like this:

Set default policy to DROP
DROP all invalid pakets
ACCEPT all established an related
Then a fairly long list of rules and chains to filter what NEW packets should be allowed out At the end, LOG those that haven't been matched by a previous rule in an unauthorized_outgoing chain.

Now I am getting a small number of packets in that chain (about 15-20 per hour, and the server does about 30mbps), like this one:

kernel: [12817249.101873] [fw] UNAUTH. OUTGOING CONN.IN= OUT=eth0 SRC=188.xx.xx.xx DST=80.xx.xx.xx LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=59528 DF PROTO=TCP SPT=80 DPT=16258 WINDOW=6432 RES=0x00 ACK URGP=0

I am wondering about a few things in that packet:

* Its source port is 80 - the server being a HTTP proxy, this packet is (fairly) likely a reply to another connection
* The packet goes into the unauthorized_outgoing logging chain, so the packet doesn't have the state established, related, or invalid : it must be "new". However the packet doesn't have the SYN bit set - it's just an ACK.

I came across the appendix B in the Linux Packet Filtering and Iptables tutorial "State NEW packets but no SYN bit set" at http://www.linuxtopia.org/Linux_Firewall_iptables/x6193.html that describes a similar "feature", and that it might be triggered by a bad Microsoft TCP/IP implementation. However, my packets are in the output chain, not the input.

I have added the following rule just before the regular logging chain:

iptables -A OUTPUT -p tcp ! --syn -m state --state NEW -j new_outgoing_without_syn

It actually captures all the packets that would have otherwise arrived in my unauthorized_outgoing chain - so, in a way, I have "solved" my problem, as I don't get the false positives anymore.

kernel: [12203442.354426] [fw] NEW W/O SYN.IN= OUT=eth0 SRC=188.xx.xx.xx DST=88.xx.xx.xx LEN=1492 TOS=0x00 PREC=0x00 TTL=64 ID=5298 DF PROTO=TCP SPT=80 DPT=2373 WINDOW=14 RES=0x00 ACK URGP=0

However, I'm unsure about the origin of those packets.
Do you think my "fix" is correct? What explanation for such packets could there be?

NB:

Linux p12 2.6.32-35-server #78-Ubuntu SMP Tue Oct 11 16:26:12 UTC 2011 x86_64 GNU/Linux iptables v1.4.4

Tuned TCP stack according to recommendations for varnish:

net.ipv4.tcp_syn_retries=2
net.ipv4.tcp_synack_retries=2
net.ipv4.tcp_max_orphans=262144
net.ipv4.tcp_fin_timeout=3
net.ipv4.tcp_max_syn_backlog=262144
net.ipv4.tcp_wmem=4096 65536 16777216
net.ipv4.tcp_rmem=4096 87380 16777216
net.core.wmem_max=16777216
net.core.rmem_max=16777216
net.core.netdev_max_backlog=2500
net.ipv4.tcp_no_metrics_save=1
net.core.somaxconn=262144
net.ipv4.tcp_tw_recycle=0
net.ipv4.tcp_tw_reuse=0

Thank you,

Yann

It might be a case of Fast Retransmit as per RFC 2001. The duplicate ACK is supposed to be sent by the receiver if segments are received out-of-order. As far as I'm aware, if three such ACKs are received by the sender - all referring to the same segment - then it is assumed by the sender that the segment was never received and that it should be retransmitted as quickly as possible. At that point, the sender would also increase the congestion window and reduce the transmission rate.

My tentative guess would be that, if more than three are seen for the same segment, they are deemed INVALID by the connection tracking system. You could use tcpdump to evaluate this possibility. Alternatively, perhaps someone with a strong knowledge of conntrack internals could confirm one way or the other.

Cheers,

--Kerin