netfilter: xt_TCPMSS: Avoid violating RFC 879 in absence of MSS option

As reported in bug #662, the clamp-mss-to-pmtu option of the xt_TCPMSS target can cause issues connecting to websites if there was no MSS option present in the original SYN packet from the client. In these cases, it adds an MSS higher than the default specified in RFC 879. Fix this by never setting a value > 536 IFF none was specified by the client.

This closes bug #662.

Phil

index a75240f..53af7db 100644
@@ -125,6 +125,13 @@ tcpmss_mangle_packet(struct sk_buff *skb,
 
 	skb_put(skb, TCPOLEN_MSS);
 
+	/*
+	 * RFC 879 states that the default MSS is 536 without specific
+	 * knowledge that the destination host is prepared to accept larger.
+	 * Since no MSS was provided, we MUST NOT set a value > 536.
+	 */
+	newmss = min(newmss, (u16)536);
+
 	opt = (u_int8_t *)tcph + sizeof(struct tcphdr);
 	memmove(opt + TCPOLEN_MSS, opt, tcplen - sizeof(struct tcphdr));
 

Applied to the nf tree, thanks Phil.

BTW, this target does not seem to make safe fragmentation handling. We need a patch similar to:

commit bc6bcb59dd7c184d229f9e86d08aa56059938a4c

Author: Pablo Neira Ayuso <[email protected]>
Date:   Tue May 7 03:22:18 2013 +0200

netfilter: xt_TCPOPTSTRIP: fix possible mangling beyond packet boundary