State resolution packet queue for IPsec

I did a implementation of a state resolution packet queue for IPsec. The original idea is described in git commit 14e50e57a ([XFRM]: Allow packet drops during larval state resolution.)

Since we don't have any notifiers for xfrm state resolution, I used a timer that does a relookup in given intervals. This is mainly to speed up the start of tcp connections, so I tried to do the relookup in a higher rate that tcp would resend the syn packet. However, the relookup intervals, the queue size and the timeout value are choosen by gut instincts, so could be suboptimal. If anyone has some insight on good choices of these values, please let me know. All other comment are welcome too, of course.

Subject: [PATCH] xfrm: Add a state resolution packet queue

As the default, we blackhole packets until the key manager resolves the states. This patch implements a packet queue where IPsec packets are queued until the states are resolved. We generate a dummy xfrm bundle, the output routine of the returned route enqueues the packet to a per policy queue and arms a timer that checks for state resolution when dst_output() is called. Once the states are resolved, the packets are sent out of the queue. If the states are not resolved after some time, the queue is flushed.

This patch keeps the defaut behaviour to blackhole packets as long as we have no states. To enable the packet queue the sysctl xfrm_larval_drop must be switched off.

---
 include/net/dst.h      |    1 +
 include/net/xfrm.h     |    7 ++
 net/xfrm/xfrm_policy.c |  229 +++++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 233 insertions(+), 4 deletions(-)

index 9a78810..3da47e0 100644
@@ -61,6 +61,7 @@ struct dst_entry {
 #define DST_NOPEER		0x0040
 #define DST_FAKE_RTABLE		0x0080
 #define DST_XFRM_TUNNEL		0x0100
+#define DST_XFRM_QUEUE		0x0200
 
 	unsigned short		pending_confirm;
 

index 63445ed..4273a8f 100644
@@ -501,6 +501,12 @@ struct xfrm_policy_walk {
 	u32 seq;
 };
 
+struct xfrm_policy_queue {
+	struct sk_buff_head	hold_queue;
+	struct timer_list	hold_timer;
+	unsigned long		timeout;
+};
+
 struct xfrm_policy {
 #ifdef CONFIG_NET_NS
 	struct net		*xp_net;

@@ -522,6 +528,7 @@ struct xfrm_policy {
 	struct xfrm_lifetime_cfg lft;
 	struct xfrm_lifetime_cur curlft;
 	struct xfrm_policy_walk_entry walk;
+	struct xfrm_policy_queue polq;
 	u8			type;
 	u8			action;
 	u8			flags;

1.7.9.5

From: Steffen Klassert <[email protected]>
Date: Mon, 21 Jan 2013 10:48:48 +0100

This is probably the best way to solve this problem without adding xfrm_state resolution notifications.

But really why would notifications be so bad even if they would not be fine grained?

Any time a state is resolved, any state, you run what you have put currently into this new timer function.

Most of the time, defer list is empty, so no work is done. Otherwise quick notification is essential and worth running the list.

Then there are no time based heuristics to come up with at all.

When we use state resolution notifiers, we have two problems. First, we would probably not dequeue the packets on the same cpu we enqueued them, this can introduce packet reorder. The timer will at least try to run on the local cpu.

Second, as you already mentioned, even with state resolution notifiers, we don't know if the inserted state is the one we need and if our state bundle is complete with this state. We would need to do a lookup whenever a state is inserted, so we hog the cpu that tries to insert the states we need.

With the timer, we don't have these two problems for the price that we have take a good choice on the relookup intervals. Not sure what's better here.

I'd apply the timer version to ipsec-next if there are no objections. We can still tune it if something does not perform well.

From: Steffen Klassert <[email protected]>
Date: Mon, 4 Feb 2013 09:25:27 +0100

Ok.