Skip to contentSkip to conversation

NFQUEUE: introduce CPU fanout

PATCH Applied Holger Eitzenberger 2 participants 9 messages Mar 23, 2013 – Apr 2, 2013

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v2 cover NFQUEUE: introduce CPU fanout

Hi,

this is v2 of the patchset which tries to improve NFQUEUE performance if the --queue-balance argument is used for steering packets to several NFQUEUEs. By changing the way which NFQUEUE is assigned I'm able to improve the performance if the processes reading on the NFQUEUs are pinned correctly.

Changes from v1:

* 1/3: fold 'flags' into 'bypass'. Not sure whether I like it.
* 2/3: tailing 'else' avoided in nfqueue_hash().
* 2/3: tried to avoid indentation uglyness in nfqueue_hash().

Current NFQUEUE target uses a hash, computed over source and destination address (and other parameters), for steering the packet to the actual NFQUEUE. This however forgets about the fact that the packet eventually is handled by a particular CPU on user request.

If e. g.

1) IRQ affinity is used to handle packets on a particular CPU already (both single-queue or multi-queue case)

and/or

2) RPS is used to steer packets to a specific softirq

the target easily chooses an NFQUEUE which is not handled by a process pinned to the same CPU.

The idea is therefore to use the CPU index for determining the NFQUEUE handling the packet.

E. g. when having a system with 4 CPUs, 4 MQ queues and 4 NFQUEUEs it looks like this:

 +-----+  +-----+  +-----+  +-----+
 |NFQ#0|  |NFQ#1|  |NFQ#2|  |NFQ#3|
 +-----+  +-----+  +-----+  +-----+
    ^        ^        ^        ^
    |        |NFQUEUE |        |
    +        +        +        +
 +-----+  +-----+  +-----+  +-----+
 |rx-0 |  |rx-1 |  |rx-2 |  |rx-3 |
 +-----+  +-----+  +-----+  +-----+

The NFQUEUEs not necessarily have to start with number 0, setups with less NFQUEUEs than packet-handling CPUs are not a problem as well.

The first patch extends the NFQUEUE target to accept a new NFQ_FLAG_CPU_FANOUT flag. If this is specified the target uses the CPU index for determining the NFQUEUE being used. I have to introduce rev3 for this, sorry.

The 2nd patch coalesces rev1 and rev3 hashing by introducing nfqueue_hash(), then used in both revisions.

The 3rd patch extends iptables userspace to accept the --queue-cpu-fanout argument, needs --queue-balance.

Thank you.

/Holger

#1 lore raw

PATCH v2 1/3 NFQUEUE: introduce CPU fanout

The 'flags' are folded into _v2 'bypass'.

include/uapi/linux/netfilter/xt_NFQUEUE.h +9 -0
Index: net-next/include/uapi/linux/netfilter/xt_NFQUEUE.h
===================================================================
@@ -26,4 +26,13 @@
 	__u16 bypass;
 };
 
+struct xt_NFQ_info_v3 {
+	__u16 queuenum;
+	__u16 queues_total;
+	__u16 flags;
+#define NFQ_FLAG_BYPASS		0x01 /* for compatibility with v2 */
+#define NFQ_FLAG_CPU_FANOUT	0x02 /* use current CPU (no hashing) */
+#define NFQ_FLAG_MASK		0x03
+};
+
 #endif /* _XT_NFQ_TARGET_H */
net/netfilter/xt_NFQUEUE.c +39 -2

63 lines not on this page · raw message

maintainer Applied
#2 lore raw
Quoted 1 line · Holger Eitzenberger

The 'flags' are folded into _v2 'bypass'.

Applied, thanks Holger.

#3 lore raw

PATCH v2 2/3 NFQUEUE: coalesce IPv4 and IPv6 hashing

In reply to Holger Eitzenberger

Because rev1 and rev3 of the target share the same hashing generalize it by introduing nfqueue_hash().

net/netfilter/xt_NFQUEUE.c +22 -20
Index: net-next/net/netfilter/xt_NFQUEUE.c
===================================================================
@@ -76,22 +76,31 @@
 }
 #endif
 
-static unsigned int
-nfqueue_tg_v1(struct sk_buff *skb, const struct xt_action_param *par)
+static u32
+nfqueue_hash(const struct sk_buff *skb, const struct xt_action_param *par)
 {
 	const struct xt_NFQ_info_v1 *info = par->targinfo;
 	u32 queue = info->queuenum;
 
-	if (info->queues_total > 1) {
-		if (par->family == NFPROTO_IPV4)
-			queue = (((u64) hash_v4(skb) * info->queues_total) >>
-				 32) + queue;
+	if (par->family == NFPROTO_IPV4)
+		queue += ((u64) hash_v4(skb) * info->queues_total) >> 32;
 #if IS_ENABLED(CONFIG_IP6_NF_IPTABLES)
-		else if (par->family == NFPROTO_IPV6)
-			queue = (((u64) hash_v6(skb) * info->queues_total) >>
-				 32) + queue;
+	else if (par->family == NFPROTO_IPV6)
+		queue += ((u64) hash_v6(skb) * info->queues_total) >> 32;
 #endif
-	}
+
+	return queue;
+}
+
+static unsigned int
+nfqueue_tg_v1(struct sk_buff *skb, const struct xt_action_param *par)
+{
+	const struct xt_NFQ_info_v1 *info = par->targinfo;
+	u32 queue = info->queuenum;
+
+	if (info->queues_total > 1)
+		queue = nfqueue_hash(skb, par);
+
 	return NF_QUEUE_NR(queue);
 }
 
@@ -144,17 +153,10 @@
 			int cpu = smp_processor_id();
 
 			queue = info->queuenum + cpu % info->queues_total;
-		} else {
-			if (par->family == NFPROTO_IPV4)
-				queue = (((u64) hash_v4(skb) * info->queues_total) >>
-						 32) + queue;
-#if IS_ENABLED(CONFIG_IP6_NF_IPTABLES)
-			else if (par->family == NFPROTO_IPV6)
-				queue = (((u64) hash_v6(skb) * info->queues_total) >>
-						 32) + queue;
-#endif
-		}
+		} else
+			queue = nfqueue_hash(skb, par);
 	}
+
 	return NF_QUEUE_NR(queue);
 }
 
#4 lore raw

Re: [PATCH v2 2/3] NFQUEUE: coalesce IPv4 and IPv6 hashing

Quoted 2 lines · Holger Eitzenberger

Because rev1 and rev3 of the target share the same hashing generalize it by introduing nfqueue_hash().

Also applied, thanks.

#5 lore raw

PATCH v2 3/3 NFQUEUE: add --queue-cpu-fanout parameter

In reply to Holger Eitzenberger
extensions/libxt_NFQUEUE.c +60 -3
Index: iptables/extensions/libxt_NFQUEUE.c
===================================================================
@@ -13,8 +13,10 @@
 	O_QUEUE_NUM = 0,
 	O_QUEUE_BALANCE,
 	O_QUEUE_BYPASS,
+	O_QUEUE_CPU_FANOUT,
 	F_QUEUE_NUM     = 1 << O_QUEUE_NUM,
 	F_QUEUE_BALANCE = 1 << O_QUEUE_BALANCE,
+	F_QUEUE_CPU_FANOUT = 1 << O_QUEUE_CPU_FANOUT,
 };
 
 static void NFQUEUE_help(void)
@@ -37,7 +39,15 @@
 {
 	NFQUEUE_help_v1();
 	printf(
-"  --queue-bypass		Bypass Queueing if no queue instance exists.\n");
+"  --queue-bypass		Bypass Queueing if no queue instance exists.\n"
+"  --queue-cpu-fanout	Use current CPU (no hashing)\n");
+}
+
+static void NFQUEUE_help_v3(void)
+{
+	NFQUEUE_help_v2();
+	printf(
+"  --queue-cpu-fanout	Use current CPU (no hashing)\n");
 }
 
 #define s struct xt_NFQ_info
@@ -48,6 +58,8 @@
 	{.name = "queue-balance", .id = O_QUEUE_BALANCE,
 	 .type = XTTYPE_UINT16RC, .excl = F_QUEUE_NUM},
 	{.name = "queue-bypass", .id = O_QUEUE_BYPASS, .type = XTTYPE_NONE},
+	{.name = "queue-cpu-fanout", .id = O_QUEUE_CPU_FANOUT,
+	 .type = XTTYPE_NONE, .also = O_QUEUE_BALANCE},
 	XTOPT_TABLEEND,
 };
 #undef s
@@ -92,6 +104,18 @@
 	}
 }
 
+static void NFQUEUE_parse_v3(struct xt_option_call *cb)
+{
+	struct xt_NFQ_info_v3 *info = cb->data;
+
+	NFQUEUE_parse_v2(cb);
+	switch (cb->entry->id) {
+	case O_QUEUE_CPU_FANOUT:
+		info->flags |= NFQ_FLAG_CPU_FANOUT;
+		break;
+	}
+}
+
 static void NFQUEUE_print(const void *ip,
                           const struct xt_entry_target *target, int numeric)
 {
@@ -120,10 +144,20 @@
 	const struct xt_NFQ_info_v2 *info = (void *) target->data;
 
 	NFQUEUE_print_v1(ip, target, numeric);
-	if (info->bypass)
+	if (info->bypass & NFQ_FLAG_BYPASS)
 		printf(" bypass");
 }
 
+static void NFQUEUE_print_v3(const void *ip,
+                             const struct xt_entry_target *target, int numeric)
+{
+	const struct xt_NFQ_info_v3 *info = (void *)target->data;
+
+	NFQUEUE_print_v2(ip, target, numeric);
+	if (info->flags & NFQ_FLAG_CPU_FANOUT)
+		printf(" cpu-fanout");
+}
+
 static void NFQUEUE_save(const void *ip, const struct xt_entry_target *target)
 {
 	const struct xt_NFQ_info *tinfo =
@@ -151,10 +185,20 @@
 
 	NFQUEUE_save_v1(ip, target);
 
-	if (info->bypass)
+	if (info->bypass & NFQ_FLAG_BYPASS)
 		printf(" --queue-bypass");
 }
 
+static void NFQUEUE_save_v3(const void *ip,
+			    const struct xt_entry_target *target)
+{
+	const struct xt_NFQ_info_v3 *info = (void *)target->data;
+
+	NFQUEUE_save_v2(ip, target);
+	if (info->flags & NFQ_FLAG_CPU_FANOUT)
+		printf(" --queue-cpu-fanout");
+}
+
 static void NFQUEUE_init_v1(struct xt_entry_target *t)
 {
 	struct xt_NFQ_info_v1 *tinfo = (void *)t->data;
@@ -199,6 +243,19 @@
 	.save		= NFQUEUE_save_v2,
 	.x6_parse	= NFQUEUE_parse_v2,
 	.x6_options	= NFQUEUE_opts,
+},{
+	.family		= NFPROTO_UNSPEC,
+	.revision	= 3,
+	.name		= "NFQUEUE",
+	.version	= XTABLES_VERSION,
+	.size		= XT_ALIGN(sizeof(struct xt_NFQ_info_v3)),
+	.userspacesize	= XT_ALIGN(sizeof(struct xt_NFQ_info_v3)),
+	.help		= NFQUEUE_help_v3,
+	.init		= NFQUEUE_init_v1,
+	.print		= NFQUEUE_print_v3,
+	.save		= NFQUEUE_save_v3,
+	.x6_parse	= NFQUEUE_parse_v3,
+	.x6_options	= NFQUEUE_opts,
 }
 };
 
include/linux/netfilter/xt_NFQUEUE.h +9 -0
Index: iptables/include/linux/netfilter/xt_NFQUEUE.h
===================================================================
@@ -26,4 +26,13 @@
 	__u16 bypass;
 };
 
+struct xt_NFQ_info_v3 {
+	__u16 queuenum;
+	__u16 queues_total;
+	__u16 flags;
+#define NFQ_FLAG_BYPASS		0x01 /* for compatibility with v2 */
+#define NFQ_FLAG_CPU_FANOUT	0x02 /* use current CPU (no hashing) */
+#define NFQ_FLAG_MASK		0x03
+};
+
 #endif /* _XT_NFQ_TARGET_H */
#6 lore raw

Re: [PATCH v2 3/3] NFQUEUE: add --queue-cpu-fanout parameter

Hi Holger,

Quoted 34 lines · Holger Eitzenberger

Index: iptables/extensions/libxt_NFQUEUE.c
===================================================================

extensions/libxt_NFQUEUE.c +11 -1
@@ -13,8 +13,10 @@
 	O_QUEUE_NUM = 0,
 	O_QUEUE_BALANCE,
 	O_QUEUE_BYPASS,
+	O_QUEUE_CPU_FANOUT,
 	F_QUEUE_NUM     = 1 << O_QUEUE_NUM,
 	F_QUEUE_BALANCE = 1 << O_QUEUE_BALANCE,
+	F_QUEUE_CPU_FANOUT = 1 << O_QUEUE_CPU_FANOUT,
 };
 
 static void NFQUEUE_help(void)
@@ -37,7 +39,15 @@
 {
 	NFQUEUE_help_v1();
 	printf(
-"  --queue-bypass		Bypass Queueing if no queue instance exists.\n");
+"  --queue-bypass		Bypass Queueing if no queue instance exists.\n"
+"  --queue-cpu-fanout	Use current CPU (no hashing)\n");
+}
+
+static void NFQUEUE_help_v3(void)
+{
+	NFQUEUE_help_v2();
+	printf(
+"  --queue-cpu-fanout	Use current CPU (no hashing)\n");
 }
 
 #define s struct xt_NFQ_info

extensions/libxt_NFQUEUE.c line 62

Quoted 5 lines
extensions/libxt_NFQUEUE.c +2 -0
@@ -48,6 +58,8 @@
 	{.name = "queue-balance", .id = O_QUEUE_BALANCE,
 	 .type = XTTYPE_UINT16RC, .excl = F_QUEUE_NUM},
 	{.name = "queue-bypass", .id = O_QUEUE_BYPASS, .type = XTTYPE_NONE},
+	{.name = "queue-cpu-fanout", .id = O_QUEUE_CPU_FANOUT,
+	 .type = XTTYPE_NONE, .also = O_QUEUE_BALANCE},

I think we have to add O_QUEUE_CPU_FANOUT here to make sure both queue-balance and queue-cpu-fanout are not used both incorrectly together. That also needs some code a new .x6_fcheck function to validate this.

Could you also send me the corresponding manpage update for this?

Thanks!

#7 lore raw

Re: [PATCH v2 3/3] NFQUEUE: add --queue-cpu-fanout parameter

Hi Pablo,

Quoted 4 lines

I think we have to add O_QUEUE_CPU_FANOUT here to make sure both queue-balance and queue-cpu-fanout are not used both incorrectly together. That also needs some code a new .x6_fcheck function to validate this.

it is currently that --queue-cpu-fanout *requires* --queue-balance, as the CPU fanout works on top of the balanced queues.

Or possibly I miss your point.

Quoted 1 line

Could you also send me the corresponding manpage update for this?

Sorry for lacking that.

I've respinned the 3/3 patch to include the man page update as well. I was unsure about using bold font for referencing the --queue-balance parameter. Checking the man pages of other extensions it seems as if it is done quite differently throughout.

Patch is attached to this email.

/Holger

NFQUEUE: add --queue-cpu-fanout parameter

extensions/libxt_NFQUEUE.c +60 -3
Index: iptables/extensions/libxt_NFQUEUE.c
===================================================================
@@ -13,8 +13,10 @@
 	O_QUEUE_NUM = 0,
 	O_QUEUE_BALANCE,
 	O_QUEUE_BYPASS,
+	O_QUEUE_CPU_FANOUT,
 	F_QUEUE_NUM     = 1 << O_QUEUE_NUM,
 	F_QUEUE_BALANCE = 1 << O_QUEUE_BALANCE,
+	F_QUEUE_CPU_FANOUT = 1 << O_QUEUE_CPU_FANOUT,
 };
 
 static void NFQUEUE_help(void)
@@ -37,7 +39,15 @@
 {
 	NFQUEUE_help_v1();
 	printf(
-"  --queue-bypass		Bypass Queueing if no queue instance exists.\n");
+"  --queue-bypass		Bypass Queueing if no queue instance exists.\n"
+"  --queue-cpu-fanout	Use current CPU (no hashing)\n");
+}
+
+static void NFQUEUE_help_v3(void)
+{
+	NFQUEUE_help_v2();
+	printf(
+"  --queue-cpu-fanout	Use current CPU (no hashing)\n");
 }
 
 #define s struct xt_NFQ_info
@@ -48,6 +58,8 @@
 	{.name = "queue-balance", .id = O_QUEUE_BALANCE,
 	 .type = XTTYPE_UINT16RC, .excl = F_QUEUE_NUM},
 	{.name = "queue-bypass", .id = O_QUEUE_BYPASS, .type = XTTYPE_NONE},
+	{.name = "queue-cpu-fanout", .id = O_QUEUE_CPU_FANOUT,
+	 .type = XTTYPE_NONE, .also = F_QUEUE_BALANCE},
 	XTOPT_TABLEEND,
 };
 #undef s
@@ -92,6 +104,18 @@
 	}
 }
 
+static void NFQUEUE_parse_v3(struct xt_option_call *cb)
+{
+	struct xt_NFQ_info_v3 *info = cb->data;
+
+	NFQUEUE_parse_v2(cb);
+	switch (cb->entry->id) {
+	case O_QUEUE_CPU_FANOUT:
+		info->flags |= NFQ_FLAG_CPU_FANOUT;
+		break;
+	}
+}
+
 static void NFQUEUE_print(const void *ip,
                           const struct xt_entry_target *target, int numeric)
 {
@@ -120,10 +144,20 @@
 	const struct xt_NFQ_info_v2 *info = (void *) target->data;
 
 	NFQUEUE_print_v1(ip, target, numeric);
-	if (info->bypass)
+	if (info->bypass & NFQ_FLAG_BYPASS)
 		printf(" bypass");
 }
 
+static void NFQUEUE_print_v3(const void *ip,
+                             const struct xt_entry_target *target, int numeric)
+{
+	const struct xt_NFQ_info_v3 *info = (void *)target->data;
+
+	NFQUEUE_print_v2(ip, target, numeric);
+	if (info->flags & NFQ_FLAG_CPU_FANOUT)
+		printf(" cpu-fanout");
+}
+
 static void NFQUEUE_save(const void *ip, const struct xt_entry_target *target)
 {
 	const struct xt_NFQ_info *tinfo =
@@ -151,10 +185,20 @@
 
 	NFQUEUE_save_v1(ip, target);
 
-	if (info->bypass)
+	if (info->bypass & NFQ_FLAG_BYPASS)
 		printf(" --queue-bypass");
 }
 
+static void NFQUEUE_save_v3(const void *ip,
+			    const struct xt_entry_target *target)
+{
+	const struct xt_NFQ_info_v3 *info = (void *)target->data;
+
+	NFQUEUE_save_v2(ip, target);
+	if (info->flags & NFQ_FLAG_CPU_FANOUT)
+		printf(" --queue-cpu-fanout");
+}
+
 static void NFQUEUE_init_v1(struct xt_entry_target *t)
 {
 	struct xt_NFQ_info_v1 *tinfo = (void *)t->data;
@@ -199,6 +243,19 @@
 	.save		= NFQUEUE_save_v2,
 	.x6_parse	= NFQUEUE_parse_v2,
 	.x6_options	= NFQUEUE_opts,
+},{
+	.family		= NFPROTO_UNSPEC,
+	.revision	= 3,
+	.name		= "NFQUEUE",
+	.version	= XTABLES_VERSION,
+	.size		= XT_ALIGN(sizeof(struct xt_NFQ_info_v3)),
+	.userspacesize	= XT_ALIGN(sizeof(struct xt_NFQ_info_v3)),
+	.help		= NFQUEUE_help_v3,
+	.init		= NFQUEUE_init_v1,
+	.print		= NFQUEUE_print_v3,
+	.save		= NFQUEUE_save_v3,
+	.x6_parse	= NFQUEUE_parse_v3,
+	.x6_options	= NFQUEUE_opts,
 }
 };
 
include/linux/netfilter/xt_NFQUEUE.h +9 -0
Index: iptables/include/linux/netfilter/xt_NFQUEUE.h
===================================================================
@@ -26,4 +26,13 @@
 	__u16 bypass;
 };
 
+struct xt_NFQ_info_v3 {
+	__u16 queuenum;
+	__u16 queues_total;
+	__u16 flags;
+#define NFQ_FLAG_BYPASS		0x01 /* for compatibility with v2 */
+#define NFQ_FLAG_CPU_FANOUT	0x02 /* use current CPU (no hashing) */
+#define NFQ_FLAG_MASK		0x03
+};
+
 #endif /* _XT_NFQ_TARGET_H */
extensions/libxt_NFQUEUE.man +8 -0
Index: iptables/extensions/libxt_NFQUEUE.man
===================================================================
@@ -23,3 +23,11 @@
 By default, if no userspace program is listening on an NFQUEUE, then all packets that are to be queued
 are dropped.  When this option is used, the NFQUEUE rule is silently bypassed instead. The packet
 will move on to the next rule.
+.PP
+.TP
+\fB\-\-queue\-cpu-fanout\fP
+When used together with \fB--queue-balance\fP this will use the CPU ID
+as an index into the NFQUEUEs.  This is a performance improvement if
+all packet handling CPUs have a corresponding NFQUEUE.
+
+Requires \fB--queue-balance\fP to be specified.
#8 lore raw

Re: [PATCH v2 3/3] NFQUEUE: add --queue-cpu-fanout parameter

Hi Holger,

Quoted 11 lines · Holger Eitzenberger

Hi Pablo,

Quoted 4 lines

I think we have to add O_QUEUE_CPU_FANOUT here to make sure both queue-balance and queue-cpu-fanout are not used both incorrectly together. That also needs some code a new .x6_fcheck function to validate this.

it is currently that --queue-cpu-fanout *requires* --queue-balance, as the CPU fanout works on top of the balanced queues.

Or possibly I miss your point.

Forget it, your patch was just fine, sorry.

Quoted 5 lines
Quoted 1 line

Could you also send me the corresponding manpage update for this?

Sorry for lacking that.

I've respinned the 3/3 patch to include the man page update as well.

Thanks!

I have pushed this to the -next branch of iptables. Will merge by when 3.10-rc1 is released.