Skip to contentSkip to conversation

rps: introduce a new sysctl switch rps_workaround_buggy_driver

PATCH In review Li Yu 3 participants 4 messages Apr 5, 2012 – Apr 6, 2012

Subsystems (2)
Networking Network drivers
lore
Share & tools
lore

Li Yu

author
#0 lore raw

PATCH 1/1 rps: introduce a new sysctl switch rps_workaround_buggy_driver

We encountered a buggy NIC driver or hardware/firmware, it keeps non-zero constant skb->rxhash for long time, so if we enabled RPS, the targeted CPU keeps same for long time too.

This patch introduces a sysctl switch to workaround for such problem, if the switch was on, RPS core discards the skb->rxhash that is computed by NIC hardware.

Hope this patch also can help others, thanks.

Signed-off-by Li Yu <[email protected]>

include/linux/skbuff.h +0 -0
index 192250b..4c28ce0 100644
@@ -628,9 +628,13 @@ extern unsigned int   skb_find_text(struct sk_buff

*skb, unsigned int from,
unsigned int to, struct ts_config *config, struct ts_state *state);

+extern int rps_workaround_buggy_driver;
extern void __skb_get_rxhash(struct sk_buff *skb); static inline __u32 skb_get_rxhash(struct sk_buff *skb) {
+ if (unlikely(rps_workaround_buggy_driver))
+ skb->rxhash = 0;
+
if (!skb->rxhash)
__skb_get_rxhash(skb);

net/core/dev.c +2 -0
index 723a406..9d1e728 100644
@@ -176,6 +176,8 @@
 #define PTYPE_HASH_SIZE	(16)
 #define PTYPE_HASH_MASK	(PTYPE_HASH_SIZE - 1)
 
+int rps_workaround_buggy_driver = 0;
+
 static DEFINE_SPINLOCK(ptype_lock);
 static struct list_head ptype_base[PTYPE_HASH_SIZE] __read_mostly;
 static struct list_head ptype_all __read_mostly;	/* Taps */
net/core/sysctl_net_core.c +7 -0
index 0c28508..065ea7c 100644
@@ -172,6 +172,13 @@ static struct ctl_table net_core_table[] = {
 		.mode		= 0644,
 		.proc_handler	= rps_sock_flow_sysctl
 	},
+	{
+		.procname	= "rps_workaround_buggy_driver",
+		.data		= &rps_workaround_buggy_driver,
+		.maxlen		= sizeof(int),
+		.mode		= 0644,
+		.proc_handler	= proc_dointvec
+	},
 #endif
 #endif /* CONFIG_NET */
 	{
maintainer
#1 lore raw

From: Li Yu <[email protected]>
Date: Thu, 05 Apr 2012 18:31:00 +0800

Quoted 5 lines

We encountered a buggy NIC driver or hardware/firmware, it keeps non-zero constant skb->rxhash for long time, so if we enabled RPS, the targeted CPU keeps same for long time too.

Quoted 7 lines

This patch introduces a sysctl switch to workaround for such problem, if the switch was on, RPS core discards the skb->rxhash that is computed by NIC hardware.

Hope this patch also can help others, thanks.

Signed-off-by Li Yu <[email protected]>

No way, we fix the drivers not add workarounds like this.

#2 lore raw
In reply to Li Yu
Quoted 9 lines · Li Yu

We encountered a buggy NIC driver or hardware/firmware, it keeps non-zero constant skb->rxhash for long time, so if we enabled RPS, the targeted CPU keeps same for long time too.

This patch introduces a sysctl switch to workaround for such problem, if the switch was on, RPS core discards the skb->rxhash that is computed by NIC hardware.

Hope this patch also can help others, thanks.

Really ?

to disable this driver rxhash, you should try :

ethtool -K eth0 rxhash off

author
#3 lore raw
Quoted 9 lines · Eric Dumazet
Quoted 8 lines · Li Yu

We encountered a buggy NIC driver or hardware/firmware, it keeps non-zero constant skb->rxhash for long time, so if we enabled RPS, the targeted CPU keeps same for long time too.

This patch introduces a sysctl switch to workaround for such problem, if the switch was on, RPS core discards the skb->rxhash that is computed by NIC hardware.

Quoted 9 lines
Quoted 1 line

Hope this patch also can help others, thanks.

Really ?

to disable this driver rxhash, you should try :

ethtool -K eth0 rxhash off

Great! I really did not know this new option ago, and wrote a kprobe workaround module. It seem that ethtool 2.6.39 have such support at least.

Thank you very much!

Yu

Quoted 2 lines