bonding: modify the current and add new hash functions
Subsystems (4)
v2 cover bonding: modify the current and add new hash functions
Hi all,
This is a complete remake of my old patch that modified the bonding hash functions to use skb_flow_dissect which was suggested by Eric Dumazet. This time around I've left the old modes although using a new hash function again suggested by Eric, which is the same for all modes. The only difference is the way the headers are obtained. The old modes obtain them as before in order to address concerns about speed, but the 2 new ones use skb_flow_dissect. The unification of the hash function allows to remove a pointer from struct bonding and also a few extra functions that dealt with it. Two new functions are added which take care of the hashing based on bond->params.xmit_policy only:bond_xmit_hash() - global function, used by XOR and 3ad modes bond_flow_dissect() - used by bond_xmit_hash() to obtain the necessary headers and combine them according to bond->params.xmit_policy. Also factor out the ports extraction from skb_flow_dissect and add a new function - skb_flow_get_ports() which can be re-used.
v2: add the flow_dissector patch and use skb_flow_get_ports in patch 02
Best regards,
Nikolay Aleksandrov
Nikolay Aleksandrov (3):
flow_dissector: factor out the ports extraction in skb_flow_get_ports
bonding: modify the old and add new xmit hash policies
bonding: document the new xmit policy modes and update the changed
ones
Documentation/networking/bonding.txt | 66 ++++++------ drivers/net/bonding/bond_3ad.c | 2 +- drivers/net/bonding/bond_main.c | 197 ++++++++++++----------------------- drivers/net/bonding/bond_sysfs.c | 2 - drivers/net/bonding/bonding.h | 3 +- include/net/flow_keys.h | 1 + include/uapi/linux/if_bonding.h | 2 + net/core/flow_dissector.c | 38 +++++-- 8 files changed, 136 insertions(+), 175 deletions(-)
Signature · 1 line
1.8.1.4
PATCH v2 1/3 flow_dissector: factor out the ports extraction in skb_flow_get_ports
Factor out the code that extracts the ports from skb_flow_dissect and add a new function skb_flow_get_ports which can be re-used.
---
v2: new patch
This seems like a good idea because there're other users that can re-use it later as well.
include/net/flow_keys.h | 1 + net/core/flow_dissector.c | 38 +++++++++++++++++++++++++++----------- 2 files changed, 28 insertions(+), 11 deletions(-)
include/net/flow_keys.h +1 -0
@@ -14,4 +14,5 @@ struct flow_keys { };bool skb_flow_dissect (const struct sk_buff *skb,struct flow_keys *flow); +__be32 skb_flow_get_ports (const struct sk_buff *skb,int nhoff,u8 ip_proto);#endif
net/core/flow_dissector.c +27 -11
@@ -25,9 +25,34 @@ static void iph_to_flow_copy_addrs(struct flow_keys *flow, const struct iphdr *imemcpy (&flow->src, &iph->saddr,sizeof (flow->src) +sizeof (flow->dst)); } +/** +* skb_flow_get_ports - extract the upper layer ports and return them +* @skb: buffer to extract the ports from +* @nhoff: network header offset +* @ip_proto: protocol for which to get port offset +* +* The function will try to retrieve the ports at offset nhoff + poff where poff +* is the protocol port offset returned from proto_ports_offset +*/ +__be32 skb_flow_get_ports (const struct sk_buff *skb,int nhoff,u8 ip_proto) +{ +int poff =proto_ports_offset (ip_proto); + +if (poff >=0 ) { +__be32 *ports, _ports; + + nhoff += poff; + ports =skb_header_pointer (skb, nhoff,sizeof (_ports), &_ports); +if (ports) +return *ports; + } + +return 0 ; +} +bool skb_flow_dissect (const struct sk_buff *skb,struct flow_keys *flow) { -int poff, nhoff =skb_network_offset (skb); +int nhoff =skb_network_offset (skb);u8 ip_proto;__be16 proto = skb->protocol;
@@ -150,16 +175,7 @@ ipv6: } flow->ip_proto = ip_proto; - poff =proto_ports_offset (ip_proto); -if (poff >=0 ) { -__be32 *ports, _ports; - - nhoff += poff; - ports =skb_header_pointer (skb, nhoff,sizeof (_ports), &_ports); -if (ports) - flow->ports = *ports; - } - + flow->ports =skb_flow_get_ports (skb, nhoff, ip_proto); flow->thoff = (u16 ) nhoff;return true;
Signature · 1 line
1.8.1.4
Re: [PATCH net-next v2 1/3] flow_dissector: factor out the ports extraction in skb_flow_get_ports
Quoted 29 lines · Nikolay Aleksandrov
Factor out the code that extracts the ports from
skb_flow_dissectand add a new functionskb_flow_get_portswhich can be re-used.Suggested-by Veaceslav Falico <[email protected]>Signed-off-by Nikolay Aleksandrov <[email protected]>---
v2: new patch
This seems like a good idea because there're other users that can re-use it later as well.include/net/flow_keys.h | 1 + net/core/flow_dissector.c | 38 +++++++++++++++++++++++++++----------- 2 files changed, 28 insertions(+), 11 deletions(-)include/net/flow_keys.h +1
-0@@ -14,4 +14,5 @@ struct flow_keys { };bool skb_flow_dissect (const struct sk_buff *skb,struct flow_keys *flow); +__be32 skb_flow_get_ports (const struct sk_buff *skb,int nhoff,u8 ip_proto);#endif net/core/flow_dissector.c +0
-0
net/core/flow_dissector.c static void iph_to_flow_copy_addrs(struct flow_keys *flow, const struct iphdr *i · line 31
Quoted 7 lines
net/core/flow_dissector.c +4
-0@@ -25,9 +25,34 @@ static void iph_to_flow_copy_addrs(struct flow_keys *flow, const struct iphdr *imemcpy (&flow->src, &iph->saddr,sizeof (flow->src) +sizeof (flow->dst)); } +/** +* skb_flow_get_ports - extract the upper layer ports and return them +* @skb: buffer to extract the ports from +* @nhoff: network header offset
I hurried too much, this should've said "offset to the end of the network header" i.e. nhoff + sizeof(network header), I'll wait until tomorrow to see if there aren't any other comments and will edit this and re-post a v3.
Nik
Re: [PATCH net-next v2 1/3] flow_dissector: factor out the ports extraction in skb_flow_get_ports
Quoted 41 lines · Nikolay Aleksandrov
Quoted 29 lines · Nikolay Aleksandrov
Factor out the code that extracts the ports from
skb_flow_dissectand add a new functionskb_flow_get_portswhich can be re-used.Suggested-by Veaceslav Falico <[email protected]>Signed-off-by Nikolay Aleksandrov <[email protected]>---
v2: new patch
This seems like a good idea because there're other users that can re-use it later as well.include/net/flow_keys.h | 1 + net/core/flow_dissector.c | 38 +++++++++++++++++++++++++++----------- 2 files changed, 28 insertions(+), 11 deletions(-)include/net/flow_keys.h +1
-0@@ -14,4 +14,5 @@ struct flow_keys { };bool skb_flow_dissect (const struct sk_buff *skb,struct flow_keys *flow); +__be32 skb_flow_get_ports (const struct sk_buff *skb,int nhoff,u8 ip_proto);#endif net/core/flow_dissector.c +0
-0net/core/flow_dissector.c static void iph_to_flow_copy_addrs(struct flow_keys *flow, const struct iphdr *i · line 31
Quoted 7 lines
net/core/flow_dissector.c +4
-0@@ -25,9 +25,34 @@ static void iph_to_flow_copy_addrs(struct flow_keys *flow, const struct iphdr *imemcpy (&flow->src, &iph->saddr,sizeof (flow->src) +sizeof (flow->dst)); } +/** +* skb_flow_get_ports - extract the upper layer ports and return them +* @skb: buffer to extract the ports from +* @nhoff: network header offset I hurried too much, this should've said "offset to the end of the network header" i.e. nhoff + sizeof(network header), I'll wait until tomorrow to see if there aren't any other comments and will edit this and re-post a v3.
Quoted 2 lines
Nik
Ugh, nevermind this email, I'm going crazy about semantics today :-) Everything is fine.
PATCH v2 2/3 bonding: modify the old and add new xmit hash policies
In reply to Nikolay AleksandrovThis patch adds two new hash policy modes which use skb_flow_dissect:
3 - Encapsulated layer 2+3
4 - Encapsulated layer 3+4
There should be a good improvement for tunnel users in those modes. It also changes the old hash functions to:
hash ^= (__force u32)flow.dst ^ (__force u32)flow.src; hash ^= (hash >> 16);
hash ^= (hash >> 8);
Where hash will be initialized either to L2 hash, that is SRCMAC[5] XOR DSTMAC[5], or to flow->ports which should be extracted from the upper layer. Flow's dst and src are also extracted based on the xmit policy either directly from the buffer or by using skb_flow_dissect, but in both cases if the protocol is IPv6 then dst and src are obtained by ipv6_addr_hash() on the real addresses. In case of a non-dissectable packet, the algorithms fall back to L2 hashing.
The bond_set_mode_ops() function is now obsolete and thus deleted because it was used only to set the proper hash policy. Also we trim a pointer from struct bonding because we no longer need to keep the hash function, now there's only a single hash function - bond_xmit_hash that works based on bond->params.xmit_policy.
The hash function and skb_flow_dissect were suggested by Eric Dumazet. The layer names were suggested by Andy Gospodarek, because I suck at semantics.
---
v2: fix a bug in bond_flow_dissect which might've caused the use of
uninitalized flow_keys and make use of skb_flow_get_ports
One line is intentionally left at 82 chars since it's the whole function and IMO looks better that way.
drivers/net/bonding/bond_3ad.c | 2 +- drivers/net/bonding/bond_main.c | 197 ++++++++++++++------------------------- drivers/net/bonding/bond_sysfs.c | 2 - drivers/net/bonding/bonding.h | 3 +- include/uapi/linux/if_bonding.h | 2 + 5 files changed, 72 insertions(+), 134 deletions(-)
drivers/net/bonding/bond_3ad.c +1 -1
@@ -2442,7 +2442,7 @@ int bond_3ad_xmit_xor(struct sk_buff *skb, struct net_device *dev) goto out; } - slave_agg_no = bond->xmit_hash_policy (skb, slaves_in_agg); + slave_agg_no =bond_xmit_hash (bond, skb, slaves_in_agg);bond_for_each_slave (bond, slave) {struct aggregator *agg =SLAVE_AD_INFO (slave).port.aggregator;
drivers/net/bonding/bond_main.c +68 -129
269 lines not on this page · raw message
drivers/net/bonding/bond_sysfs.c +0 -2
@@ -352,7 +352,6 @@ static ssize_t bonding_store_mode(struct device *d,/* don't cache arp_validate between modes */ bond->params.arp_validate =BOND_ARP_VALIDATE_NONE ; bond->params.mode = new_value; -bond_set_mode_ops (bond, bond->params.mode);pr_info ("%s: setting mode to %s (%d).\n" , bond->dev->name, bond_mode_tbl[new_value].modename, new_value);
@@ -392,7 +391,6 @@ static ssize_t bonding_store_xmit_hash(struct device *d, ret = -EINVAL ; }else { bond->params.xmit_policy = new_value; -bond_set_mode_ops (bond, bond->params.mode);pr_info ("%s: setting xmit hash policy to %s (%d).\n" , bond->dev->name, xmit_hashtype_tbl[new_value].modename, new_value);
drivers/net/bonding/bonding.h +1 -2
@@ -245,7 +245,6 @@ struct bonding {char proc_file_name[IFNAMSIZ ];#endif /* CONFIG_PROC_FS */ struct list_head bond_list; -int (*xmit_hash_policy)(struct sk_buff *,int );u16 rr_tx_counter;struct ad_bond_info ad_info;struct alb_bond_info alb_info;
@@ -446,7 +445,7 @@ int bond_release(struct net_device *bond_dev, struct net_device *slave_dev);void bond_mii_monitor (struct work_struct *);void bond_loadbalance_arp_mon (struct work_struct *);void bond_activebackup_arp_mon (struct work_struct *); -void bond_set_mode_ops (struct bonding *bond,int mode); +int bond_xmit_hash (struct bonding *bond,struct sk_buff *skb,int count);int bond_parse_parm (const char *mode_arg,const struct bond_parm_tbl *tbl);void bond_select_active_slave (struct bonding *bond);void bond_change_active_slave (struct bonding *bond,struct slave *new_active);
include/uapi/linux/if_bonding.h +2 -0
@@ -91,6 +91,8 @@#define BOND_XMIT_POLICY_LAYER2 0/* layer 2 (MAC only), default */ #define BOND_XMIT_POLICY_LAYER34 1/* layer 3+4 (IP ^ (TCP || UDP)) */ #define BOND_XMIT_POLICY_LAYER23 2/* layer 2+3 (IP ^ MAC) */ +#define BOND_XMIT_POLICY_ENCAP23 3/* encapsulated layer 2+3 */ +#define BOND_XMIT_POLICY_ENCAP34 4/* encapsulated layer 3+4 */ typedef struct ifbond { __s32 bond_mode;
Signature · 1 line
1.8.1.4
PATCH v2 3/3 bonding: document the new xmit policy modes and update the changed ones
In reply to Nikolay AleksandrovAdd new documentation for encap2+3 and encap3+4, also update the formula for the old modes due to the changes.
---
v2: no change
Documentation/networking/bonding.txt | 66 ++++++++++++++++++++---------------- 1 file changed, 36 insertions(+), 30 deletions(-)
Documentation/networking/bonding.txt +36 -30
@@ -743,21 +743,16 @@ xmit_hash_policy protocol information to generate the hash. Uses XOR of hardware MAC addresses and IP addresses to - generate the hash. The IPv4 formula is + generate the hash. The formula is - (((source IP XOR dest IP) AND 0xffff) XOR - ( source MAC XOR destination MAC )) - modulo slave count + hash = source MAC XOR destination MAC + hash = hash XOR source IP XOR destination IP + hash = hash XOR (hash RSHIFT 16) + hash = hash XOR (hash RSHIFT 8) + And then hash is reduced modulo slave count. - The IPv6 formula is - - hash = (source ip quad 2 XOR dest IP quad 2) XOR - (source ip quad 3 XOR dest IP quad 3) XOR - (source ip quad 4 XOR dest IP quad 4) - - (((hash >> 24) XOR (hash >> 16) XOR (hash >> 8) XOR hash) - XOR (source MAC XOR destination MAC)) - modulo slave count + If the protocol is IPv6 then the source and destination + addresses are first hashed using ipv6_addr_hash. This algorithm will place all traffic to a particular network peer on the same slave. For non-IP traffic,
@@ -779,21 +774,16 @@ xmit_hash_policy slaves, although a single connection will not span multiple slaves. - The formula for unfragmented IPv4 TCP and UDP packets is - - ((source port XOR dest port) XOR - ((source IP XOR dest IP) AND 0xffff) - modulo slave count + The formula for unfragmented TCP and UDP packets is - The formula for unfragmented IPv6 TCP and UDP packets is + hash = source port, destination port (as in the header) + hash = hash XOR source IP XOR destination IP + hash = hash XOR (hash RSHIFT 16) + hash = hash XOR (hash RSHIFT 8) + And then hash is reduced modulo slave count. - hash = (source port XOR dest port) XOR - ((source ip quad 2 XOR dest IP quad 2) XOR - (source ip quad 3 XOR dest IP quad 3) XOR - (source ip quad 4 XOR dest IP quad 4)) - - ((hash >> 24) XOR (hash >> 16) XOR (hash >> 8) XOR hash) - modulo slave count + If the protocol is IPv6 then the source and destination + addresses are first hashed using ipv6_addr_hash. For fragmented TCP or UDP packets and all other IPv4 and IPv6 protocol traffic, the source and destination port
@@ -801,10 +791,6 @@ xmit_hash_policy formula is the same as for the layer2 transmit hash policy. - The IPv4 policy is intended to mimic the behavior of - certain switches, notably Cisco switches with PFC2 as - well as some Foundry and IBM products. - This algorithm is not fully 802.3ad compliant. A single TCP or UDP conversation containing both fragmented and unfragmented packets will see packets
@@ -815,6 +801,26 @@ xmit_hash_policy conversations. Other implementations of 802.3ad may or may not tolerate this noncompliance. + encap2+3 + + This policy uses the same formula as layer2+3 but it + relies on skb_flow_dissect to obtain the header fields + which might result in the use of inner headers if an + encapsulation protocol is used. For example this will + improve the performance for tunnel users because the + packets will be distributed according to the encapsulated + flows. + + encap3+4 + + This policy uses the same formula as layer3+4 but it + relies on skb_flow_dissect to obtain the header fields + which might result in the use of inner headers if an + encapsulation protocol is used. For example this will + improve the performance for tunnel users because the + packets will be distributed according to the encapsulated + flows. + The default value is layer2. This option was added in bonding version 2.6.3. In earlier versions of bonding, this parameter does not exist, and the layer2 policy is the only policy. The
Signature · 1 line
1.8.1.4
Self-nack on this patchset as the flow_dissector patch will make nhoff different from before, that is the port offset will not be added to it, and thus will make flow_keys' th_off different in skb_flow_dissect. Again, I will wait until tomorrow to see if there're any other comments and will re-post a v3 with this bug fixed.
Nik