net: Pass full skb hash to ndo_rx_flow_steer

Currently, for aRFS the index into the flow table is passed to ndo_rx_flow_steer as the flow ID of a connection. This is the skb->hash & the table mask. It looks like the backend can accept the full skb->hash as the flow ID which should reduce the number of collisions in the hardware tables.

This patch provides the skb->hash to the driver for flow steering. Expiration of HW steered flows was also updated.

With a hash collision in RFS, ndo_rx_flow_steer will continue to be called with different CPUs, but now with different flow_ids. If this is still too much device interaction, then it might make sense for the driver to do its own lookup in its structure to see if a matching filter is already installed for a given flow_id, an if it is just refresh a timestamp to avoid expiration (based on looking at sfc driver).

I don't currently have any HW to test this, if someone could try this on hardware with aRFS and provide feedback that would be appreciated.

Signed-off-by Tom Herbert <[email protected]>

Currently, for aRFS the index into the flow table is passed to ndo_rx_flow_steer as the flow ID of a connection. This is the skb->hash & the table mask. It looks like the backend can accept the full skb->hash as the flow ID which should reduce the number of collisions in the hardware tables.

This patch provides the skb->hash to the driver for flow steering. Expiration of HW steered flows was also updated.

With a hash collision in RFS, ndo_rx_flow_steer will continue to be called with different CPUs, but now with different flow_ids. If this is still too much device interaction, then it might make sense for the driver to do its own lookup in its structure to see if a matching filter is already installed for a given flow_id, an if it is just refresh a timestamp to avoid expiration (based on looking at sfc driver).

I don't currently have any HW to test this, if someone could try this on hardware with aRFS and provide feedback that would be appreciated.

Signed-off-by Tom Herbert <[email protected]>

---
 net/core/dev.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

net/core/dev.c +0 -0

index 1ab168e..cb1e06d 100644

net/core/dev.c +0 -1

                cpu = ACCESS_ONCE(rflow->cpu);
                if (rflow->filter == filter_id && cpu != RPS_NO_CPU &&
                    ((int)(per_cpu(softnet_data, cpu).input_queue_head -
--

2.1.0.rc2.206.gedb03e5

Quoted 30 lines · Tom Herbert

Currently, for aRFS the index into the flow table is passed to ndo_rx_flow_steer as the flow ID of a connection. This is the skb->hash & the table mask. It looks like the backend can accept the full skb->hash as the flow ID which should reduce the number of collisions in the hardware tables.

This patch provides the skb->hash to the driver for flow steering. Expiration of HW steered flows was also updated.

With a hash collision in RFS, ndo_rx_flow_steer will continue to be called with different CPUs, but now with different flow_ids. If this is still too much device interaction, then it might make sense for the driver to do its own lookup in its structure to see if a matching filter is already installed for a given flow_id, an if it is just refresh a timestamp to avoid expiration (based on looking at sfc driver).

I don't currently have any HW to test this, if someone could try this on hardware with aRFS and provide feedback that would be appreciated.

Signed-off-by Tom Herbert <[email protected]>

---
 net/core/dev.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

net/core/dev.c +0 -0

index 1ab168e..cb1e06d 100644

net/core/dev.c set_rps_cpu(struct net_device *dev, struct sk_buff *skb, · line 3061

Quoted 6 lines

net/core/dev.c +2 -1

@@ -3057,7 +3057,8 @@ set_rps_cpu(struct net_device *dev, struct sk_buff *skb,
                        goto out;
                flow_id = skb_get_hash(skb) & flow_table->mask;
                rc = dev->netdev_ops->ndo_rx_flow_steer(dev, skb,
-                                                       rxq_index, flow_id);
+                                                       rxq_index,
+                                                       skb_get_hash(skb));

Can gcc CSE this? Not that it matters that much.

net/core/dev.c bool rps_may_expire_flow(struct net_device *dev, u16 rxq_index, · line 3200

Quoted 11 lines

net/core/dev.c +2 -2

                if (rc < 0)
                        goto out;
                old_rflow = rflow;

@@ -3195,8 +3196,8 @@ bool rps_may_expire_flow(struct net_device *dev, u16 rxq_index,
 
        rcu_read_lock();
        flow_table = rcu_dereference(rxqueue->rps_flow_table);
-       if (flow_table && flow_id <= flow_table->mask) {
-               rflow = &flow_table->flows[flow_id];
+       if (flow_table) {
+               rflow = &flow_table->flows[flow_id & flow_table->mask];

I think this is nicer, but why will it help? If there's a collision in the low bits of the hash, we'll still think that the flow is expired.

Is there a real LRU-ish hash table that could be subbed in easily?

There ought to be a data structure that's barely more complicated than this kind of hash table but that gives much better behavior when the number of flows is smaller than the table size.

--Andy

Quoted 6 lines

net/core/dev.c +0 -1

                cpu = ACCESS_ONCE(rflow->cpu);
                if (rflow->filter == filter_id && cpu != RPS_NO_CPU &&
                    ((int)(per_cpu(softnet_data, cpu).input_queue_head -
--

2.1.0.rc2.206.gedb03e5

--
Andy Lutomirski
AMA Capital Management, LLC