Skip to content

net: openvswitch: Decouple flow operations from RTNL

Adrian Moreno lkml netdev 2 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v4 Latest4 patches 4 repliesIn review
  2. v32 patches 6 repliesSuperseded

Changes in v4 from the cover letter

- Further split patches: move lockdep checks to an independent patch (Paolo)
- Add an additional patch to reduce the nested rcu callback scheduling (Paolo)
- consolidate flow_table_put after if/else (Paolo)

Patches in v4 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/4net: openvswitch: make flow_table an rcu pointernet/openvswitch/datapath.c net/openvswitch/datapath.h net/openvswitch/flow_table.c +1 more+105 -381 reply
2/4net: openvswitch: add per-flow_table lockdep checksnet/openvswitch/datapath.c net/openvswitch/flow.c net/openvswitch/flow.h +2 more+137 -851 reply
3/4net: openvswitch: decouple flow_table from ovs_mutexnet/openvswitch/datapath.c net/openvswitch/flow_table.c net/openvswitch/flow_table.h+181 -761 reply
4/4net: openvswitch: avoid double-rcu wait periodnet/openvswitch/flow_table.c+15 -121 reply

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v4 yet.

Read the v4 thread

Cover letter [PATCH net-next v4 0/4] net: openvswitch: Decouple flow operations from RTNL

When RTNL is contended, network-related control-plane operations can be delayed.

In such scenario, if OVS control-plane operations (such as vport creation) take a bit longer, it's acceptable. However, flow installation operations happen as part of upcall processing, executed in the context of of handler threads. If they get delayed, it affects the data-plane and can even result in packet drops.

Because flow operations also use ovs_mutex for concurrency protection and given RTNL can nest under ovs_mutex, contention can be easily transferred from RTNL to ovs_mutex, causing delay in flow operations.

In order to protect flow operations from RTNL delays, this series decouples them from ovs_mutex. First, the flow_table is converted into an rcu-protected pointer. Then two locking mechanisms are introduced: a per-table mutex and a refcount.

The mutex protects the flow_table against concurrent modifications, while the refcount is used to extend the lifetime of the flow_table beyond the rcu read-protected region used to dereference it.

The following is an example of how concurrent datapath deletion and flow modifcation would work.
Datapath deletion:

  ovs_lock();
  table = rcu_dereference_protected(dp->table, ...);
  rcu_assign_pointer(dp->table, NULL);
  ovs_flow_tbl_put(table);
  ovs_unlock();

Flow modification:

  rcu_read_lock();
  dp = get_dp(...);
  table = rcu_dereference(dp->table);
  ovs_flow_tbl_get(table);
  rcu_read_unlock();

  mutex_lock(&table->lock);
  /* Perform modifications on the flow_table */
  mutex_unlock(&table->lock);
  ovs_flow_tbl_put(table);

v4:
- Further split patches: move lockdep checks to an independent patch (Paolo)
- Add an additional patch to reduce the nested rcu callback scheduling (Paolo)
- consolidate flow_table_put after if/else (Paolo)

v3:
- Split in 2 patches (Paolo)
- Improve locking in get_dp_stats (Paolo and Sashiko)
- Use __always_unused in lockdep stubs (Paolo)
- Use READ_ONCE/WRITE_ONCE for table->count (Aaron)
- Take a reference in ovs_dp_masks_rebalance (Aaron)

v2: Fix argument in ovs_flow_tbl_put (sparse)
Remove rcu checks in ovs_dp_masks_rebalance

Adrian Moreno (4):
net: openvswitch: make flow_table an rcu pointer
net: openvswitch: add per-flow_table lockdep checks
net: openvswitch: decouple flow_table from ovs_mutex
net: openvswitch: avoid double-rcu wait period

 net/openvswitch/datapath.c   | 294 ++++++++++++++++++++++++-----------
 net/openvswitch/datapath.h   |   2 +-
 net/openvswitch/flow.c       |  13 +-
 net/openvswitch/flow.h       |   9 +-
 net/openvswitch/flow_table.c | 211 ++++++++++++++++---------
 net/openvswitch/flow_table.h |  56 ++++++-
 6 files changed, 406 insertions(+), 179 deletions(-)
Signature · 1 line
2.54.0