Skip to content

RFC psp: use virt cookie as Rx steering hint

Jakub Kicinski netdev 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest6 patches 11 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/6psp: steer Rx queues with the virtualization cookieDocumentation/netlink/specs/psp.yaml Documentation/networking/psp.rst drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp_rxtx.c +8 more+373 -265 replies
2/6netdevsim: support PSP VC based queue steeringdrivers/net/netdevsim/netdev.c drivers/net/netdevsim/netdevsim.h drivers/net/netdevsim/psp.c+29 -60 replies
3/6selftests: drv-net: psp: move the PSP test plumbing into psp_lib.pyMAINTAINERS tools/testing/selftests/drivers/net/Makefile tools/testing/selftests/drivers/net/psp.py +1 more+265 -2060 replies
4/6selftests: drv-net: psp_steer: test PSP VC based queue steeringtools/testing/selftests/drivers/net/Makefile tools/testing/selftests/drivers/net/psp_lib.py tools/testing/selftests/drivers/net/psp_responder.c +1 more+304 -00 replies
5/6selftests: drv-net: psp_steer: test where PSP steering sits in the Rx pipelinetools/testing/selftests/drivers/net/psp_steer.py+164 -70 replies
6/6selftests: drv-net: psp_steer: cover corner cases and racestools/testing/selftests/drivers/net/psp_steer.py+70 -100 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [RFC net-next 0/6] psp: use virt cookie as Rx steering hint

Hi!

This PoC series uses a field of the PSP header intended for tunnels to auto-steer Rx traffic. Various attempts have been made at trying to get Rx traffic to land close to the core where the application runs. By default RSS picks the Rx queue based on the flow hash. I'm not going to cover all previous solutions in detail but broadly
- we have RFS in SW which looks on which CPU Tx happens and backlogs Rx packets there, it is quite efficient. aRFS is built on top of RFS but tries to program flows into the NIC. Some NICs have a "cache" and try to automatically remember the flow to queue association.

All those solutions are entirely local to the receiver. Ideally we would want the solution to look something like TCP timestamp option - we send an opaque cookie to the peer, and the peer echoes it back to us. Our NIC can steer based on that echoed cookie.

This patch set implements exactly that using the optional PSP Virtualization Cookie field. The PSP standard doesn't have much to say about this field:

Virtualization Cookie - 64b
An optional field, present if and only if V is set.
It may contain a Virtual Network Identifier (VNI) or other data, as defined by the implementation.

IOW it's a field in the header which can be used as VNI, so presumably most PSP-capable NICs will be able to feed it into some TCAM lookup. The main use of this field is when PSP is used for tunneling. Nothing in the standard precludes it's use in transport mode.

This patchset splits this field as follows:

63 48 47 32 31 16 15 0

  +---------------+---------------+---------------+---------------+
  |    reserved   |    req qid    |    reserved   |    dst qid    |
  +---------------+---------------+---------------+---------------+

"req" is the value we want sender to put in "dst" when they respond.

When the feature is enabled we expect the NIC to create low-priority steering rules matching on "dst" (bottom 16b of the Virt Cookie). The mapping is direct today so dst=2 means queue=2 within the receiving interface (see doc in patch 1 for more info). Any explicit steering rules (ethtool, TC etc) still take precedence over PSP steering, we are only overriding the RSS queue assignment.

I'm sharing this as an RFC because I _think_ it's a good idea (feedback most welcome). We need some vendor cooperation to get this implemented - specifically IDK how to make mlx5 (the only PSP-capable NIC I have access to) to do the steering :(

Jakub Kicinski (6):
psp: steer Rx queues with the virtualization cookie
netdevsim: support PSP VC based queue steering
selftests: drv-net: psp: move the PSP test plumbing into psp_lib.py
selftests: drv-net: psp_steer: test PSP VC based queue steering
selftests: drv-net: psp_steer: test where PSP steering sits in the Rx
pipeline
selftests: drv-net: psp_steer: cover corner cases and races

 MAINTAINERS                                   |   1 +
 Documentation/netlink/specs/psp.yaml          |  42 ++
 Documentation/networking/psp.rst              |  75 +++
 tools/testing/selftests/drivers/net/Makefile  |   5 +
 include/net/psp/types.h                       |  93 +++-
 drivers/net/netdevsim/netdevsim.h             |   4 +-
 include/net/psp/functions.h                   |  54 +-
 include/uapi/linux/psp.h                      |  17 +
 .../mellanox/mlx5/core/en_accel/psp_rxtx.c    |   2 +-
 drivers/net/netdevsim/netdev.c                |   5 +-
 drivers/net/netdevsim/psp.c                   |  26 +-
 net/psp/psp-nl-gen.c                          |   5 +-
 net/psp/psp_main.c                            |  53 +-
 net/psp/psp_nl.c                              |  21 +-
 net/psp/psp_sock.c                            |  35 ++
 .../selftests/drivers/net/psp_responder.c     |  42 ++
 tools/testing/selftests/drivers/net/psp.py    | 287 +++--------
 .../testing/selftests/drivers/net/psp_lib.py  | 184 +++++++
 .../selftests/drivers/net/psp_steer.py        | 473 ++++++++++++++++++
 19 files changed, 1187 insertions(+), 237 deletions(-)
 create mode 100644 tools/testing/selftests/drivers/net/psp_lib.py
 create mode 100644 tools/testing/selftests/drivers/net/psp_steer.py
Signature · 1 line
2.55.0