netkit: Support for io_uring zero-copy and AF_XDP
Status from this message: This series was applied to netdev/net-next.git (main)
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v11 Latest14 patches 15 repliesApplied 14/14
- v1014 patches 7 repliesSuperseded
- v914 patches 3 repliesSuperseded
Earlier versions · 8
| Version | Posted | Patches | Replies | Status |
|---|---|---|---|---|
| v8 | 16 patches | 24 replies | Superseded | |
| v7 | 16 patches | 28 replies | Applied 16/16 | |
| v6 | 16 patches | 1 reply | Superseded | |
| v5 | 16 patches | 11 replies | Superseded | |
| v4 | 14 patches | 13 replies | Superseded 2 of 14 applied | |
| v3 | 15 patches | 38 replies | Superseded | |
| v2 | 15 patches | 2 replies | Superseded | |
| v1 | 20 patches | 43 replies | Superseded |
Changes in v11 from the cover letter
- Fix missing mp_ops->uninstall upon unlease path (Gemini)
- Fix dma device retrieval on tx queue when rx queue is leased (Gemini)
- Rework ethtool channel checks to check rx/tx individually (Gemini)
- The remainder of the Gemini findings were non-issues
- Add more extensive net selftests around queue leasing corner cases
- Rebase and retested everything with mlx5 + bnxt_en
Patches in v11 All 14 applied
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/14 | net: Add queue-create operationDocumentation/ include/ net/ +3 more | +95 | 0 replies | Applied |
| 2/14 | net: Implement netdev_nl_queue_create_doitDocumentation/ include/ include/ +6 more | +323 | 0 replies | Applied |
| 3/14 | net: Add lease info to queue-get responseinclude/ net/ net/ | +130 | 9 replies | Applied |
| 4/14 | net, ethtool: Disallow leased real rxqs to be resizednet/ net/ | +29 | 0 replies | Applied |
| 5/14 | net: Slightly simplify net_mp_{open,close}_rxqinclude/ io_uring/ net/ +1 more | +13 | 0 replies | Applied |
| 6/14 | net: Proxy netif_mp_{open,close}_rxq for leased queuesnet/ net/ net/ | +95 | 0 replies | Applied |
| 7/14 | net: Proxy netdev_queue_get_dma_dev for leased queuesinclude/ io_uring/ net/ +1 more | +40 | 0 replies | Applied |
| 8/14 | xsk: Extend xsk_rcv_check validationnet/ | +26 | 0 replies | Applied |
| 9/14 | xsk: Proxy pool management for leased queuesnet/ | +35 | 0 replies | Applied |
| 10/14 | netkit: Add single device mode for netkitDocumentation/ drivers/ include/ | +99 | 0 replies | Applied |
| 11/14 | netkit: Implement rtnl_link_ops->alloc and ndo_queue_createdrivers/ | +114 | 0 replies | Applied |
| 12/14 | netkit: Add netkit notifier to check for unregistering devicesdrivers/ include/ net/ | +75 | 0 replies | Applied |
| 13/14 | netkit: Add xsk support for af_xdp applicationsdrivers/ | +86 | 2 replies | Applied |
| 14/14 | selftests/net: Add queue leasing tests with netkittools/ tools/ tools/ | +1410 | 3 replies | Applied |
Reviews trailers per patch
13 of 14 patches have a Reviewed-by or Acked-by.
| Patch | Stanislav Fomichev | Nikolay Aleksandrov | Jordan Rife |
|---|---|---|---|
| 1/14 | A-b | R-b | |
| 2/14 | R-b | ||
| 3/14 | R-b | ||
| 4/14 | R-b | ||
| 5/14 | R-b | ||
| 6/14 | R-b | ||
| 7/14 | R-b | ||
| 8/14 | A-b | R-b | |
| 9/14 | A-b | R-b | |
| 10/14 | R-b | R-b | |
| 11/14 | R-b | ||
| 12/14 | R-b | ||
| 13/14 | R-b | ||
| 14/14 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 3 reviewers.
Cover letter [PATCH net-next v11 00/14] netkit: Support for io_uring zero-copy and AF_XDP
Containers use virtual netdevs to route traffic from a physical netdev in the host namespace. They do not have access to the physical netdev in the host and thus can't use memory providers or AF_XDP that require reconfiguring/restarting queues in the physical netdev.
This patchset adds the concept of queue leasing to virtual netdevs that allow containers to use memory providers and AF_XDP at native speed. Leased queues are bound to a real queue in a physical netdev and act as a proxy.
Memory providers and AF_XDP operations take an ifindex and queue id, so containers would pass in an ifindex for a virtual netdev and a queue id of a leased queue, which then gets proxied to the underlying real queue.
We have implemented support for this concept in netkit and tested the latter against Nvidia ConnectX-6 (mlx5) as well as Broadcom BCM957504 (bnxt_en) 100G NICs. For more details see the individual patches.
v10->v11:
- Fix missing mp_ops->uninstall upon unlease path (Gemini)
- Fix dma device retrieval on tx queue when rx queue is leased (Gemini)
- Rework ethtool channel checks to check rx/tx individually (Gemini)
- The remainder of the Gemini findings were non-issues
- Add more extensive net selftests around queue leasing corner cases
- Rebase and retested everything with mlx5 + bnxt_env9->v10:
- Fix netkit selftest ruff checks (Jakub)
- Pass in net arg in netdev_put_lock (Gemini)
- Make the ethtool channel busy checks direction-aware (Gemini)
- Fix list corruption in netkit_check_lease_unregister (Gemini)
- Fix netkit_set_headroom in single device mode (Gemini)
- Fix xsk_clear_pool_at_qid bounds to use num_{rx,tx}_queues (Gemini)
- Fix netkit xsk teardown state checks (Gemini)
- The remainder of the Gemini findings were non-issues
- Rebase and retested everything with mlx5 + bnxt_env8->v9:
- Use ynl --family netdev in commit desc (Jakub)
- Update docs and comments about locking (Jakub)
- Propagate extack to the driver in ndo_queue_create (Jakub)
- Move function to net/core/dev.h core where possible (Jakub)
- Add comment about lease in netdev_rx_queue struct (Jakub)
- Add min: 0 check to netns id in policy (Jakub)
- Drop ifindex == ifindex_lease test in netdev_nl_queue_create_doit (Jakub)
- Detailed extack errors in netkit's ndo_queue_create (Jakub)
- Refactor lease dump into own function in netdev_nl_queue_fill_one (Jakub)
- Dump mp and xsk info for both queues in netdev_nl_queue_fill_one (Jakub)
- Replace the net_ prefix with netif_ for net_mp_{open,close}_rxq (Jakub)
- Remove ifq_idx naming cleanup from net_mp_{open,close}_rxq (Jakub)
- Rework the mp entry points to have explicit code that deals with the lease (Jakub)
- Fix locking in netdev_queue_get_dma_dev (Jakub)
- Remove unneeded carrier flap in netkit_queue_create (Jakub)
- Drop base net selftests that were already merged
- Add more nekit queue leasing coverage of base functionality (Jakub)
- Remove any mp leftovers upon unlease found via test cases
- Rebase and retested everything with mlx5 + bnxt_env7->v8:
- Rework and refactor net_mp_{open,close}_rxq patch (Jakub)
- Moved queue lease netlink command from env into test (Jakub)
- Support netns_id also in queue create call and not only dump
- Rebase and retested everything with mlx5 + bnxt_env6->v7:
- Add xsk_dev_queue_valid real_num_rx_queues check given bound xs->queue_id could be from a TX queue (Claude)
- Fix up exception path in queue leasing selftest (Claude)
- Rebase and retested everything with mlx5 + bnxt_env5->v6:
- Fix nest_queue test in netdev_nl_queue_fill_one (Jakub/Claude)
- Fix netdev notifier locking leak (Jakub/Claude)
- Drop NETREG_UNREGISTERING WARN_ON_ONCE to avoid confusion (Stan)
- Remove slipped-in .gitignore cruft in net selftest (Stan)
- Fix Pylint warnings in net selftest (Jakub)
- Rebase and retested everything with mlx5 + bnxt_env4->v5:
- Rework of the core API into queue-create op (Jakub)
- Rename from queue peering to queue leasing (Jakub)
- Add net selftests for queue leasing (Stan, Jakub)
- Move netkit_queue_get_dma_dev into core (Jakub)
- Dropped netkit_get_channels (Jakub)
- Moved ndo_queue_create back to return index or error (Jakub)
- Inline __netdev_rx_queue_{peer,unpeer} helpers (Jakub)
- Adding helpers in patches where they are used (Jakub)
- Undo inline for netdev_put_lock (Jakub)
- Factoring out checks whether device can lease (Jakub)
- Fix up return codes in netdev_nl_bind_queue_doit (Jakub)
- Reject when AF_XDP or mp already bound (Jakub)
- Switch some error cases to NL_SET_BAD_ATTR() (Jakub)
- Rebase and retested everything with mlx5 + bnxt_env3->v4:
- ndo_queue_create store dst queue via arg (Nikolay)
- Small nits like a spelling issue + rev xmas (Nikolay)
- admin-perm flag in bind-queue spec (Jakub)
- Fix potential ABBA deadlock situation in bind (Jakub, Paolo, Stan)
- Add a peer dev_tracker to not reuse the sysfs one (Jakub)
- New patch (12/14) to handle the underlying device going away (Jakub)
- Improve commit message on queue-get (Jakub)
- Do not expose phys dev info from container on queue-get (Jakub)
- Add netif_put_rx_queue_peer_locked to simplify code (Stan)
- Rework xsk handling to simplify the code and drop a few patches
- Rebase and retested everything with mlx5 + bnxt_env2->v3:
- Use netdev_ops_assert_locked instead of netdev_assert_locked (syzbot)
- Add missing netdev_lockdep_set_classes in netkitv1->v2:
- Removed bind sample ynl code (Stan)
- Reworked netdev locking to have consistent order (Stan, Kuba)
- Return 'not supported' in API patch (Stan)
- Improved ynl documentation (Kuba)
- Added 'max: s32-max' in ynl spec for ifindex (Kuba)
- Added also queue type in ynl to have user specify rx to make it obvious (Kuba)
- Use of netdev_hold (Kuba)
- Avoid static inlines from another header (Kuba)
- Squashed some commits (Kuba, Stan)
- Removed ndo_{peer,unpeer}_queues callback and simplified
code (Kuba)
- Improved commit messages (Toke, Kuba, Stan, zf)
- Got rid of locking genl_sk_priv_get (Stan)
- Removed af_xdp cleanup churn (Maciej)
- Added netdev locking asserts (Stan)
- Reject ethtool ioctl path queue resizing (Kuba)
- Added kdoc for ndo_queue_create (Stan)
- Uninvert logic in netkit single dev mode (Jordan)
- Added binding support for multiple queues
Daniel Borkmann (10):
net: Add queue-create operation
net: Implement netdev_nl_queue_create_doit
net: Add lease info to queue-get response
net, ethtool: Disallow leased real rxqs to be resized
net: Slightly simplify net_mp_{open,close}_rxq
xsk: Extend xsk_rcv_check validation
xsk: Proxy pool management for leased queues
netkit: Add single device mode for netkit
netkit: Add netkit notifier to check for unregistering devices
netkit: Add xsk support for af_xdp applications
David Wei (4):
net: Proxy netif_mp_{open,close}_rxq for leased queues
net: Proxy netdev_queue_get_dma_dev for leased queues
netkit: Implement rtnl_link_ops->alloc and ndo_queue_create selftests/net: Add queue leasing tests with netkit
Documentation/netlink/specs/netdev.yaml | 46 + Documentation/netlink/specs/rt-link.yaml | 11 + Documentation/networking/netdevices.rst | 6 + drivers/net/netkit.c | 412 ++++- include/linux/netdevice.h | 11 +- include/net/netdev_queues.h | 23 +- include/net/netdev_rx_queue.h | 29 +- include/net/page_pool/memory_provider.h | 8 +- include/uapi/linux/if_link.h | 6 + include/uapi/linux/netdev.h | 11 + io_uring/zcrx.c | 12 +- net/core/dev.c | 18 +- net/core/dev.h | 12 + net/core/devmem.c | 6 +- net/core/netdev-genl-gen.c | 20 + net/core/netdev-genl-gen.h | 2 + net/core/netdev-genl.c | 238 ++- net/core/netdev_queues.c | 103 +- net/core/netdev_rx_queue.c | 205 ++- net/ethtool/channels.c | 28 +- net/ethtool/ioctl.c | 21 +- net/xdp/xsk.c | 76 +- tools/include/uapi/linux/netdev.h | 11 + .../testing/selftests/drivers/net/hw/Makefile | 1 + .../drivers/net/hw/lib/py/__init__.py | 4 +- .../selftests/drivers/net/hw/nk_qlease.py | 1407 +++++++++++++++++ 26 files changed, 2558 insertions(+), 169 deletions(-) create mode 100755 tools/testing/selftests/drivers/net/hw/nk_qlease.py
Signature · 1 line
2.43.0