Skip to content

devlink: add generic device max_sfs parameter

Tariq Toukan linux-doc linux-rdma lkml netdev 5 versions – Applied

Status from this message: This series was applied to netdev/net-next.git (main)

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v5 Latest2 patches 2 repliesApplied 2/2
  2. v42 patches 1 replySuperseded
  3. v32 patches 2 repliesSuperseded
Earlier versions · 2
VersionPostedPatchesRepliesStatus
v22 patches2 repliesSuperseded
v12 patches3 repliesSuperseded

Changes in v5 from the cover letter

- rebase again
- patch 02: always enable per_pf_num_sf when setting SFs > 0 because other PFs might be using SFs as well (sashiko)
- patch 02: use NL_SET_ERR_MSG_MOD in max_sfs set

Patches in v5 All 2 applied

#Subject and filesSizeRepliesApplied
1/2devlink: add generic device max_sfs parameterDocumentation/networking/devlink/devlink-params.rst include/net/devlink.h net/devlink/param.c+15 -00 repliesApplied
2/2net/mlx5: implement max_sfs parameterDocumentation/networking/devlink/mlx5.rst drivers/net/ethernet/mellanox/mlx5/core/lib/nv_param.c+128 -42 repliesApplied

Reviews trailers per patch

All 2 patches have a Reviewed-by or Acked-by.

PatchDavid AhernJiri PirkoAleksandr LoktionovAlexander Lobakin
1/2R-b R-b R-b R-b
2/2R-b R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 4 reviewers.

Read the v5 thread

Cover letter [PATCH net-next V5 0/2] devlink: add generic device max_sfs parameter

Hi,

This series by Nikolay introduces a new generic devlink device parameter, max_sfs, to control the number of light-weight NIC subfunctions (SFs) that can be created on a device.

The first patch adds the generic devlink parameter and infrastructure support.
The second patch implements support for the parameter in the mlx5 driver.

With this addition, users can enable or disable SF creation directly via devlink, without relying on external vendor-specific tools.

Regards,
Tariq

Notes about a few items from a previous Sashiko review:

Quoted 4 lines

Should there be a validate callback analogous to mlx5_devlink_total_vfs_validate() that reads the cap and rejects values above the device-reported maximum, so the documented "device-specific max" is actually enforced?

N: I don't know of such cap, if there was I'd have added it.

Quoted 7 lines

mlx5_devlink_total_vfs_set() rejects with -EOPNOTSUPP and an extack "SRIOV is not per PF on this device" when sriov_support or per_pf_total_vf_supported is clear, but no equivalent per_pf_num_sf_supported (or any SF-related) capability bit is added to nv_global_pci_cap_bits or queried here. On hardware that lacks the feature, the user only sees the generic firmware error "Failed to change ... global PCI configuration".

N: I don't know of such bit, if there was such bit I'd have added it.

Quoted 10 lines

At this point, a successful write of per_pf_num_sf has already been committed to non-volatile firmware storage. If the subsequent mlx5_nv_param_read_per_host_pf_conf() or the second mlx5_nv_param_write() fails (for example, transient firmware/PCIe issue), is there a path that rolls back per_pf_num_sf?

Because these are permanent parameters that "require a reboot to take effect", a half-applied state (e.g., per_pf_num_sf=1 but pf_total_sf_en=0/total_sf=0) appears to persist across reboots until the user issues another successful set.

N: That is expected and in line with the rest of the code.

Quoted 5 lines

The commit message says max_sfs is to "control the total light-weight NIC subfunctions"; the BAR-size side-effect is not mentioned, and any previously configured log_sf_bar_size is overwritten on every max_sfs set. Should that behavior be documented or split out from the count knob?

N: It should be documented when it is split out, at present time - no.

Quoted 8 lines

Does this unconditionally overwrite any existing configuration for the SubFunction BAR size?
When a user configures a custom log_sf_bar_size via firmware tools like mlxconfig, and then later uses devlink to change max_sfs, it appears this line discards any non-zero value already present in the configuration data and forcibly replaces it with MLX5_DEFAULT_LOG_SF_BAR_SIZE. Should this code preserve the existing log_sf_bar_size if it was already configured?

N: Yes, it does and it is expected and intentional.

V5:
- rebase again
- patch 02: always enable per_pf_num_sf when setting SFs > 0 because other PFs might be using SFs as well (sashiko)
- patch 02: use NL_SET_ERR_MSG_MOD in max_sfs set

V4:
https://lore.kernel.org/netdev/[email protected]/
- rebase due to build failure because of commit:
d603517771d8 ("devlink: pass param values by pointer") fixed in patch 02
- added Alexander Lobakin's reviewed-by tags (please let me know if that is not ok, the change in patch 02 is minor and mechanical due to the commit above)

V3, all changes are in patch 02 (mlx5 implementation):
https://lore.kernel.org/netdev/[email protected]
- Cap max_sfs at U16_MAX using a validate callback (sashiko)
- Change the warning message to match the docs (sashiko)
- On get verify that per_pf_num_sf & pf_total_sf_en are set otherwise return 0 since it means SFs are not properly enabled (sashiko)
- Define the default log bar size and use it instead of raw value (sashiko)

V2:
https://lore.kernel.org/netdev/[email protected]/
- Add missing ` (Aleksandr Loktionov).
- Add review tag to patch 1.

V1:
https://lore.kernel.org/all/[email protected]/

Nikolay Aleksandrov (2):
devlink: add generic device max_sfs parameter
net/mlx5: implement max_sfs parameter

 .../networking/devlink/devlink-params.rst     |   6 +
 Documentation/networking/devlink/mlx5.rst     |   7 +-
 .../mellanox/mlx5/core/lib/nv_param.c         | 125 +++++++++++++++++-
 include/net/devlink.h                         |   4 +
 net/devlink/param.c                           |   5 +
 5 files changed, 143 insertions(+), 4 deletions(-)

base-commit: b0057c68df711bf6a62033c072ac61c4f9d3cbc1

Signature · 1 line
2.44.0