Skip to content

devlink rate police limiter

Dima Chumak netdev 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest5 patches 15 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/5devlink: Introduce limit_type attr for rate objectsdrivers/net/ethernet/mellanox/mlx5/core/esw/qos.c include/net/devlink.h include/uapi/linux/devlink.h +1 more+140 -300 replies
1/5uapi: devlink.h DEVLINK_ATTR_RATE_LIMIT_TYPEinclude/uapi/linux/devlink.h+7 -00 replies
2/5devlink: Introduce police rate limit typeinclude/net/devlink.h include/uapi/linux/devlink.h net/core/devlink.c+268 -30 replies
2/5devlink: Add port rate limit_type supportdevlink/devlink.c man/man8/devlink-rate.8+120 -180 replies
3/5netdevsim: Support devlink rate limit_type policedrivers/net/netdevsim/dev.c drivers/net/netdevsim/netdevsim.h+207 -150 replies
3/5utils: Add get_size64()include/utils.h lib/utils_math.c+31 -00 replies
4/5selftest: netdevsim: Add devlink rate police sub-testtools/testing/selftests/drivers/net/netdevsim/devlink.sh+200 -150 replies
4/5uapi: devlink.h DEVLINK_RATE_LIMIT_TYPE_POLICEinclude/uapi/linux/devlink.h+8 -00 replies
5/5Documentation: devlink rate objects limit_typeDocumentation/networking/devlink/devlink-port.rst Documentation/networking/devlink/netdevsim.rst+43 -40 replies
5/5devlink: Introduce port rate limit_type policedevlink/devlink.c man/man8/devlink-rate.8+401 -60 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH net-next 0/5] devlink rate police limiter

Currently, kernel provides a way to limit tx rate of a VF via devlink rate function of a port. The underlying mechanism is a shaper applied to all traffic passing through the target VF or a group of VFs. By its essence, a shaper naturally works with outbound traffic, and in practice, it's rarely seen to be implemented for inbound traffic. Nevertheless, there is a user request to have a mechanism for limiting inbound traffic as well. It is usually done by using some form of traffic policing, dropping excess packets over the configured limit that set by a user. Thus, introducing another limiting mechanism to the port function can help close this gap.

This series introduces devlink attrs, along with their ops, to manage rate policing of a single port as well as a port group. It is based on the existing notion of leaf and node rate objects, and extends their attributes to support both RX and TX limiting, for a number of packets per second and/or a number of bytes per second. Additionally, there is a second set of parameters for specifying the size of buffering performed, called "burst", that controls the allowed level of spikes in traffic before it starts getting dropped.

A new sub-type of a devlink_rate object is introduced, called "limit_type". It can be either "shaping", the default, or "police". A single leaf or a node object can be switched from one limit type to another, but it cannot do both types of rate limiting simultaneously. A node and a leaf object that have parent-child relationship must have the same limit type. In other words, it's only possible to group rate objects of the same limit type as their group's limit_type.

devlink_ops extended with following callbacks:
- rate_{leaf|node}_tx_{burst|pkts|pkts_burst}_set
- rate_{leaf|node}_rx_{max|burst|pkts|pkts_burst}_set

UAPI provides:
- setting tx_{burst|pkts|pkts_burst} and rx_{max|burst|pkts|pkts_burst} of a rate object

Added devlink_rate police attrs support for netdevsim driver.

Issues/open questions:
- Current implementation requires a user to set both "rate" and "burst" parameters explicitly, in order to activate police rate limiting. For example, "rx_max 200Mbit rx_burst 16mb". Is it necessary to automagically deduce "burst" value when it's omitted by the user? For example when user only sets "rx_max 200Mbit".
- If answer is positive to the first question, at which level it's better to be done, at user-space iproute2, at kernel devlink core or at vendor driver that implements devlink_ops for police attrs?

CLI examples:

  $ devlink port function rate show
  netdevsim/netdevsim10/128: type leaf limit_type unset
  netdevsim/netdevsim10/129: type leaf limit_type unset
  netdevsim/netdevsim10/130: type leaf limit_type unset

# Set police rate limiting of inbound traffic

  $ devlink port function rate set netdevsim/netdevsim10/128 \
            limit_type police rx_max 100mbit rx_burst 10mbit
  $ devlink port function rate show
  netdevsim/netdevsim10/128: type leaf limit_type police rx_max 100Mbit rx_burst 10485Kbit

# Set shaping rate limiting of outbound traffic (default limit_type)

  $ devlink port function rate set netdevsim/netdevsim10/129 tx_max 200mbit
  $ devlink port function rate show
  netdevsim/netdevsim10/129: type leaf limit_type shaping tx_max 200Mbit

# Attempt to set police attr with the default shaping limit_type

  $ devlink port function rate set netdevsim/netdevsim10/129 rx_max 400mbit
  Unsupported option "rx_max" for limit_type "shaping"

# Set police rate attr for a port that already has active shaping

  $ devlink port function rate set netdevsim/netdevsim10/129 limit_type police rx_max 400mbit
  Error: devlink: Cannot change limit_type of the rate leaf object, reset current rate attributes first.
  kernel answers: Device or resource busy

# Create a rate group

  $ devlink port function rate add netdevsim/netdevsim10/g1 \
            limit_type police rx_max 1Gbit
  $ devlink port function rate show
  netdevsim/netdevsim10/g1: type node limit_type police rx_max 1Gbit

# Add port to the group

  $ devlink port function rate set netdevsim/netdevsim10/128 parent g1
  $ devlink port function rate show
  netdevsim/netdevsim10/g1: type node limit_type police rx_max 1Gbit
  netdevsim/netdevsim10/128: type leaf limit_type police rx_max 100Mbit rx_burst 10485Kbit parent g1
  netdevsim/netdevsim10/129: type leaf limit_type shaping tx_max 200Mbit
  netdevsim/netdevsim10/130: type leaf limit_type unset

# Try to add a port with a non-matching limit_type to the group

  $ devlink port function rate set netdevsim/netdevsim10/129 parent g1
  Error: devlink: Parent and object should be of the same limit_type.
  kernel answers: Invalid argument

# Adding a port with "unset" limit_type to a group inherits the # group's limit_type

  $ devlink port function rate set netdevsim/netdevsim10/130 parent g1
  $ devlink port function rate show
  netdevsim/netdevsim10/130: type leaf limit_type police parent g1
  # Set all police parameters
  $ devlink port func rate set netdevsim/netdevsim10/130 \
            limit_type police tx_max 10GBps tx_burst 1gb \
                              rx_max 25GBps rx_burst 2gb \
                              tx_pkts 10000 tx_pkts_burst 1gb \
                              rx_pkts 20000 rx_pkts_burst 2gb

Dima Chumak (5):
devlink: Introduce limit_type attr for rate objects
devlink: Introduce police rate limit type
netdevsim: Support devlink rate limit_type police
selftest: netdevsim: Add devlink rate police sub-test
Documentation: devlink rate objects limit_type

 .../networking/devlink/devlink-port.rst       |  44 ++-
 .../networking/devlink/netdevsim.rst          |   3 +-
 .../net/ethernet/mellanox/mlx5/core/esw/qos.c |  28 +-
 drivers/net/netdevsim/dev.c                   | 211 ++++++++++-
 drivers/net/netdevsim/netdevsim.h             |  11 +-
 include/net/devlink.h                         |  52 ++-
 include/uapi/linux/devlink.h                  |  15 +
 net/core/devlink.c                            | 336 ++++++++++++++++--
 .../drivers/net/netdevsim/devlink.sh          | 215 ++++++++++-
 9 files changed, 853 insertions(+), 62 deletions(-)
Signature · 1 line
2.36.1