mlx5 updates 2020-12-01
Subsystems (5)
From: Saeed Mahameed <[email protected]>
Hi Jakub,
v1->v2: Removed merge commit of mlx5-next.
v2->v3:
- Add accuracy improvement measurements.
- Apply the accurate stamping only on PTP port and not all UDP.
This series adds port tx timestamping support and some misc updates. For more information please see tag log below.
Please pull and let me know if there is any problem.
Thanks,
Saeed.
---
The following changes since commit 8e98387b16b88440b06e57965f6b2d789acd9451:
Merge branch 'mlx5-next' of git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux (2020-12-07 18:36:21 -0800)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux.git tags/mlx5-updates-2020-12-01
for you to fetch changes up to 2f6b379cca4cdf5e04f759c2a15933f82dc6ab0b:
net/mlx5e: Fill mlx5e_create_cq_param in a function (2020-12-08 11:28:49 -0800)
----------------------------------------------------------------
mlx5-updates-2020-12-01
mlx5e port TX timestamping support and MISC updates
1) Add support for port TX timestamping, for better PTP accuracy.
Currently in mlx5 HW TX timestamping is done on CQE (TX completion) generation, which much earlier than when the packet actually goes out to the wire, in this series Eran implements the option to do timestamping on the port using a special SQ (Send Queue), such Send Queue will generate 2 CQEs (TX completions), the original one and a new one when the packet leaves the port, due to the nature of this special handling, such mechanism is an opt-in only and it is off by default to avoid any performance degradation on normal traffic flows.
This patchset improves TX Hardware timestamping offset to be less than 40ns at a 100Gbps line rate, compared to 600ns before.
With that, making our HW compliant with G.8273.2 class C, and allow Linux systems to be deployed in the 5G telco edge, where this standard is a must.
2) Misc updates and trivial improvements.
----------------------------------------------------------------
Aya Levin (3):
net/mlx5e: Allow CQ outside of channel context net/mlx5e: Allow RQ outside of channel context net/mlx5e: Split between RX/TX tunnel FW support indication
Eran Ben Elisha (6):
net/mlx5e: Allow SQ outside of channel context net/mlx5e: Change skb fifo push/pop API to be used without SQ net/mlx5e: Split SW group counters update function net/mlx5e: Move MLX5E_RX_ERR_CQE macro
net/mlx5e: Add TX PTP port object support net/mlx5e: Add TX port timestamp support
Maxim Mikityanskiy (1):
net/mlx5e: Fill mlx5e_create_cq_param in a function
Shay Drory (1):
net/mlx5: Arm only EQs with EQEs
Tariq Toukan (1):
net/mlx5e: Free drop RQ in a dedicated function
YueHaibing (2):
net/mlx5e: Remove duplicated include
net/mlx5: Fix passing zero to 'PTR_ERR'
Zhu Yanjun (1):
net/mlx5e: remove unnecessary memset
drivers/net/ethernet/mellanox/mlx5/core/Makefile | 2 +- drivers/net/ethernet/mellanox/mlx5/core/en.h | 63 ++- drivers/net/ethernet/mellanox/mlx5/core/en/fs.h | 3 +- .../net/ethernet/mellanox/mlx5/core/en/health.c | 16 +- .../net/ethernet/mellanox/mlx5/core/en/health.h | 7 +- .../net/ethernet/mellanox/mlx5/core/en/params.h | 10 + drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c | 529 +++++++++++++++++++++ drivers/net/ethernet/mellanox/mlx5/core/en/ptp.h | 63 +++ .../ethernet/mellanox/mlx5/core/en/reporter_rx.c | 52 +- .../ethernet/mellanox/mlx5/core/en/reporter_tx.c | 215 +++++++-- drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h | 19 +- .../net/ethernet/mellanox/mlx5/core/en/xsk/setup.c | 9 +- .../mellanox/mlx5/core/en_accel/tls_rxtx.c | 2 +- .../net/ethernet/mellanox/mlx5/core/en_ethtool.c | 33 ++ drivers/net/ethernet/mellanox/mlx5/core/en_fs.c | 20 +- drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 253 ++++++---- drivers/net/ethernet/mellanox/mlx5/core/en_rx.c | 29 +- drivers/net/ethernet/mellanox/mlx5/core/en_stats.c | 403 +++++++++++----- drivers/net/ethernet/mellanox/mlx5/core/en_stats.h | 11 + drivers/net/ethernet/mellanox/mlx5/core/en_tx.c | 84 +++- drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c | 5 +- drivers/net/ethernet/mellanox/mlx5/core/eq.c | 6 +- .../mellanox/mlx5/core/esw/acl/egress_lgcy.c | 2 +- .../mellanox/mlx5/core/esw/acl/egress_ofld.c | 2 +- .../mellanox/mlx5/core/esw/acl/ingress_lgcy.c | 2 +- .../mellanox/mlx5/core/esw/acl/ingress_ofld.c | 2 +- .../ethernet/mellanox/mlx5/core/eswitch_offloads.c | 1 - 27 files changed, 1493 insertions(+), 350 deletions(-) create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en/ptp.h
PATCH v3 4/15 net/mlx5e: Allow SQ outside of channel context
From: Eran Ben Elisha <[email protected]>
In order to be able to create an SQ outside of a channel context, remove sq->channel direct pointer. This requires adding a direct pointer to:
netdevice, priv and mlx5_core in order to support SQs that are part of mlx5e_channel. Use channel_stats from the corresponding CQ.
--- drivers/net/ethernet/mellanox/mlx5/core/en.h | 4 +++- .../ethernet/mellanox/mlx5/core/en/health.c | 4 +--- .../ethernet/mellanox/mlx5/core/en/health.h | 2 +- .../mellanox/mlx5/core/en/reporter_rx.c | 2 +- .../mellanox/mlx5/core/en/reporter_tx.c | 20 +++++++++---------- .../mellanox/mlx5/core/en_accel/tls_rxtx.c | 2 +- .../net/ethernet/mellanox/mlx5/core/en_main.c | 9 +++++---- 7 files changed, 22 insertions(+), 21 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en.h +3 -1
@@ -369,10 +369,12 @@ struct mlx5e_txqsq {unsigned int hw_mtu;struct hwtstamp_config *tstamp;struct mlx5_clock *clock; +struct net_device *netdev; +struct mlx5_core_dev *mdev; +struct mlx5e_priv *priv;/* control path */ struct mlx5_wq_ctrl wq_ctrl; -struct mlx5e_channel *channel;int ch_ix;int txq_ix;u32 rate_limit;
drivers/net/ethernet/mellanox/mlx5/core/en/health.c +1 -3
@@ -157,10 +157,8 @@ void mlx5e_health_channels_update(struct mlx5e_priv *priv) DEVLINK_HEALTH_REPORTER_STATE_HEALTHY); } -int mlx5e_health_sq_to_ready (struct mlx5e_channel *channel,u32 sqn) +int mlx5e_health_sq_to_ready (struct mlx5_core_dev *mdev,struct net_device *dev,u32 sqn) { -struct mlx5_core_dev *mdev = channel->mdev; -struct net_device *dev = channel->netdev;struct mlx5e_modify_sq_param msp = {};int err;
drivers/net/ethernet/mellanox/mlx5/core/en/health.h +1 -1
@@ -42,7 +42,7 @@ struct mlx5e_err_ctx {void *ctx; }; -int mlx5e_health_sq_to_ready (struct mlx5e_channel *channel,u32 sqn); +int mlx5e_health_sq_to_ready (struct mlx5_core_dev *mdev,struct net_device *dev,u32 sqn);int mlx5e_health_channel_eq_recover (struct net_device *dev,struct mlx5_eq_comp *eq,struct mlx5e_ch_stats *stats);int mlx5e_health_recover_channels (struct mlx5e_priv *priv);
drivers/net/ethernet/mellanox/mlx5/core/en/reporter_rx.c +1 -1
@@ -87,7 +87,7 @@ static int mlx5e_rx_reporter_err_icosq_cqe_recover(void *ctx)/* At this point, both the rq and the icosq are disabled */ - err =mlx5e_health_sq_to_ready (icosq->channel, icosq->sqn); + err =mlx5e_health_sq_to_ready (mdev, dev, icosq->sqn);if (err) goto out;
drivers/net/ethernet/mellanox/mlx5/core/en/reporter_tx.c +10 -10
@@ -15,7 +15,7 @@ static int mlx5e_wait_for_sq_flush(struct mlx5e_txqsq *sq)msleep (20 ); } -netdev_err (sq->channel->netdev, +netdev_err (sq->netdev,"Wait for SQ 0x%x flush timeout (sq cc = 0x%x, sq pc = 0x%x)\n" , sq->sqn, sq->cc, sq->pc);
@@ -41,8 +41,8 @@ static int mlx5e_tx_reporter_err_cqe_recover(void *ctx)int err; sq = ctx; - mdev = sq->channel->mdev; - dev = sq->channel->netdev; + mdev = sq->mdev; + dev = sq->netdev;if (!test_bit (MLX5E_SQ_STATE_RECOVERING , &sq->state))return 0 ;
@@ -68,7 +68,7 @@ static int mlx5e_tx_reporter_err_cqe_recover(void *ctx)* pending WQEs. SQ can safely reset the SQ. */ - err =mlx5e_health_sq_to_ready (sq->channel, sq->sqn); + err =mlx5e_health_sq_to_ready (mdev, dev, sq->sqn);if (err) goto out;
@@ -99,8 +99,8 @@ static int mlx5e_tx_reporter_timeout_recover(void *ctx) to_ctx = ctx; sq = to_ctx->sq; eq = sq->cq.mcq.eq; - priv = sq->channel->priv; - err =mlx5e_health_channel_eq_recover (sq->channel->netdev, eq, sq->channel->stats); + priv = sq->priv; + err =mlx5e_health_channel_eq_recover (sq->netdev, eq, sq->cq.ch_stats);if (!err) { to_ctx->status =0 ;/* this sq recovered */ return err;
@@ -144,8 +144,8 @@ static int mlx5e_tx_reporter_build_diagnose_output(struct devlink_fmsg *fmsg, struct mlx5e_txqsq *sq, int tc) { - struct mlx5e_priv *priv = sq->channel->priv; bool stopped = netif_xmit_stopped(sq->txq); +struct mlx5e_priv *priv = sq->priv;u8 state;int err;
@@ -396,8 +396,8 @@ static int mlx5e_tx_reporter_dump(struct devlink_health_reporter *reporter,void mlx5e_reporter_tx_err_cqe (struct mlx5e_txqsq *sq) { -struct mlx5e_priv *priv = sq->channel->priv;char err_str[MLX5E_REPORTER_PER_Q_MAX_LEN ]; +struct mlx5e_priv *priv = sq->priv;struct mlx5e_err_ctx err_ctx = {}; err_ctx.ctx = sq;
@@ -410,9 +410,9 @@ void mlx5e_reporter_tx_err_cqe(struct mlx5e_txqsq *sq)int mlx5e_reporter_tx_timeout (struct mlx5e_txqsq *sq) { -struct mlx5e_priv *priv = sq->channel->priv;char err_str[MLX5E_REPORTER_PER_Q_MAX_LEN ];struct mlx5e_tx_timeout_ctx to_ctx = {}; +struct mlx5e_priv *priv = sq->priv;struct mlx5e_err_ctx err_ctx = {}; to_ctx.sq = sq;
@@ -421,7 +421,7 @@ int mlx5e_reporter_tx_timeout(struct mlx5e_txqsq *sq) err_ctx.dump = mlx5e_tx_reporter_dump_sq;snprintf (err_str,sizeof (err_str),"TX timeout on queue: %d, SQ: 0x%x, CQ: 0x%x, SQ Cons: 0x%x SQ Prod: 0x%x, usecs since last trans: %u" , - sq->channel->ix, sq->sqn, sq->cq.mcq.cqn, sq->cc, sq->pc, + sq->ch_ix, sq->sqn, sq->cq.mcq.cqn, sq->cc, sq->pc,jiffies_to_usecs (jiffies - sq->txq->trans_start));mlx5e_health_report (priv, priv->tx_reporter, err_str, &err_ctx);
drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls_rxtx.c +1 -1
@@ -276,7 +276,7 @@ bool mlx5e_tls_handle_tx_skb(struct net_device *netdev, struct mlx5e_txqsq *sq,if (WARN_ON_ONCE (tls_ctx->netdev != netdev)) goto err_out; -if (mlx5_accel_is_ktls_tx (sq->channel->mdev)) +if (mlx5_accel_is_ktls_tx (sq->mdev))return mlx5e_ktls_handle_tx_skb (tls_ctx, sq, skb, datalen, state);/* FPGA */
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +5 -4
@@ -1132,7 +1132,9 @@ static int mlx5e_alloc_txqsq(struct mlx5e_channel *c, sq->tstamp = c->tstamp; sq->clock = &mdev->clock; sq->mkey_be = c->mkey_be; - sq->channel = c; + sq->netdev = c->netdev; + sq->mdev = c->mdev; + sq->priv = c->priv; sq->ch_ix = c->ix; sq->txq_ix = txq_ix; sq->uar_map = mdev->mlx5e_res.bfreg.map;
@@ -1332,7 +1334,7 @@ static int mlx5e_open_txqsq(struct mlx5e_channel *c,void mlx5e_activate_txqsq (struct mlx5e_txqsq *sq) { - sq->txq =netdev_get_tx_queue (sq->channel->netdev, sq->txq_ix); + sq->txq =netdev_get_tx_queue (sq->netdev, sq->txq_ix);set_bit (MLX5E_SQ_STATE_ENABLED , &sq->state);netdev_tx_reset_queue (sq->txq);netif_tx_start_queue (sq->txq);
@@ -1370,8 +1372,7 @@ static void mlx5e_deactivate_txqsq(struct mlx5e_txqsq *sq)static void mlx5e_close_txqsq (struct mlx5e_txqsq *sq) { -struct mlx5e_channel *c = sq->channel; -struct mlx5_core_dev *mdev = c->mdev; +struct mlx5_core_dev *mdev = sq->mdev;struct mlx5_rate_limit rl = {0 };cancel_work_sync (&sq->dim.work);
Signature · 1 line
2.26.2
PATCH v3 7/15 net/mlx5e: Move MLX5E_RX_ERR_CQE macro
In reply to [email protected]From: Eran Ben Elisha <[email protected]>
MLX5E_RX_ERR_CQE Macro is used only in data-path, move it to the appropriate header file.
--- drivers/net/ethernet/mellanox/mlx5/core/en/health.h | 2 -- drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h | 2 ++ 2 files changed, 2 insertions(+), 2 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en/health.h +0 -2
@@ -7,8 +7,6 @@#include "en.h" #include "diag/rsc_dump.h" -#define MLX5E_RX_ERR_CQE (cqe) (get_cqe_opcode(cqe) != MLX5_CQE_RESP_SEND) -static inline bool cqe_syndrome_needs_recover (u8 syndrome) { return syndrome == MLX5_CQE_SYNDROME_LOCAL_QP_OP_ERR ||
drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h +2 -0
@@ -24,6 +24,8 @@#define INL_HDR_START_SZ (sizeof(((struct mlx5_wqe_eth_seg *)NULL)->inline_hdr.start)) +#define MLX5E_RX_ERR_CQE (cqe) (get_cqe_opcode(cqe) != MLX5_CQE_RESP_SEND) +enum mlx5e_icosq_wqe_type {MLX5E_ICOSQ_WQE_NOP ,MLX5E_ICOSQ_WQE_UMR_RX ,
Signature · 1 line
2.26.2
PATCH v3 8/15 net/mlx5e: Add TX PTP port object support
In reply to [email protected]From: Eran Ben Elisha <[email protected]>
Add TX PTP port object support for better TX timestamping accuracy. Currently, driver supports CQE based TX port timestamp. Device also offers TX port timestamp, which has less jitter and better reflects the actual time of a packet's transmit.
Define new driver layout called ptpsq, on which driver will create SQs that will support TX port timestamp for their transmitted packets. Driver to identify PTP TX skbs and steer them to these dedicated SQs as part of the select queue ndo.
Driver to hold ptpsq per TC and report them atnetif_set_real_num_tx_queues().
Add support for all needed functionality in order to xmit and poll completions received via ptpsq.
Add ptpsq to the TX reporter recover, diagnose and dump methods.
Creation of ptpsqs is disabled by default, and can be enabled via tx_port_ts private flag.
This patch steer all timestamp related packets to a ptpsq, but it does not open the port timestamp support for it. The support will be added in the following patch.
--- .../net/ethernet/mellanox/mlx5/core/Makefile | 2 +- drivers/net/ethernet/mellanox/mlx5/core/en.h | 29 +- .../ethernet/mellanox/mlx5/core/en/params.h | 8 + .../net/ethernet/mellanox/mlx5/core/en/ptp.c | 360 ++++++++++++++++++ .../net/ethernet/mellanox/mlx5/core/en/ptp.h | 48 +++ .../mellanox/mlx5/core/en/reporter_tx.c | 166 ++++++-- .../ethernet/mellanox/mlx5/core/en_ethtool.c | 33 ++ .../net/ethernet/mellanox/mlx5/core/en_main.c | 83 ++-- .../ethernet/mellanox/mlx5/core/en_stats.c | 96 +++++ .../ethernet/mellanox/mlx5/core/en_stats.h | 3 + .../net/ethernet/mellanox/mlx5/core/en_tx.c | 63 ++- 11 files changed, 831 insertions(+), 60 deletions(-) create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en/ptp.h
drivers/net/ethernet/mellanox/mlx5/core/Makefile +1 -1
@@ -25,7 +25,7 @@ mlx5_core-$(CONFIG_MLX5_CORE_EN) += en_main.o en_common.o en_fs.o en_ethtool.o \ en_tx.o en_rx.o en_dim.o en_txrx.o en/xdp.o en_stats.o \ en_selftest.o en/port.o en/monitor_stats.o en/health.o \ en/reporter_tx.o en/reporter_rx.o en/params.o en/xsk/pool.o \ - en/xsk/setup.o en/xsk/rx.o en/xsk/tx.o en/devlink.o + en/xsk/setup.o en/xsk/rx.o en/xsk/tx.o en/devlink.o en/ptp.o # # Netdev extra
drivers/net/ethernet/mellanox/mlx5/core/en.h +28 -1
@@ -227,6 +227,7 @@ enum mlx5e_priv_flag {MLX5E_PFLAG_RX_NO_CSUM_COMPLETE ,MLX5E_PFLAG_XDP_TX_MPWQE ,MLX5E_PFLAG_SKB_TX_MPWQE , +MLX5E_PFLAG_TX_PORT_TS , MLX5E_NUM_PFLAGS, /* Keep last */ };
@@ -338,6 +339,8 @@ struct mlx5e_skb_fifo {u16 mask; }; +struct mlx5e_ptpsq ; +struct mlx5e_txqsq {/* data path */
@@ -385,6 +388,7 @@ struct mlx5e_txqsq {int txq_ix;u32 rate_limit;struct work_struct recover_work; +struct mlx5e_ptpsq *ptpsq; }____cacheline_aligned_in_smp ;struct mlx5e_dma_info {
@@ -692,8 +696,11 @@ struct mlx5e_channel {int cpu; }; +struct mlx5e_port_ptp ; +struct mlx5e_channels {struct mlx5e_channel **c; +struct mlx5e_port_ptp *port_ptp;unsigned int num;struct mlx5e_params params; };
@@ -708,6 +715,11 @@ struct mlx5e_channel_stats {struct mlx5e_xdpsq_stats xsksq; }____cacheline_aligned_in_smp ; +struct mlx5e_port_ptp_stats { +struct mlx5e_ch_stats ch; +struct mlx5e_sq_stats sq[MLX5E_MAX_NUM_TC ]; +}____cacheline_aligned_in_smp ; +enum {MLX5E_STATE_OPENED ,MLX5E_STATE_DESTROYING ,
@@ -777,8 +789,10 @@ struct mlx5e_scratchpad {struct mlx5e_priv {/* priv data path fields - start */ -struct mlx5e_txqsq *txq2sq[MLX5E_MAX_NUM_CHANNELS *MLX5E_MAX_NUM_TC ]; +/* +1 for port ptp ts */ +struct mlx5e_txqsq *txq2sq[(MLX5E_MAX_NUM_CHANNELS +1 ) *MLX5E_MAX_NUM_TC ];int channel_tc2realtxq[MLX5E_MAX_NUM_CHANNELS ][MLX5E_MAX_NUM_TC ]; +int port_ptp_tc2realtxq[MLX5E_MAX_NUM_TC ];#ifdef CONFIG_MLX5_CORE_EN_DCB struct mlx5e_dcbx_dp dcbx_dp;#endif
@@ -813,12 +827,15 @@ struct mlx5e_priv {struct net_device *netdev;struct mlx5e_stats stats;struct mlx5e_channel_stats channel_stats[MLX5E_MAX_NUM_CHANNELS ]; +struct mlx5e_port_ptp_stats port_ptp_stats;u16 max_nch;u8 max_opened_tc; +bool port_ptp_opened;struct hwtstamp_config tstamp;u16 q_counter;u16 drop_rq_q_counter;struct notifier_block events_nb; +int num_tc_x_num_ch;struct udp_tunnel_nic_info nic_info;#ifdef CONFIG_MLX5_CORE_EN_DCB
@@ -993,7 +1010,17 @@ void mlx5e_deactivate_icosq(struct mlx5e_icosq *icosq);int mlx5e_modify_sq (struct mlx5_core_dev *mdev,u32 sqn,struct mlx5e_modify_sq_param *p);void mlx5e_activate_txqsq (struct mlx5e_txqsq *sq); +void mlx5e_deactivate_txqsq (struct mlx5e_txqsq *sq); +void mlx5e_free_txqsq (struct mlx5e_txqsq *sq);void mlx5e_tx_disable_queue (struct netdev_queue *txq); +int mlx5e_alloc_txqsq_db (struct mlx5e_txqsq *sq,int numa); +void mlx5e_free_txqsq_db (struct mlx5e_txqsq *sq); +struct mlx5e_create_sq_param ; +int mlx5e_create_sq_rdy (struct mlx5_core_dev *mdev, +struct mlx5e_sq_param *param, +struct mlx5e_create_sq_param *csp, +u32 *sqn); +void mlx5e_tx_err_cqe_work (struct work_struct *recover_work);static inline bool mlx5_tx_swp_supported (struct mlx5_core_dev *mdev) {
drivers/net/ethernet/mellanox/mlx5/core/en/params.h +8 -0
@@ -41,6 +41,14 @@ struct mlx5e_channel_param {struct mlx5e_sq_param async_icosq; }; +struct mlx5e_create_sq_param { +struct mlx5_wq_ctrl *wq_ctrl; +u32 cqn; +u32 tisn; +u8 tis_lst_sz; +u8 min_inline_mode; +}; + static inline bool mlx5e_qid_get_ch_if_in_group(struct mlx5e_params *params, u16 qid, enum mlx5e_rq_group group,
drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c new file +360 -0
360 lines not on this page · raw message
drivers/net/ethernet/mellanox/mlx5/core/en/ptp.h new file +48 -0
48 lines not on this page · raw message
drivers/net/ethernet/mellanox/mlx5/core/en/reporter_tx.c +134 -32
236 lines not on this page · raw message
drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c +33 -0
@@ -1944,6 +1944,38 @@ static int set_pflag_skb_tx_mpwqe(struct net_device *netdev, bool enable)return set_pflag_tx_mpwqe_common (netdev,MLX5E_PFLAG_SKB_TX_MPWQE , enable); } +static int set_pflag_tx_port_ts (struct net_device *netdev,bool enable) +{ +struct mlx5e_priv *priv =netdev_priv (netdev); +struct mlx5_core_dev *mdev = priv->mdev; +struct mlx5e_channels new_channels = {}; +int err; + +if (!MLX5_CAP_GEN (mdev, ts_cqe_to_dest_cqn)) +return -EOPNOTSUPP ; + + new_channels.params = priv->channels.params; +MLX5E_SET_PFLAG (&new_channels.params,MLX5E_PFLAG_TX_PORT_TS , enable); +/* No need to verify SQ stop room as +* ptpsq.txqsq.stop_room <= generic_sq->stop_room, and both +* has the same log_sq_size. +*/ + +if (!test_bit (MLX5E_STATE_OPENED , &priv->state)) { + priv->channels.params = new_channels.params; + err =mlx5e_num_channels_changed (priv); + goto out; + } + + err =mlx5e_safe_switch_channels (priv, &new_channels, + mlx5e_num_channels_changed_ctx,NULL ); +out: +if (!err) + priv->port_ptp_opened = true; + +return err; +} +static const struct pflag_desc mlx5e_priv_flags[MLX5E_NUM_PFLAGS ] = { {"rx_cqe_moder" , set_pflag_rx_cqe_based_moder }, {"tx_cqe_moder" , set_pflag_tx_cqe_based_moder },
@@ -1952,6 +1984,7 @@ static const struct pflag_desc mlx5e_priv_flags[MLX5E_NUM_PFLAGS] = { { "rx_no_csum_complete", set_pflag_rx_no_csum_complete }, { "xdp_tx_mpwqe", set_pflag_xdp_tx_mpwqe }, { "skb_tx_mpwqe", set_pflag_skb_tx_mpwqe }, + { "tx_port_ts", set_pflag_tx_port_ts }, }; static int mlx5e_handle_pflag(struct net_device *netdev,
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +58 -25
208 lines not on this page · raw message
drivers/net/ethernet/mellanox/mlx5/core/en_stats.c +96 -0
132 lines not on this page · raw message
drivers/net/ethernet/mellanox/mlx5/core/en_stats.h +3 -0
@@ -51,6 +51,9 @@#define MLX5E_DECLARE_XSKSQ_STAT (type, fld) "tx%d_xsk_"#fld, offsetof(type, fld)#define MLX5E_DECLARE_CH_STAT (type, fld) "ch%d_"#fld, offsetof(type, fld) +#define MLX5E_DECLARE_PTP_TX_STAT (type, fld) "ptp_tx%d_"#fld, offsetof(type, fld) +#define MLX5E_DECLARE_PTP_CH_STAT (type, fld) "ptp_ch_"#fld, offsetof(type, fld) +struct counter_desc {char format[ETH_GSTRING_LEN ];size_t offset;/* Byte offset */
drivers/net/ethernet/mellanox/mlx5/core/en_tx.c +62 -1
88 lines not on this page · raw message
Signature · 1 line
2.26.2
PATCH v3 9/15 net/mlx5e: Add TX port timestamp support
In reply to [email protected]From: Eran Ben Elisha <[email protected]>
Transmitted packet timestamping accuracy can be improved when using timestamp from the port, instead of packet CQE creation timestamp, as it better reflects the actual time of a packet's transmit.
TX port timestamping is supported starting from ConnectX6-DX hardware. Although at the original completion, only CQE timestamp can be attached, we are able to get TX port timestamping via an additional completion over a special CQ associated with the SQ (in addition to the regular CQ).
Driver to ignore the original packet completion timestamp, and report back the timestamp of the special CQ completion. If the absolute timestamp
diff between the two completions is greater than 1 / 128 second, ignore the TX port timestamp as it has a jitter which is too big. No skb will be generate out of the extra completion.
Allocate additional CQ per ptpsq, to receive the TX port timestamp.
Driver to hold an skb FIFO in order to map between transmitted skb to the two expected completions. When using ptpsq, hold double refcount on the skb, to gaurantee it will not get released before both completions arrive.
Expose dedicated counters of the ptp additional CQ and connect it to the TX health reporter.
This patch improves TX Hardware timestamping offset to be less than 40ns at a 100Gbps line rate, compared to 600ns before.
With that, making our HW compliant with G.8273.2 class C, and allow Linux systems to be deployed in the 5G telco edge, where this standard is a must.
--- drivers/net/ethernet/mellanox/mlx5/core/en.h | 1 + .../ethernet/mellanox/mlx5/core/en/params.h | 1 + .../net/ethernet/mellanox/mlx5/core/en/ptp.c | 173 +++++++++++++++++- .../net/ethernet/mellanox/mlx5/core/en/ptp.h | 15 ++ .../mellanox/mlx5/core/en/reporter_tx.c | 37 +++- .../net/ethernet/mellanox/mlx5/core/en_main.c | 1 + .../ethernet/mellanox/mlx5/core/en_stats.c | 21 ++- .../ethernet/mellanox/mlx5/core/en_stats.h | 8 + .../net/ethernet/mellanox/mlx5/core/en_tx.c | 12 +- 9 files changed, 262 insertions(+), 7 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en.h +1 -0
@@ -718,6 +718,7 @@ struct mlx5e_channel_stats {struct mlx5e_port_ptp_stats {struct mlx5e_ch_stats ch;struct mlx5e_sq_stats sq[MLX5E_MAX_NUM_TC ]; +struct mlx5e_ptp_cq_stats cq[MLX5E_MAX_NUM_TC ]; }____cacheline_aligned_in_smp ;enum {
drivers/net/ethernet/mellanox/mlx5/core/en/params.h +1 -0
@@ -44,6 +44,7 @@ struct mlx5e_channel_param {struct mlx5e_create_sq_param {struct mlx5_wq_ctrl *wq_ctrl;u32 cqn; +u32 ts_cqe_to_dest_cqn;u32 tisn;u8 tis_lst_sz;u8 min_inline_mode;
drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c +171 -2
230 lines not on this page · raw message
drivers/net/ethernet/mellanox/mlx5/core/en/ptp.h +15 -0
@@ -10,6 +10,11 @@struct mlx5e_ptpsq {struct mlx5e_txqsq txqsq; +struct mlx5e_cq ts_cq; +u16 skb_fifo_cc; +u16 skb_fifo_pc; +struct mlx5e_skb_fifo skb_fifo; +struct mlx5e_ptp_cq_stats *cq_stats; };struct mlx5e_port_ptp {
@@ -45,4 +50,14 @@ void mlx5e_port_ptp_close(struct mlx5e_port_ptp *c);void mlx5e_ptp_activate_channel (struct mlx5e_port_ptp *c);void mlx5e_ptp_deactivate_channel (struct mlx5e_port_ptp *c); +enum { +MLX5E_SKB_CB_CQE_HWTSTAMP =BIT (0 ), +MLX5E_SKB_CB_PORT_HWTSTAMP =BIT (1 ), +}; + +void mlx5e_skb_cb_hwtstamp_handler (struct sk_buff *skb,int hwtstamp_type, +ktime_t hwtstamp, +struct mlx5e_ptp_cq_stats *cq_stats); + +void mlx5e_skb_cb_hwtstamp_init (struct sk_buff *skb);#endif /* __MLX5_EN_PTP_H__ */
drivers/net/ethernet/mellanox/mlx5/core/en/reporter_tx.c +34 -3
@@ -228,9 +228,19 @@ mlx5e_tx_reporter_build_diagnose_output_ptpsq(struct devlink_fmsg *fmsg,if (err)return err; - err =mlx5e_tx_reporter_build_diagnose_output_sq_common (fmsg, - &ptpsq->txqsq, - tc); + err =mlx5e_tx_reporter_build_diagnose_output_sq_common (fmsg, &ptpsq->txqsq, tc); +if (err) +return err; + + err =mlx5e_health_fmsg_named_obj_nest_start (fmsg,"Port TS" ); +if (err) +return err; + + err =mlx5e_health_cq_diag_fmsg (&ptpsq->ts_cq, fmsg); +if (err) +return err; + + err =mlx5e_health_fmsg_named_obj_nest_end (fmsg);if (err)return err;
@@ -270,6 +280,23 @@ mlx5e_tx_reporter_diagnose_generic_txqsq(struct devlink_fmsg *fmsg,return mlx5e_health_fmsg_named_obj_nest_end (fmsg); } +static int +mlx5e_tx_reporter_diagnose_generic_tx_port_ts (struct devlink_fmsg *fmsg, +struct mlx5e_ptpsq *ptpsq) +{ +int err; + + err =mlx5e_health_fmsg_named_obj_nest_start (fmsg,"Port TS" ); +if (err) +return err; + + err =mlx5e_health_cq_common_diag_fmsg (&ptpsq->ts_cq, fmsg); +if (err) +return err; + +return mlx5e_health_fmsg_named_obj_nest_end (fmsg); +} +static int mlx5e_tx_reporter_diagnose_common_config (struct devlink_health_reporter *reporter,struct devlink_fmsg *fmsg)
@@ -301,6 +328,10 @@ mlx5e_tx_reporter_diagnose_common_config(struct devlink_health_reporter *reporteif (err)return err; + err =mlx5e_tx_reporter_diagnose_generic_tx_port_ts (fmsg, generic_ptpsq); +if (err) +return err; + err =mlx5e_health_fmsg_named_obj_nest_end (fmsg);if (err)return err;
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +1 -0
@@ -1206,6 +1206,7 @@ static int mlx5e_create_sq(struct mlx5_core_dev *mdev,MLX5_SET (sqc, sqc, tis_lst_sz, csp->tis_lst_sz);MLX5_SET (sqc, sqc, tis_num_0, csp->tisn);MLX5_SET (sqc, sqc, cqn, csp->cqn); +MLX5_SET (sqc, sqc, ts_cqe_to_dest_cqn, csp->ts_cqe_to_dest_cqn);if (MLX5_CAP_ETH (mdev, wqe_inline_mode) ==MLX5_CAP_INLINE_MODE_VPORT_CONTEXT )MLX5_SET (sqc, sqc, min_wqe_inline_mode, csp->min_inline_mode);
drivers/net/ethernet/mellanox/mlx5/core/en_stats.c +20 -1
@@ -1733,6 +1733,13 @@ static const struct counter_desc ptp_ch_stats_desc[] = { { MLX5E_DECLARE_PTP_CH_STAT(struct mlx5e_ch_stats, eq_rearm) }, }; +static const struct counter_desc ptp_cq_stats_desc[] = { + { MLX5E_DECLARE_PTP_CQ_STAT(struct mlx5e_ptp_cq_stats, cqe) }, + { MLX5E_DECLARE_PTP_CQ_STAT(struct mlx5e_ptp_cq_stats, err_cqe) }, + { MLX5E_DECLARE_PTP_CQ_STAT(struct mlx5e_ptp_cq_stats, abort) }, + { MLX5E_DECLARE_PTP_CQ_STAT(struct mlx5e_ptp_cq_stats, abort_abs_diff_ns) }, +}; +#define NUM_RQ_STATS ARRAY_SIZE(rq_stats_desc)#define NUM_SQ_STATS ARRAY_SIZE(sq_stats_desc)#define NUM_XDPSQ_STATS ARRAY_SIZE(xdpsq_stats_desc)
@@ -1742,11 +1749,13 @@ static const struct counter_desc ptp_ch_stats_desc[] = {#define NUM_CH_STATS ARRAY_SIZE(ch_stats_desc)#define NUM_PTP_SQ_STATS ARRAY_SIZE(ptp_sq_stats_desc)#define NUM_PTP_CH_STATS ARRAY_SIZE(ptp_ch_stats_desc) +#define NUM_PTP_CQ_STATS ARRAY_SIZE(ptp_cq_stats_desc)static MLX5E_DECLARE_STATS_GRP_OP_NUM_STATS (ptp) {return priv->port_ptp_opened ? -NUM_PTP_CH_STATS + (NUM_PTP_SQ_STATS * priv->max_opened_tc) : +NUM_PTP_CH_STATS + + ((NUM_PTP_SQ_STATS +NUM_PTP_CQ_STATS ) * priv->max_opened_tc) :0 ; }
@@ -1766,6 +1775,10 @@ static MLX5E_DECLARE_STATS_GRP_OP_FILL_STRS(ptp)sprintf (data + (idx++) *ETH_GSTRING_LEN , ptp_sq_stats_desc[i].format, tc); +for (tc =0 ; tc < priv->max_opened_tc; tc++) +for (i =0 ; i <NUM_PTP_CQ_STATS ; i++) +sprintf (data + (idx++) *ETH_GSTRING_LEN , + ptp_cq_stats_desc[i].format, tc);return idx; }
@@ -1787,6 +1800,12 @@ static MLX5E_DECLARE_STATS_GRP_OP_FILL_STATS(ptp)MLX5E_READ_CTR64_CPU (&priv->port_ptp_stats.sq[tc], ptp_sq_stats_desc, i); +for (tc =0 ; tc < priv->max_opened_tc; tc++) +for (i =0 ; i <NUM_PTP_CQ_STATS ; i++) + data[idx++] = +MLX5E_READ_CTR64_CPU (&priv->port_ptp_stats.cq[tc], + ptp_cq_stats_desc, i); +return idx; }
drivers/net/ethernet/mellanox/mlx5/core/en_stats.h +8 -0
@@ -53,6 +53,7 @@#define MLX5E_DECLARE_PTP_TX_STAT (type, fld) "ptp_tx%d_"#fld, offsetof(type, fld)#define MLX5E_DECLARE_PTP_CH_STAT (type, fld) "ptp_ch_"#fld, offsetof(type, fld) +#define MLX5E_DECLARE_PTP_CQ_STAT (type, fld) "ptp_cq%d_"#fld, offsetof(type, fld)struct counter_desc {char format[ETH_GSTRING_LEN ];
@@ -401,6 +402,13 @@ struct mlx5e_ch_stats {u64 eq_rearm; }; +struct mlx5e_ptp_cq_stats { +u64 cqe; +u64 err_cqe; +u64 abort; +u64 abort_abs_diff_ns; +}; +struct mlx5e_stats {struct mlx5e_sw_stats sw;struct mlx5e_qcounter_stats qcnt;
drivers/net/ethernet/mellanox/mlx5/core/en_tx.c +11 -1
@@ -463,6 +463,12 @@ mlx5e_txwqe_complete(struct mlx5e_txqsq *sq, struct sk_buff *skb,mlx5e_tx_check_stop (sq); +if (unlikely (sq->ptpsq)) { +mlx5e_skb_cb_hwtstamp_init (skb); +mlx5e_skb_fifo_push (&sq->ptpsq->skb_fifo, skb); +skb_get (skb); + } + send_doorbell =__netdev_tx_sent_queue (sq->txq, attr->num_bytes, xmit_more);if (send_doorbell)mlx5e_notify_hw (wq, sq->pc, sq->uar_map, cseg);
@@ -768,7 +774,11 @@ static void mlx5e_consume_skb(struct mlx5e_txqsq *sq, struct sk_buff *skb,u64 ts =get_cqe_ts (cqe); hwts.hwtstamp =mlx5_timecounter_cyc2time (sq->clock, ts); -skb_tstamp_tx (skb, &hwts); +if (sq->ptpsq) +mlx5e_skb_cb_hwtstamp_handler (skb,MLX5E_SKB_CB_CQE_HWTSTAMP , + hwts.hwtstamp, sq->ptpsq->cq_stats); +else +skb_tstamp_tx (skb, &hwts); }napi_consume_skb (skb, napi_budget);
Signature · 1 line
2.26.2
PATCH v3 6/15 net/mlx5e: Split SW group counters update function
In reply to [email protected]From: Eran Ben Elisha <[email protected]>
SW group counter update function aggregates sw stats out of many mlx5e_*_stats resides in a given mlx5e_channel_stats struct. Split the function into a few helper functions.
This will be used later in the series to calculate specific mlx5e_*_stats which are not defined inside mlx5e_channel_stats.
--- .../ethernet/mellanox/mlx5/core/en_stats.c | 288 ++++++++++-------- 1 file changed, 161 insertions(+), 127 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en_stats.c +161 -127
@@ -248,6 +248,160 @@ static MLX5E_DECLARE_STATS_GRP_OP_FILL_STATS(sw)return idx; } +static void mlx5e_stats_grp_sw_update_stats_xdp_red (struct mlx5e_sw_stats *s, +struct mlx5e_xdpsq_stats *xdpsq_red_stats) +{ + s->tx_xdp_xmit += xdpsq_red_stats->xmit; + s->tx_xdp_mpwqe += xdpsq_red_stats->mpwqe; + s->tx_xdp_inlnw += xdpsq_red_stats->inlnw; + s->tx_xdp_nops += xdpsq_red_stats->nops; + s->tx_xdp_full += xdpsq_red_stats->full; + s->tx_xdp_err += xdpsq_red_stats->err; + s->tx_xdp_cqes += xdpsq_red_stats->cqes; +} + +static void mlx5e_stats_grp_sw_update_stats_xdpsq (struct mlx5e_sw_stats *s, +struct mlx5e_xdpsq_stats *xdpsq_stats) +{ + s->rx_xdp_tx_xmit += xdpsq_stats->xmit; + s->rx_xdp_tx_mpwqe += xdpsq_stats->mpwqe; + s->rx_xdp_tx_inlnw += xdpsq_stats->inlnw; + s->rx_xdp_tx_nops += xdpsq_stats->nops; + s->rx_xdp_tx_full += xdpsq_stats->full; + s->rx_xdp_tx_err += xdpsq_stats->err; + s->rx_xdp_tx_cqe += xdpsq_stats->cqes; +} + +static void mlx5e_stats_grp_sw_update_stats_xsksq (struct mlx5e_sw_stats *s, +struct mlx5e_xdpsq_stats *xsksq_stats) +{ + s->tx_xsk_xmit += xsksq_stats->xmit; + s->tx_xsk_mpwqe += xsksq_stats->mpwqe; + s->tx_xsk_inlnw += xsksq_stats->inlnw; + s->tx_xsk_full += xsksq_stats->full; + s->tx_xsk_err += xsksq_stats->err; + s->tx_xsk_cqes += xsksq_stats->cqes; +} + +static void mlx5e_stats_grp_sw_update_stats_xskrq (struct mlx5e_sw_stats *s, +struct mlx5e_rq_stats *xskrq_stats) +{ + s->rx_xsk_packets += xskrq_stats->packets; + s->rx_xsk_bytes += xskrq_stats->bytes; + s->rx_xsk_csum_complete += xskrq_stats->csum_complete; + s->rx_xsk_csum_unnecessary += xskrq_stats->csum_unnecessary; + s->rx_xsk_csum_unnecessary_inner += xskrq_stats->csum_unnecessary_inner; + s->rx_xsk_csum_none += xskrq_stats->csum_none; + s->rx_xsk_ecn_mark += xskrq_stats->ecn_mark; + s->rx_xsk_removed_vlan_packets += xskrq_stats->removed_vlan_packets; + s->rx_xsk_xdp_drop += xskrq_stats->xdp_drop; + s->rx_xsk_xdp_redirect += xskrq_stats->xdp_redirect; + s->rx_xsk_wqe_err += xskrq_stats->wqe_err; + s->rx_xsk_mpwqe_filler_cqes += xskrq_stats->mpwqe_filler_cqes; + s->rx_xsk_mpwqe_filler_strides += xskrq_stats->mpwqe_filler_strides; + s->rx_xsk_oversize_pkts_sw_drop += xskrq_stats->oversize_pkts_sw_drop; + s->rx_xsk_buff_alloc_err += xskrq_stats->buff_alloc_err; + s->rx_xsk_cqe_compress_blks += xskrq_stats->cqe_compress_blks; + s->rx_xsk_cqe_compress_pkts += xskrq_stats->cqe_compress_pkts; + s->rx_xsk_congst_umr += xskrq_stats->congst_umr; + s->rx_xsk_arfs_err += xskrq_stats->arfs_err; +} + +static void mlx5e_stats_grp_sw_update_stats_rq_stats (struct mlx5e_sw_stats *s, +struct mlx5e_rq_stats *rq_stats) +{ + s->rx_packets += rq_stats->packets; + s->rx_bytes += rq_stats->bytes; + s->rx_lro_packets += rq_stats->lro_packets; + s->rx_lro_bytes += rq_stats->lro_bytes; + s->rx_ecn_mark += rq_stats->ecn_mark; + s->rx_removed_vlan_packets += rq_stats->removed_vlan_packets; + s->rx_csum_none += rq_stats->csum_none; + s->rx_csum_complete += rq_stats->csum_complete; + s->rx_csum_complete_tail += rq_stats->csum_complete_tail; + s->rx_csum_complete_tail_slow += rq_stats->csum_complete_tail_slow; + s->rx_csum_unnecessary += rq_stats->csum_unnecessary; + s->rx_csum_unnecessary_inner += rq_stats->csum_unnecessary_inner; + s->rx_xdp_drop += rq_stats->xdp_drop; + s->rx_xdp_redirect += rq_stats->xdp_redirect; + s->rx_wqe_err += rq_stats->wqe_err; + s->rx_mpwqe_filler_cqes += rq_stats->mpwqe_filler_cqes; + s->rx_mpwqe_filler_strides += rq_stats->mpwqe_filler_strides; + s->rx_oversize_pkts_sw_drop += rq_stats->oversize_pkts_sw_drop; + s->rx_buff_alloc_err += rq_stats->buff_alloc_err; + s->rx_cqe_compress_blks += rq_stats->cqe_compress_blks; + s->rx_cqe_compress_pkts += rq_stats->cqe_compress_pkts; + s->rx_cache_reuse += rq_stats->cache_reuse; + s->rx_cache_full += rq_stats->cache_full; + s->rx_cache_empty += rq_stats->cache_empty; + s->rx_cache_busy += rq_stats->cache_busy; + s->rx_cache_waive += rq_stats->cache_waive; + s->rx_congst_umr += rq_stats->congst_umr; + s->rx_arfs_err += rq_stats->arfs_err; + s->rx_recover += rq_stats->recover; +#ifdef CONFIG_MLX5_EN_TLS + s->rx_tls_decrypted_packets += rq_stats->tls_decrypted_packets; + s->rx_tls_decrypted_bytes += rq_stats->tls_decrypted_bytes; + s->rx_tls_ctx += rq_stats->tls_ctx; + s->rx_tls_del += rq_stats->tls_del; + s->rx_tls_resync_req_pkt += rq_stats->tls_resync_req_pkt; + s->rx_tls_resync_req_start += rq_stats->tls_resync_req_start; + s->rx_tls_resync_req_end += rq_stats->tls_resync_req_end; + s->rx_tls_resync_req_skip += rq_stats->tls_resync_req_skip; + s->rx_tls_resync_res_ok += rq_stats->tls_resync_res_ok; + s->rx_tls_resync_res_skip += rq_stats->tls_resync_res_skip; + s->rx_tls_err += rq_stats->tls_err; +#endif +} + +static void mlx5e_stats_grp_sw_update_stats_ch_stats (struct mlx5e_sw_stats *s, +struct mlx5e_ch_stats *ch_stats) +{ + s->ch_events += ch_stats->events; + s->ch_poll += ch_stats->poll; + s->ch_arm += ch_stats->arm; + s->ch_aff_change += ch_stats->aff_change; + s->ch_force_irq += ch_stats->force_irq; + s->ch_eq_rearm += ch_stats->eq_rearm; +} + +static void mlx5e_stats_grp_sw_update_stats_sq (struct mlx5e_sw_stats *s, +struct mlx5e_sq_stats *sq_stats) +{ + s->tx_packets += sq_stats->packets; + s->tx_bytes += sq_stats->bytes; + s->tx_tso_packets += sq_stats->tso_packets; + s->tx_tso_bytes += sq_stats->tso_bytes; + s->tx_tso_inner_packets += sq_stats->tso_inner_packets; + s->tx_tso_inner_bytes += sq_stats->tso_inner_bytes; + s->tx_added_vlan_packets += sq_stats->added_vlan_packets; + s->tx_nop += sq_stats->nop; + s->tx_mpwqe_blks += sq_stats->mpwqe_blks; + s->tx_mpwqe_pkts += sq_stats->mpwqe_pkts; + s->tx_queue_stopped += sq_stats->stopped; + s->tx_queue_wake += sq_stats->wake; + s->tx_queue_dropped += sq_stats->dropped; + s->tx_cqe_err += sq_stats->cqe_err; + s->tx_recover += sq_stats->recover; + s->tx_xmit_more += sq_stats->xmit_more; + s->tx_csum_partial_inner += sq_stats->csum_partial_inner; + s->tx_csum_none += sq_stats->csum_none; + s->tx_csum_partial += sq_stats->csum_partial; +#ifdef CONFIG_MLX5_EN_TLS + s->tx_tls_encrypted_packets += sq_stats->tls_encrypted_packets; + s->tx_tls_encrypted_bytes += sq_stats->tls_encrypted_bytes; + s->tx_tls_ctx += sq_stats->tls_ctx; + s->tx_tls_ooo += sq_stats->tls_ooo; + s->tx_tls_dump_bytes += sq_stats->tls_dump_bytes; + s->tx_tls_dump_packets += sq_stats->tls_dump_packets; + s->tx_tls_resync_bytes += sq_stats->tls_resync_bytes; + s->tx_tls_skip_no_sync_data += sq_stats->tls_skip_no_sync_data; + s->tx_tls_drop_no_sync_data += sq_stats->tls_drop_no_sync_data; + s->tx_tls_drop_bypass_req += sq_stats->tls_drop_bypass_req; +#endif + s->tx_cqes += sq_stats->cqes; +} +static MLX5E_DECLARE_STATS_GRP_OP_UPDATE_STATS (sw) {struct mlx5e_sw_stats *s = &priv->stats.sw;
@@ -258,139 +412,19 @@ static MLX5E_DECLARE_STATS_GRP_OP_UPDATE_STATS(sw)for (i =0 ; i < priv->max_nch; i++) {struct mlx5e_channel_stats *channel_stats = &priv->channel_stats[i]; -struct mlx5e_xdpsq_stats *xdpsq_red_stats = &channel_stats->xdpsq; -struct mlx5e_xdpsq_stats *xdpsq_stats = &channel_stats->rq_xdpsq; -struct mlx5e_xdpsq_stats *xsksq_stats = &channel_stats->xsksq; -struct mlx5e_rq_stats *xskrq_stats = &channel_stats->xskrq; -struct mlx5e_rq_stats *rq_stats = &channel_stats->rq; -struct mlx5e_ch_stats *ch_stats = &channel_stats->ch;int j; - s->rx_packets += rq_stats->packets; - s->rx_bytes += rq_stats->bytes; - s->rx_lro_packets += rq_stats->lro_packets; - s->rx_lro_bytes += rq_stats->lro_bytes; - s->rx_ecn_mark += rq_stats->ecn_mark; - s->rx_removed_vlan_packets += rq_stats->removed_vlan_packets; - s->rx_csum_none += rq_stats->csum_none; - s->rx_csum_complete += rq_stats->csum_complete; - s->rx_csum_complete_tail += rq_stats->csum_complete_tail; - s->rx_csum_complete_tail_slow += rq_stats->csum_complete_tail_slow; - s->rx_csum_unnecessary += rq_stats->csum_unnecessary; - s->rx_csum_unnecessary_inner += rq_stats->csum_unnecessary_inner; - s->rx_xdp_drop += rq_stats->xdp_drop; - s->rx_xdp_redirect += rq_stats->xdp_redirect; - s->rx_xdp_tx_xmit += xdpsq_stats->xmit; - s->rx_xdp_tx_mpwqe += xdpsq_stats->mpwqe; - s->rx_xdp_tx_inlnw += xdpsq_stats->inlnw; - s->rx_xdp_tx_nops += xdpsq_stats->nops; - s->rx_xdp_tx_full += xdpsq_stats->full; - s->rx_xdp_tx_err += xdpsq_stats->err; - s->rx_xdp_tx_cqe += xdpsq_stats->cqes; - s->rx_wqe_err += rq_stats->wqe_err; - s->rx_mpwqe_filler_cqes += rq_stats->mpwqe_filler_cqes; - s->rx_mpwqe_filler_strides += rq_stats->mpwqe_filler_strides; - s->rx_oversize_pkts_sw_drop += rq_stats->oversize_pkts_sw_drop; - s->rx_buff_alloc_err += rq_stats->buff_alloc_err; - s->rx_cqe_compress_blks += rq_stats->cqe_compress_blks; - s->rx_cqe_compress_pkts += rq_stats->cqe_compress_pkts; - s->rx_cache_reuse += rq_stats->cache_reuse; - s->rx_cache_full += rq_stats->cache_full; - s->rx_cache_empty += rq_stats->cache_empty; - s->rx_cache_busy += rq_stats->cache_busy; - s->rx_cache_waive += rq_stats->cache_waive; - s->rx_congst_umr += rq_stats->congst_umr; - s->rx_arfs_err += rq_stats->arfs_err; - s->rx_recover += rq_stats->recover; -#ifdef CONFIG_MLX5_EN_TLS - s->rx_tls_decrypted_packets += rq_stats->tls_decrypted_packets; - s->rx_tls_decrypted_bytes += rq_stats->tls_decrypted_bytes; - s->rx_tls_ctx += rq_stats->tls_ctx; - s->rx_tls_del += rq_stats->tls_del; - s->rx_tls_resync_req_pkt += rq_stats->tls_resync_req_pkt; - s->rx_tls_resync_req_start += rq_stats->tls_resync_req_start; - s->rx_tls_resync_req_end += rq_stats->tls_resync_req_end; - s->rx_tls_resync_req_skip += rq_stats->tls_resync_req_skip; - s->rx_tls_resync_res_ok += rq_stats->tls_resync_res_ok; - s->rx_tls_resync_res_skip += rq_stats->tls_resync_res_skip; - s->rx_tls_err += rq_stats->tls_err; -#endif - s->ch_events += ch_stats->events; - s->ch_poll += ch_stats->poll; - s->ch_arm += ch_stats->arm; - s->ch_aff_change += ch_stats->aff_change; - s->ch_force_irq += ch_stats->force_irq; - s->ch_eq_rearm += ch_stats->eq_rearm; +mlx5e_stats_grp_sw_update_stats_rq_stats (s, &channel_stats->rq); +mlx5e_stats_grp_sw_update_stats_xdpsq (s, &channel_stats->rq_xdpsq); +mlx5e_stats_grp_sw_update_stats_ch_stats (s, &channel_stats->ch);/* xdp redirect */ - s->tx_xdp_xmit += xdpsq_red_stats->xmit; - s->tx_xdp_mpwqe += xdpsq_red_stats->mpwqe; - s->tx_xdp_inlnw += xdpsq_red_stats->inlnw; - s->tx_xdp_nops += xdpsq_red_stats->nops; - s->tx_xdp_full += xdpsq_red_stats->full; - s->tx_xdp_err += xdpsq_red_stats->err; - s->tx_xdp_cqes += xdpsq_red_stats->cqes; +mlx5e_stats_grp_sw_update_stats_xdp_red (s, &channel_stats->xdpsq);/* AF_XDP zero-copy */ - s->rx_xsk_packets += xskrq_stats->packets; - s->rx_xsk_bytes += xskrq_stats->bytes; - s->rx_xsk_csum_complete += xskrq_stats->csum_complete; - s->rx_xsk_csum_unnecessary += xskrq_stats->csum_unnecessary; - s->rx_xsk_csum_unnecessary_inner += xskrq_stats->csum_unnecessary_inner; - s->rx_xsk_csum_none += xskrq_stats->csum_none; - s->rx_xsk_ecn_mark += xskrq_stats->ecn_mark; - s->rx_xsk_removed_vlan_packets += xskrq_stats->removed_vlan_packets; - s->rx_xsk_xdp_drop += xskrq_stats->xdp_drop; - s->rx_xsk_xdp_redirect += xskrq_stats->xdp_redirect; - s->rx_xsk_wqe_err += xskrq_stats->wqe_err; - s->rx_xsk_mpwqe_filler_cqes += xskrq_stats->mpwqe_filler_cqes; - s->rx_xsk_mpwqe_filler_strides += xskrq_stats->mpwqe_filler_strides; - s->rx_xsk_oversize_pkts_sw_drop += xskrq_stats->oversize_pkts_sw_drop; - s->rx_xsk_buff_alloc_err += xskrq_stats->buff_alloc_err; - s->rx_xsk_cqe_compress_blks += xskrq_stats->cqe_compress_blks; - s->rx_xsk_cqe_compress_pkts += xskrq_stats->cqe_compress_pkts; - s->rx_xsk_congst_umr += xskrq_stats->congst_umr; - s->rx_xsk_arfs_err += xskrq_stats->arfs_err; - s->tx_xsk_xmit += xsksq_stats->xmit; - s->tx_xsk_mpwqe += xsksq_stats->mpwqe; - s->tx_xsk_inlnw += xsksq_stats->inlnw; - s->tx_xsk_full += xsksq_stats->full; - s->tx_xsk_err += xsksq_stats->err; - s->tx_xsk_cqes += xsksq_stats->cqes; +mlx5e_stats_grp_sw_update_stats_xskrq (s, &channel_stats->xskrq); +mlx5e_stats_grp_sw_update_stats_xsksq (s, &channel_stats->xsksq);for (j =0 ; j < priv->max_opened_tc; j++) { -struct mlx5e_sq_stats *sq_stats = &channel_stats->sq[j]; - - s->tx_packets += sq_stats->packets; - s->tx_bytes += sq_stats->bytes; - s->tx_tso_packets += sq_stats->tso_packets; - s->tx_tso_bytes += sq_stats->tso_bytes; - s->tx_tso_inner_packets += sq_stats->tso_inner_packets; - s->tx_tso_inner_bytes += sq_stats->tso_inner_bytes; - s->tx_added_vlan_packets += sq_stats->added_vlan_packets; - s->tx_nop += sq_stats->nop; - s->tx_mpwqe_blks += sq_stats->mpwqe_blks; - s->tx_mpwqe_pkts += sq_stats->mpwqe_pkts; - s->tx_queue_stopped += sq_stats->stopped; - s->tx_queue_wake += sq_stats->wake; - s->tx_queue_dropped += sq_stats->dropped; - s->tx_cqe_err += sq_stats->cqe_err; - s->tx_recover += sq_stats->recover; - s->tx_xmit_more += sq_stats->xmit_more; - s->tx_csum_partial_inner += sq_stats->csum_partial_inner; - s->tx_csum_none += sq_stats->csum_none; - s->tx_csum_partial += sq_stats->csum_partial; -#ifdef CONFIG_MLX5_EN_TLS - s->tx_tls_encrypted_packets += sq_stats->tls_encrypted_packets; - s->tx_tls_encrypted_bytes += sq_stats->tls_encrypted_bytes; - s->tx_tls_ctx += sq_stats->tls_ctx; - s->tx_tls_ooo += sq_stats->tls_ooo; - s->tx_tls_dump_bytes += sq_stats->tls_dump_bytes; - s->tx_tls_dump_packets += sq_stats->tls_dump_packets; - s->tx_tls_resync_bytes += sq_stats->tls_resync_bytes; - s->tx_tls_skip_no_sync_data += sq_stats->tls_skip_no_sync_data; - s->tx_tls_drop_no_sync_data += sq_stats->tls_drop_no_sync_data; - s->tx_tls_drop_bypass_req += sq_stats->tls_drop_bypass_req; -#endif - s->tx_cqes += sq_stats->cqes; +mlx5e_stats_grp_sw_update_stats_sq (s, &channel_stats->sq[j]);/* https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92657 */ barrier ();
Signature · 1 line
2.26.2
PATCH v3 1/15 net/mlx5e: Free drop RQ in a dedicated function
In reply to [email protected]From: Tariq Toukan <[email protected]>
The drop RQ has very limited objects to be freed, and differs from regular RQs in the context that it is freed from. Add a dedicated function for it, use it where needed, and remove the drop_rq-specific checks in the generic function.
--- .../net/ethernet/mellanox/mlx5/core/en_main.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +10 -8
@@ -613,14 +613,11 @@ static int mlx5e_alloc_rq(struct mlx5e_channel *c,static void mlx5e_free_rq (struct mlx5e_rq *rq) { -struct mlx5e_channel *c = rq->channel; -struct bpf_prog *old_prog =NULL ; +struct bpf_prog *old_prog;int i; -/* drop_rq has neither channel nor xdp_prog. */ -if (c) - old_prog =rcu_dereference_protected (rq->xdp_prog, -lockdep_is_held (&c->priv->state_lock)); + old_prog =rcu_dereference_protected (rq->xdp_prog, +lockdep_is_held (&rq->channel->priv->state_lock));if (old_prog)bpf_prog_put (old_prog);
@@ -3196,6 +3193,11 @@ int mlx5e_close(struct net_device *netdev)return err; } +static void mlx5e_free_drop_rq (struct mlx5e_rq *rq) +{ +mlx5_wq_destroy (&rq->wq_ctrl); +} +static int mlx5e_alloc_drop_rq (struct mlx5_core_dev *mdev,struct mlx5e_rq *rq,struct mlx5e_rq_param *param)
@@ -3263,7 +3265,7 @@ int mlx5e_open_drop_rq(struct mlx5e_priv *priv,return 0 ; err_free_rq: -mlx5e_free_rq (drop_rq); +mlx5e_free_drop_rq (drop_rq); err_destroy_cq:mlx5e_destroy_cq (cq);
@@ -3277,7 +3279,7 @@ int mlx5e_open_drop_rq(struct mlx5e_priv *priv,void mlx5e_close_drop_rq (struct mlx5e_rq *drop_rq) {mlx5e_destroy_rq (drop_rq); -mlx5e_free_rq (drop_rq); +mlx5e_free_drop_rq (drop_rq);mlx5e_destroy_cq (&drop_rq->cq);mlx5e_free_cq (&drop_rq->cq); }
Signature · 1 line
2.26.2
PATCH v3 5/15 net/mlx5e: Change skb fifo push/pop API to be used without SQ
In reply to [email protected]From: Eran Ben Elisha <[email protected]>
The skb fifo push/pop API used pre-defined attributes within the mlx5e_txqsq.
In order to share the skb fifo API with other non-SQ use cases, change the API input to get newly defined mlx5e_skb_fifo struct.
--- drivers/net/ethernet/mellanox/mlx5/core/en.h | 10 ++++++++-- drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h | 15 +++++++++------ drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 13 ++++++++----- drivers/net/ethernet/mellanox/mlx5/core/en_tx.c | 6 +++--- 4 files changed, 28 insertions(+), 16 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en.h +8 -2
@@ -331,6 +331,13 @@ struct mlx5e_tx_mpwqe {u8 inline_on; }; +struct mlx5e_skb_fifo { +struct sk_buff **fifo; +u16 *pc; +u16 *cc; +u16 mask; +}; +struct mlx5e_txqsq {/* data path */
@@ -351,11 +358,10 @@ struct mlx5e_txqsq {/* read only */ struct mlx5_wq_cyc wq;u32 dma_fifo_mask; -u16 skb_fifo_mask;struct mlx5e_sq_stats *stats;struct {struct mlx5e_sq_dma *dma_fifo; -struct sk_buff **skb_fifo; +struct mlx5e_skb_fifo skb_fifo;struct mlx5e_tx_wqe_info *wqe_info; } db;void __iomem *uar_map;
drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h +9 -6
@@ -250,21 +250,24 @@ mlx5e_dma_push(struct mlx5e_txqsq *sq, dma_addr_t addr, u32 size, dma->type = map_type; } -static inline struct sk_buff **mlx5e_skb_fifo_get (struct mlx5e_txqsq *sq,u16 i) +static inline +struct sk_buff **mlx5e_skb_fifo_get (struct mlx5e_skb_fifo *fifo,u16 i) { -return &sq->db.skb_fifo[i & sq->skb_fifo_mask]; +return &fifo->fifo[i & fifo->mask]; } -static inline void mlx5e_skb_fifo_push (struct mlx5e_txqsq *sq,struct sk_buff *skb) +static inline +void mlx5e_skb_fifo_push (struct mlx5e_skb_fifo *fifo,struct sk_buff *skb) { -struct sk_buff **skb_item =mlx5e_skb_fifo_get (sq, sq->skb_fifo_pc++); +struct sk_buff **skb_item =mlx5e_skb_fifo_get (fifo, (*fifo->pc)++); *skb_item = skb; } -static inline struct sk_buff *mlx5e_skb_fifo_pop (struct mlx5e_txqsq *sq) +static inline +struct sk_buff *mlx5e_skb_fifo_pop (struct mlx5e_skb_fifo *fifo) { -return *mlx5e_skb_fifo_get (sq, sq->skb_fifo_cc++); +return *mlx5e_skb_fifo_get (fifo, (*fifo->cc)++); } static inline void
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +8 -5
@@ -1086,7 +1086,7 @@ static void mlx5e_free_icosq(struct mlx5e_icosq *sq)static void mlx5e_free_txqsq_db (struct mlx5e_txqsq *sq) {kvfree (sq->db.wqe_info); -kvfree (sq->db.skb_fifo); +kvfree (sq->db.skb_fifo.fifo);kvfree (sq->db.dma_fifo); }
@@ -1098,19 +1098,22 @@ static int mlx5e_alloc_txqsq_db(struct mlx5e_txqsq *sq, int numa) sq->db.dma_fifo =kvzalloc_node (array_size (df_sz,sizeof (*sq->db.dma_fifo)),GFP_KERNEL , numa); - sq->db.skb_fifo =kvzalloc_node (array_size (df_sz, -sizeof (*sq->db.skb_fifo)), + sq->db.skb_fifo.fifo =kvzalloc_node (array_size (df_sz, +sizeof (*sq->db.skb_fifo.fifo)),GFP_KERNEL , numa); sq->db.wqe_info =kvzalloc_node (array_size (wq_sz,sizeof (*sq->db.wqe_info)),GFP_KERNEL , numa); -if (!sq->db.dma_fifo || !sq->db.skb_fifo || !sq->db.wqe_info) { +if (!sq->db.dma_fifo || !sq->db.skb_fifo.fifo || !sq->db.wqe_info) {mlx5e_free_txqsq_db (sq);return -ENOMEM ; } sq->dma_fifo_mask = df_sz -1 ; - sq->skb_fifo_mask = df_sz -1 ; + + sq->db.skb_fifo.pc = &sq->skb_fifo_pc; + sq->db.skb_fifo.cc = &sq->skb_fifo_cc; + sq->db.skb_fifo.mask = df_sz -1 ;return 0 ; }
drivers/net/ethernet/mellanox/mlx5/core/en_tx.c +3 -3
@@ -579,7 +579,7 @@ mlx5e_sq_xmit_mpwqe(struct mlx5e_txqsq *sq, struct sk_buff *skb, goto err_unmap;mlx5e_dma_push (sq, txd.dma_addr, txd.len,MLX5E_DMA_MAP_SINGLE ); -mlx5e_skb_fifo_push (sq, skb); +mlx5e_skb_fifo_push (&sq->db.skb_fifo, skb);mlx5e_tx_mpwqe_add_dseg (sq, &txd);
@@ -719,7 +719,7 @@ static void mlx5e_tx_wi_consume_fifo_skbs(struct mlx5e_txqsq *sq, struct mlx5e_tint i;for (i =0 ; i < wi->num_fifo_pkts; i++) { -struct sk_buff *skb =mlx5e_skb_fifo_pop (sq); +struct sk_buff *skb =mlx5e_skb_fifo_pop (&sq->db.skb_fifo);mlx5e_consume_skb (sq, skb, cqe, napi_budget); }
@@ -839,7 +839,7 @@ static void mlx5e_tx_wi_kfree_fifo_skbs(struct mlx5e_txqsq *sq, struct mlx5e_tx_int i;for (i =0 ; i < wi->num_fifo_pkts; i++) -dev_kfree_skb_any (mlx5e_skb_fifo_pop (sq)); +dev_kfree_skb_any (mlx5e_skb_fifo_pop (&sq->db.skb_fifo)); }void mlx5e_free_txqsq_descs (struct mlx5e_txqsq *sq)
Signature · 1 line
2.26.2
PATCH v3 11/15 net/mlx5e: Remove duplicated include
In reply to [email protected]From: YueHaibing <[email protected]>
Remove duplicated include.
--- drivers/net/ethernet/mellanox/mlx5/core/en_rx.c | 1 - 1 file changed, 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +0 -1
@@ -52,7 +52,6 @@#include "en/xsk/rx.h" #include "en/health.h" #include "en/params.h" -#include "en/txrx.h" static struct sk_buff * mlx5e_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi,
Signature · 1 line
2.26.2
PATCH v3 12/15 net/mlx5: Arm only EQs with EQEs
In reply to [email protected]From: Shay Drory <[email protected]>
Currently, when more than one EQ is sharing an IRQ, and this IRQ is being interrupted, all the EQs sharing the IRQ will be armed. This is done regardless of whether an EQ has EQE.
When multiple EQs are sharing an IRQ, one or more EQs can have valid EQEs.
--- drivers/net/ethernet/mellanox/mlx5/core/eq.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/eq.c +2 -4
@@ -136,7 +136,7 @@ static int mlx5_eq_comp_int(struct notifier_block *nb, eqe =next_eqe_sw (eq);if (!eqe) - goto out; +return 0 ; do { struct mlx5_core_cq *cq;
@@ -161,8 +161,6 @@ static int mlx5_eq_comp_int(struct notifier_block *nb, ++eq->cons_index; }while ((++num_eqes <MLX5_EQ_POLLING_BUDGET ) && (eqe =next_eqe_sw (eq))); - -out:eq_update_ci (eq,1 );if (cqn !=-1 )
@@ -250,9 +248,9 @@ static int mlx5_eq_async_int(struct notifier_block *nb, ++eq->cons_index; }while ((++num_eqes <MLX5_EQ_POLLING_BUDGET ) && (eqe =next_eqe_sw (eq))); +eq_update_ci (eq,1 ); out: -eq_update_ci (eq,1 );mlx5_eq_async_int_unlock (eq_async, recovery, &flags);return unlikely (recovery) ? num_eqes :0 ;
Signature · 1 line
2.26.2
PATCH v3 3/15 net/mlx5e: Allow RQ outside of channel context
In reply to [email protected]From: Aya Levin <[email protected]>
In order to be able to create an RQ outside of a channel context, remove rq->channel direct pointer. This requires adding a direct pointer to:
ICOSQ and priv in order to support RQs that are part of mlx5e_channel. Use channel_stats from the corresponding CQ.
--- drivers/net/ethernet/mellanox/mlx5/core/en.h | 3 +- .../ethernet/mellanox/mlx5/core/en/health.c | 9 ++-- .../ethernet/mellanox/mlx5/core/en/health.h | 3 +- .../mellanox/mlx5/core/en/reporter_rx.c | 50 ++++++++++--------- .../mellanox/mlx5/core/en/reporter_tx.c | 2 +- .../net/ethernet/mellanox/mlx5/core/en_main.c | 23 ++++----- .../net/ethernet/mellanox/mlx5/core/en_rx.c | 22 ++++---- 7 files changed, 59 insertions(+), 53 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en.h +2 -1
@@ -595,7 +595,6 @@ struct mlx5e_rq {u8 map_dir;/* dma map direction */ } buff; -struct mlx5e_channel *channel;struct device *pdev;struct net_device *netdev;struct mlx5e_rq_stats *stats;
@@ -604,6 +603,8 @@ struct mlx5e_rq {struct mlx5e_page_cache page_cache;struct hwtstamp_config *tstamp;struct mlx5_clock *clock; +struct mlx5e_icosq *icosq; +struct mlx5e_priv *priv;mlx5e_fp_handle_rx_cqe handle_rx_cqe;mlx5e_fp_post_rx_wqes post_wqes;
drivers/net/ethernet/mellanox/mlx5/core/en/health.c +5 -4
@@ -205,21 +205,22 @@ int mlx5e_health_recover_channels(struct mlx5e_priv *priv)return err; } -int mlx5e_health_channel_eq_recover (struct mlx5_eq_comp *eq,struct mlx5e_channel *channel) +int mlx5e_health_channel_eq_recover (struct net_device *dev,struct mlx5_eq_comp *eq, +struct mlx5e_ch_stats *stats) {u32 eqe_count; -netdev_err (channel->netdev,"EQ 0x%x: Cons = 0x%x, irqn = 0x%x\n" , +netdev_err (dev,"EQ 0x%x: Cons = 0x%x, irqn = 0x%x\n" , eq->core.eqn, eq->core.cons_index, eq->core.irqn); eqe_count =mlx5_eq_poll_irq_disabled (eq);if (!eqe_count)return -EIO ; -netdev_err (channel->netdev,"Recovered %d eqes on EQ 0x%x\n" , +netdev_err (dev,"Recovered %d eqes on EQ 0x%x\n" , eqe_count, eq->core.eqn); - channel->stats->eq_rearm++; + stats->eq_rearm++;return 0 ; }
drivers/net/ethernet/mellanox/mlx5/core/en/health.h +2 -1
@@ -43,7 +43,8 @@ struct mlx5e_err_ctx { };int mlx5e_health_sq_to_ready (struct mlx5e_channel *channel,u32 sqn); -int mlx5e_health_channel_eq_recover (struct mlx5_eq_comp *eq,struct mlx5e_channel *channel); +int mlx5e_health_channel_eq_recover (struct net_device *dev,struct mlx5_eq_comp *eq, +struct mlx5e_ch_stats *stats);int mlx5e_health_recover_channels (struct mlx5e_priv *priv); int mlx5e_health_report(struct mlx5e_priv *priv, struct devlink_health_reporter *reporter, char *err_str,
drivers/net/ethernet/mellanox/mlx5/core/en/reporter_rx.c +27 -23
106 lines not on this page · raw message
drivers/net/ethernet/mellanox/mlx5/core/en/reporter_tx.c +1 -1
@@ -100,7 +100,7 @@ static int mlx5e_tx_reporter_timeout_recover(void *ctx) sq = to_ctx->sq; eq = sq->cq.mcq.eq; priv = sq->channel->priv; - err =mlx5e_health_channel_eq_recover (eq, sq->channel); + err =mlx5e_health_channel_eq_recover (sq->channel->netdev, eq, sq->channel->stats);if (!err) { to_ctx->status =0 ;/* this sq recovered */ return err;
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +10 -13
@@ -412,9 +412,10 @@ static int mlx5e_alloc_rq(struct mlx5e_channel *c, rq->wq_type = params->rq_wq_type; rq->pdev = c->pdev; rq->netdev = c->netdev; + rq->priv = c->priv; rq->tstamp = c->tstamp; rq->clock = &mdev->clock; - rq->channel = c; + rq->icosq = &c->icosq; rq->ix = c->ix; rq->mdev = mdev; rq->hw_mtu =MLX5E_SW2HW_MTU (params, params->sw_mtu);
@@ -617,7 +618,7 @@ static void mlx5e_free_rq(struct mlx5e_rq *rq)int i; old_prog =rcu_dereference_protected (rq->xdp_prog, -lockdep_is_held (&rq->channel->priv->state_lock)); +lockdep_is_held (&rq->priv->state_lock));if (old_prog)bpf_prog_put (old_prog);
@@ -717,9 +718,7 @@ int mlx5e_modify_rq_state(struct mlx5e_rq *rq, int curr_state, int next_state)static int mlx5e_modify_rq_scatter_fcs (struct mlx5e_rq *rq,bool enable) { -struct mlx5e_channel *c = rq->channel; -struct mlx5e_priv *priv = c->priv; -struct mlx5_core_dev *mdev = priv->mdev; +struct mlx5_core_dev *mdev = rq->mdev;void *in;void *rqc;
@@ -748,8 +747,7 @@ static int mlx5e_modify_rq_scatter_fcs(struct mlx5e_rq *rq, bool enable)static int mlx5e_modify_rq_vsd (struct mlx5e_rq *rq,bool vsd) { -struct mlx5e_channel *c = rq->channel; -struct mlx5_core_dev *mdev = c->mdev; +struct mlx5_core_dev *mdev = rq->mdev;void *in;void *rqc;int inlen;
@@ -783,7 +781,6 @@ static void mlx5e_destroy_rq(struct mlx5e_rq *rq)int mlx5e_wait_for_min_rx_wqes (struct mlx5e_rq *rq,int wait_time) {unsigned long exp_time = jiffies +msecs_to_jiffies (wait_time); -struct mlx5e_channel *c = rq->channel;u16 min_wqes =mlx5_min_rx_wqes (rq->wq_type,mlx5e_rqwq_get_size (rq));
@@ -794,8 +791,8 @@ int mlx5e_wait_for_min_rx_wqes(struct mlx5e_rq *rq, int wait_time)msleep (20 ); }while (time_before (jiffies, exp_time)); -netdev_warn (c->netdev,"Failed to get min RX wqes on Channel[%d] RQN[0x%x] wq cur_sz(%d) min_rx_wqes(%d)\n" , - c->ix, rq->rqn,mlx5e_rqwq_get_cur_sz (rq), min_wqes); +netdev_warn (rq->netdev,"Failed to get min RX wqes on Channel[%d] RQN[0x%x] wq cur_sz(%d) min_rx_wqes(%d)\n" , + rq->ix, rq->rqn,mlx5e_rqwq_get_cur_sz (rq), min_wqes);mlx5e_reporter_rx_timeout (rq);return -ETIMEDOUT ;
@@ -910,7 +907,7 @@ int mlx5e_open_rq(struct mlx5e_channel *c, struct mlx5e_params *params,void mlx5e_activate_rq (struct mlx5e_rq *rq) {set_bit (MLX5E_RQ_STATE_ENABLED , &rq->state); -mlx5e_trigger_irq (&rq->channel->icosq); +mlx5e_trigger_irq (rq->icosq); }void mlx5e_deactivate_rq (struct mlx5e_rq *rq)
@@ -922,7 +919,7 @@ void mlx5e_deactivate_rq(struct mlx5e_rq *rq)void mlx5e_close_rq (struct mlx5e_rq *rq) {cancel_work_sync (&rq->dim.work); -cancel_work_sync (&rq->channel->icosq.recover_work); +cancel_work_sync (&rq->icosq->recover_work);cancel_work_sync (&rq->recover_work);mlx5e_destroy_rq (rq);mlx5e_free_rx_descs (rq);
@@ -4411,7 +4408,7 @@ static void mlx5e_rq_replace_xdp_prog(struct mlx5e_rq *rq, struct bpf_prog *progstruct bpf_prog *old_prog; old_prog =rcu_replace_pointer (rq->xdp_prog, prog, -lockdep_is_held (&rq->channel->priv->state_lock)); +lockdep_is_held (&rq->priv->state_lock));if (old_prog)bpf_prog_put (old_prog); }
drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +12 -10
@@ -503,7 +503,7 @@ static int mlx5e_alloc_rx_mpwqe(struct mlx5e_rq *rq, u16 ix) {struct mlx5e_mpw_info *wi = &rq->mpwqe.info[ix];struct mlx5e_dma_info *dma_info = &wi->umr.dma_info[0 ]; -struct mlx5e_icosq *sq = &rq->channel->icosq; +struct mlx5e_icosq *sq = rq->icosq;struct mlx5_wq_cyc *wq = &sq->wq;struct mlx5e_umr_wqe *umr_wqe;u16 xlt_offset = ix << (MLX5E_LOG_ALIGNED_MPWQE_PPW -1 );
@@ -713,9 +713,9 @@ int mlx5e_poll_ico_cq(struct mlx5e_cq *cq) INDIRECT_CALLABLE_SCOPE bool mlx5e_post_rx_mpwqes(struct mlx5e_rq *rq) { -struct mlx5e_icosq *sq = &rq->channel->icosq;struct mlx5_wq_ll *wq = &rq->mpwqe.wq;u8 umr_completed = rq->mpwqe.umr_completed; +struct mlx5e_icosq *sq = rq->icosq;int alloc_err =0 ;u8 missing, i;u16 head;
@@ -1218,11 +1218,12 @@ mlx5e_skb_from_cqe_nonlinear(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe,static void trigger_report (struct mlx5e_rq *rq,struct mlx5_cqe64 *cqe) {struct mlx5_err_cqe *err_cqe = (struct mlx5_err_cqe *)cqe; +struct mlx5e_priv *priv = rq->priv;if (cqe_syndrome_needs_recover (err_cqe->syndrome) && !test_and_set_bit (MLX5E_RQ_STATE_RECOVERING , &rq->state)) {mlx5e_dump_error_cqe (&rq->cq, rq->rqn, err_cqe); -queue_work (rq->channel->priv->wq, &rq->recover_work); +queue_work (priv->wq, &rq->recover_work); } }
@@ -1771,8 +1772,9 @@ static void mlx5e_ipsec_handle_rx_cqe(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqint mlx5e_rq_set_handlers (struct mlx5e_rq *rq,struct mlx5e_params *params,bool xsk) { +struct net_device *netdev = rq->netdev;struct mlx5_core_dev *mdev = rq->mdev; -struct mlx5e_channel *c = rq->channel; +struct mlx5e_priv *priv = rq->priv;switch (rq->wq_type) {case MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ :
@@ -1784,15 +1786,15 @@ int mlx5e_rq_set_handlers(struct mlx5e_rq *rq, struct mlx5e_params *params, bool rq->post_wqes = mlx5e_post_rx_mpwqes; rq->dealloc_wqe = mlx5e_dealloc_rx_mpwqe; - rq->handle_rx_cqe = c->priv->profile->rx_handlers->handle_rx_cqe_mpwqe; + rq->handle_rx_cqe = priv->profile->rx_handlers->handle_rx_cqe_mpwqe;#ifdef CONFIG_MLX5_EN_IPSEC if (MLX5_IPSEC_DEV (mdev)) { -netdev_err (c->netdev,"MPWQE RQ with IPSec offload not supported\n" ); +netdev_err (netdev,"MPWQE RQ with IPSec offload not supported\n" );return -EINVAL ; }#endif if (!rq->handle_rx_cqe) { -netdev_err (c->netdev,"RX handler of MPWQE RQ is not set\n" ); +netdev_err (netdev,"RX handler of MPWQE RQ is not set\n" );return -EINVAL ; }break ;
@@ -1807,13 +1809,13 @@ int mlx5e_rq_set_handlers(struct mlx5e_rq *rq, struct mlx5e_params *params, bool#ifdef CONFIG_MLX5_EN_IPSEC if ((mlx5_fpga_ipsec_device_caps (mdev) &MLX5_ACCEL_IPSEC_CAP_DEVICE ) && - c->priv->ipsec) + priv->ipsec) rq->handle_rx_cqe = mlx5e_ipsec_handle_rx_cqe; else#endif - rq->handle_rx_cqe = c->priv->profile->rx_handlers->handle_rx_cqe; + rq->handle_rx_cqe = priv->profile->rx_handlers->handle_rx_cqe;if (!rq->handle_rx_cqe) { -netdev_err (c->netdev,"RX handler of RQ is not set\n" ); +netdev_err (netdev,"RX handler of RQ is not set\n" );return -EINVAL ; } }
Signature · 1 line
2.26.2
PATCH v3 13/15 net/mlx5: Fix passing zero to 'PTR_ERR'
In reply to [email protected]From: YueHaibing <[email protected]>
Fix smatch warnings:
drivers/net/ethernet/mellanox/mlx5/core/esw/acl/egress_lgcy.c:105 esw_acl_egress_lgcy_setup() warn: passing zero to 'PTR_ERR' drivers/net/ethernet/mellanox/mlx5/core/esw/acl/egress_ofld.c:177 esw_acl_egress_ofld_setup() warn: passing zero to 'PTR_ERR' drivers/net/ethernet/mellanox/mlx5/core/esw/acl/ingress_lgcy.c:184 esw_acl_ingress_lgcy_setup() warn: passing zero to 'PTR_ERR' drivers/net/ethernet/mellanox/mlx5/core/esw/acl/ingress_ofld.c:262 esw_acl_ingress_ofld_setup() warn: passing zero to 'PTR_ERR'
esw_acl_table_create() never returns NULL, so
NULL test should be removed.
--- drivers/net/ethernet/mellanox/mlx5/core/esw/acl/egress_lgcy.c | 2 +- drivers/net/ethernet/mellanox/mlx5/core/esw/acl/egress_ofld.c | 2 +- drivers/net/ethernet/mellanox/mlx5/core/esw/acl/ingress_lgcy.c | 2 +- drivers/net/ethernet/mellanox/mlx5/core/esw/acl/ingress_ofld.c | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/esw/acl/egress_lgcy.c +1 -1
@@ -101,7 +101,7 @@ int esw_acl_egress_lgcy_setup(struct mlx5_eswitch *esw, vport->egress.acl =esw_acl_table_create (esw, vport->vport,MLX5_FLOW_NAMESPACE_ESW_EGRESS , table_size); -if (IS_ERR_OR_NULL (vport->egress.acl)) { +if (IS_ERR (vport->egress.acl)) { err =PTR_ERR (vport->egress.acl); vport->egress.acl =NULL ; goto out;
drivers/net/ethernet/mellanox/mlx5/core/esw/acl/egress_ofld.c +1 -1
@@ -173,7 +173,7 @@ int esw_acl_egress_ofld_setup(struct mlx5_eswitch *esw, struct mlx5_vport *vport table_size++; vport->egress.acl =esw_acl_table_create (esw, vport->vport,MLX5_FLOW_NAMESPACE_ESW_EGRESS , table_size); -if (IS_ERR_OR_NULL (vport->egress.acl)) { +if (IS_ERR (vport->egress.acl)) { err =PTR_ERR (vport->egress.acl); vport->egress.acl =NULL ;return err;
drivers/net/ethernet/mellanox/mlx5/core/esw/acl/ingress_lgcy.c +1 -1
@@ -180,7 +180,7 @@ int esw_acl_ingress_lgcy_setup(struct mlx5_eswitch *esw, vport->ingress.acl =esw_acl_table_create (esw, vport->vport,MLX5_FLOW_NAMESPACE_ESW_INGRESS , table_size); -if (IS_ERR_OR_NULL (vport->ingress.acl)) { +if (IS_ERR (vport->ingress.acl)) { err =PTR_ERR (vport->ingress.acl); vport->ingress.acl =NULL ;return err;
drivers/net/ethernet/mellanox/mlx5/core/esw/acl/ingress_ofld.c +1 -1
@@ -258,7 +258,7 @@ int esw_acl_ingress_ofld_setup(struct mlx5_eswitch *esw, vport->ingress.acl =esw_acl_table_create (esw, vport->vport,MLX5_FLOW_NAMESPACE_ESW_INGRESS , num_ftes); -if (IS_ERR_OR_NULL (vport->ingress.acl)) { +if (IS_ERR (vport->ingress.acl)) { err =PTR_ERR (vport->ingress.acl); vport->ingress.acl =NULL ;return err;
Signature · 1 line
2.26.2
PATCH v3 14/15 net/mlx5e: Split between RX/TX tunnel FW support indication
In reply to [email protected]From: Aya Levin <[email protected]>
Use the new FW caps to advertise for ip-in-ip tunnel support separately for RX and TX.
--- .../net/ethernet/mellanox/mlx5/core/en/fs.h | 3 +- .../net/ethernet/mellanox/mlx5/core/en_fs.c | 20 +++++++---- .../net/ethernet/mellanox/mlx5/core/en_main.c | 34 ++++++++++++++++--- 3 files changed, 43 insertions(+), 14 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en/fs.h +1 -2
@@ -287,8 +287,7 @@ void mlx5e_disable_cvlan_filter(struct mlx5e_priv *priv);int mlx5e_create_flow_steering (struct mlx5e_priv *priv);void mlx5e_destroy_flow_steering (struct mlx5e_priv *priv); -bool mlx5e_tunnel_proto_supported (struct mlx5_core_dev *mdev,u8 proto_type); -bool mlx5e_any_tunnel_proto_supported (struct mlx5_core_dev *mdev); +u8 mlx5e_get_proto_by_tunnel_type (enum mlx5e_tunnel_types tt);#endif /* __MLX5E_FLOW_STEER_H__ */
drivers/net/ethernet/mellanox/mlx5/core/en_fs.c +13 -7
@@ -772,25 +772,31 @@ static struct mlx5e_etype_proto ttc_tunnel_rules[] = { }; -bool mlx5e_tunnel_proto_supported (struct mlx5_core_dev *mdev,u8 proto_type) +u8 mlx5e_get_proto_by_tunnel_type (enum mlx5e_tunnel_types tt) +{ +return ttc_tunnel_rules[tt].proto; +} + +static bool mlx5e_tunnel_proto_supported_rx (struct mlx5_core_dev *mdev,u8 proto_type) {switch (proto_type) {case IPPROTO_GRE :return MLX5_CAP_ETH (mdev, tunnel_stateless_gre);case IPPROTO_IPIP :case IPPROTO_IPV6 : -return MLX5_CAP_ETH (mdev, tunnel_stateless_ip_over_ip); +return (MLX5_CAP_ETH (mdev, tunnel_stateless_ip_over_ip) || +MLX5_CAP_ETH (mdev, tunnel_stateless_ip_over_ip_rx));default :return false; } } -bool mlx5e_any_tunnel_proto_supported (struct mlx5_core_dev *mdev) +static bool mlx5e_tunnel_any_rx_proto_supported (struct mlx5_core_dev *mdev) {int tt;for (tt =0 ; tt <MLX5E_NUM_TUNNEL_TT ; tt++) { -if (mlx5e_tunnel_proto_supported (mdev, ttc_tunnel_rules[tt].proto)) +if (mlx5e_tunnel_proto_supported_rx (mdev, ttc_tunnel_rules[tt].proto))return true; }return false;
@@ -798,7 +804,7 @@ bool mlx5e_any_tunnel_proto_supported(struct mlx5_core_dev *mdev)bool mlx5e_tunnel_inner_ft_supported (struct mlx5_core_dev *mdev) { -return (mlx5e_any_tunnel_proto_supported (mdev) && +return (mlx5e_tunnel_any_rx_proto_supported (mdev) &&MLX5_CAP_FLOWTABLE_NIC_RX (mdev, ft_field_support.inner_ip_version)); }
@@ -899,8 +905,8 @@ static int mlx5e_generate_ttc_table_rules(struct mlx5e_priv *priv, dest.type =MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE ; dest.ft = params->inner_ttc->ft.t;for (tt =0 ; tt <MLX5E_NUM_TUNNEL_TT ; tt++) { -if (!mlx5e_tunnel_proto_supported (priv->mdev, - ttc_tunnel_rules[tt].proto)) +if (!mlx5e_tunnel_proto_supported_rx (priv->mdev, + ttc_tunnel_rules[tt].proto))continue ; trules[tt] = mlx5e_generate_ttc_rule(priv, ft, &dest, ttc_tunnel_rules[tt].etype,
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +29 -5
@@ -4280,6 +4280,20 @@ int mlx5e_get_vf_stats(struct net_device *dev, }#endif +static bool mlx5e_tunnel_proto_supported_tx (struct mlx5_core_dev *mdev,u8 proto_type) +{ +switch (proto_type) { +case IPPROTO_GRE : +return MLX5_CAP_ETH (mdev, tunnel_stateless_gre); +case IPPROTO_IPIP : +case IPPROTO_IPV6 : +return (MLX5_CAP_ETH (mdev, tunnel_stateless_ip_over_ip) || +MLX5_CAP_ETH (mdev, tunnel_stateless_ip_over_ip_tx)); +default : +return false; + } +} +static bool mlx5e_gre_tunnel_inner_proto_offload_supported (struct mlx5_core_dev *mdev,struct sk_buff *skb) {
@@ -4322,7 +4336,7 @@ static netdev_features_t mlx5e_tunnel_features_check(struct mlx5e_priv *priv,break ;case IPPROTO_IPIP :case IPPROTO_IPV6 : -if (mlx5e_tunnel_proto_supported (priv->mdev,IPPROTO_IPIP )) +if (mlx5e_tunnel_proto_supported_tx (priv->mdev,IPPROTO_IPIP ))return features;break ;case IPPROTO_UDP :
@@ -4882,6 +4896,17 @@ void mlx5e_vxlan_set_netdev_info(struct mlx5e_priv *priv) priv->netdev->udp_tunnel_nic_info = &priv->nic_info; } +static bool mlx5e_tunnel_any_tx_proto_supported (struct mlx5_core_dev *mdev) +{ +int tt; + +for (tt =0 ; tt <MLX5E_NUM_TUNNEL_TT ; tt++) { +if (mlx5e_tunnel_proto_supported_tx (mdev,mlx5e_get_proto_by_tunnel_type (tt))) +return true; + } +return (mlx5_vxlan_allowed (mdev->vxlan) ||mlx5_geneve_tx_allowed (mdev)); +} +static void mlx5e_build_nic_netdev (struct net_device *netdev) {struct mlx5e_priv *priv =netdev_priv (netdev);
@@ -4927,8 +4952,7 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev)mlx5e_vxlan_set_netdev_info (priv); -if (mlx5_vxlan_allowed (mdev->vxlan) ||mlx5_geneve_tx_allowed (mdev) || -mlx5e_any_tunnel_proto_supported (mdev)) { +if (mlx5e_tunnel_any_tx_proto_supported (mdev)) { netdev->hw_enc_features |=NETIF_F_HW_CSUM ; netdev->hw_enc_features |=NETIF_F_TSO ; netdev->hw_enc_features |=NETIF_F_TSO6 ;
@@ -4945,7 +4969,7 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev)NETIF_F_GSO_UDP_TUNNEL_CSUM ; } -if (mlx5e_tunnel_proto_supported (mdev,IPPROTO_GRE )) { +if (mlx5e_tunnel_proto_supported_tx (mdev,IPPROTO_GRE )) { netdev->hw_features |=NETIF_F_GSO_GRE |NETIF_F_GSO_GRE_CSUM ; netdev->hw_enc_features |= NETIF_F_GSO_GRE |
@@ -4954,7 +4978,7 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev)NETIF_F_GSO_GRE_CSUM ; } -if (mlx5e_tunnel_proto_supported (mdev,IPPROTO_IPIP )) { +if (mlx5e_tunnel_proto_supported_tx (mdev,IPPROTO_IPIP )) { netdev->hw_features |=NETIF_F_GSO_IPXIP4 |NETIF_F_GSO_IPXIP6 ; netdev->hw_enc_features |= NETIF_F_GSO_IPXIP4 |
Signature · 1 line
2.26.2
PATCH v3 2/15 net/mlx5e: Allow CQ outside of channel context
In reply to [email protected]From: Aya Levin <[email protected]>
In order to be able to create a CQ outside of a channel context, remove cq->channel direct pointer. This requires adding a direct pointer to channel statistics, netdevice, priv and to mlx5_core in order to support CQs that are a part of mlx5e_channel.
In addition, parameters the were previously derived from the channel like napi, NUMA node, channel stats and index are now assembled in struct mlx5e_create_cq_param which is given to mlx5e_open_cq() instead of channel pointer. Generalizing mlx5e_open_cq() allows opening CQ outside of channel context which will be used in following patches in the patch-set.
--- drivers/net/ethernet/mellanox/mlx5/core/en.h | 16 ++++- .../ethernet/mellanox/mlx5/core/en/health.c | 3 +- .../net/ethernet/mellanox/mlx5/core/en/txrx.h | 2 +- .../mellanox/mlx5/core/en/xsk/setup.c | 12 +++- .../net/ethernet/mellanox/mlx5/core/en_main.c | 67 ++++++++++++------- .../net/ethernet/mellanox/mlx5/core/en_rx.c | 6 +- .../net/ethernet/mellanox/mlx5/core/en_tx.c | 3 +- .../net/ethernet/mellanox/mlx5/core/en_txrx.c | 5 +- 8 files changed, 73 insertions(+), 41 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en.h +13 -3
@@ -282,10 +282,12 @@ struct mlx5e_cq {u16 event_ctr;struct napi_struct *napi;struct mlx5_core_cq mcq; -struct mlx5e_channel *channel; +struct mlx5e_ch_stats *ch_stats;/* control */ +struct net_device *netdev;struct mlx5_core_dev *mdev; +struct mlx5e_priv *priv;struct mlx5_wq_ctrl wq_ctrl; }____cacheline_aligned_in_smp ;
@@ -923,9 +925,17 @@ int mlx5e_open_xdpsq(struct mlx5e_channel *c, struct mlx5e_params *params, struct mlx5e_xdpsq *sq, bool is_redirect);void mlx5e_close_xdpsq (struct mlx5e_xdpsq *sq); +struct mlx5e_create_cq_param { +struct napi_struct *napi; +struct mlx5e_ch_stats *ch_stats; +int node; +int ix; +}; +struct mlx5e_cq_param ; -int mlx5e_open_cq (struct mlx5e_channel *c,struct dim_cq_moder moder, -struct mlx5e_cq_param *param,struct mlx5e_cq *cq); +int mlx5e_open_cq (struct mlx5e_priv *priv,struct dim_cq_moder moder, +struct mlx5e_cq_param *param,struct mlx5e_create_cq_param *ccp, +struct mlx5e_cq *cq);void mlx5e_close_cq (struct mlx5e_cq *cq);int mlx5e_open_locked (struct net_device *netdev);
drivers/net/ethernet/mellanox/mlx5/core/en/health.c +1 -2
@@ -37,13 +37,12 @@ int mlx5e_health_fmsg_named_obj_nest_end(struct devlink_fmsg *fmsg)int mlx5e_health_cq_diag_fmsg (struct mlx5e_cq *cq,struct devlink_fmsg *fmsg) { -struct mlx5e_priv *priv = cq->channel->priv;u32 out[MLX5_ST_SZ_DW (query_cq_out)] = {};u8 hw_status;void *cqc;int err; - err =mlx5_core_query_cq (priv->mdev, &cq->mcq, out); + err =mlx5_core_query_cq (cq->mdev, &cq->mcq, out);if (err)return err;
drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h +1 -1
@@ -308,7 +308,7 @@ static inline void mlx5e_dump_error_cqe(struct mlx5e_cq *cq, u32 qn, ci =mlx5_cqwq_ctr2ix (wq, wq->cc -1 ); - netdev_err(cq->channel->netdev, + netdev_err(cq->netdev, "Error cqe on cqn 0x%x, ci 0x%x, qn 0x%x, opcode 0x%x, syndrome 0x%x, vendor syndrome 0x%x\n", cq->mcq.cqn, ci, qn, get_cqe_opcode((struct mlx5_cqe64 *)err_cqe),
drivers/net/ethernet/mellanox/mlx5/core/en/xsk/setup.c +10 -2
@@ -48,9 +48,15 @@ int mlx5e_open_xsk(struct mlx5e_priv *priv, struct mlx5e_params *params, struct mlx5e_xsk_param *xsk, struct xsk_buff_pool *pool, struct mlx5e_channel *c) { + struct mlx5e_create_cq_param ccp = {};struct mlx5e_channel_param *cparam;int err; + ccp.napi = &c->napi; + ccp.ch_stats = c->stats; + ccp.node =cpu_to_node (c->cpu); + ccp.ix = c->ix; +if (!mlx5e_validate_xsk_param (params, xsk, priv->mdev))return -EINVAL ;
@@ -60,7 +66,8 @@ int mlx5e_open_xsk(struct mlx5e_priv *priv, struct mlx5e_params *params,mlx5e_build_xsk_cparam (priv, params, xsk, cparam); - err =mlx5e_open_cq (c, params->rx_cq_moderation, &cparam->rq.cqp, &c->xskrq.cq); + err =mlx5e_open_cq (c->priv, params->rx_cq_moderation, &cparam->rq.cqp, &ccp, + &c->xskrq.cq);if (unlikely (err)) goto err_free_cparam;
@@ -68,7 +75,8 @@ int mlx5e_open_xsk(struct mlx5e_priv *priv, struct mlx5e_params *params,if (unlikely (err)) goto err_close_rx_cq; - err =mlx5e_open_cq (c, params->tx_cq_moderation, &cparam->xdp_sq.cqp, &c->xsksq.cq); + err =mlx5e_open_cq (c->priv, params->tx_cq_moderation, &cparam->xdp_sq.cqp, &ccp, + &c->xsksq.cq);if (unlikely (err)) goto err_close_rq;
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +42 -25
155 lines not on this page · raw message
drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +3 -3
@@ -670,13 +670,13 @@ int mlx5e_poll_ico_cq(struct mlx5e_cq *cq) sqcc += wi->num_wqebbs;if (last_wqe &&unlikely (get_cqe_opcode (cqe) !=MLX5_CQE_REQ )) { -netdev_WARN_ONCE (cq->channel->netdev, +netdev_WARN_ONCE (cq->netdev,"Bad OP in ICOSQ CQE: 0x%x\n" ,get_cqe_opcode (cqe));mlx5e_dump_error_cqe (&sq->cq, sq->sqn, (struct mlx5_err_cqe *)cqe);if (!test_and_set_bit (MLX5E_SQ_STATE_RECOVERING , &sq->state)) -queue_work (cq->channel->priv->wq, &sq->recover_work); +queue_work (cq->priv->wq, &sq->recover_work);break ; }
@@ -697,7 +697,7 @@ int mlx5e_poll_ico_cq(struct mlx5e_cq *cq)break ;#endif default : -netdev_WARN_ONCE (cq->channel->netdev, +netdev_WARN_ONCE (cq->netdev,"Bad WQE type in ICOSQ WQE info: 0x%x\n" , wi->wqe_type); }
drivers/net/ethernet/mellanox/mlx5/core/en_tx.c +1 -2
@@ -805,8 +805,7 @@ bool mlx5e_poll_tx_cq(struct mlx5e_cq *cq, int napi_budget)mlx5e_dump_error_cqe (&sq->cq, sq->sqn, (struct mlx5_err_cqe *)cqe);mlx5_wq_cyc_wqe_dump (&sq->wq, ci, wi->num_wqebbs); -queue_work (cq->channel->priv->wq, - &sq->recover_work); +queue_work (cq->priv->wq, &sq->recover_work); } stats->cqe_err++; }
drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c +2 -3
@@ -221,14 +221,13 @@ void mlx5e_completion_event(struct mlx5_core_cq *mcq, struct mlx5_eqe *eqe)napi_schedule (cq->napi); cq->event_ctr++; - cq->channel->stats->events++; + cq->ch_stats->events++; }void mlx5e_cq_error_event (struct mlx5_core_cq *mcq,enum mlx5_event event) { struct mlx5e_cq *cq = container_of(mcq, struct mlx5e_cq, mcq); -struct mlx5e_channel *c = cq->channel; -struct net_device *netdev = c->netdev; +struct net_device *netdev = cq->netdev;netdev_err (netdev,"%s: cqn=0x%.6x event=0x%.2x\n" , __func__, mcq->cqn, event);
Signature · 1 line
2.26.2
PATCH v3 15/15 net/mlx5e: Fill mlx5e_create_cq_param in a function
In reply to [email protected]From: Maxim Mikityanskiy <[email protected]>
Create a function to fill the fields of struct mlx5e_create_cq_param based on a channel. The purpose is code reuse between normal CQs, XSK CQs and the upcoming QoS CQs.
--- .../net/ethernet/mellanox/mlx5/core/en/params.h | 1 + .../ethernet/mellanox/mlx5/core/en/xsk/setup.c | 7 ++----- .../net/ethernet/mellanox/mlx5/core/en_main.c | 17 ++++++++++++----- 3 files changed, 15 insertions(+), 10 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en/params.h +1 -0
@@ -111,6 +111,7 @@ u16 mlx5e_get_rq_headroom(struct mlx5_core_dev *mdev,/* Build queue parameters */ +void mlx5e_build_create_cq_param (struct mlx5e_create_cq_param *ccp,struct mlx5e_channel *c); void mlx5e_build_rq_param(struct mlx5e_priv *priv, struct mlx5e_params *params, struct mlx5e_xsk_param *xsk,
drivers/net/ethernet/mellanox/mlx5/core/en/xsk/setup.c +2 -5
@@ -48,14 +48,11 @@ int mlx5e_open_xsk(struct mlx5e_priv *priv, struct mlx5e_params *params, struct mlx5e_xsk_param *xsk, struct xsk_buff_pool *pool, struct mlx5e_channel *c) { - struct mlx5e_create_cq_param ccp = {}; struct mlx5e_channel_param *cparam; +struct mlx5e_create_cq_param ccp;int err; - ccp.napi = &c->napi; - ccp.ch_stats = c->stats; - ccp.node =cpu_to_node (c->cpu); - ccp.ix = c->ix; +mlx5e_build_create_cq_param (&ccp, c);if (!mlx5e_validate_xsk_param (params, xsk, priv->mdev))return -EINVAL ;
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +12 -5
@@ -1806,18 +1806,25 @@ static int mlx5e_set_tx_maxrate(struct net_device *dev, int index, u32 rate)return err; } +void mlx5e_build_create_cq_param (struct mlx5e_create_cq_param *ccp,struct mlx5e_channel *c) +{ + *ccp = (struct mlx5e_create_cq_param ) { + .napi = &c->napi, + .ch_stats = c->stats, + .node =cpu_to_node (c->cpu), + .ix = c->ix, + }; +} +static int mlx5e_open_queues (struct mlx5e_channel *c,struct mlx5e_params *params,struct mlx5e_channel_param *cparam) {struct dim_cq_moder icocq_moder = {0 ,0 }; -struct mlx5e_create_cq_param ccp = {}; +struct mlx5e_create_cq_param ccp;int err; - ccp.napi = &c->napi; - ccp.ch_stats = c->stats; - ccp.node =cpu_to_node (c->cpu); - ccp.ix = c->ix; +mlx5e_build_create_cq_param (&ccp, c); err =mlx5e_open_cq (c->priv, icocq_moder, &cparam->icosq.cqp, &ccp, &c->async_icosq.cq);
Signature · 1 line
2.26.2
PATCH v3 10/15 net/mlx5e: remove unnecessary memset
In reply to [email protected]From: Zhu Yanjun <[email protected]>
Since kvzalloc will initialize the allocated memory, it is not necessary to initialize it once again.
--- drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c | 1 - 1 file changed, 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c +0 -1
@@ -1680,7 +1680,6 @@ static int esw_create_restore_table(struct mlx5_eswitch *esw) goto out_free; } -memset (flow_group_in,0 , inlen); match_criteria =MLX5_ADDR_OF (create_flow_group_in, flow_group_in, match_criteria); misc = MLX5_ADDR_OF(fte_match_param, match_criteria,
Signature · 1 line
2.26.2
David Miller
maintainerFrom: [email protected]
Date: Tue, 8 Dec 2020 11:35:40 -0800
Quoted 6 lines
From: Saeed Mahameed <[email protected]>
Hi Jakub,
v1->v2: Removed merge commit of mlx5-next.
Quoted 8 lines
v2->v3:
- Add accuracy improvement measurements.
- Apply the accurate stamping only on PTP port and not all UDP.This series adds port tx timestamping support and some misc updates. For more information please see tag log below.
Please pull and let me know if there is any problem.
Pulled, thanks.