bnxt_en: Add kTLS TX offload support
Michael Chan
maintainerv6 cover bnxt_en: Add kTLS TX offload support
This patchset adds kTLS offload support for TX direction. A number of new files are added:
bnxt_mpc.[ch] handle midpath channels (MPCs) used to offload kTLS connections to the chip's crypto blocks without going through FW.
bnxt_crypto.[ch] handle the crypto interface and resources.
bnxt_ktls.[ch] handle kTLS offload.
A new CONFIG_BNXT_TLS is added to enable all of the above. The first 6 patches add the MPC logic including resource accounting and reservations. The next 5 patches add the crypto logic to handle the crypto resources and to send/receive control data using the MPCs. The last 4 patches add kTLS offload for the TX direction.
There will be a follow-on patchset to make the TX offload more complete and to add the RX direction offload.
v6:
Fix more Sashiko reported issues:
Patch #1:
Make sure mpc capability is only updated during reset or ifdown/up.
Patch #10:
Handle both long and short MPC completions. Leave one empty slot in the MPC ring just like TX ring. Use memory barrier when updating tx_cons for MPC ring.
Patch #11:
Disable NAPI during MPC ring restart.
Add bnxt_free_mpc_entries() called during shutdown like other rings. Simplify bnxt_xmit_crypto_cmd() error path.
Patch #13:
Goto kick_pending if kTLS transmit failed to flush the DB. Handle the case of BPF truncating the TX packet.
Patch #14:
Clear is_push and inline_data_bds during TX ring free.
v5:
https://lore.kernel.org/netdev/[email protected]/
Fix most valid Sashiko reported issues or add comments to clarify:
Patch #2:
Clarify in the commit log that kTLS, MPC rings are only supported on P5_PLUS chips in response to some Sashiko comments.
Patch #3:
Improve the ethtool -L MPC calculation to use the actual MPC rings during ring check.
Add a comment in the code to explain the bnxt_trim_mpc_ring() call.
Patch #5:
Check for P5_PLUS before allocating mpc_info to more clearly document that MPC rings are only supported on P5_PLUS chips.
Patch #8:
Pre-existing Sashiko issue not addressed.
Patch #9:
More clearly document and explain that bnxt_clear_crypto() has no concurrent readers/writers.
Add comment to explain that the bnxt_free_one_ctx() caller is reponsible to check the epoch value when needed.
Patch #10:
Add description in commit log that crypto engines use short completions only.
Already explained that only P5_PLUS supports MPC.
Patch #11:
Defer the MPC ring reset to bnxt_sp_task() just like other resets to avoid any possibility od deadlock.
Add more comments to explain the MPC ring disable, stop, start.
Patch #12:
The bnxt_drv_busy() is the existing driver scheme to wait for activities to stop during shutdown.
Patch #13:
The kTLS software counters match the other driver software counters using plain u64. We can add u64_stats_sync for 32-bit systems but it makes more sense to add it to all the existing counters as well.
Use BNXT_NO_FW_ACCESS() to fix possible massive AER timeout.
Patch #15:
Use bool instead of bit fields to keep track of kTLS states (requested by Paolo).
v4:
https://lore.kernel.org/netdev/[email protected]/
Fix kerneldoc prototype warning and uninitialized variable warnings reported by Jakub.
Fix most valid Sashiko reported issues.
v3:
https://lore.kernel.org/netdev/[email protected]/
Fix most AI reported issues from Jakub.
v2:
https://lore.kernel.org/netdev/[email protected]/
Fix unused variable compile warnings in patch 10 and 12 by reorganizing the patches (reported by Jakub)
Fix some error recovery issues in patch 12
v1:
https://lore.kernel.org/netdev/[email protected]/
Michael Chan (15):
bnxt_en: Add Midpath channel information
bnxt_en: Account for the MPC TX and CP rings
bnxt_en: Set default MPC ring count
bnxt_en: Rename xdp_tx_lock to tx_lock
bnxt_en: Allocate and free MPC software structures
bnxt_en: Allocate and free MPC channels from firmware
bnxt_en: Allocate crypto structure and backing store
bnxt_en: Reserve crypto RX and TX key contexts on a PF
bnxt_en: Add infrastructure for crypto key context IDs
bnxt_en: Add MPC transmit and completion functions
bnxt_en: Add crypto MPC transmit/completion infrastructure
bnxt_en: Support kTLS TX offload by implementing .tls_dev_add/del()
bnxt_en: Implement kTLS TX normal path
bnxt_en: Add support for inline transmit BDs
bnxt_en: Add kTLS retransmission support
drivers/net/ethernet/broadcom/Kconfig | 9 + drivers/net/ethernet/broadcom/bnxt/Makefile | 1 + drivers/net/ethernet/broadcom/bnxt/bnxt.c | 303 +++++-- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 96 ++- .../net/ethernet/broadcom/bnxt/bnxt_crypto.c | 618 ++++++++++++++ .../net/ethernet/broadcom/bnxt/bnxt_crypto.h | 230 +++++ .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 61 +- drivers/net/ethernet/broadcom/bnxt/bnxt_gso.c | 2 +- .../net/ethernet/broadcom/bnxt/bnxt_ktls.c | 578 +++++++++++++ .../net/ethernet/broadcom/bnxt/bnxt_ktls.h | 174 ++++ drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c | 795 ++++++++++++++++++ drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h | 232 +++++ .../net/ethernet/broadcom/bnxt/bnxt_sriov.c | 6 +- drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c | 8 +- include/linux/bnxt/hsi.h | 37 + 15 files changed, 3083 insertions(+), 67 deletions(-) create mode 100644 drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c create mode 100644 drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.h create mode 100644 drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c create mode 100644 drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.h create mode 100644 drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c create mode 100644 drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h
Signature · 1 line
2.51.0
Michael Chan
maintainerPATCH v6 1/15 bnxt_en: Add Midpath channel information
Midpath channels (MPCs) are rings for hardware control paths. These control paths are used to offload kTLS directly to the hardware without going through firmware. This patch adds the basic information structures for these MPCs.
An MPC is basically a TX and completion ring pair with a HW TLS block as the destination. Two MPC channel types are used to offload connections to the TX crypto engine (TCE) and the RX crypto engine (RCE) respectively. In the driver, we re-use the bnxt_tx_ring_info and bnxt_cp_ring_info control structs for the MPCs.
This patch also adds the CONFIG_BNXT_TLS Kconfig option to conditionally include the MPC logic. The first few patches in the series add the MPC support. kTLS support will be added later in the series.
---
v6:
Make sure mpc capability is only updated during reset or ifdown/up.
--- drivers/net/ethernet/broadcom/Kconfig | 9 ++++ drivers/net/ethernet/broadcom/bnxt/Makefile | 1 + drivers/net/ethernet/broadcom/bnxt/bnxt.c | 12 +++++ drivers/net/ethernet/broadcom/bnxt/bnxt.h | 2 + drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c | 26 ++++++++++ drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h | 47 +++++++++++++++++++ 6 files changed, 97 insertions(+) create mode 100644 drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c create mode 100644 drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h
drivers/net/ethernet/broadcom/Kconfig +9 -0
@@ -255,6 +255,15 @@ config BNXT_HWMON Say Y if you want to expose the thermal sensor data on NetXtreme-C/E devices, via the hwmon sysfs interface. +config BNXT_TLS + bool "Broadcom NetXtreme-C/E TLS offload support" + default y + depends on BNXT && TLS_DEVICE + depends on TLS=y || BNXT=m + help + Say Y if you want to enable Transport Layer Security (TLS) hardware + encryption and decryption offload on supported NetXtreme-C/E devices. + config BNGE tristate "Broadcom ThorUltra Ethernet device support" depends on PCI
drivers/net/ethernet/broadcom/bnxt/Makefile +1 -0
@@ -5,3 +5,4 @@ bnxt_en-y := bnxt.o bnxt_hwrm.o bnxt_sriov.o bnxt_ethtool.o bnxt_dcb.o bnxt_ulp. bnxt_en-$(CONFIG_BNXT_FLOWER_OFFLOAD) += bnxt_tc.o bnxt_en-$(CONFIG_DEBUG_FS) += bnxt_debugfs.o bnxt_en-$(CONFIG_BNXT_HWMON) += bnxt_hwmon.o +bnxt_en-$(CONFIG_BNXT_TLS) += bnxt_mpc.o
drivers/net/ethernet/broadcom/bnxt/bnxt.c +12 -0
@@ -76,6 +76,7 @@#include "bnxt_hwmon.h" #include "bnxt_gso.h" #include <net/tso.h> +#include "bnxt_mpc.h" #define BNXT_TX_TIMEOUT (5 * HZ)#define BNXT_DEF_MSG_ENABLE (NETIF_MSG_DRV | NETIF_MSG_HW | \
@@ -9953,6 +9954,15 @@ static int __bnxt_hwrm_func_qcaps(struct bnxt *bp) } bp->tso_max_segs =le16_to_cpu (resp->max_tso_segs); +/* mpc capability can only change during FW reset and ifdown/ifup */ +if (!test_bit (BNXT_STATE_OPEN , &bp->state) || +test_bit (BNXT_STATE_IN_FW_RESET , &bp->state)) { +if (resp->mpc_chnls_cap) +bnxt_alloc_mpc_info (bp, resp->mpc_chnls_cap); +else +bnxt_free_mpc_info (bp); + } + hwrm_func_qcaps_exit:hwrm_req_drop (bp, req);return rc;
@@ -16569,6 +16579,7 @@ static void bnxt_remove_one(struct pci_dev *pdev) bp->ptp_cfg =NULL ;kfree (bp->fw_health); bp->fw_health =NULL ; +bnxt_free_mpc_info (bp);bnxt_cleanup_pci (bp);bnxt_free_ctx_mem (bp, true);bnxt_free_crash_dump_mem (bp);
@@ -17240,6 +17251,7 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)bnxt_ethtool_free (bp);kfree (bp->fw_health); bp->fw_health =NULL ; +bnxt_free_mpc_info (bp);bnxt_cleanup_pci (bp);bnxt_free_ctx_mem (bp, true);bnxt_free_crash_dump_mem (bp);
drivers/net/ethernet/broadcom/bnxt/bnxt.h +2 -0
@@ -2453,6 +2453,8 @@ struct bnxt {u8 tph_mode; +struct bnxt_mpc_info *mpc_info; +unsigned int current_interval;#define BNXT_TIMER_INTERVAL HZ
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c new file +26 -0
@@ -0,0 +1,26 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright (c) 2026 Broadcom Inc. */ + +#include <linux/stddef.h> +#include <linux/types.h> +#include <linux/slab.h> +#include <linux/bnxt/hsi.h> + +#include "bnxt.h" +#include "bnxt_mpc.h" + +void bnxt_alloc_mpc_info (struct bnxt *bp,u8 mpc_chnls_cap) +{ +if (!bp->mpc_info) + bp->mpc_info =kzalloc_obj (*bp->mpc_info); +if (bp->mpc_info) + bp->mpc_info->mpc_chnls_cap = mpc_chnls_cap; +else +netdev_warn (bp->dev,"Unable to allocate MPC info\n" ); +} + +void bnxt_free_mpc_info (struct bnxt *bp) +{ +kfree (bp->mpc_info); + bp->mpc_info =NULL ; +}
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h new file +47 -0
47 lines not on this page · raw message
Signature · 1 line
2.51.0
Michael Chan
maintainerPATCH v6 2/15 bnxt_en: Account for the MPC TX and CP rings
In reply to Michael ChanModify bnxt_cp_rings_in_use(), bnxt_get_max_func_cp_rings_for_en(), and _bnxt_get_max_rings() to account for any TX rings and CP rings used by MPCs. Add a new helper bnxt_total_tx_rings() to include MPC TX rings. Ring reservations will now include the MPC rings.
Note that for bnxt_check_rings() that is called before we commit the new rings requested by the user running ethtool -L, the next patch will calculate the proposed MPC rings to be checked.
Note that only the newer FW APIs and the newer P5-plus chips support MPC rings. The MPC values will always be zero in the non-P5 plus code paths.
---
v5:
https://lore.kernel.org/netdev/[email protected]/
Clarify in the commit log that kTLS, MPC rings are only supported on P5_PLUS chips in response to some Sashiko comments.
--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 65 +++++++++++++------ drivers/net/ethernet/broadcom/bnxt/bnxt.h | 3 +- .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 5 +- drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c | 21 ++++++ drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h | 12 ++++ .../net/ethernet/broadcom/bnxt/bnxt_sriov.c | 6 +- drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c | 4 +- 7 files changed, 90 insertions(+), 26 deletions(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +45 -20
@@ -7762,6 +7762,11 @@ static void bnxt_hwrm_ring_free(struct bnxt *bp, bool close_path) } } +int bnxt_total_tx_rings (struct bnxt *bp) +{ +return bp->tx_nr_rings +bnxt_mpc_tx_rings_in_use (bp); +} +static int __bnxt_trim_rings (struct bnxt *bp,int *rx,int *tx,int max,bool shared); static int bnxt_trim_rings(struct bnxt *bp, int *rx, int *tx, int max,
@@ -7802,19 +7807,28 @@ static int bnxt_hwrm_get_rings(struct bnxt *bp) stats =le16_to_cpu (resp->alloc_stat_ctx); hw_resc->resv_irqs = cp;if (bp->flags &BNXT_FLAG_CHIP_P5_PLUS ) { +int mpc_tx =bnxt_mpc_tx_rings_in_use (bp); +int mpc_cp =bnxt_mpc_cp_rings_in_use (bp);int rx = hw_resc->resv_rx_rings;int tx = hw_resc->resv_tx_rings; +int cp_p5; +if (tx <= mpc_tx || cp <= mpc_cp) { + rc = -ENOMEM ; + goto get_rings_exit; + } + tx -= mpc_tx; + cp_p5 = cp - mpc_cp;if (bp->flags &BNXT_FLAG_AGG_RINGS ) rx >>=1 ; -if (cp < (rx + tx)) { - rc =__bnxt_trim_rings (bp, &rx, &tx, cp, false); +if (cp_p5 < (rx + tx)) { + rc =__bnxt_trim_rings (bp, &rx, &tx, cp_p5, false);if (rc) goto get_rings_exit;if (bp->flags &BNXT_FLAG_AGG_RINGS ) rx <<=1 ; hw_resc->resv_rx_rings = rx; - hw_resc->resv_tx_rings = tx; + hw_resc->resv_tx_rings = tx + mpc_tx; } hw_resc->resv_irqs =le16_to_cpu (resp->alloc_msix); hw_resc->resv_hw_ring_grps = rx;
@@ -8006,7 +8020,7 @@ static int bnxt_cp_rings_in_use(struct bnxt *bp)return bnxt_nq_rings_in_use (bp); cp = bp->tx_nr_rings + bp->rx_nr_rings; -return cp; +return cp +bnxt_mpc_cp_rings_in_use (bp); }static int bnxt_get_func_stat_ctxs (struct bnxt *bp)
@@ -8064,7 +8078,7 @@ static void bnxt_get_total_resources(struct bnxt *bp, struct bnxt_hw_rings *hwr) hwr->cp_p5 =0 ;if (bp->flags &BNXT_FLAG_CHIP_P5_PLUS ) hwr->cp_p5 =bnxt_cp_rings_in_use (bp); - hwr->tx = bp->tx_nr_rings; + hwr->tx =bnxt_total_tx_rings (bp); hwr->rx = bp->rx_nr_rings; hwr->grp = hwr->rx; hwr->vnic =bnxt_get_total_vnics (bp, hwr->rx);
@@ -8170,8 +8184,10 @@ static int __bnxt_reserve_rings(struct bnxt *bp) hwr.rx = bp->rx_nr_rings;if (bp->flags &BNXT_FLAG_SHARED_RINGS ) sh = true; -if (bp->flags &BNXT_FLAG_CHIP_P5_PLUS ) - hwr.cp_p5 = hwr.rx + hwr.tx; +if (bp->flags &BNXT_FLAG_CHIP_P5_PLUS ) { + hwr.cp_p5 = hwr.rx + hwr.tx +bnxt_mpc_cp_rings_in_use (bp); + hwr.tx +=bnxt_mpc_tx_rings_in_use (bp); + } hwr.vnic =bnxt_get_total_vnics (bp, hwr.rx);
@@ -8208,6 +8224,9 @@ static int __bnxt_reserve_rings(struct bnxt *bp)if (bnxt_ulp_registered (edev) && hwr.stat >bnxt_get_ulp_stat_ctxs (bp)) hwr.stat -=bnxt_get_ulp_stat_ctxs (bp); hwr.cp =min_t (int, hwr.cp, hwr.stat); + hwr.tx -=bnxt_mpc_tx_rings_in_use (bp); +if (hwr.tx <0 ) +return -ENOMEM ; rc =bnxt_trim_rings (bp, &rx_rings, &hwr.tx, hwr.cp, sh);if (bp->flags &BNXT_FLAG_AGG_RINGS ) hwr.rx = rx_rings <<1 ;
@@ -11538,14 +11557,15 @@ unsigned int bnxt_get_max_func_cp_rings(struct bnxt *bp)return bp->hw_resc.max_cp_rings; } -static unsigned int bnxt_get_max_func_cp_rings_for_en (struct bnxt *bp) +static unsigned int bnxt_get_max_func_cp_rings_for_en (struct bnxt *bp, +unsigned int mpc_cp) {unsigned int cp = bp->hw_resc.max_cp_rings;if (!(bp->flags &BNXT_FLAG_CHIP_P5_PLUS )) cp -=bnxt_get_ulp_msix_num (bp); -return cp; +return mpc_cp >= cp ?0 : cp - mpc_cp; }static unsigned int bnxt_get_max_func_irqs (struct bnxt *bp)
@@ -11567,7 +11587,7 @@ unsigned int bnxt_get_avail_cp_rings_for_en(struct bnxt *bp) {unsigned int cp; - cp =bnxt_get_max_func_cp_rings_for_en (bp); + cp =bnxt_get_max_func_cp_rings_for_en (bp,bnxt_mpc_cp_rings_in_use (bp));if (bp->flags &BNXT_FLAG_CHIP_P5_PLUS )return cp - bp->rx_nr_rings - bp->tx_nr_rings;else
@@ -14907,11 +14927,11 @@ static void bnxt_sp_task(struct work_struct *work) }static void _bnxt_get_max_rings (struct bnxt *bp,int *max_rx,int *max_tx, -int *max_cp); +int *max_cp,int mpc_tx,int mpc_cp);/* Under netdev instance lock */ int bnxt_check_rings (struct bnxt *bp,int tx,int rx,bool sh,int tcs, -int tx_xdp) +int tx_xdp,int mpc_tx,int mpc_cp) {int max_rx, max_tx, max_cp, tx_sets =1 , tx_cp;struct bnxt_hw_rings hwr = {0 };
@@ -14921,7 +14941,7 @@ int bnxt_check_rings(struct bnxt *bp, int tx, int rx, bool sh, int tcs,if (tcs) tx_sets = tcs; -_bnxt_get_max_rings (bp, &max_rx, &max_tx, &max_cp); +_bnxt_get_max_rings (bp, &max_rx, &max_tx, &max_cp, mpc_tx, mpc_cp);if (max_rx < rx_rings)return -ENOMEM ;
@@ -14947,8 +14967,10 @@ int bnxt_check_rings(struct bnxt *bp, int tx, int rx, bool sh, int tcs, hwr.grp = rx; hwr.rss_ctx =bnxt_get_total_rss_ctxs (bp, &hwr); } -if (bp->flags &BNXT_FLAG_CHIP_P5_PLUS ) - hwr.cp_p5 = hwr.tx + rx; +if (bp->flags &BNXT_FLAG_CHIP_P5_PLUS ) { + hwr.cp_p5 = hwr.tx + rx + mpc_cp; + hwr.tx += mpc_tx; + } rc =bnxt_hwrm_check_rings (bp, &hwr);if (!rc &&pci_msix_can_alloc_dyn (bp->pdev)) {if (!bnxt_ulp_registered (bp->edev[BNXT_AUXDEV_RDMA ])) {
@@ -15669,7 +15691,9 @@ int bnxt_setup_mq_tc(struct net_device *dev, u8 tc) sh = true; rc =bnxt_check_rings (bp, bp->tx_nr_rings_per_tc, bp->rx_nr_rings, - sh, tc, bp->tx_nr_rings_xdp); + sh, tc, bp->tx_nr_rings_xdp, +bnxt_mpc_tx_rings_in_use (bp), +bnxt_mpc_cp_rings_in_use (bp));if (rc)return rc;
@@ -16644,14 +16668,14 @@ static int bnxt_get_max_irq(struct pci_dev *pdev) }static void _bnxt_get_max_rings (struct bnxt *bp,int *max_rx,int *max_tx, -int *max_cp) +int *max_cp,int mpc_tx,int mpc_cp) {struct bnxt_hw_resc *hw_resc = &bp->hw_resc;int max_ring_grps =0 , max_irq; - *max_tx = hw_resc->max_tx_rings; + *max_tx =max (0 , (int )hw_resc->max_tx_rings - mpc_tx); *max_rx = hw_resc->max_rx_rings; - *max_cp =bnxt_get_max_func_cp_rings_for_en (bp); + *max_cp =bnxt_get_max_func_cp_rings_for_en (bp, mpc_cp); max_irq = min_t(int, bnxt_get_max_func_irqs(bp) - bnxt_get_ulp_msix_num_in_use(bp), hw_resc->max_stat_ctxs -
@@ -16683,7 +16707,8 @@ int bnxt_get_max_rings(struct bnxt *bp, int *max_rx, int *max_tx, bool shared) {int rx, tx, cp; -_bnxt_get_max_rings (bp, &rx, &tx, &cp); +_bnxt_get_max_rings (bp, &rx, &tx, &cp,bnxt_mpc_tx_rings_in_use (bp), +bnxt_mpc_cp_rings_in_use (bp)); *max_rx = rx; *max_tx = tx;if (!rx || !tx || !cp)
drivers/net/ethernet/broadcom/bnxt/bnxt.h +2 -1
@@ -2993,6 +2993,7 @@ int bnxt_hwrm_vnic_cfg(struct bnxt *bp, struct bnxt_vnic_info *vnic);int bnxt_hwrm_vnic_alloc (struct bnxt *bp,struct bnxt_vnic_info *vnic,unsigned int start_rx_ring_idx,unsigned int nr_rings); +int bnxt_total_tx_rings (struct bnxt *bp);int __bnxt_hwrm_get_tx_rings (struct bnxt *bp,u16 fid,int *tx_rings);int bnxt_nq_rings_in_use (struct bnxt *bp);int bnxt_hwrm_set_coal (struct bnxt *);
@@ -3041,7 +3042,7 @@ int bnxt_dbg_hwrm_rd_reg(struct bnxt *bp, u32 reg_off, u16 num_words,void bnxt_fw_exception (struct bnxt *bp);void bnxt_fw_reset (struct bnxt *bp);int bnxt_check_rings (struct bnxt *bp,int tx,int rx,bool sh,int tcs, -int tx_xdp); +int tx_xdp,int mpc_tx,int mpc_cp);int bnxt_fw_init_one (struct bnxt *bp);bool bnxt_hwrm_reset_permitted (struct bnxt *bp);void bnxt_set_cp_rings (struct bnxt *bp,bool sh);
drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c +4 -1
@@ -37,6 +37,7 @@#include "bnxt_nvm_defs.h" /* NVRAM content constant and structure defs */ #include "bnxt_fw_hdr.h" /* Firmware hdr constant and structure defs */ #include "bnxt_coredump.h" +#include "bnxt_mpc.h" #define BNXT_NVM_ERR_MSG (dev, extack, msg) \ do { \
@@ -959,6 +960,7 @@ static int bnxt_set_channels(struct net_device *dev,struct bnxt *bp =netdev_priv (dev);int req_tx_rings, req_rx_rings, tcs;u32 new_tbl_size =0 , old_tbl_size; +int mpc_per_type =0 , mpc_cp =0 ;bool sh = false;int tx_xdp =0 ;int rc =0 ;
@@ -993,7 +995,8 @@ static int bnxt_set_channels(struct net_device *dev, tx_xdp = req_rx_rings; } - rc =bnxt_check_rings (bp, req_tx_rings, req_rx_rings, sh, tcs, tx_xdp); + rc =bnxt_check_rings (bp, req_tx_rings, req_rx_rings, sh, tcs, tx_xdp, + mpc_per_type *BNXT_MPC_TYPE_MAX , mpc_cp);if (rc) {netdev_warn (dev,"Unable to allocate the requested rings\n" );return rc;
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +21 -0
@@ -24,3 +24,24 @@ void bnxt_free_mpc_info(struct bnxt *bp)kfree (bp->mpc_info); bp->mpc_info =NULL ; } + +int bnxt_mpc_tx_rings_in_use (struct bnxt *bp) +{ +struct bnxt_mpc_info *mpc = bp->mpc_info; +int i, mpc_tx =0 ; + +if (!mpc) +return 0 ; +for (i =0 ; i <BNXT_MPC_TYPE_MAX ; i++) + mpc_tx += mpc->mpc_ring_count[i]; +return mpc_tx; +} + +int bnxt_mpc_cp_rings_in_use (struct bnxt *bp) +{ +struct bnxt_mpc_info *mpc = bp->mpc_info; + +if (!mpc) +return 0 ; +return mpc->mpc_cp_rings; +}
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h +12 -0
@@ -35,6 +35,8 @@ struct bnxt_mpc_info {#ifdef CONFIG_BNXT_TLS void bnxt_alloc_mpc_info (struct bnxt *bp,u8 mpc_chnls_cap);void bnxt_free_mpc_info (struct bnxt *bp); +int bnxt_mpc_tx_rings_in_use (struct bnxt *bp); +int bnxt_mpc_cp_rings_in_use (struct bnxt *bp);#else static inline void bnxt_alloc_mpc_info (struct bnxt *bp,u8 mpc_chnls_cap) {
@@ -43,5 +45,15 @@ static inline void bnxt_alloc_mpc_info(struct bnxt *bp, u8 mpc_chnls_cap)static inline void bnxt_free_mpc_info (struct bnxt *bp) { } + +static inline int bnxt_mpc_tx_rings_in_use (struct bnxt *bp) +{ +return 0 ; +} + +static inline int bnxt_mpc_cp_rings_in_use (struct bnxt *bp) +{ +return 0 ; +}#endif /* CONFIG_BNXT_TLS */ #endif /* BNXT_MPC_H */
drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c +3 -3
@@ -640,7 +640,7 @@ static int bnxt_hwrm_func_vf_resc_cfg(struct bnxt *bp, int num_vfs, bool reset) vf_rx_rings = hw_resc->max_rx_rings - bp->rx_nr_rings *2 ; else vf_rx_rings = hw_resc->max_rx_rings - bp->rx_nr_rings; - vf_tx_rings = hw_resc->max_tx_rings - bp->tx_nr_rings; + vf_tx_rings = hw_resc->max_tx_rings -bnxt_total_tx_rings (bp); vf_vnics = hw_resc->max_vnics - bp->nr_vnics; vf_rss = hw_resc->max_rsscos_ctxs - bp->rsscos_nr_ctxs;
@@ -903,8 +903,8 @@ static int bnxt_sriov_enable(struct bnxt *bp, int *num_vfs) avail_cp < min_rx_rings) rx_ok =0 ; -if (hw_resc->max_tx_rings - bp->tx_nr_rings >= min_tx_rings && - avail_cp >= min_tx_rings) +if (hw_resc->max_tx_rings -bnxt_total_tx_rings (bp) >= + min_tx_rings && avail_cp >= min_tx_rings) tx_ok =1 ; if (hw_resc->max_rsscos_ctxs - bp->rsscos_nr_ctxs >=
drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c +3 -1
@@ -19,6 +19,7 @@#include <net/page_pool/helpers.h> #include <linux/bnxt/hsi.h> #include "bnxt.h" +#include "bnxt_mpc.h" #include "bnxt_xdp.h" DEFINE_STATIC_KEY_FALSE (bnxt_xdp_locking_key);
@@ -410,7 +411,8 @@ static int bnxt_xdp_set(struct bnxt *bp, struct bpf_prog *prog)if (!tc) tc =1 ; rc =bnxt_check_rings (bp, bp->tx_nr_rings_per_tc, bp->rx_nr_rings, - true, tc, tx_xdp); + true, tc, tx_xdp,bnxt_mpc_tx_rings_in_use (bp), +bnxt_mpc_cp_rings_in_use (bp));if (rc) {netdev_warn (dev,"Unable to reserve enough TX rings to support XDP.\n" );return rc;
Signature · 1 line
2.51.0
Michael Chan
maintainerPATCH v6 3/15 bnxt_en: Set default MPC ring count
In reply to Michael ChanIf the firmware supports MPC channels and CONFIG_BNXT_TLS is set, set the default number of MPC channels. These MPC rings will share MSIX with the TX rings. The number of MPC channels for each type must not exceed the ethtool TX channel count. When changing ethtool channels, bnxt_calc_dflt_mpc_rings() will determine the default MPC rings for the newly requested channel count and we call bnxt_check_rings() to check before we commit. bnxt_set_dflt_mpc_rings() will set the committed MPC rings for the new committed channel count.
We also add bnxt_trim_mpc_rings() to make final adjustments in case the number of reserved TX channels is less than expected.
---
v5:
https://lore.kernel.org/netdev/[email protected]/
Improve the ethtool -L MPC calculation to use the actual MPC rings during ring check.
Add a comment in the code to explain the bnxt_trim_mpc_ring() call.
v3:
https://lore.kernel.org/netdev/[email protected]/
Use proper int type for min_t().
v2:
https://lore.kernel.org/netdev/[email protected]/
--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 11 +++ .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 8 ++- drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c | 72 +++++++++++++++++++ drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h | 27 +++++++ 4 files changed, 117 insertions(+), 1 deletion(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +11 -0
@@ -13219,6 +13219,7 @@ static int __bnxt_open_nic(struct bnxt *bp, bool irq_re_init, bool link_re_init)return rc;bnxt_adj_tx_rings (bp); +bnxt_trim_mpc_rings (bp); rc =bnxt_alloc_mem (bp, irq_re_init);if (rc) {netdev_err (bp->dev,"bnxt_alloc_mem err: %x\n" , rc);
@@ -16771,6 +16772,7 @@ static void bnxt_trim_dflt_sh_rings(struct bnxt *bp) bp->rx_nr_rings = bp->cp_nr_rings; bp->tx_nr_rings_per_tc = bp->cp_nr_rings; bp->tx_nr_rings =bnxt_tx_nr_rings (bp); +bnxt_trim_mpc_rings (bp); }static void bnxt_adj_dflt_rings (struct bnxt *bp,bool sh)
@@ -16822,6 +16824,8 @@ static int bnxt_set_dflt_rings(struct bnxt *bp, bool sh)bnxt_set_dflt_ulp_stat_ctxs (bp); } +bnxt_set_dflt_mpc_rings (bp); + rc =__bnxt_reserve_rings (bp);if (rc && rc != -ENODEV )netdev_warn (bp->dev,"Unable to reserve tx rings\n" );
@@ -16836,6 +16840,11 @@ static int bnxt_set_dflt_rings(struct bnxt *bp, bool sh)if (rc && rc != -ENODEV )netdev_warn (bp->dev,"2nd rings reservation failed.\n" );bnxt_adj_tx_rings (bp); +/* Re-cap MPC ring counts against the reduced TX rings. +* Any extra reserved MPC rings will be reconciled on +* the next open. +*/ +bnxt_trim_mpc_rings (bp); }if (BNXT_CHIP_TYPE_NITRO_A0 (bp)) { bp->rx_nr_rings++;
@@ -16870,6 +16879,7 @@ static int bnxt_init_dflt_ring_mode(struct bnxt *bp) goto init_dflt_ring_err;bnxt_adj_tx_rings (bp); +bnxt_trim_mpc_rings (bp);bnxt_set_dflt_rfs (bp);
@@ -17213,6 +17223,7 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)* limited MSIX, so we re-initialize the TX rings per TC. */ bp->tx_nr_rings_per_tc = bp->tx_nr_rings; +bnxt_trim_mpc_rings (bp);if (BNXT_PF (bp)) {if (!bnxt_pf_wq) {
drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c +7 -1
@@ -960,7 +960,7 @@ static int bnxt_set_channels(struct net_device *dev,struct bnxt *bp =netdev_priv (dev);int req_tx_rings, req_rx_rings, tcs;u32 new_tbl_size =0 , old_tbl_size; -int mpc_per_type =0 , mpc_cp =0 ; +int mpc_per_type, mpc_cp;bool sh = false;int tx_xdp =0 ;int rc =0 ;
@@ -995,6 +995,10 @@ static int bnxt_set_channels(struct net_device *dev, tx_xdp = req_rx_rings; } +bnxt_calc_dflt_mpc_rings (bp, + req_tx_rings * (tcs >1 ? tcs :1 ) + tx_xdp, + req_tx_rings, req_rx_rings, &mpc_per_type, + &mpc_cp); rc =bnxt_check_rings (bp, req_tx_rings, req_rx_rings, sh, tcs, tx_xdp, mpc_per_type *BNXT_MPC_TYPE_MAX , mpc_cp);if (rc) {
@@ -1054,6 +1058,8 @@ static int bnxt_set_channels(struct net_device *dev,bnxt_set_cp_rings (bp, sh); +bnxt_set_dflt_mpc_rings (bp); +/* After changing number of rx channels, update NTUPLE feature. */ netdev_update_features (dev);if (netif_running (dev)) {
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +72 -0
81 lines not on this page · raw message
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h +27 -0
@@ -17,6 +17,11 @@ enum bnxt_mpc_type {#define BNXT_MAX_MPC 8 +#define BNXT_MIN_MPC_TCE 1 +#define BNXT_MIN_MPC_RCE 1 +#define BNXT_DFLT_MPC_TCE BNXT_MAX_MPC +#define BNXT_DFLT_MPC_RCE BNXT_MAX_MPC +struct bnxt_mpc_info {u8 mpc_chnls_cap;u8 mpc_cp_rings;
@@ -37,6 +42,11 @@ void bnxt_alloc_mpc_info(struct bnxt *bp, u8 mpc_chnls_cap);void bnxt_free_mpc_info (struct bnxt *bp);int bnxt_mpc_tx_rings_in_use (struct bnxt *bp);int bnxt_mpc_cp_rings_in_use (struct bnxt *bp); +void bnxt_trim_mpc_rings (struct bnxt *bp); +void bnxt_calc_dflt_mpc_rings (struct bnxt *bp,int tx_nr_rings, +int tx_nr_rings_per_tc,int rx_nr_rings, +int *mpc_per_type,int *mpc_cp); +void bnxt_set_dflt_mpc_rings (struct bnxt *bp);#else static inline void bnxt_alloc_mpc_info (struct bnxt *bp,u8 mpc_chnls_cap) {
@@ -55,5 +65,22 @@ static inline int bnxt_mpc_cp_rings_in_use(struct bnxt *bp) {return 0 ; } + +static inline void bnxt_trim_mpc_rings (struct bnxt *bp) +{ +} + +static inline void bnxt_calc_dflt_mpc_rings (struct bnxt *bp,int tx_nr_rings, +int tx_nr_rings_per_tc, +int rx_nr_rings,int *mpc_per_type, +int *mpc_cp) +{ + *mpc_per_type =0 ; + *mpc_cp =0 ; +} + +static inline void bnxt_set_dflt_mpc_rings (struct bnxt *bp) +{ +}#endif /* CONFIG_BNXT_TLS */ #endif /* BNXT_MPC_H */
Signature · 1 line
2.51.0
Michael Chan
maintainerPATCH v6 4/15 bnxt_en: Rename xdp_tx_lock to tx_lock
In reply to Michael Chanxdp_tx_lock in struct bnxt_tx_ring_info is used to serialize XDP_REDIRECT on the same TX ring. MPCs will also need this lock for a similar purpose to serialize sending multiple messages on the same MPC, so rename it to tx_lock.
--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 2 +- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 4 ++-- drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c | 4 ++-- 3 files changed, 5 insertions(+), 5 deletions(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +1 -1
@@ -4154,7 +4154,7 @@ static int bnxt_alloc_tx_rings(struct bnxt *bp) } qidx = bp->tc_to_qidx[j]; ring->queue_id = bp->q_info[qidx].queue_id; -spin_lock_init (&txr->xdp_tx_lock); +spin_lock_init (&txr->tx_lock);if (i < bp->tx_nr_rings_xdp)continue ;if (BNXT_RING_TO_TC_OFF (bp, i) == (bp->tx_nr_rings_per_tc -1 ))
drivers/net/ethernet/broadcom/bnxt/bnxt.h +2 -2
@@ -1014,8 +1014,8 @@ struct bnxt_tx_ring_info {u32 dev_state;struct bnxt_ring_struct tx_ring_struct; -/* Synchronize simultaneous xdp_xmit on same ring */ -spinlock_t xdp_tx_lock; +/* Synchronize simultaneous xdp_xmit on same ring or for MPC ring */ +spinlock_t tx_lock; };#define BNXT_LEGACY_COAL_CMPL_PARAMS \
drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c +2 -2
@@ -352,7 +352,7 @@ int bnxt_xdp_xmit(struct net_device *dev, int num_frames,return -EINVAL ;if (static_branch_unlikely (&bnxt_xdp_locking_key)) -spin_lock (&txr->xdp_tx_lock); +spin_lock (&txr->tx_lock);for (i =0 ; i < num_frames; i++) {struct xdp_frame *xdp = frames[i];
@@ -377,7 +377,7 @@ int bnxt_xdp_xmit(struct net_device *dev, int num_frames, }if (static_branch_unlikely (&bnxt_xdp_locking_key)) -spin_unlock (&txr->xdp_tx_lock); +spin_unlock (&txr->tx_lock);return nxmit; }
Signature · 1 line
2.51.0
Michael Chan
maintainerPATCH v6 5/15 bnxt_en: Allocate and free MPC software structures
In reply to Michael ChanEach MPC consists of a special TX ring and a completion ring. Use existing structs bnxt_tx_ring_info and bnxt_cp_ring_info as control structures. The 2 MPC channels to TCE and RCE that share the MSIX will use a shared completion ring.
---
v5:
https://lore.kernel.org/netdev/[email protected]/
Check for P5_PLUS before allocating mpc_info to more clearly document that MPC rings are only supported on P5_PLUS chips.
--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 35 ++- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 12 +- drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c | 208 ++++++++++++++++++ drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h | 46 ++++ 4 files changed, 294 insertions(+), 7 deletions(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +29 -6
@@ -3685,7 +3685,7 @@ static size_t __bnxt_copy_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem,return total_len; } -static void bnxt_free_ring (struct bnxt *bp,struct bnxt_ring_mem_info *rmem) +void bnxt_free_ring (struct bnxt *bp,struct bnxt_ring_mem_info *rmem) {struct pci_dev *pdev = bp->pdev;int i;
@@ -3718,7 +3718,7 @@ static void bnxt_free_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem) } } -static int bnxt_alloc_ring (struct bnxt *bp,struct bnxt_ring_mem_info *rmem) +int bnxt_alloc_ring (struct bnxt *bp,struct bnxt_ring_mem_info *rmem) {struct pci_dev *pdev = bp->pdev;u64 valid_bit =0 ;
@@ -4325,6 +4325,8 @@ static int bnxt_alloc_cp_rings(struct bnxt *bp) (!sh && i >= bp->rx_nr_rings)) { cp_count += tcs; tx =1 ; +if (bnxt_napi_has_mpc (bp, i)) + cp_count++; } cpr->cp_ring_arr =kzalloc_objs (*cpr, cp_count);
@@ -4346,6 +4348,11 @@ static int bnxt_alloc_cp_rings(struct bnxt *bp) }else {int n, tc = k - rx; +/* MPC rings are at the highest k indices */ +if (tc >= tcs) { +bnxt_set_mpc_cp_ring (bp, i, cpr2); +continue ; + } n =BNXT_TC_TO_RING_BASE (bp, tc) + j; bp->tx_ring[n].tx_cpr = cpr2; cpr2->cp_ring_type =BNXT_NQ_HDL_TYPE_TX ;
@@ -4478,6 +4485,7 @@ static void bnxt_init_ring_struct(struct bnxt *bp) rmem->vmem = (void **)&txr->tx_buf_ring; } } +bnxt_init_mpc_ring_struct (bp); }static void bnxt_init_rxbd_pages (struct bnxt_ring_struct *ring,u32 type)
@@ -5557,6 +5565,7 @@ static void bnxt_init_l2_fltr_tbl(struct bnxt *bp)static void bnxt_free_mem (struct bnxt *bp,bool irq_re_init) {bnxt_free_vnic_attributes (bp); +bnxt_free_mpc_rings (bp);bnxt_free_tx_rings (bp);bnxt_free_rx_rings (bp);bnxt_free_cp_rings (bp);
@@ -5570,6 +5579,7 @@ static void bnxt_free_mem(struct bnxt *bp, bool irq_re_init)bnxt_free_port_stats (bp);bnxt_free_ring_grps (bp);bnxt_free_vnics (bp); +bnxt_free_mpcs (bp);kfree (bp->tx_ring_map); bp->tx_ring_map =NULL ;kfree (bp->tx_ring);
@@ -5679,6 +5689,10 @@ static int bnxt_alloc_mem(struct bnxt *bp, bool irq_re_init) txr->tx_cpr = &bnapi2->cp_ring; } + rc =bnxt_alloc_mpcs (bp); +if (rc) + goto alloc_mem_err; + rc =bnxt_alloc_stats (bp);if (rc) goto alloc_mem_err;
@@ -5707,6 +5721,10 @@ static int bnxt_alloc_mem(struct bnxt *bp, bool irq_re_init)if (rc) goto alloc_mem_err; + rc =bnxt_alloc_mpc_rings (bp); +if (rc) + goto alloc_mem_err; + rc =bnxt_alloc_cp_rings (bp);if (rc) goto alloc_mem_err;
@@ -7268,10 +7286,15 @@ static int hwrm_ring_alloc_send_msg(struct bnxt *bp, req->cmpl_ring_id =cpu_to_le16 (bnxt_cp_ring_for_tx (bp, txr)); req->length =cpu_to_le32 (bp->tx_ring_mask +1 ); req->stat_ctx_id =cpu_to_le32 (grp_info->fw_stats_ctx); - req->queue_id =cpu_to_le16 (ring->queue_id); -if (bp->flags &BNXT_FLAG_TX_COAL_CMPL ) - req->cmpl_coal_cnt = -RING_ALLOC_REQ_CMPL_COAL_CNT_COAL_64 ; +if (ring->queue_id ==BNXT_MPC_QUEUE_ID ) { + req->mpc_chnls_type = ring->mpc_chnl_type; + req->enables |=cpu_to_le32 (RING_ALLOC_REQ_ENABLES_MPC_CHNLS_TYPE ); + }else { + req->queue_id =cpu_to_le16 (ring->queue_id); +if (bp->flags &BNXT_FLAG_TX_COAL_CMPL ) + req->cmpl_coal_cnt = +RING_ALLOC_REQ_CMPL_COAL_CNT_COAL_64 ; + }if ((bp->fw_cap &BNXT_FW_CAP_TX_TS_CMP ) && bp->ptp_cfg) flags |=RING_ALLOC_REQ_FLAGS_TX_PKT_TS_CMPL_ENABLE ; req->flags =cpu_to_le16 (flags);
drivers/net/ethernet/broadcom/bnxt/bnxt.h +11 -1
@@ -686,6 +686,7 @@ struct nqe_cn {#define BNXT_NQ_HDL_TYPE_SHIFT 24#define BNXT_NQ_HDL_TYPE_RX 0x00#define BNXT_NQ_HDL_TYPE_TX 0x01 +#define BNXT_NQ_HDL_TYPE_MP 0x02#define BNXT_NQ_HDL_IDX (hdl) ((hdl) & BNXT_NQ_HDL_IDX_MASK)#define BNXT_NQ_HDL_TYPE (hdl) (((hdl) & BNXT_NQ_HDL_TYPE_MASK) >> \
@@ -951,6 +952,8 @@ struct bnxt_ring_struct { };u32 handle;u8 queue_id; +#define BNXT_MPC_QUEUE_ID 0xff +u8 mpc_chnl_type; };struct tx_push_bd {
@@ -991,12 +994,16 @@ struct bnxt_tx_ring_info {u16 tx_cons;u16 tx_hw_cons;u16 txq_index; +/* index for tx_ring[] or tx_mpc_ring[] in struct bnxt_napi */ u8 tx_napi_idx;u8 kick_pending;struct bnxt_db_info tx_db;struct tx_bd *tx_desc_ring[MAX_TX_PAGES ]; -struct bnxt_sw_tx_bd *tx_buf_ring; +union { +struct bnxt_sw_tx_bd *tx_buf_ring; +struct bnxt_sw_mpc_tx_bd *tx_mpc_buf_ring; + };dma_addr_t tx_desc_mapping[MAX_TX_PAGES ];
@@ -1242,6 +1249,7 @@ struct bnxt_napi {struct bnxt_cp_ring_info cp_ring;struct bnxt_rx_ring_info *rx_ring;struct bnxt_tx_ring_info *tx_ring[BNXT_MAX_TXR_PER_NAPI ]; +struct bnxt_tx_ring_info **tx_mpc_ring;void (*tx_int)(struct bnxt *,struct bnxt_napi *,int budget);
@@ -2965,6 +2973,8 @@ int bnxt_alloc_rx_data(struct bnxt *bp, struct bnxt_rx_ring_info *rxr,void bnxt_reuse_rx_data (struct bnxt_rx_ring_info *rxr,u16 cons,void *data);u32 bnxt_fw_health_readl (struct bnxt *bp,int reg_idx);bool bnxt_bs_trace_avail (struct bnxt *bp,u16 type); +void bnxt_free_ring (struct bnxt *bp,struct bnxt_ring_mem_info *rmem); +int bnxt_alloc_ring (struct bnxt *bp,struct bnxt_ring_mem_info *rmem);void bnxt_set_tpa_flags (struct bnxt *bp);void bnxt_set_ring_params (struct bnxt *);void bnxt_set_rx_skb_mode (struct bnxt *bp,bool page_mode);
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +208 -0
229 lines not on this page · raw message
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h +46 -0
74 lines not on this page · raw message
Signature · 1 line
2.51.0
Michael Chan
maintainerPATCH v6 6/15 bnxt_en: Allocate and free MPC channels from firmware
In reply to Michael ChanAllocate and free the reserved MPC TX rings and completion rings from firmware. MPC backing store memory also needs to be configured in order to successfully allocate the MPC TX rings.
--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 30 ++++-- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 6 ++ drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c | 93 +++++++++++++++++++ drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h | 16 ++++ 4 files changed, 136 insertions(+), 9 deletions(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +21 -9
@@ -7489,8 +7489,7 @@ static int bnxt_hwrm_rx_agg_ring_alloc(struct bnxt *bp,return 0 ; } -static int bnxt_hwrm_cp_ring_alloc_p5 (struct bnxt *bp, -struct bnxt_cp_ring_info *cpr) +int bnxt_hwrm_cp_ring_alloc_p5 (struct bnxt *bp,struct bnxt_cp_ring_info *cpr) {const u32 type =HWRM_RING_ALLOC_CMPL ;struct bnxt_napi *bnapi = cpr->bnapi;
@@ -7508,8 +7507,8 @@ static int bnxt_hwrm_cp_ring_alloc_p5(struct bnxt *bp,return 0 ; } -static int bnxt_hwrm_tx_ring_alloc (struct bnxt *bp, -struct bnxt_tx_ring_info *txr,u32 tx_idx) +int bnxt_hwrm_tx_ring_alloc (struct bnxt *bp,struct bnxt_tx_ring_info *txr, +u32 tx_idx) {struct bnxt_ring_struct *ring = &txr->tx_ring_struct;const u32 type =HWRM_RING_ALLOC_TX ;
@@ -7594,6 +7593,9 @@ static int bnxt_hwrm_ring_alloc(struct bnxt *bp) goto err_out; } } + + rc =bnxt_hwrm_mpc_ring_alloc (bp); + err_out:return rc; }
@@ -7651,9 +7653,8 @@ static int hwrm_ring_free_send_msg(struct bnxt *bp,return 0 ; } -static void bnxt_hwrm_tx_ring_free (struct bnxt *bp, -struct bnxt_tx_ring_info *txr, -bool close_path) +void bnxt_hwrm_tx_ring_free (struct bnxt *bp,struct bnxt_tx_ring_info *txr, +bool close_path) {struct bnxt_ring_struct *ring = &txr->tx_ring_struct;u32 cmpl_ring_id;
@@ -7712,8 +7713,7 @@ static void bnxt_hwrm_rx_agg_ring_free(struct bnxt *bp, bp->grp_info[grp_idx].agg_fw_ring_id =INVALID_HW_RING_ID ; } -static void bnxt_hwrm_cp_ring_free (struct bnxt *bp, -struct bnxt_cp_ring_info *cpr) +void bnxt_hwrm_cp_ring_free (struct bnxt *bp,struct bnxt_cp_ring_info *cpr) {struct bnxt_ring_struct *ring;
@@ -7747,6 +7747,8 @@ static void bnxt_hwrm_ring_free(struct bnxt *bp, bool close_path)if (!bp->bnapi)return ; +bnxt_hwrm_mpc_ring_free (bp, close_path); +for (i =0 ; i < bp->tx_nr_rings; i++)bnxt_hwrm_tx_ring_free (bp, &bp->tx_ring[i], close_path);
@@ -9384,12 +9386,21 @@ static int bnxt_hwrm_func_backing_store_cfg_v2(struct bnxt *bp,static int bnxt_backing_store_cfg_v2 (struct bnxt *bp) { +struct bnxt_mpc_info *mpc = bp->mpc_info;struct bnxt_ctx_mem_info *ctx = bp->ctx;struct bnxt_ctx_mem_type *ctxm;u16 last_type =BNXT_CTX_INV ;int rc =0 ;u16 type; +if (mpc && mpc->mpc_chnls_cap) { + ctxm = &ctx->ctx_arr[BNXT_CTX_MTQM ]; + rc =bnxt_setup_ctxm_pg_tbls (bp, ctxm, ctxm->max_entries,1 ); +if (rc) +return rc; + last_type =BNXT_CTX_MTQM ; + } +for (type =BNXT_CTX_SRT ; type <=BNXT_CTX_QPC ; type++) { ctxm = &ctx->ctx_arr[type];if (!bnxt_bs_trace_avail (bp, type))
@@ -11392,6 +11403,7 @@ static int bnxt_init_nic(struct bnxt *bp, bool irq_re_init)bnxt_init_cp_rings (bp);bnxt_init_rx_rings (bp);bnxt_init_tx_rings (bp); +bnxt_init_mpc_rings (bp);bnxt_init_ring_grps (bp, irq_re_init);bnxt_init_vnics (bp);
drivers/net/ethernet/broadcom/bnxt/bnxt.h +6 -0
@@ -3003,6 +3003,12 @@ int bnxt_hwrm_vnic_cfg(struct bnxt *bp, struct bnxt_vnic_info *vnic);int bnxt_hwrm_vnic_alloc (struct bnxt *bp,struct bnxt_vnic_info *vnic,unsigned int start_rx_ring_idx,unsigned int nr_rings); +int bnxt_hwrm_cp_ring_alloc_p5 (struct bnxt *bp,struct bnxt_cp_ring_info *cpr); +int bnxt_hwrm_tx_ring_alloc (struct bnxt *bp,struct bnxt_tx_ring_info *txr, +u32 tx_idx); +void bnxt_hwrm_tx_ring_free (struct bnxt *bp,struct bnxt_tx_ring_info *txr, +bool close_path); +void bnxt_hwrm_cp_ring_free (struct bnxt *bp,struct bnxt_cp_ring_info *cpr);int bnxt_total_tx_rings (struct bnxt *bp);int __bnxt_hwrm_get_tx_rings (struct bnxt *bp,u16 fid,int *tx_rings);int bnxt_nq_rings_in_use (struct bnxt *bp);
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +93 -0
96 lines not on this page · raw message
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h +16 -0
@@ -61,6 +61,9 @@ int bnxt_alloc_mpcs(struct bnxt *bp);void bnxt_free_mpcs (struct bnxt *bp);int bnxt_alloc_mpc_rings (struct bnxt *bp);void bnxt_free_mpc_rings (struct bnxt *bp); +void bnxt_init_mpc_rings (struct bnxt *bp); +int bnxt_hwrm_mpc_ring_alloc (struct bnxt *bp); +void bnxt_hwrm_mpc_ring_free (struct bnxt *bp,bool close_path);#else static inline void bnxt_alloc_mpc_info (struct bnxt *bp,u8 mpc_chnls_cap) {
@@ -128,5 +131,18 @@ static inline int bnxt_alloc_mpc_rings(struct bnxt *bp)static inline void bnxt_free_mpc_rings (struct bnxt *bp) { } + +static inline void bnxt_init_mpc_rings (struct bnxt *bp) +{ +} + +static inline int bnxt_hwrm_mpc_ring_alloc (struct bnxt *bp) +{ +return 0 ; +} + +static inline void bnxt_hwrm_mpc_ring_free (struct bnxt *bp,bool close_path) +{ +}#endif /* CONFIG_BNXT_TLS */ #endif /* BNXT_MPC_H */
Signature · 1 line
2.51.0
Michael Chan
maintainerRe: [PATCH net-next v6 06/15] bnxt_en: Allocate and free MPC channels from firmware
Responding to 2 High Sashiko comments:
https://sashiko.dev/#/patchset/20260810051358.1244418-1-michael.chan%40broadcom.com
Quoted 14 lines · Michael Chan
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +14
-0+void bnxt_init_mpc_rings (struct bnxt *bp) +{ +struct bnxt_mpc_info *mpc = bp->mpc_info; +int i, j; + +if (!mpc) +return ; + +for (i =0 ; i <BNXT_MPC_TYPE_MAX ; i++) { +int num = mpc->mpc_ring_count[i]; + +for (j =0 ; j < num; j++) { +struct bnxt_tx_ring_info *txr = &mpc->mpc_rings[i][j]; +struct bnxt_ring_struct *ring = &txr->tx_ring_struct;
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c void bnxt_free_mpc_rings(struct bnxt *bp) · line 352
Quoted 10 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +10
-0+ + txr->tx_prod =0 ; + txr->tx_cons =0 ; + txr->tx_hw_cons =0 ; +WRITE_ONCE (txr->dev_state,0 ); + ring->fw_ring_id =INVALID_HW_RING_ID ; + } + } +} +
Sashiko:
When an MPC ring processes a transmission, will the generated completion in __bnxt_poll_work() route to the wrong ring? ...
Response:
No, MPC transmissions generate new completion types (CMP_TYPE_MPC_CMP_SHORT and CMP_TYPE_MPC_CMP_LONG), not the existing TX completions.
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c void bnxt_free_mpc_rings(struct bnxt *bp) · line 360
Quoted 8 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +8
-0+static int bnxt_hwrm_one_mpc_ring_alloc (struct bnxt *bp, +struct bnxt_tx_ring_info *txr) +{ +struct bnxt_cp_ring_info *cpr = txr->tx_cpr; +struct bnxt_ring_struct *ring; +int rc; + + ring = &cpr->cp_ring_struct;
Sashiko:
Will txr->tx_cpr be NULL here during driver initialization? During driver load, bnxt_alloc_mem() allocates MPC rings via bnxt_alloc_mpcs(), which leaves txr->bnapi as NULL.
Immediately after, bnxt_alloc_all_cp_arrays() calls bnxt_set_mpc_cp_ring(), which expects txr->bnapi to be initialized to match and set txr->tx_cpr. Because txr->bnapi is still NULL, the match fails and txr->tx_cpr remains NULL. ...
Response:
No, bnxt_alloc_all_cp_arrays() does not call bnxt_set_mpc_cp_ring(). It is called by bnxt_alloc_cp_rings() much later. By then txr->napi is already initialized and txr->tx_cpr will be set correctly. So txr->tx_cpr will not be NULL when we get here.
Jakub Kicinski
maintainerRe: [PATCH net-next v6 06/15] bnxt_en: Allocate and free MPC channels from firmware
Quoted 2 lines · On Tue, 11 Aug 2026 23:36:38 -0700 Michael Chan wrote:
Responding to 2 High Sashiko comments:
https://sashiko.dev/#/patchset/20260810051358.1244418-1-michael.chan%40broadcom.com
Hi Michael, please TAL at this one, FWIW:
https://netdev-ai.bots.linux.dev/sashiko/#/patchset/[email protected]
Michael Chan
maintainerPATCH v6 7/15 bnxt_en: Allocate crypto structure and backing store
In reply to Michael ChanIf the chip and firmware support crypto (TLS) offload, allocate a bp->crypto_info software structure and backing store to support the RX and TX contexts. Each offloaded TLS connection requires a backing store context for each direction.
bp->crypto_info will stay persistent even if kTLS is no longer supported after a FW reset. This makes it easier to avoid NULL dereference of bp->crypto_info during FW reset and kTLS offload race conditions.
---
v4:
https://lore.kernel.org/netdev/[email protected]/
Clear BNXT_FW_CAP_KTLS in bnxt_free_cryto_info()
v3:
https://lore.kernel.org/netdev/[email protected]/
bp->crypto_info and bp->ktls_info now stay persistent for the driver's lifetime. max keys are refreshed after re-init.
v2:
https://lore.kernel.org/netdev/20260512212105.3488258-8-michael.chan@broadcom.
--- drivers/net/ethernet/broadcom/bnxt/Makefile | 2 +- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 21 +++++ drivers/net/ethernet/broadcom/bnxt/bnxt.h | 4 + .../net/ethernet/broadcom/bnxt/bnxt_crypto.c | 83 +++++++++++++++++++ .../net/ethernet/broadcom/bnxt/bnxt_crypto.h | 47 +++++++++++ include/linux/bnxt/hsi.h | 37 +++++++++ 6 files changed, 193 insertions(+), 1 deletion(-) create mode 100644 drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c create mode 100644 drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.h
drivers/net/ethernet/broadcom/bnxt/Makefile +1 -1
@@ -5,4 +5,4 @@ bnxt_en-y := bnxt.o bnxt_hwrm.o bnxt_sriov.o bnxt_ethtool.o bnxt_dcb.o bnxt_ulp. bnxt_en-$(CONFIG_BNXT_FLOWER_OFFLOAD) += bnxt_tc.o bnxt_en-$(CONFIG_DEBUG_FS) += bnxt_debugfs.o bnxt_en-$(CONFIG_BNXT_HWMON) += bnxt_hwmon.o -bnxt_en-$(CONFIG_BNXT_TLS) += bnxt_mpc.o +bnxt_en-$(CONFIG_BNXT_TLS) += bnxt_mpc.o bnxt_crypto.o
drivers/net/ethernet/broadcom/bnxt/bnxt.c +21 -0
@@ -77,6 +77,7 @@#include "bnxt_gso.h" #include <net/tso.h> #include "bnxt_mpc.h" +#include "bnxt_crypto.h" #define BNXT_TX_TIMEOUT (5 * HZ)#define BNXT_DEF_MSG_ENABLE (NETIF_MSG_DRV | NETIF_MSG_HW | \
@@ -9386,6 +9387,7 @@ static int bnxt_hwrm_func_backing_store_cfg_v2(struct bnxt *bp,static int bnxt_backing_store_cfg_v2 (struct bnxt *bp) { +struct bnxt_crypto_info *crypto = bp->crypto_info;struct bnxt_mpc_info *mpc = bp->mpc_info;struct bnxt_ctx_mem_info *ctx = bp->ctx;struct bnxt_ctx_mem_type *ctxm;
@@ -9393,6 +9395,19 @@ static int bnxt_backing_store_cfg_v2(struct bnxt *bp)int rc =0 ;u16 type; +if (BNXT_SUPPORTS_KTLS (bp)) { + ctxm = &ctx->ctx_arr[BNXT_CTX_TCK ]; + rc =bnxt_setup_ctxm_pg_tbls (bp, ctxm, +BNXT_TCK (crypto).max_ctx,1 ); +if (rc) +return rc; + ctxm = &ctx->ctx_arr[BNXT_CTX_RCK ]; + rc =bnxt_setup_ctxm_pg_tbls (bp, ctxm, +BNXT_RCK (crypto).max_ctx,1 ); +if (rc) +return rc; + last_type =BNXT_CTX_RCK ; + }if (mpc && mpc->mpc_chnls_cap) { ctxm = &ctx->ctx_arr[BNXT_CTX_MTQM ]; rc =bnxt_setup_ctxm_pg_tbls (bp, ctxm, ctxm->max_entries,1 );
@@ -9935,6 +9950,10 @@ static int __bnxt_hwrm_func_qcaps(struct bnxt *bp) bp->fw_cap |=BNXT_FW_CAP_BACKING_STORE_V2 ;if (flags_ext &FUNC_QCAPS_RESP_FLAGS_EXT_TX_COAL_CMPL_CAP ) bp->flags |=BNXT_FLAG_TX_COAL_CMPL ; +if (flags_ext &FUNC_QCAPS_RESP_FLAGS_EXT_KTLS_SUPPORTED ) +bnxt_alloc_crypto_info (bp, resp); +else + bp->fw_cap &= ~BNXT_FW_CAP_KTLS ; flags_ext2 =le32_to_cpu (resp->flags_ext2);if (flags_ext2 &FUNC_QCAPS_RESP_FLAGS_EXT2_RX_ALL_PKTS_TIMESTAMPS_SUPPORTED )
@@ -16639,6 +16658,7 @@ static void bnxt_remove_one(struct pci_dev *pdev) bp->ptp_cfg =NULL ;kfree (bp->fw_health); bp->fw_health =NULL ; +bnxt_free_crypto_info (bp);bnxt_free_mpc_info (bp);bnxt_cleanup_pci (bp);bnxt_free_ctx_mem (bp, true);
@@ -17322,6 +17342,7 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)bnxt_ethtool_free (bp);kfree (bp->fw_health); bp->fw_health =NULL ; +bnxt_free_crypto_info (bp);bnxt_free_mpc_info (bp);bnxt_cleanup_pci (bp);bnxt_free_ctx_mem (bp, true);
drivers/net/ethernet/broadcom/bnxt/bnxt.h +4 -0
@@ -2462,6 +2462,7 @@ struct bnxt {u8 tph_mode;struct bnxt_mpc_info *mpc_info; +struct bnxt_crypto_info *crypto_info;unsigned int current_interval;#define BNXT_TIMER_INTERVAL HZ
@@ -2549,6 +2550,7 @@ struct bnxt {#define BNXT_FW_CAP_NPAR_1_2 BIT_ULL(42)#define BNXT_FW_CAP_MIRROR_ON_ROCE BIT_ULL(43)#define BNXT_FW_CAP_PTP_PTM BIT_ULL(44) +#define BNXT_FW_CAP_KTLS BIT_ULL(45)u32 fw_dbg_cap;
@@ -2574,6 +2576,8 @@ struct bnxt { ((bp)->fw_cap & BNXT_FW_CAP_SW_MAX_RESOURCE_LIMITS)#define BNXT_MIRROR_ON_ROCE_CAP (bp) \ ((bp)->fw_cap & BNXT_FW_CAP_MIRROR_ON_ROCE) +#define BNXT_SUPPORTS_KTLS (bp) \ + ((bp)->fw_cap & BNXT_FW_CAP_KTLS)u32 hwrm_spec_code;u16 hwrm_cmd_seq;
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c new file +83 -0
83 lines not on this page · raw message
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.h new file +47 -0
47 lines not on this page · raw message
include/linux/bnxt/hsi.h +37 -0
@@ -3837,6 +3837,43 @@ struct hwrm_func_ptp_ext_qcfg_output {u8 valid; }; +/* hwrm_func_key_ctx_alloc_input (size:384b/48B) */ +struct hwrm_func_key_ctx_alloc_input { +__le16 req_type; +__le16 cmpl_ring; +__le16 seq_id; +__le16 target_id; +__le64 resp_addr; +__le16 fid; +__le16 num_key_ctxs; +__le32 dma_bufr_size_bytes; +u8 key_ctx_type; +#define FUNC_KEY_CTX_ALLOC_REQ_KEY_CTX_TYPE_TX 0x0UL +#define FUNC_KEY_CTX_ALLOC_REQ_KEY_CTX_TYPE_RX 0x1UL +#define FUNC_KEY_CTX_ALLOC_REQ_KEY_CTX_TYPE_QUIC_TX 0x2UL +#define FUNC_KEY_CTX_ALLOC_REQ_KEY_CTX_TYPE_QUIC_RX 0x3UL +#define FUNC_KEY_CTX_ALLOC_REQ_KEY_CTX_TYPE_LAST FUNC_KEY_CTX_ALLOC_REQ_KEY_CTX_TYPE_QUIC_RX +u8 unused_0[7 ]; +__le64 host_dma_addr; +__le32 partition_start_xid; +u8 unused_1[4 ]; +}; + +/* hwrm_func_key_ctx_alloc_output (size:192b/24B) */ +struct hwrm_func_key_ctx_alloc_output { +__le16 error_code; +__le16 req_type; +__le16 seq_id; +__le16 resp_len; +__le16 num_key_ctxs_allocated; +u8 flags; +#define FUNC_KEY_CTX_ALLOC_RESP_FLAGS_KEY_CTXS_CONTIGUOUS 0x1UL +u8 unused_0; +__le32 partition_start_xid; +u8 unused_1[7 ]; +u8 valid; +}; +/* hwrm_func_backing_store_cfg_v2_input (size:512b/64B) */ struct hwrm_func_backing_store_cfg_v2_input {__le16 req_type;
Signature · 1 line
2.51.0
Michael Chan
maintainerPATCH v6 8/15 bnxt_en: Reserve crypto RX and TX key contexts on a PF
In reply to Michael ChanIf kTLS crypto offload is supported, reserve RX and TX key contexts. These keys will later be allocated during run-time to support offloading TX and RX kTLS connections.
---
v3:
https://lore.kernel.org/netdev/[email protected]/
Add missing ASSETS_TEST flag when checking reserved crypto keys.
v2:
https://lore.kernel.org/netdev/[email protected]/
--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 22 +++++++++++-- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 11 +++++++ .../net/ethernet/broadcom/bnxt/bnxt_crypto.c | 32 +++++++++++++++++++ .../net/ethernet/broadcom/bnxt/bnxt_crypto.h | 7 ++++ 4 files changed, 70 insertions(+), 2 deletions(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +20 -2
@@ -7801,6 +7801,7 @@ static int bnxt_trim_rings(struct bnxt *bp, int *rx, int *tx, int max,static int bnxt_hwrm_get_rings (struct bnxt *bp) {struct bnxt_hw_resc *hw_resc = &bp->hw_resc; +struct bnxt_hw_crypto_resc *crypto_resc;struct hwrm_func_qcfg_output *resp;struct hwrm_func_qcfg_input *req;int rc;
@@ -7861,6 +7862,10 @@ static int bnxt_hwrm_get_rings(struct bnxt *bp) } hw_resc->resv_cp_rings = cp; hw_resc->resv_stat_ctxs = stats; + + crypto_resc = &hw_resc->crypto_resc; + crypto_resc->resv_tx_key_ctxs =le32_to_cpu (resp->num_ktls_tx_key_ctxs); + crypto_resc->resv_rx_key_ctxs =le32_to_cpu (resp->num_ktls_rx_key_ctxs); } get_rings_exit:hwrm_req_drop (bp, req);
@@ -7931,8 +7936,9 @@ __bnxt_hwrm_reserve_pf_rings(struct bnxt *bp, struct bnxt_hw_rings *hwr) } req->num_stat_ctxs =cpu_to_le16 (hwr->stat); req->num_vnics =cpu_to_le16 (hwr->vnic); +bnxt_hwrm_reserve_pf_key_ctxs (bp, req); } - req->enables =cpu_to_le32 (enables); + req->enables |=cpu_to_le32 (enables);return req; }
@@ -8334,7 +8340,7 @@ static int bnxt_hwrm_check_vf_rings(struct bnxt *bp, struct bnxt_hw_rings *hwr)static int bnxt_hwrm_check_pf_rings (struct bnxt *bp,struct bnxt_hw_rings *hwr) {struct hwrm_func_cfg_input *req; -u32 flags; +u32 flags, flags2 =0 ; req =__bnxt_hwrm_reserve_pf_rings (bp, hwr); flags =FUNC_CFG_REQ_FLAGS_TX_ASSETS_TEST ;
@@ -8348,9 +8354,14 @@ static int bnxt_hwrm_check_pf_rings(struct bnxt *bp, struct bnxt_hw_rings *hwr)FUNC_CFG_REQ_FLAGS_NQ_ASSETS_TEST ; else flags |= FUNC_CFG_REQ_FLAGS_RING_GRP_ASSETS_TEST; +if (req->enables & +cpu_to_le32 (FUNC_CFG_REQ_ENABLES_KTLS_TX_KEY_CTXS | +FUNC_CFG_REQ_ENABLES_KTLS_RX_KEY_CTXS )) + flags2 |=FUNC_CFG_REQ_FLAGS2_KTLS_KEY_CTX_ASSETS_TEST ; } req->flags =cpu_to_le32 (flags); + req->flags2 =cpu_to_le32 (flags2);return hwrm_req_send_silent (bp, req); }
@@ -9764,6 +9775,7 @@ int bnxt_hwrm_func_resc_qcaps(struct bnxt *bp, bool all)struct hwrm_func_resource_qcaps_output *resp;struct hwrm_func_resource_qcaps_input *req;struct bnxt_hw_resc *hw_resc = &bp->hw_resc; +struct bnxt_hw_crypto_resc *crypto_resc;int rc; rc =hwrm_req_init (bp, req,HWRM_FUNC_RESOURCE_QCAPS );
@@ -9801,6 +9813,12 @@ int bnxt_hwrm_func_resc_qcaps(struct bnxt *bp, bool all) hw_resc->max_vnics * BNXT_LARGE_RSS_TO_VNIC_RATIO) bp->rss_cap |= BNXT_RSS_CAP_LARGE_RSS_CTX; + crypto_resc = &hw_resc->crypto_resc; + crypto_resc->min_tx_key_ctxs =le32_to_cpu (resp->min_ktls_tx_key_ctxs); + crypto_resc->max_tx_key_ctxs =le32_to_cpu (resp->max_ktls_tx_key_ctxs); + crypto_resc->min_rx_key_ctxs =le32_to_cpu (resp->min_ktls_rx_key_ctxs); + crypto_resc->max_rx_key_ctxs =le32_to_cpu (resp->max_ktls_rx_key_ctxs); +if (bp->flags &BNXT_FLAG_CHIP_P5_PLUS ) {u16 max_msix =le16_to_cpu (resp->max_msix);
drivers/net/ethernet/broadcom/bnxt/bnxt.h +11 -0
@@ -1364,6 +1364,15 @@ struct bnxt_hw_rings {int rss_ctx; }; +struct bnxt_hw_crypto_resc { +u32 min_tx_key_ctxs; +u32 max_tx_key_ctxs; +u32 resv_tx_key_ctxs; +u32 min_rx_key_ctxs; +u32 max_rx_key_ctxs; +u32 resv_rx_key_ctxs; +}; +struct bnxt_hw_resc {u16 min_rsscos_ctxs;u16 max_rsscos_ctxs;
@@ -1398,6 +1407,8 @@ struct bnxt_hw_resc {u32 max_tx_wm_flows;u32 max_rx_em_flows;u32 max_rx_wm_flows; + +struct bnxt_hw_crypto_resc crypto_resc; };#define BNXT_LARGE_RSS_TO_VNIC_RATIO 7
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +32 -0
@@ -81,3 +81,35 @@ void bnxt_free_crypto_info(struct bnxt *bp) bp->crypto_info =NULL ; bp->fw_cap &= ~BNXT_FW_CAP_KTLS ; } + +/** +* bnxt_hwrm_reserve_pf_key_ctxs - Reserve key contexts with firmware +* @bp: pointer to bnxt device +* @req: pointer to HWRM function config request +* +* Populates the firmware request with key context reservation parameters +* for crypto offload based on current max settings and capabilities. +* +* Context: Process context during device configuration +*/ +void bnxt_hwrm_reserve_pf_key_ctxs (struct bnxt *bp, +struct hwrm_func_cfg_input *req) +{ +struct bnxt_crypto_info *crypto = bp->crypto_info; +struct bnxt_hw_resc *hw_resc = &bp->hw_resc; +struct bnxt_hw_crypto_resc *crypto_resc; +u32 tx, rx; + +if (!crypto || !BNXT_SUPPORTS_KTLS (bp)) +return ; + + crypto_resc = &hw_resc->crypto_resc; + tx =min (BNXT_TCK (crypto).max_ctx, crypto_resc->max_tx_key_ctxs); + rx =min (BNXT_RCK (crypto).max_ctx, crypto_resc->max_rx_key_ctxs); + req->num_ktls_tx_key_ctxs =cpu_to_le32 (tx); + req->num_ktls_rx_key_ctxs =cpu_to_le32 (rx); +if (tx) + req->enables |=cpu_to_le32 (FUNC_CFG_REQ_ENABLES_KTLS_TX_KEY_CTXS ); +if (rx) + req->enables |=cpu_to_le32 (FUNC_CFG_REQ_ENABLES_KTLS_RX_KEY_CTXS ); +}
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.h +7 -0
@@ -34,6 +34,8 @@ struct bnxt_crypto_info {void bnxt_alloc_crypto_info (struct bnxt *bp,struct hwrm_func_qcaps_output *resp);void bnxt_free_crypto_info (struct bnxt *bp); +void bnxt_hwrm_reserve_pf_key_ctxs (struct bnxt *bp, +struct hwrm_func_cfg_input *req);#else static inline void bnxt_alloc_crypto_info (struct bnxt *bp,struct hwrm_func_qcaps_output *resp)
@@ -43,5 +45,10 @@ static inline void bnxt_alloc_crypto_info(struct bnxt *bp,static inline void bnxt_free_crypto_info (struct bnxt *bp) { } + +static inline void bnxt_hwrm_reserve_pf_key_ctxs (struct bnxt *bp, +struct hwrm_func_cfg_input *req) +{ +}#endif /* CONFIG_BNXT_TLS */ #endif /* BNXT_CRYPTO_H */
Signature · 1 line
2.51.0
Michael Chan
maintainerPATCH v6 9/15 bnxt_en: Add infrastructure for crypto key context IDs
In reply to Michael ChanEach kTLS connection requires a crypto key context ID (KID). These KIDs are allocated from the firmware in batches. Add data structure to store these IDs. The bnxt_kid_info structure stores a batch of IDs and it can be linked as we allocate more batches. There is a bitmap in the structure to keep track of which ones are in use. Add APIs to allocate and free these KIDs.
Once allocated, these KIDs are not freed during run-time. They are re-used for new connections. FW reset or HWRM_FUNC_RESET will free all KIDs. Call bnxt_clear_crypto() to clear all KIDs in the driver's structures during these reset events.
---
v5:
https://lore.kernel.org/netdev/[email protected]/
More clearly document and explain that bnxt_clear_crypto() has no concurrent readers/writers.
Add comment to explain that the bnxt_free_one_ctx() caller is reponsible to check the epoch value when needed.
v4:
https://lore.kernel.org/netdev/[email protected]/
Fix kernel doc for bnxt_free_one_kctx().
Add BNXT_NEXT_EPOCH() to mask the epoch value.
v3:
https://lore.kernel.org/netdev/[email protected]/
Use a larger (12-bit) epoch value for the keys.
Improve comments, kerneldoc, and dmesg.
Make sure bnxt_clear_crypto() is called during shutdown with the BNXT_STATE_OPEN flag cleared.
v2:
https://lore.kernel.org/netdev/[email protected]/
--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 10 + .../net/ethernet/broadcom/bnxt/bnxt_crypto.c | 285 ++++++++++++++++++ .../net/ethernet/broadcom/bnxt/bnxt_crypto.h | 70 +++++ 3 files changed, 365 insertions(+)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +10 -0
@@ -12922,6 +12922,8 @@ static int bnxt_hwrm_if_change(struct bnxt *bp, bool up)bnxt_ulp_irq_stop (bp);bnxt_free_ctx_mem (bp, false);bnxt_dcb_free (bp); +if (fw_reset || caps_change) +bnxt_clear_crypto (bp); rc =bnxt_fw_init_one (bp);if (rc) {clear_bit (BNXT_STATE_FW_RESET_DET , &bp->state);
@@ -14659,6 +14661,7 @@ static void bnxt_fw_reset_close(struct bnxt *bp)bnxt_hwrm_func_drv_unrgtr (bp);if (pci_is_enabled (bp->pdev))pci_disable_device (bp->pdev); +bnxt_clear_crypto (bp);bnxt_free_ctx_mem (bp, false); }
@@ -17318,6 +17321,12 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) rc =bnxt_dl_register (bp);if (rc) goto init_err_dl; + rc =bnxt_crypto_init (bp); +if (rc) { +bnxt_free_crypto_info (bp); +netdev_warn (bp->dev,"Failed to initialize crypto offload, err = %d\n" , + rc); + }INIT_LIST_HEAD (&bp->usr_fltr_list);
@@ -17544,6 +17553,7 @@ static pci_ers_result_t bnxt_io_error_detected(struct pci_dev *pdev,if (pci_is_enabled (pdev))pci_disable_device (pdev); +bnxt_clear_crypto (bp);bnxt_free_ctx_mem (bp, false);netdev_unlock (netdev);
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +285 -0
312 lines not on this page · raw message
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.h +70 -0
104 lines not on this page · raw message
Signature · 1 line
2.51.0
Michael Chan
maintainerPATCH v6 10/15 bnxt_en: Add MPC transmit and completion functions
In reply to Michael ChanAdd transmit, ring selection, and completion functions for midpath rings. These will be used to send control data to the crypto engines.
All TCE/RCE completions are short (16-byte) completions. The long completions logic in bnxt_mpc_cmp() is added for completeness only.
---
v6:
Handle both long and short MPC completions. Leave one empty slot in the MPC ring just like TX ring. Use memory barrier when updating tx_cons for MPC ring.
v5:
https://lore.kernel.org/netdev/[email protected]/
Add description in commit log that crypto engines use short completions only.
v3:
https://lore.kernel.org/netdev/[email protected]/
Use WRITE_ONCE() for TX prod/cons.
v2:
https://lore.kernel.org/netdev/[email protected]/
Fix unused variable warnings
v1:
https://lore.kernel.org/netdev/[email protected]/
--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 4 + drivers/net/ethernet/broadcom/bnxt/bnxt.h | 3 + drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c | 158 ++++++++++++++++++ drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h | 65 +++++++ 4 files changed, 230 insertions(+)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +4 -0
@@ -3115,6 +3115,10 @@ static int __bnxt_poll_work(struct bnxt *bp, struct bnxt_cp_ring_info *cpr, rx_pkts++; else if (rc == -EBUSY) /* partial completion */ break; + }else if (cmp_type ==CMP_TYPE_MPC_CMP_SHORT || + cmp_type ==CMP_TYPE_MPC_CMP_LONG ) { +if (bnxt_mpc_cmp (bp, cpr, &raw_cons)) +break ; }else if (unlikely (cmp_type ==CMPL_BASE_TYPE_HWRM_DONE || cmp_type ==CMPL_BASE_TYPE_HWRM_FWD_REQ || cmp_type ==CMPL_BASE_TYPE_HWRM_ASYNC_EVENT )) {
drivers/net/ethernet/broadcom/bnxt/bnxt.h +3 -0
@@ -47,6 +47,7 @@ struct tx_bd {__le32 tx_bd_len_flags_type;#define TX_BD_TYPE (0x3f << 0)#define TX_BD_TYPE_SHORT_TX_BD (0x00 << 0) +#define TX_BD_TYPE_MPC_TX_BD (0x08 << 0)#define TX_BD_TYPE_LONG_TX_BD (0x10 << 0)#define TX_BD_FLAGS_PACKET_END (1 << 6)#define TX_BD_FLAGS_NO_CMPL (1 << 7)
@@ -160,6 +161,8 @@ struct tx_cmp {#define CMP_TYPE_RX_TPA_AGG_CMP 22#define CMP_TYPE_RX_L2_V3_CMP 23#define CMP_TYPE_RX_L2_TPA_START_V3_CMP 25 +#define CMP_TYPE_MPC_CMP_SHORT 30 +#define CMP_TYPE_MPC_CMP_LONG 31#define CMP_TYPE_STATUS_CMP 32#define CMP_TYPE_REMOTE_DRIVER_REQ 34#define CMP_TYPE_REMOTE_DRIVER_RESP 36
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +158 -0
161 lines not on this page · raw message
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h +65 -0
87 lines not on this page · raw message
Signature · 1 line
2.51.0
Michael Chan
maintainerRe: [PATCH net-next v6 10/15] bnxt_en: Add MPC transmit and completion functions
Responding to some Sashiko comments:
https://sashiko.dev/#/patchset/20260810051358.1244418-1-michael.chan%40broadcom.com
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c void bnxt_hwrm_mpc_ring_free(struct bnxt *bp, bool close_path) · line 454
Quoted 3 lines · Michael Chan
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +3
-0+int bnxt_start_xmit_mpc (struct bnxt *bp,struct bnxt_tx_ring_info *txr, +void *data,unsigned int len,unsigned long handle) +{
(prod);
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c void bnxt_hwrm_mpc_ring_free(struct bnxt *bp, bool close_path) · line 493
Quoted 11 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +11
-0+if (bd_space < bds) { +unsigned int len0 = bd_space *sizeof (*txbd); + +memcpy (txbd, data, len0); + prod += bd_space; + txbd = &txr->tx_desc_ring[TX_RING (bp, prod)][TX_IDX (prod)]; + bds -= bd_space; + len -= len0; + data += len0; + } +memcpy (txbd, data, len);
Sashiko:
Does this code adequately bounds-check the remaining length when it exceeds the size of a single descriptor page?
If the provided length is large enough to span more than two pages...
Response:
Yes, the largest MPC command is the ce_add_cmd and the size is about 72 bytes. It will not span more than 2 pages.
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c void bnxt_hwrm_mpc_ring_free(struct bnxt *bp, bool close_path) · line 569
Quoted 12 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +11
-0+int bnxt_mpc_cmp (struct bnxt *bp,struct bnxt_cp_ring_info *cpr,u32 *raw_cons) +{ + tx_cons = txr->tx_cons; +if (TX_OPAQUE_RING (opaque) != txr->tx_napi_idx) { +netdev_warn (bp->dev,"Wrong opaque %x, expected ring %x, cons idx %x\n" , + opaque, txr->tx_napi_idx, txr->tx_cons); + goto cmp_done; + } + mpc_buf = &txr->tx_mpc_buf_ring[RING_TX (bp, tx_cons)]; + mpc_buf->handle =0 ; + tx_cons += mpc_buf->inline_bds;
Sashiko:
The docstring for bnxt_start_xmit_mpc() states that bnxt_crypto_mpc_cmp() will be called with the handle passed back upon successful hardware completion.
However, mpc_buf->handle is cleared here,...
Response:
The changes are staged. The next patch #11 will hook this up and call bnxt_crypto_mpc_cmp() with the handle passed back.
Michael Chan
maintainerPATCH v6 11/15 bnxt_en: Add crypto MPC transmit/completion infrastructure
In reply to Michael ChanAdd infrastructure to support sending crypto commands using the midpath channels (MPCs). bnxt_xmit_crypto_cmd() is used to send a crypto command and sleep with timeout until the completion is received. If it times out, we recover by resetting the MPC. The reset is deferred to bnxt_sp_task() similar to other resets to prevent deadlock with a concurent close. The concurrent close may have acquired the netdev_lock and waiting for the MPC operation to finish.
Use refcount to handle MPC completions and possible timeout race conditions.
The next patch will use this infrastructure to offload kTLS connections.
---
v6:
Disable NAPI during MPC ring restart.
Add bnxt_free_mpc_entries() called during shutdown like other rings. Simplify bnxt_xmit_crypto_cmd() error path.
v5:
https://lore.kernel.org/netdev/[email protected]/
Defer the MPC ring reset to bnxt_sp_task() just like other resets to avoid any possibility od deadlock.
Add more comments to explain the MPC ring disable, stop, start.
v4:
https://lore.kernel.org/netdev/[email protected]/
Clear MPC entries in bnxt_mpc_ring_reset() if device is going down.
Only clear MPC handle in bnxt_mpc_cmp() if the ring is in normal state to ensure proper ref counting during reset.
v3:
https://lore.kernel.org/netdev/[email protected]/
Multiple improvements for the MPC timeout logic, including the use of refcount to terminate the timeout instead of the arbitrary 200msec poll wait, add synchronize_net().
v2:
https://lore.kernel.org/netdev/[email protected]/
--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 17 +- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 4 + .../net/ethernet/broadcom/bnxt/bnxt_crypto.c | 139 ++++++++++- .../net/ethernet/broadcom/bnxt/bnxt_crypto.h | 95 ++++++++ drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c | 219 +++++++++++++++++- drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h | 19 ++ 6 files changed, 488 insertions(+), 5 deletions(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +14 -3
@@ -394,7 +394,7 @@ static void __bnxt_queue_sp_work(struct bnxt *bp)schedule_work (&bp->sp_task); } -static void bnxt_queue_sp_work (struct bnxt *bp,unsigned int event) +void bnxt_queue_sp_work (struct bnxt *bp,unsigned int event) {set_bit (event, &bp->sp_event);__bnxt_queue_sp_work (bp);
@@ -3642,6 +3642,7 @@ static void bnxt_free_skbs(struct bnxt *bp) {bnxt_free_tx_skbs (bp);bnxt_free_rx_skbs (bp); +bnxt_free_mpc_entries (bp); }static void bnxt_init_ctx_mem (struct bnxt_ctx_mem_type *ctxm,void *p,int len)
@@ -7731,7 +7732,7 @@ void bnxt_hwrm_cp_ring_free(struct bnxt *bp, struct bnxt_cp_ring_info *cpr) ring->fw_ring_id =INVALID_HW_RING_ID ; } -static void bnxt_clear_one_cp_ring (struct bnxt *bp,struct bnxt_cp_ring_info *cpr) +void bnxt_clear_one_cp_ring (struct bnxt *bp,struct bnxt_cp_ring_info *cpr) {struct bnxt_ring_struct *ring = &cpr->cp_ring_struct;int i, size = ring->ring_mem.page_size;
@@ -14448,7 +14449,7 @@ static int bnxt_hwrm_rx_ring_reset(struct bnxt *bp, int ring_nr)return hwrm_req_send_silent (bp, req); } -static void bnxt_reset_task (struct bnxt *bp,bool silent) +void bnxt_reset_task (struct bnxt *bp,bool silent) {if (!silent)bnxt_dbg_dump_states (bp);
@@ -14897,6 +14898,13 @@ static void bnxt_ulp_restart(struct bnxt *bp)bnxt_ulp_start (bp); } +static void bnxt_mpc_ring_reset_sp (struct bnxt *bp) +{ +bnxt_rtnl_lock_sp (bp); +bnxt_mpc_ring_reset_task (bp); +bnxt_rtnl_unlock_sp (bp); +} +static void bnxt_sp_task (struct work_struct *work) { struct bnxt *bp = container_of(work, struct bnxt, sp_task);
@@ -14989,6 +14997,9 @@ static void bnxt_sp_task(struct work_struct *work)if (test_and_clear_bit (BNXT_RST_RING_SP_EVENT , &bp->sp_event))bnxt_rx_ring_reset (bp); +if (test_and_clear_bit (BNXT_MPC_RESET_SP_EVENT , &bp->sp_event)) +bnxt_mpc_ring_reset_sp (bp); +if (test_and_clear_bit (BNXT_FW_RESET_NOTIFY_SP_EVENT , &bp->sp_event)) {if (test_bit (BNXT_STATE_FW_FATAL_COND , &bp->state) ||test_bit (BNXT_STATE_FW_NON_FATAL_COND , &bp->state))
drivers/net/ethernet/broadcom/bnxt/bnxt.h +4 -0
@@ -2674,6 +2674,7 @@ struct bnxt {#define BNXT_THERMAL_THRESHOLD_SP_EVENT 22#define BNXT_FW_ECHO_REQUEST_SP_EVENT 23#define BNXT_RESTART_ULP_SP_EVENT 24 +#define BNXT_MPC_RESET_SP_EVENT 25struct delayed_work fw_reset_task;int fw_reset_state;
@@ -2986,6 +2987,7 @@ static inline enum pkt_hash_types bnxt_rss_ext_op(struct bnxt *bp,extern const u16 bnxt_bstore_to_trace[];extern const u16 bnxt_lhint_arr[]; +void bnxt_queue_sp_work (struct bnxt *bp,unsigned int event);int bnxt_alloc_rx_data (struct bnxt *bp,struct bnxt_rx_ring_info *rxr,u16 prod,gfp_t gfp);void bnxt_reuse_rx_data (struct bnxt_rx_ring_info *rxr,u16 cons,void *data);
@@ -3027,6 +3029,7 @@ int bnxt_hwrm_tx_ring_alloc(struct bnxt *bp, struct bnxt_tx_ring_info *txr,void bnxt_hwrm_tx_ring_free (struct bnxt *bp,struct bnxt_tx_ring_info *txr,bool close_path);void bnxt_hwrm_cp_ring_free (struct bnxt *bp,struct bnxt_cp_ring_info *cpr); +void bnxt_clear_one_cp_ring (struct bnxt *bp,struct bnxt_cp_ring_info *cpr);int bnxt_total_tx_rings (struct bnxt *bp);int __bnxt_hwrm_get_tx_rings (struct bnxt *bp,u16 fid,int *tx_rings);int bnxt_nq_rings_in_use (struct bnxt *bp);
@@ -3073,6 +3076,7 @@ void bnxt_sync_ring_stats(struct bnxt *bp);bool bnxt_rfs_capable (struct bnxt *bp,bool new_rss_ctx);int bnxt_dbg_hwrm_rd_reg (struct bnxt *bp,u32 reg_off,u16 num_words,u32 *reg_buf); +void bnxt_reset_task (struct bnxt *bp,bool silent);void bnxt_fw_exception (struct bnxt *bp);void bnxt_fw_reset (struct bnxt *bp); int bnxt_check_rings(struct bnxt *bp, int tx, int rx, bool sh, int tcs,
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +138 -1
183 lines not on this page · raw message
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.h +95 -0
123 lines not on this page · raw message
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +218 -1
248 lines not on this page · raw message
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h +19 -0
@@ -22,6 +22,8 @@ enum bnxt_mpc_type {#define BNXT_DFLT_MPC_TCE BNXT_MAX_MPC#define BNXT_DFLT_MPC_RCE BNXT_MAX_MPC +#define BNXT_MPC_TMO_MSECS 1000 +struct bnxt_mpc_info {u8 mpc_chnls_cap;u8 mpc_cp_rings;
@@ -109,6 +111,9 @@ void bnxt_hwrm_mpc_ring_free(struct bnxt *bp, bool close_path);struct bnxt_tx_ring_info *bnxt_select_mpc_ring (struct bnxt *bp,int ring_type);int bnxt_start_xmit_mpc (struct bnxt *bp,struct bnxt_tx_ring_info *txr,void *data,unsigned int len,unsigned long handle); +void bnxt_free_mpc_entries (struct bnxt *bp); +void bnxt_mpc_ring_reset_task (struct bnxt *bp); +int bnxt_mpc_timeout (struct bnxt *bp,struct bnxt_tx_ring_info *txr);int bnxt_mpc_cmp (struct bnxt *bp,struct bnxt_cp_ring_info *cpr,u32 *raw_cons);#else static inline void bnxt_alloc_mpc_info (struct bnxt *bp,u8 mpc_chnls_cap)
@@ -204,6 +209,20 @@ static inline int bnxt_start_xmit_mpc(struct bnxt *bp,return -EOPNOTSUPP ; } +static inline void bnxt_free_mpc_entries (struct bnxt *bp) +{ +} + +static inline void bnxt_mpc_ring_reset_task (struct bnxt *bp) +{ +} + +static inline int bnxt_mpc_timeout (struct bnxt *bp, +struct bnxt_tx_ring_info *txr) +{ +return -EOPNOTSUPP ; +} +static inline int bnxt_mpc_cmp (struct bnxt *bp,struct bnxt_cp_ring_info *cpr,u32 *raw_cons) {
Signature · 1 line
2.51.0
Michael Chan
maintainerRe: [PATCH net-next v6 11/15] bnxt_en: Add crypto MPC transmit/completion infrastructure
Responding to Sashiko comments:
https://sashiko.dev/#/patchset/20260810051358.1244418-1-michael.chan%40broadcom.com
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c int bnxt_key_ctx_alloc_one(struct bnxt *bp, struct bnxt_kctx *kctx, u8 kind, · line 427
Quoted 5 lines · Michael Chan
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +4
-0+int bnxt_xmit_crypto_cmd (struct bnxt *bp,struct bnxt_tx_ring_info *txr, +void *cmd,unsigned int len,unsigned int tmo) + spin_lock_bh(&txr->tx_lock); + rc = bnxt_start_xmit_mpc(bp, txr, cmd, len, handle);
Sashiko:
Since bnxt_start_xmit_mpc() only checks txr->dev_state against BNXT_DEV_STATE_CLOSING and lacks a check for the global BNXT_STATE_OPEN flag, can this proceed after the device is closed?
When the device is closed, bnxt_close_nic() frees the MPC ring memory but does not update dev_state to BNXT_DEV_STATE_CLOSING. If a caller asynchronously invokes this while the device is being closed, could it end up writing to the freed tx_desc_ring memory and cause a use-after-free?
Response:BNXT_DEV_STATE_CLOSING is set during MPC timeout/reset, not during close. During close, it will wait for all pending kTLS MPC commands to finish or abort before proceeding to free the MPC rings.
Quoted 6 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +6
-0+static bool bnxt_disable_mpc_ring (struct bnxt_mpc_info *mpc,int mpc_ring) +{ +struct bnxt_tx_ring_info *txr; +bool disabled = false; +int i; +
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c int bnxt_start_xmit_mpc(struct bnxt *bp, struct bnxt_tx_ring_info *txr, · line 525
Quoted 10 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +10
-0+for (i =0 ; i <BNXT_MPC_TYPE_MAX ; i++) { +if (mpc_ring >= mpc->mpc_ring_count[i]) +continue ; + txr = &mpc->mpc_rings[i][mpc_ring]; +spin_lock_bh (&txr->tx_lock); +if (!READ_ONCE (txr->dev_state)) { + disabled = true; +WRITE_ONCE (txr->dev_state,BNXT_DEV_STATE_CLOSING ); + } +spin_unlock_bh (&txr->tx_lock);
Sashiko:
If the first ring type (e.g., i=0) is already marked as CLOSING from a previous timeout, the disabled variable will remain false. Does breaking early here leave subsequent ring types active (dev_state == 0)? Concurrently, the workqueue task bnxt_mpc_ring_reset_task() observes the first ring as CLOSING and tears down all ring types sharing the same index via bnxt_mpc_ring_stop().
Since subsequent ring types are still active, could this allow concurrent calls to bnxt_xmit_crypto_cmd() to write to the ring memory while it is being freed without lock synchronization?
Response:
This is indeed possible if we are in the middle here and a concurrent workqueue is resetting another MPC ring and sees that this ring is in the middle of being disabled. I should be able to come up with a simple fix for this.
Michael Chan
maintainerPATCH v6 12/15 bnxt_en: Support kTLS TX offload by implementing .tls_dev_add/del()
In reply to Michael ChanAdd basic infrastructure to allocate and free kTLS context IDs (KIDs) to support kTLS TX offload. To offload a connection in .tls_dev_add(), the first step is to allocate a KID. After that the kTLS offload command is sent to the HW via MPC using the function bnxt_xmit_crypto_cmd() introduced in the last patch.
In .tls_dev_del(), we send the delete command to the HW using the same bnxt_xmit_crypto_cmd(). After that we free the KID, making it available for new offload. There is extra logic to handle ifdown, FW reset, and device reconfiguration while deleting the connection.
bnxt_ktls_init() assigns bnxt_ktls_ops to the netdev and sets up the TLS TX offload feature. bnxt_ktls_init() will be called in the next patch.
---
v4:
https://lore.kernel.org/netdev/[email protected]/
Fix uninitialized kctx_tx variable warning.
v3:
https://lore.kernel.org/netdev/[email protected]/
Use wait_event() in bnxt_ktls_dev_del() in case the device is in transient reconfig state instead of poll wait.
Call bnxt_crypto_del_all() after BNXT_STATE_OPEN is cleared as documented.
Use memzero_explicit() to clear sensitive TLS parameters.
v2:
https://lore.kernel.org/netdev/[email protected]/
Fix unused variable warning
Fix error recovery issues
v1:
https://lore.kernel.org/netdev/[email protected]/
--- drivers/net/ethernet/broadcom/bnxt/Makefile | 2 +- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 16 +- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 1 + .../net/ethernet/broadcom/bnxt/bnxt_crypto.c | 82 ++++- .../net/ethernet/broadcom/bnxt/bnxt_crypto.h | 11 + .../net/ethernet/broadcom/bnxt/bnxt_ktls.c | 345 ++++++++++++++++++ .../net/ethernet/broadcom/bnxt/bnxt_ktls.h | 112 ++++++ 7 files changed, 566 insertions(+), 3 deletions(-) create mode 100644 drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c create mode 100644 drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.h
drivers/net/ethernet/broadcom/bnxt/Makefile +1 -1
@@ -5,4 +5,4 @@ bnxt_en-y := bnxt.o bnxt_hwrm.o bnxt_sriov.o bnxt_ethtool.o bnxt_dcb.o bnxt_ulp. bnxt_en-$(CONFIG_BNXT_FLOWER_OFFLOAD) += bnxt_tc.o bnxt_en-$(CONFIG_DEBUG_FS) += bnxt_debugfs.o bnxt_en-$(CONFIG_BNXT_HWMON) += bnxt_hwmon.o -bnxt_en-$(CONFIG_BNXT_TLS) += bnxt_mpc.o bnxt_crypto.o +bnxt_en-$(CONFIG_BNXT_TLS) += bnxt_mpc.o bnxt_crypto.o bnxt_ktls.o
drivers/net/ethernet/broadcom/bnxt/bnxt.c +15 -1
@@ -78,6 +78,7 @@#include <net/tso.h> #include "bnxt_mpc.h" #include "bnxt_crypto.h" +#include "bnxt_ktls.h" #define BNXT_TX_TIMEOUT (5 * HZ)#define BNXT_DEF_MSG_ENABLE (NETIF_MSG_DRV | NETIF_MSG_HW | \
@@ -13351,6 +13352,7 @@ static int __bnxt_open_nic(struct bnxt *bp, bool irq_re_init, bool link_re_init)static_branch_disable (&bnxt_xdp_locking_key); }set_bit (BNXT_STATE_OPEN , &bp->state); +bnxt_ktls_wake (bp);bnxt_enable_int (bp);/* Enable TX queues */ bnxt_tx_enable (bp);
@@ -13492,7 +13494,8 @@ static int bnxt_open(struct net_device *dev)static bool bnxt_drv_busy (struct bnxt *bp) {return (test_bit (BNXT_STATE_IN_SP_TASK , &bp->state) || -test_bit (BNXT_STATE_READ_STATS , &bp->state)); +test_bit (BNXT_STATE_READ_STATS , &bp->state) || +bnxt_ktls_busy (bp)); } static void bnxt_get_ring_stats(struct bnxt *bp,
@@ -13510,9 +13513,20 @@ static void __bnxt_close_nic(struct bnxt *bp, bool irq_re_init,clear_bit (BNXT_STATE_OPEN , &bp->state);smp_mb__after_atomic (); +/* Wake any kTLS delete waiting on a reconfig so it re-evaluates and +* either keeps waiting for the reopen or aborts (ifdown / FW reset). +*/ +bnxt_ktls_wake (bp);while (bnxt_drv_busy (bp))msleep (20 ); +/* Delete all crypto connections and KIDs only on ifdown and FW reset, +* not ethtool config changes. +*/ +if (!netif_running (bp->dev) || +test_bit (BNXT_STATE_IN_FW_RESET , &bp->state)) +bnxt_crypto_del_all (bp); +if (BNXT_SUPPORTS_MULTI_RSS_CTX (bp))bnxt_clear_rss_ctxs (bp);/* Flush rings and disable interrupts */
drivers/net/ethernet/broadcom/bnxt/bnxt.h +1 -0
@@ -2477,6 +2477,7 @@ struct bnxt {struct bnxt_mpc_info *mpc_info;struct bnxt_crypto_info *crypto_info; +struct bnxt_tls_info *ktls_info;unsigned int current_interval;#define BNXT_TIMER_INTERVAL HZ
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +81 -1
106 lines not on this page · raw message
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.h +11 -0
@@ -148,6 +148,8 @@ struct bnxt_crypto_cmd_ctx {#ifdef CONFIG_BNXT_TLS void bnxt_alloc_crypto_info (struct bnxt *bp,struct hwrm_func_qcaps_output *resp); +int bnxt_crypto_del (struct bnxt *bp,u8 type,u8 kind,u32 kid); +void bnxt_crypto_del_all (struct bnxt *bp);void bnxt_clear_crypto (struct bnxt *bp);void bnxt_free_crypto_info (struct bnxt *bp); void bnxt_hwrm_reserve_pf_key_ctxs(struct bnxt *bp,
@@ -167,6 +169,15 @@ static inline void bnxt_alloc_crypto_info(struct bnxt *bp, { } +static inline int bnxt_crypto_del (struct bnxt *bp,u8 type,u8 kind,u32 kid) +{ +return -EOPNOTSUPP ; +} + +static inline void bnxt_crypto_del_all (struct bnxt *bp) +{ +} +static inline void bnxt_clear_crypto (struct bnxt *bp) { }
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c new file +345 -0
345 lines not on this page · raw message
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.h new file +112 -0
112 lines not on this page · raw message
Signature · 1 line
2.51.0
Michael Chan
maintainerPATCH v6 13/15 bnxt_en: Implement kTLS TX normal path
In reply to Michael ChanOffload TLS encryption of TX packets to the hardware if the TCP sequence number is the expected one. Fall back to software encryption otherwise. Implement all the TLS TX logic to check the TCP sequence number and set up the BD in the new function bnxt_ktls_xmit(). The stored TCP sequence number is only updated when BD is in the ring and guaranteed to be transmitted.
Basic kTLS statistics reporting for ethtool -S is also added. Because there can be multiple TX rings (if there are MQPRIO TCs) sharing the same bnxt_cp_ring_info containing the bnxt_sw_stats, we need to store the kTLS software TX stats in bnxt_tx_ring_info instead.
Note that the software data counters are 64-bit counters are incremented and read without synchronization just like the other existing software counters in the driver.
The next patches will add support for the exception path with out-of-order TCP sequence number.
---
v6:
Goto kick_pending if kTLS transmit failed to flush the DB. Handle the case of BPF truncating the TX packet.
v5:
https://lore.kernel.org/netdev/[email protected]/
Use BNXT_NO_FW_ACCESS() to fix possible massive AER timeout.
v3:
https://lore.kernel.org/netdev/[email protected]/
Use per TX ring TLS stats structures.
Defer tcp sequence and stats update until the BD is on the ring.
Increment the drop counter if kTLS drops the packet.
v2:
https://lore.kernel.org/netdev/[email protected]/
--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 23 ++++- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 41 +++++++- .../net/ethernet/broadcom/bnxt/bnxt_crypto.c | 1 + .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 46 +++++++++ drivers/net/ethernet/broadcom/bnxt/bnxt_gso.c | 2 +- .../net/ethernet/broadcom/bnxt/bnxt_ktls.c | 93 +++++++++++++++++++ .../net/ethernet/broadcom/bnxt/bnxt_ktls.h | 60 ++++++++++++ 7 files changed, 258 insertions(+), 8 deletions(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +20 -3
@@ -481,6 +481,7 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)dma_addr_t mapping;unsigned int length, pad =0 ;u32 len, free_size, vlan_tag_flags, cfa_action, flags; +struct bnxt_ktls_offload_ctx_tx *kctx_tx =NULL ;struct bnxt_ptp_cfg *ptp = bp->ptp_cfg;struct pci_dev *pdev = bp->pdev;u16 prod, last_frag, txts_prod;
@@ -488,6 +489,7 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)struct bnxt_sw_tx_bd *tx_buf;__le32 lflags =0 ;skb_frag_t *frag; +u32 kid =0 ; i =skb_get_queue_mapping (skb);if (unlikely (i >= bp->tx_nr_rings)) {
@@ -527,6 +529,12 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)return NETDEV_TX_BUSY ; } + skb =bnxt_ktls_xmit (bp, txr, skb, &lflags, &kid, &kctx_tx); +if (unlikely (!skb)) { +dev_core_stats_tx_dropped_inc (dev); + goto tx_kick_pending; + } + length = skb->len; len =skb_headlen (skb); last_frag =skb_shinfo (skb)->nr_frags;
@@ -675,7 +683,7 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev) prod =NEXT_TX (prod); txbd1 =bnxt_init_ext_bd (bp, txr, prod, lflags, vlan_tag_flags, - cfa_action); + cfa_action, kid);if (skb_is_gso (skb)) {bool udp_gso = !!(skb_shinfo (skb)->gso_type &SKB_GSO_UDP_L4 );
@@ -696,9 +704,10 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev) txbd1->tx_bd_hsize_lflags |=cpu_to_le32 (TX_BD_FLAGS_LSO |TX_BD_FLAGS_T_IPID | - (hdr_len << (TX_BD_HSIZE_SHIFT -1 ))); + ((hdr_len >>1 ) <<TX_BD_HSIZE_SHIFT )); length =skb_shinfo (skb)->gso_size; - txbd1->tx_bd_mss =cpu_to_le32 (length); + txbd1->tx_bd_kid_mss =cpu_to_le32 (BNXT_TX_KID_HI (kid) | + (length &TX_BD_MSS )); length += hdr_len; }else if (skb->ip_summed ==CHECKSUM_PARTIAL ) { txbd1->tx_bd_hsize_lflags |=
@@ -751,6 +760,9 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev) prod =NEXT_TX (prod);WRITE_ONCE (txr->tx_prod, prod); +/* Commit the kTLS state now that the BD is in the ring. */ +bnxt_ktls_xmit_commit (txr, kctx_tx); +if (!netdev_xmit_more () ||netif_xmit_stopped (txq)) {bnxt_txr_db_kick (bp, txr, prod); }else {
@@ -4087,6 +4099,8 @@ static void bnxt_free_tx_rings(struct bnxt *bp)bnxt_free_tx_inline_buf (txr, pdev); +bnxt_ktls_free_tx_ring_stats (txr); + ring = &txr->tx_ring_struct;bnxt_free_ring (bp, &ring->ring_mem);
@@ -4162,6 +4176,9 @@ static int bnxt_alloc_tx_rings(struct bnxt *bp) qidx = bp->tc_to_qidx[j]; ring->queue_id = bp->q_info[qidx].queue_id;spin_lock_init (&txr->tx_lock); + rc =bnxt_ktls_alloc_tx_ring_stats (bp, txr); +if (rc) +return rc;if (i < bp->tx_nr_rings_xdp)continue ;if (BNXT_RING_TO_TC_OFF (bp, i) == (bp->tx_nr_rings_per_tc -1 ))
drivers/net/ethernet/broadcom/bnxt/bnxt.h +37 -4
79 lines not on this page · raw message
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +1 -0
@@ -574,6 +574,7 @@ int bnxt_crypto_init(struct bnxt *bp)if (rc)return rc; +bnxt_ktls_init (bp);return 0 ; }
drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c +46 -0
82 lines not on this page · raw message
drivers/net/ethernet/broadcom/bnxt/bnxt_gso.c +1 -1
@@ -168,7 +168,7 @@ netdev_tx_t bnxt_sw_udp_gso_xmit(struct bnxt *bp, prod =NEXT_TX (prod);bnxt_init_ext_bd (bp, txr, prod, csum, - vlan_tag_flags, cfa_action); + vlan_tag_flags, cfa_action,0 );/* set dma_unmap_len on the LAST BD touching each * region. Since completions are in-order, the last segment
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c +93 -0
102 lines not on this page · raw message
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.h +60 -0
77 lines not on this page · raw message
Signature · 1 line
2.51.0
Michael Chan
maintainerPATCH v6 14/15 bnxt_en: Add support for inline transmit BDs
In reply to Michael ChanNewer chips (P5_PLUS) support inline transmit BDs that contain extra data. One such use case is to transmit out-of-sequence kTLS packets with encryption enabled. To account for these inline BDs during TX completion, we add the inline_data_bds field to struct bnxt_sw_tx_bd (tx_buf). tx_buf->is_push will always be set when sending these inline BDs as the operation is similar to push packets. tx_buf->skb will always be NULL as there is no associated SKB.
The next patch will make use of this feature.
---
v6:
Clear is_push and inline_data_bds during TX ring free.
--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 21 ++++++++++++++++++--- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 1 + 2 files changed, 19 insertions(+), 3 deletions(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +18 -3
@@ -848,7 +848,7 @@ static bool __bnxt_tx_int(struct bnxt *bp, struct bnxt_tx_ring_info *txr, head_buf = tx_buf; skb = tx_buf->skb; -if (unlikely (!skb)) { +if (unlikely (!skb && !tx_buf->is_push)) {bnxt_sched_reset_txr (bp, txr, cons);return rc; }
@@ -860,13 +860,22 @@ static bool __bnxt_tx_int(struct bnxt *bp, struct bnxt_tx_ring_info *txr, } cons =NEXT_TX (cons); - tx_pkts++; - tx_bytes += skb->len; +if (skb) { + tx_pkts++; + tx_bytes += skb->len; + } tx_buf->skb =NULL ; tx_buf->is_ts_pkt =0 ;if (tx_buf->is_push) { tx_buf->is_push =0 ; + cons += tx_buf->inline_data_bds; + tx_buf->inline_data_bds =0 ; +if (!skb) { +/* presync BD */ + cons =NEXT_TX (cons); +continue ; + } goto next_tx_int; }
@@ -3481,6 +3490,10 @@ static void bnxt_free_one_tx_ring_skbs(struct bnxt *bp, skb = tx_buf->skb;if (!skb) { +if (tx_buf->is_push) { + tx_buf->is_push =0 ; + tx_buf->inline_data_bds =0 ; + } i++;continue ; }
@@ -3489,6 +3502,8 @@ static void bnxt_free_one_tx_ring_skbs(struct bnxt *bp,if (tx_buf->is_push) {dev_kfree_skb (skb); + tx_buf->is_push =0 ; + tx_buf->inline_data_bds =0 ; i +=2 ;continue ; }
drivers/net/ethernet/broadcom/bnxt/bnxt.h +1 -0
@@ -918,6 +918,7 @@ struct bnxt_sw_tx_bd {u8 is_push;u8 is_sw_gso;u8 action; +u8 inline_data_bds;unsigned short nr_frags;union {u16 rx_prod;
Signature · 1 line
2.51.0
Michael Chan
maintainerPATCH v6 15/15 bnxt_en: Add kTLS retransmission support
In reply to Michael ChanIf TCP retransmits a TLS packet that requires encryption by the NIC, the TCP sequence number will go backwards and the hardware will require some assistance from the driver. The driver needs to retrieve the TLS record that covers the byte sequence of the retransmitted packet. If the retransmitted packet does not include the tag, the hardware can simply encrypt the packet using the informtaion in the TLS record.
The driver provides the TLS record information for the retransmitted packet in the presync TX BD. The presync TX BD introduced in the last patch is treated very much like a TX push BD with inline data. The only exception is that no SKB will be stored for the presync TX BD.
Retransmission that includes the TLS tag will be handled in future patches.
---
v5:
https://lore.kernel.org/netdev/[email protected]/
Use bool instead of bit fields to keep track of kTLS states (requested by Paolo).
v3:
https://lore.kernel.org/netdev/[email protected]/
Unwind the TX ring properly if the TLS packet cannot be sent.
Improve the OOO TLS counters.
Fix endianness of the record sequence number.
Check valid return address from skb_frag_address_safe().
v2:
https://lore.kernel.org/netdev/[email protected]/
--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 14 +- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 4 + .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 4 + .../net/ethernet/broadcom/bnxt/bnxt_ktls.c | 152 +++++++++++++++++- .../net/ethernet/broadcom/bnxt/bnxt_ktls.h | 2 + 5 files changed, 167 insertions(+), 9 deletions(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +11 -3
@@ -482,9 +482,9 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)unsigned int length, pad =0 ;u32 len, free_size, vlan_tag_flags, cfa_action, flags;struct bnxt_ktls_offload_ctx_tx *kctx_tx =NULL ; +u16 prod, start_prod, last_frag, txts_prod;struct bnxt_ptp_cfg *ptp = bp->ptp_cfg;struct pci_dev *pdev = bp->pdev; -u16 prod, last_frag, txts_prod;struct bnxt_tx_ring_info *txr;struct bnxt_sw_tx_bd *tx_buf;__le32 lflags =0 ;
@@ -500,7 +500,6 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev) txq =netdev_get_tx_queue (dev, i); txr = &bp->tx_ring[bp->tx_ring_map[i]]; - prod = txr->tx_prod;#if (MAX_SKB_FRAGS >TX_MAX_FRAGS )if (skb_shinfo (skb)->nr_frags >TX_MAX_FRAGS ) {
@@ -529,12 +528,14 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)return NETDEV_TX_BUSY ; } + start_prod = txr->tx_prod; skb =bnxt_ktls_xmit (bp, txr, skb, &lflags, &kid, &kctx_tx);if (unlikely (!skb)) {dev_core_stats_tx_dropped_inc (dev); goto tx_kick_pending; } + prod = txr->tx_prod; length = skb->len; len =skb_headlen (skb); last_frag =skb_shinfo (skb)->nr_frags;
@@ -817,9 +818,16 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)/* set SKB to err so PTP worker will clean up */ ptp->txts_req[txts_prod].tx_skb =ERR_PTR (-EIO ); } + txr->tx_buf_ring[RING_TX (bp, txr->tx_prod)].skb =NULL ; +/* Unwind any kTLS presync BDs */ +if (unlikely (txr->tx_prod != start_prod)) { + tx_buf = &txr->tx_buf_ring[RING_TX (bp, start_prod)]; + tx_buf->is_push =0 ; + tx_buf->inline_data_bds =0 ; +WRITE_ONCE (txr->tx_prod, start_prod); + }if (txr->kick_pending)bnxt_txr_db_kick (bp, txr, txr->tx_prod); - txr->tx_buf_ring[RING_TX (bp, txr->tx_prod)].skb =NULL ;dev_core_stats_tx_dropped_inc (dev);return NETDEV_TX_OK ; }
drivers/net/ethernet/broadcom/bnxt/bnxt.h +4 -0
@@ -1193,6 +1193,10 @@ struct bnxt_cmn_sw_stats {enum bnxt_ktls_data_counters {BNXT_KTLS_TX_PKTS =0 ,BNXT_KTLS_TX_BYTES , +BNXT_KTLS_TX_OOO_PKTS , +BNXT_KTLS_TX_OOO_FALLBACK_NO_SYNC , +BNXT_KTLS_TX_OOO_FALLBACK_NO_SPACE , +BNXT_KTLS_TX_OOO_FALLBACK_NO_HDR ,BNXT_KTLS_MAX_DATA_COUNTERS , };
drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c +4 -0
@@ -359,6 +359,10 @@ static const char *const bnxt_ring_drv_stats_arr[] = {static const char *const bnxt_ktls_data_stats[] = { [BNXT_KTLS_TX_PKTS ] ="tx_tls_encrypted_packets" , [BNXT_KTLS_TX_BYTES ] ="tx_tls_encrypted_bytes" , + [BNXT_KTLS_TX_OOO_PKTS ] ="tx_tls_ooo_packets" , + [BNXT_KTLS_TX_OOO_FALLBACK_NO_SYNC ] ="tx_tls_ooo_fallback_no_sync" , + [BNXT_KTLS_TX_OOO_FALLBACK_NO_SPACE ] ="tx_tls_ooo_fallback_no_space" , + [BNXT_KTLS_TX_OOO_FALLBACK_NO_HDR ] ="tx_tls_ooo_fallback_no_hdr" , };/* kTLS control plane counter strings indexed by enum bnxt_ktls_ctrl_counters */
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c +146 -6
187 lines not on this page · raw message
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.h +2 -0
@@ -42,6 +42,8 @@ struct bnxt_ktls_offload_ctx_tx {u32 next_tcp_seq_no;/* staged tcp seq no */ u32 kid;u32 pending_bytes;/* staged payload bytes */ +bool pending_fwd; +bool pending_ooo; };struct bnxt_ktls_tx_driver_state {
Signature · 1 line
2.51.0