bnxt_en: Add kTLS TX offload support
Michael Chan
maintainerv2 cover bnxt_en: Add kTLS TX offload support
This patchset adds kTLS offload support for TX direction. A number of new files are added:
bnxt_mpc.[ch] handle midpath channels (MPCs) used to offload kTLS connections to the chip's crypto blocks without going through FW.
bnxt_crypto.[ch] handle the crypto interface and resources.
bnxt_ktls.[ch] handle kTLS offload.
A new CONFIG_BNXT_TLS is added to enable all of the above. The first 6 patches add the MPC logic including resource accounting and reservations. The next 5 patches add the crypto logic to handle the crypto resources and to send/receive control data using the MPCs. The last 4 patches add kTLS offload for the TX direction.
There will be a follow-on patchset to make the TX offload more complete and to add the RX direction offload.
v2:
Fix unused variable compile warnings in patch 10 and 12 by reorganizing the patches (reported by Jakub)
Fix some error recovery issues in patch 12
v1:
https://lore.kernel.org/netdev/[email protected]/
Michael Chan (15):
bnxt_en: Add Midpath channel information
bnxt_en: Account for the MPC TX and CP rings
bnxt_en: Set default MPC ring count
bnxt_en: Rename xdp_tx_lock to tx_lock
bnxt_en: Allocate and free MPC software structures
bnxt_en: Allocate and free MPC channels from firmware
bnxt_en: Allocate crypto structure and backing store
bnxt_en: Reserve crypto RX and TX key contexts on a PF
bnxt_en: Add infrastructure for crypto key context IDs
bnxt_en: Add MPC transmit and completion functions
bnxt_en: Add crypto MPC transmit/completion infrastructure
bnxt_en: Support kTLS TX offload by implementing .tls_dev_add/del()
bnxt_en: Implement kTLS TX normal path
bnxt_en: Add support for inline transmit BDs
bnxt_en: Add kTLS retransmission support
drivers/net/ethernet/broadcom/Kconfig | 9 + drivers/net/ethernet/broadcom/bnxt/Makefile | 1 + drivers/net/ethernet/broadcom/bnxt/bnxt.c | 206 ++++- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 84 ++- .../net/ethernet/broadcom/bnxt/bnxt_crypto.c | 575 ++++++++++++++ .../net/ethernet/broadcom/bnxt/bnxt_crypto.h | 223 ++++++ .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 50 ++ drivers/net/ethernet/broadcom/bnxt/bnxt_gso.c | 2 +- .../net/ethernet/broadcom/bnxt/bnxt_ktls.c | 458 +++++++++++ .../net/ethernet/broadcom/bnxt/bnxt_ktls.h | 128 ++++ drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c | 711 ++++++++++++++++++ drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h | 210 ++++++ .../net/ethernet/broadcom/bnxt/bnxt_sriov.c | 6 +- drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c | 4 +- include/linux/bnxt/hsi.h | 37 + 15 files changed, 2654 insertions(+), 50 deletions(-) create mode 100644 drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c create mode 100644 drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.h create mode 100644 drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c create mode 100644 drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.h create mode 100644 drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c create mode 100644 drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h
Signature · 1 line
2.51.0
Michael Chan
maintainerPATCH v2 1/15 bnxt_en: Add Midpath channel information
Midpath channels (MPCs) are rings for hardware control paths. These control paths are used to offload kTLS directly to the hardware without going through firmware. This patch adds the basic information structures for these MPCs.
An MPC is basically a TX and completion ring pair with a HW TLS block as the destination. Two MPC channel types are used to offload connections to the TX crypto engine (TCE) and the RX crypto engine (RCE) respectively. In the driver, we re-use the bnxt_tx_ring_info and bnxt_cp_ring_info control structs for the MPCs.
This patch also adds the CONFIG_BNXT_TLS Kconfig option to conditionally include the MPC logic. The first few patches in the series add the MPC support. kTLS support will be added later in the series.
--- drivers/net/ethernet/broadcom/Kconfig | 9 ++++ drivers/net/ethernet/broadcom/bnxt/Makefile | 1 + drivers/net/ethernet/broadcom/bnxt/bnxt.c | 8 ++++ drivers/net/ethernet/broadcom/bnxt/bnxt.h | 2 + drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c | 26 ++++++++++ drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h | 47 +++++++++++++++++++ 6 files changed, 93 insertions(+) create mode 100644 drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c create mode 100644 drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h
drivers/net/ethernet/broadcom/Kconfig +9 -0
@@ -254,6 +254,15 @@ config BNXT_HWMON Say Y if you want to expose the thermal sensor data on NetXtreme-C/E devices, via the hwmon sysfs interface. +config BNXT_TLS + bool "Broadcom NetXtreme-C/E TLS offload support" + default y + depends on BNXT && TLS_DEVICE + depends on TLS=y || BNXT=m + help + Say Y if you want to enable Transport Layer Security (TLS) hardware + encryption and decryption offload on supported NetXtreme-C/E devices. + config BNGE tristate "Broadcom ThorUltra Ethernet device support" depends on PCI
drivers/net/ethernet/broadcom/bnxt/Makefile +1 -0
@@ -5,3 +5,4 @@ bnxt_en-y := bnxt.o bnxt_hwrm.o bnxt_sriov.o bnxt_ethtool.o bnxt_dcb.o bnxt_ulp. bnxt_en-$(CONFIG_BNXT_FLOWER_OFFLOAD) += bnxt_tc.o bnxt_en-$(CONFIG_DEBUG_FS) += bnxt_debugfs.o bnxt_en-$(CONFIG_BNXT_HWMON) += bnxt_hwmon.o +bnxt_en-$(CONFIG_BNXT_TLS) += bnxt_mpc.o
drivers/net/ethernet/broadcom/bnxt/bnxt.c +8 -0
@@ -76,6 +76,7 @@#include "bnxt_hwmon.h" #include "bnxt_gso.h" #include <net/tso.h> +#include "bnxt_mpc.h" #define BNXT_TX_TIMEOUT (5 * HZ)#define BNXT_DEF_MSG_ENABLE (NETIF_MSG_DRV | NETIF_MSG_HW | \
@@ -9937,6 +9938,11 @@ static int __bnxt_hwrm_func_qcaps(struct bnxt *bp) } bp->tso_max_segs =le16_to_cpu (resp->max_tso_segs); +if (resp->mpc_chnls_cap) +bnxt_alloc_mpc_info (bp, resp->mpc_chnls_cap); +else +bnxt_free_mpc_info (bp); + hwrm_func_qcaps_exit:hwrm_req_drop (bp, req);return rc;
@@ -16485,6 +16491,7 @@ static void bnxt_remove_one(struct pci_dev *pdev) bp->ptp_cfg =NULL ;kfree (bp->fw_health); bp->fw_health =NULL ; +bnxt_free_mpc_info (bp);bnxt_cleanup_pci (bp);bnxt_free_ctx_mem (bp, true);bnxt_free_crash_dump_mem (bp);
@@ -17156,6 +17163,7 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)bnxt_ethtool_free (bp);kfree (bp->fw_health); bp->fw_health =NULL ; +bnxt_free_mpc_info (bp);bnxt_cleanup_pci (bp);bnxt_free_ctx_mem (bp, true);bnxt_free_crash_dump_mem (bp);
drivers/net/ethernet/broadcom/bnxt/bnxt.h +2 -0
@@ -2451,6 +2451,8 @@ struct bnxt {u8 tph_mode; +struct bnxt_mpc_info *mpc_info; +unsigned int current_interval;#define BNXT_TIMER_INTERVAL HZ
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c new file +26 -0
@@ -0,0 +1,26 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright (c) 2026 Broadcom Inc. */ + +#include <linux/stddef.h> +#include <linux/types.h> +#include <linux/slab.h> +#include <linux/bnxt/hsi.h> + +#include "bnxt.h" +#include "bnxt_mpc.h" + +void bnxt_alloc_mpc_info (struct bnxt *bp,u8 mpc_chnls_cap) +{ +if (!bp->mpc_info) + bp->mpc_info =kzalloc_obj (*bp->mpc_info); +if (bp->mpc_info) + bp->mpc_info->mpc_chnls_cap = mpc_chnls_cap; +else +netdev_warn (bp->dev,"Unable to allocate MPC info\n" ); +} + +void bnxt_free_mpc_info (struct bnxt *bp) +{ +kfree (bp->mpc_info); + bp->mpc_info =NULL ; +}
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h new file +47 -0
47 lines not on this page · raw message
Signature · 1 line
2.51.0
Michael Chan
maintainerPATCH v2 2/15 bnxt_en: Account for the MPC TX and CP rings
In reply to Michael ChanModify bnxt_cp_rings_in_use(), bnxt_get_max_func_cp_rings_for_en(), and _bnxt_get_max_rings() to account for any TX rings and CP rings used by MPCs. Add a new helper bnxt_total_tx_rings() to include MPC TX rings. Ring reservations will now include the MPC rings.
--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 45 ++++++++++++++----- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 1 + drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c | 21 +++++++++ drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h | 12 +++++ .../net/ethernet/broadcom/bnxt/bnxt_sriov.c | 6 +-- 5 files changed, 71 insertions(+), 14 deletions(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +34 -11
@@ -7746,6 +7746,11 @@ static void bnxt_hwrm_ring_free(struct bnxt *bp, bool close_path) } } +int bnxt_total_tx_rings (struct bnxt *bp) +{ +return bp->tx_nr_rings +bnxt_mpc_tx_rings_in_use (bp); +} +static int __bnxt_trim_rings (struct bnxt *bp,int *rx,int *tx,int max,bool shared); static int bnxt_trim_rings(struct bnxt *bp, int *rx, int *tx, int max,
@@ -7786,19 +7791,28 @@ static int bnxt_hwrm_get_rings(struct bnxt *bp) stats =le16_to_cpu (resp->alloc_stat_ctx); hw_resc->resv_irqs = cp;if (bp->flags &BNXT_FLAG_CHIP_P5_PLUS ) { +int mpc_tx =bnxt_mpc_tx_rings_in_use (bp); +int mpc_cp =bnxt_mpc_cp_rings_in_use (bp);int rx = hw_resc->resv_rx_rings;int tx = hw_resc->resv_tx_rings; +int cp_p5; +if (tx <= mpc_tx || cp <= mpc_cp) { + rc = -ENOMEM ; + goto get_rings_exit; + } + tx -= mpc_tx; + cp_p5 = cp - mpc_cp;if (bp->flags &BNXT_FLAG_AGG_RINGS ) rx >>=1 ; -if (cp < (rx + tx)) { - rc =__bnxt_trim_rings (bp, &rx, &tx, cp, false); +if (cp_p5 < (rx + tx)) { + rc =__bnxt_trim_rings (bp, &rx, &tx, cp_p5, false);if (rc) goto get_rings_exit;if (bp->flags &BNXT_FLAG_AGG_RINGS ) rx <<=1 ; hw_resc->resv_rx_rings = rx; - hw_resc->resv_tx_rings = tx; + hw_resc->resv_tx_rings = tx + mpc_tx; } hw_resc->resv_irqs =le16_to_cpu (resp->alloc_msix); hw_resc->resv_hw_ring_grps = rx;
@@ -7990,7 +8004,7 @@ static int bnxt_cp_rings_in_use(struct bnxt *bp)return bnxt_nq_rings_in_use (bp); cp = bp->tx_nr_rings + bp->rx_nr_rings; -return cp; +return cp +bnxt_mpc_cp_rings_in_use (bp); }static int bnxt_get_func_stat_ctxs (struct bnxt *bp)
@@ -8048,7 +8062,7 @@ static void bnxt_get_total_resources(struct bnxt *bp, struct bnxt_hw_rings *hwr) hwr->cp_p5 =0 ;if (bp->flags &BNXT_FLAG_CHIP_P5_PLUS ) hwr->cp_p5 =bnxt_cp_rings_in_use (bp); - hwr->tx = bp->tx_nr_rings; + hwr->tx =bnxt_total_tx_rings (bp); hwr->rx = bp->rx_nr_rings; hwr->grp = hwr->rx; hwr->vnic =bnxt_get_total_vnics (bp, hwr->rx);
@@ -8154,8 +8168,10 @@ static int __bnxt_reserve_rings(struct bnxt *bp) hwr.rx = bp->rx_nr_rings;if (bp->flags &BNXT_FLAG_SHARED_RINGS ) sh = true; -if (bp->flags &BNXT_FLAG_CHIP_P5_PLUS ) - hwr.cp_p5 = hwr.rx + hwr.tx; +if (bp->flags &BNXT_FLAG_CHIP_P5_PLUS ) { + hwr.cp_p5 = hwr.rx + hwr.tx +bnxt_mpc_cp_rings_in_use (bp); + hwr.tx +=bnxt_mpc_tx_rings_in_use (bp); + } hwr.vnic =bnxt_get_total_vnics (bp, hwr.rx);
@@ -8192,6 +8208,9 @@ static int __bnxt_reserve_rings(struct bnxt *bp)if (bnxt_ulp_registered (edev) && hwr.stat >bnxt_get_ulp_stat_ctxs (bp)) hwr.stat -=bnxt_get_ulp_stat_ctxs (bp); hwr.cp =min_t (int, hwr.cp, hwr.stat); + hwr.tx -=bnxt_mpc_tx_rings_in_use (bp); +if (hwr.tx <0 ) +return -ENOMEM ; rc =bnxt_trim_rings (bp, &rx_rings, &hwr.tx, hwr.cp, sh);if (bp->flags &BNXT_FLAG_AGG_RINGS ) hwr.rx = rx_rings <<1 ;
@@ -11468,12 +11487,13 @@ unsigned int bnxt_get_max_func_cp_rings(struct bnxt *bp)static unsigned int bnxt_get_max_func_cp_rings_for_en (struct bnxt *bp) { +unsigned int mpc_cp = (unsigned int )bnxt_mpc_cp_rings_in_use (bp);unsigned int cp = bp->hw_resc.max_cp_rings;if (!(bp->flags &BNXT_FLAG_CHIP_P5_PLUS )) cp -=bnxt_get_ulp_msix_num (bp); -return cp; +return mpc_cp >= cp ?0 : cp - mpc_cp; }static unsigned int bnxt_get_max_func_irqs (struct bnxt *bp)
@@ -14868,8 +14888,10 @@ int bnxt_check_rings(struct bnxt *bp, int tx, int rx, bool sh, int tcs, hwr.grp = rx; hwr.rss_ctx =bnxt_get_total_rss_ctxs (bp, &hwr); } -if (bp->flags &BNXT_FLAG_CHIP_P5_PLUS ) - hwr.cp_p5 = hwr.tx + rx; +if (bp->flags &BNXT_FLAG_CHIP_P5_PLUS ) { + hwr.cp_p5 = hwr.tx + rx +bnxt_mpc_cp_rings_in_use (bp); + hwr.tx +=bnxt_mpc_tx_rings_in_use (bp); + } rc =bnxt_hwrm_check_rings (bp, &hwr);if (!rc &&pci_msix_can_alloc_dyn (bp->pdev)) {if (!bnxt_ulp_registered (bp->edev[BNXT_AUXDEV_RDMA ])) {
@@ -16561,7 +16583,8 @@ static void _bnxt_get_max_rings(struct bnxt *bp, int *max_rx, int *max_tx,struct bnxt_hw_resc *hw_resc = &bp->hw_resc;int max_ring_grps =0 , max_irq; - *max_tx = hw_resc->max_tx_rings; + *max_tx =max (0 , (int )hw_resc->max_tx_rings - +bnxt_mpc_tx_rings_in_use (bp)); *max_rx = hw_resc->max_rx_rings; *max_cp =bnxt_get_max_func_cp_rings_for_en (bp); max_irq = min_t(int, bnxt_get_max_func_irqs(bp) -
drivers/net/ethernet/broadcom/bnxt/bnxt.h +1 -0
@@ -2989,6 +2989,7 @@ int bnxt_hwrm_vnic_cfg(struct bnxt *bp, struct bnxt_vnic_info *vnic);int bnxt_hwrm_vnic_alloc (struct bnxt *bp,struct bnxt_vnic_info *vnic,unsigned int start_rx_ring_idx,unsigned int nr_rings); +int bnxt_total_tx_rings (struct bnxt *bp);int __bnxt_hwrm_get_tx_rings (struct bnxt *bp,u16 fid,int *tx_rings);int bnxt_nq_rings_in_use (struct bnxt *bp);int bnxt_hwrm_set_coal (struct bnxt *);
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +21 -0
@@ -24,3 +24,24 @@ void bnxt_free_mpc_info(struct bnxt *bp)kfree (bp->mpc_info); bp->mpc_info =NULL ; } + +int bnxt_mpc_tx_rings_in_use (struct bnxt *bp) +{ +struct bnxt_mpc_info *mpc = bp->mpc_info; +int i, mpc_tx =0 ; + +if (!mpc) +return 0 ; +for (i =0 ; i <BNXT_MPC_TYPE_MAX ; i++) + mpc_tx += mpc->mpc_ring_count[i]; +return mpc_tx; +} + +int bnxt_mpc_cp_rings_in_use (struct bnxt *bp) +{ +struct bnxt_mpc_info *mpc = bp->mpc_info; + +if (!mpc) +return 0 ; +return mpc->mpc_cp_rings; +}
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h +12 -0
@@ -35,6 +35,8 @@ struct bnxt_mpc_info {#ifdef CONFIG_BNXT_TLS void bnxt_alloc_mpc_info (struct bnxt *bp,u8 mpc_chnls_cap);void bnxt_free_mpc_info (struct bnxt *bp); +int bnxt_mpc_tx_rings_in_use (struct bnxt *bp); +int bnxt_mpc_cp_rings_in_use (struct bnxt *bp);#else static inline void bnxt_alloc_mpc_info (struct bnxt *bp,u8 mpc_chnls_cap) {
@@ -43,5 +45,15 @@ static inline void bnxt_alloc_mpc_info(struct bnxt *bp, u8 mpc_chnls_cap)static inline void bnxt_free_mpc_info (struct bnxt *bp) { } + +static inline int bnxt_mpc_tx_rings_in_use (struct bnxt *bp) +{ +return 0 ; +} + +static inline int bnxt_mpc_cp_rings_in_use (struct bnxt *bp) +{ +return 0 ; +}#endif /* CONFIG_BNXT_TLS */ #endif /* BNXT_MPC_H */
drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c +3 -3
@@ -640,7 +640,7 @@ static int bnxt_hwrm_func_vf_resc_cfg(struct bnxt *bp, int num_vfs, bool reset) vf_rx_rings = hw_resc->max_rx_rings - bp->rx_nr_rings *2 ; else vf_rx_rings = hw_resc->max_rx_rings - bp->rx_nr_rings; - vf_tx_rings = hw_resc->max_tx_rings - bp->tx_nr_rings; + vf_tx_rings = hw_resc->max_tx_rings -bnxt_total_tx_rings (bp); vf_vnics = hw_resc->max_vnics - bp->nr_vnics; vf_rss = hw_resc->max_rsscos_ctxs - bp->rsscos_nr_ctxs;
@@ -903,8 +903,8 @@ static int bnxt_sriov_enable(struct bnxt *bp, int *num_vfs) avail_cp < min_rx_rings) rx_ok =0 ; -if (hw_resc->max_tx_rings - bp->tx_nr_rings >= min_tx_rings && - avail_cp >= min_tx_rings) +if (hw_resc->max_tx_rings -bnxt_total_tx_rings (bp) >= + min_tx_rings && avail_cp >= min_tx_rings) tx_ok =1 ; if (hw_resc->max_rsscos_ctxs - bp->rsscos_nr_ctxs >=
Signature · 1 line
2.51.0
Michael Chan
maintainerPATCH v2 3/15 bnxt_en: Set default MPC ring count
In reply to Michael ChanIf the firmware supports MPC channels and CONFIG_BNXT_TLS is set, set the default number of MPC channels. These MPC rings will share MSIX with the TX rings. The number of MPC channels for each type must not exceed the ethtool TX channel count. bnxt_set_dflt_mpc_rings() will determine the count for each MPC channel type and it cannot be directly controlled by the user.
We also add bnxt_trim_mpc_rings() to make final adjustments in case the number of reserved TX channels is less than expected.
--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 7 +++ .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 3 + drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c | 62 +++++++++++++++++++ drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h | 15 +++++ 4 files changed, 87 insertions(+)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +7 -0
@@ -13147,6 +13147,7 @@ static int __bnxt_open_nic(struct bnxt *bp, bool irq_re_init, bool link_re_init)return rc;bnxt_adj_tx_rings (bp); +bnxt_trim_mpc_rings (bp); rc =bnxt_alloc_mem (bp, irq_re_init);if (rc) {netdev_err (bp->dev,"bnxt_alloc_mem err: %x\n" , rc);
@@ -16681,6 +16682,7 @@ static void bnxt_trim_dflt_sh_rings(struct bnxt *bp) bp->rx_nr_rings = bp->cp_nr_rings; bp->tx_nr_rings_per_tc = bp->cp_nr_rings; bp->tx_nr_rings =bnxt_tx_nr_rings (bp); +bnxt_trim_mpc_rings (bp); }static void bnxt_adj_dflt_rings (struct bnxt *bp,bool sh)
@@ -16732,6 +16734,8 @@ static int bnxt_set_dflt_rings(struct bnxt *bp, bool sh)bnxt_set_dflt_ulp_stat_ctxs (bp); } +bnxt_set_dflt_mpc_rings (bp); + rc =__bnxt_reserve_rings (bp);if (rc && rc != -ENODEV )netdev_warn (bp->dev,"Unable to reserve tx rings\n" );
@@ -16746,6 +16750,7 @@ static int bnxt_set_dflt_rings(struct bnxt *bp, bool sh)if (rc && rc != -ENODEV )netdev_warn (bp->dev,"2nd rings reservation failed.\n" );bnxt_adj_tx_rings (bp); +bnxt_trim_mpc_rings (bp); }if (BNXT_CHIP_TYPE_NITRO_A0 (bp)) { bp->rx_nr_rings++;
@@ -16780,6 +16785,7 @@ static int bnxt_init_dflt_ring_mode(struct bnxt *bp) goto init_dflt_ring_err;bnxt_adj_tx_rings (bp); +bnxt_trim_mpc_rings (bp);bnxt_set_dflt_rfs (bp);
@@ -17123,6 +17129,7 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)* limited MSIX, so we re-initialize the TX rings per TC. */ bp->tx_nr_rings_per_tc = bp->tx_nr_rings; +bnxt_trim_mpc_rings (bp);if (BNXT_PF (bp)) {if (!bnxt_pf_wq) {
drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c +3 -0
@@ -37,6 +37,7 @@#include "bnxt_nvm_defs.h" /* NVRAM content constant and structure defs */ #include "bnxt_fw_hdr.h" /* Firmware hdr constant and structure defs */ #include "bnxt_coredump.h" +#include "bnxt_mpc.h" #define BNXT_NVM_ERR_MSG (dev, extack, msg) \ do { \
@@ -1050,6 +1051,8 @@ static int bnxt_set_channels(struct net_device *dev,bnxt_set_cp_rings (bp, sh); +bnxt_set_dflt_mpc_rings (bp); +/* After changing number of rx channels, update NTUPLE feature. */ netdev_update_features (dev);if (netif_running (dev)) {
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +62 -0
71 lines not on this page · raw message
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h +15 -0
@@ -17,6 +17,11 @@ enum bnxt_mpc_type {#define BNXT_MAX_MPC 8 +#define BNXT_MIN_MPC_TCE 1 +#define BNXT_MIN_MPC_RCE 1 +#define BNXT_DFLT_MPC_TCE BNXT_MAX_MPC +#define BNXT_DFLT_MPC_RCE BNXT_MAX_MPC +struct bnxt_mpc_info {u8 mpc_chnls_cap;u8 mpc_cp_rings;
@@ -37,6 +42,8 @@ void bnxt_alloc_mpc_info(struct bnxt *bp, u8 mpc_chnls_cap);void bnxt_free_mpc_info (struct bnxt *bp);int bnxt_mpc_tx_rings_in_use (struct bnxt *bp);int bnxt_mpc_cp_rings_in_use (struct bnxt *bp); +void bnxt_trim_mpc_rings (struct bnxt *bp); +void bnxt_set_dflt_mpc_rings (struct bnxt *bp);#else static inline void bnxt_alloc_mpc_info (struct bnxt *bp,u8 mpc_chnls_cap) {
@@ -55,5 +62,13 @@ static inline int bnxt_mpc_cp_rings_in_use(struct bnxt *bp) {return 0 ; } + +static inline void bnxt_trim_mpc_rings (struct bnxt *bp) +{ +} + +static inline void bnxt_set_dflt_mpc_rings (struct bnxt *bp) +{ +}#endif /* CONFIG_BNXT_TLS */ #endif /* BNXT_MPC_H */
Signature · 1 line
2.51.0
Jakub Kicinski
maintainerRe: [PATCH net-next v2 03/15] bnxt_en: Set default MPC ring count
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---bnxt_en: Set default MPC ring count
This patch adds bnxt_set_dflt_mpc_rings() to compute a default number of MPC channels when the firmware supports MPC and CONFIG_BNXT_TLS is set, and bnxt_trim_mpc_rings() to cap the per-type counts to the current tx_nr_rings_per_tc.
Quoted 11 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +3
-0@@ -45,3 +46,64 @@ int bnxt_mpc_cp_rings_in_use(struct bnxt *bp)return 0 ;return mpc->mpc_cp_rings; } + +void bnxt_trim_mpc_rings (struct bnxt *bp) +{
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c int bnxt_mpc_cp_rings_in_use(struct bnxt *bp) · line 61
Quoted 10 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +10
-0+struct bnxt_mpc_info *mpc = bp->mpc_info; +int max = bp->tx_nr_rings_per_tc; +u8 max_cp =0 ; +int i; + +if (!mpc) +return ; + +for (i =0 ; i <BNXT_MPC_TYPE_MAX ; i++) { + mpc->mpc_ring_count[i] =min_t (u8, mpc->mpc_ring_count[i], max);
[Medium]
Can the u8 cast in this min_t() truncate the cap? bp->tx_nr_rings_per_tc is an int and max is declared as int here, but min_t(u8, ...) casts both operands to u8 before comparing.
For values of bp->tx_nr_rings_per_tc that are a multiple of 256, the truncated max becomes 0 and every mpc_ring_count[i] gets zeroed, which also makes mpc_cp_rings 0 and disables MPC even though plenty of TX rings are available. For values in the [257..263] range the truncated max is [1..7], which would cap the count below BNXT_MAX_MPC (8). The same pattern repeats every 256.
Modern P5/P7 deployments with high CPU counts and large user-selected TX channel counts via ethtool -L can land in those ranges.
Would using min_t(int, ...) be more appropriate here, given the result is bounded by BNXT_MAX_MPC == 8 and therefore still fits in u8?
Quoted 5 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +5
-0+ max_cp =max (max_cp, mpc->mpc_ring_count[i]); + } + mpc->mpc_cp_rings = max_cp; +} +
[ ... ]
Signature · 1 line
pw-bot: cr
Michael Chan
maintainerPATCH v2 4/15 bnxt_en: Rename xdp_tx_lock to tx_lock
In reply to Michael Chanxdp_tx_lock in struct bnxt_tx_ring_info is used to serialize XDP_REDIRECT on the same TX ring. MPCs will also need this lock for a similar purpose to serialize sending multiple messages on the same MPC, so rename it to tx_lock.
--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 2 +- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 4 ++-- drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c | 4 ++-- 3 files changed, 5 insertions(+), 5 deletions(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +1 -1
@@ -4148,7 +4148,7 @@ static int bnxt_alloc_tx_rings(struct bnxt *bp) } qidx = bp->tc_to_qidx[j]; ring->queue_id = bp->q_info[qidx].queue_id; -spin_lock_init (&txr->xdp_tx_lock); +spin_lock_init (&txr->tx_lock);if (i < bp->tx_nr_rings_xdp)continue ;if (BNXT_RING_TO_TC_OFF (bp, i) == (bp->tx_nr_rings_per_tc -1 ))
drivers/net/ethernet/broadcom/bnxt/bnxt.h +2 -2
@@ -1013,8 +1013,8 @@ struct bnxt_tx_ring_info {u32 dev_state;struct bnxt_ring_struct tx_ring_struct; -/* Synchronize simultaneous xdp_xmit on same ring */ -spinlock_t xdp_tx_lock; +/* Synchronize simultaneous xdp_xmit on same ring or for MPC ring */ +spinlock_t tx_lock; };#define BNXT_LEGACY_COAL_CMPL_PARAMS \
drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c +2 -2
@@ -351,7 +351,7 @@ int bnxt_xdp_xmit(struct net_device *dev, int num_frames,return -EINVAL ;if (static_branch_unlikely (&bnxt_xdp_locking_key)) -spin_lock (&txr->xdp_tx_lock); +spin_lock (&txr->tx_lock);for (i =0 ; i < num_frames; i++) {struct xdp_frame *xdp = frames[i];
@@ -376,7 +376,7 @@ int bnxt_xdp_xmit(struct net_device *dev, int num_frames, }if (static_branch_unlikely (&bnxt_xdp_locking_key)) -spin_unlock (&txr->xdp_tx_lock); +spin_unlock (&txr->tx_lock);return nxmit; }
Signature · 1 line
2.51.0
Michael Chan
maintainerPATCH v2 5/15 bnxt_en: Allocate and free MPC software structures
In reply to Michael ChanEach MPC consists of a special TX ring and a completion ring. Use existing structs bnxt_tx_ring_info and bnxt_cp_ring_info as control structures. The 2 MPC channels to TCE and RCE that share the MSIX will use a shared completion ring.
--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 35 ++- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 12 +- drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c | 205 ++++++++++++++++++ drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h | 46 ++++ 4 files changed, 291 insertions(+), 7 deletions(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +29 -6
@@ -3679,7 +3679,7 @@ static size_t __bnxt_copy_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem,return total_len; } -static void bnxt_free_ring (struct bnxt *bp,struct bnxt_ring_mem_info *rmem) +void bnxt_free_ring (struct bnxt *bp,struct bnxt_ring_mem_info *rmem) {struct pci_dev *pdev = bp->pdev;int i;
@@ -3712,7 +3712,7 @@ static void bnxt_free_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem) } } -static int bnxt_alloc_ring (struct bnxt *bp,struct bnxt_ring_mem_info *rmem) +int bnxt_alloc_ring (struct bnxt *bp,struct bnxt_ring_mem_info *rmem) {struct pci_dev *pdev = bp->pdev;u64 valid_bit =0 ;
@@ -4319,6 +4319,8 @@ static int bnxt_alloc_cp_rings(struct bnxt *bp) (!sh && i >= bp->rx_nr_rings)) { cp_count += tcs; tx =1 ; +if (bnxt_napi_has_mpc (bp, i)) + cp_count++; } cpr->cp_ring_arr =kzalloc_objs (*cpr, cp_count);
@@ -4340,6 +4342,11 @@ static int bnxt_alloc_cp_rings(struct bnxt *bp) }else {int n, tc = k - rx; +/* MPC rings are at the highest k indices */ +if (tc >= tcs) { +bnxt_set_mpc_cp_ring (bp, i, cpr2); +continue ; + } n =BNXT_TC_TO_RING_BASE (bp, tc) + j; bp->tx_ring[n].tx_cpr = cpr2; cpr2->cp_ring_type =BNXT_NQ_HDL_TYPE_TX ;
@@ -4472,6 +4479,7 @@ static void bnxt_init_ring_struct(struct bnxt *bp) rmem->vmem = (void **)&txr->tx_buf_ring; } } +bnxt_init_mpc_ring_struct (bp); }static void bnxt_init_rxbd_pages (struct bnxt_ring_struct *ring,u32 type)
@@ -5548,6 +5556,7 @@ static void bnxt_init_l2_fltr_tbl(struct bnxt *bp)static void bnxt_free_mem (struct bnxt *bp,bool irq_re_init) {bnxt_free_vnic_attributes (bp); +bnxt_free_mpc_rings (bp);bnxt_free_tx_rings (bp);bnxt_free_rx_rings (bp);bnxt_free_cp_rings (bp);
@@ -5561,6 +5570,7 @@ static void bnxt_free_mem(struct bnxt *bp, bool irq_re_init)bnxt_free_port_stats (bp);bnxt_free_ring_grps (bp);bnxt_free_vnics (bp); +bnxt_free_mpcs (bp);kfree (bp->tx_ring_map); bp->tx_ring_map =NULL ;kfree (bp->tx_ring);
@@ -5670,6 +5680,10 @@ static int bnxt_alloc_mem(struct bnxt *bp, bool irq_re_init) txr->tx_cpr = &bnapi2->cp_ring; } + rc =bnxt_alloc_mpcs (bp); +if (rc) + goto alloc_mem_err; + rc =bnxt_alloc_stats (bp);if (rc) goto alloc_mem_err;
@@ -5698,6 +5712,10 @@ static int bnxt_alloc_mem(struct bnxt *bp, bool irq_re_init)if (rc) goto alloc_mem_err; + rc =bnxt_alloc_mpc_rings (bp); +if (rc) + goto alloc_mem_err; + rc =bnxt_alloc_cp_rings (bp);if (rc) goto alloc_mem_err;
@@ -7252,10 +7270,15 @@ static int hwrm_ring_alloc_send_msg(struct bnxt *bp, req->cmpl_ring_id =cpu_to_le16 (bnxt_cp_ring_for_tx (bp, txr)); req->length =cpu_to_le32 (bp->tx_ring_mask +1 ); req->stat_ctx_id =cpu_to_le32 (grp_info->fw_stats_ctx); - req->queue_id =cpu_to_le16 (ring->queue_id); -if (bp->flags &BNXT_FLAG_TX_COAL_CMPL ) - req->cmpl_coal_cnt = -RING_ALLOC_REQ_CMPL_COAL_CNT_COAL_64 ; +if (ring->queue_id ==BNXT_MPC_QUEUE_ID ) { + req->mpc_chnls_type = ring->mpc_chnl_type; + req->enables |=cpu_to_le32 (RING_ALLOC_REQ_ENABLES_MPC_CHNLS_TYPE ); + }else { + req->queue_id =cpu_to_le16 (ring->queue_id); +if (bp->flags &BNXT_FLAG_TX_COAL_CMPL ) + req->cmpl_coal_cnt = +RING_ALLOC_REQ_CMPL_COAL_CNT_COAL_64 ; + }if ((bp->fw_cap &BNXT_FW_CAP_TX_TS_CMP ) && bp->ptp_cfg) flags |=RING_ALLOC_REQ_FLAGS_TX_PKT_TS_CMPL_ENABLE ; req->flags =cpu_to_le16 (flags);
drivers/net/ethernet/broadcom/bnxt/bnxt.h +11 -1
@@ -686,6 +686,7 @@ struct nqe_cn {#define BNXT_NQ_HDL_TYPE_SHIFT 24#define BNXT_NQ_HDL_TYPE_RX 0x00#define BNXT_NQ_HDL_TYPE_TX 0x01 +#define BNXT_NQ_HDL_TYPE_MP 0x02#define BNXT_NQ_HDL_IDX (hdl) ((hdl) & BNXT_NQ_HDL_IDX_MASK)#define BNXT_NQ_HDL_TYPE (hdl) (((hdl) & BNXT_NQ_HDL_TYPE_MASK) >> \
@@ -950,6 +951,8 @@ struct bnxt_ring_struct { };u32 handle;u8 queue_id; +#define BNXT_MPC_QUEUE_ID 0xff +u8 mpc_chnl_type; };struct tx_push_bd {
@@ -990,12 +993,16 @@ struct bnxt_tx_ring_info {u16 tx_cons;u16 tx_hw_cons;u16 txq_index; +/* index for tx_ring[] or tx_mpc_ring[] in struct bnxt_napi */ u8 tx_napi_idx;u8 kick_pending;struct bnxt_db_info tx_db;struct tx_bd *tx_desc_ring[MAX_TX_PAGES ]; -struct bnxt_sw_tx_bd *tx_buf_ring; +union { +struct bnxt_sw_tx_bd *tx_buf_ring; +struct bnxt_sw_mpc_tx_bd *tx_mpc_buf_ring; + };dma_addr_t tx_desc_mapping[MAX_TX_PAGES ];
@@ -1241,6 +1248,7 @@ struct bnxt_napi {struct bnxt_cp_ring_info cp_ring;struct bnxt_rx_ring_info *rx_ring;struct bnxt_tx_ring_info *tx_ring[BNXT_MAX_TXR_PER_NAPI ]; +struct bnxt_tx_ring_info **tx_mpc_ring;void (*tx_int)(struct bnxt *,struct bnxt_napi *,int budget);
@@ -2961,6 +2969,8 @@ int bnxt_alloc_rx_data(struct bnxt *bp, struct bnxt_rx_ring_info *rxr,void bnxt_reuse_rx_data (struct bnxt_rx_ring_info *rxr,u16 cons,void *data);u32 bnxt_fw_health_readl (struct bnxt *bp,int reg_idx);bool bnxt_bs_trace_avail (struct bnxt *bp,u16 type); +void bnxt_free_ring (struct bnxt *bp,struct bnxt_ring_mem_info *rmem); +int bnxt_alloc_ring (struct bnxt *bp,struct bnxt_ring_mem_info *rmem);void bnxt_set_tpa_flags (struct bnxt *bp);void bnxt_set_ring_params (struct bnxt *);void bnxt_set_rx_skb_mode (struct bnxt *bp,bool page_mode);
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +205 -0
220 lines not on this page · raw message
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h +46 -0
71 lines not on this page · raw message
Signature · 1 line
2.51.0
Michael Chan
maintainerPATCH v2 6/15 bnxt_en: Allocate and free MPC channels from firmware
In reply to Michael ChanAllocate and free the reserved MPC TX rings and completion rings from firmware. MPC backing store memory also needs to be configured in order to successfully allocate the MPC TX rings.
--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 30 ++++-- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 6 ++ drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c | 92 +++++++++++++++++++ drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h | 16 ++++ 4 files changed, 135 insertions(+), 9 deletions(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +21 -9
@@ -7473,8 +7473,7 @@ static int bnxt_hwrm_rx_agg_ring_alloc(struct bnxt *bp,return 0 ; } -static int bnxt_hwrm_cp_ring_alloc_p5 (struct bnxt *bp, -struct bnxt_cp_ring_info *cpr) +int bnxt_hwrm_cp_ring_alloc_p5 (struct bnxt *bp,struct bnxt_cp_ring_info *cpr) {const u32 type =HWRM_RING_ALLOC_CMPL ;struct bnxt_napi *bnapi = cpr->bnapi;
@@ -7492,8 +7491,8 @@ static int bnxt_hwrm_cp_ring_alloc_p5(struct bnxt *bp,return 0 ; } -static int bnxt_hwrm_tx_ring_alloc (struct bnxt *bp, -struct bnxt_tx_ring_info *txr,u32 tx_idx) +int bnxt_hwrm_tx_ring_alloc (struct bnxt *bp,struct bnxt_tx_ring_info *txr, +u32 tx_idx) {struct bnxt_ring_struct *ring = &txr->tx_ring_struct;const u32 type =HWRM_RING_ALLOC_TX ;
@@ -7578,6 +7577,9 @@ static int bnxt_hwrm_ring_alloc(struct bnxt *bp) goto err_out; } } + + rc =bnxt_hwrm_mpc_ring_alloc (bp); + err_out:return rc; }
@@ -7635,9 +7637,8 @@ static int hwrm_ring_free_send_msg(struct bnxt *bp,return 0 ; } -static void bnxt_hwrm_tx_ring_free (struct bnxt *bp, -struct bnxt_tx_ring_info *txr, -bool close_path) +void bnxt_hwrm_tx_ring_free (struct bnxt *bp,struct bnxt_tx_ring_info *txr, +bool close_path) {struct bnxt_ring_struct *ring = &txr->tx_ring_struct;u32 cmpl_ring_id;
@@ -7696,8 +7697,7 @@ static void bnxt_hwrm_rx_agg_ring_free(struct bnxt *bp, bp->grp_info[grp_idx].agg_fw_ring_id =INVALID_HW_RING_ID ; } -static void bnxt_hwrm_cp_ring_free (struct bnxt *bp, -struct bnxt_cp_ring_info *cpr) +void bnxt_hwrm_cp_ring_free (struct bnxt *bp,struct bnxt_cp_ring_info *cpr) {struct bnxt_ring_struct *ring;
@@ -7731,6 +7731,8 @@ static void bnxt_hwrm_ring_free(struct bnxt *bp, bool close_path)if (!bp->bnapi)return ; +bnxt_hwrm_mpc_ring_free (bp, close_path); +for (i =0 ; i < bp->tx_nr_rings; i++)bnxt_hwrm_tx_ring_free (bp, &bp->tx_ring[i], close_path);
@@ -9368,12 +9370,21 @@ static int bnxt_hwrm_func_backing_store_cfg_v2(struct bnxt *bp,static int bnxt_backing_store_cfg_v2 (struct bnxt *bp) { +struct bnxt_mpc_info *mpc = bp->mpc_info;struct bnxt_ctx_mem_info *ctx = bp->ctx;struct bnxt_ctx_mem_type *ctxm;u16 last_type =BNXT_CTX_INV ;int rc =0 ;u16 type; +if (mpc && mpc->mpc_chnls_cap) { + ctxm = &ctx->ctx_arr[BNXT_CTX_MTQM ]; + rc =bnxt_setup_ctxm_pg_tbls (bp, ctxm, ctxm->max_entries,1 ); +if (rc) +return rc; + last_type =BNXT_CTX_MTQM ; + } +for (type =BNXT_CTX_SRT ; type <=BNXT_CTX_QPC ; type++) { ctxm = &ctx->ctx_arr[type];if (!bnxt_bs_trace_avail (bp, type))
@@ -11320,6 +11331,7 @@ static int bnxt_init_nic(struct bnxt *bp, bool irq_re_init)bnxt_init_cp_rings (bp);bnxt_init_rx_rings (bp);bnxt_init_tx_rings (bp); +bnxt_init_mpc_rings (bp);bnxt_init_ring_grps (bp, irq_re_init);bnxt_init_vnics (bp);
drivers/net/ethernet/broadcom/bnxt/bnxt.h +6 -0
@@ -2999,6 +2999,12 @@ int bnxt_hwrm_vnic_cfg(struct bnxt *bp, struct bnxt_vnic_info *vnic);int bnxt_hwrm_vnic_alloc (struct bnxt *bp,struct bnxt_vnic_info *vnic,unsigned int start_rx_ring_idx,unsigned int nr_rings); +int bnxt_hwrm_cp_ring_alloc_p5 (struct bnxt *bp,struct bnxt_cp_ring_info *cpr); +int bnxt_hwrm_tx_ring_alloc (struct bnxt *bp,struct bnxt_tx_ring_info *txr, +u32 tx_idx); +void bnxt_hwrm_tx_ring_free (struct bnxt *bp,struct bnxt_tx_ring_info *txr, +bool close_path); +void bnxt_hwrm_cp_ring_free (struct bnxt *bp,struct bnxt_cp_ring_info *cpr);int bnxt_total_tx_rings (struct bnxt *bp);int __bnxt_hwrm_get_tx_rings (struct bnxt *bp,u16 fid,int *tx_rings);int bnxt_nq_rings_in_use (struct bnxt *bp);
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +92 -0
95 lines not on this page · raw message
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h +16 -0
@@ -58,6 +58,9 @@ int bnxt_alloc_mpcs(struct bnxt *bp);void bnxt_free_mpcs (struct bnxt *bp);int bnxt_alloc_mpc_rings (struct bnxt *bp);void bnxt_free_mpc_rings (struct bnxt *bp); +void bnxt_init_mpc_rings (struct bnxt *bp); +int bnxt_hwrm_mpc_ring_alloc (struct bnxt *bp); +void bnxt_hwrm_mpc_ring_free (struct bnxt *bp,bool close_path);#else static inline void bnxt_alloc_mpc_info (struct bnxt *bp,u8 mpc_chnls_cap) {
@@ -116,5 +119,18 @@ static inline int bnxt_alloc_mpc_rings(struct bnxt *bp)static inline void bnxt_free_mpc_rings (struct bnxt *bp) { } + +static inline void bnxt_init_mpc_rings (struct bnxt *bp) +{ +} + +static inline int bnxt_hwrm_mpc_ring_alloc (struct bnxt *bp) +{ +return 0 ; +} + +static inline void bnxt_hwrm_mpc_ring_free (struct bnxt *bp,bool close_path) +{ +}#endif /* CONFIG_BNXT_TLS */ #endif /* BNXT_MPC_H */
Signature · 1 line
2.51.0
Michael Chan
maintainerPATCH v2 7/15 bnxt_en: Allocate crypto structure and backing store
In reply to Michael ChanIf the chip and firmware support crypto (TLS) offload, allocate a bp->crypto_info software structure and backing store to support the RX and TX contexts. Each offloaded TLS connection requires a backing store context for each direction.
--- drivers/net/ethernet/broadcom/bnxt/Makefile | 2 +- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 21 +++++ drivers/net/ethernet/broadcom/bnxt/bnxt.h | 1 + .../net/ethernet/broadcom/bnxt/bnxt_crypto.c | 78 +++++++++++++++++++ .../net/ethernet/broadcom/bnxt/bnxt_crypto.h | 47 +++++++++++ include/linux/bnxt/hsi.h | 37 +++++++++ 6 files changed, 185 insertions(+), 1 deletion(-) create mode 100644 drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c create mode 100644 drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.h
drivers/net/ethernet/broadcom/bnxt/Makefile +1 -1
@@ -5,4 +5,4 @@ bnxt_en-y := bnxt.o bnxt_hwrm.o bnxt_sriov.o bnxt_ethtool.o bnxt_dcb.o bnxt_ulp. bnxt_en-$(CONFIG_BNXT_FLOWER_OFFLOAD) += bnxt_tc.o bnxt_en-$(CONFIG_DEBUG_FS) += bnxt_debugfs.o bnxt_en-$(CONFIG_BNXT_HWMON) += bnxt_hwmon.o -bnxt_en-$(CONFIG_BNXT_TLS) += bnxt_mpc.o +bnxt_en-$(CONFIG_BNXT_TLS) += bnxt_mpc.o bnxt_crypto.o
drivers/net/ethernet/broadcom/bnxt/bnxt.c +21 -0
@@ -77,6 +77,7 @@#include "bnxt_gso.h" #include <net/tso.h> #include "bnxt_mpc.h" +#include "bnxt_crypto.h" #define BNXT_TX_TIMEOUT (5 * HZ)#define BNXT_DEF_MSG_ENABLE (NETIF_MSG_DRV | NETIF_MSG_HW | \
@@ -9370,6 +9371,7 @@ static int bnxt_hwrm_func_backing_store_cfg_v2(struct bnxt *bp,static int bnxt_backing_store_cfg_v2 (struct bnxt *bp) { +struct bnxt_crypto_info *crypto = bp->crypto_info;struct bnxt_mpc_info *mpc = bp->mpc_info;struct bnxt_ctx_mem_info *ctx = bp->ctx;struct bnxt_ctx_mem_type *ctxm;
@@ -9377,6 +9379,19 @@ static int bnxt_backing_store_cfg_v2(struct bnxt *bp)int rc =0 ;u16 type; +if (crypto) { + ctxm = &ctx->ctx_arr[BNXT_CTX_TCK ]; + rc =bnxt_setup_ctxm_pg_tbls (bp, ctxm, +BNXT_TCK (crypto).max_ctx,1 ); +if (rc) +return rc; + ctxm = &ctx->ctx_arr[BNXT_CTX_RCK ]; + rc =bnxt_setup_ctxm_pg_tbls (bp, ctxm, +BNXT_RCK (crypto).max_ctx,1 ); +if (rc) +return rc; + last_type =BNXT_CTX_RCK ; + }if (mpc && mpc->mpc_chnls_cap) { ctxm = &ctx->ctx_arr[BNXT_CTX_MTQM ]; rc =bnxt_setup_ctxm_pg_tbls (bp, ctxm, ctxm->max_entries,1 );
@@ -9919,6 +9934,10 @@ static int __bnxt_hwrm_func_qcaps(struct bnxt *bp) bp->fw_cap |=BNXT_FW_CAP_BACKING_STORE_V2 ;if (flags_ext &FUNC_QCAPS_RESP_FLAGS_EXT_TX_COAL_CMPL_CAP ) bp->flags |=BNXT_FLAG_TX_COAL_CMPL ; +if (flags_ext &FUNC_QCAPS_RESP_FLAGS_EXT_KTLS_SUPPORTED ) +bnxt_alloc_crypto_info (bp, resp); +else +bnxt_free_crypto_info (bp); flags_ext2 =le32_to_cpu (resp->flags_ext2);if (flags_ext2 &FUNC_QCAPS_RESP_FLAGS_EXT2_RX_ALL_PKTS_TIMESTAMPS_SUPPORTED )
@@ -16549,6 +16568,7 @@ static void bnxt_remove_one(struct pci_dev *pdev) bp->ptp_cfg =NULL ;kfree (bp->fw_health); bp->fw_health =NULL ; +bnxt_free_crypto_info (bp);bnxt_free_mpc_info (bp);bnxt_cleanup_pci (bp);bnxt_free_ctx_mem (bp, true);
@@ -17228,6 +17248,7 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)bnxt_ethtool_free (bp);kfree (bp->fw_health); bp->fw_health =NULL ; +bnxt_free_crypto_info (bp);bnxt_free_mpc_info (bp);bnxt_cleanup_pci (bp);bnxt_free_ctx_mem (bp, true);
drivers/net/ethernet/broadcom/bnxt/bnxt.h +1 -0
@@ -2460,6 +2460,7 @@ struct bnxt {u8 tph_mode;struct bnxt_mpc_info *mpc_info; +struct bnxt_crypto_info *crypto_info;unsigned int current_interval;#define BNXT_TIMER_INTERVAL HZ
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c new file +78 -0
78 lines not on this page · raw message
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.h new file +47 -0
47 lines not on this page · raw message
include/linux/bnxt/hsi.h +37 -0
@@ -3837,6 +3837,43 @@ struct hwrm_func_ptp_ext_qcfg_output {u8 valid; }; +/* hwrm_func_key_ctx_alloc_input (size:384b/48B) */ +struct hwrm_func_key_ctx_alloc_input { +__le16 req_type; +__le16 cmpl_ring; +__le16 seq_id; +__le16 target_id; +__le64 resp_addr; +__le16 fid; +__le16 num_key_ctxs; +__le32 dma_bufr_size_bytes; +u8 key_ctx_type; +#define FUNC_KEY_CTX_ALLOC_REQ_KEY_CTX_TYPE_TX 0x0UL +#define FUNC_KEY_CTX_ALLOC_REQ_KEY_CTX_TYPE_RX 0x1UL +#define FUNC_KEY_CTX_ALLOC_REQ_KEY_CTX_TYPE_QUIC_TX 0x2UL +#define FUNC_KEY_CTX_ALLOC_REQ_KEY_CTX_TYPE_QUIC_RX 0x3UL +#define FUNC_KEY_CTX_ALLOC_REQ_KEY_CTX_TYPE_LAST FUNC_KEY_CTX_ALLOC_REQ_KEY_CTX_TYPE_QUIC_RX +u8 unused_0[7 ]; +__le64 host_dma_addr; +__le32 partition_start_xid; +u8 unused_1[4 ]; +}; + +/* hwrm_func_key_ctx_alloc_output (size:192b/24B) */ +struct hwrm_func_key_ctx_alloc_output { +__le16 error_code; +__le16 req_type; +__le16 seq_id; +__le16 resp_len; +__le16 num_key_ctxs_allocated; +u8 flags; +#define FUNC_KEY_CTX_ALLOC_RESP_FLAGS_KEY_CTXS_CONTIGUOUS 0x1UL +u8 unused_0; +__le32 partition_start_xid; +u8 unused_1[7 ]; +u8 valid; +}; +/* hwrm_func_backing_store_cfg_v2_input (size:512b/64B) */ struct hwrm_func_backing_store_cfg_v2_input {__le16 req_type;
Signature · 1 line
2.51.0
Jakub Kicinski
maintainerRe: [PATCH net-next v2 07/15] bnxt_en: Allocate crypto structure and backing store
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---bnxt_en: Allocate crypto structure and backing store
If the chip and firmware support crypto (TLS) offload, allocate a bp->crypto_info software structure and backing store to support the RX and TX contexts. Each offloaded TLS connection requires a backing store context for each direction.
Quoted 5 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c new file +0
-0
[ ... ]
Quoted 11 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c new file +11
-0+static u32 bnxt_get_max_crypto_key_ctx (struct bnxt *bp,int key_type) +{ +u32 fw_maj =BNXT_FW_MAJ (bp); + +if (key_type ==BNXT_TX_CRYPTO_KEY_TYPE ) +return (fw_maj <233 ) ?BNXT_MAX_TX_CRYPTO_KEYS_PRE_233FW : +BNXT_MAX_TX_CRYPTO_KEYS ; + +return (fw_maj <233 ) ?BNXT_MAX_RX_CRYPTO_KEYS_PRE_233FW : +BNXT_MAX_RX_CRYPTO_KEYS ; +}
[ ... ]
Quoted 17 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c new file +17
-0+void bnxt_alloc_crypto_info (struct bnxt *bp, +struct hwrm_func_qcaps_output *resp) +{ +u16 max_keys =le16_to_cpu (resp->max_key_ctxs_alloc); +struct bnxt_crypto_info *crypto = bp->crypto_info; + +if (BNXT_VF (bp)) +return ; +if (!crypto) { +struct bnxt_kctx *kctx; +int i; + + crypto =kzalloc_obj (*crypto); +if (!crypto) { +netdev_warn (bp->dev, +"Unable to allocate crypto info\n" ); +return ;
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c line 64
Quoted 10 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c new file +10
-0+ } +for (i =0 ; i <BNXT_MAX_CRYPTO_KEY_TYPE ; i++) { + kctx = &crypto->kctx[i]; + kctx->type = i; + kctx->max_ctx =bnxt_get_max_crypto_key_ctx (bp, i); + } + bp->crypto_info = crypto; + } + crypto->max_key_ctxs_alloc = max_keys; +}
[Medium]
Should the kctx[].max_ctx values be refreshed on every call, rather than only when bp->crypto_info is first allocated?
bnxt_alloc_crypto_info() can be invoked multiple times for a given bp. __bnxt_hwrm_func_qcaps() is the caller, and it runs from paths such as bnxt_resume(), bnxt_restore_pf_fw_resources(), bnxt_io_resume(), and during SR-IOV enable, in addition to initial probe.
On the second and later calls, the crypto pointer is non-NULL, so the loop that calls bnxt_get_max_crypto_key_ctx(bp, i) is skipped and only crypto->max_key_ctxs_alloc is refreshed.
If BNXT_FW_MAJ(bp) crosses the 233 boundary between two qcaps queries on the same driver instance (for example after a firmware update applied across suspend/resume, or after a firmware reset that re-reads fw_ver_code), the cached kctx[].max_ctx retains the pre-233 value of 65535 instead of being recomputed to 204800.
bnxt_backing_store_cfg_v2() then sizes the TCK/RCK backing stores from those cached values:
if (crypto) {
ctxm = &ctx->ctx_arr[BNXT_CTX_TCK];
rc = bnxt_setup_ctxm_pg_tbls(bp, ctxm,
BNXT_TCK(crypto).max_ctx, 1);
...
ctxm = &ctx->ctx_arr[BNXT_CTX_RCK];
rc = bnxt_setup_ctxm_pg_tbls(bp, ctxm,
BNXT_RCK(crypto).max_ctx, 1);and bnxt_setup_ctxm_pg_tbls() applies clamp_t(u32, entries, ctxm->min_entries, ctxm->max_entries), so the stale 65535 caps the allocation even if firmware now advertises a larger ctxm->max_entries.
Would moving the kctx[] initialization out of the if (!crypto) block, or running it unconditionally, be a better fit for the function name and kerneldoc which describe it as 'Allocate and initialize crypto offload context' without noting the once-only initialization?
Michael Chan
maintainerPATCH v2 8/15 bnxt_en: Reserve crypto RX and TX key contexts on a PF
In reply to Michael ChanIf kTLS crypto offload is supported, reserve RX and TX key contexts. These keys will later be allocated during run-time to support offloading TX and RX kTLS connections.
--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 15 ++++++++- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 11 +++++++ .../net/ethernet/broadcom/bnxt/bnxt_crypto.c | 31 +++++++++++++++++++ .../net/ethernet/broadcom/bnxt/bnxt_crypto.h | 7 +++++ 4 files changed, 63 insertions(+), 1 deletion(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +14 -1
@@ -7785,6 +7785,7 @@ static int bnxt_trim_rings(struct bnxt *bp, int *rx, int *tx, int max,static int bnxt_hwrm_get_rings (struct bnxt *bp) {struct bnxt_hw_resc *hw_resc = &bp->hw_resc; +struct bnxt_hw_crypto_resc *crypto_resc;struct hwrm_func_qcfg_output *resp;struct hwrm_func_qcfg_input *req;int rc;
@@ -7845,6 +7846,10 @@ static int bnxt_hwrm_get_rings(struct bnxt *bp) } hw_resc->resv_cp_rings = cp; hw_resc->resv_stat_ctxs = stats; + + crypto_resc = &hw_resc->crypto_resc; + crypto_resc->resv_tx_key_ctxs =le32_to_cpu (resp->num_ktls_tx_key_ctxs); + crypto_resc->resv_rx_key_ctxs =le32_to_cpu (resp->num_ktls_rx_key_ctxs); } get_rings_exit:hwrm_req_drop (bp, req);
@@ -7915,8 +7920,9 @@ __bnxt_hwrm_reserve_pf_rings(struct bnxt *bp, struct bnxt_hw_rings *hwr) } req->num_stat_ctxs =cpu_to_le16 (hwr->stat); req->num_vnics =cpu_to_le16 (hwr->vnic); +bnxt_hwrm_reserve_pf_key_ctxs (bp, req); } - req->enables =cpu_to_le32 (enables); + req->enables |=cpu_to_le32 (enables);return req; }
@@ -9748,6 +9754,7 @@ int bnxt_hwrm_func_resc_qcaps(struct bnxt *bp, bool all)struct hwrm_func_resource_qcaps_output *resp;struct hwrm_func_resource_qcaps_input *req;struct bnxt_hw_resc *hw_resc = &bp->hw_resc; +struct bnxt_hw_crypto_resc *crypto_resc;int rc; rc =hwrm_req_init (bp, req,HWRM_FUNC_RESOURCE_QCAPS );
@@ -9785,6 +9792,12 @@ int bnxt_hwrm_func_resc_qcaps(struct bnxt *bp, bool all) hw_resc->max_vnics * BNXT_LARGE_RSS_TO_VNIC_RATIO) bp->rss_cap |= BNXT_RSS_CAP_LARGE_RSS_CTX; + crypto_resc = &hw_resc->crypto_resc; + crypto_resc->min_tx_key_ctxs =le32_to_cpu (resp->min_ktls_tx_key_ctxs); + crypto_resc->max_tx_key_ctxs =le32_to_cpu (resp->max_ktls_tx_key_ctxs); + crypto_resc->min_rx_key_ctxs =le32_to_cpu (resp->min_ktls_rx_key_ctxs); + crypto_resc->max_rx_key_ctxs =le32_to_cpu (resp->max_ktls_rx_key_ctxs); +if (bp->flags &BNXT_FLAG_CHIP_P5_PLUS ) {u16 max_msix =le16_to_cpu (resp->max_msix);
drivers/net/ethernet/broadcom/bnxt/bnxt.h +11 -0
@@ -1362,6 +1362,15 @@ struct bnxt_hw_rings {int rss_ctx; }; +struct bnxt_hw_crypto_resc { +u32 min_tx_key_ctxs; +u32 max_tx_key_ctxs; +u32 resv_tx_key_ctxs; +u32 min_rx_key_ctxs; +u32 max_rx_key_ctxs; +u32 resv_rx_key_ctxs; +}; +struct bnxt_hw_resc {u16 min_rsscos_ctxs;u16 max_rsscos_ctxs;
@@ -1396,6 +1405,8 @@ struct bnxt_hw_resc {u32 max_tx_wm_flows;u32 max_rx_em_flows;u32 max_rx_wm_flows; + +struct bnxt_hw_crypto_resc crypto_resc; };#define BNXT_LARGE_RSS_TO_VNIC_RATIO 7
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +31 -0
@@ -76,3 +76,34 @@ void bnxt_free_crypto_info(struct bnxt *bp)kfree (bp->crypto_info); bp->crypto_info =NULL ; } + +/** +* bnxt_hwrm_reserve_pf_key_ctxs - Reserve key contexts with firmware +* @bp: pointer to bnxt device +* @req: pointer to HWRM function config request +* +* Populates the firmware request with key context reservation parameters +* for crypto offload. Calculates the minimum of driver requirements and +* firmware capabilities. +* +* Context: Process context during device configuration +*/ +void bnxt_hwrm_reserve_pf_key_ctxs (struct bnxt *bp, +struct hwrm_func_cfg_input *req) +{ +struct bnxt_crypto_info *crypto = bp->crypto_info; +struct bnxt_hw_resc *hw_resc = &bp->hw_resc; +struct bnxt_hw_crypto_resc *crypto_resc; +u32 tx, rx; + +if (!crypto) +return ; + + crypto_resc = &hw_resc->crypto_resc; + tx =min (BNXT_TCK (crypto).max_ctx, crypto_resc->max_tx_key_ctxs); + rx =min (BNXT_RCK (crypto).max_ctx, crypto_resc->max_rx_key_ctxs); + req->num_ktls_tx_key_ctxs =cpu_to_le32 (tx); + req->num_ktls_rx_key_ctxs =cpu_to_le32 (rx); + req->enables |=cpu_to_le32 (FUNC_CFG_REQ_ENABLES_KTLS_TX_KEY_CTXS | +FUNC_CFG_REQ_ENABLES_KTLS_RX_KEY_CTXS ); +}
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.h +7 -0
@@ -34,6 +34,8 @@ struct bnxt_crypto_info {void bnxt_alloc_crypto_info (struct bnxt *bp,struct hwrm_func_qcaps_output *resp);void bnxt_free_crypto_info (struct bnxt *bp); +void bnxt_hwrm_reserve_pf_key_ctxs (struct bnxt *bp, +struct hwrm_func_cfg_input *req);#else static inline void bnxt_alloc_crypto_info (struct bnxt *bp,struct hwrm_func_qcaps_output *resp)
@@ -43,5 +45,10 @@ static inline void bnxt_alloc_crypto_info(struct bnxt *bp,static inline void bnxt_free_crypto_info (struct bnxt *bp) { } + +static inline void bnxt_hwrm_reserve_pf_key_ctxs (struct bnxt *bp, +struct hwrm_func_cfg_input *req) +{ +}#endif /* CONFIG_BNXT_TLS */ #endif /* BNXT_CRYPTO_H */
Signature · 1 line
2.51.0
Jakub Kicinski
maintainerRe: [PATCH net-next v2 08/15] bnxt_en: Reserve crypto RX and TX key contexts on a PF
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---bnxt_en: Reserve crypto RX and TX key contexts on a PF
This patch teaches the bnxt_en PF reservation path to populate num_ktls_tx_key_ctxs / num_ktls_rx_key_ctxs in the HWRM_FUNC_CFG request and to record the reserved counts returned by HWRM_FUNC_QCFG, so kTLS keys can later be allocated at run-time.
Quoted 4 lines
drivers/net/ethernet/broadcom/bnxt/bnxt.c +0
-0
[ ... ]
drivers/net/ethernet/broadcom/bnxt/bnxt.c __bnxt_hwrm_reserve_pf_rings(struct bnxt *bp, struct bnxt_hw_rings *hwr) · line 7927
Quoted 10 lines
drivers/net/ethernet/broadcom/bnxt/bnxt.c +2
-1@@ -7915,8 +7920,9 @@ __bnxt_hwrm_reserve_pf_rings(struct bnxt *bp, struct bnxt_hw_rings *hwr) } req->num_stat_ctxs =cpu_to_le16 (hwr->stat); req->num_vnics =cpu_to_le16 (hwr->vnic); +bnxt_hwrm_reserve_pf_key_ctxs (bp, req); } - req->enables =cpu_to_le32 (enables); + req->enables |=cpu_to_le32 (enables);return req; }
[Medium]
After this change, every caller of __bnxt_hwrm_reserve_pf_rings() that takes the BNXT_NEW_RM(bp) path picks up a kTLS key reservation. One of those callers is bnxt_hwrm_check_pf_rings(), which is the feasibility check used by bnxt_check_rings() (ethtool -L, ethtool -G, channel resize, etc.):
static int bnxt_hwrm_check_pf_rings(struct bnxt *bp,
struct bnxt_hw_rings *hwr)
{
struct hwrm_func_cfg_input *req;
u32 flags;
req = __bnxt_hwrm_reserve_pf_rings(bp, hwr);
flags = FUNC_CFG_REQ_FLAGS_TX_ASSETS_TEST;
if (BNXT_NEW_RM(bp)) {
flags |= FUNC_CFG_REQ_FLAGS_RX_ASSETS_TEST |
FUNC_CFG_REQ_FLAGS_CMPL_ASSETS_TEST |
FUNC_CFG_REQ_FLAGS_STAT_CTX_ASSETS_TEST |
FUNC_CFG_REQ_FLAGS_VNIC_ASSETS_TEST;
...
}
req->flags = cpu_to_le32(flags);
return hwrm_req_send_silent(bp, req);
}This sets the *_ASSETS_TEST bits in req->flags, but it never sets FUNC_CFG_REQ_FLAGS2_KTLS_KEY_CTX_ASSETS_TEST in req->flags2 (a grep of drivers/net/ethernet/broadcom/bnxt confirms that flag is not set anywhere in the driver).
Does that mean firmware will treat the tx/rx/cmpl/stat/vnic resources as test-only while still committing the kTLS key reservation as a side effect of every bnxt_check_rings() call?
If so, would this also cause an ethtool ring/channel resize to be rejected on a chip whose kTLS key pool is currently exhausted, even though the user only intended to probe the ring resources?
Should bnxt_hwrm_check_pf_rings() either skip bnxt_hwrm_reserve_pf_key_ctxs(), or set FUNC_CFG_REQ_FLAGS2_KTLS_KEY_CTX_ASSETS_TEST in req->flags2 so the kTLS portion of the request is also evaluated as test-only?
[Low]
On a kTLS-capable PF (bp->crypto_info != NULL), bnxt_hwrm_reserve_pf_key_ctxs() unconditionally sets FUNC_CFG_REQ_ENABLES_KTLS_TX_KEY_CTXS | FUNC_CFG_REQ_ENABLES_KTLS_RX_KEY_CTXS in req->enables, and the change from "=" to "|=" on the final assignment preserves them.
That makes the early-return short-circuit in bnxt_hwrm_reserve_pf_rings() unreachable on such PFs:
req = __bnxt_hwrm_reserve_pf_rings(bp, hwr);
if (!req)
return -ENOMEM;
if (!req->enables) {
hwrm_req_drop(bp, req);
return 0;
}
rc = hwrm_req_send(bp, req);
...
return bnxt_hwrm_get_rings(bp);Is it intended that every previously-no-op call now performs an HWRM round-trip plus a follow-up bnxt_hwrm_get_rings() on kTLS-capable PFs?
If not, would it be appropriate to also gate the kTLS enables/values on whether any of the original "enables" bits were going to be set, so the no-op path is preserved?
[ ... ]
Michael Chan
maintainerPATCH v2 9/15 bnxt_en: Add infrastructure for crypto key context IDs
In reply to Michael ChanEach kTLS connection requires a crypto key context ID (KID). These KIDs are allocated from the firmware in batches. Add data structure to store these IDs. The bnxt_kid_info structure stores a batch of IDs and it can be linked as we allocate more batches. There is a bitmap in the structure to keep track of which ones are in use. Add APIs to allocate and free these KIDs.
--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 3 + .../net/ethernet/broadcom/bnxt/bnxt_crypto.c | 257 ++++++++++++++++++ .../net/ethernet/broadcom/bnxt/bnxt_crypto.h | 66 +++++ 3 files changed, 326 insertions(+)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +3 -0
@@ -17219,6 +17219,9 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) rc =bnxt_dl_register (bp);if (rc) goto init_err_dl; + rc =bnxt_crypto_init (bp); +if (rc) +bnxt_free_crypto_info (bp);INIT_LIST_HEAD (&bp->usr_fltr_list);
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +257 -0
284 lines not on this page · raw message
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.h +66 -0
100 lines not on this page · raw message
Signature · 1 line
2.51.0
Jakub Kicinski
maintainerRe: [PATCH net-next v2 09/15] bnxt_en: Add infrastructure for crypto key context IDs
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
Subject: bnxt_en: Add infrastructure for crypto key context IDs
This patch adds a kid_info data structure plus APIs to allocate and free crypto key context IDs (KIDs) for kTLS in the bnxt_en driver. Batches of KIDs are obtained from the firmware and tracked in an RCU-walked linked list with a per-batch bitmap.
Reviewing the changes raised a few questions, inline below.
drivers/net/ethernet/broadcom/bnxt/bnxt.c static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) · line 17224
Quoted 11 lines
drivers/net/ethernet/broadcom/bnxt/bnxt.c +3
-0@@ -17219,6 +17219,9 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) rc =bnxt_dl_register (bp);if (rc) goto init_err_dl; + rc =bnxt_crypto_init (bp); +if (rc) +bnxt_free_crypto_info (bp);
[Low]
The other init steps in this function uniformly do an "if (rc) goto init_err_*;" with at least a netdev_warn() logged.
Here, on failure rc is silently overwritten by the next assignment (rc = register_netdev(dev) further down), so a transient HWRM error or DMA-slice ENOMEM during probe ends up leaving the device up with kTLS permanently disabled and no message in dmesg.
Additionally, if the TX HWRM_FUNC_KEY_CTX_ALLOC succeeded but the RX one failed, the in-file comment
/* Note that the driver does not free the key contexts. They are freed
* by the FW during FLR and HWRM_FUNC_RESET.
*/
implies the firmware-side TX KIDs remain allocated until the next FLR/FUNC_RESET, while the driver has dropped its bookkeeping for them. Would it be reasonable to log a netdev_warn() and/or short-circuit the probe path here?
Quoted 7 lines
drivers/net/ethernet/broadcom/bnxt/bnxt.c +0
-0INIT_LIST_HEAD (&bp->usr_fltr_list);drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +0
-0
[ ... ]
Quoted 11 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +11
-0+void bnxt_clear_crypto (struct bnxt *bp) +{ +struct bnxt_crypto_info *crypto = bp->crypto_info; +struct bnxt_kid_info *kid, *tmp; +struct bnxt_kctx *kctx; +int i; + +if (!crypto) +return ; + +/* Shutting down or FW reset, no need to protect the lists. */
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c void bnxt_alloc_crypto_info(struct bnxt *bp, · line 104
Quoted 10 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +10
-0+for (i =0 ; i <BNXT_MAX_CRYPTO_KEY_TYPE ; i++) { + kctx = &crypto->kctx[i]; +list_for_each_entry_safe (kid, tmp, &kctx->list, list) { +list_del (&kid->list); +kfree (kid); + } + kctx->total_alloc =0 ; + kctx->epoch++; + } +}
[Medium]
The kid nodes are published into kctx->list with list_add_tail_rcu() in bnxt_key_ctx_store() and read by bnxt_kid_valid(), bnxt_alloc_one_kctx() and bnxt_free_one_kctx() under list_for_each_entry_rcu() inside rcu_read_lock().
Here they are torn down with plain list_del() followed by an immediate kfree(), with no list_del_rcu(), no synchronize_rcu() and no kfree_rcu() (struct bnxt_kid_info has no rcu_head). The comment says "no need to protect the lists" because the function "assumes serialization (called during shutdown)", but the kerneldoc above also lists "firmware reset" as a use case.
If a future caller invokes bnxt_clear_crypto() from a path where an RCU reader is concurrently walking kctx->list, list_del() will turn kid->next into LIST_POISON1 and the immediate kfree() will return the memory the reader is about to dereference.
Would it be safer to use list_del_rcu() plus kfree_rcu() (adding an rcu_head to bnxt_kid_info) so the API is self-contained, regardless of caller serialization assumptions?
[Low]
Also on the epoch bump: kctx->epoch is a u8, and BNXT_KID_EPOCH_MASK is 0xff000000 (8 bits). Each call to bnxt_clear_crypto() does kctx->epoch++, so after 256 shutdown/reset cycles the value wraps back to a previously-used epoch.
The kerneldoc states that the increment is used to "invalidate any outstanding key references", but a stale KID encoding the wrapped epoch will silently pass the "epoch != kctx->epoch" check in bnxt_kid_valid() and be matched against a freshly-allocated KID range.
Should the epoch field be wider, given that there are 24 free bits in the KID encoding?
Quoted 4 lines
+
/**
*bnxt_free_crypto_info- Free crypto offload resources
* @bp: pointer to bnxt device
[ ... ]
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c void bnxt_hwrm_reserve_pf_key_ctxs(struct bnxt *bp, · line 161
Quoted 10 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +10
-0+static int bnxt_key_ctx_store (struct bnxt_kctx *kctx,__le32 *key_buf,u32 num, +bool contig,u8 kind,u32 *id) +{ +struct bnxt_kid_info *kid; +u32 i; + +for (i =0 ; i < num; ) { + kid =kzalloc_obj (*kid); +if (!kid) +return -ENOMEM ;
[Low]
On a kzalloc failure partway through a non-contiguous batch, this returns -ENOMEM with kids from earlier iterations already linked into kctx->list. The remaining keys in key_buf[i..num-1], and in the contig=true case all "num" keys, were already allocated by the firmware, but the driver now has no record of them.
Per the comment
/* Note that the driver does not free the key contexts. They are
* freed by the FW during FLR and HWRM_FUNC_RESET.
*/
these IDs stay allocated on the firmware side until the next reset.
Is there a way to either record them or release them via HWRM on the error path so the usable KID pool isn't permanently shrunk under sustained ENOMEM during kTLS bring-up?
Quoted 27 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +27
-0+ kid->start_id =le32_to_cpu (key_buf[i]); + kid->type = kctx->type; + kid->kind = kind; +if (contig) + kid->count = num; +else + kid->count =1 ; +bitmap_set (kid->ids,0 , kid->count); +if (id && !i) { +clear_bit (0 , kid->ids); + *id =BNXT_SET_KID (kctx, kid->start_id); + } +spin_lock (&kctx->lock); +list_add_tail_rcu (&kid->list, &kctx->list); + kctx->total_alloc += kid->count; +spin_unlock (&kctx->lock); + i += kid->count; + } +return 0 ; +} + +/* Note that the driver does not free the key contexts. They are freed +* by the FW during FLR and HWRM_FUNC_RESET. +*/ +static int bnxt_hwrm_key_ctx_alloc (struct bnxt *bp,struct bnxt_kctx *kctx, +u8 kind,u32 num,u32 *id) +{
[ ... ]
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c void bnxt_hwrm_reserve_pf_key_ctxs(struct bnxt *bp, · line 231
Quoted 10 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +10
-0+ num_alloc =le16_to_cpu (resp->num_key_ctxs_allocated); +if (num_alloc > num) +netdev_warn (bp->dev, +"FW allocated more keys (%d) than requested (%d)\n" , + num_alloc, num); +else + num = num_alloc; + contig = resp->flags & +FUNC_KEY_CTX_ALLOC_RESP_FLAGS_KEY_CTXS_CONTIGUOUS ; + rc =bnxt_key_ctx_store (kctx, key_buf, num, contig, kind, id);
[Medium]
What happens if the firmware returns success withnum_key_ctxs_allocated == 0?
The "else num = num_alloc;" branch sets num to 0 and then calls bnxt_key_ctx_store(kctx, key_buf, 0, contig, kind, id). Inside bnxt_key_ctx_store():
for (i = 0; i < num; ) {
...
if (id && !i) {
clear_bit(0, kid->ids);
*id = BNXT_SET_KID(kctx, kid->start_id);
}
...
}
return 0;the loop body never executes, so *id is never written and the function still returns 0. bnxt_hwrm_key_ctx_alloc() then falls through to key_alloc_exit_wake with rc==0, and bnxt_key_ctx_alloc_one() returns 0 on the "if (!rc) return 0;" path.
Would it be appropriate to either treat num_alloc == 0 as an error when an id was requested, or guarantee *id is initialised before the function returns success?
Quoted 8 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +8
-0+ +key_alloc_exit_wake: +if (pending_count >=BNXT_KCTX_ALLOC_PENDING_MAX ) +wake_up_all (&kctx->alloc_pending_wq); +key_alloc_exit: +hwrm_req_drop (bp, req); +return rc; +}
[ ... ]
Quoted 4 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +4
-0+void bnxt_free_one_kctx (struct bnxt_kctx *kctx,u32 id) +{ +struct bnxt_kid_info *kid; +
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c void bnxt_hwrm_reserve_pf_key_ctxs(struct bnxt *bp, · line 306
Quoted 10 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +10
-0+ id =BNXT_KID_HW (id); +rcu_read_lock (); +list_for_each_entry_rcu (kid, &kctx->list, list) { +if (id >= kid->start_id && id < kid->start_id + kid->count) { +set_bit (id - kid->start_id, kid->ids); +break ; + } + } +rcu_read_unlock (); +}
[Low]bnxt_kid_valid() rejects stale IDs via "epoch != kctx->epoch", but bnxt_free_one_kctx() strips the epoch with BNXT_KID_HW(id) without any such check.
If a caller passes an id captured before a bnxt_clear_crypto() epoch bump, set_bit() will mark a currently in-use slot as free, and a subsequent bnxt_alloc_one_kctx() can hand the same hardware KID to a second kTLS connection — which for AES-GCM means (key, nonce) reuse.
The single in-tree consumer (introduced later in the series) does call bnxt_kid_valid() before bnxt_free_one_kctx(), so this is not currently reachable, but would a defensive epoch check inside this function close the API hazard?
Quoted 18 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +18
-0+ +#define BNXT_KCTX_ALLOC_RETRY_MAX 3 + +int bnxt_key_ctx_alloc_one (struct bnxt *bp,struct bnxt_kctx *kctx,u8 kind, +u32 *id) +{ +int rc, retry =0 ; + +while (retry++ <BNXT_KCTX_ALLOC_RETRY_MAX ) { + rc =bnxt_alloc_one_kctx (kctx, kind, id); +if (!rc) +return 0 ; + +if ((kctx->total_alloc +BNXT_KID_BATCH_SIZE ) > kctx->max_ctx) +return -ENOSPC ; + +if (!BNXT_KCTX_ALLOC_OK (kctx)) { + wait_event(kctx->alloc_pending_wq,
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c void bnxt_hwrm_reserve_pf_key_ctxs(struct bnxt *bp, · line 334
Quoted 10 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +10
-0+ BNXT_KCTX_ALLOC_OK(kctx)); +continue ; + } + rc =bnxt_hwrm_key_ctx_alloc (bp, kctx, kind, +BNXT_KID_BATCH_SIZE , id); +if (!rc) +return 0 ; + } +return -EAGAIN ; +}
[Medium]
The struct definition documents kctx->lock as
/* to serialize update to the linked list and total_alloc */ spinlock_t lock;
and bnxt_key_ctx_store() does
spin_lock(&kctx->lock); list_add_tail_rcu(&kid->list, &kctx->list); kctx->total_alloc += kid->count; spin_unlock(&kctx->lock);
but here total_alloc is read with no lock and no READ_ONCE(). Multiple callers can each observe a stale value and proceed to issue their own HWRM allocation, which combined with the BNXT_KCTX_ALLOC_PENDING_MAX gate (8 in flight) can ask the firmware for up to
8 * BNXT_KID_BATCH_SIZE = 1024 IDs beyond max_ctx before any of them update the counter.
Should this read be serialised under kctx->lock, or otherwise made race-free with respect to the writers?
[Low]
On the BNXT_KCTX_ALLOC_OK gate itself, the check-then-act here is
non-atomic: BNXT_KCTX_ALLOC_OK reads alloc_pending and bnxt_hwrm_key_ctx_alloc() does a separate atomic_inc_return(). Many threads can simultaneously observe alloc_pending < 8 and then all proceed to inc, raising it arbitrarily above the cap.
In addition, the wake_up_all() in bnxt_hwrm_key_ctx_alloc() depends on its own pending_count (post-inc) being >= BNXT_KCTX_ALLOC_PENDING_MAX:
pending_count = atomic_inc_return(&kctx->alloc_pending); rc = hwrm_req_send(bp, req); atomic_dec(&kctx->alloc_pending); ... key_alloc_exit_wake: if (pending_count >= BNXT_KCTX_ALLOC_PENDING_MAX) wake_up_all(&kctx->alloc_pending_wq);
If a burst of callers all happen to record pending_count values < 8 by each racing past the check, none of them wake the wait queue, and a waiter parked in wait_event(BNXT_KCTX_ALLOC_OK) only gets released when some later caller happens to increment to >= 8.
Was this latency tolerated intentionally, or is the gate intended to be exact?
Quoted 4 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.h +0
-0
[ ... ]
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.h enum bnxt_crypto_type { · line 51
Quoted 2 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.h +2
-0+#define BNXT_SET_KID (kctx, kid) \ + ((kid) | ((kctx)->epoch << BNXT_KID_EPOCH_SHIFT))
[Low]kctx->epoch is u8 and is promoted to (signed) int before the shift. For epoch >= 128, "(int)epoch << 24" sets the sign bit of int, which is undefined behaviour per C99/C11 6.5.7p4.
In combination with the 8-bit epoch wraparound noted earlier on bnxt_clear_crypto(), epoch >= 128 is reachable, and a UBSan-instrumented kernel will splat here.
Would casting epoch to u32 before the shift be acceptable?
Quoted 4 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.h +4
-0+ +#define BNXT_KCTX_ALLOC_OK (kctx) \ + (atomic_read(&((kctx)->alloc_pending)) < BNXT_KCTX_ALLOC_PENDING_MAX) +
[ ... ]
Michael Chan
maintainerPATCH v2 10/15 bnxt_en: Add MPC transmit and completion functions
In reply to Michael ChanAdd transmit, ring selection, and completion functions for midpath rings. These will be used to send control data to the crypto engines.
---
v2:
Fix unused variable warnings
v1:
https://lore.kernel.org/netdev/[email protected]/
--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 3 + drivers/net/ethernet/broadcom/bnxt/bnxt.h | 2 + drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c | 138 ++++++++++++++++++ drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h | 65 +++++++++ 4 files changed, 208 insertions(+)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +3 -0
@@ -3109,6 +3109,9 @@ static int __bnxt_poll_work(struct bnxt *bp, struct bnxt_cp_ring_info *cpr, rx_pkts++; else if (rc == -EBUSY) /* partial completion */ break; + }else if (cmp_type ==CMP_TYPE_MPC_CMP ) { +if (bnxt_mpc_cmp (bp, cpr, &raw_cons)) +break ; }else if (unlikely (cmp_type ==CMPL_BASE_TYPE_HWRM_DONE || cmp_type ==CMPL_BASE_TYPE_HWRM_FWD_REQ || cmp_type ==CMPL_BASE_TYPE_HWRM_ASYNC_EVENT )) {
drivers/net/ethernet/broadcom/bnxt/bnxt.h +2 -0
@@ -47,6 +47,7 @@ struct tx_bd {__le32 tx_bd_len_flags_type;#define TX_BD_TYPE (0x3f << 0)#define TX_BD_TYPE_SHORT_TX_BD (0x00 << 0) +#define TX_BD_TYPE_MPC_TX_BD (0x08 << 0)#define TX_BD_TYPE_LONG_TX_BD (0x10 << 0)#define TX_BD_FLAGS_PACKET_END (1 << 6)#define TX_BD_FLAGS_NO_CMPL (1 << 7)
@@ -160,6 +161,7 @@ struct tx_cmp {#define CMP_TYPE_RX_TPA_AGG_CMP 22#define CMP_TYPE_RX_L2_V3_CMP 23#define CMP_TYPE_RX_L2_TPA_START_V3_CMP 25 +#define CMP_TYPE_MPC_CMP 30#define CMP_TYPE_STATUS_CMP 32#define CMP_TYPE_REMOTE_DRIVER_REQ 34#define CMP_TYPE_REMOTE_DRIVER_RESP 36
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +138 -0
141 lines not on this page · raw message
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h +65 -0
87 lines not on this page · raw message
Signature · 1 line
2.51.0
Jakub Kicinski
maintainerRe: [PATCH net-next v2 10/15] bnxt_en: Add MPC transmit and completion functions
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---bnxt_en: Add MPC transmit and completion functions
This patch adds transmit, ring selection, and completion handling helpers for the bnxt midpath rings (MPC), to be used for sending control data to the crypto engines.
Quoted 4 lines
drivers/net/ethernet/broadcom/bnxt/bnxt.c +0
-0
[ ... ]
Quoted 5 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +0
-0@@ -404,3 +404,141 @@ void bnxt_hwrm_mpc_ring_free(struct bnxt *bp, bool close_path)
[ ... ]
Quoted 3 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +3
-0+int bnxt_start_xmit_mpc (struct bnxt *bp,struct bnxt_tx_ring_info *txr, +void *data,unsigned int len,unsigned long handle) +{
[ ... ]
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c void bnxt_hwrm_mpc_ring_free(struct bnxt *bp, bool close_path) · line 464
Quoted 3 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +3
-0+memcpy (txbd, data, len); + prod += bds; + txr->tx_prod = prod;
[Low]
The rest of the bnxt tx fast paths consistently use WRITE_ONCE() when updating txr->tx_prod and txr->tx_cons, since bnxt_tx_avail() reads both with READ_ONCE():
static inline u32 bnxt_tx_avail(struct bnxt *bp,
const struct bnxt_tx_ring_info *txr)
{
u32 used = READ_ONCE(txr->tx_prod) - READ_ONCE(txr->tx_cons);
...
}The new MPC paths use plain stores for txr->tx_prod here in bnxt_start_xmit_mpc() and for txr->tx_cons in bnxt_mpc_cmp() (below).
Since bnxt_mpc_cmp() runs in NAPI context without tx_lock, while the producer side (in a follow-up patch) takes tx_lock around bnxt_start_xmit_mpc(), the producer's READ_ONCE(tx_cons) inside bnxt_tx_avail() is paired with the consumer's plain store, which would be flagged by KCSAN.
Should these stores use WRITE_ONCE() to match the convention in the rest of the driver?
Quoted 6 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +6
-0+ +/* Sync BD data before updating doorbell */ +wmb (); +bnxt_db_write (bp, &txr->tx_db, prod); + +return 0 ;
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c void bnxt_hwrm_mpc_ring_free(struct bnxt *bp, bool close_path) · line 480
Quoted 10 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +10
-0+} + +static bool bnxt_mpc_unsolicit (struct mpc_cmp *mpcmp) +{ +u32 client =MPC_CMP_CLIENT_TYPE (mpcmp); + +if (client !=MPC_CMP_CLIENT_TCE && client !=MPC_CMP_CLIENT_RCE ) +return false; +return MPC_CMP_UNSOLICIT_SUBTYPE (mpcmp); +}
[Medium]
The header in this same patch declares three subtypes:
#define MPC_CMP_SUBTYPE_SOLICITED (0x0 << 8)
#define MPC_CMP_SUBTYPE_ERR (0x1 << 8)
#define MPC_CMP_SUBTYPE_RESYNC (0x2 << 8)but MPC_CMP_UNSOLICIT_SUBTYPE() only matches MPC_CMP_SUBTYPE_ERR:
#define MPC_CMP_UNSOLICIT_SUBTYPE(mpcmp) \
((le32_to_cpu((mpcmp)->mpc_cmp_client_subtype_type) & \
MPC_CMP_SUBTYPE) == MPC_CMP_SUBTYPE_ERR)Should a RESYNC completion (which by name is not a response to a host TX) also be treated as unsolicited? As written, bnxt_mpc_unsolicit() returns false on RESYNC and bnxt_mpc_cmp() then advances tx_cons and clears mpc_buf->handle for a slot the host never submitted.
Would a check like != MPC_CMP_SUBTYPE_SOLICITED (or matching both ERR and RESYNC) be more in line with the macro name and the defined subtypes?
Quoted 3 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +3
-0+ +int bnxt_mpc_cmp (struct bnxt *bp,struct bnxt_cp_ring_info *cpr,u32 *raw_cons) +{
[ ... ]
Quoted 8 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +8
-0+if (!bnxt_mpc_unsolicit (mpcmp)) { +struct bnxt_sw_mpc_tx_bd *mpc_buf; +struct bnxt_tx_ring_info *txr; +u16 tx_cons; +u32 opaque; + + opaque = mpcmp->mpc_cmp_opaque; + txr = bnapi->tx_mpc_ring[client];
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c void bnxt_hwrm_mpc_ring_free(struct bnxt *bp, bool close_path) · line 539
Quoted 10 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +10
-0+ tx_cons = txr->tx_cons; +if (TX_OPAQUE_RING (opaque) != txr->tx_napi_idx) +netdev_warn (bp->dev,"Wrong opaque %x, expected ring %x, idx %x\n" , + opaque, txr->tx_napi_idx, txr->tx_cons); + mpc_buf = &txr->tx_mpc_buf_ring[RING_TX (bp, tx_cons)]; + mpc_buf->handle =0 ; + tx_cons += mpc_buf->inline_bds; + txr->tx_cons = tx_cons; + txr->tx_hw_cons =RING_TX (bp, tx_cons); + }
[Medium]
When the opaque ring index returned by the device does not match the locally-resolved txr (TX_OPAQUE_RING(opaque) != txr->tx_napi_idx), control just falls through after netdev_warn() and the code still dereferences txr->tx_mpc_buf_ring[RING_TX(bp, tx_cons)], clears mpc_buf->handle, and advances txr->tx_cons by mpc_buf->inline_bds.
If this check ever fires (HW bug, memory corruption, compromised firmware), would the driver be corrupting its own ring bookkeeping for the locally-resolved txr — clearing a handle for a buffer that was not actually completed and skipping over slots whose handles are then never released?
Given that handle holds a kernel pointer/cookie used by the crypto path, should this branch return early (or otherwise avoid touching tx_cons / tx_mpc_buf_ring) instead of continuing as if the completion were valid?
[Low]
Same WRITE_ONCE() concern as in bnxt_start_xmit_mpc() — txr->tx_cons is written here with a plain store while producers read it via READ_ONCE(txr->tx_cons) in bnxt_tx_avail(). Should this be WRITE_ONCE(txr->tx_cons, tx_cons) for consistency with the rest of the driver?
Quoted 10 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +5
-0+ +cmp_done: + *raw_cons = tmp_raw_cons; +return 0 ; +}drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h +0
-0@@ -30,11 +30,53 @@ struct bnxt_mpc_info {
[ ... ]
Quoted 10 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h +10
-0+struct mpc_cmp { +__le32 mpc_cmp_client_subtype_type; +#define MPC_CMP_TYPE (0x3f << 0) +#define MPC_CMP_TYPE_MID_PATH_SHORT 0x1e +#define MPC_CMP_TYPE_MID_PATH_LONG 0x1f +#define MPC_CMP_SUBTYPE 0xf00 +#define MPC_CMP_SUBTYPE_SFT 8 +#define MPC_CMP_SUBTYPE_SOLICITED (0x0 << 8) +#define MPC_CMP_SUBTYPE_ERR (0x1 << 8) +#define MPC_CMP_SUBTYPE_RESYNC (0x2 << 8)
[ ... ]
Quoted 3 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h +3
-0+#define MPC_CMP_UNSOLICIT_SUBTYPE (mpcmp) \ + ((le32_to_cpu((mpcmp)->mpc_cmp_client_subtype_type) & \ + MPC_CMP_SUBTYPE) == MPC_CMP_SUBTYPE_ERR)
[ ... ]
Michael Chan
maintainerPATCH v2 11/15 bnxt_en: Add crypto MPC transmit/completion infrastructure
In reply to Michael ChanAdd infrastructure to support sending crypto commands using the midpath channels (MPCs). bnxt_xmit_crypto_cmd() is used to send a crypto command and sleep with timeout until the completion is received.
If it times out, we recover by resetting the MPC. The next patch will
use this infrastructure to offload kTLS connections.
--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 4 +- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 2 + .../net/ethernet/broadcom/bnxt/bnxt_crypto.c | 145 ++++++++++++++- .../net/ethernet/broadcom/bnxt/bnxt_crypto.h | 92 ++++++++++ drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c | 171 +++++++++++++++++- drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h | 9 + 6 files changed, 418 insertions(+), 5 deletions(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +2 -2
@@ -7714,7 +7714,7 @@ void bnxt_hwrm_cp_ring_free(struct bnxt *bp, struct bnxt_cp_ring_info *cpr) ring->fw_ring_id =INVALID_HW_RING_ID ; } -static void bnxt_clear_one_cp_ring (struct bnxt *bp,struct bnxt_cp_ring_info *cpr) +void bnxt_clear_one_cp_ring (struct bnxt *bp,struct bnxt_cp_ring_info *cpr) {struct bnxt_ring_struct *ring = &cpr->cp_ring_struct;int i, size = ring->ring_mem.page_size;
@@ -14368,7 +14368,7 @@ static int bnxt_hwrm_rx_ring_reset(struct bnxt *bp, int ring_nr)return hwrm_req_send_silent (bp, req); } -static void bnxt_reset_task (struct bnxt *bp,bool silent) +void bnxt_reset_task (struct bnxt *bp,bool silent) {if (!silent)bnxt_dbg_dump_states (bp);
drivers/net/ethernet/broadcom/bnxt/bnxt.h +2 -0
@@ -3019,6 +3019,7 @@ int bnxt_hwrm_tx_ring_alloc(struct bnxt *bp, struct bnxt_tx_ring_info *txr,void bnxt_hwrm_tx_ring_free (struct bnxt *bp,struct bnxt_tx_ring_info *txr,bool close_path);void bnxt_hwrm_cp_ring_free (struct bnxt *bp,struct bnxt_cp_ring_info *cpr); +void bnxt_clear_one_cp_ring (struct bnxt *bp,struct bnxt_cp_ring_info *cpr);int bnxt_total_tx_rings (struct bnxt *bp);int __bnxt_hwrm_get_tx_rings (struct bnxt *bp,u16 fid,int *tx_rings);int bnxt_nq_rings_in_use (struct bnxt *bp);
@@ -3064,6 +3065,7 @@ void bnxt_get_ring_drv_stats(struct bnxt *bp,bool bnxt_rfs_capable (struct bnxt *bp,bool new_rss_ctx);int bnxt_dbg_hwrm_rd_reg (struct bnxt *bp,u32 reg_off,u16 num_words,u32 *reg_buf); +void bnxt_reset_task (struct bnxt *bp,bool silent);void bnxt_fw_exception (struct bnxt *bp);void bnxt_fw_reset (struct bnxt *bp); int bnxt_check_rings(struct bnxt *bp, int tx, int rx, bool sh, int tcs,
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +144 -1
189 lines not on this page · raw message
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.h +92 -0
114 lines not on this page · raw message
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +169 -2
211 lines not on this page · raw message
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h +9 -0
@@ -22,6 +22,8 @@ enum bnxt_mpc_type {#define BNXT_DFLT_MPC_TCE BNXT_MAX_MPC#define BNXT_DFLT_MPC_RCE BNXT_MAX_MPC +#define BNXT_MPC_TMO_MSECS 1000 +struct bnxt_mpc_info {u8 mpc_chnls_cap;u8 mpc_cp_rings;
@@ -106,6 +108,7 @@ void bnxt_hwrm_mpc_ring_free(struct bnxt *bp, bool close_path);struct bnxt_tx_ring_info *bnxt_select_mpc_ring (struct bnxt *bp,int ring_type);int bnxt_start_xmit_mpc (struct bnxt *bp,struct bnxt_tx_ring_info *txr,void *data,unsigned int len,unsigned long handle); +int bnxt_mpc_timeout (struct bnxt *bp,struct bnxt_tx_ring_info *txr);int bnxt_mpc_cmp (struct bnxt *bp,struct bnxt_cp_ring_info *cpr,u32 *raw_cons);#else static inline void bnxt_alloc_mpc_info (struct bnxt *bp,u8 mpc_chnls_cap)
@@ -192,6 +195,12 @@ static inline int bnxt_start_xmit_mpc(struct bnxt *bp,return -EOPNOTSUPP ; } +static inline int bnxt_mpc_timeout (struct bnxt *bp, +struct bnxt_tx_ring_info *txr) +{ +return -EOPNOTSUPP ; +} +static inline int bnxt_mpc_cmp (struct bnxt *bp,struct bnxt_cp_ring_info *cpr,u32 *raw_cons) {
Signature · 1 line
2.51.0
Jakub Kicinski
maintainerRe: [PATCH net-next v2 11/15] bnxt_en: Add crypto MPC transmit/completion infrastructure
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---bnxt_en: Add crypto MPC transmit/completion infrastructure
This patch adds infrastructure to send crypto commands over the midpath channels (MPCs). bnxt_xmit_crypto_cmd() submits a command and sleeps with a timeout for the completion; on timeout the MPC ring is reset.
Quoted 4 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +0
-0
[ ... ]
Quoted 4 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +0
-0@@ -113,8 +130,13 @@ void bnxt_clear_crypto(struct bnxt *bp)*/ void bnxt_free_crypto_info (struct bnxt *bp) {
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c void bnxt_clear_crypto(struct bnxt *bp) · line 141
Quoted 10 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +6
-1+struct bnxt_crypto_info *crypto = bp->crypto_info; + +if (!crypto) +return ;bnxt_clear_crypto (bp); -kfree (bp->crypto_info); +kmem_cache_destroy (crypto->mpc_cache); +kfree (crypto); bp->crypto_info =NULL ; }
[High]
Is there a guarantee that no bnxt_xmit_crypto_cmd() caller is still in flight when bnxt_free_crypto_info() runs?
bnxt_clear_crypto() does not appear to wait for callers currently sleeping in wait_for_completion_timeout() or in the 200 ms poll inside bnxt_crypto_timeout(), and there is no refcount or wait_event tying ctx lifetime to cache destruction.
If a caller is mid-flight during teardown, can kmem_cache_destroy() trigger the "Slab cache still has objects" warning, and can the surviving caller later call kmem_cache_free() on a destroyed cache?
Quoted 11 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +7
-0@@ -333,6 +355,90 @@ int bnxt_key_ctx_alloc_one(struct bnxt *bp, struct bnxt_kctx *kctx, u8 kind,return -EAGAIN ; } +#define BNXT_CMD_CTX_RETRY_MAX 10 + +static void bnxt_crypto_timeout (struct bnxt *bp,struct bnxt_tx_ring_info *txr, +struct bnxt_crypto_cmd_ctx *ctx) +{ +int i; +
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c int bnxt_key_ctx_alloc_one(struct bnxt *bp, struct bnxt_kctx *kctx, u8 kind, · line 374
Quoted 10 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +10
-0+bnxt_mpc_timeout (bp, txr); +/* Wait for the ctx to complete before proceeding */ +for (i =0 ; i <BNXT_CMD_CTX_RETRY_MAX && + !(ctx->status &BNXT_CMD_CTX_COMPLETED ); i++) +msleep (20 ); +if (!(ctx->status &BNXT_CMD_CTX_COMPLETED )) +netdev_warn (bp->dev, +"Timed out waiting for cmd_ctx to complete for MPC ring %d\n" , + txr->txq_index); +}
[Critical]
After this 200 ms poll the caller proceeds to kmem_cache_free(ctx) unconditionally. Are there paths where bnxt_mpc_timeout() does not actually clean up ctx->handle from tx_mpc_buf_ring[] within 200 ms, leaving the freed pointer reachable?
bnxt_mpc_timeout() returns -EINVAL when txr->tx_ring_struct.queue_id is not BNXT_MPC_QUEUE_ID, and this rc is not checked here:
int bnxt_mpc_timeout(struct bnxt *bp, struct bnxt_tx_ring_info *txr)
{
if (txr->tx_ring_struct.queue_id == BNXT_MPC_QUEUE_ID)
return bnxt_mpc_ring_reset(bp, txr->txq_index);
return -EINVAL;
}bnxt_mpc_ring_reset() also returns 0 immediately when bnxt_disable_mpc_ring() loses the race because dev_state was already set:
if (!bnxt_disable_mpc_ring(mpc, mpc_ring)) return 0;
In both cases, no bnxt_clear_one_mpc_entries() runs, the loser's poll exits after 200 ms, and the freed ctx is still pointed to from tx_mpc_buf_ring[i].handle. When NAPI later observes the completion (or the winner's bnxt_clear_one_mpc_entries() iterates), bnxt_crypto_mpc_cmp() will write ctx->status, memcpy into ctx->ce_cmp, and complete(&ctx->cmp) on freed slab memory.
Should the timeout path arrange that the handle is removed (or the ctx guaranteed completed) before kmem_cache_free()?
Quoted 7 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +7
-0+#define BNXT_XMIT_CRYPTO_RETRY_MAX 10 +#define BNXT_XMIT_CRYPTO_MIN_TMO 100 +#define BNXT_XMIT_CRYPTO_MAX_TMO 150 + +int bnxt_xmit_crypto_cmd (struct bnxt *bp,struct bnxt_tx_ring_info *txr, +void *cmd,unsigned int len,unsigned int tmo) +{
[ ... ]
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c int bnxt_key_ctx_alloc_one(struct bnxt *bp, struct bnxt_kctx *kctx, u8 kind, · line 422
Quoted 8 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +8
-0+ tmo_left =wait_for_completion_timeout (&ctx->cmp,msecs_to_jiffies (tmo)); +if (!tmo_left) { +netdev_warn (bp->dev,"crypto MP cmd %08x timed out\n" , + *((u32 *)cmd)); +bnxt_crypto_timeout (bp, txr, ctx); + rc = -ETIMEDOUT ; + goto xmit_done; + }
[Critical]
Even outside the timeout path, is the handshake between bnxt_crypto_mpc_cmp() and the waiter safe? bnxt_crypto_mpc_cmp() does:
ctx = (void *)handle; ctx->status = BNXT_CMD_CTX_COMPLETED; ... memcpy(&ctx->ce_cmp, cmpl[0].cmpl, len); cmp_done: complete(&ctx->cmp);
The timeout poll loop above exits as soon as it observes (ctx->status & BNXT_CMD_CTX_COMPLETED). If the waiter happened to time out at exactly the same time as a real completion, can the poll observe status set by mpc_cmp(), then proceed to kmem_cache_free(ctx) while the mpc_cmp() side is still inside complete(&ctx->cmp) and is touching ctx->cmp.wait on another CPU?
[ ... ]
Quoted 2 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +2
-0+xmit_done: +if (rc) {
[ ... ]
Quoted 14 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +13
-0+if (ctx) +kmem_cache_free (crypto->mpc_cache, ctx); +return rc; +} +void bnxt_crypto_mpc_cmp (struct bnxt *bp,u32 client,unsigned long handle, +struct bnxt_cmpl_entry cmpl[],u32 entries) +{ +struct bnxt_crypto_cmd_ctx *ctx; +struct ce_cmpl *cmp =NULL ; +u32 len, kid; + +if (likely (cmpl)) + cmp = cmpl[0 ].cmpl;
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c int bnxt_crypto_init(struct bnxt *bp) · line 492
Quoted 10 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +10
-0+if (!handle || entries !=1 ) { +if (entries !=1 && cmpl) { +netdev_warn (bp->dev,"Invalid entries %d with handle %lx cmpl %08x in %s()\n" , + entries, handle, *(u32 *)cmp, __func__); + } +if (!handle) +return ; + } + ctx = (void *)handle; + ctx->status =BNXT_CMD_CTX_COMPLETED ;
[Medium]
When entries != 1 with a valid cmpl, the function emits a warning and falls through to set ctx->status = BNXT_CMD_CTX_COMPLETED without setting BNXT_CMD_CTX_ERROR (BNXT_CMD_CTX_ERROR is only set on the kid/client mismatch branch below). The waiter checks:
if (ctx->status == BNXT_CMD_CTX_COMPLETED && CE_CMPL_STATUS(&ctx->ce_cmp) == CE_CMPL_STATUS_OK) rc = 0;
Should the "invalid entries" branch also OR in BNXT_CMD_CTX_ERROR so that the caller does not see a successful return for a completion the driver itself flagged as malformed?
Quoted 4 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +4
-0+if (unlikely (!cmpl)) { + ctx->status |=BNXT_CMD_CTX_RESET ; + goto cmp_done; + }
[ ... ]
Quoted 4 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +0
-0
[ ... ]
Quoted 22 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +22
-0+/* Returns true if the ring is successfully marked as closing. */ +static bool bnxt_disable_mpc_ring (struct bnxt_mpc_info *mpc,int mpc_ring) +{ +struct bnxt_tx_ring_info *txr; +bool disabled = false; +int i; + +for (i =0 ; i <BNXT_MPC_TYPE_MAX ; i++) { +if (mpc_ring >= mpc->mpc_ring_count[i]) +continue ; + txr = &mpc->mpc_rings[i][mpc_ring]; +spin_lock_bh (&txr->tx_lock); +if (!READ_ONCE (txr->dev_state)) { + disabled = true; +WRITE_ONCE (txr->dev_state,BNXT_DEV_STATE_CLOSING ); + } +spin_unlock_bh (&txr->tx_lock); +if (!disabled) +break ; + } +return disabled; +}
[ ... ]
Quoted 39 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +38
-0+static void bnxt_clear_one_mpc_entries (struct bnxt *bp, +struct bnxt_tx_ring_info *txr) +{ +struct bnxt_sw_mpc_tx_bd *tx_buf; +unsigned long handle; +int i, max_idx; +u32 client; + + max_idx = bp->tx_nr_pages *TX_DESC_CNT ; + +for (i =0 ; i < max_idx; i++) { + tx_buf = &txr->tx_mpc_buf_ring[i]; + handle = tx_buf->handle; +if (handle) { + client = txr->tx_ring_struct.mpc_chnl_type; +bnxt_crypto_mpc_cmp (bp, client, handle,NULL ,0 ); + tx_buf->handle =0 ; + } + } +} +static void bnxt_mpc_ring_stop (struct bnxt *bp,struct bnxt_mpc_info *mpc, +int mpc_ring) +{ +struct bnxt_tx_ring_info *txr; +struct bnxt_cp_ring_info *cpr; +int i; + +for (i =0 ; i <BNXT_MPC_TYPE_MAX ; i++) { +if (mpc->mpc_ring_count[i] > mpc_ring) { + txr = &mpc->mpc_rings[i][mpc_ring]; +bnxt_hwrm_tx_ring_free (bp, txr, true); + } + } +/* CP rings must be freed at the end to guarantee that the HWRM_DONE +* responses for HWRM_RING_FREE can still be seen on the CP rings. +*/ +for (i =0 ; i <BNXT_MPC_TYPE_MAX ; i++) { +if (mpc->mpc_ring_count[i] > mpc_ring) {
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c int bnxt_start_xmit_mpc(struct bnxt *bp, struct bnxt_tx_ring_info *txr, · line 575
Quoted 10 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +10
-0+ txr = &mpc->mpc_rings[i][mpc_ring]; + cpr = txr->tx_cpr; +if (cpr) { +bnxt_hwrm_cp_ring_free (bp, cpr); +bnxt_clear_one_cp_ring (bp, cpr); + } +bnxt_clear_one_mpc_entries (bp, txr); + } + } +}
[Critical]
Is bnxt_disable_mpc_ring() sufficient to fence the NAPI poller before the ring teardown begins?
bnxt_disable_mpc_ring() takes spin_lock_bh(&txr->tx_lock) and flips dev_state to BNXT_DEV_STATE_CLOSING, but the NAPI consumer in bnxt_mpc_cmp() reads dev_state with bare READ_ONCE() and never takes tx_lock:
mpc_buf = &txr->tx_mpc_buf_ring[RING_TX(bp, tx_cons)]; if (!READ_ONCE(txr->dev_state)) handle = mpc_buf->handle; mpc_buf->handle = 0; ... bnxt_crypto_mpc_cmp(bp, client, handle, cmpl_entry_arr, cmpl_num);
There is no synchronize_net() / napi_synchronize() between the dev_state flip and bnxt_mpc_ring_stop()'s teardown.
A NAPI poll already past the READ_ONCE(dev_state) check will have captured handle = mpc_buf->handle and is about to call bnxt_crypto_mpc_cmp() while, on another CPU, bnxt_clear_one_mpc_entries() walks tx_mpc_buf_ring, reads the same still-non-zero tx_buf->handle, and also calls bnxt_crypto_mpc_cmp() on it. Two complete()s on the same ctx; if the waiter freed ctx after the first one, the second is a use-after-free.
Separately, bnxt_clear_one_cp_ring() memset()s cp_desc_ring pages to zero before NAPI is fenced, so can NAPI observe partially-zeroed completion entries while iterating?
The L2 close path in bnxt.c uses the explicit pattern of WRITE_ONCE on dev_state followed by synchronize_net() (with the comment "guarantee no more DMA on that ring after seeing the completion") before tearing the ring down. Should the MPC reset path do the same?
Quoted 12 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +12
-0+static int bnxt_mpc_ring_reset (struct bnxt *bp,int mpc_ring) +{ +struct bnxt_mpc_info *mpc = bp->mpc_info; +int rc; + +if (!mpc) +return 0 ; +if (mpc_ring >= mpc->mpc_cp_rings) +return -EINVAL ; + +if (!bnxt_disable_mpc_ring (mpc, mpc_ring)) +return 0 ;
[ ... ]
Quoted 6 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +6
-0+int bnxt_mpc_timeout (struct bnxt *bp,struct bnxt_tx_ring_info *txr) +{ +if (txr->tx_ring_struct.queue_id ==BNXT_MPC_QUEUE_ID ) +return bnxt_mpc_ring_reset (bp, txr->txq_index); +return -EINVAL ; +}
[ ... ]
Quoted 19 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c +4
-1@@ -529,14 +693,17 @@ int bnxt_mpc_cmp(struct bnxt *bp, struct bnxt_cp_ring_info *cpr, u32 *raw_cons) txr = bnapi->tx_mpc_ring[client]; tx_cons = txr->tx_cons;if (TX_OPAQUE_RING (opaque) != txr->tx_napi_idx) -netdev_warn (bp->dev,"Wrong opaque %x, expected ring %x, idx %x\n" , +netdev_warn (bp->dev,"Wrong opaque %x, expected ring %x, cons idx %x\n" , opaque, txr->tx_napi_idx, txr->tx_cons); mpc_buf = &txr->tx_mpc_buf_ring[RING_TX (bp, tx_cons)]; +if (!READ_ONCE (txr->dev_state)) + handle = mpc_buf->handle; mpc_buf->handle =0 ; tx_cons += mpc_buf->inline_bds; txr->tx_cons = tx_cons; txr->tx_hw_cons =RING_TX (bp, tx_cons); } +bnxt_crypto_mpc_cmp (bp, client, handle, cmpl_entry_arr, cmpl_num); cmp_done: *raw_cons = tmp_raw_cons;
Michael Chan
maintainerPATCH v2 12/15 bnxt_en: Support kTLS TX offload by implementing .tls_dev_add/del()
In reply to Michael ChanAdd basic infrastructure to allocate and free kTLS context IDs (KIDs) to support kTLS TX offload. To offload a connection in .tls_dev_add(), the first step is to allocate a KID. After that the kTLS offload command is sent to the HW via MPC using the function bnxt_xmit_crypto_cmd() introduced in the last patch.
In .tls_dev_del(), we send the delete command to the HW using the same bnxt_xmit_crypto_cmd(). After that we free the KID, making it available for new offload.
bnxt_ktls_init() assigns bnxt_ktls_ops to the netdev and sets up the TLS TX offload feature. bnxt_ktls_init() will be called in the next patch.
---
v2:
Fix unused variable warning
Fix error recovery issues
v1:
https://lore.kernel.org/netdev/[email protected]/
--- drivers/net/ethernet/broadcom/bnxt/Makefile | 2 +- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 6 +- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 1 + .../net/ethernet/broadcom/bnxt/bnxt_crypto.c | 65 ++++ .../net/ethernet/broadcom/bnxt/bnxt_crypto.h | 13 +- .../net/ethernet/broadcom/bnxt/bnxt_ktls.c | 289 ++++++++++++++++++ .../net/ethernet/broadcom/bnxt/bnxt_ktls.h | 90 ++++++ 7 files changed, 463 insertions(+), 3 deletions(-) create mode 100644 drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c create mode 100644 drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.h
drivers/net/ethernet/broadcom/bnxt/Makefile +1 -1
@@ -5,4 +5,4 @@ bnxt_en-y := bnxt.o bnxt_hwrm.o bnxt_sriov.o bnxt_ethtool.o bnxt_dcb.o bnxt_ulp. bnxt_en-$(CONFIG_BNXT_FLOWER_OFFLOAD) += bnxt_tc.o bnxt_en-$(CONFIG_DEBUG_FS) += bnxt_debugfs.o bnxt_en-$(CONFIG_BNXT_HWMON) += bnxt_hwmon.o -bnxt_en-$(CONFIG_BNXT_TLS) += bnxt_mpc.o bnxt_crypto.o +bnxt_en-$(CONFIG_BNXT_TLS) += bnxt_mpc.o bnxt_crypto.o bnxt_ktls.o
drivers/net/ethernet/broadcom/bnxt/bnxt.c +5 -1
@@ -78,6 +78,7 @@#include <net/tso.h> #include "bnxt_mpc.h" #include "bnxt_crypto.h" +#include "bnxt_ktls.h" #define BNXT_TX_TIMEOUT (5 * HZ)#define BNXT_DEF_MSG_ENABLE (NETIF_MSG_DRV | NETIF_MSG_HW | \
@@ -13411,7 +13412,8 @@ static int bnxt_open(struct net_device *dev)static bool bnxt_drv_busy (struct bnxt *bp) {return (test_bit (BNXT_STATE_IN_SP_TASK , &bp->state) || -test_bit (BNXT_STATE_READ_STATS , &bp->state)); +test_bit (BNXT_STATE_READ_STATS , &bp->state) || +bnxt_ktls_busy (bp)); } static void bnxt_get_ring_stats(struct bnxt *bp,
@@ -13491,6 +13493,7 @@ static int bnxt_close(struct net_device *dev) {struct bnxt *bp =netdev_priv (dev); +bnxt_crypto_del_all (bp);bnxt_close_nic (bp, true, true);bnxt_hwrm_shutdown_link (bp);bnxt_hwrm_if_change (bp, false);
@@ -14567,6 +14570,7 @@ static void bnxt_fw_reset_close(struct bnxt *bp) bp->fw_reset_min_dsecs =0 ;bnxt_fw_fatal_close (bp); } +bnxt_crypto_del_all (bp);__bnxt_close_nic (bp, true, false);bnxt_vf_reps_free (bp);bnxt_clear_int_mode (bp);
drivers/net/ethernet/broadcom/bnxt/bnxt.h +1 -0
@@ -2474,6 +2474,7 @@ struct bnxt {struct bnxt_mpc_info *mpc_info;struct bnxt_crypto_info *crypto_info; +struct bnxt_tls_info *ktls_info;unsigned int current_interval;#define BNXT_TIMER_INTERVAL HZ
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +65 -0
89 lines not on this page · raw message
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.h +12 -1
@@ -40,7 +40,7 @@ struct bnxt_kctx {wait_queue_head_t alloc_pending_wq; }; -#define BNXT_KID_HW_MASK 0xffffff +#define BNXT_KID_HW_MASK 0x0fffff#define BNXT_KID_HW (kid) ((kid) & BNXT_KID_HW_MASK)#define BNXT_KID_EPOCH_MASK 0xff000000#define BNXT_KID_EPOCH_SHIFT 24
@@ -141,6 +141,8 @@ struct bnxt_crypto_cmd_ctx {#ifdef CONFIG_BNXT_TLS void bnxt_alloc_crypto_info (struct bnxt *bp,struct hwrm_func_qcaps_output *resp); +int bnxt_crypto_del (struct bnxt *bp,u8 type,u8 kind,u32 kid); +void bnxt_crypto_del_all (struct bnxt *bp);void bnxt_clear_crypto (struct bnxt *bp);void bnxt_free_crypto_info (struct bnxt *bp); void bnxt_hwrm_reserve_pf_key_ctxs(struct bnxt *bp,
@@ -160,6 +162,15 @@ static inline void bnxt_alloc_crypto_info(struct bnxt *bp, { } +static inline int bnxt_crypto_del (struct bnxt *bp,u8 type,u8 kind,u32 kid) +{ +return -EOPNOTSUPP ; +} + +static inline void bnxt_crypto_del_all (struct bnxt *bp) +{ +} +static inline void bnxt_clear_crypto (struct bnxt *bp) { }
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c new file +289 -0
289 lines not on this page · raw message
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.h new file +90 -0
90 lines not on this page · raw message
Signature · 1 line
2.51.0
Jakub Kicinski
maintainerRe: [PATCH net-next v2 12/15] bnxt_en: Support kTLS TX offload by implementing .tls_dev_add/del()
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---bnxt_en: Support kTLS TX offload by implementing .tls_dev_add/del()
This patch adds the basic infrastructure for kTLS TX offload in bnxt_en by allocating and freeing kTLS context IDs (KIDs) via MPC commands, and introduces bnxt_ktls_init() that wires bnxt_ktls_ops onto the netdev for later use.
A few questions follow inline.
Quoted 4 lines
drivers/net/ethernet/broadcom/bnxt/bnxt.c +0
-0
[ ... ]
drivers/net/ethernet/broadcom/bnxt/bnxt.c static int bnxt_close(struct net_device *dev) · line 13499
Quoted 8 lines
drivers/net/ethernet/broadcom/bnxt/bnxt.c +1
-0@@ -13491,6 +13493,7 @@ static int bnxt_close(struct net_device *dev) {struct bnxt *bp =netdev_priv (dev); +bnxt_crypto_del_all (bp);bnxt_close_nic (bp, true, true);bnxt_hwrm_shutdown_link (bp);bnxt_hwrm_if_change (bp, false);
[High]
Is the placement of bnxt_crypto_del_all() before bnxt_close_nic() safe against concurrent tls_dev_add/tls_dev_del?
At this point BNXT_STATE_OPEN is still set and the pending counter has not been drained yet (that happens inside __bnxt_close_nic()). The net/tls core only holds device_offload_lock for read on the add path and does not synchronize with .ndo_stop, so bnxt_ktls_dev_add() can run in parallel with bnxt_crypto_del_all() and call into bnxt_key_ctx_alloc_one(), which modifies kctx->list with list_add_tail_rcu() under kctx->lock.
bnxt_crypto_del_all() walks the same list with plain list_for_each_entry() (no rcu_read_lock, no kctx->lock), while the alloc/free helpers explicitly use list_*_rcu primitives:
Quoted 8 lines
drivers/net/ethernet/broadcom/bnxt/bnxt.c +8
-0+void bnxt_crypto_del_all (struct bnxt *bp) +{ +struct bnxt_crypto_info *crypto = bp->crypto_info; +struct bnxt_kid_info *kid; +struct bnxt_kctx *kctx; +int i; + +if (!crypto)
drivers/net/ethernet/broadcom/bnxt/bnxt.c
Quoted 10 lines
drivers/net/ethernet/broadcom/bnxt/bnxt.c +10
-0+return ; + +/* Shutting down, no need to protect the lists. */ +for (i =0 ; i <BNXT_MAX_CRYPTO_KEY_TYPE ; i++) { + kctx = &crypto->kctx[i]; +list_for_each_entry (kid, &kctx->list, list) +bnxt_crypto_del_all_kids (bp, kid); + kctx->epoch++; + } +}
The "Shutting down, no need to protect the lists" comment seems to assume no concurrent mutation, but with this call site preceding the OPEN-flag clear and pending-drain, can a concurrent tls_dev_add insert a new kid mid-traversal? The same applies to the call from bnxt_fw_reset_close() below.
A second concern is in bnxt_crypto_del_all_kids():
drivers/net/ethernet/broadcom/bnxt/bnxt.c
Quoted 13 lines
drivers/net/ethernet/broadcom/bnxt/bnxt.c +13
-0+static void bnxt_crypto_del_all_kids (struct bnxt *bp,struct bnxt_kid_info *kid) +{ +int i, rc; + +for (i =0 ; i < kid->count; i++) { +if (!test_bit (i, kid->ids)) { + rc =bnxt_crypto_del (bp, kid->type, kid->kind, + kid->start_id + i); +if (!rc) +set_bit (i, kid->ids); + } + } +}
The sequence test_bit(i) -> bnxt_crypto_del() (which can sleep inside bnxt_xmit_crypto_cmd()) -> set_bit(i) is not atomic. If a concurrent bnxt_free_one_kctx() does set_bit(i) and a subsequent bnxt_alloc_one_kctx() reissues the same KID via test_and_clear_bit(i) during that sleep, can the trailing set_bit() in this loop then mark an in-use slot as free, so the next allocation hands the same KID out to a second TLS socket?
Quoted 8 lines
drivers/net/ethernet/broadcom/bnxt/bnxt.c +1
-0@@ -14567,6 +14570,7 @@ static void bnxt_fw_reset_close(struct bnxt *bp) bp->fw_reset_min_dsecs =0 ;bnxt_fw_fatal_close (bp); } +bnxt_crypto_del_all (bp);__bnxt_close_nic (bp, true, false);bnxt_vf_reps_free (bp);bnxt_clear_int_mode (bp);
[ ... ]
Quoted 2 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c new file +0
-0
[ ... ]
Quoted 46 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c new file +46
-0+static int bnxt_crypto_add (struct bnxt *bp,enum tls_offload_ctx_dir direction, +struct tls_crypto_info *crypto_info,u32 tcp_seq_no, +u32 kid) +{ +struct bnxt_tx_ring_info *txr; +struct ce_add_cmd cmd = {0 }; +u32 data; + +if (direction ==TLS_OFFLOAD_CTX_DIR_TX ) { + txr =bnxt_select_mpc_ring (bp,BNXT_MPC_TCE_TYPE ); + cmd.ctx_kind =CE_ADD_CMD_CTX_KIND_CK_TX ; + }else { +return -EOPNOTSUPP ; + } +if (!txr) +return -ENODEV ; + + data =CE_ADD_CMD_OPCODE_ADD | (BNXT_KID_HW (kid) <<CE_ADD_CMD_KID_SFT ); +switch (crypto_info->cipher_type) { +case TLS_CIPHER_AES_GCM_128 : { +struct tls12_crypto_info_aes_gcm_128 *aes; + + aes = (void *)crypto_info; + data |=CE_ADD_CMD_ALGORITHM_AES_GCM_128 ; +if (crypto_info->version ==TLS_1_3_VERSION ) + data |=CE_ADD_CMD_VERSION_TLS1_3 ; +memcpy (&cmd.session_key, aes->key,sizeof (aes->key)); +memcpy (&cmd.salt, aes->salt,sizeof (aes->salt)); +memcpy (&cmd.addl_iv, aes->iv,sizeof (aes->iv)); +bnxt_copy_tls_mp_data (cmd.record_seq_num, aes->rec_seq, +sizeof (aes->rec_seq)); +break ; + } +case TLS_CIPHER_AES_GCM_256 : { +struct tls12_crypto_info_aes_gcm_256 *aes; + + aes = (void *)crypto_info; + data |=CE_ADD_CMD_ALGORITHM_AES_GCM_256 ; +if (crypto_info->version ==TLS_1_3_VERSION ) + data |=CE_ADD_CMD_VERSION_TLS1_3 ; +memcpy (&cmd.session_key, aes->key,sizeof (aes->key)); +memcpy (&cmd.salt, aes->salt,sizeof (aes->salt)); +memcpy (&cmd.addl_iv, aes->iv,sizeof (aes->iv)); +bnxt_copy_tls_mp_data (cmd.record_seq_num, aes->rec_seq, +sizeof (aes->rec_seq)); +break ;
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c line 138
Quoted 10 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c new file +10
-0+ } +default : +return -EOPNOTSUPP ; + } + cmd.ver_algo_kid_opcode =cpu_to_le32 (data); + cmd.pkt_tcp_seq_num =cpu_to_le32 (tcp_seq_no); + cmd.tls_header_tcp_seq_num = cmd.pkt_tcp_seq_num; +return bnxt_xmit_crypto_cmd (bp, txr, &cmd,sizeof (cmd), +BNXT_MPC_TMO_MSECS ); +}
[High]
The cmd buffer here is on the kernel stack and ends up holding a cleartext copy of the AES-GCM session key in cmd.session_key. After bnxt_xmit_crypto_cmd() returns, the function returns without zeroing the buffer, so the key bytes remain in the freed stack frame until they are overwritten.
Should this end with a memzero_explicit(&cmd, sizeof(cmd)) before returning, similar to how mlx5e_ktls_add_tx() clears its key material?
[ ... ]
Quoted 26 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c new file +26
-0+static int bnxt_ktls_dev_add (struct net_device *dev,struct sock *sk, +enum tls_offload_ctx_dir direction, +struct tls_crypto_info *crypto_info, +u32 start_offload_tcp_sn) +{ +struct bnxt_ktls_offload_ctx_tx *kctx_tx; +struct bnxt *bp =netdev_priv (dev); +struct bnxt_crypto_info *crypto; +struct tls_context *tls_ctx; +struct bnxt_tls_info *ktls; +struct bnxt_kctx *kctx; +u32 kid; +int rc; + +BUILD_BUG_ON (sizeof (struct bnxt_ktls_offload_ctx_tx ) > +TLS_DRIVER_STATE_SIZE_TX ); + + ktls = bp->ktls_info; +if (direction ==TLS_OFFLOAD_CTX_DIR_RX ) +return -EOPNOTSUPP ; + +if (!bnxt_ktls_cipher_supported (bp, crypto_info)) { +atomic64_inc (&ktls->counters[BNXT_KTLS_ERR_INVALID_CIPHER ]); +return -EOPNOTSUPP ; + } +
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c line 189
Quoted 10 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c new file +10
-0+atomic_inc (&ktls->pending); +/* Make sure bnxt_close_nic() sees pending before we check the +* BNXT_STATE_OPEN flag. +*/ +smp_mb__after_atomic (); +if (!test_bit (BNXT_STATE_OPEN , &bp->state)) { +atomic64_inc (&ktls->counters[BNXT_KTLS_ERR_STATE_NOT_OPEN ]); + rc = -ENODEV ; + goto exit; + }
[Medium, Low]
Two related questions about the ktls dereferences in this function.
bp->ktls_info is read once into ktls and then immediately dereferenced without a NULL guard. bnxt_ktls_init() (in a later patch in this series) only assigns dev->tlsdev_ops when bp->ktls_info is non-NULL, but does the driver ever clear dev->tlsdev_ops on teardown? bnxt_free_ktls_info() sets bp->ktls_info = NULL and is reachable from the FW-reset path via __bnxt_hwrm_func_qcaps() -> bnxt_alloc_crypto_info() -> bnxt_free_* without unregistering the netdev. Can a tls_dev_add arriving in that window dereference NULL?
Separately, the comment on smp_mb__after_atomic() says "pending is the gate that protects ktls" but the BNXT_KTLS_ERR_INVALID_CIPHER counter is incremented before atomic_inc(&ktls->pending). Should that counter update be moved past the pending barrier so it follows the same ordering as the other ktls dereferences in this function?
[ ... ]
Quoted 22 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c new file +22
-0+#define KTLS_RETRY_MAX 100 + +static void bnxt_ktls_dev_del (struct net_device *dev, +struct tls_context *tls_ctx, +enum tls_offload_ctx_dir direction) +{ +struct bnxt_ktls_offload_ctx_tx *kctx_tx; +struct bnxt *bp =netdev_priv (dev); +struct bnxt_crypto_info *crypto; +struct bnxt_tls_info *ktls; +struct bnxt_kctx *kctx; +int retry_cnt =0 ; +u8 kind; +u32 kid; + + ktls = bp->ktls_info; +retry: +while (!test_bit (BNXT_STATE_OPEN , &bp->state)) { +if (!netif_running (dev)) +return ; +/* Prevent infinite loop if device is down but netif_running() +* returns true.
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c line 248
Quoted 10 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c new file +10
-0+*/ +if (retry_cnt >KTLS_RETRY_MAX ) { +atomic64_inc (&ktls->counters[BNXT_KTLS_ERR_RETRY_EXCEEDED ]); +netdev_warn (dev,"%s retry max %d exceeded, state %lx\n" , + __func__, retry_cnt, bp->state); +return ; + } + retry_cnt++; +msleep (100 ); + }
[High]
This loop can sleep up to 100 * 100 ms = ~10 seconds while holding device_offload_lock from the net/tls caller. tls_device_down() takes that lock for write and walks every offloaded TLS context on the netdev, so a FW-reset that overlaps teardown of N TLS sockets could multiply the delay across all of them and stall every other tls_set_device_offload() on the system for that period.
Could this be coupled to the FW-reset completion path via wait_event() or a completion instead of an open-coded msleep retry on a state bit?
The fixed retry budget also means that if the device stays out of BNXT_STATE_OPEN longer than ~10 seconds, the function returns having done nothing -- the HW key context stays programmed and the in-driver KID slot is leaked until driver unload. Is that intended, and where is the leaked slot reclaimed in that case?
[ ... ]
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c line 270
Quoted 12 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c new file +12
-0+ crypto = bp->crypto_info; + kctx_tx =__tls_driver_ctx (tls_ctx,TLS_OFFLOAD_CTX_DIR_TX ); + kid = kctx_tx->kid; + kctx = &crypto->kctx[BNXT_TX_CRYPTO_KEY_TYPE ]; + kind =BNXT_CTX_KIND_CK_TX ; +atomic64_inc (&ktls->counters[BNXT_KTLS_TX_DEL ]); +if (bnxt_kid_valid (kctx, kid) && + !bnxt_crypto_del (bp, kctx->type, kind, kid)) +bnxt_free_one_kctx (kctx, kid); + +atomic_dec (&ktls->pending); +}
[Medium]bnxt_free_one_kctx() is only called when bnxt_crypto_del() returns 0. bnxt_crypto_del() can return -ENODEV when bnxt_select_mpc_ring() returns NULL, and any nonzero value from bnxt_xmit_crypto_cmd() (timeout, MPC ring busy, completion error, allocation failure).
In all of those cases the bit corresponding to this KID in kid->ids is never cleared back to "free", so the slot stays consumed. The close-time fallback bnxt_crypto_del_all_kids() also only sets the bit on rc == 0, so it does not reclaim slots that fail repeatedly either.
Can the KID pool slowly drain over runtime ifdown/ifup cycles with intermittent MPC errors? Should the slot be released on the local side even when the HW delete command fails?
[ ... ]
Michael Chan
maintainerPATCH v2 13/15 bnxt_en: Implement kTLS TX normal path
In reply to Michael ChanOffload TLS encryption of TX packets to the hardware if the TCP sequence number is the expected one. Fall back to software encryption otherwise. Implement all the TLS TX logic to check the TCP sequence number and set up the BD in the new function bnxt_ktls_xmit().
Basic kTLS statistics reporting for ethtool -S is also added. The next patches will add support for the exception path with out-of-order TCP sequence number.
--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 10 +++- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 39 +++++++++++++-- .../net/ethernet/broadcom/bnxt/bnxt_crypto.c | 1 + .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 45 +++++++++++++++++ drivers/net/ethernet/broadcom/bnxt/bnxt_gso.c | 2 +- .../net/ethernet/broadcom/bnxt/bnxt_ktls.c | 49 +++++++++++++++++++ .../net/ethernet/broadcom/bnxt/bnxt_ktls.h | 38 ++++++++++++++ 7 files changed, 177 insertions(+), 7 deletions(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +8 -2
@@ -488,6 +488,7 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)struct bnxt_sw_tx_bd *tx_buf;__le32 lflags =0 ;skb_frag_t *frag; +u32 kid =0 ; i =skb_get_queue_mapping (skb);if (unlikely (i >= bp->tx_nr_rings)) {
@@ -527,6 +528,10 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)return NETDEV_TX_BUSY ; } + skb =bnxt_ktls_xmit (bp, txr, skb, &lflags, &kid); +if (unlikely (!skb)) +return NETDEV_TX_OK ; + length = skb->len; len =skb_headlen (skb); last_frag =skb_shinfo (skb)->nr_frags;
@@ -675,7 +680,7 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev) prod =NEXT_TX (prod); txbd1 =bnxt_init_ext_bd (bp, txr, prod, lflags, vlan_tag_flags, - cfa_action); + cfa_action, kid);if (skb_is_gso (skb)) {bool udp_gso = !!(skb_shinfo (skb)->gso_type &SKB_GSO_UDP_L4 );
@@ -698,7 +703,8 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)TX_BD_FLAGS_T_IPID | (hdr_len << (TX_BD_HSIZE_SHIFT - 1))); length =skb_shinfo (skb)->gso_size; - txbd1->tx_bd_mss =cpu_to_le32 (length); + txbd1->tx_bd_kid_mss =cpu_to_le32 (BNXT_TX_KID_HI (kid) | + length); length += hdr_len; }else if (skb->ip_summed ==CHECKSUM_PARTIAL ) { txbd1->tx_bd_hsize_lflags |=
drivers/net/ethernet/broadcom/bnxt/bnxt.h +35 -4
@@ -101,10 +101,18 @@ struct tx_bd_ext {#define TX_BD_FLAGS_LSO (1 << 5)#define TX_BD_FLAGS_IPID_FMT (1 << 6)#define TX_BD_FLAGS_T_IPID (1 << 7) +#define TX_BD_FLAGS_CRYPTO_EN (1 << 15)#define TX_BD_HSIZE (0xff << 16)#define TX_BD_HSIZE_SHIFT 16 - -__le32 tx_bd_mss; +#define TX_BD_KID_LO (0x7f << 25) +#define TX_BD_KID_LO_MASK 0x7f +#define TX_BD_KID_LO_SHIFT 25 + +__le32 tx_bd_kid_mss; +#define TX_BD_MSS 0x7fff +#define TX_BD_KID_HI (0x1ffff << 15) +#define TX_BD_KID_HI_MASK 0xffff80 +#define TX_BD_KID_HI_SHIFT 8__le32 tx_bd_cfa_action;#define TX_BD_CFA_ACTION (0xffff << 16)#define TX_BD_CFA_ACTION_SHIFT 16
@@ -122,6 +130,16 @@ struct tx_bd_ext { };#define BNXT_TX_PTP_IS_SET (lflags) ((lflags) & cpu_to_le32(TX_BD_FLAGS_STAMP)) +#define BNXT_TX_KID_LO (kid) (((kid) & TX_BD_KID_LO_MASK) << TX_BD_KID_LO_SHIFT) +#define BNXT_TX_KID_HI (kid) (((kid) & TX_BD_KID_HI_MASK) << TX_BD_KID_HI_SHIFT) + +struct tx_bd_presync { +__le32 tx_bd_len_flags_type; +#define TX_BD_TYPE_PRESYNC_TX_BD (0x09 << 0) +u32 tx_bd_opaque; +__le32 tx_bd_kid; +u32 tx_bd_unused; +};struct rx_bd {__le32 rx_bd_len_flags_type;
@@ -1165,10 +1183,23 @@ struct bnxt_cmn_sw_stats {u64 missed_irqs; }; +/* Data plane kTLS counters */ +enum bnxt_ktls_data_counters { +BNXT_KTLS_TX_PKTS =0 , +BNXT_KTLS_TX_BYTES , + +BNXT_KTLS_MAX_DATA_COUNTERS , +}; + +struct bnxt_tls_sw_stats { +u64 counters[BNXT_KTLS_MAX_DATA_COUNTERS ]; +}; +struct bnxt_sw_stats {struct bnxt_rx_sw_stats rx;struct bnxt_tx_sw_stats tx;struct bnxt_cmn_sw_stats cmn; +struct bnxt_tls_sw_stats tls; };struct bnxt_total_ring_drv_stats {
@@ -2878,14 +2909,14 @@ static inline u32 bnxt_tx_avail(struct bnxt *bp,static inline struct tx_bd_ext *bnxt_init_ext_bd (struct bnxt *bp,struct bnxt_tx_ring_info *txr,u16 prod,__le32 lflags,u32 vlan_tag_flags, -u32 cfa_action) +u32 cfa_action,u32 kid) {struct tx_bd_ext *txbd1; txbd1 = (struct tx_bd_ext *) &txr->tx_desc_ring[TX_RING (bp, prod)][TX_IDX (prod)]; txbd1->tx_bd_hsize_lflags = lflags; - txbd1->tx_bd_mss =0 ; + txbd1->tx_bd_kid_mss =cpu_to_le32 (BNXT_TX_KID_HI (kid)); txbd1->tx_bd_cfa_meta =cpu_to_le32 (vlan_tag_flags); txbd1->tx_bd_cfa_action =cpu_to_le32 (cfa_action <<TX_BD_CFA_ACTION_SHIFT );
drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c +1 -0
@@ -533,6 +533,7 @@ int bnxt_crypto_init(struct bnxt *bp)if (rc)return rc; +bnxt_ktls_init (bp);return 0 ; }
drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c +45 -0
81 lines not on this page · raw message
drivers/net/ethernet/broadcom/bnxt/bnxt_gso.c +1 -1
@@ -168,7 +168,7 @@ netdev_tx_t bnxt_sw_udp_gso_xmit(struct bnxt *bp, prod =NEXT_TX (prod);bnxt_init_ext_bd (bp, txr, prod, csum, - vlan_tag_flags, cfa_action); + vlan_tag_flags, cfa_action,0 );/* set dma_unmap_len on the LAST BD touching each * region. Since completions are in-order, the last segment
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c +49 -0
58 lines not on this page · raw message
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.h +38 -0
@@ -63,6 +63,28 @@ struct ce_add_cmd {u8 addl_iv[8 ]; }; +struct crypto_prefix_cmd { +__le32 flags; +#define CRYPTO_PREFIX_CMD_FLAGS_UPDATE_IN_ORDER_VAR 0x1UL +#define CRYPTO_PREFIX_CMD_FLAGS_FULL_REPLAY_RETRAN 0x2UL +__le32 header_tcp_seq_num; +__le32 start_tcp_seq_num; +__le32 end_tcp_seq_num; +u8 explicit_nonce[8 ]; +u8 record_seq_num[8 ]; +}; + +#define CRYPTO_PREFIX_CMD_FLAGS_UPDATE_IN_ORDER_VAR_LE \ + cpu_to_le32(CRYPTO_PREFIX_CMD_FLAGS_UPDATE_IN_ORDER_VAR) + +#define CRYPTO_PREFIX_CMD_SIZE ((u32)sizeof(struct crypto_prefix_cmd)) +#define CRYPTO_PREFIX_CMD_BDS (CRYPTO_PREFIX_CMD_SIZE / sizeof(struct tx_bd)) +#define CRYPTO_PRESYNC_BDS (CRYPTO_PREFIX_CMD_BDS + 1) + +#define CRYPTO_PRESYNC_BD_CMD \ + (cpu_to_le32((CRYPTO_PREFIX_CMD_SIZE << TX_BD_LEN_SHIFT) | \ + TX_BD_CNT(CRYPTO_PRESYNC_BDS) | TX_BD_TYPE_PRESYNC_TX_BD)) +static inline bool bnxt_ktls_busy (struct bnxt *bp) {return bp->ktls_info &&atomic_read (&bp->ktls_info->pending) >0 ;
@@ -72,6 +94,9 @@ static inline bool bnxt_ktls_busy(struct bnxt *bp)int bnxt_alloc_ktls_info (struct bnxt *bp);void bnxt_free_ktls_info (struct bnxt *bp);int bnxt_ktls_init (struct bnxt *bp); +struct sk_buff *bnxt_ktls_xmit (struct bnxt *bp,struct bnxt_tx_ring_info *txr, +struct sk_buff *skb,__le32 *lflags,u32 *kid); +void bnxt_get_ring_tls_stats (struct bnxt *bp,struct bnxt_tls_sw_stats *stats);#else static inline int bnxt_alloc_ktls_info (struct bnxt *bp) {
@@ -86,5 +111,18 @@ static inline int bnxt_ktls_init(struct bnxt *bp) {return -EOPNOTSUPP ; } + +static inline struct sk_buff *bnxt_ktls_xmit (struct bnxt *bp, +struct bnxt_tx_ring_info *txr, +struct sk_buff *skb, +__le32 *lflags,u32 *kid) +{ +return skb; +} + +static inline void bnxt_get_ring_tls_stats (struct bnxt *bp, +struct bnxt_tls_sw_stats *stats) +{ +}#endif /* CONFIG_BNXT_TLS */ #endif /* BNXT_KTLS_H */
Signature · 1 line
2.51.0
Jakub Kicinski
maintainerRe: [PATCH net-next v2 13/15] bnxt_en: Implement kTLS TX normal path
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
Subject: bnxt_en: Implement kTLS TX normal path
This patch offloads TLS encryption of TX packets to hardware when the TCP sequence number matches what the driver expects, otherwise falling back to software encryption. It also adds the bnxt_ktls_xmit() helper and basic kTLS counters in ethtool -S. A few questions inline below.
Quoted 13 lines
drivers/net/ethernet/broadcom/bnxt/bnxt.c +1
-0@@ -488,6 +488,7 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)struct bnxt_sw_tx_bd *tx_buf;__le32 lflags =0 ;skb_frag_t *frag; +u32 kid =0 ; i =skb_get_queue_mapping (skb);if (unlikely (i >= bp->tx_nr_rings)) {
drivers/net/ethernet/broadcom/bnxt/bnxt.c static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev) · line 534
Quoted 7 lines
drivers/net/ethernet/broadcom/bnxt/bnxt.c +4
-0@@ -527,6 +528,10 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)return NETDEV_TX_BUSY ; } + skb =bnxt_ktls_xmit (bp, txr, skb, &lflags, &kid); +if (unlikely (!skb)) +return NETDEV_TX_OK ; +
[Low]
Should this drop path also bump the device tx_dropped counter? The adjacent drop paths in bnxt_start_xmit() (queue out of range, the skb_linearize() failure, and tx_kick_pending) all call dev_core_stats_tx_dropped_inc(dev) before returning NETDEV_TX_OK, but this new branch returns silently. tls_sw_fallback() has already freed the original skb when it returns NULL (kmalloc(GFP_ATOMIC) of sg_in, alloc_skb() of nskb, or fill_sg_in() can each fail), so these drops would otherwise be invisible in tx_dropped — exactly the counter operators look at when memory pressure causes TLS fallback to fail.
Quoted 13 lines
drivers/net/ethernet/broadcom/bnxt/bnxt.c +1
-1length = skb->len; len =skb_headlen (skb); last_frag =skb_shinfo (skb)->nr_frags;@@ -675,7 +680,7 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev) prod =NEXT_TX (prod); txbd1 =bnxt_init_ext_bd (bp, txr, prod, lflags, vlan_tag_flags, - cfa_action); + cfa_action, kid);if (skb_is_gso (skb)) {bool udp_gso = !!(skb_shinfo (skb)->gso_type &SKB_GSO_UDP_L4 );
drivers/net/ethernet/broadcom/bnxt/bnxt.c static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev) · line 707
Quoted 6 lines
drivers/net/ethernet/broadcom/bnxt/bnxt.c +2
-1@@ -698,7 +703,8 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)TX_BD_FLAGS_T_IPID | (hdr_len << (TX_BD_HSIZE_SHIFT - 1))); length =skb_shinfo (skb)->gso_size; - txbd1->tx_bd_mss =cpu_to_le32 (length); + txbd1->tx_bd_kid_mss =cpu_to_le32 (BNXT_TX_KID_HI (kid) | + length);
[Low]
Should length be masked with TX_BD_MSS here? The patch redefines
#define TX_BD_MSS 0x7fff
#define TX_BD_KID_HI (0x1ffff << 15)so the MSS field is now only the low 15 bits of the word, with the KID HI bits sharing the same 32-bit slot. BNXT_TX_KID_LO/HI both apply defensive masks (TX_BD_KID_LO_MASK, TX_BD_KID_HI_MASK) but the length term in this OR does not. If gso_size ever has bit 15 or higher set, those bits spill into KID_HI and corrupt the crypto context lookup on a TLS packet. Would something like
txbd1->tx_bd_kid_mss = cpu_to_le32(BNXT_TX_KID_HI(kid) |
(length & TX_BD_MSS));or a BUILD_BUG_ON-style guard be appropriate?
Quoted 8 lines
drivers/net/ethernet/broadcom/bnxt/bnxt.c +0
-0length += hdr_len; }else if (skb->ip_summed ==CHECKSUM_PARTIAL ) { txbd1->tx_bd_hsize_lflags |=drivers/net/ethernet/broadcom/bnxt/bnxt.h +0
-0
drivers/net/ethernet/broadcom/bnxt/bnxt.h struct tx_bd_ext { · line 106
Quoted 6 lines
drivers/net/ethernet/broadcom/bnxt/bnxt.h +1
-0@@ -101,10 +101,18 @@ struct tx_bd_ext {#define TX_BD_FLAGS_LSO (1 << 5)#define TX_BD_FLAGS_IPID_FMT (1 << 6)#define TX_BD_FLAGS_T_IPID (1 << 7) +#define TX_BD_FLAGS_CRYPTO_EN (1 << 15)#define TX_BD_HSIZE (0xff << 16)#define TX_BD_HSIZE_SHIFT 16
[Low]
Does placing TX_BD_FLAGS_CRYPTO_EN at bit 15 collide with the LSO hdr_len encoding in bnxt_start_xmit()? The LSO path writes:
txbd1->tx_bd_hsize_lflags |= cpu_to_le32(TX_BD_FLAGS_LSO |
TX_BD_FLAGS_T_IPID |
(hdr_len << (TX_BD_HSIZE_SHIFT - 1)));with TX_BD_HSIZE_SHIFT == 16, so this is hdr_len << 15. The LSB of hdr_len lands directly on bit 15 — the new CRYPTO_EN flag. Today all hdr_len sources are even (TCP doff*4, IPv4 base 20, IPv6 40, UDP 8, Ethernet 14, VLAN 4) so bit 0 is always 0, but the encoding contract is now fragile — any future encapsulation that yields an odd hdr_len or a refactor that changes the shift would silently set CRYPTO_EN with kid=0 on a non-TLS LSO segment. Would encoding as ((hdr_len >> 1) << TX_BD_HSIZE_SHIFT) (or masking with TX_BD_HSIZE) make this less prone to future breakage?
Quoted 11 lines
drivers/net/ethernet/broadcom/bnxt/bnxt.h +9
-2- -__le32 tx_bd_mss; +#define TX_BD_KID_LO (0x7f << 25) +#define TX_BD_KID_LO_MASK 0x7f +#define TX_BD_KID_LO_SHIFT 25 + +__le32 tx_bd_kid_mss; +#define TX_BD_MSS 0x7fff +#define TX_BD_KID_HI (0x1ffff << 15) +#define TX_BD_KID_HI_MASK 0xffff80 +#define TX_BD_KID_HI_SHIFT 8
[ ... ]
Quoted 8 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c +0
-0@@ -653,6 +682,16 @@ static void bnxt_get_ethtool_stats(struct net_device *dev,for (i =0 ; i <BNXT_NUM_RING_DRV_STATS ; i++, j++, curr++, prev++) buf[j] = *curr + *prev;
drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c static void bnxt_get_ethtool_stats(struct net_device *dev, · line 694
Quoted 10 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c +10
-0+if (bp->ktls_info) { +struct bnxt_tls_info *ktls = bp->ktls_info; +struct bnxt_tls_sw_stats tls_stats = {}; + +bnxt_get_ring_tls_stats (bp, &tls_stats); +for (i =0 ; i <ARRAY_SIZE (bnxt_ktls_data_stats); i++, j++) + buf[j] = tls_stats.counters[i]; +for (i =0 ; i <ARRAY_SIZE (bnxt_ktls_ctrl_stats); i++, j++) + buf[j] =atomic64_read (&ktls->counters[i]); + }
[High]
Can this block dereference a freed bp->bnapi when ethtool -S runs on a closed interface? The function starts with:
if (!bp->bnapi) {
j += bnxt_get_num_ring_stats(bp);
goto skip_ring_stats;
}skip_ring_stats:
[ ... existing ring drv stats accumulator ... ]
if (bp->ktls_info) {
...
bnxt_get_ring_tls_stats(bp, &tls_stats);bp->bnapi is freed and cleared in bnxt_free_mem() during interface close, but bp->ktls_info has a longer lifetime — it is allocated at probe and freed only at remove. The new ktls block sits after the skip_ring_stats label, so it executes even when the !bp->bnapi guard fired. bnxt_get_ring_tls_stats() then loops bp->cp_nr_rings (which is not zeroed on close) and dereferences bp->bnapi[i]:
for (i = 0; i < bp->cp_nr_rings; i++) {
ring_stats = &bp->bnapi[i]->cp_ring.sw_stats->tls;Should the ktls block be guarded by the same bp->bnapi check, and/or should bnxt_get_ring_tls_stats() bail out when bp->bnapi is NULL?
Quoted 28 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c +0
-0if (bp->flags &BNXT_FLAG_PORT_STATS ) {u64 *port_stats = bp->port_stats.sw_stats;drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c +17
-0@@ -287,3 +288,51 @@ int bnxt_ktls_init(struct bnxt *bp) dev->features |=NETIF_F_HW_TLS_TX ;return 0 ; } + +struct sk_buff *bnxt_ktls_xmit (struct bnxt *bp,struct bnxt_tx_ring_info *txr, +struct sk_buff *skb,__le32 *lflags,u32 *kid) +{ +struct bnxt_sw_stats *sw_stats = txr->tx_cpr->sw_stats; +struct bnxt_tls_info *ktls = bp->ktls_info; +struct bnxt_ktls_offload_ctx_tx *kctx_tx; +struct tls_context *tls_ctx; +u32 seq, payload_len; + +if (!IS_ENABLED (CONFIG_TLS_DEVICE ) || !ktls || + !tls_is_skb_tx_device_offloaded (skb)) +return skb; + + seq =ntohl (tcp_hdr (skb)->seq); + tls_ctx =tls_get_ctx (skb->sk); + kctx_tx =__tls_driver_ctx (tls_ctx,TLS_OFFLOAD_CTX_DIR_TX );
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c int bnxt_ktls_init(struct bnxt *bp) · line 317
Quoted 10 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c +10
-0+ payload_len = skb->len -skb_tcp_all_headers (skb); +if (!payload_len) +return skb; +if (kctx_tx->tcp_seq_no == seq) { + kctx_tx->tcp_seq_no += payload_len; + *kid =BNXT_KID_HW (kctx_tx->kid); + *lflags |=cpu_to_le32 (TX_BD_FLAGS_CRYPTO_EN | +BNXT_TX_KID_LO (*kid)); + sw_stats->tls.counters[BNXT_KTLS_TX_PKTS ]++; + sw_stats->tls.counters[BNXT_KTLS_TX_BYTES ] += payload_len;
[High]
Can advancing kctx_tx->tcp_seq_no here desynchronise driver and HW state if the packet is dropped later in bnxt_start_xmit()? The sequence and stats are bumped before the BD is built, but the caller can still drop the skb after this returns — skb_pad() failure goes to tx_kick_pending, dma_map_single() failure to tx_free, the bnxt_lhint_arr oversize check and skb_frag_dma_map() failure to tx_dma_error. On any of those drops the BD never reaches HW, so the HW-side TCP sequence variable remains at the old position, while the driver-tracked kctx_tx->tcp_seq_no has already advanced by payload_len.
The next aligned TLS data send will then hit:
if (kctx_tx->tcp_seq_no == seq) {
...
*lflags |= cpu_to_le32(TX_BD_FLAGS_CRYPTO_EN | ...);with seq matching the inflated tracker, so the driver flags it as in-order and HW encrypts from a stale TLS record state. The follow-up patch that adds bnxt_ktls_tx_ooo() handles seq < kctx_tx->tcp_seq_no via tls_get_record(), but does not appear to detect this driver-ahead case.
Would it be safer to defer the kctx_tx->tcp_seq_no advance and the counter updates until after the BD is committed (near the doorbell), or to roll them back on the failure paths?
[Medium]
Are these u64 increments safe against concurrent xmit on different TX queues that share the same sw_stats? bnxt_alloc_cp_rings() aliases all sub-CP rings of a bnapi:
cpr2->sw_stats = cpr->sw_stats;
so multiple bnxt_tx_ring_info entries in one bnapi have distinct txr->tx_cpr but the same txr->tx_cpr->sw_stats. Different netdev TX queues hold different __netif_tx_lock instances, so two CPUs can transmit concurrently on TX queues mapped to the same bnapi and race on these counters with plain ++ and +=. The reader bnxt_get_ring_tls_stats() also reads them from process context with no synchronisation, and on 32-bit the BNXT_KTLS_TX_BYTES u64 update is a torn read/write. The pre-existing bnxt_sw_stats fields are only written from a single NAPI context per bnapi, which is why the aliasing was previously safe; the new TLS counters introduce a new multi-writer pattern.
Would u64_stats_sync (or per-CPU + SNMP_ADD_STATS64-style helpers) be more appropriate here, in line with the networking guidance for shared per-bnapi 64-bit counters?
Quoted 21 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c +21
-0+ }else { + skb =tls_encrypt_skb (skb); +if (!skb) +return NULL ; + } +return skb; +} + +void bnxt_get_ring_tls_stats (struct bnxt *bp,struct bnxt_tls_sw_stats *stats) +{ +struct bnxt_tls_sw_stats *ring_stats; +int i, j; + +if (!bp->ktls_info) +return ; +for (i =0 ; i < bp->cp_nr_rings; i++) { + ring_stats = &bp->bnapi[i]->cp_ring.sw_stats->tls; +for (j =0 ; j <BNXT_KTLS_MAX_DATA_COUNTERS ; j++) + stats->counters[j] += ring_stats->counters[j]; + } +}
[ ... ]
Michael Chan
maintainerPATCH v2 14/15 bnxt_en: Add support for inline transmit BDs
In reply to Michael ChanNewer chips (P5_PLUS) support inline transmit BDs that contain extra data. One such use case is to transmit out-of-sequence kTLS packets with encryption enabled. To account for these inline BDs during TX completion, we add the inline_data_bds field to struct bnxt_sw_tx_bd (tx_buf). tx_buf->is_push will always be set when sending these inline BDs as the operation is similar to push packets. tx_buf->skb will always be NULL as there is no associated SKB.
The next patch will make use of this feature.
--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 15 ++++++++++++--- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 1 + 2 files changed, 13 insertions(+), 3 deletions(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +12 -3
@@ -842,7 +842,7 @@ static bool __bnxt_tx_int(struct bnxt *bp, struct bnxt_tx_ring_info *txr, head_buf = tx_buf; skb = tx_buf->skb; -if (unlikely (!skb)) { +if (unlikely (!skb && !tx_buf->is_push)) {bnxt_sched_reset_txr (bp, txr, cons);return rc; }
@@ -854,13 +854,22 @@ static bool __bnxt_tx_int(struct bnxt *bp, struct bnxt_tx_ring_info *txr, } cons =NEXT_TX (cons); - tx_pkts++; - tx_bytes += skb->len; +if (skb) { + tx_pkts++; + tx_bytes += skb->len; + } tx_buf->skb =NULL ; tx_buf->is_ts_pkt =0 ;if (tx_buf->is_push) { tx_buf->is_push =0 ; + cons += tx_buf->inline_data_bds; + tx_buf->inline_data_bds =0 ; +if (!skb) { +/* presync BD */ + cons =NEXT_TX (cons); +continue ; + } goto next_tx_int; }
drivers/net/ethernet/broadcom/bnxt/bnxt.h +1 -0
@@ -917,6 +917,7 @@ struct bnxt_sw_tx_bd {u8 is_push;u8 is_sw_gso;u8 action; +u8 inline_data_bds;unsigned short nr_frags;union {u16 rx_prod;
Signature · 1 line
2.51.0
Michael Chan
maintainerPATCH v2 15/15 bnxt_en: Add kTLS retransmission support
In reply to Michael ChanIf TCP retransmits a TLS packet that requires encryption by the NIC, the TCP sequence number will go backwards and the hardware will require some assistance from the driver. The driver needs to retrieve the TLS record that covers the byte sequence of the retransmitted packet. If the retransmitted packet does not include the tag, the hardware can simply encrypt the packet using the informtaion in the TLS record.
The driver provides the TLS record information for the retransmitted packet in the presync TX BD. The presync TX BD introduced in the last patch is treated very much like a TX push BD with inline data. The only exception is that no SKB will be stored for the presync TX BD.
Retransmission that includes the TLS tag will be handled in future patches.
--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 2 +- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 2 + .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 2 + .../net/ethernet/broadcom/bnxt/bnxt_ktls.c | 126 +++++++++++++++++- 4 files changed, 128 insertions(+), 4 deletions(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +1 -1
@@ -499,7 +499,6 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev) txq =netdev_get_tx_queue (dev, i); txr = &bp->tx_ring[bp->tx_ring_map[i]]; - prod = txr->tx_prod;#if (MAX_SKB_FRAGS >TX_MAX_FRAGS )if (skb_shinfo (skb)->nr_frags >TX_MAX_FRAGS ) {
@@ -532,6 +531,7 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)if (unlikely (!skb))return NETDEV_TX_OK ; + prod = txr->tx_prod; length = skb->len; len =skb_headlen (skb); last_frag =skb_shinfo (skb)->nr_frags;
drivers/net/ethernet/broadcom/bnxt/bnxt.h +2 -0
@@ -1188,6 +1188,8 @@ struct bnxt_cmn_sw_stats {enum bnxt_ktls_data_counters {BNXT_KTLS_TX_PKTS =0 ,BNXT_KTLS_TX_BYTES , +BNXT_KTLS_TX_OOO_PKTS , +BNXT_KTLS_TX_DROP_NO_SYNC ,BNXT_KTLS_MAX_DATA_COUNTERS , };
drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c +2 -0
@@ -359,6 +359,8 @@ static const char *const bnxt_ring_drv_stats_arr[] = {static const char *const bnxt_ktls_data_stats[] = { [BNXT_KTLS_TX_PKTS ] ="tx_tls_encrypted_packets" , [BNXT_KTLS_TX_BYTES ] ="tx_tls_encrypted_bytes" , + [BNXT_KTLS_TX_OOO_PKTS ] ="tx_tls_ooo_packets" , + [BNXT_KTLS_TX_DROP_NO_SYNC ] ="tx_tls_drop_no_sync" , };/* kTLS control plane counter strings indexed by enum bnxt_ktls_ctrl_counters */
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c +123 -3
144 lines not on this page · raw message
Signature · 1 line
2.51.0
Jakub Kicinski
maintainerRe: [PATCH net-next v2 15/15] bnxt_en: Add kTLS retransmission support
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---bnxt_en: Add kTLS retransmission support
Adds a TX path for kTLS retransmissions that retrieves the TLS record covering a retransmitted byte range and emits a presync TX BD carrying the crypto prefix command so the NIC can re-encrypt the packet. The patch also adds two new ethtool -S counters (tx_tls_ooo_packets and tx_tls_drop_no_sync) and reorders the read of txr->tx_prod in bnxt_start_xmit() so it happens after bnxt_ktls_xmit() may have advanced the producer.
A few questions on the new code below.
Quoted 13 lines
drivers/net/ethernet/broadcom/bnxt/bnxt.c +0
-1@@ -499,7 +499,6 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev) txq =netdev_get_tx_queue (dev, i); txr = &bp->tx_ring[bp->tx_ring_map[i]]; - prod = txr->tx_prod;#if (MAX_SKB_FRAGS >TX_MAX_FRAGS )if (skb_shinfo (skb)->nr_frags >TX_MAX_FRAGS ) {
drivers/net/ethernet/broadcom/bnxt/bnxt.c static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev) · line 537
Quoted 7 lines
drivers/net/ethernet/broadcom/bnxt/bnxt.c +1
-0@@ -532,6 +531,7 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)if (unlikely (!skb))return NETDEV_TX_OK ; + prod = txr->tx_prod; length = skb->len; len =skb_headlen (skb); last_frag =skb_shinfo (skb)->nr_frags;
[High]
What unwinds the presync BDs if a DMA mapping fails later in bnxt_start_xmit()?
By the time bnxt_ktls_xmit() returns, bnxt_ktls_tx_ooo() has already called bnxt_ktls_pre_xmit() which advances txr->tx_prod past the CRYPTO_PRESYNC_BDS presync entries and (when fwd is true) updates kctx_tx->tcp_seq_no = end_seq. After that, bnxt_start_xmit() still has to dma_map_single(skb->data, ...) and skb_frag_dma_map() each fragment.
If either mapping fails, control jumps to tx_dma_error / tx_free / tx_kick_pending, which clears only the head data BD slot at txr->tx_prod and does not move tx_prod back past the presync BDs nor revert kctx_tx->tcp_seq_no.
Once a subsequent successful TX advances tx_prod and rings the doorbell, won't HW read the orphan presync BDs and apply the captured TLS crypto state to whichever data BDs follow, even if those belong to an unrelated packet on the ring?
drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c static const char *const bnxt_ring_drv_stats_arr[] = { · line 364
Quoted 11 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c +2
-0@@ -359,6 +359,8 @@ static const char *const bnxt_ring_drv_stats_arr[] = {static const char *const bnxt_ktls_data_stats[] = { [BNXT_KTLS_TX_PKTS ] ="tx_tls_encrypted_packets" , [BNXT_KTLS_TX_BYTES ] ="tx_tls_encrypted_bytes" , + [BNXT_KTLS_TX_OOO_PKTS ] ="tx_tls_ooo_packets" , + [BNXT_KTLS_TX_DROP_NO_SYNC ] ="tx_tls_drop_no_sync" , };
[Low]
Is the "tx_tls_drop_no_sync" name accurate? BNXT_KTLS_TX_DROP_NO_SYNC is bumped in bnxt_ktls_tx_ooo() when tls_get_record() returns NULL or a record with num_frags == 0, after which -EPROTO is returned and the caller falls back to tls_encrypt_skb() rather than dropping the packet. Would something like "tx_tls_resync_no_record" be more descriptive of what the counter actually counts, given that ethtool -S strings are hard to rename once shipped?
Quoted 5 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c +0
-0@@ -289,6 +289,116 @@ int bnxt_ktls_init(struct bnxt *bp)
[ ... ]
Quoted 4 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c +4
-0+static int bnxt_ktls_tx_ooo (struct bnxt *bp,struct bnxt_tx_ring_info *txr, +struct sk_buff *skb,u32 payload_len,u32 seq, +struct tls_context *tls_ctx) +{
[ ... ]
Quoted 10 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c +10
-0+spin_lock_irqsave (&tx_tls_ctx->lock, flags); + record =tls_get_record (tx_tls_ctx, seq, &rec_sn); +if (!record || !record->num_frags) { + rc = -EPROTO ; + sw_stats->tls.counters[BNXT_KTLS_TX_DROP_NO_SYNC ]++; + goto unlock_exit; + } + hdr_tcp_seq =tls_record_start_seq (record); + hdr =skb_frag_address_safe (&record->frags[0 ]); +
[ ... ]
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c int bnxt_ktls_init(struct bnxt *bp) · line 388
Quoted 8 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c +8
-0+if (tls_ctx->prot_info.version ==TLS_1_2_VERSION ) { +u32 nonce_bytes = tls_ctx->prot_info.iv_size; +u32 retrans_off = seq - hdr_tcp_seq; + +if (retrans_off >5 && retrans_off <5 + nonce_bytes) + nonce_bytes = retrans_off -5 ; +memcpy (pcmd.explicit_nonce, hdr +5 , nonce_bytes); + }
[High]
Can hdr be NULL here? skb_frag_address_safe() is documented to return NULL when page_address(page) returns NULL, e.g. for highmem pages on 32-bit kernels with CONFIG_HIGHMEM that have not been kmap'd.
TLS records are allocated via sk_page_frag_refill() with the socket's GFP flags, which can produce highmem pages, and the existing in-tree caller in bnxt_start_xmit() does:
fptr = skb_frag_address_safe(frag); if (!fptr) goto normal_tx;
For TLS 1.2, hdr + 5 is then dereferenced unconditionally in memcpy(pcmd.explicit_nonce, hdr + 5, nonce_bytes). Should there be a NULL check here too, similar to the bnxt_start_xmit() pattern?
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c int bnxt_ktls_init(struct bnxt *bp) · line 389
Quoted 1 line
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c +1
-0+memcpy (&pcmd.record_seq_num[0 ], &rec_sn,sizeof (rec_sn));
[High]
Does this copy the record sequence number with the byte order the NIC expects on big-endian hosts?
rec_sn comes from tls_get_record() which returns offload_ctx-> unacked_record_sn, a host-byte-order u64 initialised via be64_to_cpu(rcd_sn) - 1 in net/tls/tls_device.c and incremented as plain C arithmetic. The steady-state setup path bnxt_crypto_add() populates the same record_seq_num field via bnxt_copy_tls_mp_data() which is commented "Copy in reverse byte order" and reverses the wire-format big-endian aes->rec_seq into little-endian.
On a little-endian host memcpy of rec_sn yields LE bytes, which matches the HW format. On a big-endian host memcpy yields BE bytes, so the NIC sees the wrong record sequence number for the AEAD computation on every retransmission. Would something like
__le64 le_rec_sn = cpu_to_le64(rec_sn); memcpy(&pcmd.record_seq_num[0], &le_rec_sn, sizeof(le_rec_sn));
be more correct?
Quoted 14 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c +11
-0+ rc =0 ; +bnxt_ktls_pre_xmit (bp, txr, kctx_tx->kid, &pcmd); + +if (fwd) + kctx_tx->tcp_seq_no = end_seq; + +unlock_exit: +spin_unlock_irqrestore (&tx_tls_ctx->lock, flags); +return rc; +} +struct sk_buff *bnxt_ktls_xmit (struct bnxt *bp,struct bnxt_tx_ring_info *txr,struct sk_buff *skb,__le32 *lflags,u32 *kid) {
[ ... ]
Quoted 12 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c +5
-3@@ -316,9 +427,18 @@ struct sk_buff *bnxt_ktls_xmit(struct bnxt *bp, struct bnxt_tx_ring_info *txr, sw_stats->tls.counters[BNXT_KTLS_TX_PKTS ]++; sw_stats->tls.counters[BNXT_KTLS_TX_BYTES ] += payload_len; }else { - skb =tls_encrypt_skb (skb); -if (!skb) -return NULL ; + sw_stats->tls.counters[BNXT_KTLS_TX_OOO_PKTS ]++; + + rc =bnxt_ktls_tx_ooo (bp, txr, skb, payload_len, seq, tls_ctx); +if (rc) +return tls_encrypt_skb (skb);
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c struct sk_buff *bnxt_ktls_xmit(struct bnxt *bp, struct bnxt_tx_ring_info *txr, · line 444
Quoted 10 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c +7
-0+ + *kid =BNXT_KID_HW (kctx_tx->kid); + *lflags |=cpu_to_le32 (TX_BD_FLAGS_CRYPTO_EN | +BNXT_TX_KID_LO (*kid)); + sw_stats->tls.counters[BNXT_KTLS_TX_PKTS ]++; + sw_stats->tls.counters[BNXT_KTLS_TX_BYTES ] += payload_len; +return skb; }return skb; }
[Low]
How is an operator expected to break the OOO counter down by outcome?
BNXT_KTLS_TX_OOO_PKTS is incremented unconditionally before calling bnxt_ktls_tx_ooo(), so it counts every OOO attempt. On success the same packet then bumps BNXT_KTLS_TX_PKTS and BNXT_KTLS_TX_BYTES, while on -EPROTO it also bumps BNXT_KTLS_TX_DROP_NO_SYNC.
The two SW-fallback cases (-ENOSPC for ring-full and -EOPNOTSUPP for tag-bearing retransmissions) are not visible in any counter. Given the commit message defers tag-bearing retransmissions to future patches, -EOPNOTSUPP is expected to be the principal SW-fallback cause for now.
Would it make sense to either move the OOO_PKTS increment to the success branch or add per-failure-cause counters such as tx_tls_ooo_fallback_no_sync, tx_tls_ooo_fallback_tag_retrans and tx_tls_ooo_fallback_no_space, before the strings are picked up by tooling?
Michael Chan
maintainerRe: [PATCH net-next v2 15/15] bnxt_en: Add kTLS retransmission support
Quoted 18 lines · Jakub Kicinski
A few questions on the new code below.
Quoted 16 lines
drivers/net/ethernet/broadcom/bnxt/bnxt.c +0
-1@@ -499,7 +499,6 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev) txq =netdev_get_tx_queue (dev, i); txr = &bp->tx_ring[bp->tx_ring_map[i]]; - prod = txr->tx_prod;#if (MAX_SKB_FRAGS >TX_MAX_FRAGS )if (skb_shinfo (skb)->nr_frags >TX_MAX_FRAGS ) {@@ -532,6 +531,7 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)if (unlikely (!skb))return NETDEV_TX_OK ;
Quoted 9 lines
drivers/net/ethernet/broadcom/bnxt/bnxt.c static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev) · line 537
Quoted 4 lines
drivers/net/ethernet/broadcom/bnxt/bnxt.c +1
-0+ prod = txr->tx_prod; length = skb->len; len =skb_headlen (skb); last_frag =skb_shinfo (skb)->nr_frags;[High]
What unwinds the presync BDs if a DMA mapping fails later inbnxt_start_xmit()?
The unwind is indeed missing. The cleanup path (e.g. during TX timeout) is also missing the proper accounting for the extra inline BDs. I will fix it in v3.
Quoted 8 lines
[Low]
Is the "tx_tls_drop_no_sync" name accurate?BNXT_KTLS_TX_DROP_NO_SYNCis bumped inbnxt_ktls_tx_ooo()whentls_get_record()returns NULL or a record withnum_frags== 0, after which-EPROTOis returned and the caller falls back totls_encrypt_skb()rather than dropping the packet. Would something like "tx_tls_resync_no_record" be more descriptive of what the counter actually counts, given that ethtool -S strings are hard to rename once shipped?
I will go over these counters and make any necessary adjustments.
Quoted 10 lines
[ ... ]
drivers/net/ethernet/broadcom/bnxt/bnxt.c
Quoted 8 lines
drivers/net/ethernet/broadcom/bnxt/bnxt.c +8
-0+if (tls_ctx->prot_info.version ==TLS_1_2_VERSION ) { +u32 nonce_bytes = tls_ctx->prot_info.iv_size; +u32 retrans_off = seq - hdr_tcp_seq; + +if (retrans_off >5 && retrans_off <5 + nonce_bytes) + nonce_bytes = retrans_off -5 ; +memcpy (pcmd.explicit_nonce, hdr +5 , nonce_bytes); + }
Quoted 4 lines
[High]
Can hdr be NULL here?skb_frag_address_safe()is documented to return NULL whenpage_address(page)returns NULL, e.g. for highmem pages on 32-bit kernels withCONFIG_HIGHMEMthat have not been kmap'd.
I think that's right. For 32-bit highmem, it can be NULL so I need to check for NULL.
Quoted 6 lines
Quoted 2 lines
+
memcpy(&pcmd.record_seq_num[0], &rec_sn, sizeof(rec_sn));[High]
Does this copy the record sequence number with the byte order the NIC expects on big-endian hosts?
Endian conversion is indeed needed here.
Quoted 3 lines
[Low]
How is an operator expected to break the OOO counter down by outcome?
The TX path is not quite complete and some TX counters are deferred to future patches. But I will make them more complete for this patchset in v3. I will also review all the other AI comments and make changes for v3. Thanks.