eth: bnxt: improve the timing of stats
Jakub Kicinski
maintainerPATCH v2 1/1 eth: bnxt: improve the timing of stats
Kernel selftests wait 1.25x of the promised stats refresh time (as read from ethtool -c). bnxt reports 1sec by default, but the stats update process has two steps. First device DMAs the new values, then the service task performs update in full-width SW counters. So the worst case delay is actually 2x.
Note that the behavior is different for ring stats and port stats. Port stats are fetched synchronously by the service worker, so there's no risk of doubling up the delay there.
The problem of stale stats impacts not only tests but real workloads which monitor egress bandwidth of a NIC. The inaccuracy causes double counting in the next cycle and spurious overload alarms.
Try to read from the DMA buffer more aggressively, to mitigate timing issues between DMA and service task. The SW update should be cheap.
---
CC: [email protected]
CC: [email protected]
v2:
- split the accumulate into port and ring
- make the sync only cover rings
- remove sync from callbacks which use port stats (which are fetched synchronously by the service worker)
v1: https://lore.kernel.org/[email protected]
With this patch I had a 50 clean runs of ntuple.py in a row. Previously it'd fail within 5 runs at most.
Hopefully this is good enough, in the past I sent an RFC to convert the driver to use SW stats for everything. That felt a little drastic.
--- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 5 ++ drivers/net/ethernet/broadcom/bnxt/bnxt.c | 48 ++++++++++++++++++- .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 1 + 3 files changed, 53 insertions(+), 1 deletion(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.h +5 -0
@@ -2620,6 +2620,10 @@ struct bnxt {#define BNXT_MIN_STATS_COAL_TICKS 250000#define BNXT_MAX_STATS_COAL_TICKS 1000000 +/* Protects stats_updated_jiffies and writes to sw_stats */ +spinlock_t stats_lock; +unsigned long stats_updated_jiffies; +struct work_struct sp_task;unsigned long sp_event;#define BNXT_RX_NTP_FLTR_SP_EVENT 1
@@ -3027,6 +3031,7 @@ void bnxt_reenable_sriov(struct bnxt *bp);void bnxt_close_nic (struct bnxt *,bool ,bool );void bnxt_get_ring_drv_stats (struct bnxt *bp,struct bnxt_total_ring_drv_stats *stats); +void bnxt_sync_ring_stats (struct bnxt *bp);bool bnxt_rfs_capable (struct bnxt *bp,bool new_rss_ctx);int bnxt_dbg_hwrm_rd_reg (struct bnxt *bp,u32 reg_off,u16 num_words,u32 *reg_buf);
drivers/net/ethernet/broadcom/bnxt/bnxt.c +47 -1
97 lines not on this page · raw message
drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c +1 -0
@@ -606,6 +606,7 @@ static void bnxt_get_ethtool_stats(struct net_device *dev, goto skip_ring_stats; } +bnxt_sync_ring_stats (bp); tpa_stats =bnxt_get_num_tpa_ring_stats (bp);for (i =0 ; i < bp->cp_nr_rings; i++) {struct bnxt_napi *bnapi = bp->bnapi[i];
Signature · 1 line
2.54.0
Quoted 15 lines · Jakub Kicinski
Kernel selftests wait 1.25x of the promised stats refresh time (as read from ethtool -c). bnxt reports 1sec by default, but the stats update process has two steps. First device DMAs the new values, then the service task performs update in full-width SW counters. So the worst case delay is actually 2x.
Note that the behavior is different for ring stats and port stats. Port stats are fetched synchronously by the service worker, so there's no risk of doubling up the delay there.
The problem of stale stats impacts not only tests but real workloads which monitor egress bandwidth of a NIC. The inaccuracy causes double counting in the next cycle and spurious overload alarms.
Quoted 6 lines
Try to read from the DMA buffer more aggressively, to mitigate timing issues between DMA and service task. The SW update should be cheap.
Fixes 51f307856b60 ("bnxt_en: Allow statistics DMA to be configurable using ethtool -C.")Signed-off-by Jakub Kicinski <[email protected]>
Thanks.
This patch was applied to netdev/net.git (main) by Jakub Kicinski <[email protected]>:
Hello:
This patch was applied to netdev/net.git (main)
by Jakub Kicinski <[email protected]>:
Quoted 11 lines · On Fri, 19 Jun 2026 12:15:38 -0700 you wrote:
Kernel selftests wait 1.25x of the promised stats refresh time (as read from ethtool -c). bnxt reports 1sec by default, but the stats update process has two steps. First device DMAs the new values, then the service task performs update in full-width SW counters. So the worst case delay is actually 2x.
Note that the behavior is different for ring stats and port stats. Port stats are fetched synchronously by the service worker, so there's no risk of doubling up the delay there.
[...]
Here is the summary with links:
- [net,v2] eth: bnxt: improve the timing of stats
https://git.kernel.org/netdev/net/c/40529e58629b
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html