net: xilinx: axienet: Add statistics support
v3 cover net: xilinx: axienet: Add statistics support
Add support for hardware statistics counters (if they are enabled) in the AXI Ethernet driver. Unfortunately, the implementation is complicated a bit since the hardware might only support 32-bit counters.
Changes in v3:
- Use explicit mutex_lock/unlock instead of guard() in __axienet_device_reset to make it clear that we need to hold lp->stats_lock for the whole function.
Changes in v2:
- Switch to a seqlock-based implementation to allow fresher updates (rather than always using stale counter values from the previous refresh).
- Take stats_lock unconditionally in __axienet_device_reset
- Fix documentation mismatch
Sean Anderson (2):
net: xilinx: axienet: Report RxRject as rx_dropped
net: xilinx: axienet: Add statistics support
drivers/net/ethernet/xilinx/xilinx_axienet.h | 85 ++++++ .../net/ethernet/xilinx/xilinx_axienet_main.c | 267 +++++++++++++++++- 2 files changed, 348 insertions(+), 4 deletions(-)
Signature · 1 line
2.35.1.1320.gc452695387.dirty
PATCH v3 1/2 net: xilinx: axienet: Report RxRject as rx_dropped
The Receive Frame Rejected interrupt is asserted whenever there was a receive error (bad FCS, bad length, etc.) or whenever the frame was dropped due to a mismatched address. So this is really a combination of rx_otherhost_dropped, rx_length_errors, rx_frame_errors, and rx_crc_errors. Mismatched addresses are common and aren't really errors at all (much like how fragments are normal on half-duplex links). To avoid confusion, report these events as rx_dropped. This better reflects what's going on: the packet was received by the MAC but dropped before being processed.
---
(no changes since v1)
drivers/net/ethernet/xilinx/xilinx_axienet_main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/xilinx/xilinx_axienet_main.c +1 -1
@@ -1296,7 +1296,7 @@ static irqreturn_t axienet_eth_irq(int irq, void *_ndev) ndev->stats.rx_missed_errors++;if (pending &XAE_INT_RXRJECT_MASK ) - ndev->stats.rx_frame_errors++; + ndev->stats.rx_dropped++;axienet_iow (lp,XAE_IS_OFFSET , pending);return IRQ_HANDLED ;
Signature · 1 line
2.35.1.1320.gc452695387.dirty
PATCH v3 2/2 net: xilinx: axienet: Add statistics support
In reply to Sean AndersonAdd support for reading the statistics counters, if they are enabled. The counters may be 64-bit, but we can't detect this statically as there's no ability bit for it and the counters are read-only. Therefore, we assume the counters are 32-bits by default. To ensure we don't miss an overflow, we read all counters at 13-second intervals. This should be often enough to ensure the bytes counters don't wrap at 2.5 Gbit/s.
Another complication is that the counters may be reset when the device is reset (depending on configuration). To ensure the counters persist across link up/down (including suspend/resume), we maintain our own versions along with the last counter value we saw. Because we might wait up to 100 ms for the reset to complete, we use a mutex to protect writing hw_stats. We can't sleep in ndo_get_stats64, so we use a seqlock to protect readers.
We don't bother disabling the refresh work when we detect 64-bit counters. This is because the reset issue requires us to read hw_stat_base and reset_in_progress anyway, which would still require the seqcount. And I don't think skipping the task is worth the extra bookkeeping.
We can't use the byte counters for either get_stats64 or get_eth_mac_stats. This is because the byte counters include everything in the frame (destination address to FCS, inclusive). But rtnl_link_stats64 wants bytes excluding the FCS, and ethtool_eth_mac_stats wants to exclude the L2 overhead (addresses and length/type). It might be possible to calculate the byte values Linux expects based on the frame counters, but I think it is simpler to use the existing software counters.
---
Changes in v3:
- Use explicit mutex_lock/unlock instead of guard() in __axienet_device_reset to make it clear that we need to hold lp->stats_lock for the whole function.
Changes in v2:
- Switch to a seqlock-based implementation to allow fresher updates (rather than always using stale counter values from the previous refresh).
- Take stats_lock unconditionally in __axienet_device_reset
- Fix documentation mismatch
drivers/net/ethernet/xilinx/xilinx_axienet.h | 85 ++++++ .../net/ethernet/xilinx/xilinx_axienet_main.c | 265 +++++++++++++++++- 2 files changed, 347 insertions(+), 3 deletions(-)
drivers/net/ethernet/xilinx/xilinx_axienet.h +85 -0
@@ -156,6 +156,7 @@#define XAE_TPID0_OFFSET 0x00000028/* VLAN TPID0 register */ #define XAE_TPID1_OFFSET 0x0000002C/* VLAN TPID1 register */ #define XAE_PPST_OFFSET 0x00000030/* PCS PMA Soft Temac Status Reg */ +#define XAE_STATS_OFFSET 0x00000200/* Statistics counters */ #define XAE_RCW0_OFFSET 0x00000400/* Rx Configuration Word 0 */ #define XAE_RCW1_OFFSET 0x00000404/* Rx Configuration Word 1 */ #define XAE_TC_OFFSET 0x00000408/* Tx Configuration */
@@ -163,6 +164,7 @@#define XAE_EMMC_OFFSET 0x00000410/* EMAC mode configuration */ #define XAE_PHYC_OFFSET 0x00000414/* RGMII/SGMII configuration */ #define XAE_ID_OFFSET 0x000004F8/* Identification register */ +#define XAE_ABILITY_OFFSET 0x000004FC/* Ability Register offset */ #define XAE_MDIO_MC_OFFSET 0x00000500/* MII Management Config */ #define XAE_MDIO_MCR_OFFSET 0x00000504/* MII Management Control */ #define XAE_MDIO_MWD_OFFSET 0x00000508/* MII Management Write Data */
@@ -283,6 +285,16 @@#define XAE_PHYC_SGLINKSPD_100 0x40000000/* SGMII link 100 Mbit */ #define XAE_PHYC_SGLINKSPD_1000 0x80000000/* SGMII link 1000 Mbit */ +/* Bit masks for Axi Ethernet ability register */ +#define XAE_ABILITY_PFC BIT(16) +#define XAE_ABILITY_FRAME_FILTER BIT(10) +#define XAE_ABILITY_HALF_DUPLEX BIT(9) +#define XAE_ABILITY_STATS BIT(8) +#define XAE_ABILITY_2_5G BIT(3) +#define XAE_ABILITY_1G BIT(2) +#define XAE_ABILITY_100M BIT(1) +#define XAE_ABILITY_10M BIT(0) +/* Bit masks for Axi Ethernet MDIO interface MC register */ #define XAE_MDIO_MC_MDIOEN_MASK 0x00000040/* MII management enable */ #define XAE_MDIO_MC_CLOCK_DIVIDE_MAX 0x3F/* Maximum MDIO divisor */
@@ -331,6 +343,7 @@#define XAE_FEATURE_FULL_RX_CSUM BIT(2)#define XAE_FEATURE_FULL_TX_CSUM BIT(3)#define XAE_FEATURE_DMA_64BIT BIT(4) +#define XAE_FEATURE_STATS BIT(5)#define XAE_NO_CSUM_OFFLOAD 0
@@ -344,6 +357,61 @@#define XLNX_MII_STD_SELECT_REG 0x11#define XLNX_MII_STD_SELECT_SGMII BIT(0) +/* enum temac_stat - TEMAC statistics counters +* +* Index of statistics counters within the TEMAC. This must match the +* order/offset of hardware registers exactly. +*/ +enum temac_stat { +STAT_RX_BYTES =0 , +STAT_TX_BYTES , +STAT_UNDERSIZE_FRAMES , +STAT_FRAGMENT_FRAMES , +STAT_RX_64_BYTE_FRAMES , +STAT_RX_65_127_BYTE_FRAMES , +STAT_RX_128_255_BYTE_FRAMES , +STAT_RX_256_511_BYTE_FRAMES , +STAT_RX_512_1023_BYTE_FRAMES , +STAT_RX_1024_MAX_BYTE_FRAMES , +STAT_RX_OVERSIZE_FRAMES , +STAT_TX_64_BYTE_FRAMES , +STAT_TX_65_127_BYTE_FRAMES , +STAT_TX_128_255_BYTE_FRAMES , +STAT_TX_256_511_BYTE_FRAMES , +STAT_TX_512_1023_BYTE_FRAMES , +STAT_TX_1024_MAX_BYTE_FRAMES , +STAT_TX_OVERSIZE_FRAMES , +STAT_RX_GOOD_FRAMES , +STAT_RX_FCS_ERRORS , +STAT_RX_BROADCAST_FRAMES , +STAT_RX_MULTICAST_FRAMES , +STAT_RX_CONTROL_FRAMES , +STAT_RX_LENGTH_ERRORS , +STAT_RX_VLAN_FRAMES , +STAT_RX_PAUSE_FRAMES , +STAT_RX_CONTROL_OPCODE_ERRORS , +STAT_TX_GOOD_FRAMES , +STAT_TX_BROADCAST_FRAMES , +STAT_TX_MULTICAST_FRAMES , +STAT_TX_UNDERRUN_ERRORS , +STAT_TX_CONTROL_FRAMES , +STAT_TX_VLAN_FRAMES , +STAT_TX_PAUSE_FRAMES , +STAT_TX_SINGLE_COLLISION_FRAMES , +STAT_TX_MULTIPLE_COLLISION_FRAMES , +STAT_TX_DEFERRED_FRAMES , +STAT_TX_LATE_COLLISIONS , +STAT_TX_EXCESS_COLLISIONS , +STAT_TX_EXCESS_DEFERRAL , +STAT_RX_ALIGNMENT_ERRORS , +STAT_TX_PFC_FRAMES , +STAT_RX_PFC_FRAMES , +STAT_USER_DEFINED0 , +STAT_USER_DEFINED1 , +STAT_USER_DEFINED2 , +STAT_COUNT , +}; +/** * struct axidma_bd - Axi Dma buffer descriptor layout * @next: MM2S/S2MM Next Descriptor Pointer
@@ -434,6 +502,16 @@ struct skbuf_dma_descriptor {* @tx_packets: TX packet count for statistics * @tx_bytes: TX byte count for statistics * @tx_stat_sync: Synchronization object for TX stats +* @hw_stat_base: Base offset for statistics counters. This may be nonzero if +* the statistics counteres were reset or wrapped around. +* @hw_last_counter: Last-seen value of each statistic counter +* @reset_in_progress: Set while we are performing a reset and statistics +* counters may be invalid +* @hw_stats_seqcount: Sequence counter for @hw_stat_base, @hw_last_counter, +* and @reset_in_progress. +* @stats_lock: Lock for @hw_stats_seqcount +* @stats_work: Work for reading the hardware statistics counters often enough +* to catch overflows. * @dma_err_task: Work structure to process Axi DMA errors * @tx_irq: Axidma TX IRQ number * @rx_irq: Axidma RX IRQ number
@@ -505,6 +583,13 @@ struct axienet_local {u64_stats_t tx_bytes;struct u64_stats_sync tx_stat_sync; +u64 hw_stat_base[STAT_COUNT ]; +u64 hw_last_counter[STAT_COUNT ]; +seqcount_mutex_t hw_stats_seqcount; +struct mutex stats_lock; +struct delayed_work stats_work; +bool reset_in_progress; +struct work_struct dma_err_task;int tx_irq;
drivers/net/ethernet/xilinx/xilinx_axienet_main.c +262 -3
338 lines not on this page · raw message
Signature · 1 line
2.35.1.1320.gc452695387.dirty
Jakub Kicinski
maintainerdrivers/net/ethernet/xilinx/xilinx_axienet.h struct axienet_local { · line 587
Quoted 2 lines · On Thu, 15 Aug 2024 10:40:31 -0400 Sean Anderson wrote:
drivers/net/ethernet/xilinx/xilinx_axienet.h +2
-0+u64 hw_stat_base[STAT_COUNT ]; +u64 hw_last_counter[STAT_COUNT ];
I think hw_last_counter has to be u32..
Quoted 15 lines
drivers/net/ethernet/xilinx/xilinx_axienet.h +5
-0+seqcount_mutex_t hw_stats_seqcount; +struct mutex stats_lock; +struct delayed_work stats_work; +bool reset_in_progress; +struct work_struct dma_err_task;int tx_irq;drivers/net/ethernet/xilinx/xilinx_axienet_main.c +0
-0@@ -519,11 +519,55 @@ static void axienet_setoptions(struct net_device *ndev, u32 options) lp->options |= options; }
drivers/net/ethernet/xilinx/xilinx_axienet_main.c static void axienet_setoptions(struct net_device *ndev, u32 options) · line 530
Quoted 10 lines
drivers/net/ethernet/xilinx/xilinx_axienet_main.c +9
-0+static u64 axienet_stat (struct axienet_local *lp,enum temac_stat stat) +{ +u32 counter; + +if (lp->reset_in_progress) +return lp->hw_stat_base[stat]; + + counter =axienet_ior (lp,XAE_STATS_OFFSET + stat *8 ); +return lp->hw_stat_base[stat] + (counter - lp->hw_last_counter[stat]);
.. or you need to cast the (counter - lp->...) result to u32. Otherwise counter's type is getting bumped to 64 and you're just doing 64b math here.
Quoted 15 lines
drivers/net/ethernet/xilinx/xilinx_axienet_main.c +15
-0+} + +static void axienet_stats_update (struct axienet_local *lp,bool reset) +{ +enum temac_stat stat; + +write_seqcount_begin (&lp->hw_stats_seqcount); + lp->reset_in_progress = reset; +for (stat =0 ; stat <STAT_COUNT ; stat++) { +u32 counter =axienet_ior (lp,XAE_STATS_OFFSET + stat *8 ); + + lp->hw_stat_base[stat] += counter - lp->hw_last_counter[stat]; + lp->hw_last_counter[stat] = counter; + } +write_seqcount_end (&lp->hw_stats_seqcount);
Quoted 22 lines · Jakub Kicinski
drivers/net/ethernet/xilinx/xilinx_axienet.h struct axienet_local { · line 587
Quoted 2 lines · On Thu, 15 Aug 2024 10:40:31 -0400 Sean Anderson wrote:
drivers/net/ethernet/xilinx/xilinx_axienet.h +2
-0+u64 hw_stat_base[STAT_COUNT ]; +u64 hw_last_counter[STAT_COUNT ];I think
hw_last_counterhas to be u32..Quoted 16 lines
drivers/net/ethernet/xilinx/xilinx_axienet.h +5
-0+seqcount_mutex_t hw_stats_seqcount; +struct mutex stats_lock; +struct delayed_work stats_work; +bool reset_in_progress; +struct work_struct dma_err_task;int tx_irq;drivers/net/ethernet/xilinx/xilinx_axienet_main.c +0
-0@@ -519,11 +519,55 @@ static void axienet_setoptions(struct net_device *ndev, u32 options) lp->options |= options; }
Quoted 13 lines
drivers/net/ethernet/xilinx/xilinx_axienet_main.c static void axienet_setoptions(struct net_device *ndev, u32 options) · line 530
Quoted 9 lines
drivers/net/ethernet/xilinx/xilinx_axienet_main.c +9
-0+static u64 axienet_stat (struct axienet_local *lp,enum temac_stat stat) +{ +u32 counter; + +if (lp->reset_in_progress) +return lp->hw_stat_base[stat]; + + counter =axienet_ior (lp,XAE_STATS_OFFSET + stat *8 ); +return lp->hw_stat_base[stat] + (counter - lp->hw_last_counter[stat]);.. or you need to cast the (counter - lp->...) result to u32. Otherwise counter's type is getting bumped to 64 and you're just doing 64b math here.
Yes, good catch. I think I changed the type while refactoring and forgot about it.
I think I'll expose the byte counters for the next revision to make it easier to test rollover. I tried testing the packet counters overnight, but I'm using a trial license for these devices at the moment and it expired before the counters rolled over.
--Sean
Quoted 15 lines
Quoted 15 lines
drivers/net/ethernet/xilinx/xilinx_axienet_main.c +15
-0+} + +static void axienet_stats_update (struct axienet_local *lp,bool reset) +{ +enum temac_stat stat; + +write_seqcount_begin (&lp->hw_stats_seqcount); + lp->reset_in_progress = reset; +for (stat =0 ; stat <STAT_COUNT ; stat++) { +u32 counter =axienet_ior (lp,XAE_STATS_OFFSET + stat *8 ); + + lp->hw_stat_base[stat] += counter - lp->hw_last_counter[stat]; + lp->hw_last_counter[stat] = counter; + } +write_seqcount_end (&lp->hw_stats_seqcount);