Skip to contentSkip to conversation

net: xilinx: axienet: Add statistics support

PATCH Superseded Sean Anderson 2 participants 5 messages Aug 15, 2024 – Aug 20, 2024

Subsystems (4)

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v3 cover net: xilinx: axienet: Add statistics support

Add support for hardware statistics counters (if they are enabled) in the AXI Ethernet driver. Unfortunately, the implementation is complicated a bit since the hardware might only support 32-bit counters.

Changes in v3:
- Use explicit mutex_lock/unlock instead of guard() in __axienet_device_reset to make it clear that we need to hold lp->stats_lock for the whole function.

Changes in v2:
- Switch to a seqlock-based implementation to allow fresher updates (rather than always using stale counter values from the previous refresh).
- Take stats_lock unconditionally in __axienet_device_reset
- Fix documentation mismatch

Sean Anderson (2):
net: xilinx: axienet: Report RxRject as rx_dropped
net: xilinx: axienet: Add statistics support

 drivers/net/ethernet/xilinx/xilinx_axienet.h  |  85 ++++++
 .../net/ethernet/xilinx/xilinx_axienet_main.c | 267 +++++++++++++++++-
 2 files changed, 348 insertions(+), 4 deletions(-)
Signature · 1 line
2.35.1.1320.gc452695387.dirty
#1 lore raw

PATCH v3 1/2 net: xilinx: axienet: Report RxRject as rx_dropped

The Receive Frame Rejected interrupt is asserted whenever there was a receive error (bad FCS, bad length, etc.) or whenever the frame was dropped due to a mismatched address. So this is really a combination of rx_otherhost_dropped, rx_length_errors, rx_frame_errors, and rx_crc_errors. Mismatched addresses are common and aren't really errors at all (much like how fragments are normal on half-duplex links). To avoid confusion, report these events as rx_dropped. This better reflects what's going on: the packet was received by the MAC but dropped before being processed.

---

(no changes since v1)

 drivers/net/ethernet/xilinx/xilinx_axienet_main.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/xilinx/xilinx_axienet_main.c +1 -1
index ca04c298daa2..b2d7c396e2e3 100644
@@ -1296,7 +1296,7 @@ static irqreturn_t axienet_eth_irq(int irq, void *_ndev)
 		ndev->stats.rx_missed_errors++;
 
 	if (pending & XAE_INT_RXRJECT_MASK)
-		ndev->stats.rx_frame_errors++;
+		ndev->stats.rx_dropped++;
 
 	axienet_iow(lp, XAE_IS_OFFSET, pending);
 	return IRQ_HANDLED;
Signature · 1 line
2.35.1.1320.gc452695387.dirty
#2 lore raw

PATCH v3 2/2 net: xilinx: axienet: Add statistics support

In reply to Sean Anderson

Add support for reading the statistics counters, if they are enabled. The counters may be 64-bit, but we can't detect this statically as there's no ability bit for it and the counters are read-only. Therefore, we assume the counters are 32-bits by default. To ensure we don't miss an overflow, we read all counters at 13-second intervals. This should be often enough to ensure the bytes counters don't wrap at 2.5 Gbit/s.

Another complication is that the counters may be reset when the device is reset (depending on configuration). To ensure the counters persist across link up/down (including suspend/resume), we maintain our own versions along with the last counter value we saw. Because we might wait up to 100 ms for the reset to complete, we use a mutex to protect writing hw_stats. We can't sleep in ndo_get_stats64, so we use a seqlock to protect readers.

We don't bother disabling the refresh work when we detect 64-bit counters. This is because the reset issue requires us to read hw_stat_base and reset_in_progress anyway, which would still require the seqcount. And I don't think skipping the task is worth the extra bookkeeping.

We can't use the byte counters for either get_stats64 or get_eth_mac_stats. This is because the byte counters include everything in the frame (destination address to FCS, inclusive). But rtnl_link_stats64 wants bytes excluding the FCS, and ethtool_eth_mac_stats wants to exclude the L2 overhead (addresses and length/type). It might be possible to calculate the byte values Linux expects based on the frame counters, but I think it is simpler to use the existing software counters.

---

Changes in v3:
- Use explicit mutex_lock/unlock instead of guard() in __axienet_device_reset to make it clear that we need to hold lp->stats_lock for the whole function.

Changes in v2:
- Switch to a seqlock-based implementation to allow fresher updates (rather than always using stale counter values from the previous refresh).
- Take stats_lock unconditionally in __axienet_device_reset
- Fix documentation mismatch

 drivers/net/ethernet/xilinx/xilinx_axienet.h  |  85 ++++++
 .../net/ethernet/xilinx/xilinx_axienet_main.c | 265 +++++++++++++++++-
 2 files changed, 347 insertions(+), 3 deletions(-)
drivers/net/ethernet/xilinx/xilinx_axienet.h +85 -0
index 0d5b300107e0..373c56080dde 100644
@@ -156,6 +156,7 @@
 #define XAE_TPID0_OFFSET	0x00000028 /* VLAN TPID0 register */
 #define XAE_TPID1_OFFSET	0x0000002C /* VLAN TPID1 register */
 #define XAE_PPST_OFFSET		0x00000030 /* PCS PMA Soft Temac Status Reg */
+#define XAE_STATS_OFFSET	0x00000200 /* Statistics counters */
 #define XAE_RCW0_OFFSET		0x00000400 /* Rx Configuration Word 0 */
 #define XAE_RCW1_OFFSET		0x00000404 /* Rx Configuration Word 1 */
 #define XAE_TC_OFFSET		0x00000408 /* Tx Configuration */
@@ -163,6 +164,7 @@
 #define XAE_EMMC_OFFSET		0x00000410 /* EMAC mode configuration */
 #define XAE_PHYC_OFFSET		0x00000414 /* RGMII/SGMII configuration */
 #define XAE_ID_OFFSET		0x000004F8 /* Identification register */
+#define XAE_ABILITY_OFFSET	0x000004FC /* Ability Register offset */
 #define XAE_MDIO_MC_OFFSET	0x00000500 /* MII Management Config */
 #define XAE_MDIO_MCR_OFFSET	0x00000504 /* MII Management Control */
 #define XAE_MDIO_MWD_OFFSET	0x00000508 /* MII Management Write Data */
@@ -283,6 +285,16 @@
 #define XAE_PHYC_SGLINKSPD_100		0x40000000 /* SGMII link 100 Mbit */
 #define XAE_PHYC_SGLINKSPD_1000		0x80000000 /* SGMII link 1000 Mbit */
 
+/* Bit masks for Axi Ethernet ability register */
+#define XAE_ABILITY_PFC			BIT(16)
+#define XAE_ABILITY_FRAME_FILTER	BIT(10)
+#define XAE_ABILITY_HALF_DUPLEX		BIT(9)
+#define XAE_ABILITY_STATS		BIT(8)
+#define XAE_ABILITY_2_5G		BIT(3)
+#define XAE_ABILITY_1G			BIT(2)
+#define XAE_ABILITY_100M		BIT(1)
+#define XAE_ABILITY_10M			BIT(0)
+
 /* Bit masks for Axi Ethernet MDIO interface MC register */
 #define XAE_MDIO_MC_MDIOEN_MASK		0x00000040 /* MII management enable */
 #define XAE_MDIO_MC_CLOCK_DIVIDE_MAX	0x3F	   /* Maximum MDIO divisor */
@@ -331,6 +343,7 @@
 #define XAE_FEATURE_FULL_RX_CSUM	BIT(2)
 #define XAE_FEATURE_FULL_TX_CSUM	BIT(3)
 #define XAE_FEATURE_DMA_64BIT		BIT(4)
+#define XAE_FEATURE_STATS		BIT(5)
 
 #define XAE_NO_CSUM_OFFLOAD		0
 
@@ -344,6 +357,61 @@
 #define XLNX_MII_STD_SELECT_REG		0x11
 #define XLNX_MII_STD_SELECT_SGMII	BIT(0)
 
+/* enum temac_stat - TEMAC statistics counters
+ *
+ * Index of statistics counters within the TEMAC. This must match the
+ * order/offset of hardware registers exactly.
+ */
+enum temac_stat {
+	STAT_RX_BYTES = 0,
+	STAT_TX_BYTES,
+	STAT_UNDERSIZE_FRAMES,
+	STAT_FRAGMENT_FRAMES,
+	STAT_RX_64_BYTE_FRAMES,
+	STAT_RX_65_127_BYTE_FRAMES,
+	STAT_RX_128_255_BYTE_FRAMES,
+	STAT_RX_256_511_BYTE_FRAMES,
+	STAT_RX_512_1023_BYTE_FRAMES,
+	STAT_RX_1024_MAX_BYTE_FRAMES,
+	STAT_RX_OVERSIZE_FRAMES,
+	STAT_TX_64_BYTE_FRAMES,
+	STAT_TX_65_127_BYTE_FRAMES,
+	STAT_TX_128_255_BYTE_FRAMES,
+	STAT_TX_256_511_BYTE_FRAMES,
+	STAT_TX_512_1023_BYTE_FRAMES,
+	STAT_TX_1024_MAX_BYTE_FRAMES,
+	STAT_TX_OVERSIZE_FRAMES,
+	STAT_RX_GOOD_FRAMES,
+	STAT_RX_FCS_ERRORS,
+	STAT_RX_BROADCAST_FRAMES,
+	STAT_RX_MULTICAST_FRAMES,
+	STAT_RX_CONTROL_FRAMES,
+	STAT_RX_LENGTH_ERRORS,
+	STAT_RX_VLAN_FRAMES,
+	STAT_RX_PAUSE_FRAMES,
+	STAT_RX_CONTROL_OPCODE_ERRORS,
+	STAT_TX_GOOD_FRAMES,
+	STAT_TX_BROADCAST_FRAMES,
+	STAT_TX_MULTICAST_FRAMES,
+	STAT_TX_UNDERRUN_ERRORS,
+	STAT_TX_CONTROL_FRAMES,
+	STAT_TX_VLAN_FRAMES,
+	STAT_TX_PAUSE_FRAMES,
+	STAT_TX_SINGLE_COLLISION_FRAMES,
+	STAT_TX_MULTIPLE_COLLISION_FRAMES,
+	STAT_TX_DEFERRED_FRAMES,
+	STAT_TX_LATE_COLLISIONS,
+	STAT_TX_EXCESS_COLLISIONS,
+	STAT_TX_EXCESS_DEFERRAL,
+	STAT_RX_ALIGNMENT_ERRORS,
+	STAT_TX_PFC_FRAMES,
+	STAT_RX_PFC_FRAMES,
+	STAT_USER_DEFINED0,
+	STAT_USER_DEFINED1,
+	STAT_USER_DEFINED2,
+	STAT_COUNT,
+};
+
 /**
  * struct axidma_bd - Axi Dma buffer descriptor layout
  * @next:         MM2S/S2MM Next Descriptor Pointer
@@ -434,6 +502,16 @@ struct skbuf_dma_descriptor {
  * @tx_packets: TX packet count for statistics
  * @tx_bytes:	TX byte count for statistics
  * @tx_stat_sync: Synchronization object for TX stats
+ * @hw_stat_base: Base offset for statistics counters. This may be nonzero if
+ *                the statistics counteres were reset or wrapped around.
+ * @hw_last_counter: Last-seen value of each statistic counter
+ * @reset_in_progress: Set while we are performing a reset and statistics
+ *                     counters may be invalid
+ * @hw_stats_seqcount: Sequence counter for @hw_stat_base, @hw_last_counter,
+ *                     and @reset_in_progress.
+ * @stats_lock: Lock for @hw_stats_seqcount
+ * @stats_work: Work for reading the hardware statistics counters often enough
+ *              to catch overflows.
  * @dma_err_task: Work structure to process Axi DMA errors
  * @tx_irq:	Axidma TX IRQ number
  * @rx_irq:	Axidma RX IRQ number
@@ -505,6 +583,13 @@ struct axienet_local {
 	u64_stats_t tx_bytes;
 	struct u64_stats_sync tx_stat_sync;
 
+	u64 hw_stat_base[STAT_COUNT];
+	u64 hw_last_counter[STAT_COUNT];
+	seqcount_mutex_t hw_stats_seqcount;
+	struct mutex stats_lock;
+	struct delayed_work stats_work;
+	bool reset_in_progress;
+
 	struct work_struct dma_err_task;
 
 	int tx_irq;
drivers/net/ethernet/xilinx/xilinx_axienet_main.c +262 -3

338 lines not on this page · raw message

Signature · 1 line
2.35.1.1320.gc452695387.dirty
maintainer
#3 lore raw

drivers/net/ethernet/xilinx/xilinx_axienet.h struct axienet_local { · line 587

Quoted 2 lines · On Thu, 15 Aug 2024 10:40:31 -0400 Sean Anderson wrote:
drivers/net/ethernet/xilinx/xilinx_axienet.h +2 -0
+	u64 hw_stat_base[STAT_COUNT];
+	u64 hw_last_counter[STAT_COUNT];

I think hw_last_counter has to be u32..

Quoted 15 lines
drivers/net/ethernet/xilinx/xilinx_axienet.h +5 -0
+	seqcount_mutex_t hw_stats_seqcount;
+	struct mutex stats_lock;
+	struct delayed_work stats_work;
+	bool reset_in_progress;
+
 	struct work_struct dma_err_task;
 
 	int tx_irq;
drivers/net/ethernet/xilinx/xilinx_axienet_main.c +0 -0
index b2d7c396e2e3..9353a4f0ab1b 100644
@@ -519,11 +519,55 @@ static void axienet_setoptions(struct net_device *ndev, u32 options)
 	lp->options |= options;
 }

drivers/net/ethernet/xilinx/xilinx_axienet_main.c static void axienet_setoptions(struct net_device *ndev, u32 options) · line 530

Quoted 10 lines
drivers/net/ethernet/xilinx/xilinx_axienet_main.c +9 -0
 
+static u64 axienet_stat(struct axienet_local *lp, enum temac_stat stat)
+{
+	u32 counter;
+
+	if (lp->reset_in_progress)
+		return lp->hw_stat_base[stat];
+
+	counter = axienet_ior(lp, XAE_STATS_OFFSET + stat * 8);
+	return lp->hw_stat_base[stat] + (counter - lp->hw_last_counter[stat]);

.. or you need to cast the (counter - lp->...) result to u32. Otherwise counter's type is getting bumped to 64 and you're just doing 64b math here.

Quoted 15 lines
drivers/net/ethernet/xilinx/xilinx_axienet_main.c +15 -0
+}
+
+static void axienet_stats_update(struct axienet_local *lp, bool reset)
+{
+	enum temac_stat stat;
+
+	write_seqcount_begin(&lp->hw_stats_seqcount);
+	lp->reset_in_progress = reset;
+	for (stat = 0; stat < STAT_COUNT; stat++) {
+		u32 counter = axienet_ior(lp, XAE_STATS_OFFSET + stat * 8);
+
+		lp->hw_stat_base[stat] += counter - lp->hw_last_counter[stat];
+		lp->hw_last_counter[stat] = counter;
+	}
+	write_seqcount_end(&lp->hw_stats_seqcount);
#4 lore raw
Quoted 22 lines · Jakub Kicinski

drivers/net/ethernet/xilinx/xilinx_axienet.h struct axienet_local { · line 587

Quoted 2 lines · On Thu, 15 Aug 2024 10:40:31 -0400 Sean Anderson wrote:
drivers/net/ethernet/xilinx/xilinx_axienet.h +2 -0
+	u64 hw_stat_base[STAT_COUNT];
+	u64 hw_last_counter[STAT_COUNT];

I think hw_last_counter has to be u32..

Quoted 16 lines
drivers/net/ethernet/xilinx/xilinx_axienet.h +5 -0
+	seqcount_mutex_t hw_stats_seqcount;
+	struct mutex stats_lock;
+	struct delayed_work stats_work;
+	bool reset_in_progress;
+
 	struct work_struct dma_err_task;
 
 	int tx_irq;
drivers/net/ethernet/xilinx/xilinx_axienet_main.c +0 -0
index b2d7c396e2e3..9353a4f0ab1b 100644
@@ -519,11 +519,55 @@ static void axienet_setoptions(struct net_device *ndev, u32 options)
 	lp->options |= options;
 }
 
Quoted 13 lines

drivers/net/ethernet/xilinx/xilinx_axienet_main.c static void axienet_setoptions(struct net_device *ndev, u32 options) · line 530

Quoted 9 lines
drivers/net/ethernet/xilinx/xilinx_axienet_main.c +9 -0
+static u64 axienet_stat(struct axienet_local *lp, enum temac_stat stat)
+{
+	u32 counter;
+
+	if (lp->reset_in_progress)
+		return lp->hw_stat_base[stat];
+
+	counter = axienet_ior(lp, XAE_STATS_OFFSET + stat * 8);
+	return lp->hw_stat_base[stat] + (counter - lp->hw_last_counter[stat]);

.. or you need to cast the (counter - lp->...) result to u32. Otherwise counter's type is getting bumped to 64 and you're just doing 64b math here.

Yes, good catch. I think I changed the type while refactoring and forgot about it.

I think I'll expose the byte counters for the next revision to make it easier to test rollover. I tried testing the packet counters overnight, but I'm using a trial license for these devices at the moment and it expired before the counters rolled over.

--Sean

Quoted 15 lines
Quoted 15 lines
drivers/net/ethernet/xilinx/xilinx_axienet_main.c +15 -0
+}
+
+static void axienet_stats_update(struct axienet_local *lp, bool reset)
+{
+	enum temac_stat stat;
+
+	write_seqcount_begin(&lp->hw_stats_seqcount);
+	lp->reset_in_progress = reset;
+	for (stat = 0; stat < STAT_COUNT; stat++) {
+		u32 counter = axienet_ior(lp, XAE_STATS_OFFSET + stat * 8);
+
+		lp->hw_stat_base[stat] += counter - lp->hw_last_counter[stat];
+		lp->hw_last_counter[stat] = counter;
+	}
+	write_seqcount_end(&lp->hw_stats_seqcount);