Skip to contentSkip to conversation

i40e: add runtime-tunable ATR sample rate

PATCH Superseded [email protected] 2 participants 7 messages Jul 23 – Sep 11

Subsystems (5)

v4 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

v4 cover i40e: add runtime-tunable ATR sample rate

From: Mohammad Heib <[email protected]>

This series makes the Flow Director ATR sample rate configurable at runtime through a devlink parameter.

On systems with many queues and high-rate TCP workloads, the fixed default sampling interval can cause frequent Flow Director reprogramming and TCP packet reordering. The optimal interval depends on the workload and system configuration.

Patch 1 moves atr_sample_rate from per-ring to PF level, since it is a global policy.

Patch 2 exposes it as a devlink runtime parameter so administrators can tune it without rebuilding the driver.

Patch 3 fixes a pre-existing issue where re-enabling ATR via ethtool private flags did not trigger a ring reset, leaving fd_ena stale in the TX queue HW context.

Changes since v3:
- Added READ_ONCE() in i40e_atr() for lockless access to pf->atr_sample_rate (patch 1).
- Added WRITE_ONCE()/READ_ONCE() in devlink setter/getter (patch 2)
- New patch: trigger PF reset when re-enabling ATR (patch 3)

---
Mohammad Heib (3):
i40e: move ATR sample rate from ring to PF level
i40e: add devlink parameter for Flow Director ATR sample rate
i40e: trigger PF reset when re-enabling ATR via ethtool

 Documentation/networking/devlink/i40e.rst     | 20 +++++++++++
 drivers/net/ethernet/intel/i40e/i40e.h        |  1 +
 .../net/ethernet/intel/i40e/i40e_devlink.c    | 36 +++++++++++++++++++
 .../net/ethernet/intel/i40e/i40e_ethtool.c    |  7 ++++
 drivers/net/ethernet/intel/i40e/i40e_main.c   |  9 ++---
 drivers/net/ethernet/intel/i40e/i40e_txrx.c   |  6 ++--
 drivers/net/ethernet/intel/i40e/i40e_txrx.h   |  3 +-
 7 files changed, 72 insertions(+), 10 deletions(-)
Signature · 1 line
2.53.0
#1 lore raw

PATCH v4 1/3 i40e: move ATR sample rate from ring to PF level

From: Mohammad Heib <[email protected]>

The ATR sample rate is currently stored per-ring and initialized when each TX ring is configured. Since the sample rate is a global policy that applies uniformly across all rings, it makes more sense to store it at the PF level.

Move atr_sample_rate from struct i40e_ring to struct i40e_pf and initialize it once during i40e_sw_init(). Update i40e_atr() to reference the PF-level field. Change atr_count from u8 to u32 to match the sample rate type.

---
 drivers/net/ethernet/intel/i40e/i40e.h      | 1 +
 drivers/net/ethernet/intel/i40e/i40e_main.c | 9 +++------
 drivers/net/ethernet/intel/i40e/i40e_txrx.c | 6 ++++--
 drivers/net/ethernet/intel/i40e/i40e_txrx.h | 3 +--
 4 files changed, 9 insertions(+), 10 deletions(-)
drivers/net/ethernet/intel/i40e/i40e.h +1 -0
index 1b6a8fbaa648..88eb40ee45f0 100644
@@ -487,6 +487,7 @@ struct i40e_pf {
 	u16 rss_size_max;          /* HW defined max RSS queues */
 	u16 fdir_pf_filter_count;  /* num of guaranteed filters for this PF */
 	u16 num_alloc_vsi;         /* num VSIs this driver supports */
+	u32 atr_sample_rate;
 	bool wol_en;
 
 	struct hlist_head fdir_filter_list;
drivers/net/ethernet/intel/i40e/i40e_main.c +3 -6
index 4e5307355484..22466460e40b 100644
@@ -3457,12 +3457,7 @@ static int i40e_configure_tx_ring(struct i40e_ring *ring)
 		ring->xsk_pool = i40e_xsk_pool(ring);
 
 	/* some ATR related tx ring init */
-	if (test_bit(I40E_FLAG_FD_ATR_ENA, vsi->back->flags)) {
-		ring->atr_sample_rate = I40E_DEFAULT_ATR_SAMPLE_RATE;
-		ring->atr_count = 0;
-	} else {
-		ring->atr_sample_rate = 0;
-	}
+	ring->atr_count = 0;
 
 	/* configure XPS */
 	i40e_config_xps_tx_ring(ring);
@@ -12749,6 +12744,8 @@ static int i40e_sw_init(struct i40e_pf *pf)
 		}
 	}
 
+	pf->atr_sample_rate = I40E_DEFAULT_ATR_SAMPLE_RATE;
+
 	if ((pf->hw.func_caps.fd_filters_guaranteed > 0) ||
 	    (pf->hw.func_caps.fd_filters_best_effort > 0)) {
 		set_bit(I40E_FLAG_FD_ATR_ENA, pf->flags);
drivers/net/ethernet/intel/i40e/i40e_txrx.c +4 -2
index 61525ab7d21e..48d05c2e698e 100644
@@ -2871,6 +2871,7 @@ static void i40e_atr(struct i40e_ring *tx_ring, struct sk_buff *skb,
 	struct tcphdr *th;
 	unsigned int hlen;
 	u32 flex_ptype, dtype_cmd;
+	u32 sample_rate;
 	int l4_proto;
 	u16 i;
 
@@ -2882,7 +2883,8 @@ static void i40e_atr(struct i40e_ring *tx_ring, struct sk_buff *skb,
 		return;
 
 	/* if sampling is disabled do nothing */
-	if (!tx_ring->atr_sample_rate)
+	sample_rate = READ_ONCE(pf->atr_sample_rate);
+	if (!sample_rate)
 		return;
 
 	/* Currently only IPv4/IPv6 with TCP is supported */
@@ -2934,7 +2936,7 @@ static void i40e_atr(struct i40e_ring *tx_ring, struct sk_buff *skb,
 	if (!th->fin &&
 	    !th->syn &&
 	    !th->rst &&
-	    (tx_ring->atr_count < tx_ring->atr_sample_rate))
+	    (tx_ring->atr_count < sample_rate))
 		return;
 
 	tx_ring->atr_count = 0;
drivers/net/ethernet/intel/i40e/i40e_txrx.h +1 -2
index bb741ff3e5f2..be587f804e7a 100644
@@ -372,8 +372,7 @@ struct i40e_ring {
 	u16 next_to_clean;
 	u16 xdp_tx_active;
 
-	u8 atr_sample_rate;
-	u8 atr_count;
+	u32 atr_count;
 
 	bool ring_active;		/* is ring online or not */
 	bool arm_wb;		/* do something to arm write back */
Signature · 1 line
2.53.0
Tested-by on v4 1/3
#2 lore raw

RE: [Intel-wired-lan] [PATCH iwl-next v4 1/3] i40e: move ATR sample rate from ring to PF level

Quoted 11 lines

-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of [email protected]
Sent: 23 July 2026 19:34
To: [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nguyen, Anthony L <[email protected]>; Kitszel, Przemyslaw <[email protected]>; [email protected]; [email protected]; Mohammad Heib <[email protected]>
Subject: [Intel-wired-lan] [PATCH iwl-next v4 1/3] i40e: move ATR sample rate from ring to PF level

From: Mohammad Heib <[email protected]>

The ATR sample rate is currently stored per-ring and initialized when each TX ring is configured. Since the sample rate is a global policy that applies uniformly across all rings, it makes more sense to store it at the PF level.

Quoted 8 lines

Move atr_sample_rate from struct i40e_ring to struct i40e_pf and initialize it once during i40e_sw_init(). Update i40e_atr() to reference the PF-level field. Change atr_count from u8 to u32 to match the sample rate type.

---
drivers/net/ethernet/intel/i40e/i40e.h | 1 +
drivers/net/ethernet/intel/i40e/i40e_main.c | 9 +++------ drivers/net/ethernet/intel/i40e/i40e_txrx.c | 6 ++++-- drivers/net/ethernet/intel/i40e/i40e_txrx.h | 3 +--

4 files changed, 9 insertions(+), 10 deletions(-)
#3 lore raw

PATCH v4 2/3 i40e: add devlink parameter for Flow Director ATR sample rate

In reply to [email protected]

From: Mohammad Heib <[email protected]>

The i40e driver uses Flow Director ATR to periodically update flow steering information for active TCP flows. The update frequency is currently controlled by I40E_DEFAULT_ATR_SAMPLE_RATE and is fixed at driver build time.

On systems with a large number of queues and high-rate TCP workloads, the default sampling interval can result in frequent Flow Director reprogramming for long-lived flows.

The amount of TCP packet reordering observed on some systems is sensitive to the ATR sampling interval. Increasing the interval reduces Flow Director programming activity and can significantly reduce the associated reordering.

Since the optimal sampling interval depends on the workload and system configuration, a single fixed value is not suitable for all deployments.

Add a devlink parameter to allow administrators to tune the ATR sample rate at runtime without rebuilding the driver or disabling ATR functionality entirely.

---
 Documentation/networking/devlink/i40e.rst     | 20 +++++++++++
 .../net/ethernet/intel/i40e/i40e_devlink.c    | 36 +++++++++++++++++++
 2 files changed, 56 insertions(+)
Documentation/networking/devlink/i40e.rst +20 -0
index 51c887f0dc83..2cea98b631ba 100644
@@ -40,6 +40,26 @@ Parameters
 
         The default value is ``0`` (internal calculation is used).
 
+.. list-table:: Driver specific parameters implemented
+    :widths: 5 5 90
+
+    * - Name
+      - Mode
+      - Description
+    * - ``atr_sample_rate``
+      - runtime
+      - Controls how frequently Flow Director ATR updates flow steering
+        information for active TCP flows.
+
+        ATR programs Flow Director entries based on sampled transmitted
+        packets. The sampling interval is specified as the number of
+        transmitted packets between ATR updates.
+
+        Lower values increase Flow Director programming activity, while
+        higher values reduce the update frequency.
+
+        Setting to ``0`` disables ATR sampling (no filters will be programmed)
+        The default value is ``20``.
 
 Info versions
 =============
drivers/net/ethernet/intel/i40e/i40e_devlink.c +36 -0
index 229179ccc131..6df7eb217073 100644
@@ -33,12 +33,48 @@ static int i40e_max_mac_per_vf_get(struct devlink *devlink,
 	return 0;
 }
 
+static int i40e_atr_sample_rate_set(struct devlink *devlink,
+				    u32 id,
+				    struct devlink_param_gset_ctx *ctx,
+				    struct netlink_ext_ack *extack)
+{
+	struct i40e_pf *pf = devlink_priv(devlink);
+	u32 sample_rate = ctx->val.vu32;
+
+	WRITE_ONCE(pf->atr_sample_rate, sample_rate);
+	return 0;
+}
+
+static int i40e_atr_sample_rate_get(struct devlink *devlink,
+				    u32 id,
+				    struct devlink_param_gset_ctx *ctx,
+				    struct netlink_ext_ack *extack)
+{
+	struct i40e_pf *pf = devlink_priv(devlink);
+
+	ctx->val.vu32 = READ_ONCE(pf->atr_sample_rate);
+
+	return 0;
+}
+
+enum i40e_dl_param_id {
+	I40E_DEVLINK_PARAM_ID_BASE = DEVLINK_PARAM_GENERIC_ID_MAX,
+	I40E_DEVLINK_PARAM_ID_ATR_SAMPLE_RATE,
+};
+
 static const struct devlink_param i40e_dl_params[] = {
 	DEVLINK_PARAM_GENERIC(MAX_MAC_PER_VF,
 			      BIT(DEVLINK_PARAM_CMODE_RUNTIME),
 			      i40e_max_mac_per_vf_get,
 			      i40e_max_mac_per_vf_set,
 			      NULL),
+	DEVLINK_PARAM_DRIVER(I40E_DEVLINK_PARAM_ID_ATR_SAMPLE_RATE,
+			     "atr_sample_rate",
+			     DEVLINK_PARAM_TYPE_U32,
+			     BIT(DEVLINK_PARAM_CMODE_RUNTIME),
+			     i40e_atr_sample_rate_get,
+			     i40e_atr_sample_rate_set,
+			     NULL),
 };
 
 static void i40e_info_get_dsn(struct i40e_pf *pf, char *buf, size_t len)
Signature · 1 line
2.53.0
Tested-by on v4 2/3
#4 lore raw

RE: [Intel-wired-lan] [PATCH iwl-next v4 2/3] i40e: add devlink parameter for Flow Director ATR sample rate

Quoted 17 lines

-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of [email protected]
Sent: 23 July 2026 19:34
To: [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nguyen, Anthony L <[email protected]>; Kitszel, Przemyslaw <[email protected]>; [email protected]; [email protected]; Mohammad Heib <[email protected]>
Subject: [Intel-wired-lan] [PATCH iwl-next v4 2/3] i40e: add devlink parameter for Flow Director ATR sample rate

From: Mohammad Heib <[email protected]>

The i40e driver uses Flow Director ATR to periodically update flow steering information for active TCP flows. The update frequency is currently controlled by I40E_DEFAULT_ATR_SAMPLE_RATE and is fixed at driver build time.

On systems with a large number of queues and high-rate TCP workloads, the default sampling interval can result in frequent Flow Director reprogramming for long-lived flows.

The amount of TCP packet reordering observed on some systems is sensitive to the ATR sampling interval. Increasing the interval reduces Flow Director programming activity and can significantly reduce the associated reordering.

Since the optimal sampling interval depends on the workload and system configuration, a single fixed value is not suitable for all deployments.

Quoted 8 lines

Add a devlink parameter to allow administrators to tune the ATR sample rate at runtime without rebuilding the driver or disabling ATR functionality entirely.

---
Documentation/networking/devlink/i40e.rst | 20 +++++++++++
.../net/ethernet/intel/i40e/i40e_devlink.c | 36 +++++++++++++++++++

2 files changed, 56 insertions(+)
#5 lore raw

PATCH v4 3/3 i40e: trigger PF reset when re-enabling ATR via ethtool

In reply to [email protected]

From: Mohammad Heib <[email protected]>

Toggling flow-director-atr via ethtool private flags does not trigger a ring reset. If ATR was disabled while fd_ena was cleared in the TX queue HW context (e.g. after a ring reset with both ATR and ntuple off), re-enabling ATR causes i40e_atr() to inject FDIR descriptors into a queue where fd_ena is still 0.

Trigger a PF reset when ATR is re-enabled so that fd_ena is re-evaluated based on the current flag state.

---
 drivers/net/ethernet/intel/i40e/i40e_ethtool.c | 7 +++++++
 1 file changed, 7 insertions(+)
drivers/net/ethernet/intel/i40e/i40e_ethtool.c +7 -0
index 6d2b076049f7..bb37f8cc4643 100644
@@ -5301,6 +5301,13 @@ static int i40e_set_priv_flags(struct net_device *dev, u32 flags)
 	    test_bit(I40E_FLAG_SOURCE_PRUNING_DIS, changed_flags))
 		reset_needed = BIT(__I40E_PF_RESET_REQUESTED);
 
+	/* Re-enabling ATR requires a reset to update fd_ena in the
+	 * TX queue HW context.
+	 */
+	if (test_bit(I40E_FLAG_FD_ATR_ENA, changed_flags) &&
+	    test_bit(I40E_FLAG_FD_ATR_ENA, new_flags))
+		reset_needed = BIT(__I40E_PF_RESET_REQUESTED);
+
 	/* Before we finalize any flag changes, we need to perform some
 	 * checks to ensure that the changes are supported and safe.
 	 */
Signature · 1 line
2.53.0
Tested-by on v4 3/3
#6 lore raw

RE: [Intel-wired-lan] [PATCH iwl-next v4 3/3] i40e: trigger PF reset when re-enabling ATR via ethtool

Quoted 9 lines

-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of [email protected]
Sent: 23 July 2026 19:34
To: [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nguyen, Anthony L <[email protected]>; Kitszel, Przemyslaw <[email protected]>; [email protected]; [email protected]; Mohammad Heib <[email protected]>
Subject: [Intel-wired-lan] [PATCH iwl-next v4 3/3] i40e: trigger PF reset when re-enabling ATR via ethtool

From: Mohammad Heib <[email protected]>

Quoted 9 lines

Toggling flow-director-atr via ethtool private flags does not trigger a ring reset. If ATR was disabled while fd_ena was cleared in the TX queue HW context (e.g. after a ring reset with both ATR and ntuple off), re-enabling ATR causes i40e_atr() to inject FDIR descriptors into a queue where fd_ena is still 0.

Trigger a PF reset when ATR is re-enabled so that fd_ena is re-evaluated based on the current flag state.

---
drivers/net/ethernet/intel/i40e/i40e_ethtool.c | 7 +++++++

1 file changed, 7 insertions(+)