Skip to contentSkip to conversation

iavf: cap advertised max_pkt_size at the single-buffer HW limit

PATCH In review Dave πŸ§”β€ Butler 3 participants 3 messages Jun 16 – Jun 17

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 iavf: cap advertised max_pkt_size at the single-buffer HW limit

From: Dave Butler <[email protected]>

Since commit 5fa4caff59f2 ("iavf: switch to Page Pool") iavf_configure_queues() advertises max_pkt_size to the PF as:

	max_frame = LIBIE_MAX_RX_FRM_LEN(adapter->rx_rings->pp->p.offset);
	max_frame = min_not_zero(adapter->vf_res->max_mtu, max_frame);

LIBIE_MAX_RX_FRM_LEN (16382) is the multi-descriptor scatter/gather frame ceiling, not a single-queue value, and it exceeds the E810 MAC frame size maximum of 9728. Per the E810 datasheet (613875-009 section 13.2.2.17.1) the Tx frame-size register PRTDCB_TDPUC.MAX_TXFRAME has a maximum of 0x2600 (9728); larger frames are discarded. The in-tree ice driver encodes the same value as ICE_AQ_SET_MAC_FRAME_SIZE_MAX (== LIBIE_MAX_RX_BUF_LEN == 9728), and the VF clamped max_frame to IAVF_MAX_RXBUFFER (9728) before this commit.

When the PF advertises vf_res->max_mtu as 0, min_not_zero() leaves max_frame at 16382. The Linux ice PF advertises max_mtu = port MAC frame size (<= 9728), so a VF behind ice never sends more than that. The ESXi "icen" PF on E810 advertises max_mtu as 0, so the VF sends max_pkt_size = 16382, which icen rejects while programming the queue context for VIRTCHNL_OP_CONFIG_VSI_QUEUES (opcode 6):

icen_ConfigureTxQueue: VSI 8: Failed to set LAN Tx queue context for
absolute Tx queue 64, Error: ICE_ERR_PARAM
indrv_SendMsgToVf: VF 0: Failed opcode 6, Error -5

iavf 0000:03:00.0: PF returned error -5 (IAVF_ERR_PARAM) to our request 6 iavf 0000:03:00.0 ethX: NETDEV WATCHDOG: transmit queue N timed out

The VF's queues never come up; under SR-IOV passthrough the mis-programmed queue can also trigger a fatal IOMMU fault in the guest. Forcing only max_pkt_size back to 9728 (and leaving the Page Pool rx_buf_len/ databuffer_size untouched) makes the VF come up; databuffer_size is not involved. This was confirmed on two E810 NVM revisions (3.00 and 4.51) and two icen versions (1.14.2.0 and the latest 2.3.3.0): all reject the unpatched VF and accept the patched one, so the trigger is the icen PF behaviour, not the firmware or icen revision. Reported by several users on E810 + ESXi icen with v6.10+ guests:

Cap max_frame at the single-buffer hardware limit, restoring the pre-Page-Pool behaviour while keeping the Page Pool rx_buf_len unchanged.

Fixes 5fa4caff59f2 ("iavf: switch to Page Pool")
---
 drivers/net/ethernet/intel/iavf/iavf_virtchnl.c | 8 ++++++++
 1 file changed, 8 insertions(+)
drivers/net/ethernet/intel/iavf/iavf_virtchnl.c +8 -0
index 4f2defd2331b..2632eeed776c 100644
@@ -382,6 +382,14 @@ void iavf_configure_queues(struct iavf_adapter *adapter)
 
 	max_frame = LIBIE_MAX_RX_FRM_LEN(adapter->rx_rings->pp->p.offset);
 	max_frame = min_not_zero(adapter->vf_res->max_mtu, max_frame);
+	/* The PF programs max_pkt_size into the per-queue Rx context "rxmax".
+	 * LIBIE_MAX_RX_FRM_LEN is the multi-descriptor (S/G) frame ceiling
+	 * (16382), but that exceeds the E810 max MAC frame size (9728); some
+	 * PFs reject the out-of-range value with VIRTCHNL_STATUS_ERR_PARAM.
+	 * Cap it at the single-buffer HW limit (== the MAC frame max),
+	 * restoring the pre-Page-Pool behaviour.
+	 */
+	max_frame = min(max_frame, LIBIE_MAX_RX_BUF_LEN);
 
 	if (adapter->current_op != VIRTCHNL_OP_UNKNOWN) {
 		/* bail because we already have a command pending */

base-commit: fbc6a80cb5d3fd4ac4b56e8c9d791dd17be890c4

Signature Β· 1 line
2.43.0
Acked-by on v1 1/1
#1 lore raw
Quoted 45 lines Β· Dave πŸ§” Butler

From: Dave Butler <[email protected]>

Since commit 5fa4caff59f2 ("iavf: switch to Page Pool") iavf_configure_queues() advertises max_pkt_size to the PF as:

	max_frame = LIBIE_MAX_RX_FRM_LEN(adapter->rx_rings->pp->p.offset);
	max_frame = min_not_zero(adapter->vf_res->max_mtu, max_frame);

LIBIE_MAX_RX_FRM_LEN (16382) is the multi-descriptor scatter/gather frame ceiling, not a single-queue value, and it exceeds the E810 MAC frame size maximum of 9728. Per the E810 datasheet (613875-009 section 13.2.2.17.1) the Tx frame-size register PRTDCB_TDPUC.MAX_TXFRAME has a maximum of 0x2600 (9728); larger frames are discarded. The in-tree ice driver encodes the same value as ICE_AQ_SET_MAC_FRAME_SIZE_MAX (== LIBIE_MAX_RX_BUF_LEN == 9728), and the VF clamped max_frame to IAVF_MAX_RXBUFFER (9728) before this commit.

When the PF advertises vf_res->max_mtu as 0, min_not_zero() leaves max_frame at 16382. The Linux ice PF advertises max_mtu = port MAC frame size (<= 9728), so a VF behind ice never sends more than that. The ESXi "icen" PF on E810 advertises max_mtu as 0, so the VF sends max_pkt_size = 16382, which icen rejects while programming the queue context for VIRTCHNL_OP_CONFIG_VSI_QUEUES (opcode 6):

icen_ConfigureTxQueue: VSI 8: Failed to set LAN Tx queue context for
absolute Tx queue 64, Error: ICE_ERR_PARAM
indrv_SendMsgToVf: VF 0: Failed opcode 6, Error -5

iavf 0000:03:00.0: PF returned error -5 (IAVF_ERR_PARAM) to our request 6 iavf 0000:03:00.0 ethX: NETDEV WATCHDOG: transmit queue N timed out

The VF's queues never come up; under SR-IOV passthrough the mis-programmed queue can also trigger a fatal IOMMU fault in the guest. Forcing only max_pkt_size back to 9728 (and leaving the Page Pool rx_buf_len/ databuffer_size untouched) makes the VF come up; databuffer_size is not involved. This was confirmed on two E810 NVM revisions (3.00 and 4.51) and two icen versions (1.14.2.0 and the latest 2.3.3.0): all reject the unpatched VF and accept the patched one, so the trigger is the icen PF behaviour, not the firmware or icen revision. Reported by several users on E810 + ESXi icen with v6.10+ guests:

Quoted 6 lines

Cap max_frame at the single-buffer hardware limit, restoring the pre-Page-Pool behaviour while keeping the Page Pool rx_buf_len unchanged.

Fixes 5fa4caff59f2 ("iavf: switch to Page Pool")

---

I suppose it makes sense to restore the previous behavior pre-pagepool even if this smells like a bug on the PF side advertising 0 as the maximum size.

The only real downside here would be if the iAVF interface is ever used again with another device that supports a larger maximum frame size. I suppose we can always handle that in the future with an additional check based on some other data. One possibility is to only apply the final maximum here if the PF actually sends us 0? Mayube not worth it until/if we actually have such devices.

Quoted 24 lines
 drivers/net/ethernet/intel/iavf/iavf_virtchnl.c | 8 ++++++++
 1 file changed, 8 insertions(+)
drivers/net/ethernet/intel/iavf/iavf_virtchnl.c +8 -0
index 4f2defd2331b..2632eeed776c 100644
@@ -382,6 +382,14 @@ void iavf_configure_queues(struct iavf_adapter *adapter)
 
 	max_frame = LIBIE_MAX_RX_FRM_LEN(adapter->rx_rings->pp->p.offset);
 	max_frame = min_not_zero(adapter->vf_res->max_mtu, max_frame);
+	/* The PF programs max_pkt_size into the per-queue Rx context "rxmax".
+	 * LIBIE_MAX_RX_FRM_LEN is the multi-descriptor (S/G) frame ceiling
+	 * (16382), but that exceeds the E810 max MAC frame size (9728); some
+	 * PFs reject the out-of-range value with VIRTCHNL_STATUS_ERR_PARAM.
+	 * Cap it at the single-buffer HW limit (== the MAC frame max),
+	 * restoring the pre-Page-Pool behaviour.
+	 */
+	max_frame = min(max_frame, LIBIE_MAX_RX_BUF_LEN);
 
 	if (adapter->current_op != VIRTCHNL_OP_UNKNOWN) {
 		/* bail because we already have a command pending */

base-commit: fbc6a80cb5d3fd4ac4b56e8c9d791dd17be890c4

Reviewed-by on v1 1/1
#2 lore raw
In reply to Dave πŸ§”β€ Butler
Quoted 104 lines

-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Dave πŸ§”β€ Butler
Sent: Tuesday, June 16, 2026 10:02 PM
To: [email protected]; [email protected]; Lobakin, Aleksander <[email protected]>; Nguyen, Anthony L <[email protected]>; Kitszel, Przemyslaw <[email protected]>
Cc: [email protected]; [email protected]
Subject: [Intel-wired-lan] [PATCH net] iavf: cap advertised max_pkt_size at the single-buffer HW limit

From: Dave Butler <[email protected]>

Since commit 5fa4caff59f2 ("iavf: switch to Page Pool") iavf_configure_queues() advertises max_pkt_size to the PF as:

	max_frame = LIBIE_MAX_RX_FRM_LEN(adapter->rx_rings->pp-
Quoted 1 line

p.offset);

	max_frame = min_not_zero(adapter->vf_res->max_mtu, max_frame);

LIBIE_MAX_RX_FRM_LEN (16382) is the multi-descriptor scatter/gather frame ceiling, not a single-queue value, and it exceeds the E810 MAC frame size maximum of 9728. Per the E810 datasheet (613875-009 section 13.2.2.17.1) the Tx frame-size register PRTDCB_TDPUC.MAX_TXFRAME has a maximum of
0x2600 (9728); larger frames are discarded. The in-tree ice driver encodes the same value as ICE_AQ_SET_MAC_FRAME_SIZE_MAX (== LIBIE_MAX_RX_BUF_LEN == 9728), and the VF clamped max_frame to IAVF_MAX_RXBUFFER (9728) before this commit.

When the PF advertises vf_res->max_mtu as 0, min_not_zero() leaves max_frame at 16382. The Linux ice PF advertises max_mtu = port MAC frame size (<= 9728), so a VF behind ice never sends more than that. The ESXi "icen" PF on E810 advertises max_mtu as 0, so the VF sends max_pkt_size = 16382, which icen rejects while programming the queue context for VIRTCHNL_OP_CONFIG_VSI_QUEUES (opcode 6):

icen_ConfigureTxQueue: VSI 8: Failed to set LAN Tx queue
context for
absolute Tx queue 64, Error:
ICE_ERR_PARAM

	indrv_SendMsgToVf: VF 0: Failed opcode 6, Error -5

	iavf 0000:03:00.0: PF returned error -5 (IAVF_ERR_PARAM) to our

request 6
iavf 0000:03:00.0 ethX: NETDEV WATCHDOG: transmit queue N timed
out

The VF's queues never come up; under SR-IOV passthrough the mis- programmed queue can also trigger a fatal IOMMU fault in the guest. Forcing only max_pkt_size back to 9728 (and leaving the Page Pool rx_buf_len/ databuffer_size untouched) makes the VF come up; databuffer_size is not involved. This was confirmed on two E810 NVM revisions (3.00 and 4.51) and two icen versions (1.14.2.0 and the latest 2.3.3.0): all reject the unpatched VF and accept the patched one, so the trigger is the icen PF behaviour, not the firmware or icen revision. Reported by several users on
E810 + ESXi icen with v6.10+ guests:

driver-issue-on-Linux-6-12/m-p/1737490

enabled-vms-network-adaptor-goes-d.html

Cap max_frame at the single-buffer hardware limit, restoring the pre- Page-Pool behaviour while keeping the Page Pool rx_buf_len unchanged.

Fixes 5fa4caff59f2 ("iavf: switch to Page Pool")
---
 drivers/net/ethernet/intel/iavf/iavf_virtchnl.c | 8 ++++++++
 1 file changed, 8 insertions(+)
drivers/net/ethernet/intel/iavf/iavf_virtchnl.c +1 -0
index 4f2defd2331b..2632eeed776c 100644
@@ -382,6 +382,14 @@ void iavf_configure_queues(struct iavf_adapter
 *adapter)
 
 	max_frame = LIBIE_MAX_RX_FRM_LEN(adapter->rx_rings->pp-
 >p.offset);
 	max_frame = min_not_zero(adapter->vf_res->max_mtu, max_frame);
+	/* The PF programs max_pkt_size into the per-queue Rx context

"rxmax".

drivers/net/ethernet/intel/iavf/iavf_virtchnl.c +1 -0
+	 * LIBIE_MAX_RX_FRM_LEN is the multi-descriptor (S/G) frame

ceiling

drivers/net/ethernet/intel/iavf/iavf_virtchnl.c +1 -0
+	 * (16382), but that exceeds the E810 max MAC frame size

(9728); some

drivers/net/ethernet/intel/iavf/iavf_virtchnl.c +1 -0
+	 * PFs reject the out-of-range value with

VIRTCHNL_STATUS_ERR_PARAM.

drivers/net/ethernet/intel/iavf/iavf_virtchnl.c +4 -0
+	 * Cap it at the single-buffer HW limit (== the MAC frame max),
+	 * restoring the pre-Page-Pool behaviour.
+	 */
+	max_frame = min(max_frame, LIBIE_MAX_RX_BUF_LEN);
 
 	if (adapter->current_op != VIRTCHNL_OP_UNKNOWN) {
 		/* bail because we already have a command pending */
Quoted 3 lines

base-commit: fbc6a80cb5d3fd4ac4b56e8c9d791dd17be890c4 --
2.43.0