ibmvnic: honour flexible in-range MTU values
cover ibmvnic: honour flexible in-range MTU values
ibmvnic currently only keeps an MTU of 1500 or 9000. Any other in-range value is accepted by ip(8) and then silently reverted: the vnicserver answers REQ_MTU with PARTIALSUCCESS and the size the backing device settled on, and the driver treats that as rejection and falls back.
Customers want flexible in-range MTUs (for example 1414, or values in the 2000..8000 range) rather than being forced onto the 1500/9000 buckets. Today those requests either snap back quietly or, in the mid-range case, can fail pool allocation and leave the interface down.
This series makes those requests stick, hardens the reset path an MTU change goes through, and (patch 6) skips the reset when the backing device and current buffers already cover the new MTU.
Posted as a single net-next series so the flexible-MTU change and the reset-path hardening can be reviewed together. Patches 1-5 address user-visible bugs and carry Fixes: tags; patch 6 is the follow-on optimization. Happy to resplit 1-5 to net and leave 6 for net-next if that is preferred.
Patch 2 is the functional fix: keep the requested MTU when PARTIALSUCCESS covers it, and converge on the device size when it does not.
Patch 1 is a prerequisite. Once req_mtu may sit below the backing buffer size, the existing pool-entry guard (which budgets with req_mtu) overshoots and mid-range MTU changes can fail allocation. It must not be backported without patch 2, or separated from it.
Patches 3-5 clean up side effects of the MTU reset: skip unmaps for buffers the VIOS already forgot across a CRQ reconnect, and allocate new pools / long term buffers before releasing the old ones so a failed realloc refuses the MTU change instead of stranding the interface.
Patch 6 records the covering backing size and skips wait_for_reset() when the new MTU fits in both that size and the current buffers.
Tested on PowerVM ibmvnic with MTU sweeps including 1414 and the 2000..9000 range that previously failed to allocate.
Mingming Cao (6):
ibmvnic: cap rx pool entries against the real buffer size
ibmvnic: honour the requested mtu instead of reverting to a fallback
ibmvnic: do not unmap long term buffers across a crq reconnect
ibmvnic: allocate new buffer pools before releasing the old ones
ibmvnic: allocate a new long term buffer before freeing the old one
ibmvnic: change the mtu without a reset where the buffers allow it
drivers/net/ethernet/ibm/ibmvnic.c | 188 ++++++++++++++++++++--------- drivers/net/ethernet/ibm/ibmvnic.h | 3 + 2 files changed, 133 insertions(+), 58 deletions(-)
Signature · 1 line
2.50.1 (Apple Git-155)
PATCH 1/6 ibmvnic: cap rx pool entries against the real buffer size
Changing the mtu from 1500 to a value in the 2000..8000 range can fail pool allocation and leave the interface down:
ibmvnic 30000008: Couldn't alloc long term buffer __alloc_pages: ... order:N, mode:0x...
(9000 happens to succeed, which is how the middle of the range stood out.)
That is the non-bucket case this series exists to allow: the guest mtu is not one of the sizes the backing buffers were sized for. Until the next patch, req_mtu was always forced back to a size the device carries, so the pool guard happened to measure the right buffer. Once arbitrary in-range mtus are honoured, that luck goes away.
send_request_cap() limits how many entries a pool may hold so the whole pool fits in one long term buffer set:
max_entries = IBMVNIC_LTB_SET_SIZE /
(adapter->req_mtu + IBMVNIC_BUFFER_HLEN);The rx buffers are not sized from req_mtu. init_rx_pools() sizes them from cur_rx_buf_sz, which comes from the login response and describes the backing device. When the VIOS has the device at jumbo (cur_rx_buf_sz around 9014) but the guest mtu is still smaller, the guard measures a buffer much smaller than the one really allocated. The entry count passes through untouched and the pool asks for several times the space the guard assumed - enough that dma_alloc_coherent() / __alloc_pages() refuses the request on kernels whose per-pool LTB budget is a single allocation rather than a large set.
With a backing size of 9014 the buffers are ALIGN(9014, L1_CACHE_BYTES) = 9088 bytes, while an mtu of 2000 leaves the guard working from 2514. The overshoot only shrinks as req_mtu approaches the backing size, so the largest mtus are the safe ones and everything below drifts.
Cap the count against the size the buffers are really allocated at. reuse_rx_pools() has to compare against the same number or it would see a difference on every reset and reallocate pools that are already the right shape. clean_rx_pools() must walk rx_pool->size for the same reason, or a clamped pool would be cleaned past its end.
Comes ahead of the next patch, which is what lets req_mtu sit below the backing size. Keep the two together when backporting.
--- drivers/net/ethernet/ibm/ibmvnic.c | 26 +++++++++++++++++++++----- 1 file changed, 21 insertions(+), 5 deletions(-)
drivers/net/ethernet/ibm/ibmvnic.c +21 -5
@@ -1021,6 +1021,24 @@ static void release_rx_pools(struct ibmvnic_adapter *adapter) adapter->prev_rx_pool_size =0 ; } +/** +* rx_pool_entries() - Number of buffers one rx pool may hold +* @adapter: ibmvnic adapter +* +* send_request_cap() budgets the entry count with req_mtu, but the +* buffers are sized from cur_rx_buf_sz. Cap against that size here. +*/ +static u64 rx_pool_entries (struct ibmvnic_adapter *adapter) +{ +u64 buff_size =ALIGN (adapter->cur_rx_buf_sz,L1_CACHE_BYTES ); + +if (!buff_size) +return adapter->req_rx_add_entries_per_subcrq; + +return min_t (u64, adapter->req_rx_add_entries_per_subcrq, +IBMVNIC_LTB_SET_SIZE / buff_size); +} +/** * reuse_rx_pools() - Check if the existing rx pools can be reused. * @adapter: ibmvnic adapter
@@ -1048,7 +1066,7 @@ static bool reuse_rx_pools(struct ibmvnic_adapter *adapter) new_num_pools = adapter->req_rx_queues; old_pool_size = adapter->prev_rx_pool_size; - new_pool_size = adapter->req_rx_add_entries_per_subcrq; + new_pool_size =rx_pool_entries (adapter); old_buff_size = adapter->prev_rx_buf_sz; new_buff_size = adapter->cur_rx_buf_sz;
@@ -1082,7 +1100,7 @@ static int init_rx_pools(struct net_device *netdev)u64 buff_size;int i, j, rc; - pool_size = adapter->req_rx_add_entries_per_subcrq; + pool_size =rx_pool_entries (adapter); num_pools = adapter->req_rx_queues; buff_size = adapter->cur_rx_buf_sz;
@@ -2000,7 +2018,6 @@ static void clean_rx_pools(struct ibmvnic_adapter *adapter) {struct ibmvnic_rx_pool *rx_pool;struct ibmvnic_rx_buff *rx_buff; -u64 rx_entries;int rx_scrqs;int i, j;
@@ -2008,7 +2025,6 @@ static void clean_rx_pools(struct ibmvnic_adapter *adapter)return ; rx_scrqs = adapter->num_active_rx_pools; - rx_entries = adapter->req_rx_add_entries_per_subcrq;/* Free any remaining skbs in the rx buffer pools */ for (i =0 ; i < rx_scrqs; i++) {
@@ -2017,7 +2033,7 @@ static void clean_rx_pools(struct ibmvnic_adapter *adapter)continue ;netdev_dbg (adapter->netdev,"Cleaning rx_pool[%d]\n" , i); -for (j =0 ; j < rx_entries; j++) { +for (j =0 ; j < rx_pool->size; j++) { rx_buff = &rx_pool->rx_buff[j];if (rx_buff && rx_buff->skb) {dev_kfree_skb_any (rx_buff->skb);
Signature · 1 line
2.50.1 (Apple Git-155)
PATCH 2/6 ibmvnic: honour the requested mtu instead of reverting to a fallback
In reply to Mingming CaoNo mtu other than 1500 or 9000 can be set on an ibmvnic interface. Any other value is accepted by ip(8) but silently reverts:
# ip link set mtu 1414 env8 ibmvnic 30000008: req=1428, rsp=1514 in mtu queue, retrying. mtu of 1428 is not supported. Reverting. ibmvnic 30000008: req=1514, rsp=1514 in mtu queue, retrying. mtu of 1514 is not supported. Reverting. # ip link show env8 | grep -o 'mtu [0-9]*' mtu 1500
The backing device runs at one of a small set of fixed sizes, so the vnicserver answers every REQ_MTU between the advertised minimum and maximum with PARTIALSUCCESS and the size it settled on: 1514 for anything up to 1514, 9014 for anything above that. It is reporting what the backing device carries. Requests outside the advertised range never get here, since dev_set_mtu() rejects them against netdev->min_mtu and netdev->max_mtu.
The driver read that as a rejection. It reverted req_mtu to fallback.mtu and re-ran the exchange, which is why the log above shows the revert twice: the fallback is in range as well, so it partially succeeds too and the mtu settles on whatever the fallback was.
Treat a PARTIALSUCCESS that covers the request as the confirmation it is, and keep the requested value.
The response can also come back below the request. The advertised maximum is the protocol ceiling rather than a promise about the current backing configuration, so a device that is not set up for the larger size answers with the smaller one it does carry. That does not cover the request and must not be published, so it keeps the existing retry, which now also stores the response value rather than reverting to a fallback. It therefore converges on a size the device can carry instead of re-requesting the same number. Other capabilities are unchanged.
Later patches in this series make the resets that follow a real mtu change safe (and, on net-next, skip them when the buffers already fit).
--- drivers/net/ethernet/ibm/ibmvnic.c | 34 ++++++++++++++++++------------ 1 file changed, 20 insertions(+), 14 deletions(-)
drivers/net/ethernet/ibm/ibmvnic.c +20 -14
@@ -5502,12 +5502,17 @@ static void handle_request_cap_rsp(union ibmvnic_crq *crq, {struct device *dev = &adapter->vdev->dev;u64 *req_value; +u64 rsp_value;char *name; +u16 cap;atomic_dec (&adapter->running_cap_crqs);netdev_dbg (adapter->netdev,"Outstanding request-caps: %d\n" ,atomic_read (&adapter->running_cap_crqs)); -switch (be16_to_cpu (crq->request_capability_rsp.capability)) { + + cap =be16_to_cpu (crq->request_capability_rsp.capability); + +switch (cap) {case REQ_TX_QUEUES : req_value = &adapter->req_tx_queues; name ="tx" ;
@@ -5546,21 +5551,22 @@ static void handle_request_cap_rsp(union ibmvnic_crq *crq,case SUCCESS :break ;case PARTIALSUCCESS : -dev_info (dev,"req=%lld, rsp=%ld in %s queue, retrying.\n" , - *req_value, - (long )be64_to_cpu (crq->request_capability_rsp.number), - name); - -if (be16_to_cpu (crq->request_capability_rsp.capability) == -REQ_MTU ) { -pr_err ("mtu of %llu is not supported. Reverting.\n" , - *req_value); - *req_value = adapter->fallback.mtu; - }else { - *req_value = -be64_to_cpu (crq->request_capability_rsp.number); + rsp_value =be64_to_cpu (crq->request_capability_rsp.number); + +/* Covering PARTIALSUCCESS: keep the request. Otherwise +* retry with rsp_value. +*/ +if (cap ==REQ_MTU && rsp_value >= *req_value) { +netdev_dbg (adapter->netdev, +"backing mtu %llu covers requested %llu\n" , + rsp_value, *req_value); +break ; } +dev_info (dev,"req=%lld, rsp=%lld in %s queue, retrying.\n" , + *req_value, rsp_value, name); + *req_value = rsp_value; +send_request_cap (adapter,1 );return ;default :
Signature · 1 line
2.50.1 (Apple Git-155)
PATCH 3/6 ibmvnic: do not unmap long term buffers across a crq reconnect
In reply to Mingming CaoOnce in-range mtus are honoured, changing the mtu - especially up to jumbo - becomes a normal admin action rather than a rare failover path. Those changes can make the VIOS reconfigure the backing device and re-establish the crq connection. Every long term buffer mapping belongs to the connection it was registered on, so the VIOS drops all of them, and the driver then asks it to unmap buffers it no longer knows about:
ibmvnic 30000003 env3: MTU change 1400->9000: slow path (reset required) ibmvnic 30000003: Partner initialization complete
ibmvnic 30000003: Partner protocol version is 1
ibmvnic 30000003: Error 4 in REQUEST_UNMAP_RSP
ibmvnic 30000003: Error 4 in REQUEST_UNMAP_RSP
...
Error 4 is H_PARAMETER, one per buffer still on the books. Traffic keeps flowing; the damage is to the log. A single jumbo mtu change prints a flood of these lines, which drowns out real failures and makes every reset look broken.
free_long_term_buff() decides from reset_reason alone, listing the resets after which the VIOS is known to have unmapped everything. That cannot describe this case. The connection is re-established by the partner partway through the reset, so whether an unmap is still valid depends on when it is sent rather than on why the reset was started, which is why the errors come and go between otherwise identical runs.
Give the connection a generation, bump it whenever the crq goes away, and record it in each long term buffer as that buffer is mapped. One whose generation no longer matches was mapped on a connection that has since gone, so release it locally and leave the VIOS alone. The field fits in the padding that already followed map_id, so the struct stays 32 bytes and the few hundred mappings an adapter can hold cost nothing extra.
The reset_reason tests are now redundant, since all three tear the connection down, but leave them for the moment.
--- drivers/net/ethernet/ibm/ibmvnic.c | 30 +++++++++++++++++++++++------- drivers/net/ethernet/ibm/ibmvnic.h | 2 ++ 2 files changed, 25 insertions(+), 7 deletions(-)
drivers/net/ethernet/ibm/ibmvnic.c +23 -7
@@ -495,6 +495,9 @@ static int alloc_long_term_buff(struct ibmvnic_adapter *adapter, adapter->fw_done_rc =0 ;reinit_completion (&adapter->fw_done); +/* Snapshot gen before the map wait so a reconnect mid-wait is stale. */ + ltb->crq_gen = adapter->crq.gen; + rc =send_request_map (adapter, ltb->addr, ltb->size, ltb->map_id);if (rc) {dev_err (dev,"send_request_map failed, rc = %d\n" , rc);
@@ -529,11 +532,12 @@ static void free_long_term_buff(struct ibmvnic_adapter *adapter,if (!ltb->buff)return ; -/* VIOS automatically unmaps the long term buffer at remote -* end for the following resets: -* FAILOVER, MOBILITY, TIMEOUT. +/* Skip unmap if mapped on a prior crq generation, or after resets +* where the VIOS has already dropped mappings (FAILOVER/MOBILITY/ +* TIMEOUT). */ -if (adapter->reset_reason !=VNIC_RESET_FAILOVER && +if (ltb->crq_gen == adapter->crq.gen && + adapter->reset_reason !=VNIC_RESET_FAILOVER && adapter->reset_reason !=VNIC_RESET_MOBILITY && adapter->reset_reason !=VNIC_RESET_TIMEOUT )send_request_unmap (adapter, ltb->map_id);
@@ -5974,6 +5978,18 @@ static int handle_query_phys_parms_rsp(union ibmvnic_crq *crq,return rc; } +/** +* ibmvnic_crq_deactivate() - Mark the crq connection inactive +* @crq: crq queue +* +* Bump gen so LTB mappings from the old connection can be freed locally. +*/ +static void ibmvnic_crq_deactivate (struct ibmvnic_crq_queue *crq) +{ + crq->active = false; + crq->gen++; +} +static void ibmvnic_handle_crq (union ibmvnic_crq *crq,struct ibmvnic_adapter *adapter) {
@@ -6036,7 +6052,7 @@ static void ibmvnic_handle_crq(union ibmvnic_crq *crq,return ;case IBMVNIC_CRQ_XPORT_EVENT :netif_carrier_off (netdev); - adapter->crq.active = false; +ibmvnic_crq_deactivate (&adapter->crq);/* terminate any thread waiting for a response * from the device */
@@ -6241,7 +6257,7 @@ static int ibmvnic_reset_crq(struct ibmvnic_adapter *adapter)memset (crq->msgs,0 ,PAGE_SIZE ); crq->cur =0 ; - crq->active = false; +ibmvnic_crq_deactivate (crq);/* And re-open it again */ rc = plpar_hcall_norets(H_REG_CRQ, vdev->unit_address,
@@ -6276,7 +6292,7 @@ static void release_crq_queue(struct ibmvnic_adapter *adapter) DMA_BIDIRECTIONAL);free_page ((unsigned long )crq->msgs); crq->msgs =NULL ; - crq->active = false; +ibmvnic_crq_deactivate (crq); }static int init_crq_queue (struct ibmvnic_adapter *adapter)
drivers/net/ethernet/ibm/ibmvnic.h +2 -0
@@ -794,6 +794,7 @@ struct ibmvnic_crq_queue {/* Used for serialization of msgs, cur */ spinlock_t lock;bool active; +u32 gen;/* bumped when the crq connection drops */ char name[32 ]; };
@@ -839,6 +840,7 @@ struct ibmvnic_long_term_buff {dma_addr_t addr;u64 size;u8 map_id; +u32 crq_gen;/* crq.gen when this buffer was mapped */ };struct ibmvnic_ltb_set {
Signature · 1 line
2.50.1 (Apple Git-155)
PATCH 4/6 ibmvnic: allocate new buffer pools before releasing the old ones
In reply to Mingming CaoAn mtu change goes through wait_for_reset() and then init_rx_pools() / init_tx_pools(). Those helpers release the existing pools before allocating their replacements:
release_rx_pools(adapter);
adapter->rx_pool = kzalloc_objs(struct ibmvnic_rx_pool, num_pools);
if (!adapter->rx_pool) {
dev_err(dev, "Failed to allocate rx pools\n");
return -ENOMEM;
}If that allocation fails mid-reset the interface is left with no pools and stays down after what should have been a refused mtu change, when it could have kept running with the pools it still had.
Allocate first and only release once the allocation has succeeded. The tx side has to swap ->tx_pool and ->tso_pool in together, so that release_tx_pools() never sees one set without the other.
These are small allocations and failing them is unlikely, so this is about the ordering rather than about any failure seen in the field. alloc_long_term_buff() has the same shape for a much larger allocation on the mtu-grow path and is dealt with next.
--- drivers/net/ethernet/ibm/ibmvnic.c | 37 ++++++++++++++++-------------- 1 file changed, 20 insertions(+), 17 deletions(-)
drivers/net/ethernet/ibm/ibmvnic.c +20 -17
@@ -1098,6 +1098,7 @@ static int init_rx_pools(struct net_device *netdev) {struct ibmvnic_adapter *adapter =netdev_priv (netdev);struct device *dev = &adapter->vdev->dev; +struct ibmvnic_rx_pool *new_rx_pool;struct ibmvnic_rx_pool *rx_pool;u64 num_pools;u64 pool_size;/* # of buffers in one pool */
@@ -1113,15 +1114,16 @@ static int init_rx_pools(struct net_device *netdev) goto update_ltb; } -/* Allocate/populate the pools. */ -release_rx_pools (adapter); - - adapter->rx_pool = kzalloc_objs(struct ibmvnic_rx_pool, num_pools); -if (!adapter->rx_pool) { +/* Allocate before release so a failure keeps the old pools. */ + new_rx_pool = kzalloc_objs(struct ibmvnic_rx_pool, num_pools); +if (!new_rx_pool) {dev_err (dev,"Failed to allocate rx pools\n" );return -ENOMEM ; } +release_rx_pools (adapter); + adapter->rx_pool = new_rx_pool; +/* Set num_active_rx_pools early. If we fail below after partial * allocation, release_rx_pools() will know how many to look for. */
@@ -1333,6 +1335,8 @@ static int init_tx_pools(struct net_device *netdev) {struct ibmvnic_adapter *adapter =netdev_priv (netdev);struct device *dev = &adapter->vdev->dev; +struct ibmvnic_tx_pool *new_tso_pool; +struct ibmvnic_tx_pool *new_tx_pool;int num_pools;u64 pool_size;/* # of buffers in pool */ u64 buff_size;
@@ -1349,26 +1353,25 @@ static int init_tx_pools(struct net_device *netdev) goto update_ltb; } -/* Allocate/populate the pools. */ -release_tx_pools (adapter); - +/* Allocate before release so a failure keeps the old pools. */ pool_size = adapter->req_tx_entries_per_subcrq; num_pools = adapter->num_active_tx_scrqs; - adapter->tx_pool = kzalloc_objs(struct ibmvnic_tx_pool, num_pools); -if (!adapter->tx_pool) + new_tx_pool = kzalloc_objs(struct ibmvnic_tx_pool, num_pools); +if (!new_tx_pool)return -ENOMEM ; - adapter->tso_pool = kzalloc_objs(struct ibmvnic_tx_pool, num_pools); -/* To simplify release_tx_pools() ensure that ->tx_pool and -* ->tso_pool are either both NULL or both non-NULL. -*/ -if (!adapter->tso_pool) { -kfree (adapter->tx_pool); - adapter->tx_pool =NULL ; + new_tso_pool = kzalloc_objs(struct ibmvnic_tx_pool, num_pools); +if (!new_tso_pool) { +kfree (new_tx_pool);return -ENOMEM ; } +/* Swap both in together for release_tx_pools(). */ +release_tx_pools (adapter); + adapter->tx_pool = new_tx_pool; + adapter->tso_pool = new_tso_pool; +/* Set num_active_tx_pools early. If we fail below after partial * allocation, release_tx_pools() will know how many to look for. */
Signature · 1 line
2.50.1 (Apple Git-155)
PATCH 5/6 ibmvnic: allocate a new long term buffer before freeing the old one
In reply to Mingming CaoWhen an mtu change grows the pools, alloc_long_term_buff() releases the existing buffer as soon as it knows the size has changed, and only then asks for the replacement:
if (!reuse_ltb(ltb, size)) {
prev = ltb->size;
free_long_term_buff(adapter, ltb);
}
if (ltb->buff) {
...
} else {
ltb->buff = dma_alloc_coherent(dev, size, <b->addr,
GFP_KERNEL);
if (!ltb->buff) {
dev_err(dev, "Couldn't alloc long term buffer\n");
return -ENOMEM;
}This one is megabytes of physically contiguous memory, so it can fail while there is plenty free. By then the old buffer is gone and the pool it belongs to has nothing, which takes the interface down.
Allocate into a local buffer and only commit it once the allocation has succeeded. A failure now returns with the old buffer still in place, so the mtu change is refused and the interface keeps running at the size it had.
Both buffers are live across the swap, so an mtu increase now needs the old and the new at once. That makes the allocation somewhat more likely to fail in the low memory conditions this is meant to survive, but failing with the old buffer intact is still the better outcome: the caller can decline the change instead of losing the interface.
The new map id is taken before the old one is released, so the two differ across the swap. That is harmless with 255 of them.
--- drivers/net/ethernet/ibm/ibmvnic.c | 30 ++++++++++++++++-------------- 1 file changed, 16 insertions(+), 14 deletions(-)
drivers/net/ethernet/ibm/ibmvnic.c +16 -14
@@ -455,6 +455,7 @@ static bool reuse_ltb(struct ibmvnic_long_term_buff *ltb, int size)static int alloc_long_term_buff (struct ibmvnic_adapter *adapter,struct ibmvnic_long_term_buff *ltb,int size) { +struct ibmvnic_long_term_buff new_ltb = {};struct device *dev = &adapter->vdev->dev;u64 prev =0 ;int rc;
@@ -464,28 +465,29 @@ static int alloc_long_term_buff(struct ibmvnic_adapter *adapter,"LTB size changed from 0x%llx to 0x%x, reallocating\n" , ltb->size, size); prev = ltb->size; -free_long_term_buff (adapter, ltb); - } -if (ltb->buff) { -dev_dbg (dev,"Reusing LTB [map %d, size 0x%llx]\n" , - ltb->map_id, ltb->size); - }else { - ltb->buff =dma_alloc_coherent (dev, size, <b->addr, -GFP_KERNEL ); -if (!ltb->buff) { +/* Allocate first so failure leaves the old buffer in place. */ + new_ltb.buff =dma_alloc_coherent (dev, size, &new_ltb.addr, +GFP_KERNEL ); +if (!new_ltb.buff) {dev_err (dev,"Couldn't alloc long term buffer\n" );return -ENOMEM ; } - ltb->size = size; + new_ltb.size = size; - ltb->map_id =find_first_zero_bit (adapter->map_ids, -MAX_MAP_ID ); -bitmap_set (adapter->map_ids, ltb->map_id,1 ); + new_ltb.map_id =find_first_zero_bit (adapter->map_ids, +MAX_MAP_ID ); +bitmap_set (adapter->map_ids, new_ltb.map_id,1 );dev_dbg (dev,"Allocated new LTB [map %d, size 0x%llx was 0x%llx]\n" , - ltb->map_id, ltb->size, prev); + new_ltb.map_id, new_ltb.size, prev); + +free_long_term_buff (adapter, ltb); + *ltb = new_ltb; + }else { +dev_dbg (dev,"Reusing LTB [map %d, size 0x%llx]\n" , + ltb->map_id, ltb->size); }/* Ensure ltb is zeroed - specially when reusing it. */
Signature · 1 line
2.50.1 (Apple Git-155)
PATCH 6/6 ibmvnic: change the mtu without a reset where the buffers allow it
In reply to Mingming CaoWith the earlier patches in this series, in-range mtus are honoured instead of snapped to 1500 or 9000. Every such change still cycles the adapter through wait_for_reset(), which tears down the CRQ, logs in again and rebuilds every queue. The link goes down for around two seconds each time, even when the change needs nothing from the VIOS.
Most of them do not. The backing device runs at one of a small set of fixed sizes and the vnicserver reports that size when it answers REQ_MTU, so any mtu within it is already carried end to end. Record it as adapter->backing_mtu (set when the covering PARTIALSUCCESS path from the honour-mtu patch accepts a request) and skip the reset when the new mtu is within it and the tx buffers are large enough as they are, which covers every decrease and the increases that stay inside the current buffer size.
Both conditions are needed. Buffer sizes are rounded up to a cache line, so buffers alone would also admit an mtu somewhat past what the backing device carries: with a 1514 backing mtu, everything up to 1532 aligns to the same 1536-byte buffer. Those frames would leave the partition and be dropped by a backing device still configured for the smaller size.
An increase past backing_mtu still resets, since it has to be put to the vnicserver rather than assumed. Whatever comes back becomes the new backing_mtu, so if the device does move up, later changes within the larger size settle without a reset, and if it does not, the mtu is renegotiated down to what the device carries and the fast path keeps measuring against the truth.
Buffers are left at their existing size on a decrease rather than shrunk. reuse_tx_pools() compares prev_mtu against req_mtu, so the next reset for any reason reallocates them.
--- drivers/net/ethernet/ibm/ibmvnic.c | 35 +++++++++++++++++++++++++++--- drivers/net/ethernet/ibm/ibmvnic.h | 1 + 2 files changed, 33 insertions(+), 3 deletions(-)
drivers/net/ethernet/ibm/ibmvnic.c +32 -3
@@ -3705,8 +3705,34 @@ out:static int ibmvnic_change_mtu (struct net_device *netdev,int new_mtu) {struct ibmvnic_adapter *adapter =netdev_priv (netdev); +u64 new_mtu_with_hdr = new_mtu +ETH_HLEN ; +u64 old_buff_size, new_buff_size; + +if (adapter->req_mtu == new_mtu_with_hdr) +return 0 ; + + old_buff_size =ALIGN (adapter->prev_mtu +VLAN_HLEN ,L1_CACHE_BYTES ); + new_buff_size =ALIGN (new_mtu_with_hdr +VLAN_HLEN ,L1_CACHE_BYTES ); + +/* Skip the reset when backing_mtu and the current buffers already +* cover the new mtu. Keep desired.mtu in sync with req_mtu. +*/ +if (new_mtu_with_hdr <= adapter->backing_mtu && + new_buff_size <= old_buff_size) { +netdev_dbg (netdev,"mtu %u->%d without reset\n" , + netdev->mtu, new_mtu); + +WRITE_ONCE (netdev->mtu, new_mtu); + adapter->req_mtu = new_mtu_with_hdr; + adapter->desired.mtu = new_mtu_with_hdr; + +return 0 ; + } + +netdev_dbg (netdev,"mtu %u->%d needs larger buffers, resetting\n" , + netdev->mtu, new_mtu); - adapter->desired.mtu = new_mtu +ETH_HLEN ; + adapter->desired.mtu = new_mtu_with_hdr;return wait_for_reset (adapter); }
@@ -5558,14 +5584,17 @@ static void handle_request_cap_rsp(union ibmvnic_crq *crq,switch (crq->request_capability_rsp.rc.code) {case SUCCESS : +if (cap ==REQ_MTU ) + adapter->backing_mtu = *req_value;break ;case PARTIALSUCCESS : rsp_value =be64_to_cpu (crq->request_capability_rsp.number); -/* Covering PARTIALSUCCESS: keep the request. Otherwise -* retry with rsp_value. +/* Covering PARTIALSUCCESS: keep the request and record +* backing_mtu. Otherwise retry with rsp_value. */ if (cap ==REQ_MTU && rsp_value >= *req_value) { + adapter->backing_mtu = rsp_value;netdev_dbg (adapter->netdev,"backing mtu %llu covers requested %llu\n" , rsp_value, *req_value);
drivers/net/ethernet/ibm/ibmvnic.h +1 -0
@@ -1020,6 +1020,7 @@ struct ibmvnic_adapter {u64 max_mtu;u64 req_mtu;u64 prev_mtu; +u64 backing_mtu;/* mtu the backing device currently carries */ u64 max_multicast_filters;u64 vlan_header_insertion;u64 rx_vlan_header_insertion;
Signature · 1 line
2.50.1 (Apple Git-155)
Jakub Kicinski
maintainerQuoted 1 line · On Wed, 5 Aug 2026 15:43:55 -0700 Mingming Cao wrote:
Subject: [PATCH net-next v1 0/6] ibmvnic: honour flexible in-range MTU values
Warning: drivers/net/ethernet/ibm/ibmvnic.c:1031 No description found for return value of 'rx_pool_entries'
please DO NOT repost this until your previous series is reviewed and merged. Please read:
https://www.kernel.org/doc/html/latest/process/maintainer-netdev.html
There aren't supposed to be more than 15 outstanding patches form one person or company to one tree. We are flooded with AI-assisted code so reviews are slow. Your place in review queue depends on your credibility with the community so please help review other patches or just wait your turn.
Quoted 5 lines · Jakub Kicinski
Quoted 1 line · On Wed, 5 Aug 2026 15:43:55 -0700 Mingming Cao wrote:
Subject: [PATCH net-next v1 0/6] ibmvnic: honour flexible in-range MTU values
Warning:
drivers/net/ethernet/ibm/ibmvnic.c:1031No description found for return value of 'rx_pool_entries'
Quoted 8 lines
please DO NOT repost this until your previous series is reviewed and merged. Please read:
https://www.kernel.org/doc/html/latest/process/maintainer-netdev.html
There aren't supposed to be more than 15 outstanding patches form one person or company to one tree. We are flooded with AI-assisted code so reviews are slow. Your place in review queue depends on your credibility with the community so please help review other patches or just wait your turn.
Hi Jakub,
Thanks for the feedback.
Sorry for violating the outstanding patch limit. I wasn't aware of that rule, so thanks for pointing me to it.
I understand the review queue is already quite busy. I'll wait for the ibmveth MQ support series (v4, 14 patches) to progress through review before reposting this one. In the meantime, I'm working through Sakashi's review comments on the ibmveth VQ v4 series.
For the ibmvnic MTU support series, I'll hold off on posting a v2 for now and will address the kernel-doc warning in the next revision.
Thanks again for the guidance.
Mingming