ibmveth: Add multi-queue RX support
v4 cover ibmveth: Add multi-queue RX support
Hi,
Power11 PHYP adds Virtual Ethernet multi-queue (MQ) RX: multiple logical-LAN RX queues, per-queue buffer posting, and completion delivery. Guest Linux did not use that; ibmveth still registered one RX queue even when PHYP was MQ-capable.
This series adds the ibmveth MQ client for net-next. When PHYP advertises IBMVETH_ILLAN_RX_MULTI_QUEUE_SUPPORT via H_ILLAN_ATTRIBUTES, probe enables MQ with a default RX count of min(num_online_cpus(), 8) (same cap as TX today); ethtool -L can raise RX up to 16. Packets are received on per-queue NAPI. Older firmware without the bit is unchanged. Queue selection remains firmware-defined (PHYP hash). Ethtool RSS hash get/set for that algorithm is deferred to a follow-up series so this one stays MQ datapath only.
User-visible bits: ethtool -l/-L (channels), ethtool -S and ndo_get_stats64 (per-queue + aggregate), and a read-only debugfs buffer_pools dump (v3's multi-line sysfs dump moved to debugfs; the historical queue-0 poolN/ sysfs ABI is unchanged).
Background:
ibmveth today uses one logical LAN, one set of buffer pools, and one NAPI context. PHYP MQ mode gives each RX queue its own handle (post via H_ADD_LOGICAL_LAN_BUFFERS_QUEUE, subordinate register via H_REG_LOGICAL_LAN_QUEUE). Legacy firmware keeps the original hcall path.
Series layout (14 patches):
1-2 Hypercall wrappers; MQ adapter layout (MAX_RX_QUEUES stays 1) 3-8 Refactor open/close into queue-aware helpers (still SQ runtime) 9 Enable MQ datapath at probe/open (subordinate register helpers
land here with first use; probe pool-kobj fail paths put already-created pool kobjects)
10-11 Per-queue RX/TX stats; debugfs buffer_pools 12-13 Incremental RX resize; ethtool set_channels
14 Poll/close fixes after aggressive ethtool -L
Patch 2 is layout-only. Statistics types land with first use (hcall_stats in 7, rx/tx qstats in 10). Patches 3-8 do not change runtime behaviour until patch 9. v3's separate big-bang open/close pipeline patch is gone; wiring is incremental across 3-8.
Testing:
ppc64le PowerVM LPAR, MQ-capable firmware:
* ethtool -L cycling (16/1/8/11/1/3/16/8/1) with ping
* ethtool -L under iperf3; link down/up during traffic
* ifdown/ifup under iperf3 RX+TX (SQ and MQ)
* Legacy firmware (no MQ bit): open/close/stress on helper path
* allmodconfig W=1 clean at each patch (bisect-friendly restack)
Changes in v4:
Addresses Simon's v3 review and related fixes:
* First-use helpers/includes (irqdomain.h with first dispose); no unused statics; dropped orphan open/close pipeline patch
* Open/close unwind (free LAN before RX pools); no double TX teardown
* MQ open: replenish all queues before PHYP unmask; H_FUNCTION on subordinate register is a hard open failure
* Resize/set_channels hardenings; stats probe-lifetime + sum-on-read; buffer_pools diagnostic on debugfs
* Patch 9: put already-created pool kobjects on probe failure paths
* Patch 14: correlator skip, skb tailroom check, napi_complete_done shutdown return < budget
Comments welcome.
---
Previous versions
v3: https://lore.kernel.org/r/[email protected]
v2: https://lore.kernel.org/r/[email protected]
v1: https://lore.kernel.org/r/[email protected]
v3 review (Simon, from 05/15):
https://lore.kernel.org/r/[email protected]
PATCH v4 1/14 ibmveth: Add MQ RX hypercall wrappers and call definitions
Single-queue ibmveth only needs h_register_logical_lan() plus legacy buffer add/free calls. MQ RX uses per-queue handles, so the driver must also be able to register/deregister subordinate queues and post buffers against a specific queue handle.
Add the PHYP call IDs for:
H_REG_LOGICAL_LAN_QUEUE
H_ADD_LOGICAL_LAN_BUFFERS_QUEUE
H_FREE_LOGICAL_LAN_QUEUE
and add ibmveth.h wrapper helpers (h_reg_logical_lan_queue(), h_add_logical_lan_buffers_queue(), h_free_logical_lan_queue()) with argument ordering and return semantics matching the existing ibmveth hcall wrappers. Also add h_register_logical_lan_with_handle() so queue 0 can capture the PHYP queue handle in MQ mode.
This patch is intentionally plumbing only: no runtime behavior change yet. Legacy firmware keeps H_REGISTER_LOGICAL_LAN and the existing buffer hcalls. The new wrappers are used only when a later commit sets multi_queue from H_ILLAN_ATTRIBUTES.
---
Changes in v4:
- Document @queue_handle and @irq in h_reg_logical_lan_queue() kdoc.
- Wrap h_register_logical_lan_with_handle() prototype for readability / checkpatch.
- Drop unused H_FREE_LOGICAL_LAN_BUFFER_QUEUE wrapper/opcode (no caller; buffer return is local harvest + queue free).
arch/powerpc/include/asm/hvcall.h | 5 +- drivers/net/ethernet/ibm/ibmveth.h | 137 +++++++++++++++++++++++++++++ 2 files changed, 141 insertions(+), 1 deletion(-)
arch/powerpc/include/asm/hvcall.h +4 -1
@@ -362,7 +362,10 @@#define H_GUEST_DELETE 0x488#define H_PKS_WRAP_OBJECT 0x490#define H_PKS_UNWRAP_OBJECT 0x494 -#define MAX_HCALL_OPCODE H_PKS_UNWRAP_OBJECT +#define H_REG_LOGICAL_LAN_QUEUE 0x49C +#define H_ADD_LOGICAL_LAN_BUFFERS_QUEUE 0x4A0 +#define H_FREE_LOGICAL_LAN_QUEUE 0x4A8 +#define MAX_HCALL_OPCODE H_FREE_LOGICAL_LAN_QUEUE/* Scope args for H_SCM_UNBIND_ALL */ #define H_UNBIND_SCOPE_ALL (0x1)
drivers/net/ethernet/ibm/ibmveth.h +137 -0
143 lines not on this page · raw message
Signature · 1 line
2.50.1 (Apple Git-155)
Jakub Kicinski
maintainerRe: [PATCH net-next v4 01/14] ibmveth: Add MQ RX hypercall wrappers and call definitions
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
ibmveth: Add MQ RX hypercall wrappers and call definitions
This adds three new PHYP call IDs for multi-queue receive support and four static inline hcall wrapper helpers in ibmveth.h, bumping MAX_HCALL_OPCODE to the highest new opcode. No callers are added by this patch.
A couple of questions below about the new wrappers and about the hvcall.h hunk.
arch/powerpc/include/asm/hvcall.h line 368
Quoted 13 lines
arch/powerpc/include/asm/hvcall.h +4
-1@@ -362,7 +362,10 @@#define H_GUEST_DELETE 0x488#define H_PKS_WRAP_OBJECT 0x490#define H_PKS_UNWRAP_OBJECT 0x494 -#define MAX_HCALL_OPCODE H_PKS_UNWRAP_OBJECT +#define H_REG_LOGICAL_LAN_QUEUE 0x49C +#define H_ADD_LOGICAL_LAN_BUFFERS_QUEUE 0x4A0 +#define H_FREE_LOGICAL_LAN_QUEUE 0x4A8 +#define MAX_HCALL_OPCODE H_FREE_LOGICAL_LAN_QUEUE
[Severity: Low]
Could the changelog cite the PAPR/PHYP document and revision that assigns 0x49C, 0x4A0 and 0x4A8? These become architecture-wide constants shared with the KVM hcall emulation, so it would help to have the provenance recorded in the commit.
There are also holes left at 0x498 and at 0x4A4 (the latter from dropping H_FREE_LOGICAL_LAN_BUFFER_QUEUE in v4). Are those numbers reserved by the architecture for something else, or are they simply unused?
Separately, MAX_HCALL_OPCODE is not driver-local. It sizes kvm_arch.enabled_hcalls:
arch/powerpc/include/asm/kvm_host.h:
DECLARE_BITMAP(enabled_hcalls, MAX_HCALL_OPCODE/4 + 1);and it bounds the userspace-visible opcode range accepted by KVM_CAP_PPC_ENABLE_HCALL:
arch/powerpc/kvm/powerpc.c:kvm_vm_ioctl_enable_cap() {
...
if (hcall > MAX_HCALL_OPCODE || (hcall & 3) ||
cap->args[1] > 1)
...
}It also sizes the per-CPU hcall statistics array in arch/powerpc/platforms/pseries/hvCall_inst.c and gates kvmppc_pseries_do_hcall() and kvmppc_h_pr(). As far as I can tell the bump is behaviourally inert for these new opcodes, since kvm_vm_ioctl_enable_cap() still rejects them via kvmppc_book3s_hcall_implemented(), and the KVM dispatch path ends up returning RESUME_HOST either way. Would it be worth mentioning the cross-subsystem effect in the changelog so the powerpc/KVM side has something to ack against?
drivers/net/ethernet/ibm/ibmveth.h static inline long h_add_logical_lan_buffers(unsigned long unit_address, · line 70
Quoted 13 lines
arch/powerpc/include/asm/hvcall.h +0
-0/* Scope args for H_SCM_UNBIND_ALL */ #define H_UNBIND_SCOPE_ALL (0x1)drivers/net/ethernet/ibm/ibmveth.h +2
-0@@ -66,6 +66,143 @@ static inline long h_add_logical_lan_buffers(unsigned long unit_address, desc5, desc6, desc7, desc8); } +/** +* h_reg_logical_lan_queue - Register a subordinate receive queue
[Severity: Low]
This isn't a bug, but the naming here diverges from the existing wrappers in this file, which all spell it out as h_register_logical_lan() and h_free_logical_lan(). Would h_register_logical_lan_queue() read more consistently?
[ ... ]
drivers/net/ethernet/ibm/ibmveth.h static inline long h_add_logical_lan_buffers(unsigned long unit_address, · line 92
Quoted 5 lines
drivers/net/ethernet/ibm/ibmveth.h +5
-0+static inline long h_reg_logical_lan_queue (unsigned long unit_address, +unsigned long buffer_list, +unsigned long rec_queue, +unsigned long *queue_handle, +unsigned long *irq)
^^^^^^^^^^^^^^^^^^^^^^^^^^
[Severity: Low]
This helper returns the queue handle as unsigned long *, which matches retbuf's element type and matches the other output parameters in this header (h_send_logical_lan()'s corellator_out, h_illan_attributes()'s ret_attributes).
But h_register_logical_lan_with_handle() below declares the same conceptual value, the PHYP queue handle read out of retbuf[0], as u64 *. Can these two be made to agree? As it stands, callers have to pick a different storage type depending on which of the two helpers they call, even though both are returning the same thing.
The commit message says the new wrappers have "argument ordering and return semantics matching the existing ibmveth hcall wrappers", which holds for this one but not for the u64 * variant.
[ ... ]
drivers/net/ethernet/ibm/ibmveth.h static inline long h_add_logical_lan_buffers(unsigned long unit_address, · line 168
Quoted 8 lines
drivers/net/ethernet/ibm/ibmveth.h +8
-0+static inline long h_free_logical_lan_queue (unsigned long unit_address, +unsigned long queue_handle) +{ +unsigned long retbuf[PLPAR_HCALL9_BUFSIZE ]; + +return plpar_hcall9 (H_FREE_LOGICAL_LAN_QUEUE , + retbuf, unit_address, queue_handle); +}
[Severity: Low]
The kerneldoc above documents no output data for this hcall, and retbuf is never read. Why not use plpar_hcall_norets() here, the way the existing counterpart in this file does?
#define h_free_logical_lan(ua) \
plpar_hcall_norets(H_FREE_LOGICAL_LAN, ua)That would drop the dead nine-entry stack buffer.
[ ... ]
drivers/net/ethernet/ibm/ibmveth.h static inline long h_add_logical_lan_buffers(unsigned long unit_address, · line 191
Quoted 7 lines
drivers/net/ethernet/ibm/ibmveth.h +7
-0+static inline long +h_register_logical_lan_with_handle (unsigned long unit_address, +unsigned long buffer_list, +unsigned long rec_queue, +unsigned long filter_list, +unsigned long mac_address, +u64 *queue_handle)
^^^^^^^^^^^^^^^^^^
[Severity: Low]
This is the u64 * side of the inconsistency noted above. Both types are 64-bit on ppc64, which is the only configuration that builds ibmveth, and the callers added later in the series do compile against both prototypes, so this looks like a readability issue rather than a functional one. Could both helpers use unsigned long * for the handle?
drivers/net/ethernet/ibm/ibmveth.h static inline long h_add_logical_lan_buffers(unsigned long unit_address, · line 204
Quoted 13 lines
drivers/net/ethernet/ibm/ibmveth.h +13
-0+{ +unsigned long retbuf[PLPAR_HCALL9_BUFSIZE ]; +long rc; + + rc =plpar_hcall9 (H_REGISTER_LOGICAL_LAN , retbuf, + unit_address, buffer_list, rec_queue, + filter_list, mac_address); + +if (rc ==H_SUCCESS && queue_handle) + *queue_handle = retbuf[0 ]; + +return rc; +}
Thanks.
Signature · 1 line
pw-bot: cr
Re: [PATCH net-next v4 01/14] ibmveth: Add MQ RX hypercall wrappers and call definitions
Hi Jakub,
Thanks for the review, and I appreciate the detailed review across the whole series.
More broadly, I agree with the overall direction of your review. For v5 I plan to fold the follow-up fixes back into the patches where they belong and make the changelogs clearer, so the series reads more cleanly end to end.
Quoted 21 lines · Jakub Kicinski
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
ibmveth: Add MQ RX hypercall wrappers and call definitionsThis adds three new PHYP call IDs for multi-queue receive support and four static inline hcall wrapper helpers in
ibmveth.h, bumpingMAX_HCALL_OPCODEto the highest new opcode. No callers are added by this patch.A couple of questions below about the new wrappers and about the
hvcall.hhunk.Quoted 10 lines
arch/powerpc/include/asm/hvcall.h +1
-1@@ -362,7 +362,10 @@#define H_GUEST_DELETE 0x488#define H_PKS_WRAP_OBJECT 0x490#define H_PKS_UNWRAP_OBJECT 0x494 -#define MAX_HCALL_OPCODE H_PKS_UNWRAP_OBJECT +#define H_REG_LOGICAL_LAN_QUEUE 0x49C
Quoted 8 lines
arch/powerpc/include/asm/hvcall.h line 368
Quoted 3 lines
arch/powerpc/include/asm/hvcall.h +3
-0+#define H_ADD_LOGICAL_LAN_BUFFERS_QUEUE 0x4A0 +#define H_FREE_LOGICAL_LAN_QUEUE 0x4A8 +#define MAX_HCALL_OPCODE H_FREE_LOGICAL_LAN_QUEUE[Severity: Low]
Could the changelog cite the PAPR/PHYP document and revision that assigns0x49C,0x4A0and0x4A8? These become architecture-wide constants shared with the KVM hcall emulation, so it would help to have the provenance recorded in the commit.
Agreed. The first PAPR revision that contains the MQ hcalls is PAPR 11.20.00, and in v5 I will cite that revision explicitly in the patch-1 changelog.
Quoted 3 lines
There are also holes left at
0x498and at0x4A4(the latter from droppingH_FREE_LOGICAL_LAN_BUFFER_QUEUEin v4). Are those numbers reserved by the architecture for something else, or are they simply unused?
For the gaps, 0x4A4 remains reserved for H_FREE_LOGICAL_LAN_BUFFER_QUEUE, and 0x498 is reserved. This series does not need H_FREE_LOGICAL_LAN_BUFFER_QUEUE for the normal queue teardown path, so I am not adding it here.
Quoted 17 lines
Separately,
MAX_HCALL_OPCODEis not driver-local. It sizes kvm_arch.enabled_hcalls:arch/powerpc/include/asm/kvm_host.h: DECLARE_BITMAP(enabled_hcalls, MAX_HCALL_OPCODE/4 + 1);and it bounds the userspace-visible opcode range accepted by
KVM_CAP_PPC_ENABLE_HCALL:arch/powerpc/kvm/powerpc.c:kvm_vm_ioctl_enable_cap() { ... if (hcall > MAX_HCALL_OPCODE || (hcall & 3) || cap->args[1] > 1) ... }
Quoted 8 lines
It also sizes the per-CPU hcall statistics array in
arch/powerpc/platforms/pseries/hvCall_inst.cand gateskvmppc_pseries_do_hcall()andkvmppc_h_pr(). As far as I can tell the bump is behaviourally inert for these new opcodes, sincekvm_vm_ioctl_enable_cap()still rejects them viakvmppc_book3s_hcall_implemented(), and the KVM dispatch path ends up returningRESUME_HOSTeither way. Would it be worth mentioning the cross-subsystem effect in the changelog so the powerpc/KVM side has something to ack against?
Sure, for MAX_HCALL_OPCODE, I agree that the bump has a cross-subsystem effect because it sizes KVM's enabled-hcall bitmap and bounds KVM_CAP_PPC_ENABLE_HCALL. In v5 I will call that out explicitly in the patch-1 changelog rather than leaving it implicit.
Quoted 19 lines
drivers/net/ethernet/ibm/ibmveth.h static inline long h_add_logical_lan_buffers(unsigned long unit_address, · line 70
Quoted 13 lines
arch/powerpc/include/asm/hvcall.h +0
-0/* Scope args for H_SCM_UNBIND_ALL */ #define H_UNBIND_SCOPE_ALL (0x1)drivers/net/ethernet/ibm/ibmveth.h +2
-0@@ -66,6 +66,143 @@ static inline long h_add_logical_lan_buffers(unsigned long unit_address, desc5, desc6, desc7, desc8); } +/** +* h_reg_logical_lan_queue - Register a subordinate receive queue [Severity: Low]
This isn't a bug, but the naming here diverges from the existing wrappers in this file, which all spell it out ash_register_logical_lan()andh_free_logical_lan(). Wouldh_register_logical_lan_queue()read more consistently?
Quoted 1 line
[ ... ]
Agreed. In v5 I will rename it to h_register_logical_lan_queue() so it matches the existing wrapper naming.
Quoted 19 lines
drivers/net/ethernet/ibm/ibmveth.h static inline long h_add_logical_lan_buffers(unsigned long unit_address, · line 92
Quoted 5 lines
drivers/net/ethernet/ibm/ibmveth.h +5
-0+static inline long h_reg_logical_lan_queue (unsigned long unit_address, +unsigned long buffer_list, +unsigned long rec_queue, +unsigned long *queue_handle, +unsigned long *irq)^^^^^^^^^^^^^^^^^^^^^^^^^^[Severity: Low]
This helper returns the queue handle as unsigned long *, which matches retbuf's element type and matches the other output parameters in this header (h_send_logical_lan()'scorellator_out, h_illan_attributes()'sret_attributes).But
h_register_logical_lan_with_handle()below declares the same conceptual value, the PHYP queue handle read out of retbuf[0], as u64 *. Can these two be made to agree? As it stands, callers have to pick a different storage type depending on which of the two helpers they call, even though both are returning the same thing.
Quoted 5 lines
The commit message says the new wrappers have "argument ordering and return semantics matching the existing ibmveth hcall wrappers", which holds for this one but not for the u64 * variant.
[ ... ]
Agreed. In v5 both h_register_logical_lan_queue() and h_register_logical_lan_with_handle() will use unsigned long *queue_handle, matching the sibling wrappers and avoiding different caller-side storage depending on which helper is used.
Quoted 13 lines
drivers/net/ethernet/ibm/ibmveth.h static inline long h_add_logical_lan_buffers(unsigned long unit_address, · line 168
Quoted 8 lines
drivers/net/ethernet/ibm/ibmveth.h +8
-0+static inline long h_free_logical_lan_queue (unsigned long unit_address, +unsigned long queue_handle) +{ +unsigned long retbuf[PLPAR_HCALL9_BUFSIZE ]; + +return plpar_hcall9 (H_FREE_LOGICAL_LAN_QUEUE , + retbuf, unit_address, queue_handle); +}[Severity: Low]
The kerneldoc above documents no output data for this hcall, and retbuf is never read. Why not useplpar_hcall_norets()here, the way the existing counterpart in this file does?
Quoted 6 lines
#define h_free_logical_lan(ua) \ plpar_hcall_norets(H_FREE_LOGICAL_LAN, ua)That would drop the dead nine-entry stack buffer.
[ ... ]
Agreed. In v5 I will switch h_free_logical_lan_queue() to plpar_hcall_norets() and drop the unused return buffer.
Quoted 9 lines
drivers/net/ethernet/ibm/ibmveth.h static inline long h_add_logical_lan_buffers(unsigned long unit_address, · line 191
Quoted 7 lines
drivers/net/ethernet/ibm/ibmveth.h +7
-0+static inline long +h_register_logical_lan_with_handle (unsigned long unit_address, +unsigned long buffer_list, +unsigned long rec_queue, +unsigned long filter_list, +unsigned long mac_address, +u64 *queue_handle)^^^^^^^^^^^^^^^^^^
Quoted 6 lines
[Severity: Low]
This is the u64 * side of the inconsistency noted above. Both types are 64-bit on ppc64, which is the only configuration that builds ibmveth, and the callers added later in the series do compile against both prototypes, so this looks like a readability issue rather than a functional one. Could both helpers use unsigned long * for the handle?
|In v5 both helpers will use unsigned long *queue_handle.|
Thanks again for the detailed review,
Mingming
PATCH v4 2/14 ibmveth: Prepare MQ RX adapter data structures
In reply to Mingming CaoMQ RX needs per-queue state for NAPI, queue handles/IRQs, RX rings, buffer-list DMA mappings, and buffer pools. The current driver stores most of this as single instances tied to queue 0.
Convert those fields to queue-indexed layouts sized by IBMVETH_MAX_RX_QUEUES:
rx_queue[]
napi[]
queue_handle[] / queue_irq[]
buffer_list_addr[] / buffer_list_dma[]
rx_buff_pool[queue][pool]
and add multi_queue / num_rx_queues to track MQ capability and how many RX queues are active. Keep IBMVETH_MAX_RX_QUEUES at 1 for now so this remains a structural preparation patch; later enablement raises the limit when multi-queue RX is actually turned on.
This patch keeps behavior unchanged by mechanically switching existing references to index 0 (for example rx_queue -> rx_queue[0], rx_buff_pool[pool] -> rx_buff_pool[0][pool], napi -> napi[0]). open/poll/close still drive a single RX queue only.
Per-queue statistics structs and hcall_stats are introduced later with their first use (register helpers / statistics collection).
---
Changes in v4:
- Keep IBMVETH_MAX_RX_QUEUES at 1 until MQ enablement (same idea as v3, but v3 also planted unused stats types here).
- Layout-only: queue-indexed adapter fields only. Do not introduce hcall_stats / rx_qstats here (first-use).
- Defer hcall_stats to patch 7 (register helpers) and rx/tx qstats + NUM macros to patch 10 (statistics collection).
- Subject: "Prepare MQ RX adapter data structures" (was "...and statistics structures" in earlier drafts).
drivers/net/ethernet/ibm/ibmveth.c | 212 ++++++++++++++++------------- drivers/net/ethernet/ibm/ibmveth.h | 18 ++- 2 files changed, 128 insertions(+), 102 deletions(-)
drivers/net/ethernet/ibm/ibmveth.c +115 -97
@@ -101,7 +101,9 @@ static struct ibmveth_stat ibmveth_stats[] = {/* simple methods of getting data from the current rxq entry */ static inline u32 ibmveth_rxq_flags (struct ibmveth_adapter *adapter) { -return be32_to_cpu (adapter->rx_queue.queue_addr[adapter->rx_queue.index].flags_off); +struct ibmveth_rx_q *rxq = &adapter->rx_queue[0 ]; + +return be32_to_cpu (rxq->queue_addr[rxq->index].flags_off); }static inline int ibmveth_rxq_toggle (struct ibmveth_adapter *adapter)
@@ -112,7 +114,7 @@ static inline int ibmveth_rxq_toggle(struct ibmveth_adapter *adapter)static inline int ibmveth_rxq_pending_buffer (struct ibmveth_adapter *adapter) { -return ibmveth_rxq_toggle (adapter) == adapter->rx_queue.toggle; +return ibmveth_rxq_toggle (adapter) == adapter->rx_queue[0 ].toggle; }static inline int ibmveth_rxq_buffer_valid (struct ibmveth_adapter *adapter)
@@ -132,7 +134,9 @@ static inline int ibmveth_rxq_large_packet(struct ibmveth_adapter *adapter)static inline int ibmveth_rxq_frame_length (struct ibmveth_adapter *adapter) { -return be32_to_cpu (adapter->rx_queue.queue_addr[adapter->rx_queue.index].length); +struct ibmveth_rx_q *rxq = &adapter->rx_queue[0 ]; + +return be32_to_cpu (rxq->queue_addr[rxq->index].length); }static inline int ibmveth_rxq_csum_good (struct ibmveth_adapter *adapter)
@@ -386,7 +390,7 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,*/ static void ibmveth_update_rx_no_buffer (struct ibmveth_adapter *adapter) { -__be64 *p = adapter->buffer_list_addr +4096 -8 ; +__be64 *p = adapter->buffer_list_addr[0 ] +4096 -8 ; adapter->rx_no_buffer =be64_to_cpup (p); }
@@ -399,7 +403,7 @@ static void ibmveth_replenish_task(struct ibmveth_adapter *adapter) adapter->replenish_task_cycles++;for (i = (IBMVETH_NUM_BUFF_POOLS -1 ); i >=0 ; i--) { -struct ibmveth_buff_pool *pool = &adapter->rx_buff_pool[i]; +struct ibmveth_buff_pool *pool = &adapter->rx_buff_pool[0 ][i];if (pool->active && (atomic_read (&pool->available) < pool->threshold))
@@ -463,12 +467,12 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,struct sk_buff *skb;if (WARN_ON (pool >=IBMVETH_NUM_BUFF_POOLS ) || -WARN_ON (index >= adapter->rx_buff_pool[pool].size)) { +WARN_ON (index >= adapter->rx_buff_pool[0 ][pool].size)) {schedule_work (&adapter->work);return -EINVAL ; } - skb = adapter->rx_buff_pool[pool].skbuff[index]; + skb = adapter->rx_buff_pool[0 ][pool].skbuff[index];if (WARN_ON (!skb)) {schedule_work (&adapter->work);return -EFAULT ;
@@ -482,42 +486,44 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,/* remove the skb pointer to mark free. actual freeing is done * by upper level networking after gro_receive */ - adapter->rx_buff_pool[pool].skbuff[index] =NULL ; + adapter->rx_buff_pool[0 ][pool].skbuff[index] =NULL ;dma_unmap_single (&adapter->vdev->dev, - adapter->rx_buff_pool[pool].dma_addr[index], - adapter->rx_buff_pool[pool].buff_size, + adapter->rx_buff_pool[0 ][pool].dma_addr[index], + adapter->rx_buff_pool[0 ][pool].buff_size,DMA_FROM_DEVICE ); } - free_index = adapter->rx_buff_pool[pool].producer_index; - adapter->rx_buff_pool[pool].producer_index++; -if (adapter->rx_buff_pool[pool].producer_index >= - adapter->rx_buff_pool[pool].size) - adapter->rx_buff_pool[pool].producer_index =0 ; - adapter->rx_buff_pool[pool].free_map[free_index] = index; + free_index = adapter->rx_buff_pool[0 ][pool].producer_index; + adapter->rx_buff_pool[0 ][pool].producer_index++; +if (adapter->rx_buff_pool[0 ][pool].producer_index >= + adapter->rx_buff_pool[0 ][pool].size) + adapter->rx_buff_pool[0 ][pool].producer_index =0 ; + adapter->rx_buff_pool[0 ][pool].free_map[free_index] = index;mb (); -atomic_dec (&(adapter->rx_buff_pool[pool].available)); +atomic_dec (&adapter->rx_buff_pool[0 ][pool].available);return 0 ; }/* get the current buffer on the rx queue */ -static inline struct sk_buff *ibmveth_rxq_get_buffer (struct ibmveth_adapter *adapter) +static inline struct sk_buff * +ibmveth_rxq_get_buffer (struct ibmveth_adapter *adapter) { -u64 correlator = adapter->rx_queue.queue_addr[adapter->rx_queue.index].correlator; +struct ibmveth_rx_q *rxq = &adapter->rx_queue[0 ]; +u64 correlator = rxq->queue_addr[rxq->index].correlator;unsigned int pool = correlator >>32 ;unsigned int index = correlator &0xffffffffUL ;if (WARN_ON (pool >=IBMVETH_NUM_BUFF_POOLS ) || -WARN_ON (index >= adapter->rx_buff_pool[pool].size)) { +WARN_ON (index >= adapter->rx_buff_pool[0 ][pool].size)) {schedule_work (&adapter->work);return NULL ; } -return adapter->rx_buff_pool[pool].skbuff[index]; +return adapter->rx_buff_pool[0 ][pool].skbuff[index]; }/**
@@ -538,14 +544,16 @@ static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,u64 cor;int rc; - cor = adapter->rx_queue.queue_addr[adapter->rx_queue.index].correlator; +struct ibmveth_rx_q *rxq = &adapter->rx_queue[0 ]; + + cor = rxq->queue_addr[rxq->index].correlator; rc =ibmveth_remove_buffer_from_pool (adapter, cor, reuse);if (unlikely (rc))return rc; -if (++adapter->rx_queue.index == adapter->rx_queue.num_slots) { - adapter->rx_queue.index =0 ; - adapter->rx_queue.toggle = !adapter->rx_queue.toggle; +if (++adapter->rx_queue[0 ].index == adapter->rx_queue[0 ].num_slots) { + adapter->rx_queue[0 ].index =0 ; + adapter->rx_queue[0 ].toggle = !adapter->rx_queue[0 ].toggle; }return 0 ;
@@ -595,7 +603,7 @@ static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,*/ retry: rc =h_register_logical_lan (adapter->vdev->unit_address, - adapter->buffer_list_dma, rxq_desc.desc, + adapter->buffer_list_dma[0 ], rxq_desc.desc, adapter->filter_list_dma, mac_address);if (rc !=H_SUCCESS && try_again) {
@@ -623,14 +631,14 @@ static int ibmveth_open(struct net_device *netdev)netdev_dbg (netdev,"open starting\n" ); -napi_enable (&adapter->napi); +napi_enable (&adapter->napi[0 ]);for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) - rxq_entries += adapter->rx_buff_pool[i].size; + rxq_entries += adapter->rx_buff_pool[0 ][i].size; rc = -ENOMEM ; - adapter->buffer_list_addr = (void *)get_zeroed_page (GFP_KERNEL ); -if (!adapter->buffer_list_addr) { + adapter->buffer_list_addr[0 ] = (void *)get_zeroed_page (GFP_KERNEL ); +if (!adapter->buffer_list_addr[0 ]) {netdev_err (netdev,"unable to allocate list pages\n" ); goto out; }
@@ -643,17 +651,18 @@ static int ibmveth_open(struct net_device *netdev) dev = &adapter->vdev->dev; - adapter->rx_queue.queue_len =sizeof (struct ibmveth_rx_q_entry ) * + adapter->rx_queue[0 ].queue_len =sizeof (struct ibmveth_rx_q_entry ) * rxq_entries; - adapter->rx_queue.queue_addr = -dma_alloc_coherent (dev, adapter->rx_queue.queue_len, - &adapter->rx_queue.queue_dma,GFP_KERNEL ); -if (!adapter->rx_queue.queue_addr) + adapter->rx_queue[0 ].queue_addr = +dma_alloc_coherent (dev, adapter->rx_queue[0 ].queue_len, + &adapter->rx_queue[0 ].queue_dma,GFP_KERNEL ); +if (!adapter->rx_queue[0 ].queue_addr) goto out_free_filter_list; - adapter->buffer_list_dma =dma_map_single (dev, - adapter->buffer_list_addr,4096 ,DMA_BIDIRECTIONAL ); -if (dma_mapping_error (dev, adapter->buffer_list_dma)) { + adapter->buffer_list_dma[0 ] = +dma_map_single (dev, adapter->buffer_list_addr[0 ], +4096 ,DMA_BIDIRECTIONAL ); +if (dma_mapping_error (dev, adapter->buffer_list_dma[0 ])) {netdev_err (netdev,"unable to map buffer list pages\n" ); goto out_free_queue_mem; }
@@ -670,19 +679,21 @@ static int ibmveth_open(struct net_device *netdev) goto out_free_tx_ltb; } - adapter->rx_queue.index =0 ; - adapter->rx_queue.num_slots = rxq_entries; - adapter->rx_queue.toggle =1 ; + adapter->rx_queue[0 ].index =0 ; + adapter->rx_queue[0 ].num_slots = rxq_entries; + adapter->rx_queue[0 ].toggle =1 ; mac_address =ether_addr_to_u64 (netdev->dev_addr); rxq_desc.fields.flags_len =IBMVETH_BUF_VALID | - adapter->rx_queue.queue_len; - rxq_desc.fields.address = adapter->rx_queue.queue_dma; + adapter->rx_queue[0 ].queue_len; + rxq_desc.fields.address = adapter->rx_queue[0 ].queue_dma; -netdev_dbg (netdev,"buffer list @ 0x%p\n" , adapter->buffer_list_addr); +netdev_dbg (netdev,"buffer list @ 0x%p\n" , + adapter->buffer_list_addr[0 ]);netdev_dbg (netdev,"filter list @ 0x%p\n" , adapter->filter_list_addr); -netdev_dbg (netdev,"receive q @ 0x%p\n" , adapter->rx_queue.queue_addr); +netdev_dbg (netdev,"receive q @ 0x%p\n" , + adapter->rx_queue[0 ].queue_addr);h_vio_signal (adapter->vdev->unit_address,VIO_IRQ_DISABLE );
@@ -693,7 +704,7 @@ static int ibmveth_open(struct net_device *netdev) lpar_rc);netdev_err (netdev,"buffer TCE:0x%llx filter TCE:0x%llx rxq " "desc:0x%llx MAC:0x%llx\n" , - adapter->buffer_list_dma, + adapter->buffer_list_dma[0 ], adapter->filter_list_dma, rxq_desc.desc, mac_address);
@@ -702,11 +713,11 @@ static int ibmveth_open(struct net_device *netdev) }for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) { -if (!adapter->rx_buff_pool[i].active) +if (!adapter->rx_buff_pool[0 ][i].active)continue ; -if (ibmveth_alloc_buffer_pool (&adapter->rx_buff_pool[i])) { +if (ibmveth_alloc_buffer_pool (&adapter->rx_buff_pool[0 ][i])) {netdev_err (netdev,"unable to alloc pool\n" ); - adapter->rx_buff_pool[i].active =0 ; + adapter->rx_buff_pool[0 ][i].active =0 ; rc = -ENOMEM ; goto out_free_buffer_pools; }
@@ -738,9 +749,9 @@ static int ibmveth_open(struct net_device *netdev) out_free_buffer_pools:while (--i >=0 ) { -if (adapter->rx_buff_pool[i].active) +if (adapter->rx_buff_pool[0 ][i].active)ibmveth_free_buffer_pool (adapter, - &adapter->rx_buff_pool[i]); + &adapter->rx_buff_pool[0 ][i]); } out_unmap_filter_list: dma_unmap_single(dev, adapter->filter_list_dma, 4096,
@@ -752,18 +763,18 @@ static int ibmveth_open(struct net_device *netdev) } out_unmap_buffer_list: -dma_unmap_single (dev, adapter->buffer_list_dma,4096 , +dma_unmap_single (dev, adapter->buffer_list_dma[0 ],4096 ,DMA_BIDIRECTIONAL ); out_free_queue_mem: -dma_free_coherent (dev, adapter->rx_queue.queue_len, - adapter->rx_queue.queue_addr, - adapter->rx_queue.queue_dma); +dma_free_coherent (dev, adapter->rx_queue[0 ].queue_len, + adapter->rx_queue[0 ].queue_addr, + adapter->rx_queue[0 ].queue_dma); out_free_filter_list:free_page ((unsigned long )adapter->filter_list_addr); out_free_buffer_list: -free_page ((unsigned long )adapter->buffer_list_addr); +free_page ((unsigned long )adapter->buffer_list_addr[0 ]); out: -napi_disable (&adapter->napi); +napi_disable (&adapter->napi[0 ]);return rc; }
@@ -776,7 +787,7 @@ static int ibmveth_close(struct net_device *netdev)netdev_dbg (netdev,"close starting\n" ); -napi_disable (&adapter->napi); +napi_disable (&adapter->napi[0 ]);netif_tx_stop_all_queues (netdev);
@@ -795,22 +806,22 @@ static int ibmveth_close(struct net_device *netdev)ibmveth_update_rx_no_buffer (adapter); -dma_unmap_single (dev, adapter->buffer_list_dma,4096 , +dma_unmap_single (dev, adapter->buffer_list_dma[0 ],4096 ,DMA_BIDIRECTIONAL ); -free_page ((unsigned long )adapter->buffer_list_addr); +free_page ((unsigned long )adapter->buffer_list_addr[0 ]);dma_unmap_single (dev, adapter->filter_list_dma,4096 ,DMA_BIDIRECTIONAL );free_page ((unsigned long )adapter->filter_list_addr); -dma_free_coherent (dev, adapter->rx_queue.queue_len, - adapter->rx_queue.queue_addr, - adapter->rx_queue.queue_dma); +dma_free_coherent (dev, adapter->rx_queue[0 ].queue_len, + adapter->rx_queue[0 ].queue_addr, + adapter->rx_queue[0 ].queue_dma);for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) -if (adapter->rx_buff_pool[i].active) +if (adapter->rx_buff_pool[0 ][i].active)ibmveth_free_buffer_pool (adapter, - &adapter->rx_buff_pool[i]); + &adapter->rx_buff_pool[0 ][i]);for (i =0 ; i < netdev->real_num_tx_queues; i++)ibmveth_free_tx_ltb (adapter, i);
@@ -1448,7 +1459,7 @@ static void ibmveth_rx_csum_helper(struct sk_buff *skb,static int ibmveth_poll (struct napi_struct *napi,int budget) {struct ibmveth_adapter *adapter = - container_of(napi, struct ibmveth_adapter, napi); + container_of(napi, struct ibmveth_adapter, napi[0]);struct net_device *netdev = adapter->netdev;int frames_processed =0 ;unsigned long lpar_rc;
@@ -1573,11 +1584,11 @@ static irqreturn_t ibmveth_interrupt(int irq, void *dev_instance)struct ibmveth_adapter *adapter =netdev_priv (netdev);unsigned long lpar_rc; -if (napi_schedule_prep (&adapter->napi)) { +if (napi_schedule_prep (&adapter->napi[0 ])) { lpar_rc =h_vio_signal (adapter->vdev->unit_address,VIO_IRQ_DISABLE );WARN_ON (lpar_rc !=H_SUCCESS ); -__napi_schedule (&adapter->napi); +__napi_schedule (&adapter->napi[0 ]); }return IRQ_HANDLED ; }
@@ -1645,7 +1656,7 @@ static int ibmveth_change_mtu(struct net_device *dev, int new_mtu)int need_restart =0 ;for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) -if (new_mtu_oh <= adapter->rx_buff_pool[i].buff_size) +if (new_mtu_oh <= adapter->rx_buff_pool[0 ][i].buff_size)break ;if (i ==IBMVETH_NUM_BUFF_POOLS )
@@ -1660,9 +1671,9 @@ static int ibmveth_change_mtu(struct net_device *dev, int new_mtu)/* Look for an active buffer pool that can hold the new MTU */ for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) { - adapter->rx_buff_pool[i].active =1 ; + adapter->rx_buff_pool[0 ][i].active =1 ; -if (new_mtu_oh <= adapter->rx_buff_pool[i].buff_size) { +if (new_mtu_oh <= adapter->rx_buff_pool[0 ][i].buff_size) {WRITE_ONCE (dev->mtu, new_mtu);vio_cmo_set_dev_desired (viodev, ibmveth_get_desired_dma
@@ -1693,8 +1704,7 @@ static void ibmveth_poll_controller(struct net_device *dev)* * @vdev: struct vio_dev for the device whose desired IO mem is to be returned * -* Return value: -* Number of bytes of IO data the driver will need to perform well. +* Return: Number of bytes of IO data the driver will need to perform well. */ static unsigned long ibmveth_get_desired_dma (struct vio_dev *vdev) {
@@ -1720,12 +1730,12 @@ static unsigned long ibmveth_get_desired_dma(struct vio_dev *vdev)for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) {/* add the size of the active receive buffers */ -if (adapter->rx_buff_pool[i].active) +if (adapter->rx_buff_pool[0 ][i].active) ret += - adapter->rx_buff_pool[i].size * -IOMMU_PAGE_ALIGN (adapter->rx_buff_pool[i]. + adapter->rx_buff_pool[0 ][i].size * +IOMMU_PAGE_ALIGN (adapter->rx_buff_pool[0 ][i]. buff_size, tbl); - rxqentries += adapter->rx_buff_pool[i].size; + rxqentries += adapter->rx_buff_pool[0 ][i].size; }/* add the size of the receive queue entries */ ret += IOMMU_PAGE_ALIGN(
@@ -1844,7 +1854,7 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id) adapter->mcastFilterSize =be32_to_cpu (*mcastFilterSize_p);ibmveth_init_link_settings (netdev); -netif_napi_add_weight (netdev, &adapter->napi, ibmveth_poll,16 ); +netif_napi_add_weight (netdev, &adapter->napi[0 ], ibmveth_poll,16 ); netdev->irq = dev->irq; netdev->netdev_ops = &ibmveth_netdev_ops;
@@ -1876,6 +1886,10 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id) netdev->features |=NETIF_F_FRAGLIST ; } +/* Initialize queue count - always 1 for now */ + adapter->multi_queue =0 ; + adapter->num_rx_queues =IBMVETH_DEFAULT_RX_QUEUES ; +if (ret ==H_SUCCESS && (ret_attr &IBMVETH_ILLAN_RX_MULTI_BUFF_SUPPORT )) { adapter->rx_buffers_per_hcall =IBMVETH_MAX_RX_PER_HCALL ;
@@ -1898,10 +1912,10 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)memcpy (pool_count, pool_count_cmo,sizeof (pool_count));for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) { -struct kobject *kobj = &adapter->rx_buff_pool[i].kobj; +struct kobject *kobj = &adapter->rx_buff_pool[0 ][i].kobj;int error; -ibmveth_init_buffer_pool (&adapter->rx_buff_pool[i], i, +ibmveth_init_buffer_pool (&adapter->rx_buff_pool[0 ][i], i, pool_count[i], pool_size[i], pool_active[i]); error = kobject_init_and_add(kobj, &ktype_veth_pool,
@@ -1949,7 +1963,7 @@ static void ibmveth_remove(struct vio_dev *dev)cancel_work_sync (&adapter->work);for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) -kobject_put (&adapter->rx_buff_pool[i].kobj); +kobject_put (&adapter->rx_buff_pool[0 ][i].kobj);unregister_netdev (netdev);
@@ -2035,11 +2049,12 @@ static ssize_t veth_pool_store(struct kobject *kobj, struct attribute *attr,/* Make sure there is a buffer pool with buffers that can hold a packet of the size of the MTU */ for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) { -if (pool == &adapter->rx_buff_pool[i]) +if (pool == &adapter->rx_buff_pool[0 ][i])continue ; -if (!adapter->rx_buff_pool[i].active) +if (!adapter->rx_buff_pool[0 ][i].active)continue ; -if (mtu <= adapter->rx_buff_pool[i].buff_size) +if (mtu <= + adapter->rx_buff_pool[0 ][i].buff_size)break ; }
@@ -2213,11 +2228,11 @@ static void ibmveth_remove_buffer_from_pool_test(struct kunit *test)/* Set sane values for buffer pools */ for (int i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) -ibmveth_init_buffer_pool (&adapter->rx_buff_pool[i], i, +ibmveth_init_buffer_pool (&adapter->rx_buff_pool[0 ][i], i, pool_count[i], pool_size[i], pool_active[i]); - pool = &adapter->rx_buff_pool[0 ]; + pool = &adapter->rx_buff_pool[0 ][0 ]; pool->skbuff =kunit_kcalloc (test, pool->size,sizeof (void *),GFP_KERNEL );KUNIT_ASSERT_NOT_ERR_OR_NULL (test, pool->skbuff);
@@ -2225,7 +2240,7 @@ static void ibmveth_remove_buffer_from_pool_test(struct kunit *test)KUNIT_EXPECT_EQ (test, -EINVAL ,ibmveth_remove_buffer_from_pool (adapter, correlator, false));KUNIT_EXPECT_EQ (test, -EINVAL ,ibmveth_remove_buffer_from_pool (adapter, correlator, true)); - correlator = ((u64 )0 <<32 ) | adapter->rx_buff_pool[0 ].size; + correlator = ((u64 )0 <<32 ) | adapter->rx_buff_pool[0 ][0 ].size;KUNIT_EXPECT_EQ (test, -EINVAL ,ibmveth_remove_buffer_from_pool (adapter, correlator, false));KUNIT_EXPECT_EQ (test, -EINVAL ,ibmveth_remove_buffer_from_pool (adapter, correlator, true));
@@ -2258,30 +2273,33 @@ static void ibmveth_rxq_get_buffer_test(struct kunit *test)INIT_WORK (&adapter->work, ibmveth_reset_kunit); - adapter->rx_queue.queue_len =1 ; - adapter->rx_queue.index =0 ; - adapter->rx_queue.queue_addr =kunit_kzalloc (test,sizeof (struct ibmveth_rx_q_entry ), -GFP_KERNEL ); -KUNIT_ASSERT_NOT_ERR_OR_NULL (test, adapter->rx_queue.queue_addr); + adapter->rx_queue[0 ].queue_len =1 ; + adapter->rx_queue[0 ].index =0 ; + adapter->rx_queue[0 ].queue_addr = +kunit_kzalloc (test,sizeof (struct ibmveth_rx_q_entry ), +GFP_KERNEL ); +KUNIT_ASSERT_NOT_ERR_OR_NULL (test, adapter->rx_queue[0 ].queue_addr);/* Set sane values for buffer pools */ for (int i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) -ibmveth_init_buffer_pool (&adapter->rx_buff_pool[i], i, +ibmveth_init_buffer_pool (&adapter->rx_buff_pool[0 ][i], i, pool_count[i], pool_size[i], pool_active[i]); - pool = &adapter->rx_buff_pool[0 ]; + pool = &adapter->rx_buff_pool[0 ][0 ]; pool->skbuff =kunit_kcalloc (test, pool->size,sizeof (void *),GFP_KERNEL );KUNIT_ASSERT_NOT_ERR_OR_NULL (test, pool->skbuff); - adapter->rx_queue.queue_addr[0 ].correlator = (u64 )IBMVETH_NUM_BUFF_POOLS <<32 |0 ; + adapter->rx_queue[0 ].queue_addr[0 ].correlator = + (u64 )IBMVETH_NUM_BUFF_POOLS <<32 |0 ;KUNIT_EXPECT_PTR_EQ (test,NULL ,ibmveth_rxq_get_buffer (adapter)); - adapter->rx_queue.queue_addr[0 ].correlator = (u64 )0 <<32 | adapter->rx_buff_pool[0 ].size; + adapter->rx_queue[0 ].queue_addr[0 ].correlator = + (u64 )0 <<32 | adapter->rx_buff_pool[0 ][0 ].size;KUNIT_EXPECT_PTR_EQ (test,NULL ,ibmveth_rxq_get_buffer (adapter)); pool->skbuff[0 ] = skb; - adapter->rx_queue.queue_addr[0 ].correlator = (u64 )0 <<32 |0 ; + adapter->rx_queue[0 ].queue_addr[0 ].correlator = (u64 )0 <<32 |0 ;KUNIT_EXPECT_PTR_EQ (test, skb,ibmveth_rxq_get_buffer (adapter));flush_work (&adapter->work);
drivers/net/ethernet/ibm/ibmveth.h +13 -5
@@ -258,6 +258,8 @@ static inline long h_illan_attributes(unsigned long unit_address,#define IBMVETH_MAX_TX_BUF_SIZE (1024 * 64)#define IBMVETH_MAX_QUEUES 16U#define IBMVETH_DEFAULT_QUEUES 8U +#define IBMVETH_MAX_RX_QUEUES 1U +#define IBMVETH_DEFAULT_RX_QUEUES 1U#define IBMVETH_MAX_RX_PER_HCALL 8Ustatic int pool_size[] = {512 ,1024 *2 ,1024 *16 ,1024 *32 ,1024 *64 };
@@ -294,18 +296,23 @@ struct ibmveth_rx_q {struct ibmveth_adapter {struct vio_dev *vdev;struct net_device *netdev; -struct napi_struct napi; +struct napi_struct napi[IBMVETH_MAX_RX_QUEUES ];struct work_struct work;unsigned int mcastFilterSize; -void *buffer_list_addr; +void *buffer_list_addr[IBMVETH_MAX_RX_QUEUES ];void *filter_list_addr;void *tx_ltb_ptr[IBMVETH_MAX_QUEUES ];unsigned int tx_ltb_size;dma_addr_t tx_ltb_dma[IBMVETH_MAX_QUEUES ]; -dma_addr_t buffer_list_dma; +dma_addr_t buffer_list_dma[IBMVETH_MAX_RX_QUEUES ];dma_addr_t filter_list_dma; -struct ibmveth_buff_pool rx_buff_pool[IBMVETH_NUM_BUFF_POOLS ]; -struct ibmveth_rx_q rx_queue; +struct ibmveth_buff_pool + rx_buff_pool[IBMVETH_MAX_RX_QUEUES ][IBMVETH_NUM_BUFF_POOLS ]; +struct ibmveth_rx_q rx_queue[IBMVETH_MAX_RX_QUEUES ]; +u64 queue_handle[IBMVETH_MAX_RX_QUEUES ]; +unsigned int queue_irq[IBMVETH_MAX_RX_QUEUES ]; +int multi_queue; +unsigned int num_rx_queues;int rx_csum;int large_send;bool is_active_trunk;
@@ -325,6 +332,7 @@ struct ibmveth_adapter {u64 tx_send_failed;u64 tx_large_packets;u64 rx_large_packets; +/* Ethtool settings */ u8 duplex;u32 speed;
Signature · 1 line
2.50.1 (Apple Git-155)
Jakub Kicinski
maintainerRe: [PATCH net-next v4 02/14] ibmveth: Prepare MQ RX adapter data structures
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
ibmveth: Prepare MQ RX adapter data structures
This converts the single-instance RX state in struct ibmveth_adapter (napi, rx_queue, buffer_list_addr/dma, rx_buff_pool) into queue-indexed arrays sized by the new IBMVETH_MAX_RX_QUEUES, adds multi_queue and num_rx_queues, and mechanically rewrites all existing references to index 0. IBMVETH_MAX_RX_QUEUES stays at 1 in this patch.
A couple of observations below, neither of which changes runtime behaviour today.
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0
[ ... ]
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter, · line 495
Quoted 13 lines
drivers/net/ethernet/ibm/ibmveth.c +3
-2@@ -482,42 +486,44 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,mb (); -atomic_dec (&(adapter->rx_buff_pool[pool].available)); +atomic_dec (&adapter->rx_buff_pool[0 ][pool].available);return 0 ; }/* get the current buffer on the rx queue */ -static inline struct sk_buff *ibmveth_rxq_get_buffer (struct ibmveth_adapter *adapter) +static inline struct sk_buff * +ibmveth_rxq_get_buffer (struct ibmveth_adapter *adapter)
[Severity: Low]
This isn't a bug, but the patch carries a few edits that are not part of the stated mechanical index-0 conversion:
- dropping the redundant parentheses in the atomic_dec() call in ibmveth_remove_buffer_from_pool()
- reflowing the ibmveth_rxq_get_buffer() prototype
- rewriting the kernel-doc of ibmveth_get_desired_dma() from "Return value:" to "Return:"
- inserting a blank line in struct ibmveth_adapter after rx_large_packets
The commit message says:
This patch keeps behavior unchanged by mechanically switching existing references to index 0
Could these cosmetic edits move to a separate cleanup patch, or be mentioned in the changelog? As it stands each hunk has to first be classified as conversion or cleanup before the "behavior unchanged" claim can be confirmed.
[ ... ]
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +1
-2@@ -1693,8 +1704,7 @@ static void ibmveth_poll_controller(struct net_device *dev)* * @vdev: struct vio_dev for the device whose desired IO mem is to be returned * -* Return value: -* Number of bytes of IO data the driver will need to perform well. +* Return: Number of bytes of IO data the driver will need to perform well. */ static unsigned long ibmveth_get_desired_dma (struct vio_dev *vdev) {
[ ... ]
Quoted 13 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-2@@ -1898,10 +1912,10 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)memcpy (pool_count, pool_count_cmo,sizeof (pool_count));for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) { -struct kobject *kobj = &adapter->rx_buff_pool[i].kobj; +struct kobject *kobj = &adapter->rx_buff_pool[0 ][i].kobj;int error; -ibmveth_init_buffer_pool (&adapter->rx_buff_pool[i], i, +ibmveth_init_buffer_pool (&adapter->rx_buff_pool[0 ][i], i, pool_count[i], pool_size[i], pool_active[i]); error = kobject_init_and_add(kobj, &ktype_veth_pool,
[ ... ]
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.h +0
-0
[ ... ]
drivers/net/ethernet/ibm/ibmveth.h struct ibmveth_rx_q { · line 298
Quoted 6 lines
drivers/net/ethernet/ibm/ibmveth.h +3
-2@@ -294,18 +296,23 @@ struct ibmveth_rx_q { -struct ibmveth_buff_pool rx_buff_pool[IBMVETH_NUM_BUFF_POOLS ]; -struct ibmveth_rx_q rx_queue; +struct ibmveth_buff_pool + rx_buff_pool[IBMVETH_MAX_RX_QUEUES ][IBMVETH_NUM_BUFF_POOLS ]; +struct ibmveth_rx_q rx_queue[IBMVETH_MAX_RX_QUEUES ];
[Severity: Low]
Is replicating the whole struct ibmveth_buff_pool per queue the layout you want here?
struct ibmveth_buff_pool embeds a struct kobject, which is the per-pool sysfs tuning object. Only row 0's kobjects are ever initialized and registered, in ibmveth_probe():
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
struct kobject *kobj = &adapter->rx_buff_pool[0][i].kobj;
...
error = kobject_init_and_add(kobj, &ktype_veth_pool,
&dev->dev.kobj, "pool%d", i);and only row 0's are dropped, in ibmveth_remove():
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) kobject_put(&adapter->rx_buff_pool[0][i].kobj);
So every row above 0 carries a kobject that is never initialized and never used.
Later in the series IBMVETH_MAX_RX_QUEUES is raised to 16U, at which point the netdev private area unconditionally holds 16 x 5 pool structs regardless of num_rx_queues, of which 75 embedded kobjects are dead weight.
The follow-on code also shows that only part of the struct is really
per-queue: ibmveth_alloc_single_rx_queue() copies size, index, buff_size, threshold and active from row 0 into each new row, so the pool configuration is shared while free_map/dma_addr/skbuff/producer_index/ consumer_index/available are the genuinely per-queue state.
Would it be cleaner to split the struct into one shared, sysfs-visible configuration object plus a small per-queue state array, given this patch is the one that fixes the layout for the rest of the series?
Is it also intentional that the per-pool sysfs tuning interface now implicitly means "queue 0 configures all queues"? If so, could that be stated in the changelog?
Quoted 8 lines
drivers/net/ethernet/ibm/ibmveth.h +1
-0@@ -325,6 +332,7 @@ struct ibmveth_adapter {u64 tx_send_failed;u64 tx_large_packets;u64 rx_large_packets; +/* Ethtool settings */ u8 duplex;u32 speed;
Re: [PATCH net-next v4 02/14] ibmveth: Prepare MQ RX adapter data structures
Quoted 5 lines · Jakub Kicinski
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
ibmveth: Prepare MQ RX adapter data structures
Quoted 8 lines
This converts the single-instance RX state in struct
ibmveth_adapter(napi,rx_queue, buffer_list_addr/dma,rx_buff_pool) into queue-indexed arrays sized by the newIBMVETH_MAX_RX_QUEUES, addsmulti_queueandnum_rx_queues, and mechanically rewrites all existing references to index 0.IBMVETH_MAX_RX_QUEUESstays at 1 in this patch.A couple of observations below, neither of which changes runtime behaviour today.
Hi Jakub,
Thanks for the notes.
Quoted 36 lines
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0[ ... ]
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter, · line 495
Quoted 13 lines
drivers/net/ethernet/ibm/ibmveth.c +3
-2@@ -482,42 +486,44 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,mb (); -atomic_dec (&(adapter->rx_buff_pool[pool].available)); +atomic_dec (&adapter->rx_buff_pool[0 ][pool].available);return 0 ; }/* get the current buffer on the rx queue */ -static inline struct sk_buff *ibmveth_rxq_get_buffer (struct ibmveth_adapter *adapter) +static inline struct sk_buff * +ibmveth_rxq_get_buffer (struct ibmveth_adapter *adapter)[Severity: Low]
This isn't a bug, but the patch carries a few edits that are not part of the stated mechanical index-0 conversion:- dropping the redundant parentheses in the
atomic_dec()call inibmveth_remove_buffer_from_pool()
- reflowing theibmveth_rxq_get_buffer()prototype
- rewriting the kernel-doc ofibmveth_get_desired_dma()from "Return value:" to "Return:"
- inserting a blank line in structibmveth_adapterafterrx_large_packetsThe commit message says:
This patch keeps behavior unchanged by mechanically switching existing references to index 0
Quoted 6 lines
Could these cosmetic edits move to a separate cleanup patch, or be mentioned in the changelog? As it stands each hunk has to first be classified as conversion or cleanup before the "behavior unchanged" claim can be confirmed.
[ ... ]
Agreed. Those edits are not part of the stated index-0 conversion, so for v5 I will drop them from patch 2. If they are still worth doing, I can send them later as a separate cleanup patch.
Quoted 53 lines
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +1
-2@@ -1693,8 +1704,7 @@ static void ibmveth_poll_controller(struct net_device *dev)* * @vdev: struct vio_dev for the device whose desired IO mem is to be returned * -* Return value: -* Number of bytes of IO data the driver will need to perform well. +* Return: Number of bytes of IO data the driver will need to perform well. */ static unsigned long ibmveth_get_desired_dma (struct vio_dev *vdev) {[ ... ]
Quoted 13 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-2@@ -1898,10 +1912,10 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)memcpy (pool_count, pool_count_cmo,sizeof (pool_count));for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) { -struct kobject *kobj = &adapter->rx_buff_pool[i].kobj; +struct kobject *kobj = &adapter->rx_buff_pool[0 ][i].kobj;int error; -ibmveth_init_buffer_pool (&adapter->rx_buff_pool[i], i, +ibmveth_init_buffer_pool (&adapter->rx_buff_pool[0 ][i], i, pool_count[i], pool_size[i], pool_active[i]); error = kobject_init_and_add(kobj, &ktype_veth_pool,[ ... ]
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.h +0
-0[ ... ]
drivers/net/ethernet/ibm/ibmveth.h struct ibmveth_rx_q { · line 298
Quoted 6 lines
drivers/net/ethernet/ibm/ibmveth.h +3
-2@@ -294,18 +296,23 @@ struct ibmveth_rx_q { -struct ibmveth_buff_pool rx_buff_pool[IBMVETH_NUM_BUFF_POOLS ]; -struct ibmveth_rx_q rx_queue; +struct ibmveth_buff_pool + rx_buff_pool[IBMVETH_MAX_RX_QUEUES ][IBMVETH_NUM_BUFF_POOLS ]; +struct ibmveth_rx_q rx_queue[IBMVETH_MAX_RX_QUEUES ];[Severity: Low]
Is replicating the whole structibmveth_buff_poolper queue the layout you want here?struct
ibmveth_buff_poolembeds a struct kobject, which is the per-pool sysfs tuning object. Only row 0's kobjects are ever initialized and registered, inibmveth_probe():for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) { struct kobject *kobj = &adapter->rx_buff_pool[0][i].kobj; ... error = kobject_init_and_add(kobj, &ktype_veth_pool, &dev->dev.kobj, "pool%d", i);
Quoted 7 lines
and only row 0's are dropped, in
ibmveth_remove():for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) kobject_put(&adapter->rx_buff_pool[0][i].kobj);So every row above 0 carries a kobject that is never initialized and never used.
Correct. Only row 0's pool kobjects are initialized; rows 1..N-1 carry unused embedded kobjects in the current layout.
Quoted 11 lines
Later in the series
IBMVETH_MAX_RX_QUEUESis raised to 16U, at which point the netdev private area unconditionally holds 16 x 5 pool structs regardless ofnum_rx_queues, of which 75 embedded kobjects are dead weight.The follow-on code also shows that only part of the struct is really
per-queue:ibmveth_alloc_single_rx_queue()copies size, index,buff_size, threshold and active from row 0 into each new row, so the pool configuration is shared while free_map/dma_addr/skbuff/producer_index/ consumer_index/available are the genuinely per-queue state.
Only part of ibmveth_buff_pool is truly per-queue; the sysfs-visible tuning fields are effectively shared in this series.
Quoted 3 lines
Would it be cleaner to split the struct into one shared, sysfs-visible configuration object plus a small per-queue state array, given this patch is the one that fixes the layout for the rest of the series?
Longer term, splitting this into a shared sysfs-visible configuration object plus smaller per-queue runtime state would be cleaner. I am deferring that redesign for this series.
Quoted 4 lines
Is it also intentional that the per-pool sysfs tuning interface now implicitly means "queue 0 configures all queues"? If so, could that be stated in the changelog?
Yes, that is intentional for this series. Queue 0 is acting as the shared pool configuration/template surface rather than as a queue-0-only tuning interface. In v5 I will make that explicit in the changelog and cover letter:
- only queue 0's poolN sysfs nodes are registered
- those nodes represent the shared pool geometry/template policy (buff_size, size, active)
- open() and resize copy that template into each RX queue's pool row
- queues 1..N do not get separate pool sysfs controls in this series
Thanks again for the review,
Mingming
PATCH v4 3/14 ibmveth: Refactor RX resource allocation for MQ RX bring-up
In reply to Mingming Caoibmveth_open() allocates the filter list and every RX queue inline. That is already a long sequence and would get uglier once we loop over num_rx_queues, especially on error unwind.
Pull the RX bits into helpers and wire them into open()/close() in the same patch:
ibmveth_alloc_filter_list() / ibmveth_free_filter_list()
- shared multicast filter list (one per adapter, not per queue)
ibmveth_alloc_rx_queues() / ibmveth_cleanup_rx_resources()
- per-queue buffer lists and RX rings, looping [0, num_rx_queues)
alloc_rx_queues() rolls back on failure so open() does not need nested goto chains for every queue index. open-failure and close release the same resources through the same helpers.
Runtime behavior stays single-queue (num_rx_queues is still 1). Buffer pools, IRQ, TX LTB, and PHYP registration remain inline for later helper patches.
Also set rc = -ENOMEM before the TX LTB allocation loop so a failed ibmveth_allocate_tx_ltb() still returns a useful errno after the RX allocation blocks move into helpers.
---
Changes in v4:
- Introduce RX/filter allocation helpers in the same patch that wires their first open/close callers; v3 left unused statics ahead of the old open/close pipeline patch.
- Preserve correct -ENOMEM return on TX LTB allocation failure after the RX helper extract.
- Drop reliance on v3's separate "open/close pipeline" patch for this wiring.
drivers/net/ethernet/ibm/ibmveth.c | 268 ++++++++++++++++++++--------- 1 file changed, 190 insertions(+), 78 deletions(-)
drivers/net/ethernet/ibm/ibmveth.c +190 -78
@@ -151,6 +151,184 @@ static unsigned int ibmveth_real_max_tx_queues(void)return min (n_cpu,IBMVETH_MAX_QUEUES ); } +/** +* ibmveth_alloc_filter_list - Allocate and map filter list +* @adapter: ibmveth adapter structure +* +* Return: 0 on success, negative error code on failure +*/ +static int +ibmveth_alloc_filter_list (struct ibmveth_adapter *adapter) +{ +struct device *dev = &adapter->vdev->dev; +struct net_device *netdev = adapter->netdev; + + adapter->filter_list_addr = (void *)get_zeroed_page (GFP_KERNEL ); +if (!adapter->filter_list_addr) { +netdev_err (netdev,"unable to allocate filter pages\n" ); +return -ENOMEM ; + } + + adapter->filter_list_dma =dma_map_single (dev, + adapter->filter_list_addr, +4096 ,DMA_BIDIRECTIONAL ); +if (dma_mapping_error (dev, adapter->filter_list_dma)) { +netdev_err (netdev,"unable to map filter list pages\n" ); +free_page ((unsigned long )adapter->filter_list_addr); + adapter->filter_list_addr =NULL ; +return -ENOMEM ; + } + +netdev_dbg (netdev,"filter list @ 0x%p (DMA: 0x%llx)\n" , + adapter->filter_list_addr, + (unsigned long long )adapter->filter_list_dma); + +return 0 ; +} + +/** +* ibmveth_free_filter_list - Free filter list resources +* @adapter: ibmveth adapter structure +*/ +static void +ibmveth_free_filter_list (struct ibmveth_adapter *adapter) +{ +struct device *dev = &adapter->vdev->dev; + +if (adapter->filter_list_dma) { +dma_unmap_single (dev, adapter->filter_list_dma,4096 , +DMA_BIDIRECTIONAL ); + adapter->filter_list_dma =0 ; + } + +if (adapter->filter_list_addr) { +free_page ((unsigned long )adapter->filter_list_addr); + adapter->filter_list_addr =NULL ; + } +} + +/** +* ibmveth_alloc_rx_queues - Allocate per-queue RX resources +* @adapter: ibmveth adapter structure +* @rxq_entries: Number of entries per RX queue +* +* Return: 0 on success, negative error code on failure +*/ +static int +ibmveth_alloc_rx_queues (struct ibmveth_adapter *adapter,int rxq_entries) +{ +struct device *dev = &adapter->vdev->dev; +struct net_device *netdev = adapter->netdev; +int i; + +for (i =0 ; i < adapter->num_rx_queues; i++) { + adapter->buffer_list_addr[i] = + (void *)get_zeroed_page (GFP_KERNEL ); +if (!adapter->buffer_list_addr[i]) { +netdev_err (netdev, +"unable to allocate buffer list for queue %d\n" , + i); + goto err_cleanup; + } + + adapter->rx_queue[i].queue_len = +sizeof (struct ibmveth_rx_q_entry ) * rxq_entries; + adapter->rx_queue[i].queue_addr = +dma_alloc_coherent (dev, adapter->rx_queue[i].queue_len, + &adapter->rx_queue[i].queue_dma, +GFP_KERNEL ); +if (!adapter->rx_queue[i].queue_addr) { +netdev_err (netdev, +"unable to allocate RX queue for queue %d\n" , + i); + goto err_cleanup; + } + + adapter->buffer_list_dma[i] = +dma_map_single (dev, adapter->buffer_list_addr[i], +4096 ,DMA_BIDIRECTIONAL ); +if (dma_mapping_error (dev, adapter->buffer_list_dma[i])) { +netdev_err (netdev, +"unable to map buffer list for queue %d\n" , + i); + adapter->buffer_list_dma[i] =0 ; + goto err_cleanup; + } + + adapter->rx_queue[i].index =0 ; + adapter->rx_queue[i].num_slots = rxq_entries; + adapter->rx_queue[i].toggle =1 ; + +netdev_dbg (netdev,"queue %d: buffer_list @ 0x%p (DMA: 0x%llx), rx_queue @ 0x%p (DMA: 0x%llx), %llu entries\n" , + i, adapter->buffer_list_addr[i], + (unsigned long long )adapter->buffer_list_dma[i], + adapter->rx_queue[i].queue_addr, + (unsigned long long )adapter->rx_queue[i].queue_dma, + (unsigned long long )rxq_entries); + } + +netdev_dbg (netdev,"allocated %d RX queue(s) with %d entries each\n" , + adapter->num_rx_queues, rxq_entries); + +return 0 ; + +err_cleanup: +/* Clean up previously allocated queues */ +for (; i >=0 ; i--) { +if (adapter->buffer_list_dma[i]) { +dma_unmap_single (dev, adapter->buffer_list_dma[i], +4096 ,DMA_BIDIRECTIONAL ); + adapter->buffer_list_dma[i] =0 ; + } +if (adapter->rx_queue[i].queue_addr) { +dma_free_coherent (dev, adapter->rx_queue[i].queue_len, + adapter->rx_queue[i].queue_addr, + adapter->rx_queue[i].queue_dma); + adapter->rx_queue[i].queue_addr =NULL ; + } +if (adapter->buffer_list_addr[i]) { +free_page ((unsigned long )adapter->buffer_list_addr[i]); + adapter->buffer_list_addr[i] =NULL ; + } + } + +return -ENOMEM ; +} + +/** +* ibmveth_cleanup_rx_resources - Free all RX queue resources +* @adapter: ibmveth adapter structure +*/ +static void +ibmveth_cleanup_rx_resources (struct ibmveth_adapter *adapter) +{ +struct device *dev = &adapter->vdev->dev; +int i; + +netdev_dbg (adapter->netdev,"cleaning up %d RX queue(s)\n" , + adapter->num_rx_queues); + +for (i =0 ; i < adapter->num_rx_queues; i++) { +if (adapter->buffer_list_dma[i]) { +dma_unmap_single (dev, adapter->buffer_list_dma[i], +4096 ,DMA_BIDIRECTIONAL ); + adapter->buffer_list_dma[i] =0 ; + } + +if (adapter->rx_queue[i].queue_addr) { +dma_free_coherent (dev, adapter->rx_queue[i].queue_len, + adapter->rx_queue[i].queue_addr, + adapter->rx_queue[i].queue_dma); + adapter->rx_queue[i].queue_addr =NULL ; + } + +if (adapter->buffer_list_addr[i]) { +free_page ((unsigned long )adapter->buffer_list_addr[i]); + adapter->buffer_list_addr[i] =NULL ; + } + } +} +/* setup the initial settings for a buffer pool */ static void ibmveth_init_buffer_pool(struct ibmveth_buff_pool *pool, u32 pool_index, u32 pool_size,
@@ -627,74 +805,34 @@ static int ibmveth_open(struct net_device *netdev)int rc;union ibmveth_buf_desc rxq_desc;int i; -struct device *dev;netdev_dbg (netdev,"open starting\n" );napi_enable (&adapter->napi[0 ]); -for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) +for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) rxq_entries += adapter->rx_buff_pool[0 ][i].size; - rc = -ENOMEM ; - adapter->buffer_list_addr[0 ] = (void *)get_zeroed_page (GFP_KERNEL ); -if (!adapter->buffer_list_addr[0 ]) { -netdev_err (netdev,"unable to allocate list pages\n" ); + rc =ibmveth_alloc_filter_list (adapter); +if (rc) goto out; - } - adapter->filter_list_addr = (void *)get_zeroed_page (GFP_KERNEL ); -if (!adapter->filter_list_addr) { -netdev_err (netdev,"unable to allocate filter pages\n" ); - goto out_free_buffer_list; - } - - dev = &adapter->vdev->dev; - - adapter->rx_queue[0 ].queue_len =sizeof (struct ibmveth_rx_q_entry ) * - rxq_entries; - adapter->rx_queue[0 ].queue_addr = -dma_alloc_coherent (dev, adapter->rx_queue[0 ].queue_len, - &adapter->rx_queue[0 ].queue_dma,GFP_KERNEL ); -if (!adapter->rx_queue[0 ].queue_addr) + rc =ibmveth_alloc_rx_queues (adapter, rxq_entries); +if (rc) goto out_free_filter_list; - adapter->buffer_list_dma[0 ] = -dma_map_single (dev, adapter->buffer_list_addr[0 ], -4096 ,DMA_BIDIRECTIONAL ); -if (dma_mapping_error (dev, adapter->buffer_list_dma[0 ])) { -netdev_err (netdev,"unable to map buffer list pages\n" ); - goto out_free_queue_mem; - } - - adapter->filter_list_dma =dma_map_single (dev, - adapter->filter_list_addr,4096 ,DMA_BIDIRECTIONAL ); -if (dma_mapping_error (dev, adapter->filter_list_dma)) { -netdev_err (netdev,"unable to map filter list pages\n" ); - goto out_unmap_buffer_list; - } - + rc = -ENOMEM ;for (i =0 ; i < netdev->real_num_tx_queues; i++) {if (ibmveth_allocate_tx_ltb (adapter, i)) goto out_free_tx_ltb; } - adapter->rx_queue[0 ].index =0 ; - adapter->rx_queue[0 ].num_slots = rxq_entries; - adapter->rx_queue[0 ].toggle =1 ; - mac_address =ether_addr_to_u64 (netdev->dev_addr); rxq_desc.fields.flags_len =IBMVETH_BUF_VALID | adapter->rx_queue[0 ].queue_len; rxq_desc.fields.address = adapter->rx_queue[0 ].queue_dma; -netdev_dbg (netdev,"buffer list @ 0x%p\n" , - adapter->buffer_list_addr[0 ]); -netdev_dbg (netdev,"filter list @ 0x%p\n" , adapter->filter_list_addr); -netdev_dbg (netdev,"receive q @ 0x%p\n" , - adapter->rx_queue[0 ].queue_addr); -h_vio_signal (adapter->vdev->unit_address,VIO_IRQ_DISABLE ); lpar_rc =ibmveth_register_logical_lan (adapter, rxq_desc, mac_address);
@@ -709,7 +847,7 @@ static int ibmveth_open(struct net_device *netdev) rxq_desc.desc, mac_address); rc = -ENONET ; - goto out_unmap_filter_list; + goto out_free_tx_ltb; }for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) {
@@ -736,8 +874,6 @@ static int ibmveth_open(struct net_device *netdev) goto out_free_buffer_pools; } - rc = -ENOMEM ; -netdev_dbg (netdev,"initial replenish cycle\n" );ibmveth_interrupt (netdev->irq, netdev);
@@ -753,26 +889,12 @@ static int ibmveth_open(struct net_device *netdev)ibmveth_free_buffer_pool (adapter, &adapter->rx_buff_pool[0 ][i]); } -out_unmap_filter_list: -dma_unmap_single (dev, adapter->filter_list_dma,4096 , -DMA_BIDIRECTIONAL ); - out_free_tx_ltb: -while (--i >=0 ) { +while (--i >=0 )ibmveth_free_tx_ltb (adapter, i); - } - -out_unmap_buffer_list: -dma_unmap_single (dev, adapter->buffer_list_dma[0 ],4096 , -DMA_BIDIRECTIONAL ); -out_free_queue_mem: -dma_free_coherent (dev, adapter->rx_queue[0 ].queue_len, - adapter->rx_queue[0 ].queue_addr, - adapter->rx_queue[0 ].queue_dma); +ibmveth_cleanup_rx_resources (adapter); out_free_filter_list: -free_page ((unsigned long )adapter->filter_list_addr); -out_free_buffer_list: -free_page ((unsigned long )adapter->buffer_list_addr[0 ]); +ibmveth_free_filter_list (adapter); out:napi_disable (&adapter->napi[0 ]);return rc;
@@ -781,7 +903,6 @@ static int ibmveth_open(struct net_device *netdev)static int ibmveth_close (struct net_device *netdev) {struct ibmveth_adapter *adapter =netdev_priv (netdev); -struct device *dev = &adapter->vdev->dev;long lpar_rc;int i;
@@ -806,17 +927,8 @@ static int ibmveth_close(struct net_device *netdev)ibmveth_update_rx_no_buffer (adapter); -dma_unmap_single (dev, adapter->buffer_list_dma[0 ],4096 , -DMA_BIDIRECTIONAL ); -free_page ((unsigned long )adapter->buffer_list_addr[0 ]); - -dma_unmap_single (dev, adapter->filter_list_dma,4096 , -DMA_BIDIRECTIONAL ); -free_page ((unsigned long )adapter->filter_list_addr); - -dma_free_coherent (dev, adapter->rx_queue[0 ].queue_len, - adapter->rx_queue[0 ].queue_addr, - adapter->rx_queue[0 ].queue_dma); +ibmveth_cleanup_rx_resources (adapter); +ibmveth_free_filter_list (adapter);for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++)if (adapter->rx_buff_pool[0 ][i].active)
Signature · 1 line
2.50.1 (Apple Git-155)
Jakub Kicinski
maintainerRe: [PATCH net-next v4 03/14] ibmveth: Refactor RX resource allocation for MQ RX bring-up
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
ibmveth: Refactor RX resource allocation for MQ RX bring-up
This moves the filter list and per-queue RX ring allocation out of ibmveth_open() into four new helpers, and wires them into both the open error unwind and ibmveth_close(). Runtime behaviour is meant to stay single-queue since num_rx_queues is still 1.
Quoted 25 lines
drivers/net/ethernet/ibm/ibmveth.c +17
-0@@ -151,6 +151,184 @@ static unsigned int ibmveth_real_max_tx_queues(void)return min (n_cpu,IBMVETH_MAX_QUEUES ); } +/** +* ibmveth_alloc_filter_list - Allocate and map filter list +* @adapter: ibmveth adapter structure +* +* Return: 0 on success, negative error code on failure +*/ +static int +ibmveth_alloc_filter_list (struct ibmveth_adapter *adapter) +{ +struct device *dev = &adapter->vdev->dev; +struct net_device *netdev = adapter->netdev; + + adapter->filter_list_addr = (void *)get_zeroed_page (GFP_KERNEL ); +if (!adapter->filter_list_addr) { +netdev_err (netdev,"unable to allocate filter pages\n" ); +return -ENOMEM ; + }
drivers/net/ethernet/ibm/ibmveth.c static unsigned int ibmveth_real_max_tx_queues(void) · line 180
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+ + adapter->filter_list_dma =dma_map_single (dev, + adapter->filter_list_addr, +4096 ,DMA_BIDIRECTIONAL ); +if (dma_mapping_error (dev, adapter->filter_list_dma)) { +netdev_err (netdev,"unable to map filter list pages\n" ); +free_page ((unsigned long )adapter->filter_list_addr); + adapter->filter_list_addr =NULL ; +return -ENOMEM ; + }
[Severity: Medium]
Should this error path also clear adapter->filter_list_dma? On failure dma_map_single() has stored DMA_MAPPING_ERROR, which is ~(dma_addr_t)0 and therefore non-zero, and only filter_list_addr is reset here.
The sibling helper added in the same patch does zero the cookie in the same situation:
if (dma_mapping_error(dev, adapter->buffer_list_dma[i])) {
...
adapter->buffer_list_dma[i] = 0;
goto err_cleanup;
}so the two new helpers appear to disagree on the convention.
Quoted 12 lines
drivers/net/ethernet/ibm/ibmveth.c +12
-0+ +netdev_dbg (netdev,"filter list @ 0x%p (DMA: 0x%llx)\n" , + adapter->filter_list_addr, + (unsigned long long )adapter->filter_list_dma); + +return 0 ; +} + +/** +* ibmveth_free_filter_list - Free filter list resources +* @adapter: ibmveth adapter structure +*/
drivers/net/ethernet/ibm/ibmveth.c static unsigned int ibmveth_real_max_tx_queues(void) · line 202
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+static void +ibmveth_free_filter_list (struct ibmveth_adapter *adapter) +{ +struct device *dev = &adapter->vdev->dev; + +if (adapter->filter_list_dma) { +dma_unmap_single (dev, adapter->filter_list_dma,4096 , +DMA_BIDIRECTIONAL ); + adapter->filter_list_dma =0 ; + }
[Severity: Medium]
Can this unmap the DMA_MAPPING_ERROR sentinel? ibmveth_free_filter_list() treats any non-zero filter_list_dma as a live mapping, and the mapping-error branch of ibmveth_alloc_filter_list() leaves the sentinel behind.
The state looks reachable without dev_close(). ibmveth_set_csum_offload(), ibmveth_set_tso(), ibmveth_change_mtu() and veth_pool_store() all call ibmveth_close() and then ibmveth_open() directly while netif_running() is still true:
if (netif_running(dev)) {
restart = 1;
ibmveth_close(dev);
}
...
if (restart)
rc2 = ibmveth_open(dev);If that re-open fails at the dma_mapping_error() check in ibmveth_alloc_filter_list(), open() returns through the out: label without calling ibmveth_free_filter_list(), so the adapter is left with filter_list_addr == NULL and filter_list_dma == DMA_MAPPING_ERROR while the netdev is still marked running. A later "ip link set dev ethX down" then reaches ndo_stop == ibmveth_close() -> ibmveth_free_filter_list() and unmaps that invalid address.
On pseries the dma_iommu path catches this in iommu_free_check(), which emits a ratelimited "iommu_free: invalid entry" message and a WARN_ON(1) (a panic with panic_on_warn set), so no TCE state is changed, but the backtrace is still user triggerable with CAP_NET_ADMIN plus a mapping failure.
Setting adapter->filter_list_dma = 0 in that error branch, matching ibmveth_alloc_rx_queues(), would make the guard here reliable.
This is still present unchanged at the end of the series (a3781f4ae789), so a later patch does not appear to address it.
Quoted 7 lines
drivers/net/ethernet/ibm/ibmveth.c +7
-0+ +if (adapter->filter_list_addr) { +free_page ((unsigned long )adapter->filter_list_addr); + adapter->filter_list_addr =NULL ; + } +} +
[ ... ]
Quoted 11 lines
drivers/net/ethernet/ibm/ibmveth.c +11
-0+ adapter->buffer_list_dma[i] = +dma_map_single (dev, adapter->buffer_list_addr[i], +4096 ,DMA_BIDIRECTIONAL ); +if (dma_mapping_error (dev, adapter->buffer_list_dma[i])) { +netdev_err (netdev, +"unable to map buffer list for queue %d\n" , + i); + adapter->buffer_list_dma[i] =0 ; + goto err_cleanup; + } +
[ ... ]
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-5@@ -627,74 +805,34 @@ static int ibmveth_open(struct net_device *netdev) - rc = -ENOMEM ; - adapter->buffer_list_addr[0 ] = (void *)get_zeroed_page (GFP_KERNEL ); -if (!adapter->buffer_list_addr[0 ]) { -netdev_err (netdev,"unable to allocate list pages\n" ); + rc =ibmveth_alloc_filter_list (adapter); +if (rc) goto out; - }
[ ... ]
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-3@@ -753,26 +889,12 @@ static int ibmveth_open(struct net_device *netdev) +ibmveth_cleanup_rx_resources (adapter); out_free_filter_list: -free_page ((unsigned long )adapter->filter_list_addr); -out_free_buffer_list: -free_page ((unsigned long )adapter->buffer_list_addr[0 ]); +ibmveth_free_filter_list (adapter); out:napi_disable (&adapter->napi[0 ]);return rc;
[ ... ]
Re: [PATCH net-next v4 03/14] ibmveth: Refactor RX resource allocation for MQ RX bring-up
Quoted 9 lines · Jakub Kicinski
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
ibmveth: Refactor RX resource allocation for MQ RX bring-upThis moves the filter list and per-queue RX ring allocation out of
ibmveth_open()into four new helpers, and wires them into both the open error unwind andibmveth_close(). Runtime behaviour is meant to stay single-queue sincenum_rx_queuesis still 1.
Hi Jakub,
Thanks for the review.
Quoted 37 lines
Quoted 25 lines
drivers/net/ethernet/ibm/ibmveth.c +17
-0@@ -151,6 +151,184 @@ static unsigned int ibmveth_real_max_tx_queues(void)return min (n_cpu,IBMVETH_MAX_QUEUES ); } +/** +* ibmveth_alloc_filter_list - Allocate and map filter list +* @adapter: ibmveth adapter structure +* +* Return: 0 on success, negative error code on failure +*/ +static int +ibmveth_alloc_filter_list (struct ibmveth_adapter *adapter) +{ +struct device *dev = &adapter->vdev->dev; +struct net_device *netdev = adapter->netdev; + + adapter->filter_list_addr = (void *)get_zeroed_page (GFP_KERNEL ); +if (!adapter->filter_list_addr) { +netdev_err (netdev,"unable to allocate filter pages\n" ); +return -ENOMEM ; + }drivers/net/ethernet/ibm/ibmveth.c static unsigned int ibmveth_real_max_tx_queues(void) · line 180
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+ + adapter->filter_list_dma =dma_map_single (dev, + adapter->filter_list_addr, +4096 ,DMA_BIDIRECTIONAL ); +if (dma_mapping_error (dev, adapter->filter_list_dma)) { +netdev_err (netdev,"unable to map filter list pages\n" ); +free_page ((unsigned long )adapter->filter_list_addr); + adapter->filter_list_addr =NULL ; +return -ENOMEM ; + }[Severity: Medium]
Quoted 4 lines
Should this error path also clear
adapter->filter_list_dma? On failuredma_map_single()has storedDMA_MAPPING_ERROR, which is ~(dma_addr_t)0 and therefore non-zero, and onlyfilter_list_addris reset here.
Yes. In v5 I will clear filter_list_dma on mapping error, matching the existing buffer_list_dma convention.
Quoted 65 lines
The sibling helper added in the same patch does zero the cookie in the same situation:
if (dma_mapping_error(dev, adapter->buffer_list_dma[i])) { ... adapter->buffer_list_dma[i] = 0; goto err_cleanup; }so the two new helpers appear to disagree on the convention.
Quoted 12 lines
drivers/net/ethernet/ibm/ibmveth.c +12
-0+ +netdev_dbg (netdev,"filter list @ 0x%p (DMA: 0x%llx)\n" , + adapter->filter_list_addr, + (unsigned long long )adapter->filter_list_dma); + +return 0 ; +} + +/** +* ibmveth_free_filter_list - Free filter list resources +* @adapter: ibmveth adapter structure +*/ drivers/net/ethernet/ibm/ibmveth.c static unsigned int ibmveth_real_max_tx_queues(void) · line 202
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+static void +ibmveth_free_filter_list (struct ibmveth_adapter *adapter) +{ +struct device *dev = &adapter->vdev->dev; + +if (adapter->filter_list_dma) { +dma_unmap_single (dev, adapter->filter_list_dma,4096 , +DMA_BIDIRECTIONAL ); + adapter->filter_list_dma =0 ; + }[Severity: Medium]
Can this unmap the
DMA_MAPPING_ERRORsentinel?ibmveth_free_filter_list()treats any non-zerofilter_list_dmaas a live mapping, and the mapping-error branch ofibmveth_alloc_filter_list()leaves the sentinel behind.The state looks reachable without
dev_close().ibmveth_set_csum_offload(),ibmveth_set_tso(),ibmveth_change_mtu()andveth_pool_store()all callibmveth_close()and thenibmveth_open()directly whilenetif_running()is still true:if (netif_running(dev)) { restart = 1; ibmveth_close(dev); } ... if (restart) rc2 = ibmveth_open(dev);If that re-open fails at the
dma_mapping_error()check inibmveth_alloc_filter_list(),open()returns through the out: label without callingibmveth_free_filter_list(), so the adapter is left withfilter_list_addr== NULL andfilter_list_dma==DMA_MAPPING_ERRORwhile the netdev is still marked running. A later "ip link set dev ethX down" then reachesndo_stop==ibmveth_close()->ibmveth_free_filter_list()and unmaps that invalid address.On pseries the
dma_iommupath catches this iniommu_free_check(), which emits a ratelimited "iommu_free: invalid entry" message and aWARN_ON(1)(a panic withpanic_on_warnset), so no TCE state is changed, but the backtrace is still user triggerable withCAP_NET_ADMINplus a mapping failure.
Quoted 5 lines
Setting
adapter->filter_list_dma= 0 in that error branch, matchingibmveth_alloc_rx_queues(), would make the guard here reliable.This is still present unchanged at the end of the series (
a3781f4ae789), so a later patch does not appear to address it.
Yes. The failure path here is:
- dma_map_single() fails for the filter page
- open/reopen leaves filter_list_addr = NULL
- filter_list_dma is still left at DMA_MAPPING_ERROR
- a later free path sees a non-zero DMA handle and calls dma_unmap_single() on the sentinel value
That leads to the later bad unmap/WARN.
In v5 I will free the page and set adapter->filter_list_dma = 0 on mapping error while keeping filter_list_addr = NULL, so the free path remains consistent and only unmaps real mappings.
Quoted 34 lines
Quoted 7 lines
drivers/net/ethernet/ibm/ibmveth.c +7
-0+ +if (adapter->filter_list_addr) { +free_page ((unsigned long )adapter->filter_list_addr); + adapter->filter_list_addr =NULL ; + } +} +[ ... ]
Quoted 11 lines
drivers/net/ethernet/ibm/ibmveth.c +11
-0+ adapter->buffer_list_dma[i] = +dma_map_single (dev, adapter->buffer_list_addr[i], +4096 ,DMA_BIDIRECTIONAL ); +if (dma_mapping_error (dev, adapter->buffer_list_dma[i])) { +netdev_err (netdev, +"unable to map buffer list for queue %d\n" , + i); + adapter->buffer_list_dma[i] =0 ; + goto err_cleanup; + } +[ ... ]
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-5@@ -627,74 +805,34 @@ static int ibmveth_open(struct net_device *netdev) - rc = -ENOMEM ; - adapter->buffer_list_addr[0 ] = (void *)get_zeroed_page (GFP_KERNEL ); -if (!adapter->buffer_list_addr[0 ]) { -netdev_err (netdev,"unable to allocate list pages\n" ); + rc =ibmveth_alloc_filter_list (adapter); +if (rc) goto out; - }[ ... ]
Quoted 11 lines
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-3@@ -753,26 +889,12 @@ static int ibmveth_open(struct net_device *netdev) +ibmveth_cleanup_rx_resources (adapter); out_free_filter_list: -free_page ((unsigned long )adapter->filter_list_addr); -out_free_buffer_list: -free_page ((unsigned long )adapter->buffer_list_addr[0 ]); +ibmveth_free_filter_list (adapter); out:napi_disable (&adapter->napi[0 ]);return rc;[ ... ]
Thanks again for the review,
Mingming
PATCH v4 4/14 ibmveth: Refactor buffer pool management for per-queue MQ RX
In reply to Mingming CaoLegacy ibmveth uses five adapter-level RX buffer pools (512 B through 64 KiB). pool_active[] enables the standard-MTU pools by default; larger pools activate when MTU requires them. With single-queue RX that set is shared on one completion path.
MQ requires the same pool model per queue: buffers post with H_ADD_LOGICAL_LAN_BUFFERS_QUEUE against a queue handle and completions return on that queue. Sharing pools across queues would mix ownership and break queue-local replenish/drain/teardown.
Refactor around queue-local pools:
rx_buff_pool[queue][pool] ibmveth_alloc_queue_buffer_pools() ibmveth_free_queue_buffer_pools() ibmveth_alloc_buffer_pools() / ibmveth_free_buffer_pools()
Queue 0 remains the template for pool geometry (size, buff_size, threshold, index, active). For queues 1..N we copy metadata from queue 0, then allocate backing arrays/skbs per queue.
Wire the helpers into open()/close() in the same patch. Runtime remains single-queue (num_rx_queues is still 1).
Error handling is queue-safe:
- allocation failure unwinds only what that queue allocated, then prior queues in the caller
- free paths release by real allocations (free_map/dma_addr/skbuff),
not only pool->active
That allocation-based free check is intentional: later resize and failure paths can leave memory allocated after active was cleared.
Also disable the 64 KiB pool by default at standard MTU. MTU changes activate it when required; leaving it enabled would pin about 16 MiB per RX queue in MQ mode.
---
Changes in v4:
- Introduce the pool helpers in the same patch that wires their first open/close callers, instead of leaving unused statics.
- Copy pool->index when cloning queue-0 geometry to later queues (needed for correlators; also required by incremental resize).
drivers/net/ethernet/ibm/ibmveth.c | 164 +++++++++++++++++++++++++---- drivers/net/ethernet/ibm/ibmveth.h | 2 +- 2 files changed, 145 insertions(+), 21 deletions(-)
drivers/net/ethernet/ibm/ibmveth.c +144 -20
@@ -625,6 +625,145 @@ static void ibmveth_free_buffer_pool(struct ibmveth_adapter *adapter, } } +/** +* ibmveth_alloc_queue_buffer_pools - Allocate buffer pools for a single queue +* @adapter: ibmveth adapter structure +* @queue: queue index +* +* Allocates all active buffer pools for the specified queue. +* Pool metadata must be initialized before calling this function. +* +* Return: 0 on success, negative error code on failure +*/ +static int ibmveth_alloc_queue_buffer_pools (struct ibmveth_adapter *adapter, +int queue) +{ +struct net_device *netdev = adapter->netdev; +int i; + +for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) { +struct ibmveth_buff_pool *bpool = + &adapter->rx_buff_pool[queue][i]; + +if (!bpool->active) +continue ; + +if (ibmveth_alloc_buffer_pool (bpool)) { +netdev_err (netdev, +"pool %d/%d alloc failed (size=%u count=%u)\n" , + i, queue, + bpool->buff_size, + bpool->size); + bpool->active =0 ; + +/* Free pools allocated so far for this queue */ +while (--i >=0 ) { +struct ibmveth_buff_pool *fpool = + &adapter->rx_buff_pool[queue][i]; + +if (fpool->active) +ibmveth_free_buffer_pool (adapter, + fpool); + } +return -ENOMEM ; + } + } + +return 0 ; +} + +/** +* ibmveth_free_queue_buffer_pools - Free buffer pools for a single queue +* @adapter: ibmveth adapter structure +* @queue: queue index +* +* Frees all active buffer pools for the specified queue. +*/ +static void ibmveth_free_queue_buffer_pools (struct ibmveth_adapter *adapter, +int queue) +{ +int i; + +for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) { +struct ibmveth_buff_pool *pool = + &adapter->rx_buff_pool[queue][i]; + +/* Free pool if it has allocated memory, regardless of +* active flag. Pools may have memory allocated but not +* marked active during queue scale-up, so we must check +* for actual allocations. +*/ +if (pool->free_map || pool->dma_addr || pool->skbuff) +ibmveth_free_buffer_pool (adapter, pool); + } +} + +/** +* ibmveth_alloc_buffer_pools - Allocate buffer pools for all queues +* @adapter: ibmveth adapter structure +* +* Initializes pool metadata for queues 1-N from queue 0 settings, +* then allocates buffer pools for all queues using the helper function. +* +* Return: 0 on success, negative error code on failure +*/ +static int +ibmveth_alloc_buffer_pools (struct ibmveth_adapter *adapter) +{ +struct net_device *netdev = adapter->netdev; +int i, q, rc; + +/* Initialize pool metadata for queues 1-15 from queue 0 settings */ +for (q =1 ; q < adapter->num_rx_queues; q++) { +for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) { +struct ibmveth_buff_pool *src = + &adapter->rx_buff_pool[0 ][i]; +struct ibmveth_buff_pool *dst = + &adapter->rx_buff_pool[q][i]; + + dst->size = src->size; + dst->index = src->index; + dst->buff_size = src->buff_size; + dst->threshold = src->threshold; + dst->active = src->active; + } + } + +/* Allocate actual buffers for all queues */ +for (q =0 ; q < adapter->num_rx_queues; q++) { + rc =ibmveth_alloc_queue_buffer_pools (adapter, q); +if (rc) { +/* Free pools for all previous queues */ +while (--q >=0 ) +ibmveth_free_queue_buffer_pools (adapter, q); +return rc; + } + } + +netdev_dbg (netdev,"allocated buffer pools for %d queue(s)\n" , + adapter->num_rx_queues); +return 0 ; +} + +/** +* ibmveth_free_buffer_pools - Free buffer pools for all queues +* @adapter: ibmveth adapter structure +* +* Frees buffer pools for all queues using the helper function. +*/ +static void +ibmveth_free_buffer_pools (struct ibmveth_adapter *adapter) +{ +int q; + +/* Free buffer pools for all queues */ +for (q =0 ; q < adapter->num_rx_queues; q++) +ibmveth_free_queue_buffer_pools (adapter, q); + +netdev_dbg (adapter->netdev,"freed buffer pools for %d queue(s)\n" , + adapter->num_rx_queues); +} +/** * ibmveth_remove_buffer_from_pool - remove a buffer from a pool * @adapter: adapter instance
@@ -850,16 +989,9 @@ static int ibmveth_open(struct net_device *netdev) goto out_free_tx_ltb; } -for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) { -if (!adapter->rx_buff_pool[0 ][i].active) -continue ; -if (ibmveth_alloc_buffer_pool (&adapter->rx_buff_pool[0 ][i])) { -netdev_err (netdev,"unable to alloc pool\n" ); - adapter->rx_buff_pool[0 ][i].active =0 ; - rc = -ENOMEM ; - goto out_free_buffer_pools; - } - } + rc =ibmveth_alloc_buffer_pools (adapter); +if (rc) + goto out_free_tx_ltb;netdev_dbg (netdev,"registering irq 0x%x\n" , netdev->irq); rc = request_irq(netdev->irq, ibmveth_interrupt, 0, netdev->name,
@@ -884,11 +1016,7 @@ static int ibmveth_open(struct net_device *netdev)return 0 ; out_free_buffer_pools: -while (--i >=0 ) { -if (adapter->rx_buff_pool[0 ][i].active) -ibmveth_free_buffer_pool (adapter, - &adapter->rx_buff_pool[0 ][i]); - } +ibmveth_free_buffer_pools (adapter); out_free_tx_ltb:while (--i >=0 )ibmveth_free_tx_ltb (adapter, i);
@@ -927,14 +1055,10 @@ static int ibmveth_close(struct net_device *netdev)ibmveth_update_rx_no_buffer (adapter); +ibmveth_free_buffer_pools (adapter);ibmveth_cleanup_rx_resources (adapter);ibmveth_free_filter_list (adapter); -for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) -if (adapter->rx_buff_pool[0 ][i].active) -ibmveth_free_buffer_pool (adapter, - &adapter->rx_buff_pool[0 ][i]); -for (i =0 ; i < netdev->real_num_tx_queues; i++)ibmveth_free_tx_ltb (adapter, i);
drivers/net/ethernet/ibm/ibmveth.h +1 -1
@@ -265,7 +265,7 @@ static inline long h_illan_attributes(unsigned long unit_address,static int pool_size[] = {512 ,1024 *2 ,1024 *16 ,1024 *32 ,1024 *64 };static int pool_count[] = {256 ,512 ,256 ,256 ,256 };static int pool_count_cmo[] = {256 ,512 ,256 ,256 ,64 }; -static int pool_active[] = {1 ,1 ,0 ,0 ,1 }; +static int pool_active[] = {1 ,1 ,0 ,0 ,0 };#define IBM_VETH_INVALID_MAP ((u16)0xffff)
Signature · 1 line
2.50.1 (Apple Git-155)
Jakub Kicinski
maintainerRe: [PATCH net-next v4 04/14] ibmveth: Refactor buffer pool management for per-queue MQ RX
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
ibmveth: Refactor buffer pool management for per-queue MQ RX
This reworks the five RX buffer pools into per-queue arrays (rx_buff_pool[queue][pool]) with new alloc/free helpers, wires those helpers into ibmveth_open()/ibmveth_close(), and clones pool geometry from queue 0 to queues 1..N. It also flips the default active flag of the 64 KiB pool to 0.
A few questions below.
Quoted 38 lines
drivers/net/ethernet/ibm/ibmveth.c +30
-0@@ -625,6 +625,145 @@ static void ibmveth_free_buffer_pool(struct ibmveth_adapter *adapter, } } +/** +* ibmveth_alloc_queue_buffer_pools - Allocate buffer pools for a single queue +* @adapter: ibmveth adapter structure +* @queue: queue index +* +* Allocates all active buffer pools for the specified queue. +* Pool metadata must be initialized before calling this function. +* +* Return: 0 on success, negative error code on failure +*/ +static int ibmveth_alloc_queue_buffer_pools (struct ibmveth_adapter *adapter, +int queue) +{ +struct net_device *netdev = adapter->netdev; +int i; + +for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) { +struct ibmveth_buff_pool *bpool = + &adapter->rx_buff_pool[queue][i]; + +if (!bpool->active) +continue ; + +if (ibmveth_alloc_buffer_pool (bpool)) { +netdev_err (netdev, +"pool %d/%d alloc failed (size=%u count=%u)\n" , + i, queue, + bpool->buff_size, + bpool->size); + bpool->active =0 ;
drivers/net/ethernet/ibm/ibmveth.c static void ibmveth_free_buffer_pool(struct ibmveth_adapter *adapter, · line 667
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+ +/* Free pools allocated so far for this queue */ +while (--i >=0 ) { +struct ibmveth_buff_pool *fpool = + &adapter->rx_buff_pool[queue][i]; + +if (fpool->active) +ibmveth_free_buffer_pool (adapter, + fpool); + }
[Severity: Low]
Is the free predicate here meant to differ from the one used by the sibling helper added in the same patch?
This unwind path keys off fpool->active, while ibmveth_free_queue_buffer_pools() deliberately frees by allocation presence:
if (pool->free_map || pool->dma_addr || pool->skbuff) ibmveth_free_buffer_pool(adapter, pool);
and the changelog states "free paths release by real allocations (free_map/dma_addr/skbuff), not only pool->active".
So on the queue that fails, a pool holding allocations with active == 0 is never released, which is exactly the case the allocation-based predicate was added for. Since a failing ndo_open means ibmveth_close() is never called, would those free_map/dma_addr/skbuff arrays be leaked?
ibmveth_free_buffer_pool() already NULLs all three pointers and is idempotent, so could the unwind just call ibmveth_free_queue_buffer_pools() on the same queue (or use the same predicate)? The same helper is reused by the incremental resize path later in the series, where pools can hold memory while active is clear.
drivers/net/ethernet/ibm/ibmveth.c static void ibmveth_free_buffer_pool(struct ibmveth_adapter *adapter, · line 680
Quoted 13 lines
drivers/net/ethernet/ibm/ibmveth.c +13
-0+return -ENOMEM ; + } + } + +return 0 ; +} + +/** +* ibmveth_free_queue_buffer_pools - Free buffer pools for a single queue +* @adapter: ibmveth adapter structure +* @queue: queue index +* +* Frees all active buffer pools for the specified queue.
^^^^^^^^^^^^^^^^^^^^^^^^^^
[Severity: Low]
The kernel-doc summary says "all active buffer pools", but the body intentionally ignores the active flag and frees on allocation presence, and the in-body comment says the opposite of the summary. Could the summary be reworded to say pools that have allocations? Later MQ patches reuse this helper and depend on that distinction.
Quoted 20 lines
drivers/net/ethernet/ibm/ibmveth.c +20
-0+*/ +static void ibmveth_free_queue_buffer_pools (struct ibmveth_adapter *adapter, +int queue) +{ +int i; + +for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) { +struct ibmveth_buff_pool *pool = + &adapter->rx_buff_pool[queue][i]; + +/* Free pool if it has allocated memory, regardless of +* active flag. Pools may have memory allocated but not +* marked active during queue scale-up, so we must check +* for actual allocations. +*/ +if (pool->free_map || pool->dma_addr || pool->skbuff) +ibmveth_free_buffer_pool (adapter, pool); + } +} +
[ ... ]
drivers/net/ethernet/ibm/ibmveth.c static void ibmveth_free_buffer_pool(struct ibmveth_adapter *adapter, · line 716
Quoted 7 lines
drivers/net/ethernet/ibm/ibmveth.c +7
-0+static int +ibmveth_alloc_buffer_pools (struct ibmveth_adapter *adapter) +{ +struct net_device *netdev = adapter->netdev; +int i, q, rc; + +/* Initialize pool metadata for queues 1-15 from queue 0 settings */
^^^^^^^^^^
[Severity: Low]
IBMVETH_MAX_RX_QUEUES is 1U at this commit, so queues 1-15 do not exist yet and this loop body cannot run. Should the comment say "queues 1..N" until the bound is raised later in the series?
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-0+for (q =1 ; q < adapter->num_rx_queues; q++) { +for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) { +struct ibmveth_buff_pool *src = + &adapter->rx_buff_pool[0 ][i];
drivers/net/ethernet/ibm/ibmveth.c static void ibmveth_free_buffer_pool(struct ibmveth_adapter *adapter, · line 730
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+struct ibmveth_buff_pool *dst = + &adapter->rx_buff_pool[q][i]; + + dst->size = src->size; + dst->index = src->index; + dst->buff_size = src->buff_size; + dst->threshold = src->threshold; + dst->active = src->active; + } + }
[Severity: Low]
This turns the per-pool sysfs knobs into a per-queue template. Is that intended to stay undocumented?
Only queue 0 gets sysfs objects, in ibmveth_probe():
struct kobject *kobj = &adapter->rx_buff_pool[0][i].kobj; ... error = kobject_init_and_add(kobj, &ktype_veth_pool, &dev->dev.kobj, "pool%d", i);
and veth_pool_show()/veth_pool_store() read and write that single instance. After this change the queue-0 values are cloned above and each queue allocates its own copy, so writing num = 4096 to one pool multiplies the real allocation by adapter->num_rx_queues while sysfs still reports the single-queue value.
The multiplier is 1 everywhere in this series (IBMVETH_DEFAULT_RX_QUEUES is 1), so nothing changes today. Should these knobs stay adapter-wide (counts divided across queues), gain per-queue nodes, or should the multiplier be documented?
Quoted 17 lines
drivers/net/ethernet/ibm/ibmveth.c +17
-0+ +/* Allocate actual buffers for all queues */ +for (q =0 ; q < adapter->num_rx_queues; q++) { + rc =ibmveth_alloc_queue_buffer_pools (adapter, q); +if (rc) { +/* Free pools for all previous queues */ +while (--q >=0 ) +ibmveth_free_queue_buffer_pools (adapter, q); +return rc; + } + } + +netdev_dbg (netdev,"allocated buffer pools for %d queue(s)\n" , + adapter->num_rx_queues); +return 0 ; +} +
[ ... ]
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_close(struct net_device *netdev) · line 937
Quoted 13 lines
drivers/net/ethernet/ibm/ibmveth.c +1
-5@@ -927,14 +1055,10 @@ static int ibmveth_close(struct net_device *netdev)ibmveth_update_rx_no_buffer (adapter); +ibmveth_free_buffer_pools (adapter);ibmveth_cleanup_rx_resources (adapter);ibmveth_free_filter_list (adapter); -for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) -if (adapter->rx_buff_pool[0 ][i].active) -ibmveth_free_buffer_pool (adapter, - &adapter->rx_buff_pool[0 ][i]); -
[Severity: Low]
The teardown order changed here, which the changelog does not mention (it only says the helpers are wired into open()/close()).
Baseline:
ibmveth_update_rx_no_buffer()
ibmveth_cleanup_rx_resources()
ibmveth_free_filter_list()
free active pools
New:
ibmveth_update_rx_no_buffer()
ibmveth_free_buffer_pools()
ibmveth_cleanup_rx_resources()
ibmveth_free_filter_list()
h_free_logical_lan(), napi_disable() and free_irq() have all run by this point, so neither PHYP nor NAPI still references the pool buffers or the RX queue, and the swap looks safe. Could the reorder be called out in the changelog, since a future per-queue drain may need the RX completion queue still present while pools are released?
drivers/net/ethernet/ibm/ibmveth.h static inline long h_illan_attributes(unsigned long unit_address, · line 268
Quoted 13 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0for (i =0 ; i < netdev->real_num_tx_queues; i++)ibmveth_free_tx_ltb (adapter, i);drivers/net/ethernet/ibm/ibmveth.h +1
-1@@ -265,7 +265,7 @@ static inline long h_illan_attributes(unsigned long unit_address,static int pool_size[] = {512 ,1024 *2 ,1024 *16 ,1024 *32 ,1024 *64 };static int pool_count[] = {256 ,512 ,256 ,256 ,256 };static int pool_count_cmo[] = {256 ,512 ,256 ,256 ,64 }; -static int pool_active[] = {1 ,1 ,0 ,0 ,1 }; +static int pool_active[] = {1 ,1 ,0 ,0 ,0 };
^^^
[Severity: High]
Does this drop large-receive support for existing single-queue users?
Pools 2 (16 KiB) and 3 (32 KiB) are already inactive by default, so with the 64 KiB pool off the largest buffer ever posted at MTU 1500 becomes 2048 bytes. ibmveth_probe() seeds each pool from this array:
ibmveth_init_buffer_pool(&adapter->rx_buff_pool[0][i], i, pool_count[i], pool_size[i], pool_active[i]);
and the new helper skips inactive pools, so nothing larger is handed to PHYP:
if (!bpool->active) continue;
ibmveth_poll() does receive hypervisor-aggregated frames longer than the MTU:
if ((length > netdev->mtu + ETH_HLEN) || lrg_pkt ||
iph_check == 0xffff) {
ibmveth_rx_mss_helper(skb, mss, lrg_pkt);
adapter->rx_large_packets++;
}Those arrive as single non-scatter buffers, and TSO/large-send is enabled by default in ibmveth_probe() when firmware reports IBMVETH_ILLAN_LRG_SND_SUPPORT. With no buffer larger than 2 KiB posted, are such frames simply dropped by PHYP and counted in the no-buffer counter read by ibmveth_update_rx_no_buffer()?
The changelog says:
"MTU changes activate it when required; leaving it enabled would pin about 16 MiB per RX queue in MQ mode."
Is the first half accurate? ibmveth_change_mtu() activates pools only up to the first pool whose buff_size covers the new MTU:
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
adapter->rx_buff_pool[0][i].active = 1;
if (new_mtu_oh <= adapter->rx_buff_pool[0][i].buff_size) {
...
return 0;
}
}At MTU 1500 that stops at the 2 KiB pool, so the 64 KiB pool is never re-enabled; ibmveth_set_tso() does not touch pool->active either. Recovery then requires a manual write to pool4/active in sysfs.
The second half describes MQ, but IBMVETH_DEFAULT_RX_QUEUES is 1 both at this commit and at the end of the series (a3781f4ae789 still has pool_active[] = { 1, 1, 0, 0, 0}), so the only configurations affected are today's single-queue ones. Would it be better to keep the default at 1 and scale the 64 KiB count (or deactivate it) when RX queues are actually scaled up?
Note also that commit cd7c7ec3687986 ("ibmveth: change rx buffer default allocation for CMO") enabled this pool on purpose and added pool_count_cmo[] = { 256, 512, 256, 256, 64 } specifically to reduce the 64 KiB count under CMO, which suggests the default was deliberate.
As a side effect, ibmveth_get_desired_dma() sums only active pools:
if (adapter->rx_buff_pool[0][i].active) ret += adapter->rx_buff_pool[0][i].size * IOMMU_PAGE_ALIGN(adapter->rx_buff_pool[0][i].buff_size, tbl);
so does the CMO entitlement the driver requests shrink silently as well?
Quoted 3 lines
drivers/net/ethernet/ibm/ibmveth.h +0
-0#define IBM_VETH_INVALID_MAP ((u16)0xffff)
Re: [PATCH net-next v4 04/14] ibmveth: Refactor buffer pool management for per-queue MQ RX
Quoted 5 lines · Jakub Kicinski
his is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
ibmveth: Refactor buffer pool management for per-queue MQ RX
Quoted 7 lines
This reworks the five RX buffer pools into per-queue arrays (rx_buff_pool[queue][pool]) with new alloc/free helpers, wires those helpers into ibmveth_open()/ibmveth_close(), and clones pool geometry from queue 0 to queues 1..N. It also flips the default active flag of the 64 KiB pool to 0.
A few questions below.
Hi Jakub,
Thanks for the careful read.
Quoted 68 lines
Quoted 38 lines
drivers/net/ethernet/ibm/ibmveth.c +30
-0@@ -625,6 +625,145 @@ static void ibmveth_free_buffer_pool(struct ibmveth_adapter *adapter, } } +/** +* ibmveth_alloc_queue_buffer_pools - Allocate buffer pools for a single queue +* @adapter: ibmveth adapter structure +* @queue: queue index +* +* Allocates all active buffer pools for the specified queue. +* Pool metadata must be initialized before calling this function. +* +* Return: 0 on success, negative error code on failure +*/ +static int ibmveth_alloc_queue_buffer_pools (struct ibmveth_adapter *adapter, +int queue) +{ +struct net_device *netdev = adapter->netdev; +int i; + +for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) { +struct ibmveth_buff_pool *bpool = + &adapter->rx_buff_pool[queue][i]; + +if (!bpool->active) +continue ; + +if (ibmveth_alloc_buffer_pool (bpool)) { +netdev_err (netdev, +"pool %d/%d alloc failed (size=%u count=%u)\n" , + i, queue, + bpool->buff_size, + bpool->size); + bpool->active =0 ;drivers/net/ethernet/ibm/ibmveth.c static void ibmveth_free_buffer_pool(struct ibmveth_adapter *adapter, · line 667
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+ +/* Free pools allocated so far for this queue */ +while (--i >=0 ) { +struct ibmveth_buff_pool *fpool = + &adapter->rx_buff_pool[queue][i]; + +if (fpool->active) +ibmveth_free_buffer_pool (adapter, + fpool); + }[Severity: Low]
Is the free predicate here meant to differ from the one used by the sibling helper added in the same patch?
This unwind path keys off
fpool->active, whileibmveth_free_queue_buffer_pools()deliberately frees by allocation presence:if (pool->free_map || pool->dma_addr || pool->skbuff) ibmveth_free_buffer_pool(adapter, pool);and the changelog states "free paths release by real allocations (free_map/dma_addr/skbuff), not only
pool->active".So on the queue that fails, a pool holding allocations with active == 0 is never released, which is exactly the case the allocation-based predicate was added for. Since a failing
ndo_openmeansibmveth_close()is never called, would those free_map/dma_addr/skbuff arrays be leaked?
Quoted 5 lines
ibmveth_free_buffer_pool()already NULLs all three pointers and is idempotent, so could the unwind just callibmveth_free_queue_buffer_pools()on the same queue (or use the same predicate)? The same helper is reused by the incremental resize path later in the series, where pools can hold memory while active is clear.
Agreed. That unwind is wrong because allocation state and active can diverge on the failure path. I will make the failure unwind match ibmveth_free_queue_buffer_pools() by freeing on allocation presence rather than active, so partially allocated pools are not skipped just because active was cleared early.
Quoted 15 lines
drivers/net/ethernet/ibm/ibmveth.c static void ibmveth_free_buffer_pool(struct ibmveth_adapter *adapter, · line 680
Quoted 13 lines
drivers/net/ethernet/ibm/ibmveth.c +13
-0+return -ENOMEM ; + } + } + +return 0 ; +} + +/** +* ibmveth_free_queue_buffer_pools - Free buffer pools for a single queue +* @adapter: ibmveth adapter structure +* @queue: queue index +* +* Frees all active buffer pools for the specified queue. ^^^^^^^^^^^^^^^^^^^^^^^^^^
Quoted 7 lines
[Severity: Low]
The kernel-doc summary says "all active buffer pools", but the body intentionally ignores the active flag and frees on allocation presence, and the in-body comment says the opposite of the summary. Could the summary be reworded to say pools that have allocations? Later MQ patches reuse this helper and depend on that distinction.
Agreed. I will update the kerneldoc to match the actual allocation-presence free rule.
Quoted 31 lines
Quoted 20 lines
drivers/net/ethernet/ibm/ibmveth.c +20
-0+*/ +static void ibmveth_free_queue_buffer_pools (struct ibmveth_adapter *adapter, +int queue) +{ +int i; + +for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) { +struct ibmveth_buff_pool *pool = + &adapter->rx_buff_pool[queue][i]; + +/* Free pool if it has allocated memory, regardless of +* active flag. Pools may have memory allocated but not +* marked active during queue scale-up, so we must check +* for actual allocations. +*/ +if (pool->free_map || pool->dma_addr || pool->skbuff) +ibmveth_free_buffer_pool (adapter, pool); + } +} +[ ... ]
drivers/net/ethernet/ibm/ibmveth.c static void ibmveth_free_buffer_pool(struct ibmveth_adapter *adapter, · line 716
Quoted 7 lines
drivers/net/ethernet/ibm/ibmveth.c +7
-0+static int +ibmveth_alloc_buffer_pools (struct ibmveth_adapter *adapter) +{ +struct net_device *netdev = adapter->netdev; +int i, q, rc; + +/* Initialize pool metadata for queues 1-15 from queue 0 settings */ ^^^^^^^^^^
Quoted 5 lines
[Severity: Low]
IBMVETH_MAX_RX_QUEUESis 1U at this commit, so queues 1-15 do not exist yet and this loop body cannot run. Should the comment say "queues 1..N" until the bound is raised later in the series?
Agreed. I will make that comment generic rather than hard-coding a queue range.
Quoted 32 lines
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-0+for (q =1 ; q < adapter->num_rx_queues; q++) { +for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) { +struct ibmveth_buff_pool *src = + &adapter->rx_buff_pool[0 ][i];drivers/net/ethernet/ibm/ibmveth.c static void ibmveth_free_buffer_pool(struct ibmveth_adapter *adapter, · line 730
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+struct ibmveth_buff_pool *dst = + &adapter->rx_buff_pool[q][i]; + + dst->size = src->size; + dst->index = src->index; + dst->buff_size = src->buff_size; + dst->threshold = src->threshold; + dst->active = src->active; + } + }[Severity: Low]
This turns the per-pool sysfs knobs into a per-queue template. Is that intended to stay undocumented?
Only queue 0 gets sysfs objects, in
ibmveth_probe():struct kobject *kobj = &adapter->rx_buff_pool[0][i].kobj; ... error = kobject_init_and_add(kobj, &ktype_veth_pool, &dev->dev.kobj, "pool%d", i);and veth_pool_show()/veth_pool_store() read and write that single instance. After this change the queue-0 values are cloned above and each queue allocates its own copy, so writing num = 4096 to one pool multiplies the real allocation by
adapter->num_rx_queueswhile sysfs still reports the single-queue value.
Quoted 4 lines
The multiplier is 1 everywhere in this series (
IBMVETH_DEFAULT_RX_QUEUESis 1), so nothing changes today. Should these knobs stay adapter-wide (counts divided across queues), gain per-queue nodes, or should the multiplier be documented?
For this series, queue 0's pool sysfs nodes remain the shared configuration/template interface: they define the pool geometry copied into each RX queue rather than queue-0-only state. Later in the series I add debugfs visibility for per-queue runtime information, but not separate per-queue pool sysfs controls. I will make that split explicit in the patch and cover letter so the multiplier effect is documented rather than implicit.
Quoted 51 lines
Quoted 17 lines
drivers/net/ethernet/ibm/ibmveth.c +17
-0+ +/* Allocate actual buffers for all queues */ +for (q =0 ; q < adapter->num_rx_queues; q++) { + rc =ibmveth_alloc_queue_buffer_pools (adapter, q); +if (rc) { +/* Free pools for all previous queues */ +while (--q >=0 ) +ibmveth_free_queue_buffer_pools (adapter, q); +return rc; + } + } + +netdev_dbg (netdev,"allocated buffer pools for %d queue(s)\n" , + adapter->num_rx_queues); +return 0 ; +} +[ ... ]
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_close(struct net_device *netdev) · line 937
Quoted 13 lines
drivers/net/ethernet/ibm/ibmveth.c +1
-5@@ -927,14 +1055,10 @@ static int ibmveth_close(struct net_device *netdev)ibmveth_update_rx_no_buffer (adapter); +ibmveth_free_buffer_pools (adapter);ibmveth_cleanup_rx_resources (adapter);ibmveth_free_filter_list (adapter); -for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) -if (adapter->rx_buff_pool[0 ][i].active) -ibmveth_free_buffer_pool (adapter, - &adapter->rx_buff_pool[0 ][i]); -[Severity: Low]
The teardown order changed here, which the changelog does not mention (it only says the helpers are wired into open()/close()).
Baseline:
ibmveth_update_rx_no_buffer()
ibmveth_cleanup_rx_resources()
ibmveth_free_filter_list()
free active poolsNew:
ibmveth_update_rx_no_buffer()
ibmveth_free_buffer_pools()
ibmveth_cleanup_rx_resources()
ibmveth_free_filter_list()
Quoted 5 lines
h_free_logical_lan(),napi_disable()andfree_irq()have all run by this point, so neither PHYP nor NAPI still references the pool buffers or the RX queue, and the swap looks safe. Could the reorder be called out in the changelog, since a future per-queue drain may need the RX completion queue still present while pools are released?
Agreed. The reorder looks safe in the current shutdown path, since LAN teardown, NAPI shutdown, and IRQ teardown have already happened by then, but it is still a real teardown-sequencing change. I will call that out in the changelog, especially since later per-queue drain/free ordering may care whether the RX completion queue is still present when pools are released.
Quoted 41 lines
drivers/net/ethernet/ibm/ibmveth.h static inline long h_illan_attributes(unsigned long unit_address, · line 268
Quoted 13 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0for (i =0 ; i < netdev->real_num_tx_queues; i++)ibmveth_free_tx_ltb (adapter, i);drivers/net/ethernet/ibm/ibmveth.h +1
-1@@ -265,7 +265,7 @@ static inline long h_illan_attributes(unsigned long unit_address,static int pool_size[] = {512 ,1024 *2 ,1024 *16 ,1024 *32 ,1024 *64 };static int pool_count[] = {256 ,512 ,256 ,256 ,256 };static int pool_count_cmo[] = {256 ,512 ,256 ,256 ,64 }; -static int pool_active[] = {1 ,1 ,0 ,0 ,1 }; +static int pool_active[] = {1 ,1 ,0 ,0 ,0 };^^^[Severity: High]
Does this drop large-receive support for existing single-queue users?
Pools 2 (16 KiB) and 3 (32 KiB) are already inactive by default, so with the 64 KiB pool off the largest buffer ever posted at MTU 1500 becomes 2048 bytes.
ibmveth_probe()seeds each pool from this array:ibmveth_init_buffer_pool(&adapter->rx_buff_pool[0][i], i, pool_count[i], pool_size[i], pool_active[i]);and the new helper skips inactive pools, so nothing larger is handed to PHYP:
if (!bpool->active) continue;
ibmveth_poll()does receive hypervisor-aggregated frames longer than the MTU:if ((length > netdev->mtu + ETH_HLEN) || lrg_pkt || iph_check == 0xffff) { ibmveth_rx_mss_helper(skb, mss, lrg_pkt); adapter->rx_large_packets++; }
Quoted 5 lines
Those arrive as single non-scatter buffers, and TSO/large-send is enabled by default in
ibmveth_probe()when firmware reportsIBMVETH_ILLAN_LRG_SND_SUPPORT. With no buffer larger than 2 KiB posted, are such frames simply dropped by PHYP and counted in the no-buffer counter read byibmveth_update_rx_no_buffer()?
Agreed. With the default RX queue count still at 1, disabling the 64 KiB pool here changes the historical single-queue default rather than only reducing future MQ memory use. That makes any large-receive dependency on the 64 KiB pool a regression risk for existing single-queue users, not just an MQ tuning choice.
Quoted 17 lines
The changelog says:
"MTU changes activate it when required; leaving it enabled would pin about 16 MiB per RX queue in MQ mode."
Is the first half accurate?
ibmveth_change_mtu()activates pools only up to the first pool whosebuff_sizecovers the new MTU:for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) { adapter->rx_buff_pool[0][i].active = 1; if (new_mtu_oh <= adapter->rx_buff_pool[0][i].buff_size) { ... return 0; } }
Quoted 3 lines
At MTU 1500 that stops at the 2 KiB pool, so the 64 KiB pool is never re-enabled;
ibmveth_set_tso()does not touchpool->activeeither. Recovery then requires a manual write to pool4/active in sysfs.
You are right that the changelog claim about MTU changes re-enabling it was not accurate for the MTU 1500 case: ibmveth_change_mtu() stops once the first pool covers the MTU, so pool 4 does not come back automatically there.
Quoted 7 lines
The second half describes MQ, but
IBMVETH_DEFAULT_RX_QUEUESis 1 both at this commit and at the end of the series (a3781f4ae789still has pool_active[] = { 1, 1, 0, 0, 0}), so the only configurations affected are today's single-queue ones. Would it be better to keep the default at 1 and scale the 64 KiB count (or deactivate it) when RX queues are actually scaled up?
Yes. If MQ memory pressure still needs separate handling, that should be done only when RX queues are actually scaled up, not by changing the single-queue default.
Quoted 6 lines
Note also that commit
cd7c7ec3687986("ibmveth: change rx buffer default allocation for CMO") enabled this pool on purpose and added pool_count_cmo[] = { 256, 512, 256, 256, 64 } specifically to reduce the 64 KiB count under CMO, which suggests the default was deliberate.
Quoted 8 lines
As a side effect,
ibmveth_get_desired_dma()sums only active pools:if (adapter->rx_buff_pool[0][i].active) ret += adapter->rx_buff_pool[0][i].size * IOMMU_PAGE_ALIGN(adapter->rx_buff_pool[0][i].buff_size, tbl);so does the CMO entitlement the driver requests shrink silently as well?
Yes. Because desired DMA is summed only over active pools, disabling the
64 KiB pool here also shrinks the requested CMO footprint....
and the earlier CMO change also suggests the 64 KiB default was
intentional rather than incidental.
In v5 I will restore the default pool activation so the 64 KiB pool remains enabled in the normal single-queue baseline.
Thanks again for the careful review,
Mingming
Quoted 2 lines
Quoted 2 lines
drivers/net/ethernet/ibm/ibmveth.h +0
-0#define IBM_VETH_INVALID_MAP ((u16)0xffff)
PATCH v4 5/14 ibmveth: Refactor RX interrupt control for MQ RX queues
In reply to Mingming CaoQueue 0 and subordinate RX queues use different interrupt control interfaces in PHYP:
- queue 0: h_vio_signal() after h_register_logical_lan()
- queue N: H_VIOCTL against the queue handle/hwirq mapping
The current code is single-queue oriented and cannot safely scale to multiple RX queues in poll completion and open/close IRQ setup.
Introduce queue-indexed interrupt helpers and wire them into open()/close()/poll()/interrupt in the same patch:
ibmveth_toggle_irq() / enable_irq() / disable_irq() ibmveth_setup_rx_interrupts() / ibmveth_cleanup_rx_interrupts() ibmveth_schedule_rx_queue()
These helpers centralize queue0-vs-subordinate dispatch. request_irq() uses &adapter->napi[i] as the per-queue cookie so the handler can resolve the queue index.
Move napi_enable() into setup_rx_interrupts() (after LAN registration and buffer-pool allocation): request_irq -> napi_enable. In this single-queue tree, setup does not yet unmask PHYP; the first poll/kick still enables queue 0 via schedule_rx_queue().
Factor process-context RX kicks (open, resume, pool sysfs, netpoll) into ibmveth_schedule_rx_queue(); keep ibmveth_interrupt() as a thin IRQ-only wrapper.
cleanup_rx_interrupts() masks PHYP and synchronizes IRQs before napi_disable (intentional storm-safety vs classic close ordering).
Runtime remains single-queue (num_rx_queues is still 1).
---
Changes in v4:
- Include irq.h / irqdomain.h with first irq_dispose_mapping() use.
- Introduce IRQ helpers in the same patch that wires open/close/poll callers, instead of leaving unused statics.
- Factor process-context RX kicks into ibmveth_schedule_rx_queue(); keep ibmveth_interrupt() as the IRQ-only wrapper.
- On cleanup, mask PHYP and synchronize_irq before napi_disable (storm-safety; not fully behavior-preserving vs classic close).
- Leave queue_irq[0] set after cleanup (queue 0 uses netdev->irq; next open reuses it). Only subordinate virqs are disposed.
drivers/net/ethernet/ibm/ibmveth.c | 258 +++++++++++++++++++++++++---- 1 file changed, 225 insertions(+), 33 deletions(-)
drivers/net/ethernet/ibm/ibmveth.c +225 -33
@@ -21,6 +21,8 @@#include <linux/skbuff.h> #include <linux/init.h> #include <linux/interrupt.h> +#include <linux/irq.h> +#include <linux/irqdomain.h> #include <linux/mm.h> #include <linux/pm.h> #include <linux/ethtool.h>
@@ -329,6 +331,203 @@ ibmveth_cleanup_rx_resources(struct ibmveth_adapter *adapter) } } +/** +* ibmveth_toggle_irq - Common helper to enable/disable queue interrupts +* @adapter: ibmveth adapter structure +* @queue_index: Index of the queue (0 for primary, 1+ for subordinate) +* @enable: true to enable, false to disable +* +* For queue 0 (primary), uses h_vio_signal() as it's registered via +* h_register_logical_lan(). For subordinate queues (1+), uses H_VIOCTL +* with H_ENABLE/DISABLE_VIO_INTERRUPT for per-queue interrupt control. +* +* Return: 0 on success, error code otherwise +*/ +static int +ibmveth_toggle_irq (struct ibmveth_adapter *adapter,int queue_index, +bool enable) +{ +unsigned long rc; +unsigned long irq = adapter->queue_irq[queue_index]; +const char *action = enable ?"enable" :"disable" ; + +if (queue_index ==0 ) { +/* Primary queue: use h_vio_signal() */ + rc =h_vio_signal (adapter->vdev->unit_address, + enable ?VIO_IRQ_ENABLE :VIO_IRQ_DISABLE ); + }else { +/* Subordinate queues: use H_VIOCTL with hardware IRQ */ +struct irq_data *irq_data =irq_get_irq_data (irq); +irq_hw_number_t hwirq; +u64 vioctl_cmd = enable ?H_ENABLE_VIO_INTERRUPT : +H_DISABLE_VIO_INTERRUPT ; + +if (!irq_data) { +netdev_err (adapter->netdev, +"Failed to get IRQ data for queue %d (virq=%lu)\n" , + queue_index, irq); +return -EINVAL ; + } + + hwirq =irqd_to_hwirq (irq_data); + rc =plpar_hcall_norets (H_VIOCTL , + adapter->vdev->unit_address, + vioctl_cmd, + hwirq,0 ,0 ); + +if (rc ==H_PARAMETER ) { +/* H_PARAMETER is non-fatal when IRQ is already in +* the requested state. +*/ +netdev_warn_once (adapter->netdev, +"H_VIOCTL %s IRQ returned H_PARAMETER for queue %d (hwirq=%lu)\n" , + action, queue_index, hwirq); +return 0 ; + } + } + +if (rc) +netdev_err (adapter->netdev, +"Failed to %s IRQ for queue %d, rc=%ld\n" , + action, queue_index, rc); +return rc; +} + +/** +* ibmveth_disable_irq - Disable interrupt for a specific queue +* @adapter: ibmveth adapter structure +* @queue_index: Index of the queue (0 for primary, 1+ for subordinate) +* +* Return: 0 on success, error code otherwise +*/ +static int +ibmveth_disable_irq (struct ibmveth_adapter *adapter,int queue_index) +{ +return ibmveth_toggle_irq (adapter, queue_index, false); +} + +/** +* ibmveth_enable_irq - Enable interrupt for a specific queue +* @adapter: ibmveth adapter structure +* @queue_index: Index of the queue (0 for primary, 1+ for subordinate) +* +* Return: 0 on success, error code otherwise +*/ +static int +ibmveth_enable_irq (struct ibmveth_adapter *adapter,int queue_index) +{ +return ibmveth_toggle_irq (adapter, queue_index, true); +} + +/** +* ibmveth_setup_rx_interrupts - Register IRQs and enable NAPI +* @adapter: ibmveth adapter structure +* +* Registers interrupt handlers for all RX queues and enables NAPI polling. +* On error, cleans up any successfully registered IRQs before returning. +* +* Return: 0 on success, negative error code on failure +*/ +static int +ibmveth_setup_rx_interrupts (struct ibmveth_adapter *adapter) +{ +struct net_device *netdev = adapter->netdev; +int i, rc; + +for (i =0 ; i < adapter->num_rx_queues; i++) { +if (!adapter->queue_irq[i]) { +netdev_err (netdev,"queue %d has invalid IRQ (0)\n" , i); + rc = -EINVAL ; + goto err_free_irqs; + } + + rc =request_irq (adapter->queue_irq[i], ibmveth_interrupt, +0 , netdev->name, &adapter->napi[i]); +if (rc) { +netdev_err (netdev, +"request_irq() failed for irq 0x%x queue %d: %d\n" , + adapter->queue_irq[i], i, rc); + goto err_free_irqs; + } + } + +for (i =0 ; i < adapter->num_rx_queues; i++) +napi_enable (&adapter->napi[i]); + +return 0 ; + +err_free_irqs: +while (--i >=0 ) +free_irq (adapter->queue_irq[i], &adapter->napi[i]); +return rc; +} + +/** +* ibmveth_cleanup_rx_interrupts - Mask PHYP, disable NAPI, free IRQs +* @adapter: ibmveth adapter structure +* +* Tears down RX interrupt delivery for all queues. Mask PHYP before +* napi_disable so ibmveth_interrupt cannot return IRQ_HANDLED without +* masking (same storm window as scale-down). Safe for close and for +* open failure after setup_rx_interrupts() already unmasked PHYP. +*/ +static void +ibmveth_cleanup_rx_interrupts (struct ibmveth_adapter *adapter) +{ +int i; + +for (i =0 ; i < adapter->num_rx_queues; i++) { +if (adapter->queue_irq[i]) { +ibmveth_disable_irq (adapter, i); +synchronize_irq (adapter->queue_irq[i]); + } + } + +for (i =0 ; i < adapter->num_rx_queues; i++) +napi_disable (&adapter->napi[i]); + +for (i =0 ; i < adapter->num_rx_queues; i++) { +if (adapter->queue_irq[i]) +free_irq (adapter->queue_irq[i], &adapter->napi[i]); + } + +/* Dispose IRQ mappings for subordinate queues (1-15). +* Queue 0 uses netdev->irq from device tree, not irq_create_mapping(). +*/ +for (i =1 ; i < adapter->num_rx_queues; i++) { +if (adapter->queue_irq[i]) { +irq_dispose_mapping (adapter->queue_irq[i]); + adapter->queue_irq[i] =0 ; + } + } + +/* Queue 0 uses netdev->irq; leave queue_irq[0] for next open. */ +} + +/** +* ibmveth_schedule_rx_queue - Mask PHYP IRQ and schedule NAPI for one RX queue +* @adapter: ibmveth adapter structure +* @qindex: RX queue index +* +* Shared by the IRQ handler and process-context kick paths (open, resume, +* pool sysfs, netpoll). Keep ibmveth_interrupt() as the IRQ-only wrapper. +*/ +static void ibmveth_schedule_rx_queue (struct ibmveth_adapter *adapter, +int qindex) +{ +struct napi_struct *napi = &adapter->napi[qindex]; +unsigned long lpar_rc; + +if (WARN_ON (qindex <0 || qindex >= adapter->num_rx_queues)) +return ; + +if (napi_schedule_prep (napi)) { + lpar_rc =ibmveth_disable_irq (adapter, qindex); +WARN_ON (lpar_rc !=H_SUCCESS ); +__napi_schedule (napi); + } +} +/* setup the initial settings for a buffer pool */ static void ibmveth_init_buffer_pool(struct ibmveth_buff_pool *pool, u32 pool_index, u32 pool_size,
@@ -947,8 +1146,6 @@ static int ibmveth_open(struct net_device *netdev)netdev_dbg (netdev,"open starting\n" ); -napi_enable (&adapter->napi[0 ]); -for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) rxq_entries += adapter->rx_buff_pool[0 ][i].size;
@@ -972,7 +1169,8 @@ static int ibmveth_open(struct net_device *netdev) adapter->rx_queue[0 ].queue_len; rxq_desc.fields.address = adapter->rx_queue[0 ].queue_dma; -h_vio_signal (adapter->vdev->unit_address,VIO_IRQ_DISABLE ); + adapter->queue_irq[0 ] = netdev->irq; +ibmveth_disable_irq (adapter,0 ); lpar_rc =ibmveth_register_logical_lan (adapter, rxq_desc, mac_address);
@@ -993,21 +1191,16 @@ static int ibmveth_open(struct net_device *netdev)if (rc) goto out_free_tx_ltb; -netdev_dbg (netdev,"registering irq 0x%x\n" , netdev->irq); - rc =request_irq (netdev->irq, ibmveth_interrupt,0 , netdev->name, - netdev); -if (rc !=0 ) { -netdev_err (netdev,"unable to request irq 0x%x, rc %d\n" , - netdev->irq, rc); + rc =ibmveth_setup_rx_interrupts (adapter); +if (rc) {do { lpar_rc =h_free_logical_lan (adapter->vdev->unit_address); }while (H_IS_LONG_BUSY (lpar_rc) || (lpar_rc ==H_BUSY )); - goto out_free_buffer_pools; }netdev_dbg (netdev,"initial replenish cycle\n" ); -ibmveth_interrupt (netdev->irq, netdev); +ibmveth_schedule_rx_queue (adapter,0 );netif_tx_start_all_queues (netdev);
@@ -1024,7 +1217,6 @@ static int ibmveth_open(struct net_device *netdev) out_free_filter_list:ibmveth_free_filter_list (adapter); out: -napi_disable (&adapter->napi[0 ]);return rc; }
@@ -1036,11 +1228,10 @@ static int ibmveth_close(struct net_device *netdev)netdev_dbg (netdev,"close starting\n" ); -napi_disable (&adapter->napi[0 ]); -netif_tx_stop_all_queues (netdev); -h_vio_signal (adapter->vdev->unit_address,VIO_IRQ_DISABLE ); +/* PHYP mask + napi_disable + free_irq live in cleanup_rx_interrupts */ +ibmveth_cleanup_rx_interrupts (adapter); do { lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
@@ -1051,8 +1242,6 @@ static int ibmveth_close(struct net_device *netdev) "continuing with close\n", lpar_rc); } -free_irq (netdev->irq, netdev); -ibmveth_update_rx_no_buffer (adapter);ibmveth_free_buffer_pools (adapter);
@@ -1798,15 +1987,14 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)/* We think we are done - reenable interrupts, * then check once more to make sure we are done. */ - lpar_rc =h_vio_signal (adapter->vdev->unit_address,VIO_IRQ_ENABLE ); + lpar_rc =ibmveth_enable_irq (adapter,0 );if (WARN_ON (lpar_rc !=H_SUCCESS )) {schedule_work (&adapter->work); goto out; }if (ibmveth_rxq_pending_buffer (adapter) &&napi_schedule (napi)) { - lpar_rc =h_vio_signal (adapter->vdev->unit_address, -VIO_IRQ_DISABLE ); + lpar_rc =ibmveth_disable_irq (adapter,0 ); goto restart_poll; }
@@ -1816,16 +2004,16 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)static irqreturn_t ibmveth_interrupt (int irq,void *dev_instance) { -struct net_device *netdev = dev_instance; +struct napi_struct *napi = dev_instance; +struct net_device *netdev = napi->dev;struct ibmveth_adapter *adapter =netdev_priv (netdev); -unsigned long lpar_rc; +int qindex; -if (napi_schedule_prep (&adapter->napi[0 ])) { - lpar_rc =h_vio_signal (adapter->vdev->unit_address, -VIO_IRQ_DISABLE ); -WARN_ON (lpar_rc !=H_SUCCESS ); -__napi_schedule (&adapter->napi[0 ]); - } + qindex = napi - adapter->napi; +if (WARN_ON (qindex <0 || qindex >= adapter->num_rx_queues)) +return IRQ_NONE ; + +ibmveth_schedule_rx_queue (adapter, qindex);return IRQ_HANDLED ; }
@@ -1930,8 +2118,10 @@ static int ibmveth_change_mtu(struct net_device *dev, int new_mtu)#ifdef CONFIG_NET_POLL_CONTROLLER static void ibmveth_poll_controller (struct net_device *dev) { -ibmveth_replenish_task (netdev_priv (dev)); -ibmveth_interrupt (dev->irq, dev); +struct ibmveth_adapter *adapter =netdev_priv (dev); + +ibmveth_replenish_task (adapter); +ibmveth_schedule_rx_queue (adapter,0 ); }#endif
@@ -2343,8 +2533,8 @@ static ssize_t veth_pool_store(struct kobject *kobj, struct attribute *attr, }rtnl_unlock (); -/* kick the interrupt handler to allocate/deallocate pools */ -ibmveth_interrupt (netdev->irq, netdev); +/* kick RX processing to allocate/deallocate pools */ +ibmveth_schedule_rx_queue (adapter,0 );return count; unlock_err:
@@ -2384,7 +2574,9 @@ static struct kobj_type ktype_veth_pool = {static int ibmveth_resume (struct device *dev) {struct net_device *netdev =dev_get_drvdata (dev); -ibmveth_interrupt (netdev->irq, netdev); +struct ibmveth_adapter *adapter =netdev_priv (netdev); + +ibmveth_schedule_rx_queue (adapter,0 );return 0 ; }
Signature · 1 line
2.50.1 (Apple Git-155)
Jakub Kicinski
maintainerRe: [PATCH net-next v4 05/14] ibmveth: Refactor RX interrupt control for MQ RX queues
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
ibmveth: Refactor RX interrupt control for MQ RX queues
This patch introduces queue-indexed RX interrupt helpers (toggle/enable/ disable, setup/cleanup, schedule) and wires them into open(), close(), poll(), the IRQ handler and the process-context RX kick paths. request_irq() now passes &adapter->napi[i] as the cookie so the handler can derive the queue index, and napi_enable()/napi_disable() move into the new setup and cleanup helpers. Runtime behaviour stays single-queue.
A few questions below.
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0
[ ... ]
Quoted 47 lines
drivers/net/ethernet/ibm/ibmveth.c +43
-0@@ -329,6 +331,203 @@ ibmveth_cleanup_rx_resources(struct ibmveth_adapter *adapter) } } +/** +* ibmveth_toggle_irq - Common helper to enable/disable queue interrupts +* @adapter: ibmveth adapter structure +* @queue_index: Index of the queue (0 for primary, 1+ for subordinate) +* @enable: true to enable, false to disable +* +* For queue 0 (primary), uses h_vio_signal() as it's registered via +* h_register_logical_lan(). For subordinate queues (1+), uses H_VIOCTL +* with H_ENABLE/DISABLE_VIO_INTERRUPT for per-queue interrupt control. +* +* Return: 0 on success, error code otherwise +*/ +static int +ibmveth_toggle_irq (struct ibmveth_adapter *adapter,int queue_index, +bool enable) +{ +unsigned long rc; +unsigned long irq = adapter->queue_irq[queue_index]; +const char *action = enable ?"enable" :"disable" ; + +if (queue_index ==0 ) { +/* Primary queue: use h_vio_signal() */ + rc =h_vio_signal (adapter->vdev->unit_address, + enable ?VIO_IRQ_ENABLE :VIO_IRQ_DISABLE ); + }else { +/* Subordinate queues: use H_VIOCTL with hardware IRQ */ +struct irq_data *irq_data =irq_get_irq_data (irq); +irq_hw_number_t hwirq; +u64 vioctl_cmd = enable ?H_ENABLE_VIO_INTERRUPT : +H_DISABLE_VIO_INTERRUPT ; + +if (!irq_data) { +netdev_err (adapter->netdev, +"Failed to get IRQ data for queue %d (virq=%lu)\n" , + queue_index, irq); +return -EINVAL ; + } + + hwirq =irqd_to_hwirq (irq_data); + rc =plpar_hcall_norets (H_VIOCTL , + adapter->vdev->unit_address, + vioctl_cmd, + hwirq,0 ,0 );
drivers/net/ethernet/ibm/ibmveth.c ibmveth_cleanup_rx_resources(struct ibmveth_adapter *adapter) · line 386
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+ +if (rc ==H_PARAMETER ) { +/* H_PARAMETER is non-fatal when IRQ is already in +* the requested state. +*/ +netdev_warn_once (adapter->netdev, +"H_VIOCTL %s IRQ returned H_PARAMETER for queue %d (hwirq=%lu)\n" , + action, queue_index, hwirq); +return 0 ; + }
[Severity: Medium]
Is folding H_PARAMETER into success correct for both directions here?
hvcall.h documents H_PARAMETER as "Parameter invalid, out-of-range or conflicting", and ibmveth_toggle_irq() cannot distinguish "already in the requested state" from a bad hwirq, wrong unit address, or a queue that is no longer registered.
In the enable direction the swallowed status bypasses the recovery in ibmveth_poll():
lpar_rc = ibmveth_enable_irq(adapter, 0);
if (WARN_ON(lpar_rc != H_SUCCESS)) {
schedule_work(&adapter->work);
goto out;
}so NAPI can complete with the queue still masked at the hypervisor and nothing left to reschedule it, i.e. that queue stops receiving, with a single warn_once as the only trace.
In the disable direction, ibmveth_schedule_rx_queue() goes on to __napi_schedule() and ibmveth_cleanup_rx_interrupts() goes on to synchronize_irq()/free_irq() while believing delivery is masked.
For comparison, ibmvnic treats any non-zero rc from the same hypercalls as an error in both directions, in disable_scrq_irq() and enable_scrq_irq().
The branch is dead in this commit (num_rx_queues is 1), but it becomes live once the MQ patches land, where setup, resize and poll all depend on this return value. Would restricting the folding to the disable direction, plus a rate-limited per-queue message instead of netdev_warn_once(), be closer to the intent?
drivers/net/ethernet/ibm/ibmveth.c ibmveth_cleanup_rx_resources(struct ibmveth_adapter *adapter) · line 394
Quoted 8 lines
drivers/net/ethernet/ibm/ibmveth.c +8
-0+ } + +if (rc) +netdev_err (adapter->netdev, +"Failed to %s IRQ for queue %d, rc=%ld\n" , + action, queue_index, rc); +return rc; +}
[Severity: Medium]
Does this mix two error domains in one int return?
rc holds an unsigned long hypervisor status from h_vio_signal() or plpar_hcall_norets() (H_BUSY 1, H_HARDWARE -1, H_FUNCTION -2, H_PARAMETER -4), truncated to int on return, while the irq_get_irq_data() failure path returns -EINVAL.
The callers then disagree about what they got. ibmveth_schedule_rx_queue() and ibmveth_poll() do:
unsigned long lpar_rc; lpar_rc = ibmveth_disable_irq(adapter, qindex); WARN_ON(lpar_rc != H_SUCCESS);
while ibmveth_setup_rx_interrupts()/ibmveth_open() treat the same value as an errno. One printk formats it with %ld, the other site logs it with %d.
By the end of the series this reaches userspace: the scale-up loop in ibmveth_resize_rx_queues_incremental() does
rc = ibmveth_enable_irq(adapter, i);
if (rc) { ... goto cleanup_new_queues; }and cleanup_new_queues ends with return rc, which flows through ibmveth_resize_rx_channels() -> ibmveth_set_channels() to ethtool. H_PARAMETER (-4) then surfaces as -EINTR, and H_BUSY (1) or H_CLOSED (2) as a positive return that ethtool reads as success even though the queue count was rolled back.
Since this commit establishes the helper API for the whole series, would it make sense to settle the convention now, either normalizing to errno or keeping the raw hcall status with a matching type, name, kernel-doc and format specifier?
[ ... ]
Quoted 24 lines
drivers/net/ethernet/ibm/ibmveth.c +24
-0+/** +* ibmveth_setup_rx_interrupts - Register IRQs and enable NAPI +* @adapter: ibmveth adapter structure +* +* Registers interrupt handlers for all RX queues and enables NAPI polling. +* On error, cleans up any successfully registered IRQs before returning. +* +* Return: 0 on success, negative error code on failure +*/ +static int +ibmveth_setup_rx_interrupts (struct ibmveth_adapter *adapter) +{ +struct net_device *netdev = adapter->netdev; +int i, rc; + +for (i =0 ; i < adapter->num_rx_queues; i++) { +if (!adapter->queue_irq[i]) { +netdev_err (netdev,"queue %d has invalid IRQ (0)\n" , i); + rc = -EINVAL ; + goto err_free_irqs; + } + + rc =request_irq (adapter->queue_irq[i], ibmveth_interrupt, +0 , netdev->name, &adapter->napi[i]);
drivers/net/ethernet/ibm/ibmveth.c ibmveth_cleanup_rx_resources(struct ibmveth_adapter *adapter) · line 455
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+if (rc) { +netdev_err (netdev, +"request_irq() failed for irq 0x%x queue %d: %d\n" , + adapter->queue_irq[i], i, rc); + goto err_free_irqs; + } + } + +for (i =0 ; i < adapter->num_rx_queues; i++) +napi_enable (&adapter->napi[i]);
[Severity: Medium]
Can an interrupt arrive in the window between these two loops?
request_irq() unmasks the virq via irq_startup(), and the LAN is already registered with the hypervisor by the time ibmveth_setup_rx_interrupts() runs, yet NAPI is only enabled in the second loop. Previously napi_enable(&adapter->napi[0]) was the first statement of ibmveth_open(), so this window did not exist.
If an interrupt (including one latched at the controller while the line was shut down by the previous close's free_irq()) is delivered inside the window, ibmveth_interrupt() -> ibmveth_schedule_rx_queue() finds napi_schedule_prep() failing:
if (napi_schedule_prep(napi)) {
lpar_rc = ibmveth_disable_irq(adapter, qindex);
WARN_ON(lpar_rc != H_SUCCESS);
__napi_schedule(napi);
}so nothing is scheduled and nothing is masked, while the handler still returns IRQ_HANDLED. For a level-asserted hypervisor source that repeats until napi_enable() runs.
This is also the inverse of the invariant the later scale-up path in this series states, "publish -> replenish -> napi_enable -> enable_irq ... so ibmveth_interrupt() cannot run on an unpublished, empty, or NAPI-disabled queue". Would enabling a queue's NAPI before installing its handler restore that ordering?
Quoted 8 lines
drivers/net/ethernet/ibm/ibmveth.c +8
-0+ +return 0 ; + +err_free_irqs: +while (--i >=0 ) +free_irq (adapter->queue_irq[i], &adapter->napi[i]); +return rc; +}
drivers/net/ethernet/ibm/ibmveth.c ibmveth_cleanup_rx_resources(struct ibmveth_adapter *adapter) · line 473
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+ +/** +* ibmveth_cleanup_rx_interrupts - Mask PHYP, disable NAPI, free IRQs +* @adapter: ibmveth adapter structure +* +* Tears down RX interrupt delivery for all queues. Mask PHYP before +* napi_disable so ibmveth_interrupt cannot return IRQ_HANDLED without +* masking (same storm window as scale-down). Safe for close and for +* open failure after setup_rx_interrupts() already unmasked PHYP. +*/
[Severity: Low]
Do these comments describe what the commit actually does?
"Safe for close and for open failure after setup_rx_interrupts() already unmasked PHYP" - ibmveth_setup_rx_interrupts() only does request_irq() and napi_enable(), and the changelog agrees ("setup does not yet unmask PHYP"). No ibmveth_open() failure path calls this helper either; setup rolls back through its own err_free_irqs label and open goes to out_free_buffer_pools/out_free_tx_ltb/out.
"Mask PHYP before napi_disable so ibmveth_interrupt cannot return IRQ_HANDLED without masking" - see the question on the mask/napi_disable ordering below; an in-flight ibmveth_poll() can undo the mask.
"Dispose IRQ mappings for subordinate queues (1-15)" sits above a loop that starts at 1, but ibmveth.h defines IBMVETH_MAX_RX_QUEUES as 1U and num_rx_queues is set to IBMVETH_DEFAULT_RX_QUEUES, so the loop cannot iterate in this commit.
And ibmveth_toggle_irq()'s "Return: 0 on success, error code otherwise" conflicts with returning raw H_* codes that callers compare against H_SUCCESS.
The first and third become accurate later in the series; the second and fourth remain as they are. Could the comments be trimmed to what this commit provides?
drivers/net/ethernet/ibm/ibmveth.c ibmveth_cleanup_rx_resources(struct ibmveth_adapter *adapter) · line 484
Quoted 11 lines
drivers/net/ethernet/ibm/ibmveth.c +11
-0+static void +ibmveth_cleanup_rx_interrupts (struct ibmveth_adapter *adapter) +{ +int i; + +for (i =0 ; i < adapter->num_rx_queues; i++) { +if (adapter->queue_irq[i]) { +ibmveth_disable_irq (adapter, i); +synchronize_irq (adapter->queue_irq[i]); + } + }
[Severity: Medium]
Can a running poll undo this mask before free_irq()?
synchronize_irq() waits for the hardirq handler only, not for an ibmveth_poll() already running in softirq context, and that poll unconditionally re-arms delivery on its completion path:
lpar_rc = ibmveth_enable_irq(adapter, 0);
Sequence with RX traffic flowing during "ip link set ethN down":
CPU0 ibmveth_close() -> ibmveth_cleanup_rx_interrupts()
ibmveth_disable_irq(adapter, 0); /* PHYP masked */
synchronize_irq(queue_irq[0]); /* no hardirq running */
napi_disable(&adapter->napi[0]); /* blocks on the poll */
CPU1 softirq ibmveth_poll()
lpar_rc = ibmveth_enable_irq(adapter, 0); /* PHYP unmasked */ napi_complete_done(...)
CPU0 then resumes and calls free_irq() with the source unmasked, and nothing re-checks or re-masks after napi_disable(). If an interrupt lands in that window, ibmveth_interrupt() -> ibmveth_schedule_rx_queue() takes the napi_schedule_prep() failure path, so no mask is applied while the handler still returns IRQ_HANDLED - the storm window the new kernel-doc says this ordering avoids. The window is bounded by the following free_irq(), but the teardown no longer reliably ends with the source masked.
Should ibmveth_poll() avoid re-enabling while teardown is in progress, or should cleanup re-mask after napi_disable()?
drivers/net/ethernet/ibm/ibmveth.c ibmveth_cleanup_rx_resources(struct ibmveth_adapter *adapter) · line 492
Quoted 8 lines
drivers/net/ethernet/ibm/ibmveth.c +8
-0+ +for (i =0 ; i < adapter->num_rx_queues; i++) +napi_disable (&adapter->napi[i]); + +for (i =0 ; i < adapter->num_rx_queues; i++) { +if (adapter->queue_irq[i]) +free_irq (adapter->queue_irq[i], &adapter->napi[i]); + }
[Severity: High]
This isn't a bug introduced by this patch, but the refactor makes the unbalanced teardown harder to see and hands the same invariant to the rest of the series, so it seems worth raising here.
napi_disable() runs unconditionally for every queue, and free_irq() is gated only on queue_irq[i]. After a failed ibmveth_open() that leaves the interface administratively up, NAPI was never enabled and no handler was installed, yet both run on the next ndo_stop.
napi_disable_locked() does:
while (val & (NAPIF_STATE_SCHED | NAPIF_STATE_NPSVC)) usleep_range(20, 200);
and sets SCHED|NPSVC when it completes, so a second napi_disable() with no intervening napi_enable() loops forever, under rtnl_lock() plus the netdev lock. free_irq() on a never-requested IRQ additionally warns with "Trying to free already-free IRQ".
Trigger in this tree: veth_pool_store() calls ibmveth_close() and then ibmveth_open() directly; if open fails (-ENOMEM from ibmveth_alloc_rx_queues(), -ENONET from ibmveth_register_logical_lan(), or request_irq() failure inside ibmveth_setup_rx_interrupts()) it restores the pool values and returns the error with the netdev still up. The same close()/open() pattern exists in ibmveth_set_csum_offload() and ibmveth_set_tso(). A later "ip link set dev ethN down" then reaches ibmveth_cleanup_rx_interrupts() and hangs.
In the baseline, open() did napi_enable() at entry and napi_disable() at out:, and close() called free_irq(netdev->irq, netdev) unconditionally, so the same hang existed. What changes here is that queue_irq[0] is published in open() before request_irq() can succeed and is deliberately never cleared, so the new "if (adapter->queue_irq[i])" guard means "we have a virq number", not "a handler is installed". Could the helper track whether napi_enable()/request_irq() actually ran for each queue?
Quoted 52 lines
drivers/net/ethernet/ibm/ibmveth.c +38
-2+ +/* Dispose IRQ mappings for subordinate queues (1-15). +* Queue 0 uses netdev->irq from device tree, not irq_create_mapping(). +*/ +for (i =1 ; i < adapter->num_rx_queues; i++) { +if (adapter->queue_irq[i]) { +irq_dispose_mapping (adapter->queue_irq[i]); + adapter->queue_irq[i] =0 ; + } + } + +/* Queue 0 uses netdev->irq; leave queue_irq[0] for next open. */ +} + +/** +* ibmveth_schedule_rx_queue - Mask PHYP IRQ and schedule NAPI for one RX queue +* @adapter: ibmveth adapter structure +* @qindex: RX queue index +* +* Shared by the IRQ handler and process-context kick paths (open, resume, +* pool sysfs, netpoll). Keep ibmveth_interrupt() as the IRQ-only wrapper. +*/ +static void ibmveth_schedule_rx_queue (struct ibmveth_adapter *adapter, +int qindex) +{ +struct napi_struct *napi = &adapter->napi[qindex]; +unsigned long lpar_rc; + +if (WARN_ON (qindex <0 || qindex >= adapter->num_rx_queues)) +return ; + +if (napi_schedule_prep (napi)) { + lpar_rc =ibmveth_disable_irq (adapter, qindex); +WARN_ON (lpar_rc !=H_SUCCESS ); +__napi_schedule (napi); + } +} +/* setup the initial settings for a buffer pool */ static void ibmveth_init_buffer_pool(struct ibmveth_buff_pool *pool, u32 pool_index, u32 pool_size,@@ -947,8 +1146,6 @@ static int ibmveth_open(struct net_device *netdev)netdev_dbg (netdev,"open starting\n" ); -napi_enable (&adapter->napi[0 ]); -for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) rxq_entries += adapter->rx_buff_pool[0 ][i].size;
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_open(struct net_device *netdev) · line 1173
Quoted 6 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-1@@ -972,7 +1169,8 @@ static int ibmveth_open(struct net_device *netdev) adapter->rx_queue[0 ].queue_len; rxq_desc.fields.address = adapter->rx_queue[0 ].queue_dma; -h_vio_signal (adapter->vdev->unit_address,VIO_IRQ_DISABLE ); + adapter->queue_irq[0 ] = netdev->irq; +ibmveth_disable_irq (adapter,0 );
Related to the teardown question above: queue_irq[0] is published here, before h_register_logical_lan(), the buffer pool allocation and request_irq(), and it is never cleared on any failure path.
Quoted 51 lines
drivers/net/ethernet/ibm/ibmveth.c +5
-12lpar_rc =ibmveth_register_logical_lan (adapter, rxq_desc, mac_address);@@ -993,21 +1191,16 @@ static int ibmveth_open(struct net_device *netdev)if (rc) goto out_free_tx_ltb; -netdev_dbg (netdev,"registering irq 0x%x\n" , netdev->irq); - rc =request_irq (netdev->irq, ibmveth_interrupt,0 , netdev->name, - netdev); -if (rc !=0 ) { -netdev_err (netdev,"unable to request irq 0x%x, rc %d\n" , - netdev->irq, rc); + rc =ibmveth_setup_rx_interrupts (adapter); +if (rc) {do { lpar_rc =h_free_logical_lan (adapter->vdev->unit_address); }while (H_IS_LONG_BUSY (lpar_rc) || (lpar_rc ==H_BUSY )); - goto out_free_buffer_pools; }netdev_dbg (netdev,"initial replenish cycle\n" ); -ibmveth_interrupt (netdev->irq, netdev); +ibmveth_schedule_rx_queue (adapter,0 );netif_tx_start_all_queues (netdev);@@ -1024,7 +1217,6 @@ static int ibmveth_open(struct net_device *netdev) out_free_filter_list:ibmveth_free_filter_list (adapter); out: -napi_disable (&adapter->napi[0 ]);return rc; }@@ -1036,11 +1228,10 @@ static int ibmveth_close(struct net_device *netdev)netdev_dbg (netdev,"close starting\n" ); -napi_disable (&adapter->napi[0 ]); -netif_tx_stop_all_queues (netdev); -h_vio_signal (adapter->vdev->unit_address,VIO_IRQ_DISABLE ); +/* PHYP mask + napi_disable + free_irq live in cleanup_rx_interrupts */ +ibmveth_cleanup_rx_interrupts (adapter); do { lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_close(struct net_device *netdev) · line 1055
Quoted 5 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-2@@ -1051,8 +1242,6 @@ static int ibmveth_close(struct net_device *netdev) "continuing with close\n", lpar_rc); } -free_irq (netdev->irq, netdev); -
Also part of the teardown question: free_irq() now happens inside ibmveth_cleanup_rx_interrupts(), i.e. before h_free_logical_lan(), where the baseline order was napi_disable -> tx stop -> h_vio_signal(DISABLE) -> h_free_logical_lan -> free_irq. The changelog mentions the mask/napi_disable reordering but not this one. Was the free_irq/h_free_logical_lan inversion intended?
Quoted 11 lines
drivers/net/ethernet/ibm/ibmveth.c +1
-1ibmveth_update_rx_no_buffer (adapter);ibmveth_free_buffer_pools (adapter);@@ -1798,15 +1987,14 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)/* We think we are done - reenable interrupts, * then check once more to make sure we are done. */ - lpar_rc =h_vio_signal (adapter->vdev->unit_address,VIO_IRQ_ENABLE ); + lpar_rc =ibmveth_enable_irq (adapter,0 );if (WARN_ON (lpar_rc !=H_SUCCESS )) {schedule_work (&adapter->work);
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_poll(struct napi_struct *napi, int budget) · line 2000
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +1
-2goto out; }if (ibmveth_rxq_pending_buffer (adapter) &&napi_schedule (napi)) { - lpar_rc =h_vio_signal (adapter->vdev->unit_address, -VIO_IRQ_DISABLE ); + lpar_rc =ibmveth_disable_irq (adapter,0 ); goto restart_poll; }
[ ... ]
Cross-instance finding from sashiko-gemini (e3f4ca1fbf9d3f3114fde9e903e88e76f2f763bdede83ba454f20f9e157db105):
[Severity: Medium]
Incomplete Refactoring / Future Out-of-Bounds Access in ibmveth_poll
Re: [PATCH net-next v4 05/14] ibmveth: Refactor RX interrupt control for MQ RX queues
Quoted 5 lines · Jakub Kicinski
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
ibmveth: Refactor RX interrupt control for MQ RX queues
Quoted 8 lines
This patch introduces queue-indexed RX interrupt helpers (toggle/enable/ disable, setup/cleanup, schedule) and wires them into
open(),close(),poll(), the IRQ handler and the process-context RX kick paths.request_irq()now passes &adapter->napi[i] as the cookie so the handler can derive the queue index, and napi_enable()/napi_disable() move into the new setup and cleanup helpers. Runtime behaviour stays single-queue.A few questions below.
Hi Jakub,
Thanks for the review.
Quoted 88 lines
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0[ ... ]
Quoted 47 lines
drivers/net/ethernet/ibm/ibmveth.c +43
-0@@ -329,6 +331,203 @@ ibmveth_cleanup_rx_resources(struct ibmveth_adapter *adapter) } } +/** +* ibmveth_toggle_irq - Common helper to enable/disable queue interrupts +* @adapter: ibmveth adapter structure +* @queue_index: Index of the queue (0 for primary, 1+ for subordinate) +* @enable: true to enable, false to disable +* +* For queue 0 (primary), uses h_vio_signal() as it's registered via +* h_register_logical_lan(). For subordinate queues (1+), uses H_VIOCTL +* with H_ENABLE/DISABLE_VIO_INTERRUPT for per-queue interrupt control. +* +* Return: 0 on success, error code otherwise +*/ +static int +ibmveth_toggle_irq (struct ibmveth_adapter *adapter,int queue_index, +bool enable) +{ +unsigned long rc; +unsigned long irq = adapter->queue_irq[queue_index]; +const char *action = enable ?"enable" :"disable" ; + +if (queue_index ==0 ) { +/* Primary queue: use h_vio_signal() */ + rc =h_vio_signal (adapter->vdev->unit_address, + enable ?VIO_IRQ_ENABLE :VIO_IRQ_DISABLE ); + }else { +/* Subordinate queues: use H_VIOCTL with hardware IRQ */ +struct irq_data *irq_data =irq_get_irq_data (irq); +irq_hw_number_t hwirq; +u64 vioctl_cmd = enable ?H_ENABLE_VIO_INTERRUPT : +H_DISABLE_VIO_INTERRUPT ; + +if (!irq_data) { +netdev_err (adapter->netdev, +"Failed to get IRQ data for queue %d (virq=%lu)\n" , + queue_index, irq); +return -EINVAL ; + } + + hwirq =irqd_to_hwirq (irq_data); + rc =plpar_hcall_norets (H_VIOCTL , + adapter->vdev->unit_address, + vioctl_cmd, + hwirq,0 ,0 );drivers/net/ethernet/ibm/ibmveth.c ibmveth_cleanup_rx_resources(struct ibmveth_adapter *adapter) · line 386
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+ +if (rc ==H_PARAMETER ) { +/* H_PARAMETER is non-fatal when IRQ is already in +* the requested state. +*/ +netdev_warn_once (adapter->netdev, +"H_VIOCTL %s IRQ returned H_PARAMETER for queue %d (hwirq=%lu)\n" , + action, queue_index, hwirq); +return 0 ; + }[Severity: Medium]
Is foldingH_PARAMETERinto success correct for both directions here?
hvcall.hdocumentsH_PARAMETERas "Parameter invalid, out-of-range or conflicting", andibmveth_toggle_irq()cannot distinguish "already in the requested state" from a bad hwirq, wrong unit address, or a queue that is no longer registered.In the enable direction the swallowed status bypasses the recovery in
ibmveth_poll():lpar_rc = ibmveth_enable_irq(adapter, 0); if (WARN_ON(lpar_rc != H_SUCCESS)) { schedule_work(&adapter->work); goto out; }so NAPI can complete with the queue still masked at the hypervisor and nothing left to reschedule it, i.e. that queue stops receiving, with a single
warn_onceas the only trace.In the disable direction,
ibmveth_schedule_rx_queue()goes on to__napi_schedule()andibmveth_cleanup_rx_interrupts()goes on to synchronize_irq()/free_irq() while believing delivery is masked.
Quoted 8 lines
For comparison, ibmvnic treats any non-zero rc from the same hypercalls as an error in both directions, in
disable_scrq_irq()andenable_scrq_irq().The branch is dead in this commit (
num_rx_queuesis 1), but it becomes live once the MQ patches land, where setup, resize and poll all depend on this return value. Would restricting the folding to the disable direction, plus a rate-limited per-queue message instead ofnetdev_warn_once(), be closer to the intent?
Agreed. Treating H_PARAMETER as success on the enable side is wrong. For the disable side, I do not want to assume it is harmless without firmer justification, so if any special handling remains there I will make it explicit and better diagnosed rather than silently folding it away.
Quoted 38 lines
drivers/net/ethernet/ibm/ibmveth.c ibmveth_cleanup_rx_resources(struct ibmveth_adapter *adapter) · line 394
Quoted 8 lines
drivers/net/ethernet/ibm/ibmveth.c +8
-0+ } + +if (rc) +netdev_err (adapter->netdev, +"Failed to %s IRQ for queue %d, rc=%ld\n" , + action, queue_index, rc); +return rc; +}[Severity: Medium]
Does this mix two error domains in one int return?rc holds an unsigned long hypervisor status from
h_vio_signal()orplpar_hcall_norets()(H_BUSY1,H_HARDWARE-1,H_FUNCTION-2,H_PARAMETER-4), truncated to int on return, while theirq_get_irq_data()failure path returns-EINVAL.The callers then disagree about what they got.
ibmveth_schedule_rx_queue()andibmveth_poll()do:unsigned long lpar_rc; lpar_rc = ibmveth_disable_irq(adapter, qindex); WARN_ON(lpar_rc != H_SUCCESS);while ibmveth_setup_rx_interrupts()/ibmveth_open() treat the same value as an errno. One printk formats it with %ld, the other site logs it with %d.
By the end of the series this reaches userspace: the scale-up loop in
ibmveth_resize_rx_queues_incremental()doesrc = ibmveth_enable_irq(adapter, i); if (rc) { ... goto cleanup_new_queues; }and
cleanup_new_queuesends with return rc, which flows throughibmveth_resize_rx_channels()->ibmveth_set_channels()to ethtool.H_PARAMETER(-4) then surfaces as-EINTR, andH_BUSY(1) orH_CLOSED(2) as a positive return that ethtool reads as success even though the queue count was rolled back.
Quoted 6 lines
Since this commit establishes the helper API for the whole series, would it make sense to settle the convention now, either normalizing to errno or keeping the raw hcall status with a matching type, name, kernel-doc and format specifier?
[ ... ]
Agreed. These helpers should return one error domain only. In v5 I will keep the IRQ helper contract errno-based so later resize / ethtool-facing paths do not have to interpret mixed H_*/errno returns.
Quoted 58 lines
Quoted 24 lines
drivers/net/ethernet/ibm/ibmveth.c +24
-0+/** +* ibmveth_setup_rx_interrupts - Register IRQs and enable NAPI +* @adapter: ibmveth adapter structure +* +* Registers interrupt handlers for all RX queues and enables NAPI polling. +* On error, cleans up any successfully registered IRQs before returning. +* +* Return: 0 on success, negative error code on failure +*/ +static int +ibmveth_setup_rx_interrupts (struct ibmveth_adapter *adapter) +{ +struct net_device *netdev = adapter->netdev; +int i, rc; + +for (i =0 ; i < adapter->num_rx_queues; i++) { +if (!adapter->queue_irq[i]) { +netdev_err (netdev,"queue %d has invalid IRQ (0)\n" , i); + rc = -EINVAL ; + goto err_free_irqs; + } + + rc =request_irq (adapter->queue_irq[i], ibmveth_interrupt, +0 , netdev->name, &adapter->napi[i]);drivers/net/ethernet/ibm/ibmveth.c ibmveth_cleanup_rx_resources(struct ibmveth_adapter *adapter) · line 455
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+if (rc) { +netdev_err (netdev, +"request_irq() failed for irq 0x%x queue %d: %d\n" , + adapter->queue_irq[i], i, rc); + goto err_free_irqs; + } + } + +for (i =0 ; i < adapter->num_rx_queues; i++) +napi_enable (&adapter->napi[i]);[Severity: Medium]
Can an interrupt arrive in the window between these two loops?
request_irq()unmasks the virq viairq_startup(), and the LAN is already registered with the hypervisor by the timeibmveth_setup_rx_interrupts()runs, yet NAPI is only enabled in the second loop. Previouslynapi_enable(&adapter->napi[0])was the first statement ofibmveth_open(), so this window did not exist.If an interrupt (including one latched at the controller while the line was shut down by the previous close's
free_irq()) is delivered inside the window,ibmveth_interrupt()->ibmveth_schedule_rx_queue()findsnapi_schedule_prep()failing:if (napi_schedule_prep(napi)) { lpar_rc = ibmveth_disable_irq(adapter, qindex); WARN_ON(lpar_rc != H_SUCCESS); __napi_schedule(napi); }so nothing is scheduled and nothing is masked, while the handler still returns
IRQ_HANDLED. For a level-asserted hypervisor source that repeats untilnapi_enable()runs.
Quoted 5 lines
This is also the inverse of the invariant the later scale-up path in this series states, "publish -> replenish ->
napi_enable->enable_irq... soibmveth_interrupt()cannot run on an unpublished, empty, or NAPI-disabled queue". Would enabling a queue's NAPI before installing its handler restore that ordering?
I agree the ordering needs to be coherent. Open and scale-up are not identical cases, and the ordering here came from keeping the tested open path with PHYP delivery still masked until ibmveth_enable_irq(). I will clarify that invariant in the changelog and make sure the open-path comments do not overstate it.
Quoted 37 lines
Quoted 8 lines
drivers/net/ethernet/ibm/ibmveth.c +8
-0+ +return 0 ; + +err_free_irqs: +while (--i >=0 ) +free_irq (adapter->queue_irq[i], &adapter->napi[i]); +return rc; +}drivers/net/ethernet/ibm/ibmveth.c ibmveth_cleanup_rx_resources(struct ibmveth_adapter *adapter) · line 473
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+ +/** +* ibmveth_cleanup_rx_interrupts - Mask PHYP, disable NAPI, free IRQs +* @adapter: ibmveth adapter structure +* +* Tears down RX interrupt delivery for all queues. Mask PHYP before +* napi_disable so ibmveth_interrupt cannot return IRQ_HANDLED without +* masking (same storm window as scale-down). Safe for close and for +* open failure after setup_rx_interrupts() already unmasked PHYP. +*/ [Severity: Low]
Do these comments describe what the commit actually does?"Safe for close and for open failure after
setup_rx_interrupts()already unmasked PHYP" -ibmveth_setup_rx_interrupts()only doesrequest_irq()andnapi_enable(), and the changelog agrees ("setup does not yet unmask PHYP"). Noibmveth_open()failure path calls this helper either; setup rolls back through its ownerr_free_irqslabel and open goes to out_free_buffer_pools/out_free_tx_ltb/out."Mask PHYP before
napi_disablesoibmveth_interruptcannot returnIRQ_HANDLEDwithout masking" - see the question on the mask/napi_disable ordering below; an in-flightibmveth_poll()can undo the mask."Dispose IRQ mappings for subordinate queues (1-15)" sits above a loop that starts at 1, but
ibmveth.hdefinesIBMVETH_MAX_RX_QUEUESas 1U andnum_rx_queuesis set toIBMVETH_DEFAULT_RX_QUEUES, so the loop cannot iterate in this commit.
Quoted 7 lines
And ibmveth_toggle_irq()'s "Return: 0 on success, error code otherwise" conflicts with returning raw H_* codes that callers compare against
H_SUCCESS.The first and third become accurate later in the series; the second and fourth remain as they are. Could the comments be trimmed to what this commit provides?
Agreed. I will trim the kerneldoc/comments so they match the actual behavior in this patch rather than describing stronger later-series guarantees.
Quoted 39 lines
drivers/net/ethernet/ibm/ibmveth.c ibmveth_cleanup_rx_resources(struct ibmveth_adapter *adapter) · line 484
Quoted 11 lines
drivers/net/ethernet/ibm/ibmveth.c +11
-0+static void +ibmveth_cleanup_rx_interrupts (struct ibmveth_adapter *adapter) +{ +int i; + +for (i =0 ; i < adapter->num_rx_queues; i++) { +if (adapter->queue_irq[i]) { +ibmveth_disable_irq (adapter, i); +synchronize_irq (adapter->queue_irq[i]); + } + }[Severity: Medium]
Can a running poll undo this mask beforefree_irq()?
synchronize_irq()waits for the hardirq handler only, not for anibmveth_poll()already running in softirq context, and that poll unconditionally re-arms delivery on its completion path:lpar_rc = ibmveth_enable_irq(adapter, 0);Sequence with RX traffic flowing during "ip link set ethN down":
CPU0
ibmveth_close()->ibmveth_cleanup_rx_interrupts()
ibmveth_disable_irq(adapter, 0); /* PHYP masked */
synchronize_irq(queue_irq[0]); /* no hardirq running */
napi_disable(&adapter->napi[0]); /* blocks on the poll */CPU1 softirq
ibmveth_poll()lpar_rc = ibmveth_enable_irq(adapter, 0); /* PHYP unmasked */ napi_complete_done(...)CPU0 then resumes and calls
free_irq()with the source unmasked, and nothing re-checks or re-masks afternapi_disable(). If an interrupt lands in that window,ibmveth_interrupt()->ibmveth_schedule_rx_queue()takes thenapi_schedule_prep()failure path, so no mask is applied while the handler still returnsIRQ_HANDLED- the storm window the new kernel-doc says this ordering avoids. The window is bounded by the followingfree_irq(), but the teardown no longer reliably ends with the source masked.
Quoted 2 lines
Should
ibmveth_poll()avoid re-enabling while teardown is in progress, or should cleanup re-mask afternapi_disable()?
Agreed. That teardown race is real. Later shutdown/quiesce fixes stop poll from re-enabling delivery while teardown is in progress, and I will fold that behavior back into the owning path rather than leaving it as a tail fixup.
Quoted 37 lines
drivers/net/ethernet/ibm/ibmveth.c ibmveth_cleanup_rx_resources(struct ibmveth_adapter *adapter) · line 492
Quoted 8 lines
drivers/net/ethernet/ibm/ibmveth.c +8
-0+ +for (i =0 ; i < adapter->num_rx_queues; i++) +napi_disable (&adapter->napi[i]); + +for (i =0 ; i < adapter->num_rx_queues; i++) { +if (adapter->queue_irq[i]) +free_irq (adapter->queue_irq[i], &adapter->napi[i]); + }[Severity: High]
This isn't a bug introduced by this patch, but the refactor makes the unbalanced teardown harder to see and hands the same invariant to the rest of the series, so it seems worth raising here.
napi_disable()runs unconditionally for every queue, andfree_irq()is gated only on queue_irq[i]. After a failedibmveth_open()that leaves the interface administratively up, NAPI was never enabled and no handler was installed, yet both run on the nextndo_stop.
napi_disable_locked()does:while (val & (NAPIF_STATE_SCHED | NAPIF_STATE_NPSVC)) usleep_range(20, 200);and sets SCHED|NPSVC when it completes, so a second
napi_disable()with no interveningnapi_enable()loops forever, underrtnl_lock()plus the netdev lock.free_irq()on a never-requested IRQ additionally warns with "Trying to free already-free IRQ".Trigger in this tree:
veth_pool_store()callsibmveth_close()and thenibmveth_open()directly; if open fails (-ENOMEMfromibmveth_alloc_rx_queues(),-ENONETfromibmveth_register_logical_lan(), orrequest_irq()failure insideibmveth_setup_rx_interrupts()) it restores the pool values and returns the error with the netdev still up. The same close()/open() pattern exists inibmveth_set_csum_offload()andibmveth_set_tso(). A later "ip link set dev ethN down" then reachesibmveth_cleanup_rx_interrupts()and hangs.
Quoted 7 lines
In the baseline,
open()didnapi_enable()at entry andnapi_disable()at out:, andclose()calledfree_irq(netdev->irq, netdev)unconditionally, so the same hang existed. What changes here is that queue_irq[0] is published inopen()beforerequest_irq()can succeed and is deliberately never cleared, so the new "if (adapter->queue_irq[i])" guard means "we have a virq number", not "a handler is installed". Could the helper track whether napi_enable()/request_irq() actually ran for each queue?
Agreed. This one is high priority. The close/cleanup path needs explicit state tracking so a second teardown after close()+failed-open does not walk partially initialized state as if it were live. In v5 I will keep the opened / rx_irq_setup state tracking so cleanup is idempotent and resize/open paths key off real setup state rather than bare IFF_UP.
Quoted 115 lines
Quoted 51 lines
drivers/net/ethernet/ibm/ibmveth.c +38
-2+ +/* Dispose IRQ mappings for subordinate queues (1-15). +* Queue 0 uses netdev->irq from device tree, not irq_create_mapping(). +*/ +for (i =1 ; i < adapter->num_rx_queues; i++) { +if (adapter->queue_irq[i]) { +irq_dispose_mapping (adapter->queue_irq[i]); + adapter->queue_irq[i] =0 ; + } + } + +/* Queue 0 uses netdev->irq; leave queue_irq[0] for next open. */ +} + +/** +* ibmveth_schedule_rx_queue - Mask PHYP IRQ and schedule NAPI for one RX queue +* @adapter: ibmveth adapter structure +* @qindex: RX queue index +* +* Shared by the IRQ handler and process-context kick paths (open, resume, +* pool sysfs, netpoll). Keep ibmveth_interrupt() as the IRQ-only wrapper. +*/ +static void ibmveth_schedule_rx_queue (struct ibmveth_adapter *adapter, +int qindex) +{ +struct napi_struct *napi = &adapter->napi[qindex]; +unsigned long lpar_rc; + +if (WARN_ON (qindex <0 || qindex >= adapter->num_rx_queues)) +return ; + +if (napi_schedule_prep (napi)) { + lpar_rc =ibmveth_disable_irq (adapter, qindex); +WARN_ON (lpar_rc !=H_SUCCESS ); +__napi_schedule (napi); + } +} +/* setup the initial settings for a buffer pool */ static void ibmveth_init_buffer_pool(struct ibmveth_buff_pool *pool, u32 pool_index, u32 pool_size,@@ -947,8 +1146,6 @@ static int ibmveth_open(struct net_device *netdev)netdev_dbg (netdev,"open starting\n" ); -napi_enable (&adapter->napi[0 ]); -for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) rxq_entries += adapter->rx_buff_pool[0 ][i].size;drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_open(struct net_device *netdev) · line 1173
Quoted 6 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-1@@ -972,7 +1169,8 @@ static int ibmveth_open(struct net_device *netdev) adapter->rx_queue[0 ].queue_len; rxq_desc.fields.address = adapter->rx_queue[0 ].queue_dma; -h_vio_signal (adapter->vdev->unit_address,VIO_IRQ_DISABLE ); + adapter->queue_irq[0 ] = netdev->irq; +ibmveth_disable_irq (adapter,0 );Related to the teardown question above: queue_irq[0] is published here, before
h_register_logical_lan(), the buffer pool allocation andrequest_irq(), and it is never cleared on any failure path.Quoted 54 lines
drivers/net/ethernet/ibm/ibmveth.c +5
-12lpar_rc =ibmveth_register_logical_lan (adapter, rxq_desc, mac_address);@@ -993,21 +1191,16 @@ static int ibmveth_open(struct net_device *netdev)if (rc) goto out_free_tx_ltb; -netdev_dbg (netdev,"registering irq 0x%x\n" , netdev->irq); - rc =request_irq (netdev->irq, ibmveth_interrupt,0 , netdev->name, - netdev); -if (rc !=0 ) { -netdev_err (netdev,"unable to request irq 0x%x, rc %d\n" , - netdev->irq, rc); + rc =ibmveth_setup_rx_interrupts (adapter); +if (rc) {do { lpar_rc =h_free_logical_lan (adapter->vdev->unit_address); }while (H_IS_LONG_BUSY (lpar_rc) || (lpar_rc ==H_BUSY )); - goto out_free_buffer_pools; }netdev_dbg (netdev,"initial replenish cycle\n" ); -ibmveth_interrupt (netdev->irq, netdev); +ibmveth_schedule_rx_queue (adapter,0 );netif_tx_start_all_queues (netdev);@@ -1024,7 +1217,6 @@ static int ibmveth_open(struct net_device *netdev) out_free_filter_list:ibmveth_free_filter_list (adapter); out: -napi_disable (&adapter->napi[0 ]);return rc; }@@ -1036,11 +1228,10 @@ static int ibmveth_close(struct net_device *netdev)netdev_dbg (netdev,"close starting\n" ); -napi_disable (&adapter->napi[0 ]); -netif_tx_stop_all_queues (netdev); -h_vio_signal (adapter->vdev->unit_address,VIO_IRQ_DISABLE ); +/* PHYP mask + napi_disable + free_irq live in cleanup_rx_interrupts */ +ibmveth_cleanup_rx_interrupts (adapter); do { lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);@@ -1051,8 +1242,6 @@ static int ibmveth_close(struct net_device *netdev) "continuing with close\n", lpar_rc); }
Quoted 8 lines
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_close(struct net_device *netdev) · line 1055
Quoted 2 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-2-free_irq (netdev->irq, netdev); -Also part of the teardown question:
free_irq()now happens insideibmveth_cleanup_rx_interrupts(), i.e. beforeh_free_logical_lan(), where the baseline order wasnapi_disable-> tx stop ->h_vio_signal(DISABLE)->h_free_logical_lan->free_irq. The changelog mentions the mask/napi_disable reordering but not this one. Was the free_irq/h_free_logical_lan inversion intended?
That ordering is intentional. Once PHYP delivery is masked, the Linux handler is no longer needed for h_free_logical_lan(), and freeing the IRQ first avoids leaving a handler around while the LAN handles are being torn down. The separate queue_irq[0] point is the lifecycle issue above: it can mean "have a virq number" before request_irq() succeeds, which is why cleanup needs rx_irq_setup / opened rather than queue_irq[] alone. I will call that ordering out in the changelog.
Quoted 23 lines
Quoted 21 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-3ibmveth_update_rx_no_buffer (adapter);ibmveth_free_buffer_pools (adapter);@@ -1798,15 +1987,14 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)/* We think we are done - reenable interrupts, * then check once more to make sure we are done. */ - lpar_rc =h_vio_signal (adapter->vdev->unit_address,VIO_IRQ_ENABLE ); + lpar_rc =ibmveth_enable_irq (adapter,0 );if (WARN_ON (lpar_rc !=H_SUCCESS )) {schedule_work (&adapter->work); goto out; }if (ibmveth_rxq_pending_buffer (adapter) &&napi_schedule (napi)) { - lpar_rc =h_vio_signal (adapter->vdev->unit_address, -VIO_IRQ_DISABLE ); + lpar_rc =ibmveth_disable_irq (adapter,0 ); goto restart_poll; }[ ... ]
Quoted 3 lines
Cross-instance finding from sashiko-gemini (
e3f4ca1fbf9d3f3114fde9e903e88e76f2f763bdede83ba454f20f9e157db105):
[Severity: Medium]
Incomplete Refactoring / Future Out-of-Bounds Access inibmveth_poll
Agreed that the poll-side MQ wiring needed follow-up hardening, but I do not want to overclaim that as fully owned by patch 5. The queue-index / completion cleanup belongs with the later poll-path work, so I would treat that as a follow-on fix rather than claim it is resolved here.
Thanks again for the detailed review,
Mingming
PATCH v4 6/14 ibmveth: Refactor TX resource allocation in open/close paths
In reply to Mingming CaoSame story as the RX refactor: pull TX LTB alloc/free out of open/close into helpers and wire them in this patch.
ibmveth_alloc_tx_resources()
ibmveth_free_tx_resources()
They wrap the existing per-queue allocate_tx_ltb() / free_tx_ltb() primitives. alloc_tx_resources() allocates every TX queue and unwinds partial failure itself; free_tx_resources() walks real_num_tx_queues. TX was already multi-queue capable via ethtool -L, so there is no MQ RX behaviour change here.
Also tighten TX LTB lifetime: free_tx_ltb() is safe on an unallocated slot and clears tx_ltb_dma[] after unmap; allocate_tx_ltb() clears tx_ltb_dma[] after DMA-map failure.
Move TX LTB allocation to the end of open(), after LAN registration, RX pools, RX interrupt setup, and the initial replenish kick. A late alloc_tx_resources() failure jumps to out_cleanup_rx_interrupts and must not call free_tx_resources() again: alloc already freed any partial TX LTBs. That also retires a mid-open TX leak where the old index-based while (--i) free could skip successfully allocated LTBs.
close() uses free_tx_resources() and cleanup_rx_interrupts().
---
Changes in v4:
- Introduce the TX resource helpers in the same patch that wires their first open/close callers.
- Do not free TX LTBs again after a failed alloc_tx_resources(); harden free_tx_ltb() against unset slots.
- Move TX allocation after RX IRQ setup / replenish kick so open() failure unwind no longer depends on a shared loop index (also fixes a mid-open TX LTB leak).
drivers/net/ethernet/ibm/ibmveth.c | 90 ++++++++++++++++++++++-------- 1 file changed, 66 insertions(+), 24 deletions(-)
drivers/net/ethernet/ibm/ibmveth.c +66 -24
@@ -1077,8 +1077,14 @@ static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,static void ibmveth_free_tx_ltb (struct ibmveth_adapter *adapter,int idx) { -dma_unmap_single (&adapter->vdev->dev, adapter->tx_ltb_dma[idx], - adapter->tx_ltb_size,DMA_TO_DEVICE ); +if (!adapter->tx_ltb_ptr[idx]) +return ; + +if (adapter->tx_ltb_dma[idx]) { +dma_unmap_single (&adapter->vdev->dev, adapter->tx_ltb_dma[idx], + adapter->tx_ltb_size,DMA_TO_DEVICE ); + adapter->tx_ltb_dma[idx] =0 ; + }kfree (adapter->tx_ltb_ptr[idx]); adapter->tx_ltb_ptr[idx] =NULL ; }
@@ -1101,12 +1107,54 @@ static int ibmveth_allocate_tx_ltb(struct ibmveth_adapter *adapter, int idx) "unable to DMA map tx long term buffer\n");kfree (adapter->tx_ltb_ptr[idx]); adapter->tx_ltb_ptr[idx] =NULL ; + adapter->tx_ltb_dma[idx] =0 ;return -ENOMEM ; }return 0 ; } +/** +* ibmveth_alloc_tx_resources - Allocate TX resources for all queues +* @adapter: ibmveth adapter structure +* +* Allocates TX Long Term Buffers (LTBs) for all TX queues. +* +* Return: 0 on success, -ENOMEM on failure +*/ +static int ibmveth_alloc_tx_resources (struct ibmveth_adapter *adapter) +{ +struct net_device *netdev = adapter->netdev; +int i; + +for (i =0 ; i < netdev->real_num_tx_queues; i++) { +if (ibmveth_allocate_tx_ltb (adapter, i)) + goto err_free_ltbs; + } + +return 0 ; + +err_free_ltbs: +while (--i >=0 ) +ibmveth_free_tx_ltb (adapter, i); +return -ENOMEM ; +} + +/** +* ibmveth_free_tx_resources - Free TX resources for all queues +* @adapter: ibmveth adapter structure +* +* Frees TX Long Term Buffers (LTBs) for all TX queues. +*/ +static void ibmveth_free_tx_resources (struct ibmveth_adapter *adapter) +{ +struct net_device *netdev = adapter->netdev; +int i; + +for (i =0 ; i < netdev->real_num_tx_queues; i++) +ibmveth_free_tx_ltb (adapter, i); +} +static int ibmveth_register_logical_lan (struct ibmveth_adapter *adapter,union ibmveth_buf_desc rxq_desc,u64 mac_address) {
@@ -1157,12 +1205,6 @@ static int ibmveth_open(struct net_device *netdev)if (rc) goto out_free_filter_list; - rc = -ENOMEM ; -for (i =0 ; i < netdev->real_num_tx_queues; i++) { -if (ibmveth_allocate_tx_ltb (adapter, i)) - goto out_free_tx_ltb; - } - mac_address =ether_addr_to_u64 (netdev->dev_addr); rxq_desc.fields.flags_len = IBMVETH_BUF_VALID |
@@ -1184,35 +1226,38 @@ static int ibmveth_open(struct net_device *netdev) rxq_desc.desc, mac_address); rc = -ENONET ; - goto out_free_tx_ltb; + goto out_free_queue_mem; } rc =ibmveth_alloc_buffer_pools (adapter);if (rc) - goto out_free_tx_ltb; + goto out_unregister_lan; rc =ibmveth_setup_rx_interrupts (adapter); -if (rc) { -do { - lpar_rc =h_free_logical_lan (adapter->vdev->unit_address); - }while (H_IS_LONG_BUSY (lpar_rc) || (lpar_rc ==H_BUSY )); - goto out_free_buffer_pools; - } +if (rc) + goto out_unregister_lan;netdev_dbg (netdev,"initial replenish cycle\n" );ibmveth_schedule_rx_queue (adapter,0 ); + rc =ibmveth_alloc_tx_resources (adapter); +if (rc) + goto out_cleanup_rx_interrupts; +netif_tx_start_all_queues (netdev);netdev_dbg (netdev,"open complete\n" );return 0 ; -out_free_buffer_pools: +out_cleanup_rx_interrupts: +ibmveth_cleanup_rx_interrupts (adapter); +out_unregister_lan: +do { + lpar_rc =h_free_logical_lan (adapter->vdev->unit_address); + }while (H_IS_LONG_BUSY (lpar_rc) || (lpar_rc ==H_BUSY ));ibmveth_free_buffer_pools (adapter); -out_free_tx_ltb: -while (--i >=0 ) -ibmveth_free_tx_ltb (adapter, i); +out_free_queue_mem:ibmveth_cleanup_rx_resources (adapter); out_free_filter_list:ibmveth_free_filter_list (adapter);
@@ -1224,13 +1269,13 @@ static int ibmveth_close(struct net_device *netdev) {struct ibmveth_adapter *adapter =netdev_priv (netdev);long lpar_rc; -int i;netdev_dbg (netdev,"close starting\n" );netif_tx_stop_all_queues (netdev);/* PHYP mask + napi_disable + free_irq live in cleanup_rx_interrupts */ +ibmveth_free_tx_resources (adapter);ibmveth_cleanup_rx_interrupts (adapter); do {
@@ -1248,9 +1293,6 @@ static int ibmveth_close(struct net_device *netdev)ibmveth_cleanup_rx_resources (adapter);ibmveth_free_filter_list (adapter); -for (i =0 ; i < netdev->real_num_tx_queues; i++) -ibmveth_free_tx_ltb (adapter, i); -netdev_dbg (netdev,"close complete\n" );return 0 ;
Signature · 1 line
2.50.1 (Apple Git-155)
Jakub Kicinski
maintainerRe: [PATCH net-next v4 06/14] ibmveth: Refactor TX resource allocation in open/close paths
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
ibmveth: Refactor TX resource allocation in open/close paths
This moves the per-queue TX long term buffer alloc/free loops out of ibmveth_open() and ibmveth_close() into ibmveth_alloc_tx_resources() and ibmveth_free_tx_resources(), relocates the TX allocation to the end of open(), and reworks the open() error unwind labels.
A few questions on the changelog and on the new ordering below.
[Severity: Medium]
The changelog says:
That also retires a mid-open TX leak where the old index-based while (--i) free could skip successfully allocated LTBs.
Is that leak real? In the parent commit the TX loop re-initializes the shared index before allocating anything:
rc = -ENOMEM;
for (i = 0; i < netdev->real_num_tx_queues; i++) {
if (ibmveth_allocate_tx_ltb(adapter, i))
goto out_free_tx_ltb;
}and nothing between that loop and the later gotos (mac_address/rxq_desc setup, ibmveth_register_logical_lan(), ibmveth_alloc_buffer_pools(), ibmveth_setup_rx_interrupts()) reassigns i. On every path that reaches out_free_tx_ltb, i is either the failing index or real_num_tx_queues, so while (--i >= 0) frees exactly what was allocated. Paths taken before the TX loop jump to out_free_filter_list, which is below the TX label.
The new helper also uses the same index-based unwind that the changelog calls broken:
err_free_ltbs:
while (--i >= 0) ibmveth_free_tx_ltb(adapter, i);
Could this paragraph be dropped or reworded to describe the actual motivation (removing the dependence on a shared loop index and matching the RX helpers)? As written it reads like a leak fix that a stable maintainer might want to backport, and there is no Fixes: tag.
Quoted 8 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0@@ -1077,8 +1077,14 @@ static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,static void ibmveth_free_tx_ltb (struct ibmveth_adapter *adapter,int idx) {
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter, · line 1087
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +8
-2-dma_unmap_single (&adapter->vdev->dev, adapter->tx_ltb_dma[idx], - adapter->tx_ltb_size,DMA_TO_DEVICE ); +if (!adapter->tx_ltb_ptr[idx]) +return ; + +if (adapter->tx_ltb_dma[idx]) { +dma_unmap_single (&adapter->vdev->dev, adapter->tx_ltb_dma[idx], + adapter->tx_ltb_size,DMA_TO_DEVICE ); + adapter->tx_ltb_dma[idx] =0 ; + }
[Severity: Low]
Is the inner test on tx_ltb_dma[idx] wanted here? It treats DMA address 0 as a "not mapped" sentinel, and the only architecturally valid check for a bad handle is dma_mapping_error() at map time. If a mapping ever legitimately returned IOVA 0, the TCE entry would leak while kfree() below still released the buffer.
On pseries this cannot happen today, since arch/powerpc/kernel/iommu.c:iommu_table_reserve_pages() reserves entry 0:
if (tbl->it_offset == 0) set_bit(0, tbl->it_map);
The test also looks redundant: ibmveth_allocate_tx_ltb() already kfree()s the buffer and clears tx_ltb_ptr[idx] on dma_mapping_error(), so tx_ltb_ptr[idx] != NULL already implies a valid mapping and the new early return covers the stated "safe on an unallocated slot" goal. Would dropping the zero test and relying on the pointer check be simpler?
Quoted 3 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0kfree (adapter->tx_ltb_ptr[idx]); adapter->tx_ltb_ptr[idx] =NULL ; }
[ ... ]
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_open(struct net_device *netdev) · line 1234
Quoted 12 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-2@@ -1184,35 +1226,38 @@ static int ibmveth_open(struct net_device *netdev) rxq_desc.desc, mac_address); rc = -ENONET ; - goto out_free_tx_ltb; + goto out_free_queue_mem; } rc =ibmveth_alloc_buffer_pools (adapter);if (rc) - goto out_free_tx_ltb; + goto out_unregister_lan;
[Severity: Low]
This changes hypervisor-visible behaviour on an RX error path, which the changelog does not mention while stating "there is no MQ RX behaviour change here".
In the parent commit an ibmveth_alloc_buffer_pools() failure went to out_free_tx_ltb and then to ibmveth_cleanup_rx_resources(), leaving the logical LAN registered with PHYP while the RX ring was freed and the buffer list unmapped. With the new label the h_free_logical_lan() loop runs first, which is the safer order.
Is this an intentional fix folded into a TX-only refactor? If so, could it be split out with its own changelog text (the next patch in the series reorders open() so this path disappears entirely, so the fix becomes hard to find later).
Quoted 19 lines
drivers/net/ethernet/ibm/ibmveth.c +6
-6rc =ibmveth_setup_rx_interrupts (adapter); -if (rc) { -do { - lpar_rc =h_free_logical_lan (adapter->vdev->unit_address); - }while (H_IS_LONG_BUSY (lpar_rc) || (lpar_rc ==H_BUSY )); - goto out_free_buffer_pools; - } +if (rc) + goto out_unregister_lan;netdev_dbg (netdev,"initial replenish cycle\n" );ibmveth_schedule_rx_queue (adapter,0 ); + rc =ibmveth_alloc_tx_resources (adapter); +if (rc) + goto out_cleanup_rx_interrupts; +netif_tx_start_all_queues (netdev);
[ ... ]
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_close(struct net_device *netdev) · line 1279
Quoted 13 lines
drivers/net/ethernet/ibm/ibmveth.c +1
-1@@ -1224,13 +1269,13 @@ static int ibmveth_close(struct net_device *netdev) {struct ibmveth_adapter *adapter =netdev_priv (netdev);long lpar_rc; -int i;netdev_dbg (netdev,"close starting\n" );netif_tx_stop_all_queues (netdev);/* PHYP mask + napi_disable + free_irq live in cleanup_rx_interrupts */ +ibmveth_free_tx_resources (adapter);ibmveth_cleanup_rx_interrupts (adapter);
[Severity: High]
Can a transmit already in flight write into the TX LTB after it has been unmapped and freed here?
netif_tx_stop_all_queues() only sets __QUEUE_STATE_DRV_XOFF. It does not take __netif_tx_lock and does not wait for an ndo_start_xmit that is already running on another CPU (netif_tx_disable() would). ibmveth_close() is also called directly, bypassing dev_close() -> dev_deactivate_many() and its synchronize_net() / some_qdisc_is_busy() wait, from:
ibmveth_set_csum_offload()
ibmveth_set_tso()
ibmveth_change_mtu()
veth_pool_store()
all of which do:
if (netif_running(dev)) {
restart = 1;
ibmveth_close(dev);
}So a transmitter that already passed netif_xmit_stopped() can be inside ibmveth_start_xmit(), which copies into the LTB with no NULL check and no lock:
memcpy(adapter->tx_ltb_ptr[queue_num], skb->data, skb_headlen(skb)); ... memcpy(adapter->tx_ltb_ptr[queue_num] + total_bytes, skb_frag_address_safe(frag), skb_frag_size(frag)); ... desc.fields.address = adapter->tx_ltb_dma[queue_num];
while ibmveth_free_tx_ltb() concurrently does:
dma_unmap_single(...); adapter->tx_ltb_dma[idx] = 0; kfree(adapter->tx_ltb_ptr[idx]); adapter->tx_ltb_ptr[idx] = NULL;
That looks like either a write of up to tx_ltb_size bytes into freed heap memory, or a NULL dereference on the cleared pointer.
A second question on the same move: the DMA unmap now happens before the h_free_logical_lan() loop below, so a racing ibmveth_send() can hand PHYP a descriptor whose address is 0 or an already unmapped TCE while the logical LAN is still registered. Previously the TX LTBs were freed at the very end of close(), after h_free_logical_lan(), free_irq()/napi_disable() and the buffer pool teardown, which incidentally left a large gap between the XOFF and the free. Was the inversion intended?
Would keeping ibmveth_free_tx_resources() after the h_free_logical_lan() loop, switching to netif_tx_disable(), and/or adding an
if (unlikely(!adapter->tx_ltb_ptr[queue_num]))
bail-out in ibmveth_start_xmit() be preferable? The last one would also cover the mirror window that the new open() ordering creates, where the LAN is registered and RX interrupts/NAPI are live while tx_ltb_ptr[] is still NULL during the sleeping GFP_KERNEL allocation in ibmveth_alloc_tx_resources():
ibmveth_schedule_rx_queue(adapter, 0); rc = ibmveth_alloc_tx_resources(adapter);
[Severity: Low]
The comment about PHYP mask, napi_disable and free_irq now sits above ibmveth_free_tx_resources() instead of above the ibmveth_cleanup_rx_interrupts() call it describes, so it reads as documentation of the TX free. Could it move below the TX free?
Quoted 2 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0do {
[ ... ]
Quoted 6 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-3-for (i =0 ; i < netdev->real_num_tx_queues; i++) -ibmveth_free_tx_ltb (adapter, i); -netdev_dbg (netdev,"close complete\n" );return 0 ;
Re: [PATCH net-next v4 06/14] ibmveth: Refactor TX resource allocation in open/close paths
Quoted 11 lines · Jakub Kicinski
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
ibmveth: Refactor TX resource allocation in open/close pathsThis moves the per-queue TX long term buffer alloc/free loops out of
ibmveth_open()andibmveth_close()intoibmveth_alloc_tx_resources()andibmveth_free_tx_resources(), relocates the TX allocation to the end ofopen(), and reworks theopen()error unwind labels.A few questions on the changelog and on the new ordering below.
Hi Jakub,
Thanks for the review.
Quoted 25 lines
[Severity: Medium]
The changelog says:That also retires a mid-open TX leak where the old index-based while (--i) free could skip successfully allocated LTBs.
Is that leak real? In the parent commit the TX loop re-initializes the shared index before allocating anything:
rc = -ENOMEM; for (i = 0; i < netdev->real_num_tx_queues; i++) { if (ibmveth_allocate_tx_ltb(adapter, i)) goto out_free_tx_ltb; }and nothing between that loop and the later gotos (mac_address/rxq_desc setup,
ibmveth_register_logical_lan(),ibmveth_alloc_buffer_pools(),ibmveth_setup_rx_interrupts()) reassigns i. On every path that reachesout_free_tx_ltb, i is either the failing index orreal_num_tx_queues, so while (--i >= 0) frees exactly what was allocated. Paths taken before the TX loop jump toout_free_filter_list, which is below the TX label.The new helper also uses the same index-based unwind that the changelog calls broken:
Quoted 8 lines
err_free_ltbs:while (--i >= 0) ibmveth_free_tx_ltb(adapter, i);Could this paragraph be dropped or reworded to describe the actual motivation (removing the dependence on a shared loop index and matching the RX helpers)? As written it reads like a leak fix that a stable maintainer might want to backport, and there is no Fixes: tag.
Agreed for this patch. I will drop that paragraph and reword the motivation in terms of helper extraction, avoiding dependence on shared open-path loop state, and matching the RX helper structure, rather than presenting it as a leak fix here.
Quoted 28 lines
Quoted 8 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0@@ -1077,8 +1077,14 @@ static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,static void ibmveth_free_tx_ltb (struct ibmveth_adapter *adapter,int idx) {drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter, · line 1087
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +8
-2-dma_unmap_single (&adapter->vdev->dev, adapter->tx_ltb_dma[idx], - adapter->tx_ltb_size,DMA_TO_DEVICE ); +if (!adapter->tx_ltb_ptr[idx]) +return ; + +if (adapter->tx_ltb_dma[idx]) { +dma_unmap_single (&adapter->vdev->dev, adapter->tx_ltb_dma[idx], + adapter->tx_ltb_size,DMA_TO_DEVICE ); + adapter->tx_ltb_dma[idx] =0 ; + }[Severity: Low]
Is the inner test on tx_ltb_dma[idx] wanted here? It treats DMA address 0 as a "not mapped" sentinel, and the only architecturally valid check for a bad handle isdma_mapping_error()at map time. If a mapping ever legitimately returned IOVA 0, the TCE entry would leak whilekfree()below still released the buffer.On pseries this cannot happen today, since arch/powerpc/kernel/iommu.c:iommu_table_reserve_pages() reserves entry 0:
Quoted 8 lines
if (tbl->it_offset == 0) set_bit(0, tbl->it_map);The test also looks redundant:
ibmveth_allocate_tx_ltb()already kfree()s the buffer and clears tx_ltb_ptr[idx] ondma_mapping_error(), so tx_ltb_ptr[idx] != NULL already implies a valid mapping and the new early return covers the stated "safe on an unallocated slot" goal. Would dropping the zero test and relying on the pointer check be simpler?
Agreed. In v5 I will just key the free path off allocation state instead of treating DMA address 0 as a sentinel.
Quoted 28 lines
Quoted 3 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0kfree (adapter->tx_ltb_ptr[idx]); adapter->tx_ltb_ptr[idx] =NULL ; }[ ... ]
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_open(struct net_device *netdev) · line 1234
Quoted 12 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-2@@ -1184,35 +1226,38 @@ static int ibmveth_open(struct net_device *netdev) rxq_desc.desc, mac_address); rc = -ENONET ; - goto out_free_tx_ltb; + goto out_free_queue_mem; } rc =ibmveth_alloc_buffer_pools (adapter);if (rc) - goto out_free_tx_ltb; + goto out_unregister_lan;[Severity: Low]
This changes hypervisor-visible behaviour on an RX error path, which the changelog does not mention while stating "there is no MQ RX behaviour change here".In the parent commit an
ibmveth_alloc_buffer_pools()failure went toout_free_tx_ltband then toibmveth_cleanup_rx_resources(), leaving the logical LAN registered with PHYP while the RX ring was freed and the buffer list unmapped. With the new label theh_free_logical_lan()loop runs first, which is the safer order.
Quoted 4 lines
Is this an intentional fix folded into a TX-only refactor? If so, could it be split out with its own changelog text (the next patch in the series reorders
open()so this path disappears entirely, so the fix becomes hard to find later).
Yes, that change was intentional, and I agree it needs to be called out more clearly. Freeing the LAN before tearing down the RX resources is the safer order on that failure path, so in v5 I will say that explicitly in the changelog instead of letting it read like a pure TX refactor.
Quoted 85 lines
Quoted 19 lines
drivers/net/ethernet/ibm/ibmveth.c +6
-6rc =ibmveth_setup_rx_interrupts (adapter); -if (rc) { -do { - lpar_rc =h_free_logical_lan (adapter->vdev->unit_address); - }while (H_IS_LONG_BUSY (lpar_rc) || (lpar_rc ==H_BUSY )); - goto out_free_buffer_pools; - } +if (rc) + goto out_unregister_lan;netdev_dbg (netdev,"initial replenish cycle\n" );ibmveth_schedule_rx_queue (adapter,0 ); + rc =ibmveth_alloc_tx_resources (adapter); +if (rc) + goto out_cleanup_rx_interrupts; +netif_tx_start_all_queues (netdev);[ ... ]
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_close(struct net_device *netdev) · line 1279
Quoted 13 lines
drivers/net/ethernet/ibm/ibmveth.c +1
-1@@ -1224,13 +1269,13 @@ static int ibmveth_close(struct net_device *netdev) {struct ibmveth_adapter *adapter =netdev_priv (netdev);long lpar_rc; -int i;netdev_dbg (netdev,"close starting\n" );netif_tx_stop_all_queues (netdev);/* PHYP mask + napi_disable + free_irq live in cleanup_rx_interrupts */ +ibmveth_free_tx_resources (adapter);ibmveth_cleanup_rx_interrupts (adapter);[Severity: High]
Can a transmit already in flight write into the TX LTB after it has been unmapped and freed here?
netif_tx_stop_all_queues()only sets__QUEUE_STATE_DRV_XOFF. It does not take__netif_tx_lockand does not wait for anndo_start_xmitthat is already running on another CPU (netif_tx_disable()would).ibmveth_close()is also called directly, bypassingdev_close()->dev_deactivate_many()and itssynchronize_net()/some_qdisc_is_busy()wait, from:
ibmveth_set_csum_offload()
ibmveth_set_tso()
ibmveth_change_mtu()
veth_pool_store()all of which do:
if (netif_running(dev)) { restart = 1; ibmveth_close(dev); }So a transmitter that already passed
netif_xmit_stopped()can be insideibmveth_start_xmit(), which copies into the LTB with no NULL check and no lock:memcpy(adapter->tx_ltb_ptr[queue_num], skb->data, skb_headlen(skb)); ... memcpy(adapter->tx_ltb_ptr[queue_num] + total_bytes, skb_frag_address_safe(frag), skb_frag_size(frag)); ... desc.fields.address = adapter->tx_ltb_dma[queue_num];while
ibmveth_free_tx_ltb()concurrently does:dma_unmap_single(...); adapter->tx_ltb_dma[idx] = 0; kfree(adapter->tx_ltb_ptr[idx]); adapter->tx_ltb_ptr[idx] = NULL;That looks like either a write of up to
tx_ltb_sizebytes into freed heap memory, or a NULL dereference on the cleared pointer.A second question on the same move: the DMA unmap now happens before the
h_free_logical_lan()loop below, so a racingibmveth_send()can hand PHYP a descriptor whose address is 0 or an already unmapped TCE while the logical LAN is still registered. Previously the TX LTBs were freed at the very end ofclose(), afterh_free_logical_lan(), free_irq()/napi_disable() and the buffer pool teardown, which incidentally left a large gap between the XOFF and the free. Was the inversion intended?
Quoted 6 lines
Would keeping
ibmveth_free_tx_resources()after theh_free_logical_lan()loop, switching tonetif_tx_disable(), and/or adding anif (unlikely(!adapter->tx_ltb_ptr[queue_num]))bail-out in
ibmveth_start_xmit()be preferable?
Agreed. This is the main correctness issue in the patch. In v5 I will quiesce TX before freeing the LTB state and move the TX free back after h_free_logical_lan(). I will also add the !tx_ltb_ptr[q] guard in ibmveth_start_xmit(), but I think it reads better as the answer to your open-side mirror-window point below.
Quoted 9 lines
The last one would also
cover the mirror window that the newopen()ordering creates, where the LAN is registered and RX interrupts/NAPI are live while tx_ltb_ptr[] is still NULL during the sleepingGFP_KERNELallocation inibmveth_alloc_tx_resources():ibmveth_schedule_rx_queue(adapter, 0); rc = ibmveth_alloc_tx_resources(adapter);
Yes, and that is how I plan to cover the open-side window for this series. netif_tx_start_all_queues() still runs only after ibmveth_alloc_tx_resources() succeeds, and the !tx_ltb_ptr[q] guard in ibmveth_start_xmit() closes the remaining mirror case without needing to reorder open in this patch.
Quoted 6 lines
[Severity: Low]
The comment about PHYP mask,napi_disableandfree_irqnow sits aboveibmveth_free_tx_resources()instead of above theibmveth_cleanup_rx_interrupts()call it describes, so it reads as documentation of the TX free. Could it move below the TX free?
Agreed. I will clean up that misplaced comment so the code and
commentary match.
Thanks,
Mingming
Quoted 11 lines
Quoted 3 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0do {[ ... ]
Quoted 6 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-3-for (i =0 ; i < netdev->real_num_tx_queues; i++) -ibmveth_free_tx_ltb (adapter, i); -netdev_dbg (netdev,"close complete\n" );return 0 ;
PATCH v4 7/14 ibmveth: Add RX queue register/deregister helpers for MQ
In reply to Mingming CaoMQ RX changes queue lifecycle from one adapter-level register/free pair to a mixed model:
- queue 0: registered via h_register_logical_lan*()
- queues 1..N: registered via H_REG_LOGICAL_LAN_QUEUE
This patch extracts the queue-0 control-plane helpers used by open/close today and wires them in the same commit:
ibmveth_register_rx_queues()
ibmveth_free_all_queues()
Also update ibmveth_register_logical_lan() so that when multi_queue is later enabled, queue 0 uses h_register_logical_lan_with_handle() and stores queue_handle[0]. Runtime remains single-queue: multi_queue is still false, so queue 0 keeps the legacy h_register_logical_lan() flow and no subordinate queue is registered. Subordinate register/deregister helpers arrive with the MQ enablement patch that first calls them.
Introduce adapter->hcall_stats here for register/free path accounting (first use).
Open/close unwind:
- failures after successful LAN registration enter out_unregister_queues, then fall through to out_free_buffer_pools, so RX pools are not leaked;
- free_all_queues() runs before free_buffer_pools() on open failure and close, so PHYP cannot retain a registered LAN while its DMA-backed RX pools are released.
Failures before registration go directly to out_free_buffer_pools. RX interrupt masking before napi_disable is already handled by cleanup_rx_interrupts() from the IRQ-helper patch.
With the helper patches through this one, open/close follow the MQ-ready bring-up/teardown pipeline (still single-queue until MQ enablement):
ibmveth_open():
1. ibmveth_alloc_filter_list()
2. ibmveth_alloc_rx_queues() - buffer lists + RX rings
3. ibmveth_alloc_buffer_pools() - guest RX memory before PHYP
4. ibmveth_register_rx_queues() - PHYP registration (no IRQ enable)
5. netif_set_real_num_rx_queues()
6. ibmveth_setup_rx_interrupts() - request_irq + napi_enable
7. initial replenish kick - schedule_rx_queue(0)
8. ibmveth_alloc_tx_resources()
ibmveth_close():
1. netif_tx_stop_all_queues()
2. ibmveth_free_tx_resources()
3. ibmveth_cleanup_rx_interrupts() - mask PHYP, napi_disable, free_irq
4. ibmveth_free_all_queues() - H_FREE_LOGICAL_LAN
5. ibmveth_free_buffer_pools()
6. ibmveth_cleanup_rx_resources()
7. ibmveth_free_filter_list()
---
Changes in v4:
- Introduce register/free helpers in the same patch that wires their first open/close callers; keep subordinate-only helpers deferred until MQ enablement.
- Introduce adapter->hcall_stats here (first use); not in patch 2.
- Correct open/close unwind so free_all_queues() precedes free_buffer_pools().
- Drop the orphaned big-bang "open/close pipeline" patch from v3; that wiring is incremental across helper patches 3-7 instead.
drivers/net/ethernet/ibm/ibmveth.c | 170 ++++++++++++++++++++--------- drivers/net/ethernet/ibm/ibmveth.h | 14 +++ 2 files changed, 133 insertions(+), 51 deletions(-)
drivers/net/ethernet/ibm/ibmveth.c +119 -51
@@ -434,6 +434,9 @@ ibmveth_setup_rx_interrupts(struct ibmveth_adapter *adapter)struct net_device *netdev = adapter->netdev;int i, rc; +for (i =0 ; i < adapter->num_rx_queues; i++) +napi_enable (&adapter->napi[i]); +for (i =0 ; i < adapter->num_rx_queues; i++) {if (!adapter->queue_irq[i]) {netdev_err (netdev,"queue %d has invalid IRQ (0)\n" , i);
@@ -451,14 +454,13 @@ ibmveth_setup_rx_interrupts(struct ibmveth_adapter *adapter) } } -for (i =0 ; i < adapter->num_rx_queues; i++) -napi_enable (&adapter->napi[i]); -return 0 ; err_free_irqs:while (--i >=0 )free_irq (adapter->queue_irq[i], &adapter->napi[i]); +for (i =0 ; i < adapter->num_rx_queues; i++) +napi_disable (&adapter->napi[i]);return rc; }
@@ -1156,9 +1158,14 @@ static void ibmveth_free_tx_resources(struct ibmveth_adapter *adapter) }static int ibmveth_register_logical_lan (struct ibmveth_adapter *adapter, -union ibmveth_buf_desc rxq_desc,u64 mac_address) +union ibmveth_buf_desc rxq_desc, +u64 mac_address) {int rc, try_again =1 ; +unsigned long ua = adapter->vdev->unit_address; +unsigned long buf_dma = adapter->buffer_list_dma[0 ]; +unsigned long filter_dma = adapter->filter_list_dma; +u64 *qh0 = &adapter->queue_handle[0 ];/* * After a kexec the adapter will still be open, so our attempt to
@@ -1166,13 +1173,25 @@ static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,* try again, but only once. */ retry: - rc =h_register_logical_lan (adapter->vdev->unit_address, - adapter->buffer_list_dma[0 ], rxq_desc.desc, - adapter->filter_list_dma, mac_address); +/* In multi-queue mode, obtain a queue handle for queue 0 so all RX +* queues can use the same per-queue buffer hypercalls. +*/ +if (adapter->multi_queue) { + rc =h_register_logical_lan_with_handle (ua, buf_dma, + rxq_desc.desc, + filter_dma, + mac_address, + qh0); + }else { + rc =h_register_logical_lan (ua, buf_dma, rxq_desc.desc, + filter_dma, mac_address); + } + adapter->hcall_stats.reg_lan++;if (rc !=H_SUCCESS && try_again) {do { rc =h_free_logical_lan (adapter->vdev->unit_address); + adapter->hcall_stats.free_lan++; }while (H_IS_LONG_BUSY (rc) || (rc ==H_BUSY )); try_again =0 ;
@@ -1182,14 +1201,89 @@ static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,return rc; } +/** +* ibmveth_free_all_queues - Free all RX queues at once +* @adapter: ibmveth adapter structure +* +* Uses H_FREE_LOGICAL_LAN to free all queues in one hypercall. +* Used during interface close and registration error cleanup. +* +* Clears queue handles only; queue_irq[] is released by +* ibmveth_cleanup_rx_interrupts(). +*/ +static void ibmveth_free_all_queues (struct ibmveth_adapter *adapter) +{ +unsigned long lpar_rc; +int i; + +netdev_dbg (adapter->netdev,"freeing all RX queues at once\n" ); + +do { + lpar_rc =h_free_logical_lan (adapter->vdev->unit_address); + adapter->hcall_stats.free_lan++; + }while (H_IS_LONG_BUSY (lpar_rc) || (lpar_rc ==H_BUSY )); + +if (lpar_rc !=H_SUCCESS ) { +netdev_err (adapter->netdev, +"h_free_logical_lan failed: %ld\n" , lpar_rc); + } + +for (i =0 ; i < adapter->num_rx_queues; i++) + adapter->queue_handle[i] =0 ; +} + +/** +* ibmveth_register_rx_queues - Register RX queues with hypervisor +* @adapter: ibmveth adapter structure +* @mac_address: MAC address for device registration +* +* Registers queue 0 via ibmveth_register_logical_lan(). Subordinate queue +* registration is added when multi-queue RX is enabled. +* +* Return: 0 on success, -ENONET if queue 0 registration fails +*/ +static int +ibmveth_register_rx_queues (struct ibmveth_adapter *adapter,u64 mac_address) +{ +struct net_device *netdev = adapter->netdev; +union ibmveth_buf_desc rxq_desc; +unsigned long lpar_rc; +int rc; + + rxq_desc.fields.flags_len =IBMVETH_BUF_VALID | + adapter->rx_queue[0 ].queue_len; + rxq_desc.fields.address = adapter->rx_queue[0 ].queue_dma; + adapter->queue_irq[0 ] = netdev->irq; + + rc =ibmveth_disable_irq (adapter,0 ); +if (rc !=H_SUCCESS ) +netdev_dbg (netdev, +"Failed to disable IRQ for queue 0 before registration, rc=%d\n" , + rc); + + lpar_rc =ibmveth_register_logical_lan (adapter, rxq_desc, mac_address); +if (lpar_rc !=H_SUCCESS ) { +netdev_err (netdev, +"h_register_logical_lan failed: %ld\n" , lpar_rc); +netdev_err (netdev, +"buffer TCE:0x%llx filter TCE:0x%llx rxq desc:0x%llx MAC:0x%llx\n" , + adapter->buffer_list_dma[0 ], + adapter->filter_list_dma, + rxq_desc.desc, mac_address); +return -ENONET ; + } + +netdev_dbg (netdev, +"registered 1 RX queue with hypervisor (single-queue mode)\n" ); +return 0 ; +} +static int ibmveth_open (struct net_device *netdev) {struct ibmveth_adapter *adapter =netdev_priv (netdev); -u64 mac_address; +u64 mac_address =ether_addr_to_u64 (netdev->dev_addr);int rxq_entries =1 ; -unsigned long lpar_rc;int rc; -union ibmveth_buf_desc rxq_desc;int i;netdev_dbg (netdev,"open starting\n" );
@@ -1205,37 +1299,23 @@ static int ibmveth_open(struct net_device *netdev)if (rc) goto out_free_filter_list; - mac_address =ether_addr_to_u64 (netdev->dev_addr); - - rxq_desc.fields.flags_len =IBMVETH_BUF_VALID | - adapter->rx_queue[0 ].queue_len; - rxq_desc.fields.address = adapter->rx_queue[0 ].queue_dma; - - adapter->queue_irq[0 ] = netdev->irq; -ibmveth_disable_irq (adapter,0 ); - - lpar_rc =ibmveth_register_logical_lan (adapter, rxq_desc, mac_address); - -if (lpar_rc !=H_SUCCESS ) { -netdev_err (netdev,"h_register_logical_lan failed with %ld\n" , - lpar_rc); -netdev_err (netdev,"buffer TCE:0x%llx filter TCE:0x%llx rxq " -"desc:0x%llx MAC:0x%llx\n" , - adapter->buffer_list_dma[0 ], - adapter->filter_list_dma, - rxq_desc.desc, - mac_address); - rc = -ENONET ; + rc =ibmveth_alloc_buffer_pools (adapter); +if (rc) goto out_free_queue_mem; - } - rc =ibmveth_alloc_buffer_pools (adapter); + rc =ibmveth_register_rx_queues (adapter, mac_address);if (rc) - goto out_unregister_lan; + goto out_free_buffer_pools; + + rc =netif_set_real_num_rx_queues (netdev, adapter->num_rx_queues); +if (rc) { +netdev_err (netdev,"failed to set number of rx queues\n" ); + goto out_unregister_queues; + } rc =ibmveth_setup_rx_interrupts (adapter);if (rc) - goto out_unregister_lan; + goto out_unregister_queues;netdev_dbg (netdev,"initial replenish cycle\n" );ibmveth_schedule_rx_queue (adapter,0 );
@@ -1252,10 +1332,9 @@ static int ibmveth_open(struct net_device *netdev) out_cleanup_rx_interrupts:ibmveth_cleanup_rx_interrupts (adapter); -out_unregister_lan: -do { - lpar_rc =h_free_logical_lan (adapter->vdev->unit_address); - }while (H_IS_LONG_BUSY (lpar_rc) || (lpar_rc ==H_BUSY )); +out_unregister_queues: +ibmveth_free_all_queues (adapter); +out_free_buffer_pools:ibmveth_free_buffer_pools (adapter); out_free_queue_mem:ibmveth_cleanup_rx_resources (adapter);
@@ -1268,7 +1347,6 @@ static int ibmveth_open(struct net_device *netdev)static int ibmveth_close (struct net_device *netdev) {struct ibmveth_adapter *adapter =netdev_priv (netdev); -long lpar_rc;netdev_dbg (netdev,"close starting\n" );
@@ -1277,18 +1355,8 @@ static int ibmveth_close(struct net_device *netdev)/* PHYP mask + napi_disable + free_irq live in cleanup_rx_interrupts */ ibmveth_free_tx_resources (adapter);ibmveth_cleanup_rx_interrupts (adapter); - -do { - lpar_rc =h_free_logical_lan (adapter->vdev->unit_address); - }while (H_IS_LONG_BUSY (lpar_rc) || (lpar_rc ==H_BUSY )); - -if (lpar_rc !=H_SUCCESS ) { -netdev_err (netdev,"h_free_logical_lan failed with %lx, " -"continuing with close\n" , lpar_rc); - } -ibmveth_update_rx_no_buffer (adapter); - +ibmveth_free_all_queues (adapter);ibmveth_free_buffer_pools (adapter);ibmveth_cleanup_rx_resources (adapter);ibmveth_free_filter_list (adapter);
drivers/net/ethernet/ibm/ibmveth.h +14 -0
@@ -269,6 +269,17 @@ static int pool_active[] = { 1, 1, 0, 0, 0};#define IBM_VETH_INVALID_MAP ((u16)0xffff) +struct ibmveth_hcall_stats { +u64 reg_lan_queue;/* H_REG_LOGICAL_LAN_QUEUE */ +u64 reg_lan;/* H_REGISTER_LOGICAL_LAN */ +u64 add_bufs_queue;/* H_ADD_LOGICAL_LAN_BUFFERS_QUEUE */ +u64 add_bufs;/* H_ADD_LOGICAL_LAN_BUFFERS */ +u64 add_buf;/* H_ADD_LOGICAL_LAN_BUFFER */ +u64 free_lan_queue;/* H_FREE_LOGICAL_LAN_QUEUE */ +u64 free_lan;/* H_FREE_LOGICAL_LAN */ +u64 send_lan;/* H_SEND_LOGICAL_LAN */ +}; +struct ibmveth_buff_pool {u32 size;u32 index;
@@ -333,6 +344,9 @@ struct ibmveth_adapter {u64 tx_large_packets;u64 rx_large_packets; +/* Multi-queue statistics */ +struct ibmveth_hcall_stats hcall_stats; +/* Ethtool settings */ u8 duplex;u32 speed;
Signature · 1 line
2.50.1 (Apple Git-155)
Jakub Kicinski
maintainerRe: [PATCH net-next v4 07/14] ibmveth: Add RX queue register/deregister helpers for MQ
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
ibmveth: Add RX queue register/deregister helpers for MQ
This patch factors the queue-0 control-plane work out of ibmveth_open()/ibmveth_close() into ibmveth_register_rx_queues() and ibmveth_free_all_queues(), teaches ibmveth_register_logical_lan() to use h_register_logical_lan_with_handle() once multi_queue is set, and adds adapter->hcall_stats. Runtime behaviour stays single-queue.
A few questions about the new open/close teardown pipeline below.
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0
[ ... ]
Quoted 24 lines
drivers/net/ethernet/ibm/ibmveth.c +20
-0@@ -1182,14 +1201,89 @@ static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,return rc; } +/** +* ibmveth_free_all_queues - Free all RX queues at once +* @adapter: ibmveth adapter structure +* +* Uses H_FREE_LOGICAL_LAN to free all queues in one hypercall. +* Used during interface close and registration error cleanup. +* +* Clears queue handles only; queue_irq[] is released by +* ibmveth_cleanup_rx_interrupts(). +*/ +static void ibmveth_free_all_queues (struct ibmveth_adapter *adapter) +{ +unsigned long lpar_rc; +int i; + +netdev_dbg (adapter->netdev,"freeing all RX queues at once\n" ); + + do { + lpar_rc = h_free_logical_lan(adapter->vdev->unit_address); + adapter->hcall_stats.free_lan++;
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter, · line 1233
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+ }while (H_IS_LONG_BUSY (lpar_rc) || (lpar_rc ==H_BUSY )); + +if (lpar_rc !=H_SUCCESS ) { +netdev_err (adapter->netdev, +"h_free_logical_lan failed: %ld\n" , lpar_rc); + } + +for (i =0 ; i < adapter->num_rx_queues; i++) + adapter->queue_handle[i] =0 ; +}
[Severity: Low]
Where is the firmware contract for this documented? The helper issues one adapter-level h_free_logical_lan() and then clears queue_handle[] for all num_rx_queues entries, which assumes that the single hcall also deregisters the subordinate queues that later patches register with H_REG_LOGICAL_LAN_QUEUE.
The header already provides a per-queue wrapper that this path never uses:
static inline long h_free_logical_lan_queue(unsigned long unit_address,
unsigned long queue_handle)
The assumption cannot be exercised at this commit because multi_queue is still 0, and the helper is unchanged at the end of the series, so nothing in the series validates it. The changelog justifies the new ordering with "PHYP cannot retain a registered LAN while its DMA-backed RX pools are released"; if H_FREE_LOGICAL_LAN does not release subordinate queue registrations, then ibmveth_free_buffer_pools() and ibmveth_cleanup_rx_resources() would unmap memory PHYP still owns.
Could the PAPR reference for the subordinate-queue semantics be stated in the kerneldoc or the changelog before the MQ enablement patch relies on it?
[ ... ]
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0@@ -1252,10 +1332,9 @@ static int ibmveth_open(struct net_device *netdev) out_cleanup_rx_interrupts:ibmveth_cleanup_rx_interrupts (adapter);
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_open(struct net_device *netdev) · line 1340
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +3
-4-out_unregister_lan: -do { - lpar_rc =h_free_logical_lan (adapter->vdev->unit_address); - }while (H_IS_LONG_BUSY (lpar_rc) || (lpar_rc ==H_BUSY )); +out_unregister_queues: +ibmveth_free_all_queues (adapter); +out_free_buffer_pools:ibmveth_free_buffer_pools (adapter); out_free_queue_mem:ibmveth_cleanup_rx_resources (adapter);
[Severity: High]
This isn't a bug introduced by this patch, but should the unwind here leave state that a later ibmveth_close() can walk again?
ibmveth_set_csum_offload(), ibmveth_set_tso() and ibmveth_change_mtu() call ibmveth_close()/ibmveth_open() directly instead of dev_close()/dev_open(), so when the re-open fails the error is only returned as an errno while IFF_UP and netif_running() stay set:
if (restart) rc2 = ibmveth_open(dev); return rc1 ? rc1 : rc2;
After that failed open, every RX resource is freed (buffer_list_addr[] is NULLed by ibmveth_cleanup_rx_resources()), NAPI is left disabled, and queue_irq[0] is deliberately kept populated. The next ndo_stop, for example retrying the same ethtool command or an "ip link set dev X down", re-enters ibmveth_close() -> ibmveth_cleanup_rx_interrupts(), which does:
for (i = 0; i < adapter->num_rx_queues; i++)
napi_disable(&adapter->napi[i]);
for (i = 0; i < adapter->num_rx_queues; i++) {
if (adapter->queue_irq[i])
free_irq(adapter->queue_irq[i], &adapter->napi[i]);
}Can the second napi_disable() ever return? napi_disable_locked() leaves NAPIF_STATE_SCHED|NAPIF_STATE_NPSVC set, and the second call waits in usleep_range() for bits that no poll will clear, with rtnl held by the ndo_stop caller. The free_irq() that follows also refers to an IRQ that is no longer requested.
If it did get past those, ibmveth_update_rx_no_buffer() would dereference the now-NULL buffer_list_addr[0]:
__be64 *p = adapter->buffer_list_addr[0] + 4096 - 8;
and ibmveth_free_all_queues() would issue H_FREE_LOGICAL_LAN for a LAN that is not registered.
The baseline close() already had an unconditional napi_disable(), so the hang predates this patch, but this patch centralizes and extends the unconditional teardown (now also free_all_queues() plus the NULL-able buffer_list_addr[0] read) without an "already torn down" guard, and the same structure survives to the end of the series. Would a state check at the top of ibmveth_close(), or switching those callers to dev_close()/dev_open(), be preferable?
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_close(struct net_device *netdev) · line 1357
Quoted 12 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-1@@ -1268,7 +1347,6 @@ static int ibmveth_open(struct net_device *netdev)static int ibmveth_close (struct net_device *netdev) {struct ibmveth_adapter *adapter =netdev_priv (netdev); -long lpar_rc;netdev_dbg (netdev,"close starting\n" );@@ -1277,18 +1355,8 @@ static int ibmveth_close(struct net_device *netdev)/* PHYP mask + napi_disable + free_irq live in cleanup_rx_interrupts */ ibmveth_free_tx_resources (adapter);ibmveth_cleanup_rx_interrupts (adapter);
[Severity: Medium]
Does the PHYP mask taken by ibmveth_cleanup_rx_interrupts() actually stay in effect until free_irq()?
The helper masks queue 0 and calls synchronize_irq() before napi_disable():
ibmveth_disable_irq(adapter, i); synchronize_irq(adapter->queue_irq[i]); ... for (i = 0; i < adapter->num_rx_queues; i++) napi_disable(&adapter->napi[i]);
But a poll that is already in flight unmasks it again on completion, with no coordination against teardown:
ibmveth_poll()
if (!napi_complete_done(napi, frames_processed)) goto out; /* We think we are done - reenable interrupts, ... */ lpar_rc = ibmveth_enable_irq(adapter, 0);
So napi_disable() can return with the source unmasked. In the window before free_irq(), an RX interrupt is then delivered and ibmveth_interrupt() -> ibmveth_schedule_rx_queue() sees napi_schedule_prep() fail and returns IRQ_HANDLED without masking, since masking only happens on the napi_schedule_prep() success branch. Can a VIO source with pending RX re-assert repeatedly until free_irq() shuts the line down, and can ibmveth_free_all_queues() then issue H_FREE_LOGICAL_LAN while the source is still unmasked?
The kerneldoc in cleanup_rx_interrupts() states "Mask PHYP before napi_disable so ibmveth_interrupt cannot return IRQ_HANDLED without masking", which does not seem to hold when the poll itself is the entity that unmasks. The pre-series code did napi_disable() first, then h_vio_signal(VIO_IRQ_DISABLE), then free_irq(); this patch propagates the new ordering into the open-failure unwind as well and places ibmveth_free_all_queues() behind it, and ibmveth_poll() still unmasks unconditionally at the end of the series.
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_close(struct net_device *netdev) · line 1359
Quoted 13 lines
drivers/net/ethernet/ibm/ibmveth.c +1
-11- -do { - lpar_rc =h_free_logical_lan (adapter->vdev->unit_address); - }while (H_IS_LONG_BUSY (lpar_rc) || (lpar_rc ==H_BUSY )); - -if (lpar_rc !=H_SUCCESS ) { -netdev_err (netdev,"h_free_logical_lan failed with %lx, " -"continuing with close\n" , lpar_rc); - } -ibmveth_update_rx_no_buffer (adapter); - +ibmveth_free_all_queues (adapter);
[Severity: Low]
Was moving ibmveth_update_rx_no_buffer() ahead of the H_FREE_LOGICAL_LAN intentional?
Previously the counter was read after h_free_logical_lan() returned, so the hypervisor was no longer writing it. ibmveth_update_rx_no_buffer() reads the last 8 bytes of the DMA-mapped buffer_list page that PHYP updates:
__be64 *p = adapter->buffer_list_addr[0] + 4096 - 8; adapter->rx_no_buffer = be64_to_cpup(p);
At the new call site the LAN is still registered, so frames can still be dropped and counted after the snapshot, and those drops are lost from the rx_no_buffer value reported by ethtool -S.
The 7-step close pipeline in the changelog does not mention ibmveth_update_rx_no_buffer() at all:
ibmveth_close():
1. netif_tx_stop_all_queues()
2. ibmveth_free_tx_resources()
3. ibmveth_cleanup_rx_interrupts() - mask PHYP, napi_disable, free_irq
4. ibmveth_free_all_queues() - H_FREE_LOGICAL_LAN
5. ibmveth_free_buffer_pools()
6. ibmveth_cleanup_rx_resources()
7. ibmveth_free_filter_list()
Could the step list be updated to include it and to say where it belongs relative to H_FREE_LOGICAL_LAN? The ordering is unchanged through the end of the series.
Quoted 3 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0ibmveth_free_buffer_pools (adapter);ibmveth_cleanup_rx_resources (adapter);ibmveth_free_filter_list (adapter);
[ ... ]
Re: [PATCH net-next v4 07/14] ibmveth: Add RX queue register/deregister helpers for MQ
Quoted 5 lines · Jakub Kicinski
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
ibmveth: Add RX queue register/deregister helpers for MQ
Quoted 7 lines
This patch factors the queue-0 control-plane work out of ibmveth_open()/ibmveth_close() into
ibmveth_register_rx_queues()andibmveth_free_all_queues(), teachesibmveth_register_logical_lan()to useh_register_logical_lan_with_handle()oncemulti_queueis set, and addsadapter->hcall_stats. Runtime behaviour stays single-queue.A few questions about the new open/close teardown pipeline below.
Hi Jakub,
Thanks for the review.
Quoted 60 lines
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0[ ... ]
Quoted 24 lines
drivers/net/ethernet/ibm/ibmveth.c +20
-0@@ -1182,14 +1201,89 @@ static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,return rc; } +/** +* ibmveth_free_all_queues - Free all RX queues at once +* @adapter: ibmveth adapter structure +* +* Uses H_FREE_LOGICAL_LAN to free all queues in one hypercall. +* Used during interface close and registration error cleanup. +* +* Clears queue handles only; queue_irq[] is released by +* ibmveth_cleanup_rx_interrupts(). +*/ +static void ibmveth_free_all_queues (struct ibmveth_adapter *adapter) +{ +unsigned long lpar_rc; +int i; + +netdev_dbg (adapter->netdev,"freeing all RX queues at once\n" ); + + do { + lpar_rc = h_free_logical_lan(adapter->vdev->unit_address); + adapter->hcall_stats.free_lan++;drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter, · line 1233
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+ }while (H_IS_LONG_BUSY (lpar_rc) || (lpar_rc ==H_BUSY )); + +if (lpar_rc !=H_SUCCESS ) { +netdev_err (adapter->netdev, +"h_free_logical_lan failed: %ld\n" , lpar_rc); + } + +for (i =0 ; i < adapter->num_rx_queues; i++) + adapter->queue_handle[i] =0 ; +}[Severity: Low]
Where is the firmware contract for this documented? The helper issues one adapter-levelh_free_logical_lan()and then clears queue_handle[] for allnum_rx_queuesentries, which assumes that the single hcall also deregisters the subordinate queues that later patches register withH_REG_LOGICAL_LAN_QUEUE.The header already provides a per-queue wrapper that this path never uses:
static inline long h_free_logical_lan_queue(unsigned long
unit_address,
unsigned longqueue_handle)The assumption cannot be exercised at this commit because
multi_queueis still 0, and the helper is unchanged at the end of the series, so nothing in the series validates it. The changelog justifies the new ordering with "PHYP cannot retain a registered LAN while its DMA-backed RX pools are released"; ifH_FREE_LOGICAL_LANdoes not release subordinate queue registrations, thenibmveth_free_buffer_pools()andibmveth_cleanup_rx_resources()would unmap memory PHYP still owns.
Quoted 4 lines
Could the PAPR reference for the subordinate-queue semantics be stated in the kerneldoc or the changelog before the MQ enablement patch relies on it?
[ ... ]
Yes. For full teardown, that is the PHYP/PAPR contract we are relying
on: one H_FREE_LOGICAL_LAN tears down the LAN and any subordinate queues under it. H_FREE_LOGICAL_LAN_QUEUE is for incremental queue removal during scale-down, not for full adapter teardown. I will make that explicit in the kerneldoc and changelog.
Quoted 56 lines
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0@@ -1252,10 +1332,9 @@ static int ibmveth_open(struct net_device *netdev) out_cleanup_rx_interrupts:ibmveth_cleanup_rx_interrupts (adapter);drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_open(struct net_device *netdev) · line 1340
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +3
-4-out_unregister_lan: -do { - lpar_rc =h_free_logical_lan (adapter->vdev->unit_address); - }while (H_IS_LONG_BUSY (lpar_rc) || (lpar_rc ==H_BUSY )); +out_unregister_queues: +ibmveth_free_all_queues (adapter); +out_free_buffer_pools:ibmveth_free_buffer_pools (adapter); out_free_queue_mem:ibmveth_cleanup_rx_resources (adapter);[Severity: High]
This isn't a bug introduced by this patch, but should the unwind here leave state that a lateribmveth_close()can walk again?
ibmveth_set_csum_offload(),ibmveth_set_tso()andibmveth_change_mtu()call ibmveth_close()/ibmveth_open() directly instead of dev_close()/dev_open(), so when the re-open fails the error is only returned as an errno whileIFF_UPandnetif_running()stay set:if (restart) rc2 = ibmveth_open(dev); return rc1 ? rc1 : rc2;After that failed open, every RX resource is freed (buffer_list_addr[] is NULLed by
ibmveth_cleanup_rx_resources()), NAPI is left disabled, and queue_irq[0] is deliberately kept populated. The nextndo_stop, for example retrying the same ethtool command or an "ip link set dev X down", re-entersibmveth_close()->ibmveth_cleanup_rx_interrupts(), which does:for (i = 0; i < adapter->num_rx_queues; i++) napi_disable(&adapter->napi[i]); for (i = 0; i < adapter->num_rx_queues; i++) { if (adapter->queue_irq[i]) free_irq(adapter->queue_irq[i], &adapter->napi[i]); }Can the second
napi_disable()ever return?napi_disable_locked()leaves NAPIF_STATE_SCHED|NAPIF_STATE_NPSVC set, and the second call waits inusleep_range()for bits that no poll will clear, with rtnl held by thendo_stopcaller. Thefree_irq()that follows also refers to an IRQ that is no longer requested.If it did get past those,
ibmveth_update_rx_no_buffer()would dereference the now-NULL buffer_list_addr[0]:__be64 *p = adapter->buffer_list_addr[0] + 4096 - 8;and
ibmveth_free_all_queues()would issueH_FREE_LOGICAL_LANfor a LAN that is not registered.
Quoted 7 lines
The baseline
close()already had an unconditionalnapi_disable(), so the hang predates this patch, but this patch centralizes and extends the unconditional teardown (now alsofree_all_queues()plus the NULL-able buffer_list_addr[0] read) without an "already torn down" guard, and the same structure survives to the end of the series. Would a state check at the top ofibmveth_close(), or switching those callers to dev_close()/dev_open(), be preferable?
Agreed. This one is high priority. I'm planning to add explicit opened / rx_irq_setup state so a second teardown does not walk partially initialized RX state as if it were live, and to fold that back into the owning lifecycle patch rather than leave it as a tail fixup.
Quoted 43 lines
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_close(struct net_device *netdev) · line 1357
Quoted 12 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-1@@ -1268,7 +1347,6 @@ static int ibmveth_open(struct net_device *netdev)static int ibmveth_close (struct net_device *netdev) {struct ibmveth_adapter *adapter =netdev_priv (netdev); -long lpar_rc;netdev_dbg (netdev,"close starting\n" );@@ -1277,18 +1355,8 @@ static int ibmveth_close(struct net_device *netdev)/* PHYP mask + napi_disable + free_irq live in cleanup_rx_interrupts */ ibmveth_free_tx_resources (adapter);ibmveth_cleanup_rx_interrupts (adapter);[Severity: Medium]
Does the PHYP mask taken byibmveth_cleanup_rx_interrupts()actually stay in effect untilfree_irq()?The helper masks queue 0 and calls
synchronize_irq()beforenapi_disable():ibmveth_disable_irq(adapter, i); synchronize_irq(adapter->queue_irq[i]); ... for (i = 0; i < adapter->num_rx_queues; i++) napi_disable(&adapter->napi[i]);But a poll that is already in flight unmasks it again on completion, with no coordination against teardown:
ibmveth_poll()if (!napi_complete_done(napi, frames_processed)) goto out; /* We think we are done - reenable interrupts, ... */ lpar_rc = ibmveth_enable_irq(adapter, 0);So
napi_disable()can return with the source unmasked. In the window beforefree_irq(), an RX interrupt is then delivered andibmveth_interrupt()->ibmveth_schedule_rx_queue()seesnapi_schedule_prep()fail and returnsIRQ_HANDLEDwithout masking, since masking only happens on thenapi_schedule_prep()success branch. Can a VIO source with pending RX re-assert repeatedly untilfree_irq()shuts the line down, and canibmveth_free_all_queues()then issueH_FREE_LOGICAL_LANwhile the source is still unmasked?
Quoted 8 lines
The kerneldoc in
cleanup_rx_interrupts()states "Mask PHYP beforenapi_disablesoibmveth_interruptcannot returnIRQ_HANDLEDwithout masking", which does not seem to hold when the poll itself is the entity that unmasks. The pre-series code didnapi_disable()first, thenh_vio_signal(VIO_IRQ_DISABLE), thenfree_irq(); this patch propagates the new ordering into the open-failure unwind as well and placesibmveth_free_all_queues()behind it, andibmveth_poll()still unmasks unconditionally at the end of the series.
Agreed. This is the same shutdown race as in 05/14. I'm planning to make poll complete without ibmveth_enable_irq() when shutdown is in progress, so I would not overclaim patch 7 alone as fully solving it. That should be folded back with the lifecycle/shutdown fixes rather than left as a tail fixup.
Quoted 42 lines
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_close(struct net_device *netdev) · line 1359
Quoted 13 lines
drivers/net/ethernet/ibm/ibmveth.c +1
-11- -do { - lpar_rc =h_free_logical_lan (adapter->vdev->unit_address); - }while (H_IS_LONG_BUSY (lpar_rc) || (lpar_rc ==H_BUSY )); - -if (lpar_rc !=H_SUCCESS ) { -netdev_err (netdev,"h_free_logical_lan failed with %lx, " -"continuing with close\n" , lpar_rc); - } -ibmveth_update_rx_no_buffer (adapter); - +ibmveth_free_all_queues (adapter);[Severity: Low]
Was movingibmveth_update_rx_no_buffer()ahead of theH_FREE_LOGICAL_LANintentional?Previously the counter was read after
h_free_logical_lan()returned, so the hypervisor was no longer writing it.ibmveth_update_rx_no_buffer()reads the last 8 bytes of the DMA-mappedbuffer_listpage that PHYP updates:__be64 *p = adapter->buffer_list_addr[0] + 4096 - 8; adapter->rx_no_buffer = be64_to_cpup(p);At the new call site the LAN is still registered, so frames can still be dropped and counted after the snapshot, and those drops are lost from the
rx_no_buffervalue reported by ethtool -S.The 7-step close pipeline in the changelog does not mention
ibmveth_update_rx_no_buffer()at all:
ibmveth_close():1.
netif_tx_stop_all_queues()
2.ibmveth_free_tx_resources()
3.ibmveth_cleanup_rx_interrupts()- mask PHYP,napi_disable,free_irq
4.ibmveth_free_all_queues()-H_FREE_LOGICAL_LAN
5.ibmveth_free_buffer_pools()
6.ibmveth_cleanup_rx_resources()
7.ibmveth_free_filter_list()
Quoted 3 lines
Could the step list be updated to include it and to say where it belongs relative to
H_FREE_LOGICAL_LAN? The ordering is unchanged through the end of the series.
Agreed. There are really two separate points there. The NULL-deref side needs fixing, and I'm planning to make that accounting path NULL-safe and queue-local. Separately, if we keep the diagnostic snapshot before H_FREE_LOGICAL_LAN, that should be called out explicitly as intentional "last glimpse" accounting and included in the close step list, rather than left ambiguous in the changelog.
Thanks,
Mingming
Quoted 4 lines
Quoted 3 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0ibmveth_free_buffer_pools (adapter);ibmveth_cleanup_rx_resources (adapter);ibmveth_free_filter_list (adapter);[ ... ]
PATCH v4 8/14 ibmveth: Add queue-aware RX buffer submit helper for MQ
In reply to Mingming CaoReplenish is the last open-path hypervisor call that still needs per-queue awareness before MQ is turned on. Today ibmveth_replenish_buffer_pool() calls h_add_logical_lan_buffer() or h_add_logical_lan_buffers() directly; MQ posts via H_ADD_LOGICAL_LAN_BUFFERS_QUEUE against adapter->queue_handle[].
Add ibmveth_add_logical_lan_buffers() to pick the hcall:multi_queue uses h_add_logical_lan_buffers_queue() (up to 12 buffers, IOBAs packed with odd counts in the upper 32 bits); legacy uses the existing single- and multi-buffer hcalls. Count add_buf/add_bufs/ add_bufs_queue in hcall_stats.
Thread queue_index through the RX helpers used by poll and replenish:
ibmveth_rxq_* accessors ibmveth_remove_buffer_from_pool() ibmveth_rxq_get_buffer() / ibmveth_rxq_harvest_buffer() ibmveth_replenish_buffer_pool() / ibmveth_replenish_task()
and update open/poll/netpoll callers in the same patch so arity stays consistent. Also add per-queue replenish_lock so later concurrent NAPI/resize paths can serialize buffer posting.
Until MQ enablement, callers still pass queue 0 and legacy hcalls remain the live path.
H_FUNCTION handling is split:
- multi_queue: hard-fail and break
- legacy multi-buffer LPM fallback: set rx_buffers_per_hcall = 1 and break so the next replenish_buffer_pool() re-samples batch as 1 (do not continue with a stale local batch)
---
Changes in v4:
- Introduce queue-aware replenish/poll helpers with their first callers in the same patch; do not leave a 2-arg replenish call ahead of the signature change.
- Restore the pre-MQ LPM H_FUNCTION break instead of continue; do not loop forever on a stale local batch size.
- Fold per-queue replenish_lock into this patch.
- Update kdoc for MQ parameters on remove_buffer_from_pool / rxq_harvest_buffer.
drivers/net/ethernet/ibm/ibmveth.c | 341 +++++++++++++++++++---------- drivers/net/ethernet/ibm/ibmveth.h | 7 +- 2 files changed, 237 insertions(+), 111 deletions(-)
drivers/net/ethernet/ibm/ibmveth.c +231 -110
@@ -101,49 +101,58 @@ static struct ibmveth_stat ibmveth_stats[] = { };/* simple methods of getting data from the current rxq entry */ -static inline u32 ibmveth_rxq_flags (struct ibmveth_adapter *adapter) +static inline u32 ibmveth_rxq_flags (struct ibmveth_adapter *adapter, +int queue_index) { -struct ibmveth_rx_q *rxq = &adapter->rx_queue[0 ]; +struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];return be32_to_cpu (rxq->queue_addr[rxq->index].flags_off); } -static inline int ibmveth_rxq_toggle (struct ibmveth_adapter *adapter) +static inline int ibmveth_rxq_toggle (struct ibmveth_adapter *adapter, +int queue_index) { -return (ibmveth_rxq_flags (adapter) &IBMVETH_RXQ_TOGGLE ) >> -IBMVETH_RXQ_TOGGLE_SHIFT ; +return (ibmveth_rxq_flags (adapter, queue_index) &IBMVETH_RXQ_TOGGLE ) >> +IBMVETH_RXQ_TOGGLE_SHIFT ; } -static inline int ibmveth_rxq_pending_buffer (struct ibmveth_adapter *adapter) +static inline int ibmveth_rxq_pending_buffer (struct ibmveth_adapter *adapter, +int queue_index) { -return ibmveth_rxq_toggle (adapter) == adapter->rx_queue[0 ].toggle; +return ibmveth_rxq_toggle (adapter, queue_index) == + adapter->rx_queue[queue_index].toggle; } -static inline int ibmveth_rxq_buffer_valid (struct ibmveth_adapter *adapter) +static inline int ibmveth_rxq_buffer_valid (struct ibmveth_adapter *adapter, +int queue_index) { -return ibmveth_rxq_flags (adapter) &IBMVETH_RXQ_VALID ; +return ibmveth_rxq_flags (adapter, queue_index) &IBMVETH_RXQ_VALID ; } -static inline int ibmveth_rxq_frame_offset (struct ibmveth_adapter *adapter) +static inline int ibmveth_rxq_frame_offset (struct ibmveth_adapter *adapter, +int queue_index) { -return ibmveth_rxq_flags (adapter) &IBMVETH_RXQ_OFF_MASK ; +return ibmveth_rxq_flags (adapter, queue_index) &IBMVETH_RXQ_OFF_MASK ; } -static inline int ibmveth_rxq_large_packet (struct ibmveth_adapter *adapter) +static inline int ibmveth_rxq_large_packet (struct ibmveth_adapter *adapter, +int queue_index) { -return ibmveth_rxq_flags (adapter) &IBMVETH_RXQ_LRG_PKT ; +return ibmveth_rxq_flags (adapter, queue_index) &IBMVETH_RXQ_LRG_PKT ; } -static inline int ibmveth_rxq_frame_length (struct ibmveth_adapter *adapter) +static inline int ibmveth_rxq_frame_length (struct ibmveth_adapter *adapter, +int queue_index) { -struct ibmveth_rx_q *rxq = &adapter->rx_queue[0 ]; +struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];return be32_to_cpu (rxq->queue_addr[rxq->index].length); } -static inline int ibmveth_rxq_csum_good (struct ibmveth_adapter *adapter) +static inline int ibmveth_rxq_csum_good (struct ibmveth_adapter *adapter, +int queue_index) { -return ibmveth_rxq_flags (adapter) &IBMVETH_RXQ_CSUM_GOOD ; +return ibmveth_rxq_flags (adapter, queue_index) &IBMVETH_RXQ_CSUM_GOOD ; }static unsigned int ibmveth_real_max_tx_queues (void )
@@ -260,6 +269,7 @@ ibmveth_alloc_rx_queues(struct ibmveth_adapter *adapter, int rxq_entries) adapter->rx_queue[i].index =0 ; adapter->rx_queue[i].num_slots = rxq_entries; adapter->rx_queue[i].toggle =1 ; +spin_lock_init (&adapter->rx_queue[i].replenish_lock); netdev_dbg(netdev, "queue %d: buffer_list @ 0x%p (DMA: 0x%llx), rx_queue @ 0x%p (DMA: 0x%llx), %llu entries\n", i, adapter->buffer_list_addr[i],
@@ -588,11 +598,75 @@ static inline void ibmveth_flush_buffer(void *addr, unsigned long length) asm("dcbf %0,%1,1" ::"b" (addr),"r" (offset)); } +/** +* ibmveth_add_logical_lan_buffers - Add receive buffers to hypervisor +* @adapter: ibmveth adapter structure +* @descs: array of buffer descriptors to add +* @filled: number of valid descriptors in the array +* @buff_size: size of each buffer (multi-queue mode only) +* @queue_index: RX queue index +* +* Return: hypervisor return code +*/ +static long ibmveth_add_logical_lan_buffers (struct ibmveth_adapter *adapter, +union ibmveth_buf_desc *descs, +int filled, +unsigned long buff_size, +int queue_index) +{ +struct vio_dev *vdev = adapter->vdev; +unsigned long rc; + +if (adapter->multi_queue) { +unsigned long buffersznum = (buff_size <<32 ) | filled; +unsigned long ioba[IBMVETH_MAX_RX_PER_HCALL /2 ] = {0 }; +unsigned long handle = adapter->queue_handle[queue_index]; +int i; + +/* Pack descriptor addresses into ioba pairs. +* Each ioba holds two 32-bit addresses packed into 64 bits: +* - Even descriptors (0,2,4...) go in high 32 bits +* - Odd descriptors (1,3,5...) go in low 32 bits +*/ +for (i =0 ; i < filled && i <IBMVETH_MAX_RX_PER_HCALL ; i++) { +int pair_idx = i /2 ; +int is_high = (i %2 ==0 ); + +if (is_high) + ioba[pair_idx] = (unsigned long ) + descs[i].fields.address <<32 ; +else + ioba[pair_idx] |= descs[i].fields.address; + } + + rc =h_add_logical_lan_buffers_queue (vdev->unit_address, + handle, + buffersznum, + ioba[0 ], ioba[1 ], ioba[2 ], + ioba[3 ], ioba[4 ], ioba[5 ]); + adapter->hcall_stats.add_bufs_queue++; + }else if (filled ==1 ) { + rc =h_add_logical_lan_buffer (vdev->unit_address, + descs[0 ].desc); + adapter->hcall_stats.add_buf++; + }else { + rc =h_add_logical_lan_buffers (vdev->unit_address, + descs[0 ].desc, descs[1 ].desc, + descs[2 ].desc, descs[3 ].desc, + descs[4 ].desc, descs[5 ].desc, + descs[6 ].desc, descs[7 ].desc); + adapter->hcall_stats.add_bufs++; + } + +return rc; +} +/* replenish the buffers for a pool. note that we don't need to * skb_reserve these since they are used for incoming... */ static void ibmveth_replenish_buffer_pool (struct ibmveth_adapter *adapter, -struct ibmveth_buff_pool *pool) +struct ibmveth_buff_pool *pool, +int queue_index) {union ibmveth_buf_desc descs[IBMVETH_MAX_RX_PER_HCALL ] = {0 };u32 remaining = pool->size -atomic_read (&pool->available);
@@ -678,24 +752,16 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,if (!filled)break ; -/* single buffer case*/ -if (filled ==1 ) - lpar_rc =h_add_logical_lan_buffer (vdev->unit_address, - descs[0 ].desc); -else -/* Multi-buffer hcall */ - lpar_rc =h_add_logical_lan_buffers (vdev->unit_address, - descs[0 ].desc, - descs[1 ].desc, - descs[2 ].desc, - descs[3 ].desc, - descs[4 ].desc, - descs[5 ].desc, - descs[6 ].desc, - descs[7 ].desc); + lpar_rc =ibmveth_add_logical_lan_buffers (adapter, descs, + filled, + pool->buff_size, + queue_index); +if (lpar_rc !=H_SUCCESS ) {dev_warn_ratelimited (dev, -"RX h_add_logical_lan failed: filled=%u, rc=%lu, batch=%u\n" , +"RX h_add_logical_lan %s failed: filled=%u, rc=%lu, batch=%u\n" , + adapter->multi_queue ? +"_queue" :"" , filled, lpar_rc, batch); goto hcall_failure; }
@@ -736,24 +802,23 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter, } adapter->replenish_add_buff_failure += filled; -/* -* If multi rx buffers hcall is no longer supported by FW -* e.g. in the case of Live Partition Migration -*/ -if (batch >1 && lpar_rc ==H_FUNCTION ) { -/* -* Instead of retry submit single buffer individually -* here just set the max rx buffer per hcall to 1 -* buffers will be respleshed next time -* when ibmveth_replenish_buffer_pool() is called again -* with single-buffer case -*/ -netdev_info (adapter->netdev, -"RX Multi buffers not supported by FW, rc=%lu\n" , - lpar_rc); - adapter->rx_buffers_per_hcall =1 ; -netdev_info (adapter->netdev, -"Next rx replesh will fall back to single-buffer hcall\n" ); +if (lpar_rc ==H_FUNCTION ) { +if (adapter->multi_queue) { +netdev_err (adapter->netdev, +"MQ buffer add H_FUNCTION (q=%d, batch=%d)\n" , + queue_index, batch); + }else if (batch >1 ) { +/* +* Live Partition Migration may drop multi- +* buffer support. Fall back to single-buffer +* on the next replenish; do not continue with +* a stale local batch size (infinite loop). +*/ +netdev_warn (adapter->netdev, +"Legacy batch add H_FUNCTION (batch=%d), fallback\n" , + batch); + adapter->rx_buffers_per_hcall =1 ; + } }break ; }
@@ -775,21 +840,33 @@ static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter) }/* replenish routine */ -static void ibmveth_replenish_task (struct ibmveth_adapter *adapter) +static void ibmveth_replenish_task (struct ibmveth_adapter *adapter, +int queue_index) { +struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index]; +unsigned long flags;int i; +if (queue_index >= adapter->num_rx_queues) +return ; + adapter->replenish_task_cycles++; +spin_lock_irqsave (&rxq->replenish_lock, flags); +for (i = (IBMVETH_NUM_BUFF_POOLS -1 ); i >=0 ; i--) { -struct ibmveth_buff_pool *pool = &adapter->rx_buff_pool[0 ][i]; +struct ibmveth_buff_pool *pool = + &adapter->rx_buff_pool[queue_index][i];if (pool->active && (atomic_read (&pool->available) < pool->threshold)) -ibmveth_replenish_buffer_pool (adapter, pool); +ibmveth_replenish_buffer_pool (adapter, pool, + queue_index); }ibmveth_update_rx_no_buffer (adapter); + +spin_unlock_irqrestore (&rxq->replenish_lock, flags); }/* empty and free ana buffer pool - also used to do cleanup in error paths */
@@ -969,6 +1046,7 @@ ibmveth_free_buffer_pools(struct ibmveth_adapter *adapter)* ibmveth_remove_buffer_from_pool - remove a buffer from a pool * @adapter: adapter instance * @correlator: identifies pool and index +* @queue_index: RX queue index (0..num_rx_queues-1) * @reuse: whether to reuse buffer * * Return:
@@ -977,7 +1055,8 @@ ibmveth_free_buffer_pools(struct ibmveth_adapter *adapter)* * %-EFAULT - pool and index map to null skb */ static int ibmveth_remove_buffer_from_pool (struct ibmveth_adapter *adapter, -u64 correlator,bool reuse) +u64 correlator,int queue_index, +bool reuse) {unsigned int pool = correlator >>32 ;unsigned int index = correlator &0xffffffffUL ;
@@ -985,12 +1064,12 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,struct sk_buff *skb;if (WARN_ON (pool >=IBMVETH_NUM_BUFF_POOLS ) || -WARN_ON (index >= adapter->rx_buff_pool[0 ][pool].size)) { +WARN_ON (index >= adapter->rx_buff_pool[queue_index][pool].size)) {schedule_work (&adapter->work);return -EINVAL ; } - skb = adapter->rx_buff_pool[0 ][pool].skbuff[index]; + skb = adapter->rx_buff_pool[queue_index][pool].skbuff[index];if (WARN_ON (!skb)) {schedule_work (&adapter->work);return -EFAULT ;
@@ -1004,50 +1083,55 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,/* remove the skb pointer to mark free. actual freeing is done * by upper level networking after gro_receive */ - adapter->rx_buff_pool[0 ][pool].skbuff[index] =NULL ; +struct ibmveth_buff_pool *bpool = + &adapter->rx_buff_pool[queue_index][pool]; + + bpool->skbuff[index] =NULL ;dma_unmap_single (&adapter->vdev->dev, - adapter->rx_buff_pool[0 ][pool].dma_addr[index], - adapter->rx_buff_pool[0 ][pool].buff_size, + bpool->dma_addr[index], + bpool->buff_size,DMA_FROM_DEVICE ); } - free_index = adapter->rx_buff_pool[0 ][pool].producer_index; - adapter->rx_buff_pool[0 ][pool].producer_index++; -if (adapter->rx_buff_pool[0 ][pool].producer_index >= - adapter->rx_buff_pool[0 ][pool].size) - adapter->rx_buff_pool[0 ][pool].producer_index =0 ; - adapter->rx_buff_pool[0 ][pool].free_map[free_index] = index; + free_index = adapter->rx_buff_pool[queue_index][pool].producer_index; + adapter->rx_buff_pool[queue_index][pool].producer_index++; +if (adapter->rx_buff_pool[queue_index][pool].producer_index >= + adapter->rx_buff_pool[queue_index][pool].size) + adapter->rx_buff_pool[queue_index][pool].producer_index =0 ; + adapter->rx_buff_pool[queue_index][pool].free_map[free_index] = index;mb (); -atomic_dec (&adapter->rx_buff_pool[0 ][pool].available); +atomic_dec (&adapter->rx_buff_pool[queue_index][pool].available);return 0 ; }/* get the current buffer on the rx queue */ static inline struct sk_buff * -ibmveth_rxq_get_buffer (struct ibmveth_adapter *adapter) +ibmveth_rxq_get_buffer (struct ibmveth_adapter *adapter, +int queue_index) { -struct ibmveth_rx_q *rxq = &adapter->rx_queue[0 ]; +struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];u64 correlator = rxq->queue_addr[rxq->index].correlator;unsigned int pool = correlator >>32 ;unsigned int index = correlator &0xffffffffUL ;if (WARN_ON (pool >=IBMVETH_NUM_BUFF_POOLS ) || -WARN_ON (index >= adapter->rx_buff_pool[0 ][pool].size)) { +WARN_ON (index >= adapter->rx_buff_pool[queue_index][pool].size)) {schedule_work (&adapter->work);return NULL ; } -return adapter->rx_buff_pool[0 ][pool].skbuff[index]; +return adapter->rx_buff_pool[queue_index][pool].skbuff[index]; }/** * ibmveth_rxq_harvest_buffer - Harvest buffer from pool * * @adapter: pointer to adapter +* @queue_index: RX queue index to harvest from * @reuse: whether to reuse buffer * * Context: called from ibmveth_poll
@@ -1057,21 +1141,20 @@ ibmveth_rxq_get_buffer(struct ibmveth_adapter *adapter)* * other - non-zero return from ibmveth_remove_buffer_from_pool */ static int ibmveth_rxq_harvest_buffer (struct ibmveth_adapter *adapter, -bool reuse) +int queue_index,bool reuse) { +struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];u64 cor;int rc; -struct ibmveth_rx_q *rxq = &adapter->rx_queue[0 ]; - cor = rxq->queue_addr[rxq->index].correlator; - rc =ibmveth_remove_buffer_from_pool (adapter, cor, reuse); + rc =ibmveth_remove_buffer_from_pool (adapter, cor, queue_index, reuse);if (unlikely (rc))return rc; -if (++adapter->rx_queue[0 ].index == adapter->rx_queue[0 ].num_slots) { - adapter->rx_queue[0 ].index =0 ; - adapter->rx_queue[0 ].toggle = !adapter->rx_queue[0 ].toggle; +if (++rxq->index == rxq->num_slots) { + rxq->index =0 ; + rxq->toggle = !rxq->toggle; }return 0 ;
@@ -1993,34 +2076,45 @@ static void ibmveth_rx_csum_helper(struct sk_buff *skb,static int ibmveth_poll (struct napi_struct *napi,int budget) { -struct ibmveth_adapter *adapter = - container_of(napi, struct ibmveth_adapter, napi[0]); -struct net_device *netdev = adapter->netdev; +struct net_device *netdev = napi->dev; +struct ibmveth_adapter *adapter =netdev_priv (netdev);int frames_processed =0 ;unsigned long lpar_rc; +int queue_index, rc;u16 mss =0 ; + queue_index = napi - adapter->napi; + +if (WARN_ON (queue_index <0 || queue_index >= adapter->num_rx_queues)) +return 0 ; + restart_poll:while (frames_processed < budget) { -if (!ibmveth_rxq_pending_buffer (adapter)) +if (!ibmveth_rxq_pending_buffer (adapter, queue_index))break ;smp_rmb (); -if (!ibmveth_rxq_buffer_valid (adapter)) { +if (!ibmveth_rxq_buffer_valid (adapter, queue_index)) {wmb ();/* suggested by larson1 */ adapter->rx_invalid_buffer++;netdev_dbg (netdev,"recycling invalid buffer\n" ); -if (unlikely (ibmveth_rxq_harvest_buffer (adapter, true))) + rc =ibmveth_rxq_harvest_buffer (adapter, + queue_index, true); +if (unlikely (rc))break ; }else {struct sk_buff *skb, *new_skb; -int length =ibmveth_rxq_frame_length (adapter); -int offset =ibmveth_rxq_frame_offset (adapter); -int csum_good =ibmveth_rxq_csum_good (adapter); -int lrg_pkt =ibmveth_rxq_large_packet (adapter); +int length =ibmveth_rxq_frame_length (adapter, + queue_index); +int offset =ibmveth_rxq_frame_offset (adapter, + queue_index); +int csum_good =ibmveth_rxq_csum_good (adapter, + queue_index); +int lrg_pkt =ibmveth_rxq_large_packet (adapter, + queue_index);__sum16 iph_check =0 ; - skb =ibmveth_rxq_get_buffer (adapter); + skb =ibmveth_rxq_get_buffer (adapter, queue_index);if (unlikely (!skb))break ;
@@ -2045,12 +2139,18 @@ static int ibmveth_poll(struct napi_struct *napi, int budget) length);if (rx_flush)ibmveth_flush_buffer (skb->data, - length + offset); -if (unlikely (ibmveth_rxq_harvest_buffer (adapter, true))) + length + offset); + rc =ibmveth_rxq_harvest_buffer (adapter, + queue_index, + true); +if (unlikely (rc))break ; skb = new_skb; }else { -if (unlikely (ibmveth_rxq_harvest_buffer (adapter, false))) + rc =ibmveth_rxq_harvest_buffer (adapter, + queue_index, + false); +if (unlikely (rc))break ;skb_reserve (skb, offset); }
@@ -2086,7 +2186,7 @@ static int ibmveth_poll(struct napi_struct *napi, int budget) } } -ibmveth_replenish_task (adapter); +ibmveth_replenish_task (adapter, queue_index);if (frames_processed == budget) goto out;
@@ -2097,14 +2197,19 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)/* We think we are done - reenable interrupts, * then check once more to make sure we are done. */ - lpar_rc =ibmveth_enable_irq (adapter,0 ); -if (WARN_ON (lpar_rc !=H_SUCCESS )) { + lpar_rc =ibmveth_enable_irq (adapter, queue_index); +if (lpar_rc !=H_SUCCESS ) { +netdev_err (netdev, +"Failed to enable IRQ for queue %d (rc=0x%lx), scheduling reset\n" , + queue_index, lpar_rc);schedule_work (&adapter->work); goto out; } -if (ibmveth_rxq_pending_buffer (adapter) &&napi_schedule (napi)) { - lpar_rc =ibmveth_disable_irq (adapter,0 ); +if (ibmveth_rxq_pending_buffer (adapter, queue_index) && +napi_schedule (napi)) { + lpar_rc =ibmveth_disable_irq (adapter, queue_index); +WARN_ON (lpar_rc !=H_SUCCESS ); goto restart_poll; }
@@ -2229,9 +2334,13 @@ static int ibmveth_change_mtu(struct net_device *dev, int new_mtu)static void ibmveth_poll_controller (struct net_device *dev) {struct ibmveth_adapter *adapter =netdev_priv (dev); +int i; -ibmveth_replenish_task (adapter); -ibmveth_schedule_rx_queue (adapter,0 ); +for (i =0 ; i < adapter->num_rx_queues; i++) +ibmveth_replenish_task (adapter, i); + +for (i =0 ; i < adapter->num_rx_queues; i++) +ibmveth_schedule_rx_queue (adapter, i); }#endif
@@ -2428,7 +2537,7 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)if (ret ==H_SUCCESS && (ret_attr &IBMVETH_ILLAN_RX_MULTI_BUFF_SUPPORT )) { - adapter->rx_buffers_per_hcall =IBMVETH_MAX_RX_PER_HCALL ; + adapter->rx_buffers_per_hcall =IBMVETH_MAX_RX_REGULAR ;netdev_dbg (netdev,"RX Multi-buffer hcall supported by FW, batch set to %u\n" , adapter->rx_buffers_per_hcall);
@@ -2775,17 +2884,29 @@ static void ibmveth_remove_buffer_from_pool_test(struct kunit *test)KUNIT_ASSERT_NOT_ERR_OR_NULL (test, pool->skbuff); correlator = ((u64 )IBMVETH_NUM_BUFF_POOLS <<32 ) |0 ; -KUNIT_EXPECT_EQ (test, -EINVAL ,ibmveth_remove_buffer_from_pool (adapter, correlator, false)); -KUNIT_EXPECT_EQ (test, -EINVAL ,ibmveth_remove_buffer_from_pool (adapter, correlator, true)); +KUNIT_EXPECT_EQ (test, -EINVAL , +ibmveth_remove_buffer_from_pool (adapter, + correlator,0 , false)); +KUNIT_EXPECT_EQ (test, -EINVAL , +ibmveth_remove_buffer_from_pool (adapter, + correlator,0 , true)); correlator = ((u64 )0 <<32 ) | adapter->rx_buff_pool[0 ][0 ].size; -KUNIT_EXPECT_EQ (test, -EINVAL ,ibmveth_remove_buffer_from_pool (adapter, correlator, false)); -KUNIT_EXPECT_EQ (test, -EINVAL ,ibmveth_remove_buffer_from_pool (adapter, correlator, true)); +KUNIT_EXPECT_EQ (test, -EINVAL , +ibmveth_remove_buffer_from_pool (adapter, + correlator,0 , false)); +KUNIT_EXPECT_EQ (test, -EINVAL , +ibmveth_remove_buffer_from_pool (adapter, + correlator,0 , true)); correlator = (u64 )0 |0 ; pool->skbuff[0 ] =NULL ; -KUNIT_EXPECT_EQ (test, -EFAULT ,ibmveth_remove_buffer_from_pool (adapter, correlator, false)); -KUNIT_EXPECT_EQ (test, -EFAULT ,ibmveth_remove_buffer_from_pool (adapter, correlator, true)); +KUNIT_EXPECT_EQ (test, -EFAULT , +ibmveth_remove_buffer_from_pool (adapter, + correlator,0 , false)); +KUNIT_EXPECT_EQ (test, -EFAULT , +ibmveth_remove_buffer_from_pool (adapter, + correlator,0 , true));flush_work (&adapter->work); }
@@ -2830,15 +2951,15 @@ static void ibmveth_rxq_get_buffer_test(struct kunit *test) adapter->rx_queue[0 ].queue_addr[0 ].correlator = (u64 )IBMVETH_NUM_BUFF_POOLS <<32 |0 ; -KUNIT_EXPECT_PTR_EQ (test,NULL ,ibmveth_rxq_get_buffer (adapter)); +KUNIT_EXPECT_PTR_EQ (test,NULL ,ibmveth_rxq_get_buffer (adapter,0 )); adapter->rx_queue[0 ].queue_addr[0 ].correlator = (u64 )0 <<32 | adapter->rx_buff_pool[0 ][0 ].size; -KUNIT_EXPECT_PTR_EQ (test,NULL ,ibmveth_rxq_get_buffer (adapter)); +KUNIT_EXPECT_PTR_EQ (test,NULL ,ibmveth_rxq_get_buffer (adapter,0 )); pool->skbuff[0 ] = skb; adapter->rx_queue[0 ].queue_addr[0 ].correlator = (u64 )0 <<32 |0 ; -KUNIT_EXPECT_PTR_EQ (test, skb,ibmveth_rxq_get_buffer (adapter)); +KUNIT_EXPECT_PTR_EQ (test, skb,ibmveth_rxq_get_buffer (adapter,0 ));flush_work (&adapter->work); }
drivers/net/ethernet/ibm/ibmveth.h +6 -1
@@ -14,6 +14,8 @@#ifndef _IBMVETH_H#define _IBMVETH_H +#include <linux/spinlock_types.h> +/* constants for H_MULTICAST_CTRL */ #define IbmVethMcastReceptionModifyBit 0x80000UL#define IbmVethMcastReceptionEnableBit 0x20000UL
@@ -260,7 +262,9 @@ static inline long h_illan_attributes(unsigned long unit_address,#define IBMVETH_DEFAULT_QUEUES 8U#define IBMVETH_MAX_RX_QUEUES 1U#define IBMVETH_DEFAULT_RX_QUEUES 1U -#define IBMVETH_MAX_RX_PER_HCALL 8U +#define IBMVETH_MAX_RX_REGULAR 8U +#define IBMVETH_MAX_RX_QUEUE 12U +#define IBMVETH_MAX_RX_PER_HCALL 12Ustatic int pool_size[] = {512 ,1024 *2 ,1024 *16 ,1024 *32 ,1024 *64 };static int pool_count[] = {256 ,512 ,256 ,256 ,256 };
@@ -302,6 +306,7 @@ struct ibmveth_rx_q {dma_addr_t queue_dma;u32 queue_len;struct ibmveth_rx_q_entry *queue_addr; +spinlock_t replenish_lock;/* per-queue buffer replenish */ };struct ibmveth_adapter {
Signature · 1 line
2.50.1 (Apple Git-155)
Jakub Kicinski
maintainerRe: [PATCH net-next v4 08/14] ibmveth: Add queue-aware RX buffer submit helper for MQ
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
ibmveth: Add queue-aware RX buffer submit helper for MQ
This adds ibmveth_add_logical_lan_buffers() to choose between the legacy single/multi-buffer hcalls and the new per-queue buffer hcall, threads a queue_index argument through the RX accessors, harvest and replenish helpers, and adds a per-queue replenish_lock plus reworked H_FUNCTION handling.
A few questions below about the new failure handling, the new lock and the logging changes.
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0
[ ... ]
Quoted 24 lines
drivers/net/ethernet/ibm/ibmveth.c +20
-0@@ -588,11 +598,75 @@ static inline void ibmveth_flush_buffer(void *addr, unsigned long length) asm("dcbf %0,%1,1" ::"b" (addr),"r" (offset)); } +/** +* ibmveth_add_logical_lan_buffers - Add receive buffers to hypervisor +* @adapter: ibmveth adapter structure +* @descs: array of buffer descriptors to add +* @filled: number of valid descriptors in the array +* @buff_size: size of each buffer (multi-queue mode only) +* @queue_index: RX queue index +* +* Return: hypervisor return code +*/ +static long ibmveth_add_logical_lan_buffers (struct ibmveth_adapter *adapter, +union ibmveth_buf_desc *descs, +int filled, +unsigned long buff_size, +int queue_index) +{ +struct vio_dev *vdev = adapter->vdev; +unsigned long rc; + +if (adapter->multi_queue) {
[ ... ]
Quoted 8 lines
drivers/net/ethernet/ibm/ibmveth.c +8
-0+ rc =h_add_logical_lan_buffers_queue (vdev->unit_address, + handle, + buffersznum, + ioba[0 ], ioba[1 ], ioba[2 ], + ioba[3 ], ioba[4 ], ioba[5 ]); + adapter->hcall_stats.add_bufs_queue++; + }else if (filled ==1 ) { + rc = h_add_logical_lan_buffer(vdev->unit_address,
drivers/net/ethernet/ibm/ibmveth.c static inline void ibmveth_flush_buffer(void *addr, unsigned long length) · line 659
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+ descs[0].desc); + adapter->hcall_stats.add_buf++; + }else { + rc =h_add_logical_lan_buffers (vdev->unit_address, + descs[0 ].desc, descs[1 ].desc, + descs[2 ].desc, descs[3 ].desc, + descs[4 ].desc, descs[5 ].desc, + descs[6 ].desc, descs[7 ].desc); + adapter->hcall_stats.add_bufs++; + }
[Severity: Medium]
These hcall_stats fields are plain non-atomic u64 counters on the adapter, but the only serialization this patch adds is the per-queue rx_queue[i].replenish_lock taken in ibmveth_replenish_task().
Two NAPI instances replenishing different queues hold different locks, so can these increments lose counts once MQ is enabled later in the series and num_rx_queues becomes larger than 1?
The same question applies to the other adapter-wide counters reached from this path:
adapter->replenish_task_cycles++; adapter->replenish_add_buff_success += filled; adapter->replenish_add_buff_failure += filled;
and to ibmveth_update_rx_no_buffer(), which ibmveth_replenish_task() calls while holding queue N's lock only, although it rewrites adapter-global state for every queue:
adapter->rx_no_buffer = 0;
for (i = 0; i < adapter->num_rx_queues; i++) {
...
adapter->rx_qstats[i].no_buffer_drops = drops;
adapter->rx_no_buffer += drops;
}Would rx_no_buffer and rx_qstats[].no_buffer_drops be clobbered when two queues replenish concurrently? ibmveth_close() calls the same helper with no lock held at all.
Quoted 11 lines
/* replenish the buffers for a pool. note that we don't need to
*skb_reservethese since they are used for incoming...
*/
static void ibmveth_replenish_buffer_pool(structibmveth_adapter*adapter,drivers/net/ethernet/ibm/ibmveth.c +2
-1- struct ibmveth_buff_pool *pool) + struct ibmveth_buff_pool *pool, + int queue_index) {union ibmveth_buf_desc descs[IBMVETH_MAX_RX_PER_HCALL ] = {0 };u32 remaining = pool->size -atomic_read (&pool->available);@@ -678,24 +752,16 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
[ ... ]
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-0+ lpar_rc =ibmveth_add_logical_lan_buffers (adapter, descs, + filled, + pool->buff_size, + queue_index);
drivers/net/ethernet/ibm/ibmveth.c static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter, · line 767
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-1+if (lpar_rc !=H_SUCCESS ) {dev_warn_ratelimited (dev, -"RX h_add_logical_lan failed: filled=%u, rc=%lu, batch=%u\n" , +"RX h_add_logical_lan %s failed: filled=%u, rc=%lu, batch=%u\n" , + adapter->multi_queue ? +"_queue" :"" , filled, lpar_rc, batch); goto hcall_failure; }
[Severity: Low]
The interpolated hcall name never matches an actual symbol. In MQ mode this prints
RX h_add_logical_lan _queue failed: ...
with a stray space, and in legacy mode
RX h_add_logical_lan failed: ...
with a double space. Would a full literal per branch read better?
The two new H_FUNCTION messages added below also print batch with %d, although batch is declared u32 in this function ("u32 i, filled, batch;"), while this warning uses %u for the same variable.
Quoted 18 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-14@@ -736,24 +802,23 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter, } adapter->replenish_add_buff_failure += filled; -/* -* If multi rx buffers hcall is no longer supported by FW -* e.g. in the case of Live Partition Migration -*/ -if (batch >1 && lpar_rc ==H_FUNCTION ) { -/* -* Instead of retry submit single buffer individually -* here just set the max rx buffer per hcall to 1 -* buffers will be respleshed next time -* when ibmveth_replenish_buffer_pool() is called again -* with single-buffer case -*/ - netdev_info(adapter->netdev, - "RX Multi buffers not supported by FW, rc=%lu\n",
drivers/net/ethernet/ibm/ibmveth.c static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter, · line 810
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +6
-4- lpar_rc); - adapter->rx_buffers_per_hcall =1 ; -netdev_info (adapter->netdev, -"Next rx replesh will fall back to single-buffer hcall\n" ); +if (lpar_rc ==H_FUNCTION ) { +if (adapter->multi_queue) { +netdev_err (adapter->netdev, +"MQ buffer add H_FUNCTION (q=%d, batch=%d)\n" , + queue_index, batch); + }else if (batch >1 ) {
[Severity: High]
The multi_queue arm logs and breaks without changing any state:adapter->multi_queue stays set, rx_buffers_per_hcall is untouched, there is no fallback to h_add_logical_lan_buffer()/h_add_logical_lan_buffers(), and no schedule_work(&adapter->work).
ibmveth_add_logical_lan_buffers() picks the hcall purely on adapter->multi_queue, and that field is only written in ibmveth_probe(), which is not re-run on resume (ibmveth_resume() only calls ibmveth_schedule_rx_queue()).
So once the hypervisor stops implementing H_ADD_LOGICAL_LAN_BUFFERS_QUEUE
- exactly the Live Partition Migration case the legacy arm below exists for
- can this queue ever post another RX buffer again? pool->available stays at 0, PHYP drops every inbound frame, and the interface still reports the link up, so recovery would need an administrative down/up.
Every other unrecoverable condition in this driver schedules the reset work (ibmveth_remove_buffer_from_pool(), ibmveth_rxq_get_buffer(), and the new enable_irq failure path in ibmveth_poll()). Should this arm do the same, or fall back to the legacy hcalls the way the legacy arm downgrades batch?
Separately, netdev_err() here is not rate limited, while the warning for the same failure a few lines above uses dev_warn_ratelimited(), so a dried-out pool can emit several console lines per poll cycle per pool.
This arm appears unchanged through the end of the series.
Quoted 39 lines
drivers/net/ethernet/ibm/ibmveth.c +22
-2+/* +* Live Partition Migration may drop multi- +* buffer support. Fall back to single-buffer +* on the next replenish; do not continue with +* a stale local batch size (infinite loop). +*/ +netdev_warn (adapter->netdev, +"Legacy batch add H_FUNCTION (batch=%d), fallback\n" , + batch); + adapter->rx_buffers_per_hcall =1 ; + } }break ; }@@ -775,21 +840,33 @@ static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter) }/* replenish routine */ -static void ibmveth_replenish_task (struct ibmveth_adapter *adapter) +static void ibmveth_replenish_task (struct ibmveth_adapter *adapter, +int queue_index) { +struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index]; +unsigned long flags;int i; +if (queue_index >= adapter->num_rx_queues) +return ; + adapter->replenish_task_cycles++; +spin_lock_irqsave (&rxq->replenish_lock, flags); +for (i = (IBMVETH_NUM_BUFF_POOLS -1 ); i >=0 ; i--) { -struct ibmveth_buff_pool *pool = &adapter->rx_buff_pool[0 ][i]; +struct ibmveth_buff_pool *pool = + &adapter->rx_buff_pool[queue_index][i]; if (pool->active &&
drivers/net/ethernet/ibm/ibmveth.c static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter) · line 870
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-1(atomic_read(&pool->available) < pool->threshold)) -ibmveth_replenish_buffer_pool (adapter, pool); +ibmveth_replenish_buffer_pool (adapter, pool, + queue_index); }ibmveth_update_rx_no_buffer (adapter); + +spin_unlock_irqrestore (&rxq->replenish_lock, flags); }
[Severity: High]
Can this lock deadlock against itself when netconsole runs over this ibmveth device?
ibmveth_replenish_buffer_pool() printk()s from inside the locked region:netdev_info() on an invalid map index, dev_warn_ratelimited() on hcall failure, and the two new H_FUNCTION messages. With netconsole on this interface a printk from there re-enters the driver:
netconsole write_msg()
netpoll_send_udp()
netpoll_send_skb()
__netpoll_send_skb()
netpoll_poll_dev()
ops->ndo_poll_controller() -> ibmveth_poll_controller()
ibmveth_replenish_task(adapter, i)__netpoll_send_skb() calls netpoll_poll_dev() whenever the xmit does not go through on the first try, and netpoll_poll_dev() only declines when down_trylock(&ni->dev_lock) fails, when !netif_running(dev), or when netif_local_xmit_active(dev):
if (!ni || down_trylock(&ni->dev_lock))
return;
if (!netif_running(dev) || netif_local_xmit_active(dev)) {
up(&ni->dev_lock);
return;
}
...
if (ops->ndo_poll_controller)
ops->ndo_poll_controller(dev);None of those covers the replenish path, so ibmveth_poll_controller() loops ibmveth_replenish_task() over every queue and re-takes the same replenish_lock on the same CPU, with interrupts already disabled.
Before this patch ibmveth_poll_controller() also called ibmveth_replenish_task(), but with no lock the recursion was harmless.
This code appears unchanged through the end of the series.
[Severity: Medium]
Does this need to be spin_lock_irqsave()? The lock now brackets the whole multi-pool loop, so every netdev_alloc_skb(), every dma_map_single() and every H_ADD_LOGICAL_LAN_BUFFER[S][_QUEUE] hypervisor call for all five pools runs with local interrupts hard-disabled.
On a cold fill from ibmveth_open() or the first NAPI poll that is roughly 768 allocations and DMA maps and around 96 hcalls per queue at the default MTU (pool_count[] = { 256, 512, 256, 256, 256 }), and more when the large pools are active.
None of the protected state is written from hardirq context, and the commit message says the lock exists only "so later concurrent NAPI/resize paths can serialize buffer posting", so would spin_lock_bh(), dropping and re-acquiring per batch, or preparing buffers outside the critical section be enough?
Quoted 2 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0/* empty and free ana buffer pool - also used to do cleanup in error paths */
[ ... ]
Quoted 1 line
drivers/net/ethernet/ibm/ibmveth.c +0
-0@@ -1004,50 +1083,55 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,
[ ... ]
Quoted 7 lines
drivers/net/ethernet/ibm/ibmveth.c +1
-6- free_index = adapter->rx_buff_pool[0 ][pool].producer_index; - adapter->rx_buff_pool[0 ][pool].producer_index++; -if (adapter->rx_buff_pool[0 ][pool].producer_index >= - adapter->rx_buff_pool[0 ][pool].size) - adapter->rx_buff_pool[0 ][pool].producer_index =0 ; - adapter->rx_buff_pool[0 ][pool].free_map[free_index] = index; + free_index = adapter->rx_buff_pool[queue_index][pool].producer_index;
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter, · line 1106
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +6
-1+ adapter->rx_buff_pool[queue_index][pool].producer_index++; +if (adapter->rx_buff_pool[queue_index][pool].producer_index >= + adapter->rx_buff_pool[queue_index][pool].size) + adapter->rx_buff_pool[queue_index][pool].producer_index =0 ; + adapter->rx_buff_pool[queue_index][pool].free_map[free_index] = index;mb (); -atomic_dec (&adapter->rx_buff_pool[0 ][pool].available); +atomic_dec (&adapter->rx_buff_pool[queue_index][pool].available);
[Severity: High]
This isn't a bug introduced by this patch, but the new replenish_lock only covers the producer half of this bookkeeping, so it is worth asking whether the consumer half is still unprotected.
ibmveth_remove_buffer_from_pool() mutates free_map[], producer_index, skbuff[], dma_addr[] and available for the same pool that ibmveth_replenish_buffer_pool() mutates under replenish_lock, but takes no lock:
producer: ibmveth_poll_controller()
ibmveth_replenish_task() /* holds replenish_lock */
ibmveth_replenish_buffer_pool()
consumer: ibmveth_poll() /* holds nothing */
ibmveth_rxq_harvest_buffer()
ibmveth_remove_buffer_from_pool()
netpoll_poll_dev() serializes netpoll users against each other via ni->dev_lock, and __napi_poll()'s netpoll_poll_lock() only stops netpoll from re-polling a NAPI instance that is already running; neither prevents ndo_poll_controller from running on one CPU while the same queue's NAPI poll runs on another.
Can free_map[]/producer_index/consumer_index then hand the same pool index out twice, so a buffer is posted to the hypervisor while its skb has already gone up the stack, or lose a free index permanently?
Since the future work described in the commit message ("so later concurrent NAPI/resize paths can serialize buffer posting") builds on this lock, should the consumer side take it as well?
[ ... ]
Quoted 19 lines
drivers/net/ethernet/ibm/ibmveth.c +8
-3@@ -1993,34 +2076,45 @@ static void ibmveth_rx_csum_helper(struct sk_buff *skb,static int ibmveth_poll (struct napi_struct *napi,int budget) { -struct ibmveth_adapter *adapter = - container_of(napi, struct ibmveth_adapter, napi[0]); -struct net_device *netdev = adapter->netdev; +struct net_device *netdev = napi->dev; +struct ibmveth_adapter *adapter =netdev_priv (netdev);int frames_processed =0 ;unsigned long lpar_rc; +int queue_index, rc;u16 mss =0 ; + queue_index = napi - adapter->napi; + +if (WARN_ON (queue_index <0 || queue_index >= adapter->num_rx_queues)) +return 0 ; +
[ ... ]
Quoted 13 lines
drivers/net/ethernet/ibm/ibmveth.c +5
-2@@ -2097,14 +2197,19 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)/* We think we are done - reenable interrupts, * then check once more to make sure we are done. */ - lpar_rc =ibmveth_enable_irq (adapter,0 ); -if (WARN_ON (lpar_rc !=H_SUCCESS )) { + lpar_rc =ibmveth_enable_irq (adapter, queue_index); +if (lpar_rc !=H_SUCCESS ) { +netdev_err (netdev, +"Failed to enable IRQ for queue %d (rc=0x%lx), scheduling reset\n" , + queue_index, lpar_rc);schedule_work (&adapter->work); goto out;
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_poll(struct napi_struct *napi, int budget) · line 2214
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-2} -if (ibmveth_rxq_pending_buffer (adapter) &&napi_schedule (napi)) { - lpar_rc =ibmveth_disable_irq (adapter,0 ); +if (ibmveth_rxq_pending_buffer (adapter, queue_index) && +napi_schedule (napi)) { + lpar_rc =ibmveth_disable_irq (adapter, queue_index); +WARN_ON (lpar_rc !=H_SUCCESS ); goto restart_poll; }
[Severity: Low]
Two diagnostic changes here are not described in the commit message, which only says callers were updated "so arity stays consistent": the WARN_ON() around the ibmveth_enable_irq() failure is replaced by an unconditional netdev_err(), which drops the taint and backtrace that test harnesses key on, and a new WARN_ON() is added on the ibmveth_disable_irq() result. Would it help to mention or split out that change?
Also, the new guard added at the top of ibmveth_poll()
if (WARN_ON(queue_index < 0 || queue_index >= adapter->num_rx_queues)) return 0;
returns less than budget from a NAPI poll callback without calling napi_complete_done(), which leaves the instance scheduled with the hypervisor IRQ still masked. The guard is unreachable at this commit, since IBMVETH_MAX_RX_QUEUES is 1 and queue_index is always 0, but the same early return is still there at the end of the series. Should it complete NAPI before returning?
[ ... ]
Quoted 15 lines
drivers/net/ethernet/ibm/ibmveth.c +6
-2@@ -2229,9 +2334,13 @@ static int ibmveth_change_mtu(struct net_device *dev, int new_mtu)static void ibmveth_poll_controller (struct net_device *dev) {struct ibmveth_adapter *adapter =netdev_priv (dev); +int i; -ibmveth_replenish_task (adapter); -ibmveth_schedule_rx_queue (adapter,0 ); +for (i =0 ; i < adapter->num_rx_queues; i++) +ibmveth_replenish_task (adapter, i); + +for (i =0 ; i < adapter->num_rx_queues; i++) +ibmveth_schedule_rx_queue (adapter, i); }#endif
[ ... ]
Re: [PATCH net-next v4 08/14] ibmveth: Add queue-aware RX buffer submit helper for MQ
Quoted 5 lines · Jakub Kicinski
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
ibmveth: Add queue-aware RX buffer submit helper for MQ
Quoted 8 lines
This adds
ibmveth_add_logical_lan_buffers()to choose between the legacy single/multi-buffer hcalls and the new per-queue buffer hcall, threads aqueue_indexargument through the RX accessors, harvest and replenish helpers, and adds a per-queuereplenish_lockplus reworkedH_FUNCTIONhandling.A few questions below about the new failure handling, the new lock and the logging changes.
Hi Jakub,
Thanks for the review.
This patch introduces more than helper reshaping. It establishes the replenish locking and error-handling model that later MQ RX behavior depends on.
Quoted 78 lines
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0[ ... ]
Quoted 24 lines
drivers/net/ethernet/ibm/ibmveth.c +20
-0@@ -588,11 +598,75 @@ static inline void ibmveth_flush_buffer(void *addr, unsigned long length) asm("dcbf %0,%1,1" ::"b" (addr),"r" (offset)); } +/** +* ibmveth_add_logical_lan_buffers - Add receive buffers to hypervisor +* @adapter: ibmveth adapter structure +* @descs: array of buffer descriptors to add +* @filled: number of valid descriptors in the array +* @buff_size: size of each buffer (multi-queue mode only) +* @queue_index: RX queue index +* +* Return: hypervisor return code +*/ +static long ibmveth_add_logical_lan_buffers (struct ibmveth_adapter *adapter, +union ibmveth_buf_desc *descs, +int filled, +unsigned long buff_size, +int queue_index) +{ +struct vio_dev *vdev = adapter->vdev; +unsigned long rc; + +if (adapter->multi_queue) {[ ... ]
Quoted 8 lines
drivers/net/ethernet/ibm/ibmveth.c +8
-0+ rc =h_add_logical_lan_buffers_queue (vdev->unit_address, + handle, + buffersznum, + ioba[0 ], ioba[1 ], ioba[2 ], + ioba[3 ], ioba[4 ], ioba[5 ]); + adapter->hcall_stats.add_bufs_queue++; + }else if (filled ==1 ) { + rc = h_add_logical_lan_buffer(vdev->unit_address,drivers/net/ethernet/ibm/ibmveth.c static inline void ibmveth_flush_buffer(void *addr, unsigned long length) · line 659
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+ descs[0].desc); + adapter->hcall_stats.add_buf++; + }else { + rc =h_add_logical_lan_buffers (vdev->unit_address, + descs[0 ].desc, descs[1 ].desc, + descs[2 ].desc, descs[3 ].desc, + descs[4 ].desc, descs[5 ].desc, + descs[6 ].desc, descs[7 ].desc); + adapter->hcall_stats.add_bufs++; + }[Severity: Medium]
These
hcall_statsfields are plain non-atomic u64 counters on the adapter, but the only serialization this patch adds is the per-queue rx_queue[i].replenish_lock taken inibmveth_replenish_task().Two NAPI instances replenishing different queues hold different locks, so can these increments lose counts once MQ is enabled later in the series and
num_rx_queuesbecomes larger than 1?The same question applies to the other adapter-wide counters reached from this path:
adapter->replenish_task_cycles++; adapter->replenish_add_buff_success += filled; adapter->replenish_add_buff_failure += filled;and to
ibmveth_update_rx_no_buffer(), whichibmveth_replenish_task()calls while holding queue N's lock only, although it rewrites adapter-global state for every queue:adapter->rx_no_buffer = 0; for (i = 0; i < adapter->num_rx_queues; i++) { ... adapter->rx_qstats[i].no_buffer_drops = drops; adapter->rx_no_buffer += drops; }
Quoted 3 lines
Would
rx_no_bufferand rx_qstats[].no_buffer_drops be clobbered when two queues replenish concurrently?ibmveth_close()calls the same helper with no lock held at all.
Yes, once MQ is live those adapter-global counters can lose updates across queues. I agree they need a better concurrency model, but I see that as broader accounting follow-up work rather than one of the main correctness blockers in this series. The pool-availability path is already using atomic_t for correctness; these adapter-global counters are diagnostic/accounting state and should either move to a per-queue model or use a safer aggregation strategy.
Quoted 36 lines
Quoted 11 lines
/* replenish the buffers for a pool. note that we don't need to
*skb_reservethese since they are used for incoming...
*/
static void ibmveth_replenish_buffer_pool(structibmveth_adapter*adapter,drivers/net/ethernet/ibm/ibmveth.c +2
-1- struct ibmveth_buff_pool *pool) + struct ibmveth_buff_pool *pool, + int queue_index) {union ibmveth_buf_desc descs[IBMVETH_MAX_RX_PER_HCALL ] = {0 };u32 remaining = pool->size -atomic_read (&pool->available);@@ -678,24 +752,16 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,[ ... ]
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-0+ lpar_rc =ibmveth_add_logical_lan_buffers (adapter, descs, + filled, + pool->buff_size, + queue_index);drivers/net/ethernet/ibm/ibmveth.c static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter, · line 767
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-1+if (lpar_rc !=H_SUCCESS ) {dev_warn_ratelimited (dev, -"RX h_add_logical_lan failed: filled=%u, rc=%lu, batch=%u\n" , +"RX h_add_logical_lan %s failed: filled=%u, rc=%lu, batch=%u\n" , + adapter->multi_queue ? +"_queue" :"" , filled, lpar_rc, batch); goto hcall_failure; }[Severity: Low]
The interpolated hcall name never matches an actual symbol. In MQ mode this prints
RX
h_add_logical_lan_queuefailed: ...with a stray space, and in legacy mode
Quoted 7 lines
RX
h_add_logical_lanfailed: ...with a double space. Would a full literal per branch read better?
The two new
H_FUNCTIONmessages added below also print batch with %d, although batch is declared u32 in this function ("u32 i, filled, batch;"), while this warning uses %u for the same variable.
Yes. Full literal hcall names would read better here, and batch should use %u. I'll fold that cleanup back into the owning patch rather than leave it as a tail fixup.
Quoted 52 lines
Quoted 18 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-14@@ -736,24 +802,23 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter, } adapter->replenish_add_buff_failure += filled; -/* -* If multi rx buffers hcall is no longer supported by FW -* e.g. in the case of Live Partition Migration -*/ -if (batch >1 && lpar_rc ==H_FUNCTION ) { -/* -* Instead of retry submit single buffer individually -* here just set the max rx buffer per hcall to 1 -* buffers will be respleshed next time -* when ibmveth_replenish_buffer_pool() is called again -* with single-buffer case -*/ - netdev_info(adapter->netdev, - "RX Multi buffers not supported by FW, rc=%lu\n",drivers/net/ethernet/ibm/ibmveth.c static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter, · line 810
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +6
-4- lpar_rc); - adapter->rx_buffers_per_hcall =1 ; -netdev_info (adapter->netdev, -"Next rx replesh will fall back to single-buffer hcall\n" ); +if (lpar_rc ==H_FUNCTION ) { +if (adapter->multi_queue) { +netdev_err (adapter->netdev, +"MQ buffer add H_FUNCTION (q=%d, batch=%d)\n" , + queue_index, batch); + }else if (batch >1 ) {[Severity: High]
The
multi_queuearm logs and breaks without changing any state:adapter->multi_queuestays set,rx_buffers_per_hcallis untouched, there is no fallback to h_add_logical_lan_buffer()/h_add_logical_lan_buffers(), and noschedule_work(&adapter->work).
ibmveth_add_logical_lan_buffers()picks the hcall purely onadapter->multi_queue, and that field is only written inibmveth_probe(), which is not re-run on resume (ibmveth_resume()only callsibmveth_schedule_rx_queue()).So once the hypervisor stops implementing
H_ADD_LOGICAL_LAN_BUFFERS_QUEUE
- exactly the Live Partition Migration case the legacy arm below exists for
- can this queue ever post another RX buffer again?pool->availablestays at 0, PHYP drops every inbound frame, and the interface still reports the link up, so recovery would need an administrative down/up.Every other unrecoverable condition in this driver schedules the reset work (
ibmveth_remove_buffer_from_pool(),ibmveth_rxq_get_buffer(), and the newenable_irqfailure path inibmveth_poll()). Should this arm do the same, or fall back to the legacy hcalls the way the legacy arm downgrades batch?
Quoted 5 lines
Separately,
netdev_err()here is not rate limited, while the warning for the same failure a few lines above usesdev_warn_ratelimited(), so a dried-out pool can emit several console lines per poll cycle per pool.This arm appears unchanged through the end of the series.
Yes. As written, the MQ H_FUNCTION path can strand the queue in a permanent dry-pool state.
I'm planning to add real recovery here rather than paper over it by shrinking the batch size, because that still leaves the driver calling the dead queue-buffer hcall.
The safer direction is to treat this as an unexpected MQ H_FUNCTION event, mark fallback, and recover through the normal reset path rather than trying to half-switch modes on the live datapath. I'll also make that path rate-limited.
With the clarified project assumption that HMC blocks LPM into LPARs without MQ-capable veth, I do not think this should be framed as a supported in-place MQ-to-SQ downgrade path.
Quoted 83 lines
Quoted 39 lines
drivers/net/ethernet/ibm/ibmveth.c +22
-2+/* +* Live Partition Migration may drop multi- +* buffer support. Fall back to single-buffer +* on the next replenish; do not continue with +* a stale local batch size (infinite loop). +*/ +netdev_warn (adapter->netdev, +"Legacy batch add H_FUNCTION (batch=%d), fallback\n" , + batch); + adapter->rx_buffers_per_hcall =1 ; + } }break ; }@@ -775,21 +840,33 @@ static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter) }/* replenish routine */ -static void ibmveth_replenish_task (struct ibmveth_adapter *adapter) +static void ibmveth_replenish_task (struct ibmveth_adapter *adapter, +int queue_index) { +struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index]; +unsigned long flags;int i; +if (queue_index >= adapter->num_rx_queues) +return ; + adapter->replenish_task_cycles++; +spin_lock_irqsave (&rxq->replenish_lock, flags); +for (i = (IBMVETH_NUM_BUFF_POOLS -1 ); i >=0 ; i--) { -struct ibmveth_buff_pool *pool = &adapter->rx_buff_pool[0 ][i]; +struct ibmveth_buff_pool *pool = + &adapter->rx_buff_pool[queue_index][i]; if (pool->active &&drivers/net/ethernet/ibm/ibmveth.c static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter) · line 870
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-1(atomic_read(&pool->available) < pool->threshold)) -ibmveth_replenish_buffer_pool (adapter, pool); +ibmveth_replenish_buffer_pool (adapter, pool, + queue_index); }ibmveth_update_rx_no_buffer (adapter); + +spin_unlock_irqrestore (&rxq->replenish_lock, flags); }[Severity: High]
Can this lock deadlock against itself when netconsole runs over this ibmveth device?
ibmveth_replenish_buffer_pool()printk()s from inside the locked region:netdev_info()on an invalid map index,dev_warn_ratelimited()on hcall failure, and the two newH_FUNCTIONmessages. With netconsole on this interface a printk from there re-enters the driver:netconsole
write_msg()
netpoll_send_udp()netpoll_send_skb() __netpoll_send_skb() netpoll_poll_dev() ops->ndo_poll_controller() -> ibmveth_poll_controller() ibmveth_replenish_task(adapter, i)
__netpoll_send_skb()callsnetpoll_poll_dev()whenever the xmit does not go through on the first try, andnetpoll_poll_dev()only declines whendown_trylock(&ni->dev_lock)fails, when !netif_running(dev), or whennetif_local_xmit_active(dev):if (!ni || down_trylock(&ni->dev_lock)) return; if (!netif_running(dev) || netif_local_xmit_active(dev)) { up(&ni->dev_lock); return; } ... if (ops->ndo_poll_controller) ops->ndo_poll_controller(dev);
Quoted 8 lines
None of those covers the replenish path, so
ibmveth_poll_controller()loopsibmveth_replenish_task()over every queue and re-takes the samereplenish_lockon the same CPU, with interrupts already disabled.Before this patch
ibmveth_poll_controller()also calledibmveth_replenish_task(), but with no lock the recursion was harmless.This code appears unchanged through the end of the series.
Yes. That deadlock looks plausible with netconsole on this device.
I'm planning to move the logging and recovery actions out from under replenish_lock, because that is the real issue here.
The helper should return outcome/state while the actual logging and reset scheduling happen after unlock, so netconsole cannot recurse back into the same queue's replenish path while the lock is held.
Quoted 12 lines
[Severity: Medium]
Does this need to be
spin_lock_irqsave()? The lock now brackets the whole multi-pool loop, so everynetdev_alloc_skb(), everydma_map_single()and every H_ADD_LOGICAL_LAN_BUFFER[S][_QUEUE] hypervisor call for all five pools runs with local interrupts hard-disabled.On a cold fill from
ibmveth_open()or the first NAPI poll that is roughly 768 allocations and DMA maps and around 96 hcalls per queue at the default MTU (pool_count[] = { 256, 512, 256, 256, 256 }), and more when the large pools are active.
Quoted 5 lines
None of the protected state is written from hardirq context, and the commit message says the lock exists only "so later concurrent NAPI/resize paths can serialize buffer posting", so would
spin_lock_bh(), dropping and re-acquiring per batch, or preparing buffers outside the critical section be enough?
Fair point. That spin_lock_irqsave() critical section is heavier than I would like, but I still see it as a secondary issue compared to the main correctness problems in this patch.
I do not want to rewrite that lock protocol in this series while the higher-priority fixes here are the MQ H_FUNCTION recovery, no logging under replenish_lock, and making the producer/consumer locking model complete.
Quoted 53 lines
Quoted 2 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0/* empty and free ana buffer pool - also used to do cleanup in error paths */ [ ... ]
Quoted 1 line
drivers/net/ethernet/ibm/ibmveth.c +0
-0@@ -1004,50 +1083,55 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,[ ... ]
Quoted 7 lines
drivers/net/ethernet/ibm/ibmveth.c +1
-6- free_index = adapter->rx_buff_pool[0 ][pool].producer_index; - adapter->rx_buff_pool[0 ][pool].producer_index++; -if (adapter->rx_buff_pool[0 ][pool].producer_index >= - adapter->rx_buff_pool[0 ][pool].size) - adapter->rx_buff_pool[0 ][pool].producer_index =0 ; - adapter->rx_buff_pool[0 ][pool].free_map[free_index] = index; + free_index = adapter->rx_buff_pool[queue_index][pool].producer_index;drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter, · line 1106
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +6
-1+ adapter->rx_buff_pool[queue_index][pool].producer_index++; +if (adapter->rx_buff_pool[queue_index][pool].producer_index >= + adapter->rx_buff_pool[queue_index][pool].size) + adapter->rx_buff_pool[queue_index][pool].producer_index =0 ; + adapter->rx_buff_pool[queue_index][pool].free_map[free_index] = index;mb (); -atomic_dec (&adapter->rx_buff_pool[0 ][pool].available); +atomic_dec (&adapter->rx_buff_pool[queue_index][pool].available);[Severity: High]
This isn't a bug introduced by this patch, but the new
replenish_lockonly covers the producer half of this bookkeeping, so it is worth asking whether the consumer half is still unprotected.
ibmveth_remove_buffer_from_pool()mutates free_map[],producer_index, skbuff[], dma_addr[] and available for the same pool thatibmveth_replenish_buffer_pool()mutates underreplenish_lock, but takes no lock:producer:
ibmveth_poll_controller()
ibmveth_replenish_task()/* holdsreplenish_lock*/
ibmveth_replenish_buffer_pool()consumer:
ibmveth_poll()/* holds nothing */
ibmveth_rxq_harvest_buffer()
ibmveth_remove_buffer_from_pool()
netpoll_poll_dev()serializes netpoll users against each other viani->dev_lock, and __napi_poll()'snetpoll_poll_lock()only stops netpoll from re-polling a NAPI instance that is already running; neither preventsndo_poll_controllerfrom running on one CPU while the same queue's NAPI poll runs on another.Can free_map[]/producer_index/consumer_index then hand the same pool index out twice, so a buffer is posted to the hypervisor while its skb has already gone up the stack, or lose a free index permanently?
Quoted 5 lines
Since the future work described in the commit message ("so later concurrent NAPI/resize paths can serialize buffer posting") builds on this lock, should the consumer side take it as well?
[ ... ]
Yes. That race looks real, and the consumer side should participate in the same per-queue locking protocol.
I'm planning to serialize harvest-side mutation with the same lock so the producer/consumer model is actually complete.
The important point here is that the race is not "two NAPIs on one queue"; it is harvest on one CPU versus netpoll-driven replenish on the same queue.
Quoted 58 lines
Quoted 19 lines
drivers/net/ethernet/ibm/ibmveth.c +8
-3@@ -1993,34 +2076,45 @@ static void ibmveth_rx_csum_helper(struct sk_buff *skb,static int ibmveth_poll (struct napi_struct *napi,int budget) { -struct ibmveth_adapter *adapter = - container_of(napi, struct ibmveth_adapter, napi[0]); -struct net_device *netdev = adapter->netdev; +struct net_device *netdev = napi->dev; +struct ibmveth_adapter *adapter =netdev_priv (netdev);int frames_processed =0 ;unsigned long lpar_rc; +int queue_index, rc;u16 mss =0 ; + queue_index = napi - adapter->napi; + +if (WARN_ON (queue_index <0 || queue_index >= adapter->num_rx_queues)) +return 0 ; +[ ... ]
Quoted 13 lines
drivers/net/ethernet/ibm/ibmveth.c +5
-2@@ -2097,14 +2197,19 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)/* We think we are done - reenable interrupts, * then check once more to make sure we are done. */ - lpar_rc =ibmveth_enable_irq (adapter,0 ); -if (WARN_ON (lpar_rc !=H_SUCCESS )) { + lpar_rc =ibmveth_enable_irq (adapter, queue_index); +if (lpar_rc !=H_SUCCESS ) { +netdev_err (netdev, +"Failed to enable IRQ for queue %d (rc=0x%lx), scheduling reset\n" , + queue_index, lpar_rc);schedule_work (&adapter->work); goto out;drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_poll(struct napi_struct *napi, int budget) · line 2214
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-2} -if (ibmveth_rxq_pending_buffer (adapter) &&napi_schedule (napi)) { - lpar_rc =ibmveth_disable_irq (adapter,0 ); +if (ibmveth_rxq_pending_buffer (adapter, queue_index) && +napi_schedule (napi)) { + lpar_rc =ibmveth_disable_irq (adapter, queue_index); +WARN_ON (lpar_rc !=H_SUCCESS ); goto restart_poll; }[Severity: Low]
Two diagnostic changes here are not described in the commit message, which only says callers were updated "so arity stays consistent": the
WARN_ON()around theibmveth_enable_irq()failure is replaced by an unconditionalnetdev_err(), which drops the taint and backtrace that test harnesses key on, and a newWARN_ON()is added on theibmveth_disable_irq()result. Would it help to mention or split out that change?Also, the new guard added at the top of
ibmveth_poll()if (WARN_ON(queue_index < 0 || queue_index >= adapter->num_rx_queues)) return 0;
Quoted 8 lines
returns less than budget from a NAPI poll callback without calling
napi_complete_done(), which leaves the instance scheduled with the hypervisor IRQ still masked. The guard is unreachable at this commit, sinceIBMVETH_MAX_RX_QUEUESis 1 andqueue_indexis always 0, but the same early return is still there at the end of the series. Should it complete NAPI before returning?[ ... ]
Yes. The bad-queue-index guard should complete NAPI before returning, otherwise that defensive path can leave the instance scheduled with the IRQ still masked.
The diagnostic reshaping around the enable/disable IRQ paths belongs with the owning poll work rather than being overclaimed here.
Thanks,
Mingming
Quoted 16 lines
Quoted 15 lines
drivers/net/ethernet/ibm/ibmveth.c +6
-2@@ -2229,9 +2334,13 @@ static int ibmveth_change_mtu(struct net_device *dev, int new_mtu)static void ibmveth_poll_controller (struct net_device *dev) {struct ibmveth_adapter *adapter =netdev_priv (dev); +int i; -ibmveth_replenish_task (adapter); -ibmveth_schedule_rx_queue (adapter,0 ); +for (i =0 ; i < adapter->num_rx_queues; i++) +ibmveth_replenish_task (adapter, i); + +for (i =0 ; i < adapter->num_rx_queues; i++) +ibmveth_schedule_rx_queue (adapter, i); }#endif [ ... ]
PATCH v4 9/14 ibmveth: Enable multi-queue RX receive path
In reply to Mingming CaoThis is where MQ actually receives packets. If firmware sets IBMVETH_ILLAN_RX_MULTI_QUEUE_SUPPORT in H_ILLAN_ATTRIBUTES, probe sets multi_queue and num_rx_queues to min(num_online_cpus(), IBMVETH_DEFAULT_QUEUES), matching the existing TX default (cap 8). Up to IBMVETH_MAX_RX_QUEUES (16) remains available via ethtool -L. Otherwise we stay at one queue like today.
Enable the live multi-queue RX path using helpers introduced earlier:
- Probe: MQ capability bit, multi_queue/num_rx_queues, IBMVETH_MAX_RX_QUEUES=16, alloc_etherdev_mqs RX count, NAPI for all queues, MQ vs regular rx_buffers_per_hcall
- ibmveth_register_logical_lan_queue() / ibmveth_register_single_rx_queue() for subordinate queues via H_REG_LOGICAL_LAN_QUEUE (irq_create_mapping for subordinate virqs)
- Wire subordinates into ibmveth_register_rx_queues()
- ibmveth_dispose_subordinate_irq_mappings() on partial failure
- setup_rx_interrupts(): after request_irq + napi_enable, PHYP
enable_irq for all queues when multi_queue && num_rx_queues > 1
- MQ open replenishes every RX queue before setup_rx_interrupts()
unmasks PHYP; SQ keeps the classic setup-then-schedule kick
- ibmveth_dispose_subordinate_irq_mappings() on setup failure
(queue 0 uses netdev->irq and is never disposed)Subordinate register error handling:
- H_FUNCTION is a hard open failure: honest netdev_err, then the generic failure/params logs, without clearing multi_queue or claiming single-queue fallback
- Register each subordinate once; no blind try_again retry
- No caller-side "Invalid hypervisor return" log; the callee already reports the hcall rc
setup_rx_interrupts() failure paths dispose subordinate virq mappings from both err_free_irqs and err_disable_napi so request_irq failure after successful subordinate registration cannot leak Linux mappings.
Hot-path netdev->stats TX/RX accounting moves to the next patch (per-queue qstats).
Legacy firmware without the MQ bit is unchanged.
On probe failure after pool kobjects were created, put them before free_netdev() so earlier pools cannot leak.
---
Changes in v4:
- Fold subordinate register helpers and their review fixes into the MQ enablement patch that first uses them.
- Prefer request_irq -> napi_enable -> PHYP enable on MQ open.
- Preserve open unwind so set_real_num_rx / IRQ failures free LAN before buffer pools.
- MQ open replenishes every queue before setup_rx_interrupts() unmasks PHYP (drop avoidance during open; PHYP only interrupts after a successful enqueue). SQ keeps classic setup-then-schedule kick.
- Dispose subordinate virq mappings on setup_rx_interrupts() request_irq failure (err_free_irqs), matching err_disable_napi.
- Open unwind: setup/cleanup own subordinate dispose; skip duplicate dispose on those paths.
- H_FUNCTION on subordinate register is a hard open failure (no blind retry / no fake single-queue fallback).
- Note: hot-path netdev->stats accounting moves to the next patch (qstats).
- Put already-created pool kobjects on probe kobject_init_and_add / set_real_num_tx_queues / register_netdev failure (bisect-safe).
drivers/net/ethernet/ibm/ibmveth.c | 456 ++++++++++++++++++++++++----- drivers/net/ethernet/ibm/ibmveth.h | 3 +- 2 files changed, 380 insertions(+), 79 deletions(-)
drivers/net/ethernet/ibm/ibmveth.c +378 -78
@@ -30,6 +30,7 @@#include <linux/ip.h> #include <linux/ipv6.h> #include <linux/slab.h> +#include <linux/spinlock.h> #include <asm/hvcall.h> #include <linux/atomic.h> #include <asm/vio.h>
@@ -45,7 +46,6 @@ static unsigned long ibmveth_get_desired_dma(struct vio_dev *vdev);static struct kobj_type ktype_veth_pool; -static const char ibmveth_driver_name[] ="ibmveth" ;static const char ibmveth_driver_string[] ="IBM Power Virtual Ethernet Driver" ;#define ibmveth_driver_version "1.06"
@@ -97,7 +97,17 @@ static struct ibmveth_stat ibmveth_stats[] = { { "fw_enabled_ipv6_csum", IBMVETH_STAT_OFF(fw_ipv6_csum_support) }, { "tx_large_packets", IBMVETH_STAT_OFF(tx_large_packets) }, { "rx_large_packets", IBMVETH_STAT_OFF(rx_large_packets) }, - { "fw_enabled_large_send", IBMVETH_STAT_OFF(fw_large_send_support) } + { "fw_enabled_large_send", IBMVETH_STAT_OFF(fw_large_send_support) }, + { "hcall_reg_lan_queue", IBMVETH_STAT_OFF(hcall_stats.reg_lan_queue) }, + { "hcall_reg_lan", IBMVETH_STAT_OFF(hcall_stats.reg_lan) }, + { "hcall_add_bufs_queue", + IBMVETH_STAT_OFF(hcall_stats.add_bufs_queue) }, + { "hcall_add_bufs", IBMVETH_STAT_OFF(hcall_stats.add_bufs) }, + { "hcall_add_buf", IBMVETH_STAT_OFF(hcall_stats.add_buf) }, + { "hcall_free_lan_queue", + IBMVETH_STAT_OFF(hcall_stats.free_lan_queue) }, + { "hcall_free_lan", IBMVETH_STAT_OFF(hcall_stats.free_lan) }, + { "hcall_send_lan", IBMVETH_STAT_OFF(hcall_stats.send_lan) }, };/* simple methods of getting data from the current rxq entry */
@@ -429,12 +439,64 @@ ibmveth_enable_irq(struct ibmveth_adapter *adapter, int queue_index)return ibmveth_toggle_irq (adapter, queue_index, true); } +/** +* ibmveth_dispose_subordinate_irq_mapping - Drop one subordinate virq mapping +* @adapter: ibmveth adapter structure +* @queue_idx: RX queue index (1..N) +* +* Subordinate queues get mappings from irq_create_mapping() during PHYP +* registration. Queue 0 uses netdev->irq from device tree and is left alone. +* +* Bound against IBMVETH_MAX_RX_QUEUES, not num_rx_queues: scale-down and +* scale-up fail paths dispose queues that are no longer in the published +* live set but still own a virq in queue_irq[]. The bulk helper still +* iterates only 1..num_rx_queues-1 for close/open-fail cleanup. +* +* Linux virq lifetime is owned by interrupt cleanup helpers. Call this only +* after free_irq() when a handler was installed, or from registration failure +* cleanup before request_irq(). +*/ +static void +ibmveth_dispose_subordinate_irq_mapping (struct ibmveth_adapter *adapter, +int queue_idx) +{ +if (queue_idx <=0 || queue_idx >=IBMVETH_MAX_RX_QUEUES ) +return ; + +if (adapter->queue_irq[queue_idx]) { +irq_dispose_mapping (adapter->queue_irq[queue_idx]); + adapter->queue_irq[queue_idx] =0 ; + } +} + +/** +* ibmveth_dispose_subordinate_irq_mappings - Drop virq mappings for queues 1..N +* @adapter: ibmveth adapter structure +* +* Bulk helper for paths that registered subordinate queues but never installed +* Linux IRQ handlers. +*/ +static void +ibmveth_dispose_subordinate_irq_mappings (struct ibmveth_adapter *adapter) +{ +int i; + +for (i =1 ; i < adapter->num_rx_queues; i++) +ibmveth_dispose_subordinate_irq_mapping (adapter, i); +} +/** * ibmveth_setup_rx_interrupts - Register IRQs and enable NAPI * @adapter: ibmveth adapter structure * -* Registers interrupt handlers for all RX queues and enables NAPI polling. -* On error, cleans up any successfully registered IRQs before returning. +* Registers interrupt handlers for all RX queues, enables NAPI, then +* enables hypervisor interrupt delivery for multi-queue mode after +* every queue has a Linux handler installed. For multi-queue open the +* caller should replenish RX buffers before this helper so traffic +* during open is not dropped (PHYP only interrupts after a successful +* enqueue, which needs buffers). Single-queue open leaves PHYP masked +* here and kicks NAPI afterward (classic path: first poll posts then +* enables). * * Return: 0 on success, negative error code on failure */
@@ -442,12 +504,9 @@ static int ibmveth_setup_rx_interrupts(struct ibmveth_adapter *adapter) {struct net_device *netdev = adapter->netdev; -int i, rc; +int i, rc, num = adapter->num_rx_queues; -for (i =0 ; i < adapter->num_rx_queues; i++) -napi_enable (&adapter->napi[i]); - -for (i =0 ; i < adapter->num_rx_queues; i++) { +for (i =0 ; i < num; i++) {if (!adapter->queue_irq[i]) {netdev_err (netdev,"queue %d has invalid IRQ (0)\n" , i); rc = -EINVAL ;
@@ -464,13 +523,42 @@ ibmveth_setup_rx_interrupts(struct ibmveth_adapter *adapter) } } +for (i =0 ; i < num; i++) +napi_enable (&adapter->napi[i]); + +if (adapter->multi_queue && num >1 ) { +for (i =0 ; i < num; i++) { + rc =ibmveth_enable_irq (adapter, i); +if (rc) { +netdev_err (netdev, +"Failed to enable IRQ for queue %d, rc=%d\n" , + i, rc); +while (--i >=0 ) +ibmveth_disable_irq (adapter, i); + rc = -EIO ; + goto err_disable_napi; + } + } + } +return 0 ; +err_disable_napi: +/* PHYP unmask was rolled back above; disable NAPI before free_irq */ +for (i =0 ; i < num; i++) +napi_disable (&adapter->napi[i]); +for (i =0 ; i < num; i++) { +if (adapter->queue_irq[i]) +free_irq (adapter->queue_irq[i], &adapter->napi[i]); + } + goto err_dispose_mappings; + err_free_irqs:while (--i >=0 )free_irq (adapter->queue_irq[i], &adapter->napi[i]); -for (i =0 ; i < adapter->num_rx_queues; i++) -napi_disable (&adapter->napi[i]); +err_dispose_mappings: +/* Both setup failure paths own subordinate virq disposal. */ +ibmveth_dispose_subordinate_irq_mappings (adapter);return rc; }
@@ -503,15 +591,7 @@ ibmveth_cleanup_rx_interrupts(struct ibmveth_adapter *adapter)free_irq (adapter->queue_irq[i], &adapter->napi[i]); } -/* Dispose IRQ mappings for subordinate queues (1-15). -* Queue 0 uses netdev->irq from device tree, not irq_create_mapping(). -*/ -for (i =1 ; i < adapter->num_rx_queues; i++) { -if (adapter->queue_irq[i]) { -irq_dispose_mapping (adapter->queue_irq[i]); - adapter->queue_irq[i] =0 ; - } - } +ibmveth_dispose_subordinate_irq_mappings (adapter);/* Queue 0 uses netdev->irq; leave queue_irq[0] for next open. */ }
@@ -521,8 +601,8 @@ ibmveth_cleanup_rx_interrupts(struct ibmveth_adapter *adapter)* @adapter: ibmveth adapter structure * @qindex: RX queue index * -* Shared by the IRQ handler and process-context kick paths (open, resume, -* pool sysfs, netpoll). Keep ibmveth_interrupt() as the IRQ-only wrapper. +* Shared by the IRQ handler and process-context kick sites (open, resume, +* pool sysfs, poll_controller). */ static void ibmveth_schedule_rx_queue (struct ibmveth_adapter *adapter,int qindex)
@@ -834,9 +914,15 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,*/ static void ibmveth_update_rx_no_buffer (struct ibmveth_adapter *adapter) { -__be64 *p = adapter->buffer_list_addr[0 ] +4096 -8 ; +int i; - adapter->rx_no_buffer =be64_to_cpup (p); + adapter->rx_no_buffer =0 ; +for (i =0 ; i < adapter->num_rx_queues; i++) { +__be64 *p = adapter->buffer_list_addr[i] +4096 -8 ; +u64 drops =be64_to_cpup (p); + + adapter->rx_no_buffer += drops; + } }/* replenish routine */
@@ -847,8 +933,12 @@ static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,unsigned long flags;int i; -if (queue_index >= adapter->num_rx_queues) +if (queue_index >= adapter->num_rx_queues) { +netdev_dbg (adapter->netdev, +"Skipping replenish for freed queue %d (num_queues=%d)\n" , + queue_index, adapter->num_rx_queues);return ; + } adapter->replenish_task_cycles++;
@@ -858,7 +948,7 @@ static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,struct ibmveth_buff_pool *pool = &adapter->rx_buff_pool[queue_index][i]; -if (pool->active && +if (pool->active && pool->free_map && (atomic_read (&pool->available) < pool->threshold))ibmveth_replenish_buffer_pool (adapter, pool, queue_index);
@@ -1284,6 +1374,137 @@ static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,return rc; } +/** +* ibmveth_register_logical_lan_queue - Register subordinate queue with +* hypervisor +* @adapter: ibmveth adapter structure +* @rxq_desc: Receive queue descriptor +* @queue_index: RX queue index (1..N for subordinate queues) +* +* Registers a subordinate receive queue using H_REG_LOGICAL_LAN_QUEUE. +* On success, stores the queue handle and virtual IRQ in the adapter. +* If IRQ mapping fails after a successful hypervisor registration, the +* queue is freed before returning. +* +* Return: H_SUCCESS on success, negative errno on IRQ mapping failure, +* hypervisor error code otherwise +*/ +static int +ibmveth_register_logical_lan_queue (struct ibmveth_adapter *adapter, +union ibmveth_buf_desc rxq_desc, +int queue_index) +{ +unsigned long handle, hwirq; +unsigned int virq; +long lpar_rc; + +netdev_dbg (adapter->netdev, +"Attempting to register queue %d: unit_addr=0x%x buffer_list_dma=0x%llx rxq_desc=0x%llx\n" , + queue_index, adapter->vdev->unit_address, + (unsigned long long )adapter->buffer_list_dma[queue_index], + (unsigned long long )rxq_desc.desc); + + lpar_rc =h_reg_logical_lan_queue (adapter->vdev->unit_address, + adapter->buffer_list_dma[queue_index], + rxq_desc.desc, &handle, &hwirq); + adapter->hcall_stats.reg_lan_queue++; + +if (lpar_rc ==H_SUCCESS ) { + virq =irq_create_mapping (NULL , hwirq); +if (!virq) { +unsigned long free_rc; +unsigned long ua = adapter->vdev->unit_address; + +netdev_err (adapter->netdev, +"Failed to map IRQ for queue %d (hwirq=%lu)\n" , + queue_index, hwirq); +do { + free_rc =h_free_logical_lan_queue (ua, handle); + }while (H_IS_LONG_BUSY (free_rc) || + (free_rc ==H_BUSY )); + adapter->hcall_stats.free_lan_queue++; +if (free_rc !=H_SUCCESS ) +netdev_err (adapter->netdev, +"h_free_logical_lan_queue failed for queue %d after IRQ map failure: rc=0x%lx\n" , + queue_index, free_rc); +return -EINVAL ; + } + + adapter->queue_handle[queue_index] = handle; + adapter->queue_irq[queue_index] = virq; + +netdev_dbg (adapter->netdev, +"queue %d registered: handle=0x%llx irq=%u\n" , + queue_index, adapter->queue_handle[queue_index], + adapter->queue_irq[queue_index]); +return H_SUCCESS ; + } + +/* +* H_FUNCTION means firmware rejected this subordinate register +* (MQ unsupported). That is a hard open failure: do not clear +* multi_queue or claim single-queue fallback. Keep a specific +* log, then the generic failure lines below (no early return). +*/ +if (lpar_rc ==H_FUNCTION ) +netdev_err (adapter->netdev, +"h_reg_logical_lan_queue H_FUNCTION for queue %d (firmware MQ unsupported)\n" , + queue_index); + +netdev_err (adapter->netdev, +"h_reg_logical_lan_queue failed for queue %d with %ld\n" , + queue_index, lpar_rc); +netdev_err (adapter->netdev, +"queue %d params: unit_addr=0x%x buffer_list_dma=0x%llx rxq_desc=0x%llx\n" , + queue_index, adapter->vdev->unit_address, + (unsigned long long )adapter->buffer_list_dma[queue_index], + (unsigned long long )rxq_desc.desc); + +return lpar_rc; +} + +/** +* ibmveth_register_single_rx_queue - Register one subordinate RX queue +* @adapter: ibmveth adapter structure +* @queue_idx: Queue index to register (1..N) +* @mac_address: MAC address (unused; reserved for API symmetry) +* +* Builds the queue descriptor and registers with the hypervisor via +* ibmveth_register_logical_lan_queue(). +* +* Return: 0 on success, -EINVAL if @queue_idx is invalid, -EIO on failure +*/ +static int +ibmveth_register_single_rx_queue (struct ibmveth_adapter *adapter, +int queue_idx,u64 mac_address) +{ +struct net_device *netdev = adapter->netdev; +union ibmveth_buf_desc rxq_desc; +long lpar_rc; + + (void )mac_address; + +if (WARN_ON (queue_idx <1 || queue_idx >=IBMVETH_MAX_RX_QUEUES )) +return -EINVAL ; + + rxq_desc.fields.flags_len =IBMVETH_BUF_VALID | + adapter->rx_queue[queue_idx].queue_len; + rxq_desc.fields.address = adapter->rx_queue[queue_idx].queue_dma; + + lpar_rc =ibmveth_register_logical_lan_queue (adapter, rxq_desc, + queue_idx); +if (lpar_rc !=H_SUCCESS ) { +netdev_err (netdev,"Failed to register queue %d: rc=0x%lx\n" , + queue_idx, lpar_rc); +return -EIO ; + } + +netdev_dbg (netdev,"Registered queue %d with handle 0x%llx\n" , + queue_idx, adapter->queue_handle[queue_idx]); + +return 0 ; +} +/** * ibmveth_free_all_queues - Free all RX queues at once * @adapter: ibmveth adapter structure
@@ -1292,7 +1513,8 @@ static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,* Used during interface close and registration error cleanup. * * Clears queue handles only; queue_irq[] is released by -* ibmveth_cleanup_rx_interrupts(). +* ibmveth_cleanup_rx_interrupts() on close, or by +* ibmveth_dispose_subordinate_irq_mappings() on partial register failure. */ static void ibmveth_free_all_queues (struct ibmveth_adapter *adapter) {
@@ -1320,10 +1542,11 @@ static void ibmveth_free_all_queues(struct ibmveth_adapter *adapter)* @adapter: ibmveth adapter structure * @mac_address: MAC address for device registration * -* Registers queue 0 via ibmveth_register_logical_lan(). Subordinate queue -* registration is added when multi-queue RX is enabled. +* Registers queue 0 via ibmveth_register_logical_lan(), then subordinate +* queues 1..N when multi-queue mode is enabled. * -* Return: 0 on success, -ENONET if queue 0 registration fails +* Return: 0 on success, -ENONET if queue 0 registration fails, -EIO on +* subordinate queue registration failure */ static int ibmveth_register_rx_queues (struct ibmveth_adapter *adapter,u64 mac_address)
@@ -1331,7 +1554,7 @@ ibmveth_register_rx_queues(struct ibmveth_adapter *adapter, u64 mac_address)struct net_device *netdev = adapter->netdev;union ibmveth_buf_desc rxq_desc;unsigned long lpar_rc; -int rc; +int i, rc; rxq_desc.fields.flags_len =IBMVETH_BUF_VALID | adapter->rx_queue[0 ].queue_len;
@@ -1356,9 +1579,31 @@ ibmveth_register_rx_queues(struct ibmveth_adapter *adapter, u64 mac_address)return -ENONET ; } +if (adapter->num_rx_queues ==1 || !adapter->multi_queue) { +netdev_dbg (netdev, +"registered 1 RX queue with hypervisor (single-queue mode)\n" ); +return 0 ; + } + +netdev_dbg (netdev,"Registering %d subordinate queues (1-%d)\n" , + adapter->num_rx_queues -1 , adapter->num_rx_queues -1 ); + +for (i =1 ; i < adapter->num_rx_queues; i++) { + rc =ibmveth_register_single_rx_queue (adapter, i, mac_address); +if (rc) + goto err_unregister; + } +netdev_dbg (netdev, -"registered 1 RX queue with hypervisor (single-queue mode)\n" ); +"registered %d RX queues with hypervisor (multi-queue mode)\n" , + adapter->num_rx_queues); +return 0 ; + +err_unregister: +ibmveth_dispose_subordinate_irq_mappings (adapter); +ibmveth_free_all_queues (adapter); +return rc; }static int ibmveth_open (struct net_device *netdev)
@@ -1396,12 +1641,29 @@ static int ibmveth_open(struct net_device *netdev) goto out_unregister_queues; } +/* +* MQ: post buffers before setup_rx_interrupts() unmasks PHYP +* (avoids drops if traffic arrives during open; PHYP allows +* either order). Single-queue keeps the classic kick: setup +* (no unmask) then schedule_rx_queue() so the first poll +* replenishes and enables. +*/ +if (adapter->multi_queue && adapter->num_rx_queues >1 ) { +for (i =0 ; i < adapter->num_rx_queues; i++) { +netdev_dbg (netdev, +"initial replenish cycle for queue %d\n" , i); +ibmveth_replenish_task (adapter, i); + } + } + rc =ibmveth_setup_rx_interrupts (adapter);if (rc) - goto out_unregister_queues; + goto out_free_all_queues;/* setup already disposed IRQs */ -netdev_dbg (netdev,"initial replenish cycle\n" ); -ibmveth_schedule_rx_queue (adapter,0 ); +if (!(adapter->multi_queue && adapter->num_rx_queues >1 )) { +netdev_dbg (netdev,"initial replenish cycle\n" ); +ibmveth_schedule_rx_queue (adapter,0 ); + } rc =ibmveth_alloc_tx_resources (adapter);if (rc)
@@ -1415,7 +1677,10 @@ static int ibmveth_open(struct net_device *netdev) out_cleanup_rx_interrupts:ibmveth_cleanup_rx_interrupts (adapter); + goto out_free_all_queues;/* cleanup already disposed IRQs */ out_unregister_queues: +ibmveth_dispose_subordinate_irq_mappings (adapter); +out_free_all_queues:ibmveth_free_all_queues (adapter); out_free_buffer_pools:ibmveth_free_buffer_pools (adapter);
@@ -1702,6 +1967,11 @@ static int ibmveth_set_features(struct net_device *dev,return rc1 ? rc1 : rc2; } +/* +* Sum per-queue counters for rare ethtool reads. Do not write adapter +* globals on the hot path (ibmvnic-style); with qstats allocated for the +* adapter lifetime, these sums remain meaningful across ifdown/up. +*/ static void ibmveth_get_strings (struct net_device *dev,u32 stringset,u8 *data) {int i;
@@ -1838,12 +2108,15 @@ static int ibmveth_send(struct ibmveth_adapter *adapter,return 1 ; } + adapter->hcall_stats.send_lan++;return 0 ; }static int ibmveth_is_packet_unsupported (struct sk_buff *skb, -struct net_device *netdev) +struct ibmveth_adapter *adapter, +int queue_num) { +struct net_device *netdev = adapter->netdev;struct ethhdr *ether_header;int ret =0 ;
@@ -1851,7 +2124,6 @@ static int ibmveth_is_packet_unsupported(struct sk_buff *skb,if (ether_addr_equal (ether_header->h_dest, netdev->dev_addr)) {netdev_dbg (netdev,"veth doesn't support loopback packets, dropping packet.\n" ); - netdev->stats.tx_dropped++; ret = -EOPNOTSUPP ; }
@@ -1867,7 +2139,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,int i, queue_num =skb_get_queue_mapping (skb);unsigned long mss =0 ; -if (ibmveth_is_packet_unsupported (skb, netdev)) +if (ibmveth_is_packet_unsupported (skb, adapter, queue_num)) goto out;/* veth can't checksum offload UDP */ if (skb->ip_summed == CHECKSUM_PARTIAL &&
@@ -1878,7 +2150,6 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb, skb_checksum_help(skb)) {netdev_err (netdev,"tx: failed to checksum packet\n" ); - netdev->stats.tx_dropped++; goto out; }
@@ -1901,7 +2172,6 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,if (skb->ip_summed ==CHECKSUM_PARTIAL &&skb_is_gso (skb)) {if (adapter->fw_large_send_support) { mss = (unsigned long )skb_shinfo (skb)->gso_size; - adapter->tx_large_packets++; }else if (!skb_is_gso_v6 (skb)) {/* Put -1 in the IP checksum to tell phyp it * is a largesend packet. Put the mss in
@@ -1910,7 +2180,6 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,ip_hdr (skb)->check =0xffff ;tcp_hdr (skb)->check =cpu_to_be16 (skb_shinfo (skb)->gso_size); - adapter->tx_large_packets++; } }
@@ -1918,7 +2187,6 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,if (unlikely (skb->len > adapter->tx_ltb_size)) {netdev_err (adapter->netdev,"tx: packet size (%u) exceeds ltb (%u)\n" , skb->len, adapter->tx_ltb_size); - netdev->stats.tx_dropped++; goto out; }memcpy (adapter->tx_ltb_ptr[queue_num], skb->data,skb_headlen (skb));
@@ -1935,7 +2203,6 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,if (unlikely (total_bytes != skb->len)) {netdev_err (adapter->netdev,"tx: incorrect packet len copied into ltb (%u != %u)\n" , skb->len, total_bytes); - netdev->stats.tx_dropped++; goto out; } desc.fields.flags_len = desc_flags | skb->len;
@@ -1944,18 +2211,13 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,dma_wmb ();if (ibmveth_send (adapter, desc.desc, mss)) { - adapter->tx_send_failed++; - netdev->stats.tx_dropped++; }else { - netdev->stats.tx_packets++; - netdev->stats.tx_bytes += skb->len; } out:dev_consume_skb_any (skb);return NETDEV_TX_OK ; - }static void ibmveth_rx_mss_helper (struct sk_buff *skb,u16 mss,int lrg_pkt)
@@ -2180,8 +2442,6 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)napi_gro_receive (napi, skb);/* send it up */ - netdev->stats.rx_packets++; - netdev->stats.rx_bytes += length; frames_processed++; } }
@@ -2357,8 +2617,7 @@ static unsigned long ibmveth_get_desired_dma(struct vio_dev *vdev)struct ibmveth_adapter *adapter;struct iommu_table *tbl;unsigned long ret; -int i; -int rxqentries =1 ; +int i, q; tbl =get_iommu_table_base (&vdev->dev);
@@ -2373,18 +2632,25 @@ static unsigned long ibmveth_get_desired_dma(struct vio_dev *vdev)/* add size of mapped tx buffers */ ret +=IOMMU_PAGE_ALIGN (IBMVETH_MAX_TX_BUF_SIZE , tbl); -for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) { -/* add the size of the active receive buffers */ -if (adapter->rx_buff_pool[0 ][i].active) - ret += - adapter->rx_buff_pool[0 ][i].size * -IOMMU_PAGE_ALIGN (adapter->rx_buff_pool[0 ][i]. - buff_size, tbl); - rxqentries += adapter->rx_buff_pool[0 ][i].size; - } -/* add the size of the receive queue entries */ - ret +=IOMMU_PAGE_ALIGN ( - rxqentries *sizeof (struct ibmveth_rx_q_entry ), tbl); +for (q =0 ; q < adapter->num_rx_queues; q++) { +int rxqentries =1 ; + +for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) { +/* add the size of the active receive buffers */ +struct ibmveth_buff_pool *bpool = + &adapter->rx_buff_pool[q][i]; + +/* add the size of the active receive buffers */ +if (bpool->active) + ret += bpool->size * +IOMMU_PAGE_ALIGN (bpool->buff_size, tbl); + rxqentries += bpool->size; + } + +/* add the size of the receive queue entries */ + ret +=IOMMU_PAGE_ALIGN (rxqentries * +sizeof (struct ibmveth_rx_q_entry ), tbl); + }return ret; }
@@ -2449,9 +2715,18 @@ static const struct net_device_ops ibmveth_netdev_ops = {#endif }; +static void ibmveth_put_pool_kobjs (struct ibmveth_adapter *adapter, +int pools_ready) +{ +int i; + +for (i =0 ; i < pools_ready; i++) +kobject_put (&adapter->rx_buff_pool[0 ][i].kobj); +} +static int ibmveth_probe (struct vio_dev *dev,const struct vio_device_id *id) { -int rc, i, mac_len; +int rc, i, mac_len, pools_ready =0 ;struct net_device *netdev;struct ibmveth_adapter *adapter;unsigned char *mac_addr_p;
@@ -2486,7 +2761,8 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)return -EINVAL ; } - netdev =alloc_etherdev_mqs (sizeof (struct ibmveth_adapter ),IBMVETH_MAX_QUEUES ,1 ); + netdev =alloc_etherdev_mqs (sizeof (struct ibmveth_adapter ), +IBMVETH_MAX_QUEUES ,IBMVETH_MAX_RX_QUEUES );if (!netdev)return -ENOMEM ;
@@ -2499,7 +2775,10 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id) adapter->mcastFilterSize =be32_to_cpu (*mcastFilterSize_p);ibmveth_init_link_settings (netdev); -netif_napi_add_weight (netdev, &adapter->napi[0 ], ibmveth_poll,16 ); +for (i =0 ; i <IBMVETH_MAX_RX_QUEUES ; i++) +netif_napi_add_weight (netdev, &adapter->napi[i], + ibmveth_poll,16 ); + netdev->irq = dev->irq; netdev->netdev_ops = &ibmveth_netdev_ops;
@@ -2531,16 +2810,27 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id) netdev->features |=NETIF_F_FRAGLIST ; } -/* Initialize queue count - always 1 for now */ - adapter->multi_queue =0 ; - adapter->num_rx_queues =IBMVETH_DEFAULT_RX_QUEUES ; +if (ret ==H_SUCCESS && + (ret_attr &IBMVETH_ILLAN_RX_MULTI_QUEUE_SUPPORT )) { + adapter->multi_queue =1 ; + adapter->num_rx_queues =min (num_online_cpus (), +IBMVETH_DEFAULT_QUEUES ); +netdev_dbg (netdev,"RX multi queue mode enabled: %d queues\n" , + adapter->num_rx_queues); + }else { + adapter->multi_queue =0 ; + adapter->num_rx_queues =IBMVETH_DEFAULT_RX_QUEUES ; + }if (ret ==H_SUCCESS && (ret_attr &IBMVETH_ILLAN_RX_MULTI_BUFF_SUPPORT )) { - adapter->rx_buffers_per_hcall =IBMVETH_MAX_RX_REGULAR ; +if (adapter->multi_queue) + adapter->rx_buffers_per_hcall =IBMVETH_MAX_RX_QUEUE ; +else + adapter->rx_buffers_per_hcall =IBMVETH_MAX_RX_REGULAR ;netdev_dbg (netdev,"RX Multi-buffer hcall supported by FW, batch set to %u\n" , - adapter->rx_buffers_per_hcall); + adapter->rx_buffers_per_hcall); }else { adapter->rx_buffers_per_hcall =1 ; netdev_dbg(netdev,
@@ -2558,15 +2848,24 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) {struct kobject *kobj = &adapter->rx_buff_pool[0 ][i].kobj; -int error;ibmveth_init_buffer_pool (&adapter->rx_buff_pool[0 ][i], i, pool_count[i], pool_size[i], pool_active[i]); - error =kobject_init_and_add (kobj, &ktype_veth_pool, - &dev->dev.kobj,"pool%d" , i); -if (!error) -kobject_uevent (kobj,KOBJ_ADD ); + rc =kobject_init_and_add (kobj, &ktype_veth_pool, + &dev->dev.kobj,"pool%d" , i); +if (rc) { +dev_err (&dev->dev, +"failed to create pool%d kobject: %d\n" , i, rc); +/* init_and_add takes a ref even on failure */ +kobject_put (kobj); +ibmveth_put_pool_kobjs (adapter, pools_ready); +free_netdev (netdev); +return rc; + } + + pools_ready++; +kobject_uevent (kobj,KOBJ_ADD ); } rc = netif_set_real_num_tx_queues(netdev, min(num_online_cpus(),
@@ -2574,6 +2873,7 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)if (rc) {netdev_dbg (netdev,"failed to set number of tx queues rc=%d\n" , rc); +ibmveth_put_pool_kobjs (adapter, pools_ready);free_netdev (netdev);return rc; }
@@ -2590,6 +2890,7 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)if (rc) {netdev_dbg (netdev,"failed to register netdev rc=%d\n" , rc); +ibmveth_put_pool_kobjs (adapter, pools_ready);free_netdev (netdev);return rc; }
@@ -2761,7 +3062,6 @@ static ssize_t veth_pool_store(struct kobject *kobj, struct attribute *attr,return rc; } -#define ATTR (_name, _mode) \ struct attribute veth_##_name##_attr = { \ .name = __stringify(_name), .mode = _mode, \
drivers/net/ethernet/ibm/ibmveth.h +2 -1
@@ -30,6 +30,7 @@#define IbmVethMcastRemoveFilter 0x2UL#define IbmVethMcastClearFilterTable 0x3UL +#define IBMVETH_ILLAN_RX_MULTI_QUEUE_SUPPORT 0x0000000000080000UL#define IBMVETH_ILLAN_RX_MULTI_BUFF_SUPPORT 0x0000000000040000UL#define IBMVETH_ILLAN_LRG_SR_ENABLED 0x0000000000010000UL#define IBMVETH_ILLAN_LRG_SND_SUPPORT 0x0000000000008000UL
@@ -260,7 +261,7 @@ static inline long h_illan_attributes(unsigned long unit_address,#define IBMVETH_MAX_TX_BUF_SIZE (1024 * 64)#define IBMVETH_MAX_QUEUES 16U#define IBMVETH_DEFAULT_QUEUES 8U -#define IBMVETH_MAX_RX_QUEUES 1U +#define IBMVETH_MAX_RX_QUEUES 16U#define IBMVETH_DEFAULT_RX_QUEUES 1U#define IBMVETH_MAX_RX_REGULAR 8U#define IBMVETH_MAX_RX_QUEUE 12U
Signature · 1 line
2.50.1 (Apple Git-155)
Jakub Kicinski
maintainerRe: [PATCH net-next v4 09/14] ibmveth: Enable multi-queue RX receive path
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
ibmveth: Enable multi-queue RX receive path
This patch turns on the multi-queue RX path: probe latches multi_queue and num_rx_queues from the H_ILLAN_ATTRIBUTES MQ bit, subordinate queues are registered with H_REG_LOGICAL_LAN_QUEUE, and open replenishes every queue before setup_rx_interrupts() unmasks PHYP. A few questions below on the new ordering, the CMO entitlement accounting and the probe unwind.
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0
[ ... ]
Quoted 7 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-1@@ -97,7 +97,17 @@ static struct ibmveth_stat ibmveth_stats[] = { { "fw_enabled_ipv6_csum", IBMVETH_STAT_OFF(fw_ipv6_csum_support) }, { "tx_large_packets", IBMVETH_STAT_OFF(tx_large_packets) }, { "rx_large_packets", IBMVETH_STAT_OFF(rx_large_packets) }, - { "fw_enabled_large_send", IBMVETH_STAT_OFF(fw_large_send_support) } + { "fw_enabled_large_send", IBMVETH_STAT_OFF(fw_large_send_support) }, + { "hcall_reg_lan_queue", IBMVETH_STAT_OFF(hcall_stats.reg_lan_queue) },
drivers/net/ethernet/ibm/ibmveth.c static struct ibmveth_stat ibmveth_stats[] = { · line 111
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +9
-0+ { "hcall_reg_lan", IBMVETH_STAT_OFF(hcall_stats.reg_lan) }, + { "hcall_add_bufs_queue", + IBMVETH_STAT_OFF(hcall_stats.add_bufs_queue) }, + { "hcall_add_bufs", IBMVETH_STAT_OFF(hcall_stats.add_bufs) }, + { "hcall_add_buf", IBMVETH_STAT_OFF(hcall_stats.add_buf) }, + { "hcall_free_lan_queue", + IBMVETH_STAT_OFF(hcall_stats.free_lan_queue) }, + { "hcall_free_lan", IBMVETH_STAT_OFF(hcall_stats.free_lan) }, + { "hcall_send_lan", IBMVETH_STAT_OFF(hcall_stats.send_lan) }, };
[Severity: Low]
Should hcall_send_lan be exported through ethtool -S? It is incremented once per successfully transmitted packet on the ibmveth_send() success path:
drivers/net/ethernet/ibm/ibmveth.c:ibmveth_send() {
...
adapter->hcall_stats.send_lan++;
return 0;
}That makes it the same value as the standard rtnl_link_stats64 tx_packets counter reported by ndo_get_stats64 / ip -s link, and Documentation/networking/statistics.rst asks that counters with a standard uAPI not be duplicated as driver-private ethtool -S strings. These strings become uAPI once merged.
The commit message subject is "Enable multi-queue RX receive path" and does not mention that eight new statistics strings are being added. Could the stats additions be split out or at least described?
Quoted 6 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-0@@ -429,12 +439,64 @@ ibmveth_enable_irq(struct ibmveth_adapter *adapter, int queue_index)return ibmveth_toggle_irq (adapter, queue_index, true); } +/** +* ibmveth_dispose_subordinate_irq_mapping - Drop one subordinate virq mapping
drivers/net/ethernet/ibm/ibmveth.c ibmveth_enable_irq(struct ibmveth_adapter *adapter, int queue_index) · line 453
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+* @adapter: ibmveth adapter structure +* @queue_idx: RX queue index (1..N) +* +* Subordinate queues get mappings from irq_create_mapping() during PHYP +* registration. Queue 0 uses netdev->irq from device tree and is left alone. +* +* Bound against IBMVETH_MAX_RX_QUEUES, not num_rx_queues: scale-down and +* scale-up fail paths dispose queues that are no longer in the published +* live set but still own a virq in queue_irq[]. The bulk helper still +* iterates only 1..num_rx_queues-1 for close/open-fail cleanup.
[Severity: Low]
This isn't a bug, but do the "scale-down and scale-up fail paths" exist at this commit? ibmveth_resize_rx_queues_incremental() and ibmveth_setup_single_rx_interrupt() arrive in a later patch, and the only caller here is the bulk helper bounded by num_rx_queues.
The same applies to the comment added above ibmveth_get_strings() in this patch:
/*
* Sum per-queue counters for rare ethtool reads. Do not write adapter
* globals on the hot path (ibmvnic-style); with qstats allocated for the
* adapter lifetime, these sums remain meaningful across ifdown/up. */
ibmveth_get_strings() only memcpy()s string tables here, and no rx_qstats/tx_qstats members exist in struct ibmveth_adapter until the following patch. Could both comments move to the patches that add the code they describe, so the intermediate commits read correctly?
Quoted 19 lines
drivers/net/ethernet/ibm/ibmveth.c +18
-0+for (i =0 ; i < num; i++) +napi_enable (&adapter->napi[i]); + +if (adapter->multi_queue && num >1 ) { +for (i =0 ; i < num; i++) { + rc =ibmveth_enable_irq (adapter, i); +if (rc) { +netdev_err (netdev, +"Failed to enable IRQ for queue %d, rc=%d\n" , + i, rc); +while (--i >=0 ) +ibmveth_disable_irq (adapter, i); + rc = -EIO ; + goto err_disable_napi; + } + } + } +return 0 ;
drivers/net/ethernet/ibm/ibmveth.c ibmveth_setup_rx_interrupts(struct ibmveth_adapter *adapter) · line 554
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +9
-0+err_disable_napi: +/* PHYP unmask was rolled back above; disable NAPI before free_irq */ +for (i =0 ; i < num; i++) +napi_disable (&adapter->napi[i]); +for (i =0 ; i < num; i++) { +if (adapter->queue_irq[i]) +free_irq (adapter->queue_irq[i], &adapter->napi[i]); + } + goto err_dispose_mappings;
[Severity: Low]
Can the rollback loop above be undone by an in-flight poll? Queues 0..i-1 were already unmasked, so a poll may be running when ibmveth_enable_irq() fails for queue i:
ibmveth_poll() {
...
if (!napi_complete_done(napi, frames_processed))
goto out;
...
lpar_rc = ibmveth_enable_irq(adapter, queue_index);
}There is no synchronize_irq() between the while (--i >= 0) ibmveth_disable_irq() loop and napi_disable(), so a poll that completes after the mask re-enables PHYP delivery. napi_disable() waits for the poll but does not undo that re-enable, and the following free_irq() plus ibmveth_dispose_subordinate_irq_mappings() then leave the hypervisor source enabled for a hwirq whose Linux mapping is gone.
The scale-down path added later does disable_irq() followed by synchronize_irq() for exactly this reason. Should this path do the same?
[ ... ]
Quoted 6 lines
drivers/net/ethernet/ibm/ibmveth.c +1
-1@@ -834,9 +914,15 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,*/ static void ibmveth_update_rx_no_buffer (struct ibmveth_adapter *adapter) { -__be64 *p = adapter->buffer_list_addr[0 ] +4096 -8 ; +int i;
drivers/net/ethernet/ibm/ibmveth.c static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter, · line 926
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +7
-1- adapter->rx_no_buffer =be64_to_cpup (p); + adapter->rx_no_buffer =0 ; +for (i =0 ; i < adapter->num_rx_queues; i++) { +__be64 *p = adapter->buffer_list_addr[i] +4096 -8 ; +u64 drops =be64_to_cpup (p); + + adapter->rx_no_buffer += drops; + } }
[Severity: High]
Is this cross-queue aggregation safe against the RX queue resize path? The only lock held here is the caller's own per-queue lock:
ibmveth_replenish_task(adapter, queue_index) {
spin_lock_irqsave(&rxq->replenish_lock, flags);
...
ibmveth_update_rx_no_buffer(adapter);
spin_unlock_irqrestore(&rxq->replenish_lock, flags);
}That lock does not cover other queues' buffer_list_addr[] entries, nor num_rx_queues. The ethtool -L scale-down path added later in the series publishes the new count and then frees the pages:
adapter->num_rx_queues = new_count;
for (i = new_count; i < old_count; i++) {
...
ibmveth_free_single_rx_queue(adapter, i); /* free_page() + NULL */
}If a surviving queue's poll already loaded the older, larger num_rx_queues, can this loop dereference NULL + 4096 - 8, or read a page that was just freed, from softirq context? A READ_ONCE() of the count plus a NULL check, or serializing the aggregation against the resize path, would make this explicit.
Separately, two concurrent polls on different queues both execute adapter->rx_no_buffer = 0 followed by += drops on the same field. Can that lose increments and expose a transient 0 through rx_no_buffer / ethtool -S?
drivers/net/ethernet/ibm/ibmveth.c static void ibmveth_replenish_task(struct ibmveth_adapter *adapter, · line 943
Quoted 13 lines
drivers/net/ethernet/ibm/ibmveth.c +5
-1@@ -847,8 +933,12 @@ static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,unsigned long flags;int i; -if (queue_index >= adapter->num_rx_queues) +if (queue_index >= adapter->num_rx_queues) { +netdev_dbg (adapter->netdev, +"Skipping replenish for freed queue %d (num_queues=%d)\n" , + queue_index, adapter->num_rx_queues);return ; + } adapter->replenish_task_cycles++;
[Severity: Low]
With several RX queues now polling concurrently, do the adapter-wide plain u64 counters need serialization? replenish_task_cycles is incremented before the per-queue lock is taken, and replenish_no_mem, replenish_add_buff_success/failure and the new hcall_stats fields are touched under different per-queue locks, so the lockset intersection for these adapter-global fields is empty. hcall_stats.send_lan is also incremented from ibmveth_start_xmit() on up to 16 TX queues in parallel. The visible effect is lost increments in the ethtool -S values.
[ ... ]
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter, · line 1377
Quoted 5 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-0@@ -1284,6 +1374,137 @@ static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter, + lpar_rc =h_reg_logical_lan_queue (adapter->vdev->unit_address, + adapter->buffer_list_dma[queue_index], + rxq_desc.desc, &handle, &hwirq); + adapter->hcall_stats.reg_lan_queue++;
[Severity: Low]
Should this hcall retry on a busy return? Every other long-running hcall in the driver loops:
do {
lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
} while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));including h_free_logical_lan_queue() in the IRQ-mapping failure path a few lines below this call, and h_send_logical_lan() in ibmveth_send(). As written, a transient H_BUSY / H_LONG_BUSY_ORDER_* here becomes -EIO from ibmveth_register_single_rx_queue() and a failed ifup.
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter, · line 1452
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+/* +* H_FUNCTION means firmware rejected this subordinate register +* (MQ unsupported). That is a hard open failure: do not clear +* multi_queue or claim single-queue fallback. Keep a specific +* log, then the generic failure lines below (no early return). +*/ +if (lpar_rc ==H_FUNCTION ) +netdev_err (adapter->netdev, +"h_reg_logical_lan_queue H_FUNCTION for queue %d (firmware MQ unsupported)\n" , + queue_index);
[Severity: High]
Can the adapter recover if firmware stops honouring the MQ register after probe? multi_queue and num_rx_queues are latched once in ibmveth_probe() from the H_ILLAN_ATTRIBUTES bit and are never re-read or reduced, so a H_FUNCTION here fails ibmveth_open() and every subsequent open the same way.
The reset work item only closes and reopens, it never re-reads the attributes:
ibmveth_reset() {
rtnl_lock();
dev_close(adapter->netdev);
dev_open(adapter->netdev, NULL);
rtnl_unlock();
}ethtool -L ethX rx 1 while down does not restore RX either: multi_queue stays 1, so rx_buffers_per_hcall remains IBMVETH_MAX_RX_QUEUE and ibmveth_add_logical_lan_buffers() keeps calling h_add_logical_lan_buffers_queue(), whose H_FUNCTION path has no fallback when multi_queue is set.
The driver already assumes PHYP capabilities can disappear across LPM:
/*
* Live Partition Migration may drop multi-
* buffer support. Fall back to single-buffer
* on the next replenish; ...
*/
Would clearing multi_queue and falling back to one queue on H_FUNCTION be preferable to leaving the interface permanently unable to open?
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter, · line 1485
Quoted 9 lines
drivers/net/ethernet/ibm/ibmveth.c +9
-0+static int +ibmveth_register_single_rx_queue (struct ibmveth_adapter *adapter, +int queue_idx,u64 mac_address) +{ +struct net_device *netdev = adapter->netdev; +union ibmveth_buf_desc rxq_desc; +long lpar_rc; + + (void )mac_address;
[Severity: Low]
This isn't a bug, but is the mac_address parameter worth keeping? H_REG_LOGICAL_LAN_QUEUE has no MAC operand and ibmveth_register_logical_lan_queue() has no MAC parameter, so callers compute ether_addr_to_u64(netdev->dev_addr) only to feed a parameter that is voided here. The (void)x; suppression cast is also not needed under the kernel's warning set.
[ ... ]
Quoted 27 lines
drivers/net/ethernet/ibm/ibmveth.c +20
-1@@ -1356,9 +1579,31 @@ ibmveth_register_rx_queues(struct ibmveth_adapter *adapter, u64 mac_address) +for (i =1 ; i < adapter->num_rx_queues; i++) { + rc =ibmveth_register_single_rx_queue (adapter, i, mac_address); +if (rc) + goto err_unregister; + } +netdev_dbg (netdev, -"registered 1 RX queue with hypervisor (single-queue mode)\n" ); +"registered %d RX queues with hypervisor (multi-queue mode)\n" , + adapter->num_rx_queues); +return 0 ; + +err_unregister: +ibmveth_dispose_subordinate_irq_mappings (adapter); +ibmveth_free_all_queues (adapter); +return rc; }@@ -1396,12 +1641,29 @@ static int ibmveth_open(struct net_device *netdev) +/* +* MQ: post buffers before setup_rx_interrupts() unmasks PHYP +* (avoids drops if traffic arrives during open; PHYP allows +* either order). Single-queue keeps the classic kick: setup +* (no unmask) then schedule_rx_queue() so the first poll +* replenishes and enables.
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_open(struct net_device *netdev) · line 1656
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +9
-0+*/ +if (adapter->multi_queue && adapter->num_rx_queues >1 ) { +for (i =0 ; i < adapter->num_rx_queues; i++) { +netdev_dbg (netdev, +"initial replenish cycle for queue %d\n" , i); +ibmveth_replenish_task (adapter, i); + } + } + rc =ibmveth_setup_rx_interrupts (adapter);
[Severity: Medium]
Is PHYP delivery for subordinate queues guaranteed to be masked in this window? Queue 0 is explicitly masked before registration:
ibmveth_register_rx_queues() {
adapter->queue_irq[0] = netdev->irq;
rc = ibmveth_disable_irq(adapter, 0);
...
}but queues 1..N get no equivalent H_VIOCTL/H_DISABLE_VIO_INTERRUPT after H_REG_LOGICAL_LAN_QUEUE plus irq_create_mapping(). Buffers are now posted for every queue here, before any handler exists, and ibmveth_setup_rx_interrupts() was also reordered so request_irq() for all queues completes before any napi_enable():
for (i = 0; i < num; i++) {
...
rc = request_irq(adapter->queue_irq[i], ibmveth_interrupt, ...);
}
for (i = 0; i < num; i++)
napi_enable(&adapter->napi[i]);If an interrupt arrives in that window, ibmveth_schedule_rx_queue() finds NAPI still disabled:
if (napi_schedule_prep(napi)) {
lpar_rc = ibmveth_disable_irq(adapter, qindex);
WARN_ON(lpar_rc != H_SUCCESS);
__napi_schedule(napi);
}so the handler returns IRQ_HANDLED without masking the hypervisor source and without scheduling a poll. That is the same storm window described in ibmveth_cleanup_rx_interrupts() ("Mask PHYP before napi_disable so ibmveth_interrupt cannot return IRQ_HANDLED without masking"). The err_free_irqs path has the same gap: it frees handlers and disposes subordinate mappings without masking already-registered subordinate queues.
The later scale-up path uses register -> request_irq -> publish -> replenish -> napi_enable -> enable_irq and documents why. Would enabling NAPI before request_irq (as the baseline did), or explicitly masking each subordinate queue right after registration, close this window?
[ ... ]
Quoted 8 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-4@@ -2373,18 +2632,25 @@ static unsigned long ibmveth_get_desired_dma(struct vio_dev *vdev)/* add size of mapped tx buffers */ ret +=IOMMU_PAGE_ALIGN (IBMVETH_MAX_TX_BUF_SIZE , tbl); -for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) { -/* add the size of the active receive buffers */ -if (adapter->rx_buff_pool[0 ][i].active) - ret +=
drivers/net/ethernet/ibm/ibmveth.c static unsigned long ibmveth_get_desired_dma(struct vio_dev *vdev) · line 2636
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-8- adapter->rx_buff_pool[0 ][i].size * -IOMMU_PAGE_ALIGN (adapter->rx_buff_pool[0 ][i]. - buff_size, tbl); - rxqentries += adapter->rx_buff_pool[0 ][i].size; - } -/* add the size of the receive queue entries */ - ret +=IOMMU_PAGE_ALIGN ( - rxqentries *sizeof (struct ibmveth_rx_q_entry ), tbl); +for (q =0 ; q < adapter->num_rx_queues; q++) { +int rxqentries =1 ;
[Severity: Medium]
Does the buffer-list term still need scaling here? The unchanged line above this loop counts it once:
ret = IBMVETH_BUFF_LIST_SIZE + IBMVETH_FILT_LIST_SIZE;
while multi-queue allocates and dma_map_single()s a distinct 4096-byte buffer-list page per queue, which ibmveth_cleanup_rx_resources() unmaps per index. That leaves the desired value short by (num_rx_queues - 1) buffer-list pages.
Also, is anything updating the CMO entitlement for the new default queue count? The VIO core samples it before the driver's probe runs:
arch/powerpc/platforms/pseries/vio.c:vio_cmo_bus_probe() {
viodev->cmo.desired =
IOMMU_PAGE_ALIGN(viodrv->get_desired_dma(viodev), tbl);
}with dev_get_drvdata() still NULL, so the single-queue sized IBMVETH_IO_ENTITLEMENT_DEFAULT is used for an adapter that will open with up to 8 RX queues, and neither ibmveth_probe() nor ibmveth_open() calls vio_cmo_set_dev_desired(). On a CMO partition, can this make dma_map_single()/dma_alloc_coherent() fail during open on adapters that worked in single-queue mode?
drivers/net/ethernet/ibm/ibmveth.c static unsigned long ibmveth_get_desired_dma(struct vio_dev *vdev) · line 2641
Quoted 5 lines
drivers/net/ethernet/ibm/ibmveth.c +5
-0+ +for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) { +/* add the size of the active receive buffers */ +struct ibmveth_buff_pool *bpool = + &adapter->rx_buff_pool[q][i];
[Severity: Medium]
Is rx_buff_pool[q][i] populated for q >= 1 when this runs? Pool metadata for queues 1..N is only filled in by ibmveth_alloc_buffer_pools(), which runs from ndo_open; ibmveth_probe() initializes only rx_buff_pool[0][i], and the private area is zeroed, so before the first open these entries have size = active = buff_size = 0.
ibmveth_change_mtu() pushes the result straight into firmware accounting:
vio_cmo_set_dev_desired(viodev, ibmveth_get_desired_dma(viodev));
Can an MTU change while the interface is down therefore lower the entitlement to roughly the single-queue value, right before an open that allocates buffers for up to 8 queues?
Quoted 13 lines
drivers/net/ethernet/ibm/ibmveth.c +9
-0@@ -2449,9 +2715,18 @@ static const struct net_device_ops ibmveth_netdev_ops = {#endif }; +static void ibmveth_put_pool_kobjs (struct ibmveth_adapter *adapter, +int pools_ready) +{ +int i; + +for (i =0 ; i < pools_ready; i++) +kobject_put (&adapter->rx_buff_pool[0 ][i].kobj); +} +
[ ... ]
Quoted 8 lines
drivers/net/ethernet/ibm/ibmveth.c +1
-3@@ -2531,16 +2810,27 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id) netdev->features |=NETIF_F_FRAGLIST ; } -/* Initialize queue count - always 1 for now */ - adapter->multi_queue =0 ; - adapter->num_rx_queues =IBMVETH_DEFAULT_RX_QUEUES ; + if (ret == H_SUCCESS &&
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id) · line 2823
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+ (ret_attr & IBMVETH_ILLAN_RX_MULTI_QUEUE_SUPPORT)) { + adapter->multi_queue = 1; + adapter->num_rx_queues = min(num_online_cpus(), + IBMVETH_DEFAULT_QUEUES); + netdev_dbg(netdev, "RX multi queue mode enabled: %d queues\n", + adapter->num_rx_queues); + } else { + adapter->multi_queue =0 ; + adapter->num_rx_queues =IBMVETH_DEFAULT_RX_QUEUES ; + }
[Severity: Medium]
Now that num_rx_queues can be greater than one here, does ibmveth_resume() need updating too? It still kicks queue 0 only:
ibmveth_resume() {
...
ibmveth_schedule_rx_queue(adapter, 0);
return 0;
}The kick exists because the poll routine is what replenishes buffers and re-arms the PHYP interrupt via napi_complete_done() -> ibmveth_enable_irq(). After a suspend/resume or partition migration, queues 1..N-1 would then have no buffers posted and no interrupt re-armed until an ifdown/ifup.
Quoted 18 lines
drivers/net/ethernet/ibm/ibmveth.c +5
-5@@ -2558,15 +2848,24 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) {struct kobject *kobj = &adapter->rx_buff_pool[0 ][i].kobj; -int error;ibmveth_init_buffer_pool (&adapter->rx_buff_pool[0 ][i], i, pool_count[i], pool_size[i], pool_active[i]); - error =kobject_init_and_add (kobj, &ktype_veth_pool, - &dev->dev.kobj,"pool%d" , i); -if (!error) -kobject_uevent (kobj,KOBJ_ADD ); + rc =kobject_init_and_add (kobj, &ktype_veth_pool, + &dev->dev.kobj,"pool%d" , i); +if (rc) { +dev_err (&dev->dev, +"failed to create pool%d kobject: %d\n" , i, rc);
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id) · line 2869
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +9
-0+/* init_and_add takes a ref even on failure */ +kobject_put (kobj); +ibmveth_put_pool_kobjs (adapter, pools_ready); +free_netdev (netdev); +return rc; + } + + pools_ready++; +kobject_uevent (kobj,KOBJ_ADD ); }
[Severity: Medium]
Is it safe to free_netdev() immediately after ibmveth_put_pool_kobjs()? The kobjects are embedded in netdev_priv(netdev), and ktype_veth_pool has no release callback:
static struct kobj_type ktype_veth_pool = {
.release = NULL,
...
};With CONFIG_DEBUG_KOBJECT_RELEASE=y the final put defers cleanup using a work item stored inside the kobject itself:
lib/kobject.c:kobject_release() {
#ifdef CONFIG_DEBUG_KOBJECT_RELEASE unsigned long delay = HZ + HZ * get_random_u32_below(4); ... INIT_DELAYED_WORK(&kobj->release, kobject_delayed_cleanup); schedule_delayed_work(&kobj->release, delay); #else kobject_cleanup(kobj);
#endif
}
free_netdev() then releases that memory with no window, so kobject_delayed_cleanup() would read kobj->state_in_sysfs, kobj->parent and kobj->name from freed storage. The same shape applies to the two other new put_pool_kobjs() call sites below.
Quoted 16 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-0@@ -2574,6 +2873,7 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)if (rc) {netdev_dbg (netdev,"failed to set number of tx queues rc=%d\n" , rc); +ibmveth_put_pool_kobjs (adapter, pools_ready);free_netdev (netdev);return rc; }@@ -2590,6 +2890,7 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)if (rc) {netdev_dbg (netdev,"failed to register netdev rc=%d\n" , rc); +ibmveth_put_pool_kobjs (adapter, pools_ready);free_netdev (netdev);return rc; }
[ ... ]
Re: [PATCH net-next v4 09/14] ibmveth: Enable multi-queue RX receive path
Quoted 2 lines · Jakub Kicinski
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
Hi Jakub,
Thanks for the review.
Quoted 49 lines
---
ibmveth: Enable multi-queue RX receive pathThis patch turns on the multi-queue RX path: probe latches
multi_queueandnum_rx_queuesfrom theH_ILLAN_ATTRIBUTESMQ bit, subordinate queues are registered withH_REG_LOGICAL_LAN_QUEUE, and open replenishes every queue beforesetup_rx_interrupts()unmasks PHYP. A few questions below on the new ordering, the CMO entitlement accounting and the probe unwind.Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0[ ... ]
Quoted 7 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-1@@ -97,7 +97,17 @@ static struct ibmveth_stat ibmveth_stats[] = { { "fw_enabled_ipv6_csum", IBMVETH_STAT_OFF(fw_ipv6_csum_support) }, { "tx_large_packets", IBMVETH_STAT_OFF(tx_large_packets) }, { "rx_large_packets", IBMVETH_STAT_OFF(rx_large_packets) }, - { "fw_enabled_large_send", IBMVETH_STAT_OFF(fw_large_send_support) } + { "fw_enabled_large_send", IBMVETH_STAT_OFF(fw_large_send_support) }, + { "hcall_reg_lan_queue", IBMVETH_STAT_OFF(hcall_stats.reg_lan_queue) },drivers/net/ethernet/ibm/ibmveth.c static struct ibmveth_stat ibmveth_stats[] = { · line 111
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +9
-0+ { "hcall_reg_lan", IBMVETH_STAT_OFF(hcall_stats.reg_lan) }, + { "hcall_add_bufs_queue", + IBMVETH_STAT_OFF(hcall_stats.add_bufs_queue) }, + { "hcall_add_bufs", IBMVETH_STAT_OFF(hcall_stats.add_bufs) }, + { "hcall_add_buf", IBMVETH_STAT_OFF(hcall_stats.add_buf) }, + { "hcall_free_lan_queue", + IBMVETH_STAT_OFF(hcall_stats.free_lan_queue) }, + { "hcall_free_lan", IBMVETH_STAT_OFF(hcall_stats.free_lan) }, + { "hcall_send_lan", IBMVETH_STAT_OFF(hcall_stats.send_lan) }, };[Severity: Low]
Shouldhcall_send_lanbe exported through ethtool -S? It is incremented once per successfully transmitted packet on theibmveth_send()success path:drivers/net/ethernet/ibm/ibmveth.c:ibmveth_send() { ... adapter->hcall_stats.send_lan++; return 0; }That makes it the same value as the standard
rtnl_link_stats64tx_packetscounter reported byndo_get_stats64/ ip -s link, andDocumentation/networking/statistics.rstasks that counters with a standard uAPI not be duplicated as driver-private ethtool -S strings. These strings become uAPI once merged.
Quoted 3 lines
The commit message subject is "Enable multi-queue RX receive path" and does not mention that eight new statistics strings are being added. Could the stats additions be split out or at least described?
They do bump on the same success path. The useful distinction here is role, not packet meaning: hcall_send_lan is a PHYP hcall/debug counter, while tx_packets is the standard stack TX success counter.
I’m planning to move the hcall_* string-table growth out of P09 so this patch does not expand ethtool -S at the same time as the MQ RX enablement change.
Quoted 31 lines
Quoted 6 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-0@@ -429,12 +439,64 @@ ibmveth_enable_irq(struct ibmveth_adapter *adapter, int queue_index)return ibmveth_toggle_irq (adapter, queue_index, true); } +/** +* ibmveth_dispose_subordinate_irq_mapping - Drop one subordinate virq mapping drivers/net/ethernet/ibm/ibmveth.c ibmveth_enable_irq(struct ibmveth_adapter *adapter, int queue_index) · line 453
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+* @adapter: ibmveth adapter structure +* @queue_idx: RX queue index (1..N) +* +* Subordinate queues get mappings from irq_create_mapping() during PHYP +* registration. Queue 0 uses netdev->irq from device tree and is left alone. +* +* Bound against IBMVETH_MAX_RX_QUEUES, not num_rx_queues: scale-down and +* scale-up fail paths dispose queues that are no longer in the published +* live set but still own a virq in queue_irq[]. The bulk helper still +* iterates only 1..num_rx_queues-1 for close/open-fail cleanup. [Severity: Low]
This isn't a bug, but do the "scale-down and scale-up fail paths" exist at this commit?ibmveth_resize_rx_queues_incremental()andibmveth_setup_single_rx_interrupt()arrive in a later patch, and the only caller here is the bulk helper bounded bynum_rx_queues.The same applies to the comment added above
ibmveth_get_strings()in this patch:/*
* Sum per-queue counters for rare ethtool reads. Do not write adapter
* globals on the hot path (ibmvnic-style); with qstats allocated for the
* adapter lifetime, these sums remain meaningful across ifdown/up. */
Quoted 4 lines
ibmveth_get_strings()only memcpy()s string tables here, and no rx_qstats/tx_qstats members exist in structibmveth_adapteruntil the following patch. Could both comments move to the patches that add the code they describe, so the intermediate commits read correctly?
Yes. Those comments should move with first use at restack, so the intermediate commits read correctly.
Quoted 49 lines
Quoted 19 lines
drivers/net/ethernet/ibm/ibmveth.c +18
-0+for (i =0 ; i < num; i++) +napi_enable (&adapter->napi[i]); + +if (adapter->multi_queue && num >1 ) { +for (i =0 ; i < num; i++) { + rc =ibmveth_enable_irq (adapter, i); +if (rc) { +netdev_err (netdev, +"Failed to enable IRQ for queue %d, rc=%d\n" , + i, rc); +while (--i >=0 ) +ibmveth_disable_irq (adapter, i); + rc = -EIO ; + goto err_disable_napi; + } + } + } +return 0 ;drivers/net/ethernet/ibm/ibmveth.c ibmveth_setup_rx_interrupts(struct ibmveth_adapter *adapter) · line 554
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +9
-0+err_disable_napi: +/* PHYP unmask was rolled back above; disable NAPI before free_irq */ +for (i =0 ; i < num; i++) +napi_disable (&adapter->napi[i]); +for (i =0 ; i < num; i++) { +if (adapter->queue_irq[i]) +free_irq (adapter->queue_irq[i], &adapter->napi[i]); + } + goto err_dispose_mappings;[Severity: Low]
Can the rollback loop above be undone by an in-flight poll? Queues 0..i-1 were already unmasked, so a poll may be running whenibmveth_enable_irq()fails for queue i:ibmveth_poll() { ... if (!napi_complete_done(napi, frames_processed)) goto out; ... lpar_rc = ibmveth_enable_irq(adapter, queue_index); }There is no
synchronize_irq()between the while (--i >= 0)ibmveth_disable_irq()loop andnapi_disable(), so a poll that completes after the mask re-enables PHYP delivery.napi_disable()waits for the poll but does not undo that re-enable, and the followingfree_irq()plusibmveth_dispose_subordinate_irq_mappings()then leave the hypervisor source enabled for a hwirq whose Linux mapping is gone.
Quoted 4 lines
The scale-down path added later does
disable_irq()followed bysynchronize_irq()for exactly this reason. Should this path do the same?[ ... ]
Yes. The enable-IRQ rollback path should synchronize before tearing down NAPI and freeing the IRQs.
I’m planning to make that failure path follow the same disable/synchronize/disable-NAPI pattern as the teardown paths, so a poll cannot re-enable delivery after rollback has started.
Quoted 44 lines
Quoted 6 lines
drivers/net/ethernet/ibm/ibmveth.c +1
-1@@ -834,9 +914,15 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,*/ static void ibmveth_update_rx_no_buffer (struct ibmveth_adapter *adapter) { -__be64 *p = adapter->buffer_list_addr[0 ] +4096 -8 ; +int i;drivers/net/ethernet/ibm/ibmveth.c static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter, · line 926
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +7
-1- adapter->rx_no_buffer =be64_to_cpup (p); + adapter->rx_no_buffer =0 ; +for (i =0 ; i < adapter->num_rx_queues; i++) { +__be64 *p = adapter->buffer_list_addr[i] +4096 -8 ; +u64 drops =be64_to_cpup (p); + + adapter->rx_no_buffer += drops; + } }[Severity: High]
Is this cross-queue aggregation safe against the RX queue resize path? The only lock held here is the caller's own per-queue lock:ibmveth_replenish_task(adapter, queue_index) { spin_lock_irqsave(&rxq->replenish_lock, flags); ... ibmveth_update_rx_no_buffer(adapter); spin_unlock_irqrestore(&rxq->replenish_lock, flags); }That lock does not cover other queues' buffer_list_addr[] entries, nor
num_rx_queues. The ethtool -L scale-down path added later in the series publishes the new count and then frees the pages:adapter->num_rx_queues = new_count; for (i = new_count; i < old_count; i++) { ... ibmveth_free_single_rx_queue(adapter, i); /* free_page() + NULL */ }If a surviving queue's poll already loaded the older, larger
num_rx_queues, can this loop dereference NULL + 4096 - 8, or read a page that was just freed, from softirq context? AREAD_ONCE()of the count plus a NULL check, or serializing the aggregation against the resize path, would make this explicit.
Quoted 4 lines
Separately, two concurrent polls on different queues both execute
adapter->rx_no_buffer= 0 followed by += drops on the same field. Can that lose increments and expose a transient 0 throughrx_no_buffer/ ethtool -S?
Yes. That cross-queue rx_no_buffer aggregation is not safe once queue resize enters the picture, and even without resize it is the wrong shape for concurrent polls.
I’m planning to make that helper queue-local and NULL-safe, so it only touches the current queue’s buffer-list slot and keeps the absolute PHYP drop count in that queue’s stats.
Then the adapter-visible rx_no_buffer total can be derived by summing on read, rather than having different queues race on one shared field.
Quoted 23 lines
drivers/net/ethernet/ibm/ibmveth.c static void ibmveth_replenish_task(struct ibmveth_adapter *adapter, · line 943
Quoted 13 lines
drivers/net/ethernet/ibm/ibmveth.c +5
-1@@ -847,8 +933,12 @@ static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,unsigned long flags;int i; -if (queue_index >= adapter->num_rx_queues) +if (queue_index >= adapter->num_rx_queues) { +netdev_dbg (adapter->netdev, +"Skipping replenish for freed queue %d (num_queues=%d)\n" , + queue_index, adapter->num_rx_queues);return ; + } adapter->replenish_task_cycles++;[Severity: Low]
With several RX queues now polling concurrently, do the adapter-wide plain u64 counters need serialization?replenish_task_cyclesis incremented before the per-queue lock is taken, andreplenish_no_mem, replenish_add_buff_success/failure and the newhcall_statsfields are touched under different per-queue locks, so the lockset intersection for these adapter-global fields is empty. hcall_stats.send_lan is also incremented fromibmveth_start_xmit()on up to 16 TX queues in parallel. The visible effect is lost increments in the ethtool -S values.
Quoted 1 line
[ ... ]
Yes, once MQ is active those adapter-global counters can lose updates across queues.
I still see that as the same broader informational-counter issue as in the earlier patches. rx_no_buffer is addressed separately by the queue-local fix above.
Quoted 9 lines
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter, · line 1377
Quoted 5 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-0@@ -1284,6 +1374,137 @@ static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter, + lpar_rc =h_reg_logical_lan_queue (adapter->vdev->unit_address, + adapter->buffer_list_dma[queue_index], + rxq_desc.desc, &handle, &hwirq); + adapter->hcall_stats.reg_lan_queue++;[Severity: Low]
Should this hcall retry on a busy return? Every other long-running hcall in the driver loops:
Quoted 8 lines
do { lpar_rc = h_free_logical_lan(adapter->vdev->unit_address); } while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));including
h_free_logical_lan_queue()in the IRQ-mapping failure path a few lines below this call, andh_send_logical_lan()inibmveth_send(). As written, a transientH_BUSY/ H_LONG_BUSY_ORDER_* here becomes-EIOfromibmveth_register_single_rx_queue()and a failed ifup.
Yes. h_reg_logical_lan_queue() should handle H_BUSY and long-busy returns with a busy/long-busy spin on the subordinate register path.
I’m planning to add that there, but not bring back the older blind retry shape that repeated the same subordinate register attempt without changing any state.
Quoted 35 lines
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter, · line 1452
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+/* +* H_FUNCTION means firmware rejected this subordinate register +* (MQ unsupported). That is a hard open failure: do not clear +* multi_queue or claim single-queue fallback. Keep a specific +* log, then the generic failure lines below (no early return). +*/ +if (lpar_rc ==H_FUNCTION ) +netdev_err (adapter->netdev, +"h_reg_logical_lan_queue H_FUNCTION for queue %d (firmware MQ unsupported)\n" , + queue_index);[Severity: High]
Can the adapter recover if firmware stops honouring the MQ register after probe?multi_queueandnum_rx_queuesare latched once inibmveth_probe()from theH_ILLAN_ATTRIBUTESbit and are never re-read or reduced, so aH_FUNCTIONhere failsibmveth_open()and every subsequent open the same way.The reset work item only closes and reopens, it never re-reads the attributes:
ibmveth_reset() { rtnl_lock(); dev_close(adapter->netdev); dev_open(adapter->netdev, NULL); rtnl_unlock(); }ethtool -L ethX rx 1 while down does not restore RX either:
multi_queuestays 1, sorx_buffers_per_hcallremainsIBMVETH_MAX_RX_QUEUEandibmveth_add_logical_lan_buffers()keeps callingh_add_logical_lan_buffers_queue(), whoseH_FUNCTIONpath has no fallback whenmulti_queueis set.The driver already assumes PHYP capabilities can disappear across LPM:
Quoted 8 lines
/*
* Live Partition Migration may drop multi-
* buffer support. Fall back to single-buffer
* on the next replenish; ...
*/Would clearing
multi_queueand falling back to one queue onH_FUNCTIONbe preferable to leaving the interface permanently unable to open?
Yes. If subordinate queue registration starts returning H_FUNCTION after probe, the adapter can get stuck in a permanent open-fail state.
I’m planning to set a fallback latch on subordinate-register H_FUNCTION, fail the current open cleanly, and let the next open apply single-queue fallback after close has torn down the partial MQ state.
That should be framed as defensive recovery for an unexpected H_FUNCTION event, not as a supported live MQ-to-SQ mode switch on the running datapath.
Quoted 17 lines
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter, · line 1485
Quoted 9 lines
drivers/net/ethernet/ibm/ibmveth.c +9
-0+static int +ibmveth_register_single_rx_queue (struct ibmveth_adapter *adapter, +int queue_idx,u64 mac_address) +{ +struct net_device *netdev = adapter->netdev; +union ibmveth_buf_desc rxq_desc; +long lpar_rc; + + (void )mac_address;[Severity: Low]
This isn't a bug, but is themac_addressparameter worth keeping?H_REG_LOGICAL_LAN_QUEUEhas no MAC operand andibmveth_register_logical_lan_queue()has no MAC parameter, so callers computeether_addr_to_u64(netdev->dev_addr)only to feed a parameter that is voided here. The (void)x; suppression cast is also not needed under the kernel's warning set.
Quoted 1 line
[ ... ]
Yes. The unused mac_address parameter should just be dropped.
Quoted 77 lines
Quoted 26 lines
drivers/net/ethernet/ibm/ibmveth.c +20
-1@@ -1356,9 +1579,31 @@ ibmveth_register_rx_queues(struct ibmveth_adapter *adapter, u64 mac_address) +for (i =1 ; i < adapter->num_rx_queues; i++) { + rc =ibmveth_register_single_rx_queue (adapter, i, mac_address); +if (rc) + goto err_unregister; + } +netdev_dbg (netdev, -"registered 1 RX queue with hypervisor (single-queue mode)\n" ); +"registered %d RX queues with hypervisor (multi-queue mode)\n" , + adapter->num_rx_queues); +return 0 ; + +err_unregister: +ibmveth_dispose_subordinate_irq_mappings (adapter); +ibmveth_free_all_queues (adapter); +return rc; }@@ -1396,12 +1641,29 @@ static int ibmveth_open(struct net_device *netdev) +/* +* MQ: post buffers before setup_rx_interrupts() unmasks PHYP +* (avoids drops if traffic arrives during open; PHYP allows +* either order). Single-queue keeps the classic kick: setup +* (no unmask) then schedule_rx_queue() so the first poll +* replenishes and enables. drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_open(struct net_device *netdev) · line 1656
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +9
-0+*/ +if (adapter->multi_queue && adapter->num_rx_queues >1 ) { +for (i =0 ; i < adapter->num_rx_queues; i++) { +netdev_dbg (netdev, +"initial replenish cycle for queue %d\n" , i); +ibmveth_replenish_task (adapter, i); + } + } + rc =ibmveth_setup_rx_interrupts (adapter);[Severity: Medium]
Is PHYP delivery for subordinate queues guaranteed to be masked in this window? Queue 0 is explicitly masked before registration:ibmveth_register_rx_queues() { adapter->queue_irq[0] = netdev->irq; rc = ibmveth_disable_irq(adapter, 0); ... }but queues 1..N get no equivalent H_VIOCTL/H_DISABLE_VIO_INTERRUPT after
H_REG_LOGICAL_LAN_QUEUEplusirq_create_mapping(). Buffers are now posted for every queue here, before any handler exists, andibmveth_setup_rx_interrupts()was also reordered sorequest_irq()for all queues completes before anynapi_enable():for (i = 0; i < num; i++) { ... rc = request_irq(adapter->queue_irq[i], ibmveth_interrupt, ...); } for (i = 0; i < num; i++) napi_enable(&adapter->napi[i]);If an interrupt arrives in that window,
ibmveth_schedule_rx_queue()finds NAPI still disabled:if (napi_schedule_prep(napi)) { lpar_rc = ibmveth_disable_irq(adapter, qindex); WARN_ON(lpar_rc != H_SUCCESS); __napi_schedule(napi); }so the handler returns
IRQ_HANDLEDwithout masking the hypervisor source and without scheduling a poll. That is the same storm window described inibmveth_cleanup_rx_interrupts()("Mask PHYP beforenapi_disablesoibmveth_interruptcannot returnIRQ_HANDLEDwithout masking"). Theerr_free_irqspath has the same gap: it frees handlers and disposes subordinate mappings without masking already-registered subordinate queues.
Quoted 6 lines
The later scale-up path uses register ->
request_irq-> publish -> replenish ->napi_enable->enable_irqand documents why. Would enabling NAPI beforerequest_irq(as the baseline did), or explicitly masking each subordinate queue right after registration, close this window?[ ... ]
Yes. The subordinate queues need the same coherent IRQ/NAPI sequencing as queue 0.
I’m planning to keep the open/setup path on the same request_irq -> napi_enable -> PHYP enable rule here, so the first unmask does not race a disabled poll. I do not think this needs a different open-order policy from the single-queue path.
Quoted 37 lines
Quoted 8 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-4@@ -2373,18 +2632,25 @@ static unsigned long ibmveth_get_desired_dma(struct vio_dev *vdev)/* add size of mapped tx buffers */ ret +=IOMMU_PAGE_ALIGN (IBMVETH_MAX_TX_BUF_SIZE , tbl); -for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) { -/* add the size of the active receive buffers */ -if (adapter->rx_buff_pool[0 ][i].active) - ret +=drivers/net/ethernet/ibm/ibmveth.c static unsigned long ibmveth_get_desired_dma(struct vio_dev *vdev) · line 2636
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-8- adapter->rx_buff_pool[0 ][i].size * -IOMMU_PAGE_ALIGN (adapter->rx_buff_pool[0 ][i]. - buff_size, tbl); - rxqentries += adapter->rx_buff_pool[0 ][i].size; - } -/* add the size of the receive queue entries */ - ret +=IOMMU_PAGE_ALIGN ( - rxqentries *sizeof (struct ibmveth_rx_q_entry ), tbl); +for (q =0 ; q < adapter->num_rx_queues; q++) { +int rxqentries =1 ;[Severity: Medium]
Does the buffer-list term still need scaling here? The unchanged line above this loop counts it once:ret = IBMVETH_BUFF_LIST_SIZE + IBMVETH_FILT_LIST_SIZE;while multi-queue allocates and dma_map_single()s a distinct 4096-byte buffer-list page per queue, which
ibmveth_cleanup_rx_resources()unmaps per index. That leaves the desired value short by (num_rx_queues- 1) buffer-list pages.Also, is anything updating the CMO entitlement for the new default queue count? The VIO core samples it before the driver's probe runs:
arch/powerpc/platforms/pseries/vio.c:vio_cmo_bus_probe() { viodev->cmo.desired = IOMMU_PAGE_ALIGN(viodrv->get_desired_dma(viodev), tbl); }
Quoted 6 lines
with
dev_get_drvdata()still NULL, so the single-queue sizedIBMVETH_IO_ENTITLEMENT_DEFAULTis used for an adapter that will open with up to 8 RX queues, and neitheribmveth_probe()noribmveth_open()callsvio_cmo_set_dev_desired(). On a CMO partition, can this make dma_map_single()/dma_alloc_coherent() fail during open on adapters that worked in single-queue mode?
Yes. The desired-DMA calculation needs to scale the buffer-list pages by the live RX queue count, and probe needs to refresh the CMO entitlement after MQ capability detection.
I’m planning to keep the size derived from the pool0 geometry and call vio_cmo_set_dev_desired() after probe has finalized the queue count.
Quoted 12 lines
drivers/net/ethernet/ibm/ibmveth.c static unsigned long ibmveth_get_desired_dma(struct vio_dev *vdev) · line 2641
Quoted 5 lines
drivers/net/ethernet/ibm/ibmveth.c +5
-0+ +for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) { +/* add the size of the active receive buffers */ +struct ibmveth_buff_pool *bpool = + &adapter->rx_buff_pool[q][i];[Severity: Medium]
Is rx_buff_pool[q][i] populated for q >= 1 when this runs? Pool metadata for queues 1..N is only filled in byibmveth_alloc_buffer_pools(), which runs fromndo_open;ibmveth_probe()initializes only rx_buff_pool[0][i], and the private area is zeroed, so before the first open these entries have size = active =buff_size= 0.
Quoted 7 lines
ibmveth_change_mtu()pushes the result straight into firmware accounting:vio_cmo_set_dev_desired(viodev, ibmveth_get_desired_dma(viodev));Can an MTU change while the interface is down therefore lower the entitlement to roughly the single-queue value, right before an open that allocates buffers for up to 8 queues?
Yes. change_mtu() also needs to refresh the desired entitlement using the MQ-scaled geometry even while the interface is down.
Quoted 43 lines
Quoted 13 lines
drivers/net/ethernet/ibm/ibmveth.c +9
-0@@ -2449,9 +2715,18 @@ static const struct net_device_ops ibmveth_netdev_ops = {#endif }; +static void ibmveth_put_pool_kobjs (struct ibmveth_adapter *adapter, +int pools_ready) +{ +int i; + +for (i =0 ; i < pools_ready; i++) +kobject_put (&adapter->rx_buff_pool[0 ][i].kobj); +} +[ ... ]
Quoted 8 lines
drivers/net/ethernet/ibm/ibmveth.c +1
-3@@ -2531,16 +2810,27 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id) netdev->features |=NETIF_F_FRAGLIST ; } -/* Initialize queue count - always 1 for now */ - adapter->multi_queue =0 ; - adapter->num_rx_queues =IBMVETH_DEFAULT_RX_QUEUES ; + if (ret == H_SUCCESS &&drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id) · line 2823
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+ (ret_attr & IBMVETH_ILLAN_RX_MULTI_QUEUE_SUPPORT)) { + adapter->multi_queue = 1; + adapter->num_rx_queues = min(num_online_cpus(), + IBMVETH_DEFAULT_QUEUES); + netdev_dbg(netdev, "RX multi queue mode enabled: %d queues\n", + adapter->num_rx_queues); + } else { + adapter->multi_queue =0 ; + adapter->num_rx_queues =IBMVETH_DEFAULT_RX_QUEUES ; + }[Severity: Medium]
Now thatnum_rx_queuescan be greater than one here, doesibmveth_resume()need updating too? It still kicks queue 0 only:ibmveth_resume() { ... ibmveth_schedule_rx_queue(adapter, 0); return 0; }
Quoted 5 lines
The kick exists because the poll routine is what replenishes buffers and re-arms the PHYP interrupt via
napi_complete_done()->ibmveth_enable_irq(). After a suspend/resume or partition migration, queues 1..N-1 would then have no buffers posted and no interrupt re-armed until an ifdown/ifup.
Yes. resume() should kick every live RX queue, not just queue 0, so subordinate queues get replenished and re-armed too.
Quoted 66 lines
Quoted 18 lines
drivers/net/ethernet/ibm/ibmveth.c +5
-5@@ -2558,15 +2848,24 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) {struct kobject *kobj = &adapter->rx_buff_pool[0 ][i].kobj; -int error;ibmveth_init_buffer_pool (&adapter->rx_buff_pool[0 ][i], i, pool_count[i], pool_size[i], pool_active[i]); - error =kobject_init_and_add (kobj, &ktype_veth_pool, - &dev->dev.kobj,"pool%d" , i); -if (!error) -kobject_uevent (kobj,KOBJ_ADD ); + rc =kobject_init_and_add (kobj, &ktype_veth_pool, + &dev->dev.kobj,"pool%d" , i); +if (rc) { +dev_err (&dev->dev, +"failed to create pool%d kobject: %d\n" , i, rc);drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id) · line 2869
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +9
-0+/* init_and_add takes a ref even on failure */ +kobject_put (kobj); +ibmveth_put_pool_kobjs (adapter, pools_ready); +free_netdev (netdev); +return rc; + } + + pools_ready++; +kobject_uevent (kobj,KOBJ_ADD ); }[Severity: Medium]
Is it safe tofree_netdev()immediately afteribmveth_put_pool_kobjs()? The kobjects are embedded innetdev_priv(netdev), andktype_veth_poolhas no release callback:static struct kobj_type ktype_veth_pool = { .release = NULL, ... };With
CONFIG_DEBUG_KOBJECT_RELEASE=ythe final put defers cleanup using a work item stored inside the kobject itself:lib/kobject.c:kobject_release() {
#ifdef CONFIG_DEBUG_KOBJECT_RELEASE unsigned long delay = HZ + HZ * get_random_u32_below(4); ... INIT_DELAYED_WORK(&kobj->release, kobject_delayed_cleanup); schedule_delayed_work(&kobj->release, delay); #else kobject_cleanup(kobj);#endif
}
free_netdev()then releases that memory with no window, sokobject_delayed_cleanup()would readkobj->state_in_sysfs,kobj->parentandkobj->namefrom freed storage. The same shape applies to the two other newput_pool_kobjs()call sites below.Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +1
-0@@ -2574,6 +2873,7 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)if (rc) {netdev_dbg (netdev,"failed to set number of tx queues rc=%d\n" , rc); +ibmveth_put_pool_kobjs (adapter, pools_ready);free_netdev (netdev);return rc; }@@ -2590,6 +2890,7 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
Quoted 7 lines
Quoted 6 lines
drivers/net/ethernet/ibm/ibmveth.c +1
-0if (rc) {netdev_dbg (netdev,"failed to register netdev rc=%d\n" , rc); +ibmveth_put_pool_kobjs (adapter, pools_ready);free_netdev (netdev);return rc; }[ ... ]
Fair point. That DEBUG_KOBJECT_RELEASE hazard is real, but I’m treating it as a follow-up rather than blocking this MQ series.
An empty release callback is not enough by itself; this needs a safer lifetime arrangement, for example waiting out delayed release before free_netdev(), or moving the pool kobjects out of the netdev-private storage.
Thanks,
Mingming
PATCH v4 10/14 ibmveth: Add per-queue RX and TX statistics collection
In reply to Mingming CaoCount per-queue RX and TX statistics and expose them via ethtool -S and ndo_get_stats64().
RX hot path (poll / IRQ / replenish) updates rx_qstats[] for packets, bytes, polls, interrupts, large_packets, invalid_buffers, and no_buffer_drops. TX hot path updates tx_qstats[] for packets, bytes, large_packets, dropped_packets, send_failures, and checksum_offload.
Allocate qstat arrays for the adapter lifetime (probe/remove) and free them from ibmveth_remove() / ibmveth_probe_cleanup(). Adapter-level ethtool strings sum per-queue counters on read. ibmveth_update_rx_no_buffer() fills rx_qstats[i].no_buffer_drops and sums into adapter->rx_no_buffer.
Also fix get_channels() reporting: max_rx is IBMVETH_MAX_RX_QUEUES only when MQ firmware is enabled; rx_count tracks adapter->num_rx_queues.
---
Changes in v4:
- Merge v3's separate RX and TX stats commits into one patch.
- Introduce rx_queue_stats / tx_qstats / NUM macros here (first use).
- Allocate/free qstats at probe/remove instead of open/close.
- Report adapter-level ethtool strings by summing per-queue counters on read; drop aggregate_* helpers.
- Sum global rx_no_buffer across MQ queues into this statistics patch.
- Cacheline-align per-queue stats; derive field counts with offsetof so alignment padding is not counted as a statistic.
- probe_cleanup() cancels reset work, puts pool kobjects via helper from the prior patch, and frees qstats on probe failure paths.
- Keep plain u64 qstats like existing ibmveth / ibmvnic (PPC_PSERIES).
drivers/net/ethernet/ibm/ibmveth.c | 320 +++++++++++++++++++++++++++-- drivers/net/ethernet/ibm/ibmveth.h | 34 +++ 2 files changed, 338 insertions(+), 16 deletions(-)
drivers/net/ethernet/ibm/ibmveth.c +304 -16
@@ -228,6 +228,60 @@ ibmveth_free_filter_list(struct ibmveth_adapter *adapter) } } +/** +* ibmveth_alloc_rx_qstats - Allocate per-queue RX statistics +* @adapter: ibmveth adapter structure +* +* Return: 0 on success, -ENOMEM on failure +*/ +static int ibmveth_alloc_rx_qstats (struct ibmveth_adapter *adapter) +{ + adapter->rx_qstats =kcalloc (IBMVETH_MAX_RX_QUEUES , +sizeof (*adapter->rx_qstats), +GFP_KERNEL ); +if (!adapter->rx_qstats) +return -ENOMEM ; + +return 0 ; +} + +/** +* ibmveth_free_rx_qstats - Free per-queue RX statistics +* @adapter: ibmveth adapter structure +*/ +static void ibmveth_free_rx_qstats (struct ibmveth_adapter *adapter) +{ +kfree (adapter->rx_qstats); + adapter->rx_qstats =NULL ; +} + +/** +* ibmveth_alloc_tx_qstats - Allocate per-queue TX statistics +* @adapter: ibmveth adapter structure +* +* Return: 0 on success, -ENOMEM on failure +*/ +static int ibmveth_alloc_tx_qstats (struct ibmveth_adapter *adapter) +{ + adapter->tx_qstats =kcalloc (IBMVETH_MAX_QUEUES , +sizeof (*adapter->tx_qstats), +GFP_KERNEL ); +if (!adapter->tx_qstats) +return -ENOMEM ; + +return 0 ; +} + +/** +* ibmveth_free_tx_qstats - Free per-queue TX statistics +* @adapter: ibmveth adapter structure +*/ +static void ibmveth_free_tx_qstats (struct ibmveth_adapter *adapter) +{ +kfree (adapter->tx_qstats); + adapter->tx_qstats =NULL ; +} +/** * ibmveth_alloc_rx_queues - Allocate per-queue RX resources * @adapter: ibmveth adapter structure
@@ -921,6 +975,8 @@ static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter)__be64 *p = adapter->buffer_list_addr[i] +4096 -8 ;u64 drops =be64_to_cpup (p); +if (adapter->rx_qstats) + adapter->rx_qstats[i].no_buffer_drops = drops; adapter->rx_no_buffer += drops; } }
@@ -1972,22 +2028,131 @@ static int ibmveth_set_features(struct net_device *dev,* globals on the hot path (ibmvnic-style); with qstats allocated for the * adapter lifetime, these sums remain meaningful across ifdown/up. */ +static u64 ibmveth_sum_rx_invalid_buffers (struct ibmveth_adapter *adapter) +{ +u64 total =0 ; +int i; + +if (!adapter->rx_qstats) +return adapter->rx_invalid_buffer; + +for (i =0 ; i < adapter->num_rx_queues; i++) + total += adapter->rx_qstats[i].invalid_buffers; + +return total; +} + +static u64 ibmveth_sum_rx_large_packets (struct ibmveth_adapter *adapter) +{ +u64 total =0 ; +int i; + +if (!adapter->rx_qstats) +return adapter->rx_large_packets; + +for (i =0 ; i < adapter->num_rx_queues; i++) + total += adapter->rx_qstats[i].large_packets; + +return total; +} + +static u64 ibmveth_sum_tx_large_packets (struct ibmveth_adapter *adapter) +{ +struct net_device *netdev = adapter->netdev; +u64 total =0 ; +int i; + +if (!adapter->tx_qstats) +return adapter->tx_large_packets; + +for (i =0 ; i < netdev->real_num_tx_queues; i++) + total += adapter->tx_qstats[i].large_packets; + +return total; +} + +static u64 ibmveth_sum_tx_send_failed (struct ibmveth_adapter *adapter) +{ +struct net_device *netdev = adapter->netdev; +u64 total =0 ; +int i; + +if (!adapter->tx_qstats) +return adapter->tx_send_failed; + +for (i =0 ; i < netdev->real_num_tx_queues; i++) + total += adapter->tx_qstats[i].send_failures; + +return total; +} + +static u64 ibmveth_ethtool_adapter_stat (struct ibmveth_adapter *adapter, +int index) +{ +unsigned long offset = ibmveth_stats[index].offset; + +if (offset ==IBMVETH_STAT_OFF (rx_invalid_buffer)) +return ibmveth_sum_rx_invalid_buffers (adapter); +if (offset ==IBMVETH_STAT_OFF (rx_large_packets)) +return ibmveth_sum_rx_large_packets (adapter); +if (offset ==IBMVETH_STAT_OFF (tx_large_packets)) +return ibmveth_sum_tx_large_packets (adapter); +if (offset ==IBMVETH_STAT_OFF (tx_send_failed)) +return ibmveth_sum_tx_send_failed (adapter); + +return IBMVETH_GET_STAT (adapter, offset); +} +static void ibmveth_get_strings (struct net_device *dev,u32 stringset,u8 *data) { +struct ibmveth_adapter *adapter =netdev_priv (dev); +u8 *p = data;int i;if (stringset !=ETH_SS_STATS )return ; -for (i =0 ; i <ARRAY_SIZE (ibmveth_stats); i++, data +=ETH_GSTRING_LEN ) -memcpy (data, ibmveth_stats[i].name,ETH_GSTRING_LEN ); +for (i =0 ; i <ARRAY_SIZE (ibmveth_stats); i++) { +memcpy (p, ibmveth_stats[i].name,ETH_GSTRING_LEN ); + p +=ETH_GSTRING_LEN ; + } + +for (i =0 ; i < adapter->num_rx_queues; i++) { +ethtool_sprintf (&p,"rx%d_packets" , i); +ethtool_sprintf (&p,"rx%d_bytes" , i); +ethtool_sprintf (&p,"rx%d_interrupts" , i); +ethtool_sprintf (&p,"rx%d_polls" , i); +ethtool_sprintf (&p,"rx%d_large_packets" , i); +ethtool_sprintf (&p,"rx%d_invalid_buffers" , i); +ethtool_sprintf (&p,"rx%d_no_buffer_drops" , i); + } + +for (i =0 ; i < dev->real_num_tx_queues; i++) { +ethtool_sprintf (&p,"tx%d_packets" , i); +ethtool_sprintf (&p,"tx%d_bytes" , i); +ethtool_sprintf (&p,"tx%d_large_packets" , i); +ethtool_sprintf (&p,"tx%d_dropped_packets" , i); +ethtool_sprintf (&p,"tx%d_send_failures" , i); +ethtool_sprintf (&p,"tx%d_checksum_offload" , i); + } + +for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) { +ethtool_sprintf (&p,"pool%d_size" , i); +ethtool_sprintf (&p,"pool%d_active" , i); +ethtool_sprintf (&p,"pool%d_available" , i); + } }static int ibmveth_get_sset_count (struct net_device *dev,int sset) { +struct ibmveth_adapter *adapter =netdev_priv (dev); +switch (sset) {case ETH_SS_STATS : -return ARRAY_SIZE (ibmveth_stats); +return ARRAY_SIZE (ibmveth_stats) + + adapter->num_rx_queues *IBMVETH_NUM_RX_QSTATS + + dev->real_num_tx_queues *IBMVETH_NUM_TX_QSTATS + +IBMVETH_NUM_BUFF_POOLS *3 ;default :return -EOPNOTSUPP ; }
@@ -1996,21 +2161,59 @@ static int ibmveth_get_sset_count(struct net_device *dev, int sset)static void ibmveth_get_ethtool_stats (struct net_device *dev,struct ethtool_stats *stats,u64 *data) { -int i;struct ibmveth_adapter *adapter =netdev_priv (dev); +int i, j;for (i =0 ; i <ARRAY_SIZE (ibmveth_stats); i++) - data[i] =IBMVETH_GET_STAT (adapter, ibmveth_stats[i].offset); + data[i] =ibmveth_ethtool_adapter_stat (adapter, i); + +for (j =0 ; j < adapter->num_rx_queues; j++) { +if (adapter->rx_qstats) { + data[i++] = adapter->rx_qstats[j].packets; + data[i++] = adapter->rx_qstats[j].bytes; + data[i++] = adapter->rx_qstats[j].interrupts; + data[i++] = adapter->rx_qstats[j].polls; + data[i++] = adapter->rx_qstats[j].large_packets; + data[i++] = adapter->rx_qstats[j].invalid_buffers; + data[i++] = adapter->rx_qstats[j].no_buffer_drops; + }else { + i +=IBMVETH_NUM_RX_QSTATS ; + } + } + +for (j =0 ; j < dev->real_num_tx_queues; j++) { +if (adapter->tx_qstats) { + data[i++] = adapter->tx_qstats[j].packets; + data[i++] = adapter->tx_qstats[j].bytes; + data[i++] = adapter->tx_qstats[j].large_packets; + data[i++] = adapter->tx_qstats[j].dropped_packets; + data[i++] = adapter->tx_qstats[j].send_failures; + data[i++] = adapter->tx_qstats[j].checksum_offload; + }else { + i +=IBMVETH_NUM_TX_QSTATS ; + } + } + +for (j =0 ; j <IBMVETH_NUM_BUFF_POOLS ; j++) { + data[i++] = adapter->rx_buff_pool[0 ][j].size; + data[i++] = adapter->rx_buff_pool[0 ][j].active; + data[i++] =atomic_read (&adapter->rx_buff_pool[0 ][j].available); + } }static void ibmveth_get_channels (struct net_device *netdev,struct ethtool_channels *channels) { +struct ibmveth_adapter *adapter =netdev_priv (netdev); + channels->max_tx =ibmveth_real_max_tx_queues (); channels->tx_count = netdev->real_num_tx_queues; - channels->max_rx = netdev->real_num_rx_queues; - channels->rx_count = netdev->real_num_rx_queues; +if (adapter->multi_queue) + channels->max_rx =IBMVETH_MAX_RX_QUEUES ; +else + channels->max_rx =1 ; + channels->rx_count = adapter->num_rx_queues; } static int ibmveth_set_channels(struct net_device *netdev,
@@ -2124,6 +2327,8 @@ static int ibmveth_is_packet_unsupported(struct sk_buff *skb,if (ether_addr_equal (ether_header->h_dest, netdev->dev_addr)) {netdev_dbg (netdev,"veth doesn't support loopback packets, dropping packet.\n" ); +if (adapter->tx_qstats) + adapter->tx_qstats[queue_num].dropped_packets++; ret = -EOPNOTSUPP ; }
@@ -2150,6 +2355,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb, skb_checksum_help(skb)) {netdev_err (netdev,"tx: failed to checksum packet\n" ); + adapter->tx_qstats[queue_num].dropped_packets++; goto out; }
@@ -2161,6 +2367,8 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb, desc_flags |= (IBMVETH_BUF_NO_CSUM |IBMVETH_BUF_CSUM_GOOD ); + adapter->tx_qstats[queue_num].checksum_offload++; +/* Need to zero out the checksum */ buf[0 ] =0 ; buf[1 ] =0 ;
@@ -2172,6 +2380,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,if (skb->ip_summed ==CHECKSUM_PARTIAL &&skb_is_gso (skb)) {if (adapter->fw_large_send_support) { mss = (unsigned long )skb_shinfo (skb)->gso_size; + adapter->tx_qstats[queue_num].large_packets++; }else if (!skb_is_gso_v6 (skb)) {/* Put -1 in the IP checksum to tell phyp it * is a largesend packet. Put the mss in
@@ -2180,6 +2389,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,ip_hdr (skb)->check =0xffff ;tcp_hdr (skb)->check =cpu_to_be16 (skb_shinfo (skb)->gso_size); + adapter->tx_qstats[queue_num].large_packets++; } }
@@ -2187,6 +2397,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,if (unlikely (skb->len > adapter->tx_ltb_size)) {netdev_err (adapter->netdev,"tx: packet size (%u) exceeds ltb (%u)\n" , skb->len, adapter->tx_ltb_size); + adapter->tx_qstats[queue_num].dropped_packets++; goto out; }memcpy (adapter->tx_ltb_ptr[queue_num], skb->data,skb_headlen (skb));
@@ -2203,6 +2414,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,if (unlikely (total_bytes != skb->len)) {netdev_err (adapter->netdev,"tx: incorrect packet len copied into ltb (%u != %u)\n" , skb->len, total_bytes); + adapter->tx_qstats[queue_num].dropped_packets++; goto out; } desc.fields.flags_len = desc_flags | skb->len;
@@ -2211,7 +2423,11 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,dma_wmb ();if (ibmveth_send (adapter, desc.desc, mss)) { + adapter->tx_qstats[queue_num].send_failures++; + adapter->tx_qstats[queue_num].dropped_packets++; }else { + adapter->tx_qstats[queue_num].packets++; + adapter->tx_qstats[queue_num].bytes += skb->len; } out:
@@ -2350,6 +2566,9 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)if (WARN_ON (queue_index <0 || queue_index >= adapter->num_rx_queues))return 0 ; +if (adapter->rx_qstats) + adapter->rx_qstats[queue_index].polls++; + restart_poll:while (frames_processed < budget) {if (!ibmveth_rxq_pending_buffer (adapter, queue_index))
@@ -2358,7 +2577,11 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)smp_rmb ();if (!ibmveth_rxq_buffer_valid (adapter, queue_index)) {wmb ();/* suggested by larson1 */ - adapter->rx_invalid_buffer++; +if (adapter->rx_qstats) + adapter->rx_qstats[queue_index] + .invalid_buffers++; +else + adapter->rx_invalid_buffer++;netdev_dbg (netdev,"recycling invalid buffer\n" ); rc =ibmveth_rxq_harvest_buffer (adapter, queue_index, true);
@@ -2432,7 +2655,11 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)if ((length > netdev->mtu +ETH_HLEN ) || lrg_pkt || iph_check ==0xffff ) {ibmveth_rx_mss_helper (skb, mss, lrg_pkt); - adapter->rx_large_packets++; +if (adapter->rx_qstats) + adapter->rx_qstats[queue_index] + .large_packets++; +else + adapter->rx_large_packets++; }if (csum_good) {
@@ -2442,6 +2669,11 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)napi_gro_receive (napi, skb);/* send it up */ +if (adapter->rx_qstats) { + adapter->rx_qstats[queue_index].packets++; + adapter->rx_qstats[queue_index].bytes += length; + } + frames_processed++; } }
@@ -2488,6 +2720,9 @@ static irqreturn_t ibmveth_interrupt(int irq, void *dev_instance)if (WARN_ON (qindex <0 || qindex >= adapter->num_rx_queues))return IRQ_NONE ; +if (adapter->rx_qstats) + adapter->rx_qstats[qindex].interrupts++; +ibmveth_schedule_rx_queue (adapter, qindex);return IRQ_HANDLED ; }
@@ -2698,6 +2933,40 @@ static netdev_features_t ibmveth_features_check(struct sk_buff *skb,return vlan_features_check (skb, features); } +/** +* ibmveth_get_stats64 - Return aggregated per-queue statistics +* @dev: network device +* @stats: rtnl link statistics storage +* +* Sums per-queue rx_qstats and tx_qstats into the rtnl counters. +* Callers use ndo_get_stats64(); avoid updating netdev->stats on the +* xmit/poll paths to keep per-queue counters off the hot cache line. +*/ +static void ibmveth_get_stats64 (struct net_device *dev, +struct rtnl_link_stats64 *stats) +{ +struct ibmveth_adapter *adapter =netdev_priv (dev); +int i; + +if (adapter->rx_qstats) { +for (i =0 ; i < adapter->num_rx_queues; i++) { + stats->rx_packets += adapter->rx_qstats[i].packets; + stats->rx_bytes += adapter->rx_qstats[i].bytes; + } + } + +if (adapter->tx_qstats) { +for (i =0 ; i < dev->real_num_tx_queues; i++) { + stats->tx_packets += adapter->tx_qstats[i].packets; + stats->tx_bytes += adapter->tx_qstats[i].bytes; + stats->tx_dropped += + adapter->tx_qstats[i].dropped_packets; + } + } + + stats->tx_errors = dev->stats.tx_errors; +} +static const struct net_device_ops ibmveth_netdev_ops = { .ndo_open = ibmveth_open, .ndo_stop = ibmveth_close,
@@ -2710,13 +2979,14 @@ static const struct net_device_ops ibmveth_netdev_ops = { .ndo_validate_addr = eth_validate_addr, .ndo_set_mac_address = ibmveth_set_mac_addr, .ndo_features_check = ibmveth_features_check, + .ndo_get_stats64 = ibmveth_get_stats64, #ifdef CONFIG_NET_POLL_CONTROLLER .ndo_poll_controller = ibmveth_poll_controller,#endif };static void ibmveth_put_pool_kobjs (struct ibmveth_adapter *adapter, -int pools_ready) +int pools_ready) {int i;
@@ -2724,6 +2994,19 @@ static void ibmveth_put_pool_kobjs(struct ibmveth_adapter *adapter,kobject_put (&adapter->rx_buff_pool[0 ][i].kobj); } +static void ibmveth_probe_cleanup (struct ibmveth_adapter *adapter, +int pools_ready) +{ +struct net_device *netdev = adapter->netdev; + +cancel_work_sync (&adapter->work); +ibmveth_put_pool_kobjs (adapter, pools_ready); + +ibmveth_free_tx_qstats (adapter); +ibmveth_free_rx_qstats (adapter); +free_netdev (netdev); +} +static int ibmveth_probe (struct vio_dev *dev,const struct vio_device_id *id) {int rc, i, mac_len, pools_ready =0 ;
@@ -2779,6 +3062,11 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)netif_napi_add_weight (netdev, &adapter->napi[i], ibmveth_poll,16 ); +if (ibmveth_alloc_rx_qstats (adapter) || +ibmveth_alloc_tx_qstats (adapter)) { +ibmveth_probe_cleanup (adapter,0 ); +return -ENOMEM ; + } netdev->irq = dev->irq; netdev->netdev_ops = &ibmveth_netdev_ops;
@@ -2859,8 +3147,7 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id) "failed to create pool%d kobject: %d\n", i, rc);/* init_and_add takes a ref even on failure */ kobject_put (kobj); -ibmveth_put_pool_kobjs (adapter, pools_ready); -free_netdev (netdev); +ibmveth_probe_cleanup (adapter, pools_ready);return rc; }
@@ -2873,8 +3160,7 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)if (rc) {netdev_dbg (netdev,"failed to set number of tx queues rc=%d\n" , rc); -ibmveth_put_pool_kobjs (adapter, pools_ready); -free_netdev (netdev); +ibmveth_probe_cleanup (adapter, pools_ready);return rc; } adapter->tx_ltb_size =PAGE_ALIGN (IBMVETH_MAX_TX_BUF_SIZE );
@@ -2890,8 +3176,7 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)if (rc) {netdev_dbg (netdev,"failed to register netdev rc=%d\n" , rc); -ibmveth_put_pool_kobjs (adapter, pools_ready); -free_netdev (netdev); +ibmveth_probe_cleanup (adapter, pools_ready);return rc; }
@@ -2913,6 +3198,9 @@ static void ibmveth_remove(struct vio_dev *dev)unregister_netdev (netdev); +ibmveth_free_tx_qstats (adapter); +ibmveth_free_rx_qstats (adapter); +free_netdev (netdev);dev_set_drvdata (&dev->dev,NULL ); }
drivers/net/ethernet/ibm/ibmveth.h +34 -0
@@ -285,6 +285,38 @@ struct ibmveth_hcall_stats {u64 send_lan;/* H_SEND_LOGICAL_LAN */ }; +struct ibmveth_rx_queue_stats { +u64 packets; +u64 bytes; +u64 interrupts; +u64 polls; +u64 large_packets; +u64 invalid_buffers; +u64 no_buffer_drops; +}____cacheline_aligned_in_smp ; + +struct ibmveth_tx_queue_stats { +u64 packets; +u64 bytes; +u64 large_packets; +u64 dropped_packets; +u64 send_failures; +u64 checksum_offload; +}____cacheline_aligned_in_smp ; + +/* +* ethtool string count: use offsetof of the last counter so alignment +* padding from ____cacheline_aligned_in_smp is not counted. When adding +* a new counter at the end, point these at the new last field (same idea +* as sizeof(struct)/sizeof(u64) before alignment was added). +*/ +#define IBMVETH_NUM_RX_QSTATS \ + (offsetof(struct ibmveth_rx_queue_stats, no_buffer_drops) / \ + sizeof(u64) + 1) +#define IBMVETH_NUM_TX_QSTATS \ + (offsetof(struct ibmveth_tx_queue_stats, checksum_offload) / \ + sizeof(u64) + 1) +struct ibmveth_buff_pool {u32 size;u32 index;
@@ -352,6 +384,8 @@ struct ibmveth_adapter {/* Multi-queue statistics */ struct ibmveth_hcall_stats hcall_stats; +struct ibmveth_rx_queue_stats *rx_qstats; +struct ibmveth_tx_queue_stats *tx_qstats;/* Ethtool settings */ u8 duplex;
Signature · 1 line
2.50.1 (Apple Git-155)
Jakub Kicinski
maintainerRe: [PATCH net-next v4 10/14] ibmveth: Add per-queue RX and TX statistics collection
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
ibmveth: Add per-queue RX and TX statistics collection
This patch adds per-queue RX and TX counter arrays that live for the adapter lifetime, exports them through ethtool -S, and adds an ndo_get_stats64 implementation that sums them. It also changes what ibmveth_get_channels() reports for max_rx and rx_count.
A few questions below.
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0
[ ... ]
drivers/net/ethernet/ibm/ibmveth.c static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter) · line 982
Quoted 9 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-0@@ -921,6 +975,8 @@ static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter)__be64 *p = adapter->buffer_list_addr[i] +4096 -8 ;u64 drops =be64_to_cpup (p); +if (adapter->rx_qstats) + adapter->rx_qstats[i].no_buffer_drops = drops; adapter->rx_no_buffer += drops; } }
[Severity: Medium]
Can concurrent polls on different queues corrupt these counters?
ibmveth_update_rx_no_buffer() writes rx_qstats[i] for every queue, but its caller ibmveth_replenish_task(adapter, queue_index) holds only that one queue's lock:
spin_lock_irqsave(&rxq->replenish_lock, flags); ... ibmveth_update_rx_no_buffer(adapter); spin_unlock_irqrestore(&rxq->replenish_lock, flags);
So two NAPI polls on different queues run the same all-queue loop under disjoint locks (replenish_lock[0] vs replenish_lock[1]):
CPU A: reads queue 1 hypervisor value 100
CPU B: reads queue 1 hypervisor value 105, stores 105 CPU A: stores 100
Does that make rxN_no_buffer_drops move backwards as seen by ethtool? The adapter->rx_no_buffer = 0 / += drops sequence around it is also an unsynchronized read-modify-write visible to a concurrent ethtool reader.
This also seems to work against the ____cacheline_aligned_in_smp annotation added to struct ibmveth_rx_queue_stats, since every replenish cycle now dirties every queue's stats cache line from a foreign CPU.
Would passing queue_index into the helper and touching only rx_qstats[queue_index] and buffer_list_addr[queue_index] work, deriving the adapter-level rx_no_buffer by summing on read the way the patch already does for rx_large_packets and rx_invalid_buffer?
Related: at the end of the series the RX scale-down path in ibmveth_resize_rx_queues_incremental() lowers adapter->num_rx_queues and then frees a queue's buffer_list page while polls on surviving queues keep running. Can a poll that already loaded the old bound dereference buffer_list_addr[i] for a freed page here?
[Severity: Medium]
Should no_buffer_drops be accumulated rather than assigned?
The value PHYP writes into the last 8 bytes of the buffer-list page is an absolute count for the life of that page, and the page is re-obtained with get_zeroed_page(GFP_KERNEL) by ibmveth_alloc_rx_queues() on every ibmveth_open() and released by ibmveth_cleanup_rx_resources() on every ibmveth_close(), including the close/open pairs done by the reset work, ibmveth_set_csum_offload() and ibmveth_set_tso().
Since rx_qstats[i].no_buffer_drops is assigned with "= drops", does rxN_no_buffer_drops (and the recomputed adapter->rx_no_buffer) jump backwards after an ifdown/ifup, a driver reset, or an ethtool -K tso change?
That appears to contradict the comment this patch adds above the sum helpers:
* globals on the hot path (ibmvnic-style); with qstats allocated for the
* adapter lifetime, these sums remain meaningful across ifdown/up.
Quoted 17 lines
drivers/net/ethernet/ibm/ibmveth.c +13
-0@@ -1972,22 +2028,131 @@ static int ibmveth_set_features(struct net_device *dev,* globals on the hot path (ibmvnic-style); with qstats allocated for the * adapter lifetime, these sums remain meaningful across ifdown/up. */ +static u64 ibmveth_sum_rx_invalid_buffers (struct ibmveth_adapter *adapter) +{ +u64 total =0 ; +int i; + +if (!adapter->rx_qstats) +return adapter->rx_invalid_buffer; + +for (i =0 ; i < adapter->num_rx_queues; i++) + total += adapter->rx_qstats[i].invalid_buffers; + +return total; +}
[ ... ]
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-0+static u64 ibmveth_sum_tx_send_failed (struct ibmveth_adapter *adapter) +{ +struct net_device *netdev = adapter->netdev; +u64 total =0 ;
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_set_features(struct net_device *dev, · line 2087
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+int i; + +if (!adapter->tx_qstats) +return adapter->tx_send_failed; + +for (i =0 ; i < netdev->real_num_tx_queues; i++) + total += adapter->tx_qstats[i].send_failures; + +return total; +}
[Severity: Medium]
Do these sums go backwards when the queue count is reduced?
The qstats arrays persist for the adapter lifetime, but the sums are bounded by the currently configured queue count. ibmveth_set_channels() lowers netdev->real_num_tx_queues:
rc = netif_set_real_num_tx_queues(netdev, goal);
Nothing folds or clears the counters of the queues that disappear, so after "ethtool -L ethX tx N" with a smaller N, do the adapter-level ethtool strings tx_large_packets and tx_send_failed, plus the rtnl_link_stats64 tx_packets/tx_bytes/tx_dropped summed in ibmveth_get_stats64(), all drop by the traffic accounted to the removed queues, and then jump forward again when the count is raised?
Since this patch removes the hot-path netdev->stats accounting, ibmveth_get_stats64() is now the only source for the rtnl counters, so a decrease is directly user visible; tools computing deltas (ip -s link, ifstat) will read it as a counter wrap. The same applies on the RX side with adapter->num_rx_queues once the incremental RX resize commit later in this series changes it at runtime.
Quoted 41 lines
drivers/net/ethernet/ibm/ibmveth.c +32
-2+static u64 ibmveth_ethtool_adapter_stat (struct ibmveth_adapter *adapter, +int index) +{ +unsigned long offset = ibmveth_stats[index].offset; + +if (offset ==IBMVETH_STAT_OFF (rx_invalid_buffer)) +return ibmveth_sum_rx_invalid_buffers (adapter); +if (offset ==IBMVETH_STAT_OFF (rx_large_packets)) +return ibmveth_sum_rx_large_packets (adapter); +if (offset ==IBMVETH_STAT_OFF (tx_large_packets)) +return ibmveth_sum_tx_large_packets (adapter); +if (offset ==IBMVETH_STAT_OFF (tx_send_failed)) +return ibmveth_sum_tx_send_failed (adapter); + +return IBMVETH_GET_STAT (adapter, offset); +} +static void ibmveth_get_strings (struct net_device *dev,u32 stringset,u8 *data) { +struct ibmveth_adapter *adapter =netdev_priv (dev); +u8 *p = data;int i;if (stringset !=ETH_SS_STATS )return ; -for (i =0 ; i <ARRAY_SIZE (ibmveth_stats); i++, data +=ETH_GSTRING_LEN ) -memcpy (data, ibmveth_stats[i].name,ETH_GSTRING_LEN ); +for (i =0 ; i <ARRAY_SIZE (ibmveth_stats); i++) { +memcpy (p, ibmveth_stats[i].name,ETH_GSTRING_LEN ); + p +=ETH_GSTRING_LEN ; + } + +for (i =0 ; i < adapter->num_rx_queues; i++) { +ethtool_sprintf (&p,"rx%d_packets" , i); +ethtool_sprintf (&p,"rx%d_bytes" , i); +ethtool_sprintf (&p,"rx%d_interrupts" , i); +ethtool_sprintf (&p,"rx%d_polls" , i); +ethtool_sprintf (&p,"rx%d_large_packets" , i); +ethtool_sprintf (&p,"rx%d_invalid_buffers" , i); +ethtool_sprintf (&p,"rx%d_no_buffer_drops" , i);
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_set_features(struct net_device *dev, · line 2137
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+ } + +for (i =0 ; i < dev->real_num_tx_queues; i++) { +ethtool_sprintf (&p,"tx%d_packets" , i); +ethtool_sprintf (&p,"tx%d_bytes" , i); +ethtool_sprintf (&p,"tx%d_large_packets" , i); +ethtool_sprintf (&p,"tx%d_dropped_packets" , i); +ethtool_sprintf (&p,"tx%d_send_failures" , i); +ethtool_sprintf (&p,"tx%d_checksum_offload" , i); + }
[Severity: Medium]
Should the per-queue packet and byte counters use the standard per-queue statistics interface instead of driver-private ethtool strings?
rx%d_packets, rx%d_bytes, tx%d_packets and tx%d_bytes map one to one onto struct netdev_queue_stats_rx and struct netdev_queue_stats_tx, which are exported through netlink via struct netdev_stat_ops. No netdev_stat_ops is added here, so generic tooling (ynl) cannot consume these values, and Documentation/networking/statistics.rst asks that counters with a standard uAPI not be duplicated in ethtool -S.
Also, the new ibmveth_get_stats64() populates only rx/tx packets, bytes and tx_dropped:
stats->tx_errors = dev->stats.tx_errors;
After this patch, is there any remaining writer of dev->stats in this driver? The only match in the file is this read, so tx_errors appears to be permanently 0. The loss counters this patch does collect (no_buffer_drops from the hypervisor, invalid_buffers) are not mapped into rx_dropped / rx_missed_errors / rx_errors either, so ip -s link reports zero RX drops while ethtool -S shows a non-zero rx_no_buffer.
Quoted 22 lines
drivers/net/ethernet/ibm/ibmveth.c +12
-1+ +for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) { +ethtool_sprintf (&p,"pool%d_size" , i); +ethtool_sprintf (&p,"pool%d_active" , i); +ethtool_sprintf (&p,"pool%d_available" , i); + } }static int ibmveth_get_sset_count (struct net_device *dev,int sset) { +struct ibmveth_adapter *adapter =netdev_priv (dev); +switch (sset) {case ETH_SS_STATS : -return ARRAY_SIZE (ibmveth_stats); +return ARRAY_SIZE (ibmveth_stats) + + adapter->num_rx_queues *IBMVETH_NUM_RX_QSTATS + + dev->real_num_tx_queues *IBMVETH_NUM_TX_QSTATS + +IBMVETH_NUM_BUFF_POOLS *3 ;default :return -EOPNOTSUPP ; }
[ ... ]
Quoted 15 lines
drivers/net/ethernet/ibm/ibmveth.c +6
-2@@ -1996,21 +2161,59 @@ static int ibmveth_get_sset_count(struct net_device *dev, int sset)static void ibmveth_get_ethtool_stats (struct net_device *dev,struct ethtool_stats *stats,u64 *data) { -int i;struct ibmveth_adapter *adapter =netdev_priv (dev); +int i, j;for (i =0 ; i <ARRAY_SIZE (ibmveth_stats); i++) - data[i] =IBMVETH_GET_STAT (adapter, ibmveth_stats[i].offset); + data[i] =ibmveth_ethtool_adapter_stat (adapter, i); + +for (j =0 ; j < adapter->num_rx_queues; j++) { +if (adapter->rx_qstats) { + data[i++] = adapter->rx_qstats[j].packets;
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_get_sset_count(struct net_device *dev, int sset) · line 2182
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+ data[i++] = adapter->rx_qstats[j].bytes; + data[i++] = adapter->rx_qstats[j].interrupts; + data[i++] = adapter->rx_qstats[j].polls; + data[i++] = adapter->rx_qstats[j].large_packets; + data[i++] = adapter->rx_qstats[j].invalid_buffers; + data[i++] = adapter->rx_qstats[j].no_buffer_drops; + }else { + i +=IBMVETH_NUM_RX_QSTATS ; + } + }
[Severity: Low]
Can adapter->rx_qstats or adapter->tx_qstats ever be NULL here?
The allocations happen in ibmveth_probe() before register_netdev(), and a failure aborts probe with ibmveth_probe_cleanup() + return -ENOMEM. ibmveth_remove() frees them only after unregister_netdev(). So on a registered netdev the NULL case seems unreachable, which would make these else arms, the else arms in ibmveth_poll(), and the
"if (!adapter->*_qstats) return adapter-><legacy>;" early returns in the four sum helpers dead code.
The two styles also disagree: ibmveth_is_packet_unsupported() guards with "if (adapter->tx_qstats)" while ibmveth_start_xmit() dereferences adapter->tx_qstats[queue_num] unconditionally at six new sites on the same call chain. Which of the two is intended?
A side effect is that adapter->rx_invalid_buffer, rx_large_packets, tx_large_packets and tx_send_failed are never incremented again while still being listed in ibmveth_stats[].
Separately, ibmveth_ethtool_adapter_stat() dispatches by comparing ibmveth_stats[index].offset against IBMVETH_STAT_OFF() values. Would adding or reordering a field in struct ibmveth_adapter silently redirect a statistic to the wrong source here?
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+ +for (j =0 ; j < dev->real_num_tx_queues; j++) { +if (adapter->tx_qstats) { + data[i++] = adapter->tx_qstats[j].packets; + data[i++] = adapter->tx_qstats[j].bytes; + data[i++] = adapter->tx_qstats[j].large_packets; + data[i++] = adapter->tx_qstats[j].dropped_packets; + data[i++] = adapter->tx_qstats[j].send_failures; + data[i++] = adapter->tx_qstats[j].checksum_offload; + }else {
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_get_sset_count(struct net_device *dev, int sset) · line 2202
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +9
-0+ i +=IBMVETH_NUM_TX_QSTATS ; + } + } + +for (j =0 ; j <IBMVETH_NUM_BUFF_POOLS ; j++) { + data[i++] = adapter->rx_buff_pool[0 ][j].size; + data[i++] = adapter->rx_buff_pool[0 ][j].active; + data[i++] =atomic_read (&adapter->rx_buff_pool[0 ][j].available); + } }
[Severity: Medium]
Should the pool%d_* strings be queue-indexed?
The driver now keeps an independent pool set per RX queue (rx_buff_pool[queue][pool], populated for all q < adapter->num_rx_queues by ibmveth_alloc_buffer_pools() and updated per queue by ibmveth_replenish_task()), but these three entries read only rx_buff_pool[0][j] under queue-agnostic names, so the state of queues 1..N-1 is not visible. Is that intended for someone debugging drops on a non-zero queue?
Also, size and active are configuration values already exposed through the per-pool sysfs kobjects rather than statistics, and this new permanent ethtool string set is not mentioned in the commit message.
Quoted 6 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-0static void ibmveth_get_channels (struct net_device *netdev,struct ethtool_channels *channels) { +struct ibmveth_adapter *adapter =netdev_priv (netdev); + channels->max_tx =ibmveth_real_max_tx_queues ();
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_get_sset_count(struct net_device *dev, int sset) · line 2217
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +5
-2channels->tx_count = netdev->real_num_tx_queues; - channels->max_rx = netdev->real_num_rx_queues; - channels->rx_count = netdev->real_num_rx_queues; +if (adapter->multi_queue) + channels->max_rx =IBMVETH_MAX_RX_QUEUES ; +else + channels->max_rx =1 ; + channels->rx_count = adapter->num_rx_queues; }
[Severity: Low]
This isn't a bug, but would this ABI-visible get_channels() reporting change be easier to review as its own patch? It is independent of the statistics work, and if it is a fix it would want its own Fixes: tag.
There is also a mismatch it introduces: the new RX loops in ibmveth_get_strings() / ibmveth_get_sset_count() / ibmveth_get_stats64() use adapter->num_rx_queues while the TX loops use dev->real_num_tx_queues. netdev->real_num_rx_queues is only synced to adapter->num_rx_queues in ibmveth_open(), so on a never-opened interface ethtool -l reports a count the stack does not have yet.
[ ... ]
Quoted 7 lines
drivers/net/ethernet/ibm/ibmveth.c +1
-0@@ -2150,6 +2355,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb, skb_checksum_help(skb)) {netdev_err (netdev,"tx: failed to checksum packet\n" ); + adapter->tx_qstats[queue_num].dropped_packets++; goto out; }
[ ... ]
drivers/net/ethernet/ibm/ibmveth.c static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb, · line 2431
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-0@@ -2211,7 +2423,11 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,dma_wmb ();if (ibmveth_send (adapter, desc.desc, mss)) { + adapter->tx_qstats[queue_num].send_failures++; + adapter->tx_qstats[queue_num].dropped_packets++; }else { + adapter->tx_qstats[queue_num].packets++; + adapter->tx_qstats[queue_num].bytes += skb->len; }
[Severity: Medium]
These empty if/else bodies in the pre-image show that the previous commit in the series ("ibmveth: Enable multi-queue RX receive path") deleted the hot-path netdev->stats accounting, and the replacement only arrives here. ibmveth_poll() likewise increments no packet or byte counter at that commit, and no ndo_get_stats64 exists yet.
Does that leave the tree at the parent commit reporting zero rx_packets/rx_bytes/tx_packets/tx_bytes/tx_dropped through ip -s link, where the pre-series baseline reported them? Would squashing the removal into this patch, or deferring the removal until this replacement lands, keep every commit in the series bisectable for counter-related issues?
[ ... ]
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0@@ -2698,6 +2933,40 @@ static netdev_features_t ibmveth_features_check(struct sk_buff *skb,return vlan_features_check (skb, features); }
[ ... ]
drivers/net/ethernet/ibm/ibmveth.c static const struct net_device_ops ibmveth_netdev_ops = { · line 2991
Quoted 5 lines
static void ibmveth_put_pool_kobjs(struct
ibmveth_adapter*adapter,drivers/net/ethernet/ibm/ibmveth.c +1
-1- int pools_ready) + int pools_ready) {int i;
[Severity: Low]
This isn't a bug, but this whitespace-only reflow breaks the previously correct open-parenthesis alignment of the second parameter (checkpatch:
"Alignment should match open parenthesis") in a function this patch does not otherwise change. Could it be dropped?
Quoted 7 lines
drivers/net/ethernet/ibm/ibmveth.c +3
-0@@ -2724,6 +2994,19 @@ static void ibmveth_put_pool_kobjs(struct ibmveth_adapter *adapter,kobject_put (&adapter->rx_buff_pool[0 ][i].kobj); } +static void ibmveth_probe_cleanup (struct ibmveth_adapter *adapter, +int pools_ready) +{
drivers/net/ethernet/ibm/ibmveth.c static void ibmveth_put_pool_kobjs(struct ibmveth_adapter *adapter, · line 3009
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+struct net_device *netdev = adapter->netdev; + +cancel_work_sync (&adapter->work); +ibmveth_put_pool_kobjs (adapter, pools_ready); + +ibmveth_free_tx_qstats (adapter); +ibmveth_free_rx_qstats (adapter); +free_netdev (netdev); +} +
[Severity: High]
This isn't a problem introduced by this patch, since the old open-coded error paths had the same omission, but the new helper is now the single place four probe failure paths funnel through, so it looks like the natural spot to fix it.
ibmveth_probe() stores the netdev in the VIO drvdata right after alloc_etherdev_mqs():
dev_set_drvdata(&dev->dev, netdev);
ibmveth_probe_cleanup() ends with free_netdev(netdev) but never clears it, and ibmveth_remove() (the only caller of dev_set_drvdata(&dev->dev, NULL)) does not run when probe fails. So after any probe failure the vio_dev keeps a pointer to freed memory.
On FW_FEATURE_CMO systems the stale pointer is consumed on the next bind attempt, before the driver probe re-initializes it:
vio_bus_probe()
vio_cmo_bus_probe()
viodev->cmo.desired = IOMMU_PAGE_ALIGN(viodrv->get_desired_dma(viodev), tbl);
ibmveth_get_desired_dma()ibmveth_get_desired_dma() only checks for NULL:
netdev = dev_get_drvdata(&vdev->dev); if (netdev == NULL) return ...;
and then dereferences netdev->mtu and adapter->num_rx_queues, using the latter as the loop bound over adapter->rx_buff_pool[q][i]. Since that bound is read from freed memory, can this read past the end of the freed allocation as well? Would adding dev_set_drvdata(&dev->dev, NULL) to ibmveth_probe_cleanup() close this?
Quoted 15 lines
drivers/net/ethernet/ibm/ibmveth.c +5
-0static int ibmveth_probe (struct vio_dev *dev,const struct vio_device_id *id) {int rc, i, mac_len, pools_ready =0 ;@@ -2779,6 +3062,11 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)netif_napi_add_weight (netdev, &adapter->napi[i], ibmveth_poll,16 ); +if (ibmveth_alloc_rx_qstats (adapter) || +ibmveth_alloc_tx_qstats (adapter)) { +ibmveth_probe_cleanup (adapter,0 ); +return -ENOMEM ; + } netdev->irq = dev->irq; netdev->netdev_ops = &ibmveth_netdev_ops;
[ ... ]
drivers/net/ethernet/ibm/ibmveth.c static void ibmveth_remove(struct vio_dev *dev) · line 3206
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +3
-0@@ -2913,6 +3198,9 @@ static void ibmveth_remove(struct vio_dev *dev)unregister_netdev (netdev); +ibmveth_free_tx_qstats (adapter); +ibmveth_free_rx_qstats (adapter); +free_netdev (netdev);dev_set_drvdata (&dev->dev,NULL ); }
[Severity: High]
This is a pre-existing issue, since the cancel_work_sync() placement in ibmveth_remove() predates this patch, but the patch adds two new frees into exactly that window.
ibmveth_remove() calls cancel_work_sync(&adapter->work) at the top, before unregister_netdev(), so the device is still open when the cancel returns and softirq context can re-arm the work:
ibmveth_poll() /* on ibmveth_enable_irq() failure */
schedule_work(&adapter->work);
ibmveth_replenish_buffer_pool() /* on an invalid free_map index */
schedule_work(&adapter->work);
Timeline:
CPU A: cancel_work_sync() returns, queue empty
CPU B: ibmveth_poll() -> schedule_work(&adapter->work)
CPU A: unregister_netdev()
CPU A: ibmveth_free_tx_qstats() / ibmveth_free_rx_qstats() CPU A: free_netdev(netdev)
CPU B: ibmveth_reset() runs container_of(w, struct ibmveth_adapter, work)
Can the worker then dereference the freed adapter and netdev in rtnl_lock(); dev_close(adapter->netdev); dev_open(...)?
If it reaches dev_open() -> netif_tx_start_all_queues(), TX restarts with tx_qstats == NULL, and the unguarded adapter->tx_qstats[queue_num] dereferences added to ibmveth_start_xmit() by this patch would then be a NULL dereference. The new "if (adapter->rx_qstats)" hot-path guards do not help, since the adapter holding the pointer is itself inside the freed netdev.
Would moving the cancel (or disable_work_sync()) to after unregister_netdev() address this?
Re: [PATCH net-next v4 10/14] ibmveth: Add per-queue RX and TX statistics collection
Quoted 11 lines · Jakub Kicinski
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
ibmveth: Add per-queue RX and TX statistics collectionThis patch adds per-queue RX and TX counter arrays that live for the adapter lifetime, exports them through ethtool -S, and adds an
ndo_get_stats64implementation that sums them. It also changes whatibmveth_get_channels()reports formax_rxandrx_count.A few questions below.
Hi Jakub,
Thanks for the review.
Quoted 48 lines
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0[ ... ]
drivers/net/ethernet/ibm/ibmveth.c static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter) · line 982
Quoted 9 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-0@@ -921,6 +975,8 @@ static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter)__be64 *p = adapter->buffer_list_addr[i] +4096 -8 ;u64 drops =be64_to_cpup (p); +if (adapter->rx_qstats) + adapter->rx_qstats[i].no_buffer_drops = drops; adapter->rx_no_buffer += drops; } }[Severity: Medium]
Can concurrent polls on different queues corrupt these counters?
ibmveth_update_rx_no_buffer()writes rx_qstats[i] for every queue, but its calleribmveth_replenish_task(adapter, queue_index)holds only that one queue's lock:spin_lock_irqsave(&rxq->replenish_lock, flags); ... ibmveth_update_rx_no_buffer(adapter); spin_unlock_irqrestore(&rxq->replenish_lock, flags);So two NAPI polls on different queues run the same all-queue loop under disjoint locks (replenish_lock[0] vs replenish_lock[1]):
CPU A: reads queue 1 hypervisor value 100
CPU B: reads queue 1 hypervisor value 105, stores 105 CPU A: stores 100Does that make
rxN_no_buffer_dropsmove backwards as seen by ethtool? Theadapter->rx_no_buffer= 0 / += drops sequence around it is also an unsynchronized read-modify-write visible to a concurrent ethtool reader.This also seems to work against the
____cacheline_aligned_in_smpannotation added to structibmveth_rx_queue_stats, since every replenish cycle now dirties every queue's stats cache line from a foreign CPU.Would passing
queue_indexinto the helper and touching only rx_qstats[queue_index] and buffer_list_addr[queue_index] work, deriving the adapter-levelrx_no_bufferby summing on read the way the patch already does forrx_large_packetsandrx_invalid_buffer?
Quoted 5 lines
Related: at the end of the series the RX scale-down path in
ibmveth_resize_rx_queues_incremental()lowersadapter->num_rx_queuesand then frees a queue'sbuffer_listpage while polls on surviving queues keep running. Can a poll that already loaded the old bound dereference buffer_list_addr[i] for a freed page here?
Yes. This needs to become queue-local and NULL-safe.
I’m planning to update only the current queue’s slot, keep the absolute PHYP count in rx_qstats[q].no_buffer_drops, and derive the adapter rx_no_buffer total by summing on read. That avoids concurrent-poll RMW on one shared field, avoids foreign cacheline writes, and closes the scale-down freed-page hazard.
Quoted 17 lines
[Severity: Medium]
Should
no_buffer_dropsbe accumulated rather than assigned?The value PHYP writes into the last 8 bytes of the buffer-list page is an absolute count for the life of that page, and the page is re-obtained with
get_zeroed_page(GFP_KERNEL)byibmveth_alloc_rx_queues()on everyibmveth_open()and released byibmveth_cleanup_rx_resources()on everyibmveth_close(), including the close/open pairs done by the reset work,ibmveth_set_csum_offload()andibmveth_set_tso().Since rx_qstats[i].no_buffer_drops is assigned with "= drops", does
rxN_no_buffer_drops(and the recomputedadapter->rx_no_buffer) jump backwards after an ifdown/ifup, a driver reset, or an ethtool -K tso change?
Quoted 5 lines
That appears to contradict the comment this patch adds above the sum helpers:
* globals on the hot path (ibmvnic-style); with qstats allocated for the
* adapter lifetime, these sums remain meaningful across ifdown/up.
The PHYP field there is an absolute count for the life of that buffer-list page, so assigning = into the per-queue slot is correct for that page lifetime.
What needs tightening is the lifetime wording around the aggregate:
qstats live for the adapter lifetime, but a raw sum of these page- absolute values can still go backwards across ifdown/up. I’m planning to clarify that in the comments and ethtool wording rather than imply monotonic lifetime no_buffer semantics without a baseline+delta model.
Quoted 50 lines
Quoted 17 lines
drivers/net/ethernet/ibm/ibmveth.c +13
-0@@ -1972,22 +2028,131 @@ static int ibmveth_set_features(struct net_device *dev,* globals on the hot path (ibmvnic-style); with qstats allocated for the * adapter lifetime, these sums remain meaningful across ifdown/up. */ +static u64 ibmveth_sum_rx_invalid_buffers (struct ibmveth_adapter *adapter) +{ +u64 total =0 ; +int i; + +if (!adapter->rx_qstats) +return adapter->rx_invalid_buffer; + +for (i =0 ; i < adapter->num_rx_queues; i++) + total += adapter->rx_qstats[i].invalid_buffers; + +return total; +}[ ... ]
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-0+static u64 ibmveth_sum_tx_send_failed (struct ibmveth_adapter *adapter) +{ +struct net_device *netdev = adapter->netdev; +u64 total =0 ;drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_set_features(struct net_device *dev, · line 2087
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+int i; + +if (!adapter->tx_qstats) +return adapter->tx_send_failed; + +for (i =0 ; i < netdev->real_num_tx_queues; i++) + total += adapter->tx_qstats[i].send_failures; + +return total; +}[Severity: Medium]
Do these sums go backwards when the queue count is reduced?
The qstats arrays persist for the adapter lifetime, but the sums are bounded by the currently configured queue count.
ibmveth_set_channels()lowersnetdev->real_num_tx_queues:rc = netif_set_real_num_tx_queues(netdev, goal);Nothing folds or clears the counters of the queues that disappear, so after "ethtool -L ethX tx N" with a smaller N, do the adapter-level ethtool strings
tx_large_packetsandtx_send_failed, plus thertnl_link_stats64tx_packets/tx_bytes/tx_dropped summed inibmveth_get_stats64(), all drop by the traffic accounted to the removed queues, and then jump forward again when the count is raised?
Quoted 6 lines
Since this patch removes the hot-path
netdev->statsaccounting,ibmveth_get_stats64()is now the only source for the rtnl counters, so a decrease is directly user visible; tools computing deltas (ip -s link, ifstat) will read it as a counter wrap. The same applies on the RX side withadapter->num_rx_queuesonce the incremental RX resize commit later in this series changes it at runtime.
Yes. The aggregate sums should not be bounded by only the currently live queue count.
I’m planning to make the aggregate stats walk the fixed MAX queue slots so retired queues keep contributing and the adapter-visible totals do not move backwards when the active queue count shrinks.
Quoted 66 lines
Quoted 41 lines
drivers/net/ethernet/ibm/ibmveth.c +32
-2+static u64 ibmveth_ethtool_adapter_stat (struct ibmveth_adapter *adapter, +int index) +{ +unsigned long offset = ibmveth_stats[index].offset; + +if (offset ==IBMVETH_STAT_OFF (rx_invalid_buffer)) +return ibmveth_sum_rx_invalid_buffers (adapter); +if (offset ==IBMVETH_STAT_OFF (rx_large_packets)) +return ibmveth_sum_rx_large_packets (adapter); +if (offset ==IBMVETH_STAT_OFF (tx_large_packets)) +return ibmveth_sum_tx_large_packets (adapter); +if (offset ==IBMVETH_STAT_OFF (tx_send_failed)) +return ibmveth_sum_tx_send_failed (adapter); + +return IBMVETH_GET_STAT (adapter, offset); +} +static void ibmveth_get_strings (struct net_device *dev,u32 stringset,u8 *data) { +struct ibmveth_adapter *adapter =netdev_priv (dev); +u8 *p = data;int i;if (stringset !=ETH_SS_STATS )return ; -for (i =0 ; i <ARRAY_SIZE (ibmveth_stats); i++, data +=ETH_GSTRING_LEN ) -memcpy (data, ibmveth_stats[i].name,ETH_GSTRING_LEN ); +for (i =0 ; i <ARRAY_SIZE (ibmveth_stats); i++) { +memcpy (p, ibmveth_stats[i].name,ETH_GSTRING_LEN ); + p +=ETH_GSTRING_LEN ; + } + +for (i =0 ; i < adapter->num_rx_queues; i++) { +ethtool_sprintf (&p,"rx%d_packets" , i); +ethtool_sprintf (&p,"rx%d_bytes" , i); +ethtool_sprintf (&p,"rx%d_interrupts" , i); +ethtool_sprintf (&p,"rx%d_polls" , i); +ethtool_sprintf (&p,"rx%d_large_packets" , i); +ethtool_sprintf (&p,"rx%d_invalid_buffers" , i); +ethtool_sprintf (&p,"rx%d_no_buffer_drops" , i);drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_set_features(struct net_device *dev, · line 2137
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+ } + +for (i =0 ; i < dev->real_num_tx_queues; i++) { +ethtool_sprintf (&p,"tx%d_packets" , i); +ethtool_sprintf (&p,"tx%d_bytes" , i); +ethtool_sprintf (&p,"tx%d_large_packets" , i); +ethtool_sprintf (&p,"tx%d_dropped_packets" , i); +ethtool_sprintf (&p,"tx%d_send_failures" , i); +ethtool_sprintf (&p,"tx%d_checksum_offload" , i); + }[Severity: Medium]
Should the per-queue packet and byte counters use the standard per-queue statistics interface instead of driver-private ethtool strings?
rx%d_packets, rx%d_bytes, tx%d_packets and tx%d_bytes map one to one onto struct
netdev_queue_stats_rxand structnetdev_queue_stats_tx, which are exported through netlink via structnetdev_stat_ops. Nonetdev_stat_opsis added here, so generic tooling (ynl) cannot consume these values, andDocumentation/networking/statistics.rstasks that counters with a standard uAPI not be duplicated in ethtool -S.Also, the new
ibmveth_get_stats64()populates only rx/tx packets, bytes andtx_dropped:
Quoted 8 lines
stats->tx_errors = dev->stats.tx_errors;After this patch, is there any remaining writer of
dev->statsin this driver? The only match in the file is this read, sotx_errorsappears to be permanently 0. The loss counters this patch does collect (no_buffer_dropsfrom the hypervisor,invalid_buffers) are not mapped intorx_dropped/rx_missed_errors/rx_errorseither, so ip -s link reports zero RX drops while ethtool -S shows a non-zerorx_no_buffer.
Agreed in principle. The duplicated standard queue stats in -S, the stale tx_errors read, and the incomplete RX drop mapping are all real interface issues, but I still see them as broader uAPI cleanup rather than something that has to be solved in the same patch as MQ stats enablement.
I’m planning to keep the MQ correctness fixes in this series and leave the netdev_stat_ops / rtnl mapping cleanup as a follow-up, since that is really an ABI/tools migration rather than a small MQ fix.
Quoted 66 lines
Quoted 22 lines
drivers/net/ethernet/ibm/ibmveth.c +12
-1+ +for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) { +ethtool_sprintf (&p,"pool%d_size" , i); +ethtool_sprintf (&p,"pool%d_active" , i); +ethtool_sprintf (&p,"pool%d_available" , i); + } }static int ibmveth_get_sset_count (struct net_device *dev,int sset) { +struct ibmveth_adapter *adapter =netdev_priv (dev); +switch (sset) {case ETH_SS_STATS : -return ARRAY_SIZE (ibmveth_stats); +return ARRAY_SIZE (ibmveth_stats) + + adapter->num_rx_queues *IBMVETH_NUM_RX_QSTATS + + dev->real_num_tx_queues *IBMVETH_NUM_TX_QSTATS + +IBMVETH_NUM_BUFF_POOLS *3 ;default :return -EOPNOTSUPP ; }[ ... ]
Quoted 15 lines
drivers/net/ethernet/ibm/ibmveth.c +6
-2@@ -1996,21 +2161,59 @@ static int ibmveth_get_sset_count(struct net_device *dev, int sset)static void ibmveth_get_ethtool_stats (struct net_device *dev,struct ethtool_stats *stats,u64 *data) { -int i;struct ibmveth_adapter *adapter =netdev_priv (dev); +int i, j;for (i =0 ; i <ARRAY_SIZE (ibmveth_stats); i++) - data[i] =IBMVETH_GET_STAT (adapter, ibmveth_stats[i].offset); + data[i] =ibmveth_ethtool_adapter_stat (adapter, i); + +for (j =0 ; j < adapter->num_rx_queues; j++) { +if (adapter->rx_qstats) { + data[i++] = adapter->rx_qstats[j].packets;drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_get_sset_count(struct net_device *dev, int sset) · line 2182
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+ data[i++] = adapter->rx_qstats[j].bytes; + data[i++] = adapter->rx_qstats[j].interrupts; + data[i++] = adapter->rx_qstats[j].polls; + data[i++] = adapter->rx_qstats[j].large_packets; + data[i++] = adapter->rx_qstats[j].invalid_buffers; + data[i++] = adapter->rx_qstats[j].no_buffer_drops; + }else { + i +=IBMVETH_NUM_RX_QSTATS ; + } + }[Severity: Low]
Can
adapter->rx_qstatsoradapter->tx_qstatsever be NULL here?The allocations happen in
ibmveth_probe()beforeregister_netdev(), and a failure aborts probe withibmveth_probe_cleanup()+ return-ENOMEM.ibmveth_remove()frees them only afterunregister_netdev(). So on a registered netdev the NULL case seems unreachable, which would make these else arms, the else arms inibmveth_poll(), and the
"if (!adapter->*_qstats) return adapter-><legacy>;" early returns in the four sum helpers dead code.The two styles also disagree:
ibmveth_is_packet_unsupported()guards with "if (adapter->tx_qstats)" whileibmveth_start_xmit()dereferences adapter->tx_qstats[queue_num] unconditionally at six new sites on the same call chain. Which of the two is intended?
Quoted 8 lines
A side effect is that
adapter->rx_invalid_buffer,rx_large_packets,tx_large_packetsandtx_send_failedare never incremented again while still being listed in ibmveth_stats[].Separately,
ibmveth_ethtool_adapter_stat()dispatches by comparing ibmveth_stats[index].offset againstIBMVETH_STAT_OFF()values. Would adding or reordering a field in structibmveth_adaptersilently redirect a statistic to the wrong source here?
Yes. After successful probe those NULL arms are effectively dead, and the mixed guarded/unguarded style should be simplified.
I’m planning to collapse that to one defensive style at fold rather than leave the current inconsistent mix in place.
Separately, I agree the offset-based adapter-stat dispatch is brittle, even if it is not the main correctness blocker here.
Quoted 32 lines
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+ +for (j =0 ; j < dev->real_num_tx_queues; j++) { +if (adapter->tx_qstats) { + data[i++] = adapter->tx_qstats[j].packets; + data[i++] = adapter->tx_qstats[j].bytes; + data[i++] = adapter->tx_qstats[j].large_packets; + data[i++] = adapter->tx_qstats[j].dropped_packets; + data[i++] = adapter->tx_qstats[j].send_failures; + data[i++] = adapter->tx_qstats[j].checksum_offload; + }else {drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_get_sset_count(struct net_device *dev, int sset) · line 2202
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +9
-0+ i +=IBMVETH_NUM_TX_QSTATS ; + } + } + +for (j =0 ; j <IBMVETH_NUM_BUFF_POOLS ; j++) { + data[i++] = adapter->rx_buff_pool[0 ][j].size; + data[i++] = adapter->rx_buff_pool[0 ][j].active; + data[i++] =atomic_read (&adapter->rx_buff_pool[0 ][j].available); + } }[Severity: Medium]
Should the pool%d_* strings be queue-indexed?
The driver now keeps an independent pool set per RX queue (rx_buff_pool[queue][pool], populated for all q <
adapter->num_rx_queuesbyibmveth_alloc_buffer_pools()and updated per queue byibmveth_replenish_task()), but these three entries read only rx_buff_pool[0][j] under queue-agnostic names, so the state of queues 1..N-1 is not visible. Is that intended for someone debugging drops on a non-zero queue?
Quoted 3 lines
Also, size and active are configuration values already exposed through the per-pool sysfs kobjects rather than statistics, and this new permanent ethtool string set is not mentioned in the commit message.
Yes, that is intentional here: those pool%d_* entries are the shared queue-0 template view, not live per-queue runtime state.
I’m planning to make that clearer in the wording. Live per-queue state belongs in the later debugfs visibility, and I’m fine with dropping size / active from ethtool -S later if sysfs-only is preferred.
Quoted 22 lines
Quoted 6 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-0static void ibmveth_get_channels (struct net_device *netdev,struct ethtool_channels *channels) { +struct ibmveth_adapter *adapter =netdev_priv (netdev); + channels->max_tx =ibmveth_real_max_tx_queues ();drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_get_sset_count(struct net_device *dev, int sset) · line 2217
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +5
-2channels->tx_count = netdev->real_num_tx_queues; - channels->max_rx = netdev->real_num_rx_queues; - channels->rx_count = netdev->real_num_rx_queues; +if (adapter->multi_queue) + channels->max_rx =IBMVETH_MAX_RX_QUEUES ; +else + channels->max_rx =1 ; + channels->rx_count = adapter->num_rx_queues; }[Severity: Low]
This isn't a bug, but would this ABI-visible
get_channels()reporting change be easier to review as its own patch? It is independent of the statistics work, and if it is a fix it would want its own Fixes: tag.
Quoted 8 lines
There is also a mismatch it introduces: the new RX loops in
ibmveth_get_strings()/ibmveth_get_sset_count()/ibmveth_get_stats64()useadapter->num_rx_queueswhile the TX loops usedev->real_num_tx_queues.netdev->real_num_rx_queuesis only synced toadapter->num_rx_queuesinibmveth_open(), so on a never-opened interface ethtool -l reports a count the stack does not have yet.[ ... ]
Agreed. That get_channels() reporting change is separate enough that it should be split out so this patch stays stats-focused and the reporting change can be reviewed on its own.
Quoted 27 lines
Quoted 7 lines
drivers/net/ethernet/ibm/ibmveth.c +1
-0@@ -2150,6 +2355,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb, skb_checksum_help(skb)) {netdev_err (netdev,"tx: failed to checksum packet\n" ); + adapter->tx_qstats[queue_num].dropped_packets++; goto out; }[ ... ]
drivers/net/ethernet/ibm/ibmveth.c static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb, · line 2431
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-0@@ -2211,7 +2423,11 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,dma_wmb ();if (ibmveth_send (adapter, desc.desc, mss)) { + adapter->tx_qstats[queue_num].send_failures++; + adapter->tx_qstats[queue_num].dropped_packets++; }else { + adapter->tx_qstats[queue_num].packets++; + adapter->tx_qstats[queue_num].bytes += skb->len; }[Severity: Medium]
These empty if/else bodies in the pre-image show that the previous commit in the series ("ibmveth: Enable multi-queue RX receive path") deleted the hot-path
netdev->statsaccounting, and the replacement only arrives here.ibmveth_poll()likewise increments no packet or byte counter at that commit, and nondo_get_stats64exists yet.
Quoted 7 lines
Does that leave the tree at the parent commit reporting zero rx_packets/rx_bytes/tx_packets/tx_bytes/tx_dropped through ip -s link, where the pre-series baseline reported them? Would squashing the removal into this patch, or deferring the removal until this replacement lands, keep every commit in the series bisectable for counter-related issues?
[ ... ]
Fair point. If the series keeps the current split, I’m planning to note that intermediate stats hole in the cover and consider a minimal bridge when restacking so the series stays bisectable for counter behavior.
Quoted 13 lines
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0@@ -2698,6 +2933,40 @@ static netdev_features_t ibmveth_features_check(struct sk_buff *skb,return vlan_features_check (skb, features); }[ ... ]
drivers/net/ethernet/ibm/ibmveth.c static const struct net_device_ops ibmveth_netdev_ops = { · line 2991
Quoted 5 lines
static void ibmveth_put_pool_kobjs(struct
ibmveth_adapter*adapter,drivers/net/ethernet/ibm/ibmveth.c +1
-1- int pools_ready) + int pools_ready) {int i;[Severity: Low]
Quoted 4 lines
This isn't a bug, but this whitespace-only reflow breaks the previously correct open-parenthesis alignment of the second parameter (checkpatch:
"Alignment should match open parenthesis") in a function this patch does not otherwise change. Could it be dropped?
Yes. That whitespace-only reflow should just be dropped.
Quoted 48 lines
Quoted 7 lines
drivers/net/ethernet/ibm/ibmveth.c +3
-0@@ -2724,6 +2994,19 @@ static void ibmveth_put_pool_kobjs(struct ibmveth_adapter *adapter,kobject_put (&adapter->rx_buff_pool[0 ][i].kobj); } +static void ibmveth_probe_cleanup (struct ibmveth_adapter *adapter, +int pools_ready) +{drivers/net/ethernet/ibm/ibmveth.c static void ibmveth_put_pool_kobjs(struct ibmveth_adapter *adapter, · line 3009
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+struct net_device *netdev = adapter->netdev; + +cancel_work_sync (&adapter->work); +ibmveth_put_pool_kobjs (adapter, pools_ready); + +ibmveth_free_tx_qstats (adapter); +ibmveth_free_rx_qstats (adapter); +free_netdev (netdev); +} +[Severity: High]
This isn't a problem introduced by this patch, since the old open-coded error paths had the same omission, but the new helper is now the single place four probe failure paths funnel through, so it looks like the natural spot to fix it.
ibmveth_probe()stores the netdev in the VIO drvdata right afteralloc_etherdev_mqs():dev_set_drvdata(&dev->dev, netdev);
ibmveth_probe_cleanup()ends withfree_netdev(netdev)but never clears it, andibmveth_remove()(the only caller ofdev_set_drvdata(&dev->dev, NULL)) does not run when probe fails. So after any probe failure thevio_devkeeps a pointer to freed memory.On
FW_FEATURE_CMOsystems the stale pointer is consumed on the next bind attempt, before the driver probe re-initializes it:
vio_bus_probe()
vio_cmo_bus_probe()viodev->cmo.desired = IOMMU_PAGE_ALIGN(viodrv->get_desired_dma(viodev), tbl); ibmveth_get_desired_dma()
ibmveth_get_desired_dma()only checks for NULL:netdev = dev_get_drvdata(&vdev->dev); if (netdev == NULL) return ...;
Quoted 5 lines
and then dereferences
netdev->mtuandadapter->num_rx_queues, using the latter as the loop bound over adapter->rx_buff_pool[q][i]. Since that bound is read from freed memory, can this read past the end of the freed allocation as well? Would addingdev_set_drvdata(&dev->dev, NULL)toibmveth_probe_cleanup()close this?
Yes. probe_cleanup() needs to clear the VIO drvdata before free_netdev(), so a later CMO entitlement path cannot see a freed netdev.
Quoted 62 lines
Quoted 15 lines
drivers/net/ethernet/ibm/ibmveth.c +5
-0static int ibmveth_probe (struct vio_dev *dev,const struct vio_device_id *id) {int rc, i, mac_len, pools_ready =0 ;@@ -2779,6 +3062,11 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)netif_napi_add_weight (netdev, &adapter->napi[i], ibmveth_poll,16 ); +if (ibmveth_alloc_rx_qstats (adapter) || +ibmveth_alloc_tx_qstats (adapter)) { +ibmveth_probe_cleanup (adapter,0 ); +return -ENOMEM ; + } netdev->irq = dev->irq; netdev->netdev_ops = &ibmveth_netdev_ops;[ ... ]
drivers/net/ethernet/ibm/ibmveth.c static void ibmveth_remove(struct vio_dev *dev) · line 3206
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +3
-0@@ -2913,6 +3198,9 @@ static void ibmveth_remove(struct vio_dev *dev)unregister_netdev (netdev); +ibmveth_free_tx_qstats (adapter); +ibmveth_free_rx_qstats (adapter); +free_netdev (netdev);dev_set_drvdata (&dev->dev,NULL ); }[Severity: High]
This is a pre-existing issue, since the
cancel_work_sync()placement inibmveth_remove()predates this patch, but the patch adds two new frees into exactly that window.
ibmveth_remove()callscancel_work_sync(&adapter->work)at the top, beforeunregister_netdev(), so the device is still open when the cancel returns and softirq context can re-arm the work:
ibmveth_poll()/* onibmveth_enable_irq()failure */
schedule_work(&adapter->work);
ibmveth_replenish_buffer_pool()/* on an invalidfree_mapindex */
schedule_work(&adapter->work);Timeline:
CPU A:
cancel_work_sync()returns, queue empty
CPU B:ibmveth_poll()->schedule_work(&adapter->work)
CPU A:unregister_netdev()
CPU A:ibmveth_free_tx_qstats()/ibmveth_free_rx_qstats()CPU A:free_netdev(netdev)
CPU B:ibmveth_reset()runscontainer_of(w, struct ibmveth_adapter, work)Can the worker then dereference the freed adapter and netdev in
rtnl_lock();dev_close(adapter->netdev);dev_open(...)?If it reaches
dev_open()->netif_tx_start_all_queues(), TX restarts withtx_qstats== NULL, and the unguarded adapter->tx_qstats[queue_num] dereferences added toibmveth_start_xmit()by this patch would then be a NULL dereference. The new "if (adapter->rx_qstats)" hot-path guards do not help, since the adapter holding the pointer is itself inside the freed netdev.
Quoted 2 lines
Would moving the cancel (or
disable_work_sync()) to afterunregister_netdev()address this?
Yes. remove() needs to unregister the netdev first, then cancel the work, then free the qstats and netdev, so nothing can re-arm the reset work after the cancel returns.
Thanks,
Mingming
PATCH v4 11/14 ibmveth: Expose per-queue buffer pool details via debugfs
In reply to Mingming CaoAdd a read-only buffer_pools debugfs file that lists size, buff_size, active, and available for every RX queue and buffer pool. That is the MQ diagnostic view (multi-line table), which does not belong in sysfs.
Keep the historical queue-0 poolN sysfs ABI unchanged:
.../poolN/{active,num,size}
Those are one-value-per-file *configuration* knobs for buffer-pool classes on queue 0, not one directory per RX queue. With MQ, queue 0 pool geometry remains the shared template: open copies it to queues 1..N. Per-queue runtime pressure is what debugfs shows; this series does not add per-queue pool sysfs dirs.
Unlike ibmvnic (one RX pool per RX queue, sized from firmware, almost no pool sysfs), ibmveth historically has multiple size-class pools and already exported poolN via sysfs. Moving only the new all-queue dump to debugfs matches sysfs "one value per file" review feedback without breaking that config ABI.
---
Changes in v4:
- Move the all-queue buffer_pools diagnostic from sysfs to debugfs; subject updated to match.
- Keep historical queue-0 poolN/{active,num,size} sysfs as one-value config (template for MQ); do not add per-queue pool sysfs dirs.
drivers/net/ethernet/ibm/ibmveth.c | 43 ++++++++++++++++++++++++++++++ drivers/net/ethernet/ibm/ibmveth.h | 2 ++ 2 files changed, 45 insertions(+)
drivers/net/ethernet/ibm/ibmveth.c +43 -0
@@ -31,6 +31,7 @@#include <linux/ipv6.h> #include <linux/slab.h> #include <linux/spinlock.h> +#include <linux/debugfs.h> #include <asm/hvcall.h> #include <linux/atomic.h> #include <asm/vio.h>
@@ -2985,6 +2986,44 @@ static const struct net_device_ops ibmveth_netdev_ops = {#endif }; +static int ibmveth_buffer_pools_show (struct seq_file *m,void *v) +{ +struct ibmveth_adapter *adapter = m->private; +int i, j; + +seq_puts (m,"Queue Pool Size BuffSize Active Available\n" ); +seq_puts (m,"----- ---- ---- -------- ------ ---------\n" ); + +for (i =0 ; i < adapter->num_rx_queues; i++) { +for (j =0 ; j <IBMVETH_NUM_BUFF_POOLS ; j++) { +struct ibmveth_buff_pool *pool = + &adapter->rx_buff_pool[i][j]; + +seq_printf (m,"%5d %4d %4u %8u %6d %9d\n" , + i, j, pool->size, pool->buff_size, + pool->active, +atomic_read (&pool->available)); + } + } + +return 0 ; +} +DEFINE_SHOW_ATTRIBUTE (ibmveth_buffer_pools); + +static void ibmveth_debugfs_init (struct ibmveth_adapter *adapter) +{ + adapter->debugfs_dir =debugfs_create_dir (adapter->netdev->name, +NULL ); +debugfs_create_file ("buffer_pools" ,0400 , adapter->debugfs_dir, + adapter, &ibmveth_buffer_pools_fops); +} + +static void ibmveth_debugfs_exit (struct ibmveth_adapter *adapter) +{ +debugfs_remove_recursive (adapter->debugfs_dir); + adapter->debugfs_dir =NULL ; +} +static void ibmveth_put_pool_kobjs (struct ibmveth_adapter *adapter,int pools_ready) {
@@ -3182,6 +3221,8 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)netdev_dbg (netdev,"registered\n" ); +ibmveth_debugfs_init (adapter); +return 0 ; }
@@ -3193,6 +3234,8 @@ static void ibmveth_remove(struct vio_dev *dev)cancel_work_sync (&adapter->work); +ibmveth_debugfs_exit (adapter); +for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++)kobject_put (&adapter->rx_buff_pool[0 ][i].kobj);
drivers/net/ethernet/ibm/ibmveth.h +2 -0
@@ -387,6 +387,8 @@ struct ibmveth_adapter {struct ibmveth_rx_queue_stats *rx_qstats;struct ibmveth_tx_queue_stats *tx_qstats; +struct dentry *debugfs_dir; +/* Ethtool settings */ u8 duplex;u32 speed;
Signature · 1 line
2.50.1 (Apple Git-155)
Jakub Kicinski
maintainerRe: [PATCH net-next v4 11/14] ibmveth: Expose per-queue buffer pool details via debugfs
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
ibmveth: Expose per-queue buffer pool details via debugfs
This adds a read-only buffer_pools debugfs file that dumps size, buff_size, active and available for every RX queue and buffer pool, and wires creation into ibmveth_probe() and teardown into ibmveth_remove(). The existing queue-0 poolN sysfs files are left unchanged.
A few questions about the new code below.
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0
[ ... ]
Quoted 6 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-0@@ -2985,6 +2986,44 @@ static const struct net_device_ops ibmveth_netdev_ops = {#endif }; +static int ibmveth_buffer_pools_show (struct seq_file *m,void *v) +{
drivers/net/ethernet/ibm/ibmveth.c static const struct net_device_ops ibmveth_netdev_ops = { · line 3000
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+struct ibmveth_adapter *adapter = m->private; +int i, j; + +seq_puts (m,"Queue Pool Size BuffSize Active Available\n" ); +seq_puts (m,"----- ---- ---- -------- ------ ---------\n" ); + +for (i =0 ; i < adapter->num_rx_queues; i++) { +for (j =0 ; j <IBMVETH_NUM_BUFF_POOLS ; j++) { +struct ibmveth_buff_pool *pool = + &adapter->rx_buff_pool[i][j];
[Severity: Low]
Should this dump be taken under rtnl_lock()?
ibmveth_buffer_pools_show() reads adapter->num_rx_queues plus pool->size, pool->buff_size, pool->active and pool->available with no lock held, but every writer of those fields runs under RTNL:
veth_pool_store() {
...
rtnl_lock();
...
pool->active = newactive;
pool->buff_size = newbuff_size;
pool->size = newsize;
...
/* on ibmveth_open() failure, the same three fields are
* written back with the old values */
pool->active = oldactive;
pool->buff_size = oldbuff_size;
pool->size = oldsize;
...
}ibmveth_alloc_buffer_pools() likewise copies the queue-0 geometry into queues 1..N-1 field by field under RTNL from ibmveth_open(), and the reset path goes through dev_close()/dev_open() under rtnl_lock().
Can a concurrent read of buffer_pools therefore print a torn snapshot, for example active=1 paired with the previous size/buff_size, or a half-copied per-queue template, or the transient rollback state that was never an actual configuration?
There does not appear to be a memory-safety consequence here, since rx_buff_pool[] is a fixed-size array embedded in struct ibmveth_adapter and only scalar members are read, so the concern is limited to the values printed.
drivers/net/ethernet/ibm/ibmveth.c static const struct net_device_ops ibmveth_netdev_ops = { · line 3005
Quoted 5 lines
drivers/net/ethernet/ibm/ibmveth.c +5
-0+ +seq_printf (m,"%5d %4d %4u %8u %6d %9d\n" , + i, j, pool->size, pool->buff_size, + pool->active, +atomic_read (&pool->available));
[Severity: Low]
Are Active and Available meaningful while the interface is down?
ibmveth_free_buffer_pool() frees free_map, unmaps and frees all skbuffs, and frees dma_addr/skbuff, but leaves pool->active set and never decrements pool->available:
kfree(pool->free_map);
pool->free_map = NULL;
...
if (pool->skbuff) {
kfree(pool->skbuff);
pool->skbuff = NULL;
}
/* pool->active and pool->available untouched */available is only cleared later, in ibmveth_alloc_buffer_pool() via atomic_set(&pool->available, 0) on the next open.
So a read after ibmveth_close() -> ibmveth_free_buffer_pools() -> ibmveth_free_queue_buffer_pools() -> ibmveth_free_buffer_pool() reports every configured pool as Active=1 with the last non-zero Available, even though the skbuffs and DMA mappings are gone.
Related: for queues 1..N-1 the table prints all-zero rows before the first open, because the queue-0 template is only copied into those queues inside ibmveth_alloc_buffer_pools() during open.
The commit message says "Per-queue runtime pressure is what debugfs shows" - could the down/uninitialized cases be either reflected in the output or documented?
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-0+ } + } + +return 0 ;
drivers/net/ethernet/ibm/ibmveth.c static const struct net_device_ops ibmveth_netdev_ops = { · line 3019
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+} +DEFINE_SHOW_ATTRIBUTE (ibmveth_buffer_pools); + +static void ibmveth_debugfs_init (struct ibmveth_adapter *adapter) +{ + adapter->debugfs_dir =debugfs_create_dir (adapter->netdev->name, +NULL ); +debugfs_create_file ("buffer_pools" ,0400 , adapter->debugfs_dir, + adapter, &ibmveth_buffer_pools_fops); +}
[Severity: Medium]
Is the mutable interface name a safe key for a directory created directly in the debugfs root?
Three separate things follow from passing NULL as the parent and netdev->name as the name.
First, the name goes stale. The directory is created once, as the last statement of ibmveth_probe(), and the driver installs no netdev notifier and never calls debugfs_rename(). After the udev/systemd predictable-name rename that happens right after register_netdev(), or after any later ip link set dev ethN name foo, does the debugfs directory keep the obsolete name for the lifetime of the device?
Second, with the debugfs root as parent the name has to be globally unique there. Consider the sequence: first adapter probes as eth0, udev renames it to enP1p1s0, a second ibmveth adapter probes and alloc_etherdev_mqs() names it eth0 again because eth0 is free in the netdev namespace. debugfs_create_dir("eth0", NULL) then collides with the first adapter's stale directory:
fs/debugfs/inode.c:debugfs_start_creating() {
...
dentry = simple_start_creating(parent, name);
if (IS_ERR(dentry)) {
if (dentry == ERR_PTR(-EEXIST))
pr_err("'%s' already exists in '%pd'\n", name, parent);
...
}The ERR_PTR is stored in adapter->debugfs_dir and handed to debugfs_create_file() as the parent, which returns immediately:
fs/debugfs/inode.c:debugfs_start_creating() {
...
if (IS_ERR(parent))
return parent;
...
}Does the second and every subsequent adapter then end up with no buffer_pools file at all, with only that debugfs pr_err as an indication?
Third, netdev->name is read here after register_netdev() has published the device, without rtnl_lock(), while dev_change_name() rewrites net_device->name under RTNL. Can the string handed to debugfs_create_dir() be a partially updated name?
Would a driver-owned parent directory plus a stable identifier such as dev_name(&dev->dev) avoid all three?
Quoted 11 lines
drivers/net/ethernet/ibm/ibmveth.c +7
-0+ +static void ibmveth_debugfs_exit (struct ibmveth_adapter *adapter) +{ +debugfs_remove_recursive (adapter->debugfs_dir); + adapter->debugfs_dir =NULL ; +} +static void ibmveth_put_pool_kobjs (struct ibmveth_adapter *adapter,int pools_ready) {
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id) · line 3227
Quoted 7 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-0@@ -3182,6 +3221,8 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)netdev_dbg (netdev,"registered\n" ); +ibmveth_debugfs_init (adapter); +return 0 ; }
[ ... ]
Cross-instance finding from sashiko-gemini (b6bfc50872782421f7178a42615ea39fe861918e607d995ed830a5a2db8a4460):
[Severity: Medium]
Addition of driver-private ethtool -S strings for standard queue statistics.
Re: [PATCH net-next v4 11/14] ibmveth: Expose per-queue buffer pool details via debugfs
Quoted 11 lines · Jakub Kicinski
his is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
ibmveth: Expose per-queue buffer pool details via debugfsThis adds a read-only
buffer_poolsdebugfs file that dumps size,buff_size, active and available for every RX queue and buffer pool, and wires creation intoibmveth_probe()and teardown intoibmveth_remove(). The existing queue-0 poolN sysfs files are left unchanged.A few questions about the new code below.
Hi Jakub,
Thanks for the review.
Quoted 54 lines
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0[ ... ]
Quoted 6 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-0@@ -2985,6 +2986,44 @@ static const struct net_device_ops ibmveth_netdev_ops = {#endif }; +static int ibmveth_buffer_pools_show (struct seq_file *m,void *v) +{drivers/net/ethernet/ibm/ibmveth.c static const struct net_device_ops ibmveth_netdev_ops = { · line 3000
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+struct ibmveth_adapter *adapter = m->private; +int i, j; + +seq_puts (m,"Queue Pool Size BuffSize Active Available\n" ); +seq_puts (m,"----- ---- ---- -------- ------ ---------\n" ); + +for (i =0 ; i < adapter->num_rx_queues; i++) { +for (j =0 ; j <IBMVETH_NUM_BUFF_POOLS ; j++) { +struct ibmveth_buff_pool *pool = + &adapter->rx_buff_pool[i][j];[Severity: Low]
Should this dump be taken underrtnl_lock()?
ibmveth_buffer_pools_show()readsadapter->num_rx_queuespluspool->size,pool->buff_size,pool->activeandpool->availablewith no lock held, but every writer of those fields runs under RTNL:veth_pool_store() { ... rtnl_lock(); ... pool->active = newactive; pool->buff_size = newbuff_size; pool->size = newsize; ... /* on ibmveth_open() failure, the same three fields are * written back with the old values */ pool->active = oldactive; pool->buff_size = oldbuff_size; pool->size = oldsize; ... }
ibmveth_alloc_buffer_pools()likewise copies the queue-0 geometry into queues 1..N-1 field by field under RTNL fromibmveth_open(), and the reset path goes through dev_close()/dev_open() underrtnl_lock().Can a concurrent read of
buffer_poolstherefore print a torn snapshot, for example active=1 paired with the previous size/buff_size, or a half-copied per-queue template, or the transient rollback state that was never an actual configuration?
Quoted 4 lines
There does not appear to be a memory-safety consequence here, since rx_buff_pool[] is a fixed-size array embedded in struct
ibmveth_adapterand only scalar members are read, so the concern is limited to the values printed.
Yes. The writers already run under RTNL, so the dump can otherwise print a torn snapshot.
I'm planning to take rtnl_lock() around ibmveth_buffer_pools_show() so the output stays consistent.
Quoted 29 lines
drivers/net/ethernet/ibm/ibmveth.c static const struct net_device_ops ibmveth_netdev_ops = { · line 3005
Quoted 5 lines
drivers/net/ethernet/ibm/ibmveth.c +5
-0+ +seq_printf (m,"%5d %4d %4u %8u %6d %9d\n" , + i, j, pool->size, pool->buff_size, + pool->active, +atomic_read (&pool->available));[Severity: Low]
Are Active and Available meaningful while the interface is down?
ibmveth_free_buffer_pool()freesfree_map, unmaps and frees all skbuffs, and frees dma_addr/skbuff, but leavespool->activeset and never decrementspool->available:kfree(pool->free_map); pool->free_map = NULL; ... if (pool->skbuff) { kfree(pool->skbuff); pool->skbuff = NULL; } /* pool->active and pool->available untouched */available is only cleared later, in
ibmveth_alloc_buffer_pool()viaatomic_set(&pool->available, 0)on the next open.So a read after
ibmveth_close()->ibmveth_free_buffer_pools()->ibmveth_free_queue_buffer_pools()->ibmveth_free_buffer_pool()reports every configured pool as Active=1 with the last non-zero Available, even though the skbuffs and DMA mappings are gone.
Quoted 7 lines
Related: for queues 1..N-1 the table prints all-zero rows before the first open, because the queue-0 template is only copied into those queues inside
ibmveth_alloc_buffer_pools()during open.The commit message says "Per-queue runtime pressure is what debugfs shows" - could the down/uninitialized cases be either reflected in the output or documented?
Yes. The down/pre-open view needs to be more honest.
As written, the dump can present stale pool state as if it were live runtime state after close, and it can also show all-zero subordinate queues before the first open even though that is not the eventual runtime shape. I'm planning to make that state more explicit by clearing the down-state availability, treating pools without live backing state as not active runtime pools in the dump, and adding a short note when the adapter is not opened.
Quoted 58 lines
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-0+ } + } + +return 0 ;drivers/net/ethernet/ibm/ibmveth.c static const struct net_device_ops ibmveth_netdev_ops = { · line 3019
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+} +DEFINE_SHOW_ATTRIBUTE (ibmveth_buffer_pools); + +static void ibmveth_debugfs_init (struct ibmveth_adapter *adapter) +{ + adapter->debugfs_dir =debugfs_create_dir (adapter->netdev->name, +NULL ); +debugfs_create_file ("buffer_pools" ,0400 , adapter->debugfs_dir, + adapter, &ibmveth_buffer_pools_fops); +}[Severity: Medium]
Is the mutable interface name a safe key for a directory created directly in the debugfs root?Three separate things follow from passing NULL as the parent and
netdev->nameas the name.First, the name goes stale. The directory is created once, as the last statement of
ibmveth_probe(), and the driver installs no netdev notifier and never callsdebugfs_rename(). After the udev/systemd predictable-name rename that happens right afterregister_netdev(), or after any later ip link set dev ethN name foo, does the debugfs directory keep the obsolete name for the lifetime of the device?Second, with the debugfs root as parent the name has to be globally unique there. Consider the sequence: first adapter probes as eth0, udev renames it to enP1p1s0, a second ibmveth adapter probes and
alloc_etherdev_mqs()names it eth0 again because eth0 is free in the netdev namespace.debugfs_create_dir("eth0", NULL)then collides with the first adapter's stale directory:fs/debugfs/inode.c:debugfs_start_creating() { ... dentry = simple_start_creating(parent, name); if (IS_ERR(dentry)) { if (dentry == ERR_PTR(-EEXIST)) pr_err("'%s' already exists in '%pd'\n", name, parent); ... }The
ERR_PTRis stored inadapter->debugfs_dirand handed todebugfs_create_file()as the parent, which returns immediately:fs/debugfs/inode.c:debugfs_start_creating() { ... if (IS_ERR(parent)) return parent; ... }Does the second and every subsequent adapter then end up with no
buffer_poolsfile at all, with only that debugfspr_erras an indication?
Quoted 7 lines
Third,
netdev->nameis read here afterregister_netdev()has published the device, withoutrtnl_lock(), whiledev_change_name()rewritesnet_device->nameunder RTNL. Can the string handed todebugfs_create_dir()be a partially updated name?Would a driver-owned parent directory plus a stable identifier such as
dev_name(&dev->dev)avoid all three?
Yes. This is the main design issue in the patch.
Using netdev->name at the debugfs root means the directory can go stale after rename, collide with a later adapter that reuses the old name, and race with rename while the name is being read. I'm planning to move this under a driver-owned ibmveth/ parent and key each adapter directory off the stable VIO device name instead.
I agree that once the naming is fixed, leaving debugfs creation best-effort is much easier to justify. The bigger problem here is the choice of key and parent, not the lack of hard failure handling.
Quoted 20 lines
Quoted 18 lines
drivers/net/ethernet/ibm/ibmveth.c +9
-0+ +static void ibmveth_debugfs_exit (struct ibmveth_adapter *adapter) +{ +debugfs_remove_recursive (adapter->debugfs_dir); + adapter->debugfs_dir =NULL ; +} +static void ibmveth_put_pool_kobjs (struct ibmveth_adapter *adapter,int pools_ready) {@@ -3182,6 +3221,8 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)netdev_dbg (netdev,"registered\n" ); +ibmveth_debugfs_init (adapter); +return 0 ; }[ ... ]
Quoted 3 lines
Cross-instance finding from sashiko-gemini (
b6bfc50872782421f7178a42615ea39fe861918e607d995ed830a5a2db8a4460):
[Severity: Medium]
Addition of driver-private ethtool -S strings for standard queue statistic
I agree with the underlying point, but that belongs with the patch-10 stats/interface cleanup rather than this debugfs patch.
So my updated view is that the main patch-11 issue is the debugfs naming and lifetime design, while the RTNL and down/pre-open reporting points are real but lower-severity cleanup. I'm planning to tighten those pieces here, but I still think debugfs is the right place for the all-queue dump.
Thanks,
Mingming
PATCH v4 12/14 ibmveth: Implement incremental MQ RX queue resize
In reply to Mingming CaoAdd helpers to allocate, register, interrupt-setup, and free individual RX queues, plus ibmveth_resize_rx_queues_incremental() to grow or shrink the live RX queue count without tearing down surviving queues. Surviving queues keep PHYP handles, buffer pools, and IRQ state across resize.
Scale-up for each new queue:
alloc -> register -> request_irq -> publish num_rx_queues -> replenish -> napi_enable -> enable_irq
so the queue is visible and buffered before PHYP delivery is unmasked. On enable_irq failure after napi_enable, disable NAPI before teardown. Scale-down:
disable_irq+sync -> napi_disable -> drain -> deregister -> free
with PHYP mask before napi_disable/drain (same storm rule as close).
IRQ vs PHYP ownership for subordinates:
- deregister_single is PHYP-only (H_FREE_LOGICAL_LAN_QUEUE + clear queue_handle)
- Linux virq disposal lives in cleanup_single / dispose helpers
- single-slot dispose is bounded by IBMVETH_MAX_RX_QUEUES (not num_rx_queues) so scale-down / fail paths can dispose queues that left the published live set but still own a mapping
- dispose on request_irq failure where deregister no longer releases the mapping
Also add drain-path smp_rmb() before harvesting RX buffers so drain matches poll's descriptor-ordering assumptions, and replenish before re-enabling IRQ/NAPI on set_real_num_rx scale-down rollback.
Introduce ibmveth_resize_rx_channels() (validate + apply) and call it from set_channels() when the interface is up so the resize helper is not left unused. Full !IFF_UP RX stash ordering relative to TX is completed in the next patch.
---
Changes in v4:
- Copy pool->index when cloning buffer pools for incrementally added queues.
- Scale-up order: publish -> replenish -> napi_enable -> enable_irq (vs older enable-before-publish drafts); mirror NAPI-before-unmask on set_real_num_rx rollback.
- Linux-owned subordinate virq disposal; deregister is PHYP-only; dispose bound to MAX_RX_QUEUES; dispose on request_irq failure.
- Drain-path smp_rmb() before harvest.
- Mask PHYP before napi_disable/drain on scale-down and scale-up fail cleanup.
- Replenish before re-enabling IRQ/NAPI on set_real_num_rx scale-down rollback.
- Keep set_channels wiring as the following patch (same split as v3) but call resize_rx_channels() here when IFF_UP so the helper is not an unused static.
drivers/net/ethernet/ibm/ibmveth.c | 512 ++++++++++++++++++++++++++++- 1 file changed, 510 insertions(+), 2 deletions(-)
drivers/net/ethernet/ibm/ibmveth.c +510 -2
@@ -651,6 +651,58 @@ ibmveth_cleanup_rx_interrupts(struct ibmveth_adapter *adapter)/* Queue 0 uses netdev->irq; leave queue_irq[0] for next open. */ } +/** +* ibmveth_setup_single_rx_interrupt - Setup interrupt for a single RX queue +* @adapter: ibmveth adapter structure +* @queue_idx: Queue index to setup +* +* Registers the IRQ handler for one queue. Used during incremental +* scale-up when adding new RX queues. The caller publishes the queue, +* replenishes buffers, enables NAPI, then unmasks PHYP delivery. +* +* Return: 0 on success, negative error code on failure +*/ +static int +ibmveth_setup_single_rx_interrupt (struct ibmveth_adapter *adapter, +int queue_idx) +{ +struct net_device *netdev = adapter->netdev; +int rc; + + rc =request_irq (adapter->queue_irq[queue_idx], ibmveth_interrupt, +0 , netdev->name, &adapter->napi[queue_idx]); +if (rc) { +netdev_err (netdev,"request_irq() failed for queue %d: %d\n" , + queue_idx, rc); +return rc; + } + +netdev_dbg (netdev,"Setup IRQ %d for queue %d\n" , + adapter->queue_irq[queue_idx], queue_idx); +return 0 ; +} + +/** +* ibmveth_cleanup_single_rx_interrupt - Cleanup interrupt for a single RX queue +* @adapter: ibmveth adapter structure +* @queue_idx: Queue index to cleanup +* +* Frees the IRQ handler for one queue and releases the subordinate virq +* mapping. Used during incremental scale-down. +*/ +static void +ibmveth_cleanup_single_rx_interrupt (struct ibmveth_adapter *adapter, +int queue_idx) +{ +if (adapter->queue_irq[queue_idx]) { +free_irq (adapter->queue_irq[queue_idx], + &adapter->napi[queue_idx]); +ibmveth_dispose_subordinate_irq_mapping (adapter, queue_idx); +netdev_dbg (adapter->netdev, +"Freed IRQ for queue %d\n" , queue_idx); + } +} +/** * ibmveth_schedule_rx_queue - Mask PHYP IRQ and schedule NAPI for one RX queue * @adapter: ibmveth adapter structure
@@ -1189,6 +1241,141 @@ ibmveth_free_buffer_pools(struct ibmveth_adapter *adapter) adapter->num_rx_queues); } +/** +* ibmveth_alloc_single_rx_queue - Allocate resources for a single RX queue +* @adapter: ibmveth adapter structure +* @queue_idx: Queue index to allocate +* @rxq_entries: Number of RX queue entries +* +* Allocates buffer list, RX queue, and per-queue buffer pools for one queue. +* Used during incremental scale-up without affecting existing queues. +* +* Return: 0 on success, negative error code on failure +*/ +static int +ibmveth_alloc_single_rx_queue (struct ibmveth_adapter *adapter,int queue_idx, +int rxq_entries) +{ +struct device *dev = &adapter->vdev->dev; +struct net_device *netdev = adapter->netdev; +int i, rc = -ENOMEM ; + + adapter->buffer_list_addr[queue_idx] = + (void *)get_zeroed_page (GFP_KERNEL ); +if (!adapter->buffer_list_addr[queue_idx]) { +netdev_err (netdev,"unable to allocate buffer list for queue %d\n" , + queue_idx); +return -ENOMEM ; + } + + adapter->rx_queue[queue_idx].queue_len = +sizeof (struct ibmveth_rx_q_entry ) * rxq_entries; + adapter->rx_queue[queue_idx].queue_addr = +dma_alloc_coherent (dev, adapter->rx_queue[queue_idx].queue_len, + &adapter->rx_queue[queue_idx].queue_dma, +GFP_KERNEL ); +if (!adapter->rx_queue[queue_idx].queue_addr) { +netdev_err (netdev,"unable to allocate RX queue for queue %d\n" , + queue_idx); + goto out_free_buflist; + } + + adapter->buffer_list_dma[queue_idx] = +dma_map_single (dev, adapter->buffer_list_addr[queue_idx], +4096 ,DMA_BIDIRECTIONAL ); +if (dma_mapping_error (dev, adapter->buffer_list_dma[queue_idx])) { +netdev_err (netdev,"unable to map buffer list for queue %d\n" , + queue_idx); + goto out_free_rxq; + } + +for (i =0 ; i <IBMVETH_NUM_BUFF_POOLS ; i++) { +struct ibmveth_buff_pool *src = + &adapter->rx_buff_pool[0 ][i]; +struct ibmveth_buff_pool *dst = + &adapter->rx_buff_pool[queue_idx][i]; + + dst->size = src->size; + dst->index = src->index; + dst->buff_size = src->buff_size; + dst->threshold = src->threshold; + dst->active = src->active; + } + + rc =ibmveth_alloc_queue_buffer_pools (adapter, queue_idx); +if (rc) { +netdev_err (netdev, +"Failed to allocate buffer pools for queue %d\n" , + queue_idx); + goto out_unmap_buflist; + } + + adapter->rx_queue[queue_idx].index =0 ; + adapter->rx_queue[queue_idx].num_slots = rxq_entries; + adapter->rx_queue[queue_idx].toggle =1 ; +spin_lock_init (&adapter->rx_queue[queue_idx].replenish_lock); + +netdev_dbg (netdev, +"Allocated queue %d: buffer_list @ %p (DMA: 0x%llx), rx_queue @ %p (DMA: 0x%llx), %d entries\n" , + queue_idx, adapter->buffer_list_addr[queue_idx], + (unsigned long long )adapter->buffer_list_dma[queue_idx], + adapter->rx_queue[queue_idx].queue_addr, + (unsigned long long )adapter->rx_queue[queue_idx].queue_dma, + rxq_entries); + +return 0 ; + +out_unmap_buflist: +dma_unmap_single (dev, adapter->buffer_list_dma[queue_idx], +4096 ,DMA_BIDIRECTIONAL ); + adapter->buffer_list_dma[queue_idx] =0 ; +out_free_rxq: +dma_free_coherent (dev, adapter->rx_queue[queue_idx].queue_len, + adapter->rx_queue[queue_idx].queue_addr, + adapter->rx_queue[queue_idx].queue_dma); + adapter->rx_queue[queue_idx].queue_addr =NULL ; +out_free_buflist: +free_page ((unsigned long )adapter->buffer_list_addr[queue_idx]); + adapter->buffer_list_addr[queue_idx] =NULL ; +return rc; +} + +/** +* ibmveth_free_single_rx_queue - Free resources for a single RX queue +* @adapter: ibmveth adapter structure +* @queue_idx: Queue index to free +* +* Frees buffer list, RX queue, and per-queue buffer pools for one queue. +* Used during incremental scale-down without affecting remaining queues. +*/ +static void +ibmveth_free_single_rx_queue (struct ibmveth_adapter *adapter,int queue_idx) +{ +struct device *dev = &adapter->vdev->dev; + +ibmveth_free_queue_buffer_pools (adapter, queue_idx); + +if (adapter->buffer_list_dma[queue_idx]) { +dma_unmap_single (dev, adapter->buffer_list_dma[queue_idx], +4096 ,DMA_BIDIRECTIONAL ); + adapter->buffer_list_dma[queue_idx] =0 ; + } + +if (adapter->rx_queue[queue_idx].queue_addr) { +dma_free_coherent (dev, adapter->rx_queue[queue_idx].queue_len, + adapter->rx_queue[queue_idx].queue_addr, + adapter->rx_queue[queue_idx].queue_dma); + adapter->rx_queue[queue_idx].queue_addr =NULL ; + } + +if (adapter->buffer_list_addr[queue_idx]) { +free_page ((unsigned long )adapter->buffer_list_addr[queue_idx]); + adapter->buffer_list_addr[queue_idx] =NULL ; + } + +netdev_dbg (adapter->netdev,"Freed queue %d resources\n" , queue_idx); +} +/** * ibmveth_remove_buffer_from_pool - remove a buffer from a pool * @adapter: adapter instance
@@ -1307,6 +1494,51 @@ static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,return 0 ; } +/** +* ibmveth_drain_rx_queue - Drain pending buffers from an RX queue +* @adapter: ibmveth adapter structure +* @queue_index: Queue index to drain +* +* Recycles all pending buffers back to the per-queue buffer pools. +* Must be called with NAPI disabled for this queue. +* +* Return: Number of buffers drained +*/ +static int +ibmveth_drain_rx_queue (struct ibmveth_adapter *adapter,int queue_index) +{ +struct net_device *netdev = adapter->netdev; +int drained =0 ; +int limit = adapter->rx_queue[queue_index].num_slots; +int rc; + +netdev_dbg (netdev,"Draining RX queue %d (limit: %d slots)\n" , + queue_index, limit); + +while (drained < limit && +ibmveth_rxq_pending_buffer (adapter, queue_index)) { +/* Match poll-side order before harvesting completion state. */ +smp_rmb (); + rc =ibmveth_rxq_harvest_buffer (adapter, queue_index, true); +if (rc) { +netdev_err (netdev, +"Failed to harvest buffer from queue %d during drain: %d\n" , + queue_index, rc); +break ; + } + drained++; + } + +if (drained >0 ) +netdev_dbg (netdev,"Drained %d buffer(s) from RX queue %d\n" , + drained, queue_index); +else +netdev_dbg (netdev,"No buffers to drain from RX queue %d\n" , + queue_index); + +return drained; +} +static void ibmveth_free_tx_ltb (struct ibmveth_adapter *adapter,int idx) {if (!adapter->tx_ltb_ptr[idx])
@@ -1562,6 +1794,227 @@ ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter,return 0 ; } +/** +* ibmveth_deregister_single_rx_queue - Deregister one subordinate RX queue +* @adapter: ibmveth adapter structure +* @queue_idx: Queue index to deregister (1..N) +* +* Deregisters a single queue via H_FREE_LOGICAL_LAN_QUEUE. Linux IRQ handler +* teardown and subordinate virq mapping disposal are owned by interrupt +* cleanup helpers; queue 0 is freed only through ibmveth_free_all_queues() +* (H_FREE_LOGICAL_LAN). +*/ +static void +ibmveth_deregister_single_rx_queue (struct ibmveth_adapter *adapter, +int queue_idx) +{ +unsigned long lpar_rc; +unsigned long ua = adapter->vdev->unit_address; +unsigned long qh = adapter->queue_handle[queue_idx]; + +if (!qh) +return ; + +do { + lpar_rc =h_free_logical_lan_queue (ua, qh); + }while (H_IS_LONG_BUSY (lpar_rc) || (lpar_rc ==H_BUSY )); + + adapter->hcall_stats.free_lan_queue++; + +if (lpar_rc !=H_SUCCESS ) { +netdev_err (adapter->netdev, +"h_free_logical_lan_queue failed for queue %d: rc=0x%lx\n" , + queue_idx, lpar_rc); + } + + adapter->queue_handle[queue_idx] =0 ; + +netdev_dbg (adapter->netdev,"Deregistered queue %d\n" , queue_idx); +} + +/** +* ibmveth_resize_rx_queues_incremental - Resize RX queue count incrementally +* @adapter: ibmveth adapter structure +* @new_count: Target number of RX queues +* @rxq_entries: Number of entries per RX queue +* +* Adds or removes RX queues without tearing down the entire adapter. +* Active queues continue receiving during scale-up; scale-down drains +* excess queues before deregistering them with the hypervisor. +* +* Return: 0 on success, negative error code on failure +*/ +static int +ibmveth_resize_rx_queues_incremental (struct ibmveth_adapter *adapter, +int new_count,int rxq_entries) +{ +struct net_device *netdev = adapter->netdev; +u64 mac_address =ether_addr_to_u64 (netdev->dev_addr); +int old_count = adapter->num_rx_queues; +int failed_queue; +int rc, i; + +if (old_count == new_count) { +netdev_dbg (netdev,"RX queue count unchanged (%d), nothing to do\n" , + old_count); +return 0 ; + } + +if (new_count <1 || new_count >IBMVETH_MAX_RX_QUEUES ) { +netdev_err (netdev,"Invalid RX queue count %d (must be 1-%d)\n" , + new_count,IBMVETH_MAX_RX_QUEUES ); +return -EINVAL ; + } + +netdev_info (netdev,"Incrementally resizing RX queues: %d to %d\n" , + old_count, new_count); + +if (new_count > old_count) { +netdev_dbg (netdev,"Scale-up: adding queues %d-%d\n" , + old_count, new_count -1 ); + +for (i = old_count; i < new_count; i++) { + rc =ibmveth_alloc_single_rx_queue (adapter, i, + rxq_entries); +if (rc) { +netdev_err (netdev,"Failed to allocate queue %d: %d\n" , + i, rc); + goto cleanup_new_queues; + } + + rc =ibmveth_register_single_rx_queue (adapter, i, + mac_address); +if (rc) { +netdev_err (netdev,"Failed to register queue %d: %d\n" , + i, rc); +ibmveth_free_single_rx_queue (adapter, i); + goto cleanup_new_queues; + } + + rc =ibmveth_setup_single_rx_interrupt (adapter, i); +if (rc) { +netdev_err (netdev, +"Failed to setup IRQ for queue %d: %d\n" , + i, rc); +/* request_irq failed: mapped but no handler */ +ibmveth_dispose_subordinate_irq_mapping (adapter, + i); +ibmveth_deregister_single_rx_queue (adapter, i); +ibmveth_free_single_rx_queue (adapter, i); + goto cleanup_new_queues; + } + +/* +* Fully ready before PHYP delivery, matching open(): +* publish -> replenish -> napi_enable -> enable_irq. +* That way ibmveth_interrupt() cannot run on an +* unpublished, empty, or NAPI-disabled queue. +*/ + adapter->num_rx_queues = i +1 ; +ibmveth_replenish_task (adapter, i); +napi_enable (&adapter->napi[i]); + + rc =ibmveth_enable_irq (adapter, i); +if (rc) { +netdev_err (netdev, +"Failed to enable IRQ for queue %d: %d\n" , + i, rc); + adapter->num_rx_queues = i; +napi_disable (&adapter->napi[i]); +ibmveth_cleanup_single_rx_interrupt (adapter, i); +ibmveth_deregister_single_rx_queue (adapter, i); +ibmveth_free_single_rx_queue (adapter, i); + goto cleanup_new_queues; + } + } + + rc =netif_set_real_num_rx_queues (netdev, new_count); +if (rc) { +netdev_err (netdev,"Failed to set real RX queues to %d: %d\n" , + new_count, rc); + goto cleanup_new_queues; + } + }else { +netdev_dbg (netdev,"Scale-down: removing queues %d-%d\n" , + new_count, old_count -1 ); + +/* +* Mask PHYP delivery before napi_disable/drain. Otherwise +* ibmveth_interrupt returns IRQ_HANDLED without masking when +* NAPI is disabled, and the HV can storm during drain. +*/ +for (i = new_count; i < old_count; i++) { +ibmveth_disable_irq (adapter, i); +synchronize_irq (adapter->queue_irq[i]); + } + +for (i = new_count; i < old_count; i++) +napi_disable (&adapter->napi[i]); + +for (i = new_count; i < old_count; i++) +ibmveth_drain_rx_queue (adapter, i); + +synchronize_net (); + + rc =netif_set_real_num_rx_queues (netdev, new_count); +if (rc) { +netdev_err (netdev,"Failed to set real RX queues to %d: %d\n" , + new_count, rc); +for (i = new_count; i < old_count; i++) { +ibmveth_replenish_task (adapter, i); +napi_enable (&adapter->napi[i]); +ibmveth_enable_irq (adapter, i); + } +return rc; + } + + adapter->num_rx_queues = new_count; + +for (i = new_count; i < old_count; i++) { +ibmveth_cleanup_single_rx_interrupt (adapter, i); +ibmveth_deregister_single_rx_queue (adapter, i); +ibmveth_free_single_rx_queue (adapter, i); + } + } + +netdev_info (netdev,"Successfully resized to %d RX queues (incremental)\n" , + adapter->num_rx_queues); + +if (firmware_has_feature (FW_FEATURE_CMO )) +vio_cmo_set_dev_desired (adapter->vdev, +ibmveth_get_desired_dma (adapter->vdev)); + +return 0 ; + +cleanup_new_queues: + failed_queue = i; +netdev_err (netdev, +"Scale-up failed at queue %d, cleaning up queues %d-%d\n" , + failed_queue, old_count, failed_queue -1 ); +for (i = old_count; i < failed_queue; i++) { +ibmveth_disable_irq (adapter, i); +synchronize_irq (adapter->queue_irq[i]); + } + +for (i = old_count; i < failed_queue; i++) +napi_disable (&adapter->napi[i]); + +for (i = old_count; i < failed_queue; i++) +ibmveth_drain_rx_queue (adapter, i); + +synchronize_net (); + +for (i = old_count; i < failed_queue; i++) { +ibmveth_cleanup_single_rx_interrupt (adapter, i); +ibmveth_deregister_single_rx_queue (adapter, i); +ibmveth_free_single_rx_queue (adapter, i); + } + adapter->num_rx_queues = old_count; +netdev_warn (netdev,"Keeping %d queues after scale-up failure\n" , + old_count); +return rc; +} +/** * ibmveth_free_all_queues - Free all RX queues at once * @adapter: ibmveth adapter structure
@@ -2217,12 +2670,62 @@ static void ibmveth_get_channels(struct net_device *netdev, channels->rx_count = adapter->num_rx_queues; } +/** +* ibmveth_resize_rx_channels - Validate and apply a new RX queue count +* @adapter: ibmveth adapter structure +* @goal_rx: desired RX queue count +* +* When the interface is up, resize live queues via +* ibmveth_resize_rx_queues_incremental(). When down, only stash +* adapter->num_rx_queues for the next open(). +* +* Return: 0 on success, negative error code on failure +*/ +static int ibmveth_resize_rx_channels (struct ibmveth_adapter *adapter, +unsigned int goal_rx) +{ +struct net_device *netdev = adapter->netdev; +unsigned int old_rx = adapter->num_rx_queues; +int rxq_entries; +int rc; + +if (goal_rx >1 && !adapter->multi_queue) { +netdev_err (netdev, +"Cannot resize to %u RX queues: multi-queue mode not supported by firmware\n" , + goal_rx); +return -EOPNOTSUPP ; + } + +if (goal_rx <1 || goal_rx >IBMVETH_MAX_RX_QUEUES ) { +netdev_err (netdev, +"Invalid RX queue count %u (must be 1-%d)\n" , + goal_rx,IBMVETH_MAX_RX_QUEUES ); +return -EINVAL ; + } + +if (goal_rx == old_rx) +return 0 ; + +if (!(netdev->flags &IFF_UP )) { + adapter->num_rx_queues = goal_rx; +return 0 ; + } + + rxq_entries = adapter->rx_queue[0 ].num_slots; + rc =ibmveth_resize_rx_queues_incremental (adapter, goal_rx, + rxq_entries); +if (rc) +netdev_err (netdev,"Failed to resize RX queues: %d\n" , rc); +return rc; +} +static int ibmveth_set_channels (struct net_device *netdev,struct ethtool_channels *channels) {struct ibmveth_adapter *adapter =netdev_priv (netdev);unsigned int old = netdev->real_num_tx_queues, goal = channels->tx_count; +unsigned int goal_rx = channels->rx_count;int rc, i;/* If ndo_open has not been called yet then don't allocate, just set
@@ -2231,6 +2734,13 @@ static int ibmveth_set_channels(struct net_device *netdev,if (!(netdev->flags &IFF_UP ))return netif_set_real_num_tx_queues (netdev, goal); +/* Resize RX first while UP so ibmveth_resize_rx_channels() is used +* in this patch. !IFF_UP RX stash ordering lands next. +*/ + rc =ibmveth_resize_rx_channels (adapter, goal_rx); +if (rc) +return rc; +/* We have IBMVETH_MAX_QUEUES netdev_queue's allocated * but we may need to alloc/free the ltb's. */
@@ -2245,7 +2755,6 @@ static int ibmveth_set_channels(struct net_device *netdev,if (!rc)continue ; -/* if something goes wrong, free everything we just allocated */ netdev_err (netdev,"Failed to allocate more tx queues, returning to %d queues\n" , old); goal = old;
@@ -2259,7 +2768,6 @@ static int ibmveth_set_channels(struct net_device *netdev, goal = old; old = i; } -/* Free any that are no longer needed */ for (i = old; i > goal; i--) {if (adapter->tx_ltb_ptr[i -1 ])ibmveth_free_tx_ltb (adapter, i -1 );
Signature · 1 line
2.50.1 (Apple Git-155)
Jakub Kicinski
maintainerRe: [PATCH net-next v4 12/14] ibmveth: Implement incremental MQ RX queue resize
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
ibmveth: Implement incremental MQ RX queue resize
Adds per-queue alloc/register/irq-setup/free helpers plus ibmveth_resize_rx_queues_incremental() so the live RX queue count can grow or shrink without tearing down surviving queues, and wires ibmveth_resize_rx_channels() into set_channels() when the interface is up.
A few questions on the new resize paths below.
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0
[ ... ]
Quoted 26 lines
drivers/net/ethernet/ibm/ibmveth.c +22
-0@@ -1307,6 +1494,51 @@ static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,return 0 ; } +/** +* ibmveth_drain_rx_queue - Drain pending buffers from an RX queue +* @adapter: ibmveth adapter structure +* @queue_index: Queue index to drain +* +* Recycles all pending buffers back to the per-queue buffer pools. +* Must be called with NAPI disabled for this queue. +* +* Return: Number of buffers drained +*/ +static int +ibmveth_drain_rx_queue (struct ibmveth_adapter *adapter,int queue_index) +{ +struct net_device *netdev = adapter->netdev; +int drained =0 ; +int limit = adapter->rx_queue[queue_index].num_slots; +int rc; + +netdev_dbg (netdev,"Draining RX queue %d (limit: %d slots)\n" , + queue_index, limit); + + while (drained < limit &&
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter, · line 1528
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+ ibmveth_rxq_pending_buffer(adapter, queue_index)) { +/* Match poll-side order before harvesting completion state. */ +smp_rmb (); + rc =ibmveth_rxq_harvest_buffer (adapter, queue_index, true); +if (rc) { +netdev_err (netdev, +"Failed to harvest buffer from queue %d during drain: %d\n" , + queue_index, rc); +break ; + }
[Severity: Low]
Can this error arm ever run? ibmveth_rxq_harvest_buffer() normalizes every failure to a 0 return:
rc = ibmveth_remove_buffer_from_pool(adapter, cor, queue_index, reuse);
if (unlikely(rc)) {
if (rc == -EINVAL || rc == -EFAULT)
goto advance;
return rc;
}advance:
ibmveth_rxq_advance(rxq); return 0;
and ibmveth_remove_buffer_from_pool() only ever returns 0, -EINVAL or -EFAULT, so both non-zero values take the goto advance path. The netdev_err()/break looks like dead code.
Quoted 13 lines
drivers/net/ethernet/ibm/ibmveth.c +13
-0+ drained++; + } + +if (drained >0 ) +netdev_dbg (netdev,"Drained %d buffer(s) from RX queue %d\n" , + drained, queue_index); +else +netdev_dbg (netdev,"No buffers to drain from RX queue %d\n" , + queue_index); + +return drained; +} +
[ ... ]
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0@@ -1562,6 +1794,227 @@ ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter,return 0 ; }
[ ... ]
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-0+static int +ibmveth_resize_rx_queues_incremental (struct ibmveth_adapter *adapter, +int new_count,int rxq_entries) +{
[ ... ]
Quoted 3 lines
drivers/net/ethernet/ibm/ibmveth.c +3
-0+for (i = old_count; i < new_count; i++) { + rc =ibmveth_alloc_single_rx_queue (adapter, i, + rxq_entries);
[ ... ]
drivers/net/ethernet/ibm/ibmveth.c ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter, · line 1913
Quoted 7 lines
drivers/net/ethernet/ibm/ibmveth.c +7
-0+/* +* Fully ready before PHYP delivery, matching open(): +* publish -> replenish -> napi_enable -> enable_irq. +* That way ibmveth_interrupt() cannot run on an +* unpublished, empty, or NAPI-disabled queue. +*/ + adapter->num_rx_queues = i +1 ;
[Severity: Medium]
Should this publish store be ordered? adapter->num_rx_queues is the only gate that makes rx_queue[i].queue_addr/num_slots/toggle/replenish_lock, rx_buff_pool[i][*] and buffer_list_addr[i] visible to lockless readers, and it is written with a plain store here while the queues added in earlier iterations are already running NAPI with PHYP unmasked.
The consumers also use plain loads from hardirq and softirq context:
ibmveth_interrupt():
if (WARN_ON(qindex < 0 || qindex >= adapter->num_rx_queues)) return IRQ_NONE;
ibmveth_poll():
if (WARN_ON(queue_index < 0 || queue_index >= adapter->num_rx_queues)) return 0;
and ibmveth_update_rx_no_buffer() dereferences buffer_list_addr[i] for every i below the count. On powerpc, can a reader see the incremented count while still observing stale contents of that slot (for example a buffer_list_addr[i] left NULL by an earlier scale-down), or the converse stale count for the queue whose interrupt was just unmasked? Would WRITE_ONCE()/READ_ONCE() plus a release barrier before the publish be appropriate here?
drivers/net/ethernet/ibm/ibmveth.c ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter, · line 1917
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-0+ibmveth_replenish_task (adapter, i); +napi_enable (&adapter->napi[i]); + + rc =ibmveth_enable_irq (adapter, i);
[Severity: Medium]
After the unmask, should the queue be re-checked for descriptors that PHYP enqueued while delivery was masked? Buffers are posted by ibmveth_replenish_task() before ibmveth_enable_irq(), so PHYP can write descriptors during the masked window. ibmveth_poll() does exactly this compensation:
lpar_rc = ibmveth_enable_irq(adapter, queue_index);
...
if (ibmveth_rxq_pending_buffer(adapter, queue_index) &&
napi_schedule(napi)) {
ibmveth_disable_irq(adapter, queue_index);
goto restart_poll;
}Without an equivalent pending check plus napi_schedule() here (and in the scale-down rollback loop further down, which is even more exposed since PHYP may have enqueued during the disable/drain window), can the new queue sit with unharvested descriptors until unrelated traffic raises the next interrupt?
drivers/net/ethernet/ibm/ibmveth.c ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter, · line 1928
Quoted 11 lines
drivers/net/ethernet/ibm/ibmveth.c +11
-0+if (rc) { +netdev_err (netdev, +"Failed to enable IRQ for queue %d: %d\n" , + i, rc); + adapter->num_rx_queues = i; +napi_disable (&adapter->napi[i]); +ibmveth_cleanup_single_rx_interrupt (adapter, i); +ibmveth_deregister_single_rx_queue (adapter, i); +ibmveth_free_single_rx_queue (adapter, i); + goto cleanup_new_queues; + }
[Severity: Medium]
Is rc an errno at this point? ibmveth_enable_irq() -> ibmveth_toggle_irq() returns the raw plpar_hcall_norets() result and only normalizes H_PARAMETER to 0, so rc can be H_HARDWARE (-1), H_FUNCTION (-2), H_PRIVILEGE (-3) or the positive H_BUSY (1). That value is returned unchanged through ibmveth_resize_rx_channels() -> ibmveth_set_channels() into the ethtool ioctl, so userspace sees EPERM/ENOENT/ESRCH, or for a positive code a positive ioctl return that ethtool reads as success even though the resize failed and unwound.
The pre-existing ibmveth_setup_rx_interrupts() converts the same failure with rc = -EIO; should this path do the same?
Quoted 14 lines
drivers/net/ethernet/ibm/ibmveth.c +14
-0+ } + + rc =netif_set_real_num_rx_queues (netdev, new_count); +if (rc) { +netdev_err (netdev,"Failed to set real RX queues to %d: %d\n" , + new_count, rc); + goto cleanup_new_queues; + } + }else { +netdev_dbg (netdev,"Scale-down: removing queues %d-%d\n" , + new_count, old_count -1 ); + +/* +* Mask PHYP delivery before napi_disable/drain. Otherwise
drivers/net/ethernet/ibm/ibmveth.c ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter, · line 1952
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+* ibmveth_interrupt returns IRQ_HANDLED without masking when +* NAPI is disabled, and the HV can storm during drain. +*/ +for (i = new_count; i < old_count; i++) { +ibmveth_disable_irq (adapter, i); +synchronize_irq (adapter->queue_irq[i]); + } + +for (i = new_count; i < old_count; i++) +napi_disable (&adapter->napi[i]);
[Severity: Medium]
Does the mask actually hold across the drain? synchronize_irq() waits for the hardirq handler, not for an ibmveth_poll() already in flight on another CPU, and that poll re-arms delivery unconditionally:
CPU0 (resize)
ibmveth_disable_irq(adapter, i); synchronize_irq(adapter->queue_irq[i]); /* handler idle, returns */ napi_disable(&adapter->napi[i]); /* returns after napi_complete_done() */
CPU1 (ibmveth_poll for queue i, scheduled before the mask)
if (!napi_complete_done(napi, frames_processed))
goto out;
lpar_rc = ibmveth_enable_irq(adapter, queue_index); /* unmasked */
so the drain can run with PHYP delivery re-enabled. With NAPI disabled, ibmveth_schedule_rx_queue() fails napi_schedule_prep() and returns without masking, and once adapter->num_rx_queues is lowered each further delivery hits WARN_ON(qindex >= adapter->num_rx_queues) in ibmveth_interrupt() and returns IRQ_NONE. Is the storm rule stated in the comment still guaranteed? The same ordering is repeated in the cleanup_new_queues label below.
Quoted 7 lines
drivers/net/ethernet/ibm/ibmveth.c +7
-0+ +for (i = new_count; i < old_count; i++) +ibmveth_drain_rx_queue (adapter, i); + +synchronize_net (); + + rc =netif_set_real_num_rx_queues (netdev, new_count);
drivers/net/ethernet/ibm/ibmveth.c ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter, · line 1969
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+if (rc) { +netdev_err (netdev,"Failed to set real RX queues to %d: %d\n" , + new_count, rc); +for (i = new_count; i < old_count; i++) { +ibmveth_replenish_task (adapter, i); +napi_enable (&adapter->napi[i]); +ibmveth_enable_irq (adapter, i); + } +return rc; + }
[Severity: Medium]
Should the ibmveth_enable_irq() return code be checked in this rollback loop? This is the only new call site that discards it.
If the H_VIOCTL enable fails for one of these queues, the queue stays in the live set (adapter->num_rx_queues is still old_count) with NAPI enabled and buffers posted, but with PHYP delivery masked, so that RX queue is silently stalled until a close/open or another resize.
Elsewhere the driver treats this as actionable, e.g. ibmveth_poll():
lpar_rc = ibmveth_enable_irq(adapter, queue_index);
if (lpar_rc != H_SUCCESS) {
...
schedule_work(&adapter->work);
}drivers/net/ethernet/ibm/ibmveth.c ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter, · line 1971
Quoted 2 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-0+ + adapter->num_rx_queues = new_count;
[Severity: Medium]
Do the interface statistics go backwards here? ibmveth_get_stats64(), ibmveth_get_ethtool_stats(), ibmveth_sum_rx_invalid_buffers() and ibmveth_sum_rx_large_packets() all sum rx_qstats[0..num_rx_queues-1]:
if (adapter->rx_qstats) {
for (i = 0; i < adapter->num_rx_queues; i++) {
stats->rx_packets += adapter->rx_qstats[i].packets;so lowering the count drops the packets/bytes already accumulated by the removed queues from the reported totals.
In the other direction, ibmveth_alloc_single_rx_queue() resets rx_queue[queue_idx].index/num_slots/toggle and copies the pool geometry but never clears adapter->rx_qstats[queue_idx], so re-adding a previously removed index resurrects its old counters and the totals jump up.
Should the retired queues' counts be folded into an accumulator and the slot cleared on (re-)allocation, so rtnl_link_stats64 stays monotonic?
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-0+ +for (i = new_count; i < old_count; i++) { +ibmveth_cleanup_single_rx_interrupt (adapter, i); +ibmveth_deregister_single_rx_queue (adapter, i);
drivers/net/ethernet/ibm/ibmveth.c ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter, · line 1985
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+ibmveth_free_single_rx_queue (adapter, i); + } + } + +netdev_info (netdev,"Successfully resized to %d RX queues (incremental)\n" , + adapter->num_rx_queues); + +if (firmware_has_feature (FW_FEATURE_CMO )) +vio_cmo_set_dev_desired (adapter->vdev, +ibmveth_get_desired_dma (adapter->vdev));
[Severity: Medium]
Is the entitlement raised too late on CMO LPARs? All the CMO-charged DMA work for the new queues has already happened by the time this runs:dma_map_single() of the buffer-list page and dma_alloc_coherent() of the descriptor ring in ibmveth_alloc_single_rx_queue(), plus every RX skb mapping done by ibmveth_replenish_task(). Those run while the device's desired entitlement still reflects the old, smaller queue count, and vio_dma_iommu_alloc_coherent() refuses the allocation when vio_cmo_alloc() cannot cover it.
It is also skipped on every failure and rollback return, so a resize that fails leaves the entitlement untouched.
The driver's own convention is the opposite order, in ibmveth_change_mtu():
vio_cmo_set_dev_desired(viodev, ibmveth_get_desired_dma(viodev)); ... ibmveth_open(...)
Note that simply moving the call is not enough, since ibmveth_get_desired_dma() itself loops over adapter->num_rx_queues, so the target count has to be accounted before the allocations and rolled back on failure.
drivers/net/ethernet/ibm/ibmveth.c ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter, · line 1993
Quoted 8 lines
drivers/net/ethernet/ibm/ibmveth.c +8
-0+ +return 0 ; + +cleanup_new_queues: + failed_queue = i; +netdev_err (netdev, +"Scale-up failed at queue %d, cleaning up queues %d-%d\n" , + failed_queue, old_count, failed_queue -1 );
[Severity: Low]
When the first newly added queue is the one that fails, failed_queue == old_count, so this prints an inverted empty range such as "cleaning up queues 4-3" while the three loops below correctly iterate zero times.
Quoted 9 lines
drivers/net/ethernet/ibm/ibmveth.c +9
-0+for (i = old_count; i < failed_queue; i++) { +ibmveth_disable_irq (adapter, i); +synchronize_irq (adapter->queue_irq[i]); + } + +for (i = old_count; i < failed_queue; i++) +napi_disable (&adapter->napi[i]); + +for (i = old_count; i < failed_queue; i++)
drivers/net/ethernet/ibm/ibmveth.c ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter, · line 2012
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+ibmveth_drain_rx_queue (adapter, i); + +synchronize_net (); + +for (i = old_count; i < failed_queue; i++) { +ibmveth_cleanup_single_rx_interrupt (adapter, i); +ibmveth_deregister_single_rx_queue (adapter, i); +ibmveth_free_single_rx_queue (adapter, i); + } + adapter->num_rx_queues = old_count;
[Severity: High]
Can a concurrent NAPI poll dereference the pages this loop just freed? adapter->num_rx_queues is restored only after the free loop, so for the duration of that loop the indices [old_count, failed_queue) are freed but still inside the published live range.
ibmveth_free_single_rx_queue() unmaps buffer_list_dma[i], frees the coherent ring and then does:
free_page((unsigned long)adapter->buffer_list_addr[queue_idx]); adapter->buffer_list_addr[queue_idx] = NULL;
Meanwhile every surviving queue's poll reaches ibmveth_replenish_task() -> ibmveth_update_rx_no_buffer(), which walks the published range with no NULL or liveness check:
for (i = 0; i < adapter->num_rx_queues; i++) {
__be64 *p = adapter->buffer_list_addr[i] + 4096 - 8;
u64 drops = be64_to_cpup(p);So for a 2->4 scale-up where queue 2 came up and queue 3 fails, a poll on queue 0 or 1 reads buffer_list_addr[2] either as a just-freed page or as NULL + 0xff8, in softirq context. ibmveth_poll_controller() reaches the same range from netpoll without rtnl, and additionally posts buffers into pools that are being freed.
The guard in ibmveth_replenish_task():
if (queue_index >= adapter->num_rx_queues) {does not help while the count is still the higher value. The inline enable_irq failure handler above and the successful scale-down path both lower adapter->num_rx_queues before freeing; should this label do the same, i.e. set adapter->num_rx_queues = old_count before the free loop?
Quoted 5 lines
drivers/net/ethernet/ibm/ibmveth.c +5
-0+netdev_warn (netdev,"Keeping %d queues after scale-up failure\n" , + old_count); +return rc; +} +
[ ... ]
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0@@ -2217,12 +2670,62 @@ static void ibmveth_get_channels(struct net_device *netdev, channels->rx_count = adapter->num_rx_queues; }
[ ... ]
Quoted 3 lines
drivers/net/ethernet/ibm/ibmveth.c +3
-0+static int ibmveth_resize_rx_channels (struct ibmveth_adapter *adapter, +unsigned int goal_rx) +{
[ ... ]
drivers/net/ethernet/ibm/ibmveth.c static void ibmveth_get_channels(struct net_device *netdev, · line 2713
Quoted 8 lines
drivers/net/ethernet/ibm/ibmveth.c +8
-0+if (goal_rx == old_rx) +return 0 ; + +if (!(netdev->flags &IFF_UP )) { + adapter->num_rx_queues = goal_rx; +return 0 ; + } +
[Severity: Low]
This isn't a bug in the final series, but at this commit the stash branch is unreachable: the only caller, ibmveth_set_channels(), already returns early on the same !IFF_UP condition before reaching ibmveth_resize_rx_channels(), so ethtool -L <dev> rx N on a down interface returns 0 while the requested count is discarded. The next patch in the series ("ibmveth: Wire ethtool set_channels to MQ RX queue resize") replaces that early return and makes this branch live, so this only shows up at this bisect point.
Quoted 19 lines
drivers/net/ethernet/ibm/ibmveth.c +9
-0+ rxq_entries = adapter->rx_queue[0 ].num_slots; + rc =ibmveth_resize_rx_queues_incremental (adapter, goal_rx, + rxq_entries); +if (rc) +netdev_err (netdev,"Failed to resize RX queues: %d\n" , rc); +return rc; +} +static int ibmveth_set_channels (struct net_device *netdev,struct ethtool_channels *channels) {struct ibmveth_adapter *adapter =netdev_priv (netdev);unsigned int old = netdev->real_num_tx_queues, goal = channels->tx_count; +unsigned int goal_rx = channels->rx_count;int rc, i;/* If ndo_open has not been called yet then don't allocate, just set
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_set_channels(struct net_device *netdev, · line 2743
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +7
-0@@ -2231,6 +2734,13 @@ static int ibmveth_set_channels(struct net_device *netdev,if (!(netdev->flags &IFF_UP ))return netif_set_real_num_tx_queues (netdev, goal); +/* Resize RX first while UP so ibmveth_resize_rx_channels() is used +* in this patch. !IFF_UP RX stash ordering lands next. +*/ + rc =ibmveth_resize_rx_channels (adapter, goal_rx); +if (rc) +return rc; +
[Severity: Medium]
Should the RX side be unwound when the TX phase below fails? The RX resize is committed here first, and then ibmveth_allocate_tx_ltb() can return -ENOMEM or netif_set_real_num_tx_queues() can fail, in which case the function ends with return rc and no compensating ibmveth_resize_rx_channels(adapter, old_rx).
ETHTOOL_SCHANNELS then reports failure while half of the requested configuration is in effect: on scale-down the excess RX queues have already been deregistered from PHYP and freed, on scale-up new queues are registered with IRQs installed and DMA memory pinned, and ethtool -l afterwards reports the new rx_count.
Quoted 3 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0/* We have IBMVETH_MAX_QUEUES netdev_queue's allocated * but we may need to alloc/free the ltb's. */
[ ... ]
Re: [PATCH net-next v4 12/14] ibmveth: Implement incremental MQ RX queue resize
Quoted 11 lines · Jakub Kicinski
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
ibmveth: Implement incremental MQ RX queue resizeAdds per-queue alloc/register/irq-setup/free helpers plus
ibmveth_resize_rx_queues_incremental()so the live RX queue count can grow or shrink without tearing down surviving queues, and wiresibmveth_resize_rx_channels()intoset_channels()when the interface is up.A few questions on the new resize paths below.
Hi Jakub,
Thanks for the review.
Quoted 53 lines
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0[ ... ]
Quoted 26 lines
drivers/net/ethernet/ibm/ibmveth.c +22
-0@@ -1307,6 +1494,51 @@ static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,return 0 ; } +/** +* ibmveth_drain_rx_queue - Drain pending buffers from an RX queue +* @adapter: ibmveth adapter structure +* @queue_index: Queue index to drain +* +* Recycles all pending buffers back to the per-queue buffer pools. +* Must be called with NAPI disabled for this queue. +* +* Return: Number of buffers drained +*/ +static int +ibmveth_drain_rx_queue (struct ibmveth_adapter *adapter,int queue_index) +{ +struct net_device *netdev = adapter->netdev; +int drained =0 ; +int limit = adapter->rx_queue[queue_index].num_slots; +int rc; + +netdev_dbg (netdev,"Draining RX queue %d (limit: %d slots)\n" , + queue_index, limit); + + while (drained < limit &&drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter, · line 1528
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+ ibmveth_rxq_pending_buffer(adapter, queue_index)) { +/* Match poll-side order before harvesting completion state. */ +smp_rmb (); + rc =ibmveth_rxq_harvest_buffer (adapter, queue_index, true); +if (rc) { +netdev_err (netdev, +"Failed to harvest buffer from queue %d during drain: %d\n" , + queue_index, rc); +break ; + }[Severity: Low]
Can this error arm ever run?ibmveth_rxq_harvest_buffer()normalizes every failure to a 0 return:rc = ibmveth_remove_buffer_from_pool(adapter, cor, queue_index, reuse); if (unlikely(rc)) { if (rc == -EINVAL || rc == -EFAULT) goto advance; return rc; }
Quoted 8 lines
advance:
ibmveth_rxq_advance(rxq); return 0;and
ibmveth_remove_buffer_from_pool()only ever returns 0,-EINVALor-EFAULT, so both non-zero values take the goto advance path. The netdev_err()/break looks like dead code.
Yes, that error arm is effectively dead on the mailed patch.
I’ll make harvest return -EINVAL / -EFAULT honestly and keep drain tolerant of those slot-advanced cases.
Quoted 56 lines
Quoted 13 lines
drivers/net/ethernet/ibm/ibmveth.c +13
-0+ drained++; + } + +if (drained >0 ) +netdev_dbg (netdev,"Drained %d buffer(s) from RX queue %d\n" , + drained, queue_index); +else +netdev_dbg (netdev,"No buffers to drain from RX queue %d\n" , + queue_index); + +return drained; +} +[ ... ]
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0@@ -1562,6 +1794,227 @@ ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter,return 0 ; }[ ... ]
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-0+static int +ibmveth_resize_rx_queues_incremental (struct ibmveth_adapter *adapter, +int new_count,int rxq_entries) +{[ ... ]
Quoted 3 lines
drivers/net/ethernet/ibm/ibmveth.c +3
-0+for (i = old_count; i < new_count; i++) { + rc =ibmveth_alloc_single_rx_queue (adapter, i, + rxq_entries);[ ... ]
drivers/net/ethernet/ibm/ibmveth.c ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter, · line 1913
Quoted 7 lines
drivers/net/ethernet/ibm/ibmveth.c +7
-0+/* +* Fully ready before PHYP delivery, matching open(): +* publish -> replenish -> napi_enable -> enable_irq. +* That way ibmveth_interrupt() cannot run on an +* unpublished, empty, or NAPI-disabled queue. +*/ + adapter->num_rx_queues = i +1 ;[Severity: Medium]
Should this publish store be ordered?adapter->num_rx_queuesis the only gate that makes rx_queue[i].queue_addr/num_slots/toggle/replenish_lock, rx_buff_pool[i][*] and buffer_list_addr[i] visible to lockless readers, and it is written with a plain store here while the queues added in earlier iterations are already running NAPI with PHYP unmasked.The consumers also use plain loads from hardirq and softirq context:
ibmveth_interrupt():if (WARN_ON(qindex < 0 || qindex >= adapter->num_rx_queues)) return IRQ_NONE;
ibmveth_poll():if (WARN_ON(queue_index < 0 || queue_index >= adapter->num_rx_queues)) return 0;
Quoted 7 lines
and
ibmveth_update_rx_no_buffer()dereferences buffer_list_addr[i] for every i below the count. On powerpc, can a reader see the incremented count while still observing stale contents of that slot (for example a buffer_list_addr[i] left NULL by an earlier scale-down), or the converse stale count for the queue whose interrupt was just unmasked? Would WRITE_ONCE()/READ_ONCE() plus a release barrier before the publish be appropriate here?
Yes, the publication model needs to be explicit here
num_rx_queues is the visibility boundary for lockless readers, so I'm planning to publish it with ordering only after the per-queue state is ready, and use matching read-side accessors on the IRQ, poll, and other lockless reader paths.
Quoted 19 lines
drivers/net/ethernet/ibm/ibmveth.c ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter, · line 1917
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-0+ibmveth_replenish_task (adapter, i); +napi_enable (&adapter->napi[i]); + + rc =ibmveth_enable_irq (adapter, i);[Severity: Medium]
After the unmask, should the queue be re-checked for descriptors that PHYP enqueued while delivery was masked? Buffers are posted byibmveth_replenish_task()beforeibmveth_enable_irq(), so PHYP can write descriptors during the masked window.ibmveth_poll()does exactly this compensation:lpar_rc = ibmveth_enable_irq(adapter, queue_index); ... if (ibmveth_rxq_pending_buffer(adapter, queue_index) && napi_schedule(napi)) { ibmveth_disable_irq(adapter, queue_index); goto restart_poll; }
Quoted 5 lines
Without an equivalent pending check plus
napi_schedule()here (and in the scale-down rollback loop further down, which is even more exposed since PHYP may have enqueued during the disable/drain window), can the new queue sit with unharvested descriptors until unrelated traffic raises the next interrupt?
Yes, after unmask the queue needs the same pending-work kick.
The right rationale is that PHYP may already have posted descriptors while delivery was masked. I’ll kick the queue after scale-up enable_irq and after scale-down rollback unmask so pending work is serviced promptly.
Quoted 21 lines
drivers/net/ethernet/ibm/ibmveth.c ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter, · line 1928
Quoted 11 lines
drivers/net/ethernet/ibm/ibmveth.c +11
-0+if (rc) { +netdev_err (netdev, +"Failed to enable IRQ for queue %d: %d\n" , + i, rc); + adapter->num_rx_queues = i; +napi_disable (&adapter->napi[i]); +ibmveth_cleanup_single_rx_interrupt (adapter, i); +ibmveth_deregister_single_rx_queue (adapter, i); +ibmveth_free_single_rx_queue (adapter, i); + goto cleanup_new_queues; + }[Severity: Medium]
Is rc an errno at this point?ibmveth_enable_irq()->ibmveth_toggle_irq()returns the rawplpar_hcall_norets()result and only normalizesH_PARAMETERto 0, so rc can beH_HARDWARE(-1),H_FUNCTION(-2),H_PRIVILEGE(-3) or the positiveH_BUSY(1). That value is returned unchanged throughibmveth_resize_rx_channels()->ibmveth_set_channels()into the ethtool ioctl, so userspace sees EPERM/ENOENT/ESRCH, or for a positive code a positive ioctl return that ethtool reads as success even though the resize failed and unwound.
Quoted 2 lines
The pre-existing
ibmveth_setup_rx_interrupts()converts the same failure with rc =-EIO; should this path do the same?
Yes, that should return a Linux errno, not raw H_* status.
I'm planning to keep the helper contract normalized to errno and make the resize path return -EIO on enable_irq failure rather than leaking hypervisor status upward.
Quoted 41 lines
Quoted 14 lines
drivers/net/ethernet/ibm/ibmveth.c +14
-0+ } + + rc =netif_set_real_num_rx_queues (netdev, new_count); +if (rc) { +netdev_err (netdev,"Failed to set real RX queues to %d: %d\n" , + new_count, rc); + goto cleanup_new_queues; + } + }else { +netdev_dbg (netdev,"Scale-down: removing queues %d-%d\n" , + new_count, old_count -1 ); + +/* +* Mask PHYP delivery before napi_disable/drain. Otherwise drivers/net/ethernet/ibm/ibmveth.c ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter, · line 1952
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+* ibmveth_interrupt returns IRQ_HANDLED without masking when +* NAPI is disabled, and the HV can storm during drain. +*/ +for (i = new_count; i < old_count; i++) { +ibmveth_disable_irq (adapter, i); +synchronize_irq (adapter->queue_irq[i]); + } + +for (i = new_count; i < old_count; i++) +napi_disable (&adapter->napi[i]);[Severity: Medium]
Does the mask actually hold across the drain?synchronize_irq()waits for the hardirq handler, not for anibmveth_poll()already in flight on another CPU, and that poll re-arms delivery unconditionally:CPU0 (resize)
ibmveth_disable_irq(adapter, i); synchronize_irq(adapter->queue_irq[i]); /* handler idle, returns */ napi_disable(&adapter->napi[i]); /* returns after napi_complete_done() */CPU1 (
ibmveth_pollfor queue i, scheduled before the mask)
if (!napi_complete_done(napi,frames_processed))
goto out;
lpar_rc=ibmveth_enable_irq(adapter, queue_index); /* unmasked */
Quoted 7 lines
so the drain can run with PHYP delivery re-enabled. With NAPI disabled,
ibmveth_schedule_rx_queue()failsnapi_schedule_prep()and returns without masking, and onceadapter->num_rx_queuesis lowered each further delivery hitsWARN_ON(qindex >= adapter->num_rx_queues)inibmveth_interrupt()and returnsIRQ_NONE. Is the storm rule stated in the comment still guaranteed? The same ordering is repeated in thecleanup_new_queueslabel below.
Yes, the drain and rollback comments were too strong as written.
Teardown-first is still the primary fix, but poll also needs to stop re-enabling delivery while the queue is being stopped. I'm planning to tighten that stopping behavior so the drain window cannot remask PHYP during teardown.
Quoted 26 lines
Quoted 7 lines
drivers/net/ethernet/ibm/ibmveth.c +7
-0+ +for (i = new_count; i < old_count; i++) +ibmveth_drain_rx_queue (adapter, i); + +synchronize_net (); + + rc =netif_set_real_num_rx_queues (netdev, new_count);drivers/net/ethernet/ibm/ibmveth.c ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter, · line 1969
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+if (rc) { +netdev_err (netdev,"Failed to set real RX queues to %d: %d\n" , + new_count, rc); +for (i = new_count; i < old_count; i++) { +ibmveth_replenish_task (adapter, i); +napi_enable (&adapter->napi[i]); +ibmveth_enable_irq (adapter, i); + } +return rc; + }[Severity: Medium]
Should theibmveth_enable_irq()return code be checked in this rollback loop? This is the only new call site that discards it.If the
H_VIOCTLenable fails for one of these queues, the queue stays in the live set (adapter->num_rx_queuesis stillold_count) with NAPI enabled and buffers posted, but with PHYP delivery masked, so that RX queue is silently stalled until a close/open or another resize.
Quoted 7 lines
Elsewhere the driver treats this as actionable, e.g.
ibmveth_poll():lpar_rc = ibmveth_enable_irq(adapter, queue_index); if (lpar_rc != H_SUCCESS) { ... schedule_work(&adapter->work); }
Yes, rollback should not discard that return code.
I’ll check it there, log it, schedule reset work on failure, and still kick pending work on success so rollback does not leave a queue silently stalled.
Quoted 14 lines
drivers/net/ethernet/ibm/ibmveth.c ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter, · line 1971
Quoted 2 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-0+ + adapter->num_rx_queues = new_count;[Severity: Medium]
Do the interface statistics go backwards here?ibmveth_get_stats64(),ibmveth_get_ethtool_stats(),ibmveth_sum_rx_invalid_buffers()andibmveth_sum_rx_large_packets()all sum rx_qstats[0..num_rx_queues-1]:if (adapter->rx_qstats) { for (i = 0; i < adapter->num_rx_queues; i++) { stats->rx_packets += adapter->rx_qstats[i].packets;so lowering the count drops the packets/bytes already accumulated by the removed queues from the reported totals.
Quoted 7 lines
In the other direction,
ibmveth_alloc_single_rx_queue()resets rx_queue[queue_idx].index/num_slots/toggle and copies the pool geometry but never clears adapter->rx_qstats[queue_idx], so re-adding a previously removed index resurrects its old counters and the totals jump up.Should the retired queues' counts be folded into an accumulator and the slot cleared on (re-)allocation, so
rtnl_link_stats64stays monotonic?
Yes, that is a real integration issue with the stats model.
I'm planning to preserve monotonic aggregate stats across resize, either by folding retired queues into a persistent accumulator or by otherwise ensuring removed queues do not disappear from the reported totals and then reappear later.
Quoted 30 lines
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-0+ +for (i = new_count; i < old_count; i++) { +ibmveth_cleanup_single_rx_interrupt (adapter, i); +ibmveth_deregister_single_rx_queue (adapter, i);drivers/net/ethernet/ibm/ibmveth.c ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter, · line 1985
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+ibmveth_free_single_rx_queue (adapter, i); + } + } + +netdev_info (netdev,"Successfully resized to %d RX queues (incremental)\n" , + adapter->num_rx_queues); + +if (firmware_has_feature (FW_FEATURE_CMO )) +vio_cmo_set_dev_desired (adapter->vdev, +ibmveth_get_desired_dma (adapter->vdev));[Severity: Medium]
Is the entitlement raised too late on CMO LPARs? All the CMO-charged DMA work for the new queues has already happened by the time this runs:dma_map_single()of the buffer-list page anddma_alloc_coherent()of the descriptor ring inibmveth_alloc_single_rx_queue(), plus every RX skb mapping done byibmveth_replenish_task(). Those run while the device's desired entitlement still reflects the old, smaller queue count, andvio_dma_iommu_alloc_coherent()refuses the allocation whenvio_cmo_alloc()cannot cover it.It is also skipped on every failure and rollback return, so a resize that fails leaves the entitlement untouched.
The driver's own convention is the opposite order, in
ibmveth_change_mtu():
Quoted 8 lines
vio_cmo_set_dev_desired(viodev, ibmveth_get_desired_dma(viodev)); ... ibmveth_open(...)Note that simply moving the call is not enough, since
ibmveth_get_desired_dma()itself loops overadapter->num_rx_queues, so the target count has to be accounted before the allocations and rolled back on failure.
Yes, on CMO systems that entitlement update is too late.
I’ll raise the desired entitlement for the target RX queue count before scale-up allocation, and restore it on failure so the entitlement tracks the surviving geometry.
Quoted 4 lines
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-0+ +return 0 ; + +cleanup_new_queues:
Quoted 8 lines
drivers/net/ethernet/ibm/ibmveth.c ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter, · line 1993
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-0+ failed_queue = i; +netdev_err (netdev, +"Scale-up failed at queue %d, cleaning up queues %d-%d\n" , + failed_queue, old_count, failed_queue -1 );[Severity: Low]
When the first newly added queue is the one that fails,failed_queue==old_count, so this prints an inverted empty range such as "cleaning up queues 4-3" while the three loops below correctly iterate zero times.
Yes, that log should not print an inverted empty range.
I’ll log the cleanup range only when there is actually something to clean up, and otherwise say there is nothing to clean up.
Quoted 45 lines
Quoted 9 lines
drivers/net/ethernet/ibm/ibmveth.c +9
-0+for (i = old_count; i < failed_queue; i++) { +ibmveth_disable_irq (adapter, i); +synchronize_irq (adapter->queue_irq[i]); + } + +for (i = old_count; i < failed_queue; i++) +napi_disable (&adapter->napi[i]); + +for (i = old_count; i < failed_queue; i++)drivers/net/ethernet/ibm/ibmveth.c ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter, · line 2012
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +10
-0+ibmveth_drain_rx_queue (adapter, i); + +synchronize_net (); + +for (i = old_count; i < failed_queue; i++) { +ibmveth_cleanup_single_rx_interrupt (adapter, i); +ibmveth_deregister_single_rx_queue (adapter, i); +ibmveth_free_single_rx_queue (adapter, i); + } + adapter->num_rx_queues = old_count;[Severity: High]
Can a concurrent NAPI poll dereference the pages this loop just freed?adapter->num_rx_queuesis restored only after the free loop, so for the duration of that loop the indices [old_count,failed_queue) are freed but still inside the published live range.
ibmveth_free_single_rx_queue()unmaps buffer_list_dma[i], frees the coherent ring and then does:free_page((unsigned long)adapter->buffer_list_addr[queue_idx]); adapter->buffer_list_addr[queue_idx] = NULL;Meanwhile every surviving queue's poll reaches
ibmveth_replenish_task()->ibmveth_update_rx_no_buffer(), which walks the published range with no NULL or liveness check:for (i = 0; i < adapter->num_rx_queues; i++) { __be64 *p = adapter->buffer_list_addr[i] + 4096 - 8; u64 drops = be64_to_cpup(p);So for a 2->4 scale-up where queue 2 came up and queue 3 fails, a poll on queue 0 or 1 reads buffer_list_addr[2] either as a just-freed page or as NULL +
0xff8, in softirq context.ibmveth_poll_controller()reaches the same range from netpoll without rtnl, and additionally posts buffers into pools that are being freed.
Quoted 8 lines
The guard in
ibmveth_replenish_task():if (queue_index >= adapter->num_rx_queues) {does not help while the count is still the higher value. The inline
enable_irqfailure handler above and the successful scale-down path both loweradapter->num_rx_queuesbefore freeing; should this label do the same, i.e. setadapter->num_rx_queues=old_countbefore the free loop?
Yes, that is a serious publish-before-free hazard in the mailed patch.
The published live range needs to be shrunk before those queues are freed, otherwise lockless readers can still walk freed queue state. I'm planning to restore the old published count before the destroy loop in cleanup_new_queues, just like the successful scale-down path.
Quoted 27 lines
Quoted 5 lines
drivers/net/ethernet/ibm/ibmveth.c +5
-0+netdev_warn (netdev,"Keeping %d queues after scale-up failure\n" , + old_count); +return rc; +} +[ ... ]
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0@@ -2217,12 +2670,62 @@ static void ibmveth_get_channels(struct net_device *netdev, channels->rx_count = adapter->num_rx_queues; }[ ... ]
Quoted 3 lines
drivers/net/ethernet/ibm/ibmveth.c +3
-0+static int ibmveth_resize_rx_channels (struct ibmveth_adapter *adapter, +unsigned int goal_rx) +{[ ... ]
drivers/net/ethernet/ibm/ibmveth.c static void ibmveth_get_channels(struct net_device *netdev, · line 2713
Quoted 8 lines
drivers/net/ethernet/ibm/ibmveth.c +8
-0+if (goal_rx == old_rx) +return 0 ; + +if (!(netdev->flags &IFF_UP )) { + adapter->num_rx_queues = goal_rx; +return 0 ; + } +[Severity: Low]
Quoted 8 lines
This isn't a bug in the final series, but at this commit the stash branch is unreachable: the only caller,
ibmveth_set_channels(), already returns early on the same !IFF_UP condition before reachingibmveth_resize_rx_channels(), so ethtool -L <dev> rx N on a down interface returns 0 while the requested count is discarded. The next patch in the series ("ibmveth: Wire ethtoolset_channelsto MQ RX queue resize") replaces that early return and makes this branch live, so this only shows up at this bisect point.
Yes, that is a series-split artifact rather than a final-series bug.
I’ll fold the down-interface stash handling into the next patch so this intermediate commit does not accept and then discard the requested count.
Quoted 36 lines
Quoted 19 lines
drivers/net/ethernet/ibm/ibmveth.c +9
-0+ rxq_entries = adapter->rx_queue[0 ].num_slots; + rc =ibmveth_resize_rx_queues_incremental (adapter, goal_rx, + rxq_entries); +if (rc) +netdev_err (netdev,"Failed to resize RX queues: %d\n" , rc); +return rc; +} +static int ibmveth_set_channels (struct net_device *netdev,struct ethtool_channels *channels) {struct ibmveth_adapter *adapter =netdev_priv (netdev);unsigned int old = netdev->real_num_tx_queues, goal = channels->tx_count; +unsigned int goal_rx = channels->rx_count;int rc, i;/* If ndo_open has not been called yet then don't allocate, just set drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_set_channels(struct net_device *netdev, · line 2743
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +7
-0@@ -2231,6 +2734,13 @@ static int ibmveth_set_channels(struct net_device *netdev,if (!(netdev->flags &IFF_UP ))return netif_set_real_num_tx_queues (netdev, goal); +/* Resize RX first while UP so ibmveth_resize_rx_channels() is used +* in this patch. !IFF_UP RX stash ordering lands next. +*/ + rc =ibmveth_resize_rx_channels (adapter, goal_rx); +if (rc) +return rc; +[Severity: Medium]
Should the RX side be unwound when the TX phase below fails? The RX resize is committed here first, and thenibmveth_allocate_tx_ltb()can return-ENOMEMornetif_set_real_num_tx_queues()can fail, in which case the function ends with return rc and no compensatingibmveth_resize_rx_channels(adapter, old_rx).
Quoted 5 lines
ETHTOOL_SCHANNELSthen reports failure while half of the requested configuration is in effect: on scale-down the excess RX queues have already been deregistered from PHYP and freed, on scale-up new queues are registered with IRQs installed and DMA memory pinned, and ethtool -l afterwards reports the newrx_count.
Yes, the combined RX/TX transaction needs a rollback story.
I'm planning to roll RX back to the old count if the later TX phase cannot reach the requested configuration, so set_channels() does not report failure after committing only half of the change.
So my updated view is that patch 12 got materially better in follow-up on several of the biggest resize-path issues: publication ordering, teardown ordering, errno normalization, post-unmask pending handling, rollback robustness, and CMO timing. I would still avoid claiming the live-resize model is fully closed out until the remaining publication, teardown, and transaction story is tighter.
Thanks,
Mingming
Quoted 4 lines
Quoted 3 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0/* We have IBMVETH_MAX_QUEUES netdev_queue's allocated * but we may need to alloc/free the ltb's. */ [ ... ]
PATCH v4 13/14 ibmveth: Wire ethtool set_channels to MQ RX queue resize
In reply to Mingming CaoComplete set_channels() RX handling around ibmveth_resize_rx_channels():
- When the interface is down: set TX queues first, then stash the desired RX count in adapter->num_rx_queues for the next open() (open publishes via netif_set_real_num_rx_queues). While down there are no RX queue mappings, buffers, or IRQs to grow or shrink, so do not allocate immediately.
- When up: resize RX via ibmveth_resize_rx_channels(), then adjust TX LTBs with the existing stop/alloc/set_real_num_tx/free/wake path.
- Non-MQ firmware returns -EOPNOTSUPP for rx > 1.
- Validate rx_count within 1..IBMVETH_MAX_RX_QUEUES.
TX path hardening:
- Initialize i = old_tx so a scale-down path that never enters the alloc loop still has defined bounds if set_real_num_tx_queues() fails.
- Always return rc from set_channels().
---
Changes in v4:
- On !IFF_UP, stash num_rx_queues only after TX set succeeds; do not allocate live subordinate IRQs/buffers while down.
- Initialize i = old_tx on the TX adjust path.
- Always return rc from set_channels().
- Split from the resize-helper patch (same split as v3) while keeping a live caller of resize_rx_channels() in the previous patch.
drivers/net/ethernet/ibm/ibmveth.c | 51 ++++++++++++++++++++---------- 1 file changed, 34 insertions(+), 17 deletions(-)
drivers/net/ethernet/ibm/ibmveth.c +34 -17
@@ -2723,20 +2723,28 @@ static int ibmveth_set_channels(struct net_device *netdev, struct ethtool_channels *channels) {struct ibmveth_adapter *adapter =netdev_priv (netdev); -unsigned int old = netdev->real_num_tx_queues, - goal = channels->tx_count;unsigned int goal_rx = channels->rx_count; +unsigned int old_tx = netdev->real_num_tx_queues; +unsigned int goal_tx = channels->tx_count;int rc, i;/* If ndo_open has not been called yet then don't allocate, just set * desired netdev_queue's and return */ -if (!(netdev->flags &IFF_UP )) -return netif_set_real_num_tx_queues (netdev, goal); +if (!(netdev->flags &IFF_UP )) { +if (goal_tx != old_tx) { + rc =netif_set_real_num_tx_queues (netdev, goal_tx); +if (rc) +return rc; + } + +/* Stash desired RX count only after TX succeeds (or was +* already correct); open() publishes it via +* netif_set_real_num_rx_queues(). +*/ +return ibmveth_resize_rx_channels (adapter, goal_rx); + } -/* Resize RX first while UP so ibmveth_resize_rx_channels() is used -* in this patch. !IFF_UP RX stash ordering lands next. -*/ rc =ibmveth_resize_rx_channels (adapter, goal_rx);if (rc)return rc;
@@ -2744,10 +2752,17 @@ static int ibmveth_set_channels(struct net_device *netdev,/* We have IBMVETH_MAX_QUEUES netdev_queue's allocated * but we may need to alloc/free the ltb's. */ +if (goal_tx == old_tx) +return 0 ; +netif_tx_stop_all_queues (netdev); -/* Allocate any queue that we need */ -for (i = old; i < goal; i++) { +/* Allocate any queue that we need. Initialize i to old_tx so a +* scale-down path that never enters the loop still has defined +* bounds if set_real_num_tx_queues() fails. +*/ + i = old_tx; +for (; i < goal_tx; i++) {if (adapter->tx_ltb_ptr[i])continue ;
@@ -2755,20 +2770,22 @@ static int ibmveth_set_channels(struct net_device *netdev,if (!rc)continue ; +/* if something goes wrong, free everything we just allocated */ netdev_err (netdev,"Failed to allocate more tx queues, returning to %d queues\n" , - old); - goal = old; - old = i; + old_tx); + goal_tx = old_tx; + old_tx = i;break ; } - rc =netif_set_real_num_tx_queues (netdev, goal); + rc =netif_set_real_num_tx_queues (netdev, goal_tx);if (rc) {netdev_err (netdev,"Failed to set real tx queues, returning to %d queues\n" , - old); - goal = old; - old = i; + old_tx); + goal_tx = old_tx; + old_tx = i; } -for (i = old; i > goal; i--) { +/* Free any that are no longer needed */ +for (i = old_tx; i > goal_tx; i--) {if (adapter->tx_ltb_ptr[i -1 ])ibmveth_free_tx_ltb (adapter, i -1 ); }
Signature · 1 line
2.50.1 (Apple Git-155)
Jakub Kicinski
maintainerRe: [PATCH net-next v4 13/14] ibmveth: Wire ethtool set_channels to MQ RX queue resize
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
ibmveth: Wire ethtool set_channels to MQ RX queue resize
This completes the RX side of ibmveth_set_channels(): when the interface is down the TX queue count is published and the desired RX count is stashed for the next open(), and when it is up the RX resize runs through ibmveth_resize_rx_channels() before the existing TX LTB alloc/free path. It also renames the TX bookkeeping variables to old_tx/goal_tx and adds an early return when the TX count is unchanged.
A few questions below.
Quoted 2 lines
- Non-MQ firmware returns
-EOPNOTSUPPfor rx > 1.
- Validaterx_countwithin 1..IBMVETH_MAX_RX_QUEUES.
[Severity: Low]
Do these two bullets describe changes that are actually in this patch?
Both checks live in ibmveth_resize_rx_channels() in drivers/net/ethernet/ibm/ibmveth.c:
if (goal_rx > 1 && !adapter->multi_queue) {
...
return -EOPNOTSUPP;
}
if (goal_rx < 1 || goal_rx > IBMVETH_MAX_RX_QUEUES) {
...
return -EINVAL;
}and they were added by the preceding patch "ibmveth: Implement incremental MQ RX queue resize". This patch neither adds nor changes them.
Related to the split: the parent commit carried a comment in ibmveth_set_channels() that referred to the patch series itself, which this patch now deletes:
/* Resize RX first while UP so ibmveth_resize_rx_channels() is used * in this patch. !IFF_UP RX stash ordering lands next. */
Could that comment be dropped from the parent instead, so no committed state of the tree describes an unfinished series?
Quoted 11 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-2@@ -2723,20 +2723,28 @@ static int ibmveth_set_channels(struct net_device *netdev, struct ethtool_channels *channels) {struct ibmveth_adapter *adapter =netdev_priv (netdev); -unsigned int old = netdev->real_num_tx_queues, - goal = channels->tx_count;unsigned int goal_rx = channels->rx_count;
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_set_channels(struct net_device *netdev, · line 2734
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +3
-2+unsigned int old_tx = netdev->real_num_tx_queues; +unsigned int goal_tx = channels->tx_count;int rc, i;/* If ndo_open has not been called yet then don't allocate, just set * desired netdev_queue's and return */ -if (!(netdev->flags &IFF_UP )) -return netif_set_real_num_tx_queues (netdev, goal); +if (!(netdev->flags &IFF_UP )) {
[Severity: High]
Is netdev->flags & IFF_UP a reliable proxy here for "the driver holds RX resources"?
ibmveth tears down and re-creates its resources internally while IFF_UP stays set. veth_pool_store() does:
ibmveth_close(netdev);
...
if (netif_running(netdev)) {
rc = ibmveth_open(netdev);
if (rc) {
pool->active = oldactive;
...
goto unlock_err;and ibmveth_change_mtu() does the same close/open dance. If the reopen fails, for example a GFP_KERNEL failure in ibmveth_alloc_rx_queues() or ibmveth_alloc_buffer_pools(), the error is returned without clearing dev->flags, so IFF_UP remains set while ibmveth_close() -> ibmveth_cleanup_rx_interrupts() has already run:
for (i = 0; i < adapter->num_rx_queues; i++) napi_disable(&adapter->napi[i]);
and ibmveth_cleanup_rx_resources() has set buffer_list_addr[i] and rx_queue[i].queue_addr to NULL, with adapter->num_rx_queues left unchanged.
A later "ethtool -L ethX rx <smaller>" then takes the UP branch and the scale-down loop in ibmveth_resize_rx_queues_incremental() calls napi_disable(&adapter->napi[i]) a second time on an already-disabled NAPI. Does napi_disable_locked() ever return in that case?
net/core/dev.c:napi_disable_locked() {
...
while (val & (NAPIF_STATE_SCHED | NAPIF_STATE_NPSVC)) {
usleep_range(20, 200);
val = READ_ONCE(n->state);
}
...
}That would be an unkillable spin with rtnl_lock held.
The scale-up direction on the same stale state reaches ibmveth_replenish_task() -> ibmveth_update_rx_no_buffer(), which does:
for (i = 0; i < adapter->num_rx_queues; i++) {
__be64 *p = adapter->buffer_list_addr[i] + 4096 - 8;
u64 drops = be64_to_cpup(p);with buffer_list_addr[i] already NULL.
This dispatch on IFF_UP came from the preceding patch, but this patch finalizes it as the user-visible interface. Would keying the decision on an explicit "RX resources allocated" state in the adapter avoid this?
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_set_channels(struct net_device *netdev, · line 2746
Quoted 12 lines
drivers/net/ethernet/ibm/ibmveth.c +12
-0+if (goal_tx != old_tx) { + rc =netif_set_real_num_tx_queues (netdev, goal_tx); +if (rc) +return rc; + } + +/* Stash desired RX count only after TX succeeds (or was +* already correct); open() publishes it via +* netif_set_real_num_rx_queues(). +*/ +return ibmveth_resize_rx_channels (adapter, goal_rx); + }
[Severity: Medium]
Does this path need to refresh the VIO CMO DMA entitlement?
The stash branch of ibmveth_resize_rx_channels() just does:
if (!(netdev->flags & IFF_UP)) {
adapter->num_rx_queues = goal_rx;
return 0;
}and ibmveth_get_desired_dma() sizes the request per RX queue:
for (q = 0; q < adapter->num_rx_queues; q++) {The only two vio_cmo_set_dev_desired() call sites in the driver are the live resize path in ibmveth_resize_rx_queues_incremental():
if (firmware_has_feature(FW_FEATURE_CMO)) vio_cmo_set_dev_desired(adapter->vdev, ibmveth_get_desired_dma(adapter->vdev));
and ibmveth_change_mtu(), which refreshes it even while the interface is closed. ibmveth_open() does not call it.
So on a CMO LPAR, after
ip link set dev ethX down
ethtool -L ethX rx 16
ip link set dev ethX up
the entitlement is still sized for the old, smaller RX count while open() DMA-maps the larger footprint (per-queue buffer-list page, RX ring, buffer pools). Can vio_dma_iommu_map_page() -> vio_cmo_alloc() then fail and make the link fail to come up with -ENOMEM? Shrinking while down has the mirror effect of leaving entitlement reserved until the next MTU change or live resize.
[Severity: Low]
Should this branch also publish the RX count to the netdev?
TX is published immediately with netif_set_real_num_tx_queues(), but RX only reaches the stash in ibmveth_resize_rx_channels(), so netdev->real_num_rx_queues keeps its old value until the next ibmveth_open(). Meanwhile ibmveth_get_channels() reports the private counter:
channels->rx_count = adapter->num_rx_queues;
After "ip link set dev ethX down; ethtool -L ethX rx N", ethtool -l reports N while /sys/class/net/ethX/queues/rx-*, the RPS/RFS maps and netdev-genl queue enumeration still show the old count.
netif_set_real_num_rx_queues() only touches kobjects and RPS maps and allocates no driver resources, and probe already sized the netdev with alloc_etherdev_mqs(..., IBMVETH_MAX_QUEUES, IBMVETH_MAX_RX_QUEUES), so is there a reason RX cannot be published here the way TX is?
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_set_channels(struct net_device *netdev, · line 2750
Quoted 7 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-3-/* Resize RX first while UP so ibmveth_resize_rx_channels() is used -* in this patch. !IFF_UP RX stash ordering lands next. -*/ rc =ibmveth_resize_rx_channels (adapter, goal_rx);if (rc)return rc;
[Severity: High]
Can this live resize race with a surviving queue's NAPI poll?
The scale-down branch of ibmveth_resize_rx_queues_incremental() takes its grace period before lowering the count, and the frees come after the store:
synchronize_net();
rc = netif_set_real_num_rx_queues(netdev, new_count);
...
adapter->num_rx_queues = new_count;
for (i = new_count; i < old_count; i++) {
ibmveth_cleanup_single_rx_interrupt(adapter, i);
ibmveth_deregister_single_rx_queue(adapter, i);
ibmveth_free_single_rx_queue(adapter, i);
}ibmveth_free_single_rx_queue() then does:
if (adapter->buffer_list_addr[queue_idx]) {
free_page((unsigned long)adapter->buffer_list_addr[queue_idx]);
adapter->buffer_list_addr[queue_idx] = NULL;
}A surviving queue's poll -> ibmveth_replenish_task() -> ibmveth_update_rx_no_buffer() uses the count as a lockless bound and dereferences the pointer without a NULL check:
for (i = 0; i < adapter->num_rx_queues; i++) {
__be64 *p = adapter->buffer_list_addr[i] + 4096 - 8;
u64 drops = be64_to_cpup(p);The reader holds only rx_queue[k].replenish_lock, the writer holds rtnl, so the locksets do not intersect and disabling the removed queues' NAPIs does not quiesce the surviving pollers. Can this read a freed page, or dereference NULL + 4088 if the pointer load lands after the NULL store?
The scale-up side publishes the count with a plain store before the reader can see the pointer stores:
adapter->num_rx_queues = i + 1; ibmveth_replenish_task(adapter, i);
On PowerPC, can a reader observe the larger count with a stale NULL buffer_list_addr[i]?
The resize helper and its UP-path caller came from the preceding patch "ibmveth: Implement incremental MQ RX queue resize", so this is not introduced here, but this patch completes the user-facing entry point that drives it. The later patch "ibmveth: Fix MQ RX poll and shutdown hangs after queue resize" only touches correlator validation and poll shutdown and leaves ibmveth_update_rx_no_buffer() and this ordering unchanged.
[Severity: Medium]
Can the adapter-wide rx_no_buffer counter lose updates once more than one RX queue is configured through this path?
ibmveth_update_rx_no_buffer() resets and re-accumulates a shared counter and writes every queue's qstats entry:
adapter->rx_no_buffer = 0;
for (i = 0; i < adapter->num_rx_queues; i++) {
...
adapter->rx_no_buffer += drops;
}but it is called from ibmveth_replenish_task() while only that caller's per-queue lock is held:
spin_lock_irqsave(&rxq->replenish_lock, flags); ... ibmveth_update_rx_no_buffer(adapter); spin_unlock_irqrestore(&rxq->replenish_lock, flags);
With polls for queue 0 and queue 1 running on different CPUs, the locksets are disjoint, so the "= 0" and the "+=" from both CPUs interleave and ethtool -S / ndo_get_stats64() can report a value near zero or well below the real one. This came in with the earlier per-queue aggregation and MQ RX patches rather than this one, and the later "ibmveth: Fix MQ RX poll and shutdown hangs after queue resize" patch does not touch this function.
[Severity: Low]
Is a partially applied configuration possible here?
In the UP path the RX resize is committed first, including hypervisor registration, IRQ setup/teardown, buffer alloc/free and the update of adapter->num_rx_queues. If the later netif_set_real_num_tx_queues() fails, for example -ENOMEM out of netdev_queue_update_kobjects(), set_channels() returns that error with the RX count already changed, so the resulting configuration matches neither the old nor the requested one and nothing rolls RX back.
The !IFF_UP branch above has the mirror ordering: TX is committed with netif_set_real_num_tx_queues() before any RX validation runs, since the -EOPNOTSUPP and -EINVAL checks live inside ibmveth_resize_rx_channels(), which is called last. A rejected RX value would leave the TX count changed with no rollback. Today the ethtool core clamps rx_count against max_rx before the driver op runs, so this half is latent, but should the RX validation happen before TX is touched?
Quoted 7 lines
drivers/net/ethernet/ibm/ibmveth.c +3
-0@@ -2744,10 +2752,17 @@ static int ibmveth_set_channels(struct net_device *netdev,/* We have IBMVETH_MAX_QUEUES netdev_queue's allocated * but we may need to alloc/free the ltb's. */ +if (goal_tx == old_tx) +return 0 ; +
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_set_channels(struct net_device *netdev, · line 2765
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +6
-2netif_tx_stop_all_queues (netdev); -/* Allocate any queue that we need */ -for (i = old; i < goal; i++) { +/* Allocate any queue that we need. Initialize i to old_tx so a +* scale-down path that never enters the loop still has defined +* bounds if set_real_num_tx_queues() fails. +*/ + i = old_tx; +for (; i < goal_tx; i++) {
[Severity: Low]
Is there a case where i was undefined before this change?
The previous form was:
for (i = old; i < goal; i++) {The init clause runs unconditionally before the first condition test, so i was already assigned even when the body never executes. Splitting it into "i = old_tx; for (; i < goal_tx; i++)" looks semantically identical, and the new comment suggests the idiomatic form was unsafe. Could the comment be dropped or reworded?
The changelog bullet "Always return rc from set_channels()" also looks like a non-change, since the UP path already ended with "return rc;" and the down path already returned the netif_set_real_num_tx_queues() value.
In the one case where an error is actually lost, the claim does not hold. On ibmveth_allocate_tx_ltb() failure the rewind runs:
goal_tx = old_tx; old_tx = i; break; } rc = netif_set_real_num_tx_queues(netdev, goal_tx);
so the -ENOMEM is overwritten by a call that cannot fail because goal_tx now equals dev->real_num_tx_queues, and "ethtool -L ethX tx N" reports success after an allocation failure. That clobber is identical at the baseline and is not introduced here, it just means the hardening bullet does not describe it.
Separately, the genuinely new TX behaviour in this hunk, "if (goal_tx == old_tx) return 0;", which now skips netif_tx_stop_all_queues() and netif_tx_wake_all_queues() when only RX is resized, is not mentioned in the commit message. Could it be added there?
[ ... ]
Re: [PATCH net-next v4 13/14] ibmveth: Wire ethtool set_channels to MQ RX queue resize
Quoted 5 lines · Jakub Kicinski
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
ibmveth: Wire ethtoolset_channelsto MQ RX queue resize
Quoted 8 lines
This completes the RX side of
ibmveth_set_channels(): when the interface is down the TX queue count is published and the desired RX count is stashed for the nextopen(), and when it is up the RX resize runs throughibmveth_resize_rx_channels()before the existing TX LTB alloc/free path. It also renames the TX bookkeeping variables to old_tx/goal_tx and adds an early return when the TX count is unchanged.A few questions below.
Hi Jakub,
Thanks for the review.
Quoted 25 lines
Quoted 2 lines
- Non-MQ firmware returns
-EOPNOTSUPPfor rx > 1.
- Validaterx_countwithin 1..IBMVETH_MAX_RX_QUEUES.[Severity: Low]
Do these two bullets describe changes that are actually in this patch?Both checks live in
ibmveth_resize_rx_channels()indrivers/net/ethernet/ibm/ibmveth.c:if (goal_rx > 1 && !adapter->multi_queue) { ... return -EOPNOTSUPP; } if (goal_rx < 1 || goal_rx > IBMVETH_MAX_RX_QUEUES) { ... return -EINVAL; }and they were added by the preceding patch "ibmveth: Implement incremental MQ RX queue resize". This patch neither adds nor changes them.
Related to the split: the parent commit carried a comment in
ibmveth_set_channels()that referred to the patch series itself, which this patch now deletes:
Quoted 6 lines
/* Resize RX first while UP so ibmveth_resize_rx_channels() is used * in this patch. !IFF_UP RX stash ordering lands next. */Could that comment be dropped from the parent instead, so no committed state of the tree describes an unfinished series?
Yes, those bullets belong with the earlier resize helper rather than this wiring patch.
I’ll move that description to the earlier patch and drop the series-self-referential comment so no intermediate commit describes an unfinished series.
Quoted 72 lines
Quoted 11 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-2@@ -2723,20 +2723,28 @@ static int ibmveth_set_channels(struct net_device *netdev, struct ethtool_channels *channels) {struct ibmveth_adapter *adapter =netdev_priv (netdev); -unsigned int old = netdev->real_num_tx_queues, - goal = channels->tx_count;unsigned int goal_rx = channels->rx_count;drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_set_channels(struct net_device *netdev, · line 2734
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +3
-2+unsigned int old_tx = netdev->real_num_tx_queues; +unsigned int goal_tx = channels->tx_count;int rc, i;/* If ndo_open has not been called yet then don't allocate, just set * desired netdev_queue's and return */ -if (!(netdev->flags &IFF_UP )) -return netif_set_real_num_tx_queues (netdev, goal); +if (!(netdev->flags &IFF_UP )) {[Severity: High]
Isnetdev->flags&IFF_UPa reliable proxy here for "the driver holds RX resources"?ibmveth tears down and re-creates its resources internally while
IFF_UPstays set.veth_pool_store()does:ibmveth_close(netdev); ... if (netif_running(netdev)) { rc = ibmveth_open(netdev); if (rc) { pool->active = oldactive; ... goto unlock_err;and
ibmveth_change_mtu()does the same close/open dance. If the reopen fails, for example aGFP_KERNELfailure inibmveth_alloc_rx_queues()oribmveth_alloc_buffer_pools(), the error is returned without clearingdev->flags, soIFF_UPremains set whileibmveth_close()->ibmveth_cleanup_rx_interrupts()has already run:for (i = 0; i < adapter->num_rx_queues; i++) napi_disable(&adapter->napi[i]);and
ibmveth_cleanup_rx_resources()has set buffer_list_addr[i] and rx_queue[i].queue_addr to NULL, withadapter->num_rx_queuesleft unchanged.A later "ethtool -L ethX rx <smaller>" then takes the UP branch and the scale-down loop in
ibmveth_resize_rx_queues_incremental()callsnapi_disable(&adapter->napi[i])a second time on an already-disabled NAPI. Doesnapi_disable_locked()ever return in that case?net/core/dev.c:napi_disable_locked() { ... while (val & (NAPIF_STATE_SCHED | NAPIF_STATE_NPSVC)) { usleep_range(20, 200); val = READ_ONCE(n->state); } ... }That would be an unkillable spin with
rtnl_lockheld.The scale-up direction on the same stale state reaches
ibmveth_replenish_task()->ibmveth_update_rx_no_buffer(), which does:for (i = 0; i < adapter->num_rx_queues; i++) { __be64 *p = adapter->buffer_list_addr[i] + 4096 - 8; u64 drops = be64_to_cpup(p);
Quoted 5 lines
with buffer_list_addr[i] already NULL.
This dispatch on
IFF_UPcame from the preceding patch, but this patch finalizes it as the user-visible interface. Would keying the decision on an explicit "RX resources allocated" state in the adapter avoid this?
Yes, keying this off adapter->opened should avoid that stale-UP dispatch, because it tracks whether RX/TX resources are actually instantiated rather than whether the netdev still carries IFF_UP after a failed reopen.
Quoted 42 lines
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_set_channels(struct net_device *netdev, · line 2746
Quoted 12 lines
drivers/net/ethernet/ibm/ibmveth.c +12
-0+if (goal_tx != old_tx) { + rc =netif_set_real_num_tx_queues (netdev, goal_tx); +if (rc) +return rc; + } + +/* Stash desired RX count only after TX succeeds (or was +* already correct); open() publishes it via +* netif_set_real_num_rx_queues(). +*/ +return ibmveth_resize_rx_channels (adapter, goal_rx); + }[Severity: Medium]
Does this path need to refresh the VIO CMO DMA entitlement?The stash branch of
ibmveth_resize_rx_channels()just does:if (!(netdev->flags & IFF_UP)) { adapter->num_rx_queues = goal_rx; return 0; }and
ibmveth_get_desired_dma()sizes the request per RX queue:for (q = 0; q < adapter->num_rx_queues; q++) {The only two
vio_cmo_set_dev_desired()call sites in the driver are the live resize path inibmveth_resize_rx_queues_incremental():if (firmware_has_feature(FW_FEATURE_CMO)) vio_cmo_set_dev_desired(adapter->vdev, ibmveth_get_desired_dma(adapter->vdev));and
ibmveth_change_mtu(), which refreshes it even while the interface is closed.ibmveth_open()does not call it.So on a CMO LPAR, after
ip link set dev ethX down
ethtool -L ethX rx 16
ip link set dev ethX up
Quoted 6 lines
the entitlement is still sized for the old, smaller RX count while
open()DMA-maps the larger footprint (per-queue buffer-list page, RX ring, buffer pools). Canvio_dma_iommu_map_page()->vio_cmo_alloc()then fail and make the link fail to come up with-ENOMEM? Shrinking while down has the mirror effect of leaving entitlement reserved until the next MTU change or live resize.
Yes, the down-path stash also needs to refresh CMO desired state.
I’ll update the desired entitlement after stashing the new RX count, so the next open does not try to map a larger RX footprint against the old budget, and shrinking while down also drops stale over-entitlement.
Quoted 11 lines
[Severity: Low]
Should this branch also publish the RX count to the netdev?TX is published immediately with
netif_set_real_num_tx_queues(), but RX only reaches the stash inibmveth_resize_rx_channels(), sonetdev->real_num_rx_queueskeeps its old value until the nextibmveth_open(). Meanwhileibmveth_get_channels()reports the private counter:channels->rx_count = adapter->num_rx_queues;
Quoted 8 lines
After "ip link set dev ethX down; ethtool -L ethX rx N", ethtool -l reports N while
/sys/class/net/ethX/queues/rx-*, the RPS/RFS maps and netdev-genl queue enumeration still show the old count.
netif_set_real_num_rx_queues()only touches kobjects and RPS maps and allocates no driver resources, and probe already sized the netdev withalloc_etherdev_mqs(..., IBMVETH_MAX_QUEUES, IBMVETH_MAX_RX_QUEUES), so is there a reason RX cannot be published here the way TX is?
Yes, the while-down stash path should publish RX consistently as well.
I’ll publish real_num_rx_queues on the while-down stash path too, so userspace does not see the private RX count diverge from netdev core state before the next open.
Quoted 54 lines
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_set_channels(struct net_device *netdev, · line 2750
Quoted 7 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-3-/* Resize RX first while UP so ibmveth_resize_rx_channels() is used -* in this patch. !IFF_UP RX stash ordering lands next. -*/ rc =ibmveth_resize_rx_channels (adapter, goal_rx);if (rc)return rc;[Severity: High]
Can this live resize race with a surviving queue's NAPI poll?The scale-down branch of
ibmveth_resize_rx_queues_incremental()takes its grace period before lowering the count, and the frees come after the store:synchronize_net(); rc = netif_set_real_num_rx_queues(netdev, new_count); ... adapter->num_rx_queues = new_count; for (i = new_count; i < old_count; i++) { ibmveth_cleanup_single_rx_interrupt(adapter, i); ibmveth_deregister_single_rx_queue(adapter, i); ibmveth_free_single_rx_queue(adapter, i); }
ibmveth_free_single_rx_queue()then does:if (adapter->buffer_list_addr[queue_idx]) { free_page((unsigned long)adapter->buffer_list_addr[queue_idx]); adapter->buffer_list_addr[queue_idx] = NULL; }A surviving queue's poll ->
ibmveth_replenish_task()->ibmveth_update_rx_no_buffer()uses the count as a lockless bound and dereferences the pointer without a NULL check:for (i = 0; i < adapter->num_rx_queues; i++) { __be64 *p = adapter->buffer_list_addr[i] + 4096 - 8; u64 drops = be64_to_cpup(p);The reader holds only rx_queue[k].replenish_lock, the writer holds rtnl, so the locksets do not intersect and disabling the removed queues' NAPIs does not quiesce the surviving pollers. Can this read a freed page, or dereference NULL + 4088 if the pointer load lands after the NULL store?
The scale-up side publishes the count with a plain store before the reader can see the pointer stores:
adapter->num_rx_queues = i + 1; ibmveth_replenish_task(adapter, i);On PowerPC, can a reader observe the larger count with a stale NULL buffer_list_addr[i]?
Quoted 6 lines
The resize helper and its UP-path caller came from the preceding patch "ibmveth: Implement incremental MQ RX queue resize", so this is not introduced here, but this patch completes the user-facing entry point that drives it. The later patch "ibmveth: Fix MQ RX poll and shutdown hangs after queue resize" only touches correlator validation and poll shutdown and leaves
ibmveth_update_rx_no_buffer()and this ordering unchanged.
Yes, a reader can observe the larger count before the new queue state is safely published, or keep walking queue state after teardown has started. This path inherits the same publication and teardown hazards from the live resize helper, and patch 13 is where they become user reachable.
I’ll fix this the same way in the earlier resize path: make update_rx_no_buffer() queue-local, publish num_rx_queues with ordering, and unpublish before freeing queue state.
Quoted 22 lines
[Severity: Medium]
Can the adapter-widerx_no_buffercounter lose updates once more than one RX queue is configured through this path?
ibmveth_update_rx_no_buffer()resets and re-accumulates a shared counter and writes every queue's qstats entry:adapter->rx_no_buffer = 0; for (i = 0; i < adapter->num_rx_queues; i++) { ... adapter->rx_no_buffer += drops; }but it is called from
ibmveth_replenish_task()while only that caller's per-queue lock is held:spin_lock_irqsave(&rxq->replenish_lock, flags); ... ibmveth_update_rx_no_buffer(adapter); spin_unlock_irqrestore(&rxq->replenish_lock, flags);
Quoted 6 lines
With polls for queue 0 and queue 1 running on different CPUs, the locksets are disjoint, so the "= 0" and the "+=" from both CPUs interleave and ethtool -S /
ndo_get_stats64()can report a value near zero or well below the real one. This came in with the earlier per-queue aggregation and MQ RX patches rather than this one, and the later "ibmveth: Fix MQ RX poll and shutdown hangs after queue resize" patch does not touch this function.
Yes, that is the same queue-accounting bug class.
I’ll keep no_buffer refresh queue-local and derive the adapter-level total by summing on read, so concurrent replenish paths do not race on a shared clear-and-rebuild counter.
Quoted 11 lines
[Severity: Low]
Is a partially applied configuration possible here?In the UP path the RX resize is committed first, including hypervisor registration, IRQ setup/teardown, buffer alloc/free and the update of
adapter->num_rx_queues. If the laternetif_set_real_num_tx_queues()fails, for example-ENOMEMout ofnetdev_queue_update_kobjects(),set_channels()returns that error with the RX count already changed, so the resulting configuration matches neither the old nor the requested one and nothing rolls RX back.
Quoted 7 lines
The !IFF_UP branch above has the mirror ordering: TX is committed with
netif_set_real_num_tx_queues()before any RX validation runs, since the-EOPNOTSUPPand-EINVALchecks live insideibmveth_resize_rx_channels(), which is called last. A rejected RX value would leave the TX count changed with no rollback. Today the ethtool core clampsrx_countagainstmax_rxbefore the driver op runs, so this half is latent, but should the RX validation happen before TX is touched?
Yes, the combined RX/TX transaction needs a clearer rollback story.
I’ll validate both directions up front, and if the live TX phase cannot reach the requested goal I’ll roll RX back to the old count so set_channels() does not fail after committing only half the change.
Quoted 48 lines
Quoted 7 lines
drivers/net/ethernet/ibm/ibmveth.c +3
-0@@ -2744,10 +2752,17 @@ static int ibmveth_set_channels(struct net_device *netdev,/* We have IBMVETH_MAX_QUEUES netdev_queue's allocated * but we may need to alloc/free the ltb's. */ +if (goal_tx == old_tx) +return 0 ; +drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_set_channels(struct net_device *netdev, · line 2765
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +6
-2netif_tx_stop_all_queues (netdev); -/* Allocate any queue that we need */ -for (i = old; i < goal; i++) { +/* Allocate any queue that we need. Initialize i to old_tx so a +* scale-down path that never enters the loop still has defined +* bounds if set_real_num_tx_queues() fails. +*/ + i = old_tx; +for (; i < goal_tx; i++) {[Severity: Low]
Is there a case where i was undefined before this change?The previous form was:
for (i = old; i < goal; i++) {The init clause runs unconditionally before the first condition test, so i was already assigned even when the body never executes. Splitting it into "i =
old_tx; for (; i <goal_tx; i++)" looks semantically identical, and the new comment suggests the idiomatic form was unsafe. Could the comment be dropped or reworded?The changelog bullet "Always return rc from
set_channels()" also looks like a non-change, since the UP path already ended with "return rc;" and the down path already returned thenetif_set_real_num_tx_queues()value.In the one case where an error is actually lost, the claim does not hold. On
ibmveth_allocate_tx_ltb()failure the rewind runs:goal_tx = old_tx; old_tx = i; break; } rc = netif_set_real_num_tx_queues(netdev, goal_tx);so the
-ENOMEMis overwritten by a call that cannot fail becausegoal_txnow equalsdev->real_num_tx_queues, and "ethtool -L ethX tx N" reports success after an allocation failure. That clobber is identical at the baseline and is not introduced here, it just means the hardening bullet does not describe it.
Quoted 6 lines
Separately, the genuinely new TX behaviour in this hunk, "if (
goal_tx==old_tx) return 0;", which now skipsnetif_tx_stop_all_queues()andnetif_tx_wake_all_queues()when only RX is resized, is not mentioned in the commit message. Could it be added there?[ ... ]
Yes, the comment/changelog text is wrong, and the claimed error-return cleanup is overstated.
I’ll drop or reword the i = old_tx comment so it does not imply the earlier loop form was unsafe, and I’ll make the folded commit message describe the real TX-only behavior more precisely instead of claiming the error path is fully fixed in this patch.
Thanks,
Mingming
PATCH v4 14/14 ibmveth: Fix MQ RX poll and shutdown hangs after queue resize
In reply to Mingming CaoAfter aggressive ethtool -L cycling, PHYP can leave a VALID RX descriptor with a correlator that no longer matches the per-queue buffer pools. Poll treated this as fatal: ibmveth_rxq_get_buffer() WARNed and returned NULL without advancing the ring, then restart_poll retried the same slot forever.
Advance past bad correlators instead of spinning: validate correlators without WARN_ON, skip invalid slots in poll (count as invalid_buffers), and advance the RX ring when remove_buffer_from_pool cannot map the correlator. Rate-limit the bad correlator message.
Complete NAPI when the interface is down or napi_disable is pending so close/quiesce can finish. Do not restart_poll in that window. Close keeps hypervisor IRQ disable before napi_disable (via cleanup_rx_interrupts() / related cleanup helpers).
Also validate descriptor length against skb tailroom before skb_put(), and after napi_complete_done() on the budget-exhausted shutdown path return a value less than budget so NAPI does not immediately reschedule.
---
Changes in v4:
- Skip invalid correlators in poll instead of spinning (no WARN_ON; rate-limited message; advance ring when harvest cannot map).
- Validate descriptor length against skb tailroom before skb_put().
- After napi_complete_done() on the budget-exhausted shutdown path, return a value less than budget so NAPI does not immediately reschedule.
- Align KUnit comments with correlator validation (no WARN_ON).
drivers/net/ethernet/ibm/ibmveth.c | 109 ++++++++++++++++++++++------- 1 file changed, 85 insertions(+), 24 deletions(-)
drivers/net/ethernet/ibm/ibmveth.c +85 -24
@@ -1376,6 +1376,24 @@ ibmveth_free_single_rx_queue(struct ibmveth_adapter *adapter, int queue_idx)netdev_dbg (adapter->netdev,"Freed queue %d resources\n" , queue_idx); } +static bool ibmveth_rxq_correlator_valid (struct ibmveth_adapter *adapter, +int queue_index,u64 correlator) +{ +unsigned int pool = correlator >>32 ; +unsigned int index = correlator &0xffffffffUL ; + +return pool <IBMVETH_NUM_BUFF_POOLS && + index < adapter->rx_buff_pool[queue_index][pool].size; +} + +static void ibmveth_rxq_advance (struct ibmveth_rx_q *rxq) +{ +if (++rxq->index == rxq->num_slots) { + rxq->index =0 ; + rxq->toggle = !rxq->toggle; + } +} +/** * ibmveth_remove_buffer_from_pool - remove a buffer from a pool * @adapter: adapter instance
@@ -1397,17 +1415,12 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,unsigned int free_index;struct sk_buff *skb; -if (WARN_ON (pool >=IBMVETH_NUM_BUFF_POOLS ) || -WARN_ON (index >= adapter->rx_buff_pool[queue_index][pool].size)) { -schedule_work (&adapter->work); +if (!ibmveth_rxq_correlator_valid (adapter, queue_index, correlator))return -EINVAL ; - } skb = adapter->rx_buff_pool[queue_index][pool].skbuff[index]; -if (WARN_ON (!skb)) { -schedule_work (&adapter->work); +if (!skb)return -EFAULT ; - }/* if we are going to reuse the buffer then keep the pointers around * but mark index as available. replenish will see the skb pointer and
@@ -1452,11 +1465,8 @@ ibmveth_rxq_get_buffer(struct ibmveth_adapter *adapter,unsigned int pool = correlator >>32 ;unsigned int index = correlator &0xffffffffUL ; -if (WARN_ON (pool >=IBMVETH_NUM_BUFF_POOLS ) || -WARN_ON (index >= adapter->rx_buff_pool[queue_index][pool].size)) { -schedule_work (&adapter->work); +if (!ibmveth_rxq_correlator_valid (adapter, queue_index, correlator))return NULL ; - }return adapter->rx_buff_pool[queue_index][pool].skbuff[index]; }
@@ -1483,14 +1493,15 @@ static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter, cor = rxq->queue_addr[rxq->index].correlator; rc =ibmveth_remove_buffer_from_pool (adapter, cor, queue_index, reuse); -if (unlikely (rc)) +if (unlikely (rc)) { +if (rc == -EINVAL || rc == -EFAULT ) + goto advance;return rc; - -if (++rxq->index == rxq->num_slots) { - rxq->index =0 ; - rxq->toggle = !rxq->toggle; } +advance: +ibmveth_rxq_advance (rxq); +return 0 ; }
@@ -3092,11 +3103,19 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)if (WARN_ON (queue_index <0 || queue_index >= adapter->num_rx_queues))return 0 ; +if (!netif_running (netdev) ||napi_disable_pending (napi)) { +napi_complete_done (napi,0 ); +return 0 ; + } +if (adapter->rx_qstats) adapter->rx_qstats[queue_index].polls++; restart_poll:while (frames_processed < budget) { +if (!netif_running (netdev) ||napi_disable_pending (napi)) +break ; +if (!ibmveth_rxq_pending_buffer (adapter, queue_index))break ;
@@ -3126,8 +3145,45 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)__sum16 iph_check =0 ; skb =ibmveth_rxq_get_buffer (adapter, queue_index); -if (unlikely (!skb)) -break ; +if (unlikely (!skb)) { +if (net_ratelimit ()) +netdev_err (netdev, +"bad correlator on queue %d, skipping slot\n" , + queue_index); +if (adapter->rx_qstats) + adapter->rx_qstats[queue_index] + .invalid_buffers++; +else + adapter->rx_invalid_buffer++; + rc =ibmveth_rxq_harvest_buffer (adapter, + queue_index, + true); +if (unlikely (rc)) +break ; +continue ; + } + +if (unlikely ((unsigned int )offset + + (unsigned int )length > +skb_tailroom (skb))) { +if (net_ratelimit ()) +netdev_err (netdev, +"RX frame %u+%u exceeds buffer %u on queue %d, dropping\n" , + offset, length, +skb_tailroom (skb), + queue_index); +if (adapter->rx_qstats) + adapter->rx_qstats[queue_index] + .invalid_buffers++; +else + adapter->rx_invalid_buffer++; + rc =ibmveth_rxq_harvest_buffer (adapter, + queue_index, + true); +if (unlikely (rc)) +break ; +continue ; + }/* if the large packet bit is set in the rx queue * descriptor, the mss will be written by PHYP eight
@@ -3206,8 +3262,14 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)ibmveth_replenish_task (adapter, queue_index); -if (frames_processed == budget) +if (frames_processed == budget) { +if (!netif_running (netdev) ||napi_disable_pending (napi)) { +napi_complete_done (napi, frames_processed); +/* After complete_done, must not return full budget. */ +return frames_processed ? frames_processed -1 :0 ; + } goto out; + }if (!napi_complete_done (napi, frames_processed)) goto out;
@@ -3225,6 +3287,8 @@ static int ibmveth_poll(struct napi_struct *napi, int budget) }if (ibmveth_rxq_pending_buffer (adapter, queue_index) && +netif_running (netdev) && + !napi_disable_pending (napi) &&napi_schedule (napi)) { lpar_rc =ibmveth_disable_irq (adapter, queue_index);WARN_ON (lpar_rc !=H_SUCCESS );
@@ -4014,8 +4078,7 @@ static void ibmveth_reset_kunit(struct work_struct *w)* @test: pointer to kunit structure * * Tests the error returns from ibmveth_remove_buffer_from_pool. -* ibmveth_remove_buffer_from_pool also calls WARN_ON, so dmesg should be -* checked to see that these warnings happened. +* Bad correlators return -EINVAL/-EFAULT (no WARN_ON). * * Return: void */
@@ -4071,9 +4134,7 @@ static void ibmveth_remove_buffer_from_pool_test(struct kunit *test)* ibmveth_rxq_get_buffer_test - unit test for ibmveth_rxq_get_buffer * @test: pointer to kunit structure * -* Tests ibmveth_rxq_get_buffer. ibmveth_rxq_get_buffer also calls WARN_ON for -* the NULL returns, so dmesg should be checked to see that these warnings -* happened. +* Tests ibmveth_rxq_get_buffer invalid correlator returns NULL without WARN. * * Return: void */
Signature · 1 line
2.50.1 (Apple Git-155)
Jakub Kicinski
maintainerRe: [PATCH net-next v4 14/14] ibmveth: Fix MQ RX poll and shutdown hangs after queue resize
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
ibmveth: Fix MQ RX poll and shutdown hangs after queue resize
This changes ibmveth_poll() to skip RX slots whose correlator cannot be mapped to a buffer pool instead of retrying them forever, completes NAPI when the interface is going down, and adds a descriptor length check before skb_put(). The WARN_ON plus adapter reset escalation in the correlator validation paths is replaced by a rate-limited message.
No Link: tags present in the commit header.
A few questions below.
[Severity: Low]
Would it make sense to split this into separate patches with individual
Fixes: tags? There appear to be four independent changes here:
- skipping bad RX correlators and dropping the WARN_ON/reset escalation
- completing NAPI when the interface is down or napi_disable is pending
- validating the descriptor length against skb tailroom before skb_put()
- changing the return value on the budget-exhausted shutdown path
There is no Fixes: tag, so stable maintainers cannot pick up the hang fix without also picking up the policy change that removes the reset based recovery, and the skb_put() hardening is not visible as a fix on its own.
Quoted 7 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0@@ -1376,6 +1376,24 @@ ibmveth_free_single_rx_queue(struct ibmveth_adapter *adapter, int queue_idx)netdev_dbg (adapter->netdev,"Freed queue %d resources\n" , queue_idx); }
drivers/net/ethernet/ibm/ibmveth.c ibmveth_free_single_rx_queue(struct ibmveth_adapter *adapter, int queue_idx) · line 1387
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +9
-0+static bool ibmveth_rxq_correlator_valid (struct ibmveth_adapter *adapter, +int queue_index,u64 correlator) +{ +unsigned int pool = correlator >>32 ; +unsigned int index = correlator &0xffffffffUL ; + +return pool <IBMVETH_NUM_BUFF_POOLS && + index < adapter->rx_buff_pool[queue_index][pool].size; +}
[Severity: High]
This isn't a bug introduced by this patch, since the pre-existing WARN_ON checks were also bounds-only, but can a correlator naming an inactive or already freed pool still pass ibmveth_rxq_correlator_valid() and then dereference a NULL array?
ibmveth_init_buffer_pool() sets pool->size for all pools regardless of active state, while ibmveth_alloc_queue_buffer_pools() skips inactive pools, and ibmveth_free_buffer_pool() clears the arrays without clearing size or active:
kfree(pool->free_map);
pool->free_map = NULL;
...
if (pool->skbuff) {
kfree(pool->skbuff);
pool->skbuff = NULL;
}With the defaults in ibmveth.h:
static int pool_count[] = { 256, 512, 256, 256, 256 };
static int pool_active[] = { 1, 1, 0, 0, 0};a correlator of pool=2, index=0 satisfies both conditions of the new helper, and both users then index a NULL pointer before the !skb test can help:
return adapter->rx_buff_pool[queue_index][pool].skbuff[index];
and
skb = adapter->rx_buff_pool[queue_index][pool].skbuff[index];
Should the helper also require pool->active and non-NULL skbuff/dma_addr/ free_map, or should ibmveth_free_buffer_pool() clear pool->size? The same window seems to apply to pools torn down by ethtool -L scale-down, an MTU change, or sysfs deactivation.
[Severity: Low]
This is a pre-existing weakness rather than something new here, but the helper only bounds-checks, so a stale correlator that happens to land in range after an ethtool -L resize is treated as a live frame. With reuse=false the buffer can be handed to the stack while the hypervisor may still own it, and with reuse=true the index is pushed onto free_map a second time. Is there any generation counter or per-queue tag available in the correlator that could distinguish a stale descriptor from a current one?
Quoted 19 lines
drivers/net/ethernet/ibm/ibmveth.c +9
-3+ +static void ibmveth_rxq_advance (struct ibmveth_rx_q *rxq) +{ +if (++rxq->index == rxq->num_slots) { + rxq->index =0 ; + rxq->toggle = !rxq->toggle; + } +} +/** * ibmveth_remove_buffer_from_pool - remove a buffer from a pool * @adapter: adapter instance @@ -1397,17 +1415,12 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,unsigned int free_index;struct sk_buff *skb; -if (WARN_ON (pool >=IBMVETH_NUM_BUFF_POOLS ) || -WARN_ON (index >= adapter->rx_buff_pool[queue_index][pool].size)) { -schedule_work (&adapter->work);
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter, · line 1410
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-4+if (!ibmveth_rxq_correlator_valid (adapter, queue_index, correlator))return -EINVAL ; - } skb = adapter->rx_buff_pool[queue_index][pool].skbuff[index]; -if (WARN_ON (!skb)) { -schedule_work (&adapter->work); +if (!skb)return -EFAULT ; - }
[Severity: Medium]
The schedule_work(&adapter->work) escalation is dropped here and in ibmveth_rxq_get_buffer(), so nothing bounds the condition any more. The commit message describes the new skip policy but does not mention that all recovery escalation is gone.
The previous reset performed a close/open cycle, which issued h_free_logical_lan and re-registered the logical LAN, flushing every buffer registration the hypervisor still held. If the bad correlator exists because PHYP still holds buffers from a pool that ibmveth_free_buffer_pool() already unmapped and freed during an ethtool -L resize:
dma_unmap_single(&adapter->vdev->dev, pool->dma_addr[i], pool->buff_size, DMA_FROM_DEVICE); dev_kfree_skb_any(skb);
can the hypervisor keep writing into those freed pages indefinitely now that the driver only logs and advances?
Separately, this also folds together two different classes: -EINVAL for an out-of-range correlator, and -EFAULT where pool and index are in range but skbuff[index] is NULL, which indicates driver/hypervisor state desync. Is silently skipping the -EFAULT case intended?
And if the descriptor's correlator belongs to a different queue's pool, the skip never reclaims that queue's slot, so that pool's available count stays inflated and ibmveth_replenish_task() stops replenishing it:
if (pool->active && pool->free_map && (atomic_read(&pool->available) < pool->threshold))
Quoted 26 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-6/* if we are going to reuse the buffer then keep the pointers around * but mark index as available. replenish will see the skb pointer and @@ -1452,11 +1465,8 @@ ibmveth_rxq_get_buffer(struct ibmveth_adapter *adapter,unsigned int pool = correlator >>32 ;unsigned int index = correlator &0xffffffffUL ; -if (WARN_ON (pool >=IBMVETH_NUM_BUFF_POOLS ) || -WARN_ON (index >= adapter->rx_buff_pool[queue_index][pool].size)) { -schedule_work (&adapter->work); +if (!ibmveth_rxq_correlator_valid (adapter, queue_index, correlator))return NULL ; - }return adapter->rx_buff_pool[queue_index][pool].skbuff[index]; }@@ -1483,14 +1493,15 @@ static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter, cor = rxq->queue_addr[rxq->index].correlator; rc =ibmveth_remove_buffer_from_pool (adapter, cor, queue_index, reuse); -if (unlikely (rc)) +if (unlikely (rc)) { +if (rc == -EINVAL || rc == -EFAULT ) + goto advance;return rc; -
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter, · line 1506
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +3
-3-if (++rxq->index == rxq->num_slots) { - rxq->index =0 ; - rxq->toggle = !rxq->toggle; } +advance: +ibmveth_rxq_advance (rxq); +return 0 ; }
[Severity: Low]
Can ibmveth_rxq_harvest_buffer() still return non-zero after this change? ibmveth_remove_buffer_from_pool() has exactly three returns: -EINVAL, -EFAULT and 0, and both error codes now go to the advance label, so "return rc;" looks unreachable.
That would make all five "if (unlikely(rc)) break;" checks in ibmveth_poll(), including the two added by this patch, dead code, along with the harvest failure path in ibmveth_drain_rx_queue():
rc = ibmveth_rxq_harvest_buffer(adapter, queue_index, true);
if (rc) {
netdev_err(netdev,
"Failed to harvest buffer from queue %d during drain: %d\n",
queue_index, rc);
break;
}
drained++;Should unmappable slots encountered during drain still be counted as drained++ with no diagnostic? The kernel-doc above harvest still documents "other - non-zero return from ibmveth_remove_buffer_from_pool", which no longer matches the implementation.
Quoted 21 lines
drivers/net/ethernet/ibm/ibmveth.c +8
-0@@ -3092,11 +3103,19 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)if (WARN_ON (queue_index <0 || queue_index >= adapter->num_rx_queues))return 0 ; +if (!netif_running (netdev) ||napi_disable_pending (napi)) { +napi_complete_done (napi,0 ); +return 0 ; + } +if (adapter->rx_qstats) adapter->rx_qstats[queue_index].polls++; restart_poll:while (frames_processed < budget) { +if (!netif_running (netdev) ||napi_disable_pending (napi)) +break ; +if (!ibmveth_rxq_pending_buffer (adapter, queue_index))break ;
[ ... ]
Quoted 11 lines
drivers/net/ethernet/ibm/ibmveth.c +5
-2@@ -3126,8 +3145,45 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)__sum16 iph_check =0 ; skb =ibmveth_rxq_get_buffer (adapter, queue_index); -if (unlikely (!skb)) -break ; +if (unlikely (!skb)) { +if (net_ratelimit ()) +netdev_err (netdev, +"bad correlator on queue %d, skipping slot\n" , + queue_index);
[ ... ]
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_poll(struct napi_struct *napi, int budget) · line 3168
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-0+ +if (unlikely ((unsigned int )offset + + (unsigned int )length > +skb_tailroom (skb))) {
[Severity: High]
This is not a regression from this patch, since there was no length check at all before, but can this check be bypassed by 32-bit wraparound?
Both operands are cast to unsigned int, so the sum is modulo 2^32. Unlike offset, length is not masked; ibmveth_rxq_frame_length() returns the whole descriptor field:
return be32_to_cpu(rxq->queue_addr[rxq->index].length);
and in ibmveth.h that field is a full __be32 written by the hypervisor, while only offset gets IBMVETH_RXQ_OFF_MASK (0x0000FFFF).
For a corrupted length like 0xFFFFFFF0 with a small offset, the addition wraps to a small value and the guard does not fire. The copybreak test compares against "static unsigned int rx_copybreak", so 0xFFFFFFF0 is not less than rx_copybreak and no copy skb is allocated, and execution reaches:
skb_put(skb, length);
which trips skb_over_panic() -> BUG(). Would an overflow-safe form work better here, for example:
if (offset > tailroom || length > tailroom - offset)
or validating length against pool->buff_size?
[ ... ]
Quoted 17 lines
drivers/net/ethernet/ibm/ibmveth.c +7
-1@@ -3206,8 +3262,14 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)ibmveth_replenish_task (adapter, queue_index); -if (frames_processed == budget) +if (frames_processed == budget) { +if (!netif_running (netdev) ||napi_disable_pending (napi)) { +napi_complete_done (napi, frames_processed); +/* After complete_done, must not return full budget. */ +return frames_processed ? frames_processed -1 :0 ; + } goto out; + }if (!napi_complete_done (napi, frames_processed)) goto out;
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_poll(struct napi_struct *napi, int budget) · line 3294
Quoted 8 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-0@@ -3225,6 +3287,8 @@ static int ibmveth_poll(struct napi_struct *napi, int budget) }if (ibmveth_rxq_pending_buffer (adapter, queue_index) && +netif_running (netdev) && + !napi_disable_pending (napi) &&napi_schedule (napi)) { lpar_rc =ibmveth_disable_irq (adapter, queue_index);WARN_ON (lpar_rc !=H_SUCCESS );
[Severity: High]
This isn't a bug introduced by this patch, as the napi_complete_done() -> ibmveth_enable_irq() -> ibmveth_rxq_pending_buffer() sequence is unchanged, but do the new guards actually close the shutdown window the commit message describes?
The new mid-loop break also lands here with frames_processed < budget, so the quiesce case still reaches:
if (!napi_complete_done(napi, frames_processed)) goto out;
Clearing NAPI_STATE_SCHED there is what napi_disable() waits for, so ibmveth_close() can proceed past ibmveth_cleanup_rx_interrupts() into ibmveth_cleanup_rx_resources():
if (adapter->rx_queue[i].queue_addr) {
dma_free_coherent(dev, adapter->rx_queue[i].queue_len,
adapter->rx_queue[i].queue_addr,
adapter->rx_queue[i].queue_dma);
adapter->rx_queue[i].queue_addr = NULL;while the still running poll executes:
lpar_rc = ibmveth_enable_irq(adapter, queue_index);
re-arming the PHYP interrupt that close just masked, and then evaluates ibmveth_rxq_pending_buffer(), which reads rxq->queue_addr[rxq->index].flags_off - a NULL dereference or a read of freed coherent memory. ibmveth_close() has no synchronize_net() between napi_disable()/free_irq() and the frees, unlike the scale-down path.
Note the new netif_running()/napi_disable_pending() terms are evaluated after ibmveth_rxq_pending_buffer() in the same condition, so they cannot prevent that access; they only suppress the following ibmveth_disable_irq().
Related question on that suppression: in the window where __LINK_STATE_START is already cleared by __dev_close_many() but NAPI_STATE_DISABLE is not yet set, the pre-patch code re-masked PHYP via the napi_schedule() branch. With the new guards, poll now returns leaving delivery unmasked into the napi_disable()/free_irq() window, and the interrupt handler does not mask either when napi_schedule_prep() fails:
if (napi_schedule_prep(napi)) {
lpar_rc = ibmveth_disable_irq(adapter, qindex);
WARN_ON(lpar_rc != H_SUCCESS);
__napi_schedule(napi);
}Can that leave the queue interrupt storming until free_irq()?
[ ... ]
Re: [PATCH net-next v4 14/14] ibmveth: Fix MQ RX poll and shutdown hangs after queue resize
Quoted 11 lines · Jakub Kicinski
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
ibmveth: Fix MQ RX poll and shutdown hangs after queue resizeThis changes
ibmveth_poll()to skip RX slots whose correlator cannot be mapped to a buffer pool instead of retrying them forever, completes NAPI when the interface is going down, and adds a descriptor length check beforeskb_put(). TheWARN_ONplus adapter reset escalation in the correlator validation paths is replaced by a rate-limited message.
Quoted 3 lines
No Link: tags present in the commit header.
A few questions below.
Hi Jakub,
Thanks for the review and follow-up.
Quoted 5 lines
[Severity: Low]
Would it make sense to split this into separate patches with individual
Fixes: tags? There appear to be four independent changes here:
Quoted 8 lines
- skipping bad RX correlators and dropping the WARN_ON/reset escalation
- completing NAPI when the interface is down ornapi_disableis pending
- validating the descriptor length against skb tailroom beforeskb_put()
- changing the return value on the budget-exhausted shutdown pathThere is no Fixes: tag, so stable maintainers cannot pick up the hang fix without also picking up the policy change that removes the reset based recovery, and the
skb_put()hardening is not visible as a fix on its own.
I’ll fold these back into the introducing patches rather than keep a mixed trailer or invent broad Fixes: tags.
These are MQ-series issues in the new poll/resize/teardown paths, not
fixes for older pre-MQ behavior, so I do not think they should be presented as separate net backports.
The better restack is to fold each piece into the patch that introduces the behavior:
- P05: mask IRQ when napi_schedule_prep() fails
- P09: stopping guards, wrap-safe length validation, honest harvest
- P12: teardown-first resize fix, correlator liveness, residual skip+schedule_work handling
So I’ll restack this into the owning patches for v5; P14 may disappear entirely if nothing independent remains after the fold-back, rather than keeping one mixed patch with broad Fixes: tags.
Quoted 47 lines
Quoted 7 lines
drivers/net/ethernet/ibm/ibmveth.c +0
-0@@ -1376,6 +1376,24 @@ ibmveth_free_single_rx_queue(struct ibmveth_adapter *adapter, int queue_idx)netdev_dbg (adapter->netdev,"Freed queue %d resources\n" , queue_idx); }drivers/net/ethernet/ibm/ibmveth.c ibmveth_free_single_rx_queue(struct ibmveth_adapter *adapter, int queue_idx) · line 1387
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +9
-0+static bool ibmveth_rxq_correlator_valid (struct ibmveth_adapter *adapter, +int queue_index,u64 correlator) +{ +unsigned int pool = correlator >>32 ; +unsigned int index = correlator &0xffffffffUL ; + +return pool <IBMVETH_NUM_BUFF_POOLS && + index < adapter->rx_buff_pool[queue_index][pool].size; +}[Severity: High]
This isn't a bug introduced by this patch, since the pre-existingWARN_ONchecks were also bounds-only, but can a correlator naming an inactive or already freed pool still passibmveth_rxq_correlator_valid()and then dereference a NULL array?
ibmveth_init_buffer_pool()setspool->sizefor all pools regardless of active state, whileibmveth_alloc_queue_buffer_pools()skips inactive pools, andibmveth_free_buffer_pool()clears the arrays without clearing size or active:kfree(pool->free_map); pool->free_map = NULL; ... if (pool->skbuff) { kfree(pool->skbuff); pool->skbuff = NULL; }With the defaults in
ibmveth.h:static int pool_count[] = { 256, 512, 256, 256, 256 }; static int pool_active[] = { 1, 1, 0, 0, 0};a correlator of pool=2, index=0 satisfies both conditions of the new helper, and both users then index a NULL pointer before the !skb test can help:
return adapter->rx_buff_pool[queue_index][pool].skbuff[index];
Quoted 8 lines
and
skb = adapter->rx_buff_pool[queue_index][pool].skbuff[index];Should the helper also require
pool->activeand non-NULL skbuff/dma_addr/free_map, or shouldibmveth_free_buffer_pool()clearpool->size? The same window seems to apply to pools torn down by ethtool -L scale-down, an MTU change, or sysfs deactivation.
Yes. Bounds-only validation is too weak here.
I’ll make the helper require an active pool plus non-NULL skbuff/free_map state, and clear the freed pool state so it no longer looks in-range after teardown. That belongs with the resize teardown story, because the real bug is freeing pool state while stale correlators can still arrive.
Quoted 9 lines
[Severity: Low]
This is a pre-existing weakness rather than something new here, but the helper only bounds-checks, so a stale correlator that happens to land in range after an ethtool -L resize is treated as a live frame. With reuse=false the buffer can be handed to the stack while the hypervisor may still own it, and with reuse=true the index is pushed ontofree_mapa second time. Is there any generation counter or per-queue tag available in the correlator that could distinguish a stale descriptor from a current one?
No, there is no generation field in the correlator today.
I don’t plan to add one in this series. I think the right fix here is to close the ownership/teardown hole in P12 so PHYP is no longer posting into pool state we already tore down. With that fixed, the stronger liveness checks should be enough as belt-and-braces, without adding a larger generation/tag change in this cycle.
Quoted 54 lines
Quoted 19 lines
drivers/net/ethernet/ibm/ibmveth.c +9
-3+ +static void ibmveth_rxq_advance (struct ibmveth_rx_q *rxq) +{ +if (++rxq->index == rxq->num_slots) { + rxq->index =0 ; + rxq->toggle = !rxq->toggle; + } +} +/** * ibmveth_remove_buffer_from_pool - remove a buffer from a pool * @adapter: adapter instance @@ -1397,17 +1415,12 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,unsigned int free_index;struct sk_buff *skb; -if (WARN_ON (pool >=IBMVETH_NUM_BUFF_POOLS ) || -WARN_ON (index >= adapter->rx_buff_pool[queue_index][pool].size)) { -schedule_work (&adapter->work);drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter, · line 1410
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-4+if (!ibmveth_rxq_correlator_valid (adapter, queue_index, correlator))return -EINVAL ; - } skb = adapter->rx_buff_pool[queue_index][pool].skbuff[index]; -if (WARN_ON (!skb)) { -schedule_work (&adapter->work); +if (!skb)return -EFAULT ; - }[Severity: Medium]
Theschedule_work(&adapter->work)escalation is dropped here and inibmveth_rxq_get_buffer(), so nothing bounds the condition any more. The commit message describes the new skip policy but does not mention that all recovery escalation is gone.The previous reset performed a close/open cycle, which issued
h_free_logical_lanand re-registered the logical LAN, flushing every buffer registration the hypervisor still held. If the bad correlator exists because PHYP still holds buffers from a pool thatibmveth_free_buffer_pool()already unmapped and freed during an ethtool -L resize:dma_unmap_single(&adapter->vdev->dev, pool->dma_addr[i], pool->buff_size, DMA_FROM_DEVICE); dev_kfree_skb_any(skb);can the hypervisor keep writing into those freed pages indefinitely now that the driver only logs and advances?
Separately, this also folds together two different classes:
-EINVALfor an out-of-range correlator, and-EFAULTwhere pool and index are in range but skbuff[index] is NULL, which indicates driver/hypervisor state desync. Is silently skipping the-EFAULTcase intended?
Quoted 6 lines
And if the descriptor's correlator belongs to a different queue's pool, the skip never reclaims that queue's slot, so that pool's available count stays inflated and
ibmveth_replenish_task()stops replenishing it:if (pool->active && pool->free_map && (atomic_read(&pool->available) < pool->threshold))
Agreed — skip-only was not sufficient on its own.
The real fix belongs in P12, in the earlier resize/teardown path:
drain, deregister with h_free_logical_lan_queue(), then unmap/free, so PHYP ownership is released before the pool memory goes away. After that, I’ll keep residual bad-slot handling with rate-limited logging plus schedule_work() as secondary recovery, rather than treating skip-and-advance as the primary answer.
Quoted 55 lines
Quoted 26 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-6/* if we are going to reuse the buffer then keep the pointers around * but mark index as available. replenish will see the skb pointer and @@ -1452,11 +1465,8 @@ ibmveth_rxq_get_buffer(struct ibmveth_adapter *adapter,unsigned int pool = correlator >>32 ;unsigned int index = correlator &0xffffffffUL ; -if (WARN_ON (pool >=IBMVETH_NUM_BUFF_POOLS ) || -WARN_ON (index >= adapter->rx_buff_pool[queue_index][pool].size)) { -schedule_work (&adapter->work); +if (!ibmveth_rxq_correlator_valid (adapter, queue_index, correlator))return NULL ; - }return adapter->rx_buff_pool[queue_index][pool].skbuff[index]; }@@ -1483,14 +1493,15 @@ static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter, cor = rxq->queue_addr[rxq->index].correlator; rc =ibmveth_remove_buffer_from_pool (adapter, cor, queue_index, reuse); -if (unlikely (rc)) +if (unlikely (rc)) { +if (rc == -EINVAL || rc == -EFAULT ) + goto advance;return rc; -drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter, · line 1506
Quoted 10 lines
drivers/net/ethernet/ibm/ibmveth.c +3
-3-if (++rxq->index == rxq->num_slots) { - rxq->index =0 ; - rxq->toggle = !rxq->toggle; } +advance: +ibmveth_rxq_advance (rxq); +return 0 ; }[Severity: Low]
Canibmveth_rxq_harvest_buffer()still return non-zero after this change?ibmveth_remove_buffer_from_pool()has exactly three returns:-EINVAL,-EFAULTand 0, and both error codes now go to the advance label, so "return rc;" looks unreachable.That would make all five "if (unlikely(rc)) break;" checks in
ibmveth_poll(), including the two added by this patch, dead code, along with the harvest failure path inibmveth_drain_rx_queue():rc = ibmveth_rxq_harvest_buffer(adapter, queue_index, true); if (rc) { netdev_err(netdev, "Failed to harvest buffer from queue %d during drain: %d\n", queue_index, rc); break; } drained++;
Quoted 4 lines
Should unmappable slots encountered during drain still be counted as drained++ with no diagnostic? The kernel-doc above harvest still documents "other - non-zero return from
ibmveth_remove_buffer_from_pool", which no longer matches the implementation.
As mailed, not for -EINVAL/-EFAULT, so that part was misleading.
I’ll make harvest advance and return rc for -EINVAL/-EFAULT, with poll/drain continuing past those slots without treating them as successful receives. I’ll fold that into the earlier poll logic patch rather than leaving it here as a trailer cleanup.
Quoted 60 lines
Quoted 21 lines
drivers/net/ethernet/ibm/ibmveth.c +8
-0@@ -3092,11 +3103,19 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)if (WARN_ON (queue_index <0 || queue_index >= adapter->num_rx_queues))return 0 ; +if (!netif_running (netdev) ||napi_disable_pending (napi)) { +napi_complete_done (napi,0 ); +return 0 ; + } +if (adapter->rx_qstats) adapter->rx_qstats[queue_index].polls++; restart_poll:while (frames_processed < budget) { +if (!netif_running (netdev) ||napi_disable_pending (napi)) +break ; +if (!ibmveth_rxq_pending_buffer (adapter, queue_index))break ;[ ... ]
Quoted 11 lines
drivers/net/ethernet/ibm/ibmveth.c +5
-2@@ -3126,8 +3145,45 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)__sum16 iph_check =0 ; skb =ibmveth_rxq_get_buffer (adapter, queue_index); -if (unlikely (!skb)) -break ; +if (unlikely (!skb)) { +if (net_ratelimit ()) +netdev_err (netdev, +"bad correlator on queue %d, skipping slot\n" , + queue_index);[ ... ]
drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_poll(struct napi_struct *napi, int budget) · line 3168
Quoted 4 lines
drivers/net/ethernet/ibm/ibmveth.c +4
-0+ +if (unlikely ((unsigned int )offset + + (unsigned int )length > +skb_tailroom (skb))) {[Severity: High]
This is not a regression from this patch, since there was no length check at all before, but can this check be bypassed by 32-bit wraparound?Both operands are cast to unsigned int, so the sum is modulo 2^32. Unlike offset, length is not masked;
ibmveth_rxq_frame_length()returns the whole descriptor field:return be32_to_cpu(rxq->queue_addr[rxq->index].length);and in
ibmveth.hthat field is a full__be32written by the hypervisor, while only offset getsIBMVETH_RXQ_OFF_MASK(0x0000FFFF).For a corrupted length like
0xFFFFFFF0with a small offset, the addition wraps to a small value and the guard does not fire. The copybreak test compares against "static unsigned intrx_copybreak", so0xFFFFFFF0is not less thanrx_copybreakand no copy skb is allocated, and execution reaches:skb_put(skb, length);
Quoted 8 lines
which trips
skb_over_panic()->BUG(). Would an overflow-safe form work better here, for example:if (offset > tailroom || length > tailroom - offset)or validating length against
pool->buff_size?[ ... ]
Yes. The sum-based check can wrap and still let skb_put() overrun.
I’ll switch that to the overflow-safe form, i.e. reject if offset is already past tailroom or length exceeds the remaining room. That should live with the patch that introduces the MQ poll path, not as a separate end-of-series hardening patch.
Quoted 75 lines
Quoted 17 lines
drivers/net/ethernet/ibm/ibmveth.c +7
-1@@ -3206,8 +3262,14 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)ibmveth_replenish_task (adapter, queue_index); -if (frames_processed == budget) +if (frames_processed == budget) { +if (!netif_running (netdev) ||napi_disable_pending (napi)) { +napi_complete_done (napi, frames_processed); +/* After complete_done, must not return full budget. */ +return frames_processed ? frames_processed -1 :0 ; + } goto out; + }if (!napi_complete_done (napi, frames_processed)) goto out;drivers/net/ethernet/ibm/ibmveth.c static int ibmveth_poll(struct napi_struct *napi, int budget) · line 3294
Quoted 8 lines
drivers/net/ethernet/ibm/ibmveth.c +2
-0@@ -3225,6 +3287,8 @@ static int ibmveth_poll(struct napi_struct *napi, int budget) }if (ibmveth_rxq_pending_buffer (adapter, queue_index) && +netif_running (netdev) && + !napi_disable_pending (napi) &&napi_schedule (napi)) { lpar_rc =ibmveth_disable_irq (adapter, queue_index);WARN_ON (lpar_rc !=H_SUCCESS );[Severity: High]
This isn't a bug introduced by this patch, as thenapi_complete_done()->ibmveth_enable_irq()->ibmveth_rxq_pending_buffer()sequence is unchanged, but do the new guards actually close the shutdown window the commit message describes?The new mid-loop break also lands here with
frames_processed< budget, so the quiesce case still reaches:if (!napi_complete_done(napi, frames_processed)) goto out;Clearing
NAPI_STATE_SCHEDthere is whatnapi_disable()waits for, soibmveth_close()can proceed pastibmveth_cleanup_rx_interrupts()intoibmveth_cleanup_rx_resources():if (adapter->rx_queue[i].queue_addr) { dma_free_coherent(dev, adapter->rx_queue[i].queue_len, adapter->rx_queue[i].queue_addr, adapter->rx_queue[i].queue_dma); adapter->rx_queue[i].queue_addr = NULL;while the still running poll executes:
lpar_rc = ibmveth_enable_irq(adapter, queue_index);re-arming the PHYP interrupt that close just masked, and then evaluates
ibmveth_rxq_pending_buffer(), which reads rxq->queue_addr[rxq->index].flags_off - a NULL dereference or a read of freed coherent memory.ibmveth_close()has nosynchronize_net()between napi_disable()/free_irq() and the frees, unlike the scale-down path.Note the new netif_running()/napi_disable_pending() terms are evaluated after
ibmveth_rxq_pending_buffer()in the same condition, so they cannot prevent that access; they only suppress the followingibmveth_disable_irq().Related question on that suppression: in the window where
__LINK_STATE_STARTis already cleared by__dev_close_many()butNAPI_STATE_DISABLEis not yet set, the pre-patch code re-masked PHYP via thenapi_schedule()branch. With the new guards, poll now returns leaving delivery unmasked into the napi_disable()/free_irq() window, and the interrupt handler does not mask either whennapi_schedule_prep()fails:if (napi_schedule_prep(napi)) { lpar_rc = ibmveth_disable_irq(adapter, qindex); WARN_ON(lpar_rc != H_SUCCESS); __napi_schedule(napi); }
Quoted 3 lines
Can that leave the queue interrupt storming until
free_irq()?[ ... ]
Yes, the mailed guards were incomplete.
The stopping path needs to complete without re-enabling IRQs once close or napi_disable is in progress.
The schedule_rx_queue() path also still needs to mask the interrupt when napi_schedule_prep() fails, so the queue does not stay unmasked into the free_irq() window.
I’ll also keep synchronize_net() in close after RX IRQ/NAPI teardown and before freeing queue resources, so any poll instance that already passed the stopping checks is drained before the frees.
Thanks
Mingming
Jakub Kicinski
maintainerRe: [PATCH net-next v4 14/14] ibmveth: Fix MQ RX poll and shutdown hangs after queue resize
In reply to Mingming CaoQuoted 11 lines · On Thu, 30 Jul 2026 17:47:17 -0700 Mingming Cao wrote:
After aggressive ethtool -L cycling, PHYP can leave a VALID RX descriptor with a correlator that no longer matches the per-queue buffer pools. Poll treated this as fatal:
ibmveth_rxq_get_buffer()WARNed and returned NULL without advancing the ring, thenrestart_pollretried the same slot forever.Advance past bad correlators instead of spinning: validate correlators without
WARN_ON, skip invalid slots in poll (count asinvalid_buffers), and advance the RX ring whenremove_buffer_from_poolcannot map the correlator. Rate-limit the bad correlator message.
Quoted 8 lines
Complete NAPI when the interface is down or
napi_disableis pending so close/quiesce can finish. Do notrestart_pollin that window. Close keeps hypervisor IRQ disable beforenapi_disable(viacleanup_rx_interrupts()/ related cleanup helpers).Also validate descriptor length against skb tailroom before
skb_put(), and afternapi_complete_done()on the budget-exhausted shutdown path return a value less than budget so NAPI does not immediately reschedule.
This patch looks very odd, like there's a bug somewhere and instead of fixing it we're making the rx path more resilient. BTW minor coding note - you're adding a bunch of heavily indented code to an already long function. Please consider factoring things out. Similar comment can be applied to other patches.
Re: [PATCH net-next v4 14/14] ibmveth: Fix MQ RX poll and shutdown hangs after queue resize
Quoted 17 lines · Jakub Kicinski
Quoted 16 lines · On Thu, 30 Jul 2026 17:47:17 -0700 Mingming Cao wrote:
After aggressive ethtool -L cycling, PHYP can leave a VALID RX descriptor with a correlator that no longer matches the per-queue buffer pools. Poll treated this as fatal:
ibmveth_rxq_get_buffer()WARNed and returned NULL without advancing the ring, thenrestart_pollretried the same slot forever.Advance past bad correlators instead of spinning: validate correlators without
WARN_ON, skip invalid slots in poll (count asinvalid_buffers), and advance the RX ring whenremove_buffer_from_poolcannot map the correlator. Rate-limit the bad correlator message.Complete NAPI when the interface is down or
napi_disableis pending so close/quiesce can finish. Do notrestart_pollin that window. Close keeps hypervisor IRQ disable beforenapi_disable(viacleanup_rx_interrupts()/ related cleanup helpers).
Quoted 8 lines
Quoted 3 lines
Also validate descriptor length against skb tailroom before
skb_put(), and afternapi_complete_done()on the budget-exhausted shutdown path return a value less than budget so NAPI does not immediately reschedule.This patch looks very odd, like there's a bug somewhere and instead of fixing it we're making the rx path more resilient. BTW minor coding note - you're adding a bunch of heavily indented code to an already long function. Please consider factoring things out. Similar comment can be applied to other patches.
Agreed. The main fix here should be in the earlier resize/teardown path, not in a trailing resilience patch.
I’ll fold that back into the owning resize work: drain before deregister/unmap/free, publish the surviving queue range before freeing queue resources, and use the same teardown path on the scale-up enable_irq failure path. The poll-side skip logic should stay only as belt-and-braces after the teardown fix, not as the primary answer.
I’ll also move the new poll branches into small helpers so ibmveth_poll() stays a thin loop, but keep those helpers with the patches that introduce the logic rather than as a standalone trailer refactor.
I think the same cleanup principle applies to a few of the other patches as well, so I’ll fold helper extraction into the owning changes there too.
Thanks,
Mingming