Introduce napi queues support
Subsystems (4)
cover Introduce napi queues support
Introduce support for associating napi instances with corresponding RX and TX queue set. Add the capability to export napi information supported by the device. Extend the netdev_genl generic netlink family for netdev with napi data. The napi fields exposed are:
- napi id
- queue/queue-set (both RX and TX) associated with each napi instance
Additional napi fields such as PID association for napi thread etc. can be supported in a follow-on patch set.
This series only supports 'get' ability for retrieving napi fields (specifically, napi ids and queue[s]). The 'set' ability for setting queue[s] associated with a napi instance via netdev-genl will be submitted as a separate patch series.
Previous discussion at:
https://lore.kernel.org/netdev/[email protected]/T/#m00999652a8b4731fbdb7bf698d2e3666c65a60e7
$ ./cli.py --spec netdev.yaml --do dev-get --json='{"ifindex": 12}'
[{'ifindex': 12,
'xdp-features': {'xsk-zerocopy', 'basic', 'rx-sg', 'redirect'}},
{'napi-info': [{'napi-id': 600, 'rx-queues': [7], 'tx-queues': [7]},
{'napi-id': 599, 'rx-queues': [6], 'tx-queues': [6]},
{'napi-id': 598, 'rx-queues': [5], 'tx-queues': [5]},
{'napi-id': 597, 'rx-queues': [4], 'tx-queues': [4]},
{'napi-id': 596, 'rx-queues': [3], 'tx-queues': [3]},
{'napi-id': 595, 'rx-queues': [2], 'tx-queues': [2]},
{'napi-id': 594, 'rx-queues': [1], 'tx-queues': [1]},
{'napi-id': 593, 'rx-queues': [0], 'tx-queues': [0]}]}]---
Amritha Nambiar (4):
net: Introduce new napi fields for rx/tx queues
net: Add support for associating napi with queue[s]
netdev-genl: Introduce netdev dump ctx
netdev-genl: Add support for exposing napi info from netdev
Documentation/netlink/specs/netdev.yaml | 39 ++++ drivers/net/ethernet/intel/ice/ice_lib.c | 57 ++++++ drivers/net/ethernet/intel/ice/ice_lib.h | 4 drivers/net/ethernet/intel/ice/ice_main.c | 4 include/linux/netdevice.h | 18 ++ include/uapi/linux/netdev.h | 4 net/core/dev.c | 55 ++++++ net/core/netdev-genl.c | 261 ++++++++++++++++++++++++----- tools/include/uapi/linux/netdev.h | 4 9 files changed, 402 insertions(+), 44 deletions(-)
--
PATCH 1/4 net: Introduce new napi fields for rx/tx queues
Introduce new napi fields 'napi_rxq_list' and 'napi_txq_list' for rx and tx queue set associated with the napi and initialize them. Handle their removal as well.
This enables a mapping of each napi instance with the queue/queue-set on the corresponding irq line.
--- include/linux/netdevice.h | 7 +++++++ net/core/dev.c | 21 +++++++++++++++++++++ 2 files changed, 28 insertions(+)
include/linux/netdevice.h +7 -0
@@ -342,6 +342,11 @@ struct gro_list {*/ #define GRO_HASH_BUCKETS 8 +struct napi_queue { +struct list_head q_list; +u16 queue_index; +}; +/* * Structure for NAPI scheduling similar to tasklet but with weighting */
@@ -376,6 +381,8 @@ struct napi_struct {/* control-path-only fields follow */ struct list_head dev_list;struct hlist_node napi_hash_node; +struct list_head napi_rxq_list; +struct list_head napi_txq_list; };enum {
net/core/dev.c +21 -0
@@ -6401,6 +6401,9 @@ void netif_napi_add_weight(struct net_device *dev, struct napi_struct *napi,*/ if (dev->threaded &&napi_kthread_create (napi)) dev->threaded =0 ; + +INIT_LIST_HEAD (&napi->napi_rxq_list); +INIT_LIST_HEAD (&napi->napi_txq_list); }EXPORT_SYMBOL (netif_napi_add_weight);
@@ -6462,6 +6465,23 @@ static void flush_gro_hash(struct napi_struct *napi) } } +static void __napi_del_queue (struct napi_queue *napi_queue) +{ +list_del_rcu (&napi_queue->q_list); +kfree (napi_queue); +} + +static void napi_del_queues (struct napi_struct *napi) +{ +struct napi_queue *napi_queue, *n; + +list_for_each_entry_safe (napi_queue, n, &napi->napi_rxq_list, q_list) +__napi_del_queue (napi_queue); + +list_for_each_entry_safe (napi_queue, n, &napi->napi_txq_list, q_list) +__napi_del_queue (napi_queue); +} +/* Must be called in process context */ void __netif_napi_del (struct napi_struct *napi) {
@@ -6479,6 +6499,7 @@ void __netif_napi_del(struct napi_struct *napi)kthread_stop (napi->thread); napi->thread =NULL ; } +napi_del_queues (napi); }EXPORT_SYMBOL (__netif_napi_del);
Jakub Kicinski
maintainerRe: [net-next/RFC PATCH v1 1/4] net: Introduce new napi fields for rx/tx queues
Quoted 6 lines · On Thu, 01 Jun 2023 10:42:25 -0700 Amritha Nambiar wrote:
Introduce new napi fields '
napi_rxq_list' and 'napi_txq_list' for rx and tx queue set associated with the napi and initialize them. Handle their removal as well.This enables a mapping of each napi instance with the queue/queue-set on the corresponding irq line.
Wouldn't it be easier to store the NAPI instance pointer in the queue? That way we don't have to allocate memory.
Re: [net-next/RFC PATCH v1 1/4] net: Introduce new napi fields for rx/tx queues
Quoted 11 lines · Jakub Kicinski
Quoted 6 lines · On Thu, 01 Jun 2023 10:42:25 -0700 Amritha Nambiar wrote:
Introduce new napi fields '
napi_rxq_list' and 'napi_txq_list' for rx and tx queue set associated with the napi and initialize them. Handle their removal as well.This enables a mapping of each napi instance with the queue/queue-set on the corresponding irq line.
Wouldn't it be easier to store the NAPI instance pointer in the queue? That way we don't have to allocate memory.
Could you please elaborate on this so I have more clarity ? Are you suggesting that there's a way to avoid maintaining the list of queues in the napi struct?
The idea was for netdev-genl to extract information out of netdev->napi_list->napi. For tracking queues, we build a linked list of queues for the napi and store it in the napi_struct. This would also enable updating the napi<->queue[s] association (later with the 'set' command), i.e. remove the queue[s] from the existing napi instance that it is currently associated with and map with the new napi instance, by simply deleting from one list and adding to the new list.
Jakub Kicinski
maintainerRe: [net-next/RFC PATCH v1 1/4] net: Introduce new napi fields for rx/tx queues
Quoted 6 lines · Amritha
Quoted 5 lines · Jakub Kicinski
Quoted 4 lines · On Thu, 01 Jun 2023 10:42:25 -0700 Amritha Nambiar wrote:
Introduce new napi fields '
napi_rxq_list' and 'napi_txq_list' for rx and tx queue set associated with the napi and initialize them. Handle their removal as well.
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
This enables a mapping of each napi instance with the queue/queue-set on the corresponding irq line.
Wouldn't it be easier to store the NAPI instance pointer in the queue? That way we don't have to allocate memory.
Could you please elaborate on this so I have more clarity ?
First off, let's acknowledge the fact you're asking me for clarifications ~40 days after I sent the feedback.
Pause for self-reflection.
Okay.
Quoted 2 lines
Are you suggesting that there's a way to avoid maintaining the list of queues in the napi struct?
Yes, why not add the napi pointer to struct netdev_queue and netdev_rx_queue, specifically?
Quoted 8 lines
The idea was for netdev-genl to extract information out of
netdev->napi_list->napi. For tracking queues, we build a linked list of queues for the napi and store it in thenapi_struct. This would also enable updating the napi<->queue[s] association (later with the 'set' command), i.e. remove the queue[s] from the existing napi instance that it is currently associated with and map with the new napi instance, by simply deleting from one list and adding to the new list.
Right, my point is that each queue can only be serviced by a single NAPI at a time, so we have a 1:N relationship. It's easier to store the state on the side that's the N, rather than 1.
You can add list_head to the queue structs, if you prefer to be able to walk queues of a NAPI more efficiently (that said the head for the list is in "control path only section" of napi_struct so... I think you don't?)
Re: [net-next/RFC PATCH v1 1/4] net: Introduce new napi fields for rx/tx queues
Quoted 10 lines · Jakub Kicinski
Quoted 9 lines · Amritha
Quoted 8 lines · Jakub Kicinski
Quoted 6 lines · On Thu, 01 Jun 2023 10:42:25 -0700 Amritha Nambiar wrote:
Introduce new napi fields '
napi_rxq_list' and 'napi_txq_list' for rx and tx queue set associated with the napi and initialize them. Handle their removal as well.This enables a mapping of each napi instance with the queue/queue-set on the corresponding irq line.
Quoted 9 lines
Quoted 5 lines
Quoted 3 lines
Wouldn't it be easier to store the NAPI instance pointer in the queue? That way we don't have to allocate memory.
Could you please elaborate on this so I have more clarity ?
First off, let's acknowledge the fact you're asking me for clarifications ~40 days after I sent the feedback.
Sorry about that, my vacation to be blamed.
Quoted 10 lines
Pause for self-reflection.
Okay.
Quoted 2 lines
Are you suggesting that there's a way to avoid maintaining the list of queues in the napi struct?
Yes, why not add the napi pointer to struct
netdev_queueandnetdev_rx_queue, specifically?
Yes, this would achieve the queue<->napi binding for each queue. But when there are multiple queues for a NAPI, I would need to walk a list of queues for the NAPI.
Quoted 9 lines
Quoted 8 lines
The idea was for netdev-genl to extract information out of
netdev->napi_list->napi. For tracking queues, we build a linked list of queues for the napi and store it in thenapi_struct. This would also enable updating the napi<->queue[s] association (later with the 'set' command), i.e. remove the queue[s] from the existing napi instance that it is currently associated with and map with the new napi instance, by simply deleting from one list and adding to the new list.
Quoted 8 lines
Right, my point is that each queue can only be serviced by a single NAPI at a time, so we have a 1:N relationship. It's easier to store the state on the side that's the N, rather than 1.
You can add
list_headto the queue structs, if you prefer to be able to walk queues of a NAPI more efficiently (that said the head for the list is in "control path only section" ofnapi_structso... I think you don't?)
The napi pointer in the queue structs would give the napi<->queue mapping, I still need to walk the queues of a NAPI (when there are multiple queues for the NAPI), example:
'napi-id': 600, 'rx-queues': [7,6,5], 'tx-queues': [7,6,5]
in which case I would have a list of netdev queue structs within the napi_struct (instead of the list of queue indices that I currently have) to avoid memory allocation.
Does this sound right?
Jakub Kicinski
maintainerRe: [net-next/RFC PATCH v1 1/4] net: Introduce new napi fields for rx/tx queues
Quoted 23 lines · Amritha
Quoted 9 lines
Quoted 8 lines
The idea was for netdev-genl to extract information out of
netdev->napi_list->napi. For tracking queues, we build a linked list of queues for the napi and store it in thenapi_struct. This would also enable updating the napi<->queue[s] association (later with the 'set' command), i.e. remove the queue[s] from the existing napi instance that it is currently associated with and map with the new napi instance, by simply deleting from one list and adding to the new list.Quoted 8 lines
Right, my point is that each queue can only be serviced by a single NAPI at a time, so we have a 1:N relationship. It's easier to store the state on the side that's the N, rather than 1.
You can add
list_headto the queue structs, if you prefer to be able to walk queues of a NAPI more efficiently (that said the head for the list is in "control path only section" ofnapi_structso... I think you don't?)The napi pointer in the queue structs would give the napi<->queue mapping, I still need to walk the queues of a NAPI (when there are multiple queues for the NAPI), example:
'napi-id': 600, 'rx-queues': [7,6,5], 'tx-queues': [7,6,5]
Quoted 5 lines
in which case I would have a list of netdev queue structs within the
napi_struct(instead of the list of queue indices that I currently have) to avoid memory allocation.Does this sound right?
yes, I think that's fine.
If we store the NAPI pointer in the queue struct, we can still generate the same dump with the time complexity of #napis * (#max_rx + #max_tx). Which I don't think is too bad. Up to you.
Jakub Kicinski
maintainerRe: [net-next/RFC PATCH v1 1/4] net: Introduce new napi fields for rx/tx queues
Quoted 9 lines · On Wed, 12 Jul 2023 16:53:26 -0700 Jakub Kicinski wrote:
Quoted 9 lines
The napi pointer in the queue structs would give the napi<->queue mapping, I still need to walk the queues of a NAPI (when there are multiple queues for the NAPI), example:
'napi-id': 600, 'rx-queues': [7,6,5], 'tx-queues': [7,6,5]in which case I would have a list of netdev queue structs within the
napi_struct(instead of the list of queue indices that I currently have) to avoid memory allocation.
Quoted 7 lines
Quoted 1 line
Does this sound right?
yes, I think that's fine.
If we store the NAPI pointer in the queue struct, we can still generate the same dump with the time complexity of #napis * (#max_rx + #max_tx). Which I don't think is too bad. Up to you.
The more I think about it the more I feel like we should dump queues and NAPIs separately. And the queue can list the NAPI id of the NAPI instance which services it.
Are you actively working on this or should I take a stab?
Re: [net-next/RFC PATCH v1 1/4] net: Introduce new napi fields for rx/tx queues
Quoted 18 lines · Jakub Kicinski
Quoted 9 lines · On Wed, 12 Jul 2023 16:53:26 -0700 Jakub Kicinski wrote:
Quoted 9 lines
The napi pointer in the queue structs would give the napi<->queue mapping, I still need to walk the queues of a NAPI (when there are multiple queues for the NAPI), example:
'napi-id': 600, 'rx-queues': [7,6,5], 'tx-queues': [7,6,5]in which case I would have a list of netdev queue structs within the
napi_struct(instead of the list of queue indices that I currently have) to avoid memory allocation.Quoted 7 lines
Quoted 1 line
Does this sound right?
yes, I think that's fine.
If we store the NAPI pointer in the queue struct, we can still generate the same dump with the time complexity of #napis * (#max_rx + #max_tx). Which I don't think is too bad. Up to you.
Quoted 5 lines
The more I think about it the more I feel like we should dump queues and NAPIs separately. And the queue can list the NAPI id of the NAPI instance which services it.
Are you actively working on this or should I take a stab?
Hi Jakub, I have the next version of patches ready (I'll send that in a bit). I suggest if you could take a look at it and let me know your thoughts and then we can proceed from there.
About dumping queues and NAPIs separately, are you thinking about having both per-NAPI and per-queue instances, or do you think only one will suffice. The plan was to follow this work with a 'set-napi' series, something like, set-napi <napi_id> queues <q_id1, q_id2, ...>
to configure the queue[s] that are to be serviced by the napi instance.
In this case, dumping the NAPIs would be beneficial especially when there are multiple queues on the NAPI.
WRT per-queue, are there a set of parameters that needs to exposed besides what's already handled by ethtool... Also, to configure a queue on a NAPI, set-queue <qid> <napi_id>, the existing NAPIs would have to be looked up from the queue parameters dumped.
-Amritha
Jakub Kicinski
maintainerRe: [net-next/RFC PATCH v1 1/4] net: Introduce new napi fields for rx/tx queues
Quoted 3 lines · Amritha
Hi Jakub, I have the next version of patches ready (I'll send that in a bit). I suggest if you could take a look at it and let me know your thoughts and then we can proceed from there.
Great, looking forward.
Quoted 7 lines
About dumping queues and NAPIs separately, are you thinking about having both per-NAPI and per-queue instances, or do you think only one will suffice. The plan was to follow this work with a 'set-napi' series, something like,
set-napi <napi_id> queues <q_id1,q_id2, ...>
to configure the queue[s] that are to be serviced by the napi instance.
Quoted 5 lines
In this case, dumping the NAPIs would be beneficial especially when there are multiple queues on the NAPI.
WRT per-queue, are there a set of parameters that needs to exposed besides what's already handled by ethtool...
Not much at this point, maybe memory model. Maybe stats if we want to put stats in the same command. But the fact that sysfs has a bunch of per queue attributes makes me think that sooner or later we'll want queue as a full object in netlink. And starting out that way makes the whole API cleaner, at least in my opinion.
If we have another object which wants to refer to queues (e.g. page pool) it's easier to express the topology when it's clear what is an object and what's just an attribute.
Quoted 3 lines
Also, to configure a queue
on a NAPI, set-queue <qid> <napi_id>, the existing NAPIs would have to be looked up from the queue parameters dumped.
The look up should not be much of a problem.
And don't you think that:
set-queue queue 1 napi-id 101
set-queue queue 2 napi-id 101
is more natural than:
set-napi napi-id 101 queues [1, 2]
Especially in presence of conflicts. If user tries:
set-napi napi-id 101 queues [1, 2]
set-napi napi-id 102 queues [1, 2]
Do both napis now serve those queues? May seem obvious to us, but "philosophically" why does setting an attribute of object 102 change attributes of object 101?
If we ever gain the ability to create queues it will be:
create-queue napi-id xyz
which also matches set-queue more nicely than napi base API.
Re: [net-next/RFC PATCH v1 1/4] net: Introduce new napi fields for rx/tx queues
Quoted 45 lines · Jakub Kicinski
Quoted 3 lines · Amritha
Hi Jakub, I have the next version of patches ready (I'll send that in a bit). I suggest if you could take a look at it and let me know your thoughts and then we can proceed from there.
Great, looking forward.
Quoted 7 lines
About dumping queues and NAPIs separately, are you thinking about having both per-NAPI and per-queue instances, or do you think only one will suffice. The plan was to follow this work with a 'set-napi' series, something like,
set-napi <napi_id> queues <q_id1,q_id2, ...>
to configure the queue[s] that are to be serviced by the napi instance.Quoted 5 lines
In this case, dumping the NAPIs would be beneficial especially when there are multiple queues on the NAPI.
WRT per-queue, are there a set of parameters that needs to exposed besides what's already handled by ethtool...
Not much at this point, maybe memory model. Maybe stats if we want to put stats in the same command. But the fact that sysfs has a bunch of per queue attributes makes me think that sooner or later we'll want queue as a full object in netlink. And starting out that way makes the whole API cleaner, at least in my opinion.
If we have another object which wants to refer to queues (e.g. page pool) it's easier to express the topology when it's clear what is an object and what's just an attribute.
Quoted 3 lines
Also, to configure a queue
on a NAPI, set-queue <qid> <napi_id>, the existing NAPIs would have to be looked up from the queue parameters dumped.The look up should not be much of a problem.
And don't you think that:
set-queue queue 1 napi-id 101
set-queue queue 2 napi-id 101is more natural than:
set-napi napi-id 101 queues [1, 2]
Quoted 9 lines
Especially in presence of conflicts. If user tries:
set-napi napi-id 101 queues [1, 2] set-napi napi-id 102 queues [1, 2]Do both napis now serve those queues? May seem obvious to us, but "philosophically" why does setting an attribute of object 102 change attributes of object 101?
Right, I see the point. In presence of conflicts when the napi<->queue[s] mappings are updated, set-napi will impact other NAPI-IDs, while set-queue would limit the change to just the queue that is requested.
In both the cases, the underlying work remains the same:
1. Remove the queue from the existing napi instance it is associated with.
2. Driver updates queue[s]<->vector mapping and associates with new napi instance.
3. Report the impacted napi/queue back to the stack.
The 'napi-get' command would list all the napis and the updated queue[s] list.
Now, in usecases where a single poller is set to service multiple queues (say 8), with set-napi this can be done with a single command, while with set-queue this will result in 8 different requests to the driver. This is the trade-off I see if we go with set-queue.
Quoted 5 lines
If we ever gain the ability to create queues it will be:
create-queue napi-id xyz
which also matches set-queue more nicely than napi base API.
Re: [net-next/RFC PATCH v1 1/4] net: Introduce new napi fields for rx/tx queues
In reply to Jakub KicinskiQuoted 54 lines · Jakub Kicinski
Quoted 3 lines · Amritha
Hi Jakub, I have the next version of patches ready (I'll send that in a bit). I suggest if you could take a look at it and let me know your thoughts and then we can proceed from there.
Great, looking forward.
Quoted 7 lines
About dumping queues and NAPIs separately, are you thinking about having both per-NAPI and per-queue instances, or do you think only one will suffice. The plan was to follow this work with a 'set-napi' series, something like,
set-napi <napi_id> queues <q_id1,q_id2, ...>
to configure the queue[s] that are to be serviced by the napi instance.Quoted 5 lines
In this case, dumping the NAPIs would be beneficial especially when there are multiple queues on the NAPI.
WRT per-queue, are there a set of parameters that needs to exposed besides what's already handled by ethtool...
Not much at this point, maybe memory model. Maybe stats if we want to put stats in the same command. But the fact that sysfs has a bunch of per queue attributes makes me think that sooner or later we'll want queue as a full object in netlink. And starting out that way makes the whole API cleaner, at least in my opinion.
If we have another object which wants to refer to queues (e.g. page pool) it's easier to express the topology when it's clear what is an object and what's just an attribute.
Quoted 3 lines
Also, to configure a queue
on a NAPI, set-queue <qid> <napi_id>, the existing NAPIs would have to be looked up from the queue parameters dumped.The look up should not be much of a problem.
And don't you think that:
set-queue queue 1 napi-id 101
set-queue queue 2 napi-id 101is more natural than:
set-napi napi-id 101 queues [1, 2]
Especially in presence of conflicts. If user tries:
set-napi napi-id 101 queues [1, 2]
set-napi napi-id 102 queues [1, 2]Do both napis now serve those queues? May seem obvious to us, but "philosophically" why does setting an attribute of object 102 change attributes of object 101?
Quoted 6 lines
If we ever gain the ability to create queues it will be:
create-queue napi-id xyz
which also matches set-queue more nicely than napi base API.
I take it you have this path in mind as a means of creating "specialized" queues (e.g., io_uring and Rx ZC). Any slides or notes on the bigger picture?
Jakub Kicinski
maintainerRe: [net-next/RFC PATCH v1 1/4] net: Introduce new napi fields for rx/tx queues
Quoted 2 lines · On Tue, 1 Aug 2023 18:26:26 -0600 David Ahern wrote:
I take it you have this path in mind as a means of creating "specialized" queues (e.g.,
io_uringand Rx ZC).
TBH I've been thinking more about the huge page stuff and RSS context handling. Rx ZC should be a subset of what's needed for those cases.
Quoted 1 line
Any slides or notes on the bigger picture?
I don't have it well figured out, yet. The user space API is pretty easy, but shaping it in a way that makes driver's life manageable is more challenging.
PATCH 2/4 net: Add support for associating napi with queue[s]
In reply to Amritha NambiarAfter the napi context is initialized, map the napi instance with the queue/queue-set on the corresponding irq line.
--- drivers/net/ethernet/intel/ice/ice_lib.c | 57 +++++++++++++++++++++++++++++ drivers/net/ethernet/intel/ice/ice_lib.h | 4 ++ drivers/net/ethernet/intel/ice/ice_main.c | 4 ++ include/linux/netdevice.h | 11 ++++++ net/core/dev.c | 34 +++++++++++++++++ 5 files changed, 109 insertions(+), 1 deletion(-)
drivers/net/ethernet/intel/ice/ice_lib.c +57 -0
@@ -2478,6 +2478,12 @@ ice_vsi_cfg_def(struct ice_vsi *vsi, struct ice_vsi_cfg_params *params) goto unroll_vector_base;ice_vsi_map_rings_to_vectors (vsi); + +/* Associate q_vector rings to napi */ + ret =ice_vsi_add_napi_queues (vsi); +if (ret) + goto unroll_vector_base; + vsi->stat_offsets_loaded = false;if (ice_is_xdp_ena_vsi (vsi)) {
@@ -2957,6 +2963,57 @@ void ice_vsi_dis_irq(struct ice_vsi *vsi)synchronize_irq (vsi->q_vectors[i]->irq.virq); } +/** +* ice_q_vector_add_napi_queues - Add queue[s] associated with the napi +* @q_vector: q_vector pointer +* +* Associate the q_vector napi with all the queue[s] on the vector +* Returns 0 on success or < 0 on error +*/ +int ice_q_vector_add_napi_queues (struct ice_q_vector *q_vector) +{ +struct ice_rx_ring *rx_ring; +struct ice_tx_ring *tx_ring; +int ret; + +ice_for_each_rx_ring (rx_ring, q_vector->rx) { + ret =netif_napi_add_queue (&q_vector->napi, rx_ring->q_index, +NAPI_RX_CONTAINER ); +if (ret) +return ret; + } +ice_for_each_tx_ring (tx_ring, q_vector->tx) { + ret =netif_napi_add_queue (&q_vector->napi, tx_ring->q_index, +NAPI_TX_CONTAINER ); +if (ret) +return ret; + } + +return ret; +} + +/** +* ice_vsi_add_napi_queues +* @vsi: VSI pointer +* +* Associate queue[s] with napi for all vectors +* Returns 0 on success or < 0 on error +*/ +int ice_vsi_add_napi_queues (struct ice_vsi *vsi) +{ +int i, ret =0 ; + +if (!vsi->netdev) +return ret; + +ice_for_each_q_vector (vsi, i) { + ret =ice_q_vector_add_napi_queues (vsi->q_vectors[i]); +if (ret) +return ret; + } +return ret; +} +/** * ice_napi_del - Remove NAPI handler for the VSI * @vsi: VSI for which NAPI handler is to be removed
drivers/net/ethernet/intel/ice/ice_lib.h +4 -0
@@ -93,6 +93,10 @@ void ice_vsi_cfg_netdev_tc(struct ice_vsi *vsi, u8 ena_tc);struct ice_vsi *ice_vsi_setup (struct ice_pf *pf,struct ice_vsi_cfg_params *params); +int ice_q_vector_add_napi_queues (struct ice_q_vector *q_vector); + +int ice_vsi_add_napi_queues (struct ice_vsi *vsi); +void ice_napi_del (struct ice_vsi *vsi);int ice_vsi_release (struct ice_vsi *vsi);
drivers/net/ethernet/intel/ice/ice_main.c +3 -1
@@ -3348,9 +3348,11 @@ static void ice_napi_add(struct ice_vsi *vsi)if (!vsi->netdev)return ; -ice_for_each_q_vector (vsi, v_idx) +ice_for_each_q_vector (vsi, v_idx) {netif_napi_add (vsi->netdev, &vsi->q_vectors[v_idx]->napi, ice_napi_poll); +ice_q_vector_add_napi_queues (vsi->q_vectors[v_idx]); + } }/**
include/linux/netdevice.h +11 -0
@@ -342,6 +342,14 @@ struct gro_list {*/ #define GRO_HASH_BUCKETS 8 +/* +* napi queue container type +*/ +enum napi_container_type { +NAPI_RX_CONTAINER , +NAPI_TX_CONTAINER , +}; +struct napi_queue {struct list_head q_list;u16 queue_index;
@@ -2622,6 +2630,9 @@ static inline void *netdev_priv(const struct net_device *dev)*/ #define SET_NETDEV_DEVTYPE (net, devtype) ((net)->dev.type = (devtype)) +int netif_napi_add_queue (struct napi_struct *napi,u16 queue_index, +enum napi_container_type ); +/* Default NAPI poll() weight * Device drivers are strongly advised to not use bigger value */
net/core/dev.c +34 -0
@@ -6366,6 +6366,40 @@ int dev_set_threaded(struct net_device *dev, bool threaded) }EXPORT_SYMBOL (dev_set_threaded); +/** +* netif_napi_add_queue - Associate queue with the napi +* @napi: NAPI context +* @queue_index: Index of queue +* @napi_container_type: queue type as RX or TX +* +* Add queue with its corresponding napi context +*/ +int netif_napi_add_queue (struct napi_struct *napi,u16 queue_index, +enum napi_container_type type) +{ +struct napi_queue *napi_queue; + + napi_queue =kzalloc (sizeof (*napi_queue),GFP_KERNEL ); +if (!napi_queue) +return -ENOMEM ; + + napi_queue->queue_index = queue_index; + +switch (type) { +case NAPI_RX_CONTAINER : +list_add_rcu (&napi_queue->q_list, &napi->napi_rxq_list); +break ; +case NAPI_TX_CONTAINER : +list_add_rcu (&napi_queue->q_list, &napi->napi_txq_list); +break ; +default : +return -EINVAL ; + } + +return 0 ; +} +EXPORT_SYMBOL (netif_napi_add_queue); +void netif_napi_add_weight (struct net_device *dev,struct napi_struct *napi,int (*poll)(struct napi_struct *,int ),int weight) {
Re: [net-next/RFC PATCH v1 2/4] net: Add support for associating napi with queue[s]
Hi Amritha,
[This is a private test report for your RFC patch.] kernel test robot noticed the following build warnings:
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Amritha-Nambiar/net-Introduce-new-napi-fields-for-rx-tx-queues/20230602-013021 base: https://git.kernel.org/pub/scm/linux/kernel/git/tnguy/next-queue.git dev-queue patch link: https://lore.kernel.org/r/168564135094.7284.9691772825401908320.stgit%40anambiarhost.jf.intel.com patch subject: [net-next/RFC PATCH v1 2/4] net: Add support for associating napi with queue[s] config: i386-randconfig-m021-20230531 (https://download.01.org/0day-ci/archive/20230602/[email protected]/config) compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <[email protected]> | Reported-by: Dan Carpenter <[email protected]> | Closes: https://lore.kernel.org/r/[email protected]/
New smatch warnings:net/core/dev.c:6396 netif_napi_add_queue() warn: possible memory leak of 'napi_queue'
Old smatch warnings:net/core/dev.c:10454 netdev_run_todo() warn: passing freed memory 'dev'
vim +/napi_queue +6396 net/core/dev.c
9b6d735207c0eb Amritha Nambiar 2023-06-01 6377 int netif_napi_add_queue(struct napi_struct *napi, u16 queue_index,
9b6d735207c0eb Amritha Nambiar 2023-06-01 6378 enum napi_container_type type)
9b6d735207c0eb Amritha Nambiar 2023-06-01 6379 {
9b6d735207c0eb Amritha Nambiar 2023-06-01 6380 struct napi_queue *napi_queue;
9b6d735207c0eb Amritha Nambiar 2023-06-01 6381
9b6d735207c0eb Amritha Nambiar 2023-06-01 6382 napi_queue = kzalloc(sizeof(*napi_queue), GFP_KERNEL);
9b6d735207c0eb Amritha Nambiar 2023-06-01 6383 if (!napi_queue)
9b6d735207c0eb Amritha Nambiar 2023-06-01 6384 return -ENOMEM;
9b6d735207c0eb Amritha Nambiar 2023-06-01 6385
9b6d735207c0eb Amritha Nambiar 2023-06-01 6386 napi_queue->queue_index = queue_index;
9b6d735207c0eb Amritha Nambiar 2023-06-01 6387
9b6d735207c0eb Amritha Nambiar 2023-06-01 6388 switch (type) {
9b6d735207c0eb Amritha Nambiar 2023-06-01 6389 case NAPI_RX_CONTAINER:
9b6d735207c0eb Amritha Nambiar 2023-06-01 6390 list_add_rcu(&napi_queue->q_list, &napi->napi_rxq_list);
9b6d735207c0eb Amritha Nambiar 2023-06-01 6391 break;
9b6d735207c0eb Amritha Nambiar 2023-06-01 6392 case NAPI_TX_CONTAINER:
9b6d735207c0eb Amritha Nambiar 2023-06-01 6393 list_add_rcu(&napi_queue->q_list, &napi->napi_txq_list);
9b6d735207c0eb Amritha Nambiar 2023-06-01 6394 break;
9b6d735207c0eb Amritha Nambiar 2023-06-01 6395 default:9b6d735207c0eb Amritha Nambiar 2023-06-01 @6396 return -EINVAL;
This path is likely impossible, right? Smatch still wants a kfree(napi_queue); before the return.
9b6d735207c0eb Amritha Nambiar 2023-06-01 6397 } 9b6d735207c0eb Amritha Nambiar 2023-06-01 6398 9b6d735207c0eb Amritha Nambiar 2023-06-01 6399 return 0; 9b6d735207c0eb Amritha Nambiar 2023-06-01 6400 }
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Re: [net-next/RFC PATCH v1 2/4] net: Add support for associating napi with queue[s]
In reply to Amritha NambiarQuoted 4 lines · Amritha Nambiar
After the napi context is initialized, map the napi instance with the queue/queue-set on the corresponding irq line.
Signed-off-by Amritha Nambiar <[email protected]>
Hi Amritha,
some minor feedback from my side.
...
net/core/dev.c int dev_set_threaded(struct net_device *dev, bool threaded) · line 6373
Quoted 13 lines
net/core/dev.c +5
-0@@ -6366,6 +6366,40 @@ int dev_set_threaded(struct net_device *dev, bool threaded) }EXPORT_SYMBOL (dev_set_threaded); +/** +* netif_napi_add_queue - Associate queue with the napi +* @napi: NAPI context +* @queue_index: Index of queue +* @napi_container_type: queue type as RX or TX
s/@napi_container_type:/@type:/
Quoted 12 lines
net/core/dev.c +12
-0+* +* Add queue with its corresponding napi context +*/ +int netif_napi_add_queue (struct napi_struct *napi,u16 queue_index, +enum napi_container_type type) +{ +struct napi_queue *napi_queue; + + napi_queue =kzalloc (sizeof (*napi_queue),GFP_KERNEL ); +if (!napi_queue) +return -ENOMEM ; +
net/core/dev.c int dev_set_threaded(struct net_device *dev, bool threaded) · line 6395
Quoted 10 lines
net/core/dev.c +10
-0+ napi_queue->queue_index = queue_index; + +switch (type) { +case NAPI_RX_CONTAINER : +list_add_rcu (&napi_queue->q_list, &napi->napi_rxq_list); +break ; +case NAPI_TX_CONTAINER : +list_add_rcu (&napi_queue->q_list, &napi->napi_txq_list); +break ; +default :
Perhaps napi_queue is leaked here.
Quoted 12 lines
net/core/dev.c +7
-0+return -EINVAL ; + } + +return 0 ; +} +EXPORT_SYMBOL (netif_napi_add_queue); +void netif_napi_add_weight (struct net_device *dev,struct napi_struct *napi,int (*poll)(struct napi_struct *,int ),int weight) {
Re: [net-next/RFC PATCH v1 2/4] net: Add support for associating napi with queue[s]
Quoted 20 lines · Simon Horman
Quoted 4 lines · Amritha Nambiar
After the napi context is initialized, map the napi instance with the queue/queue-set on the corresponding irq line.
Signed-off-by Amritha Nambiar <[email protected]>Hi Amritha,
some minor feedback from my side.
...
Quoted 8 lines
net/core/dev.c +0
-0@@ -6366,6 +6366,40 @@ int dev_set_threaded(struct net_device *dev, bool threaded) }EXPORT_SYMBOL (dev_set_threaded);
Quoted 8 lines
net/core/dev.c int dev_set_threaded(struct net_device *dev, bool threaded) · line 6373
Quoted 5 lines
net/core/dev.c +5
-0+/** +* netif_napi_add_queue - Associate queue with the napi +* @napi: NAPI context +* @queue_index: Index of queue +* @napi_container_type: queue type as RX or TX s/@napi_container_type:/@type:/
Will fix.
Quoted 16 lines
Quoted 16 lines
net/core/dev.c +16
-0+* +* Add queue with its corresponding napi context +*/ +int netif_napi_add_queue (struct napi_struct *napi,u16 queue_index, +enum napi_container_type type) +{ +struct napi_queue *napi_queue; + + napi_queue =kzalloc (sizeof (*napi_queue),GFP_KERNEL ); +if (!napi_queue) +return -ENOMEM ; + + napi_queue->queue_index = queue_index; + +switch (type) { +case NAPI_RX_CONTAINER :
Quoted 9 lines
net/core/dev.c int dev_set_threaded(struct net_device *dev, bool threaded) · line 6395
Quoted 6 lines
net/core/dev.c +6
-0+list_add_rcu (&napi_queue->q_list, &napi->napi_rxq_list); +break ; +case NAPI_TX_CONTAINER : +list_add_rcu (&napi_queue->q_list, &napi->napi_txq_list); +break ; +default :Perhaps
napi_queueis leaked here.
My bad. Will fix in the next version.
Quoted 12 lines
Quoted 12 lines
net/core/dev.c +7
-0+return -EINVAL ; + } + +return 0 ; +} +EXPORT_SYMBOL (netif_napi_add_queue); +void netif_napi_add_weight (struct net_device *dev,struct napi_struct *napi,int (*poll)(struct napi_struct *,int ),int weight) {
Paolo Abeni
maintainerRe: [net-next/RFC PATCH v1 2/4] net: Add support for associating napi with queue[s]
In reply to Amritha NambiarQuoted 184 lines · Amritha Nambiar
After the napi context is initialized, map the napi instance with the queue/queue-set on the corresponding irq line.
Signed-off-by Amritha Nambiar <[email protected]>--- drivers/net/ethernet/intel/ice/ice_lib.c | 57 +++++++++++++++++++++++++++++ drivers/net/ethernet/intel/ice/ice_lib.h | 4 ++ drivers/net/ethernet/intel/ice/ice_main.c | 4 ++ include/linux/netdevice.h | 11 ++++++ net/core/dev.c | 34 +++++++++++++++++ 5 files changed, 109 insertions(+), 1 deletion(-)drivers/net/ethernet/intel/ice/ice_lib.c +57
-0@@ -2478,6 +2478,12 @@ ice_vsi_cfg_def(struct ice_vsi *vsi, struct ice_vsi_cfg_params *params) goto unroll_vector_base;ice_vsi_map_rings_to_vectors (vsi); + +/* Associate q_vector rings to napi */ + ret =ice_vsi_add_napi_queues (vsi); +if (ret) + goto unroll_vector_base; + vsi->stat_offsets_loaded = false;if (ice_is_xdp_ena_vsi (vsi)) {@@ -2957,6 +2963,57 @@ void ice_vsi_dis_irq(struct ice_vsi *vsi)synchronize_irq (vsi->q_vectors[i]->irq.virq); } +/** +* ice_q_vector_add_napi_queues - Add queue[s] associated with the napi +* @q_vector: q_vector pointer +* +* Associate the q_vector napi with all the queue[s] on the vector +* Returns 0 on success or < 0 on error +*/ +int ice_q_vector_add_napi_queues (struct ice_q_vector *q_vector) +{ +struct ice_rx_ring *rx_ring; +struct ice_tx_ring *tx_ring; +int ret; + +ice_for_each_rx_ring (rx_ring, q_vector->rx) { + ret =netif_napi_add_queue (&q_vector->napi, rx_ring->q_index, +NAPI_RX_CONTAINER ); +if (ret) +return ret; + } +ice_for_each_tx_ring (tx_ring, q_vector->tx) { + ret =netif_napi_add_queue (&q_vector->napi, tx_ring->q_index, +NAPI_TX_CONTAINER ); +if (ret) +return ret; + } + +return ret; +} + +/** +* ice_vsi_add_napi_queues +* @vsi: VSI pointer +* +* Associate queue[s] with napi for all vectors +* Returns 0 on success or < 0 on error +*/ +int ice_vsi_add_napi_queues (struct ice_vsi *vsi) +{ +int i, ret =0 ; + +if (!vsi->netdev) +return ret; + +ice_for_each_q_vector (vsi, i) { + ret =ice_q_vector_add_napi_queues (vsi->q_vectors[i]); +if (ret) +return ret; + } +return ret; +} +/** * ice_napi_del - Remove NAPI handler for the VSI * @vsi: VSI for which NAPI handler is to be removed drivers/net/ethernet/intel/ice/ice_lib.h +4
-0@@ -93,6 +93,10 @@ void ice_vsi_cfg_netdev_tc(struct ice_vsi *vsi, u8 ena_tc);struct ice_vsi *ice_vsi_setup (struct ice_pf *pf,struct ice_vsi_cfg_params *params); +int ice_q_vector_add_napi_queues (struct ice_q_vector *q_vector); + +int ice_vsi_add_napi_queues (struct ice_vsi *vsi); +void ice_napi_del (struct ice_vsi *vsi);int ice_vsi_release (struct ice_vsi *vsi);drivers/net/ethernet/intel/ice/ice_main.c +3
-1@@ -3348,9 +3348,11 @@ static void ice_napi_add(struct ice_vsi *vsi)if (!vsi->netdev)return ; -ice_for_each_q_vector (vsi, v_idx) +ice_for_each_q_vector (vsi, v_idx) {netif_napi_add (vsi->netdev, &vsi->q_vectors[v_idx]->napi, ice_napi_poll); +ice_q_vector_add_napi_queues (vsi->q_vectors[v_idx]); + } }/** include/linux/netdevice.h +11
-0@@ -342,6 +342,14 @@ struct gro_list {*/ #define GRO_HASH_BUCKETS 8 +/* +* napi queue container type +*/ +enum napi_container_type { +NAPI_RX_CONTAINER , +NAPI_TX_CONTAINER , +}; +struct napi_queue {struct list_head q_list;u16 queue_index;@@ -2622,6 +2630,9 @@ static inline void *netdev_priv(const struct net_device *dev)*/ #define SET_NETDEV_DEVTYPE (net, devtype) ((net)->dev.type = (devtype)) +int netif_napi_add_queue (struct napi_struct *napi,u16 queue_index, +enum napi_container_type ); +/* Default NAPI poll() weight * Device drivers are strongly advised to not use bigger value */ net/core/dev.c +27
-0@@ -6366,6 +6366,40 @@ int dev_set_threaded(struct net_device *dev, bool threaded) }EXPORT_SYMBOL (dev_set_threaded); +/** +* netif_napi_add_queue - Associate queue with the napi +* @napi: NAPI context +* @queue_index: Index of queue +* @napi_container_type: queue type as RX or TX +* +* Add queue with its corresponding napi context +*/ +int netif_napi_add_queue (struct napi_struct *napi,u16 queue_index, +enum napi_container_type type) +{ +struct napi_queue *napi_queue; + + napi_queue =kzalloc (sizeof (*napi_queue),GFP_KERNEL ); +if (!napi_queue) +return -ENOMEM ; + + napi_queue->queue_index = queue_index; + +switch (type) { +case NAPI_RX_CONTAINER : +list_add_rcu (&napi_queue->q_list, &napi->napi_rxq_list); +break ; +case NAPI_TX_CONTAINER : +list_add_rcu (&napi_queue->q_list, &napi->napi_txq_list); +break ; +default :
net/core/dev.c int dev_set_threaded(struct net_device *dev, bool threaded) · line 6405
Quoted 10 lines
net/core/dev.c +7
-0+return -EINVAL ; + } + +return 0 ; +} +EXPORT_SYMBOL (netif_napi_add_queue); +void netif_napi_add_weight (struct net_device *dev,struct napi_struct *napi,int (*poll)(struct napi_struct *,int ),int weight) {
I think this later 2 chunks are a better fit for the previous patch, so that here there will be only driver-related changes.
Also it looks like the napi-queue APIs are going to grow a bit. Perhaps it would be useful move all that new code in a separated file? dev.c is already pretty big.
Thanks!
Paolo
Re: [net-next/RFC PATCH v1 2/4] net: Add support for associating napi with queue[s]
Quoted 190 lines · Paolo Abeni
Quoted 189 lines · Amritha Nambiar
After the napi context is initialized, map the napi instance with the queue/queue-set on the corresponding irq line.
Signed-off-by Amritha Nambiar <[email protected]>--- drivers/net/ethernet/intel/ice/ice_lib.c | 57 +++++++++++++++++++++++++++++ drivers/net/ethernet/intel/ice/ice_lib.h | 4 ++ drivers/net/ethernet/intel/ice/ice_main.c | 4 ++ include/linux/netdevice.h | 11 ++++++ net/core/dev.c | 34 +++++++++++++++++ 5 files changed, 109 insertions(+), 1 deletion(-)drivers/net/ethernet/intel/ice/ice_lib.c +57
-0@@ -2478,6 +2478,12 @@ ice_vsi_cfg_def(struct ice_vsi *vsi, struct ice_vsi_cfg_params *params) goto unroll_vector_base;ice_vsi_map_rings_to_vectors (vsi); + +/* Associate q_vector rings to napi */ + ret =ice_vsi_add_napi_queues (vsi); +if (ret) + goto unroll_vector_base; + vsi->stat_offsets_loaded = false;if (ice_is_xdp_ena_vsi (vsi)) {@@ -2957,6 +2963,57 @@ void ice_vsi_dis_irq(struct ice_vsi *vsi)synchronize_irq (vsi->q_vectors[i]->irq.virq); } +/** +* ice_q_vector_add_napi_queues - Add queue[s] associated with the napi +* @q_vector: q_vector pointer +* +* Associate the q_vector napi with all the queue[s] on the vector +* Returns 0 on success or < 0 on error +*/ +int ice_q_vector_add_napi_queues (struct ice_q_vector *q_vector) +{ +struct ice_rx_ring *rx_ring; +struct ice_tx_ring *tx_ring; +int ret; + +ice_for_each_rx_ring (rx_ring, q_vector->rx) { + ret =netif_napi_add_queue (&q_vector->napi, rx_ring->q_index, +NAPI_RX_CONTAINER ); +if (ret) +return ret; + } +ice_for_each_tx_ring (tx_ring, q_vector->tx) { + ret =netif_napi_add_queue (&q_vector->napi, tx_ring->q_index, +NAPI_TX_CONTAINER ); +if (ret) +return ret; + } + +return ret; +} + +/** +* ice_vsi_add_napi_queues +* @vsi: VSI pointer +* +* Associate queue[s] with napi for all vectors +* Returns 0 on success or < 0 on error +*/ +int ice_vsi_add_napi_queues (struct ice_vsi *vsi) +{ +int i, ret =0 ; + +if (!vsi->netdev) +return ret; + +ice_for_each_q_vector (vsi, i) { + ret =ice_q_vector_add_napi_queues (vsi->q_vectors[i]); +if (ret) +return ret; + } +return ret; +} +/** * ice_napi_del - Remove NAPI handler for the VSI * @vsi: VSI for which NAPI handler is to be removed drivers/net/ethernet/intel/ice/ice_lib.h +4
-0@@ -93,6 +93,10 @@ void ice_vsi_cfg_netdev_tc(struct ice_vsi *vsi, u8 ena_tc);struct ice_vsi *ice_vsi_setup (struct ice_pf *pf,struct ice_vsi_cfg_params *params); +int ice_q_vector_add_napi_queues (struct ice_q_vector *q_vector); + +int ice_vsi_add_napi_queues (struct ice_vsi *vsi); +void ice_napi_del (struct ice_vsi *vsi);int ice_vsi_release (struct ice_vsi *vsi);drivers/net/ethernet/intel/ice/ice_main.c +3
-1@@ -3348,9 +3348,11 @@ static void ice_napi_add(struct ice_vsi *vsi)if (!vsi->netdev)return ; -ice_for_each_q_vector (vsi, v_idx) +ice_for_each_q_vector (vsi, v_idx) {netif_napi_add (vsi->netdev, &vsi->q_vectors[v_idx]->napi, ice_napi_poll); +ice_q_vector_add_napi_queues (vsi->q_vectors[v_idx]); + } }/** include/linux/netdevice.h +11
-0@@ -342,6 +342,14 @@ struct gro_list {*/ #define GRO_HASH_BUCKETS 8 +/* +* napi queue container type +*/ +enum napi_container_type { +NAPI_RX_CONTAINER , +NAPI_TX_CONTAINER , +}; +struct napi_queue {struct list_head q_list;u16 queue_index;@@ -2622,6 +2630,9 @@ static inline void *netdev_priv(const struct net_device *dev)*/ #define SET_NETDEV_DEVTYPE (net, devtype) ((net)->dev.type = (devtype)) +int netif_napi_add_queue (struct napi_struct *napi,u16 queue_index, +enum napi_container_type ); +/* Default NAPI poll() weight * Device drivers are strongly advised to not use bigger value */ net/core/dev.c +33
-0@@ -6366,6 +6366,40 @@ int dev_set_threaded(struct net_device *dev, bool threaded) }EXPORT_SYMBOL (dev_set_threaded); +/** +* netif_napi_add_queue - Associate queue with the napi +* @napi: NAPI context +* @queue_index: Index of queue +* @napi_container_type: queue type as RX or TX +* +* Add queue with its corresponding napi context +*/ +int netif_napi_add_queue (struct napi_struct *napi,u16 queue_index, +enum napi_container_type type) +{ +struct napi_queue *napi_queue; + + napi_queue =kzalloc (sizeof (*napi_queue),GFP_KERNEL ); +if (!napi_queue) +return -ENOMEM ; + + napi_queue->queue_index = queue_index; + +switch (type) { +case NAPI_RX_CONTAINER : +list_add_rcu (&napi_queue->q_list, &napi->napi_rxq_list); +break ; +case NAPI_TX_CONTAINER : +list_add_rcu (&napi_queue->q_list, &napi->napi_txq_list); +break ; +default : +return -EINVAL ; + } + +return 0 ; +} +EXPORT_SYMBOL (netif_napi_add_queue);
Quoted 9 lines
net/core/dev.c int dev_set_threaded(struct net_device *dev, bool threaded) · line 6405
Quoted 5 lines
net/core/dev.c +1
-0+void netif_napi_add_weight (struct net_device *dev,struct napi_struct *napi,int (*poll)(struct napi_struct *,int ),int weight) {I think this later 2 chunks are a better fit for the previous patch, so that here there will be only driver-related changes.
So if the later chunks are moved to the previous patch, wouldn't git bisect and build throw warnings as the kernel API (netif_napi_add_queue) would only be defined but not used. The function netif_napi_add_queue is being invoked only by the driver code. Hence, I had to move in the kernel function definition to this patch that has all the driver code.
Quoted 4 lines
Also it looks like the napi-queue APIs are going to grow a bit. Perhaps it would be useful move all that new code in a separated file?
dev.cis already pretty big.
Are you suggesting to move just the napi-queue related new code into a separate file, but keep all other napi stuff in dev.c ? In that case, currently, the new file would contain only two function definitions for netif_napi_add_queue and delete (if the napi-queue specific APIs do not grow). I agree there may be more generic napi APIs in future (not just queue info related), but wouldn't it look better if all the napi code were in a new file.
Quoted 4 lines
Thanks!
Paolo
PATCH 3/4 netdev-genl: Introduce netdev dump ctx
In reply to Amritha NambiarA comment in the definition of struct netlink_callback states "args is deprecated. Cast a struct over ctx instead for proper type safety." Introduce netdev_nl_dump_ctx structure and replace 'args' with netdev_nl_dump_ctx fields.
--- net/core/netdev-genl.c | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-)
net/core/netdev-genl.c +18 -4
@@ -8,6 +8,19 @@#include "netdev-genl-gen.h" +struct netdev_nl_dump_ctx { +int dev_entry_hash; +int dev_entry_idx; +}; + +static inline struct netdev_nl_dump_ctx * +netdev_dump_ctx (struct netlink_callback *cb) +{ +NL_ASSERT_DUMP_CTX_FITS (struct netdev_nl_dump_ctx ); + +return (struct netdev_nl_dump_ctx *)cb->ctx; +} +static int netdev_nl_dev_fill (struct net_device *netdev,struct sk_buff *rsp,u32 portid,u32 seq,int flags,u32 cmd)
@@ -91,14 +104,15 @@ int netdev_nl_dev_get_doit(struct sk_buff *skb, struct genl_info *info)int netdev_nl_dev_get_dumpit (struct sk_buff *skb,struct netlink_callback *cb) { +struct netdev_nl_dump_ctx *ctx =netdev_dump_ctx (cb);struct net *net =sock_net (skb->sk);struct net_device *netdev;int idx =0 , s_idx;int h, s_h;int err; - s_h = cb->args[0 ]; - s_idx = cb->args[1 ]; + s_h = ctx->dev_entry_hash; + s_idx = ctx->dev_entry_idx;rtnl_lock ();
@@ -126,8 +140,8 @@ int netdev_nl_dev_get_dumpit(struct sk_buff *skb, struct netlink_callback *cb)if (err != -EMSGSIZE )return err; - cb->args[1 ] = idx; - cb->args[0 ] = h; + ctx->dev_entry_idx = idx; + ctx->dev_entry_hash = h; cb->seq = net->dev_base_seq;return skb->len;
PATCH 4/4 netdev-genl: Add support for exposing napi info from netdev
In reply to Amritha NambiarAdd support in ynl/netdev.yaml for napi related information. The netdev structure tracks all the napi instances and napi fields. The napi instances and associated queue[s] can be retrieved this way.
Refactored netdev-genl to support exposing napi<->queue[s] mapping that is retained in a netdev.
--- Documentation/netlink/specs/netdev.yaml | 39 +++++ include/uapi/linux/netdev.h | 4 + net/core/netdev-genl.c | 239 ++++++++++++++++++++++++++----- tools/include/uapi/linux/netdev.h | 4 + 4 files changed, 247 insertions(+), 39 deletions(-)
Documentation/netlink/specs/netdev.yaml +39 -0
@@ -62,6 +62,44 @@ attribute-sets: type: u64 enum: xdp-act enum-as-flags: true + - + name: napi-info + doc: napi information such as napi-id, napi queues etc. + type: nest + multi-attr: true + nested-attributes: dev-napi-info + - + name: napi-id + doc: napi id + type: u32 + - + name: rx-queues + doc: list of rx queues associated with a napi + type: u16 + multi-attr: true + - + name: tx-queues + doc: list of tx queues associated with a napi + type: u16 + multi-attr: true + - + name: dev-napi-info + subset-of: dev + attributes: + - + name: napi-id + doc: napi id + type: u32 + - + name: rx-queues + doc: list rx of queues associated with a napi + type: u16 + multi-attr: true + - + name: tx-queues + doc: list tx of queues associated with a napi + type: u16 + multi-attr: true operations: list:
@@ -77,6 +115,7 @@ operations: attributes: - ifindex - xdp-features + - napi-info dump: reply: *dev-all -
include/uapi/linux/netdev.h +4 -0
@@ -41,6 +41,10 @@ enum {NETDEV_A_DEV_IFINDEX =1 ,NETDEV_A_DEV_PAD ,NETDEV_A_DEV_XDP_FEATURES , +NETDEV_A_DEV_NAPI_INFO , +NETDEV_A_DEV_NAPI_ID , +NETDEV_A_DEV_RX_QUEUES , +NETDEV_A_DEV_TX_QUEUES , __NETDEV_A_DEV_MAX,NETDEV_A_DEV_MAX = (__NETDEV_A_DEV_MAX -1 )
net/core/netdev-genl.c +200 -39
323 lines not on this page · raw message
tools/include/uapi/linux/netdev.h +4 -0
@@ -41,6 +41,10 @@ enum {NETDEV_A_DEV_IFINDEX =1 ,NETDEV_A_DEV_PAD ,NETDEV_A_DEV_XDP_FEATURES , +NETDEV_A_DEV_NAPI_INFO , +NETDEV_A_DEV_NAPI_ID , +NETDEV_A_DEV_RX_QUEUES , +NETDEV_A_DEV_TX_QUEUES , __NETDEV_A_DEV_MAX,NETDEV_A_DEV_MAX = (__NETDEV_A_DEV_MAX -1 )
Re: [net-next/RFC PATCH v1 4/4] netdev-genl: Add support for exposing napi info from netdev
Quoted 6 lines · Amritha Nambiar
Add support in
ynl/netdev.yamlfor napi related information. The netdev structure tracks all the napi instances and napi fields. The napi instances and associated queue[s] can be retrieved this way.Refactored netdev-genl to support exposing napi<->queue[s] mapping that is retained in a netdev.
Hi Amritha,
This feels like it should be two patches to me.
Though it is not something I feel strongly about.
Quoted 1 line
Signed-off-by Amritha Nambiar <[email protected]>
...
net/core/netdev-genl.c netdev_dump_ctx(struct netlink_callback *cb) · line 103
Quoted 12 lines
net/core/netdev-genl.c +12
-0+static int +netdev_nl_dev_napi_prepare_fill (struct net_device *netdev, +struct sk_buff **pskb,u32 portid,u32 seq, +int flags,u32 cmd,enum netdev_nl_type type) +{ +struct nlmsghdr *nlh; +struct sk_buff *skb = *pskb; +bool last = false; +int index =0 ; +void *hdr; +int err; +
nit: please use reverse xmas tree - longest line to shortest - for
local variable declarations in (new) Networking code.
...
Jakub Kicinski
maintainerRe: [net-next/RFC PATCH v1 4/4] netdev-genl: Add support for exposing napi info from netdev
Quoted 2 lines · On Fri, 2 Jun 2023 17:47:17 +0200 Simon Horman wrote:
This feels like it should be two patches to me.
Though it is not something I feel strongly about.
+1 I'd put the YAML and what's generated from it in one patch, and the hand-written code in another.
Re: [net-next/RFC PATCH v1 4/4] netdev-genl: Add support for exposing napi info from netdev
Quoted 6 lines · Jakub Kicinski
Quoted 2 lines · On Fri, 2 Jun 2023 17:47:17 +0200 Simon Horman wrote:
This feels like it should be two patches to me.
Though it is not something I feel strongly about.+1 I'd put the YAML and what's generated from it in one patch, and the hand-written code in another.
Will fix in the next version.
Re: [net-next/RFC PATCH v1 4/4] netdev-genl: Add support for exposing napi info from netdev
In reply to Simon HormanQuoted 5 lines · Simon Horman
Quoted 4 lines · Amritha Nambiar
Add support in
ynl/netdev.yamlfor napi related information. The netdev structure tracks all the napi instances and napi fields. The napi instances and associated queue[s] can be retrieved this way.
Quoted 8 lines
Quoted 2 lines
Refactored netdev-genl to support exposing napi<->queue[s] mapping that is retained in a netdev.
Hi Amritha,
This feels like it should be two patches to me.
Though it is not something I feel strongly about.
Thanks for pointing that out, I'll split this patch into two.
Quoted 20 lines
Quoted 1 line
Signed-off-by Amritha Nambiar <[email protected]>...
net/core/netdev-genl.c netdev_dump_ctx(struct netlink_callback *cb) · line 103
Quoted 12 lines
net/core/netdev-genl.c +12
-0+static int +netdev_nl_dev_napi_prepare_fill (struct net_device *netdev, +struct sk_buff **pskb,u32 portid,u32 seq, +int flags,u32 cmd,enum netdev_nl_type type) +{ +struct nlmsghdr *nlh; +struct sk_buff *skb = *pskb; +bool last = false; +int index =0 ; +void *hdr; +int err; +nit: please use reverse xmas tree - longest line to shortest - for
local variable declarations in (new) Networking code.
Quoted 1 line
...
Will fix in next version.
Jakub Kicinski
maintainerRe: [net-next/RFC PATCH v1 4/4] netdev-genl: Add support for exposing napi info from netdev
In reply to Amritha NambiarQuoted 20 lines · On Thu, 01 Jun 2023 10:42:41 -0700 Amritha Nambiar wrote:
Add support in
ynl/netdev.yamlfor napi related information. The netdev structure tracks all the napi instances and napi fields. The napi instances and associated queue[s] can be retrieved this way.Refactored netdev-genl to support exposing napi<->queue[s] mapping that is retained in a netdev.
Signed-off-by Amritha Nambiar <[email protected]>--- Documentation/netlink/specs/netdev.yaml | 39 +++++ include/uapi/linux/netdev.h | 4 + net/core/netdev-genl.c | 239 ++++++++++++++++++++++++++----- tools/include/uapi/linux/netdev.h | 4 + 4 files changed, 247 insertions(+), 39 deletions(-)Documentation/netlink/specs/netdev.yaml +0
-0
Documentation/netlink/specs/netdev.yaml attribute-sets: · line 69
Quoted 8 lines
Documentation/netlink/specs/netdev.yaml +5
-0@@ -62,6 +62,44 @@ attribute-sets: type: u64 enum: xdp-act enum-as-flags: true + - + name: napi-info + doc: napi information such as napi-id, napi queues etc. + type: nest + multi-attr: true
Let's make a new attr space for the napi info command. We don't reuse much of the attributes, and as the commands grow stuffing all attrs into one space makes finding stuff harder.
Documentation/netlink/specs/netdev.yaml attribute-sets: · line 70
Quoted 1 line
Documentation/netlink/specs/netdev.yaml +1
-0+ nested-attributes: dev-napi-info
And what's inside this nest should also be a separate attr space.
Documentation/netlink/specs/netdev.yaml attribute-sets: · line 78
Quoted 8 lines
Documentation/netlink/specs/netdev.yaml +8
-0+ - + name: napi-id + doc: napi id + type: u32 + - + name: rx-queues + doc: list of rx queues associated with a napi + type: u16
Make it u32, at the uAPI level we're tried to the width of fields, and u16 ends up being the same size as u32 "on the wire" due to padding.
Documentation/netlink/specs/netdev.yaml attribute-sets: · line 87
Quoted 11 lines
Documentation/netlink/specs/netdev.yaml +9
-0+ multi-attr: true + - + name: tx-queues + doc: list of tx queues associated with a napi + type: u16 + multi-attr: true + - + name: dev-napi-info + subset-of: dev
Yeah, this shouldn't be a subset just a full-on separate attr space. The handshake family may be a good example to look at, it's the biggest so far written with the new rules in mind.
Quoted 19 lines
Documentation/netlink/specs/netdev.yaml +15
-0+ attributes: + - + name: napi-id + doc: napi id + type: u32 + - + name: rx-queues + doc: list rx of queues associated with a napi + type: u16 + multi-attr: true + - + name: tx-queues + doc: list tx of queues associated with a napi + type: u16 + multi-attr: trueoperations:
list:
Documentation/netlink/specs/netdev.yaml operations: · line 118
Quoted 4 lines
Documentation/netlink/specs/netdev.yaml +1
-0@@ -77,6 +115,7 @@ operations: attributes: - ifindex - xdp-features + - napi-info
Aaah, separate command, please. Let's not stuff all the information into a single command like we did for rtnl.
Quoted 3 lines
Documentation/netlink/specs/netdev.yaml +0
-0dump: reply: *dev-all -
Re: [net-next/RFC PATCH v1 4/4] netdev-genl: Add support for exposing napi info from netdev
Quoted 26 lines · Jakub Kicinski
Quoted 25 lines · On Thu, 01 Jun 2023 10:42:41 -0700 Amritha Nambiar wrote:
Add support in
ynl/netdev.yamlfor napi related information. The netdev structure tracks all the napi instances and napi fields. The napi instances and associated queue[s] can be retrieved this way.Refactored netdev-genl to support exposing napi<->queue[s] mapping that is retained in a netdev.
Signed-off-by Amritha Nambiar <[email protected]>--- Documentation/netlink/specs/netdev.yaml | 39 +++++ include/uapi/linux/netdev.h | 4 + net/core/netdev-genl.c | 239 ++++++++++++++++++++++++++----- tools/include/uapi/linux/netdev.h | 4 + 4 files changed, 247 insertions(+), 39 deletions(-)Documentation/netlink/specs/netdev.yaml +2
-0@@ -62,6 +62,44 @@ attribute-sets: type: u64 enum: xdp-act enum-as-flags: true + - + name: napi-info
Quoted 9 lines
Documentation/netlink/specs/netdev.yaml attribute-sets: · line 69
Quoted 3 lines
Documentation/netlink/specs/netdev.yaml +3
-0+ doc: napi information such as napi-id, napi queues etc. + type: nest + multi-attr: trueLet's make a new attr space for the napi info command. We don't reuse much of the attributes, and as the commands grow stuffing all attrs into one space makes finding stuff harder.
Agree. I should not have overloaded the 'dev' command to begin with.
Quoted 4 lines
Documentation/netlink/specs/netdev.yaml attribute-sets: · line 70
Quoted 1 line
Documentation/netlink/specs/netdev.yaml +1
-0+ nested-attributes: dev-napi-infoAnd what's inside this nest should also be a separate attr space.
So, I think we could have two new commands for napi data. Would this be acceptable, a 'napi-queue-get' command for napi-queue specific information (set of TX and RX queues, IRQ number etc.), and another 'napi-info-get' for other information, such as PID for the napi thread, CPU etc.
Example:
$ ./cli.py --spec netdev.yaml --do napi-queue-get --json='{"ifindex":
12}'[{'napi-info': [{'napi-id': 600, 'rx-queues': [7], 'tx-queues': [7], 'irq': 298},
{'napi-id': 599, 'rx-queues': [6], 'tx-queues': [6],
'irq': 297},
{'napi-id': 598, 'rx-queues': [5], 'tx-queues': [5],
'irq': 296},
{'napi-id': 597, 'rx-queues': [4], 'tx-queues': [4],
'irq': 295},
{'napi-id': 596, 'rx-queues': [3], 'tx-queues': [3],
'irq': 294},
{'napi-id': 595, 'rx-queues': [2], 'tx-queues': [2],
'irq': 293},
{'napi-id': 594, 'rx-queues': [1], 'tx-queues': [1],
'irq': 292},
{'napi-id': 593, 'rx-queues': [0], 'tx-queues': [0], 'irq': 291}]}]
$ ./cli.py --spec netdev.yaml --do napi-info-get --json='{"ifindex": 12}'[{'napi-info': [{'napi-id': 600, 'pid': 68114},
{'napi-id': 599, 'pid': 68113},
{'napi-id': 598, 'pid': 68112},
{'napi-id': 597, 'pid': 68111},
{'napi-id': 596, 'pid': 68110},
{'napi-id': 595, 'pid': 68109},
{'napi-id': 594, 'pid': 68108},
{'napi-id': 593, 'pid': 68107}]}]Quoted 10 lines
Documentation/netlink/specs/netdev.yaml attribute-sets: · line 78
Quoted 9 lines
Documentation/netlink/specs/netdev.yaml +8
-0+ - + name: napi-id + doc: napi id + type: u32 + - + name: rx-queues + doc: list of rx queues associated with a napi + type: u16
Quoted 3 lines
Make it u32, at the uAPI level we're tried to the width of fields, and u16 ends up being the same size as u32 "on the wire" due to padding.
Makes sense. Will fix.
Quoted 8 lines
Quoted 6 lines
Documentation/netlink/specs/netdev.yaml +6
-0+ multi-attr: true + - + name: tx-queues + doc: list of tx queues associated with a napi + type: u16 + multi-attr: true
Quoted 8 lines
Documentation/netlink/specs/netdev.yaml attribute-sets: · line 87
Quoted 3 lines
Documentation/netlink/specs/netdev.yaml +3
-0+ - + name: dev-napi-info + subset-of: devYeah, this shouldn't be a subset just a full-on separate attr space. The handshake family may be a good example to look at, it's the biggest so far written with the new rules in mind.
Okay.
Quoted 16 lines
Quoted 16 lines
Documentation/netlink/specs/netdev.yaml +15
-0+ attributes: + - + name: napi-id + doc: napi id + type: u32 + - + name: rx-queues + doc: list rx of queues associated with a napi + type: u16 + multi-attr: true + - + name: tx-queues + doc: list tx of queues associated with a napi + type: u16 + multi-attr: true
Quoted 11 lines
Documentation/netlink/specs/netdev.yaml operations: · line 118
Quoted 7 lines
Documentation/netlink/specs/netdev.yaml +1
-0operations: list:@@ -77,6 +115,7 @@ operations: attributes: - ifindex - xdp-features + - napi-infoAaah, separate command, please. Let's not stuff all the information into a single command like we did for rtnl.
Okay.
Quoted 3 lines
Quoted 3 lines
Documentation/netlink/specs/netdev.yaml +0
-0dump: reply: *dev-all -
Jakub Kicinski
maintainerRe: [net-next/RFC PATCH v1 4/4] netdev-genl: Add support for exposing napi info from netdev
Quoted 6 lines · Amritha
So, I think we could have two new commands for napi data. Would this be acceptable, a 'napi-queue-get' command for napi-queue specific information (set of TX and RX queues, IRQ number etc.), and another 'napi-info-get' for other information, such as PID for the napi thread, CPU etc.
Quoted 6 lines
Example:
$ ./cli.py --spec netdev.yaml --do napi-queue-get --json='{"ifindex": 12}'[{'napi-info': [{'napi-id': 600, 'rx-queues': [7], 'tx-queues': [7], 'irq': 298},
I think the commands make sense. You should echo back the ifindex, tho, it threw me off initially that there's only on attribute ('napi-info') in the reply, in which case the nest would have been pointless..
Jakub Kicinski
maintainerQuoted 8 lines · On Thu, 01 Jun 2023 10:42:20 -0700 Amritha Nambiar wrote:
Introduce support for associating napi instances with corresponding RX and TX queue set. Add the capability to export napi information supported by the device. Extend the
netdev_genlgeneric netlink family for netdev with napi data. The napi fields exposed are:
- napi id
- queue/queue-set (both RX and TX) associated with each napi instance
Would you mind throwing in the IRQ vector number already? Should be pretty easy to find the IRQ from NAPI, and it'd make this code immediately very useful for IRQ pinning.
Quoted 7 lines
Additional napi fields such as PID association for napi thread etc. can be supported in a follow-on patch set.
This series only supports 'get' ability for retrieving napi fields (specifically, napi ids and queue[s]). The 'set' ability for setting queue[s] associated with a napi instance via netdev-genl will be submitted as a separate patch series.
Quoted 10 lines · Jakub Kicinski
Quoted 8 lines · On Thu, 01 Jun 2023 10:42:20 -0700 Amritha Nambiar wrote:
Introduce support for associating napi instances with corresponding RX and TX queue set. Add the capability to export napi information supported by the device. Extend the
netdev_genlgeneric netlink family for netdev with napi data. The napi fields exposed are:
- napi id
- queue/queue-set (both RX and TX) associated with each napi instance
Quoted 4 lines
Would you mind throwing in the IRQ vector number already? Should be pretty easy to find the IRQ from NAPI, and it'd make this code immediately very useful for IRQ pinning.
Sorry for the delayed response as I was on vacation.
Sure, I can add in the IRQ vector number as well in the next version of the series.
Quoted 7 lines
Quoted 7 lines
Additional napi fields such as PID association for napi thread etc. can be supported in a follow-on patch set.
This series only supports 'get' ability for retrieving napi fields (specifically, napi ids and queue[s]). The 'set' ability for setting queue[s] associated with a napi instance via netdev-genl will be submitted as a separate patch series.