net: mana: Optimize irq affinity for low vcpu configs
Subsystems (5)
PATCH v3 1/1 net: mana: Optimize irq affinity for low vcpu configs
In mana driver, the number of IRQs allocated is capped by the min(num_cpu + 1, queue count). In cases, where the IRQ count is greater than the vcpu count, we want to utilize all the vCPUs, irrespective of their NUMA/core bindings.
This is important, especially in the envs where number of vCPUs are so few that the softIRQ handling overhead on two IRQs on the same vCPU is much more than their overheads if they were spread across sibling vCPUs.
This behaviour is more evident with dynamic IRQ allocation. Since MANA IRQs are assigned at a later stage compared to static allocation, other device IRQs may already be affinitized to the vCPUs. As a result, IRQ weights become imbalanced, causing multiple MANA IRQs to land on the same vCPU, while some vCPUs have none.
In such cases when many parallel TCP connections are tested, the throughput drops significantly.
Test envs:
=======================================================
Case 1: without this patch
=======================================================
4 vcpu(2 cores), 5 MANA IRQs (1 HWC + 4 Queue)
TYPE effective vCPU aff
=======================================================
IRQ0: HWC 0
IRQ1: mana_q1 0
IRQ2: mana_q2 2
IRQ3: mana_q3 0
IRQ4: mana_q4 3
%soft on each vCPU(mpstat -P ALL 1) on receiver
vCPU 0 1 2 3
=======================================================
pass 1: 38.85 0.03 24.89 24.65
pass 2: 39.15 0.03 24.57 25.28
pass 3: 40.36 0.03 23.20 23.17
=======================================================
Case 2: with this patch
=======================================================
4 vcpu(2 cores), 5 MANA IRQs (1 HWC + 4 Queue)
TYPE effective vCPU aff
=======================================================
IRQ0: HWC 0 IRQ1: mana_q1 0 IRQ2: mana_q2 1 IRQ3: mana_q3 2 IRQ4: mana_q4 3
%soft on each vCPU(mpstat -P ALL 1) on receiver
vCPU 0 1 2 3
=======================================================
pass 1: 15.42 15.85 14.99 14.51 pass 2: 15.53 15.94 15.81 15.93 pass 3: 16.41 16.35 16.40 16.36
=======================================================
Throughput Impact(in Gbps, same env)
=======================================================
TCP conn with patch w/o patch
20480 15.65 7.73
10240 15.63 8.93
8192 15.64 9.69
6144 15.64 13.16
4096 15.69 15.75
2048 15.69 15.83
1024 15.71 15.28
---
Changes in v3
* Optimize the comments in mana_gd_setup_dyn_irqs()
* add more details in the dev_dbg for extra IRQs
---
Changes in v2
* Removed the unused skip_first_cpu variable
* fixed exit condition in irq_setup_linear() with len == 0
* changed return type of irq_setup_linear() as it will always be 0
* removed the unnecessary rcu_read_lock() in irq_setup_linear()
* added appropriate comments to indicate expected behaviour when IRQs are more than or equal to num_online_cpus()
--- .../net/ethernet/microsoft/mana/gdma_main.c | 60 ++++++++++++++++--- 1 file changed, 53 insertions(+), 7 deletions(-)
drivers/net/ethernet/microsoft/mana/gdma_main.c +53 -7
@@ -197,6 +197,8 @@ static int mana_gd_query_max_resources(struct pci_dev *pdev) }else {/* If dynamic allocation is enabled we have already allocated * hwc msi +* Also, we make sure in this case the following is always true +* (num_msix_usable - 1 HWC) <= num_online_cpus() */ gc->num_msix_usable =min (resp.max_msix,num_online_cpus () +1 ); }
@@ -1717,11 +1719,24 @@ static int irq_setup(unsigned int *irqs, unsigned int len, int node,return 0 ; } +/* should be called with cpus_read_lock() held */ +static void irq_setup_linear (unsigned int *irqs,unsigned int len) +{ +int cpu; + +for_each_online_cpu (cpu) { +if (len ==0 ) +break ; + +irq_set_affinity_and_hint (*irqs++,cpumask_of (cpu)); + len--; + } +} +static int mana_gd_setup_dyn_irqs (struct pci_dev *pdev,int nvec) {struct gdma_context *gc =pci_get_drvdata (pdev);struct gdma_irq_context *gic; -bool skip_first_cpu = false;int *irqs, irq, err, i; irqs =kmalloc_objs (int, nvec);
@@ -1729,6 +1744,8 @@ static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec)return -ENOMEM ;/* +* In this function, num_msix_usable = HWC IRQ + Queue IRQ. +* nvec is only Queue IRQ (HWC already setup). * While processing the next pci irq vector, we start with index 1, * as IRQ vector at index 0 is already processed for HWC. * However, the population of irqs array starts with index 0, to be
@@ -1767,13 +1784,42 @@ static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec)* first CPU sibling group since they are already affinitized to HWC IRQ */ cpus_read_lock (); -if (gc->num_msix_usable <=num_online_cpus ()) - skip_first_cpu = true; +if (gc->num_msix_usable <=num_online_cpus ()) { + err =irq_setup (irqs, nvec, gc->numa_node, true); +if (err) { +cpus_read_unlock (); + goto free_irq; + } + }else { +/* +* When num_msix_usable are more than num_online_cpus, our +* queue IRQs should be equal to num of online vCPUs. +* We try to make sure queue IRQs spread across all vCPUs. +* In such a case NUMA or CPU core affinity does not matter. +* Note: in this case the total mana IRQ should always be +* num_online_cpus + 1. The first HWC IRQ is already handled +* in HWC setup calls +* However, if CPUs went offline since num_msix_usable was +* computed, queue IRQs will be more than num_online_cpus(). +* In such cases remaining extra IRQs will retain their default +* affinity. +*/ +int first_unassigned =num_online_cpus (); +if (nvec > first_unassigned) { +char buf[32 ]; + +if (first_unassigned == nvec -1 ) +snprintf (buf,sizeof (buf),"%d" , + first_unassigned); +else +snprintf (buf,sizeof (buf),"%d-%d" , + first_unassigned, nvec -1 ); + +dev_dbg (&pdev->dev, +"MANA IRQ indices #%s will retain the default CPU affinity\n" , buf); + } - err =irq_setup (irqs, nvec, gc->numa_node, skip_first_cpu); -if (err) { -cpus_read_unlock (); - goto free_irq; +irq_setup_linear (irqs, nvec); }cpus_read_unlock ();
base-commit: 8415598365503ced2e3d019491b0a2756c85c494
Signature · 1 line
2.34.1
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] Missing error check for xa_store() leads to resource leak and potential kernel oops on allocation failure. --
commit 853688f50dc4ca56dd74698a3ecfdfeaf857e716
Author: Shradha Gupta <[email protected]>
net: mana: Optimize irq affinity for low vcpu configs
This patch optimizes irq affinity in the mana driver for low vCPU environments. When the number of allocated irqs is greater than the vCPU count, it ensures that queue irqs are spread across all available vCPUs to prevent softirq handling imbalances.
Quoted 3 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +0
-0
[ ... ]
drivers/net/ethernet/microsoft/mana/gdma_main.c static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec) · line 1791
Quoted 11 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +5
-2@@ -1767,13 +1784,42 @@ static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec)* first CPU sibling group since they are already affinitized to HWC IRQ */ cpus_read_lock (); -if (gc->num_msix_usable <=num_online_cpus ()) - skip_first_cpu = true; +if (gc->num_msix_usable <=num_online_cpus ()) { + err =irq_setup (irqs, nvec, gc->numa_node, true); +if (err) { +cpus_read_unlock (); + goto free_irq;
[Severity: High]
This isn't a bug introduced by this patch, but looking at the error handling path for goto free_irq, is there a potential resource leak if xa_store() fails earlier in the function?
When setting up the irqs in mana_gd_setup_dyn_irqs(), we call xa_store() with GFP_KERNEL:
xa_store(&gc->irq_contexts, i, gic, GFP_KERNEL);
However, the return value is not checked. If this allocation fails, the gic pointer is not saved. If a subsequent operation like irq_setup() fails and we jump to the free_irq label, the cleanup loop attempts to retrieve it:
gic = xa_load(&gc->irq_contexts, i);
if (WARN_ON(!gic))
continue;Because the store failed, xa_load() will return NULL, causing the loop to continue without calling free_irq() or kfree(gic). Could this result in an un-freed irq handler that might trigger a panic later if the module is unloaded and the hardware raises an interrupt?
Quoted 2 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +2
-0+ } + }else {
[ ... ]
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Quoted 50 lines · On Tue, Jun 02, 2026 at 10:28:11AM +0000, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] Missing error check forxa_store()leads to resource leak and potential kernel oops on allocation failure. --commit
853688f50dc4ca56dd74698a3ecfdfeaf857e716
Author: Shradha Gupta <[email protected]>net: mana: Optimize irq affinity for low vcpu configs
This patch optimizes irq affinity in the mana driver for low vCPU environments. When the number of allocated irqs is greater than the vCPU count, it ensures that queue irqs are spread across all available vCPUs to prevent softirq handling imbalances.
Quoted 3 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +0
-0[ ... ]
drivers/net/ethernet/microsoft/mana/gdma_main.c static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec) · line 1791
Quoted 11 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +5
-2@@ -1767,13 +1784,42 @@ static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec)* first CPU sibling group since they are already affinitized to HWC IRQ */ cpus_read_lock (); -if (gc->num_msix_usable <=num_online_cpus ()) - skip_first_cpu = true; +if (gc->num_msix_usable <=num_online_cpus ()) { + err =irq_setup (irqs, nvec, gc->numa_node, true); +if (err) { +cpus_read_unlock (); + goto free_irq;[Severity: High]
This isn't a bug introduced by this patch, but looking at the error handling path for gotofree_irq, is there a potential resource leak ifxa_store()fails earlier in the function?When setting up the irqs in
mana_gd_setup_dyn_irqs(), we callxa_store()withGFP_KERNEL:xa_store(&gc->irq_contexts, i, gic, GFP_KERNEL);However, the return value is not checked. If this allocation fails, the gic pointer is not saved. If a subsequent operation like
irq_setup()fails and we jump to thefree_irqlabel, the cleanup loop attempts to retrieve it:gic = xa_load(&gc->irq_contexts, i); if (WARN_ON(!gic)) continue;
Quoted 11 lines
Because the store failed,
xa_load()will return NULL, causing the loop to continue without callingfree_irq()or kfree(gic). Could this result in an un-freed irq handler that might trigger a panic later if the module is unloaded and the hardware raises an interrupt?Quoted 2 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +2
-0+ } + }else {[ ... ]
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Since this isn't a bug introduced by this patch, I will fix it in a seperate patch. Will submit the fix for this bug by next week.
Thanks,
Shradha.
Quoted 114 lines · Shradha Gupta
In mana driver, the number of IRQs allocated is capped by the
min(num_cpu + 1, queue count). In cases, where the IRQ count is greater than the vcpu count, we want to utilize all the vCPUs, irrespective of their NUMA/core bindings.This is important, especially in the envs where number of vCPUs are so few that the softIRQ handling overhead on two IRQs on the same vCPU is much more than their overheads if they were spread across sibling vCPUs.
This behaviour is more evident with dynamic IRQ allocation. Since MANA IRQs are assigned at a later stage compared to static allocation, other device IRQs may already be affinitized to the vCPUs. As a result, IRQ weights become imbalanced, causing multiple MANA IRQs to land on the same vCPU, while some vCPUs have none.
In such cases when many parallel TCP connections are tested, the throughput drops significantly.
Test envs:
=======================================================
Case 1: without this patch
=======================================================
4 vcpu(2 cores), 5 MANA IRQs (1 HWC + 4 Queue)TYPE effective vCPU aff
=======================================================
IRQ0: HWC 0
IRQ1:mana_q10
IRQ2:mana_q22
IRQ3:mana_q30
IRQ4:mana_q43%soft on each vCPU(mpstat -P ALL 1) on receiver
vCPU 0 1 2 3
=======================================================
pass 1: 38.85 0.03 24.89 24.65
pass 2: 39.15 0.03 24.57 25.28
pass 3: 40.36 0.03 23.20 23.17=======================================================
Case 2: with this patch
=======================================================
4 vcpu(2 cores), 5 MANA IRQs (1 HWC + 4 Queue)TYPE effective vCPU aff
=======================================================IRQ0: HWC 0 IRQ1: mana_q1 0 IRQ2: mana_q2 1 IRQ3: mana_q3 2 IRQ4: mana_q4 3%soft on each vCPU(mpstat -P ALL 1) on receiver
vCPU 0 1 2 3
=======================================================pass 1: 15.42 15.85 14.99 14.51 pass 2: 15.53 15.94 15.81 15.93 pass 3: 16.41 16.35 16.40 16.36=======================================================
ThroughputImpact(in Gbps, same env)
=======================================================
TCP conn with patch w/o patch
20480 15.65 7.73
10240 15.63 8.93
8192 15.64 9.69
6144 15.64 13.16
4096 15.69 15.75
2048 15.69 15.83
1024 15.71 15.28Fixes 755391121038 ("net: mana: Allocate MSI-X vectors dynamically")Co-developed-by Erni Sri Satya Vennela <[email protected]>Signed-off-by Erni Sri Satya Vennela <[email protected]>Signed-off-by Shradha Gupta <[email protected]>Reviewed-by Haiyang Zhang <[email protected]>Reviewed-by Simon Horman <[email protected]>---
Changes in v3
* Optimize the comments inmana_gd_setup_dyn_irqs()
* add more details in thedev_dbgfor extra IRQs
---
Changes in v2
* Removed the unusedskip_first_cpuvariable
* fixed exit condition inirq_setup_linear()with len == 0
* changed return type ofirq_setup_linear()as it will always be 0
* removed the unnecessaryrcu_read_lock()inirq_setup_linear()
* added appropriate comments to indicate expected behaviour when IRQs are more than or equal tonum_online_cpus()--- .../net/ethernet/microsoft/mana/gdma_main.c | 60 ++++++++++++++++--- 1 file changed, 53 insertions(+), 7 deletions(-)drivers/net/ethernet/microsoft/mana/gdma_main.c +5
-0@@ -197,6 +197,8 @@ static int mana_gd_query_max_resources(struct pci_dev *pdev) }else {/* If dynamic allocation is enabled we have already allocated * hwc msi +* Also, we make sure in this case the following is always true +* (num_msix_usable - 1 HWC) <= num_online_cpus() */ gc->num_msix_usable =min (resp.max_msix,num_online_cpus () +1 ); }@@ -1717,11 +1719,24 @@ static int irq_setup(unsigned int *irqs, unsigned int len, int node,return 0 ; } +/* should be called with cpus_read_lock() held */ +static void irq_setup_linear (unsigned int *irqs,unsigned int len) +{
drivers/net/ethernet/microsoft/mana/gdma_main.c static int irq_setup(unsigned int *irqs, unsigned int len, int node, · line 1734
Quoted 10 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +10
-0+int cpu; + +for_each_online_cpu (cpu) { +if (len ==0 ) +break ; + +irq_set_affinity_and_hint (*irqs++,cpumask_of (cpu)); + len--; + } +}
I would find all of this a bit easier to follow if irq_setup_linear() and irq_setup() had a mana prefix so it was more obvious these are specific to the driver. Of course irq_setup is pre-existing, and its not my driver so do as you will :)
Quoted 54 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +33
-3+static int mana_gd_setup_dyn_irqs (struct pci_dev *pdev,int nvec) {struct gdma_context *gc =pci_get_drvdata (pdev);struct gdma_irq_context *gic; -bool skip_first_cpu = false;int *irqs, irq, err, i; irqs =kmalloc_objs (int, nvec);@@ -1729,6 +1744,8 @@ static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec)return -ENOMEM ;/* +* In this function, num_msix_usable = HWC IRQ + Queue IRQ. +* nvec is only Queue IRQ (HWC already setup). * While processing the next pci irq vector, we start with index 1, * as IRQ vector at index 0 is already processed for HWC. * However, the population of irqs array starts with index 0, to be @@ -1767,13 +1784,42 @@ static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec)* first CPU sibling group since they are already affinitized to HWC IRQ */ cpus_read_lock (); -if (gc->num_msix_usable <=num_online_cpus ()) - skip_first_cpu = true; +if (gc->num_msix_usable <=num_online_cpus ()) { + err =irq_setup (irqs, nvec, gc->numa_node, true); +if (err) { +cpus_read_unlock (); + goto free_irq; + } + }else { +/* +* When num_msix_usable are more than num_online_cpus, our +* queue IRQs should be equal to num of online vCPUs. +* We try to make sure queue IRQs spread across all vCPUs. +* In such a case NUMA or CPU core affinity does not matter. +* Note: in this case the total mana IRQ should always be +* num_online_cpus + 1. The first HWC IRQ is already handled +* in HWC setup calls +* However, if CPUs went offline since num_msix_usable was +* computed, queue IRQs will be more than num_online_cpus(). +* In such cases remaining extra IRQs will retain their default +* affinity. +*/ +int first_unassigned =num_online_cpus (); +if (nvec > first_unassigned) { +char buf[32 ]; + +if (first_unassigned == nvec -1 ) +snprintf (buf,sizeof (buf),"%d" , + first_unassigned); +else +snprintf (buf,sizeof (buf),"%d-%d" , + first_unassigned, nvec -1 );
drivers/net/ethernet/microsoft/mana/gdma_main.c static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec) · line 1822
Quoted 10 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +5
-4+ +dev_dbg (&pdev->dev, +"MANA IRQ indices #%s will retain the default CPU affinity\n" , buf); + } - err =irq_setup (irqs, nvec, gc->numa_node, skip_first_cpu); -if (err) { -cpus_read_unlock (); - goto free_irq; +irq_setup_linear (irqs, nvec);
irq_setup() doesn't have a driver prefix, but is actually a static function in gdma_main.c, so its implementation is specific to this driver despite its name.
So if I understand this change correctly, if the number of usable MSI-X vectors is smaller than the number of CPUs, you contineu to use the current irq_setup logic.. otherwise you switch to the simpler "linear" logic.
I guess this means the logic and heuristic used in irq_setup() breaks down when the number of vectors is large and number of vCPU is small?
Makes sense.
Quoted 5 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +0
-0}cpus_read_unlock ();base-commit:
8415598365503ced2e3d019491b0a2756c85c494
Quoted 205 lines · Jacob Keller
Quoted 114 lines · Shradha Gupta
In mana driver, the number of IRQs allocated is capped by the
min(num_cpu + 1, queue count). In cases, where the IRQ count is greater than the vcpu count, we want to utilize all the vCPUs, irrespective of their NUMA/core bindings.This is important, especially in the envs where number of vCPUs are so few that the softIRQ handling overhead on two IRQs on the same vCPU is much more than their overheads if they were spread across sibling vCPUs.
This behaviour is more evident with dynamic IRQ allocation. Since MANA IRQs are assigned at a later stage compared to static allocation, other device IRQs may already be affinitized to the vCPUs. As a result, IRQ weights become imbalanced, causing multiple MANA IRQs to land on the same vCPU, while some vCPUs have none.
In such cases when many parallel TCP connections are tested, the throughput drops significantly.
Test envs:
=======================================================
Case 1: without this patch
=======================================================
4 vcpu(2 cores), 5 MANA IRQs (1 HWC + 4 Queue)TYPE effective vCPU aff
=======================================================
IRQ0: HWC 0
IRQ1:mana_q10
IRQ2:mana_q22
IRQ3:mana_q30
IRQ4:mana_q43%soft on each vCPU(mpstat -P ALL 1) on receiver
vCPU 0 1 2 3
=======================================================
pass 1: 38.85 0.03 24.89 24.65
pass 2: 39.15 0.03 24.57 25.28
pass 3: 40.36 0.03 23.20 23.17=======================================================
Case 2: with this patch
=======================================================
4 vcpu(2 cores), 5 MANA IRQs (1 HWC + 4 Queue)TYPE effective vCPU aff
=======================================================IRQ0: HWC 0 IRQ1: mana_q1 0 IRQ2: mana_q2 1 IRQ3: mana_q3 2 IRQ4: mana_q4 3%soft on each vCPU(mpstat -P ALL 1) on receiver
vCPU 0 1 2 3
=======================================================pass 1: 15.42 15.85 14.99 14.51 pass 2: 15.53 15.94 15.81 15.93 pass 3: 16.41 16.35 16.40 16.36=======================================================
ThroughputImpact(in Gbps, same env)
=======================================================
TCP conn with patch w/o patch
20480 15.65 7.73
10240 15.63 8.93
8192 15.64 9.69
6144 15.64 13.16
4096 15.69 15.75
2048 15.69 15.83
1024 15.71 15.28Fixes 755391121038 ("net: mana: Allocate MSI-X vectors dynamically")Co-developed-by Erni Sri Satya Vennela <[email protected]>Signed-off-by Erni Sri Satya Vennela <[email protected]>Signed-off-by Shradha Gupta <[email protected]>Reviewed-by Haiyang Zhang <[email protected]>Reviewed-by Simon Horman <[email protected]>---
Changes in v3
* Optimize the comments inmana_gd_setup_dyn_irqs()
* add more details in thedev_dbgfor extra IRQs
---
Changes in v2
* Removed the unusedskip_first_cpuvariable
* fixed exit condition inirq_setup_linear()with len == 0
* changed return type ofirq_setup_linear()as it will always be 0
* removed the unnecessaryrcu_read_lock()inirq_setup_linear()
* added appropriate comments to indicate expected behaviour when IRQs are more than or equal tonum_online_cpus()--- .../net/ethernet/microsoft/mana/gdma_main.c | 60 ++++++++++++++++--- 1 file changed, 53 insertions(+), 7 deletions(-)drivers/net/ethernet/microsoft/mana/gdma_main.c +5
-0@@ -197,6 +197,8 @@ static int mana_gd_query_max_resources(struct pci_dev *pdev) }else {/* If dynamic allocation is enabled we have already allocated * hwc msi +* Also, we make sure in this case the following is always true +* (num_msix_usable - 1 HWC) <= num_online_cpus() */ gc->num_msix_usable =min (resp.max_msix,num_online_cpus () +1 ); }@@ -1717,11 +1719,24 @@ static int irq_setup(unsigned int *irqs, unsigned int len, int node,return 0 ; } +/* should be called with cpus_read_lock() held */ +static void irq_setup_linear (unsigned int *irqs,unsigned int len) +{drivers/net/ethernet/microsoft/mana/gdma_main.c static int irq_setup(unsigned int *irqs, unsigned int len, int node, · line 1734
Quoted 10 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +10
-0+int cpu; + +for_each_online_cpu (cpu) { +if (len ==0 ) +break ; + +irq_set_affinity_and_hint (*irqs++,cpumask_of (cpu)); + len--; + } +}I would find all of this a bit easier to follow if
irq_setup_linear()andirq_setup()had a mana prefix so it was more obvious these are specific to the driver. Of courseirq_setupis pre-existing, and its not my driver so do as you will :)Quoted 54 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +33
-3+static int mana_gd_setup_dyn_irqs (struct pci_dev *pdev,int nvec) {struct gdma_context *gc =pci_get_drvdata (pdev);struct gdma_irq_context *gic; -bool skip_first_cpu = false;int *irqs, irq, err, i; irqs =kmalloc_objs (int, nvec);@@ -1729,6 +1744,8 @@ static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec)return -ENOMEM ;/* +* In this function, num_msix_usable = HWC IRQ + Queue IRQ. +* nvec is only Queue IRQ (HWC already setup). * While processing the next pci irq vector, we start with index 1, * as IRQ vector at index 0 is already processed for HWC. * However, the population of irqs array starts with index 0, to be @@ -1767,13 +1784,42 @@ static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec)* first CPU sibling group since they are already affinitized to HWC IRQ */ cpus_read_lock (); -if (gc->num_msix_usable <=num_online_cpus ()) - skip_first_cpu = true; +if (gc->num_msix_usable <=num_online_cpus ()) { + err =irq_setup (irqs, nvec, gc->numa_node, true); +if (err) { +cpus_read_unlock (); + goto free_irq; + } + }else { +/* +* When num_msix_usable are more than num_online_cpus, our +* queue IRQs should be equal to num of online vCPUs. +* We try to make sure queue IRQs spread across all vCPUs. +* In such a case NUMA or CPU core affinity does not matter. +* Note: in this case the total mana IRQ should always be +* num_online_cpus + 1. The first HWC IRQ is already handled +* in HWC setup calls +* However, if CPUs went offline since num_msix_usable was +* computed, queue IRQs will be more than num_online_cpus(). +* In such cases remaining extra IRQs will retain their default +* affinity. +*/ +int first_unassigned =num_online_cpus (); +if (nvec > first_unassigned) { +char buf[32 ]; + +if (first_unassigned == nvec -1 ) +snprintf (buf,sizeof (buf),"%d" , + first_unassigned); +else +snprintf (buf,sizeof (buf),"%d-%d" , + first_unassigned, nvec -1 );drivers/net/ethernet/microsoft/mana/gdma_main.c static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec) · line 1822
Quoted 10 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +5
-4+ +dev_dbg (&pdev->dev, +"MANA IRQ indices #%s will retain the default CPU affinity\n" , buf); + } - err =irq_setup (irqs, nvec, gc->numa_node, skip_first_cpu); -if (err) { -cpus_read_unlock (); - goto free_irq; +irq_setup_linear (irqs, nvec);
irq_setup()doesn't have a driver prefix, but is actually a static function ingdma_main.c, so its implementation is specific to this driver despite its name.So if I understand this change correctly, if the number of usable MSI-X vectors is smaller than the number of CPUs, you contineu to use the current
irq_setuplogic.. otherwise you switch to the simpler "linear" logic.
Quoted 5 lines
I guess this means the logic and heuristic used in
irq_setup()breaks down when the number of vectors is large and number of vCPU is small?Makes sense.
Hi Jacob,
Yes, that's the right understanding.
Regarding the function names, let me take that up in a seperate patch to add prefixes to all such functions.
Thanks.
Quoted 5 lines
Quoted 5 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +0
-0}cpus_read_unlock ();base-commit:
8415598365503ced2e3d019491b0a2756c85c494
Quoted 209 lines · Shradha Gupta
Quoted 208 lines · Jacob Keller
Quoted 114 lines · Shradha Gupta
In mana driver, the number of IRQs allocated is capped by the
min(num_cpu + 1, queue count). In cases, where the IRQ count is greater than the vcpu count, we want to utilize all the vCPUs, irrespective of their NUMA/core bindings.This is important, especially in the envs where number of vCPUs are so few that the softIRQ handling overhead on two IRQs on the same vCPU is much more than their overheads if they were spread across sibling vCPUs.
This behaviour is more evident with dynamic IRQ allocation. Since MANA IRQs are assigned at a later stage compared to static allocation, other device IRQs may already be affinitized to the vCPUs. As a result, IRQ weights become imbalanced, causing multiple MANA IRQs to land on the same vCPU, while some vCPUs have none.
In such cases when many parallel TCP connections are tested, the throughput drops significantly.
Test envs:
=======================================================
Case 1: without this patch
=======================================================
4 vcpu(2 cores), 5 MANA IRQs (1 HWC + 4 Queue)TYPE effective vCPU aff
=======================================================
IRQ0: HWC 0
IRQ1:mana_q10
IRQ2:mana_q22
IRQ3:mana_q30
IRQ4:mana_q43%soft on each vCPU(mpstat -P ALL 1) on receiver
vCPU 0 1 2 3
=======================================================
pass 1: 38.85 0.03 24.89 24.65
pass 2: 39.15 0.03 24.57 25.28
pass 3: 40.36 0.03 23.20 23.17=======================================================
Case 2: with this patch
=======================================================
4 vcpu(2 cores), 5 MANA IRQs (1 HWC + 4 Queue)TYPE effective vCPU aff
=======================================================IRQ0: HWC 0 IRQ1: mana_q1 0 IRQ2: mana_q2 1 IRQ3: mana_q3 2 IRQ4: mana_q4 3%soft on each vCPU(mpstat -P ALL 1) on receiver
vCPU 0 1 2 3
=======================================================pass 1: 15.42 15.85 14.99 14.51 pass 2: 15.53 15.94 15.81 15.93 pass 3: 16.41 16.35 16.40 16.36=======================================================
ThroughputImpact(in Gbps, same env)
=======================================================
TCP conn with patch w/o patch
20480 15.65 7.73
10240 15.63 8.93
8192 15.64 9.69
6144 15.64 13.16
4096 15.69 15.75
2048 15.69 15.83
1024 15.71 15.28Fixes 755391121038 ("net: mana: Allocate MSI-X vectors dynamically")Co-developed-by Erni Sri Satya Vennela <[email protected]>Signed-off-by Erni Sri Satya Vennela <[email protected]>Signed-off-by Shradha Gupta <[email protected]>Reviewed-by Haiyang Zhang <[email protected]>Reviewed-by Simon Horman <[email protected]>---
Changes in v3
* Optimize the comments inmana_gd_setup_dyn_irqs()
* add more details in thedev_dbgfor extra IRQs
---
Changes in v2
* Removed the unusedskip_first_cpuvariable
* fixed exit condition inirq_setup_linear()with len == 0
* changed return type ofirq_setup_linear()as it will always be 0
* removed the unnecessaryrcu_read_lock()inirq_setup_linear()
* added appropriate comments to indicate expected behaviour when IRQs are more than or equal tonum_online_cpus()--- .../net/ethernet/microsoft/mana/gdma_main.c | 60 ++++++++++++++++--- 1 file changed, 53 insertions(+), 7 deletions(-)drivers/net/ethernet/microsoft/mana/gdma_main.c +5
-0@@ -197,6 +197,8 @@ static int mana_gd_query_max_resources(struct pci_dev *pdev) }else {/* If dynamic allocation is enabled we have already allocated * hwc msi +* Also, we make sure in this case the following is always true +* (num_msix_usable - 1 HWC) <= num_online_cpus() */ gc->num_msix_usable =min (resp.max_msix,num_online_cpus () +1 ); }@@ -1717,11 +1719,24 @@ static int irq_setup(unsigned int *irqs, unsigned int len, int node,return 0 ; } +/* should be called with cpus_read_lock() held */ +static void irq_setup_linear (unsigned int *irqs,unsigned int len) +{drivers/net/ethernet/microsoft/mana/gdma_main.c static int irq_setup(unsigned int *irqs, unsigned int len, int node, · line 1734
Quoted 10 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +10
-0+int cpu; + +for_each_online_cpu (cpu) { +if (len ==0 ) +break ; + +irq_set_affinity_and_hint (*irqs++,cpumask_of (cpu)); + len--; + } +}I would find all of this a bit easier to follow if
irq_setup_linear()andirq_setup()had a mana prefix so it was more obvious these are specific to the driver. Of courseirq_setupis pre-existing, and its not my driver so do as you will :)Quoted 54 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +33
-3+static int mana_gd_setup_dyn_irqs (struct pci_dev *pdev,int nvec) {struct gdma_context *gc =pci_get_drvdata (pdev);struct gdma_irq_context *gic; -bool skip_first_cpu = false;int *irqs, irq, err, i; irqs =kmalloc_objs (int, nvec);@@ -1729,6 +1744,8 @@ static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec)return -ENOMEM ;/* +* In this function, num_msix_usable = HWC IRQ + Queue IRQ. +* nvec is only Queue IRQ (HWC already setup). * While processing the next pci irq vector, we start with index 1, * as IRQ vector at index 0 is already processed for HWC. * However, the population of irqs array starts with index 0, to be @@ -1767,13 +1784,42 @@ static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec)* first CPU sibling group since they are already affinitized to HWC IRQ */ cpus_read_lock (); -if (gc->num_msix_usable <=num_online_cpus ()) - skip_first_cpu = true; +if (gc->num_msix_usable <=num_online_cpus ()) { + err =irq_setup (irqs, nvec, gc->numa_node, true); +if (err) { +cpus_read_unlock (); + goto free_irq; + } + }else { +/* +* When num_msix_usable are more than num_online_cpus, our +* queue IRQs should be equal to num of online vCPUs. +* We try to make sure queue IRQs spread across all vCPUs. +* In such a case NUMA or CPU core affinity does not matter. +* Note: in this case the total mana IRQ should always be +* num_online_cpus + 1. The first HWC IRQ is already handled +* in HWC setup calls +* However, if CPUs went offline since num_msix_usable was +* computed, queue IRQs will be more than num_online_cpus(). +* In such cases remaining extra IRQs will retain their default +* affinity. +*/ +int first_unassigned =num_online_cpus (); +if (nvec > first_unassigned) { +char buf[32 ]; + +if (first_unassigned == nvec -1 ) +snprintf (buf,sizeof (buf),"%d" , + first_unassigned); +else +snprintf (buf,sizeof (buf),"%d-%d" , + first_unassigned, nvec -1 );drivers/net/ethernet/microsoft/mana/gdma_main.c static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec) · line 1822
Quoted 10 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +5
-4+ +dev_dbg (&pdev->dev, +"MANA IRQ indices #%s will retain the default CPU affinity\n" , buf); + } - err =irq_setup (irqs, nvec, gc->numa_node, skip_first_cpu); -if (err) { -cpus_read_unlock (); - goto free_irq; +irq_setup_linear (irqs, nvec);
irq_setup()doesn't have a driver prefix, but is actually a static function ingdma_main.c, so its implementation is specific to this driver despite its name.So if I understand this change correctly, if the number of usable MSI-X vectors is smaller than the number of CPUs, you contineu to use the current
irq_setuplogic.. otherwise you switch to the simpler "linear" logic.I guess this means the logic and heuristic used in
irq_setup()breaks down when the number of vectors is large and number of vCPU is small?
Quoted 8 lines
Quoted 2 lines
Makes sense.
Hi Jacob,
Yes, that's the right understanding.
Regarding the function names, let me take that up in a seperate patch to add prefixes to all such functions.
I agree. Now that you've got more than one setup method, short 'irq_setup' looks confusing, if not misleading. You need some name that distinguished numa-based and plain linear method.
Thanks,
Yury
Quoted 219 lines · Yury Norov
Quoted 209 lines · Shradha Gupta
Quoted 208 lines · Jacob Keller
Quoted 114 lines · Shradha Gupta
In mana driver, the number of IRQs allocated is capped by the
min(num_cpu + 1, queue count). In cases, where the IRQ count is greater than the vcpu count, we want to utilize all the vCPUs, irrespective of their NUMA/core bindings.This is important, especially in the envs where number of vCPUs are so few that the softIRQ handling overhead on two IRQs on the same vCPU is much more than their overheads if they were spread across sibling vCPUs.
This behaviour is more evident with dynamic IRQ allocation. Since MANA IRQs are assigned at a later stage compared to static allocation, other device IRQs may already be affinitized to the vCPUs. As a result, IRQ weights become imbalanced, causing multiple MANA IRQs to land on the same vCPU, while some vCPUs have none.
In such cases when many parallel TCP connections are tested, the throughput drops significantly.
Test envs:
=======================================================
Case 1: without this patch
=======================================================
4 vcpu(2 cores), 5 MANA IRQs (1 HWC + 4 Queue)TYPE effective vCPU aff
=======================================================
IRQ0: HWC 0
IRQ1:mana_q10
IRQ2:mana_q22
IRQ3:mana_q30
IRQ4:mana_q43%soft on each vCPU(mpstat -P ALL 1) on receiver
vCPU 0 1 2 3
=======================================================
pass 1: 38.85 0.03 24.89 24.65
pass 2: 39.15 0.03 24.57 25.28
pass 3: 40.36 0.03 23.20 23.17=======================================================
Case 2: with this patch
=======================================================
4 vcpu(2 cores), 5 MANA IRQs (1 HWC + 4 Queue)TYPE effective vCPU aff
=======================================================IRQ0: HWC 0 IRQ1: mana_q1 0 IRQ2: mana_q2 1 IRQ3: mana_q3 2 IRQ4: mana_q4 3%soft on each vCPU(mpstat -P ALL 1) on receiver
vCPU 0 1 2 3
=======================================================pass 1: 15.42 15.85 14.99 14.51 pass 2: 15.53 15.94 15.81 15.93 pass 3: 16.41 16.35 16.40 16.36=======================================================
ThroughputImpact(in Gbps, same env)
=======================================================
TCP conn with patch w/o patch
20480 15.65 7.73
10240 15.63 8.93
8192 15.64 9.69
6144 15.64 13.16
4096 15.69 15.75
2048 15.69 15.83
1024 15.71 15.28Fixes 755391121038 ("net: mana: Allocate MSI-X vectors dynamically")Co-developed-by Erni Sri Satya Vennela <[email protected]>Signed-off-by Erni Sri Satya Vennela <[email protected]>Signed-off-by Shradha Gupta <[email protected]>Reviewed-by Haiyang Zhang <[email protected]>Reviewed-by Simon Horman <[email protected]>---
Changes in v3
* Optimize the comments inmana_gd_setup_dyn_irqs()
* add more details in thedev_dbgfor extra IRQs
---
Changes in v2
* Removed the unusedskip_first_cpuvariable
* fixed exit condition inirq_setup_linear()with len == 0
* changed return type ofirq_setup_linear()as it will always be 0
* removed the unnecessaryrcu_read_lock()inirq_setup_linear()
* added appropriate comments to indicate expected behaviour when IRQs are more than or equal tonum_online_cpus()--- .../net/ethernet/microsoft/mana/gdma_main.c | 60 ++++++++++++++++--- 1 file changed, 53 insertions(+), 7 deletions(-)drivers/net/ethernet/microsoft/mana/gdma_main.c +5
-0@@ -197,6 +197,8 @@ static int mana_gd_query_max_resources(struct pci_dev *pdev) }else {/* If dynamic allocation is enabled we have already allocated * hwc msi +* Also, we make sure in this case the following is always true +* (num_msix_usable - 1 HWC) <= num_online_cpus() */ gc->num_msix_usable =min (resp.max_msix,num_online_cpus () +1 ); }@@ -1717,11 +1719,24 @@ static int irq_setup(unsigned int *irqs, unsigned int len, int node,return 0 ; } +/* should be called with cpus_read_lock() held */ +static void irq_setup_linear (unsigned int *irqs,unsigned int len) +{drivers/net/ethernet/microsoft/mana/gdma_main.c static int irq_setup(unsigned int *irqs, unsigned int len, int node, · line 1734
Quoted 10 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +10
-0+int cpu; + +for_each_online_cpu (cpu) { +if (len ==0 ) +break ; + +irq_set_affinity_and_hint (*irqs++,cpumask_of (cpu)); + len--; + } +}I would find all of this a bit easier to follow if
irq_setup_linear()andirq_setup()had a mana prefix so it was more obvious these are specific to the driver. Of courseirq_setupis pre-existing, and its not my driver so do as you will :)Quoted 54 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +33
-3+static int mana_gd_setup_dyn_irqs (struct pci_dev *pdev,int nvec) {struct gdma_context *gc =pci_get_drvdata (pdev);struct gdma_irq_context *gic; -bool skip_first_cpu = false;int *irqs, irq, err, i; irqs =kmalloc_objs (int, nvec);@@ -1729,6 +1744,8 @@ static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec)return -ENOMEM ;/* +* In this function, num_msix_usable = HWC IRQ + Queue IRQ. +* nvec is only Queue IRQ (HWC already setup). * While processing the next pci irq vector, we start with index 1, * as IRQ vector at index 0 is already processed for HWC. * However, the population of irqs array starts with index 0, to be @@ -1767,13 +1784,42 @@ static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec)* first CPU sibling group since they are already affinitized to HWC IRQ */ cpus_read_lock (); -if (gc->num_msix_usable <=num_online_cpus ()) - skip_first_cpu = true; +if (gc->num_msix_usable <=num_online_cpus ()) { + err =irq_setup (irqs, nvec, gc->numa_node, true); +if (err) { +cpus_read_unlock (); + goto free_irq; + } + }else { +/* +* When num_msix_usable are more than num_online_cpus, our +* queue IRQs should be equal to num of online vCPUs. +* We try to make sure queue IRQs spread across all vCPUs. +* In such a case NUMA or CPU core affinity does not matter. +* Note: in this case the total mana IRQ should always be +* num_online_cpus + 1. The first HWC IRQ is already handled +* in HWC setup calls +* However, if CPUs went offline since num_msix_usable was +* computed, queue IRQs will be more than num_online_cpus(). +* In such cases remaining extra IRQs will retain their default +* affinity. +*/ +int first_unassigned =num_online_cpus (); +if (nvec > first_unassigned) { +char buf[32 ]; + +if (first_unassigned == nvec -1 ) +snprintf (buf,sizeof (buf),"%d" , + first_unassigned); +else +snprintf (buf,sizeof (buf),"%d-%d" , + first_unassigned, nvec -1 );drivers/net/ethernet/microsoft/mana/gdma_main.c static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec) · line 1822
Quoted 10 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +5
-4+ +dev_dbg (&pdev->dev, +"MANA IRQ indices #%s will retain the default CPU affinity\n" , buf); + } - err =irq_setup (irqs, nvec, gc->numa_node, skip_first_cpu); -if (err) { -cpus_read_unlock (); - goto free_irq; +irq_setup_linear (irqs, nvec);
irq_setup()doesn't have a driver prefix, but is actually a static function ingdma_main.c, so its implementation is specific to this driver despite its name.So if I understand this change correctly, if the number of usable MSI-X vectors is smaller than the number of CPUs, you contineu to use the current
irq_setuplogic.. otherwise you switch to the simpler "linear" logic.I guess this means the logic and heuristic used in
irq_setup()breaks down when the number of vectors is large and number of vCPU is small?Quoted 8 lines
Quoted 2 lines
Makes sense.
Hi Jacob,
Yes, that's the right understanding.
Regarding the function names, let me take that up in a seperate patch to add prefixes to all such functions.
Quoted 6 lines
I agree. Now that you've got more than one setup method, short '
irq_setup' looks confusing, if not misleading. You need some name that distinguished numa-based and plain linear method.Thanks,
Yury
noted, now that a v4 is needed, I will make these changes there. Thanks
-Shradha
Paolo Abeni
maintainerQuoted 71 lines · Shradha Gupta
In mana driver, the number of IRQs allocated is capped by the
min(num_cpu + 1, queue count). In cases, where the IRQ count is greater than the vcpu count, we want to utilize all the vCPUs, irrespective of their NUMA/core bindings.This is important, especially in the envs where number of vCPUs are so few that the softIRQ handling overhead on two IRQs on the same vCPU is much more than their overheads if they were spread across sibling vCPUs.
This behaviour is more evident with dynamic IRQ allocation. Since MANA IRQs are assigned at a later stage compared to static allocation, other device IRQs may already be affinitized to the vCPUs. As a result, IRQ weights become imbalanced, causing multiple MANA IRQs to land on the same vCPU, while some vCPUs have none.
In such cases when many parallel TCP connections are tested, the throughput drops significantly.
Test envs:
=======================================================
Case 1: without this patch
=======================================================
4 vcpu(2 cores), 5 MANA IRQs (1 HWC + 4 Queue)TYPE effective vCPU aff
=======================================================
IRQ0: HWC 0
IRQ1:mana_q10
IRQ2:mana_q22
IRQ3:mana_q30
IRQ4:mana_q43%soft on each vCPU(mpstat -P ALL 1) on receiver
vCPU 0 1 2 3
=======================================================
pass 1: 38.85 0.03 24.89 24.65
pass 2: 39.15 0.03 24.57 25.28
pass 3: 40.36 0.03 23.20 23.17=======================================================
Case 2: with this patch
=======================================================
4 vcpu(2 cores), 5 MANA IRQs (1 HWC + 4 Queue)TYPE effective vCPU aff
=======================================================IRQ0: HWC 0 IRQ1: mana_q1 0 IRQ2: mana_q2 1 IRQ3: mana_q3 2 IRQ4: mana_q4 3%soft on each vCPU(mpstat -P ALL 1) on receiver
vCPU 0 1 2 3
=======================================================pass 1: 15.42 15.85 14.99 14.51 pass 2: 15.53 15.94 15.81 15.93 pass 3: 16.41 16.35 16.40 16.36=======================================================
ThroughputImpact(in Gbps, same env)
=======================================================
TCP conn with patch w/o patch
20480 15.65 7.73
10240 15.63 8.93
8192 15.64 9.69
6144 15.64 13.16
4096 15.69 15.75
2048 15.69 15.83
1024 15.71 15.28
Quoted 7 lines
Fixes 755391121038 ("net: mana: Allocate MSI-X vectors dynamically")Co-developed-by Erni Sri Satya Vennela <[email protected]>Signed-off-by Erni Sri Satya Vennela <[email protected]>Signed-off-by Shradha Gupta <[email protected]>Reviewed-by Haiyang Zhang <[email protected]>Reviewed-by Simon Horman <[email protected]>
Why do you consider this patch a fix? To me is a configuration improvement and should land on net-next.
drivers/net/ethernet/microsoft/mana/gdma_main.c static int irq_setup(unsigned int *irqs, unsigned int len, int node, · line 1722
Quoted 5 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +1
-0@@ -1717,11 +1719,24 @@ static int irq_setup(unsigned int *irqs, unsigned int len, int node,return 0 ; } +/* should be called with cpus_read_lock() held */
Minor nit: s/should/must/ or just drop the comment, as for_each_online_cpu() usage implies that.
drivers/net/ethernet/microsoft/mana/gdma_main.c static int irq_setup(unsigned int *irqs, unsigned int len, int node, · line 1733
Quoted 11 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +11
-0+static void irq_setup_linear (unsigned int *irqs,unsigned int len) +{ +int cpu; + +for_each_online_cpu (cpu) { +if (len ==0 ) +break ; + +irq_set_affinity_and_hint (*irqs++,cpumask_of (cpu)); + len--; + }
As this is another heuristic regarding irq spreading, why don't you implement that inside irq_setup()?
Quoted 18 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +12
-2@@ -1767,13 +1784,42 @@ static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec)* first CPU sibling group since they are already affinitized to HWC IRQ */ cpus_read_lock (); -if (gc->num_msix_usable <=num_online_cpus ()) - skip_first_cpu = true; +if (gc->num_msix_usable <=num_online_cpus ()) { + err =irq_setup (irqs, nvec, gc->numa_node, true); +if (err) { +cpus_read_unlock (); + goto free_irq; + } + }else { +/* +* When num_msix_usable are more than num_online_cpus, our +* queue IRQs should be equal to num of online vCPUs. +* We try to make sure queue IRQs spread across all vCPUs. +* In such a case NUMA or CPU core affinity does not matter.
drivers/net/ethernet/microsoft/mana/gdma_main.c static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec) · line 1808
Quoted 10 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +10
-0+* Note: in this case the total mana IRQ should always be +* num_online_cpus + 1. The first HWC IRQ is already handled +* in HWC setup calls +* However, if CPUs went offline since num_msix_usable was +* computed, queue IRQs will be more than num_online_cpus(). +* In such cases remaining extra IRQs will retain their default +* affinity. +*/ +int first_unassigned =num_online_cpus (); +if (nvec > first_unassigned) {
An empty line is needed between the variable declaration and the code.
/P
Quoted 80 lines · Paolo Abeni
Quoted 71 lines · Shradha Gupta
In mana driver, the number of IRQs allocated is capped by the
min(num_cpu + 1, queue count). In cases, where the IRQ count is greater than the vcpu count, we want to utilize all the vCPUs, irrespective of their NUMA/core bindings.This is important, especially in the envs where number of vCPUs are so few that the softIRQ handling overhead on two IRQs on the same vCPU is much more than their overheads if they were spread across sibling vCPUs.
This behaviour is more evident with dynamic IRQ allocation. Since MANA IRQs are assigned at a later stage compared to static allocation, other device IRQs may already be affinitized to the vCPUs. As a result, IRQ weights become imbalanced, causing multiple MANA IRQs to land on the same vCPU, while some vCPUs have none.
In such cases when many parallel TCP connections are tested, the throughput drops significantly.
Test envs:
=======================================================
Case 1: without this patch
=======================================================
4 vcpu(2 cores), 5 MANA IRQs (1 HWC + 4 Queue)TYPE effective vCPU aff
=======================================================
IRQ0: HWC 0
IRQ1:mana_q10
IRQ2:mana_q22
IRQ3:mana_q30
IRQ4:mana_q43%soft on each vCPU(mpstat -P ALL 1) on receiver
vCPU 0 1 2 3
=======================================================
pass 1: 38.85 0.03 24.89 24.65
pass 2: 39.15 0.03 24.57 25.28
pass 3: 40.36 0.03 23.20 23.17=======================================================
Case 2: with this patch
=======================================================
4 vcpu(2 cores), 5 MANA IRQs (1 HWC + 4 Queue)TYPE effective vCPU aff
=======================================================IRQ0: HWC 0 IRQ1: mana_q1 0 IRQ2: mana_q2 1 IRQ3: mana_q3 2 IRQ4: mana_q4 3%soft on each vCPU(mpstat -P ALL 1) on receiver
vCPU 0 1 2 3
=======================================================pass 1: 15.42 15.85 14.99 14.51 pass 2: 15.53 15.94 15.81 15.93 pass 3: 16.41 16.35 16.40 16.36=======================================================
ThroughputImpact(in Gbps, same env)
=======================================================
TCP conn with patch w/o patch
20480 15.65 7.73
10240 15.63 8.93
8192 15.64 9.69
6144 15.64 13.16
4096 15.69 15.75
2048 15.69 15.83
1024 15.71 15.28Quoted 7 lines
Fixes 755391121038 ("net: mana: Allocate MSI-X vectors dynamically")Co-developed-by Erni Sri Satya Vennela <[email protected]>Signed-off-by Erni Sri Satya Vennela <[email protected]>Signed-off-by Shradha Gupta <[email protected]>Reviewed-by Haiyang Zhang <[email protected]>Reviewed-by Simon Horman <[email protected]>
Quoted 2 lines
Why do you consider this patch a fix? To me is a configuration improvement and should land on net-next.
Hi Paolo,
This is a fix for commit 755391121038 ("net: mana: Allocate MSI-X vectors dynamically"). Before that commit, IRQs were statically allocated and clustering of MANA IRQs happened less often on low vCPU configs. With dynamic allocation, MANA IRQs are assigned at a later stage when other device IRQs have already occupied vCPUs. The NUMA-aware affinity logic in that commit increased the probability of IRQ clustering, causing a 2x throughput regression (15.65 vs 7.73 Gbps) on low vCPU Azure SKUs at high connection counts.
Quoted 10 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c static int irq_setup(unsigned int *irqs, unsigned int len, int node, · line 1722
Quoted 6 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +1
-0@@ -1717,11 +1719,24 @@ static int irq_setup(unsigned int *irqs, unsigned int len, int node,return 0 ; } +/* should be called with cpus_read_lock() held */ Minor nit: s/should/must/ or just drop the comment, as
for_each_online_cpu()usage implies that.
Thanks, will change this in next version.
Quoted 6 lines
Quoted 6 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +6
-0+static void irq_setup_linear (unsigned int *irqs,unsigned int len) +{ +int cpu; + +for_each_online_cpu (cpu) { +if (len ==0 )
Quoted 9 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c static int irq_setup(unsigned int *irqs, unsigned int len, int node, · line 1733
Quoted 5 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +5
-0+break ; + +irq_set_affinity_and_hint (*irqs++,cpumask_of (cpu)); + len--; + }As this is another heuristic regarding irq spreading, why don't you implement that inside
irq_setup()?
irq_setup() already handles multiple cases - dynamic, static, and HWC affinity logic with NUMA-aware sibling group spreading. Adding the linear case there would make it more complex and harder to follow. Keeping it as a separate function makes both, NUMA aware and linear paths easier to understand and maintain.
Happy to reconsider if you feel strongly about it.
Quoted 22 lines
Quoted 22 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +16
-2@@ -1767,13 +1784,42 @@ static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec)* first CPU sibling group since they are already affinitized to HWC IRQ */ cpus_read_lock (); -if (gc->num_msix_usable <=num_online_cpus ()) - skip_first_cpu = true; +if (gc->num_msix_usable <=num_online_cpus ()) { + err =irq_setup (irqs, nvec, gc->numa_node, true); +if (err) { +cpus_read_unlock (); + goto free_irq; + } + }else { +/* +* When num_msix_usable are more than num_online_cpus, our +* queue IRQs should be equal to num of online vCPUs. +* We try to make sure queue IRQs spread across all vCPUs. +* In such a case NUMA or CPU core affinity does not matter. +* Note: in this case the total mana IRQ should always be +* num_online_cpus + 1. The first HWC IRQ is already handled +* in HWC setup calls +* However, if CPUs went offline since num_msix_usable was
Quoted 8 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec) · line 1808
Quoted 6 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +6
-0+* computed, queue IRQs will be more than num_online_cpus(). +* In such cases remaining extra IRQs will retain their default +* affinity. +*/ +int first_unassigned =num_online_cpus (); +if (nvec > first_unassigned) {An empty line is needed between the variable declaration and the code.
noted, Thanks.
Quoted 2 lines
/P
Quoted 59 lines · Shradha Gupta
In mana driver, the number of IRQs allocated is capped by the
min(num_cpu + 1, queue count). In cases, where the IRQ count is greater than the vcpu count, we want to utilize all the vCPUs, irrespective of their NUMA/core bindings.This is important, especially in the envs where number of vCPUs are so few that the softIRQ handling overhead on two IRQs on the same vCPU is much more than their overheads if they were spread across sibling vCPUs.
This behaviour is more evident with dynamic IRQ allocation. Since MANA IRQs are assigned at a later stage compared to static allocation, other device IRQs may already be affinitized to the vCPUs. As a result, IRQ weights become imbalanced, causing multiple MANA IRQs to land on the same vCPU, while some vCPUs have none.
In such cases when many parallel TCP connections are tested, the throughput drops significantly.
Test envs:
=======================================================
Case 1: without this patch
=======================================================
4 vcpu(2 cores), 5 MANA IRQs (1 HWC + 4 Queue)TYPE effective vCPU aff
=======================================================
IRQ0: HWC 0
IRQ1:mana_q10
IRQ2:mana_q22
IRQ3:mana_q30
IRQ4:mana_q43%soft on each vCPU(mpstat -P ALL 1) on receiver
vCPU 0 1 2 3
=======================================================
pass 1: 38.85 0.03 24.89 24.65
pass 2: 39.15 0.03 24.57 25.28
pass 3: 40.36 0.03 23.20 23.17=======================================================
Case 2: with this patch
=======================================================
4 vcpu(2 cores), 5 MANA IRQs (1 HWC + 4 Queue)TYPE effective vCPU aff
=======================================================IRQ0: HWC 0 IRQ1: mana_q1 0 IRQ2: mana_q2 1 IRQ3: mana_q3 2 IRQ4: mana_q4 3%soft on each vCPU(mpstat -P ALL 1) on receiver
vCPU 0 1 2 3
=======================================================pass 1: 15.42 15.85 14.99 14.51 pass 2: 15.53 15.94 15.81 15.93 pass 3: 16.41 16.35 16.40 16.36
Quoted 11 lines
=======================================================
ThroughputImpact(in Gbps, same env)
=======================================================
TCP conn with patch w/o patch
20480 15.65 7.73
10240 15.63 8.93
8192 15.64 9.69
6144 15.64 13.16
4096 15.69 15.75
2048 15.69 15.83
1024 15.71 15.28
So, case 1 is irq_setup(), and case 2 is irq_setup_linear(). Is that correct?
On the previous round we've discussed a no-affinity case:
irq_set_affinity_and_hint(irq, NULL);
My naive view is that the more freedom you give to the scheduler in balancing the IRQ handling load, the better results you've got. But your numbers show that the 'linear' distribution is still better. Can you add the results of that experiment as the 'case 3' please? Any ideas why the linear case wins over the no-affinity?
Thanks,
Yury
Quoted 124 lines
Fixes 755391121038 ("net: mana: Allocate MSI-X vectors dynamically")Co-developed-by Erni Sri Satya Vennela <[email protected]>Signed-off-by Erni Sri Satya Vennela <[email protected]>Signed-off-by Shradha Gupta <[email protected]>Reviewed-by Haiyang Zhang <[email protected]>Reviewed-by Simon Horman <[email protected]>---
Changes in v3
* Optimize the comments inmana_gd_setup_dyn_irqs()
* add more details in thedev_dbgfor extra IRQs
---
Changes in v2
* Removed the unusedskip_first_cpuvariable
* fixed exit condition inirq_setup_linear()with len == 0
* changed return type ofirq_setup_linear()as it will always be 0
* removed the unnecessaryrcu_read_lock()inirq_setup_linear()
* added appropriate comments to indicate expected behaviour when IRQs are more than or equal tonum_online_cpus()--- .../net/ethernet/microsoft/mana/gdma_main.c | 60 ++++++++++++++++--- 1 file changed, 53 insertions(+), 7 deletions(-)drivers/net/ethernet/microsoft/mana/gdma_main.c +53
-7@@ -197,6 +197,8 @@ static int mana_gd_query_max_resources(struct pci_dev *pdev) }else {/* If dynamic allocation is enabled we have already allocated * hwc msi +* Also, we make sure in this case the following is always true +* (num_msix_usable - 1 HWC) <= num_online_cpus() */ gc->num_msix_usable =min (resp.max_msix,num_online_cpus () +1 ); }@@ -1717,11 +1719,24 @@ static int irq_setup(unsigned int *irqs, unsigned int len, int node,return 0 ; } +/* should be called with cpus_read_lock() held */ +static void irq_setup_linear (unsigned int *irqs,unsigned int len) +{ +int cpu; + +for_each_online_cpu (cpu) { +if (len ==0 ) +break ; + +irq_set_affinity_and_hint (*irqs++,cpumask_of (cpu)); + len--; + } +} +static int mana_gd_setup_dyn_irqs (struct pci_dev *pdev,int nvec) {struct gdma_context *gc =pci_get_drvdata (pdev);struct gdma_irq_context *gic; -bool skip_first_cpu = false;int *irqs, irq, err, i; irqs =kmalloc_objs (int, nvec);@@ -1729,6 +1744,8 @@ static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec)return -ENOMEM ;/* +* In this function, num_msix_usable = HWC IRQ + Queue IRQ. +* nvec is only Queue IRQ (HWC already setup). * While processing the next pci irq vector, we start with index 1, * as IRQ vector at index 0 is already processed for HWC. * However, the population of irqs array starts with index 0, to be @@ -1767,13 +1784,42 @@ static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec)* first CPU sibling group since they are already affinitized to HWC IRQ */ cpus_read_lock (); -if (gc->num_msix_usable <=num_online_cpus ()) - skip_first_cpu = true; +if (gc->num_msix_usable <=num_online_cpus ()) { + err =irq_setup (irqs, nvec, gc->numa_node, true); +if (err) { +cpus_read_unlock (); + goto free_irq; + } + }else { +/* +* When num_msix_usable are more than num_online_cpus, our +* queue IRQs should be equal to num of online vCPUs. +* We try to make sure queue IRQs spread across all vCPUs. +* In such a case NUMA or CPU core affinity does not matter. +* Note: in this case the total mana IRQ should always be +* num_online_cpus + 1. The first HWC IRQ is already handled +* in HWC setup calls +* However, if CPUs went offline since num_msix_usable was +* computed, queue IRQs will be more than num_online_cpus(). +* In such cases remaining extra IRQs will retain their default +* affinity. +*/ +int first_unassigned =num_online_cpus (); +if (nvec > first_unassigned) { +char buf[32 ]; + +if (first_unassigned == nvec -1 ) +snprintf (buf,sizeof (buf),"%d" , + first_unassigned); +else +snprintf (buf,sizeof (buf),"%d-%d" , + first_unassigned, nvec -1 ); + +dev_dbg (&pdev->dev, +"MANA IRQ indices #%s will retain the default CPU affinity\n" , buf); + } - err =irq_setup (irqs, nvec, gc->numa_node, skip_first_cpu); -if (err) { -cpus_read_unlock (); - goto free_irq; +irq_setup_linear (irqs, nvec); }cpus_read_unlock ();base-commit:
8415598365503ced2e3d019491b0a2756c85c494Signature · 1 line
2.34.1
Quoted 72 lines · Yury Norov
Quoted 59 lines · Shradha Gupta
In mana driver, the number of IRQs allocated is capped by the
min(num_cpu + 1, queue count). In cases, where the IRQ count is greater than the vcpu count, we want to utilize all the vCPUs, irrespective of their NUMA/core bindings.This is important, especially in the envs where number of vCPUs are so few that the softIRQ handling overhead on two IRQs on the same vCPU is much more than their overheads if they were spread across sibling vCPUs.
This behaviour is more evident with dynamic IRQ allocation. Since MANA IRQs are assigned at a later stage compared to static allocation, other device IRQs may already be affinitized to the vCPUs. As a result, IRQ weights become imbalanced, causing multiple MANA IRQs to land on the same vCPU, while some vCPUs have none.
In such cases when many parallel TCP connections are tested, the throughput drops significantly.
Test envs:
=======================================================
Case 1: without this patch
=======================================================
4 vcpu(2 cores), 5 MANA IRQs (1 HWC + 4 Queue)TYPE effective vCPU aff
=======================================================
IRQ0: HWC 0
IRQ1:mana_q10
IRQ2:mana_q22
IRQ3:mana_q30
IRQ4:mana_q43%soft on each vCPU(mpstat -P ALL 1) on receiver
vCPU 0 1 2 3
=======================================================
pass 1: 38.85 0.03 24.89 24.65
pass 2: 39.15 0.03 24.57 25.28
pass 3: 40.36 0.03 23.20 23.17=======================================================
Case 2: with this patch
=======================================================
4 vcpu(2 cores), 5 MANA IRQs (1 HWC + 4 Queue)TYPE effective vCPU aff
=======================================================IRQ0: HWC 0 IRQ1: mana_q1 0 IRQ2: mana_q2 1 IRQ3: mana_q3 2 IRQ4: mana_q4 3%soft on each vCPU(mpstat -P ALL 1) on receiver
vCPU 0 1 2 3
=======================================================pass 1: 15.42 15.85 14.99 14.51 pass 2: 15.53 15.94 15.81 15.93 pass 3: 16.41 16.35 16.40 16.36Quoted 11 lines
=======================================================
ThroughputImpact(in Gbps, same env)
=======================================================
TCP conn with patch w/o patch
20480 15.65 7.73
10240 15.63 8.93
8192 15.64 9.69
6144 15.64 13.16
4096 15.69 15.75
2048 15.69 15.83
1024 15.71 15.28
Quoted 2 lines
So, case 1 is
irq_setup(), and case 2 isirq_setup_linear(). Is that correct?
That is correct.
Quoted 10 lines
On the previous round we've discussed a no-affinity case:
irq_set_affinity_and_hint(irq, NULL);My naive view is that the more freedom you give to the scheduler in balancing the IRQ handling load, the better results you've got. But your numbers show that the 'linear' distribution is still better. Can you add the results of that experiment as the 'case 3' please? Any ideas why the linear case wins over the no-affinity?
Sure, I will include them in the commit message for the next version. The reason is that in some cases(reproducible testcases in Azure environments), there are existing non-MANA IRQs that are skewing the IRQ weights on individual vCPUS. And that is causing MANA queue IRQ clustering again.
I had mentioned a possibility of this in the last thread, and we were able to reproduce it through some more testcases.
Quoted 128 lines
Thanks,
YuryQuoted 124 lines
Fixes 755391121038 ("net: mana: Allocate MSI-X vectors dynamically")Co-developed-by Erni Sri Satya Vennela <[email protected]>Signed-off-by Erni Sri Satya Vennela <[email protected]>Signed-off-by Shradha Gupta <[email protected]>Reviewed-by Haiyang Zhang <[email protected]>Reviewed-by Simon Horman <[email protected]>---
Changes in v3
* Optimize the comments inmana_gd_setup_dyn_irqs()
* add more details in thedev_dbgfor extra IRQs
---
Changes in v2
* Removed the unusedskip_first_cpuvariable
* fixed exit condition inirq_setup_linear()with len == 0
* changed return type ofirq_setup_linear()as it will always be 0
* removed the unnecessaryrcu_read_lock()inirq_setup_linear()
* added appropriate comments to indicate expected behaviour when IRQs are more than or equal tonum_online_cpus()--- .../net/ethernet/microsoft/mana/gdma_main.c | 60 ++++++++++++++++--- 1 file changed, 53 insertions(+), 7 deletions(-)drivers/net/ethernet/microsoft/mana/gdma_main.c +53
-7@@ -197,6 +197,8 @@ static int mana_gd_query_max_resources(struct pci_dev *pdev) }else {/* If dynamic allocation is enabled we have already allocated * hwc msi +* Also, we make sure in this case the following is always true +* (num_msix_usable - 1 HWC) <= num_online_cpus() */ gc->num_msix_usable =min (resp.max_msix,num_online_cpus () +1 ); }@@ -1717,11 +1719,24 @@ static int irq_setup(unsigned int *irqs, unsigned int len, int node,return 0 ; } +/* should be called with cpus_read_lock() held */ +static void irq_setup_linear (unsigned int *irqs,unsigned int len) +{ +int cpu; + +for_each_online_cpu (cpu) { +if (len ==0 ) +break ; + +irq_set_affinity_and_hint (*irqs++,cpumask_of (cpu)); + len--; + } +} +static int mana_gd_setup_dyn_irqs (struct pci_dev *pdev,int nvec) {struct gdma_context *gc =pci_get_drvdata (pdev);struct gdma_irq_context *gic; -bool skip_first_cpu = false;int *irqs, irq, err, i; irqs =kmalloc_objs (int, nvec);@@ -1729,6 +1744,8 @@ static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec)return -ENOMEM ;/* +* In this function, num_msix_usable = HWC IRQ + Queue IRQ. +* nvec is only Queue IRQ (HWC already setup). * While processing the next pci irq vector, we start with index 1, * as IRQ vector at index 0 is already processed for HWC. * However, the population of irqs array starts with index 0, to be @@ -1767,13 +1784,42 @@ static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec)* first CPU sibling group since they are already affinitized to HWC IRQ */ cpus_read_lock (); -if (gc->num_msix_usable <=num_online_cpus ()) - skip_first_cpu = true; +if (gc->num_msix_usable <=num_online_cpus ()) { + err =irq_setup (irqs, nvec, gc->numa_node, true); +if (err) { +cpus_read_unlock (); + goto free_irq; + } + }else { +/* +* When num_msix_usable are more than num_online_cpus, our +* queue IRQs should be equal to num of online vCPUs. +* We try to make sure queue IRQs spread across all vCPUs. +* In such a case NUMA or CPU core affinity does not matter. +* Note: in this case the total mana IRQ should always be +* num_online_cpus + 1. The first HWC IRQ is already handled +* in HWC setup calls +* However, if CPUs went offline since num_msix_usable was +* computed, queue IRQs will be more than num_online_cpus(). +* In such cases remaining extra IRQs will retain their default +* affinity. +*/ +int first_unassigned =num_online_cpus (); +if (nvec > first_unassigned) { +char buf[32 ]; + +if (first_unassigned == nvec -1 ) +snprintf (buf,sizeof (buf),"%d" , + first_unassigned); +else +snprintf (buf,sizeof (buf),"%d-%d" , + first_unassigned, nvec -1 ); + +dev_dbg (&pdev->dev, +"MANA IRQ indices #%s will retain the default CPU affinity\n" , buf); + } - err =irq_setup (irqs, nvec, gc->numa_node, skip_first_cpu); -if (err) { -cpus_read_unlock (); - goto free_irq; +irq_setup_linear (irqs, nvec); }cpus_read_unlock ();base-commit:
8415598365503ced2e3d019491b0a2756c85c494Signature · 1 line
2.34.1