Skip to contentSkip to conversation

net/mlx5: Only consider online CPUs in affinity subset check

PATCH Superseded Fushuai Wang 2 participants 5 messages Jun 4 – Jun 5

Subsystems (4)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover net/mlx5: Only consider online CPUs in affinity subset check

From: Fushuai Wang <[email protected]>

Hi all,

When an SF is created after a CPU has been taken offline, the IRQ affinity check fails because existing IRQs in the pool may have affinity masks that include the now-offline CPU. This causes SF creation to fail even though suitable online CPUs are available.

This series fixes this issue and includes a small cleanup:

Patch 1 folds cpumask_copy() into cpumask_andnot() for better code clarity in comp_irq_request_sf().

Patch 2 filters affinity masks to only consider effective CPUs before the subset check, ensuring SF creation succeeds when CPUs have been taken offline.

--WANG

Fushuai Wang (2):
net/mlx5: Simplify cpumask operations in comp_irq_request_sf() net/mlx5: Use effective affinity mask for IRQ selection

 drivers/net/ethernet/mellanox/mlx5/core/eq.c           | 3 +--
 drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c | 5 ++++-
 2 files changed, 5 insertions(+), 3 deletions(-)
Signature · 1 line
2.36.1
#1 lore raw

PATCH v2 1/2 net/mlx5: Simplify cpumask operations in comp_irq_request_sf()

From: Fushuai Wang <[email protected]>

Combine cpumask_copy() and cpumask_andnot() into a single cpumask_andnot() since the function can take cpu_online_mask directly as the source.

---
 drivers/net/ethernet/mellanox/mlx5/core/eq.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/eq.c +1 -2
index 22a637111aa2..d11ec263d53c 100644
@@ -886,8 +886,7 @@ static int comp_irq_request_sf(struct mlx5_core_dev *dev, u16 vecidx)
 		return -ENOMEM;
 
 	af_desc->is_managed = false;
-	cpumask_copy(&af_desc->mask, cpu_online_mask);
-	cpumask_andnot(&af_desc->mask, &af_desc->mask, &table->used_cpus);
+	cpumask_andnot(&af_desc->mask, cpu_online_mask, &table->used_cpus);
 	irq = mlx5_irq_affinity_request(dev, pool, af_desc);
 	if (IS_ERR(irq)) {
 		kvfree(af_desc);
Signature · 1 line
2.36.1
#2 lore raw

PATCH v2 2/2 net/mlx5: Use effective affinity mask for IRQ selection

In reply to Fushuai Wang

From: Fushuai Wang <[email protected]>

When an SF is created after a CPU has been taken offline, the IRQ pool may contain IRQs with affinity masks that include the offline CPU. Since only online CPUs should be considered for IRQ placement, cpumask_subset() check would fail because the iter_mask contains offline CPUs that are not present in req_mask, causing SF creation to fail.

Use irq_get_effective_affinity_mask() instead, which returns the IRQ's actual effective affinity that already excludes offline CPUs.

Fixes 061f5b23588a ("net/mlx5: SF, Use all available cpu for setting cpu affinity")
---
 drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c +4 -1
index 994fe83da4be..c5d784cbc8e4 100644
@@ -105,9 +105,12 @@ irq_pool_find_least_loaded(struct mlx5_irq_pool *pool, const struct cpumask *req
 
 	lockdep_assert_held(&pool->lock);
 	xa_for_each_range(&pool->irqs, index, iter, start, end) {
-		struct cpumask *iter_mask = mlx5_irq_get_affinity_mask(iter);
+		const struct cpumask *iter_mask;
 		int iter_refcount = mlx5_irq_read_locked(iter);
 
+		iter_mask = irq_get_effective_affinity_mask(mlx5_irq_get_irq(iter));
+		if (!iter_mask)
+			continue;
 		if (!cpumask_subset(iter_mask, req_mask))
 			/* skip IRQs with a mask which is not subset of req_mask */
 			continue;
Signature · 1 line
2.36.1
#3 lore raw

Re: [PATCH v2 2/2] net/mlx5: Use effective affinity mask for IRQ selection

Quoted 10 lines · Fushuai Wang

External email: Use caution opening links or attachments

From: Fushuai Wang <[email protected]>

When an SF is created after a CPU has been taken offline, the IRQ pool may contain IRQs with affinity masks that include the offline CPU. Since only online CPUs should be considered for IRQ placement, cpumask_subset() check would fail because the iter_mask contains offline CPUs that are not present in req_mask, causing SF creation to fail.

can you please add the example/repro from the prev discussion? Sorry I wasn't clear I want it the commit message...

Quoted 16 lines

Use irq_get_effective_affinity_mask() instead, which returns the IRQ's actual effective affinity that already excludes offline CPUs.

Fixes 061f5b23588a ("net/mlx5: SF, Use all available cpu for setting cpu affinity")
---
  drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c | 5 ++++-
  1 file changed, 4 insertions(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c +0 -0
index 994fe83da4be..c5d784cbc8e4 100644

drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c irq_pool_find_least_loaded(struct mlx5_irq_pool *pool, const struct cpumask *req · line 109

Quoted 6 lines
drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c +1 -1
@@ -105,9 +105,12 @@ irq_pool_find_least_loaded(struct mlx5_irq_pool *pool, const struct cpumask *req
 
         lockdep_assert_held(&pool->lock);
         xa_for_each_range(&pool->irqs, index, iter, start, end) {
-               struct cpumask *iter_mask = mlx5_irq_get_affinity_mask(iter);
+               const struct cpumask *iter_mask;
                 int iter_refcount = mlx5_irq_read_locked(iter);

can you please keep revers xmas tree here as well?

Quoted 10 lines
drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c +3 -1
+               iter_mask = irq_get_effective_affinity_mask(mlx5_irq_get_irq(iter));
+               if (!iter_mask)
+                       continue;
                 if (!cpumask_subset(iter_mask, req_mask))
                         /* skip IRQs with a mask which is not subset of req_mask */
                         continue;
--

2.36.1

#4 lore raw
In reply to Fushuai Wang
Quoted 12 lines · Fushuai Wang

External email: Use caution opening links or attachments

From: Fushuai Wang <[email protected]>

Hi all,

When an SF is created after a CPU has been taken offline, the IRQ affinity check fails because existing IRQs in the pool may have affinity masks that include the now-offline CPU. This causes SF creation to fail even though suitable online CPUs are available.

Quoted 7 lines

This series fixes this issue and includes a small cleanup:

Patch 1 folds cpumask_copy() into cpumask_andnot() for better code clarity in comp_irq_request_sf().

Patch 2 filters affinity masks to only consider effective CPUs before the subset check, ensuring SF creation succeeds when CPUs have been taken offline.

Thanks for the patches.
two points:
1) you need to add branch name to the title (PATCH net ...)
2) you need to keep a change-log.

The first patch is net-next material, and the second patch is net. I think it would be better to send each one separately to its own branch. sorry didn't notice on prev review...

Quoted 14 lines

--WANG

Fushuai Wang (2):
net/mlx5: Simplify cpumask operations in comp_irq_request_sf() net/mlx5: Use effective affinity mask for IRQ selection

  drivers/net/ethernet/mellanox/mlx5/core/eq.c           | 3 +--
  drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c | 5 ++++-
  2 files changed, 5 insertions(+), 3 deletions(-)

--
2.36.1