Skip to contentSkip to conversation

genirq/matrix: Choose CPU for managed IRQs based on how many of them are allocated

PATCH Superseded Long Li 3 participants 4 messages Nov 2, 2018 – Nov 3, 2018 IRQ

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Long Li

author
#0 lore raw

PATCH v3 1/1 genirq/matrix: Choose CPU for managed IRQs based on how many of them are allocated

From: Long Li <[email protected]>

On a large system with multiple devices of the same class (e.g. NVMe disks, using managed IRQs), the kernel tends to concentrate their IRQs on several CPUs.

The issue is that when NVMe calls irq_matrix_alloc_managed(), the assigned CPU tends to be the first several CPUs in the cpumask, because they check for cpumap->available that will not change after managed IRQs are reserved.

For a managed IRQ, it tends to reserve more than one CPU, based on cpumask in irq_matrix_reserve_managed. But later when actually allocating CPU for this IRQ, only one CPU is allocated. Because "available" is calculated at the time managed IRQ is reserved, it tends to indicate a CPU has more IRQs than the actual number it's assigned.

To get a more even distribution for allocating managed IRQs, we need to keep track of how many of them are allocated on a given CPU. Introduce "managed_allocated" in struct cpumap to track those managed IRQs that are allocated on this CPU, and change the code to use this information for deciding how to allocate CPU for managed IRQs.

Signed-off-by Long Li <[email protected]>
---
 kernel/irq/matrix.c | 25 ++++++++++++++++++++++++-
 1 file changed, 24 insertions(+), 1 deletion(-)
kernel/irq/matrix.c +24 -1
index 6e6d467f3dec..94dd173f24d6 100644
@@ -14,6 +14,7 @@ struct cpumap {
 	unsigned int		available;
 	unsigned int		allocated;
 	unsigned int		managed;
+	unsigned int		managed_allocated;
 	bool			initialized;
 	bool			online;
 	unsigned long		alloc_map[IRQ_MATRIX_SIZE];

@@ -145,6 +146,27 @@ static unsigned int matrix_find_best_cpu(struct irq_matrix *m,
 	return best_cpu;
 }
 
+/* Find the best CPU which has the lowest number of managed IRQs allocated */
+static unsigned int matrix_find_best_cpu_managed(struct irq_matrix *m,
+						const struct cpumask *msk)
+{
+	unsigned int cpu, best_cpu, allocated = UINT_MAX;
+	struct cpumap *cm;
+
+	best_cpu = UINT_MAX;
+
+	for_each_cpu(cpu, msk) {
+		cm = per_cpu_ptr(m->maps, cpu);
+
+		if (!cm->online || cm->managed_allocated > allocated)
+			continue;
+
+		best_cpu = cpu;
+		allocated = cm->managed_allocated;
+	}
+	return best_cpu;
+}
+
 /**
  * irq_matrix_assign_system - Assign system wide entry in the matrix
  * @m:		Matrix pointer

@@ -269,7 +291,7 @@ int irq_matrix_alloc_managed(struct irq_matrix *m, const struct cpumask *msk,
 	if (cpumask_empty(msk))
 		return -EINVAL;
 
-	cpu = matrix_find_best_cpu(m, msk);
+	cpu = matrix_find_best_cpu_managed(m, msk);
 	if (cpu == UINT_MAX)
 		return -ENOSPC;
 

@@ -282,6 +304,7 @@ int irq_matrix_alloc_managed(struct irq_matrix *m, const struct cpumask *msk,
 		return -ENOSPC;
 	set_bit(bit, cm->alloc_map);
 	cm->allocated++;
+	cm->managed_allocated++;
 	m->total_allocated++;
 	*mapped_cpu = cpu;
 	trace_irq_matrix_alloc_managed(bit, cpu, m, cm);
Signature · 1 line
2.14.1
#1 lore raw

Long,

Quoted 13 lines · Long Li

/**
* irq_matrix_assign_system - Assign system wide entry in the matrix
* @m: Matrix pointer

kernel/irq/matrix.c +1 -1
@@ -269,7 +291,7 @@ int irq_matrix_alloc_managed(struct irq_matrix *m, const struct cpumask *msk,
 	if (cpumask_empty(msk))
 		return -EINVAL;
 
-	cpu = matrix_find_best_cpu(m, msk);
+	cpu = matrix_find_best_cpu_managed(m, msk);
 	if (cpu == UINT_MAX)
 		return -ENOSPC;

kernel/irq/matrix.c int irq_matrix_alloc_managed(struct irq_matrix *m, const struct cpumask *msk, · line 310

Quoted 7 lines
kernel/irq/matrix.c +1 -0
@@ -282,6 +304,7 @@ int irq_matrix_alloc_managed(struct irq_matrix *m, const struct cpumask *msk,
 		return -ENOSPC;
 	set_bit(bit, cm->alloc_map);
 	cm->allocated++;
+	cm->managed_allocated++;
 	m->total_allocated++;
 	*mapped_cpu = cpu;
 	trace_irq_matrix_alloc_managed(bit, cpu, m, cm);

so far so good. But what exactly decrements managed_allocated ?

Thanks,

tglx

#2 lore raw
Quoted 13 lines · Thomas Gleixner
Quoted 12 lines · Long Li

/**
* irq_matrix_assign_system - Assign system wide entry in the matrix
* @m: Matrix pointer

kernel/irq/matrix.c +1 -1
@@ -269,7 +291,7 @@ int irq_matrix_alloc_managed(struct irq_matrix *m, const struct cpumask *msk,
 	if (cpumask_empty(msk))
 		return -EINVAL;
 
-	cpu = matrix_find_best_cpu(m, msk);
+	cpu = matrix_find_best_cpu_managed(m, msk);
 	if (cpu == UINT_MAX)
 		return -ENOSPC;
Quoted 10 lines

kernel/irq/matrix.c int irq_matrix_alloc_managed(struct irq_matrix *m, const struct cpumask *msk, · line 310

Quoted 8 lines
kernel/irq/matrix.c +1 -0
@@ -282,6 +304,7 @@ int irq_matrix_alloc_managed(struct irq_matrix *m, const struct cpumask *msk,
 		return -ENOSPC;
 	set_bit(bit, cm->alloc_map);
 	cm->allocated++;
+	cm->managed_allocated++;
 	m->total_allocated++;
 	*mapped_cpu = cpu;
 	trace_irq_matrix_alloc_managed(bit, cpu, m, cm);

so far so good. But what exactly decrements managed_allocated ?

Another thing. If we add that counter, then it would be good to expose it in the debugfs files as well.

Thanks,

tglx

#3 lore raw
Quoted 30 lines

Subject: Re: [PATCH v3] genirq/matrix: Choose CPU for managed IRQs based on how many of them are allocated

Quoted 5 lines · Thomas Gleixner

Review int irq_matrix_alloc_managed(struct irq_matrix

Quoted 4 lines · Long Li

/**
* irq_matrix_assign_system - Assign system wide entry in the matrix
* @m: Matrix pointer

(quoted diff) +0 -0
@@ -269,7 +291,7 @@ int irq_matrix_alloc_managed(struct irq_matrix

*m, const struct cpumask *msk,

Quoted 9 lines

kernel/irq/matrix.c int irq_matrix_alloc_managed(struct irq_matrix

Quoted 9 lines
kernel/irq/matrix.c +1 -1
 	if (cpumask_empty(msk))
 		return -EINVAL;
 
-	cpu = matrix_find_best_cpu(m, msk);
+	cpu = matrix_find_best_cpu_managed(m, msk);
 	if (cpu == UINT_MAX)
 		return -ENOSPC;
 

@@ -282,6 +304,7 @@ int irq_matrix_alloc_managed(struct irq_matrix

*m, const struct cpumask *msk,

Quoted 9 lines

kernel/irq/matrix.c int irq_matrix_alloc_managed(struct irq_matrix *m, const struct cpumask *msk, · line 310

Quoted 7 lines
kernel/irq/matrix.c +1 -0
 		return -ENOSPC;
 	set_bit(bit, cm->alloc_map);
 	cm->allocated++;
+	cm->managed_allocated++;
 	m->total_allocated++;
 	*mapped_cpu = cpu;
 	trace_irq_matrix_alloc_managed(bit, cpu, m, cm);

so far so good. But what exactly decrements managed_allocated ?

Quoted 2 lines

Another thing. If we add that counter, then it would be good to expose it in the debugfs files as well.

I will send an update to address those.

Long

Quoted 4 lines

Thanks,

tglx