Skip to contentSkip to conversation

irq/affinity: Mark the pre/post vectors as regular interrupts

RFC PATCH In review Dou Liyang 3 participants 4 messages Sep 13, 2018 – Sep 20, 2018

Subsystems (3)
lore
Share & tools
lore

Dou Liyang

author
#0 lore raw

RFC PATCH 1/1 irq/affinity: Mark the pre/post vectors as regular interrupts

From: Dou Liyang <[email protected]>

As Kashyap and Sumit reported, in MSI/-x subsystem, the pre/post vectors may be used to some extra reply queues for performance. the best way to map the pre/post vectors is map them to the local numa node.

But, current Linux can't do that, because

The pre and post vectors are marked managed and their affinity mask is set to the irq default affinity mask.

The default affinity mask is by default ALL cpus, but it can be tweaked both on the kernel command line and via proc.

If that mask is only a subset of CPUs and all of them go offline then these vectors are shutdown in managed mode.

So, clear these affinity mask and check it in alloc_desc() to leave them as regular interrupts which can be affinity controlled and also can move freely on hotplug.

Note: will break the validation of affinity mask(s)

---
 kernel/irq/affinity.c |  9 ++++++---
 kernel/irq/irqdesc.c  | 24 ++++++++++--------------
 2 files changed, 16 insertions(+), 17 deletions(-)
kernel/irq/affinity.c +6 -3
index f4f29b9d90ee..ba35a5050dd2 100644
@@ -204,7 +204,7 @@ irq_create_affinity_masks(int nvecs, const struct irq_affinity *affd)
 
 	/* Fill out vectors at the beginning that don't need affinity */
 	for (curvec = 0; curvec < affd->pre_vectors; curvec++)
-		cpumask_copy(masks + curvec, irq_default_affinity);
+		cpumask_clear(masks + curvec);
 
 	/* Stabilize the cpumasks */
 	get_online_cpus();
@@ -234,10 +234,13 @@ irq_create_affinity_masks(int nvecs, const struct irq_affinity *affd)
 	/* Fill out vectors at the end that don't need affinity */
 	if (usedvecs >= affvecs)
 		curvec = affd->pre_vectors + affvecs;
-	else
+	else {
 		curvec = affd->pre_vectors + usedvecs;
+		for (; curvec < affd->pre_vectors + affvecs; curvec++)
+			cpumask_copy(masks + curvec, irq_default_affinity);
+	}
 	for (; curvec < nvecs; curvec++)
-		cpumask_copy(masks + curvec, irq_default_affinity);
+		cpumask_clear(masks + curvec);
 
 outnodemsk:
 	free_node_to_cpumask(node_to_cpumask);
kernel/irq/irqdesc.c +10 -14
index 578d0e5f1b5b..5cffa791a20b 100644
@@ -453,24 +453,20 @@ static int alloc_descs(unsigned int start, unsigned int cnt, int node,
 {
 	const struct cpumask *mask = NULL;
 	struct irq_desc *desc;
-	unsigned int flags;
+	unsigned int flags = 0;
 	int i;
 
-	/* Validate affinity mask(s) */
-	if (affinity) {
-		for (i = 0, mask = affinity; i < cnt; i++, mask++) {
-			if (cpumask_empty(mask))
-				return -EINVAL;
-		}
-	}
-
-	flags = affinity ? IRQD_AFFINITY_MANAGED | IRQD_MANAGED_SHUTDOWN : 0;
-	mask = NULL;
-
 	for (i = 0; i < cnt; i++) {
 		if (affinity) {
-			node = cpu_to_node(cpumask_first(affinity));
-			mask = affinity;
+			if (cpumask_empty(affinity)) {
+				flags = 0;
+				mask = NULL;
+			} else {
+				flags = IRQD_AFFINITY_MANAGED |
+					IRQD_MANAGED_SHUTDOWN;
+				mask = affinity;
+				node = cpu_to_node(cpumask_first(affinity));
+			}
 			affinity++;
 		}
 		desc = alloc_desc(start + i, node, flags, mask, owner);
Signature · 1 line
2.14.3
#1 lore raw
Quoted 3 lines · Dou Liyang

So, clear these affinity mask and check it in alloc_desc() to leave them as regular interrupts which can be affinity controlled and also can move freely on hotplug.

This is the wrong direction as it does not allow to do initial affinity assignement for the non-managed interrupts on allocation time. And that's what Kashyap and Sumit are looking for.

The trivial fix for the possible breakage when irq_default_affinity != cpu_possible_mask is to set the affinity for the pre/post vectors to cpu_possible_mask and be done with it.

One other thing I noticed while staring at that is the fact, that the PCI code does not care about the return value of irq_create_affinity_masks() at all. It just proceeds if masks is NULL as if the stuff was requested with the affinity descriptor pointer being NULL. I don't think that's a brilliant idea because the drivers might rely on the interrupt being managed, but it might be a non-issue and just result in bad locality.

Christoph?

So back to the problem at hand. We need to change the affinity management scheme in a way which lets us differentiate between managed and unmanaged interrupts, so it allows to automatically assign (initial) affinity to all of them.

Right now everything is bound to the cpumasks array, which does not allow to convey more information. So we need to come up with something different.

Something like the below (does not compile and is just for reference) should do the trick. I'm not sure whether we want to convey the information (masks and bitmap) in a single data structure or not, but that's an implementation detail.

Thanks,

tglx

8<---------

drivers/pci/msi.c +12 -4
@@ -535,15 +535,16 @@ static struct msi_desc *
 msi_setup_entry(struct pci_dev *dev, int nvec, const struct irq_affinity *affd)
 {
 	struct cpumask *masks = NULL;
+	unsigned long managed = 0;
 	struct msi_desc *entry;
 	u16 control;
 
 	if (affd)
-		masks = irq_create_affinity_masks(nvec, affd);
+		masks = irq_create_affinity_masks(nvec, affd, &managed);
 
 
 	/* MSI Entry Initialization */
-	entry = alloc_msi_entry(&dev->dev, nvec, masks);
+	entry = alloc_msi_entry(&dev->dev, nvec, masks, managed);
 	if (!entry)
 		goto out;
 
@@ -672,15 +673,22 @@ static int msix_setup_entries(struct pci
 			      struct msix_entry *entries, int nvec,
 			      const struct irq_affinity *affd)
 {
+	/*
+	 * MSIX_MAX_VECTORS = 2048, i.e. 256 bytes. Might need runtime
+	 * allocation. OTOH, are 2048 vectors realistic?
+	 */
+	DECLARE_BITMAP(managed, MSIX_MAX_VECTORS);
 	struct cpumask *curmsk, *masks = NULL;
 	struct msi_desc *entry;
 	int ret, i;
 
 	if (affd)
-		masks = irq_create_affinity_masks(nvec, affd);
+		masks = irq_create_affinity_masks(nvec, affd, managed);
 
 	for (i = 0, curmsk = masks; i < nvec; i++) {
-		entry = alloc_msi_entry(&dev->dev, 1, curmsk);
+		unsigned long m = test_bit(i, managed) ? 1 : 0;
+
+		entry = alloc_msi_entry(&dev->dev, 1, curmsk, m);
 		if (!entry) {
 			if (!i)
 				iounmap(base);
kernel/irq/msi.c +4 -2
@@ -27,7 +27,8 @@
  * and the affinity masks from @affinity are copied.
  */
 struct msi_desc *
-alloc_msi_entry(struct device *dev, int nvec, const struct cpumask *affinity)
+alloc_msi_entry(struct device *dev, int nvec, const struct cpumask *affinity,
+		unsigned long managed)
 {
 	struct msi_desc *desc;
 
@@ -38,6 +39,7 @@ alloc_msi_entry(struct device *dev, int
 	INIT_LIST_HEAD(&desc->list);
 	desc->dev = dev;
 	desc->nvec_used = nvec;
+	desc->managed = managed;
 	if (affinity) {
 		desc->affinity = kmemdup(affinity,
 			nvec * sizeof(*desc->affinity), GFP_KERNEL);
@@ -416,7 +418,7 @@ int msi_domain_alloc_irqs(struct irq_dom
 
 		virq = __irq_domain_alloc_irqs(domain, -1, desc->nvec_used,
 					       dev_to_node(dev), &arg, false,
-					       desc->affinity);
+					       desc->affinity, desc->managed);
 		if (virq < 0) {
 			ret = -ENOSPC;
 			if (ops->handle_error)
#2 lore raw
Quoted 2 lines

This is the wrong direction as it does not allow to do initial affinity assignement for the non-managed interrupts on allocation time. And

that's

Quoted 7 lines

what Kashyap and Sumit are looking for.

The trivial fix for the possible breakage when irq_default_affinity != cpu_possible_mask is to set the affinity for the pre/post vectors to cpu_possible_mask and be done with it.

One other thing I noticed while staring at that is the fact, that the

PCI

Quoted 1 line

code does not care about the return value of irq_create_affinity_masks()

at

Quoted 1 line

all. It just proceeds if masks is NULL as if the stuff was requested

with

Quoted 7 lines

the affinity descriptor pointer being NULL. I don't think that's a brilliant idea because the drivers might rely on the interrupt being managed, but it might be a non-issue and just result in bad locality.

Christoph?

So back to the problem at hand. We need to change the affinity

management

Quoted 3 lines

scheme in a way which lets us differentiate between managed and unmanaged
interrupts, so it allows to automatically assign (initial) affinity to

all

Quoted 3 lines

of them.

Right now everything is bound to the cpumasks array, which does not

allow

Quoted 5 lines

to convey more information. So we need to come up with something different.

Something like the below (does not compile and is just for reference) should do the trick. I'm not sure whether we want to convey the

information

Quoted 2 lines

(masks and bitmap) in a single data structure or not, but that's an implementation detail.

Dou - Do you want me to test your patch or shall I wait for next version ?

drivers/pci/msi.c static struct msi_desc * · line 535

Quoted 10 lines

Thanks,

tglx

8<---------

drivers/pci/msi.c +0 -0
@@ -535,15 +535,16 @@ static struct msi_desc *
 msi_setup_entry(struct pci_dev *dev, int nvec, const struct

irq_affinity *affd)

Quoted 45 lines
drivers/pci/msi.c +12 -4
 {
 	struct cpumask *masks = NULL;
+	unsigned long managed = 0;
 	struct msi_desc *entry;
 	u16 control;
 
 	if (affd)
-		masks = irq_create_affinity_masks(nvec, affd);
+		masks = irq_create_affinity_masks(nvec, affd, &managed);
 
 
 	/* MSI Entry Initialization */
-	entry = alloc_msi_entry(&dev->dev, nvec, masks);
+	entry = alloc_msi_entry(&dev->dev, nvec, masks, managed);
 	if (!entry)
 		goto out;
 
@@ -672,15 +673,22 @@ static int msix_setup_entries(struct pci
 			      struct msix_entry *entries, int nvec,
 			      const struct irq_affinity *affd)
 {
+	/*
+	 * MSIX_MAX_VECTORS = 2048, i.e. 256 bytes. Might need runtime
+	 * allocation. OTOH, are 2048 vectors realistic?
+	 */
+	DECLARE_BITMAP(managed, MSIX_MAX_VECTORS);
 	struct cpumask *curmsk, *masks = NULL;
 	struct msi_desc *entry;
 	int ret, i;
 
 	if (affd)
-		masks = irq_create_affinity_masks(nvec, affd);
+		masks = irq_create_affinity_masks(nvec, affd, managed);
 
 	for (i = 0, curmsk = masks; i < nvec; i++) {
-		entry = alloc_msi_entry(&dev->dev, 1, curmsk);
+		unsigned long m = test_bit(i, managed) ? 1 : 0;
+
+		entry = alloc_msi_entry(&dev->dev, 1, curmsk, m);
 		if (!entry) {
 			if (!i)
 				iounmap(base);

kernel/irq/msi.c line 30

Quoted 4 lines
kernel/irq/msi.c +0 -1
@@ -27,7 +27,8 @@
  * and the affinity masks from @affinity are copied.
  */
 struct msi_desc *
-alloc_msi_entry(struct device *dev, int nvec, const struct cpumask

*affinity)

kernel/irq/msi.c line 30

Quoted 1 line
kernel/irq/msi.c +1 -0
+alloc_msi_entry(struct device *dev, int nvec, const struct cpumask

*affinity,

Quoted 13 lines
kernel/irq/msi.c +2 -0
+		unsigned long managed)
 {
 	struct msi_desc *desc;
 
@@ -38,6 +39,7 @@ alloc_msi_entry(struct device *dev, int
 	INIT_LIST_HEAD(&desc->list);
 	desc->dev = dev;
 	desc->nvec_used = nvec;
+	desc->managed = managed;
 	if (affinity) {
 		desc->affinity = kmemdup(affinity,
 			nvec * sizeof(*desc->affinity), GFP_KERNEL);

kernel/irq/msi.c int msi_domain_alloc_irqs(struct irq_dom · line 419

Quoted 2 lines
kernel/irq/msi.c +0 -0
@@ -416,7 +418,7 @@ int msi_domain_alloc_irqs(struct irq_dom
 
 		virq = __irq_domain_alloc_irqs(domain, -1,

desc->nvec_used,

kernel/irq/msi.c int msi_domain_alloc_irqs(struct irq_dom · line 420

Quoted 1 line
kernel/irq/msi.c +0 -0
 					       dev_to_node(dev), &arg,

false,

kernel/irq/msi.c int msi_domain_alloc_irqs(struct irq_dom · line 421

Quoted 2 lines
kernel/irq/msi.c +1 -1
-					       desc->affinity);
+					       desc->affinity,

desc->managed);

Quoted 3 lines
kernel/irq/msi.c +0 -0
 		if (virq < 0) {
 			ret = -ENOSPC;
 			if (ops->handle_error)
author
#3 lore raw

Hi Kashyap,

On 2018/9/20 16:39, Kashyap Desai worte:

Quoted 2 lines

Dou - Do you want me to test your patch or shall I wait for next version ?

I'm sorry, please wait for the next version.

Thanks,
dou