Skip to contentSkip to conversation

nvme-pci: allow unmanaged interrupts

PATCH In review Ming Lei 6 participants 13 messages Jul 2, 2024 – Jul 16, 2024 NVMe

lore
Share & tools
lore

Ming Lei

author
#0 lore raw

PATCH v3 1/1 nvme-pci: allow unmanaged interrupts

From: Keith Busch <[email protected]>

People _really_ want to control their interrupt affinity in some cases, such as Openshift with Performance profile, in which each irq's affinity is completely specified from userspace. Turns out that 'isolcpus=managed_irqs' isn't enough.

Add module parameter to allow unmanaged interrupts, just as some SCSI drivers are doing.

---
v2->v3:
- rebase on for-next
- add openshift use case

v1->v2: skip the the AFFINITY vector allocation if the parameter is provided instead trying to make the vector code handle all post_vectors.

 drivers/nvme/host/pci.c | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)
drivers/nvme/host/pci.c +16 -2
index 5d8035218de9..a39c99c9b64d 100644
@@ -63,6 +63,11 @@ MODULE_PARM_DESC(sgl_threshold,
 		"Use SGLs when average request segment size is larger or equal to "
 		"this size. Use 0 to disable SGLs.");
 
+static bool managed_irqs = true;
+module_param(managed_irqs, bool, 0444);
+MODULE_PARM_DESC(managed_irqs,
+		 "set to false for user controlled irq affinity");
+
 #define NVME_PCI_MIN_QUEUE_SIZE 2
 #define NVME_PCI_MAX_QUEUE_SIZE 4095
 static int io_queue_depth_set(const char *val, const struct kernel_param *kp);
@@ -456,7 +461,7 @@ static void nvme_pci_map_queues(struct blk_mq_tag_set *set)
 		 * affinity), so use the regular blk-mq cpu mapping
 		 */
 		map->queue_offset = qoff;
-		if (i != HCTX_TYPE_POLL && offset)
+		if (managed_irqs && i != HCTX_TYPE_POLL && offset)
 			blk_mq_pci_map_queues(map, to_pci_dev(dev->dev), offset);
 		else
 			blk_mq_map_queues(map);
@@ -2226,6 +2231,7 @@ static int nvme_setup_irqs(struct nvme_dev *dev, unsigned int nr_io_queues)
 	};
 	unsigned int irq_queues, poll_queues;
 	unsigned int flags = PCI_IRQ_ALL_TYPES | PCI_IRQ_AFFINITY;
+	int ret;
 
 	/*
 	 * Poll queues don't need interrupts, but we need at least one I/O queue
@@ -2251,8 +2257,16 @@ static int nvme_setup_irqs(struct nvme_dev *dev, unsigned int nr_io_queues)
 		irq_queues += (nr_io_queues - poll_queues);
 	if (dev->ctrl.quirks & NVME_QUIRK_BROKEN_MSI)
 		flags &= ~PCI_IRQ_MSI;
-	return pci_alloc_irq_vectors_affinity(pdev, 1, irq_queues, flags,
+
+	if (managed_irqs)
+		return pci_alloc_irq_vectors_affinity(pdev, 1, irq_queues, flags,
 					      &affd);
+
+	flags &= ~PCI_IRQ_AFFINITY;
+	ret = pci_alloc_irq_vectors(pdev, 1, irq_queues, flags);
+	if (ret > 0)
+		nvme_calc_irq_sets(&affd, ret - 1);
+	return ret;
 }
 
 static unsigned int nvme_max_io_queues(struct nvme_dev *dev)
Signature · 1 line
2.44.0
#1 lore raw
Quoted 9 lines · Ming Lei

From: Keith Busch <[email protected]>

People _really_ want to control their interrupt affinity in some cases, such as Openshift with Performance profile, in which each irq's affinity is completely specified from userspace. Turns out that 'isolcpus=managed_irqs' isn't enough.

Add module parameter to allow unmanaged interrupts, just as some SCSI drivers are doing.

Same as before: hell no. We can't just add hacky global kernel parameters everywhere. We need the cpu isolation infrastructure to work properly instead of piling hacks of hacks in every relevant driver.

author
#2 lore raw
Quoted 8 lines · Christoph Hellwig
Quoted 7 lines · Ming Lei

From: Keith Busch <[email protected]>

People _really_ want to control their interrupt affinity in some cases, such as Openshift with Performance profile, in which each irq's affinity is completely specified from userspace. Turns out that 'isolcpus=managed_irqs' isn't enough.

Quoted 6 lines
Quoted 2 lines

Add module parameter to allow unmanaged interrupts, just as some SCSI drivers are doing.

Same as before: hell no. We can't just add hacky global kernel parameters everywhere. We need the cpu isolation infrastructure to work properly instead of piling hacks of hacks in every relevant driver.

Per my understanding, here cpu isolation infrastructure can't work for Openshift, in which IO workload can be run on applications which are executed on isolated CPUs, meantime userspace do expect that interrupts can be triggered on user-specified CPU cores only in controllable way.

Marcelo and Lawrence may have more input in this area.

Also irq allocation really belongs to device & driver stuff, how can that be hack? We even may not abstract public API in block layer for handling irq related thing.

Thanks,
Ming

#3 lore raw

Openshift needs the ability to dynamically move IRQs of all drivers away from a specific set of CPUs, at the point that an isolated workload starts running on those CPUs, and requires high performance guarantees, i.e. no HW interrupts to occur. To achieve this, dynamic setting of the smp_affinity for all drivers is used - at the moment, the NVME driver does not support this, so the NVME IRQs remain running on CPUs they should not be on, and so impact performance of the isolated workload.

Ability to set the smp_affinity seems like a generic feature, that should be supported by all drivers - I'm unclear how adding this feature to the NVME driver can be viewed as a hack?

Thanks,
Lawrence

-----Original Message-----
From: Ming Lei <[email protected]>
Sent: Tuesday, July 2, 2024 1:12 PM
To: Christoph Hellwig <[email protected]>
Cc: Keith Busch <[email protected]>; [email protected]; Sagi Grimberg <[email protected]>; Lawrence Troup (ltroup) <[email protected]>; Marcelo Tosatti <[email protected]>; [email protected]
Subject: Re: [PATCH V3] nvme-pci: allow unmanaged interrupts

Quoted 8 lines · Christoph Hellwig
Quoted 7 lines · Ming Lei

From: Keith Busch <[email protected]>

People _really_ want to control their interrupt affinity in some cases, such as Openshift with Performance profile, in which each irq's affinity is completely specified from userspace. Turns out that 'isolcpus=managed_irqs' isn't enough.

Quoted 6 lines
Quoted 2 lines

Add module parameter to allow unmanaged interrupts, just as some SCSI drivers are doing.

Same as before: hell no. We can't just add hacky global kernel parameters everywhere. We need the cpu isolation infrastructure to work properly instead of piling hacks of hacks in every relevant driver.

Per my understanding, here cpu isolation infrastructure can't work for Openshift, in which IO workload can be run on applications which are executed on isolated CPUs, meantime userspace do expect that interrupts can be triggered on user-specified CPU cores only in controllable way.

Marcelo and Lawrence may have more input in this area.

Also irq allocation really belongs to device & driver stuff, how can that be hack? We even may not abstract public API in block layer for handling irq related thing.

Thanks,
Ming

#4 lore raw
Quoted 1 line · Lawrence Troup (ltroup)

Openshift needs the ability to dynamically move IRQs of all drivers away from a specific set of CPUs, at the point that an isolated workload starts running on those CPUs, and requires high performance guarantees, i.e. no HW interrupts to occur. To achieve this, dynamic setting of the smp_affinity for all drivers is used - at the moment, the NVME driver does not support this, so the NVME IRQs remain running on CPUs they should not be on, and so impact performance of the isolated workload.

Then you need to create a core kernel interface that moves all managed IRQs of these CPUs. Without that you have a never ending whack a mole adding hacks to the drivers of the day that you care about.

author
#5 lore raw
Quoted 5 lines · Christoph Hellwig
Quoted 1 line · Lawrence Troup (ltroup)

Openshift needs the ability to dynamically move IRQs of all drivers away from a specific set of CPUs, at the point that an isolated workload starts running on those CPUs, and requires high performance guarantees, i.e. no HW interrupts to occur. To achieve this, dynamic setting of the smp_affinity for all drivers is used - at the moment, the NVME driver does not support this, so the NVME IRQs remain running on CPUs they should not be on, and so impact performance of the isolated workload.

Then you need to create a core kernel interface that moves all managed IRQs of these CPUs.

It is basically not doable or too hard since blk-mq queue mapping depends on managed irq affinity.

Quoted 2 lines

Without that you have a never ending whack a mole adding hacks to the drivers of the day that you care about.

So far, most popular SCSI hosts support the option via module parameter, and the only remainder could be nvme-pci.

Thanks,
Ming

#6 lore raw
Quoted 2 lines · Ming Lei

So far, most popular SCSI hosts support the option via module parameter, and the only remainder could be nvme-pci.

Until something new comes along and then we need to play this game again.

I might sound like a broken record but we need to fix this for real.

#7 lore raw
In reply to Ming Lei
Quoted 21 lines · Ming Lei
Quoted 8 lines · Christoph Hellwig
Quoted 7 lines · Ming Lei

From: Keith Busch <[email protected]>

People _really_ want to control their interrupt affinity in some cases, such as Openshift with Performance profile, in which each irq's affinity is completely specified from userspace. Turns out that 'isolcpus=managed_irqs' isn't enough.

Quoted 6 lines
Quoted 2 lines

Add module parameter to allow unmanaged interrupts, just as some SCSI drivers are doing.

Same as before: hell no. We can't just add hacky global kernel parameters everywhere. We need the cpu isolation infrastructure to work properly instead of piling hacks of hacks in every relevant driver.

Per my understanding, here cpu isolation infrastructure can't work for Openshift, in which IO workload can be run on applications which are executed on isolated CPUs, meantime userspace do expect that interrupts can be triggered on user-specified CPU cores only in controllable way.

Quoted 5 lines

Marcelo and Lawrence may have more input in this area.

Also irq allocation really belongs to device & driver stuff, how can that be hack? We even may not abstract public API in block layer for handling irq related thing.

I am confused. I though you told me that my series 'nvme-pci: honor isolcpus configuration' is not necessary. But you still need this patch to get the affinity sorted out? Wouldn't it make sense to figure out how we can make my series working also for your use case? E.g. we could introduce another HK type (io_queue) to control the affinity. This would decouple if from the managed_irq option.

author
#8 lore raw
Quoted 22 lines · Daniel Wagner
Quoted 21 lines · Ming Lei
Quoted 8 lines · Christoph Hellwig
Quoted 7 lines · Ming Lei

From: Keith Busch <[email protected]>

People _really_ want to control their interrupt affinity in some cases, such as Openshift with Performance profile, in which each irq's affinity is completely specified from userspace. Turns out that 'isolcpus=managed_irqs' isn't enough.

Quoted 6 lines
Quoted 2 lines

Add module parameter to allow unmanaged interrupts, just as some SCSI drivers are doing.

Same as before: hell no. We can't just add hacky global kernel parameters everywhere. We need the cpu isolation infrastructure to work properly instead of piling hacks of hacks in every relevant driver.

Per my understanding, here cpu isolation infrastructure can't work for Openshift, in which IO workload can be run on applications which are executed on isolated CPUs, meantime userspace do expect that interrupts can be triggered on user-specified CPU cores only in controllable way.

Quoted 8 lines
Quoted 5 lines

Marcelo and Lawrence may have more input in this area.

Also irq allocation really belongs to device & driver stuff, how can that be hack? We even may not abstract public API in block layer for handling irq related thing.

I am confused. I though you told me that my series 'nvme-pci: honor isolcpus configuration' is not necessary. But you still need this patch

Your patch fixes nothing basically, meantime it introduces regression. But I don't object the approach if blk-mq regressions can be solved.

Quoted 4 lines

to get the affinity sorted out? Wouldn't it make sense to figure out how we can make my series working also for your use case? E.g. we could introduce another HK type (io_queue) to control the affinity. This would decouple if from the managed_irq option.

Adding new HK type can't help this issue because Openshift environment needs to control each irq's affinity by themselves dynamically, and even IO workload may be run on isolated CPUs.

Thanks,
Ming

#9 lore raw
Quoted 35 lines · Ming Lei
Quoted 22 lines · Daniel Wagner
Quoted 21 lines · Ming Lei
Quoted 8 lines · Christoph Hellwig
Quoted 7 lines · Ming Lei

From: Keith Busch <[email protected]>

People _really_ want to control their interrupt affinity in some cases, such as Openshift with Performance profile, in which each irq's affinity is completely specified from userspace. Turns out that 'isolcpus=managed_irqs' isn't enough.

Quoted 6 lines
Quoted 2 lines

Add module parameter to allow unmanaged interrupts, just as some SCSI drivers are doing.

Same as before: hell no. We can't just add hacky global kernel parameters everywhere. We need the cpu isolation infrastructure to work properly instead of piling hacks of hacks in every relevant driver.

Per my understanding, here cpu isolation infrastructure can't work for Openshift, in which IO workload can be run on applications which are executed on isolated CPUs, meantime userspace do expect that interrupts can be triggered on user-specified CPU cores only in controllable way.

Quoted 8 lines
Quoted 5 lines

Marcelo and Lawrence may have more input in this area.

Also irq allocation really belongs to device & driver stuff, how can that be hack? We even may not abstract public API in block layer for handling irq related thing.

I am confused. I though you told me that my series 'nvme-pci: honor isolcpus configuration' is not necessary. But you still need this patch

Your patch fixes nothing basically, meantime it introduces regression. But I don't object the approach if blk-mq regressions can be solved.

Quoted 9 lines
Quoted 4 lines

to get the affinity sorted out? Wouldn't it make sense to figure out how we can make my series working also for your use case? E.g. we could introduce another HK type (io_queue) to control the affinity. This would decouple if from the managed_irq option.

Adding new HK type can't help this issue because Openshift environment needs to control each irq's affinity by themselves dynamically, and even IO workload may be run on isolated CPUs.

Understood, but that is what Daniel is working on, namely split the pool of available interrupts such that each side (housekeeping and isolcpus) gets a fair share of interrupts.

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke                  Kernel Storage Architect
[email protected]                                +49 911 74053 688
SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
#10 lore raw
In reply to Ming Lei
Quoted 29 lines · Ming Lei

From: Keith Busch <[email protected]>

People _really_ want to control their interrupt affinity in some cases, such as Openshift with Performance profile, in which each irq's affinity is completely specified from userspace. Turns out that 'isolcpus=managed_irqs' isn't enough.

Add module parameter to allow unmanaged interrupts, just as some SCSI drivers are doing.

---
v2->v3:
- rebase on for-next
- add openshift use case

v1->v2: skip the the AFFINITY vector allocation if the parameter is provided instead trying to make the vector code handle all post_vectors.

 drivers/nvme/host/pci.c | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)
drivers/nvme/host/pci.c +0 -0
index 5d8035218de9..a39c99c9b64d 100644

drivers/nvme/host/pci.c MODULE_PARM_DESC(sgl_threshold, · line 70

Quoted 8 lines
drivers/nvme/host/pci.c +5 -0
@@ -63,6 +63,11 @@ MODULE_PARM_DESC(sgl_threshold,
 		"Use SGLs when average request segment size is larger or equal to "
 		"this size. Use 0 to disable SGLs.");
 
+static bool managed_irqs = true;
+module_param(managed_irqs, bool, 0444);
+MODULE_PARM_DESC(managed_irqs,
+		 "set to false for user controlled irq affinity");
+

What if you set "static bool managed_irqs" to false when isolcpus is being used?

For example:

if (housekeeping_enabled(HK_TYPE_MANAGED_IRQ))

	managed_irqs = false;

Then there is no additional parameter to tune (which addresses Christoph's concern).

Quoted 42 lines
drivers/nvme/host/pci.c +11 -2
 #define NVME_PCI_MIN_QUEUE_SIZE 2
 #define NVME_PCI_MAX_QUEUE_SIZE 4095
 static int io_queue_depth_set(const char *val, const struct kernel_param *kp);
@@ -456,7 +461,7 @@ static void nvme_pci_map_queues(struct blk_mq_tag_set *set)
 		 * affinity), so use the regular blk-mq cpu mapping
 		 */
 		map->queue_offset = qoff;
-		if (i != HCTX_TYPE_POLL && offset)
+		if (managed_irqs && i != HCTX_TYPE_POLL && offset)
 			blk_mq_pci_map_queues(map, to_pci_dev(dev->dev), offset);
 		else
 			blk_mq_map_queues(map);
@@ -2226,6 +2231,7 @@ static int nvme_setup_irqs(struct nvme_dev *dev, unsigned int nr_io_queues)
 	};
 	unsigned int irq_queues, poll_queues;
 	unsigned int flags = PCI_IRQ_ALL_TYPES | PCI_IRQ_AFFINITY;
+	int ret;
 
 	/*
 	 * Poll queues don't need interrupts, but we need at least one I/O queue
@@ -2251,8 +2257,16 @@ static int nvme_setup_irqs(struct nvme_dev *dev, unsigned int nr_io_queues)
 		irq_queues += (nr_io_queues - poll_queues);
 	if (dev->ctrl.quirks & NVME_QUIRK_BROKEN_MSI)
 		flags &= ~PCI_IRQ_MSI;
-	return pci_alloc_irq_vectors_affinity(pdev, 1, irq_queues, flags,
+
+	if (managed_irqs)
+		return pci_alloc_irq_vectors_affinity(pdev, 1, irq_queues, flags,
 					      &affd);
+
+	flags &= ~PCI_IRQ_AFFINITY;
+	ret = pci_alloc_irq_vectors(pdev, 1, irq_queues, flags);
+	if (ret > 0)
+		nvme_calc_irq_sets(&affd, ret - 1);
+	return ret;
 }
 
 static unsigned int nvme_max_io_queues(struct nvme_dev *dev)
Signature · 1 line
2.44.0
#11 lore raw
Quoted 41 lines · Marcelo Tosatti
Quoted 29 lines · Ming Lei

From: Keith Busch <[email protected]>

People _really_ want to control their interrupt affinity in some cases, such as Openshift with Performance profile, in which each irq's affinity is completely specified from userspace. Turns out that 'isolcpus=managed_irqs' isn't enough.

Add module parameter to allow unmanaged interrupts, just as some SCSI drivers are doing.

---
v2->v3:
- rebase on for-next
- add openshift use case

v1->v2: skip the the AFFINITY vector allocation if the parameter is provided instead trying to make the vector code handle all post_vectors.

 drivers/nvme/host/pci.c | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)
drivers/nvme/host/pci.c +0 -0
index 5d8035218de9..a39c99c9b64d 100644

drivers/nvme/host/pci.c MODULE_PARM_DESC(sgl_threshold, · line 70

Quoted 8 lines
drivers/nvme/host/pci.c +5 -0
@@ -63,6 +63,11 @@ MODULE_PARM_DESC(sgl_threshold,
 		"Use SGLs when average request segment size is larger or equal to "
 		"this size. Use 0 to disable SGLs.");
 
+static bool managed_irqs = true;
+module_param(managed_irqs, bool, 0444);
+MODULE_PARM_DESC(managed_irqs,
+		 "set to false for user controlled irq affinity");
+

What if you set "static bool managed_irqs" to false when isolcpus is being used?

Quoted 4 lines

For example:

if (housekeeping_enabled(HK_TYPE_MANAGED_IRQ))

	managed_irqs = false;

Well, that is confusing.

Maybe just:

if (housekeeping_enabled(HK_FLAG_MISC))

	managed_irqs = false;

Instead.

Quoted 3 lines

Then there is no additional parameter to tune (which addresses Christoph's concern).

#12 lore raw
In reply to Marcelo Tosatti
Quoted 9 lines · Marcelo Tosatti

What if you set "static bool managed_irqs" to false when isolcpus is being used?

For example:

if (housekeeping_enabled(HK_TYPE_MANAGED_IRQ))

	managed_irqs = false;

Then there is no additional parameter to tune (which addresses Christoph's concern).

No, it absolutely does not. We need to still properly spread the irqs using the proper managed irq setup, we just need to do it only on the CPUs they should be spread on.