Skip to contentSkip to conversation

blk-mq: fix blk_mq_alloc_request_hctx

PATCH Superseded Ming Lei 6 participants 22 messages Jul 15, 2021 – Jul 22, 2021

Subsystems (6)
SCSI Block layer PCI Driver core +2 more subsystems

v4 of 7 Superseded Latest is v7 →

lore
Share & tools
lore

Ming Lei

author
#0 lore raw

v4 cover blk-mq: fix blk_mq_alloc_request_hctx

Hi,

blk_mq_alloc_request_hctx() is used by NVMe fc/rdma/tcp/loop to connect io queue. Also the sw ctx is chosen as the 1st online cpu in hctx->cpumask. However, all cpus in hctx->cpumask may be offline.

This usage model isn't well supported by blk-mq which supposes allocator is always done on one online CPU in hctx->cpumask. This assumption is related with managed irq, which also requires blk-mq to drain inflight request in this hctx when the last cpu in hctx->cpumask is going to offline.

However, NVMe fc/rdma/tcp/loop don't use managed irq, so we should allow them to ask for request allocation when the specified hctx is inactive (all cpus in hctx->cpumask are offline). Fix blk_mq_alloc_request_hctx() by allowing to allocate request when all CPUs of this hctx are offline.

V4:
- remove patches for cleanup queue map helpers
- take Christoph's suggestion to add field into 'struct device' for describing if managed irq is allocated from one device

V3:
- cleanup map queues helpers, and remove pci/virtio/rdma queue helpers
- store use managed irq info into qmap

V2:
- use flag of BLK_MQ_F_MANAGED_IRQ
- pass BLK_MQ_F_MANAGED_IRQ from driver explicitly
- kill BLK_MQ_F_STACKING

Ming Lei (3):
driver core: mark device as irq affinity managed if any irq is managed
blk-mq: mark if one queue map uses managed irq
blk-mq: don't deactivate hctx if managed irq isn't used

 block/blk-mq-pci.c      |  1 +
 block/blk-mq-rdma.c     |  3 +++
 block/blk-mq-virtio.c   |  1 +
 block/blk-mq.c          | 27 +++++++++++++++++----------
 block/blk-mq.h          |  8 ++++++++
 drivers/base/platform.c |  7 +++++++
 drivers/pci/msi.c       |  3 +++
 include/linux/blk-mq.h  |  3 ++-
 include/linux/device.h  |  1 +
 9 files changed, 43 insertions(+), 11 deletions(-)
Signature · 1 line
2.31.1
author
#1 lore raw

PATCH v4 1/3 driver core: mark device as irq affinity managed if any irq is managed

irq vector allocation with managed affinity may be used by driver, and blk-mq needs this info because managed irq will be shutdown when all CPUs in the affinity mask are offline.

The info of using managed irq is often produced by drivers(pci subsystem, platform device, ...), and it is consumed by blk-mq, so different subsystems are involved in this info flow

Address this issue by adding one field of .irq_affinity_managed into 'struct device'.

---
 drivers/base/platform.c | 7 +++++++
 drivers/pci/msi.c       | 3 +++
 include/linux/device.h  | 1 +
 3 files changed, 11 insertions(+)
drivers/base/platform.c +7 -0
index 8640578f45e9..d28cb91d5cf9 100644
@@ -388,6 +388,13 @@ int devm_platform_get_irqs_affinity(struct platform_device *dev,
 				ptr->irq[i], ret);
 			goto err_free_desc;
 		}
+
+		/*
+		 * mark the device as irq affinity managed if any irq affinity
+		 * descriptor is managed
+		 */
+		if (desc[i].is_managed)
+			dev->dev.irq_affinity_managed = true;
 	}
 
 	devres_add(&dev->dev, ptr);
drivers/pci/msi.c +3 -0
index 3d6db20d1b2b..7ddec90b711d 100644
@@ -1197,6 +1197,7 @@ int pci_alloc_irq_vectors_affinity(struct pci_dev *dev, unsigned int min_vecs,
 	if (flags & PCI_IRQ_AFFINITY) {
 		if (!affd)
 			affd = &msi_default_affd;
+		dev->dev.irq_affinity_managed = true;
 	} else {
 		if (WARN_ON(affd))
 			affd = NULL;
@@ -1215,6 +1216,8 @@ int pci_alloc_irq_vectors_affinity(struct pci_dev *dev, unsigned int min_vecs,
 			return nvecs;
 	}
 
+	dev->dev.irq_affinity_managed = false;
+
 	/* use legacy IRQ if allowed */
 	if (flags & PCI_IRQ_LEGACY) {
 		if (min_vecs == 1 && dev->irq) {
include/linux/device.h +1 -0
index 59940f1744c1..9ec6e671279e 100644
@@ -569,6 +569,7 @@ struct device {
 #ifdef CONFIG_DMA_OPS_BYPASS
 	bool			dma_ops_bypass : 1;
 #endif
+	bool			irq_affinity_managed : 1;
 };
 
 /**
Signature · 1 line
2.31.1
#2 lore raw

Re: [PATCH V4 1/3] driver core: mark device as irq affinity managed if any irq is managed

include/linux/device.h struct device { · line 573

Quoted 8 lines · Ming Lei
include/linux/device.h +1 -0
@@ -569,6 +569,7 @@ struct device {
 #ifdef CONFIG_DMA_OPS_BYPASS
 	bool			dma_ops_bypass : 1;
 #endif
+	bool			irq_affinity_managed : 1;
 };

No documentation for this new field?

:(

author
#3 lore raw

Re: [PATCH V4 1/3] driver core: mark device as irq affinity managed if any irq is managed

Quoted 11 lines · Greg Kroah-Hartman

include/linux/device.h struct device { · line 573

Quoted 8 lines · Ming Lei
include/linux/device.h +1 -0
@@ -569,6 +569,7 @@ struct device {
 #ifdef CONFIG_DMA_OPS_BYPASS
 	bool			dma_ops_bypass : 1;
 #endif
+	bool			irq_affinity_managed : 1;
 };

No documentation for this new field?

OK, will fix it in next version.

Thanks,
Ming

#4 lore raw

Re: [PATCH V4 1/3] driver core: mark device as irq affinity managed if any irq is managed

In reply to Ming Lei
Quoted 5 lines · Ming Lei

irq vector allocation with managed affinity may be used by driver, and blk-mq needs this info because managed irq will be shutdown when all CPUs in the affinity mask are offline.

The info of using managed irq is often produced by drivers(pci subsystem,

Add space between "drivers" and "(".
s/pci/PCI/

Does this "managed IRQ" (or "managed affinity", not sure what the correct terminology is here) have something to do with devm?

Quoted 2 lines

platform device, ...), and it is consumed by blk-mq, so different subsystems are involved in this info flow

Add period at end of sentence.

Quoted 34 lines

Address this issue by adding one field of .irq_affinity_managed into 'struct device'.

---
 drivers/base/platform.c | 7 +++++++
 drivers/pci/msi.c       | 3 +++
 include/linux/device.h  | 1 +
 3 files changed, 11 insertions(+)
drivers/base/platform.c +7 -0
index 8640578f45e9..d28cb91d5cf9 100644
@@ -388,6 +388,13 @@ int devm_platform_get_irqs_affinity(struct platform_device *dev,
 				ptr->irq[i], ret);
 			goto err_free_desc;
 		}
+
+		/*
+		 * mark the device as irq affinity managed if any irq affinity
+		 * descriptor is managed
+		 */
+		if (desc[i].is_managed)
+			dev->dev.irq_affinity_managed = true;
 	}
 
 	devres_add(&dev->dev, ptr);
drivers/pci/msi.c +0 -0
index 3d6db20d1b2b..7ddec90b711d 100644

drivers/pci/msi.c int pci_alloc_irq_vectors_affinity(struct pci_dev *dev, unsigned int min_vecs, · line 1200

Quoted 4 lines
drivers/pci/msi.c +1 -0
@@ -1197,6 +1197,7 @@ int pci_alloc_irq_vectors_affinity(struct pci_dev *dev, unsigned int min_vecs,
 	if (flags & PCI_IRQ_AFFINITY) {
 		if (!affd)
 			affd = &msi_default_affd;
+		dev->dev.irq_affinity_managed = true;

This is really opaque to me. I can't tell what the connection between PCI_IRQ_AFFINITY and irq_affinity_managed is.

AFAICT the only place irq_affinity_managed is ultimately used is blk_mq_hctx_notify_offline(), and there's no obvious connection between that and this code.

Quoted 32 lines
drivers/pci/msi.c +2 -0
 	} else {
 		if (WARN_ON(affd))
 			affd = NULL;
@@ -1215,6 +1216,8 @@ int pci_alloc_irq_vectors_affinity(struct pci_dev *dev, unsigned int min_vecs,
 			return nvecs;
 	}
 
+	dev->dev.irq_affinity_managed = false;
+
 	/* use legacy IRQ if allowed */
 	if (flags & PCI_IRQ_LEGACY) {
 		if (min_vecs == 1 && dev->irq) {
include/linux/device.h +1 -0
index 59940f1744c1..9ec6e671279e 100644
@@ -569,6 +569,7 @@ struct device {
 #ifdef CONFIG_DMA_OPS_BYPASS
 	bool			dma_ops_bypass : 1;
 #endif
+	bool			irq_affinity_managed : 1;
 };
 
 /**
Signature · 7 lines
2.31.1


_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
author
#5 lore raw

Re: [PATCH V4 1/3] driver core: mark device as irq affinity managed if any irq is managed

Quoted 9 lines · Bjorn Helgaas
Quoted 5 lines · Ming Lei

irq vector allocation with managed affinity may be used by driver, and blk-mq needs this info because managed irq will be shutdown when all CPUs in the affinity mask are offline.

The info of using managed irq is often produced by drivers(pci subsystem,

Add space between "drivers" and "(".
s/pci/PCI/

OK.

Quoted 8 lines

Does this "managed IRQ" (or "managed affinity", not sure what the correct terminology is here) have something to do with devm?

Quoted 2 lines

platform device, ...), and it is consumed by blk-mq, so different subsystems are involved in this info flow

Add period at end of sentence.

OK.

Quoted 29 lines
Quoted 29 lines

Address this issue by adding one field of .irq_affinity_managed into 'struct device'.

---
 drivers/base/platform.c | 7 +++++++
 drivers/pci/msi.c       | 3 +++
 include/linux/device.h  | 1 +
 3 files changed, 11 insertions(+)
drivers/base/platform.c +7 -0
index 8640578f45e9..d28cb91d5cf9 100644
@@ -388,6 +388,13 @@ int devm_platform_get_irqs_affinity(struct platform_device *dev,
 				ptr->irq[i], ret);
 			goto err_free_desc;
 		}
+
+		/*
+		 * mark the device as irq affinity managed if any irq affinity
+		 * descriptor is managed
+		 */
+		if (desc[i].is_managed)
+			dev->dev.irq_affinity_managed = true;
 	}
 
Quoted 13 lines

drivers/pci/msi.c int pci_alloc_irq_vectors_affinity(struct pci_dev *dev, unsigned int min_vecs, · line 1200

Quoted 10 lines
drivers/base/platform.c +0 -0
 	devres_add(&dev->dev, ptr);
drivers/pci/msi.c +1 -0
index 3d6db20d1b2b..7ddec90b711d 100644
@@ -1197,6 +1197,7 @@ int pci_alloc_irq_vectors_affinity(struct pci_dev *dev, unsigned int min_vecs,
 	if (flags & PCI_IRQ_AFFINITY) {
 		if (!affd)
 			affd = &msi_default_affd;
+		dev->dev.irq_affinity_managed = true;

This is really opaque to me. I can't tell what the connection between PCI_IRQ_AFFINITY and irq_affinity_managed is.

Comment for PCI_IRQ_AFFINITY is 'Auto-assign affinity', 'irq_affinity_managed' basically means that irq's affinity is managed by kernel.

What blk-mq needs is exactly if PCI_IRQ_AFFINITY is applied when allocating irq vectors. When PCI_IRQ_AFFINITY is used, genirq will shutdown the irq when all CPUs in the assigned affinity are offline, then blk-mq has to drain all in-flight IOs which will be completed via this irq and prevent new IO. That is the connection.

Or you think 'irq_affinity_managed' isn't named well?

Quoted 4 lines

AFAICT the only place irq_affinity_managed is ultimately used is blk_mq_hctx_notify_offline(), and there's no obvious connection between that and this code.

I believe the connection is described in comment.

Thanks,
Ming

#6 lore raw

Re: [PATCH V4 1/3] driver core: mark device as irq affinity managed if any irq is managed

Quoted 69 lines · Ming Lei
Quoted 9 lines · Bjorn Helgaas
Quoted 5 lines · Ming Lei

irq vector allocation with managed affinity may be used by driver, and blk-mq needs this info because managed irq will be shutdown when all CPUs in the affinity mask are offline.

The info of using managed irq is often produced by drivers(pci subsystem,

Add space between "drivers" and "(".
s/pci/PCI/

OK.

Quoted 7 lines

Does this "managed IRQ" (or "managed affinity", not sure what the correct terminology is here) have something to do with devm?

Quoted 2 lines

platform device, ...), and it is consumed by blk-mq, so different subsystems are involved in this info flow

Add period at end of sentence.

OK.

Quoted 28 lines
Quoted 28 lines

Address this issue by adding one field of .irq_affinity_managed into 'struct device'.

---
 drivers/base/platform.c | 7 +++++++
 drivers/pci/msi.c       | 3 +++
 include/linux/device.h  | 1 +
 3 files changed, 11 insertions(+)
drivers/base/platform.c +7 -0
index 8640578f45e9..d28cb91d5cf9 100644
@@ -388,6 +388,13 @@ int devm_platform_get_irqs_affinity(struct platform_device *dev,
 				ptr->irq[i], ret);
 			goto err_free_desc;
 		}
+
+		/*
+		 * mark the device as irq affinity managed if any irq affinity
+		 * descriptor is managed
+		 */
+		if (desc[i].is_managed)
+			dev->dev.irq_affinity_managed = true;
 	}
 
Quoted 13 lines

drivers/pci/msi.c int pci_alloc_irq_vectors_affinity(struct pci_dev *dev, unsigned int min_vecs, · line 1200

Quoted 10 lines
drivers/base/platform.c +0 -0
 	devres_add(&dev->dev, ptr);
drivers/pci/msi.c +1 -0
index 3d6db20d1b2b..7ddec90b711d 100644
@@ -1197,6 +1197,7 @@ int pci_alloc_irq_vectors_affinity(struct pci_dev *dev, unsigned int min_vecs,
 	if (flags & PCI_IRQ_AFFINITY) {
 		if (!affd)
 			affd = &msi_default_affd;
+		dev->dev.irq_affinity_managed = true;

This is really opaque to me. I can't tell what the connection between PCI_IRQ_AFFINITY and irq_affinity_managed is.

Comment for PCI_IRQ_AFFINITY is 'Auto-assign affinity', 'irq_affinity_managed' basically means that irq's affinity is managed by kernel.

Quoted 7 lines

What blk-mq needs is exactly if PCI_IRQ_AFFINITY is applied when allocating irq vectors. When PCI_IRQ_AFFINITY is used, genirq will shutdown the irq when all CPUs in the assigned affinity are offline, then blk-mq has to drain all in-flight IOs which will be completed via this irq and prevent new IO. That is the connection.

Or you think 'irq_affinity_managed' isn't named well?

I've been looking at "devm" management where there is a concept of devm-managed IRQs, and you mentioned "managed IRQ" in the commit log. But IIUC this is a completely different sort of management.

Quoted 5 lines
Quoted 3 lines

AFAICT the only place irq_affinity_managed is ultimately used is blk_mq_hctx_notify_offline(), and there's no obvious connection between that and this code.

I believe the connection is described in comment.

You mean the comment that says "hctx needn't to be deactivated in case managed irq isn't used"? Sorry, that really doesn't explain to me why pci_alloc_irq_vectors_affinity() needs to set irq_affinity_managed. There's just a lot of magic here that cannot be deduced by reading the code.

Nit: s/needn't to be/needn't be/ in that comment. Or maybe even "Deactivate hctx only when managed IRQ is used"? I still have no idea what that means, but at least it's easier to read :) Or maybe this is actually "draining a queue" instead of "deactivating"?

Bjorn

#7 lore raw

Re: [PATCH V4 1/3] driver core: mark device as irq affinity managed if any irq is managed

In reply to Ming Lei
Quoted 8 lines · Ming Lei

irq vector allocation with managed affinity may be used by driver, and blk-mq needs this info because managed irq will be shutdown when all CPUs in the affinity mask are offline.

The info of using managed irq is often produced by drivers(pci subsystem, platform device, ...), and it is consumed by blk-mq, so different subsystems are involved in this info flow

Quoted 5 lines

Address this issue by adding one field of .irq_affinity_managed into 'struct device'.

Did you consider that for PCI device we effectively have this info already:

bool dev_has_managed_msi_irq(struct device *dev)

{
	struct msi_desc *desc;

	list_for_each_entry(desc, dev_to_msi_list(dev), list) {
		if (desc->affinity && desc->affinity->is_managed)
			return true;
	}

	return false;
}

Thanks,
John

Quoted 58 lines
---
  drivers/base/platform.c | 7 +++++++
  drivers/pci/msi.c       | 3 +++
  include/linux/device.h  | 1 +
  3 files changed, 11 insertions(+)
drivers/base/platform.c +7 -0
index 8640578f45e9..d28cb91d5cf9 100644
@@ -388,6 +388,13 @@ int devm_platform_get_irqs_affinity(struct platform_device *dev,
  				ptr->irq[i], ret);
  			goto err_free_desc;
  		}
+
+		/*
+		 * mark the device as irq affinity managed if any irq affinity
+		 * descriptor is managed
+		 */
+		if (desc[i].is_managed)
+			dev->dev.irq_affinity_managed = true;
  	}
 
  	devres_add(&dev->dev, ptr);
drivers/pci/msi.c +3 -0
index 3d6db20d1b2b..7ddec90b711d 100644
@@ -1197,6 +1197,7 @@ int pci_alloc_irq_vectors_affinity(struct pci_dev *dev, unsigned int min_vecs,
  	if (flags & PCI_IRQ_AFFINITY) {
  		if (!affd)
  			affd = &msi_default_affd;
+		dev->dev.irq_affinity_managed = true;
  	} else {
  		if (WARN_ON(affd))
  			affd = NULL;
@@ -1215,6 +1216,8 @@ int pci_alloc_irq_vectors_affinity(struct pci_dev *dev, unsigned int min_vecs,
  			return nvecs;
  	}
 
+	dev->dev.irq_affinity_managed = false;
+
  	/* use legacy IRQ if allowed */
  	if (flags & PCI_IRQ_LEGACY) {
  		if (min_vecs == 1 && dev->irq) {
include/linux/device.h +1 -0
index 59940f1744c1..9ec6e671279e 100644
@@ -569,6 +569,7 @@ struct device {
  #ifdef CONFIG_DMA_OPS_BYPASS
  	bool			dma_ops_bypass : 1;
  #endif
+	bool			irq_affinity_managed : 1;
  };
 
  /**
#8 lore raw

Re: [PATCH V4 1/3] driver core: mark device as irq affinity managed if any irq is managed

Quoted 9 lines · John Garry
Quoted 5 lines

Address this issue by adding one field of .irq_affinity_managed into 'struct device'.

Did you consider that for PCI device we effectively have this info already:

bool dev_has_managed_msi_irq(struct device *dev)

Quoted 9 lines
{
	struct msi_desc *desc;

	list_for_each_entry(desc, dev_to_msi_list(dev), list) {
		if (desc->affinity && desc->affinity->is_managed)
			return true;
	}

	return false;

Just walking the list seems fine to me given that this is not a performance criticial path. But what are the locking implications?

Also does the above imply this won't work for your platform MSI case?

#9 lore raw

Re: [PATCH V4 1/3] driver core: mark device as irq affinity managed if any irq is managed

Quoted 7 lines · Christoph Hellwig
Quoted 6 lines · John Garry
Quoted 5 lines

Address this issue by adding one field of .irq_affinity_managed into 'struct device'.

Quoted 7 lines
Quoted 7 lines

Did you consider that for PCI device we effectively have this info already:

bool dev_has_managed_msi_irq(struct device *dev)

{
	struct msi_desc *desc;

	list_for_each_entry(desc, dev_to_msi_list(dev), list)

I just noticed for_each_msi_entry(), which is the same

Quoted 8 lines
Quoted 5 lines
		if (desc->affinity && desc->affinity->is_managed)
			return true;
	}

	return false;

Just walking the list seems fine to me given that this is not a performance criticial path. But what are the locking implications?

Since it would be used for sequential setup code, I didn't think any locking was required. But would need to consider where that function lived and whether it's public.

Quoted 4 lines

Also does the above imply this won't work for your platform MSI case? .

Right. I think that it may be possible to reach into the platform msi descriptors to get this info, but I am not sure it's worth it. There is only 1x user there and there is no generic .map_queues function, so could set the flag directly:

int blk_mq_pci_map_queues(struct blk_mq_queue_map *qmap, struct pci_dev *pdev,
for_each_cpu(cpu, mask)
qmap->mq_map[cpu] = qmap->queue_offset + queue;
}
+ qmap->use_managed_irq = dev_has_managed_msi_irq(&pdev->dev);
}

drivers/scsi/hisi_sas/hisi_sas_v2_hw.c +2 -0
@@ -3563,6 +3563,8 @@ static int map_queues_v2_hw(struct Scsi_Host *shost)
                       qmap->mq_map[cpu] = qmap->queue_offset + queue;
       }
 
+       qmap->use_managed_irq = 1;
+
       return 0;

}

author
#10 lore raw

Re: [PATCH V4 1/3] driver core: mark device as irq affinity managed if any irq is managed

Quoted 23 lines · John Garry
Quoted 7 lines · Christoph Hellwig
Quoted 6 lines · John Garry
Quoted 5 lines

Address this issue by adding one field of .irq_affinity_managed into 'struct device'.

Quoted 7 lines
Quoted 7 lines

Did you consider that for PCI device we effectively have this info already:

bool dev_has_managed_msi_irq(struct device *dev)

{
	struct msi_desc *desc;

	list_for_each_entry(desc, dev_to_msi_list(dev), list)

I just noticed for_each_msi_entry(), which is the same

Quoted 4 lines
Quoted 4 lines
		if (desc->affinity && desc->affinity->is_managed)
			return true;
	}
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
	return false;

Just walking the list seems fine to me given that this is not a performance criticial path. But what are the locking implications?

Since it would be used for sequential setup code, I didn't think any locking was required. But would need to consider where that function lived and whether it's public.

Yeah, the allocated irq vectors should be live when running map queues.

Quoted 22 lines
Quoted 5 lines

Also does the above imply this won't work for your platform MSI case? .

Right. I think that it may be possible to reach into the platform msi descriptors to get this info, but I am not sure it's worth it. There is only 1x user there and there is no generic .map_queues function, so could set the flag directly:

int blk_mq_pci_map_queues(struct blk_mq_queue_map *qmap, struct pci_dev *pdev,

drivers/scsi/hisi_sas/hisi_sas_v2_hw.c +1 -0
 		for_each_cpu(cpu, mask)
 			qmap->mq_map[cpu] = qmap->queue_offset + queue;
 	}
+	qmap->use_managed_irq = dev_has_managed_msi_irq(&pdev->dev);

}

drivers/scsi/hisi_sas/hisi_sas_v2_hw.c static int map_queues_v2_hw(struct Scsi_Host *shost) · line 3568

Quoted 6 lines
drivers/scsi/hisi_sas/hisi_sas_v2_hw.c +2 -0
@@ -3563,6 +3563,8 @@ static int map_queues_v2_hw(struct Scsi_Host *shost)
                       qmap->mq_map[cpu] = qmap->queue_offset + queue;
       }
 
+       qmap->use_managed_irq = 1;
+
       return 0;

virtio can be populated via platform device too, but managed irq affinity isn't used, so seems dev_has_managed_msi_irq() is fine.

Thanks,
Ming

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

#11 lore raw

Re: [PATCH V4 1/3] driver core: mark device as irq affinity managed if any irq is managed

In reply to Christoph Hellwig
Quoted 18 lines · Christoph Hellwig
Quoted 17 lines · John Garry
Quoted 5 lines

Address this issue by adding one field of .irq_affinity_managed into 'struct device'.

Did you consider that for PCI device we effectively have this info already:

bool dev_has_managed_msi_irq(struct device *dev)

{
	struct msi_desc *desc;

	list_for_each_entry(desc, dev_to_msi_list(dev), list) {
		if (desc->affinity && desc->affinity->is_managed)
			return true;
	}
Quoted 4 lines
Quoted 1 line
	return false;

Just walking the list seems fine to me given that this is not a performance criticial path. But what are the locking implications?

At the moment there are none because the list is initialized in the setup path and never modified afterwards. Though that might change sooner than later to fix the virtio wreckage vs. MSI-X.

Quoted 1 line

Also does the above imply this won't work for your platform MSI case?

The msi descriptors are attached to struct device and independent of platform/PCI/whatever.

Thanks,

tglx

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

#12 lore raw

Re: [PATCH V4 1/3] driver core: mark device as irq affinity managed if any irq is managed

Quoted 6 lines · Thomas Gleixner
Quoted 2 lines

Just walking the list seems fine to me given that this is not a performance criticial path. But what are the locking implications?

At the moment there are none because the list is initialized in the setup path and never modified afterwards. Though that might change sooner than later to fix the virtio wreckage vs. MSI-X.

What is the issue there? Either way, if we keep the helper in the IRQ code it should be easy to spot for anyone adding the locking.

Quoted 4 lines
Quoted 1 line

Also does the above imply this won't work for your platform MSI case?

The msi descriptors are attached to struct device and independent of platform/PCI/whatever.

That's what I assumed, but this text from John suggested there is something odd about the platform case:

"Did you consider that for PCI .."

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

#13 lore raw

Re: [PATCH V4 1/3] driver core: mark device as irq affinity managed if any irq is managed

+ Marc

Quoted 9 lines · Christoph Hellwig
Quoted 5 lines · Thomas Gleixner
Quoted 2 lines

Just walking the list seems fine to me given that this is not a performance criticial path. But what are the locking implications?

At the moment there are none because the list is initialized in the setup path and never modified afterwards. Though that might change sooner than later to fix the virtio wreckage vs. MSI-X.

What is the issue there? Either way, if we keep the helper in the IRQ code it should be easy to spot for anyone adding the locking.

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

Also does the above imply this won't work for your platform MSI case?

The msi descriptors are attached to struct device and independent of platform/PCI/whatever.

That's what I assumed, but this text from John suggested there is something odd about the platform case:

"Did you consider that for PCI .."
.

For this special platform MSI case there is a secondary interrupt controller (called mbigen) which generates the MSI on behalf of the device, which I think the MSI belongs to (and not the device, itself).

See "latter case" mentioned in commit 91f90daa4fb2.

I think Marc and Thomas can explain this much better than I could.

Anyway, as I mentioned earlier, I think that this specific problem is unique and can be solved without using a function which examines the struct device.msi_list .

Thanks,
John

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

#14 lore raw

Re: [PATCH V4 1/3] driver core: mark device as irq affinity managed if any irq is managed

Quoted 8 lines · John Garry
Quoted 7 lines · Christoph Hellwig
Quoted 3 lines · Thomas Gleixner
Quoted 1 line

Also does the above imply this won't work for your platform MSI case?

The msi descriptors are attached to struct device and independent of platform/PCI/whatever.

That's what I assumed, but this text from John suggested there is something odd about the platform case:

Quoted 6 lines
Quoted 2 lines

"Did you consider that for PCI .."
.

For this special platform MSI case there is a secondary interrupt controller (called mbigen) which generates the MSI on behalf of the device, which I think the MSI belongs to (and not the device, itself).

MBIGEN is a different story because it converts wired interrupts into MSI interrupts, IOW a MSI based interrupt pin extender.

I might be wrong, but I seriously doubt that any multiqueue device which wants to use affinity managed interrupts is built on top of that.

Thanks,

tglx

#15 lore raw

Re: [PATCH V4 1/3] driver core: mark device as irq affinity managed if any irq is managed

Quoted 5 lines · Thomas Gleixner
Quoted 5 lines
Quoted 5 lines

That's what I assumed, but this text from John suggested there is something odd about the platform case:

"Did you consider that for PCI .."
.

Quoted 8 lines
Quoted 3 lines

For this special platform MSI case there is a secondary interrupt controller (called mbigen) which generates the MSI on behalf of the device, which I think the MSI belongs to (and not the device, itself).

MBIGEN is a different story because it converts wired interrupts into MSI interrupts, IOW a MSI based interrupt pin extender.

I might be wrong, but I seriously doubt that any multiqueue device which wants to use affinity managed interrupts is built on top of that.

Actually the SCSI device for which platform device managed interrupts support was added in commit e15f2fa959f2 uses mbigen.

Thanks,
John

#16 lore raw

Re: [PATCH V4 1/3] driver core: mark device as irq affinity managed if any irq is managed

In reply to Christoph Hellwig
Quoted 10 lines · Christoph Hellwig
Quoted 6 lines · Thomas Gleixner
Quoted 2 lines

Just walking the list seems fine to me given that this is not a performance criticial path. But what are the locking implications?

At the moment there are none because the list is initialized in the setup path and never modified afterwards. Though that might change sooner than later to fix the virtio wreckage vs. MSI-X.

What is the issue there? Either way, if we keep the helper in the IRQ code it should be easy to spot for anyone adding the locking.

https://lore.kernel.org/r/[email protected]

TLDR: virtio allocates ONE irq on msix_enable() and then when the guest actually unmasks another entry (e.g. request_irq()), it tears down the allocated one and set's up two. On the third one this repeats ....

There are only two options:

1) allocate everything upfront, which is undesired
2) append entries, which might need locking, but I'm still trying to avoid that

There is another problem vs. vector exhaustion which can't be fixed that way, but that's a different story.

Thanks,

tglx

#17 lore raw

Re: [PATCH V4 1/3] driver core: mark device as irq affinity managed if any irq is managed

Quoted 6 lines · Thomas Gleixner

https://lore.kernel.org/r/[email protected]

TLDR: virtio allocates ONE irq on msix_enable() and then when the guest actually unmasks another entry (e.g. request_irq()), it tears down the allocated one and set's up two. On the third one this repeats ....

Quoted 8 lines

There are only two options:

1) allocate everything upfront, which is undesired
2) append entries, which might need locking, but I'm still trying to avoid that

There is another problem vs. vector exhaustion which can't be fixed that way, but that's a different story.

FTI, NVMe is similar. We need one IRQ to setup the admin queue, which is used to query/set how many I/O queues are supported. Just two steps though and not unbound.

#18 lore raw

Re: [PATCH V4 1/3] driver core: mark device as irq affinity managed if any irq is managed

Quoted 5 lines · Christoph Hellwig
Quoted 4 lines · Thomas Gleixner

https://lore.kernel.org/r/[email protected]

TLDR: virtio allocates ONE irq on msix_enable() and then when the guest

OOps, sorry that should have been VFIO not virtio.

Quoted 9 lines
Quoted 9 lines

actually unmasks another entry (e.g. request_irq()), it tears down the allocated one and set's up two. On the third one this repeats ....

There are only two options:

1) allocate everything upfront, which is undesired
2) append entries, which might need locking, but I'm still trying to avoid that

Quoted 6 lines
Quoted 2 lines

There is another problem vs. vector exhaustion which can't be fixed that way, but that's a different story.

FTI, NVMe is similar. We need one IRQ to setup the admin queue, which is used to query/set how many I/O queues are supported. Just two steps though and not unbound.

That's fine because that's controlled by the driver consistently and it (hopefully) makes sure that the admin queue is quiesced before everything is torn down after the initial query.

But that's not the case for VFIO. It tears down all in use interrupts and the guest driver is completely oblivious of that.

Assume the following situation:

1) VM boots with 8 present CPUs and 16 possible CPUs

2) The passed through card (PF or VF) supports multiqueue and the driver uses managed interrupts which e.g. allocates one queue and one interrupt per possible CPU.

Initial setup requests all the interrupts, but only the first 8 queue interrupts are unmasked and therefore reallocated by the host which works by some definition of works because the device is quiet at that point.

3) Host admin plugs the other 8 CPUs into the guest

Onlining these CPUs in the guest will unmask the dormant managed queue interrupts and cause the host to allocate the remaining 8 per queue interrupts one by one thereby tearing down _all_ previously allocated ones and then allocating one more than before.

Assume that while this goes on the guest has I/O running on the already online CPUs and their associated queues. Depending on the device this either will lose interrupts or reroute them to the legacy INTx which is not handled. This might in the best case result in a few "timedout" requests, but I managed it at least once to make the device go into lala land state, i.e. it did not recover.

The above can be fixed by adding an 'append' mode to the MSI code.

But that does not fix the overcommit issue where the host runs out of vector space. The result is simply that the guest does not know and just continues to work on device/queues which will never ever recieve an interrupt (again).

I got educated that all of this is considered unlikely and my argument that the concept of unlikely simply does not exist at cloud scale got ignored. Sure, I know it's VIRT and therefore not subject to common sense.

Thanks,

tglx

#19 lore raw

Re: [PATCH V4 1/3] driver core: mark device as irq affinity managed if any irq is managed

Quoted 3 lines · Thomas Gleixner

That's fine because that's controlled by the driver consistently and it (hopefully) makes sure that the admin queue is quiesced before everything is torn down after the initial query.

Yes, it is.

Quoted 1 line

The above can be fixed by adding an 'append' mode to the MSI code.

So IFF we get that append mode I think it would help to simplify drivers that have unmanaged pre and post vectors, and/or do the above proving.

So instead of currently requesting a single unmanaged vector, do basic setup, tear it down, request N managed vectors with an unmanaged pre-vector we could just keep the unmanaged vector, never tear it down and just append the post vectors. In the long run this could remove the need to do the pre and post vectors entirely.

author
#20 lore raw

PATCH v4 2/3 blk-mq: mark if one queue map uses managed irq

In reply to Ming Lei

Retrieve this info via the field 'irq_affinity_managed' of 'struct device' in queue map helpers.

---
 block/blk-mq-pci.c     | 1 +
 block/blk-mq-rdma.c    | 3 +++
 block/blk-mq-virtio.c  | 1 +
 include/linux/blk-mq.h | 3 ++-
 4 files changed, 7 insertions(+), 1 deletion(-)
block/blk-mq-pci.c +1 -0
index b595a94c4d16..aa0bdb80d0ce 100644
@@ -37,6 +37,7 @@ int blk_mq_pci_map_queues(struct blk_mq_queue_map *qmap, struct pci_dev *pdev,
 		for_each_cpu(cpu, mask)
 			qmap->mq_map[cpu] = qmap->queue_offset + queue;
 	}
+	qmap->use_managed_irq = pdev->dev.irq_affinity_managed;
 
 	return 0;
 
block/blk-mq-rdma.c +3 -0
index 14f968e58b8f..7b10d8bd2a37 100644
@@ -36,6 +36,9 @@ int blk_mq_rdma_map_queues(struct blk_mq_queue_map *map,
 			map->mq_map[cpu] = map->queue_offset + queue;
 	}
 
+	/* So far RDMA doesn't use managed irq */
+	map->use_managed_irq = false;
+
 	return 0;
 
 fallback:
block/blk-mq-virtio.c +1 -0
index 7b8a42c35102..b57a0aa6d900 100644
@@ -38,6 +38,7 @@ int blk_mq_virtio_map_queues(struct blk_mq_queue_map *qmap,
 		for_each_cpu(cpu, mask)
 			qmap->mq_map[cpu] = qmap->queue_offset + queue;
 	}
+	qmap->use_managed_irq = vdev->dev.irq_affinity_managed;
 
 	return 0;
 fallback:
include/linux/blk-mq.h +2 -1
index 1d18447ebebc..d54a795ec971 100644
@@ -192,7 +192,8 @@ struct blk_mq_hw_ctx {
 struct blk_mq_queue_map {
 	unsigned int *mq_map;
 	unsigned int nr_queues;
-	unsigned int queue_offset;
+	unsigned int queue_offset:31;
+	unsigned int use_managed_irq:1;
 };
 
 /**
Signature · 1 line
2.31.1
author
#21 lore raw

PATCH v4 3/3 blk-mq: don't deactivate hctx if managed irq isn't used

In reply to Ming Lei

blk-mq deactivates one hctx when the last CPU in hctx->cpumask become offline by draining all requests originated from this hctx and moving new allocation on other active hctx. This way is for avoiding inflight IO in case of managed irq because managed irq is shutdown when the last CPU in the irq's affinity becomes offline.

However, lots of drivers(nvme fc, rdma, tcp, loop, ...) don't use managed irq, so they needn't to deactivate hctx when the last CPU becomes offline. Also, some of them are the only user of blk_mq_alloc_request_hctx() which is used for connecting io queue. And their requirement is that the connect request needs to be submitted successfully via one specified hctx even though all CPUs in this hctx->cpumask have become offline.

Addressing the requirement for nvme fc/rdma/loop by allowing to allocate request from one hctx when all CPUs in this hctx are offline, since these drivers don't use managed irq.

Finally don't deactivate one hctx when it doesn't use managed irq.

---
 block/blk-mq.c | 27 +++++++++++++++++----------
 block/blk-mq.h |  8 ++++++++
 2 files changed, 25 insertions(+), 10 deletions(-)
block/blk-mq.c +17 -10
index 2c4ac51e54eb..591ab07c64d8 100644
@@ -427,6 +427,15 @@ struct request *blk_mq_alloc_request(struct request_queue *q, unsigned int op,
 }
 EXPORT_SYMBOL(blk_mq_alloc_request);
 
+static inline int blk_mq_first_mapped_cpu(struct blk_mq_hw_ctx *hctx)
+{
+	int cpu = cpumask_first_and(hctx->cpumask, cpu_online_mask);
+
+	if (cpu >= nr_cpu_ids)
+		cpu = cpumask_first(hctx->cpumask);
+	return cpu;
+}
+
 struct request *blk_mq_alloc_request_hctx(struct request_queue *q,
 	unsigned int op, blk_mq_req_flags_t flags, unsigned int hctx_idx)
 {
@@ -468,7 +477,10 @@ struct request *blk_mq_alloc_request_hctx(struct request_queue *q,
 	data.hctx = q->queue_hw_ctx[hctx_idx];
 	if (!blk_mq_hw_queue_mapped(data.hctx))
 		goto out_queue_exit;
-	cpu = cpumask_first_and(data.hctx->cpumask, cpu_online_mask);
+
+	WARN_ON_ONCE(blk_mq_hctx_use_managed_irq(data.hctx));
+
+	cpu = blk_mq_first_mapped_cpu(data.hctx);
 	data.ctx = __blk_mq_get_ctx(q, cpu);
 
 	if (!q->elevator)
@@ -1501,15 +1513,6 @@ static void __blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx)
 	hctx_unlock(hctx, srcu_idx);
 }
 
-static inline int blk_mq_first_mapped_cpu(struct blk_mq_hw_ctx *hctx)
-{
-	int cpu = cpumask_first_and(hctx->cpumask, cpu_online_mask);
-
-	if (cpu >= nr_cpu_ids)
-		cpu = cpumask_first(hctx->cpumask);
-	return cpu;
-}
-
 /*
  * It'd be great if the workqueue API had a way to pass
  * in a mask and had some smarts for more clever placement.
@@ -2556,6 +2559,10 @@ static int blk_mq_hctx_notify_offline(unsigned int cpu, struct hlist_node *node)
 	struct blk_mq_hw_ctx *hctx = hlist_entry_safe(node,
 			struct blk_mq_hw_ctx, cpuhp_online);
 
+	/* hctx needn't to be deactivated in case managed irq isn't used */
+	if (!blk_mq_hctx_use_managed_irq(hctx))
+		return 0;
+
 	if (!cpumask_test_cpu(cpu, hctx->cpumask) ||
 	    !blk_mq_last_cpu_in_hctx(cpu, hctx))
 		return 0;
block/blk-mq.h +8 -0
index d08779f77a26..7333b659d8f5 100644
@@ -119,6 +119,14 @@ static inline struct blk_mq_hw_ctx *blk_mq_map_queue(struct request_queue *q,
 	return ctx->hctxs[type];
 }
 
+static inline bool blk_mq_hctx_use_managed_irq(struct blk_mq_hw_ctx *hctx)
+{
+	if (hctx->type == HCTX_TYPE_POLL)
+		return false;
+
+	return hctx->queue->tag_set->map[hctx->type].use_managed_irq;
+}
+
 /*
  * sysfs helpers
  */
Signature · 1 line
2.31.1