Skip to contentSkip to conversation

blk-mq: fix blk_mq_alloc_request_hctx

PATCH Superseded Ming Lei 4 participants 14 messages Jul 22, 2021 – Aug 18, 2021 2 messages not in the archive

Subsystems (7)
SCSI Block layer NVMe IRQ +3 more subsystems

v6 of 7 Superseded Latest is v7 →

lore
Share & tools
lore

Ming Lei

author
#0 lore raw

v6 cover blk-mq: fix blk_mq_alloc_request_hctx

Hi,

blk_mq_alloc_request_hctx() is used by NVMe fc/rdma/tcp/loop to connect io queue. Also the sw ctx is chosen as the 1st online cpu in hctx->cpumask. However, all cpus in hctx->cpumask may be offline.

This usage model isn't well supported by blk-mq which supposes allocator is always done on one online CPU in hctx->cpumask. This assumption is related with managed irq, which also requires blk-mq to drain inflight request in this hctx when the last cpu in hctx->cpumask is going to offline.

However, NVMe fc/rdma/tcp/loop don't use managed irq, so we should allow them to ask for request allocation when the specified hctx is inactive (all cpus in hctx->cpumask are offline). Fix blk_mq_alloc_request_hctx() by allowing to allocate request when all CPUs of this hctx are offline.

Wen Xiong has verified V4 in her nvmef test.

V6:
- move device_has_managed_msi_irq() into kernel/irq/msi.c

V5:
- take John Garry's suggestion to replace device field with new helper of device_has_managed_msi_irq()
V4:
- remove patches for cleanup queue map helpers
- take Christoph's suggestion to add field into 'struct device' for describing if managed irq is allocated from one device

V3:
- cleanup map queues helpers, and remove pci/virtio/rdma queue helpers
- store use managed irq info into qmap

V2:
- use flag of BLK_MQ_F_MANAGED_IRQ
- pass BLK_MQ_F_MANAGED_IRQ from driver explicitly
- kill BLK_MQ_F_STACKING

Ming Lei (3):
genirq: add device_has_managed_msi_irq
blk-mq: mark if one queue map uses managed irq
blk-mq: don't deactivate hctx if managed irq isn't used

 block/blk-mq-pci.c                     |  2 ++
 block/blk-mq-rdma.c                    |  7 +++++++
 block/blk-mq-virtio.c                  |  2 ++
 block/blk-mq.c                         | 27 ++++++++++++++++----------
 block/blk-mq.h                         |  8 ++++++++
 drivers/scsi/hisi_sas/hisi_sas_v2_hw.c |  1 +
 include/linux/blk-mq.h                 |  3 ++-
 include/linux/msi.h                    |  5 +++++
 kernel/irq/msi.c                       | 18 +++++++++++++++++
 9 files changed, 62 insertions(+), 11 deletions(-)
Signature · 1 line
2.31.1
author
#1 lore raw

PATCH v6 1/3 genirq: add device_has_managed_msi_irq

irq vector allocation with managed affinity may be used by driver, and blk-mq needs this info for draining queue because genirq core will shutdown managed irq when all CPUs in the affinity mask are offline.

The info of using managed irq is often produced by drivers, and it is consumed by blk-mq, so different subsystems are involved in this info flow.

Address this issue by adding one helper of device_has_managed_msi_irq() which is suggested by John Garry.

---
 include/linux/msi.h |  5 +++++
 kernel/irq/msi.c    | 18 ++++++++++++++++++
 2 files changed, 23 insertions(+)
include/linux/msi.h +5 -0
index 6aff469e511d..2d0f3962d3c3 100644
@@ -59,10 +59,15 @@ struct platform_msi_priv_data;
 void __get_cached_msi_msg(struct msi_desc *entry, struct msi_msg *msg);
 #ifdef CONFIG_GENERIC_MSI_IRQ
 void get_cached_msi_msg(unsigned int irq, struct msi_msg *msg);
+bool device_has_managed_msi_irq(struct device *dev);
 #else
 static inline void get_cached_msi_msg(unsigned int irq, struct msi_msg *msg)
 {
 }
+static inline bool device_has_managed_msi_irq(struct device *dev)
+{
+	return false;
+}
 #endif
 
 typedef void (*irq_write_msi_msg_t)(struct msi_desc *desc,
kernel/irq/msi.c +18 -0
index c41965e348b5..01aae22246c8 100644
@@ -69,6 +69,24 @@ void get_cached_msi_msg(unsigned int irq, struct msi_msg *msg)
 }
 EXPORT_SYMBOL_GPL(get_cached_msi_msg);
 
+/**
+ * device_has_managed_msi_irq - Query if device has managed irq entry
+ * @dev:	Pointer to the device for which we want to query
+ *
+ * Return true if there is managed irq vector allocated on this device
+ */
+bool device_has_managed_msi_irq(struct device *dev)
+{
+	struct msi_desc *desc;
+
+	for_each_msi_entry(desc, dev) {
+		if (desc->affinity && desc->affinity->is_managed)
+			return true;
+	}
+	return false;
+}
+EXPORT_SYMBOL_GPL(device_has_managed_msi_irq);
+
 #ifdef CONFIG_GENERIC_MSI_IRQ_DOMAIN
 static inline void irq_chip_write_msi_msg(struct irq_data *data,
 					  struct msi_msg *msg)
Signature · 1 line
2.31.1
maintainer Reviewed-by on v6 1/3
#2 lore raw

Re: [PATCH V6 1/3] genirq: add device_has_managed_msi_irq

Quoted 9 lines · Ming Lei

irq vector allocation with managed affinity may be used by driver, and blk-mq needs this info for draining queue because genirq core will shutdown managed irq when all CPUs in the affinity mask are offline.

The info of using managed irq is often produced by drivers, and it is consumed by blk-mq, so different subsystems are involved in this info flow.

Address this issue by adding one helper of device_has_managed_msi_irq() which is suggested by John Garry.

This implies non-MSI irqs can't be managed, which is true right now. If the irq maintaines are fine with that:

author
#3 lore raw

PATCH v6 2/3 blk-mq: mark if one queue map uses managed irq

In reply to Ming Lei

Retrieve this info via new added helper of device_has_managed_msi_irq, then we can decide if one hctx needs to be drained before all its CPUs become offline.

---
 block/blk-mq-pci.c                     | 2 ++
 block/blk-mq-rdma.c                    | 7 +++++++
 block/blk-mq-virtio.c                  | 2 ++
 drivers/scsi/hisi_sas/hisi_sas_v2_hw.c | 1 +
 include/linux/blk-mq.h                 | 3 ++-
 5 files changed, 14 insertions(+), 1 deletion(-)
block/blk-mq-pci.c +2 -0
index b595a94c4d16..e452cda0896a 100644
@@ -8,6 +8,7 @@
 #include <linux/blk-mq-pci.h>
 #include <linux/pci.h>
 #include <linux/module.h>
+#include <linux/msi.h>
 
 #include "blk-mq.h"
 
@@ -37,6 +38,7 @@ int blk_mq_pci_map_queues(struct blk_mq_queue_map *qmap, struct pci_dev *pdev,
 		for_each_cpu(cpu, mask)
 			qmap->mq_map[cpu] = qmap->queue_offset + queue;
 	}
+	qmap->use_managed_irq = device_has_managed_msi_irq(&pdev->dev);
 
 	return 0;
 
block/blk-mq-rdma.c +7 -0
index 14f968e58b8f..19ad31c44eab 100644
@@ -36,6 +36,13 @@ int blk_mq_rdma_map_queues(struct blk_mq_queue_map *map,
 			map->mq_map[cpu] = map->queue_offset + queue;
 	}
 
+	/*
+	 * RDMA doesn't use managed irq, and nvme rdma driver can allocate
+	 * and submit requests on specified hctx via
+	 * blk_mq_alloc_request_hctx
+	 */
+	map->use_managed_irq = false;
+
 	return 0;
 
 fallback:
block/blk-mq-virtio.c +2 -0
index 7b8a42c35102..2ce39fb77dce 100644
@@ -7,6 +7,7 @@
 #include <linux/blk-mq-virtio.h>
 #include <linux/virtio_config.h>
 #include <linux/module.h>
+#include <linux/msi.h>
 #include "blk-mq.h"
 
 /**
@@ -38,6 +39,7 @@ int blk_mq_virtio_map_queues(struct blk_mq_queue_map *qmap,
 		for_each_cpu(cpu, mask)
 			qmap->mq_map[cpu] = qmap->queue_offset + queue;
 	}
+	qmap->use_managed_irq = device_has_managed_msi_irq(&vdev->dev);
 
 	return 0;
 fallback:
drivers/scsi/hisi_sas/hisi_sas_v2_hw.c +1 -0
index b0b2361e63fe..7d7df261d346 100644
@@ -3562,6 +3562,7 @@ static int map_queues_v2_hw(struct Scsi_Host *shost)
 		for_each_cpu(cpu, mask)
 			qmap->mq_map[cpu] = qmap->queue_offset + queue;
 	}
+	qmap->use_managed_irq = true;
 
 	return 0;
 
include/linux/blk-mq.h +2 -1
index 1d18447ebebc..d54a795ec971 100644
@@ -192,7 +192,8 @@ struct blk_mq_hw_ctx {
 struct blk_mq_queue_map {
 	unsigned int *mq_map;
 	unsigned int nr_queues;
-	unsigned int queue_offset;
+	unsigned int queue_offset:31;
+	unsigned int use_managed_irq:1;
 };
 
 /**
Signature · 1 line
2.31.1
#4 lore raw

Re: [PATCH V6 2/3] blk-mq: mark if one queue map uses managed irq

On Thu, Jul 22, 2021 at 05:52:45PM +0800, Ming Lei wrote:
and nvme rdma driver can allocate

block/blk-mq-rdma.c int blk_mq_rdma_map_queues(struct blk_mq_queue_map *map, · line 42

Quoted 2 lines
block/blk-mq-rdma.c +2 -0
+	 * and submit requests on specified hctx via
+	 * blk_mq_alloc_request_hctx

Why does that matter for this setting?

author
#5 lore raw

Re: [PATCH V6 2/3] blk-mq: mark if one queue map uses managed irq

Quoted 6 lines · Christoph Hellwig

On Thu, Jul 22, 2021 at 05:52:45PM +0800, Ming Lei wrote:
and nvme rdma driver can allocate

block/blk-mq-rdma.c int blk_mq_rdma_map_queues(struct blk_mq_queue_map *map, · line 42

Quoted 2 lines
block/blk-mq-rdma.c +2 -0
+	 * and submit requests on specified hctx via
+	 * blk_mq_alloc_request_hctx

Why does that matter for this setting?

blk_mq_alloc_request_hctx() has been broken for long time, which can only work if the hctx isn't driven by non-managed irq.

Thanks,
Ming

#6 lore raw

Re: [PATCH V6 2/3] blk-mq: mark if one queue map uses managed irq

Quoted 10 lines · Ming Lei
Quoted 6 lines · Christoph Hellwig

On Thu, Jul 22, 2021 at 05:52:45PM +0800, Ming Lei wrote:
and nvme rdma driver can allocate

block/blk-mq-rdma.c int blk_mq_rdma_map_queues(struct blk_mq_queue_map *map, · line 42

Quoted 2 lines
block/blk-mq-rdma.c +2 -0
+	 * and submit requests on specified hctx via
+	 * blk_mq_alloc_request_hctx

Why does that matter for this setting?

blk_mq_alloc_request_hctx() has been broken for long time, which can only work if the hctx isn't driven by non-managed irq.

The qla2xxx and lpfc drivers uses managed interrupt. I'm not quite sure if the nvme-fc queues directly map to those interrupts, though.

author
#7 lore raw

PATCH v6 3/3 blk-mq: don't deactivate hctx if managed irq isn't used

In reply to Ming Lei

blk-mq deactivates one hctx when the last CPU in hctx->cpumask become offline by draining all requests originated from this hctx and moving new allocation to other active hctx. This way is for avoiding inflight IO in case of managed irq because managed irq is shutdown when the last CPU in the irq's affinity becomes offline.

However, lots of drivers(nvme fc, rdma, tcp, loop, ...) don't use managed irq, so they needn't to deactivate hctx when the last CPU becomes offline. Also, some of them are the only user of blk_mq_alloc_request_hctx() which is used for connecting io queue. And their requirement is that the connect request needs to be submitted successfully via one specified hctx even though all CPUs in this hctx->cpumask have become offline.

Addressing the requirement for nvme fc/rdma/loop by allowing to allocate request from one hctx when all CPUs in this hctx are offline, since these drivers don't use managed irq.

Finally don't deactivate one hctx when it doesn't use managed irq.

---
 block/blk-mq.c | 27 +++++++++++++++++----------
 block/blk-mq.h |  8 ++++++++
 2 files changed, 25 insertions(+), 10 deletions(-)
block/blk-mq.c +17 -10
index 2c4ac51e54eb..591ab07c64d8 100644
@@ -427,6 +427,15 @@ struct request *blk_mq_alloc_request(struct request_queue *q, unsigned int op,
 }
 EXPORT_SYMBOL(blk_mq_alloc_request);
 
+static inline int blk_mq_first_mapped_cpu(struct blk_mq_hw_ctx *hctx)
+{
+	int cpu = cpumask_first_and(hctx->cpumask, cpu_online_mask);
+
+	if (cpu >= nr_cpu_ids)
+		cpu = cpumask_first(hctx->cpumask);
+	return cpu;
+}
+
 struct request *blk_mq_alloc_request_hctx(struct request_queue *q,
 	unsigned int op, blk_mq_req_flags_t flags, unsigned int hctx_idx)
 {
@@ -468,7 +477,10 @@ struct request *blk_mq_alloc_request_hctx(struct request_queue *q,
 	data.hctx = q->queue_hw_ctx[hctx_idx];
 	if (!blk_mq_hw_queue_mapped(data.hctx))
 		goto out_queue_exit;
-	cpu = cpumask_first_and(data.hctx->cpumask, cpu_online_mask);
+
+	WARN_ON_ONCE(blk_mq_hctx_use_managed_irq(data.hctx));
+
+	cpu = blk_mq_first_mapped_cpu(data.hctx);
 	data.ctx = __blk_mq_get_ctx(q, cpu);
 
 	if (!q->elevator)
@@ -1501,15 +1513,6 @@ static void __blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx)
 	hctx_unlock(hctx, srcu_idx);
 }
 
-static inline int blk_mq_first_mapped_cpu(struct blk_mq_hw_ctx *hctx)
-{
-	int cpu = cpumask_first_and(hctx->cpumask, cpu_online_mask);
-
-	if (cpu >= nr_cpu_ids)
-		cpu = cpumask_first(hctx->cpumask);
-	return cpu;
-}
-
 /*
  * It'd be great if the workqueue API had a way to pass
  * in a mask and had some smarts for more clever placement.
@@ -2556,6 +2559,10 @@ static int blk_mq_hctx_notify_offline(unsigned int cpu, struct hlist_node *node)
 	struct blk_mq_hw_ctx *hctx = hlist_entry_safe(node,
 			struct blk_mq_hw_ctx, cpuhp_online);
 
+	/* hctx needn't to be deactivated in case managed irq isn't used */
+	if (!blk_mq_hctx_use_managed_irq(hctx))
+		return 0;
+
 	if (!cpumask_test_cpu(cpu, hctx->cpumask) ||
 	    !blk_mq_last_cpu_in_hctx(cpu, hctx))
 		return 0;
block/blk-mq.h +8 -0
index d08779f77a26..7333b659d8f5 100644
@@ -119,6 +119,14 @@ static inline struct blk_mq_hw_ctx *blk_mq_map_queue(struct request_queue *q,
 	return ctx->hctxs[type];
 }
 
+static inline bool blk_mq_hctx_use_managed_irq(struct blk_mq_hw_ctx *hctx)
+{
+	if (hctx->type == HCTX_TYPE_POLL)
+		return false;
+
+	return hctx->queue->tag_set->map[hctx->type].use_managed_irq;
+}
+
 /*
  * sysfs helpers
  */
Signature · 1 line
2.31.1
Reviewed-by on v6 3/3
#8 lore raw

Re: [PATCH V6 3/3] blk-mq: don't deactivate hctx if managed irq isn't used

Quoted 13 lines · Ming Lei

blk-mq deactivates one hctx when the last CPU in hctx->cpumask become offline by draining all requests originated from this hctx and moving new allocation to other active hctx. This way is for avoiding inflight IO in case of managed irq because managed irq is shutdown when the last CPU in the irq's affinity becomes offline.

However, lots of drivers(nvme fc, rdma, tcp, loop, ...) don't use managed irq, so they needn't to deactivate hctx when the last CPU becomes offline. Also, some of them are the only user of blk_mq_alloc_request_hctx() which is used for connecting io queue. And their requirement is that the connect request needs to be submitted successfully via one specified hctx even though all CPUs in this hctx->cpumask have become offline.

Quoted 8 lines

Addressing the requirement for nvme fc/rdma/loop by allowing to allocate request from one hctx when all CPUs in this hctx are offline, since these drivers don't use managed irq.

Finally don't deactivate one hctx when it doesn't use managed irq.

Sorry for lateness. Just a few minor comments below, regardless:

Quoted 10 lines
---
  block/blk-mq.c | 27 +++++++++++++++++----------
  block/blk-mq.h |  8 ++++++++
  2 files changed, 25 insertions(+), 10 deletions(-)
block/blk-mq.c +0 -0
index 2c4ac51e54eb..591ab07c64d8 100644

block/blk-mq.c struct request *blk_mq_alloc_request(struct request_queue *q, unsigned int op, · line 430

Quoted 4 lines
block/blk-mq.c +1 -0
@@ -427,6 +427,15 @@ struct request *blk_mq_alloc_request(struct request_queue *q, unsigned int op,
  }
  EXPORT_SYMBOL(blk_mq_alloc_request);
 
+static inline int blk_mq_first_mapped_cpu(struct blk_mq_hw_ctx *hctx)

I don't see why it needs to be inline. I think generally we leave it to the compiler to decide.

Quoted 40 lines
block/blk-mq.c +12 -10
+{
+	int cpu = cpumask_first_and(hctx->cpumask, cpu_online_mask);
+
+	if (cpu >= nr_cpu_ids)
+		cpu = cpumask_first(hctx->cpumask);
+	return cpu;
+}
+
  struct request *blk_mq_alloc_request_hctx(struct request_queue *q,
  	unsigned int op, blk_mq_req_flags_t flags, unsigned int hctx_idx)
  {
@@ -468,7 +477,10 @@ struct request *blk_mq_alloc_request_hctx(struct request_queue *q,
  	data.hctx = q->queue_hw_ctx[hctx_idx];
  	if (!blk_mq_hw_queue_mapped(data.hctx))
  		goto out_queue_exit;
-	cpu = cpumask_first_and(data.hctx->cpumask, cpu_online_mask);
+
+	WARN_ON_ONCE(blk_mq_hctx_use_managed_irq(data.hctx));
+
+	cpu = blk_mq_first_mapped_cpu(data.hctx);
  	data.ctx = __blk_mq_get_ctx(q, cpu);
 
  	if (!q->elevator)
@@ -1501,15 +1513,6 @@ static void __blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx)
  	hctx_unlock(hctx, srcu_idx);
  }
 
-static inline int blk_mq_first_mapped_cpu(struct blk_mq_hw_ctx *hctx)
-{
-	int cpu = cpumask_first_and(hctx->cpumask, cpu_online_mask);
-
-	if (cpu >= nr_cpu_ids)
-		cpu = cpumask_first(hctx->cpumask);
-	return cpu;
-}
-
  /*
   * It'd be great if the workqueue API had a way to pass
   * in a mask and had some smarts for more clever placement.

block/blk-mq.c static int blk_mq_hctx_notify_offline(unsigned int cpu, struct hlist_node *node) · line 2564

Quoted 6 lines
block/blk-mq.c +3 -0
@@ -2556,6 +2559,10 @@ static int blk_mq_hctx_notify_offline(unsigned int cpu, struct hlist_node *node)
  	struct blk_mq_hw_ctx *hctx = hlist_entry_safe(node,
  			struct blk_mq_hw_ctx, cpuhp_online);
 
+	/* hctx needn't to be deactivated in case managed irq isn't used */
+	if (!blk_mq_hctx_use_managed_irq(hctx))
+		return 0;

In a previous version (v2) I think that we just didn't register the cpu hotplug handlers, so not sure why that changed.

Quoted 9 lines
block/blk-mq.c +1 -0
+
  	if (!cpumask_test_cpu(cpu, hctx->cpumask) ||
  	    !blk_mq_last_cpu_in_hctx(cpu, hctx))
  		return 0;
block/blk-mq.h +0 -0
index d08779f77a26..7333b659d8f5 100644

block/blk-mq.h static inline struct blk_mq_hw_ctx *blk_mq_map_queue(struct request_queue *q, · line 128

Quoted 10 lines
block/blk-mq.h +7 -0
@@ -119,6 +119,14 @@ static inline struct blk_mq_hw_ctx *blk_mq_map_queue(struct request_queue *q,
  	return ctx->hctxs[type];
  }
 
+static inline bool blk_mq_hctx_use_managed_irq(struct blk_mq_hw_ctx *hctx)
+{
+	if (hctx->type == HCTX_TYPE_POLL)
+		return false;
+
+	return hctx->queue->tag_set->map[hctx->type].use_managed_irq;
+}

This function only seems to be used in blk-mq.c, so not sure why you put it in .h file.

author
#9 lore raw

Re: [PATCH V6 3/3] blk-mq: don't deactivate hctx if managed irq isn't used

Quoted 32 lines · John Garry
Quoted 13 lines · Ming Lei

blk-mq deactivates one hctx when the last CPU in hctx->cpumask become offline by draining all requests originated from this hctx and moving new allocation to other active hctx. This way is for avoiding inflight IO in case of managed irq because managed irq is shutdown when the last CPU in the irq's affinity becomes offline.

However, lots of drivers(nvme fc, rdma, tcp, loop, ...) don't use managed irq, so they needn't to deactivate hctx when the last CPU becomes offline. Also, some of them are the only user of blk_mq_alloc_request_hctx() which is used for connecting io queue. And their requirement is that the connect request needs to be submitted successfully via one specified hctx even though all CPUs in this hctx->cpumask have become offline.

Quoted 8 lines

Addressing the requirement for nvme fc/rdma/loop by allowing to allocate request from one hctx when all CPUs in this hctx are offline, since these drivers don't use managed irq.

Finally don't deactivate one hctx when it doesn't use managed irq.

Sorry for lateness. Just a few minor comments below, regardless:

Quoted 5 lines
---
  block/blk-mq.c | 27 +++++++++++++++++----------
  block/blk-mq.h |  8 ++++++++
  2 files changed, 25 insertions(+), 10 deletions(-)
Quoted 11 lines

block/blk-mq.c struct request *blk_mq_alloc_request(struct request_queue *q, unsigned int op, · line 429

Quoted 8 lines
block/blk-mq.c +1 -0
index 2c4ac51e54eb..591ab07c64d8 100644
@@ -427,6 +427,15 @@ struct request *blk_mq_alloc_request(struct request_queue *q, unsigned int op,
  }
  EXPORT_SYMBOL(blk_mq_alloc_request);
+static inline int blk_mq_first_mapped_cpu(struct blk_mq_hw_ctx *hctx)

I don't see why it needs to be inline. I think generally we leave it to the compiler to decide.

Usually if one function is called more than two times, it will not be inlined. blk_mq_first_mapped_cpu() is called in fast path, so we need to inline it.

Quoted 39 lines
Quoted 39 lines
block/blk-mq.c +12 -10
+{
+	int cpu = cpumask_first_and(hctx->cpumask, cpu_online_mask);
+
+	if (cpu >= nr_cpu_ids)
+		cpu = cpumask_first(hctx->cpumask);
+	return cpu;
+}
+
  struct request *blk_mq_alloc_request_hctx(struct request_queue *q,
  	unsigned int op, blk_mq_req_flags_t flags, unsigned int hctx_idx)
  {
@@ -468,7 +477,10 @@ struct request *blk_mq_alloc_request_hctx(struct request_queue *q,
  	data.hctx = q->queue_hw_ctx[hctx_idx];
  	if (!blk_mq_hw_queue_mapped(data.hctx))
  		goto out_queue_exit;
-	cpu = cpumask_first_and(data.hctx->cpumask, cpu_online_mask);
+
+	WARN_ON_ONCE(blk_mq_hctx_use_managed_irq(data.hctx));
+
+	cpu = blk_mq_first_mapped_cpu(data.hctx);
  	data.ctx = __blk_mq_get_ctx(q, cpu);
  	if (!q->elevator)
@@ -1501,15 +1513,6 @@ static void __blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx)
  	hctx_unlock(hctx, srcu_idx);
  }
-static inline int blk_mq_first_mapped_cpu(struct blk_mq_hw_ctx *hctx)
-{
-	int cpu = cpumask_first_and(hctx->cpumask, cpu_online_mask);
-
-	if (cpu >= nr_cpu_ids)
-		cpu = cpumask_first(hctx->cpumask);
-	return cpu;
-}
-
  /*
   * It'd be great if the workqueue API had a way to pass
   * in a mask and had some smarts for more clever placement.
@@ -2556,6 +2559,10 @@ static int blk_mq_hctx_notify_offline(unsigned int cpu, struct hlist_node *node)
Quoted 8 lines

block/blk-mq.c static int blk_mq_hctx_notify_offline(unsigned int cpu, struct hlist_node *node) · line 2564

Quoted 5 lines
block/blk-mq.c +3 -0
  	struct blk_mq_hw_ctx *hctx = hlist_entry_safe(node,
  			struct blk_mq_hw_ctx, cpuhp_online);
+	/* hctx needn't to be deactivated in case managed irq isn't used */
+	if (!blk_mq_hctx_use_managed_irq(hctx))
+		return 0;

In a previous version (v2) I think that we just didn't register the cpu hotplug handlers, so not sure why that changed.

Sorry, forget to mention the change, inside blk_mq_realloc_hw_ctxs() we may grab one cached hctx without initializing it, so use managed irq info change may not be visible in blk_mq_init_hctx().

Given blk_mq_hctx_notify_offline() isn't in fast path, I'd rather to check in blk_mq_hctx_notify_offline(), which is more reliable than running the check in blk_mq_init_hctx().

Quoted 14 lines
Quoted 14 lines
block/blk-mq.c +1 -0
+
  	if (!cpumask_test_cpu(cpu, hctx->cpumask) ||
  	    !blk_mq_last_cpu_in_hctx(cpu, hctx))
  		return 0;
block/blk-mq.h +2 -0
index d08779f77a26..7333b659d8f5 100644
@@ -119,6 +119,14 @@ static inline struct blk_mq_hw_ctx *blk_mq_map_queue(struct request_queue *q,
  	return ctx->hctxs[type];
  }
+static inline bool blk_mq_hctx_use_managed_irq(struct blk_mq_hw_ctx *hctx)
+{
Quoted 8 lines

block/blk-mq.h static inline struct blk_mq_hw_ctx *blk_mq_map_queue(struct request_queue *q, · line 128

Quoted 5 lines
block/blk-mq.h +5 -0
+	if (hctx->type == HCTX_TYPE_POLL)
+		return false;
+
+	return hctx->queue->tag_set->map[hctx->type].use_managed_irq;
+}

This function only seems to be used in blk-mq.c, so not sure why you put it in .h file.

oops, the helper must be used in early version, I will send a new version soon.

Thanks,
Ming

#10 lore raw
In reply to Ming Lei
Quoted 1 line · Ming Lei

Wen Xiong has verified V4 in her nvmef test.

FWIW, I am testing this series right now. I observe hanging I/Os again, but I am not totally sure if my test setup is working properly. Working on it. I'll keep you posted.

A message that is not in the archive (sent off-list or not yet received): OFDADF39F5.DDB99A55-ON0025871A.00794382-0025871A.00797A2E@ibm.com

author
#12 lore raw
Quoted 9 lines · Wen Xiong
Quoted 5 lines

Wen Xiong has verified V4 in her nvmef test.

Quoted 3 lines

FWIW, I am testing this series right now. I observe hanging I/Os again, but I am not totally sure if my test setup is working properly. Working on it. I'll keep you posted.

I built the latest upstream(v5.14-rc2) + Ming's V6 + Daniel's V3: didn't work.

Hi Wenxiong,

V6 is basically same with V4, can you figure out where the failure comes?(v5.14-rc2, V6 or Daniel's V3)

I verified that V6 works as expected by tracing blk_mq_update_queue_map():

#!/usr/bin/bpftrace
#include <linux/blk-mq.h>
kprobe:blk_mq_update_queue_map

{
	@in_map[tid]=1;
	@set[tid] = (struct blk_mq_tag_set *)arg0;
	@ms[tid] = kstack;
}

kretprobe:blk_mq_update_queue_map
/@in_map[tid]==1/

{
	$s = @set[tid];
	$m = (struct blk_mq_queue_map *)$s->map;

	printf("%s %d: %s queues %d use_managed_irq %d\n",
		comm, pid, @ms[tid], $s->nr_hw_queues, $m->use_managed_irq);
}

Thanks,
Ming

A message that is not in the archive (sent off-list or not yet received): OF2C4681CD.AF20CBB4-ON0025871E.004D37B6-0025871E.004E3B9F@ibm.com

Tested-by
#14 lore raw

Hi,

Quoted 5 lines · Wen Xiong
Quoted 2 lines
Quoted 2 lines

V6 is basically same with V4, can you figure out where the failure comes?(v5.14-rc2, V6 or Daniel's V3)

Looks 3/3 was not patched cleanly in v5.14-rc2 last week. I made the changes in block/blk-mq.c.rej manually but still missed the last part of 3/3 patch.

Sorry for the long delay on my side. It took a while to get my test setup running again. The qla2xxx driver really doesn't like 'fast' remote port toggling. But that's a different story.

Anyway, it turns out that my patch series is still not working correctly. When I tested the series I deliberate forced to execute the 'revising io queue count' path in nvme_fc_recreate_io_queues by doing:

drivers/nvme/host/fc.c +1 -1
@@ -2954,7 +2954,7 @@ nvme_fc_recreate_io_queues(struct nvme_fc_ctrl *ctrl)
        if (ctrl->ctrl.queue_count == 1)
                return 0;
 
-       if (prior_ioq_cnt != nr_io_queues) {
+       if (prior_ioq_cnt != nr_io_queues - 1) {
                dev_info(ctrl->ctrl.device,
                        "reconnect: revising io queue count from %d to %d\n",
                        prior_ioq_cnt, nr_io_queues);

With this change I can't observe any I/O hanging. Without the change it hangs in the first iteration.

In Wen's setup we observed in earlier debugging sessions that the nr_io_queues does change. So this explains why Wen doesn't see any hanging I/Os.

@James, I think we need to look at bit more at the freeze code. BTW, my initial patch which just added a nvme_start_freeze() in nvme_fc_delete_association() doesn't work either for the 'prior_ioq_cnt == nr_io_queues' case.

So I think Ming series can be merged as the hanging I/Os are clearly not caused by the series.

Feel free to add

Thanks,
Daniel