Add BLK_MQ_F_NO_SCHED flag to admin tag set

PATCH New Israel Rukshin 5 participants 14 messages Oct 18, 2017 – Oct 19, 2017

Subsystems (3)

NVMe NVMe target NVMe FC transport

lore

Share & tools

lore

Israel Rukshin

author

2017-10-18 12:38 UTC #0 lore raw

cover Add BLK_MQ_F_NO_SCHED flag to admin tag set

Since commit b86dd81
"block: get rid of blk-mq default scheduler choice Kconfig entries", when setting nr_hw_queues to 1 the admin tag set uses mq-deadline scheduler. This flag is useful for admin queues that aren't used for normal IO. We already use this flag at nvme-pci.

Israel Rukshin (3):
nvme-rdma: Add BLK_MQ_F_NO_SCHED flag to admin tag set
nvme-fc: Add BLK_MQ_F_NO_SCHED flag to admin tag set
nvme-loop: Add BLK_MQ_F_NO_SCHED flag to admin tag set

 drivers/nvme/host/fc.c     | 1 +
 drivers/nvme/host/rdma.c   | 1 +
 drivers/nvme/target/loop.c | 1 +
 3 files changed, 3 insertions(+)

Signature · 1 line

1.8.3.1

Israel Rukshin

author

2017-10-18 12:38 UTC #1 lore raw

PATCH 1/3 nvme-rdma: Add BLK_MQ_F_NO_SCHED flag to admin tag set

Signed-off-by Israel Rukshin <israelr at mellanox.com>

---
 drivers/nvme/host/rdma.c | 1 +
 1 file changed, 1 insertion(+)

drivers/nvme/host/rdma.c +1 -0

index 92a03ff..ee61b98f 100644
@@ -699,6 +699,7 @@ static struct blk_mq_tag_set *nvme_rdma_alloc_tagset(struct nvme_ctrl *nctrl,
 		set->driver_data = ctrl;
 		set->nr_hw_queues = 1;
 		set->timeout = ADMIN_TIMEOUT;
+		set->flags = BLK_MQ_F_NO_SCHED;
 	} else {
 		set = &ctrl->tag_set;
 		memset(set, 0, sizeof(*set));

Signature · 1 line

1.8.3.1

Sagi Grimberg

maintainer

2017-10-18 15:30 UTC #2 lore raw

PATCH 1/3 nvme-rdma: Add BLK_MQ_F_NO_SCHED flag to admin tag set

Reviewed-by Sagi Grimberg <sagi at grimberg.me>

Israel Rukshin

author

2017-10-18 12:38 UTC #3 lore raw

PATCH 2/3 nvme-fc: Add BLK_MQ_F_NO_SCHED flag to admin tag set

In reply to Israel Rukshin

Signed-off-by Israel Rukshin <israelr at mellanox.com>

---
 drivers/nvme/host/fc.c | 1 +
 1 file changed, 1 insertion(+)

drivers/nvme/host/fc.c +1 -0

index af075e9..b383735 100644
@@ -2796,6 +2796,7 @@ enum {
 	ctrl->admin_tag_set.driver_data = ctrl;
 	ctrl->admin_tag_set.nr_hw_queues = 1;
 	ctrl->admin_tag_set.timeout = ADMIN_TIMEOUT;
+	ctrl->admin_tag_set.flags = BLK_MQ_F_NO_SCHED;
 
 	ret = blk_mq_alloc_tag_set(&ctrl->admin_tag_set);
 	if (ret)

Signature · 1 line

1.8.3.1

Sagi Grimberg

maintainer

2017-10-18 15:30 UTC #4 lore raw

PATCH 2/3 nvme-fc: Add BLK_MQ_F_NO_SCHED flag to admin tag set

Reviewed-by Sagi Grimberg <sagi at grimberg.me>

James Smart

2017-10-18 21:41 UTC #5 lore raw

PATCH 2/3 nvme-fc: Add BLK_MQ_F_NO_SCHED flag to admin tag set

In reply to Israel Rukshin

Quoted 15 lines · Israel Rukshin

Since commit b86dd81
"block: get rid of blk-mq default scheduler choice Kconfig entries", when setting nr_hw_queues to 1 the admin tag set uses mq-deadline scheduler. This flag is useful for admin queues that aren't used for normal IO.
Signed-off-by Israel Rukshin <israelr at mellanox.com>
---
  drivers/nvme/host/fc.c | 1 +
  1 file changed, 1 insertion(+)
drivers/nvme/host/fc.c +0 -0
index af075e9..b383735 100644

drivers/nvme/host/fc.c enum { · line 2802

Quoted 7 lines

drivers/nvme/host/fc.c +1 -0

@@ -2796,6 +2796,7 @@ enum {
  	ctrl->admin_tag_set.driver_data = ctrl;
  	ctrl->admin_tag_set.nr_hw_queues = 1;
  	ctrl->admin_tag_set.timeout = ADMIN_TIMEOUT;
+	ctrl->admin_tag_set.flags = BLK_MQ_F_NO_SCHED;
 
  	ret = blk_mq_alloc_tag_set(&ctrl->admin_tag_set);
  	if (ret)

Looks fine

Signed-off-by James Smart? <james.smart at broadcom.com>

Israel Rukshin

author

2017-10-18 12:38 UTC #6 lore raw

PATCH 3/3 nvme-loop: Add BLK_MQ_F_NO_SCHED flag to admin tag set

In reply to Israel Rukshin

This flag is useful for admin queues that aren't used for normal IO.

Signed-off-by Israel Rukshin <israelr at mellanox.com>

---
 drivers/nvme/target/loop.c | 1 +
 1 file changed, 1 insertion(+)

drivers/nvme/target/loop.c +1 -0

index 92628c4..c56354e 100644
@@ -365,6 +365,7 @@ static int nvme_loop_configure_admin_queue(struct nvme_loop_ctrl *ctrl)
 	ctrl->admin_tag_set.driver_data = ctrl;
 	ctrl->admin_tag_set.nr_hw_queues = 1;
 	ctrl->admin_tag_set.timeout = ADMIN_TIMEOUT;
+	ctrl->admin_tag_set.flags = BLK_MQ_F_NO_SCHED;
 
 	ctrl->queues[0].ctrl = ctrl;
 	error = nvmet_sq_init(&ctrl->queues[0].nvme_sq);

Signature · 1 line

1.8.3.1

Sagi Grimberg

maintainer

2017-10-18 15:30 UTC #7 lore raw

PATCH 3/3 nvme-loop: Add BLK_MQ_F_NO_SCHED flag to admin tag set

Reviewed-by Sagi Grimberg <sagi at grimberg.me>

Sagi Grimberg

maintainer

2017-10-18 12:50 UTC #8 lore raw

cover Add BLK_MQ_F_NO_SCHED flag to admin tag set

In reply to Israel Rukshin

Quoted 6 lines

Since commit b86dd81
"block: get rid of blk-mq default scheduler choice Kconfig entries", when setting nr_hw_queues to 1 the admin tag set uses mq-deadline scheduler. This flag is useful for admin queues that aren't used for normal IO. We already use this flag at nvme-pci.

How is the fact that we set a single hw queue has anything to do with the admin tagset?

Israel Rukshin

author

2017-10-18 13:58 UTC #9 lore raw

cover Add BLK_MQ_F_NO_SCHED flag to admin tag set

Quoted 10 lines · Sagi Grimberg

Quoted 7 lines
Since commit b86dd81
"block: get rid of blk-mq default scheduler choice Kconfig entries", when setting nr_hw_queues to 1 the admin tag set uses mq-deadline scheduler.
This flag is useful for admin queues that aren't used for normal IO. We already use this flag at nvme-pci.
How is the fact that we set a single hw queue has anything to do with the admin tagset?

Setting nr_hw_queues attribute of admin tag set to 1 influence the value of nr_hw_queues attribute of the admin queue.
At /block/elevator.c
elevator_init() selects the scheduler according to the value of nr_hw_queues
attribute of the admin queue.

if (q->mq_ops) {
     if (q->nr_hw_queues == 1)
         e = elevator_get("mq-deadline", false);
     if (!e)
         return 0;

...

Israel

Sagi Grimberg

maintainer

2017-10-18 14:31 UTC #10 lore raw

cover Add BLK_MQ_F_NO_SCHED flag to admin tag set

Quoted 11 lines

Quoted 10 lines
Quoted 7 lines
Since commit b86dd81
"block: get rid of blk-mq default scheduler choice Kconfig entries", when setting nr_hw_queues to 1 the admin tag set uses mq-deadline scheduler.
This flag is useful for admin queues that aren't used for normal IO. We already use this flag at nvme-pci.
How is the fact that we set a single hw queue has anything to do with the admin tagset?

Quoted 6 lines

Setting nr_hw_queues attribute of admin tag set to 1 influence the value of nr_hw_queues attribute of the admin queue.
At /block/elevator.c
elevator_init() selects the scheduler according to the value of nr_hw_queues
attribute of the admin queue.

Oh, I confused your meaning with setting nr_hw_queues to 1 in the IO tagset makes a difference for the admin tagset.

Israel Rukshin

author

2017-10-18 15:16 UTC #11 lore raw

cover Add BLK_MQ_F_NO_SCHED flag to admin tag set

Quoted 18 lines · Sagi Grimberg

Quoted 11 lines
Quoted 10 lines
Quoted 7 lines
Since commit b86dd81
"block: get rid of blk-mq default scheduler choice Kconfig entries", when setting nr_hw_queues to 1 the admin tag set uses mq-deadline scheduler.
This flag is useful for admin queues that aren't used for normal IO. We already use this flag at nvme-pci.
How is the fact that we set a single hw queue has anything to do with the admin tagset?
Quoted 6 lines
Setting nr_hw_queues attribute of admin tag set to 1 influence the value of nr_hw_queues attribute of the admin queue.
At /block/elevator.c
elevator_init() selects the scheduler according to the value of nr_hw_queues
attribute of the admin queue.

Quoted 2 lines

Oh, I confused your meaning with setting nr_hw_queues to 1 in the IO tagset makes a difference for the admin tagset.

No,
I just wanted to say that before commit b86dd8 there was no scheduler for the admin q.
After commit b86dd8 I saw that mq-deadline scheduler was selected for the admin q.

Max Gurtovoy

2017-10-18 20:05 UTC #12 lore raw

cover Add BLK_MQ_F_NO_SCHED flag to admin tag set

In reply to Israel Rukshin

Quoted 10 lines · Israel Rukshin

Since commit b86dd81
"block: get rid of blk-mq default scheduler choice Kconfig entries", when setting nr_hw_queues to 1 the admin tag set uses mq-deadline scheduler. This flag is useful for admin queues that aren't used for normal IO. We already use this flag at nvme-pci.
Israel Rukshin (3):
nvme-rdma: Add BLK_MQ_F_NO_SCHED flag to admin tag set
nvme-fc: Add BLK_MQ_F_NO_SCHED flag to admin tag set
nvme-loop: Add BLK_MQ_F_NO_SCHED flag to admin tag set

thanks Israel.
all 3 patches looks good to me,

Reviewed-by Max Gurtovoy <maxg at mellanox.com>

Christoph Hellwig

2017-10-19 08:59 UTC #13 lore raw

cover Add BLK_MQ_F_NO_SCHED flag to admin tag set

In reply to Israel Rukshin

Thanks Israel,

applied the whole series to nvme-4.15