blk-mq: Add support for multiple queue maps
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v3 Latest16 patches 31 repliesIn review
- v214 patches 44 repliesSuperseded
- v114 patches 24 repliesSuperseded
Changes in v3 from the cover letter
- Address review comments (mostly code comments)
- Add rq->mq_hctx cache
- Cleanup/improve blk_mq_sched_insert_requests()
- Harden the IRQ affinity sets feature
- Fix assignment of hardware queue type if already mapped
- Improve plug sorting for multiple maps
Patches in v3 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/16 | blk-mq: kill q->mq_mapblock/ block/ include/ | +7 | 1 reply | |
| 2/16 | blk-mq: abstract out queue mapblock/ block/ block/ +12 more | +64 | 2 replies | |
| 3/16 | blk-mq: provide dummy blk_mq_map_queue_type() helperblock/ | +8 | 1 reply | |
| 4/16 | blk-mq: pass in request/bio flags to queue mappingblock/ block/ block/ +4 more | +57 | 2 replies | |
| 5/16 | blk-mq: allow software queue to map to multiple hardware queuesblock/ block/ block/ +2 more | +23 | 2 replies | |
| 6/16 | blk-mq: add 'type' attribute to the sysfs hctx directoryblock/ | +10 | 4 replies | |
| 7/16 | blk-mq: support multiple hctx mapsblock/ block/ include/ | +100 | 2 replies | |
| 8/16 | blk-mq: separate number of hardware queues from nr_cpu_idsblock/ | +21 | 1 reply | |
| 9/16 | blk-mq: cache request hardware queue mappingblock/ block/ block/ +4 more | +19 | 3 replies | |
| 10/16 | blk-mq: cleanup and improve list insertionblock/ block/ block/ | +16 | 2 replies | |
| 11/16 | blk-mq: improve plug list sortingblock/ | +12 | 2 replies | |
| 12/16 | blk-mq: initial support for multiple queue mapsblock/ block/ block/ +1 more | +9 | 1 reply | |
| 13/16 | irq: add support for allocating (and affinitizing) sets of IRQsdrivers/ include/ kernel/ | +49 | 4 replies | |
| 14/16 | nvme: utilize two queue maps, one for reads and one for writesdrivers/ | +162 | 2 replies | |
| 15/16 | block: add REQ_HIPRI and inherit it from IOCB_HIPRIfs/ fs/ fs/ +1 more | +15 | 1 reply | |
| 16/16 | nvme: add separate poll queue mapdrivers/ include/ | +81 | 0 replies |
Reviews trailers per patch
All 16 patches have a Reviewed-by or Acked-by of their own; the series has one for all of them.
| Patch | Christoph Hellwig | Hannes Reinecke | Bart Van Assche | Ming Lei | Keith Busch | Sagi Grimberg |
|---|---|---|---|---|---|---|
| Series | R-b | |||||
| 1/16 | R-b | R-b | R-b | R-b | ||
| 2/16 | R-b | R-b | ||||
| 3/16 | R-b | R-b | ||||
| 4/16 | R-b | |||||
| 5/16 | R-b | |||||
| 6/16 | R-b | R-b | R-b | |||
| 7/16 | R-b | R-b | ||||
| 8/16 | R-b | R-b | R-b | |||
| 9/16 | R-b | R-b | ||||
| 10/16 | R-b | R-b | ||||
| 11/16 | R-b | R-b | ||||
| 12/16 | R-b | R-b | ||||
| 13/16 | R-b | R-b | R-b | |||
| 14/16 | R-b | R-b | ||||
| 15/16 | R-b | R-b | ||||
| 16/16 | R-b |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 6 reviewers.
Cover letter [PATCHSET v3 0/16] blk-mq: Add support for multiple queue maps
This series adds support for multiple queue maps for blk-mq. Since blk-mq was introduced, it's only support a single queue map. This means you can have 1 set of queues, and the mapping purely depends on what CPU an IO originated from. With this patch set, drivers can implement mappings that depend on both CPU and request type - and they can have multiple sets of mappings.
NVMe is used as a proof of concept. It adds support for a separate write queue set. One way to use this would be to limit the number of write queues to favor reads, since NVMe does round-robin service of queues. An easy extension of this would be to add multiple sets of queues, for prioritized IO.
NVMe also uses this feature to finally make the polling work efficiently, without triggering interrupts. This both increases performance (and decreases latency), at a lower system load. At the same time it's more flexible, as you don't have to worry about IRQ coalescing and redirection to avoid interrupts disturbing the workload. This is how polling should have worked from day 1.
This is on top of my mq-conversions branch. It can also be bound in my mq-maps branch.
Changes since v2:
- Address review comments (mostly code comments)
- Add rq->mq_hctx cache
- Cleanup/improve blk_mq_sched_insert_requests()
- Harden the IRQ affinity sets feature
- Fix assignment of hardware queue type if already mapped
- Improve plug sorting for multiple maps
Changes since v1:
- Ensure irq_calc_affinity_vectors() doesn't return more than 'maxvec'
- Rebase on top of current mq-conversions series
block/blk-flush.c | 11 +- block/blk-mq-cpumap.c | 19 +-- block/blk-mq-debugfs.c | 2 +- block/blk-mq-pci.c | 10 +- block/blk-mq-rdma.c | 4 +- block/blk-mq-sched.c | 12 +- block/blk-mq-sched.h | 2 +- block/blk-mq-sysfs.c | 10 ++ block/blk-mq-tag.c | 12 +- block/blk-mq-virtio.c | 8 +- block/blk-mq.c | 237 +++++++++++++++++++----------- block/blk-mq.h | 45 ++++-- block/blk.h | 6 +- block/kyber-iosched.c | 6 +- drivers/block/virtio_blk.c | 2 +- drivers/nvme/host/pci.c | 263 ++++++++++++++++++++++++++++++---- drivers/pci/msi.c | 14 ++ drivers/scsi/qla2xxx/qla_os.c | 5 +- drivers/scsi/scsi_lib.c | 2 +- drivers/scsi/smartpqi/smartpqi_init.c | 3 +- drivers/scsi/virtio_scsi.c | 3 +- fs/block_dev.c | 2 + fs/direct-io.c | 2 + fs/iomap.c | 9 +- include/linux/blk-mq-pci.h | 4 +- include/linux/blk-mq-virtio.h | 4 +- include/linux/blk-mq.h | 33 ++++- include/linux/blk_types.h | 4 +- include/linux/blkdev.h | 3 +- include/linux/interrupt.h | 4 + kernel/irq/affinity.c | 40 ++++-- 31 files changed, 582 insertions(+), 199 deletions(-)
Signature · 1 line
Jens Axboe