blk-mq: improvement CPU hotplug(simplified version)
Get series with b4
b4 am [email protected]Versions newest first
- v1 Latest6 patches 10 repliesIn review
Patches in v1 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/6 | blk-mq: allocate request on cpu in hctx->cpumask for blk_mq_alloc_request_hctxblock/ | +26 | 1 reply | |
| 2/6 | blk-mq: set data->ctx and data->hctx explicitly in blk_mq_get_requestblock/ | +6 | 0 replies | |
| 3/6 | blk-mq: add blk_mq_all_tag_iterblock/ block/ | +31 | 0 replies | |
| 4/6 | blk-mq: prepare for draining IO when hctx's all CPUs are offlineblock/ block/ drivers/ +3 more | +30 | 0 replies | |
| 5/6 | blk-mq: disable preempt during allocating request tagblock/ block/ block/ | +9 | 2 replies | |
| 6/6 | blk-mq: stop to allocate new requst and drain request before hctx becomes inactiveblock/ block/ block/ +1 more | +125 | 2 replies |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.
Cover letter [PATCH 0/6] blk-mq: improvement CPU hotplug(simplified version)
Hi,
Thomas mentioned:
"
That was the constraint of managed interrupts from the very beginning:
The driver/subsystem has to quiesce the interrupt line and the associated queue _before_ it gets shutdown in CPU unplug and not fiddle with it until it's restarted by the core when the CPU is plugged in again.
"
But no drivers or blk-mq do that before one hctx becomes inactive(all CPUs for one hctx are offline), and even it is worse, blk-mq stills tries to run hw queue after hctx is dead, see blk_mq_hctx_notify_dead().
This patchset tries to address the issue by the following approach:
- before the last cpu in hctx->cpumask is going to offline, mark this hctx as inactive
- disable preempt during allocating tag for request, and after tag is allocated, check if this hctx is inactive. If yes, give up this allocation and try remote allocation from online CPUs
- before hctx becomes inactive, drain all allocated requests on this hctx
Thanks John Garry for running lots of tests on arm64 with this previous version patches and co-working on investigating all kinds of issues.
Thanks Christoph for review on old versions of this patches.
https://github.com/ming1/linux/commits/v5.7-rc-blk-mq-improve-cpu-hotplug
Ming Lei (6):
blk-mq: allocate request on cpu in hctx->cpumask for
blk_mq_alloc_request_hctx
blk-mq: set data->ctx and data->hctx explicitly in blk_mq_get_request
blk-mq: add blk_mq_all_tag_iter
blk-mq: prepare for draining IO when hctx's all CPUs are offline
blk-mq: disable preempt during allocating request tag
blk-mq: stop to allocate new requst and drain request before hctx
becomes inactive
block/blk-mq-debugfs.c | 2 + block/blk-mq-tag.c | 47 +++++++++- block/blk-mq-tag.h | 2 + block/blk-mq.c | 184 ++++++++++++++++++++++++++++++++----- block/blk-mq.h | 1 + drivers/block/loop.c | 2 +- drivers/md/dm-rq.c | 2 +- include/linux/blk-mq.h | 10 ++ include/linux/cpuhotplug.h | 1 + 9 files changed, 221 insertions(+), 30 deletions(-)
Signature · 1 line
2.25.2