block: skip the blkcg walk in blk_cgroup_congested() when nothing is throttled
Status from this message: Applied, thanks!
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v2 Latest3 patches 2 repliesApplied
- v13 patches 6 repliesSuperseded
Changes in v2 from the cover letter
- Rename blkcg_congested_blkcgs to blkcg_nr_congested to make it clear that the global tracks a count rather than a boolean.
- Warn if blkcg_css_free() finds a residual congestion_count, while still dropping its contribution so it cannot disable the fast path permanently.
- Use atomic_dec_and_test() for the congestion_count 1 -> 0 transition.
Patches in v2 Applied
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/3 | blk-iolatency: clear delay state when freeing policy datablock/ | +9 | 0 replies | |
| 2/3 | blk-iocost: clear delay state when freeing policy datablock/ | +7 | 0 replies | |
| 3/3 | block: skip blkcg walk in blk_cgroup_congested() when nothing throttledblock/ block/ include/ | +57 | 0 replies |
Reviews trailers per patch
None of the 3 patches has a Reviewed-by or Acked-by of their own; the series has one for all of them.
| Patch | Tejun Heo |
|---|---|
| Series | A-b |
| 1/3 | |
| 2/3 | |
| 3/3 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [PATCH v2 0/3] block: skip the blkcg walk in blk_cgroup_congested() when nothing is throttled
blk_cgroup_congested() walks the current task's blkcg ancestor chain on every readahead decision and, once swap is in use, on every anonymous and shmem folio allocation. The answer is almost always "no", but finding that out costs two loads per level on two cold cache lines, plus an out-of-line kthread_blkcg() and an RCU read-side pair. On a fleet profile of hosts running containers with 5-10 level hierarchies it costs about as much as all of mutex_lock(), 99.4% of it under __folio_throttle_swaprate().
Patch 3 gates the walk on a global count of blkcgs with a non-zero congestion_count, so the common case is a load and a predicted branch.
That only works if the count is correctly maintained, currently two teardown paths can leave a blkcg permanently marked congested. Today that only hurts tasks in the affected cgroup, but it hurts them for the life of the cgroup - readahead cut to a single page, async readahead skipped, and a throttle scheduled on every anonymous folio allocation. With a global gate it would cost every other task on the machine the walk as well. Patches 1 and 2 fix those two paths and stand on their own as bugfixes; patch 3 depends on them.
v1 -> v2 (Tejun):
- Rename blkcg_congested_blkcgs to blkcg_nr_congested to make it clear that the global tracks a count rather than a boolean.
- Warn if blkcg_css_free() finds a residual congestion_count, while still dropping its contribution so it cannot disable the fast path permanently.
- Use atomic_dec_and_test() for the congestion_count 1 -> 0 transition.
Usama Arif (3):
blk-iolatency: clear delay state when freeing policy data
blk-iocost: clear delay state when freeing policy data
block: skip blkcg walk in blk_cgroup_congested() when nothing
throttled
block/blk-cgroup.c | 15 ++++++++++++++- block/blk-cgroup.h | 25 +++++++++++++++++++++---- block/blk-iocost.c | 7 +++++++ block/blk-iolatency.c | 9 +++++++++ include/linux/blk-cgroup.h | 23 ++++++++++++++++++++++- 5 files changed, 73 insertions(+), 6 deletions(-)
Signature · 1 line
2.53.0-Meta