block: expose blkcg io.stat to BPF
Subsystems (6)
v2 cover block: expose blkcg io.stat to BPF
Collecting cgroup statistics is expensive: the existing method is to open and parse a cgroup file for every cgroup of interest. The memory controller already has an efficient alternative through BPF; this series extends that model to the block I/O controller's per-device io.stat.
Patch 1 adds the blkcg flush and per-device iterator.
Patch 2 adds a selftest that performs direct I/O on a private loop device and compares every counter read by BPF with io.stat.
===
Changes since v1.
- Drop the blkcg acquire/release helpers. The program reads the io controller's css from cgroup->subsys[] under RCU instead.
- Remove the kfuncs that read the blkg device ID and counters; BPF programs can read those fields with BPF_CORE_READ().
- Keep the flush sleepable. It now takes a struct cgroup * and pins its CSS internally.
- Make the blkg iterator take an RCU-protected CSS.
Tested on VM with v7.2-rc5.
Ziyang Men (2):
block: add BPF kfuncs to read blkcg io.stat selftests/bpf: add test for blkcg io.stat BPF kfuncs
MAINTAINERS | 1 + block/Makefile | 3 + block/blk-cgroup.c | 2 +- block/blk-cgroup.h | 1 + block/bpf_blkcg.c | 154 ++++++++++ tools/testing/selftests/bpf/cgroup_iter_io.h | 17 ++ tools/testing/selftests/bpf/config | 1 + .../selftests/bpf/prog_tests/cgroup_iter_io.c | 277 ++++++++++++++++++ .../selftests/bpf/progs/cgroup_iter_io.c | 99 +++++++ 9 files changed, 554 insertions(+), 1 deletion(-) create mode 100644 block/bpf_blkcg.c create mode 100644 tools/testing/selftests/bpf/cgroup_iter_io.h create mode 100644 tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c create mode 100644 tools/testing/selftests/bpf/progs/cgroup_iter_io.c
Signature · 1 line
2.53.0-Meta
PATCH v2 1/2 block: add BPF kfuncs to read blkcg io.stat
Collecting cgroup statistics is expensive: the existing method is to open and parse a cgroup file for every cgroup of interest. memcg already has an efficient alternative through BPF; this series extends that idea to block. This series exposes the block I/O controller's per-device io.stat to BPF.
The flush is sleepable and takes a cgroup. It pins the I/O css before leaving RCU, then flushes it. The iterator takes the RCU-protected css and remains block-specific because each block device has its own blkg. The behavior mirrows the blkcg_print_stat().
The blkg device iterator take a RCU css.
No kfuncs are added to read the blkcg counters since user can read it using the BPF_CORE_READ.
--- MAINTAINERS | 1 + block/Makefile | 3 + block/blk-cgroup.c | 2 +- block/blk-cgroup.h | 1 + block/bpf_blkcg.c | 154 +++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 160 insertions(+), 1 deletion(-) create mode 100644 block/bpf_blkcg.c
MAINTAINERS +1 -0
@@ -6617,6 +6617,7 @@ F: block/blk-cgroup.c F: block/blk-iocost.c F: block/blk-iolatency.c F: block/blk-throttle.c +F: block/bpf_blkcg.c F: include/linux/blk-cgroup.h CONTROL GROUP - CPUSET
block/Makefile +3 -0
@@ -17,6 +17,9 @@ obj-$(CONFIG_BLK_ERROR_INJECTION) += error-injection.o obj-$(CONFIG_BLK_DEV_BSG_COMMON) += bsg.o obj-$(CONFIG_BLK_DEV_BSGLIB) += bsg-lib.o obj-$(CONFIG_BLK_CGROUP) += blk-cgroup.o +ifdef CONFIG_BPF_SYSCALL +obj-$(CONFIG_BLK_CGROUP) += bpf_blkcg.o +endif obj-$(CONFIG_BLK_CGROUP_RWSTAT) += blk-cgroup-rwstat.o obj-$(CONFIG_BLK_CGROUP_FC_APPID) += blk-cgroup-fc-appid.o obj-$(CONFIG_BLK_DEV_THROTTLING) += blk-throttle.o
block/blk-cgroup.c +1 -1
@@ -1086,7 +1086,7 @@ static void blkcg_rstat_flush(struct cgroup_subsys_state *css, int cpu)* flushing the root cgroup's stats by explicitly filling in the iostat * with disk level statistics. */ -static void blkcg_fill_root_iostats (void ) +void blkcg_fill_root_iostats (void ) {struct class_dev_iter iter;struct device *dev;
block/blk-cgroup.h +1 -0
@@ -205,6 +205,7 @@ void blkcg_deactivate_policy(struct gendisk *disk, const struct blkcg_policy *pol);const char *blkg_dev_name (struct blkcg_gq *blkg); +void blkcg_fill_root_iostats (void ); void blkcg_print_blkgs(struct seq_file *sf, struct blkcg *blkcg, u64 (*prfill)(struct seq_file *, struct blkg_policy_data *, int),
block/bpf_blkcg.c new file +154 -0
154 lines not on this page · raw message
Signature · 1 line
2.53.0-Meta
[email protected]
AI reviewRe: [PATCH v2 1/2] block: add BPF kfuncs to read blkcg io.stat
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] Concurrent execution of blkcg_fill_root_iostats() corrupts the u64_stats_sync sequence counter, leading to reader deadlocks. --
--- Patch [1]: [PATCH v2 1/2] block: add BPF kfuncs to read blkcg io.stat --- commit bdaf96b752ce960b924c3be337708d377f83207c
Author: Ziyang Men <[email protected]>
block: add BPF kfuncs to read blkcg io.stat
This commit introduces BPF kfuncs to allow reading per-device block I/O controller statistics. It exposes blkcg_fill_root_iostats and provides bpf_blkcg_flush_stats and iterator functions for blkg.
Quoted 5 lines
block/bpf_blkcg.c new file +0
-0
[ ... ]
Quoted 6 lines
block/bpf_blkcg.c new file +6
-0+__bpf_kfunc void bpf_blkcg_flush_stats(struct cgroup *cgrp) +{ +struct cgroup_subsys_state *css; + +/* Pin the css for the sleepable flush. */ +rcu_read_lock ();
block/bpf_blkcg.c line 36
Quoted 10 lines
block/bpf_blkcg.c new file +10
-0+ css =cgroup_css (cgrp, &io_cgrp_subsys); +if (css && !css_tryget (css)) + css =NULL ; +rcu_read_unlock (); + +if (!css) +return ; + +if (!css->parent) +blkcg_fill_root_iostats ();
[Severity: High]
This isn't a bug introduced by this patch, but can concurrent executions here lead to sequence counter corruption and reader deadlocks on 32-bit systems?
In block/blk-cgroup.c, blkcg_fill_root_iostats() loops over block devices and updates statistics without an overarching writer lock:
blkcg_fill_root_iostats() {
...
flags = u64_stats_update_begin_irqsave(&blkg->iostat.sync);
blkg_iostat_set(&blkg->iostat.cur, &tmp);
u64_stats_update_end_irqrestore(&blkg->iostat.sync, flags);
...
}On 32-bit architectures, u64_stats_update_begin_irqsave() maps to write_seqcount_begin(), which requires external mutual exclusion to prevent concurrent writers from corrupting the sequence counter. If the counter becomes permanently odd, readers using read_seqcount_begin() will spin infinitely.
While this bug existed previously via concurrent reads of the root cgroup's io.stat sysfs file, exposing blkcg_fill_root_iostats() via this sleepable BPF kfunc allows BPF programs to easily and repeatedly trigger this lockless code path simultaneously across multiple CPUs.
Quoted 5 lines
block/bpf_blkcg.c new file +5
-0+else +css_rstat_flush (css); + +css_put (css); +}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
[email protected]
AI reviewRe: [PATCH v2 1/2] block: add BPF kfuncs to read blkcg io.stat
In reply to Ziyang MenThis isn't a bug, but there are a few rough spots in the changelog: "mirrows" should be "mirrors", "The blkg device iterator take a RCU css." reads as a fragment that restates the earlier sentence…
Quoted 6 lines
The behavior mirrows the
blkcg_print_stat().The blkg device iterator take a RCU css.
No kfuncs are added to read the blkcg counters since user can read it using the
BPF_CORE_READ.
This isn't a bug, but there are a few rough spots in the changelog:
"mirrows" should be "mirrors", "The blkg device iterator take a RCU css." reads as a fragment that restates the earlier sentence about the iterator taking the RCU-protected css, and "since user can read it" is missing an article before "user".
Could the changelog also say why a new flush kfunc is needed rather than reusing the existing css_rstat_flush() kfunc -- is the point specifically the root-cgroup blkcg_fill_root_iostats() path?
Quoted 14 lines
block/Makefile +3
-0@@ -17,6 +17,9 @@ obj-$(CONFIG_BLK_ERROR_INJECTION) += error-injection.o obj-$(CONFIG_BLK_DEV_BSG_COMMON) += bsg.o obj-$(CONFIG_BLK_DEV_BSGLIB) += bsg-lib.o obj-$(CONFIG_BLK_CGROUP) += blk-cgroup.o +ifdef CONFIG_BPF_SYSCALL +obj-$(CONFIG_BLK_CGROUP) += bpf_blkcg.o +endif obj-$(CONFIG_BLK_CGROUP_RWSTAT) += blk-cgroup-rwstat.o obj-$(CONFIG_BLK_CGROUP_FC_APPID) += blk-cgroup-fc-appid.o obj-$(CONFIG_BLK_DEV_THROTTLING) += blk-throttle.o
[ ... ]
Quoted 37 lines
block/bpf_blkcg.c new file +31
-0@@ -0,0 +1,154 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* +* Block I/O Controller-related BPF kfuncs and auxiliary code +*/ + +#include "blk-cgroup.h" + +#include <linux/bpf.h> +#include <linux/btf_ids.h> +#include <linux/rculist.h> + +__bpf_kfunc_start_defs (); + +/** +* bpf_blkcg_flush_stats - Flush a block cgroup's io statistics +* @cgrp: cgroup to flush +* +* Propagate I/O statistics up the cgroup tree. Root statistics come from +* block devices and include all cgroups' I/O. +*/ +__bpf_kfunc void bpf_blkcg_flush_stats(struct cgroup *cgrp) +{ +struct cgroup_subsys_state *css; + +/* Pin the css for the sleepable flush. */ +rcu_read_lock (); + css =cgroup_css (cgrp, &io_cgrp_subsys); +if (css && !css_tryget (css)) + css =NULL ; +rcu_read_unlock (); +
block/bpf_blkcg.c line 41
Quoted 10 lines
block/bpf_blkcg.c new file +10
-0+if (!css) +return ; + +if (!css->parent) +blkcg_fill_root_iostats (); +else +css_rstat_flush (css); + +css_put (css); +}
This isn't a bug, but would it be simpler for bpf_blkcg_flush_stats() to take the io css directly, so the same handle feeds bpf_iter_blkg_new()? The two entry points in this file take different cgroup handles:bpf_blkcg_flush_stats() takes struct cgroup* and resolves the css internally with cgroup_css()/css_tryget(), while bpf_iter_blkg_new() below takes struct cgroup_subsys_state* directly. A program that wants to flush and then iterate therefore needs both handle types for the same cgroup. Taking struct cgroup_subsys_state* in both, as mm/bpf_memcontrol.c does for its memcg handles, would make them composable and would drop the cgroup_css() lookup.
Quoted 103 lines
block/bpf_blkcg.c new file +103
-0+ +struct bpf_iter_blkg { +__u64 __opaque[2 ]; +}__aligned(8) ; + +struct bpf_iter_blkg_kern { +struct blkcg *blkcg; +struct blkcg_gq *pos; +}__aligned(8) ; + +/** +* bpf_iter_blkg_new - Start iterating a block cgroup's per-device blkgs +* @it: iterator to initialize +* @css: the io controller's css +* +* Each blkg holds one device's io.stat counters. Offline blkgs are skipped. +* A blkg without a disk can be returned. Must run under RCU. +* +* Return: 0 on success, -EINVAL if @css is not the io controller's. +*/ +__bpf_kfunc int bpf_iter_blkg_new(struct bpf_iter_blkg *it, +struct cgroup_subsys_state *css) +{ +struct bpf_iter_blkg_kern *kit = (void *)it; + +BUILD_BUG_ON (sizeof (struct bpf_iter_blkg_kern ) >sizeof (struct bpf_iter_blkg )); +BUILD_BUG_ON (__alignof__(struct bpf_iter_blkg_kern ) != + __alignof__(struct bpf_iter_blkg )); + + kit->pos =NULL ; + +if (css->ss != &io_cgrp_subsys) { + kit->blkcg =NULL ; +return -EINVAL ; + } + + kit->blkcg =css_to_blkcg (css); +return 0 ; +} + +/** +* bpf_iter_blkg_next - Return the next online blkg of the iterated block cgroup +* @it: iterator +* +* Return: the next online blkg, or NULL when the walk is done. +*/ +__bpf_kfunc struct blkcg_gq *bpf_iter_blkg_next(struct bpf_iter_blkg *it) +{ +struct bpf_iter_blkg_kern *kit = (void *)it; +struct blkcg_gq *blkg = kit->pos; +struct hlist_node *node; + +if (!kit->blkcg) +return NULL ; + +if (!blkg) + node =rcu_dereference (hlist_first_rcu (&kit->blkcg->blkg_list)); +else + node =rcu_dereference (hlist_next_rcu (&blkg->blkcg_node)); + +/* Skip offline blkgs, matching io.stat. */ +while (node) { + blkg = hlist_entry(node, struct blkcg_gq, blkcg_node); +/* A race only changes whether this blkg is returned. */ +if (data_race (blkg->online)) { + kit->pos = blkg; +return blkg; + } + node =rcu_dereference (hlist_next_rcu (&blkg->blkcg_node)); + } + +/* The iterator must keep returning NULL after completion. */ + kit->pos =NULL ; + kit->blkcg =NULL ; +return NULL ; +} + +/** +* bpf_iter_blkg_destroy - Tear down a blkg iterator +* @it: iterator +*/ +__bpf_kfunc void bpf_iter_blkg_destroy(struct bpf_iter_blkg *it) +{ +} + +__bpf_kfunc_end_defs (); + +BTF_KFUNCS_START (bpf_blkcg_kfuncs) +BTF_ID_FLAGS (func ,bpf_blkcg_flush_stats ,KF_SLEEPABLE ) + +BTF_ID_FLAGS (func, bpf_iter_blkg_new, +KF_ITER_NEW |KF_RCU |KF_RCU_PROTECTED ) +BTF_ID_FLAGS (func, bpf_iter_blkg_next,KF_ITER_NEXT |KF_RET_NULL ) +BTF_ID_FLAGS (func, bpf_iter_blkg_destroy,KF_ITER_DESTROY ) +BTF_KFUNCS_END (bpf_blkcg_kfuncs) + +static const struct btf_kfunc_id_set bpf_blkcg_kfunc_set = { + .owner =THIS_MODULE , + .set = &bpf_blkcg_kfuncs, +}; + +static int __init bpf_blkcg_init (void ) +{
block/bpf_blkcg.c line 154
Quoted 10 lines
block/bpf_blkcg.c new file +10
-0+int err; + + err =register_btf_kfunc_id_set (BPF_PROG_TYPE_UNSPEC , + &bpf_blkcg_kfunc_set); +if (err) +pr_warn ("error while registering bpf blkcg kfuncs: %d\n" , err); + +return err; +} +late_initcall (bpf_blkcg_init);
This isn't a bug, but could a selftest under tools/testing/selftests/bpf/ be added alongside these kfuncs to lock down the RCU-protected iterator usage? Four new kfuncs are registered for BPF_PROG_TYPE_UNSPEC (all program types) with no accompanying selftest and no in-tree consumer, so nothing exercises the iterator's KF_RCU_PROTECTED contract, the offline-blkg skip, or the root-vs-non-root flush split. Comparable additions -- mm/bpf_memcontrol.c and the bpf_iter_css family -- landed with tests under tools/testing/selftests/bpf/, which is also what pins the intended usage pattern (bpf_rcu_read_lock() around new/next/destroy) for future readers.
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/32073368069
Tejun Heo
maintainerRe: [PATCH v2 1/2] block: add BPF kfuncs to read blkcg io.stat
In reply to Ziyang MenHello,
block/bpf_blkcg.c line 21
Quoted 8 lines · Ziyang Men
block/bpf_blkcg.c new file +8
-0+/** +* bpf_blkcg_flush_stats - Flush a block cgroup's io statistics +* @cgrp: cgroup to flush +* +* Propagate I/O statistics up the cgroup tree. Root statistics come from +* block devices and include all cgroups' I/O. +*/ +__bpf_kfunc void bpf_blkcg_flush_stats(struct cgroup *cgrp)
Either @css or @blkcg, probably the latter so that each operation doesn't have to verify subsys again. I'd make this and the cpu ppatches into a single series. Add blkcg cast there too.
Thanks.
Signature · 1 line
tejun
Re: [PATCH v2 1/2] block: add BPF kfuncs to read blkcg io.stat
Hi Tejun,
Quoted 15 lines · Tejun Heo
Hello,
block/bpf_blkcg.c line 21
Quoted 8 lines · Ziyang Men
block/bpf_blkcg.c new file +8
-0+/** +* bpf_blkcg_flush_stats - Flush a block cgroup's io statistics +* @cgrp: cgroup to flush +* +* Propagate I/O statistics up the cgroup tree. Root statistics come from +* block devices and include all cgroups' I/O. +*/ +__bpf_kfunc void bpf_blkcg_flush_stats(struct cgroup *cgrp)Either @css or @blkcg, probably the latter so that each operation doesn't have to verify subsys again. I'd make this and the cpu ppatches into a single series. Add blkcg cast there too.
Ok I will add the casts bpf_get_blkcg() and bpf_put_blkcg(), and remove the check of css->ss in above functoins.
Next submission of this series will be together with the cpu one.
Quoted 5 lines
Thanks.
Signature · 1 line
tejun
Thanks for your review!
Best,
Ziyang
Tejun Heo
maintainerRe: [PATCH v2 1/2] block: add BPF kfuncs to read blkcg io.stat
Quoted 2 lines · Ziyang Men
Ok I will add the casts
bpf_get_blkcg()andbpf_put_blkcg(), and remove the check ofcss->ssin above functoins.
RCU_PROTECTED is probably better than get/put().
Thanks.
Signature · 1 line
tejun
Re: [PATCH v2 1/2] block: add BPF kfuncs to read blkcg io.stat
Hi Tejun,
Quoted 5 lines · Tejun Heo
Quoted 2 lines · Ziyang Men
Ok I will add the casts
bpf_get_blkcg()andbpf_put_blkcg(), and remove the check ofcss->ssin above functoins.
RCU_PROTECTEDis probably better than get/put().
Yes we can make the blkcg* RCU_PROTECTED but then the bpf_blkcg_flush_stats() needs to take the @cgroup* since it is SLEEPABLE. Does this work for you?
Best,
Ziyang
Quoted 5 lines
Thanks.
Signature · 1 line
tejun
Tejun Heo
maintainerRe: [PATCH v2 1/2] block: add BPF kfuncs to read blkcg io.stat
Hello,
Quoted 9 lines · Ziyang Men
Quoted 5 lines · Tejun Heo
Quoted 2 lines · Ziyang Men
Ok I will add the casts
bpf_get_blkcg()andbpf_put_blkcg(), and remove the check ofcss->ssin above functoins.
RCU_PROTECTEDis probably better than get/put().Yes we can make the blkcg*
RCU_PROTECTEDbut then thebpf_blkcg_flush_stats()needs to take the @cgroup* since it is SLEEPABLE. Does this work for you?
Do we really need that function? css_rstat_flush() is already exposed and css can already be obtained, right? blkcg_fill_root_iostat() is just accessing system-wide bdev stats, which you probably don't need to begin with or if necessary just access through bdevs.
Thanks.
Signature · 1 line
tejun
Re: [PATCH v2 1/2] block: add BPF kfuncs to read blkcg io.stat
Hi Tejun,
Quoted 10 lines · Tejun Heo
Hello,
Quoted 7 lines · Ziyang Men
Quoted 5 lines · Tejun Heo
Quoted 2 lines · Ziyang Men
Ok I will add the casts
bpf_get_blkcg()andbpf_put_blkcg(), and remove the check ofcss->ssin above functoins.
RCU_PROTECTEDis probably better than get/put().
Quoted 7 lines
Quoted 2 lines
Yes we can make the blkcg*
RCU_PROTECTEDbut then thebpf_blkcg_flush_stats()needs to take the @cgroup* since it is SLEEPABLE. Does this work for you?Do we really need that function?
css_rstat_flush()is already exposed and css can already be obtained, right?blkcg_fill_root_iostat()is just accessing system-wide bdev stats, which you probably don't need to begin with or if necessary just access through bdevs.
Yes the css_rstat_flush() is exposed but currently it is marked as SLEEPABLE, so it doesn't accept a RCU_PROTECTED css we proposed. This is not a problem for cpu ones since &cgrp->self is a trusted pointer.
Best,
Ziyang
Quoted 5 lines
Thanks.
Signature · 1 line
tejun
Tejun Heo
maintainerRe: [PATCH v2 1/2] block: add BPF kfuncs to read blkcg io.stat
Hello,
Quoted 3 lines · Ziyang Men
Yes the
css_rstat_flush()is exposed but currently it is marked as SLEEPABLE, so it doesn't accept aRCU_PROTECTEDcss we proposed. This is not a problem for cpu ones since&cgrp->selfis a trusted pointer.
Yeah, bummer, so we get ref on cgroups but the subsys pointers are RCU, so that means those refs are only RCU protected. I guess we can't avoid a kfunc to get acquire a subsys css from cgroup. I wish css_rstat_flush() didn't schedule but it does need that cond_resched().
Thanks.
Signature · 1 line
tejun
Re: [PATCH v2 1/2] block: add BPF kfuncs to read blkcg io.stat
Hi Tejun,
Quoted 11 lines · Tejun Heo
Hello,
Quoted 3 lines · Ziyang Men
Yes the
css_rstat_flush()is exposed but currently it is marked as SLEEPABLE, so it doesn't accept aRCU_PROTECTEDcss we proposed. This is not a problem for cpu ones since&cgrp->selfis a trusted pointer.Yeah, bummer, so we get ref on cgroups but the subsys pointers are RCU, so that means those refs are only RCU protected. I guess we can't avoid a kfunc to get acquire a subsys css from cgroup. I wish
css_rstat_flush()didn't schedule but it does need thatcond_resched().
Yeah that would be a pity. I will add kfuncs bpf_cgroup_css() and bpf_css_release() to handle the reference for the css.
The bpf_blkcg_flush_stats() will be removed as well.
Thanks,
Ziyang
Quoted 5 lines
Thanks.
Signature · 1 line
tejun
PATCH v2 2/2 selftests/bpf: add test for blkcg io.stat BPF kfuncs
In reply to Ziyang MenAdd cgroup_iter_io to test the blkcg io.stat BPF kfuncs. The BPF program flushes the statistics, looks up the I/O css under RCU, walks its blkgs, and reads one device's counters with BPF_CORE_READ().
The test performs O_DIRECT I/O on a private loop device. It checks the device ID and counters against io.stat.
--- tools/testing/selftests/bpf/cgroup_iter_io.h | 17 ++ tools/testing/selftests/bpf/config | 1 + .../selftests/bpf/prog_tests/cgroup_iter_io.c | 277 ++++++++++++++++++ .../selftests/bpf/progs/cgroup_iter_io.c | 99 +++++++ 4 files changed, 394 insertions(+) create mode 100644 tools/testing/selftests/bpf/cgroup_iter_io.h create mode 100644 tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c create mode 100644 tools/testing/selftests/bpf/progs/cgroup_iter_io.c
tools/testing/selftests/bpf/cgroup_iter_io.h new file +17 -0
@@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) 2025 Meta Platforms, Inc. and affiliates. */ +#ifndef __CGROUP_ITER_IO_H +#define __CGROUP_ITER_IO_H + +struct io_query { +/* one device's io.stat counters */ +__u64 rbytes; +__u64 wbytes; +__u64 rios; +__u64 wios; +__u64 dbytes; +__u64 dios; +__u64 dev;/* dev_t of the device the counters belong to */ +}; + +#endif /* __CGROUP_ITER_IO_H */
tools/testing/selftests/bpf/config +1 -0
@@ -1,3 +1,4 @@ +CONFIG_BLK_CGROUP=y CONFIG_BLK_DEV_LOOP=y CONFIG_BOOTPARAM_HARDLOCKUP_PANIC=y CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC=1
tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c new file +277 -0
277 lines not on this page · raw message
tools/testing/selftests/bpf/progs/cgroup_iter_io.c new file +99 -0
99 lines not on this page · raw message
Signature · 1 line
2.53.0-Meta
[email protected]
AI reviewRe: [PATCH v2 2/2] selftests/bpf: add test for blkcg io.stat BPF kfuncs
Does the err: cleanup path leak the loop device?
Quoted 5 lines
tools/testing/selftests/bpf/cgroup_iter_io.h new file +0
-0
[ ... ]
Quoted 4 lines
tools/testing/selftests/bpf/config +0
-0
[ ... ]
Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c new file +0
-0@@ -0,0 +1,277 @@
[ ... ]
Quoted 31 lines
tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c new file +31
-0+/* Set up a loop device for cgroup-charged I/O. */ +static int loop_setup (char *loop_path,size_t sz,int *ctl_fd,int *loop_fd, +int *back_fd) +{ +char back_path[] ="/tmp/cgroup_iter_io.XXXXXX" ; +int nr; + + *ctl_fd = *loop_fd = *back_fd =-1 ; + + *ctl_fd =open ("/dev/loop-control" ,O_RDWR |O_CLOEXEC ); +if (*ctl_fd <0 ) +return -1 ; + + nr =ioctl (*ctl_fd,LOOP_CTL_GET_FREE ); +if (nr <0 ) + goto err; +snprintf (loop_path, sz,"/dev/loop%d" , nr); + + *back_fd =mkstemp (back_path); +if (*back_fd <0 ) + goto err; +unlink (back_path); +if (ftruncate (*back_fd, (off_t )IO_SIZE *4 )) + goto err; + + *loop_fd =open (loop_path,O_RDWR |O_CLOEXEC ); +if (*loop_fd <0 ) + goto err; +if (ioctl (*loop_fd,LOOP_SET_FD , *back_fd)) + goto err; +
tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c line 78
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c new file +10
-0+return 0 ; +err: +if (*loop_fd >=0 ) +close (*loop_fd); +if (*back_fd >=0 ) +close (*back_fd); +close (*ctl_fd); + *ctl_fd = *loop_fd = *back_fd =-1 ; +return -1 ; +}
Does the err: cleanup path leak the loop device?
LOOP_CTL_GET_FREE can allocate a new loop device when no unbound device exists, and nr is stored. Every failure after that point (mkstemp, ftruncate, open(/dev/loopN), LOOP_SET_FD) leaves that device behind.
The caller jumps straight to cleanup_cgroup_fd on loop_setup() failure, bypassing cleanup_loop, so loop_teardown() never runs and the device remains allocated.
Recording nr and calling ioctl(*ctl_fd, LOOP_CTL_REMOVE, nr) on the err:
path would fix this. Additionally, loop_teardown() ignores the return of both LOOP_CLR_FD and LOOP_CTL_REMOVE, so a device left in Lo_bound state also leaks silently.
Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c new file +5
-0+/* Read @dev's io.stat counters. @dev uses kernel dev_t encoding. */ +static int parse_io_stat (int cgroup_fd,__u64 dev,struct io_query *out) +{ +unsigned int want_maj = dev >>20 , want_min = dev & ((1U <<20 ) -1 ); +char buf[4096 ], *line, *saveptr;
tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c line 138
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c new file +10
-0+int fd, n, ret =-1 ; + + fd =openat (cgroup_fd,"io.stat" ,O_RDONLY ); +if (fd <0 ) +return -1 ; + n =read (fd, buf,sizeof (buf) -1 ); +close (fd); +if (n <=0 ) +return -1 ; + buf[n] ='\0' ;
Will a single 4095-byte read() always capture the complete io.stat output?
io.stat is a seq_file with one line per blkg, and seq_read_iter() copies with byte granularity, not record boundaries. Two failure modes:
1. If the cgroup's blkg list produces more than 4095 bytes, the loop device's line may not be in the buffer, and parse_io_stat() returns -1.
2. The cut can land mid-line. A truncated line like "7:0 rbytes=4194304 wbytes=41" satisfies the '< 2' check below and is accepted with the remaining counters silently zero, producing a bogus counter mismatch against the kfunc rather than a parse error.
Reading in a loop until read() returns 0, or at least treating a non- terminated final line as an error, would avoid both.
Quoted 14 lines
tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c new file +14
-0+for (line =strtok_r (buf,"\n" , &saveptr); line; + line =strtok_r (NULL ,"\n" , &saveptr)) { +unsigned long long rb =0 , wb =0 , ri =0 , wi =0 , db =0 , di =0 ; +unsigned int maj, min; + +/* Only the device id is required; missing counters stay zero. */ +if (sscanf (line, +"%u:%u rbytes=%llu wbytes=%llu rios=%llu wios=%llu dbytes=%llu dios=%llu" , + &maj, &min, &rb, &wb, &ri, &wi, &db, &di) <2 ) +continue ; +if (maj != want_maj || min != want_min) +continue ; + + out->rbytes = rb;
tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c line 163
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c new file +10
-0+ out->wbytes = wb; + out->rios = ri; + out->wios = wi; + out->dbytes = db; + out->dios = di; + ret =0 ; +break ; + } +return ret; +}
Should the partial conversion check be stricter?
Accepting '< 2' and then returning success means a line that matched the device but whose counters were not parsed is reported as all-zero rather than an error. The caller immediately uses strict equality checks:
ASSERT_EQ(q->dbytes, filev.dbytes, "dbytes");
so a parse problem is reported as a counter mismatch against the kfunc, pointing at the kernel instead of the parser.
This is reachable because blkcg_print_one_stat() emits the device name unconditionally but gates the counter group with:
if (rbytes || wbytes || rios || wios) seq_printf(s, "rbytes=%llu wbytes=%llu ... dios=%llu", ...);
A cgroup whose only traffic to a device was discard produces "MAJ:MIN " with no key=value pairs. sscanf() returns 2, parse_io_stat() returns success with dbytes=dios=0, and the match subtest fails on "dbytes" even though the kernel is correct.
Requiring all eight conversions (== 8) once maj:min matches would surface the real problem.
[ ... ]
tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c line 182
Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c new file +4
-0+if (loop_setup (loop_path,sizeof (loop_path), &ctl_fd, &loop_fd, &back_fd)) { +test__skip ();/* needs root + CONFIG_BLK_DEV_LOOP */ + goto cleanup_cgroup_fd; + }
A subsystem pattern flags this as potentially concerning:
loop_setup() collapses eight distinct failures into a single -1, and the caller turns every one of them into a bare test__skip() with no runtime reason. The source comment says "needs root + CONFIG_BLK_DEV_LOOP", but the same return value is produced by: open("/dev/loop-control") failing, LOOP_CTL_GET_FREE failing, mkstemp() failing, ftruncate() failing (ENOSPC on a small tmpfs /tmp), open("/dev/loopN") failing, and LOOP_SET_FD failing.
The LOOP_SET_FD case is a genuine race: LOOP_CTL_GET_FREE is not a reservation, so any concurrent consumer (another test_progs worker under -j, udev, systemd, losetup) can bind /dev/loopN between GET_FREE and SET_FD, and SET_FD then returns EBUSY. The result is that the whole test, including all five subtests exercising the new blkcg kfuncs, silently reports SKIP. Since nothing is printed, CI shows a green skip and a broken kfunc goes unnoticed indefinitely.
Could the errno be printed before skipping, and "prerequisite absent" (ENOENT/ENODEV/EPERM on /dev/loop-control) be distinguished from a real failure (EBUSY, ENOSPC, ENOMEM)?
[ ... ]
tools/testing/selftests/bpf/progs/cgroup_iter_io.c
Quoted 5 lines
tools/testing/selftests/bpf/progs/cgroup_iter_io.c new file +0
-0
[ ... ]
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/32073368069