Skip to contentSkip to conversation

block: expose blkcg io.stat to BPF

PATCH In review Ziyang Men 4 participants 14 messages Aug 17 – Aug 19

Subsystems (6)
lore
Share & tools
lore

Ziyang Men

author
#0 lore raw

v2 cover block: expose blkcg io.stat to BPF

Collecting cgroup statistics is expensive: the existing method is to open and parse a cgroup file for every cgroup of interest. The memory controller already has an efficient alternative through BPF; this series extends that model to the block I/O controller's per-device io.stat.

Patch 1 adds the blkcg flush and per-device iterator.

Patch 2 adds a selftest that performs direct I/O on a private loop device and compares every counter read by BPF with io.stat.

===
Changes since v1.

- Drop the blkcg acquire/release helpers. The program reads the io controller's css from cgroup->subsys[] under RCU instead.
- Remove the kfuncs that read the blkg device ID and counters; BPF programs can read those fields with BPF_CORE_READ().
- Keep the flush sleepable. It now takes a struct cgroup * and pins its CSS internally.
- Make the blkg iterator take an RCU-protected CSS.

Tested on VM with v7.2-rc5.

Ziyang Men (2):
block: add BPF kfuncs to read blkcg io.stat selftests/bpf: add test for blkcg io.stat BPF kfuncs

 MAINTAINERS                                   |   1 +
 block/Makefile                                |   3 +
 block/blk-cgroup.c                            |   2 +-
 block/blk-cgroup.h                            |   1 +
 block/bpf_blkcg.c                             | 154 ++++++++++
 tools/testing/selftests/bpf/cgroup_iter_io.h  |  17 ++
 tools/testing/selftests/bpf/config            |   1 +
 .../selftests/bpf/prog_tests/cgroup_iter_io.c | 277 ++++++++++++++++++
 .../selftests/bpf/progs/cgroup_iter_io.c      |  99 +++++++
 9 files changed, 554 insertions(+), 1 deletion(-)
 create mode 100644 block/bpf_blkcg.c
 create mode 100644 tools/testing/selftests/bpf/cgroup_iter_io.h
 create mode 100644 tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c
 create mode 100644 tools/testing/selftests/bpf/progs/cgroup_iter_io.c
Signature · 1 line
2.53.0-Meta
author
#1 lore raw

PATCH v2 1/2 block: add BPF kfuncs to read blkcg io.stat

Collecting cgroup statistics is expensive: the existing method is to open and parse a cgroup file for every cgroup of interest. memcg already has an efficient alternative through BPF; this series extends that idea to block. This series exposes the block I/O controller's per-device io.stat to BPF.

The flush is sleepable and takes a cgroup. It pins the I/O css before leaving RCU, then flushes it. The iterator takes the RCU-protected css and remains block-specific because each block device has its own blkg. The behavior mirrows the blkcg_print_stat().

The blkg device iterator take a RCU css.

No kfuncs are added to read the blkcg counters since user can read it using the BPF_CORE_READ.

Assisted-by Claude:claude-opus-5
---
 MAINTAINERS        |   1 +
 block/Makefile     |   3 +
 block/blk-cgroup.c |   2 +-
 block/blk-cgroup.h |   1 +
 block/bpf_blkcg.c  | 154 +++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 160 insertions(+), 1 deletion(-)
 create mode 100644 block/bpf_blkcg.c
MAINTAINERS +1 -0
index 2f9472c1a090..87c56e955577 100644
@@ -6617,6 +6617,7 @@ F:	block/blk-cgroup.c
 F:	block/blk-iocost.c
 F:	block/blk-iolatency.c
 F:	block/blk-throttle.c
+F:	block/bpf_blkcg.c
 F:	include/linux/blk-cgroup.h
 
 CONTROL GROUP - CPUSET
block/Makefile +3 -0
index e7bd320e3d69..572e49988c8e 100644
@@ -17,6 +17,9 @@ obj-$(CONFIG_BLK_ERROR_INJECTION) += error-injection.o
 obj-$(CONFIG_BLK_DEV_BSG_COMMON) += bsg.o
 obj-$(CONFIG_BLK_DEV_BSGLIB)	+= bsg-lib.o
 obj-$(CONFIG_BLK_CGROUP)	+= blk-cgroup.o
+ifdef CONFIG_BPF_SYSCALL
+obj-$(CONFIG_BLK_CGROUP)	+= bpf_blkcg.o
+endif
 obj-$(CONFIG_BLK_CGROUP_RWSTAT)	+= blk-cgroup-rwstat.o
 obj-$(CONFIG_BLK_CGROUP_FC_APPID) += blk-cgroup-fc-appid.o
 obj-$(CONFIG_BLK_DEV_THROTTLING)	+= blk-throttle.o
block/blk-cgroup.c +1 -1
index d9676126c5b5..8d538ad4e861 100644
@@ -1086,7 +1086,7 @@ static void blkcg_rstat_flush(struct cgroup_subsys_state *css, int cpu)
  * flushing the root cgroup's stats by explicitly filling in the iostat
  * with disk level statistics.
  */
-static void blkcg_fill_root_iostats(void)
+void blkcg_fill_root_iostats(void)
 {
 	struct class_dev_iter iter;
 	struct device *dev;
block/blk-cgroup.h +1 -0
index 615390f751aa..8c9c2a1adfaa 100644
@@ -205,6 +205,7 @@ void blkcg_deactivate_policy(struct gendisk *disk,
 			     const struct blkcg_policy *pol);
 
 const char *blkg_dev_name(struct blkcg_gq *blkg);
+void blkcg_fill_root_iostats(void);
 void blkcg_print_blkgs(struct seq_file *sf, struct blkcg *blkcg,
 		       u64 (*prfill)(struct seq_file *,
 				     struct blkg_policy_data *, int),
block/bpf_blkcg.c new file +154 -0

154 lines not on this page · raw message

Signature · 1 line
2.53.0-Meta
#2 lore raw

Re: [PATCH v2 1/2] block: add BPF kfuncs to read blkcg io.stat

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Concurrent execution of blkcg_fill_root_iostats() corrupts the u64_stats_sync sequence counter, leading to reader deadlocks. --

--- Patch [1]: [PATCH v2 1/2] block: add BPF kfuncs to read blkcg io.stat --- commit bdaf96b752ce960b924c3be337708d377f83207c
Author: Ziyang Men <[email protected]>

block: add BPF kfuncs to read blkcg io.stat

This commit introduces BPF kfuncs to allow reading per-device block I/O controller statistics. It exposes blkcg_fill_root_iostats and provides bpf_blkcg_flush_stats and iterator functions for blkg.

Quoted 5 lines
block/bpf_blkcg.c new file +0 -0
new file mode 100644
index 0000000000000..25c809f5091c2

[ ... ]

Quoted 6 lines
block/bpf_blkcg.c new file +6 -0
+__bpf_kfunc void bpf_blkcg_flush_stats(struct cgroup *cgrp)
+{
+	struct cgroup_subsys_state *css;
+
+	/* Pin the css for the sleepable flush. */
+	rcu_read_lock();

block/bpf_blkcg.c line 36

Quoted 10 lines
block/bpf_blkcg.c new file +10 -0
+	css = cgroup_css(cgrp, &io_cgrp_subsys);
+	if (css && !css_tryget(css))
+		css = NULL;
+	rcu_read_unlock();
+
+	if (!css)
+		return;
+
+	if (!css->parent)
+		blkcg_fill_root_iostats();

[Severity: High]
This isn't a bug introduced by this patch, but can concurrent executions here lead to sequence counter corruption and reader deadlocks on 32-bit systems?

In block/blk-cgroup.c, blkcg_fill_root_iostats() loops over block devices and updates statistics without an overarching writer lock:

blkcg_fill_root_iostats() {
    ...
    flags = u64_stats_update_begin_irqsave(&blkg->iostat.sync);
    blkg_iostat_set(&blkg->iostat.cur, &tmp);
    u64_stats_update_end_irqrestore(&blkg->iostat.sync, flags);
    ...
}

On 32-bit architectures, u64_stats_update_begin_irqsave() maps to write_seqcount_begin(), which requires external mutual exclusion to prevent concurrent writers from corrupting the sequence counter. If the counter becomes permanently odd, readers using read_seqcount_begin() will spin infinitely.

While this bug existed previously via concurrent reads of the root cgroup's io.stat sysfs file, exposing blkcg_fill_root_iostats() via this sleepable BPF kfunc allows BPF programs to easily and repeatedly trigger this lockless code path simultaneously across multiple CPUs.

Quoted 5 lines
block/bpf_blkcg.c new file +5 -0
+	else
+		css_rstat_flush(css);
+
+	css_put(css);
+}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
#3 lore raw

Re: [PATCH v2 1/2] block: add BPF kfuncs to read blkcg io.stat

In reply to Ziyang Men
This isn't a bug, but there are a few rough spots in the changelog: "mirrows" should be "mirrors", "The blkg device iterator take a RCU css." reads as a fragment that restates the earlier sentence…
Quoted 6 lines

The behavior mirrows the blkcg_print_stat().

The blkg device iterator take a RCU css.

No kfuncs are added to read the blkcg counters since user can read it using the BPF_CORE_READ.

This isn't a bug, but there are a few rough spots in the changelog:
"mirrows" should be "mirrors", "The blkg device iterator take a RCU css." reads as a fragment that restates the earlier sentence about the iterator taking the RCU-protected css, and "since user can read it" is missing an article before "user".

Could the changelog also say why a new flush kfunc is needed rather than reusing the existing css_rstat_flush() kfunc -- is the point specifically the root-cgroup blkcg_fill_root_iostats() path?

Quoted 14 lines
block/Makefile +3 -0
index e7bd320e3d69..572e49988c8e 100644
@@ -17,6 +17,9 @@ obj-$(CONFIG_BLK_ERROR_INJECTION) += error-injection.o
 obj-$(CONFIG_BLK_DEV_BSG_COMMON) += bsg.o
 obj-$(CONFIG_BLK_DEV_BSGLIB)	+= bsg-lib.o
 obj-$(CONFIG_BLK_CGROUP)	+= blk-cgroup.o
+ifdef CONFIG_BPF_SYSCALL
+obj-$(CONFIG_BLK_CGROUP)	+= bpf_blkcg.o
+endif
 obj-$(CONFIG_BLK_CGROUP_RWSTAT)	+= blk-cgroup-rwstat.o
 obj-$(CONFIG_BLK_CGROUP_FC_APPID) += blk-cgroup-fc-appid.o
 obj-$(CONFIG_BLK_DEV_THROTTLING)	+= blk-throttle.o

[ ... ]

Quoted 37 lines
block/bpf_blkcg.c new file +31 -0
new file mode 100644
index 000000000000..25c809f5091c
@@ -0,0 +1,154 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Block I/O Controller-related BPF kfuncs and auxiliary code
+ */
+
+#include "blk-cgroup.h"
+
+#include <linux/bpf.h>
+#include <linux/btf_ids.h>
+#include <linux/rculist.h>
+
+__bpf_kfunc_start_defs();
+
+/**
+ * bpf_blkcg_flush_stats - Flush a block cgroup's io statistics
+ * @cgrp: cgroup to flush
+ *
+ * Propagate I/O statistics up the cgroup tree. Root statistics come from
+ * block devices and include all cgroups' I/O.
+ */
+__bpf_kfunc void bpf_blkcg_flush_stats(struct cgroup *cgrp)
+{
+	struct cgroup_subsys_state *css;
+
+	/* Pin the css for the sleepable flush. */
+	rcu_read_lock();
+	css = cgroup_css(cgrp, &io_cgrp_subsys);
+	if (css && !css_tryget(css))
+		css = NULL;
+	rcu_read_unlock();
+

block/bpf_blkcg.c line 41

Quoted 10 lines
block/bpf_blkcg.c new file +10 -0
+	if (!css)
+		return;
+
+	if (!css->parent)
+		blkcg_fill_root_iostats();
+	else
+		css_rstat_flush(css);
+
+	css_put(css);
+}

This isn't a bug, but would it be simpler for bpf_blkcg_flush_stats() to take the io css directly, so the same handle feeds bpf_iter_blkg_new()? The two entry points in this file take different cgroup handles:
bpf_blkcg_flush_stats() takes struct cgroup* and resolves the css internally with cgroup_css()/css_tryget(), while bpf_iter_blkg_new() below takes struct cgroup_subsys_state* directly. A program that wants to flush and then iterate therefore needs both handle types for the same cgroup. Taking struct cgroup_subsys_state* in both, as mm/bpf_memcontrol.c does for its memcg handles, would make them composable and would drop the cgroup_css() lookup.

Quoted 103 lines
block/bpf_blkcg.c new file +103 -0
+
+struct bpf_iter_blkg {
+	__u64 __opaque[2];
+} __aligned(8);
+
+struct bpf_iter_blkg_kern {
+	struct blkcg *blkcg;
+	struct blkcg_gq *pos;
+} __aligned(8);
+
+/**
+ * bpf_iter_blkg_new - Start iterating a block cgroup's per-device blkgs
+ * @it: iterator to initialize
+ * @css: the io controller's css
+ *
+ * Each blkg holds one device's io.stat counters. Offline blkgs are skipped.
+ * A blkg without a disk can be returned. Must run under RCU.
+ *
+ * Return: 0 on success, -EINVAL if @css is not the io controller's.
+ */
+__bpf_kfunc int bpf_iter_blkg_new(struct bpf_iter_blkg *it,
+				  struct cgroup_subsys_state *css)
+{
+	struct bpf_iter_blkg_kern *kit = (void *)it;
+
+	BUILD_BUG_ON(sizeof(struct bpf_iter_blkg_kern) > sizeof(struct bpf_iter_blkg));
+	BUILD_BUG_ON(__alignof__(struct bpf_iter_blkg_kern) !=
+		     __alignof__(struct bpf_iter_blkg));
+
+	kit->pos = NULL;
+
+	if (css->ss != &io_cgrp_subsys) {
+		kit->blkcg = NULL;
+		return -EINVAL;
+	}
+
+	kit->blkcg = css_to_blkcg(css);
+	return 0;
+}
+
+/**
+ * bpf_iter_blkg_next - Return the next online blkg of the iterated block cgroup
+ * @it: iterator
+ *
+ * Return: the next online blkg, or NULL when the walk is done.
+ */
+__bpf_kfunc struct blkcg_gq *bpf_iter_blkg_next(struct bpf_iter_blkg *it)
+{
+	struct bpf_iter_blkg_kern *kit = (void *)it;
+	struct blkcg_gq *blkg = kit->pos;
+	struct hlist_node *node;
+
+	if (!kit->blkcg)
+		return NULL;
+
+	if (!blkg)
+		node = rcu_dereference(hlist_first_rcu(&kit->blkcg->blkg_list));
+	else
+		node = rcu_dereference(hlist_next_rcu(&blkg->blkcg_node));
+
+	/* Skip offline blkgs, matching io.stat. */
+	while (node) {
+		blkg = hlist_entry(node, struct blkcg_gq, blkcg_node);
+		/* A race only changes whether this blkg is returned. */
+		if (data_race(blkg->online)) {
+			kit->pos = blkg;
+			return blkg;
+		}
+		node = rcu_dereference(hlist_next_rcu(&blkg->blkcg_node));
+	}
+
+	/* The iterator must keep returning NULL after completion. */
+	kit->pos = NULL;
+	kit->blkcg = NULL;
+	return NULL;
+}
+
+/**
+ * bpf_iter_blkg_destroy - Tear down a blkg iterator
+ * @it: iterator
+ */
+__bpf_kfunc void bpf_iter_blkg_destroy(struct bpf_iter_blkg *it)
+{
+}
+
+__bpf_kfunc_end_defs();
+
+BTF_KFUNCS_START(bpf_blkcg_kfuncs)
+BTF_ID_FLAGS(func, bpf_blkcg_flush_stats, KF_SLEEPABLE)
+
+BTF_ID_FLAGS(func, bpf_iter_blkg_new,
+	     KF_ITER_NEW | KF_RCU | KF_RCU_PROTECTED)
+BTF_ID_FLAGS(func, bpf_iter_blkg_next, KF_ITER_NEXT | KF_RET_NULL)
+BTF_ID_FLAGS(func, bpf_iter_blkg_destroy, KF_ITER_DESTROY)
+BTF_KFUNCS_END(bpf_blkcg_kfuncs)
+
+static const struct btf_kfunc_id_set bpf_blkcg_kfunc_set = {
+	.owner		= THIS_MODULE,
+	.set		= &bpf_blkcg_kfuncs,
+};
+
+static int __init bpf_blkcg_init(void)
+{

block/bpf_blkcg.c line 154

Quoted 10 lines
block/bpf_blkcg.c new file +10 -0
+	int err;
+
+	err = register_btf_kfunc_id_set(BPF_PROG_TYPE_UNSPEC,
+					&bpf_blkcg_kfunc_set);
+	if (err)
+		pr_warn("error while registering bpf blkcg kfuncs: %d\n", err);
+
+	return err;
+}
+late_initcall(bpf_blkcg_init);

This isn't a bug, but could a selftest under tools/testing/selftests/bpf/ be added alongside these kfuncs to lock down the RCU-protected iterator usage? Four new kfuncs are registered for BPF_PROG_TYPE_UNSPEC (all program types) with no accompanying selftest and no in-tree consumer, so nothing exercises the iterator's KF_RCU_PROTECTED contract, the offline-blkg skip, or the root-vs-non-root flush split. Comparable additions -- mm/bpf_memcontrol.c and the bpf_iter_css family -- landed with tests under tools/testing/selftests/bpf/, which is also what pins the intended usage pattern (bpf_rcu_read_lock() around new/next/destroy) for future readers.

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/32073368069

maintainer
#4 lore raw

Re: [PATCH v2 1/2] block: add BPF kfuncs to read blkcg io.stat

In reply to Ziyang Men

Hello,

block/bpf_blkcg.c line 21

Quoted 8 lines · Ziyang Men
block/bpf_blkcg.c new file +8 -0
+/**
+ * bpf_blkcg_flush_stats - Flush a block cgroup's io statistics
+ * @cgrp: cgroup to flush
+ *
+ * Propagate I/O statistics up the cgroup tree. Root statistics come from
+ * block devices and include all cgroups' I/O.
+ */
+__bpf_kfunc void bpf_blkcg_flush_stats(struct cgroup *cgrp)

Either @css or @blkcg, probably the latter so that each operation doesn't have to verify subsys again. I'd make this and the cpu ppatches into a single series. Add blkcg cast there too.

Thanks.

Signature · 1 line
tejun
author
#5 lore raw

Re: [PATCH v2 1/2] block: add BPF kfuncs to read blkcg io.stat

Hi Tejun,

Quoted 15 lines · Tejun Heo

Hello,

block/bpf_blkcg.c line 21

Quoted 8 lines · Ziyang Men
block/bpf_blkcg.c new file +8 -0
+/**
+ * bpf_blkcg_flush_stats - Flush a block cgroup's io statistics
+ * @cgrp: cgroup to flush
+ *
+ * Propagate I/O statistics up the cgroup tree. Root statistics come from
+ * block devices and include all cgroups' I/O.
+ */
+__bpf_kfunc void bpf_blkcg_flush_stats(struct cgroup *cgrp)

Either @css or @blkcg, probably the latter so that each operation doesn't have to verify subsys again. I'd make this and the cpu ppatches into a single series. Add blkcg cast there too.

Ok I will add the casts bpf_get_blkcg() and bpf_put_blkcg(), and remove the check of css->ss in above functoins.

Next submission of this series will be together with the cpu one.

Quoted 5 lines

Thanks.

Signature · 1 line
tejun

Thanks for your review!

Best,
Ziyang

maintainer
#6 lore raw

Re: [PATCH v2 1/2] block: add BPF kfuncs to read blkcg io.stat

Quoted 2 lines · Ziyang Men

Ok I will add the casts bpf_get_blkcg() and bpf_put_blkcg(), and remove the check of css->ss in above functoins.

RCU_PROTECTED is probably better than get/put().

Thanks.

Signature · 1 line
tejun
author
#7 lore raw

Re: [PATCH v2 1/2] block: add BPF kfuncs to read blkcg io.stat

Hi Tejun,

Quoted 5 lines · Tejun Heo
Quoted 2 lines · Ziyang Men

Ok I will add the casts bpf_get_blkcg() and bpf_put_blkcg(), and remove the check of css->ss in above functoins.

RCU_PROTECTED is probably better than get/put().

Yes we can make the blkcg* RCU_PROTECTED but then the bpf_blkcg_flush_stats() needs to take the @cgroup* since it is SLEEPABLE. Does this work for you?

Best,
Ziyang

Quoted 5 lines

Thanks.

Signature · 1 line
tejun
maintainer
#8 lore raw

Re: [PATCH v2 1/2] block: add BPF kfuncs to read blkcg io.stat

Hello,

Quoted 9 lines · Ziyang Men
Quoted 5 lines · Tejun Heo
Quoted 2 lines · Ziyang Men

Ok I will add the casts bpf_get_blkcg() and bpf_put_blkcg(), and remove the check of css->ss in above functoins.

RCU_PROTECTED is probably better than get/put().

Yes we can make the blkcg* RCU_PROTECTED but then the bpf_blkcg_flush_stats() needs to take the @cgroup* since it is SLEEPABLE. Does this work for you?

Do we really need that function? css_rstat_flush() is already exposed and css can already be obtained, right? blkcg_fill_root_iostat() is just accessing system-wide bdev stats, which you probably don't need to begin with or if necessary just access through bdevs.

Thanks.

Signature · 1 line
tejun
author
#9 lore raw

Re: [PATCH v2 1/2] block: add BPF kfuncs to read blkcg io.stat

Hi Tejun,

Quoted 10 lines · Tejun Heo

Hello,

Quoted 7 lines · Ziyang Men
Quoted 5 lines · Tejun Heo
Quoted 2 lines · Ziyang Men

Ok I will add the casts bpf_get_blkcg() and bpf_put_blkcg(), and remove the check of css->ss in above functoins.

RCU_PROTECTED is probably better than get/put().

Quoted 7 lines
Quoted 2 lines

Yes we can make the blkcg* RCU_PROTECTED but then the bpf_blkcg_flush_stats() needs to take the @cgroup* since it is SLEEPABLE. Does this work for you?

Do we really need that function? css_rstat_flush() is already exposed and css can already be obtained, right? blkcg_fill_root_iostat() is just accessing system-wide bdev stats, which you probably don't need to begin with or if necessary just access through bdevs.

Yes the css_rstat_flush() is exposed but currently it is marked as SLEEPABLE, so it doesn't accept a RCU_PROTECTED css we proposed. This is not a problem for cpu ones since &cgrp->self is a trusted pointer.

Best,
Ziyang

Quoted 5 lines

Thanks.

Signature · 1 line
tejun
maintainer
#10 lore raw

Re: [PATCH v2 1/2] block: add BPF kfuncs to read blkcg io.stat

Hello,

Quoted 3 lines · Ziyang Men

Yes the css_rstat_flush() is exposed but currently it is marked as SLEEPABLE, so it doesn't accept a RCU_PROTECTED css we proposed. This is not a problem for cpu ones since &cgrp->self is a trusted pointer.

Yeah, bummer, so we get ref on cgroups but the subsys pointers are RCU, so that means those refs are only RCU protected. I guess we can't avoid a kfunc to get acquire a subsys css from cgroup. I wish css_rstat_flush() didn't schedule but it does need that cond_resched().

Thanks.

Signature · 1 line
tejun
author
#11 lore raw

Re: [PATCH v2 1/2] block: add BPF kfuncs to read blkcg io.stat

Hi Tejun,

Quoted 11 lines · Tejun Heo

Hello,

Quoted 3 lines · Ziyang Men

Yes the css_rstat_flush() is exposed but currently it is marked as SLEEPABLE, so it doesn't accept a RCU_PROTECTED css we proposed. This is not a problem for cpu ones since &cgrp->self is a trusted pointer.

Yeah, bummer, so we get ref on cgroups but the subsys pointers are RCU, so that means those refs are only RCU protected. I guess we can't avoid a kfunc to get acquire a subsys css from cgroup. I wish css_rstat_flush() didn't schedule but it does need that cond_resched().

Yeah that would be a pity. I will add kfuncs bpf_cgroup_css() and bpf_css_release() to handle the reference for the css.

The bpf_blkcg_flush_stats() will be removed as well.

Thanks,
Ziyang

Quoted 5 lines

Thanks.

Signature · 1 line
tejun
author
#12 lore raw

PATCH v2 2/2 selftests/bpf: add test for blkcg io.stat BPF kfuncs

In reply to Ziyang Men

Add cgroup_iter_io to test the blkcg io.stat BPF kfuncs. The BPF program flushes the statistics, looks up the I/O css under RCU, walks its blkgs, and reads one device's counters with BPF_CORE_READ().

The test performs O_DIRECT I/O on a private loop device. It checks the device ID and counters against io.stat.

Assisted-by Claude:claude-opus-5
---
 tools/testing/selftests/bpf/cgroup_iter_io.h  |  17 ++
 tools/testing/selftests/bpf/config            |   1 +
 .../selftests/bpf/prog_tests/cgroup_iter_io.c | 277 ++++++++++++++++++
 .../selftests/bpf/progs/cgroup_iter_io.c      |  99 +++++++
 4 files changed, 394 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/cgroup_iter_io.h
 create mode 100644 tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c
 create mode 100644 tools/testing/selftests/bpf/progs/cgroup_iter_io.c
tools/testing/selftests/bpf/cgroup_iter_io.h new file +17 -0
new file mode 100644
index 000000000000..f4bbaaccdf71
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2025 Meta Platforms, Inc. and affiliates. */
+#ifndef __CGROUP_ITER_IO_H
+#define __CGROUP_ITER_IO_H
+
+struct io_query {
+	/* one device's io.stat counters */
+	__u64 rbytes;
+	__u64 wbytes;
+	__u64 rios;
+	__u64 wios;
+	__u64 dbytes;
+	__u64 dios;
+	__u64 dev;	/* dev_t of the device the counters belong to */
+};
+
+#endif /* __CGROUP_ITER_IO_H */
tools/testing/selftests/bpf/config +1 -0
index ea7044f30adc..270e6bf9194d 100644
@@ -1,3 +1,4 @@
+CONFIG_BLK_CGROUP=y
 CONFIG_BLK_DEV_LOOP=y
 CONFIG_BOOTPARAM_HARDLOCKUP_PANIC=y
 CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC=1
tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c new file +277 -0

277 lines not on this page · raw message

tools/testing/selftests/bpf/progs/cgroup_iter_io.c new file +99 -0

99 lines not on this page · raw message

Signature · 1 line
2.53.0-Meta
#13 lore raw

Re: [PATCH v2 2/2] selftests/bpf: add test for blkcg io.stat BPF kfuncs

Does the err: cleanup path leak the loop device?
Quoted 5 lines
tools/testing/selftests/bpf/cgroup_iter_io.h new file +0 -0
new file mode 100644
index 000000000000..f4bbaaccdf71

[ ... ]

Quoted 4 lines
tools/testing/selftests/bpf/config +0 -0
index ea7044f30adc..270e6bf9194d 100644

[ ... ]

Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c new file +0 -0
new file mode 100644
index 000000000000..5d27e5d28379
@@ -0,0 +1,277 @@

[ ... ]

Quoted 31 lines
tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c new file +31 -0
+/* Set up a loop device for cgroup-charged I/O. */
+static int loop_setup(char *loop_path, size_t sz, int *ctl_fd, int *loop_fd,
+		      int *back_fd)
+{
+	char back_path[] = "/tmp/cgroup_iter_io.XXXXXX";
+	int nr;
+
+	*ctl_fd = *loop_fd = *back_fd = -1;
+
+	*ctl_fd = open("/dev/loop-control", O_RDWR | O_CLOEXEC);
+	if (*ctl_fd < 0)
+		return -1;
+
+	nr = ioctl(*ctl_fd, LOOP_CTL_GET_FREE);
+	if (nr < 0)
+		goto err;
+	snprintf(loop_path, sz, "/dev/loop%d", nr);
+
+	*back_fd = mkstemp(back_path);
+	if (*back_fd < 0)
+		goto err;
+	unlink(back_path);
+	if (ftruncate(*back_fd, (off_t)IO_SIZE * 4))
+		goto err;
+
+	*loop_fd = open(loop_path, O_RDWR | O_CLOEXEC);
+	if (*loop_fd < 0)
+		goto err;
+	if (ioctl(*loop_fd, LOOP_SET_FD, *back_fd))
+		goto err;
+

tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c line 78

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c new file +10 -0
+	return 0;
+err:
+	if (*loop_fd >= 0)
+		close(*loop_fd);
+	if (*back_fd >= 0)
+		close(*back_fd);
+	close(*ctl_fd);
+	*ctl_fd = *loop_fd = *back_fd = -1;
+	return -1;
+}

Does the err: cleanup path leak the loop device?

LOOP_CTL_GET_FREE can allocate a new loop device when no unbound device exists, and nr is stored. Every failure after that point (mkstemp, ftruncate, open(/dev/loopN), LOOP_SET_FD) leaves that device behind.

The caller jumps straight to cleanup_cgroup_fd on loop_setup() failure, bypassing cleanup_loop, so loop_teardown() never runs and the device remains allocated.

Recording nr and calling ioctl(*ctl_fd, LOOP_CTL_REMOVE, nr) on the err:
path would fix this. Additionally, loop_teardown() ignores the return of both LOOP_CLR_FD and LOOP_CTL_REMOVE, so a device left in Lo_bound state also leaks silently.

Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c new file +5 -0
+/* Read @dev's io.stat counters. @dev uses kernel dev_t encoding. */
+static int parse_io_stat(int cgroup_fd, __u64 dev, struct io_query *out)
+{
+	unsigned int want_maj = dev >> 20, want_min = dev & ((1U << 20) - 1);
+	char buf[4096], *line, *saveptr;

tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c line 138

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c new file +10 -0
+	int fd, n, ret = -1;
+
+	fd = openat(cgroup_fd, "io.stat", O_RDONLY);
+	if (fd < 0)
+		return -1;
+	n = read(fd, buf, sizeof(buf) - 1);
+	close(fd);
+	if (n <= 0)
+		return -1;
+	buf[n] = '\0';

Will a single 4095-byte read() always capture the complete io.stat output?

io.stat is a seq_file with one line per blkg, and seq_read_iter() copies with byte granularity, not record boundaries. Two failure modes:

1. If the cgroup's blkg list produces more than 4095 bytes, the loop device's line may not be in the buffer, and parse_io_stat() returns -1.

2. The cut can land mid-line. A truncated line like "7:0 rbytes=4194304 wbytes=41" satisfies the '< 2' check below and is accepted with the remaining counters silently zero, producing a bogus counter mismatch against the kfunc rather than a parse error.

Reading in a loop until read() returns 0, or at least treating a non- terminated final line as an error, would avoid both.

Quoted 14 lines
tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c new file +14 -0
+	for (line = strtok_r(buf, "\n", &saveptr); line;
+	     line = strtok_r(NULL, "\n", &saveptr)) {
+		unsigned long long rb = 0, wb = 0, ri = 0, wi = 0, db = 0, di = 0;
+		unsigned int maj, min;
+
+		/* Only the device id is required; missing counters stay zero. */
+		if (sscanf(line,
+			   "%u:%u rbytes=%llu wbytes=%llu rios=%llu wios=%llu dbytes=%llu dios=%llu",
+			   &maj, &min, &rb, &wb, &ri, &wi, &db, &di) < 2)
+			continue;
+		if (maj != want_maj || min != want_min)
+			continue;
+
+		out->rbytes = rb;

tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c line 163

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c new file +10 -0
+		out->wbytes = wb;
+		out->rios = ri;
+		out->wios = wi;
+		out->dbytes = db;
+		out->dios = di;
+		ret = 0;
+		break;
+	}
+	return ret;
+}

Should the partial conversion check be stricter?

Accepting '< 2' and then returning success means a line that matched the device but whose counters were not parsed is reported as all-zero rather than an error. The caller immediately uses strict equality checks:

	ASSERT_EQ(q->dbytes, filev.dbytes, "dbytes");

so a parse problem is reported as a counter mismatch against the kfunc, pointing at the kernel instead of the parser.

This is reachable because blkcg_print_one_stat() emits the device name unconditionally but gates the counter group with:

	if (rbytes || wbytes || rios || wios)
		seq_printf(s, "rbytes=%llu wbytes=%llu ... dios=%llu", ...);

A cgroup whose only traffic to a device was discard produces "MAJ:MIN " with no key=value pairs. sscanf() returns 2, parse_io_stat() returns success with dbytes=dios=0, and the match subtest fails on "dbytes" even though the kernel is correct.

Requiring all eight conversions (== 8) once maj:min matches would surface the real problem.

[ ... ]

tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c line 182

Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/cgroup_iter_io.c new file +4 -0
+	if (loop_setup(loop_path, sizeof(loop_path), &ctl_fd, &loop_fd, &back_fd)) {
+		test__skip();	/* needs root + CONFIG_BLK_DEV_LOOP */
+		goto cleanup_cgroup_fd;
+	}

A subsystem pattern flags this as potentially concerning:

loop_setup() collapses eight distinct failures into a single -1, and the caller turns every one of them into a bare test__skip() with no runtime reason. The source comment says "needs root + CONFIG_BLK_DEV_LOOP", but the same return value is produced by: open("/dev/loop-control") failing, LOOP_CTL_GET_FREE failing, mkstemp() failing, ftruncate() failing (ENOSPC on a small tmpfs /tmp), open("/dev/loopN") failing, and LOOP_SET_FD failing.

The LOOP_SET_FD case is a genuine race: LOOP_CTL_GET_FREE is not a reservation, so any concurrent consumer (another test_progs worker under -j, udev, systemd, losetup) can bind /dev/loopN between GET_FREE and SET_FD, and SET_FD then returns EBUSY. The result is that the whole test, including all five subtests exercising the new blkcg kfuncs, silently reports SKIP. Since nothing is printed, CI shows a green skip and a broken kfunc goes unnoticed indefinitely.

Could the errno be printed before skipping, and "prerequisite absent" (ENOENT/ENODEV/EPERM on /dev/loop-control) be distinguished from a real failure (EBUSY, ENOSPC, ENOMEM)?

[ ... ]

tools/testing/selftests/bpf/progs/cgroup_iter_io.c

Quoted 5 lines
tools/testing/selftests/bpf/progs/cgroup_iter_io.c new file +0 -0
new file mode 100644
index 000000000000..2cd538068987

[ ... ]

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/32073368069