Skip to contentSkip to conversation

A tentative patch to bypass REQ_META in blk throttle

PATCH In review Benlong Zhang 3 participants 5 messages Apr 20, 2018 – May 2, 2018

Subsystems (2)
Block layer cgroup blkio
lore
Share & tools
lore

Benlong Zhang

author
#0 lore raw

PATCH 1/1 A tentative patch to bypass REQ_META in blk throttle

One problem with cgwb is how fs should treat metadata bios. For example in xfs, the log might be partially stuck in one group, leaving threads in other groups waiting for too long. Please refer to the linux-xfs discussion:
"[PATCH V2] xfs: implement cgroup writeback support"

One approach is to correctly tag bio->bi_css from within the filesystems (for xfs log should be blkcg_root), and the other is to skip them, but relies on REQ_META being set.

It works with xfs on a cgwb porting implementation to 3.10.0. But really not sure about other filesystems...

---
 block/blk-throttle.c | 4 ++++
 1 file changed, 4 insertions(+)
block/blk-throttle.c +4 -0
index c5a1316..e2abf6e 100644
@@ -2159,6 +2159,10 @@ bool blk_throtl_bio(struct request_queue *q, struct blkcg_gq *blkg,
 	if (bio_flagged(bio, BIO_THROTTLED) || !tg->has_rules[rw])
 		goto out;
 
+	/* bypass REQ_META for fs global resource */
+	if (bio->bi_opf & REQ_META)
+		goto out;
+
 	spin_lock_irq(q->queue_lock);
 
 	throtl_update_latency_buckets(td);
Signature · 1 line
1.8.3.1
maintainer
#1 lore raw

Hello,

Quoted 6 lines · Benlong Zhang

One problem with cgwb is how fs should treat metadata bios. For example in xfs, the log might be partially stuck in one group, leaving threads in other groups waiting for too long. Please refer to the linux-xfs discussion:
"[PATCH V2] xfs: implement cgroup writeback support"

Quoted 6 lines

One approach is to correctly tag bio->bi_css from within the filesystems (for xfs log should be blkcg_root), and the other is to skip them, but relies on REQ_META being set.

It works with xfs on a cgwb porting implementation to 3.10.0. But really not sure about other filesystems...

Yeah, Josef (cc'd) is working on a similar approach but in a more generic way.

Thanks.

Signature · 1 line
tejun
#2 lore raw
Quoted 13 lines · Tejun Heo

Hello,

Quoted 10 lines · Benlong Zhang

One problem with cgwb is how fs should treat metadata bios. For example in xfs, the log might be partially stuck in one group, leaving threads in other groups waiting for too long. Please refer to the linux-xfs discussion:
"[PATCH V2] xfs: implement cgroup writeback support"

One approach is to correctly tag bio->bi_css from within the filesystems (for xfs log should be blkcg_root), and the other is to skip them, but relies on REQ_META being set.

Quoted 6 lines
Quoted 2 lines

It works with xfs on a cgwb porting implementation to 3.10.0. But really not sure about other filesystems...

Yeah, Josef (cc'd) is working on a similar approach but in a more generic way.

I thought Josef's work had more to do with tracking/managing dirty metadata (i.e., driving writeback/reclaim and whatnot). Perhaps I just haven't followed it close enough... how does that relate to cgroup bio ownership?

The issue addressed by this patch (wrt to XFS) is that log I/O that happens to be submitted via a throttled cgroup ends up blocking unthrottled tasks because the log is essentially a global/shared mechanism. We therefore want to somehow exempt metadata I/O from throttling in that particular case.

Note that I'm not familiar enough with the block layer code to know whether this patch is the right way to do that, but I think (as noted in the commit log above) that not doing the default bio association if the fs hasn't (or associating with some root group) for metadata I/O would also be sufficient. Hm?

Brian

Quoted 4 lines

Thanks.

Signature · 1 line
tejun
maintainer
#3 lore raw

Hello, Brian.

Quoted 4 lines · Brian Foster

I thought Josef's work had more to do with tracking/managing dirty metadata (i.e., driving writeback/reclaim and whatnot). Perhaps I just haven't followed it close enough... how does that relate to cgroup bio ownership?

Oh, we've been working on it for a while now but haven't posted the patches yet. We should be able to post the patches in several weeks.

Thanks.

Signature · 1 line
tejun
author
#4 lore raw
In reply to Tejun Heo

Hi Tejun,

Quoted 13 lines · Tejun Heo

Hello,

Quoted 10 lines · Benlong Zhang

One problem with cgwb is how fs should treat metadata bios. For example in xfs, the log might be partially stuck in one group, leaving threads in other groups waiting for too long. Please refer to the linux-xfs discussion:
"[PATCH V2] xfs: implement cgroup writeback support"

One approach is to correctly tag bio->bi_css from within the filesystems (for xfs log should be blkcg_root), and the other is to skip them, but relies on REQ_META being set.

Quoted 5 lines
Quoted 2 lines

It works with xfs on a cgwb porting implementation to 3.10.0. But really not sure about other filesystems...

Yeah, Josef (cc'd) is working on a similar approach but in a more generic way.

Good to know and looking forward to the code.

Thank you for your reply!

Benlong

Quoted 5 lines

Thanks.

Signature · 1 line
tejun