Skip to contentSkip to conversation

blkcg: Optimize exit to user space

PATCH In review Thomas Gleixner 2 participants 2 messages Aug 13, 2025

Subsystems (3)
Block layer cgroup cgroup blkio
lore
Share & tools
lore

Thomas Gleixner

author
#0 lore raw

PATCH 1/1 blkcg: Optimize exit to user space

blkcg uses TIF_NOTIFY_RESUME to handle throttling on exit to user space. TIF_NOTIFY_RESUME is a multiplexing TIF bit, which is utilized by other entities as well.

This results in a unconditional blkcg_maybe_throttle_current() call for every invocation of resume_user_mode_work(), which is a pointless exercise as most of the time there is no throttling work to do.

Optimize this by doing a quick check of the throttling condition before invoking it.

Signed-off-by Thomas Gleixner <[email protected]>
Cc Tejun Heo <[email protected]>
Cc Josef Bacik <[email protected]>
Cc Jens Axboe <[email protected]>
---
 block/blk-cgroup.c         |    4 ++--
 include/linux/blk-cgroup.h |   10 +++++++++-
 2 files changed, 11 insertions(+), 3 deletions(-)
block/blk-cgroup.c +2 -2
@@ -2001,7 +2001,7 @@ static void blkcg_maybe_throttle_blkg(st
 }
 
 /**
- * blkcg_maybe_throttle_current - throttle the current task if it has been marked
+ * __blkcg_maybe_throttle_current - throttle the current task if it has been marked
  *
  * This is only called if we've been marked with set_notify_resume().  Obviously
  * we can be set_notify_resume() for reasons other than blkcg throttling, so we

@@ -2010,7 +2010,7 @@ static void blkcg_maybe_throttle_blkg(st
  * to be called by people willy-nilly as it will actually do the work to
  * throttle the task if it is setup for throttling.
  */
-void blkcg_maybe_throttle_current(void)
+void __blkcg_maybe_throttle_current(void)
 {
 	struct gendisk *disk = current->throttle_disk;
 	struct blkcg *blkcg;
include/linux/blk-cgroup.h +9 -1
@@ -14,6 +14,7 @@
  * 	              Nauman Rafique <[email protected]>
  */
 
+#include <linux/sched.h>
 #include <linux/types.h>
 
 struct bio;

@@ -26,7 +27,14 @@ struct gendisk;
 extern struct cgroup_subsys_state * const blkcg_root_css;
 
 void blkcg_schedule_throttle(struct gendisk *disk, bool use_memdelay);
-void blkcg_maybe_throttle_current(void);
+void __blkcg_maybe_throttle_current(void);
+
+static inline void blkcg_maybe_throttle_current(void)
+{
+	if (unlikely(current->throttle_disk))
+		__blkcg_maybe_throttle_current();
+}
+
 bool blk_cgroup_congested(void);
 void blkcg_pin_online(struct cgroup_subsys_state *blkcg_css);
 void blkcg_unpin_online(struct cgroup_subsys_state *blkcg_css);
maintainer Acked-by on v1 1/1
#1 lore raw
Quoted 15 lines · Thomas Gleixner

blkcg uses TIF_NOTIFY_RESUME to handle throttling on exit to user space. TIF_NOTIFY_RESUME is a multiplexing TIF bit, which is utilized by other entities as well.

This results in a unconditional blkcg_maybe_throttle_current() call for every invocation of resume_user_mode_work(), which is a pointless exercise as most of the time there is no throttling work to do.

Optimize this by doing a quick check of the throttling condition before invoking it.

Signed-off-by Thomas Gleixner <[email protected]>
Cc Tejun Heo <[email protected]>
Cc Josef Bacik <[email protected]>
Cc Jens Axboe <[email protected]>

...

block/blk-cgroup.c static void blkcg_maybe_throttle_blkg(st · line 2016

Quoted 5 lines
block/blk-cgroup.c +1 -1
-void blkcg_maybe_throttle_current(void)
+void __blkcg_maybe_throttle_current(void)
 {
 	struct gendisk *disk = current->throttle_disk;
 	struct blkcg *blkcg;

A nit: __blkcg_maybe_throttle_current() ends up doing another NULL check. Maybe drop that? Other than that,

Acked-by Tejun Heo <[email protected]>

Thanks.

Signature · 1 line
tejun