Skip to contentSkip to conversation

sched: fix throttle accounting with nested bandwidth limits

PATCH In review Josh Don 2 participants 4 messages Jun 15, 2023 – Jun 16, 2023 Scheduler

lore
Share & tools
lore

Josh Don

author
#0 lore raw

PATCH 1/1 sched: fix throttle accounting with nested bandwidth limits

This fixes two issues:
- throttled_clock should only be set on the group that is actually getting throttled
- self-throttled time should only be accounted on entry/exit to throttled state when we have nested limits

Fixes 88cb2868250c ("sched: add throttled time stat for throttled children")
Fixes 3ab150d011da ("sched: don't account throttle time for empty groups")
---
 kernel/sched/fair.c | 25 +++++++++++++------------
 1 file changed, 13 insertions(+), 12 deletions(-)
kernel/sched/fair.c +13 -12
index 0219cf870cef..a5fc825a8d70 100644
@@ -4787,6 +4787,7 @@ place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int initial)
 }
 
 static void check_enqueue_throttle(struct cfs_rq *cfs_rq);
+static inline int cfs_rq_throttled(struct cfs_rq *cfs_rq);
 
 static inline bool cfs_bandwidth_used(void);
 
@@ -4879,7 +4880,7 @@ enqueue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags)
 #ifdef CONFIG_CFS_BANDWIDTH
 			struct rq *rq = rq_of(cfs_rq);
 
-			if (!cfs_rq->throttled_clock)
+			if (cfs_rq_throttled(cfs_rq) && !cfs_rq->throttled_clock)
 				cfs_rq->throttled_clock = rq_clock(rq);
 			if (!cfs_rq->throttled_clock_self)
 				cfs_rq->throttled_clock_self = rq_clock(rq);
@@ -5387,17 +5388,17 @@ static int tg_unthrottle_up(struct task_group *tg, void *data)
 		/* Add cfs_rq with load or one or more already running entities to the list */
 		if (!cfs_rq_is_decayed(cfs_rq))
 			list_add_leaf_cfs_rq(cfs_rq);
-	}
 
-	if (cfs_rq->throttled_clock_self) {
-		u64 delta = rq_clock(rq) - cfs_rq->throttled_clock_self;
+		if (cfs_rq->throttled_clock_self) {
+			u64 delta = rq_clock(rq) - cfs_rq->throttled_clock_self;
 
-		cfs_rq->throttled_clock_self = 0;
+			cfs_rq->throttled_clock_self = 0;
 
-		if (SCHED_WARN_ON((s64)delta < 0))
-			delta = 0;
+			if (SCHED_WARN_ON((s64)delta < 0))
+				delta = 0;
 
-		cfs_rq->throttled_clock_self_time += delta;
+			cfs_rq->throttled_clock_self_time += delta;
+		}
 	}
 
 	return 0;
@@ -5412,13 +5413,13 @@ static int tg_throttle_down(struct task_group *tg, void *data)
 	if (!cfs_rq->throttle_count) {
 		cfs_rq->throttled_clock_pelt = rq_clock_pelt(rq);
 		list_del_leaf_cfs_rq(cfs_rq);
+
+		SCHED_WARN_ON(cfs_rq->throttled_clock_self);
+		if (cfs_rq->nr_running)
+			cfs_rq->throttled_clock_self = rq_clock(rq);
 	}
 	cfs_rq->throttle_count++;
 
-	SCHED_WARN_ON(cfs_rq->throttled_clock_self);
-	if (cfs_rq->nr_running)
-		cfs_rq->throttled_clock_self = rq_clock(rq);
-
 	return 0;
 }
 
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
maintainer
#1 lore raw
Quoted 9 lines · Josh Don

This fixes two issues:
- throttled_clock should only be set on the group that is actually getting throttled
- self-throttled time should only be accounted on entry/exit to throttled state when we have nested limits

Fixes 88cb2868250c ("sched: add throttled time stat for throttled children")
Fixes 3ab150d011da ("sched: don't account throttle time for empty groups")

Hurmph, those are not the sha1 I have in tip/sched/core.

Also, should I rebase and just pull those patches so we can try again?

author
#2 lore raw
Quoted 8 lines · Peter Zijlstra
Quoted 6 lines · Josh Don

This fixes two issues:
- throttled_clock should only be set on the group that is actually getting throttled
- self-throttled time should only be accounted on entry/exit to throttled state when we have nested limits

Quoted 7 lines
Quoted 3 lines
Fixes 88cb2868250c ("sched: add throttled time stat for throttled children")
Fixes 3ab150d011da ("sched: don't account throttle time for empty groups")

Hurmph, those are not the sha1 I have in tip/sched/core.

Also, should I rebase and just pull those patches so we can try again?

Oh whoops, I used the SHA's from your queue.git.

Yes, if it is still possible to intercept them then that would be great. I'm OOO at the moment, but can respin those for you first thing next week.

Thanks,
Josh

maintainer
#3 lore raw
Quoted 15 lines · Josh Don
Quoted 14 lines · Peter Zijlstra
Quoted 9 lines · Josh Don

This fixes two issues:
- throttled_clock should only be set on the group that is actually getting throttled
- self-throttled time should only be accounted on entry/exit to throttled state when we have nested limits

Fixes 88cb2868250c ("sched: add throttled time stat for throttled children")
Fixes 3ab150d011da ("sched: don't account throttle time for empty groups")

Hurmph, those are not the sha1 I have in tip/sched/core.

Quoted 7 lines
Quoted 1 line

Also, should I rebase and just pull those patches so we can try again?

Oh whoops, I used the SHA's from your queue.git.

Yes, if it is still possible to intercept them then that would be great. I'm OOO at the moment, but can respin those for you first thing next week.

There's only a few patches on top, I'll make it go away.