Skip to contentSkip to conversation

Accounting forced idle time per cpu and per cgroup

PATCH Applied Cruz Zhao 6 participants 15 messages Jan 11, 2022 – Jan 18, 2022

Subsystems (2)
lore
Share & tools
lore

Cruz Zhao

author
#0 lore raw

v2 cover Accounting forced idle time per cpu and per cgroup

There are two types of forced idle time: forced idle time from cookie'd task and forced idle time form uncookie'd task. The forced idle time from uncookie'd task is actually caused by the cookie'd task in runqueue indirectly, and it's more accurate to measure the capacity loss with the sum of both.

This patch set accounts forced idle time for each cpu to measure how long the cpu is forced idle, which is displayed via via /proc/schedstat, and also accounts for each cgroup to measure how long it forced its SMT siblings into idle, which is displayed via /sys/fs/cgroup/cpuacct/cpuacct.forceidle and /sys/fs/cgroup/cpuacct/cpuacct.forceidle_percpu. It is worth noting that the forced idle time and the force idle time have different meanings.

We can get the total system forced idle time by looking at the root cgroup, and we can get how long the cgroup forced it SMT siblings into idle. If the force idle time of a cgroup is high, that can be rectified by making some changes(ie. affinity, cpu budget, etc.) to the cgroup.

Cruz Zhao (3):
sched/core: Accounting forceidle time for all tasks except idle task sched/core: Forced idle accounting per-cpu
sched/core: Force idle accounting per cgroup

 include/linux/cgroup.h    |  7 +++++
 kernel/sched/core.c       | 10 ++++--
 kernel/sched/core_sched.c | 10 ++++--
 kernel/sched/cpuacct.c    | 79 +++++++++++++++++++++++++++++++++++++++++++++++
 kernel/sched/sched.h      |  4 +++
 kernel/sched/stats.c      | 17 ++++++++--
 6 files changed, 119 insertions(+), 8 deletions(-)
Signature · 1 line
1.8.3.1
author
#1 lore raw

PATCH v2 1/3 sched/core: Accounting forceidle time for all tasks except idle task

There are two types of forced idle time: forced idle time from cookie'd task and forced idle time form uncookie'd task. The forced idle time from uncookie'd task is actually caused by the cookie'd task in runqueue indirectly, and it's more accurate to measure the capacity loss with the sum of both.

Assuming cpu x and cpu y are a pair of SMT siblings, consider the following scenarios:
1.There's a cookie'd task running on cpu x, and there're 4 uncookie'd
tasks running on cpu y. For cpu x, there will be 80% forced idle time (from uncookie'd task); for cpu y, there will be 20% forced idle time (from cookie'd task).
2.There's a uncookie'd task running on cpu x, and there're 4 cookie'd
tasks running on cpu y. For cpu x, there will be 80% forced idle time (from cookie'd task); for cpu y, there will be 20% forced idle time (from uncookie'd task).

The scenario1 can recurrent by stress-ng(scenario2 can recurrent similary):
(cookie'd)taskset -c x stress-ng -c 1 -l 100 (uncookie'd)taskset -c y stress-ng -c 4 -l 100

In the above two scenarios, the total capacity loss is 1 cpu, but in scenario1, the cookie'd forced idle time tells us 20% cpu capacity loss, in scenario2, the cookie'd forced idle time tells us 80% cpu capacity loss, which are not accurate. It'll be more accurate to measure with cookie'd forced idle time and uncookie'd forced idle time.

---
 kernel/sched/core.c       | 3 +--
 kernel/sched/core_sched.c | 2 +-
 2 files changed, 2 insertions(+), 3 deletions(-)
kernel/sched/core.c +1 -2
index 2e4ae00..e8187e7 100644
@@ -5822,8 +5822,7 @@ static inline struct task_struct *pick_task(struct rq *rq)
 	}
 
 	if (schedstat_enabled() && rq->core->core_forceidle_count) {
-		if (cookie)
-			rq->core->core_forceidle_start = rq_clock(rq->core);
+		rq->core->core_forceidle_start = rq_clock(rq->core);
 		rq->core->core_forceidle_occupation = occ;
 	}
 
kernel/sched/core_sched.c +1 -1
index 1fb4567..c8746a9 100644
@@ -277,7 +277,7 @@ void __sched_core_account_forceidle(struct rq *rq)
 		rq_i = cpu_rq(i);
 		p = rq_i->core_pick ?: rq_i->curr;
 
-		if (!p->core_cookie)
+		if (p == rq_i->idle)
 			continue;
 
 		__schedstat_add(p->stats.core_forceidle_sum, delta);
Signature · 1 line
1.8.3.1
Reviewed-by on v2 1/3
#2 lore raw

Re: [PATCH v2 1/3] sched/core: Accounting forceidle time for all tasks except idle task

Quoted 22 lines · Cruz Zhao

There are two types of forced idle time: forced idle time from cookie'd task and forced idle time form uncookie'd task. The forced idle time from uncookie'd task is actually caused by the cookie'd task in runqueue indirectly, and it's more accurate to measure the capacity loss with the sum of both.

Assuming cpu x and cpu y are a pair of SMT siblings, consider the following scenarios:
1.There's a cookie'd task running on cpu x, and there're 4 uncookie'd
tasks running on cpu y. For cpu x, there will be 80% forced idle time (from uncookie'd task); for cpu y, there will be 20% forced idle time (from cookie'd task).
2.There's a uncookie'd task running on cpu x, and there're 4 cookie'd
tasks running on cpu y. For cpu x, there will be 80% forced idle time (from cookie'd task); for cpu y, there will be 20% forced idle time (from uncookie'd task).

The scenario1 can recurrent by stress-ng(scenario2 can recurrent similary):
(cookie'd)taskset -c x stress-ng -c 1 -l 100 (uncookie'd)taskset -c y stress-ng -c 4 -l 100

Quoted 8 lines

In the above two scenarios, the total capacity loss is 1 cpu, but in scenario1, the cookie'd forced idle time tells us 20% cpu capacity loss, in scenario2, the cookie'd forced idle time tells us 80% cpu capacity loss, which are not accurate. It'll be more accurate to measure with cookie'd forced idle time and uncookie'd forced idle time.

---

Thanks,

merge bot Applied
#3 lore raw

[tip: sched/urgent] sched/core: Accounting forceidle time for all tasks except idle task

In reply to Cruz Zhao
The following commit has been merged into the sched/urgent branch of tip:

The following commit has been merged into the sched/urgent branch of tip:

Commit-ID:     b171501f258063f5c56dd2c5fdf310802d8d7dc1
Gitweb:        https://git.kernel.org/tip/b171501f258063f5c56dd2c5fdf310802d8d7dc1
Author:        Cruz Zhao <[email protected]>
AuthorDate:    Tue, 11 Jan 2022 17:55:59 +08:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Tue, 18 Jan 2022 12:09:59 +01:00

sched/core: Accounting forceidle time for all tasks except idle task

There are two types of forced idle time: forced idle time from cookie'd task and forced idle time form uncookie'd task. The forced idle time from uncookie'd task is actually caused by the cookie'd task in runqueue indirectly, and it's more accurate to measure the capacity loss with the sum of both.

Assuming cpu x and cpu y are a pair of SMT siblings, consider the following scenarios:
1.There's a cookie'd task running on cpu x, and there're 4 uncookie'd
tasks running on cpu y. For cpu x, there will be 80% forced idle time (from uncookie'd task); for cpu y, there will be 20% forced idle time (from cookie'd task).
2.There's a uncookie'd task running on cpu x, and there're 4 cookie'd
tasks running on cpu y. For cpu x, there will be 80% forced idle time (from cookie'd task); for cpu y, there will be 20% forced idle time (from uncookie'd task).

The scenario1 can recurrent by stress-ng(scenario2 can recurrent similary):
(cookie'd)taskset -c x stress-ng -c 1 -l 100 (uncookie'd)taskset -c y stress-ng -c 4 -l 100

In the above two scenarios, the total capacity loss is 1 cpu, but in scenario1, the cookie'd forced idle time tells us 20% cpu capacity loss, in scenario2, the cookie'd forced idle time tells us 80% cpu capacity loss, which are not accurate. It'll be more accurate to measure with cookie'd forced idle time and uncookie'd forced idle time.

---
 kernel/sched/core.c       | 3 +--
 kernel/sched/core_sched.c | 2 +-
 2 files changed, 2 insertions(+), 3 deletions(-)
kernel/sched/core.c +1 -2
index 83872f9..0d2ab2a 100644
@@ -5822,8 +5822,7 @@ pick_next_task(struct rq *rq, struct task_struct *prev, struct rq_flags *rf)
 	}
 
 	if (schedstat_enabled() && rq->core->core_forceidle_count) {
-		if (cookie)
-			rq->core->core_forceidle_start = rq_clock(rq->core);
+		rq->core->core_forceidle_start = rq_clock(rq->core);
 		rq->core->core_forceidle_occupation = occ;
 	}
 
kernel/sched/core_sched.c +1 -1
index 1fb4567..c8746a9 100644
@@ -277,7 +277,7 @@ void __sched_core_account_forceidle(struct rq *rq)
 		rq_i = cpu_rq(i);
 		p = rq_i->core_pick ?: rq_i->curr;
 
-		if (!p->core_cookie)
+		if (p == rq_i->idle)
 			continue;
 
 		__schedstat_add(p->stats.core_forceidle_sum, delta);
author
#4 lore raw

PATCH v2 2/3 sched/core: Forced idle accounting per-cpu

In reply to Cruz Zhao

Accounting for "forced idle" time per cpu, which is the time a cpu is forced to idle by its SMT sibling.

As it's not accurate to measure the capacity loss only by cookie'd forced idle time, and it's hard to trace the forced idle time caused by all the uncookie'd tasks, we account the forced idle time from the perspective of the cpu.

Forced idle time per cpu is displayed via /proc/schedstat, we can get the forced idle time of cpu x from the 10th column of the row for cpu x. The unit is ns. It also requires that schedstats is enabled.

---
 kernel/sched/core.c       |  7 ++++++-
 kernel/sched/core_sched.c |  7 +++++--
 kernel/sched/sched.h      |  4 ++++
 kernel/sched/stats.c      | 17 +++++++++++++++--
 4 files changed, 30 insertions(+), 5 deletions(-)
kernel/sched/core.c +6 -1
index e8187e7..a224b916 100644
@@ -285,8 +285,10 @@ static void __sched_core_flip(bool enabled)
 
 		sched_core_lock(cpu, &flags);
 
-		for_each_cpu(t, smt_mask)
+		for_each_cpu(t, smt_mask) {
 			cpu_rq(t)->core_enabled = enabled;
+			cpu_rq(t)->in_forcedidle = false;
+		}
 
 		cpu_rq(cpu)->core->core_forceidle_start = 0;
 
@@ -5690,6 +5692,7 @@ static inline struct task_struct *pick_task(struct rq *rq)
 		 * another cpu during offline.
 		 */
 		rq->core_pick = NULL;
+		rq->in_forcedidle = false;
 		return __pick_next_task(rq, prev, rf);
 	}
 
@@ -5810,9 +5813,11 @@ static inline struct task_struct *pick_task(struct rq *rq)
 
 		rq_i->core_pick = p;
 
+		rq->in_forcedidle = false;
 		if (p == rq_i->idle) {
 			if (rq_i->nr_running) {
 				rq->core->core_forceidle_count++;
+				rq_i->in_forcedidle = true;
 				if (!fi_before)
 					rq->core->core_forceidle_seq++;
 			}
kernel/sched/core_sched.c +5 -2
index c8746a9..fe04805 100644
@@ -242,7 +242,7 @@ int sched_core_share_pid(unsigned int cmd, pid_t pid, enum pid_type type,
 void __sched_core_account_forceidle(struct rq *rq)
 {
 	const struct cpumask *smt_mask = cpu_smt_mask(cpu_of(rq));
-	u64 delta, now = rq_clock(rq->core);
+	u64 delta_per_idlecpu, delta, now = rq_clock(rq->core);
 	struct rq *rq_i;
 	struct task_struct *p;
 	int i;
@@ -254,7 +254,7 @@ void __sched_core_account_forceidle(struct rq *rq)
 	if (rq->core->core_forceidle_start == 0)
 		return;
 
-	delta = now - rq->core->core_forceidle_start;
+	delta_per_idlecpu = delta = now - rq->core->core_forceidle_start;
 	if (unlikely((s64)delta <= 0))
 		return;
 
@@ -277,6 +277,9 @@ void __sched_core_account_forceidle(struct rq *rq)
 		rq_i = cpu_rq(i);
 		p = rq_i->core_pick ?: rq_i->curr;
 
+		if (rq_i->in_forcedidle)
+			rq->rq_forceidle_time += delta_per_idlecpu;
+
 		if (p == rq_i->idle)
 			continue;
 
kernel/sched/sched.h +4 -0
index de53be9..9377d91 100644
@@ -1086,6 +1086,9 @@ struct rq {
 	/* try_to_wake_up() stats */
 	unsigned int		ttwu_count;
 	unsigned int		ttwu_local;
+#ifdef CONFIG_SCHED_CORE
+	u64			rq_forceidle_time;
+#endif
 #endif
 
 #ifdef CONFIG_CPU_IDLE
@@ -1115,6 +1118,7 @@ struct rq {
 	unsigned int		core_forceidle_seq;
 	unsigned int		core_forceidle_occupation;
 	u64			core_forceidle_start;
+	bool			in_forcedidle;
 #endif
 };
 
kernel/sched/stats.c +15 -2
index 07dde29..ea22a8c 100644
@@ -108,6 +108,16 @@ void __update_stats_enqueue_sleeper(struct rq *rq, struct task_struct *p,
 	}
 }
 
+#ifdef CONFIG_SCHED_CORE
+static inline u64 get_rq_forceidle_time(struct rq *rq) {
+	return rq->rq_forceidle_time;
+}
+#else
+static inline u64 get_rq_forceidle_time(struct rq *rq) {
+	return 0;
+}
+#endif
+
 /*
  * Current schedstat API version.
  *
@@ -125,21 +135,24 @@ static int show_schedstat(struct seq_file *seq, void *v)
 		seq_printf(seq, "timestamp %lu\n", jiffies);
 	} else {
 		struct rq *rq;
+		u64 rq_forceidle_time;
 #ifdef CONFIG_SMP
 		struct sched_domain *sd;
 		int dcount = 0;
 #endif
 		cpu = (unsigned long)(v - 2);
 		rq = cpu_rq(cpu);
+		rq_forceidle_time = get_rq_forceidle_time(rq);
 
 		/* runqueue-specific stats */
 		seq_printf(seq,
-		    "cpu%d %u 0 %u %u %u %u %llu %llu %lu",
+		    "cpu%d %u 0 %u %u %u %u %llu %llu %lu %llu",
 		    cpu, rq->yld_count,
 		    rq->sched_count, rq->sched_goidle,
 		    rq->ttwu_count, rq->ttwu_local,
 		    rq->rq_cpu_time,
-		    rq->rq_sched_info.run_delay, rq->rq_sched_info.pcount);
+		    rq->rq_sched_info.run_delay, rq->rq_sched_info.pcount,
+		    rq_forceidle_time);
 
 		seq_printf(seq, "\n");
 
Signature · 1 line
1.8.3.1
#5 lore raw

Re: [PATCH v2 2/3] sched/core: Forced idle accounting per-cpu

Quoted 9 lines · Cruz Zhao

Accounting for "forced idle" time per cpu, which is the time a cpu is forced to idle by its SMT sibling.

As it's not accurate to measure the capacity loss only by cookie'd forced idle time, and it's hard to trace the forced idle time caused by all the uncookie'd tasks, we account the forced idle time from the perspective of the cpu.

Quoted 6 lines

Forced idle time per cpu is displayed via /proc/schedstat, we can get the forced idle time of cpu x from the 10th column of the row for cpu x. The unit is ns. It also requires that schedstats is enabled.

---

Two quick followup questions:

1) From your v1, I still wasn't quite sure if the per-cpu time was useful or not for you versus a single overall sum (ie. I think other metrics could be more useful for analyzing steal_cookie if that's what you're specifically interested in). Do you definitely want the per-cpu totals?

2) If your cgroup accounting patch is merged, do you still want this patch? You can grab the global values from the root cgroup (assuming you have cgroups enabled). The only potential gap is if you need per-cpu totals, though I'm working to add percpu stats to cgroup-v2:
https://lkml.kernel.org/r/%3C20220107234138.1765668-1-joshdon-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org%3E

author
#6 lore raw

Re: [PATCH v2 2/3] sched/core: Forced idle accounting per-cpu

Quoted 17 lines · Josh Don
Quoted 9 lines · Cruz Zhao

Accounting for "forced idle" time per cpu, which is the time a cpu is forced to idle by its SMT sibling.

As it's not accurate to measure the capacity loss only by cookie'd forced idle time, and it's hard to trace the forced idle time caused by all the uncookie'd tasks, we account the forced idle time from the perspective of the cpu.

Quoted 6 lines

Forced idle time per cpu is displayed via /proc/schedstat, we can get the forced idle time of cpu x from the 10th column of the row for cpu x. The unit is ns. It also requires that schedstats is enabled.

---

Quoted 8 lines

Two quick followup questions:

1) From your v1, I still wasn't quite sure if the per-cpu time was useful or not for you versus a single overall sum (ie. I think other metrics could be more useful for analyzing steal_cookie if that's what you're specifically interested in). Do you definitely want the per-cpu totals?

IMO, the per-cpu forced idle time can help us get to know whether the forced idle time is uniform among the core, or among all the cpus. IMO, it's a kind of balance.

Quoted 5 lines

2) If your cgroup accounting patch is merged, do you still want this patch? You can grab the global values from the root cgroup (assuming you have cgroups enabled). The only potential gap is if you need per-cpu totals, though I'm working to add percpu stats to cgroup-v2:
https://lkml.kernel.org/r/%3C20220107234138.1765668-1-joshdon-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org%3E

If cgroup accounting patch is merged, this patch is still needed.

Consider the following scenario:
Task p of cgroup A is running on cpu x, and it forces cpu y into idle for t ns. The forceidle time of cgroup A on cpu x increases t ns, and the forcedidle time of cpu y increases t ns.

That is, the force idle time of cgroup is defined as the forced idle time it caused, and the force idle time of cpu is defined as the time the cpu is forced into idle, which have different meanings from each other.

And for SMT > 2, we cannot caculate the forced idle time of cpu x from the cgroup interface.

Best,
Cruz Zhao

#7 lore raw

Re: [PATCH v2 2/3] sched/core: Forced idle accounting per-cpu

Quoted 9 lines
Quoted 6 lines

1) From your v1, I still wasn't quite sure if the per-cpu time was useful or not for you versus a single overall sum (ie. I think other metrics could be more useful for analyzing steal_cookie if that's what you're specifically interested in). Do you definitely want the per-cpu totals?

IMO, the per-cpu forced idle time can help us get to know whether the forced idle time is uniform among the core, or among all the cpus. IMO, it's a kind of balance.

Sure, I'm not opposed to it.

Quoted 13 lines
Quoted 5 lines

2) If your cgroup accounting patch is merged, do you still want this patch? You can grab the global values from the root cgroup (assuming you have cgroups enabled). The only potential gap is if you need per-cpu totals, though I'm working to add percpu stats to cgroup-v2:
https://lkml.kernel.org/r/%3C20220107234138.1765668-1-joshdon-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org%3E

If cgroup accounting patch is merged, this patch is still needed.

Consider the following scenario:
Task p of cgroup A is running on cpu x, and it forces cpu y into idle for t ns. The forceidle time of cgroup A on cpu x increases t ns, and the forcedidle time of cpu y increases t ns.

Quoted 6 lines

That is, the force idle time of cgroup is defined as the forced idle time it caused, and the force idle time of cpu is defined as the time the cpu is forced into idle, which have different meanings from each other.

And for SMT > 2, we cannot caculate the forced idle time of cpu x from the cgroup interface.

Ack. Note that the patch I linked above for per-cpu stats for cgroup-v2 has been nack'd, so it is unlikely we'll have kernel exports for cgroup-v2 per-cpu stats (but perhaps some export via BPF). In v2, we could at least export the cgroup sum force idle time in cpu.stat, if you feel it would be useful and want to add the accounting to rstat. If you really just want the per-cpu root stats then I'm fine with skipping cgroup for now.

maintainer
#8 lore raw

Re: [PATCH v2 2/3] sched/core: Forced idle accounting per-cpu

In reply to Cruz Zhao

kernel/sched/sched.h struct rq { · line 1121

Quoted 5 lines · Cruz Zhao
kernel/sched/sched.h +1 -0
@@ -1115,6 +1118,7 @@ struct rq {
 	unsigned int		core_forceidle_seq;
 	unsigned int		core_forceidle_occupation;
 	u64			core_forceidle_start;
+	bool			in_forcedidle;

naming is wrong

Quoted 10 lines
(quoted diff) +0 -0
 #endif
 };
 
kernel/sched/stats.c +0 -0
index 07dde29..ea22a8c 100644
@@ -108,6 +108,16 @@ void __update_stats_enqueue_sleeper(struct rq *rq, struct task_struct *p,
 	}
 }

kernel/sched/stats.c void __update_stats_enqueue_sleeper(struct rq *rq, struct task_struct *p, · line 119

Quoted 10 lines
kernel/sched/stats.c +9 -0
 
+#ifdef CONFIG_SCHED_CORE
+static inline u64 get_rq_forceidle_time(struct rq *rq) {
+	return rq->rq_forceidle_time;
+}
+#else
+static inline u64 get_rq_forceidle_time(struct rq *rq) {
+	return 0;
+}
+#endif

indent is wrong, and if you put the #ifdef inside the function it'll be smaller.

author
#9 lore raw

Re: [PATCH v2 2/3] sched/core: Forced idle accounting per-cpu

Quoted 21 lines · Peter Zijlstra

kernel/sched/sched.h struct rq { · line 1121

Quoted 5 lines · Cruz Zhao
kernel/sched/sched.h +1 -0
@@ -1115,6 +1118,7 @@ struct rq {
 	unsigned int		core_forceidle_seq;
 	unsigned int		core_forceidle_occupation;
 	u64			core_forceidle_start;
+	bool			in_forcedidle;

naming is wrong

Quoted 11 lines
(quoted diff) +0 -0
 #endif
 };
 
kernel/sched/stats.c +0 -0
index 07dde29..ea22a8c 100644
@@ -108,6 +108,16 @@ void __update_stats_enqueue_sleeper(struct rq *rq, struct task_struct *p,
 	}
 }
 
Quoted 12 lines

kernel/sched/stats.c void __update_stats_enqueue_sleeper(struct rq *rq, struct task_struct *p, · line 119

Quoted 9 lines
kernel/sched/stats.c +9 -0
+#ifdef CONFIG_SCHED_CORE
+static inline u64 get_rq_forceidle_time(struct rq *rq) {
+	return rq->rq_forceidle_time;
+}
+#else
+static inline u64 get_rq_forceidle_time(struct rq *rq) {
+	return 0;
+}
+#endif

indent is wrong, and if you put the #ifdef inside the function it'll be smaller.

Thanks for reviewing and suggestions, I'll fix these problems in the next version.

Best,
Cruz Zhao

author
#10 lore raw

PATCH v2 3/3 sched/core: Force idle accounting per cgroup

In reply to Cruz Zhao

Accounting for "force idle" time per cgroup, which is the time the tasks of the cgroup forced its SMT siblings into idle.

Force idle time per cgroup is displayed via
/sys/fs/cgroup/cpuacct/$cg/cpuacct.forceidle.
Force idle time per cgroup per cpu is displayed via
/sys/fs/cgroup/cpuacct/$cg/cpuacct.forceidle_percpu.
The unit is ns.
It also requires that schedstats is enabled.

We can get the total system forced idle time by looking at the root cgroup, and we can get how long the cgroup forced it SMT siblings into idle. If the force idle time of a cgroup is high, that can be rectified by making some changes(ie. affinity, cpu budget, etc.) to the cgroup.

---
 include/linux/cgroup.h    |  7 +++++
 kernel/sched/core_sched.c |  1 +
 kernel/sched/cpuacct.c    | 79 +++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 87 insertions(+)
include/linux/cgroup.h +7 -0
index 75c1514..0c1b616 100644
@@ -774,10 +774,17 @@ static inline struct cgroup *cgroup_get_from_id(u64 id)
 #ifdef CONFIG_CGROUP_CPUACCT
 void cpuacct_charge(struct task_struct *tsk, u64 cputime);
 void cpuacct_account_field(struct task_struct *tsk, int index, u64 val);
+#ifdef CONFIG_SCHED_CORE
+void cpuacct_account_forceidle(int cpu, struct task_struct *task, u64 cputime);
+#endif
 #else
 static inline void cpuacct_charge(struct task_struct *tsk, u64 cputime) {}
 static inline void cpuacct_account_field(struct task_struct *tsk, int index,
 					 u64 val) {}
+#ifdef CONFIG_SCHED_CORE
+static inline void cpuacct_account_forceidle(int cpu, struct task_struct *task,
+					     u64 cputime) {}
+#endif
 #endif
 
 void __cgroup_account_cputime(struct cgroup *cgrp, u64 delta_exec);
kernel/sched/core_sched.c +1 -0
index fe04805..add8672 100644
@@ -284,6 +284,7 @@ void __sched_core_account_forceidle(struct rq *rq)
 			continue;
 
 		__schedstat_add(p->stats.core_forceidle_sum, delta);
+		cpuacct_account_forceidle(i, p, delta);
 	}
 }
 
kernel/sched/cpuacct.c +79 -0

120 lines not on this page · raw message

Signature · 1 line
1.8.3.1
maintainer
#11 lore raw

Re: [PATCH v2 3/3] sched/core: Force idle accounting per cgroup

Hello,

Quoted 5 lines · Cruz Zhao
kernel/sched/cpuacct.c +5 -0
+#ifdef CONFIG_SCHED_CORE
+void cpuacct_account_forceidle(int cpu, struct task_struct *tsk, u64 cputime)
+{
+	struct cpuacct *ca;
+	u64 *fi;

kernel/sched/cpuacct.c void cpuacct_account_field(struct task_struct *tsk, int index, u64 val) · line 436

Quoted 10 lines
kernel/sched/cpuacct.c +10 -0
+
+	rcu_read_lock();
+	/*
+	 * We have hold rq->core->__lock here, which protects ca->forceidle
+	 * percpu.
+	 */
+	for (ca = task_ca(tsk); ca; ca = parent_ca(ca)) {
+		fi = per_cpu_ptr(ca->forceidle, cpu);
+		*fi += cputime;
+	}

Please don't do this. Use rstat and integrate it with other stats.

Thanks.

Signature · 1 line
tejun
author
#12 lore raw

Re: [PATCH v2 3/3] sched/core: Force idle accounting per cgroup

Quoted 21 lines · Tejun Heo

Hello,

Quoted 5 lines · Cruz Zhao
kernel/sched/cpuacct.c +5 -0
+#ifdef CONFIG_SCHED_CORE
+void cpuacct_account_forceidle(int cpu, struct task_struct *tsk, u64 cputime)
+{
+	struct cpuacct *ca;
+	u64 *fi;

kernel/sched/cpuacct.c void cpuacct_account_field(struct task_struct *tsk, int index, u64 val) · line 436

Quoted 10 lines
kernel/sched/cpuacct.c +10 -0
+
+	rcu_read_lock();
+	/*
+	 * We have hold rq->core->__lock here, which protects ca->forceidle
+	 * percpu.
+	 */
+	for (ca = task_ca(tsk); ca; ca = parent_ca(ca)) {
+		fi = per_cpu_ptr(ca->forceidle, cpu);
+		*fi += cputime;
+	}

Please don't do this. Use rstat and integrate it with other stats.

Quoted 2 lines

Thanks.

Thanks for suggestions, I'll try to do this using rstat. BTW, is it ok to integrate it with cgroup_base_stat?

Best,
Cruz Zhao

maintainer
#13 lore raw

Re: [PATCH v2 3/3] sched/core: Force idle accounting per cgroup

Quoted 2 lines · cruzzhao

Thanks for suggestions, I'll try to do this using rstat. BTW, is it ok to integrate it with cgroup_base_stat?

Yeah, I don't see a problem at least from cgroup / rstat POV.

Thanks.

Signature · 1 line
tejun
#14 lore raw

Re: [PATCH v2 3/3] sched/core: Force idle accounting per cgroup

In reply to Cruz Zhao
Thank you for the patch! Yet something to improve:

Hi Cruz,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on tip/sched/core]
[also build test ERROR on tip/master linux/master linus/master next-20220112] [cannot apply to tj-cgroup/for-next v5.16]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Cruz-Zhao/Accounting-forced-idle-time-per-cpu-and-per-cgroup/20220111-175754
base:   https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git 82762d2af31a60081162890983a83499c9c7dd74
config: ia64-randconfig-r024-20220112 (https://download.01.org/0day-ci/archive/20220113/202201130529.HUZtnC5G-lkp(a)intel.com/config)
compiler: ia64-linux-gcc (GCC) 11.2.0

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/363ef949b487441383f2719bdff3dc9156779565
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Cruz-Zhao/Accounting-forced-idle-time-per-cpu-and-per-cgroup/20220111-175754 git checkout 363ef949b487441383f2719bdff3dc9156779565
# save the config file to linux build tree
mkdir build_dir
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross O=build_dir ARCH=ia64 SHELL=/bin/bash

If you fix the issue, kindly add following tag as appropriate

All errors (new ones prefixed by >>):

kernel/sched/core_sched.c: In function '__sched_core_account_forceidle':

>> kernel/sched/core_sched.c:287:17: error: implicit declaration of function 'cpuacct_account_forceidle'; did you mean 'sched_core_account_forceidle'? [-Werror=implicit-function-declaration]
     287 |                 cpuacct_account_forceidle(i, p, delta);
         |                 ^~~~~~~~~~~~~~~~~~~~~~~~~
         |                 sched_core_account_forceidle

cc1: some warnings being treated as errors

vim +287 kernel/sched/core_sched.c

240
241 /* REQUIRES: rq->core's clock recently updated. */
242 void __sched_core_account_forceidle(struct rq *rq)
243 {
244 const struct cpumask *smt_mask = cpu_smt_mask(cpu_of(rq));
245 u64 delta_per_idlecpu, delta, now = rq_clock(rq->core);
246 struct rq *rq_i;
247 struct task_struct *p;
248 int i;
249
250 lockdep_assert_rq_held(rq);
251
252 WARN_ON_ONCE(!rq->core->core_forceidle_count);
253
254 if (rq->core->core_forceidle_start == 0)
255 return;
256
257 delta_per_idlecpu = delta = now - rq->core->core_forceidle_start;
258 if (unlikely((s64)delta <= 0))
259 return;
260
261 rq->core->core_forceidle_start = now;
262
263 if (WARN_ON_ONCE(!rq->core->core_forceidle_occupation)) {
264 /* can't be forced idle without a running task */
265 } else if (rq->core->core_forceidle_count > 1 ||
266 rq->core->core_forceidle_occupation > 1) {
267 /*
268 * For larger SMT configurations, we need to scale the charged
269 * forced idle amount since there can be more than one forced
270 * idle sibling and more than one running cookied task.
271 */
272 delta *= rq->core->core_forceidle_count;
273 delta = div_u64(delta, rq->core->core_forceidle_occupation);
274 }
275
276 for_each_cpu(i, smt_mask) {
277 rq_i = cpu_rq(i);
278 p = rq_i->core_pick ?: rq_i->curr;
279
280 if (rq_i->in_forcedidle)
281 rq->rq_forceidle_time += delta_per_idlecpu;
282
283 if (p == rq_i->idle)
284 continue;
285
286 __schedstat_add(p->stats.core_forceidle_sum, delta);

 > 287			cpuacct_account_forceidle(i, p, delta);

288 }
289 }
290

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org