Skip to contentSkip to conversation

cgroup: Track time in cgroup v2 freezer

RFC PATCH Superseded Tiffany Yang 4 participants 18 messages Jul 14, 2025 – Jul 25, 2025

Subsystems (2)

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

RFC PATCH v2 1/1 cgroup: Track time in cgroup v2 freezer

The cgroup v2 freezer controller allows user processes to be dynamically added to and removed from an interruptible frozen state from userspace. This feature is helpful for application management, as a background app can be frozen to prevent its threads from being scheduled or otherwise contending with foreground tasks for resources. However, the application is usually unaware that it was frozen, which can cause issues by disrupting any internal monitoring that it is performing.

As an example, an application may implement a watchdog thread for one of its high priority maintenance tasks that operates by checking some state of that task at a set interval to ensure it has made progress. The key challenge here is that the task is only expected to make progress when the application it belongs to has the opportunity to run, but there's no application-relative time to set the watchdog timer against. Instead, the next timeout is set relative to system time, using an approximation that assumes the application will continue to be scheduled as normal. If the task misses that approximate deadline because the application was frozen, without any way to know that, the watchdog may kill the healthy process.

Other sources of delay can cause similar issues, but this change focuses on allowing frozen time to be accounted for in particular because of how large it can grow and how unevenly it can affect applications running on the system. To allow an application to better account for the time it spends running, I propose tracking the time each cgroup spends freezing and exposing it to userland via a new core interface file in cgroupfs (cgroup.freeze.stat). I used this naming because utility controllers like "kill" and "freeze" are exposed as cgroup v2 core interface files, but I'm happy to change it if there's a convention others would prefer!

Currently, the cgroup css_set_lock is used to serialize accesses to the CGRP_FREEZE bit of cgrp->flags and the new cgroup_freezer_state counters (freeze_time_start_ns and freeze_time_total_ns). If we start to see higher contention on this lock, we may want to introduce a v2 freezer state-specific lock to avoid having to take the global lock every time a cgroup.freeze.stat file is read.

Any feedback would be much appreciated!

Thank you,
Tiffany

---
v2:
* Track per-cgroup freezing time instead of per-task frozen time as suggested by Tejun Heo

Cc John Stultz <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Stephen Boyd <[email protected]>
Cc Anna-Maria Behnsen <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Tejun Heo <[email protected]>
Cc Johannes Weiner <[email protected]>
Cc Michal Koutný <[email protected]>
Cc "Rafael J. Wysocki" <[email protected]>
Cc Pavel Machek <[email protected]>
Cc Roman Gushchin <[email protected]>
Cc Chen Ridong <[email protected]>
---
 Documentation/admin-guide/cgroup-v2.rst |  8 ++++++++
 include/linux/cgroup-defs.h             |  6 ++++++
 kernel/cgroup/cgroup.c                  | 24 ++++++++++++++++++++++++
 kernel/cgroup/freezer.c                 |  8 ++++++--
 4 files changed, 44 insertions(+), 2 deletions(-)
Documentation/admin-guide/cgroup-v2.rst +8 -0
index bd98ea3175ec..9fbf3a959bdf 100644
@@ -1018,6 +1018,14 @@ All cgroup core files are prefixed with "cgroup."
 	it's possible to delete a frozen (and empty) cgroup, as well as
 	create new sub-cgroups.
 
+  cgroup.freeze.stat
+	A read-only flat-keyed file which exists in non-root cgroups.
+	The following entry is defined:
+
+	  freeze_time_total_ns
+		Cumulative time that this cgroup has spent in the freezing
+		state, regardless of whether or not it reaches "frozen".
+
   cgroup.kill
 	A write-only single value file which exists in non-root cgroups.
 	The only allowed value is "1".
include/linux/cgroup-defs.h +6 -0
index e61687d5e496..86332d83fa22 100644
@@ -436,6 +436,12 @@ struct cgroup_freezer_state {
 	 * frozen, SIGSTOPped, and PTRACEd.
 	 */
 	int nr_frozen_tasks;
+
+	/* Time when the cgroup was requested to freeze */
+	u64 freeze_time_start_ns;
+
+	/* Total duration the cgroup has spent freezing */
+	u64 freeze_time_total_ns;
 };
 
 struct cgroup {
kernel/cgroup/cgroup.c +24 -0
index a723b7dc6e4e..1f54d16a8713 100644
@@ -4050,6 +4050,23 @@ static ssize_t cgroup_freeze_write(struct kernfs_open_file *of,
 	return nbytes;
 }
 
+static int cgroup_freeze_stat_show(struct seq_file *seq, void *v)
+{
+	struct cgroup *cgrp = seq_css(seq)->cgroup;
+	u64 freeze_time = 0;
+
+	spin_lock_irq(&css_set_lock);
+	if (test_bit(CGRP_FREEZE, &cgrp->flags))
+		freeze_time = ktime_get_ns() - cgrp->freezer.freeze_time_start_ns;
+
+	freeze_time += cgrp->freezer.freeze_time_total_ns;
+	spin_unlock_irq(&css_set_lock);
+
+	seq_printf(seq, "freeze_time_total_ns %llu\n", freeze_time);
+
+	return 0;
+}
+
 static void __cgroup_kill(struct cgroup *cgrp)
 {
 	struct css_task_iter it;
@@ -5355,6 +5372,11 @@ static struct cftype cgroup_base_files[] = {
 		.seq_show = cgroup_freeze_show,
 		.write = cgroup_freeze_write,
 	},
+	{
+		.name = "cgroup.freeze.stat",
+		.flags = CFTYPE_NOT_ON_ROOT,
+		.seq_show = cgroup_freeze_stat_show,
+	},
 	{
 		.name = "cgroup.kill",
 		.flags = CFTYPE_NOT_ON_ROOT,
@@ -5758,6 +5780,7 @@ static struct cgroup *cgroup_create(struct cgroup *parent, const char *name,
 	 * if the parent has to be frozen, the child has too.
 	 */
 	cgrp->freezer.e_freeze = parent->freezer.e_freeze;
+	cgrp->freezer.freeze_time_total_ns = 0;
 	if (cgrp->freezer.e_freeze) {
 		/*
 		 * Set the CGRP_FREEZE flag, so when a process will be
@@ -5766,6 +5789,7 @@ static struct cgroup *cgroup_create(struct cgroup *parent, const char *name,
 		 * consider it frozen immediately.
 		 */
 		set_bit(CGRP_FREEZE, &cgrp->flags);
+		cgrp->freezer.freeze_time_start_ns = ktime_get_ns();
 		set_bit(CGRP_FROZEN, &cgrp->flags);
 	}
 
kernel/cgroup/freezer.c +6 -2
index bf1690a167dd..6f3fab252140 100644
@@ -179,10 +179,14 @@ static void cgroup_do_freeze(struct cgroup *cgrp, bool freeze)
 	lockdep_assert_held(&cgroup_mutex);
 
 	spin_lock_irq(&css_set_lock);
-	if (freeze)
+	if (freeze) {
 		set_bit(CGRP_FREEZE, &cgrp->flags);
-	else
+		cgrp->freezer.freeze_time_start_ns = ktime_get_ns();
+	} else {
 		clear_bit(CGRP_FREEZE, &cgrp->flags);
+		cgrp->freezer.freeze_time_total_ns += (ktime_get_ns() -
+			cgrp->freezer.freeze_time_start_ns);
+	}
 	spin_unlock_irq(&css_set_lock);
 
 	if (freeze)
Signature · 1 line
2.50.0.727.gbf7dc18ff4-goog
maintainer
#1 lore raw

Hello Tiffany.

Quoted 4 lines · Tiffany Yang

Other sources of delay can cause similar issues, but this change focuses on allowing frozen time to be accounted for in particular because of how large it can grow and how unevenly it can affect applications running on the system.

I'd like to incorporate the reason from your other mail:

| Since there isn't yet a clear way to identify a set of "lost" time
| that everyone (or at least a wider group of users) cares about, it
| seems like iterating over components of interest is the best way 

into this commit message (because that's a stronger ponit that your use case alone).

Quoted 1 line

Any feedback would be much appreciated!

I can see benefits of this new stat field conceptually, I have some remarks to implementation and suggestions to conventions below.

Documentation/admin-guide/cgroup-v2.rst All cgroup core files are prefixed with "cgroup." · line 1021

Quoted 7 lines
Documentation/admin-guide/cgroup-v2.rst +1 -0
@@ -1018,6 +1018,14 @@ All cgroup core files are prefixed with "cgroup."
 	it's possible to delete a frozen (and empty) cgroup, as well as
 	create new sub-cgroups.
 
+  cgroup.freeze.stat

With the given implementation (and use scenario), this'd better exposed in
cgroup.freeze.stat.local

I grok the hierarchical summing would make little sense and it'd make implementaion more complex. With that I'm thinking about formulation:

Cumulative time that cgroup has spent between freezing and thawing, regardless of whether by self or ancestor cgroups. NB (not) reaching "frozen" state is not accounted here.

Documentation/admin-guide/cgroup-v2.rst All cgroup core files are prefixed with "cgroup." · line 1028

Quoted 7 lines
Documentation/admin-guide/cgroup-v2.rst +7 -0
+	A read-only flat-keyed file which exists in non-root cgroups.
+	The following entry is defined:
+
+	  freeze_time_total_ns
+		Cumulative time that this cgroup has spent in the freezing
+		state, regardless of whether or not it reaches "frozen".
+

Rather use microseconds, it's the cgroup API convention and I'm not sure nanosecods exposed here are the needed precision.

       1    _____

frozen 0 __/ \__
ab cd

Yeah, I find the mesurent between a and c the sanest.

Documentation/admin-guide/cgroup-v2.rst All cgroup core files are prefixed with "cgroup."

Quoted 11 lines
Documentation/admin-guide/cgroup-v2.rst +11 -0
+static int cgroup_freeze_stat_show(struct seq_file *seq, void *v)
+{
+	struct cgroup *cgrp = seq_css(seq)->cgroup;
+	u64 freeze_time = 0;
+
+	spin_lock_irq(&css_set_lock);
+	if (test_bit(CGRP_FREEZE, &cgrp->flags))
+		freeze_time = ktime_get_ns() - cgrp->freezer.freeze_time_start_ns;
+
+	freeze_time += cgrp->freezer.freeze_time_total_ns;
+	spin_unlock_irq(&css_set_lock);

I don't like taking this spinlock only for the matter of reading this attribute. The intention should be to keep the (un)freezeing mostly unaffected at the expense of these readers (seqcount or u64 stats?).

Alternative approach: either there's outer watcher who can be notified by cgroup.events:frozen or it's an inner watcher who couldn't actively read the field anyway. So the field could only show completed freeze/thaw cycles from the past (i.e. not substitute clock_gettime(2) when the cgroup is frozen), which could simplify querying the flag too.

Documentation/admin-guide/cgroup-v2.rst static struct cgroup *cgroup_create(struct cgroup *parent, const char *name, · line 5783

Quoted 5 lines
Documentation/admin-guide/cgroup-v2.rst +1 -0
@@ -5758,6 +5780,7 @@ static struct cgroup *cgroup_create(struct cgroup *parent, const char *name,
 	 * if the parent has to be frozen, the child has too.
 	 */
 	cgrp->freezer.e_freeze = parent->freezer.e_freeze;
+	cgrp->freezer.freeze_time_total_ns = 0;

struct cgroup is kzalloc'd, this is unnecessary

#2 lore raw
Quoted 28 lines · Michal Koutný

Hello Tiffany.

Quoted 4 lines · Tiffany Yang

Other sources of delay can cause similar issues, but this change focuses on allowing frozen time to be accounted for in particular because of how large it can grow and how unevenly it can affect applications running on the system.

I'd like to incorporate the reason from your other mail:

| Since there isn't yet a clear way to identify a set of "lost" time
| that everyone (or at least a wider group of users) cares about, it
| seems like iterating over components of interest is the best way 

into this commit message (because that's a stronger ponit that your use case alone).

Quoted 1 line

Any feedback would be much appreciated!

I can see benefits of this new stat field conceptually, I have some remarks to implementation and suggestions to conventions below.

Quoted 6 lines
Documentation/admin-guide/cgroup-v2.rst +0 -0
@@ -1018,6 +1018,14 @@ All cgroup core files are prefixed with "cgroup."
 	it's possible to delete a frozen (and empty) cgroup, as well as
 	create new sub-cgroups.
 
Quoted 6 lines

Documentation/admin-guide/cgroup-v2.rst All cgroup core files are prefixed with "cgroup." · line 1021

Quoted 1 line
Documentation/admin-guide/cgroup-v2.rst +1 -0
+  cgroup.freeze.stat

With the given implementation (and use scenario), this'd better exposed in
cgroup.freeze.stat.local

Would it be possible to add this field to either cgroup.event or cgroup.stat? Since the frozen status is already tracked in cgroup.event, this placement would maintain better cohesion with existing metrics.

This is just a suggestion.

Best regards,
Ridong

Quoted 55 lines

I grok the hierarchical summing would make little sense and it'd make implementaion more complex. With that I'm thinking about formulation:

Cumulative time that cgroup has spent between freezing and thawing, regardless of whether by self or ancestor cgroups. NB (not) reaching "frozen" state is not accounted here.

Documentation/admin-guide/cgroup-v2.rst All cgroup core files are prefixed with "cgroup." · line 1028

Quoted 7 lines
Documentation/admin-guide/cgroup-v2.rst +7 -0
+	A read-only flat-keyed file which exists in non-root cgroups.
+	The following entry is defined:
+
+	  freeze_time_total_ns
+		Cumulative time that this cgroup has spent in the freezing
+		state, regardless of whether or not it reaches "frozen".
+

Rather use microseconds, it's the cgroup API convention and I'm not sure nanosecods exposed here are the needed precision.

       1    _____

frozen 0 __/ \__
ab cd

Yeah, I find the mesurent between a and c the sanest.

Documentation/admin-guide/cgroup-v2.rst All cgroup core files are prefixed with "cgroup."

Quoted 11 lines
Documentation/admin-guide/cgroup-v2.rst +11 -0
+static int cgroup_freeze_stat_show(struct seq_file *seq, void *v)
+{
+	struct cgroup *cgrp = seq_css(seq)->cgroup;
+	u64 freeze_time = 0;
+
+	spin_lock_irq(&css_set_lock);
+	if (test_bit(CGRP_FREEZE, &cgrp->flags))
+		freeze_time = ktime_get_ns() - cgrp->freezer.freeze_time_start_ns;
+
+	freeze_time += cgrp->freezer.freeze_time_total_ns;
+	spin_unlock_irq(&css_set_lock);

I don't like taking this spinlock only for the matter of reading this attribute. The intention should be to keep the (un)freezeing mostly unaffected at the expense of these readers (seqcount or u64 stats?).

Alternative approach: either there's outer watcher who can be notified by cgroup.events:frozen or it's an inner watcher who couldn't actively read the field anyway. So the field could only show completed freeze/thaw cycles from the past (i.e. not substitute clock_gettime(2) when the cgroup is frozen), which could simplify querying the flag too.

Documentation/admin-guide/cgroup-v2.rst static struct cgroup *cgroup_create(struct cgroup *parent, const char *name, · line 5783

Quoted 5 lines
Documentation/admin-guide/cgroup-v2.rst +1 -0
@@ -5758,6 +5780,7 @@ static struct cgroup *cgroup_create(struct cgroup *parent, const char *name,
 	 * if the parent has to be frozen, the child has too.
 	 */
 	cgrp->freezer.e_freeze = parent->freezer.e_freeze;
+	cgrp->freezer.freeze_time_total_ns = 0;

struct cgroup is kzalloc'd, this is unnecessary

maintainer
#3 lore raw
Quoted 10 lines · Chen Ridong
Quoted 4 lines

With the given implementation (and use scenario), this'd better exposed in
cgroup.freeze.stat.local

Would it be possible to add this field to either cgroup.event or cgroup.stat? Since the frozen status is already tracked in cgroup.event, this placement would maintain better cohesion with existing metrics.

This is just a suggestion.

Yeah, given that the freezer is an integral part of cgroup core, using cgroup.stat[.local] probably makes more sense.

Thanks.

Signature · 1 line
tejun
#4 lore raw
Quoted 10 lines · Tejun Heo
Quoted 8 lines · Chen Ridong
Quoted 4 lines

With the given implementation (and use scenario), this'd better exposed in
cgroup.freeze.stat.local

Would it be possible to add this field to either cgroup.event or cgroup.stat? Since the frozen status is already tracked in cgroup.event, this placement would maintain better cohesion with existing metrics.

Quoted 6 lines
Quoted 1 line

This is just a suggestion.

Yeah, given that the freezer is an integral part of cgroup core, using cgroup.stat[.local] probably makes more sense.

Thanks.

One of the reasons I avoided cgroup.stat was because I interpreted its purpose to be for exposing cgroup metadata (i.e., descendants and descendants per subsystem), and I didn't think this value fit in neatly.

It doesn't seem like there currently exists a cgroup.stat.local, but if that is the preferred location for this accounting, I could create one and print it there!

Signature · 1 line
Tiffany Y. Yang
#5 lore raw
In reply to Chen Ridong

Hi Rindong,

Quoted 18 lines · Chen Ridong
Quoted 10 lines · Michal Koutný

Hello Tiffany.

Quoted 5 lines · Tiffany Yang
Documentation/admin-guide/cgroup-v2.rst +0 -0
@@ -1018,6 +1018,14 @@ All cgroup core files are prefixed with "cgroup."
  	it's possible to delete a frozen (and empty) cgroup, as well as
  	create new sub-cgroups.
Quoted 5 lines

Documentation/admin-guide/cgroup-v2.rst All cgroup core files are prefixed with "cgroup." · line 1021

Quoted 1 line
Documentation/admin-guide/cgroup-v2.rst +1 -0
+  cgroup.freeze.stat

With the given implementation (and use scenario), this'd better exposed in
cgroup.freeze.stat.local

Quoted 8 lines

Would it be possible to add this field to either cgroup.event or cgroup.stat? Since the frozen status is already tracked in cgroup.event, this placement would maintain better cohesion with existing metrics.

This is just a suggestion.

Best regards,
Ridong

Thanks for taking a look!

I don't think this would *quite* fit in cgroup.event because we're measuring when the cgroup begins freezing instead of when it reaches the frozen state. I also worry that having the value so close to cgroup.frozen would cause additional confusion about its meaning. cgroup.stat seems reasonable, but the values inside appear to be accounted for hierarchically, which wouldn't suit our use case.

Signature · 1 line
Tiffany Y. Yang
maintainer
#6 lore raw
In reply to Michal Koutný

Hello,

On Thu, Jul 17, 2025 at 02:56:13PM +0200, Michal Koutný wrote:
...

Quoted 6 lines
Quoted 1 line

+ cgroup.freeze.stat

With the given implementation (and use scenario), this'd better exposed in
cgroup.freeze.stat.local

Quoted 6 lines

I grok the hierarchical summing would make little sense and it'd make implementaion more complex. With that I'm thinking about formulation:

Cumulative time that cgroup has spent between freezing and thawing, regardless of whether by self or ancestor cgroups. NB (not) reaching "frozen" state is not accounted here.

I wonder what hierarchical summing would look like for this. It's absolute time interval measurement and I'm not sure whether summing up the descendants' durations is the best way to go about it. ie. Should it be the total duration any of the descendants are freezing or should it be sum of the freezing durations of all descendants?

Thanks.

Signature · 1 line
tejun
maintainer
#7 lore raw
Quoted 1 line · Tejun Heo

I wonder what hierarchical summing would look like for this.

So do I.
Thus I meant to expose this only in a *.local file not the hierarchical one.

But I realize it should [1] match cpu.stat[.local]:thottled_usec since they're similar quantities in principle.
- cpu.stat:thottled_usec
- sums the time the cgroup's quota was in effect
- not hierarchical (:-/)
- cpu.stat.local:thottled_usec
- not hierarchical
- sums the time cgroup's or ancestor's quota was in effect

    -> IIUC this is what's the motivation of the original patch

HTH,
Michal

[1] I'd find it more logical if
cpu.stat:thottled_usec were cpu.stat.local:thottling_usec and cpu.stat.local:thottled_usec were cpu.stat.local:throttled_usec. Only to illustrate my understanding of hierarchy in cpu.stat, it doesn't matter since it's what it is now.

#8 lore raw
Quoted 17 lines · Michal Koutný
Quoted 1 line · Tejun Heo

I wonder what hierarchical summing would look like for this.

So do I.
Thus I meant to expose this only in a *.local file not the hierarchical one.

But I realize it should [1] match cpu.stat[.local]:thottled_usec since they're similar quantities in principle.
- cpu.stat:thottled_usec
- sums the time the cgroup's quota was in effect
- not hierarchical (:-/)
- cpu.stat.local:thottled_usec
- not hierarchical
- sums the time cgroup's or ancestor's quota was in effect

    -> IIUC this is what's the motivation of the original patch
Quoted 8 lines

HTH,
Michal

[1] I'd find it more logical if
cpu.stat:thottled_usec were cpu.stat.local:thottling_usec and cpu.stat.local:thottled_usec were cpu.stat.local:throttled_usec. Only to illustrate my understanding of hierarchy in cpu.stat, it doesn't matter since it's what it is now.

Hi Michal and TJ,

I'd like to raise a separate thought unrelated to the current discussion.:)

With the recent merge of the series "cgroup: separate rstat trees," the rstat are not bound to CPU system. This makes me wonder: should we consider moving the cpu.stat and cpu.stat.local interfaces to the CPU subsystem?

The CPU subsystem could then align more closely with other resource controllers like memory or I/O subsystems. By decoupling these CPU-specific statistics from the cgroup core, it could help keep both cgroup and rstat implementations more focused.

Is there any particular reason why the CPU subsystem must remain bound to the cgroup core?

Looking forward to your insights.

Best regards,
Ridong

maintainer
#9 lore raw

cpu.stat in core or cpu controller (was Re: [RFC PATCH v2] cgroup: Track time in cgroup v2 freezer)

Quoted 3 lines · Chen Ridong

With the recent merge of the series "cgroup: separate rstat trees," the rstat are not bound to CPU system. This makes me wonder: should we consider moving the cpu.stat and cpu.stat.local interfaces to the CPU subsystem?

Note that fields printed in cpu.stat are combination of "core" and cpu controller values.

Quoted 3 lines

The CPU subsystem could then align more closely with other resource controllers like memory or I/O subsystems. By decoupling these CPU-specific statistics from the cgroup core, it could help keep both cgroup and rstat implementations more focused.

In my eyes, cpu controller is stuff encapsulated by cpu_cgrp_subsys. I'm not sure I understand what you refer to as the CPU subsystem.

One thing is how it is presented to users (filenames and content) another one is how it is implemented. The latter surely can be refactored but it's not obvious to me from the short description, sorry.

Quoted 2 lines

Is there any particular reason why the CPU subsystem must remain bound to the cgroup core?

The stuff that's bound to the core is essentially not "control" but only accounting, so with this association, the accounting can have fine granularity while control (which incurs higher overhead in principle) may remain coarse. I find it thus quite fitting that CPU stats build on top of rstat.
(Naturally, my previous claim about overhead is only rough and it's the reason for existence of adjustments like in the commit 34f26a15611af ("sched/psi: Per-cgroup PSI accounting disable/re-enable interface").)

Thats how I see it, happy to discuss possible problems you see with this.

Michal

#10 lore raw

Re: cpu.stat in core or cpu controller (was Re: [RFC PATCH v2] cgroup: Track time in cgroup v2 freezer)

Quoted 8 lines · Michal Koutný
Quoted 3 lines · Chen Ridong

With the recent merge of the series "cgroup: separate rstat trees," the rstat are not bound to CPU system. This makes me wonder: should we consider moving the cpu.stat and cpu.stat.local interfaces to the CPU subsystem?

Note that fields printed in cpu.stat are combination of "core" and cpu controller values.

Do you mean the "core" values are shown as below:
- usage_usec
- user_usec
- system_usec
- nice_usec

In the legacy cgroup, these values are in the cpuacct subsystem.

Quoted 11 lines
Quoted 3 lines

The CPU subsystem could then align more closely with other resource controllers like memory or I/O subsystems. By decoupling these CPU-specific statistics from the cgroup core, it could help keep both cgroup and rstat implementations more focused.

In my eyes, cpu controller is stuff encapsulated by cpu_cgrp_subsys. I'm not sure I understand what you refer to as the CPU subsystem.

One thing is how it is presented to users (filenames and content) another one is how it is implemented. The latter surely can be refactored but it's not obvious to me from the short description, sorry.

What I'm considering is moving the implementation of cpu.stat from cgroup_base_files to cpu_cgrp_subsys—without changing the user-facing interface (filenames and content remain the same). However, the interface would only appear if the CPU subsystem is enabled.

Currently, cpu.stat and cpu.stat.local are visible in every cgroup, even when the CPU subsystem is disabled. The only populated fields in such cases are:

- usage_usec
- user_usec
- system_usec
- nice_usec

I’m unsure whether this change would be acceptable?

Quoted 8 lines
Quoted 2 lines

Is there any particular reason why the CPU subsystem must remain bound to the cgroup core?

The stuff that's bound to the core is essentially not "control" but only accounting, so with this association, the accounting can have fine granularity while control (which incurs higher overhead in principle) may remain coarse. I find it thus quite fitting that CPU stats build on top of rstat.

The implementation would still rely on rstat, similar to memory.stat and io.stat. The goal is to decouple it from the cgroup core (cgroup.c and rstat.c) while preserving accounting granularity.

Best regards,
Ridong

Quoted 8 lines

(Naturally, my previous claim about overhead is only rough and it's the reason for existence of adjustments like in the commit 34f26a15611af ("sched/psi: Per-cgroup PSI accounting disable/re-enable interface").)

Thats how I see it, happy to discuss possible problems you see with this.

Michal

maintainer
#11 lore raw

Re: cpu.stat in core or cpu controller (was Re: [RFC PATCH v2] cgroup: Track time in cgroup v2 freezer)

On Sat, Jul 19, 2025 at 10:01:07AM +0800, Chen Ridong wrote:
...

Quoted 7 lines

What I'm considering is moving the implementation of cpu.stat from cgroup_base_files to cpu_cgrp_subsys—without changing the user-facing interface (filenames and content remain the same). However, the interface would only appear if the CPU subsystem is enabled.

Currently, cpu.stat and cpu.stat.local are visible in every cgroup, even when the CPU subsystem is disabled. The only populated fields in such cases are:

Quoted 6 lines

- usage_usec
- user_usec
- system_usec
- nice_usec

I’m unsure whether this change would be acceptable?

I don't think so and don't really see what benefits moving the stats would bring. Why would we move these?

Thanks.

Signature · 1 line
tejun
#12 lore raw

Re: cpu.stat in core or cpu controller (was Re: [RFC PATCH v2] cgroup: Track time in cgroup v2 freezer)

Quoted 14 lines · Tejun Heo

On Sat, Jul 19, 2025 at 10:01:07AM +0800, Chen Ridong wrote:
...

Quoted 12 lines

What I'm considering is moving the implementation of cpu.stat from cgroup_base_files to cpu_cgrp_subsys—without changing the user-facing interface (filenames and content remain the same). However, the interface would only appear if the CPU subsystem is enabled.

Currently, cpu.stat and cpu.stat.local are visible in every cgroup, even when the CPU subsystem is disabled. The only populated fields in such cases are:

- usage_usec
- user_usec
- system_usec
- nice_usec

Quoted 7 lines
Quoted 1 line

I’m unsure whether this change would be acceptable?

I don't think so and don't really see what benefits moving the stats would bring. Why would we move these?

Thanks.

Thank you for your attention. My intention is to better modularize the cgroup code by moving CPU subsystem-specific statistics out of the core cgroup implementation (cgroup.c and rstat.c).

Specifically, this change would allow us to:

1.Remove these CPU-specific callbacks from the core:
css_extra_stat_show()
css_local_stat_show()
2. Clean up the 'is_self' logic in rstat.c.
3. Make the stat handling consistent across subsystems (currently cpu.stat is the only subsystem-specific stat implemented in the core).

Best regards,
Ridong.

maintainer
#13 lore raw

Re: cpu.stat in core or cpu controller (was Re: [RFC PATCH v2] cgroup: Track time in cgroup v2 freezer)

Quoted 6 lines · Chen Ridong

Specifically, this change would allow us to:

1.Remove these CPU-specific callbacks from the core:
css_extra_stat_show() css_local_stat_show()
2. Clean up the 'is_self' logic in rstat.c.

If you see an option to organize the code better, why not. (At the same time, I currently also don't see the "why.)

Quoted 2 lines

3. Make the stat handling consistent across subsystems (currently cpu.stat is the only subsystem-specific stat implemented in the core).

But beware that the possibility of having cpu.stat without enabling the cpu controller on v2 is a user visible behavior and I'm quite sure some userspace relies on it, so you'd need to preserve that.

Michal

#14 lore raw

Re: cpu.stat in core or cpu controller (was Re: [RFC PATCH v2] cgroup: Track time in cgroup v2 freezer)

Quoted 12 lines · Michal Koutný
Quoted 6 lines · Chen Ridong

Specifically, this change would allow us to:

1.Remove these CPU-specific callbacks from the core:
css_extra_stat_show() css_local_stat_show()
2. Clean up the 'is_self' logic in rstat.c.

If you see an option to organize the code better, why not. (At the same time, I currently also don't see the "why.)

Quoted 7 lines
Quoted 2 lines

3. Make the stat handling consistent across subsystems (currently cpu.stat is the only subsystem-specific stat implemented in the core).

But beware that the possibility of having cpu.stat without enabling the cpu controller on v2 is a user visible behavior and I'm quite sure some userspace relies on it, so you'd need to preserve that.

This is what I worry about. Thank you for your confirmation.

Best regards,
Ridong

maintainer
#15 lore raw

Re: cpu.stat in core or cpu controller (was Re: [RFC PATCH v2] cgroup: Track time in cgroup v2 freezer)

Quoted 5 lines · Chen Ridong
Quoted 3 lines

But beware that the possibility of having cpu.stat without enabling the cpu controller on v2 is a user visible behavior and I'm quite sure some userspace relies on it, so you'd need to preserve that.

This is what I worry about. Thank you for your confirmation.

Yeah, this was an intentional decision - sacrificing a bit of code org cleanliness for everyday usefulness. Enabling CPU controller can have substantial overhead and having cpu stats available by default doesn't cost much while improving usefulness.

Thanks.

Signature · 1 line
tejun
#16 lore raw

Re: cpu.stat in core or cpu controller (was Re: [RFC PATCH v2] cgroup: Track time in cgroup v2 freezer)

Quoted 5 lines · Tejun Heo
Quoted 4 lines · Chen Ridong
Quoted 3 lines

But beware that the possibility of having cpu.stat without enabling the cpu controller on v2 is a user visible behavior and I'm quite sure some userspace relies on it, so you'd need to preserve that.

Quoted 9 lines
Quoted 1 line

This is what I worry about. Thank you for your confirmation.

Yeah, this was an intentional decision - sacrificing a bit of code org cleanliness for everyday usefulness. Enabling CPU controller can have substantial overhead and having cpu stats available by default doesn't cost much while improving usefulness.

Thanks.

Thank you Tj, This is clear now.

Best regards,
Ridong

#17 lore raw
In reply to Michal Koutný
Quoted 32 lines · Michal Koutný

I'd like to incorporate the reason from your other mail:

| Since there isn't yet a clear way to identify a set of "lost" time
| that everyone (or at least a wider group of users) cares about, it
| seems like iterating over components of interest is the best way

into this commit message (because that's a stronger ponit that your use case alone).

Quoted 1 line

Any feedback would be much appreciated!

I can see benefits of this new stat field conceptually, I have some remarks to implementation and suggestions to conventions below.

Documentation/admin-guide/cgroup-v2.rst All cgroup core files are prefixed with "cgroup." · line 1021

Quoted 7 lines
Documentation/admin-guide/cgroup-v2.rst +1 -0
@@ -1018,6 +1018,14 @@ All cgroup core files are prefixed with "cgroup."
  	it's possible to delete a frozen (and empty) cgroup, as well as
  	create new sub-cgroups.
 
+  cgroup.freeze.stat

With the given implementation (and use scenario), this'd better exposed in
cgroup.freeze.stat.local

I grok the hierarchical summing would make little sense and it'd make implementaion more complex. With that I'm thinking about formulation:

Cumulative time that cgroup has spent between freezing and thawing, regardless of whether by self or ancestor cgroups. NB (not) reaching "frozen" state is not accounted here.

Quoted 10 lines

Documentation/admin-guide/cgroup-v2.rst All cgroup core files are prefixed with "cgroup." · line 1028

Quoted 7 lines
Documentation/admin-guide/cgroup-v2.rst +7 -0
+	A read-only flat-keyed file which exists in non-root cgroups.
+	The following entry is defined:
+
+	  freeze_time_total_ns
+		Cumulative time that this cgroup has spent in the freezing
+		state, regardless of whether or not it reaches "frozen".
+

Rather use microseconds, it's the cgroup API convention and I'm not sure nanosecods exposed here are the needed precision.

Ack.

Quoted 7 lines
        1    _____

frozen 0 __/ \__
ab cd

Yeah, I find the mesurent between a and c the sanest.

Quoted 15 lines

Documentation/admin-guide/cgroup-v2.rst All cgroup core files are prefixed with "cgroup."

Quoted 11 lines
Documentation/admin-guide/cgroup-v2.rst +11 -0
+static int cgroup_freeze_stat_show(struct seq_file *seq, void *v)
+{
+	struct cgroup *cgrp = seq_css(seq)->cgroup;
+	u64 freeze_time = 0;
+
+	spin_lock_irq(&css_set_lock);
+	if (test_bit(CGRP_FREEZE, &cgrp->flags))
+		freeze_time = ktime_get_ns() - cgrp->freezer.freeze_time_start_ns;
+
+	freeze_time += cgrp->freezer.freeze_time_total_ns;
+	spin_unlock_irq(&css_set_lock);

I don't like taking this spinlock only for the matter of reading this attribute. The intention should be to keep the (un)freezeing mostly unaffected at the expense of these readers (seqcount or u64 stats?).

Ah, thank you for this suggestion! I noticed that none of the other seq_file read implementations took a lock, so I thought this might be a point of contention. I'll try a seqlock in the next version of the patch.

Quoted 5 lines

Alternative approach: either there's outer watcher who can be notified by cgroup.events:frozen or it's an inner watcher who couldn't actively read the field anyway. So the field could only show completed freeze/thaw cycles from the past (i.e. not substitute clock_gettime(2) when the cgroup is frozen), which could simplify querying the flag too.

This is a good observation. This approach does simplify things, but even though it would work for our use case, I feel like this value would be less useful for the outer watcher case, especially in the case where the cgroup never reaches the frozen state.

Quoted 8 lines

Documentation/admin-guide/cgroup-v2.rst static struct cgroup *cgroup_create(struct cgroup · line 5784

Quoted 6 lines
Documentation/admin-guide/cgroup-v2.rst +1 -0
@@ -5758,6 +5780,7 @@ static struct cgroup *cgroup_create(struct cgroup  
 *parent, const char *name,
  	 * if the parent has to be frozen, the child has too.
  	 */
  	cgrp->freezer.e_freeze = parent->freezer.e_freeze;
+	cgrp->freezer.freeze_time_total_ns = 0;

struct cgroup is kzalloc'd, this is unnecessary

Thank you for all your feedback! I'll make sure to incorporate these suggestions into the next version.

Signature · 1 line
Tiffany Y. Yang