Skip to contentSkip to conversation

cpuacct: add a branch prediction

PATCH In review Li Zefan 9 participants 42 messages Feb 26, 2009 – Mar 4, 2009

lore
Share & tools
lore

Li Zefan

author
#0 lore raw

PATCH 1/1 cpuacct: add a branch prediction

cpuacct_charge() is in fast-path, and checking of !cpuacct_susys.active always returns false after cpuacct has been initialized at system boot.

---
 kernel/sched.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
kernel/sched.c +1 -1
index 410eec4..fd2f7fc 100644
@@ -9589,7 +9589,7 @@ static void cpuacct_charge(struct task_struct *tsk, u64 cputime)
 	struct cpuacct *ca;
 	int cpu;
 
-	if (!cpuacct_subsys.active)
+	if (unlikely(!cpuacct_subsys.active))
 		return;
 
 	cpu = task_cpu(tsk);
Signature · 1 line
1.5.4.rc3
#1 lore raw
Quoted 13 lines · On Thu, 26 Feb 2009 15:40:15 +0800 Li Zefan <[email protected]> wrote:

cpuacct_charge() is in fast-path, and checking of !cpuacct_susys.active always returns false after cpuacct has been initialized at system boot.

---
 kernel/sched.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
kernel/sched.c +0 -0
index 410eec4..fd2f7fc 100644

kernel/sched.c static void cpuacct_charge(struct task_struct *tsk, u64 cputime) · line 9594

Quoted 7 lines
kernel/sched.c +1 -1
@@ -9589,7 +9589,7 @@ static void cpuacct_charge(struct task_struct *tsk, u64 cputime)
 	struct cpuacct *ca;
 	int cpu;
 
-	if (!cpuacct_subsys.active)
+	if (unlikely(!cpuacct_subsys.active))
 		return;
 

(Just curious)
I wonder "ca = task_ca(tsk)" will return NULL if cpuacct subsys is not initalized. Then, can we just remove this check ?

Thanks,
-Kame

author
#2 lore raw
Quoted 19 lines · KAMEZAWA Hiroyuki
Quoted 16 lines · On Thu, 26 Feb 2009 15:40:15 +0800 Li Zefan <[email protected]> wrote:

cpuacct_charge() is in fast-path, and checking of !cpuacct_susys.active always returns false after cpuacct has been initialized at system boot.

---
 kernel/sched.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
kernel/sched.c +0 -0
index 410eec4..fd2f7fc 100644
@@ -9589,7 +9589,7 @@ static void cpuacct_charge(struct task_struct *tsk, u64 cputime)
 	struct cpuacct *ca;
 	int cpu;
 
Quoted 6 lines

kernel/sched.c static void cpuacct_charge(struct task_struct *tsk, u64 cputime) · line 9594

Quoted 4 lines
kernel/sched.c +1 -1
-	if (!cpuacct_subsys.active)
+	if (unlikely(!cpuacct_subsys.active))
 		return;
 

(Just curious)
I wonder "ca = task_ca(tsk)" will return NULL if cpuacct subsys is not initalized.

Yes, it will be NULL, and that's why we need this check.

Quoted 2 lines

Then, can we just remove this check ?

cpuacct_charge() can be called before cpuacct is initialized, so we have to check this case here.

#3 lore raw
Quoted 27 lines · On Thu, 26 Feb 2009 16:17:31 +0800 Li Zefan <[email protected]> wrote:
Quoted 19 lines · KAMEZAWA Hiroyuki
Quoted 16 lines · On Thu, 26 Feb 2009 15:40:15 +0800 Li Zefan <[email protected]> wrote:

cpuacct_charge() is in fast-path, and checking of !cpuacct_susys.active always returns false after cpuacct has been initialized at system boot.

---
 kernel/sched.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
kernel/sched.c +0 -0
index 410eec4..fd2f7fc 100644
@@ -9589,7 +9589,7 @@ static void cpuacct_charge(struct task_struct *tsk, u64 cputime)
 	struct cpuacct *ca;
 	int cpu;
 
Quoted 6 lines

kernel/sched.c static void cpuacct_charge(struct task_struct *tsk, u64 cputime) · line 9594

Quoted 4 lines
kernel/sched.c +1 -1
-	if (!cpuacct_subsys.active)
+	if (unlikely(!cpuacct_subsys.active))
 		return;
 

(Just curious)
I wonder "ca = task_ca(tsk)" will return NULL if cpuacct subsys is not initalized.

Quoted 8 lines

Yes, it will be NULL, and that's why we need this check.

Quoted 2 lines

Then, can we just remove this check ?

cpuacct_charge() can be called before cpuacct is initialized, so we have to check this case here.

My point is,

ca = task_ca(tsk)
for (; ca; ca->parent) {
...
}

What is problem even if ca is NULL.

Thanks,
-Kame

author
#4 lore raw
Quoted 29 lines · KAMEZAWA Hiroyuki
Quoted 26 lines · On Thu, 26 Feb 2009 16:17:31 +0800 Li Zefan <[email protected]> wrote:
Quoted 25 lines · KAMEZAWA Hiroyuki
Quoted 13 lines · On Thu, 26 Feb 2009 15:40:15 +0800 Li Zefan <[email protected]> wrote:

cpuacct_charge() is in fast-path, and checking of !cpuacct_susys.active always returns false after cpuacct has been initialized at system boot.

---
 kernel/sched.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
kernel/sched.c +0 -0
index 410eec4..fd2f7fc 100644

kernel/sched.c static void cpuacct_charge(struct task_struct *tsk, u64 cputime) · line 9594

Quoted 7 lines
kernel/sched.c +1 -1
@@ -9589,7 +9589,7 @@ static void cpuacct_charge(struct task_struct *tsk, u64 cputime)
 	struct cpuacct *ca;
 	int cpu;
 
-	if (!cpuacct_subsys.active)
+	if (unlikely(!cpuacct_subsys.active))
 		return;
 

(Just curious)
I wonder "ca = task_ca(tsk)" will return NULL if cpuacct subsys is not initalized.

Quoted 9 lines
Quoted 7 lines

Yes, it will be NULL, and that's why we need this check.

Quoted 2 lines

Then, can we just remove this check ?

cpuacct_charge() can be called before cpuacct is initialized, so we have to check this case here.

My point is,

Ah, I see.

Quoted 5 lines

ca = task_ca(tsk)

for (; ca; ca->parent) {
   ...
}

ca is not checked before hierarchy support, and it's a side-effect.

Before cpuacct is initialized, css == task->cgroups->subsys[cpuacct_subsys] == NULL, but ca = task_ca(tsk) is not necessarily NULL, unless struct cgroup_subsys_state is the first member of struct cpuacct.

And the above code actually should be:

do {
	...
} while (ca->parent);
Quoted 2 lines

What is problem even if ca is NULL.

#37 lore raw
In reply to Li Zefan
Quoted 13 lines · Li Zefan

cpuacct_charge() is in fast-path, and checking of !cpuacct_susys.active always returns false after cpuacct has been initialized at system boot.

---
 kernel/sched.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
kernel/sched.c +0 -0
index 410eec4..fd2f7fc 100644

kernel/sched.c static void cpuacct_charge(struct task_struct *tsk, u64 cputime) · line 9592

Quoted 5 lines
kernel/sched.c +1 -1
@@ -9589,7 +9589,7 @@ static void cpuacct_charge(struct task_struct *tsk, u64 cputime)
 	struct cpuacct *ca;
 	int cpu;
 
-	if (!cpuacct_subsys.active)
+	if (unlikely(!cpuacct_subsys.active))

What happens if most distros make cpuacct_subsys active? Can we decide on unlikely depending on how the system is configured?

Signature · 1 line
	Balbir
author
#38 lore raw
Quoted 18 lines · Balbir Singh
Quoted 16 lines · Li Zefan

cpuacct_charge() is in fast-path, and checking of !cpuacct_susys.active always returns false after cpuacct has been initialized at system boot.

---
 kernel/sched.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
kernel/sched.c +0 -0
index 410eec4..fd2f7fc 100644
@@ -9589,7 +9589,7 @@ static void cpuacct_charge(struct task_struct *tsk, u64 cputime)
 	struct cpuacct *ca;
 	int cpu;
 
Quoted 6 lines

kernel/sched.c static void cpuacct_charge(struct task_struct *tsk, u64 cputime) · line 9592

Quoted 2 lines
kernel/sched.c +1 -1
-	if (!cpuacct_subsys.active)
+	if (unlikely(!cpuacct_subsys.active))

What happens if most distros make cpuacct_subsys active? Can we decide on unlikely depending on how the system is configured?

Note, it's ->active but not ->disabled. ;)

And note, I make it "unlikely(!cpuacct_subsys.active)", but not "unlikely(cpuacct_subsys.active)"

->active can't be configured by users. If CONFIG_CGROUP_CPUACCT is enabled, cpuacct_subsys.active will be set to true after cpuacct is initialized.

#5 lore raw
In reply to Li Zefan
Quoted 6 lines · On Thu, 26 Feb 2009 16:35:33 +0800 Li Zefan <[email protected]> wrote:

ca is not checked before hierarchy support, and it's a side-effect.

Before cpuacct is initialized, css == task->cgroups->subsys[cpuacct_subsys] == NULL, but ca = task_ca(tsk) is not necessarily NULL, unless struct cgroup_subsys_state is the first member of struct cpuacct.

Quoted 6 lines

And the above code actually should be:

do {
	...
} while (ca->parent);

I'll send no more objections to this patch itself.

But IMHO, this loop is tooo bad, I think. The hierarchy information should be gathered by read-side and the total code should be

void charge_statistics(tsk, cputime)
{
ca = task_ca(tsk)
/* ca can be null while init */

   if (likely(ca)) {
     u64 *cpuusage = percpu_ptr(ca->cpuusage, task_cpu(tsk))
     *cpuusage += cputime

}
}
(Need changes to read-side and create/destroy of cpuacct cgroups.)

Thanks,
-Kame

#41 lore raw
In reply to Balbir Singh
Quoted 18 lines · On Thu, 26 Feb 2009 14:07:02 +0530 Balbir Singh <[email protected]>…
Quoted 16 lines · Li Zefan

cpuacct_charge() is in fast-path, and checking of !cpuacct_susys.active always returns false after cpuacct has been initialized at system boot.

---
 kernel/sched.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
kernel/sched.c +0 -0
index 410eec4..fd2f7fc 100644
@@ -9589,7 +9589,7 @@ static void cpuacct_charge(struct task_struct *tsk, u64 cputime)
 	struct cpuacct *ca;
 	int cpu;
 
Quoted 6 lines

kernel/sched.c static void cpuacct_charge(struct task_struct *tsk, u64 cputime) · line 9592

Quoted 2 lines
kernel/sched.c +1 -1
-	if (!cpuacct_subsys.active)
+	if (unlikely(!cpuacct_subsys.active))

What happens if most distros make cpuacct_subsys active? Can we decide on unlikely depending on how the system is configured?

"active" just guarantees task_ca(tsk) is not NULL and subsystem is initialized.

Thanks,
-Kame

#6 lore raw
In reply to KAMEZAWA Hiroyuki
Quoted 11 lines · KAMEZAWA Hiroyuki
Quoted 8 lines · On Thu, 26 Feb 2009 16:35:33 +0800 Li Zefan <[email protected]> wrote:

ca is not checked before hierarchy support, and it's a side-effect.

Before cpuacct is initialized, css == task->cgroups->subsys[cpuacct_subsys] == NULL, but ca = task_ca(tsk) is not necessarily NULL, unless struct cgroup_subsys_state is the first member of struct cpuacct.

And the above code actually should be:

Quoted 8 lines
Quoted 4 lines

do {
      ...
} while (ca->parent);

I'll send no more objections to this patch itself.

But IMHO, this loop is tooo bad, I think. The hierarchy information should be gathered by read-side and the total code should be

How do you take care of accounting when a group gets deleted ?. Wouldn't you loose some information due to deleted groups if you want to accumulate hierarchical stats only during read ?

Regards,
Bharata.

Signature · 1 line
http://bharata.sulekha.com/blog/posts.htm
#7 lore raw

Bharata B Rao:

Quoted 19 lines
Quoted 17 lines · KAMEZAWA Hiroyuki
Quoted 14 lines · On Thu, 26 Feb 2009 16:35:33 +0800 Li Zefan <[email protected]> wrote:

ca is not checked before hierarchy support, and it's a side-effect.

Before cpuacct is initialized, css == task->cgroups->subsys[cpuacct_subsys] == NULL,
but ca = task_ca(tsk) is not necessarily NULL, unless struct cgroup_subsys_state is the
first member of struct cpuacct.

And the above code actually should be:

do {
&#160; &#160; &#160; ...
} while (ca->parent);

Quoted 7 lines
Quoted 5 lines

I'll send no more objections to this patch itself.

But IMHO, this loop is tooo bad, I think. The hierarchy information should be gathered by
read-side and the total code should be

How do you take care of accounting when a group gets deleted ?.

propagate accounts to the parent or root group if deleted.

Quoted 3 lines

Wouldn't you loose some information due to deleted groups if you want to accumulate hierarchical stats only during read ?

Taking hierarchy mutex while reading will make read-side stable. I haven't considered in detail but I wrote CSS ID and scan hierarchy code to do this kind of "visit all css object under hierarchy" jobs. I think you can catch what I mean by reading what memory.stat file of mem cgroup does. (mmotm)

Thanks,
-Kame

#39 lore raw
In reply to Li Zefan
Quoted 19 lines · Li Zefan
Quoted 18 lines · Balbir Singh
Quoted 16 lines · Li Zefan

cpuacct_charge() is in fast-path, and checking of !cpuacct_susys.active always returns false after cpuacct has been initialized at system boot.

---
 kernel/sched.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
kernel/sched.c +0 -0
index 410eec4..fd2f7fc 100644
@@ -9589,7 +9589,7 @@ static void cpuacct_charge(struct task_struct *tsk, u64 cputime)
 	struct cpuacct *ca;
 	int cpu;
 
Quoted 9 lines
Quoted 6 lines

kernel/sched.c static void cpuacct_charge(struct task_struct *tsk, u64 cputime) · line 9592

Quoted 2 lines
kernel/sched.c +1 -1
-	if (!cpuacct_subsys.active)
+	if (unlikely(!cpuacct_subsys.active))

What happens if most distros make cpuacct_subsys active? Can we decide on unlikely depending on how the system is configured?

Note, it's ->active but not ->disabled. ;)

I've seen that

Quoted 5 lines

And note, I make it "unlikely(!cpuacct_subsys.active)", but not "unlikely(cpuacct_subsys.active)"

->active can't be configured by users. If CONFIG_CGROUP_CPUACCT is enabled, cpuacct_subsys.active will be set to true after cpuacct is initialized.

My point is the likely/unlikely in this case depends on turning on/off CONFIG_CGROUP_CPUACCT? Am I missing something?

Signature · 1 line
	Balbir
#40 lore raw
Quoted 2 lines · Balbir Singh

My point is the likely/unlikely in this case depends on turning on/off CONFIG_CGROUP_CPUACCT? Am I missing something?

Yeah, that whole function is a stub for !CPUACCT

#ifdef CONFIG_CGROUP_CPUACCT
static void cpuacct_charge(struct task_struct *tsk, u64 cputime); #else
static inline void cpuacct_charge(struct task_struct *tsk, u64 cputime) {} #endif

#8 lore raw
In reply to KAMEZAWA Hiroyuki
Quoted 1 line · KAMEZAWA Hiroyuki

Taking hierarchy mutex while reading will make read-side stable.

We're talking about scheduling here, taking a mutex to stop scheduling won't work, nor will it be acceptible to use anything that will.

#9 lore raw

Peter Zijlstra wrote:

Quoted 7 lines · KAMEZAWA Hiroyuki
Quoted 2 lines

Taking hierarchy mutex while reading will make read-side stable.

We're talking about scheduling here, taking a mutex to stop scheduling won't work, nor will it be acceptible to use anything that will.

I'm sorry but I thought we are talking about cpuacct subsystem and not cpu subsystem....cpuacct subsystem's information is used for schedule ?

Is there a user of information gathered by cpuacct subsystem in the kernel ? I thought only user(reader) is the user who read cpuacct.usage file.

Thanks,
-Kame

#11 lore raw
In reply to Peter Zijlstra

Peter Zijlstra さんは書きました:

Quoted 7 lines · KAMEZAWA Hiroyuki
Quoted 2 lines

Taking hierarchy mutex while reading will make read-side stable.

We're talking about scheduling here, taking a mutex to stop scheduling won't work, nor will it be acceptible to use anything that will.

No mutex is necessary, anyway.
hierarchy-walker function completely works well under rcu read lock, if small jitter is allowed.

Thanks,
-Kame

#10 lore raw
In reply to KAMEZAWA Hiroyuki
Quoted 10 lines · KAMEZAWA Hiroyuki

Peter Zijlstra wrote:

Quoted 7 lines · KAMEZAWA Hiroyuki
Quoted 2 lines

Taking hierarchy mutex while reading will make read-side stable.

We're talking about scheduling here, taking a mutex to stop scheduling won't work, nor will it be acceptible to use anything that will.

I'm sorry but I thought we are talking about cpuacct subsystem and not cpu subsystem....cpuacct subsystem's information is used for schedule ?

Not used for scheduling, but directly generated by it.

#12 lore raw
In reply to KAMEZAWA Hiroyuki
Quoted 11 lines · KAMEZAWA Hiroyuki

Peter Zijlstra さんは書きました:

Quoted 7 lines · KAMEZAWA Hiroyuki
Quoted 2 lines

Taking hierarchy mutex while reading will make read-side stable.

We're talking about scheduling here, taking a mutex to stop scheduling won't work, nor will it be acceptible to use anything that will.

No mutex is necessary, anyway.
hierarchy-walker function completely works well under rcu read lock, if small jitter is allowed.

Right, should be doable -- and looking at the code, we have this horrible 32 bit exception in there that locks the rq in order to read the 64bit value.

Would be grand to get rid of that,. how bad would it be for userspace to get the occasionally fubarred value?

But aside from that, the cpu controller itself is also summing directly up the hierarchy, so cpuacct doing the same doesn't seem odd.

#13 lore raw

Peter Zijlstra wrote:

Quoted 13 lines · KAMEZAWA Hiroyuki
Quoted 11 lines

Peter Zijlstra wrote:

Quoted 7 lines · KAMEZAWA Hiroyuki
Quoted 2 lines

Taking hierarchy mutex while reading will make read-side stable.

We're talking about scheduling here, taking a mutex to stop scheduling won't work, nor will it be acceptible to use anything that will.

No mutex is necessary, anyway.
hierarchy-walker function completely works well under rcu read lock, if small jitter is allowed.

Quoted 8 lines

Right, should be doable -- and looking at the code, we have this horrible 32 bit exception in there that locks the rq in order to read the 64bit value.

Would be grand to get rid of that,. how bad would it be for userspace to get the occasionally fubarred value?

From view of user-support saler, if terrible broken value is reported,

it will be user-incident and annoy me(us) ;)

I'd like to get rid of rq->lock, too..Hmm.. some routine like atomic64_read() can help this ? (But I don't want to use atomic_t here..)

Quoted 3 lines

But aside from that, the cpu controller itself is also summing directly up the hierarchy, so cpuacct doing the same doesn't seem odd.

I'll post some idea if I can think of something reasonable. But I tend to hesitate to modify sched.c ;)

Thanks,
-Kame

#14 lore raw
Quoted 2 lines · KAMEZAWA Hiroyuki

Hmm.. some routine like
atomic64_read() can help this ? (But I don't want to use atomic_t here..)

Yeah, atomic64_t has been proposed numerous times, and x86 could actually implement that using cmpxchg8b, just not sure about all the other 32bit archs, and if we start using it in the scheduler, they'd better have it implemented.

Furthermore, it would require all the ops to be atomic, which would require them all to use cmpxchg8b and its a horridly expensive op to use :-(

#15 lore raw
Quoted 8 lines · Peter Zijlstra
Quoted 2 lines · KAMEZAWA Hiroyuki

Hmm.. some routine like
atomic64_read() can help this ? (But I don't want to use atomic_t here..)

Yeah, atomic64_t has been proposed numerous times, and x86 could actually implement that using cmpxchg8b, just not sure about all the other 32bit archs, and if we start using it in the scheduler, they'd better have it implemented.

I have written a working atomic64_t implementation for tip:perfcounters/core, for 32-bit x86.

Ingo

#16 lore raw
Quoted 8 lines · Ingo Molnar
Quoted 5 lines

Yeah, atomic64_t has been proposed numerous times, and x86 could actually implement that using cmpxchg8b, just not sure about all the other 32bit archs, and if we start using it in the scheduler, they'd better have it implemented.

I have written a working atomic64_t implementation for tip:perfcounters/core, for 32-bit x86.

It should also be possible to write an asm-generic atomic64_t implementation based on the parisc atomic_t hashlock code. For non-SMP configurations, it falls back on local_irq_save(), which is already the method used for atomic_t on half the embedded architectures.

	Arnd <><
#18 lore raw
In reply to KAMEZAWA Hiroyuki
Quoted 18 lines · KAMEZAWA Hiroyuki

Peter Zijlstra wrote:

Quoted 17 lines · KAMEZAWA Hiroyuki
Quoted 11 lines

Peter Zijlstra wrote:

Quoted 7 lines · KAMEZAWA Hiroyuki
Quoted 2 lines

Taking hierarchy mutex while reading will make read-side stable.

We're talking about scheduling here, taking a mutex to stop scheduling won't work, nor will it be acceptible to use anything that will.

No mutex is necessary, anyway.
hierarchy-walker function completely works well under rcu read lock, if small jitter is allowed.

Right, should be doable -- and looking at the code, we have this horrible 32 bit exception in there that locks the rq in order to read the 64bit value.

Quoted 8 lines
Quoted 4 lines

Would be grand to get rid of that,. how bad would it be for userspace to get the occasionally fubarred value?

From view of user-support saler, if terrible broken value is reported,

it will be user-incident and annoy me(us) ;)

I'd like to get rid of rq->lock, too..Hmm.. some routine like atomic64_read() can help this ? (But I don't want to use atomic_t here..)

atomic64_read() will not help you on a 32-bit machine. Here is the sequence of events that will cause the aforementioned user incidents and consequent annoyance:

o The value of the counter is (2^32)-1, or 0xffffffff.

o CPU 0 reads the high-order 32 bits of the counter, getting zero.

o CPU 1 increments the low-order 32 bits of the counter, resulting
in zero, but notes that there is a carry out of this field.

o CPU 0 reads the low-order 32 bits of the counter, getting zero.

o CPU 1 increments the high-order 32 bits of the counter, so that

	the new value of the counter is 2^32, or 0x100000000.

So CPU 0 gets a value that is -way- off.

The usual trick is something like the following for counter read:

1. Read the high-order 32 bits of the counter.

2. Do a memory barrier, smp_mb().

3. Read the low-order 32 bits of the counter.

4. Do another memory barrier, again smp_mb().

5. Read the high-order 32 bits of the counter again.

If it is the same as the value obtained in step 1 (or the previous execution of step 5), then we are done. (This works even in case of complete 64-bit overflow, though we should be very lucky to live that long!) Otherwise, go to step 2.

But it is also necessary to modify the counter update:

1. Increment the low-order 32 bits of the counter. If no overflow
occurred, we are done, otherwise, continue through this sequence of steps.

2. Do a memory barrier, smp_mb().

3. Increment the high-order 32 bits of the counter.

How to detect overflow in step 1? Well, if we are incrementing, we can just test for the new value being zero. Otherwise, if we are adding a 32-bit number, if the new value of the low-order 32 bits of counter is less than the old value, overflow occurred (but make sure that the comparison is unsigned!).

This all assumes that you are adding a 32-bit quantity to the counter. Adding 64-bit values is not much harder.

Does this approach work for you?

Thanx, Paul

Quoted 14 lines
Quoted 3 lines

But aside from that, the cpu controller itself is also summing directly up the hierarchy, so cpuacct doing the same doesn't seem odd.

I'll post some idea if I can think of something reasonable. But I tend to hesitate to modify sched.c ;)

Thanks,
-Kame

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#19 lore raw
Quoted 79 lines · On Thu, 26 Feb 2009 08:45:09 -0800 "Paul E. McKenney"…
Quoted 18 lines · KAMEZAWA Hiroyuki

Peter Zijlstra wrote:

Quoted 17 lines · KAMEZAWA Hiroyuki
Quoted 11 lines

Peter Zijlstra wrote:

Quoted 7 lines · KAMEZAWA Hiroyuki
Quoted 2 lines

Taking hierarchy mutex while reading will make read-side stable.

We're talking about scheduling here, taking a mutex to stop scheduling won't work, nor will it be acceptible to use anything that will.

No mutex is necessary, anyway.
hierarchy-walker function completely works well under rcu read lock, if small jitter is allowed.

Right, should be doable -- and looking at the code, we have this horrible 32 bit exception in there that locks the rq in order to read the 64bit value.

Quoted 8 lines
Quoted 4 lines

Would be grand to get rid of that,. how bad would it be for userspace to get the occasionally fubarred value?

From view of user-support saler, if terrible broken value is reported,

it will be user-incident and annoy me(us) ;)

I'd like to get rid of rq->lock, too..Hmm.. some routine like atomic64_read() can help this ? (But I don't want to use atomic_t here..)

atomic64_read() will not help you on a 32-bit machine. Here is the sequence of events that will cause the aforementioned user incidents and consequent annoyance:

o The value of the counter is (2^32)-1, or 0xffffffff.

o CPU 0 reads the high-order 32 bits of the counter, getting zero.

o CPU 1 increments the low-order 32 bits of the counter, resulting
in zero, but notes that there is a carry out of this field.

o CPU 0 reads the low-order 32 bits of the counter, getting zero.

o CPU 1 increments the high-order 32 bits of the counter, so that

	the new value of the counter is 2^32, or 0x100000000.

So CPU 0 gets a value that is -way- off.

The usual trick is something like the following for counter read:

1. Read the high-order 32 bits of the counter.

2. Do a memory barrier, smp_mb().

3. Read the low-order 32 bits of the counter.

4. Do another memory barrier, again smp_mb().

5. Read the high-order 32 bits of the counter again.

If it is the same as the value obtained in step 1 (or the previous execution of step 5), then we are done. (This works even in case of complete 64-bit overflow, though we should be very lucky to live that long!) Otherwise, go to step 2.

But it is also necessary to modify the counter update:

1. Increment the low-order 32 bits of the counter. If no overflow
occurred, we are done, otherwise, continue through this sequence of steps.

2. Do a memory barrier, smp_mb().

3. Increment the high-order 32 bits of the counter.

How to detect overflow in step 1? Well, if we are incrementing, we can just test for the new value being zero. Otherwise, if we are adding a 32-bit number, if the new value of the low-order 32 bits of counter is less than the old value, overflow occurred (but make sure that the comparison is unsigned!).

Quoted 5 lines

This all assumes that you are adding a 32-bit quantity to the counter. Adding 64-bit values is not much harder.

Does this approach work for you?

Thank you. I'll try some and post if it seems easy to read/merge. Hmm, but in your approach, can't we see the counter goes backword ? (if the reader see only low 32 bit is incremtend.)

Can't we use seq_counter in include/linux/seqlock.h ? There is only one writer and we don't need write-side lock.

Thanks,
-Kame

#20 lore raw
Quoted 85 lines · KAMEZAWA Hiroyuki
Quoted 82 lines · On Thu, 26 Feb 2009 08:45:09 -0800 "Paul E. McKenney"…
Quoted 18 lines · KAMEZAWA Hiroyuki

Peter Zijlstra wrote:

Quoted 17 lines · KAMEZAWA Hiroyuki
Quoted 11 lines

Peter Zijlstra wrote:

Quoted 7 lines · KAMEZAWA Hiroyuki
Quoted 2 lines

Taking hierarchy mutex while reading will make read-side stable.

We're talking about scheduling here, taking a mutex to stop scheduling won't work, nor will it be acceptible to use anything that will.

No mutex is necessary, anyway.
hierarchy-walker function completely works well under rcu read lock, if small jitter is allowed.

Right, should be doable -- and looking at the code, we have this horrible 32 bit exception in there that locks the rq in order to read the 64bit value.

Quoted 8 lines
Quoted 4 lines

Would be grand to get rid of that,. how bad would it be for userspace to get the occasionally fubarred value?

From view of user-support saler, if terrible broken value is reported,

it will be user-incident and annoy me(us) ;)

I'd like to get rid of rq->lock, too..Hmm.. some routine like atomic64_read() can help this ? (But I don't want to use atomic_t here..)

atomic64_read() will not help you on a 32-bit machine. Here is the sequence of events that will cause the aforementioned user incidents and consequent annoyance:

o The value of the counter is (2^32)-1, or 0xffffffff.

o CPU 0 reads the high-order 32 bits of the counter, getting zero.

o CPU 1 increments the low-order 32 bits of the counter, resulting
in zero, but notes that there is a carry out of this field.

o CPU 0 reads the low-order 32 bits of the counter, getting zero.

o CPU 1 increments the high-order 32 bits of the counter, so that

	the new value of the counter is 2^32, or 0x100000000.

So CPU 0 gets a value that is -way- off.

The usual trick is something like the following for counter read:

1. Read the high-order 32 bits of the counter.

2. Do a memory barrier, smp_mb().

3. Read the low-order 32 bits of the counter.

4. Do another memory barrier, again smp_mb().

5. Read the high-order 32 bits of the counter again.

If it is the same as the value obtained in step 1 (or the previous execution of step 5), then we are done. (This works even in case of complete 64-bit overflow, though we should be very lucky to live that long!) Otherwise, go to step 2.

But it is also necessary to modify the counter update:

1. Increment the low-order 32 bits of the counter. If no overflow
occurred, we are done, otherwise, continue through this sequence of steps.

2. Do a memory barrier, smp_mb().

3. Increment the high-order 32 bits of the counter.

How to detect overflow in step 1? Well, if we are incrementing, we can just test for the new value being zero. Otherwise, if we are adding a 32-bit number, if the new value of the low-order 32 bits of counter is less than the old value, overflow occurred (but make sure that the comparison is unsigned!).

This all assumes that you are adding a 32-bit quantity to the counter. Adding 64-bit values is not much harder.

Quoted 6 lines
Quoted 2 lines

Does this approach work for you?

Thank you. I'll try some and post if it seems easy to read/merge. Hmm, but in your approach, can't we see the counter goes backword ? (if the reader see only low 32 bit is incremtend.)

Ouch, indeed! The update would need to be atomic for my approach to work. My apologies for my confusion!

Quoted 2 lines

Can't we use seq_counter in include/linux/seqlock.h ? There is only one writer and we don't need write-side lock.

Yes, seqlock should work fine, good point!

Thanx, Paul

#21 lore raw

RFC PATCH 1/1 remove rq->lock from cpuacct cgroup (Was Re: [PATCH] cpuacct: add a branch prediction

Quoted 5 lines · On Thu, 26 Feb 2009 17:29:15 -0800 "Paul E. McKenney"…
Quoted 2 lines

Can't we use seq_counter in include/linux/seqlock.h ? There is only one writer and we don't need write-side lock.

Yes, seqlock should work fine, good point!

This is a trial version. (may hunk with the latest mmotm) seems overkill ?

==
From: KAMEZAWA Hiroyuki <[email protected]>

cgroup/cpuacct subsystem counts cpu usage by 64bit coutnter in per-cpu object. In read-side (via cpuacct.usage file), for reading 64bit value in safe manner, it takes rq->lock of (other) cpus.

In general, taking rq->lock of other cpus from codes not for scheduler is not good. This patch tries to remove rq->lock used in read-side.

To read 64bit value in safe, this patch uses seqcounter.

Pros.
- rq->lock is not necessary.
Cons.
- When updating counter, sequence number must be updated. (I hope this per-cpu sequence number is on cache...)
- not simple.

---
 kernel/sched.c |  141 ++++++++++++++++++++++++++++++++++++++++++---------------
 1 file changed, 105 insertions(+), 36 deletions(-)
kernel/sched.c +105 -36
Index: mmotm-2.6.29-Feb24/kernel/sched.c
===================================================================
@@ -9581,6 +9581,67 @@ struct cgroup_subsys cpu_cgroup_subsys =
 
 #ifdef CONFIG_CGROUP_CPUACCT
 
+#ifndef CONFIG_64BIT
+DEFINE_PER_CPU(struct seqcount, cpuacct_cgroup_seq);
+
+static inline void cpuacct_start_counter_update(void)
+{
+	/* This is called under rq->lock and IRQ is off */
+	struct seqcount *s = &get_cpu_var(cpuacct_cgroup_seq);
+
+	write_seqcount_begin(s);
+	put_cpu_var(cpuacct_cgroup_seq);
+}
+
+static inline void cpuacct_end_counter_update(void)
+{
+	struct seqcount *s = &get_cpu_var(cpuacct_cgroup_seq);
+
+	write_seqcount_end(s);
+	put_cpu_var(cpuacct_cgroup_seq);
+}
+
+static inline u64
+cpuacct_read_counter(u64 *val, int cpu)
+{
+	struct seqcount *s = &per_cpu(cpuacct_cgroup_seq, cpu);
+	unsigned int seq;
+	u64 data;
+
+	do {
+		seq = read_seqcount_begin(s);
+		data = *val;
+	} while (read_seqcount_retry(s, seq));
+	return data;
+}
+/* This is a special funtion called against "offline" cpus. */
+static inline void cpuacct_reset_offline_counter(u64 *val, int cpu)
+{
+	struct seqcount *s = &per_cpu(cpuacct_cgroup_seq, cpu);
+
+	preempt_disable();
+	write_seqcount_begin(s);
+	*val = 0;
+	write_seqcount_end(s);
+	preempt_enable();
+}
+#else
+static inline void cpuacct_start_counter_update(void)
+{
+}
+static inline void cpuacct_end_counter_update(void)
+{
+}
+static inline u64 cpuacct_read_counter(u64 *val, int cpu)
+{
+	return *val;
+}
+static inline void cpuacct_reset_offline_counter(u64 *val, int cpu)
+{
+	*val = 0;
+}
+#endif
+
 /*
  * CPU accounting code for task groups.
  *
@@ -9596,6 +9657,11 @@ struct cpuacct {
 	struct cpuacct *parent;
 };
 
+struct cpuacct_work {
+	struct work_struct work;
+	struct cpuacct *cpuacct;
+};
+
 struct cgroup_subsys cpuacct_subsys;
 
 /* return cpu accounting group corresponding to this container */
@@ -9643,39 +9709,29 @@ cpuacct_destroy(struct cgroup_subsys *ss
 	kfree(ca);
 }
 
+/* In 32bit enviroment, seqcounter is used for reading 64bit in safe way */
 static u64 cpuacct_cpuusage_read(struct cpuacct *ca, int cpu)
 {
 	u64 *cpuusage = percpu_ptr(ca->cpuusage, cpu);
 	u64 data;
 
-#ifndef CONFIG_64BIT
-	/*
-	 * Take rq->lock to make 64-bit read safe on 32-bit platforms.
-	 */
-	spin_lock_irq(&cpu_rq(cpu)->lock);
-	data = *cpuusage;
-	spin_unlock_irq(&cpu_rq(cpu)->lock);
-#else
-	data = *cpuusage;
-#endif
+	data = cpuacct_read_counter(cpuusage, cpu);
 
 	return data;
 }
 
-static void cpuacct_cpuusage_write(struct cpuacct *ca, int cpu, u64 val)
+/* called by per-cpu workqueue */
+static void cpuacct_cpuusage_reset_cpu(struct work_struct *work)
 {
+	struct cpuacct_work *cw = container_of(work, struct cpuacct_work, work);
+	struct cpuacct *ca = cw->cpuacct;
+	int cpu = get_cpu();
 	u64 *cpuusage = percpu_ptr(ca->cpuusage, cpu);
 
-#ifndef CONFIG_64BIT
-	/*
-	 * Take rq->lock to make 64-bit write safe on 32-bit platforms.
-	 */
-	spin_lock_irq(&cpu_rq(cpu)->lock);
-	*cpuusage = val;
-	spin_unlock_irq(&cpu_rq(cpu)->lock);
-#else
-	*cpuusage = val;
-#endif
+	cpuacct_start_counter_update();
+	*cpuusage = 0;
+	cpuacct_end_counter_update();
+	put_cpu();
 }
 
 /* return total cpu usage (in nanoseconds) of a group */
@@ -9691,23 +9747,34 @@ static u64 cpuusage_read(struct cgroup *
 	return totalcpuusage;
 }
 
-static int cpuusage_write(struct cgroup *cgrp, struct cftype *cftype,
-								u64 reset)
+static int cpuacct_cpuusage_reset(struct cgroup *cgrp, unsigned int event)
 {
 	struct cpuacct *ca = cgroup_ca(cgrp);
-	int err = 0;
-	int i;
-
-	if (reset) {
-		err = -EINVAL;
-		goto out;
+	int cpu;
+	/*
+	 * Reset All counters....doesn't need to be fast.
+	 * "ca" will be stable while doing this. We are in write() syscall.
+	 */
+	get_online_cpus();
+	/*
+	 * Because we use alloc_percpu() for allocating counter, we have
+	 * a counter per a possible cpu. Reset all online's by workqueue and
+	 * reset offline cpu's directly.
+	 */
+	for_each_possible_cpu(cpu) {
+		if (cpu_online(cpu)) {
+			struct cpuacct_work cw;
+			INIT_WORK(&cw.work, cpuacct_cpuusage_reset_cpu);
+			cw.cpuacct = ca;
+			schedule_work_on(cpu, &cw.work);
+			flush_work(&cw.work);
+		} else {
+			u64 *cpuusage = percpu_ptr(ca->cpuusage, cpu);
+			cpuacct_reset_offline_counter(cpuusage, cpu);
+		}
 	}
-
-	for_each_present_cpu(i)
-		cpuacct_cpuusage_write(ca, i, 0);
-
-out:
-	return err;
+	put_online_cpus();
+	return 0;
 }
 
 static int cpuacct_percpu_seq_read(struct cgroup *cgroup, struct cftype *cft,
@@ -9729,7 +9796,7 @@ static struct cftype files[] = {
 	{
 		.name = "usage",
 		.read_u64 = cpuusage_read,
-		.write_u64 = cpuusage_write,
+		.trigger = cpuacct_cpuusage_reset,
 	},
 	{
 		.name = "usage_percpu",
@@ -9756,10 +9823,12 @@ static void cpuacct_charge(struct task_s
 	cpu = task_cpu(tsk);
 	ca = task_ca(tsk);
 
+	cpuacct_start_counter_update();
 	for (; ca; ca = ca->parent) {
 		u64 *cpuusage = percpu_ptr(ca->cpuusage, cpu);
 		*cpuusage += cputime;
 	}
+	cpuacct_end_counter_update();
 }
 
 struct cgroup_subsys cpuacct_subsys = {
#17 lore raw
In reply to Ingo Molnar
Quoted 6 lines · Ingo Molnar
Quoted 4 lines · Peter Zijlstra
Quoted 2 lines · KAMEZAWA Hiroyuki

Hmm.. some routine like
atomic64_read() can help this ? (But I don't want to use atomic_t here..)

Quoted 7 lines
Quoted 4 lines

Yeah, atomic64_t has been proposed numerous times, and x86 could actually implement that using cmpxchg8b, just not sure about all the other 32bit archs, and if we start using it in the scheduler, they'd better have it implemented.

I have written a working atomic64_t implementation for tip:perfcounters/core, for 32-bit x86.

atomic64_t would be a big problem for 32-bit powerpc. We'd have to use an array of spinlocks, or make atomic64_t actually be 12 bytes so we have a lock word to use.

Paul.

#22 lore raw

Re: [RFC][PATCH] remove rq->lock from cpuacct cgroup (Was Re: [PATCH] cpuacct: add a branch prediction

In reply to KAMEZAWA Hiroyuki

On Fri, 2009-02-27 at 12:22 +0900, KAMEZAWA Hiroyuki wrote:

Comments below..

Quoted 40 lines

From: KAMEZAWA Hiroyuki <[email protected]>

cgroup/cpuacct subsystem counts cpu usage by 64bit coutnter in per-cpu object. In read-side (via cpuacct.usage file), for reading 64bit value in safe manner, it takes rq->lock of (other) cpus.

In general, taking rq->lock of other cpus from codes not for scheduler is not good. This patch tries to remove rq->lock used in read-side.

To read 64bit value in safe, this patch uses seqcounter.

Pros.
- rq->lock is not necessary.
Cons.
- When updating counter, sequence number must be updated. (I hope this per-cpu sequence number is on cache...)
- not simple.

---
 kernel/sched.c |  141 ++++++++++++++++++++++++++++++++++++++++++---------------
 1 file changed, 105 insertions(+), 36 deletions(-)

Index: mmotm-2.6.29-Feb24/kernel/sched.c
===================================================================

kernel/sched.c +9 -0
@@ -9581,6 +9581,67 @@ struct cgroup_subsys cpu_cgroup_subsys =
 
 #ifdef CONFIG_CGROUP_CPUACCT
 
+#ifndef CONFIG_64BIT
+DEFINE_PER_CPU(struct seqcount, cpuacct_cgroup_seq);
+
+static inline void cpuacct_start_counter_update(void)
+{
+	/* This is called under rq->lock and IRQ is off */
+	struct seqcount *s = &get_cpu_var(cpuacct_cgroup_seq);
+
+	write_seqcount_begin(s);

kernel/sched.c struct cgroup_subsys cpu_cgroup_subsys = · line 9602

Quoted 10 lines
kernel/sched.c +10 -0
+	put_cpu_var(cpuacct_cgroup_seq);
+}
+
+static inline void cpuacct_end_counter_update(void)
+{
+	struct seqcount *s = &get_cpu_var(cpuacct_cgroup_seq);
+
+	write_seqcount_end(s);
+	put_cpu_var(cpuacct_cgroup_seq);
+}

It seems odd we disable/enable preemption in both, I would expect for start to disable preemption, and have end enable it again, or use __get_cpu_var() and assume preemption is already disabled (callsites are under rq->lock, right?)

Quoted 14 lines
kernel/sched.c +14 -0
+static inline u64
+cpuacct_read_counter(u64 *val, int cpu)
+{
+	struct seqcount *s = &per_cpu(cpuacct_cgroup_seq, cpu);
+	unsigned int seq;
+	u64 data;
+
+	do {
+		seq = read_seqcount_begin(s);
+		data = *val;
+	} while (read_seqcount_retry(s, seq));
+	return data;
+}
+/* This is a special funtion called against "offline" cpus. */

kernel/sched.c struct cgroup_subsys cpu_cgroup_subsys = · line 9627

Quoted 10 lines
kernel/sched.c +10 -0
+static inline void cpuacct_reset_offline_counter(u64 *val, int cpu)
+{
+	struct seqcount *s = &per_cpu(cpuacct_cgroup_seq, cpu);
+
+	preempt_disable();
+	write_seqcount_begin(s);
+	*val = 0;
+	write_seqcount_end(s);
+	preempt_enable();
+}

And here you double disable preemption, quite useless if you take a remote cpu's per-cpu data.

Quoted 110 lines
kernel/sched.c +46 -29
+#else
+static inline void cpuacct_start_counter_update(void)
+{
+}
+static inline void cpuacct_end_counter_update(void)
+{
+}
+static inline u64 cpuacct_read_counter(u64 *val, int cpu)
+{
+	return *val;
+}
+static inline void cpuacct_reset_offline_counter(u64 *val, int cpu)
+{
+	*val = 0;
+}
+#endif
+
 /*
  * CPU accounting code for task groups.
  *
@@ -9596,6 +9657,11 @@ struct cpuacct {
 	struct cpuacct *parent;
 };
 
+struct cpuacct_work {
+	struct work_struct work;
+	struct cpuacct *cpuacct;
+};
+
 struct cgroup_subsys cpuacct_subsys;
 
 /* return cpu accounting group corresponding to this container */
@@ -9643,39 +9709,29 @@ cpuacct_destroy(struct cgroup_subsys *ss
 	kfree(ca);
 }
 
+/* In 32bit enviroment, seqcounter is used for reading 64bit in safe way */
 static u64 cpuacct_cpuusage_read(struct cpuacct *ca, int cpu)
 {
 	u64 *cpuusage = percpu_ptr(ca->cpuusage, cpu);
 	u64 data;
 
-#ifndef CONFIG_64BIT
-	/*
-	 * Take rq->lock to make 64-bit read safe on 32-bit platforms.
-	 */
-	spin_lock_irq(&cpu_rq(cpu)->lock);
-	data = *cpuusage;
-	spin_unlock_irq(&cpu_rq(cpu)->lock);
-#else
-	data = *cpuusage;
-#endif
+	data = cpuacct_read_counter(cpuusage, cpu);
 
 	return data;
 }
 
-static void cpuacct_cpuusage_write(struct cpuacct *ca, int cpu, u64 val)
+/* called by per-cpu workqueue */
+static void cpuacct_cpuusage_reset_cpu(struct work_struct *work)
 {
+	struct cpuacct_work *cw = container_of(work, struct cpuacct_work, work);
+	struct cpuacct *ca = cw->cpuacct;
+	int cpu = get_cpu();
 	u64 *cpuusage = percpu_ptr(ca->cpuusage, cpu);
 
-#ifndef CONFIG_64BIT
-	/*
-	 * Take rq->lock to make 64-bit write safe on 32-bit platforms.
-	 */
-	spin_lock_irq(&cpu_rq(cpu)->lock);
-	*cpuusage = val;
-	spin_unlock_irq(&cpu_rq(cpu)->lock);
-#else
-	*cpuusage = val;
-#endif
+	cpuacct_start_counter_update();
+	*cpuusage = 0;
+	cpuacct_end_counter_update();
+	put_cpu();
 }
 
 /* return total cpu usage (in nanoseconds) of a group */
@@ -9691,23 +9747,34 @@ static u64 cpuusage_read(struct cgroup *
 	return totalcpuusage;
 }
 
-static int cpuusage_write(struct cgroup *cgrp, struct cftype *cftype,
-								u64 reset)
+static int cpuacct_cpuusage_reset(struct cgroup *cgrp, unsigned int event)
 {
 	struct cpuacct *ca = cgroup_ca(cgrp);
-	int err = 0;
-	int i;
-
-	if (reset) {
-		err = -EINVAL;
-		goto out;
+	int cpu;
+	/*
+	 * Reset All counters....doesn't need to be fast.
+	 * "ca" will be stable while doing this. We are in write() syscall.
+	 */
+	get_online_cpus();
+	/*
+	 * Because we use alloc_percpu() for allocating counter, we have
+	 * a counter per a possible cpu. Reset all online's by workqueue and
+	 * reset offline cpu's directly.
+	 */
+	for_each_possible_cpu(cpu) {

kernel/sched.c static u64 cpuusage_read(struct cgroup * · line 9774

Quoted 10 lines
kernel/sched.c +10 -0
+		if (cpu_online(cpu)) {
+			struct cpuacct_work cw;
+			INIT_WORK(&cw.work, cpuacct_cpuusage_reset_cpu);
+			cw.cpuacct = ca;
+			schedule_work_on(cpu, &cw.work);
+			flush_work(&cw.work);
+		} else {
+			u64 *cpuusage = percpu_ptr(ca->cpuusage, cpu);
+			cpuacct_reset_offline_counter(cpuusage, cpu);
+		}

I'm not particularly convinced this is the right way, schedule_work_on() sounds way expensive for setting a variable to 0.

Furthermore, if you want something like schedule_work_on() for each cpu, there's schedule_on_each_cpu().

Quoted 35 lines
kernel/sched.c +5 -7
 	}
-
-	for_each_present_cpu(i)
-		cpuacct_cpuusage_write(ca, i, 0);
-
-out:
-	return err;
+	put_online_cpus();
+	return 0;
 }
 
 static int cpuacct_percpu_seq_read(struct cgroup *cgroup, struct cftype *cft,
@@ -9729,7 +9796,7 @@ static struct cftype files[] = {
 	{
 		.name = "usage",
 		.read_u64 = cpuusage_read,
-		.write_u64 = cpuusage_write,
+		.trigger = cpuacct_cpuusage_reset,
 	},
 	{
 		.name = "usage_percpu",
@@ -9756,10 +9823,12 @@ static void cpuacct_charge(struct task_s
 	cpu = task_cpu(tsk);
 	ca = task_ca(tsk);
 
+	cpuacct_start_counter_update();
 	for (; ca; ca = ca->parent) {
 		u64 *cpuusage = percpu_ptr(ca->cpuusage, cpu);
 		*cpuusage += cputime;
 	}
+	cpuacct_end_counter_update();
 }
 
 struct cgroup_subsys cpuacct_subsys = {
#23 lore raw

Re: [RFC][PATCH] remove rq->lock from cpuacct cgroup (Was Re: [PATCH] cpuacct: add a branch prediction

Quoted 51 lines · On Mon, 02 Mar 2009 15:56:10 +0100 Peter Zijlstra <[email protected]>…

On Fri, 2009-02-27 at 12:22 +0900, KAMEZAWA Hiroyuki wrote:

Comments below..

Quoted 47 lines

From: KAMEZAWA Hiroyuki <[email protected]>

cgroup/cpuacct subsystem counts cpu usage by 64bit coutnter in per-cpu object. In read-side (via cpuacct.usage file), for reading 64bit value in safe manner, it takes rq->lock of (other) cpus.

In general, taking rq->lock of other cpus from codes not for scheduler is not good. This patch tries to remove rq->lock used in read-side.

To read 64bit value in safe, this patch uses seqcounter.

Pros.
- rq->lock is not necessary.
Cons.
- When updating counter, sequence number must be updated. (I hope this per-cpu sequence number is on cache...)
- not simple.

---
 kernel/sched.c |  141 ++++++++++++++++++++++++++++++++++++++++++---------------
 1 file changed, 105 insertions(+), 36 deletions(-)

Index: mmotm-2.6.29-Feb24/kernel/sched.c
===================================================================

kernel/sched.c +16 -0
@@ -9581,6 +9581,67 @@ struct cgroup_subsys cpu_cgroup_subsys =
 
 #ifdef CONFIG_CGROUP_CPUACCT
 
+#ifndef CONFIG_64BIT
+DEFINE_PER_CPU(struct seqcount, cpuacct_cgroup_seq);
+
+static inline void cpuacct_start_counter_update(void)
+{
+	/* This is called under rq->lock and IRQ is off */
+	struct seqcount *s = &get_cpu_var(cpuacct_cgroup_seq);
+
+	write_seqcount_begin(s);
+	put_cpu_var(cpuacct_cgroup_seq);
+}
+
+static inline void cpuacct_end_counter_update(void)
+{
+	struct seqcount *s = &get_cpu_var(cpuacct_cgroup_seq);
+
Quoted 9 lines

kernel/sched.c struct cgroup_subsys cpu_cgroup_subsys = · line 9602

Quoted 3 lines
kernel/sched.c +3 -0
+	write_seqcount_end(s);
+	put_cpu_var(cpuacct_cgroup_seq);
+}

It seems odd we disable/enable preemption in both, I would expect for start to disable preemption, and have end enable it again, or use __get_cpu_var() and assume preemption is already disabled (callsites are under rq->lock, right?)

yes. calles are under rq->lock...ok, you're right. I'll remove preepmtp disabling codes.

Quoted 19 lines
Quoted 19 lines
kernel/sched.c +19 -0
+static inline u64
+cpuacct_read_counter(u64 *val, int cpu)
+{
+	struct seqcount *s = &per_cpu(cpuacct_cgroup_seq, cpu);
+	unsigned int seq;
+	u64 data;
+
+	do {
+		seq = read_seqcount_begin(s);
+		data = *val;
+	} while (read_seqcount_retry(s, seq));
+	return data;
+}
+/* This is a special funtion called against "offline" cpus. */
+static inline void cpuacct_reset_offline_counter(u64 *val, int cpu)
+{
+	struct seqcount *s = &per_cpu(cpuacct_cgroup_seq, cpu);
+
+	preempt_disable();
Quoted 9 lines

kernel/sched.c struct cgroup_subsys cpu_cgroup_subsys = · line 9627

Quoted 5 lines
kernel/sched.c +5 -0
+	write_seqcount_begin(s);
+	*val = 0;
+	write_seqcount_end(s);
+	preempt_enable();
+}

And here you double disable preemption, quite useless if you take a remote cpu's per-cpu data.

Ok, maybe this "reset" from a user should be under rq->lock. (But reading will not be under rq->lock.)

Quoted 115 lines
Quoted 115 lines
kernel/sched.c +51 -29
+#else
+static inline void cpuacct_start_counter_update(void)
+{
+}
+static inline void cpuacct_end_counter_update(void)
+{
+}
+static inline u64 cpuacct_read_counter(u64 *val, int cpu)
+{
+	return *val;
+}
+static inline void cpuacct_reset_offline_counter(u64 *val, int cpu)
+{
+	*val = 0;
+}
+#endif
+
 /*
  * CPU accounting code for task groups.
  *
@@ -9596,6 +9657,11 @@ struct cpuacct {
 	struct cpuacct *parent;
 };
 
+struct cpuacct_work {
+	struct work_struct work;
+	struct cpuacct *cpuacct;
+};
+
 struct cgroup_subsys cpuacct_subsys;
 
 /* return cpu accounting group corresponding to this container */
@@ -9643,39 +9709,29 @@ cpuacct_destroy(struct cgroup_subsys *ss
 	kfree(ca);
 }
 
+/* In 32bit enviroment, seqcounter is used for reading 64bit in safe way */
 static u64 cpuacct_cpuusage_read(struct cpuacct *ca, int cpu)
 {
 	u64 *cpuusage = percpu_ptr(ca->cpuusage, cpu);
 	u64 data;
 
-#ifndef CONFIG_64BIT
-	/*
-	 * Take rq->lock to make 64-bit read safe on 32-bit platforms.
-	 */
-	spin_lock_irq(&cpu_rq(cpu)->lock);
-	data = *cpuusage;
-	spin_unlock_irq(&cpu_rq(cpu)->lock);
-#else
-	data = *cpuusage;
-#endif
+	data = cpuacct_read_counter(cpuusage, cpu);
 
 	return data;
 }
 
-static void cpuacct_cpuusage_write(struct cpuacct *ca, int cpu, u64 val)
+/* called by per-cpu workqueue */
+static void cpuacct_cpuusage_reset_cpu(struct work_struct *work)
 {
+	struct cpuacct_work *cw = container_of(work, struct cpuacct_work, work);
+	struct cpuacct *ca = cw->cpuacct;
+	int cpu = get_cpu();
 	u64 *cpuusage = percpu_ptr(ca->cpuusage, cpu);
 
-#ifndef CONFIG_64BIT
-	/*
-	 * Take rq->lock to make 64-bit write safe on 32-bit platforms.
-	 */
-	spin_lock_irq(&cpu_rq(cpu)->lock);
-	*cpuusage = val;
-	spin_unlock_irq(&cpu_rq(cpu)->lock);
-#else
-	*cpuusage = val;
-#endif
+	cpuacct_start_counter_update();
+	*cpuusage = 0;
+	cpuacct_end_counter_update();
+	put_cpu();
 }
 
 /* return total cpu usage (in nanoseconds) of a group */
@@ -9691,23 +9747,34 @@ static u64 cpuusage_read(struct cgroup *
 	return totalcpuusage;
 }
 
-static int cpuusage_write(struct cgroup *cgrp, struct cftype *cftype,
-								u64 reset)
+static int cpuacct_cpuusage_reset(struct cgroup *cgrp, unsigned int event)
 {
 	struct cpuacct *ca = cgroup_ca(cgrp);
-	int err = 0;
-	int i;
-
-	if (reset) {
-		err = -EINVAL;
-		goto out;
+	int cpu;
+	/*
+	 * Reset All counters....doesn't need to be fast.
+	 * "ca" will be stable while doing this. We are in write() syscall.
+	 */
+	get_online_cpus();
+	/*
+	 * Because we use alloc_percpu() for allocating counter, we have
+	 * a counter per a possible cpu. Reset all online's by workqueue and
+	 * reset offline cpu's directly.
+	 */
+	for_each_possible_cpu(cpu) {
+		if (cpu_online(cpu)) {
+			struct cpuacct_work cw;
+			INIT_WORK(&cw.work, cpuacct_cpuusage_reset_cpu);
+			cw.cpuacct = ca;
+			schedule_work_on(cpu, &cw.work);
Quoted 9 lines

kernel/sched.c static u64 cpuusage_read(struct cgroup * · line 9774

Quoted 5 lines
kernel/sched.c +5 -0
+			flush_work(&cw.work);
+		} else {
+			u64 *cpuusage = percpu_ptr(ca->cpuusage, cpu);
+			cpuacct_reset_offline_counter(cpuusage, cpu);
+		}

I'm not particularly convinced this is the right way, schedule_work_on() sounds way expensive for setting a variable to 0.

yes, yes..

Quoted 3 lines

Furthermore, if you want something like schedule_work_on() for each cpu, there's schedule_on_each_cpu().

It can't pass arguments...Maybe I should use rq->lock here to reset other cpu's value.

Thank you for review.
-Kame

#24 lore raw

Re: [RFC][PATCH] remove rq->lock from cpuacct cgroup (Was Re: [PATCH] cpuacct: add a branch prediction

Quoted 5 lines · KAMEZAWA Hiroyuki
Quoted 3 lines

Furthermore, if you want something like schedule_work_on() for each cpu, there's schedule_on_each_cpu().

It can't pass arguments...Maybe I should use rq->lock here to reset other cpu's value.

Why bother with serializing the reset code at all?

But there's always smp_call_function() I suppose.

#25 lore raw

Re: [RFC][PATCH] remove rq->lock from cpuacct cgroup (Was Re: [PATCH] cpuacct: add a branch prediction

Quoted 11 lines · Peter Zijlstra
Quoted 6 lines · KAMEZAWA Hiroyuki
Quoted 1 line

Furthermore, if you want something like schedule_work_on() for each

cpu,

Quoted 2 lines

there's schedule_on_each_cpu().

It can't pass arguments...Maybe I should use rq->lock here to reset other cpu's value.

Why bother with serializing the reset code at all?

I don't think reset v.s. read is problem but reset v.s. increment (read-modify-write) can't be ?

Quoted 2 lines

But there's always smp_call_function() I suppose.

Hmm, I'll find some simpler way.

Thanks,
-Kame

#26 lore raw

Re: [RFC][PATCH] remove rq->lock from cpuacct cgroup (Was Re: [PATCH] cpuacct: add a branch prediction

Quoted 7 lines · KAMEZAWA Hiroyuki
Quoted 6 lines · Peter Zijlstra
Quoted 4 lines · KAMEZAWA Hiroyuki
Quoted 1 line

Furthermore, if you want something like schedule_work_on() for each

cpu,

Quoted 2 lines

there's schedule_on_each_cpu().

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

It can't pass arguments...Maybe I should use rq->lock here to reset other cpu's value.

Why bother with serializing the reset code at all?

I don't think reset v.s. read is problem but reset v.s. increment (read-modify-write) can't be ?

Sure, can be, do we care?

#27 lore raw

Re: [RFC][PATCH] remove rq->lock from cpuacct cgroup (Was Re: [PATCH] cpuacct: add a branch prediction

Quoted 11 lines · Peter Zijlstra
Quoted 10 lines · KAMEZAWA Hiroyuki
Quoted 9 lines · Peter Zijlstra
Quoted 6 lines · KAMEZAWA Hiroyuki
Quoted 1 line

Furthermore, if you want something like schedule_work_on() for each

cpu,

Quoted 2 lines

there's schedule_on_each_cpu().

It can't pass arguments...Maybe I should use rq->lock here to reset other cpu's value.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

Why bother with serializing the reset code at all?

I don't think reset v.s. read is problem but reset v.s. increment (read-modify-write) can't be ?

Sure, can be, do we care?

If small/easy code allows us to declare "there are any racy case! and you don't have to check whether you successfully reseted", it's worth to do I think.

Thanks,
-Kame

#28 lore raw

Re: [RFC][PATCH] remove rq->lock from cpuacct cgroup (Was Re: [PATCH] cpuacct: add a branch prediction

Quoted 13 lines · KAMEZAWA Hiroyuki
Quoted 12 lines · Peter Zijlstra
Quoted 11 lines · KAMEZAWA Hiroyuki
Quoted 3 lines · Peter Zijlstra
Quoted 1 line · KAMEZAWA Hiroyuki
Quoted 1 line

Furthermore, if you want something like schedule_work_on() for

each

Quoted 6 lines
Quoted 5 lines

cpu,

Quoted 2 lines

there's schedule_on_each_cpu().

It can't pass arguments...Maybe I should use rq->lock here to reset other cpu's value.

Quoted 8 lines
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

Why bother with serializing the reset code at all?

I don't think reset v.s. read is problem but reset v.s. increment (read-modify-write) can't be ?

Sure, can be, do we care?

If small/easy code allows us to declare "there are any racy case!

                                                     isn't...

Sorry.
-Kame

#29 lore raw

Re: [RFC][PATCH] remove rq->lock from cpuacct cgroup (Was Re: [PATCH] cpuacct: add a branch prediction

In reply to KAMEZAWA Hiroyuki
Quoted 14 lines · KAMEZAWA Hiroyuki
Quoted 13 lines · Peter Zijlstra
Quoted 12 lines · KAMEZAWA Hiroyuki
Quoted 11 lines · Peter Zijlstra
Quoted 6 lines · KAMEZAWA Hiroyuki
Quoted 1 line

Furthermore, if you want something like schedule_work_on() for each

cpu,

Quoted 2 lines

there's schedule_on_each_cpu().

It can't pass arguments...Maybe I should use rq->lock here to reset other cpu's value.

Why bother with serializing the reset code at all?

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

I don't think reset v.s. read is problem but reset v.s. increment (read-modify-write) can't be ?

Sure, can be, do we care?

If small/easy code allows us to declare "there are any racy case! and you don't have to check whether you successfully reseted", it's worth to do I think.

smp_call_function() it is...

#30 lore raw

PATCH 1/1 remove rq->lock from cpuacct cgroup v2

From: KAMEZAWA Hiroyuki <[email protected]>

cgroup/cpuacct subsystem counts cpu usage by 64bit coutnter in per-cpu object. In read-side (via cpuacct.usage file), for reading 64bit value in safe manner, it takes rq->lock of (other) cpus.

In general, taking rq->lock of other cpus from codes not for scheduler is not good. This patch tries to remove rq->lock in read-side.

To read 64bit value in atomic, this patch uses seqcounter.

Pros.
- rq->lock is not necessary.
Cons.
- When updating counter, sequence number must be updated. (I hope this per-cpu sequence number is on cache...)

Changelog: v1->v2
- checking calling context of all calls and avoid unnecessary preempt_disable calls.
- use on_each_cpu() instead of workqueue, at reset

---

kernel/sched.c +97 -35
Index: mmotm-2.6.29-Mar3/kernel/sched.c
===================================================================
@@ -9581,6 +9581,71 @@ struct cgroup_subsys cpu_cgroup_subsys =
 
 #ifdef CONFIG_CGROUP_CPUACCT
 
+#ifndef CONFIG_64BIT
+/* seq counter for handle 64bit counter on 32bit system */
+DEFINE_PER_CPU(struct seqcount, cpuacct_cgroup_seq);
+
+/*
+ * Counter update happens while rq->lock is held and we don't need to
+ * disable preempt explcitly.
+ */
+static inline void cpuacct_start_counter_update(void)
+{
+	/* This is called under rq->lock and IRQ is off */
+	struct seqcount *s = &__get_cpu_var(cpuacct_cgroup_seq);
+
+	write_seqcount_begin(s);
+}
+
+static inline void cpuacct_end_counter_update(void)
+{
+	struct seqcount *s = &__get_cpu_var(cpuacct_cgroup_seq);
+
+	write_seqcount_end(s);
+}
+
+static inline u64
+cpuacct_read_counter(u64 *val, int cpu)
+{
+	struct seqcount *s = &per_cpu(cpuacct_cgroup_seq, cpu);
+	unsigned int seq;
+	u64 data;
+
+	do {
+		seq = read_seqcount_begin(s);
+		data = *val;
+	} while (read_seqcount_retry(s, seq));
+	return data;
+}
+/* This is a special funtion called against "offline" cpus. */
+static inline void cpuacct_reset_offline_counter(u64 *val, int cpu)
+{
+	struct seqcount *s = &per_cpu(cpuacct_cgroup_seq, cpu);
+
+	write_seqcount_begin(s);
+	*val = 0;
+	write_seqcount_end(s);
+}
+#else
+static inline void cpuacct_start_counter_update(void)
+{
+}
+
+static inline void cpuacct_end_counter_update(void)
+{
+}
+
+static inline u64 cpuacct_read_counter(u64 *val, int cpu)
+{
+	return *val;
+}
+
+static inline void cpuacct_reset_offline_counter(u64 *val, int cpu)
+{
+	*val = 0;
+}
+#endif
+
 /*
  * CPU accounting code for task groups.
  *
@@ -9643,39 +9708,27 @@ cpuacct_destroy(struct cgroup_subsys *ss
 	kfree(ca);
 }
 
+/* In 32bit enviroment, seqcounter is used for reading 64bit in safe way */
 static u64 cpuacct_cpuusage_read(struct cpuacct *ca, int cpu)
 {
 	u64 *cpuusage = percpu_ptr(ca->cpuusage, cpu);
 	u64 data;
 
-#ifndef CONFIG_64BIT
-	/*
-	 * Take rq->lock to make 64-bit read safe on 32-bit platforms.
-	 */
-	spin_lock_irq(&cpu_rq(cpu)->lock);
-	data = *cpuusage;
-	spin_unlock_irq(&cpu_rq(cpu)->lock);
-#else
-	data = *cpuusage;
-#endif
+	data = cpuacct_read_counter(cpuusage, cpu);
 
 	return data;
 }
 
-static void cpuacct_cpuusage_write(struct cpuacct *ca, int cpu, u64 val)
+/* called by per-cpu smp call function (in non-preemptable context) */
+static void cpuacct_cpuusage_reset_cpu(void *data)
 {
+	int cpu = smp_processor_id();
+	struct cpuacct *ca = data;
 	u64 *cpuusage = percpu_ptr(ca->cpuusage, cpu);
 
-#ifndef CONFIG_64BIT
-	/*
-	 * Take rq->lock to make 64-bit write safe on 32-bit platforms.
-	 */
-	spin_lock_irq(&cpu_rq(cpu)->lock);
-	*cpuusage = val;
-	spin_unlock_irq(&cpu_rq(cpu)->lock);
-#else
-	*cpuusage = val;
-#endif
+	cpuacct_start_counter_update();
+	*cpuusage = 0;
+	cpuacct_end_counter_update();
 }
 
 /* return total cpu usage (in nanoseconds) of a group */
@@ -9691,23 +9744,30 @@ static u64 cpuusage_read(struct cgroup *
 	return totalcpuusage;
 }
 
-static int cpuusage_write(struct cgroup *cgrp, struct cftype *cftype,
-								u64 reset)
+static int cpuacct_cpuusage_reset(struct cgroup *cgrp, unsigned int event)
 {
 	struct cpuacct *ca = cgroup_ca(cgrp);
-	int err = 0;
-	int i;
+	int cpu;
+	/*
+	 * We prevent cpu hotplug while we do reset.
+	 */
+	get_online_cpus();
+	/*
+	 * clear all online cpu's status (including local one)
+	 * This reseting uses nowait smp call and counter will be cleared in
+	 * asynchronous way.
+	 */
+	on_each_cpu(cpuacct_cpuusage_reset_cpu, ca, 0);
 
-	if (reset) {
-		err = -EINVAL;
-		goto out;
+	/* clear all present but offline cpus' */
+	for_each_possible_cpu(cpu) {
+		if (!cpu_online(cpu)) {
+			u64 *cpuusage = percpu_ptr(ca->cpuusage, cpu);
+			cpuacct_reset_offline_counter(cpuusage, cpu);
+		}
 	}
-
-	for_each_present_cpu(i)
-		cpuacct_cpuusage_write(ca, i, 0);
-
-out:
-	return err;
+	put_online_cpus();
+	return 0;
 }
 
 static int cpuacct_percpu_seq_read(struct cgroup *cgroup, struct cftype *cft,
@@ -9729,7 +9789,7 @@ static struct cftype files[] = {
 	{
 		.name = "usage",
 		.read_u64 = cpuusage_read,
-		.write_u64 = cpuusage_write,
+		.trigger = cpuacct_cpuusage_reset,
 	},
 	{
 		.name = "usage_percpu",
@@ -9759,10 +9819,12 @@ static void cpuacct_charge(struct task_s
 	cpu = task_cpu(tsk);
 	ca = task_ca(tsk);
 
+	cpuacct_start_counter_update();
 	for (; ca; ca = ca->parent) {
 		u64 *cpuusage = percpu_ptr(ca->cpuusage, cpu);
 		*cpuusage += cputime;
 	}
+	cpuacct_end_counter_update();
 }
 
 struct cgroup_subsys cpuacct_subsys = {
#31 lore raw

Re: [PATCH] remove rq->lock from cpuacct cgroup v2

Quoted 17 lines · KAMEZAWA Hiroyuki

From: KAMEZAWA Hiroyuki <[email protected]>

cgroup/cpuacct subsystem counts cpu usage by 64bit coutnter in per-cpu object. In read-side (via cpuacct.usage file), for reading 64bit value in safe manner, it takes rq->lock of (other) cpus.

In general, taking rq->lock of other cpus from codes not for scheduler is not good. This patch tries to remove rq->lock in read-side.

To read 64bit value in atomic, this patch uses seqcounter.

Pros.
 - rq->lock is not necessary.
Cons.
 - When updating counter, sequence number must be updated. (I hope this per-cpu sequence number is on cache...)

Quoted 6 lines

Changelog: v1->v2
 - checking calling context of all calls and avoid unnecessary preempt_disable calls.
 - use on_each_cpu() instead of workqueue, at reset

So cpuacct->cpuusage is a 64 bit percpu counter and I see that cpuacct subsystem itself handles the following special cases:
- 32 vs 64 bit update issues
- resetting percpu counters on online and offline cpus

Tomorrow if I add other counters to cpuacct subsystem (like stime and utime), I need to do what you have done in this patch all over again for the additional counters.

Instead of subsystems handling all these percpu counter problems themselves, shouldn't we be using percpu_counter subsytem and let it handle all the issues transparently for us ? I am not sure if all these problems have been addressed in percpu_counter, but would like to know why we are not using percpu_counter for these kinds of things and enhance percpu_counter if it can't handle some of the issues which we are solving here specifically for cpuacct subsystem ?

Regards,
Bharata.

Signature · 1 line
http://bharata.sulekha.com/blog/posts.htm
#32 lore raw

Re: [PATCH] remove rq->lock from cpuacct cgroup v2

Quoted 26 lines · On Wed, 4 Mar 2009 13:24:43 +0530 Bharata B Rao <[email protected]> wrote:
Quoted 17 lines · KAMEZAWA Hiroyuki

From: KAMEZAWA Hiroyuki <[email protected]>

cgroup/cpuacct subsystem counts cpu usage by 64bit coutnter in per-cpu object. In read-side (via cpuacct.usage file), for reading 64bit value in safe manner, it takes rq->lock of (other) cpus.

In general, taking rq->lock of other cpus from codes not for scheduler is not good. This patch tries to remove rq->lock in read-side.

To read 64bit value in atomic, this patch uses seqcounter.

Pros.
 - rq->lock is not necessary.
Cons.
 - When updating counter, sequence number must be updated. (I hope this per-cpu sequence number is on cache...)

Quoted 6 lines

Changelog: v1->v2
 - checking calling context of all calls and avoid unnecessary preempt_disable calls.
 - use on_each_cpu() instead of workqueue, at reset

Quoted 9 lines

So cpuacct->cpuusage is a 64 bit percpu counter and I see that cpuacct subsystem itself handles the following special cases:
- 32 vs 64 bit update issues
- resetting percpu counters on online and offline cpus

Tomorrow if I add other counters to cpuacct subsystem (like stime and utime), I need to do what you have done in this patch all over again for the additional counters.

I'm not sure Yes or No. Is it necessary to be per-cpu ? IIUC, stime/utime update is at most once-per-tick and not so frequent as cpuacct.

Quoted 8 lines

Instead of subsystems handling all these percpu counter problems themselves, shouldn't we be using percpu_counter subsytem and let it handle all the issues transparently for us ? I am not sure if all these problems have been addressed in percpu_counter, but would like to know why we are not using percpu_counter for these kinds of things and enhance percpu_counter if it can't handle some of the issues which we are solving here specifically for cpuacct subsystem ?

At first, generic per-cpu counter sounds interesting but to be honest, some special handling is used for cpuacct based on its characteristic.

- Writer works under non-preemptable context.
- There is only one writer.

Otherwize, using generic atomic64_t.. or res_counter will be good...maybe. (res_counter updates 64bit value under spinlock lock.)

Thanks,
-Kame

Quoted 5 lines

Regards,
Bharata.

Signature · 1 line
http://bharata.sulekha.com/blog/posts.htm
#33 lore raw

Re: [PATCH] remove rq->lock from cpuacct cgroup v2

Quoted 8 lines · On Wed, 4 Mar 2009 17:20:05 +0900 KAMEZAWA Hiroyuki…

On Wed, 4 Mar 2009 13:24:43 +0530
Bharata B Rao <[email protected]> wrote:
At first, generic per-cpu counter sounds interesting but to be honest, some special handling is used for cpuacct based on its characteristic.

- Writer works under non-preemptable context.
- There is only one writer.

If utime/stime updates works on above context, using the same code will be good.

I don't use any cpuacct structure specific in routines... If you want me to rewrite it, I'll do. please request what you want.

Thanks,
-Kame

#34 lore raw

Re: [PATCH] remove rq->lock from cpuacct cgroup v2

Quoted 8 lines · KAMEZAWA Hiroyuki
Quoted 5 lines · On Wed, 4 Mar 2009 17:20:05 +0900 KAMEZAWA Hiroyuki…

On Wed, 4 Mar 2009 13:24:43 +0530
Bharata B Rao <[email protected]> wrote:
At first, generic per-cpu counter sounds interesting but to be honest, some special handling is used for cpuacct based on its characteristic.

Quoted 4 lines
Quoted 3 lines

  - Writer works under non-preemptable context.
  - There is only one writer.

If utime/stime updates works on above context, using the same code will be good.

IIUC, utime/stime updates also work under the above conditions.

Quoted 3 lines

I don't use any cpuacct structure specific in routines... If you want me to rewrite it, I'll do. please request what you want.

After looking deep into your patch, I think I could use the same seq counter introduced by you to update stime/utime also.
I guess I could use most part of your code except there is a slight difference wrt preemption disabled assumption in the write path. cpusuage updates happen under rq->lock but stime/utime updates don't. So I probably can't use cpuacct_start/end_counter_update as is.

Regards,
Bharata.

Signature · 1 line
http://bharata.sulekha.com/blog/posts.htm
#35 lore raw

Re: [PATCH] remove rq->lock from cpuacct cgroup v2

In reply to KAMEZAWA Hiroyuki
Quoted 11 lines · KAMEZAWA Hiroyuki
Quoted 8 lines · On Wed, 4 Mar 2009 13:24:43 +0530 Bharata B Rao <[email protected]> wrote:

Instead of subsystems handling all these percpu counter problems themselves, shouldn't we be using percpu_counter subsytem and let it handle all the issues transparently for us ? I am not sure if all these problems have been addressed in percpu_counter, but would like to know why we are not using percpu_counter for these kinds of things and enhance percpu_counter if it can't handle some of the issues which we are solving here specifically for cpuacct subsystem ?

Quoted 3 lines

At first, generic per-cpu counter sounds interesting but to be honest, some special handling is used for cpuacct based on its characteristic.

Just trying to understand this clearly ...

Quoted 1 line

 - Writer works under non-preemptable context.

Which means the writer is running with preemption disabled. percpu_counter writes (via percpu_counter_add) don't assume anything and disable preemption themselves. Is this the problem or is there a bigger issue here why percpu_counter can't be used ?

Quoted 1 line

 - There is only one writer.

Not sure how you have optimized for this case in cpuacct. percpu_counters use spinlocks to serialize writers. Are you saying using spinlocks for this 1 writer case is too much ? Also note that they update 32 bit percpu counters without any lock and take spinlocks only when they do a batch update to the 64bit counter.

Regards,
Bharata.

Signature · 1 line
http://bharata.sulekha.com/blog/posts.htm
#36 lore raw

Re: [PATCH] remove rq->lock from cpuacct cgroup v2

Bharata B Rao さんは書きました:

Quoted 17 lines
Quoted 11 lines · KAMEZAWA Hiroyuki
Quoted 8 lines · On Wed, 4 Mar 2009 13:24:43 +0530 Bharata B Rao <[email protected]> wrote:

Instead of subsystems handling all these percpu counter problems themselves, shouldn't we be using percpu_counter subsytem and let it handle all the issues transparently for us ? I am not sure if all these problems have been addressed in percpu_counter, but would like to know why we are not using percpu_counter for these kinds of things and enhance percpu_counter if it can't handle some of the issues which we are solving here specifically for cpuacct subsystem ?

Quoted 3 lines

At first, generic per-cpu counter sounds interesting but to be honest, some special handling is used for cpuacct based on its characteristic.

Quoted 9 lines

Just trying to understand this clearly ...

Quoted 1 line

&#160;- Writer works under non-preemptable context.

Which means the writer is running with preemption disabled. percpu_counter writes (via percpu_counter_add) don't assume anything and disable preemption themselves. Is this the problem or is there a bigger issue here why percpu_counter can't be used ?

Ah, just means it doesn't call disable_preempt() explicitly and use __get_per_cpu() (not get_cpu_var) and smp_processor_id (not get_cpu).

Quoted 3 lines
Quoted 1 line

&#160;- There is only one writer.

Not sure how you have optimized for this case in cpuacct.

Because there is only one writer, I used seq_counter not seq_lock. (there are no atomic ops by this.)

Quoted 2 lines

percpu_counters use spinlocks to serialize writers. Are you saying using spinlocks for this 1 writer case is too much ?

I'm sorry I noticed there is lib/percpu_counter now....... I think lib/percpu_counter does proper jobs.

Quoted 3 lines

Also note that they update 32 bit percpu counters without any lock and take spinlocks only when they do a batch update to the 64bit counter.

Hmm, cpuacct's one uses hierachical update of several counters at once. So, some private code like mine is not so bad, I think.

What I think now is it's ok to see your patch first and later do this kind of update to avoid locks if necessary. This patch is micro optimization and not in hurry.

Thanks,
-Kame