Skip to contentSkip to conversation

do not make cputime scaling in kernel

RFC In review Stanislaw Gruszka 4 participants 11 messages Mar 28, 2013 – Apr 11, 2013

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover do not make cputime scaling in kernel

This patch series removes cputime scaling from kernel. It can be easily done in user space using floating point if we provide sum_exec_runtime, what patches 2/4 and 3/4 do. I have procps which utilize that:

http://people.redhat.com/sgruszka/procps-use-sum_exec_runtime.patch

I will post it, if this patch set will be queued.

Change affect also getrusage() and times() syscals, but I don't think kernel give guarantees about utime/stime precision, in a matter of fact before commit b27f03d4bdc145a09fb7b0c0e004b29f1ee555fa, we do not perform any scaling and we provided raw cputime values to user space.

Providing sum_exec_runtime via proc is done against malware that utilize lot of cpu time but hide itself from top program.

This affect kernels not compiled with CONFIG_VIRT_CPU_ACCOUNTING_{GEN,NATIVE}, if user choose to compile kernel with some of those options, he/she will have more precise cputime accounting, what is documented in Kconfig.

#1 lore raw

RFC 1/4 cputime: change parameter of thread_group_cputime_adjusted

This is preparation for further changes in cputime code.

---
 fs/proc/array.c        |    5 ++++-
 include/linux/sched.h  |    2 +-
 kernel/exit.c          |    8 ++++----
 kernel/sched/cputime.c |   18 ++++++------------
 kernel/sys.c           |   18 ++++++++++--------
 5 files changed, 25 insertions(+), 26 deletions(-)
fs/proc/array.c +4 -1
index f7ed9ee..fef7850 100644
@@ -446,6 +446,7 @@ static int do_task_stat(struct seq_file *m, struct pid_namespace *ns,
 		/* add up live thread stats at the group level */
 		if (whole) {
 			struct task_struct *t = task;
+			struct task_cputime cputime;
 			do {
 				min_flt += t->min_flt;
 				maj_flt += t->maj_flt;
@@ -455,7 +456,9 @@ static int do_task_stat(struct seq_file *m, struct pid_namespace *ns,
 
 			min_flt += sig->min_flt;
 			maj_flt += sig->maj_flt;
-			thread_group_cputime_adjusted(task, &utime, &stime);
+			thread_group_cputime_adjusted(task, &cputime);
+			utime = cputime.utime;
+			stime = cputime.stime;
 			gtime += sig->gtime;
 		}
 
include/linux/sched.h +1 -1
index 10626e2..eb6005c 100644
@@ -1595,7 +1595,7 @@ static inline cputime_t task_gtime(struct task_struct *t)
 }
 #endif
 extern void task_cputime_adjusted(struct task_struct *p, cputime_t *ut, cputime_t *st);
-extern void thread_group_cputime_adjusted(struct task_struct *p, cputime_t *ut, cputime_t *st);
+extern void thread_group_cputime_adjusted(struct task_struct *p, struct task_cputime *ct);
 
 /*
  * Per process flags
kernel/exit.c +4 -4
index 51e485c..5a7023f 100644
@@ -1067,7 +1067,7 @@ static int wait_task_zombie(struct wait_opts *wo, struct task_struct *p)
 		struct signal_struct *psig;
 		struct signal_struct *sig;
 		unsigned long maxrss;
-		cputime_t tgutime, tgstime;
+		struct task_cputime tg_cputime;
 
 		/*
 		 * The resource counters for the group leader are in its
@@ -1088,12 +1088,12 @@ static int wait_task_zombie(struct wait_opts *wo, struct task_struct *p)
 		 * group, which consolidates times for all threads in the
 		 * group including the group leader.
 		 */
-		thread_group_cputime_adjusted(p, &tgutime, &tgstime);
+		thread_group_cputime_adjusted(p, &tg_cputime);
 		spin_lock_irq(&p->real_parent->sighand->siglock);
 		psig = p->real_parent->signal;
 		sig = p->signal;
-		psig->cutime += tgutime + sig->cutime;
-		psig->cstime += tgstime + sig->cstime;
+		psig->cutime += tg_cputime.utime + sig->cutime;
+		psig->cstime += tg_cputime.stime + sig->cstime;
 		psig->cgtime += task_gtime(p) + sig->gtime + sig->cgtime;
 		psig->cmin_flt +=
 			p->min_flt + sig->min_flt + sig->cmin_flt;
kernel/sched/cputime.c +6 -12
index 4006b42..a600f7f 100644
@@ -456,14 +456,9 @@ void task_cputime_adjusted(struct task_struct *p, cputime_t *ut, cputime_t *st)
 	*st = p->stime;
 }
 
-void thread_group_cputime_adjusted(struct task_struct *p, cputime_t *ut, cputime_t *st)
+void thread_group_cputime_adjusted(struct task_struct *p, struct task_cputime *cputime)
 {
-	struct task_cputime cputime;
-
-	thread_group_cputime(p, &cputime);
-
-	*ut = cputime.utime;
-	*st = cputime.stime;
+	thread_group_cputime(p, cputime);
 }
 #else /* !CONFIG_VIRT_CPU_ACCOUNTING_NATIVE */
 /*
@@ -618,12 +613,11 @@ void task_cputime_adjusted(struct task_struct *p, cputime_t *ut, cputime_t *st)
 /*
  * Must be called with siglock held.
  */
-void thread_group_cputime_adjusted(struct task_struct *p, cputime_t *ut, cputime_t *st)
+void thread_group_cputime_adjusted(struct task_struct *p, struct task_cputime *cputime)
 {
-	struct task_cputime cputime;
-
-	thread_group_cputime(p, &cputime);
-	cputime_adjust(&cputime, &p->signal->prev_cputime, ut, st);
+	thread_group_cputime(p, cputime);
+	cputime_adjust(cputime, &p->signal->prev_cputime,
+		       &cputime->utime, &cputime->stime);
 }
 #endif /* !CONFIG_VIRT_CPU_ACCOUNTING_NATIVE */
 
kernel/sys.c +10 -8
index 39c9c4a..2f555c1 100644
@@ -1045,15 +1045,16 @@ change_okay:
 
 void do_sys_times(struct tms *tms)
 {
-	cputime_t tgutime, tgstime, cutime, cstime;
+	cputime_t cutime, cstime;
+	struct task_cputime tg_cputime;
 
 	spin_lock_irq(&current->sighand->siglock);
-	thread_group_cputime_adjusted(current, &tgutime, &tgstime);
+	thread_group_cputime_adjusted(current, &tg_cputime);
 	cutime = current->signal->cutime;
 	cstime = current->signal->cstime;
 	spin_unlock_irq(&current->sighand->siglock);
-	tms->tms_utime = cputime_to_clock_t(tgutime);
-	tms->tms_stime = cputime_to_clock_t(tgstime);
+	tms->tms_utime = cputime_to_clock_t(tg_cputime.utime);
+	tms->tms_stime = cputime_to_clock_t(tg_cputime.stime);
 	tms->tms_cutime = cputime_to_clock_t(cutime);
 	tms->tms_cstime = cputime_to_clock_t(cstime);
 }
@@ -1699,8 +1700,9 @@ static void k_getrusage(struct task_struct *p, int who, struct rusage *r)
 {
 	struct task_struct *t;
 	unsigned long flags;
-	cputime_t tgutime, tgstime, utime, stime;
+	cputime_t utime, stime;
 	unsigned long maxrss = 0;
+	struct task_cputime tg_cputime;
 
 	memset((char *) r, 0, sizeof *r);
 	utime = stime = 0;
@@ -1732,9 +1734,9 @@ static void k_getrusage(struct task_struct *p, int who, struct rusage *r)
 				break;
 
 		case RUSAGE_SELF:
-			thread_group_cputime_adjusted(p, &tgutime, &tgstime);
-			utime += tgutime;
-			stime += tgstime;
+			thread_group_cputime_adjusted(p, &tg_cputime);
+			utime += tg_cputime.utime;
+			stime += tg_cputime.stime;
 			r->ru_nvcsw += p->signal->nvcsw;
 			r->ru_nivcsw += p->signal->nivcsw;
 			r->ru_minflt += p->signal->min_flt;
Signature · 1 line
1.7.1
#2 lore raw

RFC 2/4 procfs: add sum_exec_runtime to /proc/PID/stat

In reply to Stanislaw Gruszka

Allow procps programs (i.e. top) to read precise CPU time usage of processes.

---
 fs/proc/array.c |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)
fs/proc/array.c +5 -0
index fef7850..ee47b29 100644
@@ -403,6 +403,7 @@ static int do_task_stat(struct seq_file *m, struct pid_namespace *ns,
 	unsigned long rsslim = 0;
 	char tcomm[sizeof(task->comm)];
 	unsigned long flags;
+	u64 sum_exec_runtime = 0;
 
 	state = *get_task_state(task);
 	vsize = eip = esp = 0;
@@ -459,6 +460,7 @@ static int do_task_stat(struct seq_file *m, struct pid_namespace *ns,
 			thread_group_cputime_adjusted(task, &cputime);
 			utime = cputime.utime;
 			stime = cputime.stime;
+			sum_exec_runtime = cputime.sum_exec_runtime;
 			gtime += sig->gtime;
 		}
 
@@ -475,6 +477,7 @@ static int do_task_stat(struct seq_file *m, struct pid_namespace *ns,
 		min_flt = task->min_flt;
 		maj_flt = task->maj_flt;
 		task_cputime_adjusted(task, &utime, &stime);
+		sum_exec_runtime = task->se.sum_exec_runtime;
 		gtime = task_gtime(task);
 	}
 
@@ -554,6 +557,8 @@ static int do_task_stat(struct seq_file *m, struct pid_namespace *ns,
 	else
 		seq_put_decimal_ll(m, ' ', 0);
 
+	seq_put_decimal_ull(m, ' ', sum_exec_runtime);
+
 	seq_putc(m, '\n');
 	if (mm)
 		mmput(mm);
Signature · 1 line
1.7.1
#3 lore raw

RFC 3/4 sched,proc: add csum_sched_runtime

In reply to Stanislaw Gruszka

Account and export via proc scheduler calculated time of children execution.

---
 fs/proc/array.c       |    3 +++
 include/linux/sched.h |    1 +
 kernel/exit.c         |    2 ++
 3 files changed, 6 insertions(+), 0 deletions(-)
fs/proc/array.c +3 -0
index ee47b29..1444dc5 100644
@@ -404,6 +404,7 @@ static int do_task_stat(struct seq_file *m, struct pid_namespace *ns,
 	char tcomm[sizeof(task->comm)];
 	unsigned long flags;
 	u64 sum_exec_runtime = 0;
+	u64 csum_exec_runtime = 0;
 
 	state = *get_task_state(task);
 	vsize = eip = esp = 0;
@@ -442,6 +443,7 @@ static int do_task_stat(struct seq_file *m, struct pid_namespace *ns,
 		cutime = sig->cutime;
 		cstime = sig->cstime;
 		cgtime = sig->cgtime;
+		csum_exec_runtime = sig->csum_sched_runtime;
 		rsslim = ACCESS_ONCE(sig->rlim[RLIMIT_RSS].rlim_cur);
 
 		/* add up live thread stats at the group level */
@@ -558,6 +560,7 @@ static int do_task_stat(struct seq_file *m, struct pid_namespace *ns,
 		seq_put_decimal_ll(m, ' ', 0);
 
 	seq_put_decimal_ull(m, ' ', sum_exec_runtime);
+	seq_put_decimal_ull(m, ' ', csum_exec_runtime);
 
 	seq_putc(m, '\n');
 	if (mm)
include/linux/sched.h +1 -0
index eb6005c..6673a59 100644
@@ -574,6 +574,7 @@ struct signal_struct {
 	 * other than jiffies.)
 	 */
 	unsigned long long sum_sched_runtime;
+	unsigned long long csum_sched_runtime;
 
 	/*
 	 * We don't bother to synchronize most readers of this at all,
kernel/exit.c +2 -0
index 5a7023f..eda4fa7 100644
@@ -1094,6 +1094,8 @@ static int wait_task_zombie(struct wait_opts *wo, struct task_struct *p)
 		sig = p->signal;
 		psig->cutime += tg_cputime.utime + sig->cutime;
 		psig->cstime += tg_cputime.stime + sig->cstime;
+		psig->csum_sched_runtime +=
+			tg_cputime.sum_exec_runtime + sig->csum_sched_runtime;
 		psig->cgtime += task_gtime(p) + sig->gtime + sig->cgtime;
 		psig->cmin_flt +=
 			p->min_flt + sig->min_flt + sig->cmin_flt;
Signature · 1 line
1.7.1
#4 lore raw

RFC 4/4 cputime: remove scaling

In reply to Stanislaw Gruszka

Scaling cputime cause problems, bunch of them was fixed, but still is possible to hit multiplication overflow issue, which make {u,s}time values incorrect. This problem has no good solution in kernel.

This patch remove scaling code and export raw values of {u,t}ime . Procps programs can use newly introduced sum_exec_runtime to find out precisely calculated process cpu time and scale utime, stime values accordingly.

Unfortunately times(2) syscall has no such option.

This change affect kernels compiled without CONFIG_VIRT_CPU_ACCOUNTING_*.

---
 fs/proc/array.c        |    4 +-
 include/linux/sched.h  |   20 --------
 kernel/exit.c          |    2 +-
 kernel/fork.c          |    3 -
 kernel/sched/cputime.c |  117 +-----------------------------------------------
 kernel/sys.c           |    6 +-
 6 files changed, 7 insertions(+), 145 deletions(-)
fs/proc/array.c +2 -2
index 1444dc5..5feadc4 100644
@@ -459,7 +459,7 @@ static int do_task_stat(struct seq_file *m, struct pid_namespace *ns,
 
 			min_flt += sig->min_flt;
 			maj_flt += sig->maj_flt;
-			thread_group_cputime_adjusted(task, &cputime);
+			thread_group_cputime(task, &cputime);
 			utime = cputime.utime;
 			stime = cputime.stime;
 			sum_exec_runtime = cputime.sum_exec_runtime;
@@ -478,7 +478,7 @@ static int do_task_stat(struct seq_file *m, struct pid_namespace *ns,
 	if (!whole) {
 		min_flt = task->min_flt;
 		maj_flt = task->maj_flt;
-		task_cputime_adjusted(task, &utime, &stime);
+		task_cputime(task, &utime, &stime);
 		sum_exec_runtime = task->se.sum_exec_runtime;
 		gtime = task_gtime(task);
 	}
include/linux/sched.h +0 -20
index 6673a59..26257a7 100644
@@ -397,18 +397,6 @@ struct cpu_itimer {
 };
 
 /**
- * struct cputime - snaphsot of system and user cputime
- * @utime: time spent in user mode
- * @stime: time spent in system mode
- *
- * Gathers a generic snapshot of user and system time.
- */
-struct cputime {
-	cputime_t utime;
-	cputime_t stime;
-};
-
-/**
  * struct task_cputime - collected CPU time counts
  * @utime:		time spent in user mode, in &cputime_t units
  * @stime:		time spent in kernel mode, in &cputime_t units
@@ -558,9 +546,6 @@ struct signal_struct {
 	cputime_t utime, stime, cutime, cstime;
 	cputime_t gtime;
 	cputime_t cgtime;
-#ifndef CONFIG_VIRT_CPU_ACCOUNTING_NATIVE
-	struct cputime prev_cputime;
-#endif
 	unsigned long nvcsw, nivcsw, cnvcsw, cnivcsw;
 	unsigned long min_flt, maj_flt, cmin_flt, cmaj_flt;
 	unsigned long inblock, oublock, cinblock, coublock;
@@ -1159,9 +1144,6 @@ struct task_struct {
 
 	cputime_t utime, stime, utimescaled, stimescaled;
 	cputime_t gtime;
-#ifndef CONFIG_VIRT_CPU_ACCOUNTING_NATIVE
-	struct cputime prev_cputime;
-#endif
 #ifdef CONFIG_VIRT_CPU_ACCOUNTING_GEN
 	seqlock_t vtime_seqlock;
 	unsigned long long vtime_snap;
@@ -1595,8 +1577,6 @@ static inline cputime_t task_gtime(struct task_struct *t)
 	return t->gtime;
 }
 #endif
-extern void task_cputime_adjusted(struct task_struct *p, cputime_t *ut, cputime_t *st);
-extern void thread_group_cputime_adjusted(struct task_struct *p, struct task_cputime *ct);
 
 /*
  * Per process flags
kernel/exit.c +1 -1
index eda4fa7..9a7ecbb 100644
@@ -1088,7 +1088,7 @@ static int wait_task_zombie(struct wait_opts *wo, struct task_struct *p)
 		 * group, which consolidates times for all threads in the
 		 * group including the group leader.
 		 */
-		thread_group_cputime_adjusted(p, &tg_cputime);
+		thread_group_cputime(p, &tg_cputime);
 		spin_lock_irq(&p->real_parent->sighand->siglock);
 		psig = p->real_parent->signal;
 		sig = p->signal;
kernel/fork.c +0 -3
index 339f60d..2ae1706 100644
@@ -1233,9 +1233,6 @@ static struct task_struct *copy_process(unsigned long clone_flags,
 
 	p->utime = p->stime = p->gtime = 0;
 	p->utimescaled = p->stimescaled = 0;
-#ifndef CONFIG_VIRT_CPU_ACCOUNTING_NATIVE
-	p->prev_cputime.utime = p->prev_cputime.stime = 0;
-#endif
 #ifdef CONFIG_VIRT_CPU_ACCOUNTING_GEN
 	seqlock_init(&p->vtime_seqlock);
 	p->vtime_snap = 0;
kernel/sched/cputime.c +1 -116

129 lines not on this page · raw message

kernel/sys.c +3 -3
index 2f555c1..00f143e 100644
@@ -1049,7 +1049,7 @@ void do_sys_times(struct tms *tms)
 	struct task_cputime tg_cputime;
 
 	spin_lock_irq(&current->sighand->siglock);
-	thread_group_cputime_adjusted(current, &tg_cputime);
+	thread_group_cputime(current, &tg_cputime);
 	cutime = current->signal->cutime;
 	cstime = current->signal->cstime;
 	spin_unlock_irq(&current->sighand->siglock);
@@ -1708,7 +1708,7 @@ static void k_getrusage(struct task_struct *p, int who, struct rusage *r)
 	utime = stime = 0;
 
 	if (who == RUSAGE_THREAD) {
-		task_cputime_adjusted(current, &utime, &stime);
+		task_cputime(current, &utime, &stime);
 		accumulate_thread_rusage(p, r);
 		maxrss = p->signal->maxrss;
 		goto out;
@@ -1734,7 +1734,7 @@ static void k_getrusage(struct task_struct *p, int who, struct rusage *r)
 				break;
 
 		case RUSAGE_SELF:
-			thread_group_cputime_adjusted(p, &tg_cputime);
+			thread_group_cputime(p, &tg_cputime);
 			utime += tg_cputime.utime;
 			stime += tg_cputime.stime;
 			r->ru_nvcsw += p->signal->nvcsw;
Signature · 1 line
1.7.1
#5 lore raw

Re: [RFC 4/4] cputime: remove scaling

Quoted 3 lines · Stanislaw Gruszka

Scaling cputime cause problems, bunch of them was fixed, but still is possible to hit multiplication overflow issue, which make {u,s}time values incorrect. This problem has no good solution in kernel.

Wasn't 128-bit math a solution to the overflow problems? 128-bit math isn't nice, but at least for multiplication it's defensible.

Quoted 7 lines

This patch remove scaling code and export raw values of {u,t}ime . Procps programs can use newly introduced sum_exec_runtime to find out precisely calculated process cpu time and scale utime, stime values accordingly.

Unfortunately times(2) syscall has no such option.

This change affect kernels compiled without CONFIG_VIRT_CPU_ACCOUNTING_*.

So, the concern here is that 'top hiding' code can now hide again. It's also that we are not really solving the problem, we are pushing it to user-space - which in the best case gets updated to solve the problem in some similar fashion - and in the worst case does not get updated or does it in a buggy way.

So while user-space has it a bit easier because it can do floating point math, is there really no workable solution to the current kernel side integer overflow bug? I really prefer robust kernel side accounting/instrumentation.

Thanks,

Ingo

#6 lore raw

Re: [RFC 4/4] cputime: remove scaling

I have a patch that does scaling by multiply for 64-bit architectures. I probably should clean it up and send it in. I need to see if it fixes this problem.

Quoted 42 lines · Ingo Molnar
Quoted 1 line · Stanislaw Gruszka

Scaling cputime cause problems, bunch of them was fixed, but still is

possible

Quoted 1 line

to hit multiplication overflow issue, which make {u,s}time values

incorrect.

Quoted 1 line

This problem has no good solution in kernel.

Wasn't 128-bit math a solution to the overflow problems? 128-bit math isn't nice,
but at least for multiplication it's defensible.

Quoted 1 line

This patch remove scaling code and export raw values of {u,t}ime .

Procps

Quoted 1 line

programs can use newly introduced sum_exec_runtime to find out

precisely

Quoted 1 line

calculated process cpu time and scale utime, stime values

accordingly.

Quoted 4 lines

Unfortunately times(2) syscall has no such option.

This change affect kernels compiled without

CONFIG_VIRT_CPU_ACCOUNTING_*.

So, the concern here is that 'top hiding' code can now hide again. It's also that
we are not really solving the problem, we are pushing it to user-space
- which in
the best case gets updated to solve the problem in some similar fashion
- and in
the worst case does not get updated or does it in a buggy way.

So while user-space has it a bit easier because it can do floating point math, is
there really no workable solution to the current kernel side integer overflow bug?
I really prefer robust kernel side accounting/instrumentation.

Thanks,

Ingo

Signature · 1 line
Sent from my mobile phone. Please excuse brevity and lack of formatting.
#7 lore raw

Re: [RFC 4/4] cputime: remove scaling

Quoted 1 line · H. Peter Anvin

I have a patch that does scaling by multiply for 64-bit architectures. I probably should clean it up and send it in. I need to see if it fixes this problem.

Interesting. Could you attach draft patch, so I could look at it ?

Stanislaw

#8 lore raw

Re: [RFC 4/4] cputime: remove scaling

Quoted 4 lines · Stanislaw Gruszka
Quoted 1 line · H. Peter Anvin

I have a patch that does scaling by multiply for 64-bit architectures. I probably should clean it up and send it in. I need to see if it fixes this problem.

Interesting. Could you attach draft patch, so I could look at it ?

It's just the trivial extension of the code already in kernel/time.c to the 64-bit case.

However, it looks like in your case the scaling factor stime/total is dynamic. This means that doing scaling by multiplication isn't practical.

I guess I don't understand what function the scaling is supposed to serve there.

-hpa

Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.
#9 lore raw

Re: [RFC 4/4] cputime: remove scaling

In reply to Ingo Molnar
Quoted 9 lines · Ingo Molnar
Quoted 3 lines · Stanislaw Gruszka

Scaling cputime cause problems, bunch of them was fixed, but still is possible to hit multiplication overflow issue, which make {u,s}time values incorrect. This problem has no good solution in kernel.

Wasn't 128-bit math a solution to the overflow problems? 128-bit math isn't nice, but at least for multiplication it's defensible.

128 bit division is needed unfortunately. Though on 99.9% of cases, it will go through 64 bit fast path.

Quoted 8 lines
Quoted 7 lines

This patch remove scaling code and export raw values of {u,t}ime . Procps programs can use newly introduced sum_exec_runtime to find out precisely calculated process cpu time and scale utime, stime values accordingly.

Unfortunately times(2) syscall has no such option.

This change affect kernels compiled without CONFIG_VIRT_CPU_ACCOUNTING_*.

Quoted 7 lines

So, the concern here is that 'top hiding' code can now hide again. It's also that we are not really solving the problem, we are pushing it to user-space - which in the best case gets updated to solve the problem in some similar fashion - and in the worst case does not get updated or does it in a buggy way.

So while user-space has it a bit easier because it can do floating point math, is there really no workable solution to the current kernel side integer overflow bug?

I do not see any. Basically all we have make problem less reproducible or just defer it. The best solution, except full 128 bit math I found is something like this (dropping precision if values are big and overflow will happen):

u64 _scale_time(u64 rtime, u64 total, u64 time)

{
        const int zero_bits = clzll(time) + clzll(rtime);
        u64 scaled;

        if (zero_bits < 64) {
                /* Drop precision */
                const int drop_bits = 64 - zero_bits;

                time >>= drop_bits;
                rtime >>= drop_bits;
                total >>= 2*drop_bits;

                if (total == 0)
                        return time;
        }

        scaled = (time * rtime) / total;

        return scaled;
}

It defer problem to quite long period. My testing script detect failure at:

FAIL!
rtime: 1954463459156 <- 22621 days (one thread , CONFIG_HZ=1000)
total: 1771603722423
stime: 354320744484
kernel: 391351504748 <- kernel value
python: 390892691830 <- correct value

For one thread this is fine, but for 512 threads inaccuracy will happen after only 40 days (due to dropping too many of "total" variable bits).

Quoted 1 line

I really prefer robust kernel side accounting/instrumentation.

We have CONFIG_IRQ_TIME_ACCOUNTING and CONFIG_VIRT_CPU_ACCOUNTING_GEN. Perhaps we can change to use one of those options by default. I wonder if the additional performance cost related with them is really something that we should care about. Are there any measurement that show those will make performance worse ?

Stanislaw

#10 lore raw

Re: [RFC 4/4] cputime: remove scaling

Quoted 7 lines · Stanislaw Gruszka
Quoted 1 line

I really prefer robust kernel side accounting/instrumentation.

We have CONFIG_IRQ_TIME_ACCOUNTING and CONFIG_VIRT_CPU_ACCOUNTING_GEN. Perhaps we can change to use one of those options by default. I wonder if the additional performance cost related with them is really something that we should care about. Are there any measurement that show those will make performance worse ?

CONFIG_IRQ_TIME_ACCOUNTING also make use of scaling. And CONFIG_VIRT_CPU_ACCOUNTING_GEN involves too much overhead on IO-bound workloads. It's mostly good for undisturbed userspace bound workloads (few IRQs, few exceptions, few syscalls).