Skip to contentSkip to conversation

per cpu times

PATCH In review Erich Focht 6 participants 7 messages Jul 18, 2003 – Jul 23, 2003

Subsystems (4)
MM core Scheduler exec and binfmt Proc filesystem
lore
Share & tools
lore

Erich Focht

author
#0 lore raw

PATCH 1/1 per cpu times

This patch brings back the per CPU user & system times which one was used to see in /proc/PID/cpu with 2.4 kernels. Useful for SMP and NUMA scheduler development, needed for reasonable output in numabench / numa_test.

Regards,
Erich

fs/proc/array.c +23 -0
diff -urN 2.6.0-test1-ia64-0/fs/proc/array.c 2.6.0-test1-ia64-na/fs/proc/array.c
@@ -405,3 +405,26 @@
 	return sprintf(buffer,"%d %d %d %d %d %d %d\n",
 		       size, resident, shared, text, lib, data, 0);
 }
+
+#ifdef CONFIG_SMP
+int proc_pid_cpu(struct task_struct *task, char * buffer)
+{
+	int i, len;
+
+	len = sprintf(buffer,
+		"cpu  %lu %lu\n",
+		jiffies_to_clock_t(task->utime),
+		jiffies_to_clock_t(task->stime));
+		
+	for (i = 0 ; i < NR_CPUS; i++) {
+		if (cpu_online(i))
+		len += sprintf(buffer + len, "cpu%d %lu %lu\n",
+			i,
+			jiffies_to_clock_t(task->per_cpu_utime[i]),
+			jiffies_to_clock_t(task->per_cpu_stime[i]));
+
+	}
+	len += sprintf(buffer + len, "current_cpu %d\n",task_cpu(task));
+	return len;
+}
+#endif
fs/proc/base.c +10 -0
diff -urN 2.6.0-test1-ia64-0/fs/proc/base.c 2.6.0-test1-ia64-na/fs/proc/base.c
@@ -56,6 +56,7 @@
 	PROC_PID_STAT,
 	PROC_PID_STATM,
 	PROC_PID_MAPS,
+	PROC_PID_CPU,
 	PROC_PID_MOUNTS,
 	PROC_PID_WCHAN,
 #ifdef CONFIG_SECURITY

@@ -83,6 +84,9 @@
   E(PROC_PID_CMDLINE,	"cmdline",	S_IFREG|S_IRUGO),
   E(PROC_PID_STAT,	"stat",		S_IFREG|S_IRUGO),
   E(PROC_PID_STATM,	"statm",	S_IFREG|S_IRUGO),
+#ifdef CONFIG_SMP
+  E(PROC_PID_CPU,	"cpu",		S_IFREG|S_IRUGO),
+#endif
   E(PROC_PID_MAPS,	"maps",		S_IFREG|S_IRUGO),
   E(PROC_PID_MEM,	"mem",		S_IFREG|S_IRUSR|S_IWUSR),
   E(PROC_PID_CWD,	"cwd",		S_IFLNK|S_IRWXUGO),

@@ -1170,6 +1174,12 @@
 			inode->i_fop = &proc_info_file_operations;
 			ei->op.proc_read = proc_pid_stat;
 			break;
+#ifdef CONFIG_SMP
+		case PROC_PID_CPU:
+			inode->i_fop = &proc_info_file_operations;
+			ei->op.proc_read = proc_pid_cpu;
+			break;
+#endif
 		case PROC_PID_CMDLINE:
 			inode->i_fop = &proc_info_file_operations;
 			ei->op.proc_read = proc_pid_cmdline;
include/linux/sched.h +3 -0
diff -urN 2.6.0-test1-ia64-0/include/linux/sched.h 2.6.0-test1-ia64-na/include/linux/sched.h
@@ -390,6 +390,9 @@
 	struct list_head posix_timers; /* POSIX.1b Interval Timers */
 	unsigned long utime, stime, cutime, cstime;
 	u64 start_time;
+#ifdef CONFIG_SMP
+	long per_cpu_utime[NR_CPUS], per_cpu_stime[NR_CPUS];
+#endif
 /* mm fault and swap info: this can arguably be seen as either mm-specific or thread-specific */
 	unsigned long min_flt, maj_flt, nswap, cmin_flt, cmaj_flt, cnswap;
 /* process credentials */
kernel/fork.c +8 -0
diff -urN 2.6.0-test1-ia64-0/kernel/fork.c 2.6.0-test1-ia64-na/kernel/fork.c
@@ -861,6 +861,14 @@
 	p->tty_old_pgrp = 0;
 	p->utime = p->stime = 0;
 	p->cutime = p->cstime = 0;
+#ifdef CONFIG_SMP
+	{
+		int i;
+
+		for(i = 0; i < NR_CPUS; i++)
+			p->per_cpu_utime[i] = p->per_cpu_stime[i] = 0;
+	}
+#endif
 	p->array = NULL;
 	p->lock_depth = -1;		/* -1 = no lock */
 	p->start_time = get_jiffies_64();
kernel/timer.c +4 -0
diff -urN 2.6.0-test1-ia64-0/kernel/timer.c 2.6.0-test1-ia64-na/kernel/timer.c
@@ -720,6 +720,10 @@
 void update_one_process(struct task_struct *p, unsigned long user,
 			unsigned long system, int cpu)
 {
+#ifdef CONFIG_SMP
+	p->per_cpu_utime[cpu] += user;
+	p->per_cpu_stime[cpu] += system;
+#endif
 	do_process_times(p, user, system);
 	do_it_virt(p, user);
 	do_it_prof(p);
#1 lore raw
Quoted 6 lines · Erich Focht

This patch brings back the per CPU user & system times which one was used to see in /proc/PID/cpu with 2.4 kernels. Useful for SMP and NUMA scheduler development, needed for reasonable output in numabench / numa_test.

On a somewhat related note ...

We (Big Blue) have a performance reporting application that would like to know how long a task sits on a runqueue before it is actually given the CPU. In other words, it wants to know how long the 'runnable task' was delayed due to contention for the CPU(s). Of course, one could get an overall feel for this based on total runqueue length. However, this app would really like this info on a per-task basis.

Does anyone else think this type of info would be useful?

A patch to compute/export this info should be straight forward to implement.

Signature · 1 line
Mike
#2 lore raw
Quoted 11 lines · Mike Kravetz

On a somewhat related note ...
We (Big Blue) have a performance reporting application that would like to know how long a task sits on a runqueue before it is actually given the CPU. In other words, it wants to know how long the 'runnable task' was delayed due to contention for the CPU(s). Of course, one could get an overall feel for this based on total runqueue length. However, this app would really like this info on a per-task basis.
Does anyone else think this type of info would be useful? A patch to compute/export this info should be straight forward to implement.

I wrote something to collect the standard queueing statistics a while back but am not sure what I did with it. I think Rick Lindsley might still have a copy around.

-- wli

#3 lore raw

I wrote something to collect the standard queueing statistics a while back but am not sure what I did with it. I think Rick Lindsley might still have a copy around.

Actually, better than that -- it's in the mjb tree. But I'll repost it here for those who might find it generally useful. I've regenerated the patch below for 2.6.0-test1, but while it applies and compiles cleanly, I haven't really exercised it to any extent yet in 2.6.

The code adds extra fields to the stat field for each process and each cpu.

Note that per-process info is only available while that process is alive. Once a process exits, the specific information about it is lost. The extra fields for the cpu info in /proc decay over time instead of accumulating, so they are more of a "wait average" (similar to a load average) rather than a sum of wait times.

Rick

fs/proc/array.c +5 -2
diff -rup linux-2.6.0-test1/fs/proc/array.c linux-2.6.0-qs/fs/proc/array.c
@@ -336,7 +336,7 @@ int proc_pid_stat(struct task_struct *ta
 	read_unlock(&tasklist_lock);
 	res = sprintf(buffer,"%d (%s) %c %d %d %d %d %d %lu %lu \
 %lu %lu %lu %lu %lu %ld %ld %ld %ld %ld %ld %llu %lu %ld %lu %lu %lu %lu %lu \
-%lu %lu %lu %lu %lu %lu %lu %lu %d %d %lu %lu\n",
+%lu %lu %lu %lu %lu %lu %lu %lu %d %d %lu %lu %lu %lu %lu\n",
 		task->pid,
 		task->comm,
 		state,

@@ -382,7 +382,10 @@ int proc_pid_stat(struct task_struct *ta
 		task->exit_signal,
 		task_cpu(task),
 		task->rt_priority,
-		task->policy);
+		task->policy,
+		jiffies_to_clock_t(task->sched_info.inter_arrival_time),
+		jiffies_to_clock_t(task->sched_info.service_time),
+		jiffies_to_clock_t(task->sched_info.response_time));
 	if(mm)
 		mmput(mm);
 	return res;
fs/proc/proc_misc.c +9 -3
diff -rup linux-2.6.0-test1/fs/proc/proc_misc.c linux-2.6.0-qs/fs/proc/proc_misc.c
@@ -401,14 +401,20 @@ static int kstat_read_proc(char *page, c
 		jiffies_to_clock_t(idle),
 		jiffies_to_clock_t(iowait));
 	for (i = 0 ; i < NR_CPUS; i++){
-		if (!cpu_online(i)) continue;
-		len += sprintf(page + len, "cpu%d %u %u %u %u %u\n",
+		struct sched_info info;
+		if (!cpu_online(i))
+			continue;
+		cpu_sched_info(&info, i);
+		len += sprintf(page + len, "cpu%d %u %u %u %u %u %u %u %u\n",
 			i,
 			jiffies_to_clock_t(kstat_cpu(i).cpustat.user),
 			jiffies_to_clock_t(kstat_cpu(i).cpustat.nice),
 			jiffies_to_clock_t(kstat_cpu(i).cpustat.system),
 			jiffies_to_clock_t(kstat_cpu(i).cpustat.idle),
-			jiffies_to_clock_t(kstat_cpu(i).cpustat.iowait));
+			jiffies_to_clock_t(kstat_cpu(i).cpustat.iowait),
+			(uint) jiffies_to_clock_t(info.inter_arrival_time),
+			(uint) jiffies_to_clock_t(info.service_time),
+			(uint) jiffies_to_clock_t(info.response_time));
 	}
 	len += sprintf(page + len, "intr %u", sum);
include/linux/sched.h +12 -0
diff -rup linux-2.6.0-test1/include/linux/sched.h linux-2.6.0-qs/include/linux/sched.h
@@ -94,6 +94,9 @@ extern unsigned long nr_running(void);
 extern unsigned long nr_uninterruptible(void);
 extern unsigned long nr_iowait(void);
 
+struct sched_info;
+extern void cpu_sched_info(struct sched_info *, int);
+
 #include <linux/time.h>
 #include <linux/param.h>
 #include <linux/resource.h>

@@ -320,6 +323,13 @@ struct k_itimer {
 	struct sigqueue *sigq;		/* signal queue entry. */
 };
 
+struct sched_info {
+	/* running averages */
+	unsigned long response_time, inter_arrival_time, service_time;
+
+	/* timestamps */
+	unsigned long last_arrival, began_service;
+};
 
 struct io_context;			/* See blkdev.h */
 void exit_io_context(void);

@@ -344,6 +354,8 @@ struct task_struct {
 	unsigned long cpus_allowed;
 	unsigned int time_slice, first_time_slice;
 
+	struct sched_info sched_info;
+
 	struct list_head tasks;
 	struct list_head ptrace_children;
 	struct list_head ptrace_list;
kernel/sched.c +87 -2

138 lines not on this page · raw message

#4 lore raw
In reply to Mike Kravetz
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Mike" == Mike Kravetz <[email protected]> writes:

Quoted 5 lines · Erich Focht
Quoted 5 lines

This patch brings back the per CPU user & system times which one was used to see in /proc/PID/cpu with 2.4 kernels. Useful for SMP and NUMA scheduler development, needed for reasonable output in numabench / numa_test.

Mike> On a somewhat related note ...

Mike> We (Big Blue) have a performance reporting application that Mike> would like to know how long a task sits on a runqueue before it Mike> is actually given the CPU. In other words, it wants to know how Mike> long the 'runnable task' was delayed due to contention for the Mike> CPU(s). Of course, one could get an overall feel for this based Mike> on total runqueue length. However, this app would really like Mike> this info on a per-task basis.

Mike> Does anyone else think this type of info would be useful?

This is exactly what my microstate accounting patch does.

Per task figures for:
-- how long on CPU
-- how long on active queue
-- how long on expired queue
-- how long sleeping for paging
-- how long sleeping in other non-interruptible state -- how long sleeping interruptibly
-- how much time stolen for interrupts
-- how long in system call
-- how long sleeping on Futex
-- how long sleeping for epoll, poll or select

I haven't yet added time spent handling traps, so the ONCPU time includes pagefault and other trap time; also I've implemented the low level timers only for X86 and IA64. It'd be pretty trivial to add other architectures.

The most recent published version of the patch is at
http://www.ussg.iu.edu/hypermail/linux/kernel/0306.3/0636.html
(that one doesn't include all the states I mentioned, but the on-queue times *are* counted)

There'll be another patch with more states soon.

--
Dr Peter Chubb http://www.gelato.unsw.edu.au peterc AT gelato.unsw.edu.au
You are lost in a maze of BitKeeper repositories, all slightly different.

#5 lore raw
In reply to Mike Kravetz

In article <[email protected]>, Mike Kravetz <[email protected]> wrote:

| On a somewhat related note ...
| 
| We (Big Blue) have a performance reporting application that
| would like to know how long a task sits on a runqueue before
| it is actually given the CPU.  In other words, it wants to
| know how long the 'runnable task' was delayed due to contention
| for the CPU(s).  Of course, one could get an overall feel for
| this based on total runqueue length.  However, this app would
| really like this info on a per-task basis.

This is certainly a useful number. It's easy to tell when the CPU is "in use," but it's not easy to tell when it's "busy" and processes are waiting for a CPU.

Signature · 3 lines
bill davidsen <[email protected]>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.
#6 lore raw
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

"bill" == bill davidsen <[email protected]> writes:
"Mike" == Mike Kravetz <[email protected]>

Mike> On a somewhat related note ... We (Big Blue) have a Mike> performance reporting application that would like to know how Mike> long a task sits on a runqueue before it is actually given the Mike> CPU. In other words, it wants to know how long the 'runnable Mike> task' was delayed due to contention for the CPU(s). Of Mike> course, one could get an overall feel for this based on total Mike> runqueue length. However, this app would really like this Mike> info on a per-task basis.

bill> This is certainly a useful number.

This is exactly what's measured by the microstate accounting patches I've been pushing to LKML, along with a few other useful statistics.

If you try it, please let me know: see
http://marc.theaimsgroup.com/?l=linux-kernel&m=105884469205748&w=2

--
Dr Peter Chubb http://www.gelato.unsw.edu.au peterc AT gelato.unsw.edu.au
You are lost in a maze of BitKeeper repositories, all slightly different.