Skip to contentSkip to conversation

KS/Plumbers: c-state governor BOF

DISCUSSION Rik van Riel 2 participants 6 messages Aug 24, 2012 – Sep 11, 2012

lore
Share & tools
lore
#0 lore raw

After talking about my RFC patches to the c-state governor with Matthew and Arjan, it is clear that the whole concept of how things are done could use some more discussion.

Since a good number of us will be in San Diego next week, at Kernel Summit / Plumbers / etc, I will organize a c-state governor BOF for those who are interested.

Things to think about:
- what should the c-state governor do?
- how to best predict the future?
- what kinds of odd workloads do we need to accomodate?

I will talk to the LF people about getting space for such a BOF (probably after a mini-summit is done with a room, or a spare room), and will announce when and where such a BOF will take place.

Watch whatever board is used to announce BOFs next week :)

Signature · 1 line
All rights reversed
#1 lore raw
Quoted 3 lines · Rik van Riel

Since a good number of us will be in San Diego next week, at Kernel Summit / Plumbers / etc, I will organize a c-state governor BOF for those who are interested.

Here are the note from today's c-state governor BOF session:

- idle time estimator can be improved to return more conservative values
- have a trace point to verify how well idle time estimation works, returning expected idle time for the current algorithm, expected idle time for a test algorithm, and actually measured idle time
- we can objectively measure whether one idle time estimation algorithm is better than another
- example: last 8 idle times are 180 200 30 210 1000 150 10000 190
- average idle time around 1500, but not typical
- typical idle time just under 200
- almost all idle time spent in one interval
- scheduler and cpufreq people also want idle time statistics, idea is to move the idle time statistics to the scheduler
- having idle time statistics in the scheduler allows the stats to be migrated when the scheduler migrates
a task
- interface: expected idle time, demotion time
- if we sleep much more than the expected idle time, put CPU into a deeper c-state
- on x86 timed mwait could be used
- generic timer code would be usable too
- take into account both the idle statistics and power break-even

#2 lore raw
In reply to Rik van Riel
Quoted 4 lines

After talking about my RFC patches to the c-state governor with Matthew and Arjan, it is clear that the whole concept of how things are done could use some more discussion.

Quoted 8 lines

Since a good number of us will be in San Diego next week, at Kernel Summit / Plumbers / etc, I will organize a c-state governor BOF for those who are interested.

Things to think about:
- what should the c-state governor do?
- how to best predict the future?
- what kinds of odd workloads do we need to accomodate?

Hi Rik,

Just notice there is a topic to discuss menu governor at Kernel Summit. Acutally, I have posted a patchset to at May 11 2012 to bring up the topic, at that time, I only have a convinced and proved application turbostat v1 to prove that my patch are useful. I try to find other workloads to prove that the patchset are also solidated useful. But I stucked in other high priority tasks, so I move slow on it.

Quoted 1 line

From you bring up the issue I guess that you already has real workload

to show this issue.
My patchset is not only improve repeat mode failure but also improve general prediction failure. Let's have a discuss and talk about it.

Here is the patchset posted at May 11 2012.

http://lwn.net/Articles/496919/ "x86,idle: Enhance cpuidle prediction to handle its failure"
http://lkml.indiana.edu/hypermail/linux/kernel/1205.1/02267.html
"[PATCH 1/3] x86,idle: Quickly notice prediction failure for repeat mode"
http://lkml.indiana.edu/hypermail/linux/kernel/1205.1/02268.html
"[PATCH 2/3] x86,idle: Quickly notice prediction failure in general case"
http://lkml.indiana.edu/hypermail/linux/kernel/1205.1/02269.html
"[PATCH 3/3] x86,idle: Set residency to 0 if target Cstate not really enter"

Thanks
-Youquan

#3 lore raw
Quoted 28 lines · Youquan Song
Quoted 4 lines

After talking about my RFC patches to the c-state governor with Matthew and Arjan, it is clear that the whole concept of how things are done could use some more discussion.

Quoted 8 lines

Since a good number of us will be in San Diego next week, at Kernel Summit / Plumbers / etc, I will organize a c-state governor BOF for those who are interested.

Things to think about:
- what should the c-state governor do?
- how to best predict the future?
- what kinds of odd workloads do we need to accomodate?

Hi Rik,

Just notice there is a topic to discuss menu governor at Kernel Summit. Acutally, I have posted a patchset to at May 11 2012 to bring up the topic, at that time, I only have a convinced and proved application turbostat v1 to prove that my patch are useful. I try to find other workloads to prove that the patchset are also solidated useful. But I stucked in other high priority tasks, so I move slow on it.
From you bring up the issue I guess that you already has real workload
to show this issue.
My patchset is not only improve repeat mode failure but also improve general prediction failure. Let's have a discuss and talk about it.

Here is the patchset posted at May 11 2012.

Quoted 9 lines

http://lwn.net/Articles/496919/ "x86,idle: Enhance cpuidle prediction to handle its failure"
http://lkml.indiana.edu/hypermail/linux/kernel/1205.1/02267.html
"[PATCH 1/3] x86,idle: Quickly notice prediction failure for repeat mode"
http://lkml.indiana.edu/hypermail/linux/kernel/1205.1/02268.html
"[PATCH 2/3] x86,idle: Quickly notice prediction failure in general case"
http://lkml.indiana.edu/hypermail/linux/kernel/1205.1/02269.html
"[PATCH 3/3] x86,idle: Set residency to 0 if target Cstate not really enter"

Your patches could make a lot of sense when integrated with my patches:

http://people.redhat.com/riel/cstate/

However, we should probably get the tracepoint upstream first, so we can know for sure :)

Signature · 1 line
All rights reversed
#4 lore raw
Quoted 6 lines

Your patches could make a lot of sense when integrated with my patches:

http://people.redhat.com/riel/cstate/
However, we should probably get the tracepoint upstream first, so we can know for sure :)

I can not access the patches at this directory. Can you send it to me? I will look at your patches and then integrated with my patches to look what will happen tomorrow.

Do you have test case share? or ideas how to show the benefit.

I have done many test for my pathes. It show some benefit big or small in various cases, but there is no negative effect showed at least.

I have two onviced test cases to show the great benefit
1. turbostat v1 (before 3.5)
2. I write the simple test application which also show greate benefit. running it by #./idle_predict -l 8

I write a simple application using usleep which it is clear to the repeat mode prediction failure will greatly effect the application with such repeat pattern.

-----------------------
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <signal.h>
#include <sys/time.h>
#include <time.h>
#include <pthread.h>

volatile int * shutdown;
volatile long * count;
int delay = 20;
int loop = 8;

void usage(void)

{
	fprintf(stderr,
		"Usage: idle_predict [options]\n"
		"  --help	-h  Print this help\n"
		"  --thread	-n  Thread number\n"
		"  --loop     	-l  Loop times in shallow Cstate\n"
		"  --delay	-t  Sleep time (uS)in shallow

Cstate\n");
}

void *simple_loop() {
	int idle_num = 1;
	while (*shutdown) {
		*count = *count + 1;
	
		if (idle_num % loop)
			usleep(delay);
		else {
			/* sleep 1 second */
			usleep(1000000);
			idle_num = 0;
		}
		idle_num++;
	}

}

static void sighand(int sig)

{
	*shutdown = 0;
}

int main(int argc, char *argv[])

{
	sigset_t sigset;
	int signum = SIGALRM;
	int i, c, er = 0, thread_num = 8;
	pthread_t pt[1024];

	static char optstr[] = "n:l:t:h:";

	while ((c = getopt(argc, argv, optstr)) != EOF)
		switch (c) {
			case 'n':
				thread_num = atoi(optarg);
				break;
			case 'l':
				loop = atoi(optarg);
				break;
			case 't':
				delay = atoi(optarg);
				break;
			case 'h':
			default:
				usage();
				exit(1);
		}

	printf("thread=%d,loop=%d,delay=%d\n",thread_num,loop,delay);
	count = malloc(sizeof(long));
	shutdown = malloc(sizeof(int));
	*count = 0;
	*shutdown = 1;

	sigemptyset(&sigset);
	sigaddset(&sigset, signum);
	sigprocmask (SIG_BLOCK, &sigset, NULL);
	signal(SIGINT, sighand);
	signal(SIGTERM, sighand);

	for(i = 0; i < thread_num ; i++)
		pthread_create(&pt[i], NULL, simple_loop, NULL);

	for (i = 0; i < thread_num; i++)
		pthread_join(pt[i], NULL);

	exit(0);
}
#5 lore raw
Quoted 10 lines · Youquan Song
Quoted 6 lines

Your patches could make a lot of sense when integrated with my patches:

http://people.redhat.com/riel/cstate/
However, we should probably get the tracepoint upstream first, so we can know for sure :)

I can not access the patches at this directory. Can you send it to me? I will look at your patches and then integrated with my patches to look what will happen tomorrow.

Argh, fixed!

Quoted 11 lines

Do you have test case share? or ideas how to show the benefit.

I have done many test for my pathes. It show some benefit big or small in various cases, but there is no negative effect showed at least.

I have two onviced test cases to show the great benefit
1. turbostat v1 (before 3.5)
2. I write the simple test application which also show greate benefit. running it by #./idle_predict -l 8

Quoted 3 lines

I write a simple application using usleep which it is clear to the repeat mode prediction failure will greatly effect the application with such repeat pattern.

Your test looks exactly like the kind of thing that could benefit from my changes to the menu governor :)

Signature · 1 line
All rights reversed