kvm: Improving undercommit,overcommit scenarios

From: Raghavendra K T <[email protected]>

Also we do not update last boosted vcpu in failure cases.

#endif

virt/kvm/kvm_main.c +1 -0

+
 void kvm_vcpu_on_spin(struct kvm_vcpu *me)
 {
 	struct kvm *kvm = me->kvm;

@@ -1727,11 +1727,12 @@ void kvm_vcpu_on_spin(struct kvm_vcpu *me)
 				continue;
 			if (!kvm_vcpu_eligible_for_directed_yield(vcpu))
 				continue;

Quoted 17 lines · Raghavendra K T

From: Raghavendra K T <[email protected]>

Also we do not update last boosted vcpu in failure cases.

  #endif

virt/kvm/kvm_main.c +4 -1

+
  void kvm_vcpu_on_spin(struct kvm_vcpu *me)
  {
  	struct kvm *kvm = me->kvm;

@@ -1727,11 +1727,12 @@ void kvm_vcpu_on_spin(struct kvm_vcpu *me)
  				continue;
  			if (!kvm_vcpu_eligible_for_directed_yield(vcpu))
  				continue;
-			if (kvm_vcpu_yield_to(vcpu)) {
+
+			yielded = kvm_vcpu_yield_to(vcpu);
+			if (yielded > 0)

virt/kvm/kvm_main.c void kvm_vcpu_on_spin(struct kvm_vcpu *me) · line 1736

Quoted 6 lines

virt/kvm/kvm_main.c +1 -2

  				kvm->last_boosted_vcpu = i;
-				yielded = 1;
+			if (yielded)
  				break;
-			}
  		}

If yielded == -ESRCH, should we not try to yield to another vcpu?

Quoted 18 lines · Avi Kivity

Quoted 17 lines · Raghavendra K T

From: Raghavendra K T <[email protected]>

Also we do not update last boosted vcpu in failure cases.

  #endif

virt/kvm/kvm_main.c +4 -1

+
  void kvm_vcpu_on_spin(struct kvm_vcpu *me)
  {
      struct kvm *kvm = me->kvm;

@@ -1727,11 +1727,12 @@ void kvm_vcpu_on_spin(struct kvm_vcpu *me)
                  continue;
              if (!kvm_vcpu_eligible_for_directed_yield(vcpu))
                  continue;
-            if (kvm_vcpu_yield_to(vcpu)) {
+
+            yielded = kvm_vcpu_yield_to(vcpu);
+            if (yielded > 0)

Quoted 9 lines

virt/kvm/kvm_main.c void kvm_vcpu_on_spin(struct kvm_vcpu *me) · line 1736

Quoted 6 lines

virt/kvm/kvm_main.c +1 -2

                  kvm->last_boosted_vcpu = i;
-                yielded = 1;
+            if (yielded)
                  break;
-            }
          }

If yielded == -ESRCH, should we not try to yield to another vcpu?

Yes. plan is to abort the iteration. since it means we are mostly
undercommitted.

Quoted 29 lines · Raghavendra K T

Quoted 27 lines · Avi Kivity

Quoted 13 lines · Raghavendra K T

From: Raghavendra K T <[email protected]>

Also we do not update last boosted vcpu in failure cases.

  #endif

virt/kvm/kvm_main.c +1 -0

+
  void kvm_vcpu_on_spin(struct kvm_vcpu *me)
  {
      struct kvm *kvm = me->kvm;

@@ -1727,11 +1727,12 @@ void kvm_vcpu_on_spin(struct kvm_vcpu *me)
                  continue;
              if (!kvm_vcpu_eligible_for_directed_yield(vcpu))
                  continue;

virt/kvm/kvm_main.c void kvm_vcpu_on_spin(struct kvm_vcpu *me) · line 1736

Quoted 10 lines

virt/kvm/kvm_main.c +4 -3

-            if (kvm_vcpu_yield_to(vcpu)) {
+
+            yielded = kvm_vcpu_yield_to(vcpu);
+            if (yielded > 0)
                  kvm->last_boosted_vcpu = i;
-                yielded = 1;
+            if (yielded)
                  break;
-            }
          }

If yielded == -ESRCH, should we not try to yield to another vcpu?

Quoted 2 lines

Yes. plan is to abort the iteration. since it means we are mostly
undercommitted.

Sorry if it was ambiguous. I wanted to say we do not want to continue yield to another vcpu..

Quoted 33 lines · Raghavendra K T

Quoted 31 lines · Raghavendra K T

Quoted 18 lines · Avi Kivity

Quoted 17 lines · Raghavendra K T

From: Raghavendra K T <[email protected]>

Also we do not update last boosted vcpu in failure cases.

   #endif

virt/kvm/kvm_main.c +4 -1

+
   void kvm_vcpu_on_spin(struct kvm_vcpu *me)
   {
       struct kvm *kvm = me->kvm;

@@ -1727,11 +1727,12 @@ void kvm_vcpu_on_spin(struct kvm_vcpu *me)
                   continue;
               if (!kvm_vcpu_eligible_for_directed_yield(vcpu))
                   continue;
-            if (kvm_vcpu_yield_to(vcpu)) {
+
+            yielded = kvm_vcpu_yield_to(vcpu);
+            if (yielded > 0)

Quoted 9 lines

virt/kvm/kvm_main.c void kvm_vcpu_on_spin(struct kvm_vcpu *me) · line 1736

Quoted 6 lines

virt/kvm/kvm_main.c +1 -2

                   kvm->last_boosted_vcpu = i;
-                yielded = 1;
+            if (yielded)
                   break;
-            }
           }

If yielded == -ESRCH, should we not try to yield to another vcpu?

Yes. plan is to abort the iteration. since it means we are mostly
undercommitted.

Quoted 3 lines

Sorry if it was ambiguous. I wanted to say we do not want to continue yield to another vcpu..

Why not? We found that this particular vcpu is running and therefore likely not a lock holder. That says nothing about other vcpus. The next in line might be runnable-but-not-running on another runqueue.

Quoted 34 lines · Avi Kivity

Quoted 33 lines · Raghavendra K T

Quoted 31 lines · Raghavendra K T

Quoted 18 lines · Avi Kivity

Quoted 17 lines · Raghavendra K T

From: Raghavendra K T <[email protected]>

Also we do not update last boosted vcpu in failure cases.

  #endif

virt/kvm/kvm_main.c +4 -1

+
  void kvm_vcpu_on_spin(struct kvm_vcpu *me)
  {
      struct kvm *kvm = me->kvm;

@@ -1727,11 +1727,12 @@ void kvm_vcpu_on_spin(struct kvm_vcpu *me)
                  continue;
              if (!kvm_vcpu_eligible_for_directed_yield(vcpu))
                  continue;
-            if (kvm_vcpu_yield_to(vcpu)) {
+
+            yielded = kvm_vcpu_yield_to(vcpu);
+            if (yielded > 0)

Quoted 9 lines

virt/kvm/kvm_main.c void kvm_vcpu_on_spin(struct kvm_vcpu *me) · line 1736

Quoted 6 lines

virt/kvm/kvm_main.c +1 -2

                  kvm->last_boosted_vcpu = i;
-                yielded = 1;
+            if (yielded)
                  break;
-            }
          }

If yielded == -ESRCH, should we not try to yield to another vcpu?

Yes. plan is to abort the iteration. since it means we are mostly
undercommitted.

Quoted 8 lines

Quoted 3 lines

Sorry if it was ambiguous. I wanted to say we do not want to continue yield to another vcpu..

Why not? We found that this particular vcpu is running and therefore likely not a lock holder. That says nothing about other vcpus. The next in line might be runnable-but-not-running on another runqueue.

Agree that next in the line might be runnable-not-running. But here we are optimistic that, that is not the case and we save time by returning back instead of iterating, thinking we are mostly in undercommitted case and each vcpu has dedicated cpu.

Probably an alternative we have here is to look for say 2-3 successive failures before breaking out?

virt/kvm/kvm_main.c +8 -0

+/*
+ * A load of 2048 corresponds to 1:1 overcommit
+ * undercommit threshold is half the 1:1 overcommit
+ * overcommit threshold is 1.75 times of 1:1 overcommit threshold
+ */
+#define COMMIT_THRESHOLD (FIXED_1)
+#define UNDERCOMMIT_THRESHOLD (COMMIT_THRESHOLD >> 1)
+#define OVERCOMMIT_THRESHOLD ((COMMIT_THRESHOLD << 1) -

(COMMIT_THRESHOLD >> 2))

Quoted 9 lines · Raghavendra K T

virt/kvm/kvm_main.c +8 -0

+/*
+ * A load of 2048 corresponds to 1:1 overcommit
+ * undercommit threshold is half the 1:1 overcommit
+ * overcommit threshold is 1.75 times of 1:1 overcommit threshold
+ */
+#define COMMIT_THRESHOLD (FIXED_1)
+#define UNDERCOMMIT_THRESHOLD (COMMIT_THRESHOLD >> 1)
+#define OVERCOMMIT_THRESHOLD ((COMMIT_THRESHOLD << 1) -

(COMMIT_THRESHOLD >> 2))

virt/kvm/kvm_main.c bool kvm_vcpu_eligible_for_directed_yield(struct kvm_vcpu *vcpu) · line 1717

Quoted 10 lines

virt/kvm/kvm_main.c +10 -0

+
+unsigned long kvm_system_load(void)
+{
+       unsigned long load;
+
+       load = avenrun[0] + FIXED_1/200;
+       load = load / num_online_cpus();
+
+       return load;
+}

ARGH.. no that's wrong.. very wrong.

1) avenrun[] EXPORT_SYMBOL says it should be removed, that's not a
joke.

2) avenrun[] is a global load, do not ever use a global load measure

3) avenrun[] has nothing what so ever to do with runqueue lengths,
someone with a gazillion tasks in D state will get a huge load but the cpu is very idle.

overcommit_threshold => 4k window.

Quoted 23 lines · Peter Zijlstra

Quoted 9 lines · Raghavendra K T

virt/kvm/kvm_main.c +8 -0

+/*
+ * A load of 2048 corresponds to 1:1 overcommit
+ * undercommit threshold is half the 1:1 overcommit
+ * overcommit threshold is 1.75 times of 1:1 overcommit threshold
+ */
+#define COMMIT_THRESHOLD (FIXED_1)
+#define UNDERCOMMIT_THRESHOLD (COMMIT_THRESHOLD >> 1)
+#define OVERCOMMIT_THRESHOLD ((COMMIT_THRESHOLD << 1) -

(COMMIT_THRESHOLD >> 2))

virt/kvm/kvm_main.c bool kvm_vcpu_eligible_for_directed_yield(struct kvm_vcpu *vcpu) · line 1717

Quoted 10 lines

virt/kvm/kvm_main.c +10 -0

+
+unsigned long kvm_system_load(void)
+{
+       unsigned long load;
+
+       load = avenrun[0] + FIXED_1/200;
+       load = load / num_online_cpus();
+
+       return load;
+}

ARGH.. no that's wrong.. very wrong.

Quoted 2 lines

1) avenrun[] EXPORT_SYMBOL says it should be removed, that's not a
joke.

Okay.

Quoted 1 line

2) avenrun[] is a global load, do not ever use a global load measure

This makes sense. Using a local optimization that leads to near global optimization is the way to go.

Quoted 5 lines

3) avenrun[] has nothing what so ever to do with runqueue lengths,
someone with a gazillion tasks in D state will get a huge load but the cpu is very idle.

I used loadavg as an alternative measure. But the above condition poses a concern for that.

Okay, now IIUC, usage of *any* global measure is bad?

Because I was also thinking to use nrrunning()/ num_online_cpus(), to get an idea of global overcommit sense. (ofcourse since, this involves iteration over per CPU nrrunning, I wanted to calculate this periodically)

The overall logic, of having overcommit_threshold, undercommit_threshold, I wanted to use for even dynamic ple_window tuning purpose.

so logic was:
< undercommit_threshold => 16k ple_window

Quoted 1 line

overcommit_threshold => 4k window.

for in between case scale the ple_window accordingly.

The alternative was to decide depending on how ple handler succeeded in yield_to. But I thought, that is too sensitive and more overhead.

This topic may deserve different thread, but thought I shall table it here.

So, Thinking about the alternatives to implement, logic such as

(a) if(undercommitted)
just go back and spin rather than going for yield_to iteration.
(b) if (overcommitted)
better to yield rather than spinning logic

of current patches..

[ ofcourse, (a) is already met to large extent by your patches..]

So I think everything boils down to

"how do we measure these two thresholds without much overhead in a compliant way"

Ideas welcome..

Quoted 80 lines · Raghavendra K T

Quoted 23 lines · Peter Zijlstra

Quoted 9 lines · Raghavendra K T

virt/kvm/kvm_main.c +8 -0

+/*
+ * A load of 2048 corresponds to 1:1 overcommit
+ * undercommit threshold is half the 1:1 overcommit
+ * overcommit threshold is 1.75 times of 1:1 overcommit threshold
+ */
+#define COMMIT_THRESHOLD (FIXED_1)
+#define UNDERCOMMIT_THRESHOLD (COMMIT_THRESHOLD >> 1)
+#define OVERCOMMIT_THRESHOLD ((COMMIT_THRESHOLD << 1) -

(COMMIT_THRESHOLD >> 2))

virt/kvm/kvm_main.c bool kvm_vcpu_eligible_for_directed_yield(struct kvm_vcpu *vcpu) · line 1717

Quoted 10 lines

virt/kvm/kvm_main.c +10 -0

+
+unsigned long kvm_system_load(void)
+{
+       unsigned long load;
+
+       load = avenrun[0] + FIXED_1/200;
+       load = load / num_online_cpus();
+
+       return load;
+}

ARGH.. no that's wrong.. very wrong.

Quoted 2 lines

1) avenrun[] EXPORT_SYMBOL says it should be removed, that's not a
joke.

Okay.

Quoted 1 line

2) avenrun[] is a global load, do not ever use a global load measure

This makes sense. Using a local optimization that leads to near global optimization is the way to go.

Quoted 5 lines

3) avenrun[] has nothing what so ever to do with runqueue lengths,
someone with a gazillion tasks in D state will get a huge load but the cpu is very idle.

I used loadavg as an alternative measure. But the above condition poses a concern for that.

Okay, now IIUC, usage of *any* global measure is bad?

Because I was also thinking to use nrrunning()/ num_online_cpus(), to get an idea of global overcommit sense. (ofcourse since, this involves iteration over per CPU nrrunning, I wanted to calculate this periodically)

The overall logic, of having overcommit_threshold, undercommit_threshold, I wanted to use for even dynamic ple_window tuning purpose.

so logic was:
< undercommit_threshold => 16k ple_window

Quoted 1 line

overcommit_threshold => 4k window.

for in between case scale the ple_window accordingly.

The alternative was to decide depending on how ple handler succeeded in yield_to. But I thought, that is too sensitive and more overhead.

This topic may deserve different thread, but thought I shall table it here.

So, Thinking about the alternatives to implement, logic such as

(a) if(undercommitted)
just go back and spin rather than going for yield_to iteration.
(b) if (overcommitted)
better to yield rather than spinning logic

of current patches..

[ ofcourse, (a) is already met to large extent by your patches..]

So I think everything boils down to

"how do we measure these two thresholds without much overhead in a compliant way"

Quoted 2 lines

Ideas welcome..

What happened to Avi's preempt notifier idea for determining under/overcommit? If nobody has picked that up yet, then I'll go ahead and try to prototype it.

Quoted 81 lines · Andrew Jones

Quoted 80 lines · Raghavendra K T

Quoted 23 lines · Peter Zijlstra

Quoted 9 lines · Raghavendra K T

virt/kvm/kvm_main.c +8 -0

+/*
+ * A load of 2048 corresponds to 1:1 overcommit
+ * undercommit threshold is half the 1:1 overcommit
+ * overcommit threshold is 1.75 times of 1:1 overcommit threshold
+ */
+#define COMMIT_THRESHOLD (FIXED_1)
+#define UNDERCOMMIT_THRESHOLD (COMMIT_THRESHOLD >> 1)
+#define OVERCOMMIT_THRESHOLD ((COMMIT_THRESHOLD << 1) -

(COMMIT_THRESHOLD >> 2))

virt/kvm/kvm_main.c bool kvm_vcpu_eligible_for_directed_yield(struct kvm_vcpu *vcpu) · line 1717

Quoted 10 lines

virt/kvm/kvm_main.c +10 -0

+
+unsigned long kvm_system_load(void)
+{
+       unsigned long load;
+
+       load = avenrun[0] + FIXED_1/200;
+       load = load / num_online_cpus();
+
+       return load;
+}

ARGH.. no that's wrong.. very wrong.

Quoted 2 lines

1) avenrun[] EXPORT_SYMBOL says it should be removed, that's not a
joke.

Okay.

Quoted 1 line

2) avenrun[] is a global load, do not ever use a global load measure

This makes sense. Using a local optimization that leads to near global optimization is the way to go.

Quoted 5 lines

3) avenrun[] has nothing what so ever to do with runqueue lengths,
someone with a gazillion tasks in D state will get a huge load but the cpu is very idle.

I used loadavg as an alternative measure. But the above condition poses a concern for that.

Okay, now IIUC, usage of *any* global measure is bad?

Because I was also thinking to use nrrunning()/ num_online_cpus(), to get an idea of global overcommit sense. (ofcourse since, this involves iteration over per CPU nrrunning, I wanted to calculate this periodically)

The overall logic, of having overcommit_threshold, undercommit_threshold, I wanted to use for even dynamic ple_window tuning purpose.

so logic was:
< undercommit_threshold => 16k ple_window

Quoted 1 line

overcommit_threshold => 4k window.

for in between case scale the ple_window accordingly.

The alternative was to decide depending on how ple handler succeeded in yield_to. But I thought, that is too sensitive and more overhead.

This topic may deserve different thread, but thought I shall table it here.

So, Thinking about the alternatives to implement, logic such as

(a) if(undercommitted)
just go back and spin rather than going for yield_to iteration.
(b) if (overcommitted)
better to yield rather than spinning logic

of current patches..

[ ofcourse, (a) is already met to large extent by your patches..]

So I think everything boils down to

"how do we measure these two thresholds without much overhead in a compliant way"

Quoted 6 lines

Quoted 2 lines

Ideas welcome..

What happened to Avi's preempt notifier idea for determining under/overcommit? If nobody has picked that up yet, then I'll go ahead and try to prototype it.

Hi Drew,

I had assumed my priority order as
1) this patch series 2) dynamic ple window 3) preempt notifiers.

But I do not have any problem on re-prioritizing / helping on these as far as we are clear on what we are looking into.

I was thinking about preempt notifier idea as a tool to refine candidate VCPUs. But you are right, Avi, also told we can use bitmap/counter itself as an indicator to decide whether we go ahead with yield_to at all.

IMO, only patch(3) has some conflict because of various approach we can try.May be we should attack the problem via all 3 solutions at once and decide?

To be frank, within each of the approach, trying/analyzing all the
possibilities made the things slow.. (my end).

Suggestions..?

Quoted 109 lines · Raghavendra K T

Quoted 81 lines · Andrew Jones

Quoted 80 lines · Raghavendra K T

Quoted 23 lines · Peter Zijlstra

Quoted 9 lines · Raghavendra K T

virt/kvm/kvm_main.c +8 -0

+/*
+ * A load of 2048 corresponds to 1:1 overcommit
+ * undercommit threshold is half the 1:1 overcommit
+ * overcommit threshold is 1.75 times of 1:1 overcommit threshold
+ */
+#define COMMIT_THRESHOLD (FIXED_1)
+#define UNDERCOMMIT_THRESHOLD (COMMIT_THRESHOLD >> 1)
+#define OVERCOMMIT_THRESHOLD ((COMMIT_THRESHOLD << 1) -

(COMMIT_THRESHOLD >> 2))

virt/kvm/kvm_main.c bool kvm_vcpu_eligible_for_directed_yield(struct kvm_vcpu *vcpu) · line 1717

Quoted 10 lines

virt/kvm/kvm_main.c +10 -0

+
+unsigned long kvm_system_load(void)
+{
+       unsigned long load;
+
+       load = avenrun[0] + FIXED_1/200;
+       load = load / num_online_cpus();
+
+       return load;
+}

ARGH.. no that's wrong.. very wrong.

Quoted 2 lines

1) avenrun[] EXPORT_SYMBOL says it should be removed, that's not a
joke.

Okay.

Quoted 1 line

2) avenrun[] is a global load, do not ever use a global load measure

This makes sense. Using a local optimization that leads to near global optimization is the way to go.

Quoted 5 lines

3) avenrun[] has nothing what so ever to do with runqueue lengths,
someone with a gazillion tasks in D state will get a huge load but the cpu is very idle.

I used loadavg as an alternative measure. But the above condition poses a concern for that.

Okay, now IIUC, usage of *any* global measure is bad?

Because I was also thinking to use nrrunning()/ num_online_cpus(), to get an idea of global overcommit sense. (ofcourse since, this involves iteration over per CPU nrrunning, I wanted to calculate this periodically)

The overall logic, of having overcommit_threshold, undercommit_threshold, I wanted to use for even dynamic ple_window tuning purpose.

so logic was:
< undercommit_threshold => 16k ple_window

Quoted 1 line

overcommit_threshold => 4k window.

for in between case scale the ple_window accordingly.

The alternative was to decide depending on how ple handler succeeded in yield_to. But I thought, that is too sensitive and more overhead.

This topic may deserve different thread, but thought I shall table it here.

So, Thinking about the alternatives to implement, logic such as

(a) if(undercommitted)
just go back and spin rather than going for yield_to iteration.
(b) if (overcommitted)
better to yield rather than spinning logic

of current patches..

[ ofcourse, (a) is already met to large extent by your patches..]

So I think everything boils down to

"how do we measure these two thresholds without much overhead in a compliant way"

Quoted 6 lines

Quoted 2 lines

Ideas welcome..

What happened to Avi's preempt notifier idea for determining under/overcommit? If nobody has picked that up yet, then I'll go ahead and try to prototype it.

Hi Drew,

I had assumed my priority order as
1) this patch series 2) dynamic ple window 3) preempt notifiers.

But I do not have any problem on re-prioritizing / helping on these as far as we are clear on what we are looking into.

I was thinking about preempt notifier idea as a tool to refine candidate VCPUs. But you are right, Avi, also told we can use bitmap/counter itself as an indicator to decide whether we go ahead with yield_to at all.

IMO, only patch(3) has some conflict because of various approach we can try.May be we should attack the problem via all 3 solutions at once and decide?

To be frank, within each of the approach, trying/analyzing all the
possibilities made the things slow.. (my end).

Quoted 2 lines

Suggestions..?

I agree, it's a complex problem that needs lots of trial+error work. We should definitely work in parallel on multiple ideas. I'll go ahead and dig into the preempt notifiers.

Okay, now IIUC, usage of *any* global measure is bad?

Quoted 1 line · Raghavendra K T

Okay, now IIUC, usage of *any* global measure is bad?

Yep, people like to carve up their machines, esp. now that they're somewhat bigger than they used to be. This can result in very asymmetric loads, no global measure can ever deal with that.

In some special scenarios like #vcpu <= #pcpu, PLE handler may prove very costly, because there is no need to iterate over vcpus and do unsuccessful yield_to burning CPU.

Similarly, when we have large number of small guests, it is
possible that a spinning vcpu fails to yield_to any vcpu of same VM and go back and spin. This is also not effective when we are over-committed. Instead, we do a yield() so that we give chance to other VMs to run.

This patch tries to optimize above scenarios.

The first patch optimizes all the yield_to by bailing out when there is no need to continue yield_to (i.e., when there is only one task in source and target rq).

Second patch uses that in PLE handler.

Third patch uses overall system load knowledge to take decison on continuing in yield_to handler, and also yielding in overcommits. To be precise,
* loadavg is converted to a scale of 2048 / per CPU
* a load value of less than 1024 is considered as undercommit and we return from PLE handler in those cases
* a load value of greater than 3586 (1.75 * 2048) is considered as overcommit and we yield to other VMs in such cases.

(let threshold = 2048)
Rationale for using threshold/2 for undercommit limit:
Having a load below (0.5 * threshold) is used to avoid (the concern rasied by Rik)
scenarios where we still have lock holder preempted vcpu waiting to be scheduled. (scenario arises when rq length is > 1 even when we are under committed)

Rationale for using (1.75 * threshold) for overcommit scenario:
This is a heuristic where we should probably see rq length > 1 and a vcpu of a different VM is waiting to be scheduled.

Related future work (independent of this series):

- Dynamically changing PLE window depending on system load.

Result on 3.7.0-rc1 kernel shows around 146% improvement for ebizzy 1x with 32 core PLE machine with 32 vcpu guest.
I believe we should get very good improvements for overcommit (especially > 2) on large machines with small vcpu guests. (Could not test this as I do not have access to a bigger machine)

base = 3.7.0-rc1
machine: 32 core mx3850 x5 PLE mc

--+-----------+-----------+-----------+------------+-----------+
ebizzy (rec/sec higher is beter)
--+-----------+-----------+-----------+------------+-----------+
base stdev patched stdev %improve
--+-----------+-----------+-----------+------------+-----------+

1x  2543.3750    20.2903    6279.3750    82.5226   146.89143   
2x  2410.8750    96.4327    2450.7500   207.8136     1.65396
3x  2184.9167   205.5226    2178.3333    97.2034    -0.30131

--+-----------+-----------+-----------+------------+-----------+

--+-----------+-----------+-----------+------------+-----------+
dbench (throughput in MB/sec. higher is better)
--+-----------+-----------+-----------+------------+-----------+
base stdev patched stdev %improve
--+-----------+-----------+-----------+------------+-----------+

1x  5545.4330   596.4344    7042.8510  1012.0924    27.00272
2x  1993.0970    43.6548    1990.6200    75.7837    -0.12428
3x  1295.3867    22.3997    1315.5208    36.0075     1.55429

--+-----------+-----------+-----------+------------+-----------+

Changes since V1:
- Discard the idea of exporting nrrunning and optimize in core scheduler (Peter)
- Use yield() instead of schedule in overcommit scenarios (Rik)
- Use loadavg knowledge to detect undercommit/overcommit

Peter Zijlstra (1):
Bail out of yield_to when source and target runqueue has one task

Raghavendra K T (2):
Handle yield_to failure return for potential undercommit case Check system load and handle different commit cases accordingly

Please let me know your comments and suggestions.

Link for V1:
https://lkml.org/lkml/2012/9/21/168

 kernel/sched/core.c | 25 +++++++++++++++++++------
 virt/kvm/kvm_main.c | 56 ++++++++++++++++++++++++++++++++++++++++++++++----------
 2 files changed, 65 insertions(+), 16 deletions(-)

Quoted 66 lines · Raghavendra K T

In some special scenarios like #vcpu <= #pcpu, PLE handler may prove very costly, because there is no need to iterate over vcpus and do unsuccessful yield_to burning CPU.

Similarly, when we have large number of small guests, it is
possible that a spinning vcpu fails to yield_to any vcpu of same VM and go back and spin. This is also not effective when we are over-committed. Instead, we do a yield() so that we give chance to other VMs to run.

This patch tries to optimize above scenarios.

The first patch optimizes all the yield_to by bailing out when there is no need to continue yield_to (i.e., when there is only one task in source and target rq).

Second patch uses that in PLE handler.

Third patch uses overall system load knowledge to take decison on continuing in yield_to handler, and also yielding in overcommits. To be precise,
* loadavg is converted to a scale of 2048 / per CPU
* a load value of less than 1024 is considered as undercommit and we return from PLE handler in those cases
* a load value of greater than 3586 (1.75 * 2048) is considered as overcommit and we yield to other VMs in such cases.

(let threshold = 2048)
Rationale for using threshold/2 for undercommit limit:
Having a load below (0.5 * threshold) is used to avoid (the concern rasied by Rik)
scenarios where we still have lock holder preempted vcpu waiting to be scheduled. (scenario arises when rq length is > 1 even when we are under committed)

Rationale for using (1.75 * threshold) for overcommit scenario:
This is a heuristic where we should probably see rq length > 1 and a vcpu of a different VM is waiting to be scheduled.

Related future work (independent of this series):

- Dynamically changing PLE window depending on system load.

Result on 3.7.0-rc1 kernel shows around 146% improvement for ebizzy 1x with 32 core PLE machine with 32 vcpu guest.
I believe we should get very good improvements for overcommit (especially > 2) on large machines with small vcpu guests. (Could not test this as I do not have access to a bigger machine)

base = 3.7.0-rc1
machine: 32 core mx3850 x5 PLE mc

--+-----------+-----------+-----------+------------+-----------+
ebizzy (rec/sec higher is beter)
--+-----------+-----------+-----------+------------+-----------+
base stdev patched stdev %improve
--+-----------+-----------+-----------+------------+-----------+

1x  2543.3750    20.2903    6279.3750    82.5226   146.89143
2x  2410.8750    96.4327    2450.7500   207.8136     1.65396
3x  2184.9167   205.5226    2178.3333    97.2034    -0.30131

--+-----------+-----------+-----------+------------+-----------+

--+-----------+-----------+-----------+------------+-----------+
dbench (throughput in MB/sec. higher is better)
--+-----------+-----------+-----------+------------+-----------+
base stdev patched stdev %improve
--+-----------+-----------+-----------+------------+-----------+

Quoted 4 lines

1x  5545.4330   596.4344    7042.8510  1012.0924    27.00272
2x  1993.0970    43.6548    1990.6200    75.7837    -0.12428
3x  1295.3867    22.3997    1315.5208    36.0075     1.55429

--+-----------+-----------+-----------+------------+-----------+

Could you include a PLE-off result for 1x over-commit, so we know what the best possible result is?

Looks like skipping the yield_to() for rq = 1 helps, but I'd like to know if the performance is the same as PLE off for 1x. I am concerned the vcpu to task lookup is still expensive.

Based on Peter's comments I would say the 3rd patch and the 2x,3x results are not conclusive at this time.

I think we should also discuss what we think a good target is. We should know what our high-water mark is, and IMO, if we cannot get close, then I do not feel we are heading down the right path. For example, if dbench aggregate throughput for 1x with PLE off is 10000 MB/sec, then the best possible 2x,3x result, should be a little lower than that due to task switching the vcpus and sharing chaches. This should be quite evident with current PLE handler and smaller VMs (like 10 vcpus or less).