KVM: add option to advance tscdeadline hrtimer expiration (v5)

kvm_x86_ops->test_posted_interrupt() returns true/false depending whether 'vector' is set.

Next patch makes use of this interface.

Signed-off-by Marcelo Tosatti <[email protected]>

For the hrtimer which emulates the tscdeadline timer in the guest, add an option to advance expiration, and busy spin on VM-entry waiting for the actual expiration time to elapse.

This allows achieving low latencies in cyclictest (or any scenario which requires strict timing regarding timer expiration).

Reduces average cyclictest latency from 12us to 8us on Core i5 desktop.

Note: this option requires tuning to find the appropriate value for a particular hardware/guest combination. One method is to measure the average delay between apic_timer_fn and VM-entry.
Another method is to start with 1000ns, and increase the value in say 500ns increments until avg cyclictest numbers stop decreasing.

Signed-off-by Marcelo Tosatti <[email protected]>

+++ kvm/arch/x86/kvm/lapic.c

(quoted diff) +0 -0

@@ -1087,11 +1089,64 @@ static void apic_timer_expired(struct kv

+/*
+ * On APICv, this test will cause a busy wait
+ * during a higher-priority task.
+ */

(quoted diff) +2 -0

+
+static bool lapic_timer_int_injected(struct kvm_vcpu *vcpu)

arch/x86/kvm/lapic.c +2 -0

+void wait_lapic_expire(struct kvm_vcpu *vcpu)
+{

2014-12-16 09:08-0500, Marcelo Tosatti:

Quoted 10 lines

For the hrtimer which emulates the tscdeadline timer in the guest, add an option to advance expiration, and busy spin on VM-entry waiting for the actual expiration time to elapse.

This allows achieving low latencies in cyclictest (or any scenario which requires strict timing regarding timer expiration).

Reduces average cyclictest latency from 12us to 8us on Core i5 desktop.

Quoted 7 lines

Note: this option requires tuning to find the appropriate value for a particular hardware/guest combination. One method is to measure the average delay between apic_timer_fn and VM-entry.
Another method is to start with 1000ns, and increase the value in say 500ns increments until avg cyclictest numbers stop decreasing.

Signed-off-by Marcelo Tosatti <[email protected]>

Reviewed-by Radim Krčmář <[email protected]>

Quoted 2 lines

+++ kvm/arch/x86/kvm/lapic.c

(quoted diff) +0 -0

@@ -1087,11 +1089,64 @@ static void apic_timer_expired(struct kv

[...]

Quoted 4 lines

+/*
+ * On APICv, this test will cause a busy wait
+ * during a higher-priority task.
+ */

(A bit confusing ... this test doesn't busy wait.)

Quoted 2 lines

(quoted diff) +2 -0

+
+static bool lapic_timer_int_injected(struct kvm_vcpu *vcpu)

[...]

Quoted 2 lines

arch/x86/kvm/lapic.c +2 -0

+void wait_lapic_expire(struct kvm_vcpu *vcpu)
+{

[...]

arch/x86/kvm/lapic.c static void apic_timer_expired(struct kv · line 1139

Quoted 6 lines

arch/x86/kvm/lapic.c +6 -0

+	tsc_deadline = apic->lapic_timer.expired_tscdeadline;
+	apic->lapic_timer.expired_tscdeadline = 0;
+	guest_tsc = kvm_x86_ops->read_l1_tsc(vcpu, native_read_tsc());
+
+	while (guest_tsc < tsc_deadline) {
+		int delay = min(tsc_deadline - guest_tsc, 1000ULL);

Why break the __delay() loop into smaller parts?

arch/x86/kvm/lapic.c static void apic_timer_expired(struct kv · line 1141

Quoted 1 line

arch/x86/kvm/lapic.c +1 -0

+		__delay(delay);

(Does not have to call delay_tsc, but I guess it won't change.)

arch/x86/kvm/lapic.c static void apic_timer_expired(struct kv · line 1144

Quoted 4 lines

arch/x86/kvm/lapic.c +2 -0

+		guest_tsc = kvm_x86_ops->read_l1_tsc(vcpu, native_read_tsc());
+	}
 }

Btw. simple automatic delta tuning had worse results?

Quoted 1 line

Quoted 1 line

Btw. simple automatic delta tuning had worse results?

Haven't tried automatic tuning.

So what happens on a realtime environment is this: you execute the fixed number of instructions from interrupt handling all the way to VM-entry.

Well, almost fixed. Fixed is the number of apic_timer_fn plus KVM instructions. You can also execute host scheduler and timekeeping processing.

In practice, the length to execute that instruction sequence is a bell shaped normal distribution around the average (the right side is slightly higher due to host scheduler and timekeeping processing).

You want to advance the timer by the rightmost bucket, that way you guarantee lower possible latencies (which is the interest here).

That said, i don't see advantage in automatic tuning for the usecase which this targets.

Quoted 9 lines

2014-12-16 09:08-0500, Marcelo Tosatti:

arch/x86/kvm/lapic.c static void apic_timer_expired(struct kv · line 1139

Quoted 6 lines

arch/x86/kvm/lapic.c +6 -0

+	tsc_deadline = apic->lapic_timer.expired_tscdeadline;
+	apic->lapic_timer.expired_tscdeadline = 0;
+	guest_tsc = kvm_x86_ops->read_l1_tsc(vcpu, native_read_tsc());
+
+	while (guest_tsc < tsc_deadline) {
+		int delay = min(tsc_deadline - guest_tsc, 1000ULL);

Why break the __delay() loop into smaller parts?

So that you can handle interrupts, in case this code ever moves outside IRQ protected region.

Quoted 10 lines

arch/x86/kvm/lapic.c static void apic_timer_expired(struct kv · line 1141

Quoted 1 line

arch/x86/kvm/lapic.c +1 -0

+		__delay(delay);

(Does not have to call delay_tsc, but I guess it won't change.)

arch/x86/kvm/lapic.c static void apic_timer_expired(struct kv · line 1144

Quoted 4 lines

arch/x86/kvm/lapic.c +2 -0

+		guest_tsc = kvm_x86_ops->read_l1_tsc(vcpu, native_read_tsc());
+	}
 }

Btw. simple automatic delta tuning had worse results?

Haven't tried automatic tuning.

So what happens on a realtime environment is this: you execute the fixed number of instructions from interrupt handling all the way to VM-entry.

Well, almost fixed. Fixed is the number of apic_timer_fn plus KVM instructions. You can also execute host scheduler and timekeeping processing.

In practice, the length to execute that instruction sequence is a bell shaped normal distribution around the average (the right side is slightly higher due to host scheduler and timekeeping processing).

You want to advance the timer by the rightmost bucket, that way you guarantee lower possible latencies (which is the interest here).

That said, i don't see advantage in automatic tuning for the usecase which this targets.

2014-12-17 15:41-0200, Marcelo Tosatti:

Quoted 11 lines · Radim Krcmar

Quoted 9 lines

2014-12-16 09:08-0500, Marcelo Tosatti:

arch/x86/kvm/lapic.c static void apic_timer_expired(struct kv · line 1139

Quoted 6 lines

arch/x86/kvm/lapic.c +6 -0

+	tsc_deadline = apic->lapic_timer.expired_tscdeadline;
+	apic->lapic_timer.expired_tscdeadline = 0;
+	guest_tsc = kvm_x86_ops->read_l1_tsc(vcpu, native_read_tsc());
+
+	while (guest_tsc < tsc_deadline) {
+		int delay = min(tsc_deadline - guest_tsc, 1000ULL);

Why break the __delay() loop into smaller parts?

Quoted 2 lines

So that you can handle interrupts, in case this code ever moves outside IRQ protected region.

__delay() works only if it is delay_tsc(), which has this handled ... (It even considers rescheduling with unsynchronized TSC.)

delay_tsc(delay) translates roughly to

  end = read_tsc() + delay;
  while (read_tsc() < end);

so the code of our while loop has a structure like

  while ((guest_tsc = read_tsc()) < tsc_deadline) {
    end = read_tsc() + min(tsc_deadline - guest_tsc, 1000);
    while (read_tsc() < end);
  }

which complicates our original idea of

  while (read_tsc() < tsc_deadline);

(but I'm completely fine with it.)

Quoted 24 lines

Quoted 10 lines

arch/x86/kvm/lapic.c static void apic_timer_expired(struct kv · line 1141

Quoted 1 line

arch/x86/kvm/lapic.c +1 -0

+		__delay(delay);

(Does not have to call delay_tsc, but I guess it won't change.)

arch/x86/kvm/lapic.c static void apic_timer_expired(struct kv · line 1144

Quoted 4 lines

arch/x86/kvm/lapic.c +2 -0

+		guest_tsc = kvm_x86_ops->read_l1_tsc(vcpu, native_read_tsc());
+	}
 }

Btw. simple automatic delta tuning had worse results?

Haven't tried automatic tuning.

So what happens on a realtime environment is this: you execute the fixed number of instructions from interrupt handling all the way to VM-entry.

Well, almost fixed. Fixed is the number of apic_timer_fn plus KVM instructions. You can also execute host scheduler and timekeeping processing.

In practice, the length to execute that instruction sequence is a bell shaped normal distribution around the average (the right side is slightly higher due to host scheduler and timekeeping processing).

Quoted 2 lines

You want to advance the timer by the rightmost bucket, that way you guarantee lower possible latencies (which is the interest here).

(Lower latencies would likely be achieved by having a timer that issues
posted interrupts from another CPU, and the guest set to busy idle.)

Quoted 2 lines

That said, i don't see advantage in automatic tuning for the usecase which this targets.

Thanks, it doesn't make much difference in the long RT setup checklist.

---
I was asking just because I consider programming to equal automation ... If we know that we will always set this to the rightmost bucket anyway, it could be done like this

  if ((s64)(delta = guest_tsc - tsc_deadline) > 0)
    tsc_deadline_delta += delta;
  ...
  advance_ns = kvm_tsc_to_ns(tsc_deadline_delta);

instead of a script that runs a test and sets the variable. (On the other hand, it would probably have to be more complicated to
reach the same level of flexibility.)

True. I can change to a direct wait if that is preferred.

Add tracepoint to wait_lapic_expire.

Signed-off-by Marcelo Tosatti <[email protected]>

+++ kvm/arch/x86/kvm/lapic.c

arch/x86/kvm/lapic.c +1 -1

@@ -1120,7 +1120,7 @@ static bool lapic_timer_int_injected(str
 void wait_lapic_expire(struct kvm_vcpu *vcpu)
 {
 	struct kvm_lapic *apic = vcpu->arch.apic;
-	u64 guest_tsc, tsc_deadline;
+	u64 orig_guest_tsc, guest_tsc, tsc_deadline;
 
 	if (!kvm_vcpu_has_lapic(vcpu))
 		return;

arch/x86/kvm/lapic.c +2 -0

 	while (guest_tsc < tsc_deadline) {
 		int delay = min(tsc_deadline - guest_tsc, 1000ULL);

@@ -1141,6 +1141,8 @@ void wait_lapic_expire(struct kvm_vcpu *
 		__delay(delay);
 		guest_tsc = kvm_x86_ops->read_l1_tsc(vcpu, native_read_tsc());
 	}
+
+	trace_kvm_wait_lapic_expire(vcpu->vcpu_id, orig_guest_tsc - tsc_deadline);
 }