Skip to contentSkip to conversation

gettimeofday() vsyscall for kvm-clock?

DISCUSSION Andrew Theurer 2 participants 3 messages May 21, 2012

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

Wondering if a user-space gettimofday() for kvm-clock has been considered before. I am seeing a pretty large difference in performance between tsc and kvm-clock. I have to assume at least some of this is due to the mode switch for kvm-clock. Here are the results:

(this is a 16 vCPU VM on a 16 thread 2S Nehalem-EP host, looped gettimeofday() calls on all vCPUs)

tsc: .0645 usec per call
kvm-clock: .4222 usec per call (6.54x)

-Andrew Theurer

#1 lore raw
Quoted 6 lines · Andrew Theurer

Wondering if a user-space gettimofday() for kvm-clock has been considered before. I am seeing a pretty large difference in performance between tsc and kvm-clock. I have to assume at least some of this is due to the mode switch for kvm-clock. Here are the results:

Quoted 8 lines

(this is a 16 vCPU VM on a 16 thread 2S Nehalem-EP host, looped gettimeofday() calls on all vCPUs)

tsc: .0645 usec per call
kvm-clock: .4222 usec per call (6.54x)

-Andrew Theurer

https://bugzilla.redhat.com/show_bug.cgi?id=679207

"model name : Intel(R) Xeon(R) CPU E5540 @ 2.53GHz

native, gettimeofday (vsyscall): 45ns
guest, kvmclock (syscall): 198ns"

But this was before

commit 489fb490dbf8dab0249ad82b56688ae3842a79e8

Author: Glauber Costa <[email protected]>
Date:   Tue May 11 12:17:40 2010 -0400

x86, paravirt: Add a global synchronization point for pvclock

(see the full changelog for details).

Can you try disabling the global variable, to see if that makes a difference (should not be enabled in production)? Untested patch (against guest kernel) below

arch/x86/kernel/kvmclock.c +1 -1
index 086eb58..7347a31 100644
@@ -235,6 +235,6 @@ void __init kvmclock_init(void)
 	pv_info.paravirt_enabled = 1;
 	pv_info.name = "KVM";
 
-	if (kvm_para_has_feature(KVM_FEATURE_CLOCKSOURCE_STABLE_BIT))
+	//if (kvm_para_has_feature(KVM_FEATURE_CLOCKSOURCE_STABLE_BIT))
 		pvclock_set_flags(PVCLOCK_TSC_STABLE_BIT);
 }

Also, how is UP gettimeofday performance?

#2 lore raw
Quoted 29 lines · Marcelo Tosatti
Quoted 6 lines · Andrew Theurer

Wondering if a user-space gettimofday() for kvm-clock has been considered before. I am seeing a pretty large difference in performance between tsc and kvm-clock. I have to assume at least some of this is due to the mode switch for kvm-clock. Here are the results:

Quoted 8 lines

(this is a 16 vCPU VM on a 16 thread 2S Nehalem-EP host, looped gettimeofday() calls on all vCPUs)

tsc: .0645 usec per call
kvm-clock: .4222 usec per call (6.54x)

-Andrew Theurer

https://bugzilla.redhat.com/show_bug.cgi?id=679207

"model name : Intel(R) Xeon(R) CPU E5540 @ 2.53GHz

native, gettimeofday (vsyscall): 45ns
guest, kvmclock (syscall): 198ns"

But this was before

commit 489fb490dbf8dab0249ad82b56688ae3842a79e8

Author: Glauber Costa<[email protected]>
Date:   Tue May 11 12:17:40 2010 -0400
Quoted 7 lines

x86, paravirt: Add a global synchronization point for pvclock

(see the full changelog for details).

Can you try disabling the global variable, to see if that makes a difference (should not be enabled in production)? Untested patch (against guest kernel) below

The following was re-done on a 3.4 guest kernel (previously RHEL kernel):

1-way:
tsc: .0315
kvm-clock: .2112 (6.7x)

16-way:
tsc: .0432
kvm-clock: .4825 (11.1x)

Now with global var disabled:

16-way:
kvm-clock: .4628

Does not look like much of a difference.

-Andrew