xen-time: decreasing the rating of the xen clocksource below that of the tsc clocksource for dom0's
Subsystems (2)
RFC PATCH 1/1 xen-time: decreasing the rating of the xen clocksource below that of the tsc clocksource for dom0's
From: "Palik, Imre" <[email protected]>
In Dom0's the use of the TSC clocksource (whenever it is stable enough to be used) instead of the Xen clocksource should not cause any issues, as Dom0 VMs never live-migrated. The TSC clocksource is somewhat more efficient than the Xen paravirtualised clocksource, thus it should have higher rating.
--- arch/x86/xen/time.c | 4 ++++ 1 file changed, 4 insertions(+)
arch/x86/xen/time.c +4 -0
@@ -487,6 +487,10 @@ static void __init xen_time_init(void)int cpu =smp_processor_id ();struct timespec tp; +/* As Dom0 is never moved, no penalty on using TSC there */ +if (xen_initial_domain ()) + xen_clocksource.rating =275 ; +clocksource_register_hz (&xen_clocksource,NSEC_PER_SEC );if (HYPERVISOR_vcpu_op (VCPUOP_stop_periodic_timer, cpu,NULL ) ==0 ) {
Signature · 1 line
1.7.9.5
Quoted 5 lines · Imre Palik
From: "Palik, Imre" <[email protected]>
In Dom0's the use of the TSC clocksource (whenever it is stable enough to be used) instead of the Xen clocksource should not cause any issues, as Dom0 VMs never live-migrated.
Is this still true given that dom0's vcpus are migrated amongst pcpus on the host? The tsc are not synchronised on some generations of hardware so the result there would be the TSC appearing to do very odd things under dom0's feet. Does Linux cope with that or does it not matter for some other reason?
Quoted 22 lines
Cc Anthony Liguori <[email protected]>Signed-off-by Imre Palik <[email protected]>--- arch/x86/xen/time.c | 4 ++++ 1 file changed, 4 insertions(+)arch/x86/xen/time.c +4
-0@@ -487,6 +487,10 @@ static void __init xen_time_init(void)int cpu =smp_processor_id ();struct timespec tp; +/* As Dom0 is never moved, no penalty on using TSC there */ +if (xen_initial_domain ()) + xen_clocksource.rating =275 ; +clocksource_register_hz (&xen_clocksource,NSEC_PER_SEC );if (HYPERVISOR_vcpu_op (VCPUOP_stop_periodic_timer, cpu,NULL ) ==0 ) {
Quoted 8 lines
Quoted 1 line
Quoted 1 line · On 07.01.15 at 17:30, <[email protected]> wrote:
Quoted 5 lines · Imre Palik
From: "Palik, Imre" <[email protected]>
In Dom0's the use of the TSC clocksource (whenever it is stable enough to be used) instead of the Xen clocksource should not cause any issues, as Dom0 VMs never live-migrated.
Quoted 5 lines
Is this still true given that dom0's vcpus are migrated amongst pcpus on the host? The tsc are not synchronised on some generations of hardware so the result there would be the TSC appearing to do very odd things under dom0's feet. Does Linux cope with that or does it not matter for some other reason?
Indeed. The textual qualification above ("whenever it is stable enough") isn't being expressed in the code change at all.
Jan
Quoted 8 lines
Quoted 1 line
Quoted 1 line · On 07.01.15 at 17:30, <[email protected]> wrote:
Quoted 5 lines · Imre Palik
From: "Palik, Imre" <[email protected]>
In Dom0's the use of the TSC clocksource (whenever it is stable enough to be used) instead of the Xen clocksource should not cause any issues, as Dom0 VMs never live-migrated.
Quoted 5 lines
Is this still true given that dom0's vcpus are migrated amongst pcpus on the host? The tsc are not synchronised on some generations of hardware so the result there would be the TSC appearing to do very odd things under dom0's feet. Does Linux cope with that or does it not matter for some other reason?
Indeed. The textual qualification above ("whenever it is stable enough") isn't being expressed in the code change at all.
Jan
Quoted 7 lines · Ian Campbell
Quoted 5 lines · Imre Palik
From: "Palik, Imre" <[email protected]>
In Dom0's the use of the TSC clocksource (whenever it is stable enough to be used) instead of the Xen clocksource should not cause any issues, as Dom0 VMs never live-migrated.
Quoted 5 lines
Is this still true given that dom0's vcpus are migrated amongst pcpus on the host? The tsc are not synchronised on some generations of hardware so the result there would be the TSC appearing to do very odd things under dom0's feet. Does Linux cope with that or does it not matter for some other reason?
First of all, if the initial pcpus are not synchronised, linux won't consider TSC as a viable clocksource.
If the initial pcpus are synchronised, but then the dom0 vcpus are migrated to unsynchronised pcpus, then hopefully the tsc watchdog catches the issue, and the kernel falls back to the xen clocksource. The issue here is that the watchdog can only detect clock skews bigger than 0.0625s/0.5s. Hopefully this is enough to avoid the weirdest things.
Also, some parts of the kernel (e.g., scheduling) will always use the paravirt clock. No matter what priorities are set.
So it should be safe for some definition of safe.
But I was unable to test it on a hardware platform that would hit the problematic case described above.
Quoted 25 lines
Quoted 23 lines
Cc Anthony Liguori <[email protected]>Signed-off-by Imre Palik <[email protected]>--- arch/x86/xen/time.c | 4 ++++ 1 file changed, 4 insertions(+)arch/x86/xen/time.c +4
-0@@ -487,6 +487,10 @@ static void __init xen_time_init(void)int cpu =smp_processor_id ();struct timespec tp; +/* As Dom0 is never moved, no penalty on using TSC there */ +if (xen_initial_domain ()) + xen_clocksource.rating =275 ; +clocksource_register_hz (&xen_clocksource,NSEC_PER_SEC );if (HYPERVISOR_vcpu_op (VCPUOP_stop_periodic_timer, cpu,NULL ) ==0 ) {
Quoted 7 lines · Ian Campbell
Quoted 5 lines · Imre Palik
From: "Palik, Imre" <[email protected]>
In Dom0's the use of the TSC clocksource (whenever it is stable enough to be used) instead of the Xen clocksource should not cause any issues, as Dom0 VMs never live-migrated.
Quoted 5 lines
Is this still true given that dom0's vcpus are migrated amongst pcpus on the host? The tsc are not synchronised on some generations of hardware so the result there would be the TSC appearing to do very odd things under dom0's feet. Does Linux cope with that or does it not matter for some other reason?
First of all, if the initial pcpus are not synchronised, linux won't consider TSC as a viable clocksource.
If the initial pcpus are synchronised, but then the dom0 vcpus are migrated to unsynchronised pcpus, then hopefully the tsc watchdog catches the issue, and the kernel falls back to the xen clocksource. The issue here is that the watchdog can only detect clock skews bigger than 0.0625s/0.5s. Hopefully this is enough to avoid the weirdest things.
Also, some parts of the kernel (e.g., scheduling) will always use the paravirt clock. No matter what priorities are set.
So it should be safe for some definition of safe.
But I was unable to test it on a hardware platform that would hit the problematic case described above.
Quoted 25 lines
Quoted 23 lines
Cc Anthony Liguori <[email protected]>Signed-off-by Imre Palik <[email protected]>--- arch/x86/xen/time.c | 4 ++++ 1 file changed, 4 insertions(+)arch/x86/xen/time.c +4
-0@@ -487,6 +487,10 @@ static void __init xen_time_init(void)int cpu =smp_processor_id ();struct timespec tp; +/* As Dom0 is never moved, no penalty on using TSC there */ +if (xen_initial_domain ()) + xen_clocksource.rating =275 ; +clocksource_register_hz (&xen_clocksource,NSEC_PER_SEC );if (HYPERVISOR_vcpu_op (VCPUOP_stop_periodic_timer, cpu,NULL ) ==0 ) {
Quoted 14 lines · Imre Palik
Quoted 7 lines · Ian Campbell
Quoted 5 lines · Imre Palik
From: "Palik, Imre" <[email protected]>
In Dom0's the use of the TSC clocksource (whenever it is stable enough to be used) instead of the Xen clocksource should not cause any issues, as Dom0 VMs never live-migrated.
Quoted 5 lines
Is this still true given that dom0's vcpus are migrated amongst pcpus on the host? The tsc are not synchronised on some generations of hardware so the result there would be the TSC appearing to do very odd things under dom0's feet. Does Linux cope with that or does it not matter for some other reason?
Quoted 8 lines
First of all, if the initial pcpus are not synchronised, linux won't consider TSC as a viable clocksource.
If the initial pcpus are synchronised, but then the dom0 vcpus are migrated to unsynchronised pcpus, then hopefully the tsc watchdog catches the issue, and the kernel falls back to the xen clocksource. The issue here is that the watchdog can only detect clock skews bigger than 0.0625s/0.5s. Hopefully this is enough to avoid the weirdest things.
I don't think any such hardware exists. Either TSC is synchronized across all CPUs or none.
Quoted 6 lines
Also, some parts of the kernel (e.g., scheduling) will always use the paravirt clock. No matter what priorities are set.
So it should be safe for some definition of safe.
But I was unable to test it on a hardware platform that would hit the problematic case described above.
Ok. Can you list the various time sources and their ratings in the commit message for clarity. i.e, to justify 275 (below TSC = 300. above hpet = 250).
David
Quoted 14 lines · Imre Palik
Quoted 7 lines · Ian Campbell
Quoted 5 lines · Imre Palik
From: "Palik, Imre" <[email protected]>
In Dom0's the use of the TSC clocksource (whenever it is stable enough to be used) instead of the Xen clocksource should not cause any issues, as Dom0 VMs never live-migrated.
Quoted 5 lines
Is this still true given that dom0's vcpus are migrated amongst pcpus on the host? The tsc are not synchronised on some generations of hardware so the result there would be the TSC appearing to do very odd things under dom0's feet. Does Linux cope with that or does it not matter for some other reason?
Quoted 8 lines
First of all, if the initial pcpus are not synchronised, linux won't consider TSC as a viable clocksource.
If the initial pcpus are synchronised, but then the dom0 vcpus are migrated to unsynchronised pcpus, then hopefully the tsc watchdog catches the issue, and the kernel falls back to the xen clocksource. The issue here is that the watchdog can only detect clock skews bigger than 0.0625s/0.5s. Hopefully this is enough to avoid the weirdest things.
I don't think any such hardware exists. Either TSC is synchronized across all CPUs or none.
Quoted 6 lines
Also, some parts of the kernel (e.g., scheduling) will always use the paravirt clock. No matter what priorities are set.
So it should be safe for some definition of safe.
But I was unable to test it on a hardware platform that would hit the problematic case described above.
Ok. Can you list the various time sources and their ratings in the commit message for clarity. i.e, to justify 275 (below TSC = 300. above hpet = 250).
David
Quoted 5 lines · Imre Palik
From: "Palik, Imre" <[email protected]>
In Dom0's the use of the TSC clocksource (whenever it is stable enough to be used) instead of the Xen clocksource should not cause any issues, as Dom0 VMs never live-migrated.
Is this still true given that dom0's vcpus are migrated amongst pcpus on the host? The tsc are not synchronised on some generations of hardware so the result there would be the TSC appearing to do very odd things under dom0's feet. Does Linux cope with that or does it not matter for some other reason?
Quoted 22 lines
Cc Anthony Liguori <[email protected]>Signed-off-by Imre Palik <[email protected]>--- arch/x86/xen/time.c | 4 ++++ 1 file changed, 4 insertions(+)arch/x86/xen/time.c +4
-0@@ -487,6 +487,10 @@ static void __init xen_time_init(void)int cpu =smp_processor_id ();struct timespec tp; +/* As Dom0 is never moved, no penalty on using TSC there */ +if (xen_initial_domain ()) + xen_clocksource.rating =275 ; +clocksource_register_hz (&xen_clocksource,NSEC_PER_SEC );if (HYPERVISOR_vcpu_op (VCPUOP_stop_periodic_timer, cpu,NULL ) ==0 ) {
Boris Ostrovsky
reviewerQuoted 19 lines · Imre Palik
From: "Palik, Imre" <[email protected]>
In Dom0's the use of the TSC clocksource (whenever it is stable enough to be used) instead of the Xen clocksource should not cause any issues, as Dom0 VMs never live-migrated. The TSC clocksource is somewhat more efficient than the Xen paravirtualised clocksource, thus it should have higher rating.
Cc Anthony Liguori <[email protected]>Signed-off-by Imre Palik <[email protected]>--- arch/x86/xen/time.c | 4 ++++ 1 file changed, 4 insertions(+)arch/x86/xen/time.c +0
-0
arch/x86/xen/time.c static void __init xen_time_init(void) · line 492
Quoted 6 lines
arch/x86/xen/time.c +3
-0@@ -487,6 +487,10 @@ static void __init xen_time_init(void)int cpu =smp_processor_id ();struct timespec tp; +/* As Dom0 is never moved, no penalty on using TSC there */ +if (xen_initial_domain ()) + xen_clocksource.rating =275 ;
Should this only be limited to dom0? Can we do this for guests running with TSC_MODE_NEVER_EMULATE as well?
We also have disable_migrate flag for guests (but it doesn't appear to be accessible to a guest kernel).
-boris
Quoted 4 lines
arch/x86/xen/time.c +1
-0+clocksource_register_hz (&xen_clocksource,NSEC_PER_SEC );if (HYPERVISOR_vcpu_op (VCPUOP_stop_periodic_timer, cpu,NULL ) ==0 ) {
Boris Ostrovsky
reviewerQuoted 19 lines · Imre Palik
From: "Palik, Imre" <[email protected]>
In Dom0's the use of the TSC clocksource (whenever it is stable enough to be used) instead of the Xen clocksource should not cause any issues, as Dom0 VMs never live-migrated. The TSC clocksource is somewhat more efficient than the Xen paravirtualised clocksource, thus it should have higher rating.
Cc Anthony Liguori <[email protected]>Signed-off-by Imre Palik <[email protected]>--- arch/x86/xen/time.c | 4 ++++ 1 file changed, 4 insertions(+)arch/x86/xen/time.c +0
-0
arch/x86/xen/time.c static void __init xen_time_init(void) · line 492
Quoted 6 lines
arch/x86/xen/time.c +3
-0@@ -487,6 +487,10 @@ static void __init xen_time_init(void)int cpu =smp_processor_id ();struct timespec tp; +/* As Dom0 is never moved, no penalty on using TSC there */ +if (xen_initial_domain ()) + xen_clocksource.rating =275 ;
Should this only be limited to dom0? Can we do this for guests running with TSC_MODE_NEVER_EMULATE as well?
We also have disable_migrate flag for guests (but it doesn't appear to be accessible to a guest kernel).
-boris
Quoted 4 lines
arch/x86/xen/time.c +1
-0+clocksource_register_hz (&xen_clocksource,NSEC_PER_SEC );if (HYPERVISOR_vcpu_op (VCPUOP_stop_periodic_timer, cpu,NULL ) ==0 ) {