Make QEmu HPET disabled by default for KVM?
I think we have already suffered enough timer issues due to this(e.g. I can't boot up well on 2.6.18 kernel)... I have kept --no-hpet in my setup for months...
Signature · 2 lines
regards Yang, Sheng
Quoted 2 lines · Sheng Yang
I think we have already suffered enough timer issues due to this(e.g. I can't boot up well on 2.6.18 kernel)...
2.6.18 as guest or as host?
Quoted 3 lines
I have kept --no-hpet in my setup for
months...
Any details about the problems? HPET is important to some guests.
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 5 lines · On Thursday 11 March 2010 15:58:12 Avi Kivity wrote:
Quoted 2 lines · Sheng Yang
I think we have already suffered enough timer issues due to this(e.g. I can't boot up well on 2.6.18 kernel)...
2.6.18 as guest or as host?
Guest
Quoted 5 lines
Quoted 2 lines
I have kept --no-hpet in my setup for
months...Any details about the problems? HPET is important to some guests.
Seems like HPET reaction is too slow to satisfy some guests(for it would replace PIT).
Here is the thread last time.
http://thread.gmane.org/gmane.comp.emulators.kvm.devel/44899
Signature · 2 lines
regards Yang, Sheng
Quoted 6 lines · Sheng Yang
Quoted 6 lines
Quoted 3 lines
I have kept --no-hpet in my setup for
months...Any details about the problems? HPET is important to some guests.
Quoted 8 lines
Seems like HPET reaction is too slow to satisfy some guests(for it would replace PIT).
Here is the thread last time.
http://thread.gmane.org/gmane.comp.emulators.kvm.devel/44899
Thanks. We can address this in three ways: first, adjust the guest not to do timing related tests when virtualized (since no matter what we do, the tests may fail). Second, I think we should implement userspace ack notifiers (similar to tpr access notifiers already present). Third, we can implement a kernel hpet, which, after we solve the zillion bug it introduces, will also give a nice performance improvement for hpet intensive workloads.
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 13 lines · Avi Kivity
Quoted 11 lines · Sheng Yang
Quoted 4 lines
Quoted 2 lines
I have kept --no-hpet in my setup for
months...Any details about the problems? HPET is important to some guests.
Seems like HPET reaction is too slow to satisfy some guests(for it would replace PIT).
Here is the thread last time.
http://thread.gmane.org/gmane.comp.emulators.kvm.devel/44899
Quoted 8 lines
Thanks. We can address this in three ways: first, adjust the guest not to do timing related tests when virtualized (since no matter what we do, the tests may fail). Second, I think we should implement userspace ack notifiers (similar to tpr access notifiers already present). Third, we can implement a kernel hpet, which, after we solve the zillion bug it introduces, will also give a nice performance improvement for hpet intensive workloads.
Second will not solve the problem. Presence of ack notifiers will not make HPET interrupt arrive faster.
--
Gleb.
Quoted 27 lines · Gleb Natapov
Quoted 16 lines · Avi Kivity
Quoted 6 lines · Sheng Yang
Quoted 6 lines
Quoted 3 lines
I have kept --no-hpet in my setup for
months...Any details about the problems? HPET is important to some guests.
Quoted 8 lines
Seems like HPET reaction is too slow to satisfy some guests(for it would replace PIT).
Here is the thread last time.
http://thread.gmane.org/gmane.comp.emulators.kvm.devel/44899
Quoted 9 lines
Thanks. We can address this in three ways: first, adjust the guest not to do timing related tests when virtualized (since no matter what we do, the tests may fail). Second, I think we should implement userspace ack notifiers (similar to tpr access notifiers already present). Third, we can implement a kernel hpet, which, after we solve the zillion bug it introduces, will also give a nice performance improvement for hpet intensive workloads.
Quoted 3 lines
Second will not solve the problem. Presence of ack notifiers will not make HPET interrupt arrive faster.
But it will allow us to compensate for interrupts being coalesced, which may be the root of the problem.
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 22 lines · On Thursday 11 March 2010 16:31:57 Gleb Natapov wrote:
Quoted 13 lines · Avi Kivity
Quoted 11 lines · Sheng Yang
Quoted 4 lines
Quoted 2 lines
I have kept --no-hpet in my setup for
months...Any details about the problems? HPET is important to some guests.
Seems like HPET reaction is too slow to satisfy some guests(for it would replace PIT).
Here is the thread last time.
http://thread.gmane.org/gmane.comp.emulators.kvm.devel/44899
Quoted 7 lines
Thanks. We can address this in three ways: first, adjust the guest not to do timing related tests when virtualized (since no matter what we do, the tests may fail). Second, I think we should implement userspace ack notifiers (similar to tpr access notifiers already present). Third, we can implement a kernel hpet, which, after we solve the zillion bug it introduces, will also give a nice performance improvement for hpet intensive workloads.
Quoted 2 lines
Second will not solve the problem. Presence of ack notifiers will not make HPET interrupt arrive faster.
The slow may also due to lost tick. And with the lost tick, hpet is still unusable...
Signature · 2 lines
regards Yang, Sheng
Quoted 23 lines · Sheng Yang
Quoted 22 lines · On Thursday 11 March 2010 16:31:57 Gleb Natapov wrote:
Quoted 20 lines · Avi Kivity
Quoted 11 lines · Sheng Yang
Quoted 4 lines
Quoted 2 lines
I have kept --no-hpet in my setup for
months...Any details about the problems? HPET is important to some guests.
Seems like HPET reaction is too slow to satisfy some guests(for it would replace PIT).
Here is the thread last time.
http://thread.gmane.org/gmane.comp.emulators.kvm.devel/44899
Thanks. We can address this in three ways: first, adjust the guest not to do timing related tests when virtualized (since no matter what we do, the tests may fail). Second, I think we should implement userspace ack notifiers (similar to tpr access notifiers already present). Third, we can implement a kernel hpet, which, after we solve the zillion bug it introduces, will also give a nice performance improvement for hpet intensive workloads.
Quoted 6 lines
Quoted 2 lines
Second will not solve the problem. Presence of ack notifiers will not make HPET interrupt arrive faster.
The slow may also due to lost tick. And with the lost tick, hpet is still unusable...
If the problem it due to lost ticks reinjection may solve it, but only partially. What if IO thread haven't run even once during the time vcpu did clock source check? IIRC sometimes we trigger this even with in kernel PIT.
--
Gleb.
Quoted 31 lines · Gleb Natapov
Quoted 29 lines · Sheng Yang
Quoted 27 lines · On Thursday 11 March 2010 16:31:57 Gleb Natapov wrote:
Quoted 14 lines · Avi Kivity
Quoted 12 lines · Sheng Yang
Quoted 5 lines
Quoted 3 lines
I have kept --no-hpet in my setup for
months...Any details about the problems? HPET is important to some guests.
Seems like HPET reaction is too slow to satisfy some guests(for it would replace PIT).
Here is the thread last time.
http://thread.gmane.org/gmane.comp.emulators.kvm.devel/44899
Quoted 8 lines
Thanks. We can address this in three ways: first, adjust the guest not to do timing related tests when virtualized (since no matter what we do, the tests may fail). Second, I think we should implement userspace ack notifiers (similar to tpr access notifiers already present). Third, we can implement a kernel hpet, which, after we solve the zillion bug it introduces, will also give a nice performance improvement for hpet intensive workloads.
Second will not solve the problem. Presence of ack notifiers will not make HPET interrupt arrive faster.
Quoted 8 lines
Quoted 4 lines
The slow may also due to lost tick. And with the lost tick, hpet is still unusable...
If the problem it due to lost ticks reinjection may solve it, but only partially. What if IO thread haven't run even once during the time vcpu did clock source check? IIRC sometimes we trigger this even with in kernel PIT.
That is true. Reinjection can correct problems in the long term, but may fail in the short term. 10 ticks is easily short term in a heavily loaded system.
How does it happen with kernel PIT? I could understand it if we had a work item doing the injection, but everything happens either from hrtimer context or vcpu context.
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 30 lines · Avi Kivity
Quoted 20 lines · Gleb Natapov
Quoted 19 lines · Sheng Yang
Quoted 18 lines · On Thursday 11 March 2010 16:31:57 Gleb Natapov wrote:
Quoted 17 lines · Avi Kivity
Quoted 9 lines · Sheng Yang
Quoted 3 lines
Quoted 2 lines
I have kept --no-hpet in my setup for
months...Any details about the problems? HPET is important to some guests.
Seems like HPET reaction is too slow to satisfy some guests(for it would replace PIT).
Here is the thread last time.
http://thread.gmane.org/gmane.comp.emulators.kvm.devel/44899
Thanks. We can address this in three ways: first, adjust the guest not to do timing related tests when virtualized (since no matter what we do, the tests may fail). Second, I think we should implement userspace ack notifiers (similar to tpr access notifiers already present). Third, we can implement a kernel hpet, which, after we solve the zillion bug it introduces, will also give a nice performance improvement for hpet intensive workloads.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Second will not solve the problem. Presence of ack notifiers will not make HPET interrupt arrive faster.
The slow may also due to lost tick. And with the lost tick, hpet is still unusable...
If the problem it due to lost ticks reinjection may solve it, but only partially. What if IO thread haven't run even once during the time vcpu did clock source check? IIRC sometimes we trigger this even with in kernel PIT.
Quoted 8 lines
That is true. Reinjection can correct problems in the long term, but may fail in the short term. 10 ticks is easily short term in a heavily loaded system.
How does it happen with kernel PIT? I could understand it if we had a work item doing the injection, but everything happens either from hrtimer context or vcpu context.
Do we kick vcpu out of guest mode when hrtimer triggers? I don't see us doing it in __kvm_timer_fn().
--
Gleb.
Quoted 9 lines · Gleb Natapov
Quoted 9 lines
Quoted 5 lines
If the problem it due to lost ticks reinjection may solve it, but only partially. What if IO thread haven't run even once during the time vcpu did clock source check? IIRC sometimes we trigger this even with in kernel PIT.
That is true. Reinjection can correct problems in the long term, but may fail in the short term. 10 ticks is easily short term in a heavily loaded system.
Quoted 9 lines
Quoted 5 lines
How does it happen with kernel PIT? I could understand it if we had a work item doing the injection, but everything happens either from hrtimer context or vcpu context.
Do we kick vcpu out of guest mode when hrtimer triggers? I don't see us doing it in
__kvm_timer_fn().
We're always running on the same cpu as vcpu 0, so no need.
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 13 lines · Avi Kivity
Quoted 12 lines · Gleb Natapov
Quoted 12 lines
Quoted 4 lines
If the problem it due to lost ticks reinjection may solve it, but only partially. What if IO thread haven't run even once during the time vcpu did clock source check? IIRC sometimes we trigger this even with in kernel PIT.
That is true. Reinjection can correct problems in the long term, but may fail in the short term. 10 ticks is easily short term in a heavily loaded system.
How does it happen with kernel PIT? I could understand it if we had a work item doing the injection, but everything happens either from hrtimer context or vcpu context.
Quoted 6 lines
Quoted 3 lines
Do we kick vcpu out of guest mode when hrtimer triggers? I don't see us doing it in
__kvm_timer_fn().We're always running on the same cpu as vcpu 0, so no need.
Would be better to do it, though, in case we have migration races.
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 10 lines · Avi Kivity
Quoted 2 lines · Sheng Yang
I think we have already suffered enough timer issues due to this(e.g. I can't boot up well on 2.6.18 kernel)...
2.6.18 as guest or as host?
Quoted 2 lines
I have kept --no-hpet in my setup for
months...Any details about the problems? HPET is important to some guests.
As Gleb mentioned in the other thread, reinjection will introduce another set of problems.
Ideally all this timer related problems should be fixed by correlating timer interrupts and time source reads.
Since one already has to use special timer parameters (-rtc-td-hack, -no-kvm-pit-reinjection), using -no-hpet for problematic Linux guests seems fine?
Quoted 5 lines · Marcelo Tosatti
Quoted 5 lines
Quoted 5 lines
I have kept --no-hpet in my setup for
months...
Quoted 8 lines
Quoted 2 lines
Any details about the problems? HPET is important to some guests.
As Gleb mentioned in the other thread, reinjection will introduce another set of problems.
Ideally all this timer related problems should be fixed by correlating timer interrupts and time source reads.
This still needs reinjection (or slewing of the timer frequency). Correlation doesn't fix drift.
Quoted 4 lines
Since one already has to use special timer parameters (-rtc-td-hack, -no-kvm-pit-reinjection), using -no-hpet for problematic Linux guests seems fine?
Depends on how common the problematic ones are. If they're common, better to have a generic fix.
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 6 lines · Avi Kivity
Quoted 5 lines · Marcelo Tosatti
Quoted 5 lines
Quoted 4 lines
I have kept --no-hpet in my setup for
months...Any details about the problems? HPET is important to some guests.
Quoted 9 lines
Quoted 5 lines
As Gleb mentioned in the other thread, reinjection will introduce another set of problems.
Ideally all this timer related problems should be fixed by correlating timer interrupts and time source reads.
This still needs reinjection (or slewing of the timer frequency). Correlation doesn't fix drift.
But only when all time sources are synchronised and correlated with interrupts we can slew time frequency without guest noticing (and only if guest disables NTP)
Quoted 9 lines
Quoted 3 lines
Since one already has to use special timer parameters (-rtc-td-hack, -no-kvm-pit-reinjection), using -no-hpet for problematic Linux guests seems fine?
Depends on how common the problematic ones are. If they're common, better to have a generic fix.
Signature · 1 line
error compiling committee.c: too many arguments to function
--
Gleb.
Quoted 13 lines · Gleb Natapov
Quoted 12 lines · Avi Kivity
Quoted 10 lines · Marcelo Tosatti
Quoted 5 lines
Quoted 4 lines
I have kept --no-hpet in my setup for
months...Any details about the problems? HPET is important to some guests.
As Gleb mentioned in the other thread, reinjection will introduce another set of problems.
Ideally all this timer related problems should be fixed by correlating timer interrupts and time source reads.
Quoted 6 lines
Quoted 3 lines
This still needs reinjection (or slewing of the timer frequency). Correlation doesn't fix drift.
But only when all time sources are synchronised and correlated with interrupts we can slew time frequency without guest noticing (and only if guest disables NTP)
In the mean time we should definitely disable hpet by default.
Besides this we need to fully virtualize the tsc, fix win7 64bit rtc time drift and some pvclock potential issues. Before we add new timer, better fix existing ones.
What about creating a pv time keeping device that will be aware of lost ticks and host wall clock time? It's similar to hyper-v enlightenment virt timers.
Quoted 17 lines
Quoted 10 lines
Quoted 4 lines
Since one already has to use special timer parameters (-rtc-td-hack, -no-kvm-pit-reinjection), using -no-hpet for problematic Linux guests seems fine?
Depends on how common the problematic ones are. If they're common, better to have a generic fix.
--
error compilingcommittee.c: too many arguments to function--
Gleb.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 14 lines · Dor Laor
Quoted 13 lines · Gleb Natapov
Quoted 12 lines · Avi Kivity
Quoted 10 lines · Marcelo Tosatti
Quoted 5 lines
Quoted 4 lines
I have kept --no-hpet in my setup for
months...Any details about the problems? HPET is important to some guests.
As Gleb mentioned in the other thread, reinjection will introduce another set of problems.
Ideally all this timer related problems should be fixed by correlating timer interrupts and time source reads.
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
This still needs reinjection (or slewing of the timer frequency). Correlation doesn't fix drift.
But only when all time sources are synchronised and correlated with interrupts we can slew time frequency without guest noticing (and only if guest disables NTP)
In the mean time we should definitely disable hpet by default.
Definitely not. Windows needs it. Some pre-kvmclock Linux may also work with it.
Without hpet, there is no fast high resolution timer in the system.
Quoted 3 lines
Besides this we need to fully virtualize the tsc, fix win7 64bit rtc time drift and some pvclock potential issues. Before we add new timer, better fix existing ones.
What about creating a pv time keeping device that will be aware of lost ticks and host wall clock time? It's similar to hyper-v enlightenment virt timers.
That's kvmclock.
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 22 lines · Avi Kivity
Quoted 21 lines · Dor Laor
Quoted 19 lines · Gleb Natapov
Quoted 6 lines · Avi Kivity
Quoted 5 lines · Marcelo Tosatti
Quoted 5 lines
Quoted 4 lines
I have kept --no-hpet in my setup for
months...Any details about the problems? HPET is important to some guests.
Quoted 9 lines
Quoted 5 lines
As Gleb mentioned in the other thread, reinjection will introduce another set of problems.
Ideally all this timer related problems should be fixed by correlating timer interrupts and time source reads.
This still needs reinjection (or slewing of the timer frequency). Correlation doesn't fix drift.
But only when all time sources are synchronised and correlated with interrupts we can slew time frequency without guest noticing (and only if guest disables NTP)
Quoted 6 lines
Quoted 1 line
In the mean time we should definitely disable hpet by default.
Definitely not. Windows needs it. Some pre-kvmclock Linux may also work with it.
Without hpet, there is no fast high resolution timer in the system.
It's all depends on how hard would it be to re-inject to windows guest.
We still need to fix the win2k3 64 bit and win2k8 64 bit (and not win7 as I told initially) since the irq is broadcasted to all the vcpus and we do not track who acknowledged the irq.
Quoted 11 lines
Quoted 8 lines
Besides this we need to fully virtualize the tsc, fix win7 64bit rtc time drift and some pvclock potential issues. Before we add new timer, better fix existing ones.
What about creating a pv time keeping device that will be aware of lost ticks and host wall clock time? It's similar to hyper-v enlightenment virt timers.
That's kvmclock.
I meant a device that can be used to generate timeouts. We do use today pit/rtc along with kvmclock time source but it's not perfect and probably the same for hpet. This is why I tough that a pv device will be beneficial.