Skip to contentSkip to conversation

qemu-kvm upstreaming: Do we need -no-kvm-pit and -no-kvm-pit-reinjection semantics?

DISCUSSION Jan Kiszka 5 participants 19 messages Jan 19, 2012 – Jan 20, 2012 KVM

lore
Share & tools
lore

Jan Kiszka

author
#0 lore raw

Hi all,

I've finished a first version of cleaned-up in-kernel KVM PIT support. That will be rolled out once the base support for irqchip has been merged.

I'm now wondering if and how to model two control knobs we have in qemu-kvm:

o -no-kvm-pit, ie. disable the in-kernel PIT even when {A,IOA,}PIC are kernel based (default: off, ie. use in-kernel PIT)

o -no-kvm-pit-reinjection, ie. control over the lost ticks reinjection logic in the kernel (default: off, ie. do reinject)

So far I dropped the former and modeled the latter via a qdev property. But I tend to think that even the latter knob is superfluous. In that case I would also deprecate the original switches in qemu-kvm, just like recently done with -tdf.

Other thoughts?

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux
#1 lore raw
Quoted 9 lines · Jan Kiszka

Hi all,

I've finished a first version of cleaned-up in-kernel KVM PIT support. That will be rolled out once the base support for irqchip has been merged.

I'm now wondering if and how to model two control knobs we have in qemu-kvm:

o -no-kvm-pit, ie. disable the in-kernel PIT even when {A,IOA,}PIC are kernel based (default: off, ie. use in-kernel PIT)

It can be useful for debugging.

Quoted 2 lines

o -no-kvm-pit-reinjection, ie. control over the lost ticks reinjection logic in the kernel (default: off, ie. do reinject)

If the guest kernel does not compensate for lost ticks, reinjection is needed. Otherwise, it might cause problems.

Therefore this option is needed.

Quoted 16 lines

So far I dropped the former and modeled the latter via a qdev property. But I tend to think that even the latter knob is superfluous. In that case I would also deprecate the original switches in qemu-kvm, just like recently done with -tdf.

Other thoughts?

Jan

Signature · 6 lines
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
author
#2 lore raw
Quoted 6 lines · Marcelo Tosatti
Quoted 5 lines · Jan Kiszka

Hi all,

I've finished a first version of cleaned-up in-kernel KVM PIT support. That will be rolled out once the base support for irqchip has been merged.

Quoted 6 lines
Quoted 4 lines

I'm now wondering if and how to model two control knobs we have in qemu-kvm:

o -no-kvm-pit, ie. disable the in-kernel PIT even when {A,IOA,}PIC are kernel based (default: off, ie. use in-kernel PIT)

It can be useful for debugging.

When was it last? The kernel model should be stable by now, just possibly incomplete. But then there is still the full emulation in user mode, available via kernel_irqchip=off.

We can control it, the question is if we really need the granularity.

Quoted 6 lines
Quoted 3 lines

o -no-kvm-pit-reinjection, ie. control over the lost ticks reinjection logic in the kernel (default: off, ie. do reinject)

If the guest kernel does not compensate for lost ticks, reinjection is needed. Otherwise, it might cause problems.

That's why it's /on/ by default? What problems does it cause, and in which scenarios? Can't they be fixed?

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux
#3 lore raw
Quoted 12 lines · Jan Kiszka
Quoted 11 lines · Marcelo Tosatti
Quoted 9 lines · Jan Kiszka

Hi all,

I've finished a first version of cleaned-up in-kernel KVM PIT support. That will be rolled out once the base support for irqchip has been merged.

I'm now wondering if and how to model two control knobs we have in qemu-kvm:

o -no-kvm-pit, ie. disable the in-kernel PIT even when {A,IOA,}PIC are kernel based (default: off, ie. use in-kernel PIT)

Quoted 7 lines
Quoted 1 line

It can be useful for debugging.

When was it last? The kernel model should be stable by now, just possibly incomplete. But then there is still the full emulation in user mode, available via kernel_irqchip=off.

We can control it, the question is if we really need the granularity.

Right, that should be enough.

Quoted 9 lines
Quoted 7 lines
Quoted 4 lines

o -no-kvm-pit-reinjection, ie. control over the lost ticks reinjection logic in the kernel (default: off, ie. do reinject)

If the guest kernel does not compensate for lost ticks, reinjection is needed. Otherwise, it might cause problems.

That's why it's /on/ by default?

Its on by default because it was the default before the option was introduced.

Quoted 2 lines

What problems does it cause, and in which scenarios? Can't they be fixed?

If the guest compensates for lost ticks, and KVM reinjects them, guest time advances faster then it should, to the extent where NTP fails to correct it. This is the case with RHEL4.

But for example v2.4 kernel (or Windows with non-acpi HAL) do not compensate. In that case you want KVM to reinject.

I don't know of any other way to fix this.

author
#4 lore raw
Quoted 11 lines · Marcelo Tosatti
Quoted 2 lines

What problems does it cause, and in which scenarios? Can't they be fixed?

If the guest compensates for lost ticks, and KVM reinjects them, guest time advances faster then it should, to the extent where NTP fails to correct it. This is the case with RHEL4.

But for example v2.4 kernel (or Windows with non-acpi HAL) do not compensate. In that case you want KVM to reinject.

I don't know of any other way to fix this.

OK, i see. The old unsolved problem of guessing what is being executed.

Then the next question is how and where to control this. Conceptually, there should rather be a global switch say "compensate for lost ticks of periodic timers: yes/no" - instead of a per-timer knob. Didn't we discussed something like this before?

What about periodic APIC tick compensation? I suppose the kernel does not support this as no common guest makes use of this as clock source, right? Or the HPET? Once the user space model supports compensation, we need to control it as well. Individually?

I just want to avoid introducing an clumsy interface we then need to maintain for a long time.

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux
#5 lore raw
Quoted 11 lines · Jan Kiszka
Quoted 10 lines · Marcelo Tosatti
Quoted 2 lines

What problems does it cause, and in which scenarios? Can't they be fixed?

If the guest compensates for lost ticks, and KVM reinjects them, guest time advances faster then it should, to the extent where NTP fails to correct it. This is the case with RHEL4.

But for example v2.4 kernel (or Windows with non-acpi HAL) do not compensate. In that case you want KVM to reinject.

Quoted 8 lines
Quoted 1 line

I don't know of any other way to fix this.

OK, i see. The old unsolved problem of guessing what is being executed.

Then the next question is how and where to control this. Conceptually, there should rather be a global switch say "compensate for lost ticks of periodic timers: yes/no" - instead of a per-timer knob. Didn't we discussed something like this before?

I don't see the advantage of a global control versus per device control (in fact it lowers flexibility).

Quoted 3 lines

What about periodic APIC tick compensation? I suppose the kernel does not support this as no common guest makes use of this as clock source, right?

Recent guests use the APIC timer as clock event, but their time keeping algorithms are not as susceptible to lost ticks as the ones that use PIT/RTC.

Quoted 2 lines

Or the HPET? Once the user space model supports compensation, we need to control it as well. Individually?

Ulrich has posted patches for HPET compensation:

http://lists.gnu.org/archive/html/qemu-devel/2011-03/msg01989.html

Quoted 4 lines

I just want to avoid introducing an clumsy interface we then need to maintain for a long time.

Jan

If the option is a qdev property, i don't see what is clumsy about it?

author
#6 lore raw
Quoted 16 lines · Marcelo Tosatti
Quoted 15 lines · Jan Kiszka
Quoted 11 lines · Marcelo Tosatti
Quoted 2 lines

What problems does it cause, and in which scenarios? Can't they be fixed?

If the guest compensates for lost ticks, and KVM reinjects them, guest time advances faster then it should, to the extent where NTP fails to correct it. This is the case with RHEL4.

But for example v2.4 kernel (or Windows with non-acpi HAL) do not compensate. In that case you want KVM to reinject.

I don't know of any other way to fix this.

OK, i see. The old unsolved problem of guessing what is being executed.

Quoted 7 lines
Quoted 4 lines

Then the next question is how and where to control this. Conceptually, there should rather be a global switch say "compensate for lost ticks of periodic timers: yes/no" - instead of a per-timer knob. Didn't we discussed something like this before?

I don't see the advantage of a global control versus per device control (in fact it lowers flexibility).

Usability. Users should not have to care about individual tick-based clocks. They care about "my OS requires lost ticks compensation, yes or no".

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux
#7 lore raw
Quoted 22 lines · Jan Kiszka
Quoted 21 lines · Marcelo Tosatti
Quoted 19 lines · Jan Kiszka
Quoted 11 lines · Marcelo Tosatti
Quoted 2 lines

What problems does it cause, and in which scenarios? Can't they be fixed?

If the guest compensates for lost ticks, and KVM reinjects them, guest time advances faster then it should, to the extent where NTP fails to correct it. This is the case with RHEL4.

But for example v2.4 kernel (or Windows with non-acpi HAL) do not compensate. In that case you want KVM to reinject.

I don't know of any other way to fix this.

OK, i see. The old unsolved problem of guessing what is being executed.

Then the next question is how and where to control this. Conceptually, there should rather be a global switch say "compensate for lost ticks of periodic timers: yes/no" - instead of a per-timer knob. Didn't we discussed something like this before?

Quoted 5 lines
Quoted 2 lines

I don't see the advantage of a global control versus per device control (in fact it lowers flexibility).

Usability. Users should not have to care about individual tick-based clocks. They care about "my OS requires lost ticks compensation, yes or no".

FYI, at the libvirt level we model policy against individual timers, for example:

<clock offset="localtime">
<timer name="rtc" tickpolicy="catchup" track="guest"/> <timer name="pit" tickpolicy="delay"/>
</clock>

Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
author
#8 lore raw
Quoted 29 lines · Daniel P. Berrange
Quoted 22 lines · Jan Kiszka
Quoted 21 lines · Marcelo Tosatti
Quoted 19 lines · Jan Kiszka
Quoted 11 lines · Marcelo Tosatti
Quoted 2 lines

What problems does it cause, and in which scenarios? Can't they be fixed?

If the guest compensates for lost ticks, and KVM reinjects them, guest time advances faster then it should, to the extent where NTP fails to correct it. This is the case with RHEL4.

But for example v2.4 kernel (or Windows with non-acpi HAL) do not compensate. In that case you want KVM to reinject.

I don't know of any other way to fix this.

OK, i see. The old unsolved problem of guessing what is being executed.

Then the next question is how and where to control this. Conceptually, there should rather be a global switch say "compensate for lost ticks of periodic timers: yes/no" - instead of a per-timer knob. Didn't we discussed something like this before?

Quoted 5 lines
Quoted 2 lines

I don't see the advantage of a global control versus per device control (in fact it lowers flexibility).

Usability. Users should not have to care about individual tick-based clocks. They care about "my OS requires lost ticks compensation, yes or no".

Quoted 7 lines

FYI, at the libvirt level we model policy against individual timers, for example:

<clock offset="localtime">
<timer name="rtc" tickpolicy="catchup" track="guest"/> <timer name="pit" tickpolicy="delay"/>
</clock>

Are the various modes of tickpolicy fully specified somewhere?

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux
#9 lore raw
Quoted 33 lines · Jan Kiszka
Quoted 32 lines · Daniel P. Berrange
Quoted 22 lines · Jan Kiszka
Quoted 21 lines · Marcelo Tosatti
Quoted 19 lines · Jan Kiszka
Quoted 11 lines · Marcelo Tosatti
Quoted 2 lines

What problems does it cause, and in which scenarios? Can't they be fixed?

If the guest compensates for lost ticks, and KVM reinjects them, guest time advances faster then it should, to the extent where NTP fails to correct it. This is the case with RHEL4.

But for example v2.4 kernel (or Windows with non-acpi HAL) do not compensate. In that case you want KVM to reinject.

I don't know of any other way to fix this.

OK, i see. The old unsolved problem of guessing what is being executed.

Then the next question is how and where to control this. Conceptually, there should rather be a global switch say "compensate for lost ticks of periodic timers: yes/no" - instead of a per-timer knob. Didn't we discussed something like this before?

Quoted 5 lines
Quoted 2 lines

I don't see the advantage of a global control versus per device control (in fact it lowers flexibility).

Usability. Users should not have to care about individual tick-based clocks. They care about "my OS requires lost ticks compensation, yes or no".

FYI, at the libvirt level we model policy against individual timers, for example:

Quoted 6 lines
Quoted 4 lines

<clock offset="localtime">
<timer name="rtc" tickpolicy="catchup" track="guest"/> <timer name="pit" tickpolicy="delay"/>
</clock>

Are the various modes of tickpolicy fully specified somewhere?

There are some (not all that great) docs here:

http://libvirt.org/formatdomain.html#elementsTime

The meaning of the 4 policies are:

delay: continue to deliver at normal rate
catchup: deliver at higher rate to catchup
merge: ticks merged into 1 single tick
discard: all missed ticks are discarded

The original design rationale was here, though beware that some things changed between this design & the actual implementation libvirt has:

https://www.redhat.com/archives/libvir-list/2010-March/msg00304.html

Regards,
Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
author
#10 lore raw
Quoted 41 lines · Daniel P. Berrange
Quoted 33 lines · Jan Kiszka
Quoted 32 lines · Daniel P. Berrange
Quoted 22 lines · Jan Kiszka
Quoted 21 lines · Marcelo Tosatti
Quoted 19 lines · Jan Kiszka
Quoted 11 lines · Marcelo Tosatti
Quoted 2 lines

What problems does it cause, and in which scenarios? Can't they be fixed?

If the guest compensates for lost ticks, and KVM reinjects them, guest time advances faster then it should, to the extent where NTP fails to correct it. This is the case with RHEL4.

But for example v2.4 kernel (or Windows with non-acpi HAL) do not compensate. In that case you want KVM to reinject.

I don't know of any other way to fix this.

OK, i see. The old unsolved problem of guessing what is being executed.

Then the next question is how and where to control this. Conceptually, there should rather be a global switch say "compensate for lost ticks of periodic timers: yes/no" - instead of a per-timer knob. Didn't we discussed something like this before?

Quoted 5 lines
Quoted 2 lines

I don't see the advantage of a global control versus per device control (in fact it lowers flexibility).

Usability. Users should not have to care about individual tick-based clocks. They care about "my OS requires lost ticks compensation, yes or no".

FYI, at the libvirt level we model policy against individual timers, for example:

Quoted 6 lines
Quoted 4 lines

<clock offset="localtime">
<timer name="rtc" tickpolicy="catchup" track="guest"/> <timer name="pit" tickpolicy="delay"/>
</clock>

Are the various modes of tickpolicy fully specified somewhere?

Quoted 7 lines

There are some (not all that great) docs here:

http://libvirt.org/formatdomain.html#elementsTime

The meaning of the 4 policies are:

delay: continue to deliver at normal rate

What does this mean? The timer stops ticking until the guest accepts its ticks again?

Quoted 3 lines

catchup: deliver at higher rate to catchup
merge: ticks merged into 1 single tick
discard: all missed ticks are discarded

But those interpretations aren't stated in the docs. That makes it hard to map them on individual hypervisors - or model proper new hypervisor interfaces accordingly.

Quoted 9 lines

The original design rationale was here, though beware that some things changed between this design & the actual implementation libvirt has:

https://www.redhat.com/archives/libvir-list/2010-March/msg00304.html

Regards,
Daniel

Given that there is almost no tick compensation in QEMU yet (ignoring the awful RTC hack for now), this is a good time to establish a useful generic interface with the advent of the KVM device models.

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux
#11 lore raw
Quoted 44 lines · Jan Kiszka
Quoted 43 lines · Daniel P. Berrange
Quoted 33 lines · Jan Kiszka
Quoted 32 lines · Daniel P. Berrange
Quoted 22 lines · Jan Kiszka
Quoted 21 lines · Marcelo Tosatti
Quoted 19 lines · Jan Kiszka
Quoted 11 lines · Marcelo Tosatti
Quoted 2 lines

What problems does it cause, and in which scenarios? Can't they be fixed?

If the guest compensates for lost ticks, and KVM reinjects them, guest time advances faster then it should, to the extent where NTP fails to correct it. This is the case with RHEL4.

But for example v2.4 kernel (or Windows with non-acpi HAL) do not compensate. In that case you want KVM to reinject.

I don't know of any other way to fix this.

OK, i see. The old unsolved problem of guessing what is being executed.

Then the next question is how and where to control this. Conceptually, there should rather be a global switch say "compensate for lost ticks of periodic timers: yes/no" - instead of a per-timer knob. Didn't we discussed something like this before?

Quoted 5 lines
Quoted 2 lines

I don't see the advantage of a global control versus per device control (in fact it lowers flexibility).

Usability. Users should not have to care about individual tick-based clocks. They care about "my OS requires lost ticks compensation, yes or no".

FYI, at the libvirt level we model policy against individual timers, for example:

Quoted 6 lines
Quoted 4 lines

<clock offset="localtime">
<timer name="rtc" tickpolicy="catchup" track="guest"/> <timer name="pit" tickpolicy="delay"/>
</clock>

Are the various modes of tickpolicy fully specified somewhere?

There are some (not all that great) docs here:

Quoted 8 lines
Quoted 5 lines

http://libvirt.org/formatdomain.html#elementsTime

The meaning of the 4 policies are:

delay: continue to deliver at normal rate

What does this mean? The timer stops ticking until the guest accepts its ticks again?

It means that the hypervisor will not attempt to do any compensation, so the guest will see delays in its ticks being delivered & gradually drift over time.

Quoted 7 lines
Quoted 3 lines

catchup: deliver at higher rate to catchup
merge: ticks merged into 1 single tick
discard: all missed ticks are discarded

But those interpretations aren't stated in the docs. That makes it hard to map them on individual hypervisors - or model proper new hypervisor interfaces accordingly.

That's not a real problem, now I notice they are missing the docs, I can just add them in.

Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
author
#12 lore raw
Quoted 49 lines · Daniel P. Berrange
Quoted 48 lines · Jan Kiszka
Quoted 47 lines · Daniel P. Berrange
Quoted 33 lines · Jan Kiszka
Quoted 32 lines · Daniel P. Berrange
Quoted 22 lines · Jan Kiszka
Quoted 21 lines · Marcelo Tosatti
Quoted 19 lines · Jan Kiszka
Quoted 11 lines · Marcelo Tosatti
Quoted 2 lines

What problems does it cause, and in which scenarios? Can't they be fixed?

If the guest compensates for lost ticks, and KVM reinjects them, guest time advances faster then it should, to the extent where NTP fails to correct it. This is the case with RHEL4.

But for example v2.4 kernel (or Windows with non-acpi HAL) do not compensate. In that case you want KVM to reinject.

I don't know of any other way to fix this.

OK, i see. The old unsolved problem of guessing what is being executed.

Then the next question is how and where to control this. Conceptually, there should rather be a global switch say "compensate for lost ticks of periodic timers: yes/no" - instead of a per-timer knob. Didn't we discussed something like this before?

Quoted 5 lines
Quoted 2 lines

I don't see the advantage of a global control versus per device control (in fact it lowers flexibility).

Usability. Users should not have to care about individual tick-based clocks. They care about "my OS requires lost ticks compensation, yes or no".

FYI, at the libvirt level we model policy against individual timers, for example:

Quoted 6 lines
Quoted 4 lines

<clock offset="localtime">
<timer name="rtc" tickpolicy="catchup" track="guest"/> <timer name="pit" tickpolicy="delay"/>
</clock>

Are the various modes of tickpolicy fully specified somewhere?

There are some (not all that great) docs here:

http://libvirt.org/formatdomain.html#elementsTime

The meaning of the 4 policies are:

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

delay: continue to deliver at normal rate

What does this mean? The timer stops ticking until the guest accepts its ticks again?

It means that the hypervisor will not attempt to do any compensation, so the guest will see delays in its ticks being delivered & gradually drift over time.

Still, is the logic as I described? Or what is the difference to "discard".

Quoted 11 lines
Quoted 8 lines
Quoted 4 lines

catchup: deliver at higher rate to catchup
merge: ticks merged into 1 single tick
discard: all missed ticks are discarded

But those interpretations aren't stated in the docs. That makes it hard to map them on individual hypervisors - or model proper new hypervisor interfaces accordingly.

That's not a real problem, now I notice they are missing the docs, I can just add them in.

TIA, but just please more verbose. The above descriptions only help if you take real implementations of hypervisors as reference.

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux
#13 lore raw
Quoted 52 lines · Jan Kiszka
Quoted 51 lines · Daniel P. Berrange
Quoted 50 lines · Jan Kiszka
Quoted 48 lines · Daniel P. Berrange
Quoted 33 lines · Jan Kiszka
Quoted 32 lines · Daniel P. Berrange
Quoted 22 lines · Jan Kiszka
Quoted 21 lines · Marcelo Tosatti
Quoted 19 lines · Jan Kiszka
Quoted 11 lines · Marcelo Tosatti
Quoted 2 lines

What problems does it cause, and in which scenarios? Can't they be fixed?

If the guest compensates for lost ticks, and KVM reinjects them, guest time advances faster then it should, to the extent where NTP fails to correct it. This is the case with RHEL4.

But for example v2.4 kernel (or Windows with non-acpi HAL) do not compensate. In that case you want KVM to reinject.

I don't know of any other way to fix this.

OK, i see. The old unsolved problem of guessing what is being executed.

Then the next question is how and where to control this. Conceptually, there should rather be a global switch say "compensate for lost ticks of periodic timers: yes/no" - instead of a per-timer knob. Didn't we discussed something like this before?

Quoted 5 lines
Quoted 2 lines

I don't see the advantage of a global control versus per device control (in fact it lowers flexibility).

Usability. Users should not have to care about individual tick-based clocks. They care about "my OS requires lost ticks compensation, yes or no".

FYI, at the libvirt level we model policy against individual timers, for example:

Quoted 6 lines
Quoted 4 lines

<clock offset="localtime">
<timer name="rtc" tickpolicy="catchup" track="guest"/> <timer name="pit" tickpolicy="delay"/>
</clock>

Are the various modes of tickpolicy fully specified somewhere?

There are some (not all that great) docs here:

http://libvirt.org/formatdomain.html#elementsTime

The meaning of the 4 policies are:

delay: continue to deliver at normal rate

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

What does this mean? The timer stops ticking until the guest accepts its ticks again?

It means that the hypervisor will not attempt to do any compensation, so the guest will see delays in its ticks being delivered & gradually drift over time.

Still, is the logic as I described? Or what is the difference to "discard".

With 'discard', the delayed tick will be thrown away. In 'delay', the delayed tick will still be injected to the guest, possibly well after the intended injection time though, and there will be no attempt to compensate by speeding up delivery of later ticks.

Regards,
Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
author
#14 lore raw
Quoted 60 lines · Daniel P. Berrange
Quoted 59 lines · Jan Kiszka
Quoted 57 lines · Daniel P. Berrange
Quoted 44 lines · Jan Kiszka
Quoted 43 lines · Daniel P. Berrange
Quoted 33 lines · Jan Kiszka
Quoted 32 lines · Daniel P. Berrange
Quoted 22 lines · Jan Kiszka
Quoted 21 lines · Marcelo Tosatti
Quoted 19 lines · Jan Kiszka
Quoted 11 lines · Marcelo Tosatti
Quoted 2 lines

What problems does it cause, and in which scenarios? Can't they be fixed?

If the guest compensates for lost ticks, and KVM reinjects them, guest time advances faster then it should, to the extent where NTP fails to correct it. This is the case with RHEL4.

But for example v2.4 kernel (or Windows with non-acpi HAL) do not compensate. In that case you want KVM to reinject.

I don't know of any other way to fix this.

OK, i see. The old unsolved problem of guessing what is being executed.

Then the next question is how and where to control this. Conceptually, there should rather be a global switch say "compensate for lost ticks of periodic timers: yes/no" - instead of a per-timer knob. Didn't we discussed something like this before?

Quoted 5 lines
Quoted 2 lines

I don't see the advantage of a global control versus per device control (in fact it lowers flexibility).

Usability. Users should not have to care about individual tick-based clocks. They care about "my OS requires lost ticks compensation, yes or no".

FYI, at the libvirt level we model policy against individual timers, for example:

Quoted 6 lines
Quoted 4 lines

<clock offset="localtime">
<timer name="rtc" tickpolicy="catchup" track="guest"/> <timer name="pit" tickpolicy="delay"/>
</clock>

Are the various modes of tickpolicy fully specified somewhere?

There are some (not all that great) docs here:

Quoted 8 lines
Quoted 5 lines

http://libvirt.org/formatdomain.html#elementsTime

The meaning of the 4 policies are:

delay: continue to deliver at normal rate

What does this mean? The timer stops ticking until the guest accepts its ticks again?

It means that the hypervisor will not attempt to do any compensation, so the guest will see delays in its ticks being delivered & gradually drift over time.

Quoted 6 lines
Quoted 1 line

Still, is the logic as I described? Or what is the difference to "discard".

With 'discard', the delayed tick will be thrown away. In 'delay', the delayed tick will still be injected to the guest, possibly well after the intended injection time though, and there will be no attempt to compensate by speeding up delivery of later ticks.

OK, let's see if I got it:

delay - all lost ticks are replayed in a row once the guest accepts
them again
catchup - lost ticks are gradually replayed at a higher frequency than
the original tick
merge - at most one tick is replayed once the guest accepts it again
discard - no lost ticks compensation

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux
#15 lore raw
Quoted 68 lines · Jan Kiszka
Quoted 60 lines · Daniel P. Berrange
Quoted 59 lines · Jan Kiszka
Quoted 57 lines · Daniel P. Berrange
Quoted 44 lines · Jan Kiszka
Quoted 43 lines · Daniel P. Berrange
Quoted 33 lines · Jan Kiszka
Quoted 32 lines · Daniel P. Berrange
Quoted 22 lines · Jan Kiszka
Quoted 21 lines · Marcelo Tosatti
Quoted 19 lines · Jan Kiszka
Quoted 11 lines · Marcelo Tosatti
Quoted 2 lines

What problems does it cause, and in which scenarios? Can't they be
fixed?

If the guest compensates for lost ticks, and KVM reinjects them, guest time advances faster then it should, to the extent where NTP fails to correct it. This is the case with RHEL4.

But for example v2.4 kernel (or Windows with non-acpi HAL) do not compensate. In that case you want KVM to reinject.

I don't know of any other way to fix this.

OK, i see. The old unsolved problem of guessing what is being executed.

Then the next question is how and where to control this. Conceptually, there should rather be a global switch say "compensate for lost ticks of periodic timers: yes/no" - instead of a per-timer knob. Didn't we discussed something like this before?

Quoted 5 lines
Quoted 2 lines

I don't see the advantage of a global control versus per device control (in fact it lowers flexibility).

Usability. Users should not have to care about individual tick-based clocks. They care about "my OS requires lost ticks compensation, yes or no".

FYI, at the libvirt level we model policy against individual timers, for example:

Quoted 6 lines
Quoted 4 lines

<clock offset="localtime">
<timer name="rtc" tickpolicy="catchup" track="guest"/> <timer name="pit" tickpolicy="delay"/>
</clock>

Are the various modes of tickpolicy fully specified somewhere?

There are some (not all that great) docs here:

Quoted 8 lines
Quoted 5 lines

http://libvirt.org/formatdomain.html#elementsTime

The meaning of the 4 policies are:

delay: continue to deliver at normal rate

What does this mean? The timer stops ticking until the guest accepts its ticks again?

It means that the hypervisor will not attempt to do any compensation, so the guest will see delays in its ticks being delivered & gradually drift over time.

Quoted 6 lines
Quoted 1 line

Still, is the logic as I described? Or what is the difference to "discard".

With 'discard', the delayed tick will be thrown away. In 'delay', the delayed tick will still be injected to the guest, possibly well after the intended injection time though, and there will be no attempt to compensate by speeding up delivery of later ticks.

Quoted 8 lines

OK, let's see if I got it:

delay - all lost ticks are replayed in a row once the guest accepts
them again
catchup - lost ticks are gradually replayed at a higher frequency than
the original tick
merge - at most one tick is replayed once the guest accepts it again
discard - no lost ticks compensation

Yes, I think that is a good understanding.

Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
#16 lore raw
In reply to Jan Kiszka
Quoted 2 lines · Jan Kiszka

Usability. Users should not have to care about individual tick-based clocks. They care about "my OS requires lost ticks compensation, yes or no".

Conceivably an OS may require lost ticks compensation depending on boot options given to the OS telling it which clock sources to use.

However I like the idea of a global default, which you can set and all the devices inherit it unless overridden in each device.

-- Jamie

author
#17 lore raw
Quoted 9 lines · Jamie Lokier
Quoted 2 lines · Jan Kiszka

Usability. Users should not have to care about individual tick-based clocks. They care about "my OS requires lost ticks compensation, yes or no".

Conceivably an OS may require lost ticks compensation depending on boot options given to the OS telling it which clock sources to use.

However I like the idea of a global default, which you can set and all the devices inherit it unless overridden in each device.

OK, this sounds like a good option: add per-device control but also introduce global default. The latter can still be done later on.

The only problem is that we should already come up with the right, generic control switch template. "reinject=on|off", as I did it for now for the PIT, is definitely not optimal.

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux
maintainer
#18 lore raw
Quoted 6 lines · Jan Kiszka

OK, this sounds like a good option: add per-device control but also introduce global default. The latter can still be done later on.

The only problem is that we should already come up with the right, generic control switch template. "reinject=on|off", as I did it for now for the PIT, is definitely not optimal.

What about adding suboptions to -clock (like driftfix we have for -rtc)?