Skip to contentSkip to conversation

LoongArch: KVM: Reload one-shot TVAL on migration destination

PATCH Superseded Tao Cui 2 participants 5 messages Jul 15

Subsystems (3)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Tao Cui

author
#0 lore raw

PATCH 1/1 LoongArch: KVM: Reload one-shot TVAL on migration destination

From: Tao Cui <[email protected]>

kvm_restore_timer() rebuilds the remaining timer countdown from vcpu->arch.expire, which is host-internal and is not part of the migrated vCPU state. On the migration destination it is still 0, so for a one-shot timer that has not expired yet the computed delta is 0 and write_gcsr_timertick(0) injects the timer interrupt immediately instead of after the remaining time. The guest observes a premature timer event right after migration.

The expired one-shot case (TVAL = -1) is already handled earlier, so only the non-expired one-shot path is affected. When expire has not been set (i.e. on the destination), reload the remaining countdown from the migrated TVAL. The regular preempt/resume path on the source, where expire is valid, is unchanged.

Fixes a5857b9ff6e0 ("LoongArch: KVM: Implement vcpu timer operations")
---
 arch/loongarch/kvm/timer.c | 8 ++++++++
 1 file changed, 8 insertions(+)
arch/loongarch/kvm/timer.c +8 -0
index 3829f35a4070..57f53d19a00a 100644
@@ -132,6 +132,14 @@ void kvm_restore_timer(struct kvm_vcpu *vcpu)
 		 * during injecting intr async
 		 */
 		kvm_queue_irq(vcpu, INT_TI);
+	} else if (!expire) {
+		/*
+		 * One-shot timer on the migration destination: vcpu->arch.expire
+		 * is host-internal and is not migrated, so it is still 0 here.
+		 * Reload the remaining countdown from the migrated TVAL instead
+		 * of firing the timer immediately.
+		 */
+		delta = ticks;
 	}
 
 	write_gcsr_timertick(delta);
Signature · 1 line
2.43.0
maintainer
#1 lore raw
Quoted 27 lines · Tao Cui

From: Tao Cui <[email protected]>

kvm_restore_timer() rebuilds the remaining timer countdown from vcpu->arch.expire, which is host-internal and is not part of the migrated vCPU state. On the migration destination it is still 0, so for a one-shot timer that has not expired yet the computed delta is 0 and write_gcsr_timertick(0) injects the timer interrupt immediately instead of after the remaining time. The guest observes a premature timer event right after migration.

The expired one-shot case (TVAL = -1) is already handled earlier, so only the non-expired one-shot path is affected. When expire has not been set (i.e. on the destination), reload the remaining countdown from the migrated TVAL. The regular preempt/resume path on the source, where expire is valid, is unchanged.

Fixes a5857b9ff6e0 ("LoongArch: KVM: Implement vcpu timer operations")
---
  arch/loongarch/kvm/timer.c | 8 ++++++++
  1 file changed, 8 insertions(+)
arch/loongarch/kvm/timer.c +0 -0
index 3829f35a4070..57f53d19a00a 100644

arch/loongarch/kvm/timer.c void kvm_restore_timer(struct kvm_vcpu *vcpu) · line 138

Quoted 7 lines
arch/loongarch/kvm/timer.c +4 -0
@@ -132,6 +132,14 @@ void kvm_restore_timer(struct kvm_vcpu *vcpu)
  		 * during injecting intr async
  		 */
  		kvm_queue_irq(vcpu, INT_TI);
+	} else if (!expire) {
+		/*
+		 * One-shot timer on the migration destination: vcpu->arch.expire
+		 * is host-internal and is not migrated, so it is still 0 here.

Good catch, I ever noticed this issue before however without good method. My previous method is to recalculate vcpu->arch.expire in function _kvm_setcsr() when LOONGARCH_CSR_TVAL is set and value of vcpu->arch.expire is 0.

Regards
Bibo Mao

Quoted 8 lines
arch/loongarch/kvm/timer.c +4 -0
+		 * Reload the remaining countdown from the migrated TVAL instead
+		 * of firing the timer immediately.
+		 */
+		delta = ticks;
  	}
 
  	write_gcsr_timertick(delta);
author
#2 lore raw
Quoted 25 lines · Bibo Mao
Quoted 22 lines · Tao Cui

From: Tao Cui <[email protected]>

kvm_restore_timer() rebuilds the remaining timer countdown from vcpu->arch.expire, which is host-internal and is not part of the migrated vCPU state. On the migration destination it is still 0, so for a one-shot timer that has not expired yet the computed delta is 0 and write_gcsr_timertick(0) injects the timer interrupt immediately instead of after the remaining time. The guest observes a premature timer event right after migration.

The expired one-shot case (TVAL = -1) is already handled earlier, so only the non-expired one-shot path is affected. When expire has not been set (i.e. on the destination), reload the remaining countdown from the migrated TVAL. The regular preempt/resume path on the source, where expire is valid, is unchanged.

Fixes a5857b9ff6e0 ("LoongArch: KVM: Implement vcpu timer operations")

---
  arch/loongarch/kvm/timer.c | 8 ++++++++

  1 file changed, 8 insertions(+)
Quoted 14 lines

arch/loongarch/kvm/timer.c void kvm_restore_timer(struct kvm_vcpu *vcpu) · line 138

Quoted 12 lines
arch/loongarch/kvm/timer.c +4 -0
index 3829f35a4070..57f53d19a00a 100644
@@ -132,6 +132,14 @@ void kvm_restore_timer(struct kvm_vcpu *vcpu)
            * during injecting intr async
            */
           kvm_queue_irq(vcpu, INT_TI);
+    } else if (!expire) {
+        /*
+         * One-shot timer on the migration destination: vcpu->arch.expire
+         * is host-internal and is not migrated, so it is still 0 here.

Good catch, I ever noticed this issue before however without good method. My previous method is to recalculate vcpu->arch.expire in function _kvm_setcsr() when LOONGARCH_CSR_TVAL is set and value of vcpu->arch.expire is 0.

Thanks for the review and for sharing your idea. I also considered recalculating expire in _kvm_setcsr() when TVAL is restored, but ended up keeping the change within kvm_restore_timer() to avoid touching the core CSR write path. Both approaches are functionally equivalent since the first save_timer() after restore will set expire correctly either way.

That said, if you prefer the _kvm_setcsr() approach, I'm happy to rework the patch accordingly.

Thanks,
Tao

Quoted 10 lines

Regards
Bibo Mao

Quoted 7 lines
arch/loongarch/kvm/timer.c +4 -0
+         * Reload the remaining countdown from the migrated TVAL instead
+         * of firing the timer immediately.
+         */
+        delta = ticks;
       }
         write_gcsr_timertick(delta);
 
maintainer
#3 lore raw
Quoted 42 lines · Tao Cui
Quoted 25 lines · Bibo Mao
Quoted 22 lines · Tao Cui

From: Tao Cui <[email protected]>

kvm_restore_timer() rebuilds the remaining timer countdown from vcpu->arch.expire, which is host-internal and is not part of the migrated vCPU state. On the migration destination it is still 0, so for a one-shot timer that has not expired yet the computed delta is 0 and write_gcsr_timertick(0) injects the timer interrupt immediately instead of after the remaining time. The guest observes a premature timer event right after migration.

The expired one-shot case (TVAL = -1) is already handled earlier, so only the non-expired one-shot path is affected. When expire has not been set (i.e. on the destination), reload the remaining countdown from the migrated TVAL. The regular preempt/resume path on the source, where expire is valid, is unchanged.

Fixes a5857b9ff6e0 ("LoongArch: KVM: Implement vcpu timer operations")
---
   arch/loongarch/kvm/timer.c | 8 ++++++++
   1 file changed, 8 insertions(+)
Quoted 14 lines

arch/loongarch/kvm/timer.c void kvm_restore_timer(struct kvm_vcpu *vcpu) · line 138

Quoted 12 lines
arch/loongarch/kvm/timer.c +4 -0
index 3829f35a4070..57f53d19a00a 100644
@@ -132,6 +132,14 @@ void kvm_restore_timer(struct kvm_vcpu *vcpu)
            * during injecting intr async
            */
           kvm_queue_irq(vcpu, INT_TI);
+    } else if (!expire) {
+        /*
+         * One-shot timer on the migration destination: vcpu->arch.expire
+         * is host-internal and is not migrated, so it is still 0 here.

Good catch, I ever noticed this issue before however without good method. My previous method is to recalculate vcpu->arch.expire in function _kvm_setcsr() when LOONGARCH_CSR_TVAL is set and value of vcpu->arch.expire is 0.

Quoted 8 lines

Thanks for the review and for sharing your idea. I also considered recalculating expire in _kvm_setcsr() when TVAL is restored, but ended up keeping the change within kvm_restore_timer() to avoid touching the core CSR write path. Both approaches are functionally equivalent since the first save_timer() after restore will set expire correctly either way.

That said, if you prefer the _kvm_setcsr() approach, I'm happy to

The problem in _kvm_setcsr() is that there is only register LOONGARCH_CSR_TVAL however no context of LOONGARCH_CSR_TCFG register, it seems that it is better in to restore it in kvm_restore_timer() function where all registers about CPU timer are set.

How about something like this, it should work with period timer mode when migration. @@ -119,6 +119,17 @@ void kvm_restore_timer(struct kvm_vcpu *vcpu)

        delta = 0;
        now = ktime_get();
        expire = vcpu->arch.expire;
+       if (!expire) {
+             /*
+              * The field vcpu->arch.expire is host-internal and is not
+              * migrated, so it is 0 after migration. Reload the
+              * remaining countdown from the migrated TVAL.
+              */

+ if (ticks < cfg)
+ delta = tick_to_ns(vcpu, ticks);
+ expire = ktime_add_ns(ktime_get(), delta);
+ }
+

Quoted 1 line

rework the patch accordingly.

ok, that is fine. Thanks for catching this issue.

Regards
Bibo Mao

Quoted 13 lines

Thanks,
Tao

Quoted 10 lines

Regards
Bibo Mao

Quoted 7 lines
arch/loongarch/kvm/timer.c +4 -0
+         * Reload the remaining countdown from the migrated TVAL instead
+         * of firing the timer immediately.
+         */
+        delta = ticks;
       }
         write_gcsr_timertick(delta);
 
author
#4 lore raw
Quoted 61 lines · Bibo Mao
Quoted 42 lines · Tao Cui
Quoted 25 lines · Bibo Mao
Quoted 22 lines · Tao Cui

From: Tao Cui <[email protected]>

kvm_restore_timer() rebuilds the remaining timer countdown from vcpu->arch.expire, which is host-internal and is not part of the migrated vCPU state. On the migration destination it is still 0, so for a one-shot timer that has not expired yet the computed delta is 0 and write_gcsr_timertick(0) injects the timer interrupt immediately instead of after the remaining time. The guest observes a premature timer event right after migration.

The expired one-shot case (TVAL = -1) is already handled earlier, so only the non-expired one-shot path is affected. When expire has not been set (i.e. on the destination), reload the remaining countdown from the migrated TVAL. The regular preempt/resume path on the source, where expire is valid, is unchanged.

Fixes a5857b9ff6e0 ("LoongArch: KVM: Implement vcpu timer operations")

---
   arch/loongarch/kvm/timer.c | 8 ++++++++

   1 file changed, 8 insertions(+)
Quoted 14 lines

arch/loongarch/kvm/timer.c void kvm_restore_timer(struct kvm_vcpu *vcpu) · line 138

Quoted 12 lines
arch/loongarch/kvm/timer.c +4 -0
index 3829f35a4070..57f53d19a00a 100644
@@ -132,6 +132,14 @@ void kvm_restore_timer(struct kvm_vcpu *vcpu)
             * during injecting intr async
             */
            kvm_queue_irq(vcpu, INT_TI);
+    } else if (!expire) {
+        /*
+         * One-shot timer on the migration destination: vcpu->arch.expire
+         * is host-internal and is not migrated, so it is still 0 here.

Good catch, I ever noticed this issue before however without good method. My previous method is to recalculate vcpu->arch.expire in function _kvm_setcsr() when LOONGARCH_CSR_TVAL is set and value of vcpu->arch.expire is 0.

Quoted 8 lines

Thanks for the review and for sharing your idea. I also considered recalculating expire in _kvm_setcsr() when TVAL is restored, but ended up keeping the change within kvm_restore_timer() to avoid touching the core CSR write path. Both approaches are functionally equivalent since the first save_timer() after restore will set expire correctly either way.

That said, if you prefer the _kvm_setcsr() approach, I'm happy to

The problem in _kvm_setcsr() is that there is only register LOONGARCH_CSR_TVAL however no context of LOONGARCH_CSR_TCFG register, it seems that it is better in to restore it in kvm_restore_timer() function where all registers about CPU timer are set.

How about something like this, it should work with period timer mode when migration.

(quoted diff) +1 -0
@@ -119,6 +119,17 @@ void kvm_restore_timer(struct kvm_vcpu *vcpu)
         delta = 0;
         now = ktime_get();
         expire = vcpu->arch.expire;
+       if (!expire) {

Review void kvm_restore_timer(struct kvm_vcpu *vcpu) · line 132

Quoted 10 lines
(quoted diff) +10 -0
+             /*
+              * The field vcpu->arch.expire is host-internal and is not
+              * migrated, so it is 0 after migration. Reload the
+              * remaining countdown from the migrated TVAL.
+              */
+               if (ticks < cfg)
+                       delta = tick_to_ns(vcpu, ticks);
+               expire = ktime_add_ns(ktime_get(), delta);
+       }
+

Thank you for the suggestion. I agree that kvm_restore_timer() is the right place since all timer registers (TCFG/TVAL) are available there.

I've updated the patch to compute expire from the migrated TVAL before the normal path, which also fixes the period timer case during migration. Suggested-by added.

Will send v2 shortly.

Thanks,
Tao

Quoted 19 lines
Quoted 1 line

rework the patch accordingly.

ok, that is fine. Thanks for catching this issue.

Regards
Bibo Mao

Quoted 13 lines

Thanks,
Tao

Quoted 10 lines

Regards
Bibo Mao

Quoted 7 lines
arch/loongarch/kvm/timer.c +4 -0
+         * Reload the remaining countdown from the migrated TVAL instead
+         * of firing the timer immediately.
+         */
+        delta = ticks;
        }
          write_gcsr_timertick(delta);