Skip to contentSkip to conversation

xen/arm: avoid vtimer flip-flop transition in context switch

PATCH In review Wei Chen 2 participants 5 messages Jun 15, 2022 – Jun 17, 2022

lore
Share & tools
lore

Wei Chen

author
#0 lore raw

PATCH 1/1 xen/arm: avoid vtimer flip-flop transition in context switch

virt_vtimer_save is calculating the new time for the vtimer and has a potential risk of timer flip in:
"v->arch.virt_timer.cval + v->domain->arch.virt_timer_base.offset
- boot_count".
In this formula, "cval + offset" could make uint64_t overflow. Generally speaking, this is difficult to trigger. But unfortunately the problem was encountered with a platform where the timer started with a very huge initial value, like 0xF333899122223333. On this platform cval + offset is overflowing after running for a while.

So in this patch, we adjust the formula to use "offset - boot_count" first, and then use the result to plus cval. This will avoid the uint64_t overflow.

---
 xen/arch/arm/vtimer.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
xen/arch/arm/vtimer.c +3 -2
index 5bb5970f58..86e63303c8 100644
@@ -144,8 +144,9 @@ void virt_timer_save(struct vcpu *v)
     if ( (v->arch.virt_timer.ctl & CNTx_CTL_ENABLE) &&
          !(v->arch.virt_timer.ctl & CNTx_CTL_MASK))
     {
-        set_timer(&v->arch.virt_timer.timer, ticks_to_ns(v->arch.virt_timer.cval +
-                  v->domain->arch.virt_timer_base.offset - boot_count));
+        set_timer(&v->arch.virt_timer.timer,
+                  ticks_to_ns(v->domain->arch.virt_timer_base.offset -
+                              boot_count + v->arch.virt_timer.cval));
     }
 }
 
Signature · 1 line
2.25.1
#1 lore raw

Hi Wei,

Title: I don't quite understand what you mean by "flip-flop transition".

Quoted 9 lines · Wei Chen

virt_vtimer_save is calculating the new time for the vtimer and has a potential risk of timer flip in:
"v->arch.virt_timer.cval + v->domain->arch.virt_timer_base.offset
- boot_count".
In this formula, "cval + offset" could make uint64_t overflow. Generally speaking, this is difficult to trigger. But unfortunately the problem was encountered with a platform where the timer started with a very huge initial value, like 0xF333899122223333. On this platform cval + offset is overflowing after running for a while.

I am not sure how this is a problem. Yes, uint64_t will overflow with "cval + offset", but AFAIK the overall result will still be correct and not undefined.

Quoted 4 lines

So in this patch, we adjust the formula to use "offset - boot_count" first, and then use the result to plus cval. This will avoid the uint64_t overflow.

Technically, the overflow is still present because the (offset - boot_count) is a non-zero value *and* cval is a 64-bit value.

So I think the equation below should be reworked to...

Quoted 11 lines
---
  xen/arch/arm/vtimer.c | 5 +++--
  1 file changed, 3 insertions(+), 2 deletions(-)
xen/arch/arm/vtimer.c +0 -0
index 5bb5970f58..86e63303c8 100644

xen/arch/arm/vtimer.c void virt_timer_save(struct vcpu *v) · line 149

Quoted 8 lines
xen/arch/arm/vtimer.c +3 -2
@@ -144,8 +144,9 @@ void virt_timer_save(struct vcpu *v)
      if ( (v->arch.virt_timer.ctl & CNTx_CTL_ENABLE) &&
           !(v->arch.virt_timer.ctl & CNTx_CTL_MASK))
      {
-        set_timer(&v->arch.virt_timer.timer, ticks_to_ns(v->arch.virt_timer.cval +
-                  v->domain->arch.virt_timer_base.offset - boot_count));
+        set_timer(&v->arch.virt_timer.timer,
+                  ticks_to_ns(v->domain->arch.virt_timer_base.offset -
+                              boot_count + v->arch.virt_timer.cval));

... something like:

ticks_to_ns(offset - boot_count) + ticks_to_ns(cval);

The first part of the equation should always be the same. So it could be stored in struct domain.

Cheers,

Signature · 1 line
Julien Grall
author
#2 lore raw

Hi Julien,

Quoted 10 lines

-----Original Message-----
From: Julien Grall <[email protected]>
Sent: 2022年6月15日 17:47
To: Wei Chen <[email protected]>; [email protected]
Cc: nd <[email protected]>; Stefano Stabellini <[email protected]>; Bertrand Marquis <[email protected]>; Volodymyr Babchuk <[email protected]>
Subject: Re: [PATCH] xen/arm: avoid vtimer flip-flop transition in context switch

Quoted 4 lines

Hi Wei,

Title: I don't quite understand what you mean by "flip-flop transition".

Sorry for the no accurate words. I mean the time reaches to the MAX uint64_t and continue from 0. Maybe an "overflow" be better for this description.

Quoted 12 lines · Wei Chen
Quoted 10 lines

virt_vtimer_save is calculating the new time for the vtimer and has a potential risk of timer flip in:
"v->arch.virt_timer.cval + v->domain->arch.virt_timer_base.offset
- boot_count".
In this formula, "cval + offset" could make uint64_t overflow. Generally speaking, this is difficult to trigger.
But unfortunately
the problem was encountered with a platform where the timer started with a very huge initial value, like 0xF333899122223333. On this platform cval + offset is overflowing after running for a while.

Quoted 4 lines

I am not sure how this is a problem. Yes, uint64_t will overflow with "cval + offset", but AFAIK the overall result will still be correct and not undefined.

Yes, just as you said, even overflow, but the result is still correct. I had just noticed the overflow, but thought in a wrong way.

We have encountered this overflow in one RTOS guest:

PCNT:      0xf33ad45367e4a4ff
EL2_OFF:   0xf333333359e29a7f
BOOT_TICK: 0xf333333359e00000
VCNT:      0x0007a1200e020a80

If there is no timer in list, RTOS will calculate a huge number for "infinite wait", for example:
VCAL: 0x0ff7a1200e020a80
EL2_OFF + VCAL - BOOT_TICK = 0x032ad45367e4a4ff - 0xf333333359e00000 = 0xFF7A1200E04A4FF EL2_OFF - BOOT_TICK + VCAL = 0x29a7f + 0x0ff7a1200e020a80 = 0xFF7A1200E04A4FF

Quoted 8 lines
Quoted 4 lines

So in this patch, we adjust the formula to use "offset - boot_count" first, and then use the result to plus cval. This will avoid the uint64_t overflow.

Technically, the overflow is still present because the (offset - boot_count) is a non-zero value *and* cval is a 64-bit value.

Yes, GuestOS can issue any valid 64-bit value for their usage.

Quoted 25 lines

So I think the equation below should be reworked to...

Quoted 16 lines
---
  xen/arch/arm/vtimer.c | 5 +++--
  1 file changed, 3 insertions(+), 2 deletions(-)
xen/arch/arm/vtimer.c +0 -1
index 5bb5970f58..86e63303c8 100644
@@ -144,8 +144,9 @@ void virt_timer_save(struct vcpu *v)
      if ( (v->arch.virt_timer.ctl & CNTx_CTL_ENABLE) &&
           !(v->arch.virt_timer.ctl & CNTx_CTL_MASK))
      {
-        set_timer(&v->arch.virt_timer.timer, ticks_to_ns(v-

arch.virt_timer.cval +

xen/arch/arm/vtimer.c void virt_timer_save(struct vcpu *v) · line 149

Quoted 4 lines
xen/arch/arm/vtimer.c +3 -1
-                  v->domain->arch.virt_timer_base.offset - boot_count));
+        set_timer(&v->arch.virt_timer.timer,
+                  ticks_to_ns(v->domain->arch.virt_timer_base.offset -
+                              boot_count + v->arch.virt_timer.cval));

... something like:

Quoted 5 lines

ticks_to_ns(offset - boot_count) + ticks_to_ns(cval);

The first part of the equation should always be the same. So it could be stored in struct domain.

If you think there is still some values to continue this patch, I will address this comment : )

Thanks,
Wei Chen

Quoted 4 lines

Cheers,

--
Julien Grall

#3 lore raw
Quoted 1 line · Wei Chen

Hi Julien,

Hi Wei,

Quoted 45 lines
Quoted 10 lines

-----Original Message-----
From: Julien Grall <[email protected]>
Sent: 2022年6月15日 17:47
To: Wei Chen <[email protected]>; [email protected]
Cc: nd <[email protected]>; Stefano Stabellini <[email protected]>; Bertrand Marquis <[email protected]>; Volodymyr Babchuk <[email protected]>
Subject: Re: [PATCH] xen/arm: avoid vtimer flip-flop transition in context switch

Quoted 1 line
Quoted 7 lines
Quoted 3 lines

So in this patch, we adjust the formula to use "offset - boot_count" first, and then use the result to plus cval. This will avoid the uint64_t overflow.

Technically, the overflow is still present because the (offset - boot_count) is a non-zero value *and* cval is a 64-bit value.

Yes, GuestOS can issue any valid 64-bit value for their usage.

Quoted 25 lines

So I think the equation below should be reworked to...

Quoted 16 lines
---
   xen/arch/arm/vtimer.c | 5 +++--
   1 file changed, 3 insertions(+), 2 deletions(-)
xen/arch/arm/vtimer.c +0 -1
index 5bb5970f58..86e63303c8 100644
@@ -144,8 +144,9 @@ void virt_timer_save(struct vcpu *v)
       if ( (v->arch.virt_timer.ctl & CNTx_CTL_ENABLE) &&
            !(v->arch.virt_timer.ctl & CNTx_CTL_MASK))
       {
-        set_timer(&v->arch.virt_timer.timer, ticks_to_ns(v-

arch.virt_timer.cval +

xen/arch/arm/vtimer.c void virt_timer_save(struct vcpu *v) · line 149

Quoted 4 lines
xen/arch/arm/vtimer.c +3 -1
-                  v->domain->arch.virt_timer_base.offset - boot_count));
+        set_timer(&v->arch.virt_timer.timer,
+                  ticks_to_ns(v->domain->arch.virt_timer_base.offset -
+                              boot_count + v->arch.virt_timer.cval));

... something like:

Quoted 8 lines
Quoted 5 lines

ticks_to_ns(offset - boot_count) + ticks_to_ns(cval);

The first part of the equation should always be the same. So it could be stored in struct domain.

If you think there is still some values to continue this patch, I will address this comment : )

I think there are. This can be easily triggered by a vCPU setting a large cval.

Cheers,

Signature · 1 line
Julien Grall
author
#4 lore raw

Hi Julien,

Quoted 70 lines

-----Original Message-----
From: Julien Grall <[email protected]>
Sent: 2022年6月17日 16:32
To: Wei Chen <[email protected]>; [email protected]
Cc: nd <[email protected]>; Stefano Stabellini <[email protected]>; Bertrand Marquis <[email protected]>; Volodymyr Babchuk <[email protected]>
Subject: Re: [PATCH] xen/arm: avoid vtimer flip-flop transition in context switch

Quoted 1 line · Wei Chen

Hi Julien,

Hi Wei,

Quoted 5 lines
Quoted 5 lines

-----Original Message-----
From: Julien Grall <[email protected]>
Sent: 2022年6月15日 17:47
To: Wei Chen <[email protected]>; [email protected]
Cc: nd <[email protected]>; Stefano Stabellini <[email protected]>;

Bertrand

Quoted 3 lines
Quoted 3 lines

Marquis <[email protected]>; Volodymyr Babchuk <[email protected]>
Subject: Re: [PATCH] xen/arm: avoid vtimer flip-flop transition in

context

Quoted 20 lines
Quoted 9 lines

switch

Quoted 4 lines

So in this patch, we adjust the formula to use "offset - boot_count" first, and then use the result to plus cval. This will avoid the uint64_t overflow.

Technically, the overflow is still present because the (offset - boot_count) is a non-zero value *and* cval is a 64-bit value.

Yes, GuestOS can issue any valid 64-bit value for their usage.

Quoted 8 lines

So I think the equation below should be reworked to...

Quoted 6 lines
---
   xen/arch/arm/vtimer.c | 5 +++--
   1 file changed, 3 insertions(+), 2 deletions(-)
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
xen/arch/arm/vtimer.c +0 -1
index 5bb5970f58..86e63303c8 100644
@@ -144,8 +144,9 @@ void virt_timer_save(struct vcpu *v)
       if ( (v->arch.virt_timer.ctl & CNTx_CTL_ENABLE) &&
            !(v->arch.virt_timer.ctl & CNTx_CTL_MASK))
       {
-        set_timer(&v->arch.virt_timer.timer, ticks_to_ns(v-

arch.virt_timer.cval +
- v->domain->arch.virt_timer_base.offset -

boot_count));

Quoted 2 lines
Quoted 2 lines

xen/arch/arm/vtimer.c

Quoted 2 lines
xen/arch/arm/vtimer.c +2 -0
+        set_timer(&v->arch.virt_timer.timer,
+                  ticks_to_ns(v->domain->arch.virt_timer_base.offset

-

Quoted 7 lines
Quoted 7 lines
Quoted 1 line

+ boot_count + v->arch.virt_timer.cval));

... something like:

ticks_to_ns(offset - boot_count) + ticks_to_ns(cval);

The first part of the equation should always be the same. So it could

be

Quoted 9 lines
Quoted 5 lines
Quoted 2 lines

stored in struct domain.

If you think there is still some values to continue this patch, I will address this comment : )

I think there are. This can be easily triggered by a vCPU setting a large cval.

Ok, thanks! We will address it and refine the subject and descriptions.

Quoted 4 lines

Cheers,

--
Julien Grall