Skip to contentSkip to conversation

tscdeadline_latency: Some fixes of hangs

PATCH New Peter Xu 1 participant 4 messages Jul 12, 2019 KVM

lore
Share & tools
lore

Peter Xu

author
#0 lore raw

v2 cover tscdeadline_latency: Some fixes of hangs

The previous version only contain patch 1 but when at it I noticed more ways to hang it... so let's fix them all up...

v2:
- use safe_halt() as suggested [Sean]
- added two more patches

Peter Xu (3):
tscdeadline_latency: Check condition first before loop
tscdeadline_latency: Limit size
tscdeadline_latency: Stop timer when reach max loop

 x86/tscdeadline_latency.c | 27 +++++++++++++++++----------
 1 file changed, 17 insertions(+), 10 deletions(-)
Signature · 1 line
2.21.0
author
#1 lore raw

PATCH v2 1/3 tscdeadline_latency: Check condition first before loop

This patch fixes a tscdeadline_latency hang when specifying a very small breakmax value. It's easily reproduced on my host when set breakmax to e.g. 10 TSC clocks.

The problem is test_tsc_deadline_timer() can be very slow because we've got printf() in there. So when reach the main loop we might have already triggered the IRQ handler for multiple times and we might have triggered the hitmax condition which will turn IRQ off. Then with no IRQ that first HLT instruction can last forever.

Fix this by don't enable irq and use safe_halt() as suggested by Sean Christopherson.

---
 x86/tscdeadline_latency.c | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)
x86/tscdeadline_latency.c +1 -4
index 0617a1b..3ad2950 100644
@@ -70,8 +70,6 @@ static void tsc_deadline_timer_isr(isr_regs_t *regs)
 static void start_tsc_deadline_timer(void)
 {
     handle_irq(TSC_DEADLINE_TIMER_VECTOR, tsc_deadline_timer_isr);
-    irq_enable();
-
     wrmsr(MSR_IA32_TSCDEADLINE, rdmsr(MSR_IA32_TSC)+delta);
     asm volatile ("nop");
 }
@@ -116,10 +114,9 @@ int main(int argc, char **argv)
     breakmax = argc <= 3 ? 0 : atol(argv[3]);
     printf("breakmax=%d\n", breakmax);
     test_tsc_deadline_timer();
-    irq_enable();
 
     do {
-        asm volatile("hlt");
+        safe_halt();
     } while (!hitmax && table_idx < size);
 
     for (i = 0; i < table_idx; i++) {
Signature · 1 line
2.21.0
author
#2 lore raw

PATCH v2 2/3 tscdeadline_latency: Limit size

In reply to Peter Xu

When size>TABLE_SIZE it can hang as well (say, 10001), because when we get TABLE_SIZE IRQs we'll stop increasing table_idx, then we'll never be able to get out of the main loop.

---
 x86/tscdeadline_latency.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
x86/tscdeadline_latency.c +5 -1
index 3ad2950..0a3532d 100644
@@ -112,7 +112,11 @@ int main(int argc, char **argv)
     delta = argc <= 1 ? 200000 : atol(argv[1]);
     size = argc <= 2 ? TABLE_SIZE : atol(argv[2]);
     breakmax = argc <= 3 ? 0 : atol(argv[3]);
-    printf("breakmax=%d\n", breakmax);
+
+    if (size >= TABLE_SIZE)
+	    size = TABLE_SIZE;
+    printf("delta=%d, size=%d, breakmax=%d\n", delta, size, breakmax);
+
     test_tsc_deadline_timer();
 
     do {
Signature · 1 line
2.21.0
author
#3 lore raw

PATCH v2 3/3 tscdeadline_latency: Stop timer when reach max loop

In reply to Peter Xu

This fixes another possible hang of this test when delay is configured too small (e.g., 2000) so that the IRQ handler will trigger forever.

Let's stop the timer if we've collected enough data so that the program can quit always.

---
 x86/tscdeadline_latency.c | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)
x86/tscdeadline_latency.c +11 -5
index 0a3532d..a63f98b 100644
@@ -50,18 +50,24 @@ int breakmax = 0;
 
 static void tsc_deadline_timer_isr(isr_regs_t *regs)
 {
-    u64 now = rdtsc();
+    u64 now = rdtsc(), delay;
     ++tdt_count;
 
-    if (table_idx < TABLE_SIZE && tdt_count > 1)
-        table[table_idx++] = now - exptime;
+    if (tdt_count == 1)
+        /* Skip the 1st IRQ */
+        goto setup_next;
 
-    if (breakmax && tdt_count > 1 && (now - exptime) > breakmax) {
-        hitmax = 1;
+    table[table_idx++] = delay = now - exptime;
+
+    /* Stop if either we finished collection or hit max */
+    if ((breakmax && delay > breakmax) || table_idx >= TABLE_SIZE) {
+        if (breakmax)
+            hitmax = 1;
         apic_write(APIC_EOI, 0);
         return;
     }
 
+setup_next:
     exptime = now+delta;
     wrmsr(MSR_IA32_TSCDEADLINE, now+delta);
     apic_write(APIC_EOI, 0);
Signature · 1 line
2.21.0