Skip to contentSkip to conversation

COLO project queued patches 20-Oct

PATCH Applied Zhang Chen 2 participants 12 messages Oct 16, 2020 – Oct 19, 2020 Networking

lore
Share & tools
lore

Zhang Chen

author
#0 lore raw

v2 cover COLO project queued patches 20-Oct

From: Zhang Chen <[email protected]>

Hi Jason, this series include latest COLO related patches. please check and merge it.

Li Zhijian (2):
colo-compare: fix missing compare_seq initialization
colo-compare: check mark in mutual exclusion

Pan Nengyuan (1):
net/filter-rewriter: destroy g_hash_table in colo_rewriter_cleanup

Rao, Lei (3):
Optimize seq_sorter function for colo-compare Reduce the time of checkpoint for COLO
Fix the qemu crash when guest shutdown in COLO mode

Zhang Chen (4):
net/colo-compare.c: Fix compare_timeout format issue net/colo-compare.c: Change the timer clock type net/colo-compare.c: Add secondary old packet detection net/colo-compare.c: Increase default queued packet scan frequency

 migration/ram.c       | 14 ++++++++++-
 net/colo-compare.c    | 58 ++++++++++++++++++++++---------------------
 net/colo.c            |  5 +---
 net/filter-rewriter.c |  2 ++
 softmmu/vl.c          |  1 +
 5 files changed, 47 insertions(+), 33 deletions(-)
Signature · 1 line
2.17.1
author
#1 lore raw

PATCH v2 1/10 net/filter-rewriter: destroy g_hash_table in colo_rewriter_cleanup

From: Pan Nengyuan <[email protected]>

s->connection_track_table forgot to destroy in colo_rewriter_cleanup. Fix it.

---
 net/filter-rewriter.c | 2 ++
 1 file changed, 2 insertions(+)
net/filter-rewriter.c +2 -0
index dc3c27a489..e063a818b7 100644
@@ -381,6 +381,8 @@ static void colo_rewriter_cleanup(NetFilterState *nf)
         filter_rewriter_flush(nf);
         g_free(s->incoming_queue);
     }
+
+    g_hash_table_destroy(s->connection_track_table);
 }
 
 static void colo_rewriter_setup(NetFilterState *nf, Error **errp)
Signature · 1 line
2.17.1
author
#2 lore raw

PATCH v2 2/10 Optimize seq_sorter function for colo-compare

In reply to Zhang Chen

From: "Rao, Lei" <[email protected]>

The seq of tcp has been filled in fill_pkt_tcp_info, it can be used directly here.

---
 net/colo-compare.c | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)
net/colo-compare.c +2 -5
index 3a45d64175..a35c10fb59 100644
@@ -194,13 +194,10 @@ static void colo_compare_inconsistency_notify(CompareState *s)
     }
 }
 
+/* Use restricted to colo_insert_packet() */
 static gint seq_sorter(Packet *a, Packet *b, gpointer data)
 {
-    struct tcp_hdr *atcp, *btcp;
-
-    atcp = (struct tcp_hdr *)(a->transport_header);
-    btcp = (struct tcp_hdr *)(b->transport_header);
-    return ntohl(atcp->th_seq) - ntohl(btcp->th_seq);
+    return a->tcp_seq - b->tcp_seq;
 }
 
 static void fill_pkt_tcp_info(void *data, uint32_t *max_ack)
Signature · 1 line
2.17.1
author
#3 lore raw

PATCH v2 3/10 Reduce the time of checkpoint for COLO

In reply to Zhang Chen

From: "Rao, Lei" <[email protected]>

we should set ram_bulk_stage to false after ram_state_init, otherwise the bitmap will be unused in migration_bitmap_find_dirty. all pages in ram cache will be flushed to the ram of secondary guest for each checkpoint.

---
 migration/ram.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)
migration/ram.c +13 -1
index 433489d633..9cfac3d9ba 100644
@@ -3009,6 +3009,18 @@ static void decompress_data_with_multi_threads(QEMUFile *f,
     qemu_mutex_unlock(&decomp_done_lock);
 }
 
+ /*
+  * we must set ram_bulk_stage to false, otherwise in
+  * migation_bitmap_find_dirty the bitmap will be unused and
+  * all the pages in ram cache wil be flushed to the ram of
+  * secondary VM.
+  */
+static void colo_init_ram_state(void)
+{
+    ram_state_init(&ram_state);
+    ram_state->ram_bulk_stage = false;
+}
+
 /*
  * colo cache: this is for secondary VM, we cache the whole
  * memory of the secondary VM, it is need to hold the global lock
@@ -3052,7 +3064,7 @@ int colo_init_ram_cache(void)
         }
     }
 
-    ram_state_init(&ram_state);
+    colo_init_ram_state();
     return 0;
 }
 
Signature · 1 line
2.17.1
author
#4 lore raw

PATCH v2 4/10 Fix the qemu crash when guest shutdown in COLO mode

In reply to Zhang Chen

From: "Rao, Lei" <[email protected]>

In COLO mode, if the startup parameters of QEMU include "no-shutdown", QEMU will crash when the guest shutdown. The root cause is when the guest shutdown, the state of VM will switch COLO to SHUTDOWN. When do checkpoint again, the state will be changed to COLO. But the state switch is undefined in runstate_transitions_def, we should add it. This patch fixes the following:
qemu-system-x86_64: invalid runstate transition: 'shutdown' -> 'colo' Aborted

---
 softmmu/vl.c | 1 +
 1 file changed, 1 insertion(+)
softmmu/vl.c +1 -0
index 5a11a62f78..bafe7c5b70 100644
@@ -632,6 +632,7 @@ static const RunStateTransition runstate_transitions_def[] = {
     { RUN_STATE_SHUTDOWN, RUN_STATE_PAUSED },
     { RUN_STATE_SHUTDOWN, RUN_STATE_FINISH_MIGRATE },
     { RUN_STATE_SHUTDOWN, RUN_STATE_PRELAUNCH },
+    { RUN_STATE_SHUTDOWN, RUN_STATE_COLO },
 
     { RUN_STATE_DEBUG, RUN_STATE_SUSPENDED },
     { RUN_STATE_RUNNING, RUN_STATE_SUSPENDED },
Signature · 1 line
2.17.1
author
#5 lore raw

PATCH v2 5/10 colo-compare: fix missing compare_seq initialization

In reply to Zhang Chen

From: Li Zhijian <[email protected]>

Fixes f449c9e549c ("colo: compare the packet based on the tcp sequence

number")

---
 net/colo.c | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)
net/colo.c +1 -4
index a6c66d829a..ef00609848 100644
@@ -133,14 +133,11 @@ void reverse_connection_key(ConnectionKey *key)
 
 Connection *connection_new(ConnectionKey *key)
 {
-    Connection *conn = g_slice_new(Connection);
+    Connection *conn = g_slice_new0(Connection);
 
     conn->ip_proto = key->ip_proto;
     conn->processing = false;
-    conn->offset = 0;
     conn->tcp_state = TCPS_CLOSED;
-    conn->pack = 0;
-    conn->sack = 0;
     g_queue_init(&conn->primary_list);
     g_queue_init(&conn->secondary_list);
 
Signature · 1 line
2.17.1
author
#6 lore raw

PATCH v2 6/10 colo-compare: check mark in mutual exclusion

In reply to Zhang Chen

From: Li Zhijian <[email protected]>

---
 net/colo-compare.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)
net/colo-compare.c +2 -4
index a35c10fb59..8d476bbd99 100644
@@ -477,13 +477,11 @@ sec:
             colo_release_primary_pkt(s, ppkt);
             g_queue_push_head(&conn->secondary_list, spkt);
             goto pri;
-        }
-        if (mark == COLO_COMPARE_FREE_SECONDARY) {
+        } else if (mark == COLO_COMPARE_FREE_SECONDARY) {
             conn->compare_seq = spkt->seq_end;
             packet_destroy(spkt, NULL);
             goto sec;
-        }
-        if (mark == (COLO_COMPARE_FREE_PRIMARY | COLO_COMPARE_FREE_SECONDARY)) {
+        } else if (mark == (COLO_COMPARE_FREE_PRIMARY | COLO_COMPARE_FREE_SECONDARY)) {
             conn->compare_seq = ppkt->seq_end;
             colo_release_primary_pkt(s, ppkt);
             packet_destroy(spkt, NULL);
Signature · 1 line
2.17.1
author
#7 lore raw

PATCH v2 7/10 net/colo-compare.c: Fix compare_timeout format issue

In reply to Zhang Chen

From: Zhang Chen <[email protected]>

This parameter need compare with the return of qemu_clock_get_ms(), it is uint64_t. So we need fix this issue here.

Fixes 9cc43c94b31 ("net/colo-compare.c: Expose "compare_timeout" to users")
---
 net/colo-compare.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
net/colo-compare.c +6 -6
index 8d476bbd99..76b83a9ca0 100644
@@ -120,7 +120,7 @@ struct CompareState {
     SendCo out_sendco;
     SendCo notify_sendco;
     bool vnet_hdr;
-    uint32_t compare_timeout;
+    uint64_t compare_timeout;
     uint32_t expired_scan_cycle;
 
     /*
@@ -1076,9 +1076,9 @@ static void compare_get_timeout(Object *obj, Visitor *v,
                                 Error **errp)
 {
     CompareState *s = COLO_COMPARE(obj);
-    uint32_t value = s->compare_timeout;
+    uint64_t value = s->compare_timeout;
 
-    visit_type_uint32(v, name, &value, errp);
+    visit_type_uint64(v, name, &value, errp);
 }
 
 static void compare_set_timeout(Object *obj, Visitor *v,
@@ -1141,9 +1141,9 @@ static void set_max_queue_size(Object *obj, Visitor *v,
                                Error **errp)
 {
     Error *local_err = NULL;
-    uint32_t value;
+    uint64_t value;
 
-    visit_type_uint32(v, name, &value, &local_err);
+    visit_type_uint64(v, name, &value, &local_err);
     if (local_err) {
         goto out;
     }
@@ -1391,7 +1391,7 @@ static void colo_compare_init(Object *obj)
     object_property_add_str(obj, "notify_dev",
                             compare_get_notify_dev, compare_set_notify_dev);
 
-    object_property_add(obj, "compare_timeout", "uint32",
+    object_property_add(obj, "compare_timeout", "uint64",
                         compare_get_timeout,
                         compare_set_timeout, NULL, NULL);
 
Signature · 1 line
2.17.1
author
#8 lore raw

PATCH v2 8/10 net/colo-compare.c: Change the timer clock type

In reply to Zhang Chen

From: Zhang Chen <[email protected]>

The virtual clock only runs during the emulation. It stops when the virtual machine is stopped.
The host clock should be used for device models that emulate accurate real time sources. It will continue to run when the virtual machine is suspended. COLO need to know the host time here.

Fixes dd321ecfc2e ("colo-compare: Use IOThread to Check old packet

regularly and Process packets of the primary")

---
 net/colo-compare.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
net/colo-compare.c +3 -3
index 76b83a9ca0..1263203e7f 100644
@@ -900,7 +900,7 @@ static void check_old_packet_regular(void *opaque)
 
     /* if have old packet we will notify checkpoint */
     colo_old_packet_check(s);
-    timer_mod(s->packet_check_timer, qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL) +
+    timer_mod(s->packet_check_timer, qemu_clock_get_ms(QEMU_CLOCK_HOST) +
               s->expired_scan_cycle);
 }
 
@@ -934,10 +934,10 @@ static void colo_compare_timer_init(CompareState *s)
 {
     AioContext *ctx = iothread_get_aio_context(s->iothread);
 
-    s->packet_check_timer = aio_timer_new(ctx, QEMU_CLOCK_VIRTUAL,
+    s->packet_check_timer = aio_timer_new(ctx, QEMU_CLOCK_HOST,
                                 SCALE_MS, check_old_packet_regular,
                                 s);
-    timer_mod(s->packet_check_timer, qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL) +
+    timer_mod(s->packet_check_timer, qemu_clock_get_ms(QEMU_CLOCK_HOST) +
               s->expired_scan_cycle);
 }
 
Signature · 1 line
2.17.1
author
#9 lore raw

PATCH v2 9/10 net/colo-compare.c: Add secondary old packet detection

In reply to Zhang Chen

From: Zhang Chen <[email protected]>

Detect queued secondary packet to sync VM state in time.

---
 net/colo-compare.c | 25 ++++++++++++++++---------
 1 file changed, 16 insertions(+), 9 deletions(-)
net/colo-compare.c +16 -9
index 1263203e7f..0c87fd9e33 100644
@@ -636,19 +636,26 @@ void colo_compare_unregister_notifier(Notifier *notify)
 static int colo_old_packet_check_one_conn(Connection *conn,
                                           CompareState *s)
 {
-    GList *result = NULL;
-
-    result = g_queue_find_custom(&conn->primary_list,
-                                 &s->compare_timeout,
-                                 (GCompareFunc)colo_old_packet_check_one);
+    if (!g_queue_is_empty(&conn->primary_list)) {
+        if (g_queue_find_custom(&conn->primary_list,
+                                &s->compare_timeout,
+                                (GCompareFunc)colo_old_packet_check_one))
+            goto out;
+    }
 
-    if (result) {
-        /* Do checkpoint will flush old packet */
-        colo_compare_inconsistency_notify(s);
-        return 0;
+    if (!g_queue_is_empty(&conn->secondary_list)) {
+        if (g_queue_find_custom(&conn->secondary_list,
+                                &s->compare_timeout,
+                                (GCompareFunc)colo_old_packet_check_one))
+            goto out;
     }
 
     return 1;
+
+out:
+    /* Do checkpoint will flush old packet */
+    colo_compare_inconsistency_notify(s);
+    return 0;
 }
 
 /*
Signature · 1 line
2.17.1
author
#10 lore raw

PATCH v2 10/10 net/colo-compare.c: Increase default queued packet scan frequency

In reply to Zhang Chen

From: Zhang Chen <[email protected]>

In my test, use this default parameter looks better.

---
 net/colo-compare.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
net/colo-compare.c +1 -1
index 0c87fd9e33..337025b44f 100644
@@ -52,7 +52,7 @@ static NotifierList colo_compare_notifiers =
 #define COLO_COMPARE_FREE_PRIMARY     0x01
 #define COLO_COMPARE_FREE_SECONDARY   0x02
 
-#define REGULAR_PACKET_CHECK_MS 3000
+#define REGULAR_PACKET_CHECK_MS 1000
 #define DEFAULT_TIME_OUT_MS 3000
 
 /* #define DEBUG_COLO_PACKETS */
Signature · 1 line
2.17.1
Applied
#11 lore raw
In reply to Zhang Chen
Quoted 23 lines · Zhang Chen

From: Zhang Chen <[email protected]>

Hi Jason, this series include latest COLO related patches. please check and merge it.

Li Zhijian (2):
colo-compare: fix missing compare_seq initialization
colo-compare: check mark in mutual exclusion

Pan Nengyuan (1):
net/filter-rewriter: destroy g_hash_table in colo_rewriter_cleanup

Rao, Lei (3):
Optimize seq_sorter function for colo-compare Reduce the time of checkpoint for COLO
Fix the qemu crash when guest shutdown in COLO mode

Zhang Chen (4):
net/colo-compare.c: Fix compare_timeout format issue net/colo-compare.c: Change the timer clock type net/colo-compare.c: Add secondary old packet detection net/colo-compare.c: Increase default queued packet scan frequency

Quoted 7 lines
  migration/ram.c       | 14 ++++++++++-
  net/colo-compare.c    | 58 ++++++++++++++++++++++---------------------
  net/colo.c            |  5 +---
  net/filter-rewriter.c |  2 ++
  softmmu/vl.c          |  1 +
  5 files changed, 47 insertions(+), 33 deletions(-)

Applied.

Thanks