Skip to contentSkip to conversation

net/colo-compare.c: Expose "max_queue_size" to users and clean up

PATCH Superseded Zhang Chen 3 participants 13 messages Apr 11, 2020 – Jun 24, 2020 Networking

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Zhang Chen

author
#0 lore raw

cover net/colo-compare.c: Expose "max_queue_size" to users and clean up

From: Zhang Chen <[email protected]>

This series make a way to config COLO "max_queue_size" parameters according to user's scenarios and environments and do some clean up for descriptions.

Zhang Chen (2):
net/colo-compare.c: Expose compare "max_queue_size" to users qemu-options.hx: Clean up and fix typo for colo-compare

 net/colo-compare.c | 43 ++++++++++++++++++++++++++++++++++++++++++-
 qemu-options.hx    | 33 +++++++++++++++++----------------
 2 files changed, 59 insertions(+), 17 deletions(-)
Signature · 1 line
2.17.1
author
#1 lore raw

PATCH 1/2 net/colo-compare.c: Expose compare "max_queue_size" to users

From: Zhang Chen <[email protected]>

This patch allow users to set the "max_queue_size" according to their environment.

---
 net/colo-compare.c | 43 ++++++++++++++++++++++++++++++++++++++++++-
 qemu-options.hx    |  5 +++--
 2 files changed, 45 insertions(+), 3 deletions(-)
net/colo-compare.c +42 -1
index 10c0239f9d..def60ff2ea 100644
@@ -54,6 +54,7 @@ static NotifierList colo_compare_notifiers =
 static QemuMutex event_mtx;
 static QemuCond event_complete_cond;
 static int event_unhandled_count;
+static uint32_t max_queue_size;
 
 /*
  *  + CompareState ++
@@ -193,7 +194,7 @@ static void fill_pkt_tcp_info(void *data, uint32_t *max_ack)
  */
 static int colo_insert_packet(GQueue *queue, Packet *pkt, uint32_t *max_ack)
 {
-    if (g_queue_get_length(queue) <= MAX_QUEUE_SIZE) {
+    if (g_queue_get_length(queue) <= max_queue_size) {
         if (pkt->ip->ip_p == IPPROTO_TCP) {
             fill_pkt_tcp_info(pkt, max_ack);
             g_queue_insert_sorted(queue,
@@ -1051,6 +1052,37 @@ out:
     error_propagate(errp, local_err);
 }
 
+static void get_max_queue_size(Object *obj, Visitor *v,
+                               const char *name, void *opaque,
+                               Error **errp)
+{
+    uint32_t value = max_queue_size;
+
+    visit_type_uint32(v, name, &value, errp);
+}
+
+static void set_max_queue_size(Object *obj, Visitor *v,
+                               const char *name, void *opaque,
+                               Error **errp)
+{
+    Error *local_err = NULL;
+    uint32_t value;
+
+    visit_type_uint32(v, name, &value, &local_err);
+    if (local_err) {
+        goto out;
+    }
+    if (!value) {
+        error_setg(&local_err, "Property '%s.%s' requires a positive value",
+                   object_get_typename(obj), name);
+        goto out;
+    }
+    max_queue_size = value;
+
+out:
+    error_propagate(errp, local_err);
+}
+
 static void compare_pri_rs_finalize(SocketReadState *pri_rs)
 {
     CompareState *s = container_of(pri_rs, CompareState, pri_rs);
@@ -1167,6 +1199,11 @@ static void colo_compare_complete(UserCreatable *uc, Error **errp)
         s->expired_scan_cycle = REGULAR_PACKET_CHECK_MS;
     }
 
+    if (!max_queue_size) {
+        /* Set default queue size to 1024 */
+        max_queue_size = MAX_QUEUE_SIZE;
+    }
+
     if (find_and_check_chardev(&chr, s->pri_indev, errp) ||
         !qemu_chr_fe_init(&s->chr_pri_in, chr, errp)) {
         return;
@@ -1270,6 +1307,10 @@ static void colo_compare_init(Object *obj)
                         compare_get_expired_scan_cycle,
                         compare_set_expired_scan_cycle, NULL, NULL, NULL);
 
+    object_property_add(obj, "max_queue_size", "uint32",
+                        get_max_queue_size,
+                        set_max_queue_size, NULL, NULL, NULL);
+
     s->vnet_hdr = false;
     object_property_add_bool(obj, "vnet_hdr_support", compare_get_vnet_hdr,
                              compare_set_vnet_hdr, NULL);
qemu-options.hx +3 -2
index 16debd03cb..107ff3e71b 100644
@@ -4615,7 +4615,7 @@ SRST
         stored. The file format is libpcap, so it can be analyzed with
         tools such as tcpdump or Wireshark.
 
-    ``-object colo-compare,id=id,primary_in=chardevid,secondary_in=chardevid,outdev=chardevid,iothread=id[,vnet_hdr_support][,notify_dev=id][,compare_timeout=@var{ms}][,expired_scan_cycle=@var{ms}``
+    ``-object colo-compare,id=id,primary_in=chardevid,secondary_in=chardevid,outdev=chardevid,iothread=id[,vnet_hdr_support][,notify_dev=id][,compare_timeout=@var{ms}][,expired_scan_cycle=@var{ms}][,max_queue_size=@var{size}]``
         Colo-compare gets packet from primary\_inchardevid and
         secondary\_inchardevid, than compare primary packet with
         secondary packet. If the packets are same, we will output
@@ -4627,7 +4627,8 @@ SRST
         vnet\_hdr\_len. Then compare\_timeout=@var{ms} determines the
         maximum delay colo-compare wait for the packet.
         The expired\_scan\_cycle=@var{ms} to set the period of scanning
-        expired primary node network packets.
+        expired primary node network packets. The max\_queue\_size=@var{size}
+        is to set the max compare queue size depend on user environment.
         If you want to use Xen COLO, will need the notify\_dev to
         notify Xen colo-frame to do checkpoint.
 
Signature · 1 line
2.17.1
author
#2 lore raw

PATCH 2/2 qemu-options.hx: Clean up and fix typo for colo-compare

In reply to Zhang Chen

From: Zhang Chen <[email protected]>

Fix some typo and optimized some descriptions.

---
 qemu-options.hx | 32 ++++++++++++++++----------------
 1 file changed, 16 insertions(+), 16 deletions(-)
qemu-options.hx +16 -16
index 107ff3e71b..84368f15be 100644
@@ -4616,24 +4616,24 @@ SRST
         tools such as tcpdump or Wireshark.
 
     ``-object colo-compare,id=id,primary_in=chardevid,secondary_in=chardevid,outdev=chardevid,iothread=id[,vnet_hdr_support][,notify_dev=id][,compare_timeout=@var{ms}][,expired_scan_cycle=@var{ms}][,max_queue_size=@var{size}]``
-        Colo-compare gets packet from primary\_inchardevid and
-        secondary\_inchardevid, than compare primary packet with
-        secondary packet. If the packets are same, we will output
-        primary packet to outdevchardevid, else we will notify
-        colo-frame do checkpoint and send primary packet to
-        outdevchardevid. In order to improve efficiency, we need to put
-        the task of comparison in another thread. If it has the
-        vnet\_hdr\_support flag, colo compare will send/recv packet with
-        vnet\_hdr\_len. Then compare\_timeout=@var{ms} determines the
-        maximum delay colo-compare wait for the packet.
-        The expired\_scan\_cycle=@var{ms} to set the period of scanning
-        expired primary node network packets. The max\_queue\_size=@var{size}
-        is to set the max compare queue size depend on user environment.
-        If you want to use Xen COLO, will need the notify\_dev to
+        Colo-compare gets packet from primary\_in chardevid and
+        secondary\_in, then compare whether the payload of primary packet
+        and secondary packet are the same. If same, it will output
+        primary packet to out\_dev, else it will notify COLO-framework to do
+        checkpoint and send primary packet to out\_dev. In order to
+        improve efficiency, we need to put the task of comparison in
+        another iothread. If it has the vnet\_hdr\_support flag,
+        colo compare will send/recv packet with vnet\_hdr\_len.
+        The compare\_timeout=@var{ms} determines the maximum time of the
+        colo-compare hold the packet. The expired\_scan\_cycle=@var{ms}
+        is to set the period of scanning expired primary node network packets.
+        The max\_queue\_size=@var{size} is to set the max compare queue
+        size depend on user environment.
+        If user want to use Xen COLO, need to add the notify\_dev to
         notify Xen colo-frame to do checkpoint.
 
-        we must use it with the help of filter-mirror and
-        filter-redirector.
+        COLO-compare must be used with the help of filter-mirror,
+        filter-redirector and filter-rewriter.
 
         ::
 
Signature · 1 line
2.17.1
#3 lore raw
In reply to Zhang Chen

Hi Jason,

Please review this series when you free.

Thanks
Zhang Chen

Quoted 27 lines

-----Original Message-----
From: Zhang, Chen <[email protected]>
Sent: Saturday, April 11, 2020 11:38 AM
To: Jason Wang <[email protected]>; qemu-dev <qemu- [email protected]>
Cc: Zhang Chen <[email protected]>; Zhang, Chen <[email protected]>
Subject: [PATCH 0/2] net/colo-compare.c: Expose "max_queue_size" to users and clean up

From: Zhang Chen <[email protected]>

This series make a way to config COLO "max_queue_size" parameters according to user's scenarios and environments and do some clean up for descriptions.

Zhang Chen (2):
net/colo-compare.c: Expose compare "max_queue_size" to users qemu-options.hx: Clean up and fix typo for colo-compare

 net/colo-compare.c | 43
++++++++++++++++++++++++++++++++++++++++++-
 qemu-options.hx    | 33 +++++++++++++++++----------------
 2 files changed, 59 insertions(+), 17 deletions(-)

--
2.17.1

#4 lore raw
Quoted 7 lines · Chen

Hi Jason,

Please review this series when you free.

Thanks
Zhang Chen

Sure.

I wonder maybe it's better e.g you can review and collect the patches that looks good and send them to me periodically?

Thanks

#5 lore raw
Quoted 17 lines

-----Original Message-----
From: Jason Wang <[email protected]>
Sent: Thursday, April 23, 2020 4:54 PM
To: Zhang, Chen <[email protected]>; qemu-dev <qemu- [email protected]>
Cc: Zhang Chen <[email protected]>
Subject: Re: [PATCH 0/2] net/colo-compare.c: Expose "max_queue_size" to users and clean up

Quoted 4 lines · Chen

Hi Jason,

Please review this series when you free.

Quoted 8 lines
Quoted 3 lines

Thanks
Zhang Chen

Sure.

I wonder maybe it's better e.g you can review and collect the patches that looks good and send them to me periodically?

OK, I will queue COLO related patch as one series to you. Do I need send a pull request? or just a big patch set?

Thanks
Zhang Chen

Quoted 2 lines

Thanks

#6 lore raw
Quoted 19 lines · Chen
Quoted 19 lines

-----Original Message-----
From: Jason Wang <[email protected]>
Sent: Thursday, April 23, 2020 4:54 PM
To: Zhang, Chen <[email protected]>; qemu-dev <qemu- [email protected]>
Cc: Zhang Chen <[email protected]>
Subject: Re: [PATCH 0/2] net/colo-compare.c: Expose "max_queue_size" to users and clean up

Quoted 7 lines · Chen

Hi Jason,

Please review this series when you free.

Thanks
Zhang Chen

Quoted 6 lines
Quoted 4 lines

Sure.

I wonder maybe it's better e.g you can review and collect the patches that looks good and send them to me periodically?

OK, I will queue COLO related patch as one series to you. Do I need send a pull request? or just a big patch set?

I prefer big patch set.

Thanks

Quoted 5 lines

Thanks
Zhang Chen

Quoted 1 line

Thanks

#7 lore raw
Quoted 34 lines

-----Original Message-----
From: Jason Wang <[email protected]>
Sent: Thursday, April 23, 2020 5:07 PM
To: Zhang, Chen <[email protected]>; qemu-dev <qemu- [email protected]>
Cc: Zhang Chen <[email protected]>
Subject: Re: [PATCH 0/2] net/colo-compare.c: Expose "max_queue_size" to users and clean up

Quoted 21 lines · Chen
Quoted 21 lines

-----Original Message-----
From: Jason Wang <[email protected]>
Sent: Thursday, April 23, 2020 4:54 PM
To: Zhang, Chen <[email protected]>; qemu-dev <qemu- [email protected]>
Cc: Zhang Chen <[email protected]>
Subject: Re: [PATCH 0/2] net/colo-compare.c: Expose "max_queue_size" to users and clean up

Quoted 7 lines · Chen

Hi Jason,

Please review this series when you free.

Thanks
Zhang Chen

Sure.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

I wonder maybe it's better e.g you can review and collect the patches that looks good and send them to me periodically?

OK, I will queue COLO related patch as one series to you. Do I need send a pull request? or just a big patch set?

I prefer big patch set.

OK, I got it.

Thanks
Zhang Chen

Quoted 9 lines

Thanks

Quoted 5 lines

Thanks
Zhang Chen

Quoted 1 line

Thanks

#8 lore raw
In reply to Zhang Chen
Quoted 5 lines · Zhang Chen

From: Zhang Chen <[email protected]>

This series make a way to config COLO "max_queue_size" parameters according to user's scenarios and environments and do some clean up for descriptions.

Quoted 7 lines

Zhang Chen (2):
net/colo-compare.c: Expose compare "max_queue_size" to users qemu-options.hx: Clean up and fix typo for colo-compare

  net/colo-compare.c | 43 ++++++++++++++++++++++++++++++++++++++++++-
  qemu-options.hx    | 33 +++++++++++++++++----------------
  2 files changed, 59 insertions(+), 17 deletions(-)

Queued for 5.1.

Thanks

#9 lore raw
Quoted 10 lines · On Fri, 24 Apr 2020 10:19:48 +0800 Jason Wang <[email protected]> wrote:
Quoted 9 lines · Zhang Chen

From: Zhang Chen <[email protected]>

This series make a way to config COLO "max_queue_size" parameters according to user's scenarios and environments and do some clean up for descriptions.

Zhang Chen (2):
net/colo-compare.c: Expose compare "max_queue_size" to users qemu-options.hx: Clean up and fix typo for colo-compare

Quoted 10 lines
Quoted 3 lines
  net/colo-compare.c | 43 ++++++++++++++++++++++++++++++++++++++++++-
  qemu-options.hx    | 33 +++++++++++++++++----------------
  2 files changed, 59 insertions(+), 17 deletions(-)  

Queued for 5.1.

Thanks

Hi,
It looks like this hasn't been merged.

Regards,
Lukas Straub

#10 lore raw
Quoted 31 lines

-----Original Message-----
From: Lukas Straub <[email protected]>
Sent: Saturday, June 20, 2020 1:47 AM
To: Jason Wang <[email protected]>
Cc: Zhang, Chen <[email protected]>; qemu-dev <qemu- [email protected]>; Zhang Chen <[email protected]>
Subject: Re: [PATCH 0/2] net/colo-compare.c: Expose "max_queue_size" to users and clean up

Quoted 5 lines · On Fri, 24 Apr 2020 10:19:48 +0800 Jason Wang <[email protected]> wrote:
Quoted 4 lines · Zhang Chen

From: Zhang Chen <[email protected]>

This series make a way to config COLO "max_queue_size" parameters according to user's scenarios and environments and do some clean up for

descriptions.

Quoted 6 lines
Quoted 6 lines

Zhang Chen (2):
net/colo-compare.c: Expose compare "max_queue_size" to users qemu-options.hx: Clean up and fix typo for colo-compare

  net/colo-compare.c | 43

++++++++++++++++++++++++++++++++++++++++++-

Quoted 4 lines
Quoted 2 lines
  qemu-options.hx    | 33 +++++++++++++++++----------------
  2 files changed, 59 insertions(+), 17 deletions(-)
Quoted 8 lines
Quoted 5 lines

Queued for 5.1.

Thanks

Hi,
It looks like this hasn't been merged.

Thanks Lukas.
Hi Jason, I have double checked on upstream, looks missed this series.

Thanks
Zhang Chen

Quoted 3 lines

Regards,
Lukas Straub

#11 lore raw
Quoted 33 lines · Chen
Quoted 33 lines

-----Original Message-----
From: Lukas Straub <[email protected]>
Sent: Saturday, June 20, 2020 1:47 AM
To: Jason Wang <[email protected]>
Cc: Zhang, Chen <[email protected]>; qemu-dev <qemu- [email protected]>; Zhang Chen <[email protected]>
Subject: Re: [PATCH 0/2] net/colo-compare.c: Expose "max_queue_size" to users and clean up

Quoted 5 lines · On Fri, 24 Apr 2020 10:19:48 +0800 Jason Wang <[email protected]> wrote:
Quoted 4 lines · Zhang Chen

From: Zhang Chen <[email protected]>

This series make a way to config COLO "max_queue_size" parameters according to user's scenarios and environments and do some clean up for

descriptions.

Quoted 5 lines
Quoted 5 lines

Zhang Chen (2):
net/colo-compare.c: Expose compare "max_queue_size" to users qemu-options.hx: Clean up and fix typo for colo-compare

   net/colo-compare.c | 43

++++++++++++++++++++++++++++++++++++++++++-

Quoted 8 lines
Quoted 2 lines
   qemu-options.hx    | 33 +++++++++++++++++----------------
   2 files changed, 59 insertions(+), 17 deletions(-)

Queued for 5.1.

Thanks

Quoted 7 lines
Quoted 2 lines

Hi,
It looks like this hasn't been merged.

Thanks Lukas.
Hi Jason, I have double checked on upstream, looks missed this series.

Thanks
Zhang Chen

Right, but unfortunately it can't be applied cleanly on master.

Please send a new version.

Sorry.

Thanks

Quoted 3 lines
Quoted 3 lines

Regards,
Lukas Straub

#12 lore raw
Quoted 57 lines

-----Original Message-----
From: Jason Wang <[email protected]>
Sent: Tuesday, June 23, 2020 1:54 PM
To: Zhang, Chen <[email protected]>; Lukas Straub <[email protected]>
Cc: qemu-dev <[email protected]>; Zhang Chen <[email protected]>
Subject: Re: [PATCH 0/2] net/colo-compare.c: Expose "max_queue_size" to users and clean up

Quoted 34 lines · Chen
Quoted 34 lines

-----Original Message-----
From: Lukas Straub <[email protected]>
Sent: Saturday, June 20, 2020 1:47 AM
To: Jason Wang <[email protected]>
Cc: Zhang, Chen <[email protected]>; qemu-dev <qemu- [email protected]>; Zhang Chen <[email protected]>
Subject: Re: [PATCH 0/2] net/colo-compare.c: Expose "max_queue_size" to users and clean up

Quoted 6 lines · On Fri, 24 Apr 2020 10:19:48 +0800 Jason Wang <[email protected]> wrote:
Quoted 5 lines · Zhang Chen

From: Zhang Chen <[email protected]>

This series make a way to config COLO "max_queue_size" parameters according to user's scenarios and environments and do some clean up for

descriptions.

Quoted 5 lines
Quoted 5 lines

Zhang Chen (2):
net/colo-compare.c: Expose compare "max_queue_size" to users qemu-options.hx: Clean up and fix typo for colo-compare

   net/colo-compare.c | 43

++++++++++++++++++++++++++++++++++++++++++-

Quoted 8 lines
Quoted 2 lines
   qemu-options.hx    | 33 +++++++++++++++++----------------
   2 files changed, 59 insertions(+), 17 deletions(-)

Queued for 5.1.

Thanks

Quoted 7 lines
Quoted 2 lines

Hi,
It looks like this hasn't been merged.

Thanks Lukas.
Hi Jason, I have double checked on upstream, looks missed this series.

Thanks
Zhang Chen

Quoted 5 lines

Right, but unfortunately it can't be applied cleanly on master.

Please send a new version.

Sorry.

It's OK, I have send V2 version of this series, please merge it.

Thanks
Zhang Chen

Quoted 7 lines

Thanks

Quoted 3 lines
Quoted 3 lines

Regards,
Lukas Straub