Skip to contentSkip to conversation

Zero copy improvements (QIOChannel + multifd)

PATCH Superseded Leonardo Bras 3 participants 11 messages Jun 28, 2022 – Jul 4, 2022

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

cover Zero copy improvements (QIOChannel + multifd)

The first patch avoid spuriously returning 1 [*] when zero-copy flush is attempted before any buffer was sent using MSG_ZEROCOPY.

[*] zero-copy not being used, even though it's enabled and supported by kernel

The second patch should be applied on top of Juan Quintela's patchset that reduces the frequency of multifd syncs, or else it could potentially send 20 warnings per second.

									...

Leonardo Bras (2):
QIOChannelSocket: Fix zero-copy flush returning code 1 when nothing
sent
migration/multifd: Warn user when zerocopy not working

 io/channel-socket.c | 8 +++++++-
 migration/multifd.c | 3 +++
 2 files changed, 10 insertions(+), 1 deletion(-)
Signature · 1 line
2.36.1
#1 lore raw

PATCH 1/2 QIOChannelSocket: Fix zero-copy flush returning code 1 when nothing sent

If flush is called when no buffer was sent with MSG_ZEROCOPY, it currently returns 1. This return code should be used only when Linux fails to use MSG_ZEROCOPY on a lot of sendmsg().

Fix this by returning early from flush if no sendmsg(...,MSG_ZEROCOPY) was attempted.

Fixes 2bc58ffc2926 ("QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX")
---
 io/channel-socket.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
io/channel-socket.c +7 -1
index 4466bb1cd4..698c086b70 100644
@@ -716,12 +716,18 @@ static int qio_channel_socket_flush(QIOChannel *ioc,
     struct cmsghdr *cm;
     char control[CMSG_SPACE(sizeof(*serr))];
     int received;
-    int ret = 1;
+    int ret;
+
+    if (!sioc->zero_copy_queued) {
+        return 0;
+    }
 
     msg.msg_control = control;
     msg.msg_controllen = sizeof(control);
     memset(control, 0, sizeof(control));
 
+    ret = 1;
+
     while (sioc->zero_copy_sent < sioc->zero_copy_queued) {
         received = recvmsg(sioc->fd, &msg, MSG_ERRQUEUE);
         if (received < 0) {
Signature · 1 line
2.36.1
#2 lore raw

PATCH 2/2 migration/multifd: Warn user when zerocopy not working

In reply to Leonardo Bras

Some errors, like the lack of Scatter-Gather support by the network interface(NETIF_F_SG) may cause sendmsg(...,MSG_ZEROCOPY) to fail on using zero-copy, which causes it to fall back to the default copying mechanism.

After each full dirty-bitmap scan there should be a zero-copy flush happening, which checks for errors each of the previous calls to sendmsg(...,MSG_ZEROCOPY). If all of them failed to use zero-copy, then warn the user about it.

Since it happens once each full dirty-bitmap scan, even in worst case scenario it should not print a lot of warnings, and will allow tracking how many dirty-bitmap iterations were not able to use zero-copy send.

---
 migration/multifd.c | 3 +++
 1 file changed, 3 insertions(+)
migration/multifd.c +3 -0
index 684c014c86..9c62aec84e 100644
@@ -624,6 +624,9 @@ int multifd_send_sync_main(QEMUFile *f)
             if (ret < 0) {
                 error_report_err(err);
                 return -1;
+            } else if (ret == 1) {
+                warn_report("The network device is not able to use "
+                            "zero-copy-send: copying is being used");
             }
         }
     }
Signature · 1 line
2.36.1
#3 lore raw

Re: [PATCH v1 2/2] migration/multifd: Warn user when zerocopy not working

Quoted 3 lines · Leonardo Bras

Some errors, like the lack of Scatter-Gather support by the network interface(NETIF_F_SG) may cause sendmsg(...,MSG_ZEROCOPY) to fail on using zero-copy, which causes it to fall back to the default copying mechanism.

How common is this lack of SG support ? What NICs did you have that were affected ?

Quoted 8 lines

After each full dirty-bitmap scan there should be a zero-copy flush happening, which checks for errors each of the previous calls to sendmsg(...,MSG_ZEROCOPY). If all of them failed to use zero-copy, then warn the user about it.

Since it happens once each full dirty-bitmap scan, even in worst case scenario it should not print a lot of warnings, and will allow tracking how many dirty-bitmap iterations were not able to use zero-copy send.

For long running migrations which are not converging, or converging very slowly there could be 100's of passes.

Quoted 22 lines
---
 migration/multifd.c | 3 +++
 1 file changed, 3 insertions(+)
migration/multifd.c +3 -0
index 684c014c86..9c62aec84e 100644
@@ -624,6 +624,9 @@ int multifd_send_sync_main(QEMUFile *f)
             if (ret < 0) {
                 error_report_err(err);
                 return -1;
+            } else if (ret == 1) {
+                warn_report("The network device is not able to use "
+                            "zero-copy-send: copying is being used");
             }
         }
     }
Signature · 1 line
2.36.1

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#4 lore raw

Re: [PATCH v1 2/2] migration/multifd: Warn user when zerocopy not working

Quoted 8 lines · Daniel P. Berrangé
Quoted 3 lines · Leonardo Bras

Some errors, like the lack of Scatter-Gather support by the network interface(NETIF_F_SG) may cause sendmsg(...,MSG_ZEROCOPY) to fail on using zero-copy, which causes it to fall back to the default copying mechanism.

How common is this lack of SG support ? What NICs did you have that were affected ?

I am not aware of any NIC without SG available for testing, nor have any idea on how common they are.
But since we can detect sendmsg() falling back to copying we should warn the user if this ever happens.

There is also a case in IPv6 related to fragmentation that may cause MSG_ZEROCOPY to fall back to the copying mechanism, so it's also covered.

Quoted 6 lines
Quoted 6 lines

After each full dirty-bitmap scan there should be a zero-copy flush happening, which checks for errors each of the previous calls to sendmsg(...,MSG_ZEROCOPY). If all of them failed to use zero-copy, then warn the user about it.

Quoted 7 lines
Quoted 3 lines

Since it happens once each full dirty-bitmap scan, even in worst case scenario it should not print a lot of warnings, and will allow tracking how many dirty-bitmap iterations were not able to use zero-copy send.

For long running migrations which are not converging, or converging very slowly there could be 100's of passes.

I could change it so it only warns once, if that is too much output.

Best regards,
Leo

Quoted 32 lines
Quoted 22 lines
---
 migration/multifd.c | 3 +++
 1 file changed, 3 insertions(+)
migration/multifd.c +3 -1
index 684c014c86..9c62aec84e 100644
@@ -624,6 +624,9 @@ int multifd_send_sync_main(QEMUFile *f)
             if (ret < 0) {
                 error_report_err(err);
                 return -1;
+            } else if (ret == 1) {
+                warn_report("The network device is not able to use "
+                            "zero-copy-send: copying is being used");
             }
         }
     }
--
Quoted 2 lines

2.36.1

With regards,
Daniel
--

|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#5 lore raw

Re: [PATCH v1 2/2] migration/multifd: Warn user when zerocopy not working

Quoted 29 lines · Leonardo Bras Soares Passos
Quoted 8 lines · Daniel P. Berrangé
Quoted 3 lines · Leonardo Bras

Some errors, like the lack of Scatter-Gather support by the network interface(NETIF_F_SG) may cause sendmsg(...,MSG_ZEROCOPY) to fail on using zero-copy, which causes it to fall back to the default copying mechanism.

How common is this lack of SG support ? What NICs did you have that were affected ?

I am not aware of any NIC without SG available for testing, nor have any idea on how common they are.
But since we can detect sendmsg() falling back to copying we should warn the user if this ever happens.

There is also a case in IPv6 related to fragmentation that may cause MSG_ZEROCOPY to fall back to the copying mechanism, so it's also covered.

Quoted 10 lines
Quoted 9 lines

After each full dirty-bitmap scan there should be a zero-copy flush happening, which checks for errors each of the previous calls to sendmsg(...,MSG_ZEROCOPY). If all of them failed to use zero-copy, then warn the user about it.

Since it happens once each full dirty-bitmap scan, even in worst case scenario it should not print a lot of warnings, and will allow tracking how many dirty-bitmap iterations were not able to use zero-copy send.

Quoted 5 lines
Quoted 3 lines

For long running migrations which are not converging, or converging very slowly there could be 100's of passes.

I could change it so it only warns once, if that is too much output.

Well I'm mostly wondering what we're expecting the user todo with this information. Generally a log file containing warnings ends up turning into a bug report. If we think it is important for users and/or mgmt apps to be aware of this info, then it might be better to actually put a field in the query-migrate stats to report if zero-copy is being honoured or not, and just have a trace point in this location instead.

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#6 lore raw

Re: [PATCH v1 2/2] migration/multifd: Warn user when zerocopy not working

Quoted 34 lines · * Daniel P. Berrangé ([email protected]) wrote:
Quoted 33 lines · Leonardo Bras Soares Passos
Quoted 8 lines · Daniel P. Berrangé
Quoted 3 lines · Leonardo Bras

Some errors, like the lack of Scatter-Gather support by the network interface(NETIF_F_SG) may cause sendmsg(...,MSG_ZEROCOPY) to fail on using zero-copy, which causes it to fall back to the default copying mechanism.

How common is this lack of SG support ? What NICs did you have that were affected ?

I am not aware of any NIC without SG available for testing, nor have any idea on how common they are.
But since we can detect sendmsg() falling back to copying we should warn the user if this ever happens.

There is also a case in IPv6 related to fragmentation that may cause MSG_ZEROCOPY to fall back to the copying mechanism, so it's also covered.

Quoted 13 lines
Quoted 9 lines

After each full dirty-bitmap scan there should be a zero-copy flush happening, which checks for errors each of the previous calls to sendmsg(...,MSG_ZEROCOPY). If all of them failed to use zero-copy, then warn the user about it.

Since it happens once each full dirty-bitmap scan, even in worst case scenario it should not print a lot of warnings, and will allow tracking how many dirty-bitmap iterations were not able to use zero-copy send.

For long running migrations which are not converging, or converging very slowly there could be 100's of passes.

Quoted 8 lines
Quoted 1 line

I could change it so it only warns once, if that is too much output.

Well I'm mostly wondering what we're expecting the user todo with this information. Generally a log file containing warnings ends up turning into a bug report. If we think it is important for users and/or mgmt apps to be aware of this info, then it might be better to actually put a field in the query-migrate stats to report if zero-copy is being honoured or not,

Yeh just a counter would work there I think.

Quoted 2 lines

and just have a trace point in this location
instead.

Yeh.

Dave

Quoted 7 lines

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
#7 lore raw

Re: [PATCH v1 2/2] migration/multifd: Warn user when zerocopy not working

Quoted 32 lines · Dr. David Alan Gilbert
Quoted 30 lines · * Daniel P. Berrangé ([email protected]) wrote:
Quoted 29 lines · Leonardo Bras Soares Passos
Quoted 8 lines · Daniel P. Berrangé
Quoted 3 lines · Leonardo Bras

Some errors, like the lack of Scatter-Gather support by the network interface(NETIF_F_SG) may cause sendmsg(...,MSG_ZEROCOPY) to fail on using zero-copy, which causes it to fall back to the default copying mechanism.

How common is this lack of SG support ? What NICs did you have that were affected ?

I am not aware of any NIC without SG available for testing, nor have any idea on how common they are.
But since we can detect sendmsg() falling back to copying we should warn the user if this ever happens.

There is also a case in IPv6 related to fragmentation that may cause MSG_ZEROCOPY to fall back to the copying mechanism, so it's also covered.

Quoted 10 lines
Quoted 9 lines

After each full dirty-bitmap scan there should be a zero-copy flush happening, which checks for errors each of the previous calls to sendmsg(...,MSG_ZEROCOPY). If all of them failed to use zero-copy, then warn the user about it.

Since it happens once each full dirty-bitmap scan, even in worst case scenario it should not print a lot of warnings, and will allow tracking how many dirty-bitmap iterations were not able to use zero-copy send.

Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

For long running migrations which are not converging, or converging very slowly there could be 100's of passes.

I could change it so it only warns once, if that is too much output.

Well I'm mostly wondering what we're expecting the user todo with this information.

My rationale on that:
- zero-copy-send is a feature that is supposed to improve send throughput by reducing cpu usage.
- there is a chance the sendmsg(MSG_ZEROCOPY) fails to use zero-copy
- if this happens, there will be a potential throughput decrease on sendmsg()
- the user (or management app) need to know when zero-copy-send is degrading throughput, so it can be disabled
- this is also important for performance testing, given it can be confusing having zero-copy-send improving throughput in some cases, and degrading in others, without any apparent reason why.

Quoted 7 lines
Quoted 5 lines

Generally a log file containing warnings ends up turning into a bug report. If we think it is important for users and/or mgmt apps to be aware of this info, then it might be better to actually put a field in the query-migrate stats to report if zero-copy is being honoured or not,

Yeh just a counter would work there I think.

The warning idea was totally due to my inexperience on this mgmt app interface, since I had no other idea on how to deal with that.

I think having it in query-migrate is a much better idea than a warning, since it should be much easier to parse and disable zero-copy-send if desired.
Even in my current qemu test script, it's much better having it in query-migrate.

Quoted 6 lines
Quoted 3 lines

and just have a trace point in this location
instead.

Yeh.

Yeap, the counter idea seems great!
Will it be always printed there, or only when zero-copy-send is enabled?

Best regards,
Leo

Quoted 12 lines

Dave

Quoted 7 lines

With regards,
Daniel
--

|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

#8 lore raw

Re: [PATCH v1 2/2] migration/multifd: Warn user when zerocopy not working

Quoted 61 lines · * Leonardo Bras Soares Passos ([email protected]) wrote:
Quoted 32 lines · Dr. David Alan Gilbert
Quoted 30 lines · * Daniel P. Berrangé ([email protected]) wrote:
Quoted 29 lines · Leonardo Bras Soares Passos
Quoted 8 lines · Daniel P. Berrangé
Quoted 3 lines · Leonardo Bras

Some errors, like the lack of Scatter-Gather support by the network interface(NETIF_F_SG) may cause sendmsg(...,MSG_ZEROCOPY) to fail on using zero-copy, which causes it to fall back to the default copying mechanism.

How common is this lack of SG support ? What NICs did you have that were affected ?

I am not aware of any NIC without SG available for testing, nor have any idea on how common they are.
But since we can detect sendmsg() falling back to copying we should warn the user if this ever happens.

There is also a case in IPv6 related to fragmentation that may cause MSG_ZEROCOPY to fall back to the copying mechanism, so it's also covered.

Quoted 10 lines
Quoted 9 lines

After each full dirty-bitmap scan there should be a zero-copy flush happening, which checks for errors each of the previous calls to sendmsg(...,MSG_ZEROCOPY). If all of them failed to use zero-copy, then warn the user about it.

Since it happens once each full dirty-bitmap scan, even in worst case scenario it should not print a lot of warnings, and will allow tracking how many dirty-bitmap iterations were not able to use zero-copy send.

Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

For long running migrations which are not converging, or converging very slowly there could be 100's of passes.

I could change it so it only warns once, if that is too much output.

Well I'm mostly wondering what we're expecting the user todo with this information.

My rationale on that:
- zero-copy-send is a feature that is supposed to improve send throughput by reducing cpu usage.
- there is a chance the sendmsg(MSG_ZEROCOPY) fails to use zero-copy
- if this happens, there will be a potential throughput decrease on sendmsg()
- the user (or management app) need to know when zero-copy-send is degrading throughput, so it can be disabled
- this is also important for performance testing, given it can be confusing having zero-copy-send improving throughput in some cases, and degrading in others, without any apparent reason why.

Quoted 6 lines
Quoted 5 lines

Generally a log file containing warnings ends up turning into a bug report. If we think it is important for users and/or mgmt apps to be aware of this info, then it might be better to actually put a field in the query-migrate stats to report if zero-copy is being honoured or not,

Quoted 4 lines
Quoted 1 line

Yeh just a counter would work there I think.

The warning idea was totally due to my inexperience on this mgmt app interface, since I had no other idea on how to deal with that.

Yeh it's not too silly an idea!
The way some of these warning or stats get to us can be a bit random, but sometimes can confuse things.

Quoted 7 lines

I think having it in query-migrate is a much better idea than a warning, since it should be much easier to parse and disable zero-copy-send if desired.
Even in my current qemu test script, it's much better having it in query-migrate.

Quoted 1 line
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

and just have a trace point in this location
instead.

Yeh.

Yeap, the counter idea seems great!
Will it be always printed there, or only when zero-copy-send is enabled?

You could make it either if it's enabled or if it's none zero. (I guess you want it to reset to 0 at the start of a new migration).

Dave

Quoted 17 lines

Best regards,
Leo

Quoted 12 lines

Dave

Quoted 7 lines

With regards,
Daniel
--

|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
#9 lore raw

Re: [PATCH v1 2/2] migration/multifd: Warn user when zerocopy not working

Quoted 95 lines · Dr. David Alan Gilbert
Quoted 72 lines · * Leonardo Bras Soares Passos ([email protected]) wrote:
Quoted 42 lines · Dr. David Alan Gilbert
Quoted 40 lines · * Daniel P. Berrangé ([email protected]) wrote:
Quoted 38 lines · Leonardo Bras Soares Passos
Quoted 11 lines · Daniel P. Berrangé
Quoted 6 lines · Leonardo Bras

Some errors, like the lack of Scatter-Gather support by the network
interface(NETIF_F_SG) may cause sendmsg(...,MSG_ZEROCOPY) to fail on using
zero-copy, which causes it to fall back to the default copying mechanism.

How common is this lack of SG support ? What NICs did you have that were affected ?

I am not aware of any NIC without SG available for testing, nor have any idea on how common they are.
But since we can detect sendmsg() falling back to copying we should warn the user if this ever happens.

There is also a case in IPv6 related to fragmentation that may cause MSG_ZEROCOPY to fall back to the copying mechanism, so it's also covered.

Quoted 15 lines
Quoted 14 lines

After each full dirty-bitmap scan there should be a zero-copy flush
happening, which checks for errors each of the previous calls to sendmsg(...,MSG_ZEROCOPY). If all of them failed to use zero-copy, then
warn the user about it.

Since it happens once each full dirty-bitmap scan, even in worst case
scenario it should not print a lot of warnings, and will allow tracking
how many dirty-bitmap iterations were not able to use zero-copy send.

Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

For long running migrations which are not converging, or converging very slowly there could be 100's of passes.

I could change it so it only warns once, if that is too much output.

Well I'm mostly wondering what we're expecting the user todo with this information.

My rationale on that:
- zero-copy-send is a feature that is supposed to improve send throughput by reducing cpu usage.
- there is a chance the sendmsg(MSG_ZEROCOPY) fails to use zero-copy
- if this happens, there will be a potential throughput decrease on sendmsg()
- the user (or management app) need to know when zero-copy-send is degrading throughput, so it can be disabled
- this is also important for performance testing, given it can be confusing having zero-copy-send improving throughput in some cases, and degrading in others, without any apparent reason why.

Quoted 6 lines
Quoted 5 lines

Generally a log file containing warnings ends up turning into a bug report. If we think it is important for users and/or mgmt apps to be aware of this info, then it might be better to actually put a field in the query-migrate stats to report if zero-copy is being honoured or not,

Quoted 4 lines
Quoted 1 line

Yeh just a counter would work there I think.

The warning idea was totally due to my inexperience on this mgmt app interface, since I had no other idea on how to deal with that.

Yeh it's not too silly an idea!
The way some of these warning or stats get to us can be a bit random, but sometimes can confuse things.

Quoted 13 lines

I think having it in query-migrate is a much better idea than a warning, since it should be much easier to parse and disable zero-copy-send if desired.
Even in my current qemu test script, it's much better having it in query-migrate.

Quoted 6 lines
Quoted 3 lines

and just have a trace point in this location
instead.

Yeh.

Quoted 7 lines
Quoted 2 lines

Yeap, the counter idea seems great!
Will it be always printed there, or only when zero-copy-send is enabled?

You could make it either if it's enabled or if it's none zero. (I guess you want it to reset to 0 at the start of a new migration).

Dave

Thanks for this feedback!

I have everything already working, but I am struggling with a good property name.

I am currently using zero_copy_copied (or zero-copy-copied in json), but it does not look like a good Migration stat name.

Do you have any suggestion?

Best regards,
Leo

Quoted 21 lines
Quoted 21 lines

Best regards,
Leo

Quoted 15 lines

Dave

Quoted 10 lines

With regards,
Daniel
--

Quoted 6 lines

: https://berrange.com      -o-
https://www.flickr.com/photos/dberrange :|
: https://libvirt.org         -o-
https://fstop138.berrange.com :|
: https://entangle-photo.org    -o-
https://www.instagram.com/dberrange :|

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

#10 lore raw

Re: [PATCH v1 2/2] migration/multifd: Warn user when zerocopy not working

Quoted 106 lines · * Leonardo Brás ([email protected]) wrote:
Quoted 95 lines · Dr. David Alan Gilbert
Quoted 72 lines · * Leonardo Bras Soares Passos ([email protected]) wrote:
Quoted 42 lines · Dr. David Alan Gilbert
Quoted 40 lines · * Daniel P. Berrangé ([email protected]) wrote:
Quoted 38 lines · Leonardo Bras Soares Passos
Quoted 11 lines · Daniel P. Berrangé
Quoted 6 lines · Leonardo Bras

Some errors, like the lack of Scatter-Gather support by the network
interface(NETIF_F_SG) may cause sendmsg(...,MSG_ZEROCOPY) to fail on using
zero-copy, which causes it to fall back to the default copying mechanism.

How common is this lack of SG support ? What NICs did you have that were affected ?

I am not aware of any NIC without SG available for testing, nor have any idea on how common they are.
But since we can detect sendmsg() falling back to copying we should warn the user if this ever happens.

There is also a case in IPv6 related to fragmentation that may cause MSG_ZEROCOPY to fall back to the copying mechanism, so it's also covered.

Quoted 15 lines
Quoted 14 lines

After each full dirty-bitmap scan there should be a zero-copy flush
happening, which checks for errors each of the previous calls to sendmsg(...,MSG_ZEROCOPY). If all of them failed to use zero-copy, then
warn the user about it.

Since it happens once each full dirty-bitmap scan, even in worst case
scenario it should not print a lot of warnings, and will allow tracking
how many dirty-bitmap iterations were not able to use zero-copy send.

Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

For long running migrations which are not converging, or converging very slowly there could be 100's of passes.

I could change it so it only warns once, if that is too much output.

Well I'm mostly wondering what we're expecting the user todo with this information.

My rationale on that:
- zero-copy-send is a feature that is supposed to improve send throughput by reducing cpu usage.
- there is a chance the sendmsg(MSG_ZEROCOPY) fails to use zero-copy
- if this happens, there will be a potential throughput decrease on sendmsg()
- the user (or management app) need to know when zero-copy-send is degrading throughput, so it can be disabled
- this is also important for performance testing, given it can be confusing having zero-copy-send improving throughput in some cases, and degrading in others, without any apparent reason why.

Quoted 6 lines
Quoted 5 lines

Generally a log file containing warnings ends up turning into a bug report. If we think it is important for users and/or mgmt apps to be aware of this info, then it might be better to actually put a field in the query-migrate stats to report if zero-copy is being honoured or not,

Quoted 4 lines
Quoted 1 line

Yeh just a counter would work there I think.

The warning idea was totally due to my inexperience on this mgmt app interface, since I had no other idea on how to deal with that.

Yeh it's not too silly an idea!
The way some of these warning or stats get to us can be a bit random, but sometimes can confuse things.

Quoted 13 lines

I think having it in query-migrate is a much better idea than a warning, since it should be much easier to parse and disable zero-copy-send if desired.
Even in my current qemu test script, it's much better having it in query-migrate.

Quoted 6 lines
Quoted 3 lines

and just have a trace point in this location
instead.

Yeh.

Quoted 7 lines
Quoted 2 lines

Yeap, the counter idea seems great!
Will it be always printed there, or only when zero-copy-send is enabled?

You could make it either if it's enabled or if it's none zero. (I guess you want it to reset to 0 at the start of a new migration).

Dave

Thanks for this feedback!

Quoted 7 lines

I have everything already working, but I am struggling with a good property name.

I am currently using zero_copy_copied (or zero-copy-copied in json), but it does not look like a good Migration stat name.

Do you have any suggestion?

Shrug; I'm not going to fuss over the name too much as long as it's reasonable. 'zero-copied' might be OK.

Dave

Quoted 26 lines

Best regards,
Leo

Quoted 21 lines
Quoted 21 lines

Best regards,
Leo

Quoted 15 lines

Dave

Quoted 10 lines

With regards,
Daniel
--

Quoted 6 lines

: https://berrange.com      -o-
https://www.flickr.com/photos/dberrange :|
: https://libvirt.org         -o-
https://fstop138.berrange.com :|
: https://entangle-photo.org    -o-
https://www.instagram.com/dberrange :|

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK