QIOChannelSocket: Reduce ifdefs to improve readability
PATCH v2 1/2 QIOChannelSocket: Reduce ifdefs to improve readability
During implementation of MSG_ZEROCOPY feature, a lot of #ifdefs were introduced, particularly at qio_channel_socket_writev().
Rewrite some of those changes so it's easier to read.
...
--- io/channel-socket.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
io/channel-socket.c +3 -3
@@ -554,6 +554,7 @@ static ssize_t qio_channel_socket_writev(QIOChannel *ioc,size_t fdsize =sizeof (int ) * nfds;struct cmsghdr *cmsg;int sflags =0 ; +bool zero_copy_enabled = false;memset (control,0 ,CMSG_SPACE (sizeof (int ) *SOCKET_MAX_FDS ));
@@ -581,6 +582,7 @@ static ssize_t qio_channel_socket_writev(QIOChannel *ioc,#ifdef QEMU_MSG_ZEROCOPY if (flags &QIO_CHANNEL_WRITE_FLAG_ZERO_COPY ) { sflags =MSG_ZEROCOPY ; + zero_copy_enabled = true; }#endif
@@ -592,15 +594,13 @@ static ssize_t qio_channel_socket_writev(QIOChannel *ioc,return QIO_CHANNEL_ERR_BLOCK ;case EINTR : goto retry; -#ifdef QEMU_MSG_ZEROCOPY case ENOBUFS : -if (sflags &MSG_ZEROCOPY ) { +if (zero_copy_enabled) {error_setg_errno (errp, errno,"Process can't lock enough memory for using MSG_ZEROCOPY" );return -1 ; }break ; -#endif } error_setg_errno(errp, errno,
Signature · 1 line
2.36.1
PATCH v2 2/2 QIOChannelSocket: Fix zero-copy send so socket flush works
Somewhere between v6 and v7 the of the zero-copy-send patchset a crucial part of the flushing mechanism got missing: incrementing zero_copy_queued.
Without that, the flushing interface becomes a no-op, and there is no guarantee the buffer is really sent.
This can go as bad as causing a corruption in RAM during migration.
--- io/channel-socket.c | 5 +++++ 1 file changed, 5 insertions(+)
io/channel-socket.c +5 -0
@@ -607,6 +607,11 @@ static ssize_t qio_channel_socket_writev(QIOChannel *ioc, "Unable to write to socket"); return -1; } + +if (zero_copy_enabled) { + sioc->zero_copy_queued++; + } +return ret; }#else /* WIN32 */
Signature · 1 line
2.36.1
Quoted 23 lines · Leonardo Bras
During implementation of
MSG_ZEROCOPYfeature, a lot of #ifdefs were introduced, particularly atqio_channel_socket_writev().Rewrite some of those changes so it's easier to read.
...Signed-off-by Leonardo Bras <[email protected]>--- io/channel-socket.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)io/channel-socket.c +1
-0@@ -554,6 +554,7 @@ static ssize_t qio_channel_socket_writev(QIOChannel *ioc,size_t fdsize =sizeof (int ) * nfds;struct cmsghdr *cmsg;int sflags =0 ; +bool zero_copy_enabled = false;memset (control,0 ,CMSG_SPACE (sizeof (int ) *SOCKET_MAX_FDS ));
io/channel-socket.c static ssize_t qio_channel_socket_writev(QIOChannel *ioc, · line 586
Quoted 5 lines
io/channel-socket.c +1
-0@@ -581,6 +582,7 @@ static ssize_t qio_channel_socket_writev(QIOChannel *ioc,#ifdef QEMU_MSG_ZEROCOPY if (flags &QIO_CHANNEL_WRITE_FLAG_ZERO_COPY ) { sflags =MSG_ZEROCOPY ; + zero_copy_enabled = true; }
There should be a
#else
error_setg(errp, "Zero copy not supported on this platform");
return -1;#endif
io/channel-socket.c static ssize_t qio_channel_socket_writev(QIOChannel *ioc, · line 598
Quoted 10 lines
io/channel-socket.c +1
-2#endif @@ -592,15 +594,13 @@ static ssize_t qio_channel_socket_writev(QIOChannel *ioc,return QIO_CHANNEL_ERR_BLOCK ;case EINTR : goto retry; -#ifdef QEMU_MSG_ZEROCOPY case ENOBUFS : -if (sflags &MSG_ZEROCOPY ) { +if (zero_copy_enabled) {
if (flags & QIO_CHANNEL_WRITE_FLAG_ZERO_COPY)
avoids the #ifdef without needing to add yet another variable expressing what's already expressed in both 'flags' and 'sflags'.
Quoted 12 lines
io/channel-socket.c +0
-1error_setg_errno (errp, errno,"Process can't lock enough memory for using MSG_ZEROCOPY" );return -1 ; }break ; -#endif } error_setg_errno(errp, errno,Signature · 1 line
2.36.1
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Hello Daniel,
Quoted 33 lines · Daniel P. Berrangé
Quoted 23 lines · Leonardo Bras
During implementation of
MSG_ZEROCOPYfeature, a lot of #ifdefs were introduced, particularly atqio_channel_socket_writev().Rewrite some of those changes so it's easier to read.
...Signed-off-by Leonardo Bras <[email protected]>--- io/channel-socket.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)io/channel-socket.c +1
-0@@ -554,6 +554,7 @@ static ssize_t qio_channel_socket_writev(QIOChannel *ioc,size_t fdsize =sizeof (int ) * nfds;struct cmsghdr *cmsg;int sflags =0 ; +bool zero_copy_enabled = false;memset (control,0 ,CMSG_SPACE (sizeof (int ) *SOCKET_MAX_FDS ));io/channel-socket.c static ssize_t qio_channel_socket_writev(QIOChannel *ioc, · line 586
Quoted 5 lines
io/channel-socket.c +1
-0@@ -581,6 +582,7 @@ static ssize_t qio_channel_socket_writev(QIOChannel *ioc,#ifdef QEMU_MSG_ZEROCOPY if (flags &QIO_CHANNEL_WRITE_FLAG_ZERO_COPY ) { sflags =MSG_ZEROCOPY ; + zero_copy_enabled = true; }There should be a
Quoted 5 lines
#else error_setg(errp, "Zero copy not supported on this platform"); return -1;#endif
IIUC, if done as suggested, it will break every non-zero-copy call of qio_channel_socket_writev();
I think you are suggesting something like :
if (flags & QIO_CHANNEL_WRITE_FLAG_ZERO_COPY) {#ifdef QEMU_MSG_ZEROCOPY
sflags = MSG_ZEROCOPY;
zero_copy_enabled = true; // I know you suggested this out,
just for example purposes
#else
error_setg(errp, "Zero copy not supported on this platform");
return -1;
#endif
}Which is supposed to fail if QIO_CHANNEL_WRITE_FLAG_ZERO_COPY is specified, but qemu does not support it at compile time.
If I get the part above correctly, it would not be necessary, as qio_channel_socket_writev() is
called only by qio_channel_writev_full(), which tests:
if ((flags & QIO_CHANNEL_WRITE_FLAG_ZERO_COPY) &&
!qio_channel_has_feature(ioc, QIO_CHANNEL_FEATURE_WRITE_ZERO_COPY)) {
error_setg_errno(errp, EINVAL,
"Requested Zero Copy feature is not available");
return -1;
}and QIO_CHANNEL_FEATURE_WRITE_ZERO_COPY is only set in qio_channel_socket_connect_sync(), and is conditional to QEMU_MSG_ZEROCOPY being enabled during compile time. Meaning it's the same test as before mentioned, but
failing earlier.
Quoted 13 lines
io/channel-socket.c static ssize_t qio_channel_socket_writev(QIOChannel *ioc, · line 598
Quoted 10 lines
io/channel-socket.c +1
-2#endif @@ -592,15 +594,13 @@ static ssize_t qio_channel_socket_writev(QIOChannel *ioc,return QIO_CHANNEL_ERR_BLOCK ;case EINTR : goto retry; -#ifdef QEMU_MSG_ZEROCOPY case ENOBUFS : -if (sflags &MSG_ZEROCOPY ) { +if (zero_copy_enabled) {if (flags &
QIO_CHANNEL_WRITE_FLAG_ZERO_COPY)
Quoted 3 lines
avoids the #ifdef without needing to add yet another variable expressing what's already expressed in both 'flags' and 'sflags'.
Yes, it does, but at the cost of not compiling-out the zero-copy part when it's not supported,
since the QIO_CHANNEL_WRITE_FLAG_ZERO_COPY comes as a parameter. This ends up meaning there will be at least one extra test for every time this function is called (the one in the next patch).
An option would be testing sflags & MSG_ZEROCOPY, which would compile-out zero-copy code
if it's not supported, but there was a bug in some distros where MSG_ZEROCOPY is not defined,
causing the build to fail.
I understand the idea of reusing those variables instead of creating a new one, but this boolean
variable will most certainly be compiled-out in this function, and will allow compiling out the
zero-copy code where it's not supported.
Best regards,
Leo
Quoted 21 lines
Quoted 11 lines
io/channel-socket.c +0
-2error_setg_errno (errp, errno,"Process can't lock enough memory for using MSG_ZEROCOPY" );return -1 ; }break ; -#endif } error_setg_errno(errp, errno, --Quoted 2 lines
2.36.1
With regards,
Daniel
--|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Quoted 52 lines · Leonardo Bras Soares Passos
Hello Daniel,
Quoted 33 lines · Daniel P. Berrangé
Quoted 23 lines · Leonardo Bras
During implementation of
MSG_ZEROCOPYfeature, a lot of #ifdefs were introduced, particularly atqio_channel_socket_writev().Rewrite some of those changes so it's easier to read.
...Signed-off-by Leonardo Bras <[email protected]>--- io/channel-socket.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)io/channel-socket.c +1
-0@@ -554,6 +554,7 @@ static ssize_t qio_channel_socket_writev(QIOChannel *ioc,size_t fdsize =sizeof (int ) * nfds;struct cmsghdr *cmsg;int sflags =0 ; +bool zero_copy_enabled = false;memset (control,0 ,CMSG_SPACE (sizeof (int ) *SOCKET_MAX_FDS ));io/channel-socket.c static ssize_t qio_channel_socket_writev(QIOChannel *ioc, · line 586
Quoted 5 lines
io/channel-socket.c +1
-0@@ -581,6 +582,7 @@ static ssize_t qio_channel_socket_writev(QIOChannel *ioc,#ifdef QEMU_MSG_ZEROCOPY if (flags &QIO_CHANNEL_WRITE_FLAG_ZERO_COPY ) { sflags =MSG_ZEROCOPY ; + zero_copy_enabled = true; }There should be a
Quoted 5 lines
#else error_setg(errp, "Zero copy not supported on this platform"); return -1;#endif
IIUC, if done as suggested, it will break every non-zero-copy call of
qio_channel_socket_writev();I think you are suggesting something like :
if (flags & QIO_CHANNEL_WRITE_FLAG_ZERO_COPY) {#ifdef
QEMU_MSG_ZEROCOPY
sflags =MSG_ZEROCOPY;
zero_copy_enabled= true; // I know you suggested this out,
just for example purposes
Quoted 5 lines
#else
error_setg(errp, "Zero copy not supported on this platform");
return -1;
#endif
}Yes, that is what I mean.
Quoted 3 lines
Which is supposed to fail if
QIO_CHANNEL_WRITE_FLAG_ZERO_COPYis specified, but qemu does not support it at compile time.
Correct, the caller should have checked the ZERO_COPY feeature when they first opened the channel, and if they none the less pass ZERO_COPY when it isn't supported that is a programmer error that needs reporting.
Quoted 10 lines
If I get the part above correctly, it would not be necessary, as
qio_channel_socket_writev()is
called only byqio_channel_writev_full(), which tests:if ((flags & QIO_CHANNEL_WRITE_FLAG_ZERO_COPY) && !qio_channel_has_feature(ioc, QIO_CHANNEL_FEATURE_WRITE_ZERO_COPY)) { error_setg_errno(errp, EINVAL, "Requested Zero Copy feature is not available"); return -1; }
Ok, so if it is checked earlier then we merely need an assert.
if (flags & QIO_CHANNEL_WRITE_FLAG_ZERO_COPY) {
#ifdef QEMU_MSG_ZEROCOPY
sflags = MSG_ZEROCOPY;
zero_copy_enabled = true;
#else
g_assert_unreachable();#endif
Quoted 19 lines
}Quoted 11 lines
io/channel-socket.c static ssize_t qio_channel_socket_writev(QIOChannel *ioc, · line 598
Quoted 8 lines
io/channel-socket.c +1
-2@@ -592,15 +594,13 @@ static ssize_t qio_channel_socket_writev(QIOChannel *ioc,return QIO_CHANNEL_ERR_BLOCK ;case EINTR : goto retry; -#ifdef QEMU_MSG_ZEROCOPY case ENOBUFS : -if (sflags &MSG_ZEROCOPY ) { +if (zero_copy_enabled) {if (flags &
QIO_CHANNEL_WRITE_FLAG_ZERO_COPY)Quoted 3 lines
avoids the #ifdef without needing to add yet another variable expressing what's already expressed in both 'flags' and 'sflags'.
Quoted 5 lines
Yes, it does, but at the cost of not compiling-out the zero-copy part when it's not supported,
since theQIO_CHANNEL_WRITE_FLAG_ZERO_COPYcomes as a parameter. This ends up meaning there will be at least one extra test for every time this function is called (the one in the next patch).
The cost of a simple bit test is between negligible-and-non-existant with branch prediction. I doubt it would be possible to even measure it.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Quoted 1 line · Daniel P. Berrangé
[...]
Quoted 10 lines
Ok, so if it is checked earlier then we merely need an assert.
if (flags & QIO_CHANNEL_WRITE_FLAG_ZERO_COPY) { #ifdef QEMU_MSG_ZEROCOPY sflags = MSG_ZEROCOPY; zero_copy_enabled = true; #else g_assert_unreachable();#endif
Quoted 1 line
}
Ok, I will add that in the next version.
Quoted 24 lines
Quoted 18 lines
Quoted 14 lines
io/channel-socket.c static ssize_t qio_channel_socket_writev(QIOChannel *ioc, · line 598
Quoted 11 lines
io/channel-socket.c +1
-2@@ -592,15 +594,13 @@ static ssize_t qio_channel_socket_writev(QIOChannel *ioc,return QIO_CHANNEL_ERR_BLOCK ;case EINTR : goto retry; -#ifdef QEMU_MSG_ZEROCOPY case ENOBUFS : -if (sflags &MSG_ZEROCOPY ) { +if (zero_copy_enabled) {if (flags &
QIO_CHANNEL_WRITE_FLAG_ZERO_COPY)Quoted 3 lines
avoids the #ifdef without needing to add yet another variable expressing what's already expressed in both 'flags' and 'sflags'.
Quoted 5 lines
Yes, it does, but at the cost of not compiling-out the zero-copy part when it's not supported,
since theQIO_CHANNEL_WRITE_FLAG_ZERO_COPYcomes as a parameter. This ends up meaning there will be at least one extra test for every time this function is called (the one in the next patch).
Quoted 3 lines
The cost of a simple bit test is between negligible-and-non-existant with branch prediction. I doubt it would be possible to even measure it.
Yeah, you are probably right on that.
So the main learning point here is that it's not worth creating a new boolean for compiling-out
code that should not impact performance ?
I mean, if performance-wise they should be the same, then a new variable would be just a
bother for the programmer.
Best regards,
Leo
Quoted 8 lines
With regards,
Daniel
--|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Quoted 42 lines · Leonardo Bras Soares Passos
Quoted 1 line · Daniel P. Berrangé
[...]
Quoted 10 lines
Ok, so if it is checked earlier then we merely need an assert.
if (flags & QIO_CHANNEL_WRITE_FLAG_ZERO_COPY) { #ifdef QEMU_MSG_ZEROCOPY sflags = MSG_ZEROCOPY; zero_copy_enabled = true; #else g_assert_unreachable();#endif
Quoted 1 line
}Ok, I will add that in the next version.
Quoted 24 lines
Quoted 23 lines
Quoted 14 lines
io/channel-socket.c static ssize_t qio_channel_socket_writev(QIOChannel *ioc, · line 598
Quoted 11 lines
io/channel-socket.c +1
-2@@ -592,15 +594,13 @@ static ssize_t qio_channel_socket_writev(QIOChannel *ioc,return QIO_CHANNEL_ERR_BLOCK ;case EINTR : goto retry; -#ifdef QEMU_MSG_ZEROCOPY case ENOBUFS : -if (sflags &MSG_ZEROCOPY ) { +if (zero_copy_enabled) {if (flags &
QIO_CHANNEL_WRITE_FLAG_ZERO_COPY)Quoted 3 lines
avoids the #ifdef without needing to add yet another variable expressing what's already expressed in both 'flags' and 'sflags'.
Yes, it does, but at the cost of not compiling-out the zero-copy part when it's not supported,
since theQIO_CHANNEL_WRITE_FLAG_ZERO_COPYcomes as a parameter. This ends up meaning there will be at least one extra test for every time this function is called (the one in the next patch).
Quoted 8 lines
Quoted 3 lines
The cost of a simple bit test is between negligible-and-non-existant with branch prediction. I doubt it would be possible to even measure it.
Yeah, you are probably right on that.
So the main learning point here is that it's not worth creating a new boolean for compiling-out
code that should not impact performance ?
As ever "it depends" so there's no hard rule, and sometimes it can verge on bikeshed colouring :-)
I didn't like the variable in this case, because it introduces a 3rd variable to the method for representing whether zero copy is need, which is excessive. I'm not a fan of redundancy as it can often then lead to inconsistency. So it would need a compelling reason why it is better, which is difficult for such a simple method. If the code was more complex, a variable might have benefit of clarity, but in this case IMHO it was just overkill.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Quoted 56 lines · Daniel P. Berrangé
Quoted 42 lines · Leonardo Bras Soares Passos
Quoted 1 line · Daniel P. Berrangé
[...]
Quoted 10 lines
Ok, so if it is checked earlier then we merely need an assert.
if (flags & QIO_CHANNEL_WRITE_FLAG_ZERO_COPY) { #ifdef QEMU_MSG_ZEROCOPY sflags = MSG_ZEROCOPY; zero_copy_enabled = true; #else g_assert_unreachable();#endif
Quoted 1 line
}Ok, I will add that in the next version.
Quoted 24 lines
Quoted 23 lines
Quoted 14 lines
io/channel-socket.c static ssize_t qio_channel_socket_writev(QIOChannel *ioc, · line 598
Quoted 11 lines
io/channel-socket.c +1
-2@@ -592,15 +594,13 @@ static ssize_t qio_channel_socket_writev(QIOChannel *ioc,return QIO_CHANNEL_ERR_BLOCK ;case EINTR : goto retry; -#ifdef QEMU_MSG_ZEROCOPY case ENOBUFS : -if (sflags &MSG_ZEROCOPY ) { +if (zero_copy_enabled) {if (flags &
QIO_CHANNEL_WRITE_FLAG_ZERO_COPY)Quoted 3 lines
avoids the #ifdef without needing to add yet another variable expressing what's already expressed in both 'flags' and 'sflags'.
Yes, it does, but at the cost of not compiling-out the zero-copy part when it's not supported,
since theQIO_CHANNEL_WRITE_FLAG_ZERO_COPYcomes as a parameter. This ends up meaning there will be at least one extra test for every time this function is called (the one in the next patch).Quoted 8 lines
Quoted 3 lines
The cost of a simple bit test is between negligible-and-non-existant with branch prediction. I doubt it would be possible to even measure it.
Yeah, you are probably right on that.
So the main learning point here is that it's not worth creating a new boolean for compiling-out
code that should not impact performance ?As ever "it depends" so there's no hard rule, and sometimes it can verge on bikeshed colouring :-)
Quoted 7 lines
I didn't like the variable in this case, because it introduces a 3rd variable to the method for representing whether zero copy is need, which is excessive. I'm not a fan of redundancy as it can often then lead to inconsistency. So it would need a compelling reason why it is better, which is difficult for such a simple method. If the code was more complex, a variable might have benefit of clarity, but in this case IMHO it was just overkill.
I see. Thanks for the clarification!
Best regards,
Leo
Quoted 8 lines
With regards,
Daniel
--|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|