MSG_ZEROCOPY + multifd
v8 cover MSG_ZEROCOPY + multifd
This patch series intends to enable MSG_ZEROCOPY in QIOChannel, and make use of it for multifd migration performance improvement, by reducing cpu usage.
Patch #1 creates new callbacks for QIOChannel, allowing the implementation of zero copy writing.
Patch #2 implements io_writev flags and io_flush() on QIOChannelSocket, making use of MSG_ZEROCOPY on Linux.
Patch #3 adds a "zero_copy_send" migration property, only available with CONFIG_LINUX, and compiled-out in any other architectures. This migration property has to be enabled before multifd migration starts.
Patch #4 adds a helper function that allows to see if TLS is going to be used. This helper will be later used in patch #5.
Patch #5 Makes use of QIOChannelSocket zero_copy implementation on nocomp multifd migration.
Results:
In preliminary tests, the resource usage of __sys_sendmsg() reduced 15 times, and the overall migration took 13-22% less time, based in synthetic cpu workload.
In further tests, it was noted that, on multifd migration with 8 channels:
- On idle hosts, migration time reduced in 10% to 21%.
- On hosts busy with heavy cpu stress (1 stress thread per cpu, but not cpu-pinned) migration time reduced in ~25% by enabling zero-copy.
- On hosts with heavy cpu-pinned workloads (1 stress thread per cpu, cpu-pinned), migration time reducted in ~66% by enabling zero-copy.
Above tests setup:
- Sending and Receiving hosts:
- CPU : Intel(R) Xeon(R) Platinum 8276L CPU @ 2.20GHz (448 CPUS)
- Network card: E810-C (100Gbps)
- >1TB RAM
- QEMU: Upstream master branch + This patchset
- Linux: Upstream v5.15
- VM configuration:
- 28 VCPUs
- 512GB RAM
---
Changes since v7:
- Migration property renamed from zero-copy to zero-copy-send
- A few early tests added to help misconfigurations to fail earlier
- qio_channel_full*_flags() renamed back to qio_channel_full*()
- multifd_send_sync_main() reverted back to not receiving a flag, so it always sync zero-copy when enabled.
- Improve code quality on a few points
Changes since v6:
- Remove io_writev_zero_copy(), and makes use of io_writev() new flags to achieve the same results.
- Rename io_flush_zero_copy() to io_flush()
- Previous patch #2 became too small, so it was squashed in previous patch #3 (now patch #2)
Changes since v5:
- flush_zero_copy now returns -1 on fail, 0 on success, and 1 when all processed writes were not able to use zerocopy in kernel.
- qio_channel_socket_poll() removed, using qio_channel_wait() instead
- ENOBUFS is now processed inside qio_channel_socket_writev_flags()
- Most zerocopy parameter validation moved to migrate_params_check(), leaving only feature test to socket_outgoing_migration() callback
- Naming went from *zerocopy to *zero_copy or *zero-copy, due to QAPI/QMP preferences
- Improved docs
Changes since v4:
- 3 patches got splitted in 6
- Flush is used for syncing after each iteration, instead of only at the end
- If zerocopy is not available, fail in connect instead of failing on write
- 'multifd-zerocopy' property renamed to 'zerocopy'
- Fail migrations that don't support zerocopy, if it's enabled.
- Instead of checking for zerocopy at each write, save the flags in MultiFDSendParams->write_flags and use them on write
- Reorganized flag usage in QIOChannelSocket
- A lot of typos fixed
- More doc on buffer restrictions
Changes since v3:
- QIOChannel interface names changed from io_async_{writev,flush} to io_{writev,flush}_zerocopy
- Instead of falling back in case zerocopy is not implemented, return error and abort operation.
- Flush now waits as long as needed, or return error in case anything goes wrong, aborting the operation.
- Zerocopy is now conditional in multifd, being set by parameter multifd-zerocopy
- Moves zerocopy_flush to multifd_send_sync_main() from multifd_save_cleanup so migration can abort if flush goes wrong.
- Several other small improvements
Changes since v2:
- Patch #1: One more fallback
- Patch #2: Fall back to sync if fails to lock buffer memory in MSG_ZEROCOPY send.
Changes since v1:
- Reimplemented the patchset using async_write + async_flush approach.
- Implemented a flush to be able to tell whenever all data was written.
Leonardo Bras (5):
QIOChannel: Add flags on io_writev and introduce io_flush callback
QIOChannelSocket: Implement io_writev zero copy flag & io_flush for
CONFIG_LINUX
migration: Add zero-copy-send parameter for QMP/HMP for Linux
migration: Add migrate_use_tls() helper
multifd: Implement zero copy write in multifd migration
(multifd-zero-copy)
qapi/migration.json | 24 ++++++ include/io/channel-socket.h | 2 + include/io/channel.h | 38 +++++++++- migration/migration.h | 6 ++ migration/multifd.h | 4 +- chardev/char-io.c | 2 +- hw/remote/mpqemu-link.c | 2 +- io/channel-buffer.c | 1 + io/channel-command.c | 1 + io/channel-file.c | 1 + io/channel-socket.c | 110 +++++++++++++++++++++++++++- io/channel-tls.c | 1 + io/channel-websock.c | 1 + io/channel.c | 53 +++++++++++--- migration/channel.c | 3 +- migration/migration.c | 52 ++++++++++++- migration/multifd.c | 46 +++++++++--- migration/ram.c | 29 ++++++-- migration/rdma.c | 1 + migration/socket.c | 6 ++ monitor/hmp-cmds.c | 6 ++ scsi/pr-manager-helper.c | 2 +- tests/unit/test-io-channel-socket.c | 1 + 23 files changed, 353 insertions(+), 39 deletions(-)
Signature · 1 line
2.34.1
PATCH v8 1/5 QIOChannel: Add flags on io_writev and introduce io_flush callback
Add flags to io_writev and introduce io_flush as optional callback to QIOChannelClass, allowing the implementation of zero copy writes by subclasses.
How to use them:
- Write data using qio_channel_writev*(...,QIO_CHANNEL_WRITE_FLAG_ZERO_COPY),
- Wait write completion with qio_channel_flush().
Notes:
As some zero copy write implementations work asynchronously, it's recommended to keep the write buffer untouched until the return of qio_channel_flush(), to avoid the risk of sending an updated buffer instead of the buffer state during write.
As io_flush callback is optional, if a subclass does not implement it, then:
- io_flush will return 0 without changing anything.
Also, some functions like qio_channel_writev_full_all() were adapted to receive a flag parameter. That allows shared code between zero copy and non-zero copy writev, and also an easier implementation on new flags.
--- include/io/channel.h | 38 ++++++++++++++++++++- chardev/char-io.c | 2 +- hw/remote/mpqemu-link.c | 2 +- io/channel-buffer.c | 1 + io/channel-command.c | 1 + io/channel-file.c | 1 + io/channel-socket.c | 2 ++ io/channel-tls.c | 1 + io/channel-websock.c | 1 + io/channel.c | 53 +++++++++++++++++++++++------ migration/rdma.c | 1 + scsi/pr-manager-helper.c | 2 +- tests/unit/test-io-channel-socket.c | 1 + 13 files changed, 92 insertions(+), 14 deletions(-)
include/io/channel.h +37 -1
@@ -32,12 +32,15 @@ OBJECT_DECLARE_TYPE(QIOChannel, QIOChannelClass,#define QIO_CHANNEL_ERR_BLOCK -2 +#define QIO_CHANNEL_WRITE_FLAG_ZERO_COPY 0x1 +typedef enum QIOChannelFeature QIOChannelFeature ;enum QIOChannelFeature {QIO_CHANNEL_FEATURE_FD_PASS ,QIO_CHANNEL_FEATURE_SHUTDOWN ,QIO_CHANNEL_FEATURE_LISTEN , +QIO_CHANNEL_FEATURE_WRITE_ZERO_COPY , };
@@ -104,6 +107,7 @@ struct QIOChannelClass {size_t niov, int *fds, size_t nfds, + int flags, Error **errp); ssize_t (*io_readv)(QIOChannel *ioc, const struct iovec *iov,
@@ -136,6 +140,8 @@ struct QIOChannelClass { IOHandler *io_read, IOHandler *io_write, void *opaque); +int (*io_flush)(QIOChannel *ioc, +Error **errp); };/* General I/O handling functions */
@@ -228,6 +234,7 @@ ssize_t qio_channel_readv_full(QIOChannel *ioc,* @niov: the length of the @iov array * @fds: an array of file handles to send * @nfds: number of file handles in @fds +* @flags: write flags (QIO_CHANNEL_WRITE_FLAG_*) * @errp: pointer to a NULL-initialized error object * * Write data to the IO channel, reading it from the
@@ -260,6 +267,7 @@ ssize_t qio_channel_writev_full(QIOChannel *ioc,size_t niov, int *fds, size_t nfds, + int flags, Error **errp);/**
@@ -837,6 +845,7 @@ int qio_channel_readv_full_all(QIOChannel *ioc,* @niov: the length of the @iov array * @fds: an array of file handles to send * @nfds: number of file handles in @fds +* @flags: write flags (QIO_CHANNEL_WRITE_FLAG_*) * @errp: pointer to a NULL-initialized error object * *
@@ -846,6 +855,14 @@ int qio_channel_readv_full_all(QIOChannel *ioc,* to be written, yielding from the current coroutine * if required. * +* If QIO_CHANNEL_WRITE_FLAG_ZERO_COPY is passed in flags, +* instead of waiting for all requested data to be written, +* this function will wait until it's all queued for writing. +* In this case, if the buffer gets changed between queueing and +* sending, the updated buffer will be sent. If this is not a +* desired behavior, it's suggested to call qio_channel_flush() +* before reusing the buffer. +* * Returns: 0 if all bytes were written, or -1 on error */
@@ -853,6 +870,25 @@ int qio_channel_writev_full_all(QIOChannel *ioc,const struct iovec *iov, size_t niov, int *fds, size_t nfds, - Error **errp); + int flags, Error **errp); + +/** +* qio_channel_flush: +* @ioc: the channel object +* @errp: pointer to a NULL-initialized error object +* +* Will block until every packet queued with +* qio_channel_writev_full() + QIO_CHANNEL_WRITE_FLAG_ZERO_COPY +* is sent, or return in case of any error. +* +* If not implemented, acts as a no-op, and returns 0. +* +* Returns -1 if any error is found, +* 1 if every send failed to use zero copy. +* 0 otherwise. +*/ + +int qio_channel_flush (QIOChannel *ioc, +Error **errp);#endif /* QIO_CHANNEL_H */
chardev/char-io.c +1 -1
@@ -122,7 +122,7 @@ int io_channel_send_full(QIOChannel *ioc, ret =qio_channel_writev_full ( ioc, &iov,1 , - fds, nfds,NULL ); + fds, nfds,0 ,NULL );if (ret ==QIO_CHANNEL_ERR_BLOCK ) {if (offset) {return offset;
hw/remote/mpqemu-link.c +1 -1
@@ -69,7 +69,7 @@ bool mpqemu_msg_send(MPQemuMsg *msg, QIOChannel *ioc, Error **errp) }if (!qio_channel_writev_full_all (ioc, send,G_N_ELEMENTS (send), - fds, nfds, errp)) { + fds, nfds,0 , errp)) { ret = true; }else {trace_mpqemu_send_io_error (msg->cmd, msg->size, nfds);
io/channel-buffer.c +1 -0
@@ -81,6 +81,7 @@ static ssize_t qio_channel_buffer_writev(QIOChannel *ioc,size_t niov, int *fds, size_t nfds, + int flags, Error **errp) { QIOChannelBuffer *bioc = QIO_CHANNEL_BUFFER(ioc);
io/channel-command.c +1 -0
@@ -258,6 +258,7 @@ static ssize_t qio_channel_command_writev(QIOChannel *ioc,size_t niov, int *fds, size_t nfds, + int flags, Error **errp) { QIOChannelCommand *cioc = QIO_CHANNEL_COMMAND(ioc);
io/channel-file.c +1 -0
@@ -114,6 +114,7 @@ static ssize_t qio_channel_file_writev(QIOChannel *ioc,size_t niov, int *fds, size_t nfds, + int flags, Error **errp) { QIOChannelFile *fioc = QIO_CHANNEL_FILE(ioc);
io/channel-socket.c +2 -0
@@ -525,6 +525,7 @@ static ssize_t qio_channel_socket_writev(QIOChannel *ioc,size_t niov, int *fds, size_t nfds, + int flags, Error **errp) { QIOChannelSocket *sioc = QIO_CHANNEL_SOCKET(ioc);
@@ -620,6 +621,7 @@ static ssize_t qio_channel_socket_writev(QIOChannel *ioc,size_t niov, int *fds, size_t nfds, + int flags, Error **errp) { QIOChannelSocket *sioc = QIO_CHANNEL_SOCKET(ioc);
io/channel-tls.c +1 -0
@@ -301,6 +301,7 @@ static ssize_t qio_channel_tls_writev(QIOChannel *ioc,size_t niov, int *fds, size_t nfds, + int flags, Error **errp) { QIOChannelTLS *tioc = QIO_CHANNEL_TLS(ioc);
io/channel-websock.c +1 -0
@@ -1127,6 +1127,7 @@ static ssize_t qio_channel_websock_writev(QIOChannel *ioc,size_t niov, int *fds, size_t nfds, + int flags, Error **errp) { QIOChannelWebsock *wioc = QIO_CHANNEL_WEBSOCK(ioc);
io/channel.c +43 -10
105 lines not on this page · raw message
migration/rdma.c +1 -0
@@ -2833,6 +2833,7 @@ static ssize_t qio_channel_rdma_writev(QIOChannel *ioc,size_t niov, int *fds, size_t nfds, + int flags, Error **errp) { QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(ioc);
scsi/pr-manager-helper.c +1 -1
@@ -77,7 +77,7 @@ static int pr_manager_helper_write(PRManagerHelper *pr_mgr, iov.iov_base = (void *)buf; iov.iov_len = sz; n_written =qio_channel_writev_full (QIO_CHANNEL (pr_mgr->ioc), &iov,1 , - nfds ? &fd :NULL , nfds, errp); + nfds ? &fd :NULL , nfds,0 , errp);if (n_written <=0 ) {assert (n_written !=QIO_CHANNEL_ERR_BLOCK );
tests/unit/test-io-channel-socket.c +1 -0
@@ -444,6 +444,7 @@ static void test_io_channel_unix_fd_pass(void)G_N_ELEMENTS (iosend), fdsend,G_N_ELEMENTS (fdsend), +0 , &error_abort); qio_channel_readv_full(dst,
Signature · 1 line
2.34.1
Re: [PATCH v8 1/5] QIOChannel: Add flags on io_writev and introduce io_flush callback
Quoted 54 lines · Leonardo Bras
Add flags to
io_writevand introduceio_flushas optional callback to QIOChannelClass, allowing the implementation of zero copy writes by subclasses.How to use them:
- Write data using qio_channel_writev*(...,QIO_CHANNEL_WRITE_FLAG_ZERO_COPY),
- Wait write completion withqio_channel_flush().Notes:
As some zero copy write implementations work asynchronously, it's recommended to keep the write buffer untouched until the return ofqio_channel_flush(), to avoid the risk of sending an updated buffer instead of the buffer state during write.As
io_flushcallback is optional, if a subclass does not implement it, then:
-io_flushwill return 0 without changing anything.Also, some functions like
qio_channel_writev_full_all()were adapted to receive a flag parameter. That allows shared code between zero copy and non-zero copy writev, and also an easier implementation on new flags.Signed-off-by Leonardo Bras <[email protected]>--- include/io/channel.h | 38 ++++++++++++++++++++- chardev/char-io.c | 2 +- hw/remote/mpqemu-link.c | 2 +- io/channel-buffer.c | 1 + io/channel-command.c | 1 + io/channel-file.c | 1 + io/channel-socket.c | 2 ++ io/channel-tls.c | 1 + io/channel-websock.c | 1 + io/channel.c | 53 +++++++++++++++++++++++------ migration/rdma.c | 1 + scsi/pr-manager-helper.c | 2 +- tests/unit/test-io-channel-socket.c | 1 + 13 files changed, 92 insertions(+), 14 deletions(-)io/channel.c +2
-2@@ -72,18 +72,32 @@ ssize_t qio_channel_writev_full(QIOChannel *ioc,size_t niov, int *fds, size_t nfds, + int flags, Error **errp) { QIOChannelClass *klass = QIO_CHANNEL_GET_CLASS(ioc); -if ((fds || nfds) && - !qio_channel_has_feature (ioc,QIO_CHANNEL_FEATURE_FD_PASS )) { +if (fds || nfds) {
io/channel.c ssize_t qio_channel_writev_full(QIOChannel *ioc, · line 90
Quoted 10 lines
io/channel.c +10
-0+if (!qio_channel_has_feature (ioc,QIO_CHANNEL_FEATURE_FD_PASS )) { +error_setg_errno (errp,EINVAL , +"Channel does not support file descriptor passing" ); +return -1 ; + } +if (flags &QIO_CHANNEL_WRITE_FLAG_ZERO_COPY ) { +error_setg_errno (errp,EINVAL , +"Zero Copy does not support file descriptor passing" ); +return -1 ; + }
Here you gracefully reject FD passing when zero copy is requested which is good.
io/channel.c int qio_channel_writev_full_all(QIOChannel *ioc, · line 254
Quoted 10 lines
io/channel.c +5
-0+ } +@@ -235,10 +249,16 @@ int qio_channel_writev_full_all(QIOChannel *ioc, iov, niov, 0, iov_size(iov, niov)); +if (flags &QIO_CHANNEL_WRITE_FLAG_ZERO_COPY ) { +assert (fds ==NULL && nfds ==0 ); + }
But here you abort QEMU if FD passing is requested when zero copy is set.
AFAICT, if you just delete this assert, the code to gracefully report errors will do the right thing.
Without the assert:
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v8 1/5] QIOChannel: Add flags on io_writev and introduce io_flush callback
Hello Daniel, thanks for reviewing!
Quoted 84 lines · Daniel P. Berrangé
Quoted 54 lines · Leonardo Bras
Add flags to
io_writevand introduceio_flushas optional callback to QIOChannelClass, allowing the implementation of zero copy writes by subclasses.How to use them:
- Write data using qio_channel_writev*(...,QIO_CHANNEL_WRITE_FLAG_ZERO_COPY),
- Wait write completion withqio_channel_flush().Notes:
As some zero copy write implementations work asynchronously, it's recommended to keep the write buffer untouched until the return ofqio_channel_flush(), to avoid the risk of sending an updated buffer instead of the buffer state during write.As
io_flushcallback is optional, if a subclass does not implement it, then:
-io_flushwill return 0 without changing anything.Also, some functions like
qio_channel_writev_full_all()were adapted to receive a flag parameter. That allows shared code between zero copy and non-zero copy writev, and also an easier implementation on new flags.Signed-off-by Leonardo Bras <[email protected]>--- include/io/channel.h | 38 ++++++++++++++++++++- chardev/char-io.c | 2 +- hw/remote/mpqemu-link.c | 2 +- io/channel-buffer.c | 1 + io/channel-command.c | 1 + io/channel-file.c | 1 + io/channel-socket.c | 2 ++ io/channel-tls.c | 1 + io/channel-websock.c | 1 + io/channel.c | 53 +++++++++++++++++++++++------ migration/rdma.c | 1 + scsi/pr-manager-helper.c | 2 +- tests/unit/test-io-channel-socket.c | 1 + 13 files changed, 92 insertions(+), 14 deletions(-)io/channel.c +2
-2@@ -72,18 +72,32 @@ ssize_t qio_channel_writev_full(QIOChannel *ioc,size_t niov, int *fds, size_t nfds, + int flags, Error **errp) { QIOChannelClass *klass = QIO_CHANNEL_GET_CLASS(ioc); -if ((fds || nfds) && - !qio_channel_has_feature (ioc,QIO_CHANNEL_FEATURE_FD_PASS )) { +if (fds || nfds) {io/channel.c ssize_t qio_channel_writev_full(QIOChannel *ioc, · line 90
Quoted 10 lines
io/channel.c +10
-0+if (!qio_channel_has_feature (ioc,QIO_CHANNEL_FEATURE_FD_PASS )) { +error_setg_errno (errp,EINVAL , +"Channel does not support file descriptor passing" ); +return -1 ; + } +if (flags &QIO_CHANNEL_WRITE_FLAG_ZERO_COPY ) { +error_setg_errno (errp,EINVAL , +"Zero Copy does not support file descriptor passing" ); +return -1 ; + }Here you gracefully reject FD passing when zero copy is requested which is good.
io/channel.c int qio_channel_writev_full_all(QIOChannel *ioc, · line 254
Quoted 10 lines
io/channel.c +5
-0+ } +@@ -235,10 +249,16 @@ int qio_channel_writev_full_all(QIOChannel *ioc, iov, niov, 0, iov_size(iov, niov)); +if (flags &QIO_CHANNEL_WRITE_FLAG_ZERO_COPY ) { +assert (fds ==NULL && nfds ==0 ); + }But here you abort QEMU if FD passing is requested when zero copy is set.
Quoted 2 lines
AFAICT, if you just delete this assert, the code to gracefully report errors will do the right thing.
Yeah, thatś right. This test is unnecessary since qio_channel_writev_full() will be called and will return error if fds + zerocopy happens.
Good catch!
Quoted 5 lines
Without the assert:
Reviewed-by Daniel P. Berrangé <[email protected]>
Thanks!
I will wait for more feedback on other patches before sending the v9, but it should not take too long this time.
Best regards,
Leo
Re: [PATCH v8 1/5] QIOChannel: Add flags on io_writev and introduce io_flush callback
In reply to Leonardo BrasQuoted 14 lines · Leonardo Bras
Add flags to
io_writevand introduceio_flushas optional callback to QIOChannelClass, allowing the implementation of zero copy writes by subclasses.How to use them:
- Write data using qio_channel_writev*(...,QIO_CHANNEL_WRITE_FLAG_ZERO_COPY),
- Wait write completion withqio_channel_flush().Notes:
As some zero copy write implementations work asynchronously, it's recommended to keep the write buffer untouched until the return ofqio_channel_flush(), to avoid the risk of sending an updated buffer instead of the buffer state during write.
Quoted 8 lines
As
io_flushcallback is optional, if a subclass does not implement it, then:
-io_flushwill return 0 without changing anything.Also, some functions like
qio_channel_writev_full_all()were adapted to receive a flag parameter. That allows shared code between zero copy and non-zero copy writev, and also an easier implementation on new flags.Signed-off-by Leonardo Bras <[email protected]>
With Dan's comment addressed on removing the redundant assertion:
Signature · 1 line
Peter Xu
Re: [PATCH v8 1/5] QIOChannel: Add flags on io_writev and introduce io_flush callback
Hello Peter,
Quoted 23 lines · Peter Xu
Quoted 21 lines · Leonardo Bras
Add flags to
io_writevand introduceio_flushas optional callback to QIOChannelClass, allowing the implementation of zero copy writes by subclasses.How to use them:
- Write data using qio_channel_writev*(...,QIO_CHANNEL_WRITE_FLAG_ZERO_COPY),
- Wait write completion withqio_channel_flush().Notes:
As some zero copy write implementations work asynchronously, it's recommended to keep the write buffer untouched until the return ofqio_channel_flush(), to avoid the risk of sending an updated buffer instead of the buffer state during write.As
io_flushcallback is optional, if a subclass does not implement it, then:
-io_flushwill return 0 without changing anything.Also, some functions like
qio_channel_writev_full_all()were adapted to receive a flag parameter. That allows shared code between zero copy and non-zero copy writev, and also an easier implementation on new flags.
Quoted 6 lines
Quoted 1 line
Signed-off-by Leonardo Bras <[email protected]>With Dan's comment addressed on removing the redundant assertion:
Reviewed-by Peter Xu <[email protected]>
Thank you for reviewing!
I think I am now missing reviewing only on patch 5/5 before sending the next version.
Could you and/or Daniel help me with that? Just to check if I am missing anything?
Best regards,
Leo
Re: [PATCH v8 1/5] QIOChannel: Add flags on io_writev and introduce io_flush callback
In reply to Leonardo BrasQuoted 14 lines · Leonardo Bras
Add flags to
io_writevand introduceio_flushas optional callback to QIOChannelClass, allowing the implementation of zero copy writes by subclasses.How to use them:
- Write data using qio_channel_writev*(...,QIO_CHANNEL_WRITE_FLAG_ZERO_COPY),
- Wait write completion withqio_channel_flush().Notes:
As some zero copy write implementations work asynchronously, it's recommended to keep the write buffer untouched until the return ofqio_channel_flush(), to avoid the risk of sending an updated buffer instead of the buffer state during write.
Quoted 8 lines
As
io_flushcallback is optional, if a subclass does not implement it, then:
-io_flushwill return 0 without changing anything.Also, some functions like
qio_channel_writev_full_all()were adapted to receive a flag parameter. That allows shared code between zero copy and non-zero copy writev, and also an easier implementation on new flags.Signed-off-by Leonardo Bras <[email protected]>
As everybody pointed out about the missing assertion...
Re: [PATCH v8 1/5] QIOChannel: Add flags on io_writev and introduce io_flush callback
Thanks for reviewing, Juan!
Quoted 29 lines · Juan Quintela
Quoted 14 lines · Leonardo Bras
Add flags to
io_writevand introduceio_flushas optional callback to QIOChannelClass, allowing the implementation of zero copy writes by subclasses.How to use them:
- Write data using qio_channel_writev*(...,QIO_CHANNEL_WRITE_FLAG_ZERO_COPY),
- Wait write completion withqio_channel_flush().Notes:
As some zero copy write implementations work asynchronously, it's recommended to keep the write buffer untouched until the return ofqio_channel_flush(), to avoid the risk of sending an updated buffer instead of the buffer state during write.Quoted 8 lines
As
io_flushcallback is optional, if a subclass does not implement it, then:
-io_flushwill return 0 without changing anything.Also, some functions like
qio_channel_writev_full_all()were adapted to receive a flag parameter. That allows shared code between zero copy and non-zero copy writev, and also an easier implementation on new flags.Signed-off-by Leonardo Bras <[email protected]>Reviewed-by Juan Quintela <[email protected]>As everybody pointed out about the missing assertion...
PATCH v8 2/5 QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX
In reply to Leonardo BrasFor CONFIG_LINUX, implement the new zero copy flag and the optional callback io_flush on QIOChannelSocket, but enables it only when MSG_ZEROCOPY feature is available in the host kernel, which is checked on qio_channel_socket_connect_sync()
qio_channel_socket_flush() was implemented by counting how many times sendmsg(...,MSG_ZEROCOPY) was successfully called, and then reading the socket's error queue, in order to find how many of them finished sending. Flush will loop until those counters are the same, or until some error occurs.
Notes on using writev() with QIO_CHANNEL_WRITE_FLAG_ZERO_COPY:
1: Buffer
- As MSG_ZEROCOPY tells the kernel to use the same user buffer to avoid copying, some caution is necessary to avoid overwriting any buffer before it's sent. If something like this happen, a newer version of the buffer may be sent instead.
- If this is a problem, it's recommended to call qio_channel_flush() before freeing or re-using the buffer.
2: Locked memory
- When using MSG_ZERCOCOPY, the buffer memory will be locked after queued, and unlocked after it's sent.
- Depending on the size of each buffer, and how often it's sent, it may require a larger amount of locked memory than usually available to non-root user.
- If the required amount of locked memory is not available, writev_zero_copy will return an error, which can abort an operation like migration,
- Because of this, when an user code wants to add zero copy as a feature, it requires a mechanism to disable it, so it can still be accessible to less privileged users.
--- include/io/channel-socket.h | 2 + io/channel-socket.c | 108 ++++++++++++++++++++++++++++++++++-- 2 files changed, 106 insertions(+), 4 deletions(-)
include/io/channel-socket.h +2 -0
@@ -47,6 +47,8 @@ struct QIOChannelSocket {socklen_t localAddrLen;struct sockaddr_storage remoteAddr;socklen_t remoteAddrLen; +ssize_t zero_copy_queued; +ssize_t zero_copy_sent; };
io/channel-socket.c +104 -4
155 lines not on this page · raw message
Signature · 1 line
2.34.1
Re: [PATCH v8 2/5] QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX
Quoted 29 lines · Leonardo Bras
For
CONFIG_LINUX, implement the new zero copy flag and the optional callbackio_flushon QIOChannelSocket, but enables it only whenMSG_ZEROCOPYfeature is available in the host kernel, which is checked onqio_channel_socket_connect_sync()
qio_channel_socket_flush()was implemented by counting how many timessendmsg(...,MSG_ZEROCOPY)was successfully called, and then reading the socket's error queue, in order to find how many of them finished sending. Flush will loop until those counters are the same, or until some error occurs.Notes on using
writev()withQIO_CHANNEL_WRITE_FLAG_ZERO_COPY:
1: Buffer
- AsMSG_ZEROCOPYtells the kernel to use the same user buffer to avoid copying, some caution is necessary to avoid overwriting any buffer before it's sent. If something like this happen, a newer version of the buffer may be sent instead.
- If this is a problem, it's recommended to callqio_channel_flush()before freeing or re-using the buffer.2: Locked memory
- When usingMSG_ZERCOCOPY, the buffer memory will be locked after queued, and unlocked after it's sent.
- Depending on the size of each buffer, and how often it's sent, it may require a larger amount of locked memory than usually available to non-root user.
- If the required amount of locked memory is not available,writev_zero_copywill return an error, which can abort an operation like migration,
- Because of this, when an user code wants to add zero copy as a feature, it requires a mechanism to disable it, so it can still be accessible to less privileged users.
Quoted 3 lines
Signed-off-by Leonardo Bras <[email protected]>Reviewed-by Peter Xu <[email protected]>Reviewed-by Daniel P. Berrangé <[email protected]>
PATCH v8 3/5 migration: Add zero-copy-send parameter for QMP/HMP for Linux
In reply to Leonardo BrasAdd property that allows zero-copy migration of memory pages on the sending side, and also includes a helper function migrate_use_zero_copy_send() to check if it's enabled.
No code is introduced to actually do the migration, but it allow future implementations to enable/disable this feature.
On non-Linux builds this parameter is compiled-out.
--- qapi/migration.json | 24 ++++++++++++++++++++++++ migration/migration.h | 5 +++++ migration/migration.c | 32 ++++++++++++++++++++++++++++++++ migration/socket.c | 5 +++++ monitor/hmp-cmds.c | 6 ++++++ 5 files changed, 72 insertions(+)
qapi/migration.json +24 -0
@@ -741,6 +741,13 @@ # will consume more CPU. # Defaults to 1. (Since 5.0) # +# @zero-copy-send: Controls behavior on sending memory pages on migration. +# When true, enables a zero-copy mechanism for sending memory +# pages, if host supports it. +# Requires that QEMU be permitted to use locked memory for guest +# RAM pages. +# Defaults to false. (Since 7.1) +# # @block-bitmap-mapping: Maps block nodes and bitmaps on them to # aliases for the purpose of dirty bitmap migration. Such # aliases may for example be the corresponding names on the
@@ -780,6 +787,7 @@ 'xbzrle-cache-size', 'max-postcopy-bandwidth', 'max-cpu-throttle', 'multifd-compression', 'multifd-zlib-level' ,'multifd-zstd-level', + { 'name': 'zero-copy-send', 'if' : 'CONFIG_LINUX'}, 'block-bitmap-mapping' ] } ##
@@ -906,6 +914,13 @@ # will consume more CPU. # Defaults to 1. (Since 5.0) # +# @zero-copy-send: Controls behavior on sending memory pages on migration. +# When true, enables a zero-copy mechanism for sending memory +# pages, if host supports it. +# Requires that QEMU be permitted to use locked memory for guest +# RAM pages. +# Defaults to false. (Since 7.1) +# # @block-bitmap-mapping: Maps block nodes and bitmaps on them to # aliases for the purpose of dirty bitmap migration. Such # aliases may for example be the corresponding names on the
@@ -960,6 +975,7 @@ '*multifd-compression': 'MultiFDCompression', '*multifd-zlib-level': 'uint8', '*multifd-zstd-level': 'uint8', + '*zero-copy-send': { 'type': 'bool', 'if': 'CONFIG_LINUX' }, '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ] } } ##
@@ -1106,6 +1122,13 @@ # will consume more CPU. # Defaults to 1. (Since 5.0) # +# @zero-copy-send: Controls behavior on sending memory pages on migration. +# When true, enables a zero-copy mechanism for sending memory +# pages, if host supports it. +# Requires that QEMU be permitted to use locked memory for guest +# RAM pages. +# Defaults to false. (Since 7.1) +# # @block-bitmap-mapping: Maps block nodes and bitmaps on them to # aliases for the purpose of dirty bitmap migration. Such # aliases may for example be the corresponding names on the
@@ -1158,6 +1181,7 @@ '*multifd-compression': 'MultiFDCompression', '*multifd-zlib-level': 'uint8', '*multifd-zstd-level': 'uint8', + '*zero-copy-send': { 'type': 'bool', 'if': 'CONFIG_LINUX' }, '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ] } } ##
migration/migration.h +5 -0
@@ -339,6 +339,11 @@ MultiFDCompression migrate_multifd_compression(void);int migrate_multifd_zlib_level (void );int migrate_multifd_zstd_level (void ); +#ifdef CONFIG_LINUX +bool migrate_use_zero_copy_send (void ); +#else +#define migrate_use_zero_copy_send () (false) +#endif int migrate_use_xbzrle (void );uint64_t migrate_xbzrle_cache_size (void );bool migrate_colo_enabled (void );
migration/migration.c +32 -0
@@ -893,6 +893,10 @@ MigrationParameters *qmp_query_migrate_parameters(Error **errp) params->multifd_zlib_level = s->parameters.multifd_zlib_level; params->has_multifd_zstd_level = true; params->multifd_zstd_level = s->parameters.multifd_zstd_level; +#ifdef CONFIG_LINUX + params->has_zero_copy_send = true; + params->zero_copy_send = s->parameters.zero_copy_send; +#endif params->has_xbzrle_cache_size = true; params->xbzrle_cache_size = s->parameters.xbzrle_cache_size; params->has_max_postcopy_bandwidth = true;
@@ -1549,6 +1553,11 @@ static void migrate_params_test_apply(MigrateSetParameters *params,if (params->has_multifd_compression) { dest->multifd_compression = params->multifd_compression; } +#ifdef CONFIG_LINUX +if (params->has_zero_copy_send) { + dest->zero_copy_send = params->zero_copy_send; + } +#endif if (params->has_xbzrle_cache_size) { dest->xbzrle_cache_size = params->xbzrle_cache_size; }
@@ -1661,6 +1670,11 @@ static void migrate_params_apply(MigrateSetParameters *params, Error **errp)if (params->has_multifd_compression) { s->parameters.multifd_compression = params->multifd_compression; } +#ifdef CONFIG_LINUX +if (params->has_zero_copy_send) { + s->parameters.zero_copy_send = params->zero_copy_send; + } +#endif if (params->has_xbzrle_cache_size) { s->parameters.xbzrle_cache_size = params->xbzrle_cache_size;xbzrle_cache_resize (params->xbzrle_cache_size, errp);
@@ -2551,6 +2565,17 @@ int migrate_multifd_zstd_level(void)return s->parameters.multifd_zstd_level; } +#ifdef CONFIG_LINUX +bool migrate_use_zero_copy_send (void ) +{ +MigrationState *s; + + s =migrate_get_current (); + +return s->parameters.zero_copy_send; +} +#endif +int migrate_use_xbzrle (void ) {MigrationState *s;
@@ -4194,6 +4219,10 @@ static Property migration_properties[] = { DEFINE_PROP_UINT8("multifd-zstd-level", MigrationState, parameters.multifd_zstd_level, DEFAULT_MIGRATE_MULTIFD_ZSTD_LEVEL), +#ifdef CONFIG_LINUX +DEFINE_PROP_BOOL ("zero_copy_send" , MigrationState, + parameters.zero_copy_send, false), +#endifDEFINE_PROP_SIZE ("xbzrle-cache-size" , MigrationState, parameters.xbzrle_cache_size, DEFAULT_MIGRATE_XBZRLE_CACHE_SIZE),
@@ -4291,6 +4320,9 @@ static void migration_instance_init(Object *obj) params->has_multifd_compression = true; params->has_multifd_zlib_level = true; params->has_multifd_zstd_level = true; +#ifdef CONFIG_LINUX + params->has_zero_copy_send = true; +#endif params->has_xbzrle_cache_size = true; params->has_max_postcopy_bandwidth = true; params->has_max_cpu_throttle = true;
migration/socket.c +5 -0
@@ -77,6 +77,11 @@ static void socket_outgoing_migration(QIOTask *task, }else {trace_migration_socket_outgoing_connected (data->hostname); } + +if (migrate_use_zero_copy_send ()) { +error_setg (&err,"Zero copy send not available in migration" ); + } +migration_channel_connect (data->s, sioc, data->hostname, err);object_unref (OBJECT (sioc)); }
monitor/hmp-cmds.c +6 -0
@@ -1309,6 +1309,12 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDict *qdict) p->has_multifd_zstd_level = true;visit_type_uint8 (v, param, &p->multifd_zstd_level, &err);break ; +#ifdef CONFIG_LINUX +case MIGRATION_PARAMETER_ZERO_COPY_SEND : + p->has_zero_copy_send = true; +visit_type_bool (v, param, &p->zero_copy_send, &err); +break ; +#endif case MIGRATION_PARAMETER_XBZRLE_CACHE_SIZE : p->has_xbzrle_cache_size = true;if (!visit_type_size (v, param, &cache_size, &err)) {
Signature · 1 line
2.34.1
Re: [PATCH v8 3/5] migration: Add zero-copy-send parameter for QMP/HMP for Linux
Quoted 4 lines · Leonardo Bras
Add property that allows zero-copy migration of memory pages on the sending side, and also includes a helper function
migrate_use_zero_copy_send()to check if it's enabled.
Quoted 8 lines
No code is introduced to actually do the migration, but it allow future implementations to enable/disable this feature.
On non-Linux builds this parameter is compiled-out.
Signed-off-by Leonardo Bras <[email protected]>Reviewed-by Peter Xu <[email protected]>Reviewed-by Daniel P. Berrangé <[email protected]>
PATCH v8 4/5 migration: Add migrate_use_tls() helper
In reply to Leonardo BrasA lot of places check parameters.tls_creds in order to evaluate if TLS is in use, and sometimes call migrate_get_current() just for that test.
Add new helper function migrate_use_tls() in order to simplify testing for TLS usage.
--- migration/migration.h | 1 + migration/channel.c | 3 +-- migration/migration.c | 9 +++++++++ migration/multifd.c | 5 +---- 4 files changed, 12 insertions(+), 6 deletions(-)
migration/migration.h +1 -0
@@ -344,6 +344,7 @@ bool migrate_use_zero_copy_send(void);#else #define migrate_use_zero_copy_send () (false)#endif +int migrate_use_tls (void );int migrate_use_xbzrle (void );uint64_t migrate_xbzrle_cache_size (void );bool migrate_colo_enabled (void );
migration/channel.c +1 -2
@@ -38,8 +38,7 @@ void migration_channel_process_incoming(QIOChannel *ioc)trace_migration_set_incoming_channel ( ioc,object_get_typename (OBJECT (ioc))); -if (s->parameters.tls_creds && - *s->parameters.tls_creds && +if (migrate_use_tls () && !object_dynamic_cast (OBJECT (ioc),TYPE_QIO_CHANNEL_TLS )) {migration_tls_channel_process_incoming (s, ioc, &local_err);
migration/migration.c +9 -0
@@ -2576,6 +2576,15 @@ bool migrate_use_zero_copy_send(void) }#endif +int migrate_use_tls (void ) +{ +MigrationState *s; + + s =migrate_get_current (); + +return s->parameters.tls_creds && *s->parameters.tls_creds; +} +int migrate_use_xbzrle (void ) {MigrationState *s;
migration/multifd.c +1 -4
@@ -784,14 +784,11 @@ static bool multifd_channel_connect(MultiFDSendParams *p, QIOChannel *ioc, Error *error) { -MigrationState *s =migrate_get_current (); -trace_multifd_set_outgoing_channel ( ioc,object_get_typename (OBJECT (ioc)), p->tls_hostname, error);if (!error) { -if (s->parameters.tls_creds && - *s->parameters.tls_creds && +if (migrate_use_tls () && !object_dynamic_cast (OBJECT (ioc),TYPE_QIO_CHANNEL_TLS )) {multifd_tls_channel_connect (p, ioc, &error);
Signature · 1 line
2.34.1
PATCH v8 5/5 multifd: Implement zero copy write in multifd migration (multifd-zero-copy)
In reply to Leonardo BrasImplement zero copy send on nocomp_send_write(), by making use of QIOChannel writev + flags & flush interface.
Change multifd_send_sync_main() so flush_zero_copy() can be called after each iteration in order to make sure all dirty pages are sent before a new iteration is started. It will also flush at the beginning and at the end of migration.
Also make it return -1 if flush_zero_copy() fails, in order to cancel the migration process, and avoid resuming the guest in the target host without receiving all current RAM.
This will work fine on RAM migration because the RAM pages are not usually freed, and there is no problem on changing the pages content between writev_zero_copy() and the actual sending of the buffer, because this change will dirty the page and cause it to be re-sent on a next iteration anyway.
A lot of locked memory may be needed in order to use multid migration with zero-copy enabled, so disabling the feature should be necessary for low-privileged users trying to perform multifd migrations.
--- migration/multifd.h | 4 +++- migration/migration.c | 11 ++++++++++- migration/multifd.c | 41 +++++++++++++++++++++++++++++++++++------ migration/ram.c | 29 ++++++++++++++++++++++------- migration/socket.c | 5 +++-- 5 files changed, 73 insertions(+), 17 deletions(-)
migration/multifd.h +3 -1
@@ -22,7 +22,7 @@ int multifd_load_cleanup(Error **errp);bool multifd_recv_all_channels_created (void );bool multifd_recv_new_channel (QIOChannel *ioc,Error **errp);void multifd_recv_sync_main (void ); -void multifd_send_sync_main (QEMUFile *f); +int multifd_send_sync_main (QEMUFile *f);int multifd_queue_page (QEMUFile *f,RAMBlock *block,ram_addr_t offset);/* Multifd Compression flags */
@@ -96,6 +96,8 @@ typedef struct {uint32_t packet_len;/* pointer to the packet */ MultiFDPacket_t *packet; +/* multifd flags for sending ram */ +int write_flags;/* multifd flags for each packet */ uint32_t flags;/* size of the next packet that contains pages */
migration/migration.c +10 -1
@@ -1479,7 +1479,16 @@ static bool migrate_params_check(MigrationParameters *params, Error **errp)error_prepend (errp,"Invalid mapping given for block-bitmap-mapping: " );return false; } - +#ifdef CONFIG_LINUX +if (params->zero_copy_send && + (!migrate_use_multifd () || + params->multifd_compression !=MULTIFD_COMPRESSION_NONE || + (params->tls_creds && *params->tls_creds))) { +error_setg (errp, +"Zero copy only available for non-compressed non-TLS multifd migration" ); +return false; + } +#endif return true; }
migration/multifd.c +35 -6
87 lines not on this page · raw message
migration/ram.c +22 -7
@@ -2902,6 +2902,7 @@ static int ram_save_setup(QEMUFile *f, void *opaque) {RAMState **rsp = opaque;RAMBlock *block; +int ret;if (compress_threads_save_setup ()) {return -1 ;
@@ -2936,7 +2937,11 @@ static int ram_save_setup(QEMUFile *f, void *opaque)ram_control_before_iterate (f,RAM_CONTROL_SETUP );ram_control_after_iterate (f,RAM_CONTROL_SETUP ); -multifd_send_sync_main (f); + ret =multifd_send_sync_main (f); +if (ret <0 ) { +return ret; + } +qemu_put_be64 (f,RAM_SAVE_FLAG_EOS );qemu_fflush (f);
@@ -3045,7 +3050,11 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) out:if (ret >=0 &&migration_is_setup_or_active (migrate_get_current ()->state)) { -multifd_send_sync_main (rs->f); + ret =multifd_send_sync_main (rs->f); +if (ret <0 ) { +return ret; + } +qemu_put_be64 (f,RAM_SAVE_FLAG_EOS );qemu_fflush (f);ram_transferred_add (8 );
@@ -3105,13 +3114,19 @@ static int ram_save_complete(QEMUFile *f, void *opaque)ram_control_after_iterate (f,RAM_CONTROL_FINISH ); } -if (ret >=0 ) { -multifd_send_sync_main (rs->f); -qemu_put_be64 (f,RAM_SAVE_FLAG_EOS ); -qemu_fflush (f); +if (ret <0 ) { +return ret; } -return ret; + ret =multifd_send_sync_main (rs->f); +if (ret <0 ) { +return ret; + } + +qemu_put_be64 (f,RAM_SAVE_FLAG_EOS ); +qemu_fflush (f); + +return 0 ; } static void ram_save_pending(QEMUFile *f, void *opaque, uint64_t max_size,
migration/socket.c +3 -2
@@ -78,8 +78,9 @@ static void socket_outgoing_migration(QIOTask *task,trace_migration_socket_outgoing_connected (data->hostname); } -if (migrate_use_zero_copy_send ()) { -error_setg (&err,"Zero copy send not available in migration" ); +if (migrate_use_zero_copy_send () && + !qio_channel_has_feature (sioc,QIO_CHANNEL_FEATURE_WRITE_ZERO_COPY )) { +error_setg (&err,"Zero copy send feature not detected in host kernel" ); }migration_channel_connect (data->s, sioc, data->hostname, err);
Signature · 1 line
2.34.1
Re: [PATCH v8 5/5] multifd: Implement zero copy write in multifd migration (multifd-zero-copy)
Quoted 5 lines · Leonardo Bras
migration/multifd.c +2
-1-void multifd_send_sync_main (QEMUFile *f) +int multifd_send_sync_main (QEMUFile *f) {int i; +bool flush_zero_copy;
migration/multifd.c void multifd_save_cleanup(void) · line 582
Quoted 10 lines
migration/multifd.c +2
-2if (!migrate_use_multifd ()) { -return ; +return 0 ; }if (multifd_send_state->pages->num) {if (multifd_send_pages (f) <0 ) {error_report ("%s: multifd_send_pages fail" , __func__); -return ; +return 0 ;
I've not checked how it used to do if multifd_send_pages() failed, but.. should it returns -1 rather than 0 when there will be a return code?
Quoted 14 lines
} }migration/multifd.c +8
-0+ +/* +* When using zero-copy, it's necessary to flush after each iteration to +* make sure pages from earlier iterations don't end up replacing newer +* pages. +*/ + flush_zero_copy =migrate_use_zero_copy_send (); +for (i =0 ; i <migrate_multifd_channels (); i++) {MultiFDSendParams *p = &multifd_send_state->params[i];
migration/multifd.c void multifd_send_sync_main(QEMUFile *f) · line 603
Quoted 5 lines
migration/multifd.c +1
-1@@ -591,7 +600,7 @@ void multifd_send_sync_main(QEMUFile *f)if (p->quit) {error_report ("%s: channel %d has already quit" , __func__, i);qemu_mutex_unlock (&p->mutex); -return ; +return 0 ;
Same question here.
Review void multifd_send_sync_main(QEMUFile *f) · line 604
Quoted 1 line
(quoted diff) +0
-0}
The rest looks good. Thanks,
Signature · 1 line
Peter Xu
Re: [PATCH v8 5/5] multifd: Implement zero copy write in multifd migration (multifd-zero-copy)
Hello Peter, thanks for reviewing!
Quoted 8 lines · Peter Xu
Quoted 6 lines · Leonardo Bras
migration/multifd.c +2
-1-void multifd_send_sync_main (QEMUFile *f) +int multifd_send_sync_main (QEMUFile *f) {int i; +bool flush_zero_copy;
Quoted 12 lines
migration/multifd.c void multifd_save_cleanup(void) · line 582
Quoted 9 lines
if (!migrate_use_multifd()) {migration/multifd.c +2
-2-return ; +return 0 ; }if (multifd_send_state->pages->num) {if (multifd_send_pages (f) <0 ) {error_report ("%s: multifd_send_pages fail" , __func__); -return ; +return 0 ;I've not checked how it used to do if
multifd_send_pages()failed, but.. should it returns -1 rather than 0 when there will be a return code?
Yeah, that makes sense.
The point here is that I was trying not to modify much of the current behavior.
I mean, multifd_send_sync_main() would previously return void, so any other errors would not matter to the caller of this function, which will continue to run as if nothing happened.
Now, if it fails with flush_zero_copy, the operation needs to be aborted.
Maybe, I should make it different:
- In any error, return -1.
- Create/use a specific error code in the case of a failing flush_zero_copy, so I can test the return value for it on the caller function and return early.
Or alternatively, the other errors could also return early, but since this will change how the code currently works, I would probably need another patch for that change. (so it can be easily reverted if needed)
What do you think is better?
Quoted 13 lines
Quoted 13 lines
} }migration/multifd.c +8
-0+ +/* +* When using zero-copy, it's necessary to flush after each iteration to +* make sure pages from earlier iterations don't end up replacing newer +* pages. +*/ + flush_zero_copy =migrate_use_zero_copy_send (); +for (i =0 ; i <migrate_multifd_channels (); i++) {MultiFDSendParams *p = &multifd_send_state->params[i];
Quoted 8 lines
migration/multifd.c void multifd_send_sync_main(QEMUFile *f) · line 603
Quoted 6 lines
migration/multifd.c +1
-1@@ -591,7 +600,7 @@ void multifd_send_sync_main(QEMUFile *f)if (p->quit) {error_report ("%s: channel %d has already quit" , __func__, i);qemu_mutex_unlock (&p->mutex); -return ; +return 0 ;Same question here.
Please see above,
Quoted 4 lines
Review void multifd_send_sync_main(QEMUFile *f) · line 604
Quoted 2 lines
(quoted diff) +0
-0}The rest looks good. Thanks,
Thank you!
Best regards,
Leo
Re: [PATCH v8 5/5] multifd: Implement zero copy write in multifd migration (multifd-zero-copy)
Quoted 24 lines · Leonardo Bras Soares Passos
Hello Peter, thanks for reviewing!
Quoted 8 lines · Peter Xu
Quoted 6 lines · Leonardo Bras
migration/multifd.c +2
-1-void multifd_send_sync_main (QEMUFile *f) +int multifd_send_sync_main (QEMUFile *f) {int i; +bool flush_zero_copy;Quoted 12 lines
migration/multifd.c void multifd_save_cleanup(void) · line 582
Quoted 9 lines
if (!migrate_use_multifd()) {migration/multifd.c +2
-2-return ; +return 0 ; }if (multifd_send_state->pages->num) {if (multifd_send_pages (f) <0 ) {error_report ("%s: multifd_send_pages fail" , __func__); -return ; +return 0 ;I've not checked how it used to do if
multifd_send_pages()failed, but.. should it returns -1 rather than 0 when there will be a return code?
Quoted 8 lines
Yeah, that makes sense.
The point here is that I was trying not to modify much of the current behavior.I mean,
multifd_send_sync_main()would previously return void, so any other errors would not matter to the caller of this function, which will continue to run as if nothing happened.Now, if it fails with
flush_zero_copy, the operation needs to be aborted.
Right, so how I understand is we'll fail anyway, and this allows us to fail (probably) sooner.
Quoted 11 lines
Maybe, I should make it different:
- In any error, return -1.
- Create/use a specific error code in the case of a failingflush_zero_copy, so I can test the return value for it on the caller function and return early.Or alternatively, the other errors could also return early, but since this will change how the code currently works, I would probably need another patch for that change. (so it can be easily reverted if needed)
Yeah, should work too to add a patch before this one.
Quoted 2 lines
What do you think is better?
I just don't see how it could continue if e.g. multifd_send_pages() failed.
The other thing is returning zero looks weird itself when there's obviously an error. Normally we could allow that but better with a comment showing why. For this case it's more natural to me if we could just fail early.
Juan?
Signature · 1 line
Peter Xu
Re: [PATCH v8 5/5] multifd: Implement zero copy write in multifd migration (multifd-zero-copy)
In reply to Leonardo Bras Soares PassosQuoted 24 lines · Leonardo Bras Soares Passos
Hello Peter, thanks for reviewing!
Quoted 8 lines · Peter Xu
Quoted 6 lines · Leonardo Bras
migration/multifd.c +2
-1-void multifd_send_sync_main (QEMUFile *f) +int multifd_send_sync_main (QEMUFile *f) {int i; +bool flush_zero_copy;Quoted 12 lines
migration/multifd.c void multifd_save_cleanup(void) · line 582
Quoted 9 lines
if (!migrate_use_multifd()) {migration/multifd.c +2
-2-return ; +return 0 ; }if (multifd_send_state->pages->num) {if (multifd_send_pages (f) <0 ) {error_report ("%s: multifd_send_pages fail" , __func__); -return ; +return 0 ;I've not checked how it used to do if
multifd_send_pages()failed, but.. should it returns -1 rather than 0 when there will be a return code?
Quoted 2 lines
Yeah, that makes sense.
The point here is that I was trying not to modify much of the current behavior.
if (qatomic_read(&multifd_send_state->exiting)) {
return -1;
}
if (p->quit) {
error_report("%s: channel %d has already quit!", __func__, i);
qemu_mutex_unlock(&p->mutex);
return -1;
}This are the only two cases where the current code can return one error. In the 1st case we are exiting, we are already in the middle of finishing, so we don't really care.
In the second one, we have already quit, and error as already quite big.
But I agree with both comments:
- we need to improve the error paths
- leonardo changes don't affect what is already there.
Quoted 11 lines
I mean,
multifd_send_sync_main()would previously return void, so any other errors would not matter to the caller of this function, which will continue to run as if nothing happened.Now, if it fails with
flush_zero_copy, the operation needs to be aborted.Maybe, I should make it different:
- In any error, return -1.
- Create/use a specific error code in the case of a failingflush_zero_copy, so I can test the return value for it on the caller function and return early.
We need to add the check. It don't matter if the problem is zero_copy or the existing one, we are under a minor catastrophe and migration has to be aborted.
Quoted 30 lines
Or alternatively, the other errors could also return early, but since this will change how the code currently works, I would probably need another patch for that change. (so it can be easily reverted if needed)
What do you think is better?
Quoted 13 lines
Quoted 13 lines
} }migration/multifd.c +8
-0+ +/* +* When using zero-copy, it's necessary to flush after each iteration to +* make sure pages from earlier iterations don't end up replacing newer +* pages. +*/ + flush_zero_copy =migrate_use_zero_copy_send (); +for (i =0 ; i <migrate_multifd_channels (); i++) {MultiFDSendParams *p = &multifd_send_state->params[i];Quoted 8 lines
migration/multifd.c void multifd_send_sync_main(QEMUFile *f) · line 603
Quoted 6 lines
migration/multifd.c +1
-1@@ -591,7 +600,7 @@ void multifd_send_sync_main(QEMUFile *f)if (p->quit) {error_report ("%s: channel %d has already quit" , __func__, i);qemu_mutex_unlock (&p->mutex); -return ; +return 0 ;Same question here.
Quoted 6 lines
Please see above,
Quoted 4 lines
Review void multifd_send_sync_main(QEMUFile *f) · line 604
Quoted 2 lines
(quoted diff) +0
-0}The rest looks good. Thanks,
Later, Juan.
Re: [PATCH v8 5/5] multifd: Implement zero copy write in multifd migration (multifd-zero-copy)
Quoted 64 lines · Juan Quintela
Quoted 24 lines · Leonardo Bras Soares Passos
Hello Peter, thanks for reviewing!
Quoted 8 lines · Peter Xu
Quoted 6 lines · Leonardo Bras
migration/multifd.c +2
-1-void multifd_send_sync_main (QEMUFile *f) +int multifd_send_sync_main (QEMUFile *f) {int i; +bool flush_zero_copy;Quoted 12 lines
migration/multifd.c void multifd_save_cleanup(void) · line 582
Quoted 9 lines
if (!migrate_use_multifd()) {migration/multifd.c +2
-2-return ; +return 0 ; }if (multifd_send_state->pages->num) {if (multifd_send_pages (f) <0 ) {error_report ("%s: multifd_send_pages fail" , __func__); -return ; +return 0 ;I've not checked how it used to do if
multifd_send_pages()failed, but.. should it returns -1 rather than 0 when there will be a return code?Quoted 2 lines
Yeah, that makes sense.
The point here is that I was trying not to modify much of the current behavior.if (qatomic_read(&multifd_send_state->exiting)) { return -1; } if (p->quit) { error_report("%s: channel %d has already quit!", __func__, i); qemu_mutex_unlock(&p->mutex); return -1; }This are the only two cases where the current code can return one error. In the 1st case we are exiting, we are already in the middle of finishing, so we don't really care.
In the second one, we have already quit, and error as already quite big.But I agree with both comments:
- we need to improve the error paths
- leonardo changes don't affect what is already there.Quoted 11 lines
I mean,
multifd_send_sync_main()would previously return void, so any other errors would not matter to the caller of this function, which will continue to run as if nothing happened.Now, if it fails with
flush_zero_copy, the operation needs to be aborted.Maybe, I should make it different:
- In any error, return -1.
- Create/use a specific error code in the case of a failingflush_zero_copy, so I can test the return value for it on the caller function and return early.
Quoted 3 lines
We need to add the check. It don't matter if the problem is
zero_copyor the existing one, we are under a minor catastrophe and migration has to be aborted.
Ok, I will fix that so we can abort in case of any error. Maybe it's better to do that on a separated patch, before 5/5, right?
Quoted 31 lines
Quoted 31 lines
Or alternatively, the other errors could also return early, but since this will change how the code currently works, I would probably need another patch for that change. (so it can be easily reverted if needed)
What do you think is better?
Quoted 21 lines
Quoted 14 lines
} }migration/multifd.c +8
-0+ +/* +* When using zero-copy, it's necessary to flush after each iteration to +* make sure pages from earlier iterations don't end up replacing newer +* pages. +*/ + flush_zero_copy =migrate_use_zero_copy_send (); +for (i =0 ; i <migrate_multifd_channels (); i++) {MultiFDSendParams *p = &multifd_send_state->params[i];migration/multifd.c void multifd_send_sync_main(QEMUFile *f) · line 603
Quoted 5 lines
migration/multifd.c +1
-1@@ -591,7 +600,7 @@ void multifd_send_sync_main(QEMUFile *f)if (p->quit) {error_report ("%s: channel %d has already quit" , __func__, i);qemu_mutex_unlock (&p->mutex); -return ; +return 0 ;Same question here.
Quoted 9 lines
Quoted 6 lines
Please see above,
Quoted 4 lines
Review void multifd_send_sync_main(QEMUFile *f) · line 604
Quoted 2 lines
(quoted diff) +0
-0}The rest looks good. Thanks,
Later, Juan.
Thanks for the feedback!
Best regards,
Leo
Re: [PATCH v8 5/5] multifd: Implement zero copy write in multifd migration (multifd-zero-copy)
In reply to Leonardo BrasQuoted 12 lines · Leonardo Bras
Implement zero copy send on
nocomp_send_write(), by making use of QIOChannel writev + flags & flush interface.Change
multifd_send_sync_main()soflush_zero_copy()can be called after each iteration in order to make sure all dirty pages are sent before a new iteration is started. It will also flush at the beginning and at the end of migration.Also make it return -1 if
flush_zero_copy()fails, in order to cancel the migration process, and avoid resuming the guest in the target host without receiving all current RAM.
Quoted 6 lines
This will work fine on RAM migration because the RAM pages are not usually freed, and there is no problem on changing the pages content between
writev_zero_copy()and the actual sending of the buffer, because this change will dirty the page and cause it to be re-sent on a next iteration anyway.A lot of locked memory may be needed in order to use multid migration
^^^^^^
multifd.
I can fix it on the commit.
Quoted 7 lines
migration/migration.c +2
-1@@ -1479,7 +1479,16 @@ static bool migrate_params_check(MigrationParameters *params, Error **errp)error_prepend (errp,"Invalid mapping given for block-bitmap-mapping: " );return false; } - +#ifdef CONFIG_LINUX + if (params->zero_copy_send &&
migration/migration.c static bool migrate_params_check(MigrationParameters *params, Error **errp) · line 1493
Quoted 10 lines
migration/migration.c +8
-0+ (!migrate_use_multifd() || + params->multifd_compression != MULTIFD_COMPRESSION_NONE || + (params->tls_creds && *params->tls_creds))) { +error_setg (errp, +"Zero copy only available for non-compressed non-TLS multifd migration" ); +return false; + } +#endif return true; }
Test is long, but it is exactly what we need. Good.
Quoted 65 lines
migration/multifd.c +24
-4@@ -568,19 +568,28 @@ void multifd_save_cleanup(void) multifd_send_state =NULL ; } -void multifd_send_sync_main (QEMUFile *f) +int multifd_send_sync_main (QEMUFile *f) {int i; +bool flush_zero_copy;if (!migrate_use_multifd ()) { -return ; +return 0 ; }if (multifd_send_state->pages->num) {if (multifd_send_pages (f) <0 ) {error_report ("%s: multifd_send_pages fail" , __func__); -return ; +return 0 ; } } + +/* +* When using zero-copy, it's necessary to flush after each iteration to +* make sure pages from earlier iterations don't end up replacing newer +* pages. +*/ + flush_zero_copy =migrate_use_zero_copy_send (); +for (i =0 ; i <migrate_multifd_channels (); i++) {MultiFDSendParams *p = &multifd_send_state->params[i];@@ -591,7 +600,7 @@ void multifd_send_sync_main(QEMUFile *f)if (p->quit) {error_report ("%s: channel %d has already quit" , __func__, i);qemu_mutex_unlock (&p->mutex); -return ; +return 0 ; } p->packet_num = multifd_send_state->packet_num++;@@ -602,6 +611,17 @@ void multifd_send_sync_main(QEMUFile *f) ram_counters.transferred += p->packet_len;qemu_mutex_unlock (&p->mutex);qemu_sem_post (&p->sem); + +if (flush_zero_copy) { +int ret; +Error *err =NULL ; + + ret =qio_channel_flush (p->c, &err); +if (ret <0 ) { +error_report_err (err); +return -1 ; + } + } }for (i =0 ; i <migrate_multifd_channels (); i++) {MultiFDSendParams *p = &multifd_send_state->params[i];
migration/multifd.c void multifd_send_sync_main(QEMUFile *f) · line 635
Quoted 6 lines
migration/multifd.c +2
-0@@ -610,6 +630,8 @@ void multifd_send_sync_main(QEMUFile *f)qemu_sem_wait (&p->sem_sync); }trace_multifd_send_sync_main (multifd_send_state->packet_num); + +return 0 ; }
We are leaving pages is flight for potentially a lot of time. I *think* that we can sync shorter than that.
migration/multifd.c static void *multifd_send_thread(void *opaque) · line 697
Quoted 12 lines
migration/multifd.c +2
-2static void *multifd_send_thread (void *opaque)@@ -668,8 +690,8 @@ static void *multifd_send_thread(void *opaque) p->iov[0 ].iov_len = p->packet_len; p->iov[0 ].iov_base = p->packet; - ret =qio_channel_writev_all (p->c, p->iov, p->iovs_num, - &local_err); + ret =qio_channel_writev_full_all (p->c, p->iov, p->iovs_num,NULL , +0 , p->write_flags, &local_err);if (ret !=0 ) {break ; }
I still think that it would be better to have a sync here in each thread. I don't know the size, but once every couple megabytes of RAM or so.
I did a change on:
commit d48c3a044537689866fe44e65d24c7d39a68868a
Author: Juan Quintela <[email protected]> Date: Fri Nov 19 15:35:58 2021 +0100
multifd: Use a single writev on the send side
Until now, we wrote the packet header with write(), and the rest of the pages with writev(). Just increase the size of the iovec and do a single writev().
And now we need to "perserve" this header until we do the sync, otherwise we are overwritting it with other things.
What testing have you done after this commit?
Notice that it is not _complicated_ to fix it, I will try to come with some idea on monday, but basically is having an array of buffers for each thread, and store them until we call a sync().
Later, Juan.
Re: [PATCH v8 5/5] multifd: Implement zero copy write in multifd migration (multifd-zero-copy)
Hello Juan, thanks for the feedback!
Quoted 19 lines · Juan Quintela
Quoted 17 lines · Leonardo Bras
Implement zero copy send on
nocomp_send_write(), by making use of QIOChannel writev + flags & flush interface.Change
multifd_send_sync_main()soflush_zero_copy()can be called after each iteration in order to make sure all dirty pages are sent before a new iteration is started. It will also flush at the beginning and at the end of migration.Also make it return -1 if
flush_zero_copy()fails, in order to cancel the migration process, and avoid resuming the guest in the target host without receiving all current RAM.This will work fine on RAM migration because the RAM pages are not usually freed, and there is no problem on changing the pages content between
writev_zero_copy()and the actual sending of the buffer, because this change will dirty the page and cause it to be re-sent on a next iteration anyway.
Quoted 5 lines
Quoted 1 line
A lot of locked memory may be needed in order to use multid migration
^^^^^^multifd.
I can fix it on the commit.
No worries, fixed for v9.
Quoted 13 lines
Quoted 13 lines
migration/migration.c +6
-1@@ -1479,7 +1479,16 @@ static bool migrate_params_check(MigrationParameters *params, Error **errp)error_prepend (errp,"Invalid mapping given for block-bitmap-mapping: " );return false; } - +#ifdef CONFIG_LINUX +if (params->zero_copy_send && + (!migrate_use_multifd () || + params->multifd_compression !=MULTIFD_COMPRESSION_NONE || + (params->tls_creds && *params->tls_creds))) { + error_setg(errp,
Quoted 8 lines
migration/migration.c static bool migrate_params_check(MigrationParameters *params, Error **errp) · line 1493
Quoted 6 lines
migration/migration.c +4
-0+ "Zero copy only available for non-compressed non-TLS multifd migration"); + return false; + } +#endif return true; }Test is long, but it is exactly what we need. Good.
Thanks!
Quoted 82 lines
Quoted 67 lines
migration/multifd.c +24
-4@@ -568,19 +568,28 @@ void multifd_save_cleanup(void) multifd_send_state =NULL ; } -void multifd_send_sync_main (QEMUFile *f) +int multifd_send_sync_main (QEMUFile *f) {int i; +bool flush_zero_copy;if (!migrate_use_multifd ()) { -return ; +return 0 ; }if (multifd_send_state->pages->num) {if (multifd_send_pages (f) <0 ) {error_report ("%s: multifd_send_pages fail" , __func__); -return ; +return 0 ; } } + +/* +* When using zero-copy, it's necessary to flush after each iteration to +* make sure pages from earlier iterations don't end up replacing newer +* pages. +*/ + flush_zero_copy =migrate_use_zero_copy_send (); +for (i =0 ; i <migrate_multifd_channels (); i++) {MultiFDSendParams *p = &multifd_send_state->params[i];@@ -591,7 +600,7 @@ void multifd_send_sync_main(QEMUFile *f)if (p->quit) {error_report ("%s: channel %d has already quit" , __func__, i);qemu_mutex_unlock (&p->mutex); -return ; +return 0 ; } p->packet_num = multifd_send_state->packet_num++;@@ -602,6 +611,17 @@ void multifd_send_sync_main(QEMUFile *f) ram_counters.transferred += p->packet_len;qemu_mutex_unlock (&p->mutex);qemu_sem_post (&p->sem); + +if (flush_zero_copy) { +int ret; +Error *err =NULL ; + + ret =qio_channel_flush (p->c, &err); +if (ret <0 ) { +error_report_err (err); +return -1 ; + } + } }for (i =0 ; i <migrate_multifd_channels (); i++) {MultiFDSendParams *p = &multifd_send_state->params[i];migration/multifd.c void multifd_send_sync_main(QEMUFile *f) · line 635
Quoted 6 lines
migration/multifd.c +2
-0@@ -610,6 +630,8 @@ void multifd_send_sync_main(QEMUFile *f)qemu_sem_wait (&p->sem_sync); }trace_multifd_send_sync_main (multifd_send_state->packet_num); + +return 0 ; }We are leaving pages is flight for potentially a lot of time. I *think* that we can sync shorter than that.
Quoted 5 lines
migration/multifd.c +0
-0static void *multifd_send_thread (void *opaque)@@ -668,8 +690,8 @@ static void *multifd_send_thread(void *opaque) p->iov[0 ].iov_len = p->packet_len; p->iov[0 ].iov_base = p->packet;
Quoted 11 lines
migration/multifd.c static void *multifd_send_thread(void *opaque) · line 697
Quoted 7 lines
migration/multifd.c +2
-2- ret =qio_channel_writev_all (p->c, p->iov, p->iovs_num, - &local_err); + ret =qio_channel_writev_full_all (p->c, p->iov, p->iovs_num,NULL , +0 , p->write_flags, &local_err);if (ret !=0 ) {break ; }I still think that it would be better to have a sync here in each thread. I don't know the size, but once every couple megabytes of RAM or so.
This seems a good idea, since the first iteration may take a while, and we may take a lot of time to fail if something goes wrong with zerocopy at the start of iteration 1.
On the other hand, flushing takes some time, and doing it a lot may take away some of the performance improvements.
Maybe we could move the flushing to a thread of it's own, if it becomes a problem.
Quoted 13 lines
I did a change on:
commit
d48c3a044537689866fe44e65d24c7d39a68868aAuthor: Juan Quintela <[email protected]> Date: Fri Nov 19 15:35:58 2021 +0100multifd: Use a single writev on the send side
Until now, we wrote the packet header with
write(), and the rest of the pages withwritev(). Just increase the size of the iovec and do a singlewritev().
Quoted 5 lines
Signed-off-by Juan Quintela <[email protected]>Reviewed-by Dr. David Alan Gilbert <[email protected]>And now we need to "perserve" this header until we do the sync, otherwise we are overwritting it with other things.
Yeah, that is a problem I faced on non-multifd migration, and a reason why I choose to implement directly in multifd.
Quoted 2 lines
What testing have you done after this commit?
Not much, but this will probably become an issue with bigger guests.
Quoted 4 lines
Notice that it is not _complicated_ to fix it, I will try to come with some idea on monday, but basically is having an array of buffers for each thread, and store them until we call a
sync().
That will probably work, we can use MultiFDRecvParams->packet as this array, right?
We can start with a somehow small buffer size and grow if it ever gets full. (Full means a sync + g_try_malloc0 + g_free)
By my calculations, it should take 1,8kB each element on a 4k PAGESIZE.
What do you think?
Best regards,
Leo
Re: [PATCH v8 5/5] multifd: Implement zero copy write in multifd migration (multifd-zero-copy)
Quoted 189 lines · Leonardo Bras Soares Passos
Hello Juan, thanks for the feedback!
Quoted 19 lines · Juan Quintela
Quoted 17 lines · Leonardo Bras
Implement zero copy send on
nocomp_send_write(), by making use of QIOChannel writev + flags & flush interface.Change
multifd_send_sync_main()soflush_zero_copy()can be called after each iteration in order to make sure all dirty pages are sent before a new iteration is started. It will also flush at the beginning and at the end of migration.Also make it return -1 if
flush_zero_copy()fails, in order to cancel the migration process, and avoid resuming the guest in the target host without receiving all current RAM.This will work fine on RAM migration because the RAM pages are not usually freed, and there is no problem on changing the pages content between
writev_zero_copy()and the actual sending of the buffer, because this change will dirty the page and cause it to be re-sent on a next iteration anyway.Quoted 5 lines
Quoted 1 line
A lot of locked memory may be needed in order to use multid migration
^^^^^^multifd.
I can fix it on the commit.
No worries, fixed for v9.
Quoted 13 lines
Quoted 13 lines
migration/migration.c +6
-1@@ -1479,7 +1479,16 @@ static bool migrate_params_check(MigrationParameters *params, Error **errp)error_prepend (errp,"Invalid mapping given for block-bitmap-mapping: " );return false; } - +#ifdef CONFIG_LINUX +if (params->zero_copy_send && + (!migrate_use_multifd () || + params->multifd_compression !=MULTIFD_COMPRESSION_NONE || + (params->tls_creds && *params->tls_creds))) { + error_setg(errp,Quoted 8 lines
migration/migration.c static bool migrate_params_check(MigrationParameters *params, Error **errp) · line 1493
Quoted 6 lines
migration/migration.c +4
-0+ "Zero copy only available for non-compressed non-TLS multifd migration"); + return false; + } +#endif return true; }Test is long, but it is exactly what we need. Good.
Thanks!
Quoted 82 lines
Quoted 67 lines
migration/multifd.c +24
-4@@ -568,19 +568,28 @@ void multifd_save_cleanup(void) multifd_send_state =NULL ; } -void multifd_send_sync_main (QEMUFile *f) +int multifd_send_sync_main (QEMUFile *f) {int i; +bool flush_zero_copy;if (!migrate_use_multifd ()) { -return ; +return 0 ; }if (multifd_send_state->pages->num) {if (multifd_send_pages (f) <0 ) {error_report ("%s: multifd_send_pages fail" , __func__); -return ; +return 0 ; } } + +/* +* When using zero-copy, it's necessary to flush after each iteration to +* make sure pages from earlier iterations don't end up replacing newer +* pages. +*/ + flush_zero_copy =migrate_use_zero_copy_send (); +for (i =0 ; i <migrate_multifd_channels (); i++) {MultiFDSendParams *p = &multifd_send_state->params[i];@@ -591,7 +600,7 @@ void multifd_send_sync_main(QEMUFile *f)if (p->quit) {error_report ("%s: channel %d has already quit" , __func__, i);qemu_mutex_unlock (&p->mutex); -return ; +return 0 ; } p->packet_num = multifd_send_state->packet_num++;@@ -602,6 +611,17 @@ void multifd_send_sync_main(QEMUFile *f) ram_counters.transferred += p->packet_len;qemu_mutex_unlock (&p->mutex);qemu_sem_post (&p->sem); + +if (flush_zero_copy) { +int ret; +Error *err =NULL ; + + ret =qio_channel_flush (p->c, &err); +if (ret <0 ) { +error_report_err (err); +return -1 ; + } + } }for (i =0 ; i <migrate_multifd_channels (); i++) {MultiFDSendParams *p = &multifd_send_state->params[i];migration/multifd.c void multifd_send_sync_main(QEMUFile *f) · line 635
Quoted 6 lines
migration/multifd.c +2
-0@@ -610,6 +630,8 @@ void multifd_send_sync_main(QEMUFile *f)qemu_sem_wait (&p->sem_sync); }trace_multifd_send_sync_main (multifd_send_state->packet_num); + +return 0 ; }We are leaving pages is flight for potentially a lot of time. I *think* that we can sync shorter than that.
Quoted 5 lines
migration/multifd.c +0
-0static void *multifd_send_thread (void *opaque)@@ -668,8 +690,8 @@ static void *multifd_send_thread(void *opaque) p->iov[0 ].iov_len = p->packet_len; p->iov[0 ].iov_base = p->packet;Quoted 11 lines
migration/multifd.c static void *multifd_send_thread(void *opaque) · line 697
Quoted 7 lines
migration/multifd.c +2
-2- ret =qio_channel_writev_all (p->c, p->iov, p->iovs_num, - &local_err); + ret =qio_channel_writev_full_all (p->c, p->iov, p->iovs_num,NULL , +0 , p->write_flags, &local_err);if (ret !=0 ) {break ; }I still think that it would be better to have a sync here in each thread. I don't know the size, but once every couple megabytes of RAM or so.
This seems a good idea, since the first iteration may take a while, and we may take a lot of time to fail if something goes wrong with zerocopy at the start of iteration 1.
On the other hand, flushing takes some time, and doing it a lot may take away some of the performance improvements.
Maybe we could move the flushing to a thread of it's own, if it becomes a problem.
Quoted 13 lines
I did a change on:
commit
d48c3a044537689866fe44e65d24c7d39a68868aAuthor: Juan Quintela <[email protected]> Date: Fri Nov 19 15:35:58 2021 +0100multifd: Use a single writev on the send side
Until now, we wrote the packet header with
write(), and the rest of the pages withwritev(). Just increase the size of the iovec and do a singlewritev().Quoted 5 lines
Signed-off-by Juan Quintela <[email protected]>Reviewed-by Dr. David Alan Gilbert <[email protected]>And now we need to "perserve" this header until we do the sync, otherwise we are overwritting it with other things.
Yeah, that is a problem I faced on non-multifd migration, and a reason why I choose to implement directly in multifd.
Quoted 2 lines
What testing have you done after this commit?
Not much, but this will probably become an issue with bigger guests.
Quoted 1 line
Quoted 8 lines
Quoted 3 lines
Notice that it is not _complicated_ to fix it, I will try to come with some idea on monday, but basically is having an array of buffers for each thread, and store them until we call a
sync().That will probably work, we can use
MultiFDRecvParams->packetas this array, right?
We can start with a somehow small buffer size and grow if it ever gets full. (Full means a sync +g_try_malloc0+g_free)
Or maybe use the array size as a size limiter before flushing, so it will always flush after BUFFSIZE headers are sent.
Quoted 7 lines
By my calculations, it should take 1,8kB each element on a 4k PAGESIZE.
What do you think?
Best regards,
Leo
Re: [PATCH v8 5/5] multifd: Implement zero copy write in multifd migration (multifd-zero-copy)
In reply to Juan QuintelaQuoted 15 lines · Juan Quintela
I did a change on:
commit
d48c3a044537689866fe44e65d24c7d39a68868aAuthor: Juan Quintela <[email protected]> Date: Fri Nov 19 15:35:58 2021 +0100multifd: Use a single writev on the send side
Until now, we wrote the packet header with
write(), and the rest of the pages withwritev(). Just increase the size of the iovec and do a singlewritev().Signed-off-by Juan Quintela <[email protected]>Reviewed-by Dr. David Alan Gilbert <[email protected]>
Quoted 8 lines
And now we need to "perserve" this header until we do the sync, otherwise we are overwritting it with other things.
What testing have you done after this commit?
Notice that it is not _complicated_ to fix it, I will try to come with some idea on monday, but basically is having an array of buffers for each thread, and store them until we call a
sync().
Or can we conditionally merge the two write()s? IMHO the array of buffers idea sounds too complicated, and I'm not extremely sure whether it'll pay off at last. We could keep the two write()s with ZEROCOPY enabled, and use the merged version otherwise.
Btw, is there any performance measurements for above commit d48c3a044537? I had a feeling that the single write() may not help that much, because for multifd the bottleneck should be on the nic not on the processor.
IOW, we could find that the major time used does not fall into the user<->kernel switches (which is where the extra overhead of write() syscall, iiuc), but we simply blocked on any of the write()s because the socket write buffer is full... So we could have saved some cpu cycles by merging the calls, but performance-wise we may not get much.
Thanks,
Signature · 1 line
Peter Xu
Re: [PATCH v8 5/5] multifd: Implement zero copy write in multifd migration (multifd-zero-copy)
Quoted 22 lines · Peter Xu
Quoted 20 lines · Juan Quintela
I did a change on:
commit
d48c3a044537689866fe44e65d24c7d39a68868aAuthor: Juan Quintela <[email protected]> Date: Fri Nov 19 15:35:58 2021 +0100multifd: Use a single writev on the send side
Until now, we wrote the packet header with
write(), and the rest of the pages withwritev(). Just increase the size of the iovec and do a singlewritev().Signed-off-by Juan Quintela <[email protected]>Reviewed-by Dr. David Alan Gilbert <[email protected]>And now we need to "perserve" this header until we do the sync, otherwise we are overwritting it with other things.
What testing have you done after this commit?
Quoted 8 lines
Quoted 3 lines
Notice that it is not _complicated_ to fix it, I will try to come with some idea on monday, but basically is having an array of buffers for each thread, and store them until we call a
sync().Or can we conditionally merge the two write()s? IMHO the array of buffers idea sounds too complicated, and I'm not extremely sure whether it'll pay off at last. We could keep the two write()s with ZEROCOPY enabled, and use the merged version otherwise.
I think that's a great idea!
It would optimize the non-zerocopy version while letting us have a simpler zerocopy implementation.
The array of buffers implementation would either require us to have a 'large' amount of memory for keeping the headers, or having flush happening too often.
Quoted 4 lines
Btw, is there any performance measurements for above commit
d48c3a044537? I had a feeling that the singlewrite()may not help that much, because for multifd the bottleneck should be on the nic not on the processor.
I am quite curious about those numbers too.
Quoted 7 lines
IOW, we could find that the major time used does not fall into the user<->kernel switches (which is where the extra overhead of
write()syscall, iiuc), but we simply blocked on any of the write()s because the socket write buffer is full... So we could have saved some cpu cycles by merging the calls, but performance-wise we may not get much.
Quoted 5 lines
Thanks,
--
Peter Xu
Thanks Peter!