Received: from eggs.gnu.org ([208.118.235.92]:59616) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UIbUL-0006SE-4q for qemu-devel@nongnu.org; Thu, 21 Mar 2013 05:08:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UIbUJ-0005XA-U6 for qemu-devel@nongnu.org; Thu, 21 Mar 2013 05:08:45 -0400 Received: from mx1.redhat.com ([209.132.183.28]:30206) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UIbUJ-0005Ww-MM for qemu-devel@nongnu.org; Thu, 21 Mar 2013 05:08:43 -0400 From: Orit Wasserman Date: Thu, 21 Mar 2013 11:09:19 +0200 Message-Id: <1363856971-4601-1-git-send-email-owasserm@redhat.com> Subject: [Qemu-devel] [RFC 00/12] Migration: Remove copying of guest ram pages List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: Orit Wasserman , pbonzini@redhat.com, mst@redhat.com, chegu_vinod@hp.com, quintela@redhat.com In migration all data is copied to a static buffer in QEMUFile, this hurts our network bandwidth and CPU usage especially with large guests. We switched to iovec for storing different buffers to send (even a byte field is considered as a buffer) and use writev to send the iovec. writev was chosen (as apposed to sendmsg) because it supprts non socket fds. Guest memory pages are not copied by calling a new function qemu_put_buffer_no_copy. The page header data and device state data are still copied into the static buffer. This data consists of a lot of bytes and integer fields and the static buffer is used to store it during batching. Another improvement is changing qemu_putbe64/32/16 to create a single buffer instead of several byte sized buffer. Orit Wasserman (12): Add iov_writev to use writev to send iovec (also for files) Add QemuFileWritevBuffer QemuFileOps Add socket_writev_buffer function Add stdio_writev_buffer function Add block_writev_buffer function Update bytes_xfer in qemu_put_byte Store the data to send also in iovec Use writev ops instead of put_buffer ops More optimized qemu_put_be64/32/16 Add qemu_put_buffer_no_copy Use qemu_put_buffer_no_copy for guest memory pages Bye Bye put_buffer arch_init.c | 2 +- include/migration/qemu-file.h | 20 ++++--- include/qemu/iov.h | 12 ++++ savevm.c | 130 +++++++++++++++++++++++++----------------- util/iov.c | 36 ++++++++++++ 5 files changed, 139 insertions(+), 61 deletions(-) -- 1.7.11.7