Initial support for multi-process Qemu
Subsystems (2)
v11 cover Initial support for multi-process Qemu
Hello,
This is the v11 of the patchset. Thank you very much for the review of the v10 of the series. We are glad to hear the patchset is getting in a better shape.
We made changes to the following patches in this series.
[PATCH v10 06/19] multi-process: define MPQemuMsg format and transmission functions [PATCH v10 08/19] multi-process: Associate fd of a PCIDevice with its object [PATCH v10 10/19] multi-process: introduce proxy object
To touch upon the history of this project, we posted the Proof Of Concept patches before the BoF session in 2018. Subsequently, we posted 9 versions on the qemu-devel mailist. You can find them by following the links below ([1] - [9]).
Following people contributed to the design and
implementation of this project:
Jagannathan Raman <[email protected]>
Elena Ufimtseva <[email protected]>
John G Johnson <[email protected]>
Stefan Hajnoczi <[email protected]>
Konrad Wilk <[email protected]>
Kanth Ghatraju <[email protected]>
We would like to thank QEMU community for your feedback in the design and implementation of this project.
Qemu wiki page:
https://wiki.qemu.org/Features/MultiProcessQEMU
For the full concept writeup about QEMU multi-process, please refer to docs/devel/qemu-multiprocess.rst. Also see docs/qemu-multiprocess.txt for usage information.
We welcome all your ideas, concerns, and questions for this patchset. [POC]: https://www.mail-archive.com/[email protected]/msg566538.html
[1]: https://www.mail-archive.com/[email protected]/msg602285.html
[2]: https://www.mail-archive.com/[email protected]/msg624877.html
[3]: https://www.mail-archive.com/[email protected]/msg642000.html
[4]: https://www.mail-archive.com/[email protected]/msg655118.html
[5]: https://www.mail-archive.com/[email protected]/msg682429.html
[6]: https://www.mail-archive.com/[email protected]/msg697484.html
[7]: https://patchew.org/QEMU/[email protected]/
[8]: https://www.mail-archive.com/[email protected]/msg727007.html
[9]: https://www.mail-archive.com/[email protected]/msg734275.html
[10]: https://www.mail-archive.com/[email protected]/msg747638.html
Elena Ufimtseva (7):
multi-process: add qio channel function to transmit
multi-process: define MPQemuMsg format and transmission functions
multi-process: introduce proxy object
multi-process: add proxy communication functions
multi-process: Forward PCI config space acceses to the remote process
multi-process: perform device reset in the remote process
multi-process: add configure and usage information
Jagannathan Raman (11):
memory: alloc RAM from file at offset
multi-process: Add config option for multi-process QEMU
multi-process: setup PCI host bridge for remote device
multi-process: setup a machine object for remote device process
multi-process: Initialize message handler in remote device
multi-process: Associate fd of a PCIDevice with its object
multi-process: setup memory manager for remote device
multi-process: PCI BAR read/write handling for proxy & remote
endpoints
multi-process: Synchronize remote memory
multi-process: create IOHUB object to handle irq
multi-process: Retrieve PCI info from remote process
John G Johnson (1):
multi-process: add the concept description to
docs/devel/qemu-multiprocess
MAINTAINERS | 26 ++ backends/hostmem-memfd.c | 2 +- configure | 10 + docs/devel/index.rst | 1 + docs/devel/multi-process.rst | 966 ++++++++++++++++++++++++++++++++++++++++ docs/multi-process.rst | 67 +++ hw/i386/meson.build | 5 + hw/i386/remote-iohub.c | 123 +++++ hw/i386/remote-memory.c | 58 +++ hw/i386/remote-msg.c | 241 ++++++++++ hw/i386/remote-obj.c | 154 +++++++ hw/i386/remote.c | 79 ++++ hw/misc/ivshmem.c | 3 +- hw/pci-host/meson.build | 1 + hw/pci-host/remote.c | 75 ++++ hw/pci/memory-sync.c | 210 +++++++++ hw/pci/meson.build | 3 + hw/pci/proxy.c | 377 ++++++++++++++++ include/exec/memory.h | 2 + include/exec/ram_addr.h | 2 +- include/hw/i386/remote-iohub.h | 42 ++ include/hw/i386/remote-memory.h | 19 + include/hw/i386/remote-obj.h | 42 ++ include/hw/i386/remote.h | 40 ++ include/hw/pci-host/remote.h | 31 ++ include/hw/pci/memory-sync.h | 27 ++ include/hw/pci/pci_ids.h | 3 + include/hw/pci/proxy.h | 53 +++ include/io/channel.h | 24 + include/io/mpqemu-link.h | 98 ++++ include/qemu/mmap-alloc.h | 3 +- io/channel.c | 45 ++ io/meson.build | 2 + io/mpqemu-link.c | 303 +++++++++++++ meson.build | 1 + scripts/mpqemu-launcher.py | 49 ++ softmmu/memory.c | 3 +- softmmu/physmem.c | 11 +- util/mmap-alloc.c | 7 +- util/oslib-posix.c | 2 +- 40 files changed, 3197 insertions(+), 13 deletions(-) create mode 100644 docs/devel/multi-process.rst create mode 100644 docs/multi-process.rst create mode 100644 hw/i386/remote-iohub.c create mode 100644 hw/i386/remote-memory.c create mode 100644 hw/i386/remote-msg.c create mode 100644 hw/i386/remote-obj.c create mode 100644 hw/i386/remote.c create mode 100644 hw/pci-host/remote.c create mode 100644 hw/pci/memory-sync.c create mode 100644 hw/pci/proxy.c create mode 100644 include/hw/i386/remote-iohub.h create mode 100644 include/hw/i386/remote-memory.h create mode 100644 include/hw/i386/remote-obj.h create mode 100644 include/hw/i386/remote.h create mode 100644 include/hw/pci-host/remote.h create mode 100644 include/hw/pci/memory-sync.h create mode 100644 include/hw/pci/proxy.h create mode 100644 include/io/mpqemu-link.h create mode 100644 io/mpqemu-link.c create mode 100755 scripts/mpqemu-launcher.py
Signature · 1 line
1.8.3.1
PATCH v11 3/19 multi-process: setup PCI host bridge for remote device
PCI host bridge is setup for the remote device process. It is implemented using remote-pcihost object. It is an extension of the PCI host bridge setup by QEMU.
Remote-pcihost configures a PCI bus which could be used by the remote PCI device to latch on to.
--- MAINTAINERS | 8 +++++ hw/pci-host/meson.build | 1 + hw/pci-host/remote.c | 75 ++++++++++++++++++++++++++++++++++++++++++++ include/hw/pci-host/remote.h | 30 ++++++++++++++++++ 4 files changed, 114 insertions(+) create mode 100644 hw/pci-host/remote.c create mode 100644 include/hw/pci-host/remote.h
MAINTAINERS +8 -0
@@ -3095,6 +3095,14 @@ S: Maintained F: hw/semihosting/ F: include/hw/semihosting/ +Multi-process QEMU +M: Jagannathan Raman <[email protected]> +M: Elena Ufimtseva <[email protected]> +M: John G Johnson <[email protected]> +S: Maintained +F: hw/pci-host/remote.c +F: include/hw/pci-host/remote.h + Build and test automation ------------------------- Build and test automation
hw/pci-host/meson.build +1 -0
@@ -9,6 +9,7 @@ pci_ss.add(when: 'CONFIG_PCI_EXPRESS_XILINX', if_true: files('xilinx-pcie.c')) pci_ss.add(when: 'CONFIG_PCI_I440FX', if_true: files('i440fx.c')) pci_ss.add(when: 'CONFIG_PCI_SABRE', if_true: files('sabre.c')) pci_ss.add(when: 'CONFIG_XEN_IGD_PASSTHROUGH', if_true: files('xen_igd_pt.c')) +pci_ss.add(when: 'CONFIG_MPQEMU', if_true: files('remote.c')) # PPC devices pci_ss.add(when: 'CONFIG_PREP_PCI', if_true: files('prep.c'))
hw/pci-host/remote.c new file +75 -0
75 lines not on this page · raw message
include/hw/pci-host/remote.h new file +30 -0
@@ -0,0 +1,30 @@ +/* +* PCI Host for remote device +* +* Copyright © 2018, 2020 Oracle and/or its affiliates. +* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory. +* +*/ + +#ifndef REMOTE_PCIHOST_H +#define REMOTE_PCIHOST_H + +#include "exec/memory.h" +#include "hw/pci/pcie_host.h" + +#define TYPE_REMOTE_HOST_DEVICE "remote-pcihost" +#define REMOTE_HOST_DEVICE (obj) \ + OBJECT_CHECK(RemotePCIHost, (obj), TYPE_REMOTE_HOST_DEVICE) + +typedef struct RemotePCIHost { +/*< private >*/ +PCIExpressHost parent_obj; +/*< public >*/ + +MemoryRegion *mr_pci_mem; +MemoryRegion *mr_sys_io; +}RemotePCIHost ; + +#endif
Signature · 1 line
1.8.3.1
PATCH v11 8/19 multi-process: Associate fd of a PCIDevice with its object
In reply to Jagannathan RamanAssociate the file descriptor for a PCIDevice in remote process with DeviceState object.
--- MAINTAINERS | 2 + hw/i386/meson.build | 1 + hw/i386/remote-msg.c | 1 + hw/i386/remote-obj.c | 154 +++++++++++++++++++++++++++++++++++++++++++ include/hw/i386/remote-obj.h | 42 ++++++++++++ 5 files changed, 200 insertions(+) create mode 100644 hw/i386/remote-obj.c create mode 100644 include/hw/i386/remote-obj.h
MAINTAINERS +2 -0
@@ -3107,6 +3107,8 @@ F: include/hw/i386/remote.h F: io/mpqemu-link.c F: include/io/mpqemu-link.h F: hw/i386/remote-msg.c +F: include/hw/i386/remote-obj.h +F: hw/i386/remote-obj.c Build and test automation -------------------------
hw/i386/meson.build +1 -0
@@ -26,6 +26,7 @@ i386_ss.add(when: 'CONFIG_PC', if_true: files( 'port92.c')) i386_ss.add(when: 'CONFIG_MPQEMU', if_true: files('remote.c')) i386_ss.add(when: 'CONFIG_MPQEMU', if_true: files('remote-msg.c')) +i386_ss.add(when: 'CONFIG_MPQEMU', if_true: files('remote-obj.c')) subdir('kvm') subdir('xen')
hw/i386/remote-msg.c +1 -0
@@ -56,6 +56,7 @@ void coroutine_fn mpqemu_remote_msg_loop_co(void *data) } }qemu_system_shutdown_request (SHUTDOWN_CAUSE_GUEST_SHUTDOWN ); +g_free (com);return ; }
hw/i386/remote-obj.c new file +154 -0
154 lines not on this page · raw message
include/hw/i386/remote-obj.h new file +42 -0
42 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH v11 11/19 multi-process: add proxy communication functions
In reply to Jagannathan RamanFrom: Elena Ufimtseva <[email protected]>
--- include/io/mpqemu-link.h | 4 ++++ io/mpqemu-link.c | 38 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 42 insertions(+)
include/io/mpqemu-link.h +4 -0
@@ -15,6 +15,8 @@#include "qemu/thread.h" #include "io/channel.h" #include "exec/hwaddr.h" +#include "io/channel-socket.h" +#include "hw/pci/proxy.h" #define REMOTE_MAX_FDS 8
@@ -65,6 +67,8 @@ typedef struct {int num_fds; }QEMU_PACKED MPQemuMsg; +uint64_t mpqemu_msg_send_and_await_reply (MPQemuMsg *msg,PCIProxyDev *pdev, +Error **errp);void mpqemu_msg_send (MPQemuMsg *msg,QIOChannel *ioc,Error **errp);void mpqemu_msg_recv (MPQemuMsg *msg,QIOChannel *ioc,Error **errp);
io/mpqemu-link.c +38 -0
@@ -17,6 +17,7 @@#include "qemu/iov.h" #include "qemu/error-report.h" #include "qemu/main-loop.h" +#include "io/channel.h" /* * Send message over the ioc QIOChannel.
@@ -214,6 +215,43 @@ fail: } } +/* +* Called from VCPU thread in non-coroutine context. +*/ +uint64_t mpqemu_msg_send_and_await_reply (MPQemuMsg *msg,PCIProxyDev *pdev, +Error **errp) +{ +MPQemuMsg msg_reply = {0 }; +uint64_t ret =UINT64_MAX ; +Error *local_err =NULL ; + +qemu_mutex_lock (&pdev->io_mutex); +mpqemu_msg_send (msg, pdev->ioc, &local_err); +if (local_err) { +error_propagate (errp, local_err); + goto exit_send; + } + +mpqemu_msg_recv (&msg_reply, pdev->ioc, &local_err); +if (local_err) { +error_propagate (errp, local_err); + goto exit_send; + } + +if (!mpqemu_msg_valid (&msg_reply) || msg_reply.cmd !=RET_MSG ) { +error_setg (errp,"ERROR: Invalid reply received for command %d" , + msg->cmd); + goto exit_send; + }else { + ret = msg_reply.data.u64; + } + + exit_send: +qemu_mutex_unlock (&pdev->io_mutex); + +return ret; +} +bool mpqemu_msg_valid (MPQemuMsg *msg) {if (msg->cmd >=MPQEMU_CMD_MAX && msg->cmd <0 ) {
Signature · 1 line
1.8.3.1
PATCH v11 1/19 memory: alloc RAM from file at offset
In reply to Jagannathan RamanAllow RAM MemoryRegion to be created from an offset in a file, instead of allocating at offset of 0 by default. This is needed to synchronize RAM between QEMU & remote process.
--- backends/hostmem-memfd.c | 2 +- hw/misc/ivshmem.c | 3 ++- include/exec/memory.h | 2 ++ include/exec/ram_addr.h | 2 +- include/qemu/mmap-alloc.h | 3 ++- softmmu/memory.c | 3 ++- softmmu/physmem.c | 11 +++++++---- util/mmap-alloc.c | 7 ++++--- util/oslib-posix.c | 2 +- 9 files changed, 22 insertions(+), 13 deletions(-)
backends/hostmem-memfd.c +1 -1
@@ -55,7 +55,7 @@ memfd_backend_memory_alloc(HostMemoryBackend *backend, Error **errp) name =host_memory_backend_get_name (backend);memory_region_init_ram_from_fd (&backend->mr,OBJECT (backend), name, backend->size, - backend->share, fd, errp); + backend->share, fd,0 , errp);g_free (name); }
hw/misc/ivshmem.c +2 -1
@@ -494,7 +494,8 @@ static void process_msg_shmem(IVShmemState *s, int fd, Error **errp)/* mmap the region and map into the BAR2 */ memory_region_init_ram_from_fd (&s->server_bar2,OBJECT (s), -"ivshmem.bar2" , size, true, fd, &local_err); +"ivshmem.bar2" , size, true, fd,0 , + &local_err);if (local_err) {error_propagate (errp, local_err);return ;
include/exec/memory.h +2 -0
@@ -934,6 +934,7 @@ void memory_region_init_ram_from_file(MemoryRegion *mr,* @size: size of the region. * @share: %true if memory must be mmaped with the MAP_SHARED flag * @fd: the fd to mmap. +* @offset: offset within the file referenced by fd * @errp: pointer to Error*, to store an error if it happens. * * Note that this function does not do anything to cause the data in the
@@ -945,6 +946,7 @@ void memory_region_init_ram_from_fd(MemoryRegion *mr,uint64_t size, bool share, int fd, + ram_addr_t offset, Error **errp);#endif
include/exec/ram_addr.h +1 -1
@@ -121,7 +121,7 @@ RAMBlock *qemu_ram_alloc_from_file(ram_addr_t size, MemoryRegion *mr, Error **errp);RAMBlock *qemu_ram_alloc_from_fd (ram_addr_t size,MemoryRegion *mr,uint32_t ram_flags,int fd, -Error **errp); +off_t offset,Error **errp);RAMBlock *qemu_ram_alloc_from_ptr (ram_addr_t size,void *host,MemoryRegion *mr,Error **errp);
include/qemu/mmap-alloc.h +2 -1
@@ -25,7 +25,8 @@ void *qemu_ram_mmap(int fd,size_t size, size_t align, bool shared, - bool is_pmem); + bool is_pmem, + off_t start);void qemu_ram_munmap (int fd,void *ptr,size_t size);
softmmu/memory.c +2 -1
@@ -1591,6 +1591,7 @@ void memory_region_init_ram_from_fd(MemoryRegion *mr,uint64_t size, bool share, int fd, + ram_addr_t offset, Error **errp) { Error *err = NULL;
@@ -1600,7 +1601,7 @@ void memory_region_init_ram_from_fd(MemoryRegion *mr, mr->destructor = memory_region_destructor_ram; mr->ram_block =qemu_ram_alloc_from_fd (size, mr, share ?RAM_SHARED :0 , - fd, &err); + fd, offset, &err); mr->dirty_log_mask =tcg_enabled () ? (1 <<DIRTY_MEMORY_CODE ) :0 ;if (err) { mr->size =int128_zero ();
softmmu/physmem.c +7 -4
@@ -1461,6 +1461,7 @@ static void *file_ram_alloc(RAMBlock *block,ram_addr_t memory, int fd, bool truncate, + off_t offset, Error **errp) { void *area;
@@ -1511,7 +1512,8 @@ static void *file_ram_alloc(RAMBlock *block, } area =qemu_ram_mmap (fd, memory, block->mr->align, - block->flags &RAM_SHARED , block->flags &RAM_PMEM ); + block->flags &RAM_SHARED , block->flags &RAM_PMEM , + offset);if (area ==MAP_FAILED ) {error_setg_errno (errp, errno,"unable to map backing store for guest RAM" );
@@ -1943,7 +1945,7 @@ static void ram_block_add(RAMBlock *new_block, Error **errp, bool shared)#ifdef CONFIG_POSIX RAMBlock *qemu_ram_alloc_from_fd (ram_addr_t size,MemoryRegion *mr,uint32_t ram_flags,int fd, -Error **errp) +off_t offset,Error **errp) {RAMBlock *new_block;Error *local_err =NULL ;
@@ -1996,7 +1998,8 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr, new_block->used_length = size; new_block->max_length = size; new_block->flags = ram_flags; - new_block->host =file_ram_alloc (new_block, size, fd, !file_size, errp); + new_block->host =file_ram_alloc (new_block, size, fd, !file_size, offset, + errp);if (!new_block->host) {g_free (new_block);return NULL ;
@@ -2026,7 +2029,7 @@ RAMBlock *qemu_ram_alloc_from_file(ram_addr_t size, MemoryRegion *mr,return NULL ; } - block =qemu_ram_alloc_from_fd (size, mr, ram_flags, fd, errp); + block =qemu_ram_alloc_from_fd (size, mr, ram_flags, fd,0 , errp);if (!block) {if (created) {unlink (mem_path);
util/mmap-alloc.c +4 -3
@@ -86,7 +86,8 @@ void *qemu_ram_mmap(int fd,size_t size, size_t align, bool shared, - bool is_pmem) + bool is_pmem, + off_t start) { int flags;int map_sync_flags =0 ;
@@ -147,7 +148,7 @@ void *qemu_ram_mmap(int fd, offset =QEMU_ALIGN_UP ((uintptr_t )guardptr, align) - (uintptr_t )guardptr; ptr =mmap (guardptr + offset, size,PROT_READ |PROT_WRITE , - flags | map_sync_flags, fd,0 ); + flags | map_sync_flags, fd, start);if (ptr ==MAP_FAILED && map_sync_flags) {if (errno ==ENOTSUP ) {
@@ -172,7 +173,7 @@ void *qemu_ram_mmap(int fd,* we will remove these flags to handle compatibility. */ ptr =mmap (guardptr + offset, size,PROT_READ |PROT_WRITE , - flags, fd,0 ); + flags, fd, start); }if (ptr ==MAP_FAILED ) {
util/oslib-posix.c +1 -1
@@ -227,7 +227,7 @@ void *qemu_memalign(size_t alignment, size_t size)void *qemu_anon_ram_alloc (size_t size,uint64_t *alignment,bool shared) {size_t align =QEMU_VMALLOC_ALIGN ; -void *ptr =qemu_ram_mmap (-1 , size, align, shared, false); +void *ptr =qemu_ram_mmap (-1 , size, align, shared, false,0 );if (ptr ==MAP_FAILED ) {return NULL ;
Signature · 1 line
1.8.3.1
PATCH v11 10/19 multi-process: introduce proxy object
In reply to Jagannathan RamanFrom: Elena Ufimtseva <[email protected]>
Defines a PCI Device proxy object as a child of TYPE_PCI_DEVICE.
--- MAINTAINERS | 2 ++ hw/pci/meson.build | 1 + hw/pci/proxy.c | 98 ++++++++++++++++++++++++++++++++++++++++++++++++++ include/hw/pci/proxy.h | 36 +++++++++++++++++++ 4 files changed, 137 insertions(+) create mode 100644 hw/pci/proxy.c create mode 100644 include/hw/pci/proxy.h
MAINTAINERS +2 -0
@@ -3111,6 +3111,8 @@ F: include/hw/i386/remote-obj.h F: hw/i386/remote-obj.c F: include/hw/i386/remote-memory.h F: hw/i386/remote-memory.c +F: hw/pci/proxy.c +F: include/hw/pci/proxy.h Build and test automation -------------------------
hw/pci/meson.build +1 -0
@@ -12,6 +12,7 @@ pci_ss.add(files( # allow plugging PCIe devices into PCI buses, include them even if # CONFIG_PCI_EXPRESS=n. pci_ss.add(files('pcie.c', 'pcie_aer.c')) +pci_ss.add(when: 'CONFIG_MPQEMU', if_true: files('proxy.c')) softmmu_ss.add(when: 'CONFIG_PCI_EXPRESS', if_true: files('pcie_port.c', 'pcie_host.c')) softmmu_ss.add_all(when: 'CONFIG_PCI', if_true: pci_ss)
hw/pci/proxy.c new file +98 -0
98 lines not on this page · raw message
include/hw/pci/proxy.h new file +36 -0
@@ -0,0 +1,36 @@ +/* +* Copyright © 2018, 2020 Oracle and/or its affiliates. +* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory. +* +*/ + +#ifndef PROXY_H +#define PROXY_H + +#include "hw/pci/pci.h" +#include "io/channel.h" + +#define TYPE_PCI_PROXY_DEV "x-pci-proxy-dev" + +#define PCI_PROXY_DEV (obj) \ + OBJECT_CHECK(PCIProxyDev, (obj), TYPE_PCI_PROXY_DEV) +typedef struct PCIProxyDev PCIProxyDev ; + +struct PCIProxyDev { +PCIDevice parent_dev; +char *fd; + +/* +* Mutex used to protect the QIOChannel fd from +* the concurrent access by the VCPUs since proxy +* blocks while awaiting for the replies from the +* process remote. +*/ +QemuMutex io_mutex; +QIOChannel *ioc; +Error *migration_blocker; +}; + +#endif /* PROXY_H */
Signature · 1 line
1.8.3.1
PATCH v11 13/19 multi-process: PCI BAR read/write handling for proxy & remote endpoints
In reply to Jagannathan RamanProxy device object implements handler for PCI BAR writes and reads. The handler uses BAR_WRITE/BAR_READ message to communicate to the remote process with the BAR address and value to be written/read. The remote process implements handler for BAR_WRITE/BAR_READ message.
--- hw/i386/remote-msg.c | 87 ++++++++++++++++++++++++++++++++++++++++++++++++ hw/pci/proxy.c | 60 +++++++++++++++++++++++++++++++++ include/hw/pci/proxy.h | 10 ++++++ include/io/mpqemu-link.h | 10 ++++++ io/mpqemu-link.c | 6 ++++ 5 files changed, 173 insertions(+)
hw/i386/remote-msg.c +87 -0
@@ -16,11 +16,14 @@#include "qapi/error.h" #include "sysemu/runstate.h" #include "hw/pci/pci.h" +#include "exec/memattrs.h" static void process_config_write (QIOChannel *ioc,PCIDevice *dev,MPQemuMsg *msg);static void process_config_read (QIOChannel *ioc,PCIDevice *dev,MPQemuMsg *msg); +static void process_bar_write (QIOChannel *ioc,MPQemuMsg *msg,Error **errp); +static void process_bar_read (QIOChannel *ioc,MPQemuMsg *msg,Error **errp); void coroutine_fn mpqemu_remote_msg_loop_co(void *data) {
@@ -55,6 +58,12 @@ void coroutine_fn mpqemu_remote_msg_loop_co(void *data)case PCI_CONFIG_READ :process_config_read (com->ioc, pci_dev, &msg);break ; +case BAR_WRITE : +process_bar_write (com->ioc, &msg, &local_err); +break ; +case BAR_READ : +process_bar_read (com->ioc, &msg, &local_err); +break ;default : error_setg(&local_err, "Unknown command (%d) received for device %s (pid=%d)",
@@ -122,3 +131,81 @@ static void process_config_read(QIOChannel *ioc, PCIDevice *dev, getpid()); } } + +static void process_bar_write (QIOChannel *ioc,MPQemuMsg *msg,Error **errp) +{ +BarAccessMsg *bar_access = &msg->data.bar_access; +AddressSpace *as = + bar_access->memory ? &address_space_memory : &address_space_io; +MPQemuMsg ret = {0 }; +MemTxResult res; +uint64_t val; +Error *local_err =NULL ; + +if (!is_power_of_2 (bar_access->size) || + (bar_access->size >sizeof (uint64_t ))) { + ret.data.u64 =UINT64_MAX ; + goto fail; + } + + val =cpu_to_le64 (bar_access->val); + + res =address_space_rw (as, bar_access->addr,MEMTXATTRS_UNSPECIFIED , + (void *)&val, bar_access->size, true); + +if (res !=MEMTX_OK ) { +error_setg (errp,"Could not perform address space write operation," +" inaccessible address: %lx in pid %d." , + bar_access->addr,getpid ()); + ret.data.u64 =-1 ; + } + +fail: + ret.cmd =RET_MSG ; + ret.size =sizeof (ret.data.u64); + +mpqemu_msg_send (&ret, ioc, &local_err); +if (local_err) { +error_setg (errp,"Error while sending message to proxy " +"in remote process pid=%d" ,getpid ()); + } +} + +static void process_bar_read (QIOChannel *ioc,MPQemuMsg *msg,Error **errp) +{ +BarAccessMsg *bar_access = &msg->data.bar_access; +MPQemuMsg ret = {0 }; +AddressSpace *as; +MemTxResult res; +uint64_t val =0 ; +Error *local_err =NULL ; + + as = bar_access->memory ? &address_space_memory : &address_space_io; + +if (!is_power_of_2 (bar_access->size) || + (bar_access->size >sizeof (uint64_t ))) { + val =UINT64_MAX ; + goto fail; + } + + res =address_space_rw (as, bar_access->addr,MEMTXATTRS_UNSPECIFIED , + (void *)&val, bar_access->size, false); + +if (res !=MEMTX_OK ) { +error_setg (errp,"Could not perform address space read operation," +" inaccessible address: %lx in pid %d." , + bar_access->addr,getpid ()); + val =UINT64_MAX ; + } + +fail: + ret.cmd =RET_MSG ; + ret.data.u64 =le64_to_cpu (val); + ret.size =sizeof (ret.data.u64); + +mpqemu_msg_send (&ret, ioc, &local_err); +if (local_err) { +error_setg (errp,"Error while sending message to proxy " +"in remote process pid=%d" ,getpid ()); + } +}
hw/pci/proxy.c +60 -0
63 lines not on this page · raw message
include/hw/pci/proxy.h +10 -0
@@ -16,8 +16,17 @@#define PCI_PROXY_DEV (obj) \ OBJECT_CHECK(PCIProxyDev, (obj), TYPE_PCI_PROXY_DEV) +typedef struct PCIProxyDev PCIProxyDev ; +typedef struct ProxyMemoryRegion { +PCIProxyDev *dev; +MemoryRegion mr; +bool memory; +bool present; +uint8_t type; +}ProxyMemoryRegion ; +struct PCIProxyDev {PCIDevice parent_dev;char *fd;
@@ -31,6 +40,7 @@ struct PCIProxyDev {QemuMutex io_mutex;QIOChannel *ioc;Error *migration_blocker; + ProxyMemoryRegion region[PCI_NUM_REGIONS]; };#endif /* PROXY_H */
include/io/mpqemu-link.h +10 -0
@@ -36,6 +36,8 @@ typedef enum {RET_MSG ,PCI_CONFIG_WRITE ,PCI_CONFIG_READ , +BAR_WRITE , +BAR_READ ,MPQEMU_CMD_MAX , } MPQemuCmd;
@@ -51,6 +53,13 @@ typedef struct {int l; } ConfDataMsg; +typedef struct { +hwaddr addr; +uint64_t val; +unsigned size; +bool memory; +}BarAccessMsg ; +/** * MPQemuMsg: * @cmd: The remote command
@@ -70,6 +79,7 @@ typedef struct {uint64_t u64;ConfDataMsg conf_data;SyncSysmemMsg sync_sysmem; +BarAccessMsg bar_access; } data;int fds[REMOTE_MAX_FDS ];
io/mpqemu-link.c +6 -0
@@ -284,6 +284,12 @@ bool mpqemu_msg_valid(MPQemuMsg *msg)return false; }break ; +case BAR_WRITE : +case BAR_READ : +if ((msg->size !=sizeof (BarAccessMsg)) || (msg->num_fds !=0 )) { +return false; + } +break ;default :break ; }
Signature · 1 line
1.8.3.1
PATCH v11 5/19 multi-process: add qio channel function to transmit
In reply to Jagannathan RamanFrom: Elena Ufimtseva <[email protected]>
The entire array of the memory regions and file handlers. Will be used in the next patch.
--- include/io/channel.h | 24 ++++++++++++++++++++++++ io/channel.c | 45 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+)
include/io/channel.h +24 -0
@@ -773,5 +773,29 @@ void qio_channel_set_aio_fd_handler(QIOChannel *ioc, IOHandler *io_read, IOHandler *io_write, void *opaque); +/** +* qio_channel_writev_full_all: +* @ioc: the channel object +* @iov: the array of memory regions to write data from +* @niov: the length of the @iov array +* @fds: an array of file handles to send +* @nfds: number of file handles in @fds +* @errp: pointer to a NULL-initialized error object +* +* +* Behaves like qio_channel_writev_full but will attempt +* to send all data passed (file handles and memory regions). +* The function will wait for all requested data +* to be written, yielding from the current coroutine +* if required. +* +* Returns: 0 if all bytes were written, or -1 on error +*/ + +int qio_channel_writev_full_all (QIOChannel *ioc, +const struct iovec *iov, +size_t niov, +int *fds,size_t nfds, +Error **errp);#endif /* QIO_CHANNEL_H */
io/channel.c +45 -0
51 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH v11 4/19 multi-process: setup a machine object for remote device process
In reply to Jagannathan Ramanremote-machine object sets up various subsystems of the remote device process. Instantiate PCI host bridge object and initialize RAM, IO & PCI memory regions.
--- MAINTAINERS | 2 ++ hw/i386/meson.build | 1 + hw/i386/remote.c | 69 ++++++++++++++++++++++++++++++++++++++++++++ include/hw/i386/remote.h | 28 ++++++++++++++++++ include/hw/pci-host/remote.h | 1 + 5 files changed, 101 insertions(+) create mode 100644 hw/i386/remote.c create mode 100644 include/hw/i386/remote.h
MAINTAINERS +2 -0
@@ -3102,6 +3102,8 @@ M: John G Johnson <[email protected]> S: Maintained F: hw/pci-host/remote.c F: include/hw/pci-host/remote.h +F: hw/i386/remote.c +F: include/hw/i386/remote.h Build and test automation -------------------------
hw/i386/meson.build +1 -0
@@ -24,6 +24,7 @@ i386_ss.add(when: 'CONFIG_PC', if_true: files( 'pc_sysfw.c', 'acpi-build.c', 'port92.c')) +i386_ss.add(when: 'CONFIG_MPQEMU', if_true: files('remote.c')) subdir('kvm') subdir('xen')
hw/i386/remote.c new file +69 -0
69 lines not on this page · raw message
include/hw/i386/remote.h new file +28 -0
@@ -0,0 +1,28 @@ +/* +* Remote machine configuration +* +* Copyright © 2018, 2020 Oracle and/or its affiliates. +* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory. +* +*/ + +#ifndef REMOTE_MACHINE_H +#define REMOTE_MACHINE_H + +#include "qom/object.h" +#include "hw/boards.h" +#include "hw/pci-host/remote.h" + +typedef struct RemoteMachineState { +MachineState parent_obj; + +RemotePCIHost *host; +}RemoteMachineState ; + +#define TYPE_REMOTE_MACHINE "x-remote-machine" +#define REMOTE_MACHINE (obj) \ + OBJECT_CHECK(RemoteMachineState, (obj), TYPE_REMOTE_MACHINE) + +#endif
include/hw/pci-host/remote.h +1 -0
@@ -25,6 +25,7 @@ typedef struct RemotePCIHost {MemoryRegion *mr_pci_mem;MemoryRegion *mr_sys_io; + MemoryRegion *mr_sys_mem; } RemotePCIHost;#endif
Signature · 1 line
1.8.3.1
PATCH v11 12/19 multi-process: Forward PCI config space acceses to the remote process
In reply to Jagannathan RamanFrom: Elena Ufimtseva <[email protected]>
The Proxy Object sends the PCI config space accesses as messages to the remote process over the communication channel
TODO:
Investigate if the local PCI config writes can be dropped. Without the proxy local PCI config space writes for the device, the driver in the guest times out on the probing.
We have tried to only refer to the remote for the PCI config writes, but the driver timeout in the guest forced as to left this as it is (removing local PCI config only).
--- hw/i386/remote-msg.c | 62 ++++++++++++++++++++++++++++++++++++++++++++++++ hw/pci/proxy.c | 50 ++++++++++++++++++++++++++++++++++++++ include/io/mpqemu-link.h | 9 +++++++ io/mpqemu-link.c | 6 +++++ 4 files changed, 127 insertions(+)
hw/i386/remote-msg.c +62 -0
@@ -15,6 +15,12 @@#include "io/mpqemu-link.h" #include "qapi/error.h" #include "sysemu/runstate.h" +#include "hw/pci/pci.h" + +static void process_config_write (QIOChannel *ioc,PCIDevice *dev, +MPQemuMsg *msg); +static void process_config_read (QIOChannel *ioc,PCIDevice *dev, +MPQemuMsg *msg); void coroutine_fn mpqemu_remote_msg_loop_co(void *data) {
@@ -43,6 +49,12 @@ void coroutine_fn mpqemu_remote_msg_loop_co(void *data) }switch (msg.cmd) { +case PCI_CONFIG_WRITE : +process_config_write (com->ioc, pci_dev, &msg); +break ; +case PCI_CONFIG_READ : +process_config_read (com->ioc, pci_dev, &msg); +break ;default : error_setg(&local_err, "Unknown command (%d) received for device %s (pid=%d)",
@@ -60,3 +72,53 @@ void coroutine_fn mpqemu_remote_msg_loop_co(void *data)return ; } + +static void process_config_write (QIOChannel *ioc,PCIDevice *dev, +MPQemuMsg *msg) +{ +ConfDataMsg *conf = (ConfDataMsg *)&msg->data.conf_data; +MPQemuMsg ret = {0 }; +Error *local_err =NULL ; + +if ((conf->addr +sizeof (conf->val)) >pci_config_size (dev)) { +error_report ("Bad address received when writing PCI config, pid %d" , +getpid ()); + ret.data.u64 =UINT64_MAX ; + }else { +pci_default_write_config (dev, conf->addr, conf->val, conf->l); + } + + ret.cmd =RET_MSG ; + ret.size =sizeof (ret.data.u64); + +mpqemu_msg_send (&ret, ioc, &local_err); +if (local_err) { +error_report ("Could not send message to proxy from pid %d" , +getpid ()); + } +} + +static void process_config_read (QIOChannel *ioc,PCIDevice *dev, +MPQemuMsg *msg) +{ +ConfDataMsg *conf = (ConfDataMsg *)&msg->data.conf_data; +MPQemuMsg ret = {0 }; +Error *local_err =NULL ; + +if ((conf->addr +sizeof (conf->val)) >pci_config_size (dev)) { +error_report ("Bad address received when reading PCI config, pid %d" , +getpid ()); + ret.data.u64 =UINT64_MAX ; + }else { + ret.data.u64 =pci_default_read_config (dev, conf->addr, conf->l); + } + + ret.cmd =RET_MSG ; + ret.size =sizeof (ret.data.u64); + +mpqemu_msg_send (&ret, ioc, &local_err); +if (local_err) { +error_report ("Could not send message to proxy from pid %d" , +getpid ()); + } +}
hw/pci/proxy.c +50 -0
68 lines not on this page · raw message
include/io/mpqemu-link.h +9 -0
@@ -34,6 +34,8 @@ typedef enum {MPQEMU_CMD_INIT ,SYNC_SYSMEM ,RET_MSG , +PCI_CONFIG_WRITE , +PCI_CONFIG_READ ,MPQEMU_CMD_MAX , } MPQemuCmd;
@@ -43,6 +45,12 @@ typedef struct {off_t offsets[REMOTE_MAX_FDS ]; } SyncSysmemMsg; +typedef struct { +uint32_t addr; +uint32_t val; +int l; +}ConfDataMsg ; +/** * MPQemuMsg: * @cmd: The remote command
@@ -60,6 +68,7 @@ typedef struct {union {uint64_t u64; +ConfDataMsg conf_data;SyncSysmemMsg sync_sysmem; } data;
io/mpqemu-link.c +6 -0
@@ -278,6 +278,12 @@ bool mpqemu_msg_valid(MPQemuMsg *msg)return false; }break ; +case PCI_CONFIG_WRITE : +case PCI_CONFIG_READ : +if (msg->size !=sizeof (ConfDataMsg)) { +return false; + } +break ;default :break ; }
Signature · 1 line
1.8.3.1
PATCH v11 15/19 multi-process: create IOHUB object to handle irq
In reply to Jagannathan RamanIOHUB object is added to manage PCI IRQs. It uses KVM_IRQFD ioctl to create irqfd to injecting PCI interrupts to the guest. IOHUB object forwards the irqfd to the remote process. Remote process uses this fd to directly send interrupts to the guest, bypassing QEMU.
--- MAINTAINERS | 2 + hw/i386/meson.build | 1 + hw/i386/remote-iohub.c | 123 +++++++++++++++++++++++++++++++++++++++++ hw/i386/remote-msg.c | 4 ++ hw/i386/remote.c | 10 ++++ hw/pci/proxy.c | 58 +++++++++++++++++++ include/hw/i386/remote-iohub.h | 42 ++++++++++++++ include/hw/i386/remote.h | 3 + include/hw/pci/pci_ids.h | 3 + include/hw/pci/proxy.h | 5 ++ include/io/mpqemu-link.h | 1 + io/mpqemu-link.c | 5 ++ 12 files changed, 257 insertions(+) create mode 100644 hw/i386/remote-iohub.c create mode 100644 include/hw/i386/remote-iohub.h
MAINTAINERS +2 -0
@@ -3115,6 +3115,8 @@ F: hw/pci/proxy.c F: include/hw/pci/proxy.h F: hw/pci/memory-sync.c F: include/hw/pci/memory-sync.h +F: hw/i386/remote-iohub.c +F: include/hw/i386/remote-iohub.h Build and test automation -------------------------
hw/i386/meson.build +1 -0
@@ -28,6 +28,7 @@ i386_ss.add(when: 'CONFIG_MPQEMU', if_true: files('remote.c')) i386_ss.add(when: 'CONFIG_MPQEMU', if_true: files('remote-msg.c')) i386_ss.add(when: 'CONFIG_MPQEMU', if_true: files('remote-obj.c')) i386_ss.add(when: 'CONFIG_MPQEMU', if_true: files('remote-memory.c')) +i386_ss.add(when: 'CONFIG_MPQEMU', if_true: files('remote-iohub.c')) subdir('kvm') subdir('xen')
hw/i386/remote-iohub.c new file +123 -0
123 lines not on this page · raw message
hw/i386/remote-msg.c +4 -0
@@ -18,6 +18,7 @@#include "hw/pci/pci.h" #include "exec/memattrs.h" #include "hw/i386/remote-memory.h" +#include "hw/i386/remote-iohub.h" static void process_config_write (QIOChannel *ioc,PCIDevice *dev,MPQemuMsg *msg);
@@ -68,6 +69,9 @@ void coroutine_fn mpqemu_remote_msg_loop_co(void *data)case SYNC_SYSMEM :remote_sysmem_reconfig (&msg, &local_err);break ; +case SET_IRQFD : +process_set_irqfd_msg (pci_dev, &msg); +break ;default : error_setg(&local_err,
hw/i386/remote.c +10 -0
@@ -20,12 +20,15 @@#include "exec/address-spaces.h" #include "exec/memory.h" #include "qapi/error.h" +#include "hw/pci/pci_host.h" +#include "hw/i386/remote-iohub.h" static void remote_machine_init (MachineState *machine) {MemoryRegion *system_memory, *system_io, *pci_memory;RemoteMachineState *s =REMOTE_MACHINE (machine);RemotePCIHost *rem_host; +PCIHostState *pci_host; system_memory =get_system_memory (); system_io =get_system_io ();
@@ -45,6 +48,13 @@ static void remote_machine_init(MachineState *machine)memory_region_add_subregion_overlap (system_memory,0x0 , pci_memory,-1 );qdev_realize (DEVICE (rem_host),sysbus_get_default (), &error_fatal); + + pci_host =PCI_HOST_BRIDGE (rem_host); + +remote_iohub_init (&s->iohub); + +pci_bus_irqs (pci_host->bus, remote_iohub_set_irq, remote_iohub_map_irq, + &s->iohub,REMOTE_IOHUB_NB_PIRQS ); }static void remote_machine_class_init (ObjectClass *oc,void *data)
hw/pci/proxy.c +58 -0
82 lines not on this page · raw message
include/hw/i386/remote-iohub.h new file +42 -0
42 lines not on this page · raw message
include/hw/i386/remote.h +3 -0
@@ -15,11 +15,14 @@#include "hw/boards.h" #include "hw/pci-host/remote.h" #include "io/channel.h" +#include "hw/i386/remote-iohub.h" typedef struct RemoteMachineState {MachineState parent_obj;RemotePCIHost *host; + +RemoteIOHubState iohub; }RemoteMachineState ;/* Used to pass to co-routine device and ioc. */
include/hw/pci/pci_ids.h +3 -0
@@ -192,6 +192,9 @@#define PCI_DEVICE_ID_SUN_SIMBA 0x5000#define PCI_DEVICE_ID_SUN_SABRE 0xa000 +#define PCI_VENDOR_ID_ORACLE 0x108e +#define PCI_DEVICE_ID_REMOTE_IOHUB 0xb000 +#define PCI_VENDOR_ID_CMD 0x1095#define PCI_DEVICE_ID_CMD_646 0x0646
include/hw/pci/proxy.h +5 -0
@@ -12,6 +12,7 @@#include "hw/pci/pci.h" #include "io/channel.h" #include "hw/pci/memory-sync.h" +#include "qemu/event_notifier.h" #define TYPE_PCI_PROXY_DEV "x-pci-proxy-dev"
@@ -42,6 +43,10 @@ struct PCIProxyDev {QIOChannel *ioc;Error *migration_blocker;RemoteMemSync sync; +int virq; +EventNotifier intr; +EventNotifier resample; +ProxyMemoryRegion region[PCI_NUM_REGIONS ]; };
include/io/mpqemu-link.h +1 -0
@@ -38,6 +38,7 @@ typedef enum {PCI_CONFIG_READ ,BAR_WRITE ,BAR_READ , +SET_IRQFD ,MPQEMU_CMD_MAX , } MPQemuCmd;
io/mpqemu-link.c +5 -0
@@ -290,6 +290,11 @@ bool mpqemu_msg_valid(MPQemuMsg *msg)return false; }break ; +case SET_IRQFD : +if (msg->size || (msg->num_fds !=2 )) { +return false; + } +break ;default :break ; }
Signature · 1 line
1.8.3.1
PATCH v11 2/19 multi-process: Add config option for multi-process QEMU
In reply to Jagannathan RamanAdd a configuration option to separate multi-process code
--- configure | 10 ++++++++++ meson.build | 1 + 2 files changed, 11 insertions(+)
configure +10 -0
@@ -447,6 +447,7 @@ meson="" ninja="" skip_meson=no gettext="" +mpqemu="no" bogus_os="no" malloc_trim="auto"
@@ -1520,6 +1521,10 @@ for opt do ;; --disable-libdaxctl) libdaxctl=no ;; + --enable-mpqemu) mpqemu=yes + ;; + --disable-mpqemu) mpqemu=no + ;; *) echo "ERROR: unknown option $opt" echo "Try '$0 --help' for more information"
@@ -1838,6 +1843,8 @@ disabled with --disable-FEATURE, default is enabled if available: xkbcommon xkbcommon support rng-none dummy RNG, avoid using /dev/(u)random and getrandom() libdaxctl libdaxctl support + mpqemu multi-process QEMU support + NOTE: The object files are built at the place where configure is launched EOF
@@ -6737,6 +6744,9 @@ fi if test "$have_mlockall" = "yes" ; then echo "HAVE_MLOCKALL=y" >> $config_host_mak fi +if test "$mpqemu" = "yes" ; then + echo "CONFIG_MPQEMU=y" >> $config_host_mak +fi if test "$fuzzing" = "yes" ; then # If LIB_FUZZING_ENGINE is set, assume we are running on OSS-Fuzz, and the # needed CFLAGS have already been provided
meson.build +1 -0
@@ -2091,6 +2091,7 @@ endif summary_info += {'thread sanitizer': config_host.has_key('CONFIG_TSAN')} summary_info += {'rng-none': config_host.has_key('CONFIG_RNG_NONE')} summary_info += {'Linux keyring': config_host.has_key('CONFIG_SECRET_KEYRING')} +summary_info += {'Multi-Process QEMU': config_host.has_key('CONFIG_MPQEMU')} summary(summary_info, bool_yn: true) if not supported_cpus.contains(cpu)
Signature · 1 line
1.8.3.1
PATCH v11 14/19 multi-process: Synchronize remote memory
In reply to Jagannathan RamanAdd memory-listener object which is used to keep the view of the RAM in sync between QEMU and remote process.
A MemoryListener is registered for system-memory AddressSpace. The listener sends SYNC_SYSMEM message to the remote process when memory listener commits the changes to memory, the remote process receives the message and processes it in the handler for SYNC_SYSMEM message.
--- MAINTAINERS | 2 + hw/i386/remote-msg.c | 5 ++ hw/pci/memory-sync.c | 210 +++++++++++++++++++++++++++++++++++++++++++ hw/pci/meson.build | 2 + hw/pci/proxy.c | 6 ++ include/hw/pci/memory-sync.h | 27 ++++++ include/hw/pci/proxy.h | 2 + 7 files changed, 254 insertions(+) create mode 100644 hw/pci/memory-sync.c create mode 100644 include/hw/pci/memory-sync.h
MAINTAINERS +2 -0
@@ -3113,6 +3113,8 @@ F: include/hw/i386/remote-memory.h F: hw/i386/remote-memory.c F: hw/pci/proxy.c F: include/hw/pci/proxy.h +F: hw/pci/memory-sync.c +F: include/hw/pci/memory-sync.h Build and test automation -------------------------
hw/i386/remote-msg.c +5 -0
@@ -17,6 +17,7 @@#include "sysemu/runstate.h" #include "hw/pci/pci.h" #include "exec/memattrs.h" +#include "hw/i386/remote-memory.h" static void process_config_write (QIOChannel *ioc,PCIDevice *dev,MPQemuMsg *msg);
@@ -64,6 +65,10 @@ void coroutine_fn mpqemu_remote_msg_loop_co(void *data)case BAR_READ :process_bar_read (com->ioc, &msg, &local_err);break ; +case SYNC_SYSMEM : +remote_sysmem_reconfig (&msg, &local_err); +break ; +default : error_setg(&local_err, "Unknown command (%d) received for device %s (pid=%d)",
hw/pci/memory-sync.c new file +210 -0
210 lines not on this page · raw message
hw/pci/meson.build +2 -0
@@ -18,3 +18,5 @@ softmmu_ss.add_all(when: 'CONFIG_PCI', if_true: pci_ss) softmmu_ss.add(when: 'CONFIG_PCI', if_false: files('pci-stub.c')) softmmu_ss.add(when: 'CONFIG_ALL', if_true: files('pci-stub.c')) + +specific_ss.add(when: 'CONFIG_MPQEMU', if_true: files('memory-sync.c'))
hw/pci/proxy.c +6 -0
@@ -18,6 +18,8 @@#include "migration/blocker.h" #include "io/mpqemu-link.h" #include "qemu/error-report.h" +#include "hw/pci/memory-sync.h" +#include "qom/object.h" static void proxy_set_socket (PCIProxyDev *pdev,int fd,Error **errp) {
@@ -58,6 +60,8 @@ static void pci_proxy_dev_realize(PCIDevice *device, Error **errp)qemu_mutex_init (&dev->io_mutex);qio_channel_set_blocking (dev->ioc, true,NULL ); + +configure_memory_sync (&dev->sync, dev->ioc); }static void pci_proxy_dev_exit (PCIDevice *pdev)
@@ -69,6 +73,8 @@ static void pci_proxy_dev_exit(PCIDevice *pdev)migrate_del_blocker (dev->migration_blocker);error_free (dev->migration_blocker); + +deconfigure_memory_sync (&dev->sync); } static int config_op_send(PCIProxyDev *pdev, uint32_t addr, uint32_t *val,
include/hw/pci/memory-sync.h new file +27 -0
@@ -0,0 +1,27 @@ +/* +* Copyright © 2018, 2020 Oracle and/or its affiliates. +* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory. +* +*/ + +#ifndef MEMORY_SYNC_H +#define MEMORY_SYNC_H + +#include "exec/memory.h" +#include "io/channel.h" + +typedef struct RemoteMemSync { +MemoryListener listener; + +int n_mr_sections; +MemoryRegionSection *mr_sections; + +QIOChannel *ioc; +}RemoteMemSync ; + +void configure_memory_sync (RemoteMemSync *sync,QIOChannel *ioc); +void deconfigure_memory_sync (RemoteMemSync *sync); + +#endif
include/hw/pci/proxy.h +2 -0
@@ -11,6 +11,7 @@#include "hw/pci/pci.h" #include "io/channel.h" +#include "hw/pci/memory-sync.h" #define TYPE_PCI_PROXY_DEV "x-pci-proxy-dev"
@@ -40,6 +41,7 @@ struct PCIProxyDev {QemuMutex io_mutex;QIOChannel *ioc;Error *migration_blocker; +RemoteMemSync sync;ProxyMemoryRegion region[PCI_NUM_REGIONS ]; };
Signature · 1 line
1.8.3.1
PATCH v11 9/19 multi-process: setup memory manager for remote device
In reply to Jagannathan RamanSyncSysMemMsg message format is defined. It is used to send file descriptors of the RAM regions to remote device.
RAM on the remote device is configured with a set of file descriptors. Old RAM regions are deleted and new regions, each with an fd, is added to the RAM.
--- MAINTAINERS | 2 ++ hw/i386/meson.build | 1 + hw/i386/remote-memory.c | 58 +++++++++++++++++++++++++++++++++++++++++ include/hw/i386/remote-memory.h | 19 ++++++++++++++ include/io/mpqemu-link.h | 13 +++++++++ io/mpqemu-link.c | 11 ++++++++ 6 files changed, 104 insertions(+) create mode 100644 hw/i386/remote-memory.c create mode 100644 include/hw/i386/remote-memory.h
MAINTAINERS +2 -0
@@ -3109,6 +3109,8 @@ F: include/io/mpqemu-link.h F: hw/i386/remote-msg.c F: include/hw/i386/remote-obj.h F: hw/i386/remote-obj.c +F: include/hw/i386/remote-memory.h +F: hw/i386/remote-memory.c Build and test automation -------------------------
hw/i386/meson.build +1 -0
@@ -27,6 +27,7 @@ i386_ss.add(when: 'CONFIG_PC', if_true: files( i386_ss.add(when: 'CONFIG_MPQEMU', if_true: files('remote.c')) i386_ss.add(when: 'CONFIG_MPQEMU', if_true: files('remote-msg.c')) i386_ss.add(when: 'CONFIG_MPQEMU', if_true: files('remote-obj.c')) +i386_ss.add(when: 'CONFIG_MPQEMU', if_true: files('remote-memory.c')) subdir('kvm') subdir('xen')
hw/i386/remote-memory.c new file +58 -0
58 lines not on this page · raw message
include/hw/i386/remote-memory.h new file +19 -0
@@ -0,0 +1,19 @@ +/* +* Memory manager for remote device +* +* Copyright © 2018, 2020 Oracle and/or its affiliates. +* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory. +* +*/ + +#ifndef REMOTE_MEMORY_H +#define REMOTE_MEMORY_H + +#include "exec/hwaddr.h" +#include "io/mpqemu-link.h" + +void remote_sysmem_reconfig (MPQemuMsg *msg,Error **errp); + +#endif
include/io/mpqemu-link.h +13 -0
@@ -14,6 +14,7 @@#include "qom/object.h" #include "qemu/thread.h" #include "io/channel.h" +#include "exec/hwaddr.h" #define REMOTE_MAX_FDS 8
@@ -24,12 +25,22 @@* * MPQemuCmd enum type to specify the command to be executed on the remote * device. +* +* SYNC_SYSMEM Shares QEMU's RAM with remote device's RAM */ typedef enum {MPQEMU_CMD_INIT , +SYNC_SYSMEM , +RET_MSG ,MPQEMU_CMD_MAX , }MPQemuCmd ; +typedef struct { +hwaddr gpas[REMOTE_MAX_FDS ]; +uint64_t sizes[REMOTE_MAX_FDS ]; +off_t offsets[REMOTE_MAX_FDS ]; +}SyncSysmemMsg ; +/** * MPQemuMsg: * @cmd: The remote command
@@ -40,12 +51,14 @@ typedef enum {* MPQemuMsg Format of the message sent to the remote device from QEMU. * */ + typedef struct { int cmd; size_t size; union { uint64_t u64; + SyncSysmemMsg sync_sysmem; } data; int fds[REMOTE_MAX_FDS];
io/mpqemu-link.c +11 -0
@@ -233,5 +233,16 @@ bool mpqemu_msg_valid(MPQemuMsg *msg) } } +/* Verify message specific fields. */ +switch (msg->cmd) { +case SYNC_SYSMEM : +if (msg->num_fds ==0 || msg->size !=sizeof (SyncSysmemMsg)) { +return false; + } +break ; +default : +break ; + } +return true; }
Signature · 1 line
1.8.3.1
PATCH v11 7/19 multi-process: Initialize message handler in remote device
In reply to Jagannathan RamanInitializes the message handler function in the remote process. It is called whenever there's an event pending on QIOChannel that registers this function.
--- MAINTAINERS | 1 + hw/i386/meson.build | 1 + hw/i386/remote-msg.c | 61 ++++++++++++++++++++++++++++++++++++++++++++++++ include/hw/i386/remote.h | 9 +++++++ 4 files changed, 72 insertions(+) create mode 100644 hw/i386/remote-msg.c
MAINTAINERS +1 -0
@@ -3106,6 +3106,7 @@ F: hw/i386/remote.c F: include/hw/i386/remote.h F: io/mpqemu-link.c F: include/io/mpqemu-link.h +F: hw/i386/remote-msg.c Build and test automation -------------------------
hw/i386/meson.build +1 -0
@@ -25,6 +25,7 @@ i386_ss.add(when: 'CONFIG_PC', if_true: files( 'acpi-build.c', 'port92.c')) i386_ss.add(when: 'CONFIG_MPQEMU', if_true: files('remote.c')) +i386_ss.add(when: 'CONFIG_MPQEMU', if_true: files('remote-msg.c')) subdir('kvm') subdir('xen')
hw/i386/remote-msg.c new file +61 -0
61 lines not on this page · raw message
include/hw/i386/remote.h +9 -0
@@ -14,6 +14,7 @@#include "qom/object.h" #include "hw/boards.h" #include "hw/pci-host/remote.h" +#include "io/channel.h" typedef struct RemoteMachineState { MachineState parent_obj;
@@ -21,8 +22,16 @@ typedef struct RemoteMachineState {RemotePCIHost *host; } RemoteMachineState; +/* Used to pass to co-routine device and ioc. */ +typedef struct RemoteCommDev { +PCIDevice *dev; +QIOChannel *ioc; +}RemoteCommDev ; +#define TYPE_REMOTE_MACHINE "x-remote-machine"#define REMOTE_MACHINE (obj) \ OBJECT_CHECK(RemoteMachineState, (obj), TYPE_REMOTE_MACHINE) +void coroutine_fn mpqemu_remote_msg_loop_co(void *data); +#endif
Signature · 1 line
1.8.3.1
PATCH v11 17/19 multi-process: perform device reset in the remote process
In reply to Jagannathan RamanFrom: Elena Ufimtseva <[email protected]>
Perform device reset in the remote process when QEMU performs device reset. This is required to reset the internal state (like registers, etc...) of emulated devices
--- hw/i386/remote-msg.c | 23 ++++++++++++++++++++++- hw/pci/proxy.c | 20 ++++++++++++++++++++ include/io/mpqemu-link.h | 1 + 3 files changed, 43 insertions(+), 1 deletion(-)
hw/i386/remote-msg.c +22 -1
@@ -19,6 +19,7 @@#include "exec/memattrs.h" #include "hw/i386/remote-memory.h" #include "hw/i386/remote-iohub.h" +#include "sysemu/reset.h" static void process_config_write (QIOChannel *ioc,PCIDevice *dev,MPQemuMsg *msg);
@@ -26,6 +27,8 @@ static void process_config_read(QIOChannel *ioc, PCIDevice *dev, MPQemuMsg *msg);static void process_bar_write (QIOChannel *ioc,MPQemuMsg *msg,Error **errp);static void process_bar_read (QIOChannel *ioc,MPQemuMsg *msg,Error **errp); +static void process_device_reset_msg (QIOChannel *ioc,PCIDevice *dev, +Error **errp); void coroutine_fn mpqemu_remote_msg_loop_co(void *data) {
@@ -72,7 +75,9 @@ void coroutine_fn mpqemu_remote_msg_loop_co(void *data)case SET_IRQFD :process_set_irqfd_msg (pci_dev, &msg);break ; - +case DEVICE_RESET : +process_device_reset_msg (com->ioc, pci_dev, &local_err); +break ;default : error_setg(&local_err, "Unknown command (%d) received for device %s (pid=%d)",
@@ -218,3 +223,19 @@ fail: "in remote process pid=%d", getpid()); } } + +static void process_device_reset_msg (QIOChannel *ioc,PCIDevice *dev, +Error **errp) +{ +DeviceClass *dc =DEVICE_GET_CLASS (dev); +DeviceState *s =DEVICE (dev); +MPQemuMsg ret = {0 }; + +if (dc->reset) { + dc->reset (s); + } + + ret.cmd =RET_MSG ; + +mpqemu_msg_send (&ret, ioc, errp); +}
hw/pci/proxy.c +20 -0
@@ -25,6 +25,7 @@#include "util/event_notifier-posix.c" static void probe_pci_info (PCIDevice *dev,Error **errp); +static void proxy_device_reset (DeviceState *dev);static void proxy_set_socket (PCIProxyDev *pdev,int fd,Error **errp) {
@@ -198,6 +199,8 @@ static void pci_proxy_dev_class_init(ObjectClass *klass, void *data) k->config_read = pci_proxy_read_config; k->config_write = pci_proxy_write_config; + dc->reset = proxy_device_reset; +device_class_set_props (dc, proxy_properties); }
@@ -355,3 +358,20 @@ static void probe_pci_info(PCIDevice *dev, Error **errp) } } } + +static void proxy_device_reset (DeviceState *dev) +{ +PCIProxyDev *pdev =PCI_PROXY_DEV (dev); +MPQemuMsg msg = {0 }; +Error *local_err =NULL ; + + msg.cmd =DEVICE_RESET ; + msg.size =0 ; + + (void )mpqemu_msg_send_and_await_reply (&msg, pdev, &local_err); +if (local_err) { +error_report ("Failed to send DEVICE_RESET to the remote process" ); +error_free (local_err); + } + +}
include/io/mpqemu-link.h +1 -0
@@ -39,6 +39,7 @@ typedef enum {BAR_WRITE ,BAR_READ ,SET_IRQFD , +DEVICE_RESET ,MPQEMU_CMD_MAX , } MPQemuCmd;
Signature · 1 line
1.8.3.1
PATCH v11 16/19 multi-process: Retrieve PCI info from remote process
In reply to Jagannathan RamanRetrieve PCI configuration info about the remote device and configure the Proxy PCI object based on the returned information
--- hw/pci/proxy.c | 85 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 85 insertions(+)
hw/pci/proxy.c +85 -0
@@ -24,6 +24,8 @@#include "sysemu/kvm.h" #include "util/event_notifier-posix.c" +static void probe_pci_info (PCIDevice *dev,Error **errp); +static void proxy_set_socket (PCIProxyDev *pdev,int fd,Error **errp) { pdev->ioc =qio_channel_new_fd (fd, errp);
@@ -87,6 +89,7 @@ static void setup_irqfd(PCIProxyDev *dev)static void pci_proxy_dev_realize (PCIDevice *device,Error **errp) {PCIProxyDev *dev =PCI_PROXY_DEV (device); +uint8_t *pci_conf = device->config;int fd;if (dev->fd) {
@@ -114,9 +117,14 @@ static void pci_proxy_dev_realize(PCIDevice *device, Error **errp)qemu_mutex_init (&dev->io_mutex);qio_channel_set_blocking (dev->ioc, true,NULL ); + pci_conf[PCI_LATENCY_TIMER ] =0xff ; + pci_conf[PCI_INTERRUPT_PIN ] =0x01 ; +configure_memory_sync (&dev->sync, dev->ioc);setup_irqfd (dev); + +probe_pci_info (PCI_DEVICE (dev), errp); }static void pci_proxy_dev_exit (PCIDevice *pdev)
@@ -270,3 +278,80 @@ const MemoryRegionOps proxy_mr_ops = { .max_access_size = 8, }, }; + +static void probe_pci_info (PCIDevice *dev,Error **errp) +{ +PCIDeviceClass *pc =PCI_DEVICE_GET_CLASS (dev); +uint32_t orig_val, new_val, base_class, val; +PCIProxyDev *pdev =PCI_PROXY_DEV (dev); +DeviceClass *dc =DEVICE_CLASS (pc); +uint8_t type; +int i, size; +char *name; + +config_op_send (pdev,PCI_VENDOR_ID , &val,2 ,PCI_CONFIG_READ ); + pc->vendor_id = (uint16_t )val; + +config_op_send (pdev,PCI_DEVICE_ID , &val,2 ,PCI_CONFIG_READ ); + pc->device_id = (uint16_t )val; + +config_op_send (pdev,PCI_CLASS_DEVICE , &val,2 ,PCI_CONFIG_READ ); + pc->class_id = (uint16_t )val; + +config_op_send (pdev,PCI_SUBSYSTEM_ID , &val,2 ,PCI_CONFIG_READ ); + pc->subsystem_id = (uint16_t )val; + + base_class = pc->class_id >>4 ; +switch (base_class) { +case PCI_BASE_CLASS_BRIDGE : +set_bit (DEVICE_CATEGORY_BRIDGE , dc->categories); +break ; +case PCI_BASE_CLASS_STORAGE : +set_bit (DEVICE_CATEGORY_STORAGE , dc->categories); +break ; +case PCI_BASE_CLASS_NETWORK : +set_bit (DEVICE_CATEGORY_NETWORK , dc->categories); +break ; +case PCI_BASE_CLASS_INPUT : +set_bit (DEVICE_CATEGORY_INPUT , dc->categories); +break ; +case PCI_BASE_CLASS_DISPLAY : +set_bit (DEVICE_CATEGORY_DISPLAY , dc->categories); +break ; +case PCI_BASE_CLASS_PROCESSOR : +set_bit (DEVICE_CATEGORY_CPU , dc->categories); +break ; +default : +set_bit (DEVICE_CATEGORY_MISC , dc->categories); +break ; + } + +for (i =0 ; i <PCI_NUM_REGIONS ; i++) { +config_op_send (pdev,PCI_BASE_ADDRESS_0 + (4 * i), &orig_val,4 , +PCI_CONFIG_READ ); + new_val =0xffffffff ; +config_op_send (pdev,PCI_BASE_ADDRESS_0 + (4 * i), &new_val,4 , +PCI_CONFIG_WRITE ); +config_op_send (pdev,PCI_BASE_ADDRESS_0 + (4 * i), &new_val,4 , +PCI_CONFIG_READ ); + size = (~(new_val &0xFFFFFFF0 )) +1 ; +config_op_send (pdev,PCI_BASE_ADDRESS_0 + (4 * i), &orig_val,4 , +PCI_CONFIG_WRITE ); + type = (new_val &0x1 ) ? +PCI_BASE_ADDRESS_SPACE_IO :PCI_BASE_ADDRESS_SPACE_MEMORY ; + +if (size) { + pdev->region[i].dev = pdev; + pdev->region[i].present = true; +if (type ==PCI_BASE_ADDRESS_SPACE_MEMORY ) { + pdev->region[i].memory = true; + } + name =g_strdup_printf ("bar-region-%d" , i); +memory_region_init_io (&pdev->region[i].mr,OBJECT (pdev), + &proxy_mr_ops, &pdev->region[i], + name, size); +pci_register_bar (dev, i, type, &pdev->region[i].mr); +g_free (name); + } + } +}
Signature · 1 line
1.8.3.1
PATCH v11 18/19 multi-process: add the concept description to docs/devel/qemu-multiprocess
In reply to Jagannathan RamanFrom: John G Johnson <[email protected]>
--- MAINTAINERS | 1 + docs/devel/index.rst | 1 + docs/devel/multi-process.rst | 966 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 968 insertions(+) create mode 100644 docs/devel/multi-process.rst
MAINTAINERS +1 -0
@@ -3117,6 +3117,7 @@ F: hw/pci/memory-sync.c F: include/hw/pci/memory-sync.h F: hw/i386/remote-iohub.c F: include/hw/i386/remote-iohub.h +F: docs/devel/multi-process.rst Build and test automation -------------------------
docs/devel/index.rst +1 -0
@@ -34,3 +34,4 @@ Contents: clocks qom block-coroutine-wrapper + multi-process
docs/devel/multi-process.rst new file +966 -0
966 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH v11 19/19 multi-process: add configure and usage information
In reply to Jagannathan RamanFrom: Elena Ufimtseva <[email protected]>
--- MAINTAINERS | 2 ++ docs/multi-process.rst | 67 ++++++++++++++++++++++++++++++++++++++++++++++ scripts/mpqemu-launcher.py | 49 +++++++++++++++++++++++++++++++++ 3 files changed, 118 insertions(+) create mode 100644 docs/multi-process.rst create mode 100755 scripts/mpqemu-launcher.py
MAINTAINERS +2 -0
@@ -3118,6 +3118,8 @@ F: include/hw/pci/memory-sync.h F: hw/i386/remote-iohub.c F: include/hw/i386/remote-iohub.h F: docs/devel/multi-process.rst +F: scripts/mpqemu-launcher.py +F: scripts/mpqemu-launcher-perf-mode.py Build and test automation -------------------------
docs/multi-process.rst new file +67 -0
67 lines not on this page · raw message
scripts/mpqemu-launcher.py new file +49 -0
49 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH v11 6/19 multi-process: define MPQemuMsg format and transmission functions
In reply to Jagannathan RamanFrom: Elena Ufimtseva <[email protected]>
Defines MPQemuMsg, which is the message that is sent to the remote process. This message is sent over QIOChannel and is used to command the remote process to perform various tasks.
Define transmission functions used by proxy and by remote.
There are certain restrictions on where its safe to use these functions:
- From main loop in co-routine context. Will block the main loop if not in
co-routine context;
- From vCPU thread with no co-routine context and if the channel is not part
of the main loop handling;
- From IOThread within co-routine context, outside of co-routine context will
block IOThread;--- MAINTAINERS | 2 + include/io/mpqemu-link.h | 60 ++++++++++++ io/meson.build | 2 + io/mpqemu-link.c | 237 +++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 301 insertions(+) create mode 100644 include/io/mpqemu-link.h create mode 100644 io/mpqemu-link.c
MAINTAINERS +2 -0
@@ -3104,6 +3104,8 @@ F: hw/pci-host/remote.c F: include/hw/pci-host/remote.h F: hw/i386/remote.c F: include/hw/i386/remote.h +F: io/mpqemu-link.c +F: include/io/mpqemu-link.h Build and test automation -------------------------
include/io/mpqemu-link.h new file +60 -0
60 lines not on this page · raw message
io/meson.build +2 -0
@@ -13,3 +13,5 @@ io_ss.add(files( 'net-listener.c', 'task.c', )) + +io_ss.add(when: 'CONFIG_MPQEMU', if_true: files('mpqemu-link.c'))
io/mpqemu-link.c new file +237 -0
237 lines not on this page · raw message
Signature · 1 line
1.8.3.1
Re: [PATCH v11 06/19] multi-process: define MPQemuMsg format and transmission functions
Quoted 29 lines · Jagannathan Raman
io/mpqemu-link.c new file +29
-0+void mpqemu_msg_recv (MPQemuMsg *msg,QIOChannel *ioc,Error **errp) +{ +Error *local_err =NULL ; +int *fds =NULL ; +size_t nfds =0 ; +ssize_t len; + + len =mpqemu_read (ioc, (void *)msg,MPQEMU_MSG_HDR_SIZE , &fds, &nfds, + &local_err); +if (!local_err) { +if (len == -EIO ) { +error_setg (&local_err,"Connection closed." ); + goto fail; + } +if (len <0 ) { +error_setg (&local_err,"Message length is less than 0" ); + goto fail; + } +if (len !=MPQEMU_MSG_HDR_SIZE ) { +error_setg (&local_err,"Message header corrupted" ); + goto fail; + } + }else { + goto fail; + } + +if (msg->size >sizeof (msg->data)) { +error_setg (&local_err,"Invalid size for message" ); + goto fail;
io/mpqemu-link.c line 199
Quoted 10 lines
io/mpqemu-link.c new file +10
-0+ } + +if (mpqemu_read (ioc, (void *)&msg->data, msg->size,NULL ,NULL , + &local_err) <0 ) { + goto fail; + } + + msg->num_fds = nfds; +if (nfds) { +memcpy (msg->fds, fds, nfds *sizeof (int ));
I can't find anything that limits nfds to REMOTE_MAX_FDS. This looks like a buffer overflow.
Re: [PATCH v11 08/19] multi-process: Associate fd of a PCIDevice with its object
In reply to Jagannathan RamanQuoted 6 lines · Jagannathan Raman
Associate the file descriptor for a PCIDevice in remote process with DeviceState object.
Signed-off-by Elena Ufimtseva <[email protected]>Signed-off-by John G Johnson <[email protected]>Signed-off-by Jagannathan Raman <[email protected]>
Quoted 9 lines
--- MAINTAINERS | 2 + hw/i386/meson.build | 1 + hw/i386/remote-msg.c | 1 + hw/i386/remote-obj.c | 154 +++++++++++++++++++++++++++++++++++++++++++ include/hw/i386/remote-obj.h | 42 ++++++++++++ 5 files changed, 200 insertions(+) create mode 100644 hw/i386/remote-obj.c create mode 100644 include/hw/i386/remote-obj.h
Re: [PATCH v11 10/19] multi-process: introduce proxy object
In reply to Jagannathan RamanQuoted 7 lines · Jagannathan Raman
From: Elena Ufimtseva <[email protected]>
Defines a PCI Device proxy object as a child of
TYPE_PCI_DEVICE.Signed-off-by Elena Ufimtseva <[email protected]>Signed-off-by Jagannathan Raman <[email protected]>Signed-off-by John G Johnson <[email protected]>
Quoted 8 lines
--- MAINTAINERS | 2 ++ hw/pci/meson.build | 1 + hw/pci/proxy.c | 98 ++++++++++++++++++++++++++++++++++++++++++++++++++ include/hw/pci/proxy.h | 36 +++++++++++++++++++ 4 files changed, 137 insertions(+) create mode 100644 hw/pci/proxy.c create mode 100644 include/hw/pci/proxy.h
Quoted 11 lines · Jagannathan Raman
Hello,
This is the v11 of the patchset. Thank you very much for the review of the v10 of the series. We are glad to hear the patchset is getting in a better shape.
We made changes to the following patches in this series.
[PATCH v10 06/19] multi-process: define MPQemuMsg format and transmission functions [PATCH v10 08/19] multi-process: Associate fd of a PCIDevice with its object [PATCH v10 10/19] multi-process: introduce proxy object
I have reviewed these patches and posted a comment about a possible buffer overflow.
Stefan
Re: [PATCH v11 04/19] multi-process: setup a machine object for remote device process
In reply to Jagannathan RamanQuoted 87 lines · Jagannathan Raman
remote-machine object sets up various subsystems of the remote device process. Instantiate PCI host bridge object and initialize RAM, IO & PCI memory regions.
Signed-off-by John G Johnson <[email protected]>Signed-off-by Jagannathan Raman <[email protected]>Signed-off-by Elena Ufimtseva <[email protected]>Reviewed-by Stefan Hajnoczi <[email protected]>--- MAINTAINERS | 2 ++ hw/i386/meson.build | 1 + hw/i386/remote.c | 69 ++++++++++++++++++++++++++++++++++++++++++++ include/hw/i386/remote.h | 28 ++++++++++++++++++ include/hw/pci-host/remote.h | 1 + 5 files changed, 101 insertions(+)create mode 100644
hw/i386/remote.c
create mode 100644include/hw/i386/remote.hMAINTAINERS +2
-0@@ -3102,6 +3102,8 @@ M: John G Johnson <[email protected]> S: Maintained F: hw/pci-host/remote.c F: include/hw/pci-host/remote.h +F: hw/i386/remote.c +F: include/hw/i386/remote.h Build and test automation -------------------------hw/i386/meson.build +1
-0@@ -24,6 +24,7 @@ i386_ss.add(when: 'CONFIG_PC', if_true: files( 'pc_sysfw.c', 'acpi-build.c', 'port92.c')) +i386_ss.add(when: 'CONFIG_MPQEMU', if_true: files('remote.c')) subdir('kvm') subdir('xen')hw/i386/remote.c new file +38
-0@@ -0,0 +1,69 @@ +/* +* Machine for remote device +* +* This machine type is used by the remote device process in multi-process +* QEMU. QEMU device models depend on parent busses, interrupt controllers, +* memory regions, etc. The remote machine type offers this environment so +* that QEMU device models can be used as remote devices. +* +* Copyright © 2018, 2020 Oracle and/or its affiliates. +* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory. +* +*/ + +#include "qemu/osdep.h" +#include "qemu-common.h" + +#include "hw/i386/remote.h" +#include "exec/address-spaces.h" +#include "exec/memory.h" +#include "qapi/error.h" + +static void remote_machine_init (MachineState *machine) +{ +MemoryRegion *system_memory, *system_io, *pci_memory; +RemoteMachineState *s =REMOTE_MACHINE (machine); +RemotePCIHost *rem_host; + + system_memory =get_system_memory (); + system_io =get_system_io (); + + pci_memory =g_new (MemoryRegion,1 ); +memory_region_init (pci_memory,NULL ,"pci" ,UINT64_MAX ); + + rem_host =REMOTE_HOST_DEVICE (qdev_new (TYPE_REMOTE_HOST_DEVICE )); + + rem_host->mr_pci_mem = pci_memory;
hw/i386/remote.c line 48
Quoted 10 lines
hw/i386/remote.c new file +10
-0+ rem_host->mr_sys_mem = system_memory; + rem_host->mr_sys_io = system_io; + + s->host = rem_host; + +object_property_add_child (OBJECT (s),"remote-device" ,OBJECT (rem_host)); +memory_region_add_subregion_overlap (system_memory,0x0 , pci_memory,-1 ); + +qdev_realize (DEVICE (rem_host),sysbus_get_default (), &error_fatal); +}
Nothing is x86-specific in this machine. Why not make it a generic machine, like the 'none' one?
Re: [PATCH v11 05/19] multi-process: add qio channel function to transmit
In reply to Jagannathan RamanQuoted 3 lines · Jagannathan Raman
From: Elena Ufimtseva <[email protected]>
The entire array of the memory regions and file handlers.
I don't understand the description... Did you forgot the verb?
Quoted 10 lines
Will be used in the next patch.
Signed-off-by Elena Ufimtseva <[email protected]>Signed-off-by John G Johnson <[email protected]>Signed-off-by Jagannathan Raman <[email protected]>Reviewed-by Stefan Hajnoczi <[email protected]>--- include/io/channel.h | 24 ++++++++++++++++++++++++ io/channel.c | 45 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+)
Re: [PATCH v11 02/19] multi-process: Add config option for multi-process QEMU
In reply to Jagannathan RamanQuoted 35 lines · Jagannathan Raman
Add a configuration option to separate multi-process code
Signed-off-by John G Johnson <[email protected]>Signed-off-by Jagannathan Raman <[email protected]>Signed-off-by Elena Ufimtseva <[email protected]>Reviewed-by Stefan Hajnoczi <[email protected]>--- configure | 10 ++++++++++ meson.build | 1 + 2 files changed, 11 insertions(+)configure +5
-0@@ -447,6 +447,7 @@ meson="" ninja="" skip_meson=no gettext="" +mpqemu="no" bogus_os="no" malloc_trim="auto"@@ -1520,6 +1521,10 @@ for opt do ;; --disable-libdaxctl) libdaxctl=no ;; + --enable-mpqemu) mpqemu=yes + ;; + --disable-mpqemu) mpqemu=no + ;; *) echo "ERROR: unknown option $opt" echo "Try '$0 --help' for more information"
configure disabled with --disable-FEATURE, default is enabled if available: · line 1846
Quoted 4 lines
configure +1
-0@@ -1838,6 +1843,8 @@ disabled with --disable-FEATURE, default is enabled if available: xkbcommon xkbcommon support rng-none dummy RNG, avoid using /dev/(u)random and getrandom() libdaxctl libdaxctl support + mpqemu multi-process QEMU support
When I first heard about this, I was expecting something else. It could be better named as "remote-device emulation" or maybe "manifold (device) emulation" :) Anyway too late I presume.
Re: [PATCH v11 12/19] multi-process: Forward PCI config space acceses to the remote process
In reply to Philippe Mathieu-DaudéQuoted 5 lines · Philippe Mathieu-Daudé
Quoted 4 lines · Jagannathan Raman
From: Elena Ufimtseva <[email protected]>
The Proxy Object sends the PCI config space accesses as messages to the remote process over the communication channel
...
Quoted 11 lines
Quoted 11 lines
hw/i386/remote-msg.c +9
-0+static void process_config_read (QIOChannel *ioc,PCIDevice *dev, + MPQemuMsg *msg) +{ + ConfDataMsg *conf = (ConfDataMsg *)&msg->data.conf_data; + MPQemuMsg ret = { 0 }; + Error *local_err = NULL; + + if ((conf->addr + sizeof(conf->val)) > pci_config_size(dev)) { + error_report("Bad address received when reading PCI config,pid %d",
+getpid());
Quoted 8 lines
Quoted 4 lines
+ ret.data.u64 =
UINT64_MAX;
+ } else {
+ ret.data.u64 =pci_default_read_config(dev, conf->addr, conf->l);Isn't it endianess issue here? It might makes sense to declare ConfDataMsg and PCI_CONFIG_READ/PCI_CONFIG_WRITE packets in little endian.
Bah Stefan told me to look at the specs which are included in the latest patch (docs/multi-process.rst). As I process patches of this series in order I haven't looked at it yet. Still, why not put it first in your series?
Regards,
Phil.
Re: [PATCH v11 12/19] multi-process: Forward PCI config space acceses to the remote process
In reply to Jagannathan RamanQuoted 5 lines · Jagannathan Raman
From: Elena Ufimtseva <[email protected]>
The Proxy Object sends the PCI config space accesses as messages to the remote process over the communication channel
Quoted 7 lines
TODO:
Investigate if the local PCI config writes can be dropped. Without the proxy local PCI config space writes for the device, the driver in the guest times out on the probing.
We have tried to only refer to the remote for the PCI config writes, but the driver timeout in the guest forced as to left this as it is (removing local PCI config only).
I expect this TODO to be lost in git history. Better to commit it in the code and remove it once resolved IMHO.
Quoted 11 lines
Signed-off-by Elena Ufimtseva <[email protected]>Signed-off-by Jagannathan Raman <[email protected]>Signed-off-by John G Johnson <[email protected]>Reviewed-by Stefan Hajnoczi <[email protected]>--- hw/i386/remote-msg.c | 62 ++++++++++++++++++++++++++++++++++++++++++++++++ hw/pci/proxy.c | 50 ++++++++++++++++++++++++++++++++++++++ include/io/mpqemu-link.h | 9 +++++++ io/mpqemu-link.c | 6 +++++ 4 files changed, 127 insertions(+)
Please have a look at scripts/git.orderfile, it helps reviewers to avoid scrolling down/up the patches.
Quoted 64 lines
hw/i386/remote-msg.c +41
-0@@ -15,6 +15,12 @@#include "io/mpqemu-link.h" #include "qapi/error.h" #include "sysemu/runstate.h" +#include "hw/pci/pci.h" + +static void process_config_write (QIOChannel *ioc,PCIDevice *dev, +MPQemuMsg *msg); +static void process_config_read (QIOChannel *ioc,PCIDevice *dev, +MPQemuMsg *msg); void coroutine_fn mpqemu_remote_msg_loop_co(void *data) {@@ -43,6 +49,12 @@ void coroutine_fn mpqemu_remote_msg_loop_co(void *data) }switch (msg.cmd) { +case PCI_CONFIG_WRITE : +process_config_write (com->ioc, pci_dev, &msg); +break ; +case PCI_CONFIG_READ : +process_config_read (com->ioc, pci_dev, &msg); +break ;default : error_setg(&local_err, "Unknown command (%d) received for device %s (pid=%d)",@@ -60,3 +72,53 @@ void coroutine_fn mpqemu_remote_msg_loop_co(void *data)return ; } + +static void process_config_write (QIOChannel *ioc,PCIDevice *dev, +MPQemuMsg *msg) +{ +ConfDataMsg *conf = (ConfDataMsg *)&msg->data.conf_data; +MPQemuMsg ret = {0 }; +Error *local_err =NULL ; + +if ((conf->addr +sizeof (conf->val)) >pci_config_size (dev)) { +error_report ("Bad address received when writing PCI config, pid %d" , +getpid ()); + ret.data.u64 =UINT64_MAX ; + }else { +pci_default_write_config (dev, conf->addr, conf->val, conf->l); + } + + ret.cmd =RET_MSG ; + ret.size =sizeof (ret.data.u64); + +mpqemu_msg_send (&ret, ioc, &local_err); +if (local_err) { +error_report ("Could not send message to proxy from pid %d" , +getpid ()); + } +} + +static void process_config_read (QIOChannel *ioc,PCIDevice *dev, +MPQemuMsg *msg) +{
hw/i386/remote-msg.c void coroutine_fn mpqemu_remote_msg_loop_co(void *data) · line 113
Quoted 10 lines
hw/i386/remote-msg.c +10
-0+ConfDataMsg *conf = (ConfDataMsg *)&msg->data.conf_data; +MPQemuMsg ret = {0 }; +Error *local_err =NULL ; + +if ((conf->addr +sizeof (conf->val)) >pci_config_size (dev)) { +error_report ("Bad address received when reading PCI config, pid %d" , +getpid ()); + ret.data.u64 =UINT64_MAX ; + }else { + ret.data.u64 =pci_default_read_config (dev, conf->addr, conf->l);
Isn't it endianess issue here? It might makes sense to declare ConfDataMsg and PCI_CONFIG_READ/PCI_CONFIG_WRITE packets in little endian.
Quoted 100 lines
hw/i386/remote-msg.c +11
-0+ } + + ret.cmd =RET_MSG ; + ret.size =sizeof (ret.data.u64); + +mpqemu_msg_send (&ret, ioc, &local_err); +if (local_err) { +error_report ("Could not send message to proxy from pid %d" , +getpid ()); + } +}hw/pci/proxy.c +50
-0@@ -16,6 +16,8 @@#include "hw/qdev-properties.h" #include "monitor/monitor.h" #include "migration/blocker.h" +#include "io/mpqemu-link.h" +#include "qemu/error-report.h" static void proxy_set_socket (PCIProxyDev *pdev,int fd,Error **errp) {@@ -69,6 +71,51 @@ static void pci_proxy_dev_exit(PCIDevice *pdev)error_free (dev->migration_blocker); } +static int config_op_send (PCIProxyDev *pdev,uint32_t addr,uint32_t *val, +int l,unsigned int op) +{ +MPQemuMsg msg = {0 }; +uint64_t ret = -EINVAL ; +Error *local_err =NULL ; + + msg.cmd = op; + msg.data.conf_data.addr = addr; + msg.data.conf_data.val = (op ==PCI_CONFIG_WRITE ) ? *val :0 ; + msg.data.conf_data.l = l; + msg.size =sizeof (ConfDataMsg); + + ret =mpqemu_msg_send_and_await_reply (&msg, pdev, &local_err); +if (local_err) { +error_report_err (local_err); + } +if (op ==PCI_CONFIG_READ ) { + *val = (uint32_t )ret; + } + +return ret; +} + +static uint32_t pci_proxy_read_config (PCIDevice *d,uint32_t addr,int len) +{ +uint32_t val; + + (void )config_op_send (PCI_PROXY_DEV (d), addr, &val, len,PCI_CONFIG_READ ); + +return val; +} + +static void pci_proxy_write_config (PCIDevice *d,uint32_t addr,uint32_t val, +int l) +{ +/* +* Some of the functions access the copy of the remote device +* PCI config space, therefore maintain it updated. +*/ +pci_default_write_config (d, addr, val, l); + + (void )config_op_send (PCI_PROXY_DEV (d), addr, &val, l,PCI_CONFIG_WRITE ); +} +static void pci_proxy_dev_class_init (ObjectClass *klass,void *data) {DeviceClass *dc =DEVICE_CLASS (klass);@@ -76,6 +123,9 @@ static void pci_proxy_dev_class_init(ObjectClass *klass, void *data) k->realize = pci_proxy_dev_realize; k->exit = pci_proxy_dev_exit; + k->config_read = pci_proxy_read_config; + k->config_write = pci_proxy_write_config; +device_class_set_props (dc, proxy_properties); }include/io/mpqemu-link.h +2
-0@@ -34,6 +34,8 @@ typedef enum {MPQEMU_CMD_INIT ,SYNC_SYSMEM ,RET_MSG , +PCI_CONFIG_WRITE , +PCI_CONFIG_READ ,MPQEMU_CMD_MAX , } MPQemuCmd;
include/io/mpqemu-link.h typedef struct { · line 52
Quoted 8 lines
include/io/mpqemu-link.h +5
-0@@ -43,6 +45,12 @@ typedef struct {off_t offsets[REMOTE_MAX_FDS ]; } SyncSysmemMsg; +typedef struct { +uint32_t addr; +uint32_t val; +int l; +}ConfDataMsg ;
Maybe name this PciConfDataMsg?
Quoted 30 lines
include/io/mpqemu-link.h +2
-0+/** * MPQemuMsg: * @cmd: The remote command @@ -60,6 +68,7 @@ typedef struct {union {uint64_t u64; +ConfDataMsg conf_data;SyncSysmemMsg sync_sysmem; } data;io/mpqemu-link.c +6
-0@@ -278,6 +278,12 @@ bool mpqemu_msg_valid(MPQemuMsg *msg)return false; }break ; +case PCI_CONFIG_WRITE : +case PCI_CONFIG_READ : +if (msg->size !=sizeof (ConfDataMsg)) { +return false; + } +break ;default :break ; }
Re: [PATCH v11 19/19] multi-process: add configure and usage information
In reply to Jagannathan RamanHi Jagannathan,
Quoted 1 line · Jagannathan Raman
From: Elena Ufimtseva <[email protected]>
Documentation is scarce ;)
Quoted 18 lines
Signed-off-by Elena Ufimtseva <[email protected]>Signed-off-by Jagannathan Raman <[email protected]>Signed-off-by John G Johnson <[email protected]>Reviewed-by Stefan Hajnoczi <[email protected]>--- MAINTAINERS | 2 ++ docs/multi-process.rst | 67 ++++++++++++++++++++++++++++++++++++++++++++++ scripts/mpqemu-launcher.py | 49 +++++++++++++++++++++++++++++++++ 3 files changed, 118 insertions(+) create mode 100644 docs/multi-process.rst create mode 100755 scripts/mpqemu-launcher.pyMAINTAINERS +0
-0
MAINTAINERS F: include/hw/pci/memory-sync.h · line 3122
Quoted 5 lines
MAINTAINERS +2
-0@@ -3118,6 +3118,8 @@ F: include/hw/pci/memory-sync.h F: hw/i386/remote-iohub.c F: include/hw/i386/remote-iohub.h F: docs/devel/multi-process.rst +F: scripts/mpqemu-launcher.py +F: scripts/mpqemu-launcher-perf-mode.py
This one was in v7, Stefan asked about it, then the script disappeared in v8 =)
https://www.mail-archive.com/[email protected]/msg718984.html
Quoted 82 lines
MAINTAINERS +0
-0Build and test automation -------------------------docs/multi-process.rst new file +67
-0@@ -0,0 +1,67 @@ +Multi-process QEMU +================== + +This document describes how to configure and use multi-process qemu. +For the design document refer to docs/devel/qemu-multiprocess. + +1) Configuration +---------------- + +To enable support for multi-process add --enable-mpqemu +to the list of options for the "configure" script. + + +2) Usage +-------- + +Multi-process QEMU requires an orchestrator to launch. Please refer to a +light-weight python based orchestrator for mpqemu in +scripts/mpqemu-launcher.py to lauch QEMU in multi-process mode. + +Following is a description of command-line used to launch mpqemu. + +* Orchestrator: + + - The Orchestrator creates a unix socketpair + + - It launches the remote process and passes one of the + sockets to it via command-line. + + - It then launches QEMU and specifies the other socket as an option + to the Proxy device object + +* Remote Process: + + - QEMU can enter remote process mode by using the "remote" machine + option. + + - The orchestrator creates a "remote-object" with details about + the device and the file descriptor for the device + + - The remaining options are no different from how one launches QEMU with + devices. + + - Example command-line for the remote process is as follows: + + /usr/bin/qemu-system-x86_64 \ + -machine x-remote \ + -device lsi53c895a,id=lsi0 \ + -drive id=drive_image2,file=/build/ol7-nvme-test-1.qcow2 \ + -device scsi-hd,id=drive2,drive=drive_image2,bus=lsi0.0,scsi-id=0 \ + -object x-remote-object,id=robj1,devid=lsi1,fd=4, + +* QEMU: + + - Since parts of the RAM are shared between QEMU & remote process, a + memory-backend-memfd is required to facilitate this, as follows: + + -object memory-backend-memfd,id=mem,size=2G + + - A "x-pci-proxy-dev" device is created for each of the PCI devices emulated + in the remote process. A "socket" sub-option specifies the other end of + unix channel created by orchestrator. The "id" sub-option must be specified + and should be the same as the "id" specified for the remote PCI device + + - Example commandline for QEMU is as follows: + + -device x-pci-proxy-dev,id=lsi0,socket=3scripts/mpqemu-launcher.py new file +0
-0
scripts/mpqemu-launcher.py line 7
Quoted 7 lines
scripts/mpqemu-launcher.py new file +7
-0@@ -0,0 +1,49 @@ +#!/usr/bin/env python3 +import socket +import os +import subprocess +import time + +PROC_QEMU='/usr/bin/qemu-system-x86_64'
If this is a (multiarch) test, then ...
scripts/mpqemu-launcher.py line 13
Quoted 6 lines
scripts/mpqemu-launcher.py new file +6
-0+ +proxy, remote = socket.socketpair(socket.AF_UNIX, socket.SOCK_STREAM) + +remote_cmd = [ PROC_QEMU, \ + '-machine', 'x-remote', \ + '-device', 'lsi53c895a,id=lsi1', \
... I'd move it to tests/integration/multiproc-x86-lsi53c895a.py ...
scripts/mpqemu-launcher.py line 14
Quoted 1 line
scripts/mpqemu-launcher.py new file +1
-0+ '-drive', 'id=drive_image1,file=/build/ol7-nvme-test-1.qcow2', \
... use avocado.utils.vmimage (see tests/acceptance/boot_linux.py) to download a prebuilt image, ...
Quoted 26 lines
scripts/mpqemu-launcher.py new file +25
-0+ '-device', 'scsi-hd,id=drive1,drive=drive_image1,bus=lsi1.0,' \ + 'scsi-id=0', \ + '-object', \ + 'x-remote-object,id=robj1,devid=lsi1,fd='+str(remote.fileno()), \ + '-nographic', \ + ] + +proxy_cmd = [ PROC_QEMU, \ + '-name', 'OL7.4', \ + '-machine', 'q35,accel=kvm', \ + '-smp', 'sockets=1,cores=1,threads=1', \ + '-m', '2048', \ + '-object', 'memory-backend-memfd,id=sysmem-file,size=2G', \ + '-numa', 'node,memdev=sysmem-file', \ + '-device', 'virtio-scsi-pci,id=virtio_scsi_pci0', \ + '-drive', 'id=drive_image1,if=none,format=qcow2,' \ + 'file=/home/ol7-hdd-1.qcow2', \ + '-device', 'scsi-hd,id=image1,drive=drive_image1,' \ + 'bus=virtio_scsi_pci0.0', \ + '-boot', 'd', \ + '-vnc', ':0', \ + '-device', 'x-pci-proxy-dev,id=lsi1,fd='+str(proxy.fileno()), \ + ] + +
scripts/mpqemu-launcher.py line 49
Quoted 10 lines
scripts/mpqemu-launcher.py new file +10
-0+pid = os.fork(); + +if pid: + # In Proxy + print('Launching QEMU with Proxy object'); + process = subprocess.Popen(proxy_cmd, pass_fds=[proxy.fileno()]) +else: + # In remote + print('Launching Remote process'); + process = subprocess.Popen(remote_cmd, pass_fds=[remote.fileno()])
... and do something within the guest to be sure MultiProc works :)
Regards,
Phil.
Re: [PATCH v11 05/19] multi-process: add qio channel function to transmit
In reply to Jagannathan RamanQuoted 53 lines · Jagannathan Raman
From: Elena Ufimtseva <[email protected]>
The entire array of the memory regions and file handlers. Will be used in the next patch.
Signed-off-by Elena Ufimtseva <[email protected]>Signed-off-by John G Johnson <[email protected]>Signed-off-by Jagannathan Raman <[email protected]>Reviewed-by Stefan Hajnoczi <[email protected]>--- include/io/channel.h | 24 ++++++++++++++++++++++++ io/channel.c | 45 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+)include/io/channel.h +24
-0@@ -773,5 +773,29 @@ void qio_channel_set_aio_fd_handler(QIOChannel *ioc, IOHandler *io_read, IOHandler *io_write, void *opaque); +/** +* qio_channel_writev_full_all: +* @ioc: the channel object +* @iov: the array of memory regions to write data from +* @niov: the length of the @iov array +* @fds: an array of file handles to send +* @nfds: number of file handles in @fds +* @errp: pointer to a NULL-initialized error object +* +* +* Behaves like qio_channel_writev_full but will attempt +* to send all data passed (file handles and memory regions). +* The function will wait for all requested data +* to be written, yielding from the current coroutine +* if required. +* +* Returns: 0 if all bytes were written, or -1 on error +*/ + +int qio_channel_writev_full_all (QIOChannel *ioc, +const struct iovec *iov, +size_t niov, +int *fds,size_t nfds, +Error **errp);#endif /* QIO_CHANNEL_H */ io/channel.c +0
-0
io/channel.c int qio_channel_writev_all(QIOChannel *ioc, · line 199
Quoted 10 lines
io/channel.c +6
-0@@ -190,6 +190,51 @@ int qio_channel_writev_all(QIOChannel *ioc,return ret; } +int qio_channel_writev_full_all (QIOChannel *ioc, +const struct iovec *iov, +size_t niov, +int *fds,size_t nfds, +Error **errp) +{
Please make qio_channel_writev_all() call qio_channel_writev_full_all() to avoid logic duplication.
thanks
+ int ret = -1;
Quoted 45 lines
io/channel.c +38
-1+ struct iovec *local_iov = g_new(struct iovec, niov); +struct iovec *local_iov_head = local_iov; +unsigned int nlocal_iov = niov; + + nlocal_iov =iov_copy (local_iov, nlocal_iov, + iov, niov, +0 ,iov_size (iov, niov)); + +while (nlocal_iov >0 ) { +ssize_t len; + len =qio_channel_writev_full (ioc, local_iov, nlocal_iov, fds, + nfds, errp); +if (len ==QIO_CHANNEL_ERR_BLOCK ) { +if (qemu_in_coroutine ()) { +qio_channel_yield (ioc,G_IO_OUT ); + }else { +qio_channel_wait (ioc,G_IO_OUT ); + } +continue ; + } +if (len <0 ) { + goto cleanup; + } + +iov_discard_front (&local_iov, &nlocal_iov, len); + +if (len >0 ) { + fds =NULL ; + nfds =0 ; + } + } + + ret =0 ; + cleanup: +g_free (local_iov_head); +return ret; +} + ssize_t qio_channel_readv(QIOChannel *ioc, const struct iovec *iov, size_t niov, --1.8.3.1
Signature · 1 line
Marc-André Lureau