Skip to content

Initial support for multi-process Qemu

Jagannathan Raman qemu-devel 13 versions – Applied

Status from this message: Thanks, applied to my block tree:

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v21 Latest20 patches 9 repliesApplied
  2. v2020 patches 6 repliesSuperseded
  3. v1920 patches 8 repliesSuperseded
Earlier versions · 10
VersionPostedPatchesRepliesStatus
v1820 patches2 repliesSuperseded
v1720 patches0 repliesSuperseded
v1620 patches6 repliesSuperseded
v1319 patches13 repliesSuperseded
v1219 patches32 repliesSuperseded
v1119 patches11 repliesSuperseded
v820 patches11 repliesSuperseded
v550 patches63 repliesSuperseded
v4 RFC49 patches87 repliesSuperseded
v3 RFC45 patches26 repliesSuperseded

Patches in v21 Applied

#Subject and filesSizeRepliesApplied
1/20multi-process: add the concept description to docs/devel/qemu-multiprocessMAINTAINERS docs/devel/index.rst docs/devel/multi-process.rst+974 -00 replies
2/20multi-process: add configure and usage informationMAINTAINERS docs/multi-process.rst+65 -00 replies
3/20memory: alloc RAM from file at offsetbackends/hostmem-memfd.c hw/misc/ivshmem.c include/exec/memory.h +6 more+23 -130 replies
4/20multi-process: Add config option for multi-process QEMUKconfig.host configure hw/Kconfig +2 more+21 -10 replies
5/20multi-process: setup PCI host bridge for remote deviceMAINTAINERS hw/pci-host/Kconfig hw/pci-host/meson.build +3 more+111 -00 replies
6/20multi-process: setup a machine object for remote device processMAINTAINERS hw/meson.build hw/remote/machine.c +3 more+106 -00 replies
7/20io: add qio_channel_writev_full_all helperinclude/io/channel.h io/channel.c+39 -10 replies
8/20io: add qio_channel_readv_full_all_eof & qio_channel_readv_full_all helpersinclude/io/channel.h io/channel.c+134 -201 reply
9/20multi-process: define MPQemuMsg format and transmission functionsMAINTAINERS hw/remote/meson.build hw/remote/mpqemu-link.c +6 more+289 -02 replies
10/20multi-process: Initialize message handler in remote deviceMAINTAINERS hw/remote/meson.build hw/remote/message.c +1 more+68 -00 replies
11/20multi-process: Associate fd of a PCIDevice with its objectMAINTAINERS hw/remote/meson.build hw/remote/remote-obj.c+205 -00 replies
12/20multi-process: setup memory manager for remote deviceMAINTAINERS hw/remote/memory.c hw/remote/meson.build +3 more+109 -00 replies
13/20multi-process: introduce proxy objectMAINTAINERS hw/remote/meson.build hw/remote/proxy.c +1 more+135 -00 replies
14/20multi-process: add proxy communication functionshw/remote/mpqemu-link.c include/hw/remote/mpqemu-link.h+38 -00 replies
15/20multi-process: Forward PCI config space acceses to the remote processhw/remote/message.c hw/remote/mpqemu-link.c hw/remote/proxy.c +1 more+132 -10 replies
16/20multi-process: PCI BAR read/write handling for proxy & remote endpointshw/remote/message.c hw/remote/mpqemu-link.c hw/remote/proxy.c +2 more+168 -00 replies
17/20multi-process: Synchronize remote memoryMAINTAINERS hw/remote/meson.build hw/remote/message.c +4 more+270 -00 replies
18/20multi-process: create IOHUB object to handle irqMAINTAINERS hw/remote/iohub.c hw/remote/machine.c +9 more+249 -00 replies
19/20multi-process: Retrieve PCI info from remote processhw/remote/proxy.c+84 -00 replies
20/20multi-process: perform device reset in the remote processhw/remote/message.c hw/remote/proxy.c include/hw/remote/mpqemu-link.h+42 -00 replies

Reviews trailers per patch

All 20 patches have a Reviewed-by or Acked-by.

PatchStefan HajnocziDaniel P. Berrangé
1/20R-b
2/20R-b
3/20R-b
4/20R-b
5/20R-b
6/20R-b
7/20R-b A-b
8/20A-b
9/20R-b
10/20R-b
11/20R-b
12/20R-b
13/20R-b
14/20R-b
15/20R-b
16/20R-b
17/20R-b
18/20R-b
19/20R-b
20/20R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v21 thread

Cover letter [PATCH v21 00/20] Initial support for multi-process Qemu

Hi,

This is the v21 of the patchset. This version has the following changes:

[PATCH v18 08/20] io: add qio_channel_readv_full_all_eof &
qio_channel_readv_full_all helpers
- Separates out the error handling for (len = 0) & (len < 0)
- (len = 0) does not necessarily mean EOF. It could also be a case where only FDs are read, without any data with more data possibly on the way. Addresses this scenario as well, as was pointed out in the feedback from last time.

To touch upon the history of this project, we posted the Proof Of Concept patches before the BoF session in 2018. Subsequently, we have posted 20 versions on the qemu-devel mailing list. You can find them by following the links below ([1] - [20]). Following people contributed to the design and implementation of this project:
Jagannathan Raman <[email protected]>
Elena Ufimtseva <[email protected]>
John G Johnson <[email protected]>
Stefan Hajnoczi <[email protected]>
Konrad Wilk <[email protected]>
Kanth Ghatraju <[email protected]>

We would like to thank you for your feedback in the design and implementation of this project. Qemu wiki page:
https://wiki.qemu.org/Features/MultiProcessQEMU

For the full concept writeup about QEMU multi-process, please refer to docs/devel/qemu-multiprocess.rst. Also, see docs/qemu-multiprocess.txt for usage information.

Thank you for reviewing this series!

[POC]: https://www.mail-archive.com/[email protected]/msg566538.html
[1]: https://www.mail-archive.com/[email protected]/msg602285.html
[2]: https://www.mail-archive.com/[email protected]/msg624877.html
[3]: https://www.mail-archive.com/[email protected]/msg642000.html
[4]: https://www.mail-archive.com/[email protected]/msg655118.html
[5]: https://www.mail-archive.com/[email protected]/msg682429.html
[6]: https://www.mail-archive.com/[email protected]/msg697484.html
[7]: https://patchew.org/QEMU/[email protected]/
[8]: https://www.mail-archive.com/[email protected]/msg727007.html
[9]: https://www.mail-archive.com/[email protected]/msg734275.html
[10]: https://www.mail-archive.com/[email protected]/msg747638.html
[11]: https://www.mail-archive.com/[email protected]/msg750972.html
[12]: https://patchew.org/QEMU/[email protected]/
[13]: https://www.mail-archive.com/[email protected]/msg766825.html
[14]: https://www.mail-archive.com/[email protected]/msg768376.html
[15]: https://www.mail-archive.com/[email protected]/msg769178.html
[16]: https://www.mail-archive.com/[email protected]/msg771455.html
[17]: https://www.mail-archive.com/[email protected]/msg772512.html
[18]: https://www.mail-archive.com/[email protected]/msg772602.html
[19]: https://www.mail-archive.com/[email protected]/msg772980.html
[20]: https://patchew.org/QEMU/[email protected]/

Elena Ufimtseva (8):
multi-process: add configure and usage information
io: add qio_channel_writev_full_all helper
io: add qio_channel_readv_full_all_eof & qio_channel_readv_full_all
helpers
multi-process: define MPQemuMsg format and transmission functions
multi-process: introduce proxy object
multi-process: add proxy communication functions
multi-process: Forward PCI config space acceses to the remote process
multi-process: perform device reset in the remote process

Jagannathan Raman (11):
memory: alloc RAM from file at offset
multi-process: Add config option for multi-process QEMU
multi-process: setup PCI host bridge for remote device
multi-process: setup a machine object for remote device process
multi-process: Initialize message handler in remote device
multi-process: Associate fd of a PCIDevice with its object
multi-process: setup memory manager for remote device
multi-process: PCI BAR read/write handling for proxy & remote
endpoints
multi-process: Synchronize remote memory
multi-process: create IOHUB object to handle irq
multi-process: Retrieve PCI info from remote process

John G Johnson (1):
multi-process: add the concept description to
docs/devel/qemu-multiprocess

 docs/devel/index.rst                      |   1 +
 docs/devel/multi-process.rst              | 966 ++++++++++++++++++++++++++++++
 docs/multi-process.rst                    |  64 ++
 configure                                 |  10 +
 meson.build                               |   5 +-
 hw/remote/trace.h                         |   1 +
 include/exec/memory.h                     |   2 +
 include/exec/ram_addr.h                   |   2 +-
 include/hw/pci-host/remote.h              |  30 +
 include/hw/pci/pci_ids.h                  |   3 +
 include/hw/remote/iohub.h                 |  42 ++
 include/hw/remote/machine.h               |  38 ++
 include/hw/remote/memory.h                |  19 +
 include/hw/remote/mpqemu-link.h           |  99 +++
 include/hw/remote/proxy-memory-listener.h |  28 +
 include/hw/remote/proxy.h                 |  48 ++
 include/io/channel.h                      |  78 +++
 include/qemu/mmap-alloc.h                 |   4 +-
 include/sysemu/iothread.h                 |   6 +
 backends/hostmem-memfd.c                  |   2 +-
 hw/misc/ivshmem.c                         |   3 +-
 hw/pci-host/remote.c                      |  75 +++
 hw/remote/iohub.c                         | 119 ++++
 hw/remote/machine.c                       |  80 +++
 hw/remote/memory.c                        |  65 ++
 hw/remote/message.c                       | 230 +++++++
 hw/remote/mpqemu-link.c                   | 267 +++++++++
 hw/remote/proxy-memory-listener.c         | 227 +++++++
 hw/remote/proxy.c                         | 379 ++++++++++++
 hw/remote/remote-obj.c                    | 203 +++++++
 io/channel.c                              | 116 +++-
 iothread.c                                |   6 +
 softmmu/memory.c                          |   3 +-
 softmmu/physmem.c                         |  11 +-
 util/mmap-alloc.c                         |   7 +-
 util/oslib-posix.c                        |   2 +-
 Kconfig.host                              |   4 +
 MAINTAINERS                               |  24 +
 hw/Kconfig                                |   1 +
 hw/meson.build                            |   1 +
 hw/pci-host/Kconfig                       |   3 +
 hw/pci-host/meson.build                   |   1 +
 hw/remote/Kconfig                         |   4 +
 hw/remote/meson.build                     |  13 +
 hw/remote/trace-events                    |   4 +
 45 files changed, 3261 insertions(+), 35 deletions(-)
 create mode 100644 docs/devel/multi-process.rst
 create mode 100644 docs/multi-process.rst
 create mode 100644 hw/remote/trace.h
 create mode 100644 include/hw/pci-host/remote.h
 create mode 100644 include/hw/remote/iohub.h
 create mode 100644 include/hw/remote/machine.h
 create mode 100644 include/hw/remote/memory.h
 create mode 100644 include/hw/remote/mpqemu-link.h
 create mode 100644 include/hw/remote/proxy-memory-listener.h
 create mode 100644 include/hw/remote/proxy.h
 create mode 100644 hw/pci-host/remote.c
 create mode 100644 hw/remote/iohub.c
 create mode 100644 hw/remote/machine.c
 create mode 100644 hw/remote/memory.c
 create mode 100644 hw/remote/message.c
 create mode 100644 hw/remote/mpqemu-link.c
 create mode 100644 hw/remote/proxy-memory-listener.c
 create mode 100644 hw/remote/proxy.c
 create mode 100644 hw/remote/remote-obj.c
 create mode 100644 hw/remote/Kconfig
 create mode 100644 hw/remote/meson.build
 create mode 100644 hw/remote/trace-events
Signature · 1 line
1.8.3.1