Skip to content

RFC ACPI memory hotplug

Igor Mammedov qemu-devel 2 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v7 RFC Latest27 patches 111 repliesIn review
  2. v6 RFC16 patches 31 repliesApplied

Patches in v7 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/27acpi: factor out common pm_update_sci() into acpi corehw/acpi/core.c hw/acpi/ich9.c hw/acpi/piix4.c +1 more+38 -452 replies
2/27rename pci_hotplug_fn to hotplug_fn and make it available for other deviceshw/acpi/pcihp.c hw/acpi/piix4.c hw/pci/pci.c +6 more+42 -383 replies
3/27pc: add 'etc/reserved-memory-end' fw_cfg interface for SeaBIOShw/i386/pc.c hw/i386/pc_piix.c hw/i386/pc_q35.c +3 more+40 -63 replies
4/27vl: convert -m to qemu_opts_parse()qemu-options.hx vl.c+68 -1415 replies
5/27qapi: add SIZE type parser to string_input_visitorqapi/string-input-visitor.c+18 -016 replies
6/27get reference to /backend container via qemu_get_backend()include/sysemu/sysemu.h ui/console.c vl.c+16 -10 replies
7/27add memdev backend infrastructurebackends/Makefile.objs backends/hostmem.c backends/hostmem_compat_ram.c +7 more+501 -08 replies
8/27dimm: implement dimm device abstractiondefault-configs/i386-softmmu.mak default-configs/x86_64-softmmu.mak hw/Makefile.objs +3 more+251 -03 replies
9/27dimm: map DimmDevice into DimBus provided address spacehw/mem/dimm.c include/hw/mem/dimm.h+28 -00 replies
10/27dimm: add busy slot check and slot auto-allocationhw/mem/dimm.c include/hw/mem/dimm.h+54 -10 replies
11/27dimm: add busy address check and address auto-allocationhw/mem/dimm.c include/hw/mem/dimm.h+84 -00 replies
12/27dimm: add hotplug callback to DimmBushw/i386/pc.c hw/mem/dimm.c include/hw/i386/pc.h +1 more+25 -02 replies
13/27acpi: memory hotplug ACPI hardware implementationdocs/specs/acpi_mem_hotplug.txt hw/acpi/core.c include/hw/acpi/acpi.h+206 -012 replies
14/27acpi: initialize memory hotplug ACPI PIIX4 hardwarehw/acpi/piix4.c include/hw/acpi/piix4.h+18 -00 replies
15/27acpi: piix4: add memory-hotplug-io-base property to piix4_pmhw/acpi/piix4.c include/hw/acpi/acpi.h+15 -10 replies
16/27acpi: ich9: allow guest to clear SCI rised by GPEhw/acpi/ich9.c+1 -020 replies
17/27acpi: initialize memory hotplug ACPI ICH9 hardwarehw/acpi/ich9.c include/hw/acpi/ich9.h+21 -00 replies
18/27acpi: ich9: add memory-hotplug-io-base property to ich9_pmhw/acpi/ich9.c+11 -00 replies
19/27acpi: piix4/ich9: add optional vmstate field for MemHotplugState migrationhw/acpi/core.c hw/acpi/ich9.c hw/acpi/piix4.c +1 more+49 -00 replies
20/27pc: piix: make PCII440FXState type publichw/pci-host/piix.c include/hw/pci-host/piix.h+28 -210 replies
21/27pc: add memory hotplug 440fx machinehw/i386/pc.c hw/i386/pc_piix.c hw/pci-host/piix.c +2 more+87 -215 replies
22/27pc: add memory hotplug Q35 machinehw/i386/pc_q35.c hw/pci-host/q35.c include/hw/pci-host/q35.h+21 -10 replies
23/27pc: ACPI BIOS: implement memory hotplug interfacehw/i386/Makefile.objs hw/i386/acpi-build.c hw/i386/ssdt-mem.dsl +1 more+292 -16 replies
24/27pc: ACPI BIOS: add ssdt-mem.hex.generated and update ssdt-misc.hex.generatedhw/i386/ssdt-mem.hex.generated hw/i386/ssdt-misc.hex.generated+1074 -40 replies
25/27pc: ACPI BIOS: use enum for defining memory affinity flagshw/i386/acpi-build.c+15 -72 replies
26/27pc: ACPI BIOS: reserve SRAT entry for hotplug mem holehw/i386/acpi-build.c+33 -02 replies
27/27pc: ACPI BIOS: make GPE.3 handle memory hotplug event on PIIX and Q35 machineshw/i386/acpi-dsdt.dsl hw/i386/q35-acpi-dsdt.dsl+8 -20 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v7 yet.

Read the v7 thread

Cover letter [Qemu-devel] [PATCH 00/27 RFC v7] ACPI memory hotplug

---
What's new since v6:

* DIMM device is split to backend and frontend. Therefore following command/options were added for supporting it:

For memdev backend:
CLI: -memdev-add
Monitor/QMP: memdev-add
with options: 'id' and 'size'
For dimm frontend:
option "size" became readonly, pulling it's size from attached backend added option "memdev" for specifying backend by 'id'

* Added Q35 support
* Added support for 32 bit guests
* build for i386 emulator (not tested)
---

This series allows to hotplug 'arbitrary' DIMM devices specifying size, NUMA node mapping (guest side), slot and address where to map it, at runtime.

Due to ACPI limitation there is need to specify a number of possible DIMM devices. For this task -m option was extended to support following format:

-m [mem=]RamSize[,slots=N,maxmem=M]

To allow memory hotplug user must specify a pair of additional parameters:
'slots' - number of possible increments
'maxmem' - max possible total memory size QEMU is allowed to use,
including RamSize.

minimal monitor command syntax to hotplug DIMM device:

memdev-add id=memX,size=1G
device_add dimm,id=dimmX,memdev=memX

DIMM device provides following properties that could be used with device_add / -device to alter default behavior:

id - unique string identifying device [mandatory]
slot - number in range [0-slots) [optional], if not specified
the first free slot is used
node - NUMA node id [optional] (default: 0)
size - amount of memory to add, readonly derived from backing memdev start - guest's physical address where to plug DIMM [optional],
if not specified the first gap in hotplug memory region that fits DIMM is used

-device option could be used for adding potentially hotunplugable DIMMs
and also for specifying hotplugged DIMMs in migration case.

Tested guests:
- RHEL 6x64
- Windows 2012DCx64
- Windows 2008DCx64
- Windows 2008DCx32

Known limitations/bugs/TODOs:
- only hot-add supported
- max number of supported DIMM devices 255 (due to ACPI object name limit), could be increased creating several containers and putting DIMMs there. (exercise for future)
- failed hotplug action consumes 1 slot (device_add doesn't delete device properly if realize failed)
- e820 table doesn't include DIMM devices added with -device / (or after reboot devices added with device_add)
- Windows 2008 remembers DIMM configuration, so if DIMM with other start/size is added into the same slot, it refuses to use it insisting on old mapping.

Series is based on mst's PCI tree and requires following SeaBIOS patch:
http://patchwork.ozlabs.org/patch/292614/
on top of patches to load ACPI tables from QEMU
working SeaBIOS tree for testing is available at:
https://github.com/imammedo/seabios/commits/memhp-wip

QEMU git tree for testing is available at:
https://github.com/imammedo/qemu/commits/memory-hotplug-v7

Example QEMU cmd line:

  qemu-system-x86_64 -enable-kvm -monitor unix:/tmp/mon,server,nowait \ 
     -m 4096,slots=4,maxmem=8G -L /custome_seabios guest.img

PS:
Windows guest requires SRAT table for hotplug to work so add extra option:
-numa node
to QEMU command line.

Igor Mammedov (26):
acpi: factor out common pm_update_sci() into acpi core rename pci_hotplug_fn to hotplug_fn and make it available for other
devices
pc: add 'etc/reserved-memory-end' fw_cfg interface for SeaBIOS
vl: convert -m to qemu_opts_parse()
qapi: add SIZE type parser to string_input_visitor
get reference to /backend container via qemu_get_backend() add memdev backend infrastructure
dimm: map DimmDevice into DimBus provided address space
dimm: add busy slot check and slot auto-allocation
dimm: add busy address check and address auto-allocation
dimm: add hotplug callback to DimmBus
acpi: memory hotplug ACPI hardware implementation
acpi: initialize memory hotplug ACPI PIIX4 hardware
acpi: piix4: add memory-hotplug-io-base property to piix4_pm
acpi: ich9: allow guest to clear SCI rised by GPE
acpi: initialize memory hotplug ACPI ICH9 hardware
acpi: ich9: add memory-hotplug-io-base property to ich9_pm
acpi: piix4/ich9: add optional vmstate field for MemHotplugState
migration
pc: piix: make PCII440FXState type public
pc: add memory hotplug 440fx machine
pc: add memory hotplug Q35 machine
pc: ACPI BIOS: implement memory hotplug interface
pc: ACPI BIOS: add ssdt-mem.hex.generated and update
ssdt-misc.hex.generated
pc: ACPI BIOS: use enum for defining memory affinity flags
pc: ACPI BIOS: reserve SRAT entry for hotplug mem hole
pc: ACPI BIOS: make GPE.3 handle memory hotplug event on PIIX and Q35
machines

Vasilis Liaskovitis (1):
dimm: implement dimm device abstraction

 backends/Makefile.objs             |    2 +
 backends/hostmem.c                 |  275 +++++++++++
 backends/hostmem_compat_ram.c      |   42 ++
 default-configs/i386-softmmu.mak   |    1 +
 default-configs/x86_64-softmmu.mak |    1 +
 docs/specs/acpi_mem_hotplug.txt    |   38 ++
 hmp-commands.hx                    |   13 +
 hw/Makefile.objs                   |    1 +
 hw/acpi/core.c                     |  189 ++++++++
 hw/acpi/ich9.c                     |   57 ++-
 hw/acpi/pcihp.c                    |    6 +-
 hw/acpi/piix4.c                    |   91 +++--
 hw/i386/Makefile.objs              |    3 +-
 hw/i386/acpi-build.c               |   91 ++++-
 hw/i386/acpi-dsdt.dsl              |    5 +-
 hw/i386/pc.c                       |   90 ++++-
 hw/i386/pc_piix.c                  |    7 +-
 hw/i386/pc_q35.c                   |    4 +
 hw/i386/q35-acpi-dsdt.dsl          |    5 +-
 hw/i386/ssdt-mem.dsl               |   75 +++
 hw/i386/ssdt-mem.hex.generated     |  197 ++++++++
 hw/i386/ssdt-misc.dsl              |  179 ++++++++
 hw/i386/ssdt-misc.hex.generated    |  881 +++++++++++++++++++++++++++++++++++-
 hw/mem/Makefile.objs               |    1 +
 hw/mem/dimm.c                      |  321 +++++++++++++
 hw/pci-host/piix.c                 |   36 +-
 hw/pci-host/q35.c                  |   19 +-
 hw/pci/pci.c                       |   12 +-
 hw/pci/pcie.c                      |    7 +-
 hw/pci/shpc.c                      |    9 +-
 include/hw/acpi/acpi.h             |   42 ++
 include/hw/acpi/ich9.h             |    6 +
 include/hw/acpi/pcihp.h            |    2 +-
 include/hw/acpi/piix4.h            |    2 +
 include/hw/i386/pc.h               |   33 ++-
 include/hw/mem/dimm.h              |   99 ++++
 include/hw/pci-host/piix.h         |   29 ++
 include/hw/pci-host/q35.h          |    1 +
 include/hw/pci/pci.h               |   11 +-
 include/hw/pci/pci_bus.h           |    2 +-
 include/hw/qdev-core.h             |    9 +
 include/sysemu/hostmem.h           |  102 +++++
 include/sysemu/sysemu.h            |    3 +
 monitor.c                          |    1 +
 qapi-schema.json                   |   18 +
 qapi/string-input-visitor.c        |   18 +
 qemu-options.hx                    |   16 +-
 qmp-commands.hx                    |   27 ++
 ui/console.c                       |    3 +-
 vl.c                               |   98 ++++-
 50 files changed, 3040 insertions(+), 140 deletions(-)
 create mode 100644 backends/hostmem.c
 create mode 100644 backends/hostmem_compat_ram.c
 create mode 100644 docs/specs/acpi_mem_hotplug.txt
 create mode 100644 hw/i386/ssdt-mem.dsl
 create mode 100644 hw/i386/ssdt-mem.hex.generated
 create mode 100644 hw/mem/Makefile.objs
 create mode 100644 hw/mem/dimm.c
 create mode 100644 include/hw/mem/dimm.h
 create mode 100644 include/hw/pci-host/piix.h
 create mode 100644 include/sysemu/hostmem.h