Skip to contentSkip to conversation

microvm: memory config tweaks

PATCH Superseded Gerd Hoffmann 4 participants 9 messages May 28, 2020 – May 29, 2020

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover microvm: memory config tweaks

With more microvm memory config tweaks split this into its owns series, the microvm acpi patch series is already big enough ...

v2:
- use 3G split.
- add patch to move virtio-mmio region.
- pick up acks & reviews.

take care,
Gerd

Gerd Hoffmann (4):
microvm: use 3G split unconditionally
microvm: drop max-ram-below-4g support
x86: move max-ram-below-4g to pc
microvm: move virtio base to 0xfeb00000

 include/hw/i386/microvm.h |  2 +-
 include/hw/i386/pc.h      |  2 ++
 include/hw/i386/x86.h     |  4 ----
 hw/i386/microvm.c         | 35 +----------------------------
 hw/i386/pc.c              | 46 +++++++++++++++++++++++++++++++++++++++
 hw/i386/pc_piix.c         | 10 ++++-----
 hw/i386/pc_q35.c          | 10 ++++-----
 hw/i386/x86.c             | 46 ---------------------------------------
 8 files changed, 60 insertions(+), 95 deletions(-)
Signature · 1 line
2.18.4
#1 lore raw

PATCH v2 1/4 microvm: use 3G split unconditionally

Looks like the logic was copied over from q35.

q35 does this for backward compatibility, there is no reason to do this on microvm though. Also microvm doesn't need much mmio space, 1G is more than enough. Using an mmio window smaller than 1G is bad for gigabyte alignment and hugepages though. So split @ 3G unconditionally.

---
 hw/i386/microvm.c | 16 +---------------
 1 file changed, 1 insertion(+), 15 deletions(-)
hw/i386/microvm.c +1 -15
index 937db10ae6a5..44f940813b07 100644
@@ -170,23 +170,9 @@ static void microvm_memory_init(MicrovmMachineState *mms)
     MemoryRegion *ram_below_4g, *ram_above_4g;
     MemoryRegion *system_memory = get_system_memory();
     FWCfgState *fw_cfg;
-    ram_addr_t lowmem;
+    ram_addr_t lowmem = 0xc0000000; /* 3G */
     int i;
 
-    /*
-     * Check whether RAM fits below 4G (leaving 1/2 GByte for IO memory
-     * and 256 Mbytes for PCI Express Enhanced Configuration Access Mapping
-     * also known as MMCFG).
-     * If it doesn't, we need to split it in chunks below and above 4G.
-     * In any case, try to make sure that guest addresses aligned at
-     * 1G boundaries get mapped to host addresses aligned at 1G boundaries.
-     */
-    if (machine->ram_size >= 0xb0000000) {
-        lowmem = 0x80000000;
-    } else {
-        lowmem = 0xb0000000;
-    }
-
     /*
      * Handle the machine opt max-ram-below-4g.  It is basically doing
      * min(qemu limit, user limit).
Signature · 1 line
2.18.4
#2 lore raw

PATCH v2 2/4 microvm: drop max-ram-below-4g support

In reply to Gerd Hoffmann

Not useful for microvm and allows users to shoot themself into the foot (make ram + mmio overlap).

---
 hw/i386/microvm.c | 19 -------------------
 1 file changed, 19 deletions(-)
hw/i386/microvm.c +0 -19
index 44f940813b07..5e931975a06d 100644
@@ -173,25 +173,6 @@ static void microvm_memory_init(MicrovmMachineState *mms)
     ram_addr_t lowmem = 0xc0000000; /* 3G */
     int i;
 
-    /*
-     * Handle the machine opt max-ram-below-4g.  It is basically doing
-     * min(qemu limit, user limit).
-     */
-    if (!x86ms->max_ram_below_4g) {
-        x86ms->max_ram_below_4g = 4 * GiB;
-    }
-    if (lowmem > x86ms->max_ram_below_4g) {
-        lowmem = x86ms->max_ram_below_4g;
-        if (machine->ram_size - lowmem > lowmem &&
-            lowmem & (1 * GiB - 1)) {
-            warn_report("There is possibly poor performance as the ram size "
-                        " (0x%" PRIx64 ") is more then twice the size of"
-                        " max-ram-below-4g (%"PRIu64") and"
-                        " max-ram-below-4g is not a multiple of 1G.",
-                        (uint64_t)machine->ram_size, x86ms->max_ram_below_4g);
-        }
-    }
-
     if (machine->ram_size > lowmem) {
         x86ms->above_4g_mem_size = machine->ram_size - lowmem;
         x86ms->below_4g_mem_size = lowmem;
Signature · 1 line
2.18.4
Reviewed-by on v2 2/4
#3 lore raw

Re: [PATCH v2 2/4] microvm: drop max-ram-below-4g support

Quoted 31 lines · Gerd Hoffmann

Not useful for microvm and allows users to shoot themself into the foot (make ram + mmio overlap).

---
 hw/i386/microvm.c | 19 -------------------
 1 file changed, 19 deletions(-)
hw/i386/microvm.c +0 -12
index 44f940813b07..5e931975a06d 100644
@@ -173,25 +173,6 @@ static void microvm_memory_init(MicrovmMachineState *mms)
     ram_addr_t lowmem = 0xc0000000; /* 3G */
     int i;
 
-    /*
-     * Handle the machine opt max-ram-below-4g.  It is basically doing
-     * min(qemu limit, user limit).
-     */
-    if (!x86ms->max_ram_below_4g) {
-        x86ms->max_ram_below_4g = 4 * GiB;
-    }
-    if (lowmem > x86ms->max_ram_below_4g) {
-        lowmem = x86ms->max_ram_below_4g;
-        if (machine->ram_size - lowmem > lowmem &&
-            lowmem & (1 * GiB - 1)) {
-            warn_report("There is possibly poor performance as the ram size "

hw/i386/microvm.c static void microvm_memory_init(MicrovmMachineState *mms) · line 178

Quoted 10 lines
hw/i386/microvm.c +0 -7
-                        " (0x%" PRIx64 ") is more then twice the size of"
-                        " max-ram-below-4g (%"PRIu64") and"
-                        " max-ram-below-4g is not a multiple of 1G.",
-                        (uint64_t)machine->ram_size, x86ms->max_ram_below_4g);
-        }
-    }
-
     if (machine->ram_size > lowmem) {
         x86ms->above_4g_mem_size = machine->ram_size - lowmem;
         x86ms->below_4g_mem_size = lowmem;
#4 lore raw

PATCH v2 3/4 x86: move max-ram-below-4g to pc

In reply to Gerd Hoffmann

Move from X86MachineClass to PCMachineClass so it disappears from microvm machine type property list.

---
 include/hw/i386/pc.h  |  2 ++
 include/hw/i386/x86.h |  4 ----
 hw/i386/pc.c          | 46 +++++++++++++++++++++++++++++++++++++++++++
 hw/i386/pc_piix.c     | 10 +++++-----
 hw/i386/pc_q35.c      | 10 +++++-----
 hw/i386/x86.c         | 46 -------------------------------------------
 6 files changed, 58 insertions(+), 60 deletions(-)
include/hw/i386/pc.h +2 -0
index 8d764f965cd3..e6135c34d656 100644
@@ -35,6 +35,7 @@ struct PCMachineState {
     PFlashCFI01 *flash[2];
 
     /* Configuration options: */
+    uint64_t max_ram_below_4g;
     OnOffAuto vmport;
 
     bool acpi_build_enabled;
@@ -51,6 +52,7 @@ struct PCMachineState {
 };
 
 #define PC_MACHINE_ACPI_DEVICE_PROP "acpi-device"
+#define PC_MACHINE_MAX_RAM_BELOW_4G "max-ram-below-4g"
 #define PC_MACHINE_DEVMEM_REGION_SIZE "device-memory-region-size"
 #define PC_MACHINE_VMPORT           "vmport"
 #define PC_MACHINE_SMBUS            "smbus"
include/hw/i386/x86.h +0 -4
index b52285481687..b79f24e28545 100644
@@ -51,9 +51,6 @@ typedef struct {
     qemu_irq *gsi;
     GMappedFile *initrd_mapped_file;
 
-    /* Configuration options: */
-    uint64_t max_ram_below_4g;
-
     /* RAM information (sizes, addresses, configuration): */
     ram_addr_t below_4g_mem_size, above_4g_mem_size;
 
@@ -82,7 +79,6 @@ typedef struct {
     AddressSpace *ioapic_as;
 } X86MachineState;
 
-#define X86_MACHINE_MAX_RAM_BELOW_4G "max-ram-below-4g"
 #define X86_MACHINE_SMM              "smm"
 #define X86_MACHINE_ACPI             "acpi"
 
hw/i386/pc.c +46 -0

64 lines not on this page · raw message

hw/i386/pc_piix.c +5 -5
index f66e1d73ce0b..503c35f7bf4c 100644
@@ -127,11 +127,11 @@ static void pc_init1(MachineState *machine,
     if (xen_enabled()) {
         xen_hvm_init(pcms, &ram_memory);
     } else {
-        if (!x86ms->max_ram_below_4g) {
-            x86ms->max_ram_below_4g = 0xe0000000; /* default: 3.5G */
+        if (!pcms->max_ram_below_4g) {
+            pcms->max_ram_below_4g = 0xe0000000; /* default: 3.5G */
         }
-        lowmem = x86ms->max_ram_below_4g;
-        if (machine->ram_size >= x86ms->max_ram_below_4g) {
+        lowmem = pcms->max_ram_below_4g;
+        if (machine->ram_size >= pcms->max_ram_below_4g) {
             if (pcmc->gigabyte_align) {
                 if (lowmem > 0xc0000000) {
                     lowmem = 0xc0000000;
@@ -140,7 +140,7 @@ static void pc_init1(MachineState *machine,
                     warn_report("Large machine and max_ram_below_4g "
                                 "(%" PRIu64 ") not a multiple of 1G; "
                                 "possible bad performance.",
-                                x86ms->max_ram_below_4g);
+                                pcms->max_ram_below_4g);
                 }
             }
         }
hw/i386/pc_q35.c +5 -5
index 4ba8ac8774e4..90e8fb2cb737 100644
@@ -154,18 +154,18 @@ static void pc_q35_init(MachineState *machine)
     /* Handle the machine opt max-ram-below-4g.  It is basically doing
      * min(qemu limit, user limit).
      */
-    if (!x86ms->max_ram_below_4g) {
-        x86ms->max_ram_below_4g = 4 * GiB;
+    if (!pcms->max_ram_below_4g) {
+        pcms->max_ram_below_4g = 4 * GiB;
     }
-    if (lowmem > x86ms->max_ram_below_4g) {
-        lowmem = x86ms->max_ram_below_4g;
+    if (lowmem > pcms->max_ram_below_4g) {
+        lowmem = pcms->max_ram_below_4g;
         if (machine->ram_size - lowmem > lowmem &&
             lowmem & (1 * GiB - 1)) {
             warn_report("There is possibly poor performance as the ram size "
                         " (0x%" PRIx64 ") is more then twice the size of"
                         " max-ram-below-4g (%"PRIu64") and"
                         " max-ram-below-4g is not a multiple of 1G.",
-                        (uint64_t)machine->ram_size, x86ms->max_ram_below_4g);
+                        (uint64_t)machine->ram_size, pcms->max_ram_below_4g);
         }
     }
 
hw/i386/x86.c +0 -46

64 lines not on this page · raw message

Signature · 1 line
2.18.4
#5 lore raw

PATCH v2 4/4 microvm: move virtio base to 0xfeb00000

In reply to Gerd Hoffmann

Place virtio-mmio devices near the other mmio regions, next ioapic is at @ 0xfec00000.

---
 include/hw/i386/microvm.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
include/hw/i386/microvm.h +1 -1
index ba68d1f22bb3..fd34b78e0d2a 100644
@@ -26,7 +26,7 @@
 #include "hw/i386/x86.h"
 
 /* Platform virtio definitions */
-#define VIRTIO_MMIO_BASE      0xc0000000
+#define VIRTIO_MMIO_BASE      0xfeb00000
 #define VIRTIO_IRQ_BASE       5
 #define VIRTIO_NUM_TRANSPORTS 8
 #define VIRTIO_CMDLINE_MAXLEN 64
Signature · 1 line
2.18.4
#6 lore raw
In reply to Gerd Hoffmann
Patchew URL: https://patchew.org/QEMU/[email protected]/

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series failed the docker-quick@centos7 build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-centos7 V=1 NETWORK=1
time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1 === TEST SCRIPT END ===

CC x86_64-softmmu/target/i386/translate.o
CC aarch64-softmmu/hw/arm/omap1.o
/tmp/qemu-test/src/hw/i386/xen/xen-hvm.c: In function 'xen_ram_init':
/tmp/qemu-test/src/hw/i386/xen/xen-hvm.c:206:34: error: 'X86_MACHINE_MAX_RAM_BELOW_4G' undeclared (first use in this function)

                                  X86_MACHINE_MAX_RAM_BELOW_4G,
                                  ^

/tmp/qemu-test/src/hw/i386/xen/xen-hvm.c:206:34: note: each undeclared identifier is reported only once for each function it appears in make[1]: *** [hw/i386/xen/xen-hvm.o] Error 1
CC aarch64-softmmu/hw/arm/omap2.o
make[1]: *** Waiting for unfinished jobs....
CC aarch64-softmmu/hw/arm/strongarm.o
---

  CC      aarch64-softmmu/target/arm/gdbstub64.o
  CC      aarch64-softmmu/target/arm/machine.o
  CC      aarch64-softmmu/target/arm/arch_dump.o

make: *** [x86_64-softmmu/all] Error 2
make: *** Waiting for unfinished jobs....
CC aarch64-softmmu/target/arm/monitor.o
CC aarch64-softmmu/target/arm/arm-powerctl.o
---
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=74ab42ef34594fdb988f724e8efaa96d', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-fky5lt0a/src/docker-src.2020-05-28-16.55.24.5056:/var/tmp/qemu:z,ro', 'qemu:centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit status 2. filter=--filter=label=com.qemu.instance.uuid=74ab42ef34594fdb988f724e8efaa96d
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-fky5lt0a/src'
make: *** [docker-run-test-quick@centos7] Error 2

real 3m8.799s
user 0m4.488s

The full log is available at
http://patchew.org/logs/[email protected]/testing.docker-quick@centos7/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]

#7 lore raw
Quoted 16 lines · On Thu, 28 May 2020 13:58:35 -0700 (PDT) [email protected] wrote:

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series failed the docker-quick@centos7 build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-centos7 V=1 NETWORK=1
time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1 === TEST SCRIPT END ===

Quoted 5 lines

CC x86_64-softmmu/target/i386/translate.o
CC aarch64-softmmu/hw/arm/omap1.o
/tmp/qemu-test/src/hw/i386/xen/xen-hvm.c: In function 'xen_ram_init':
/tmp/qemu-test/src/hw/i386/xen/xen-hvm.c:206:34: error: 'X86_MACHINE_MAX_RAM_BELOW_4G' undeclared (first use in this function)

                                  X86_MACHINE_MAX_RAM_BELOW_4G,

it seems the build is still broken

Quoted 31 lines
                                  ^

/tmp/qemu-test/src/hw/i386/xen/xen-hvm.c:206:34: note: each undeclared identifier is reported only once for each function it appears in make[1]: *** [hw/i386/xen/xen-hvm.o] Error 1
CC aarch64-softmmu/hw/arm/omap2.o
make[1]: *** Waiting for unfinished jobs....
CC aarch64-softmmu/hw/arm/strongarm.o
---

  CC      aarch64-softmmu/target/arm/gdbstub64.o
  CC      aarch64-softmmu/target/arm/machine.o
  CC      aarch64-softmmu/target/arm/arch_dump.o

make: *** [x86_64-softmmu/all] Error 2
make: *** Waiting for unfinished jobs....
CC aarch64-softmmu/target/arm/monitor.o
CC aarch64-softmmu/target/arm/arm-powerctl.o
---
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=74ab42ef34594fdb988f724e8efaa96d', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-fky5lt0a/src/docker-src.2020-05-28-16.55.24.5056:/var/tmp/qemu:z,ro', 'qemu:centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit status 2. filter=--filter=label=com.qemu.instance.uuid=74ab42ef34594fdb988f724e8efaa96d
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-fky5lt0a/src'
make: *** [docker-run-test-quick@centos7] Error 2

real 3m8.799s
user 0m4.488s

The full log is available at
http://patchew.org/logs/[email protected]/testing.docker-quick@centos7/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]

#8 lore raw
In reply to Gerd Hoffmann
Patchew URL: https://patchew.org/QEMU/[email protected]/

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series failed the asan build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
export ARCH=x86_64
make docker-image-fedora V=1 NETWORK=1
time make docker-test-debug@fedora TARGET_LIST=x86_64-softmmu J=14 NETWORK=1 === TEST SCRIPT END ===

  CC      x86_64-softmmu/softmmu/main.o
  CC      x86_64-softmmu/gdbstub-xml.o
  CC      x86_64-softmmu/trace/generated-helpers.o
/tmp/qemu-test/src/hw/i386/xen/xen-hvm.c:206:34: error: use of undeclared identifier 'X86_MACHINE_MAX_RAM_BELOW_4G'
                                 X86_MACHINE_MAX_RAM_BELOW_4G,
                                 ^

1 error generated.
make[1]: *** [/tmp/qemu-test/src/rules.mak:69: hw/i386/xen/xen-hvm.o] Error 1 make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:527: x86_64-softmmu/all] Error 2
Traceback (most recent call last):
File "./tests/docker/docker.py", line 664, in <module>

    sys.exit(main())

---
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=0bac4918513149f2a4717d0ab9d331ee', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=x86_64-softmmu', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-sdwodinx/src/docker-src.2020-05-28-16.59.46.14444:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-debug']' returned non-zero exit status 2. filter=--filter=label=com.qemu.instance.uuid=0bac4918513149f2a4717d0ab9d331ee
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-sdwodinx/src'
make: *** [docker-run-test-debug@fedora] Error 2

real 4m15.596s
user 0m7.249s

The full log is available at
http://patchew.org/logs/[email protected]/testing.asan/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]