Skip to contentSkip to conversation

cpu: add i386 cpu hot remove support

PATCH Superseded Zhu Guihua 4 participants 26 messages Jan 14, 2015 – Feb 13, 2015 3 messages not in the archive

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Zhu Guihua

author
#0 lore raw

v2 cover cpu: add i386 cpu hot remove support

This series is based on chen fan's previous i386 cpu hot remove patchset:
https://lists.nongnu.org/archive/html/qemu-devel/2013-12/msg04266.html

Via implementing ACPI standard methods _EJ0 in ACPI table, after Guest OS remove one vCPU online, the fireware will store removed bitmap to QEMU, then QEMU could know to notify the assigned vCPU of exiting. Meanwhile, intruduce the QOM command 'device_del' to remove vCPU from QEMU itself.

The whole work is based on the new hot plug/unplug framework, ,the unplug request callback does the pre-check and send the request, unplug callback does the removal handling.

This series depends on tangchen's common hot plug/unplug enhance patchset. [RESEND PATCH v1 0/5] Common unplug and unplug request cb for memory and CPU hot-unplug
https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg00429.html

The is the second half of the previous series:
[RFC V2 00/10] cpu: add device_add foo-x86_64-cpu and i386 cpu hot remove support
https://lists.nongnu.org/archive/html/qemu-devel/2014-08/msg04779.html

If you want to test the series, you need to apply the 'device_add foo-x86_64-cpu' patchset first:
[PATCH v3 0/7] cpu: add device_add foo-x86_64-cpu support
https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg01552.html

---
Changelog since v1:
-rebase on the latest version.
-delete patch i386/cpu: add instance finalize callback, and put it into patchset
[PATCH v3 0/6] cpu: add device_add foo-x86_64-cpu support.

Changelog since RFC:
-splited the i386 cpu hot remove into single thread. -replaced apic_no with apic_id, so does the related stuff to make it
work with arbitrary CPU hotadd.
-add the icc_device_unrealize callback to handle apic unrealize. -rework on the new hot plug/unplug platform.
---

Chen Fan (2):
x86: add x86_cpu_unrealizefn() for cpu apic remove
cpu hotplug: implement function cpu_status_write() for vcpu ejection

Gu Zheng (5):
acpi/cpu: add cpu hot unplug request callback function acpi/piix4: add cpu hot unplug callback support acpi/ich9: add cpu hot unplug support
pc: add cpu hot unplug callback support
cpus: reclaim allocated vCPU objects

Zhu Guihua (4):
acpi/piix4: add cpu hot unplug request callback support acpi/ich9: add cpu hot unplug request callback support
pc: add cpu hot unplug request callback support acpi/cpu: add cpu hot unplug callback function

 cpus.c                            | 44 ++++++++++++++++++++
 hw/acpi/cpu_hotplug.c             | 88 ++++++++++++++++++++++++++++++++++++---
 hw/acpi/ich9.c                    | 17 ++++++--
 hw/acpi/piix4.c                   | 12 +++++-
 hw/core/qdev.c                    |  2 +-
 hw/cpu/icc_bus.c                  | 11 +++++
 hw/i386/acpi-dsdt-cpu-hotplug.dsl |  6 ++-
 hw/i386/kvm/apic.c                |  8 ++++
 hw/i386/pc.c                      | 62 +++++++++++++++++++++++++--
 hw/intc/apic.c                    | 10 +++++
 hw/intc/apic_common.c             | 21 ++++++----
 include/hw/acpi/cpu_hotplug.h     |  8 ++++
 include/hw/cpu/icc_bus.h          |  1 +
 include/hw/i386/apic_internal.h   |  1 +
 include/hw/qdev-core.h            |  1 +
 include/qom/cpu.h                 |  9 ++++
 include/sysemu/kvm.h              |  1 +
 kvm-all.c                         | 57 ++++++++++++++++++++++++-
 target-i386/cpu.c                 | 46 ++++++++++++++++++++
 19 files changed, 378 insertions(+), 27 deletions(-)
Signature · 1 line
1.9.3
author
#1 lore raw

PATCH v2 1/11 x86: add x86_cpu_unrealizefn() for cpu apic remove

From: Chen Fan <[email protected]>

Implement x86_cpu_unrealizefn() for corresponding x86_cpu_realizefn(), which is mostly used to clean the apic related allocation and vmstates at here.

---
 hw/cpu/icc_bus.c                | 11 ++++++++++
 hw/i386/kvm/apic.c              |  8 +++++++
 hw/intc/apic.c                  | 10 +++++++++
 hw/intc/apic_common.c           | 21 ++++++++++++-------
 include/hw/cpu/icc_bus.h        |  1 +
 include/hw/i386/apic_internal.h |  1 +
 target-i386/cpu.c               | 46 +++++++++++++++++++++++++++++++++++++++++
 7 files changed, 90 insertions(+), 8 deletions(-)
hw/cpu/icc_bus.c +11 -0
index f455a20..3e7f62f 100644
@@ -44,11 +44,22 @@ static void icc_device_realize(DeviceState *dev, Error **errp)
 
 }
 
+static void icc_device_unrealize(DeviceState *dev, Error **errp)
+{
+    ICCDeviceClass *idc = ICC_DEVICE_GET_CLASS(dev);
+
+    /* convert to QOM */
+    if (idc->unrealize) {
+        idc->unrealize(dev, errp);
+    }
+}
+
 static void icc_device_class_init(ObjectClass *oc, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(oc);
 
     dc->realize = icc_device_realize;
+    dc->unrealize = icc_device_unrealize;
     dc->bus_type = TYPE_ICC_BUS;
 }
 
hw/i386/kvm/apic.c +8 -0
index 5b47056..781c296 100644
@@ -189,11 +189,19 @@ static void kvm_apic_realize(DeviceState *dev, Error **errp)
     }
 }
 
+static void kvm_apic_unrealize(DeviceState *dev, Error **errp)
+{
+    APICCommonState *s = APIC_COMMON(dev);
+
+    object_unparent(OBJECT(&s->io_memory));
+}
+
 static void kvm_apic_class_init(ObjectClass *klass, void *data)
 {
     APICCommonClass *k = APIC_COMMON_CLASS(klass);
 
     k->realize = kvm_apic_realize;
+    k->unrealize = kvm_apic_unrealize;
     k->reset = kvm_apic_reset;
     k->set_base = kvm_apic_set_base;
     k->set_tpr = kvm_apic_set_tpr;
hw/intc/apic.c +10 -0
index 0f97b47..2f3a7a3 100644
@@ -889,11 +889,21 @@ static void apic_realize(DeviceState *dev, Error **errp)
     msi_supported = true;
 }
 
+static void apic_unrealize(DeviceState *dev, Error **errp)
+{
+    APICCommonState *s = APIC_COMMON(dev);
+
+    object_unparent(OBJECT(&s->io_memory));
+    timer_free(s->timer);
+    local_apics[s->idx] = NULL;
+}
+
 static void apic_class_init(ObjectClass *klass, void *data)
 {
     APICCommonClass *k = APIC_COMMON_CLASS(klass);
 
     k->realize = apic_realize;
+    k->unrealize = apic_unrealize;
     k->set_base = apic_set_base;
     k->set_tpr = apic_set_tpr;
     k->get_tpr = apic_get_tpr;
hw/intc/apic_common.c +13 -8
index 719f74d..912ba64 100644
@@ -299,16 +299,8 @@ static void apic_common_realize(DeviceState *dev, Error **errp)
     APICCommonState *s = APIC_COMMON(dev);
     APICCommonClass *info;
     static DeviceState *vapic;
-    static int apic_no;
     static bool mmio_registered;
 
-    if (apic_no >= MAX_APICS) {
-        error_setg(errp, "%s initialization failed.",
-                   object_get_typename(OBJECT(dev)));
-        return;
-    }
-    s->idx = apic_no++;
-
     info = APIC_COMMON_GET_CLASS(s);
     info->realize(dev, errp);
     if (!mmio_registered) {
@@ -329,6 +321,18 @@ static void apic_common_realize(DeviceState *dev, Error **errp)
 
 }
 
+static void apic_common_unrealize(DeviceState *dev, Error **errp)
+{
+    APICCommonState *s = APIC_COMMON(dev);
+    APICCommonClass *info = APIC_COMMON_GET_CLASS(s);
+
+    info->unrealize(dev, errp);
+
+    if (apic_report_tpr_access && info->enable_tpr_reporting) {
+        info->enable_tpr_reporting(s, false);
+    }
+}
+
 static int apic_pre_load(void *opaque)
 {
     APICCommonState *s = APIC_COMMON(opaque);
@@ -437,6 +441,7 @@ static void apic_common_class_init(ObjectClass *klass, void *data)
     dc->reset = apic_reset_common;
     dc->props = apic_properties_common;
     idc->realize = apic_common_realize;
+    idc->unrealize = apic_common_unrealize;
     /*
      * Reason: APIC and CPU need to be wired up by
      * x86_cpu_apic_create()
include/hw/cpu/icc_bus.h +1 -0
index 7e53afb..0b2ddd5 100644
@@ -69,6 +69,7 @@ typedef struct ICCDeviceClass {
     /*< public >*/
 
     DeviceRealize realize;
+    DeviceUnrealize unrealize;
 } ICCDeviceClass;
 
 #define TYPE_ICC_DEVICE "icc-device"
include/hw/i386/apic_internal.h +1 -0
index 00d29e2..b46a3af 100644
@@ -82,6 +82,7 @@ typedef struct APICCommonClass
     ICCDeviceClass parent_class;
 
     DeviceRealize realize;
+    DeviceUnrealize unrealize;
     void (*set_base)(APICCommonState *s, uint64_t val);
     void (*set_tpr)(APICCommonState *s, uint8_t val);
     uint8_t (*get_tpr)(APICCommonState *s);
target-i386/cpu.c +46 -0

74 lines not on this page · raw message

Signature · 1 line
1.9.3
author
#2 lore raw

PATCH v2 2/11 acpi/cpu: add cpu hot unplug request callback function

In reply to Zhu Guihua

From: Gu Zheng <[email protected]>

---
 hw/acpi/cpu_hotplug.c         | 38 +++++++++++++++++++++++++++++++++-----
 include/hw/acpi/cpu_hotplug.h |  4 ++++
 2 files changed, 37 insertions(+), 5 deletions(-)
hw/acpi/cpu_hotplug.c +33 -5
index 4047294..f8a10d2 100644
@@ -12,6 +12,11 @@
 #include "hw/hw.h"
 #include "hw/acpi/cpu_hotplug.h"
 
+typedef enum STS_OPT {
+    SET,
+    CLEAR,
+} STS_OPT;
+
 static uint64_t cpu_status_read(void *opaque, hwaddr addr, unsigned int size)
 {
     AcpiCpuHotplug *cpus = opaque;
@@ -36,8 +41,8 @@ static const MemoryRegionOps AcpiCpuHotplug_ops = {
     },
 };
 
-static void acpi_set_cpu_present_bit(AcpiCpuHotplug *g, CPUState *cpu,
-                                     Error **errp)
+static void acpi_update_cpu_present_bit(AcpiCpuHotplug *g, CPUState *cpu,
+                                        STS_OPT opt, Error **errp)
 {
     CPUClass *k = CPU_GET_CLASS(cpu);
     int64_t cpu_id;
@@ -48,13 +53,23 @@ static void acpi_set_cpu_present_bit(AcpiCpuHotplug *g, CPUState *cpu,
         return;
     }
 
-    g->sts[cpu_id / 8] |= (1 << (cpu_id % 8));
+    switch (opt) {
+    case SET:
+        g->sts[cpu_id / 8] |= (1 << (cpu_id % 8));
+        break;
+    case CLEAR:
+        g->sts[cpu_id / 8] &= ~(1 << (cpu_id % 8));
+        break;
+    default:
+        g_assert(0);
+        break;
+    }
 }
 
 void acpi_cpu_plug_cb(ACPIREGS *ar, qemu_irq irq,
                       AcpiCpuHotplug *g, DeviceState *dev, Error **errp)
 {
-    acpi_set_cpu_present_bit(g, CPU(dev), errp);
+    acpi_update_cpu_present_bit(g, CPU(dev), SET, errp);
     if (*errp != NULL) {
         return;
     }
@@ -66,13 +81,26 @@ void acpi_cpu_plug_cb(ACPIREGS *ar, qemu_irq irq,
     }
 }
 
+void acpi_cpu_unplug_request_cb(ACPIREGS *ar, qemu_irq irq,
+                                AcpiCpuHotplug *g, DeviceState *dev,
+                                Error **errp)
+{
+    acpi_update_cpu_present_bit(g, CPU(dev), CLEAR, errp);
+    if (*errp != NULL) {
+        return;
+    }
+
+    ar->gpe.sts[0] |= ACPI_CPU_HOTPLUG_STATUS;
+    acpi_update_sci(ar, irq);
+}
+
 void acpi_cpu_hotplug_init(MemoryRegion *parent, Object *owner,
                            AcpiCpuHotplug *gpe_cpu, uint16_t base)
 {
     CPUState *cpu;
 
     CPU_FOREACH(cpu) {
-        acpi_set_cpu_present_bit(gpe_cpu, cpu, &error_abort);
+        acpi_update_cpu_present_bit(gpe_cpu, cpu, SET, &error_abort);
     }
     memory_region_init_io(&gpe_cpu->io, owner, &AcpiCpuHotplug_ops,
                           gpe_cpu, "acpi-cpu-hotplug", ACPI_GPE_PROC_LEN);
include/hw/acpi/cpu_hotplug.h +4 -0
index f6d358d..8b15a3d 100644
@@ -23,6 +23,10 @@ typedef struct AcpiCpuHotplug {
 void acpi_cpu_plug_cb(ACPIREGS *ar, qemu_irq irq,
                       AcpiCpuHotplug *g, DeviceState *dev, Error **errp);
 
+void acpi_cpu_unplug_request_cb(ACPIREGS *ar, qemu_irq irq,
+                                AcpiCpuHotplug *g, DeviceState *dev,
+                                Error **errp);
+
 void acpi_cpu_hotplug_init(MemoryRegion *parent, Object *owner,
                            AcpiCpuHotplug *gpe_cpu, uint16_t base);
 #endif
Signature · 1 line
1.9.3
author
#3 lore raw

PATCH v2 3/11 acpi/piix4: add cpu hot unplug request callback support

In reply to Zhu Guihua
---
 hw/acpi/piix4.c | 2 ++
 1 file changed, 2 insertions(+)
hw/acpi/piix4.c +2 -0
index 4407388..9b75780 100644
@@ -364,6 +364,8 @@ static void piix4_device_unplug_request_cb(HotplugHandler *hotplug_dev,
     if (object_dynamic_cast(OBJECT(dev), TYPE_PCI_DEVICE)) {
         acpi_pcihp_device_unplug_cb(&s->ar, s->irq, &s->acpi_pci_hotplug, dev,
                                     errp);
+    } else if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
+        acpi_cpu_unplug_request_cb(&s->ar, s->irq, &s->gpe_cpu, dev, errp);
     } else {
         error_setg(errp, "acpi: device unplug request for not supported device"
                    " type: %s", object_get_typename(OBJECT(dev)));
Signature · 1 line
1.9.3
author
#4 lore raw

PATCH v2 4/11 acpi/ich9: add cpu hot unplug request callback support

In reply to Zhu Guihua
---
 hw/acpi/ich9.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
hw/acpi/ich9.c +7 -2
index c48d176..b795f8f 100644
@@ -304,8 +304,13 @@ void ich9_pm_device_plug_cb(ICH9LPCPMRegs *pm, DeviceState *dev, Error **errp)
 void ich9_pm_device_unplug_request_cb(ICH9LPCPMRegs *pm, DeviceState *dev,
                                       Error **errp)
 {
-    error_setg(errp, "acpi: device unplug request for not supported device"
-               " type: %s", object_get_typename(OBJECT(dev)));
+    if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
+        acpi_cpu_unplug_request_cb(&pm->acpi_regs, pm->irq,
+                                   &pm->gpe_cpu, dev, errp);
+    } else {
+        error_setg(errp, "acpi: device unplug request for not supported device"
+                   " type: %s", object_get_typename(OBJECT(dev)));
+    }
 }
 
 void ich9_pm_device_unplug_cb(ICH9LPCPMRegs *pm, DeviceState *dev,
Signature · 1 line
1.9.3
author
#5 lore raw

PATCH v2 5/11 pc: add cpu hot unplug request callback support

In reply to Zhu Guihua

Add cpu hot unplug request callback support, and a pre-check about the active cpus is added to avoid removing the last cpu.

---
 hw/i386/pc.c | 36 ++++++++++++++++++++++++++++++++++--
 1 file changed, 34 insertions(+), 2 deletions(-)
hw/i386/pc.c +34 -2
index 9bdec16..dd6e8da 100644
@@ -1694,6 +1694,34 @@ out:
     error_propagate(errp, local_err);
 }
 
+static void pc_cpu_unplug_request(HotplugHandler *hotplug_dev,
+                                  DeviceState *dev, Error **errp)
+{
+    HotplugHandlerClass *hhc;
+    Error *local_err = NULL;
+    PCMachineState *pcms;
+
+    if (smp_cpus == 1) {
+        error_setg(&local_err,
+                   "This is the last cpu, should not be removed!");
+        goto out;
+    }
+
+    pcms = PC_MACHINE(hotplug_dev);
+    if (!pcms->acpi_dev) {
+        if (dev->hotplugged) {
+            error_setg(&local_err,
+                       "cpu hot unplug is not enabled: missing acpi device");
+        }
+        goto out;
+    }
+
+    hhc = HOTPLUG_HANDLER_GET_CLASS(pcms->acpi_dev);
+    hhc->unplug_request(HOTPLUG_HANDLER(pcms->acpi_dev), dev, &local_err);
+out:
+    error_propagate(errp, local_err);
+}
+
 static void pc_machine_device_plug_cb(HotplugHandler *hotplug_dev,
                                       DeviceState *dev, Error **errp)
 {
@@ -1707,8 +1735,12 @@ static void pc_machine_device_plug_cb(HotplugHandler *hotplug_dev,
 static void pc_machine_device_unplug_request_cb(HotplugHandler *hotplug_dev,
                                                 DeviceState *dev, Error **errp)
 {
-    error_setg(errp, "acpi: device unplug request for not supported device"
-               " type: %s", object_get_typename(OBJECT(dev)));
+    if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
+        pc_cpu_unplug_request(hotplug_dev, dev, errp);
+    } else {
+        error_setg(errp, "acpi: device unplug request for not supported device"
+                   " type: %s", object_get_typename(OBJECT(dev)));
+    }
 }
 
 static void pc_machine_device_unplug_cb(HotplugHandler *hotplug_dev,
Signature · 1 line
1.9.3
author
#6 lore raw

PATCH v2 6/11 acpi/cpu: add cpu hot unplug callback function

In reply to Zhu Guihua
---
 cpus.c                        | 7 +++++++
 hw/acpi/cpu_hotplug.c         | 8 ++++++++
 include/hw/acpi/cpu_hotplug.h | 3 +++
 include/qom/cpu.h             | 9 +++++++++
 4 files changed, 27 insertions(+)
cpus.c +7 -0
index d5e35c0..1c25054 100644
@@ -1205,6 +1205,13 @@ void resume_all_vcpus(void)
     }
 }
 
+void cpu_remove(CPUState *cpu)
+{
+    cpu->stop = true;
+    cpu->exit = true;
+    qemu_cpu_kick(cpu);
+}
+
 /* For temporary buffers for forming a name */
 #define VCPU_THREAD_NAME_SIZE 16
 
hw/acpi/cpu_hotplug.c +8 -0
index f8a10d2..36feb6a 100644
@@ -94,6 +94,14 @@ void acpi_cpu_unplug_request_cb(ACPIREGS *ar, qemu_irq irq,
     acpi_update_sci(ar, irq);
 }
 
+void acpi_cpu_unplug_cb(ACPIREGS *ar, qemu_irq irq, AcpiCpuHotplug *g,
+                        DeviceState *dev, Error **errp)
+{
+    CPUState *cpu = CPU(dev);
+
+    cpu_remove(cpu);
+}
+
 void acpi_cpu_hotplug_init(MemoryRegion *parent, Object *owner,
                            AcpiCpuHotplug *gpe_cpu, uint16_t base)
 {
include/hw/acpi/cpu_hotplug.h +3 -0
index 8b15a3d..0f84adb 100644
@@ -27,6 +27,9 @@ void acpi_cpu_unplug_request_cb(ACPIREGS *ar, qemu_irq irq,
                                 AcpiCpuHotplug *g, DeviceState *dev,
                                 Error **errp);
 
+void acpi_cpu_unplug_cb(ACPIREGS *ar, qemu_irq irq,
+                        AcpiCpuHotplug *g, DeviceState *dev, Error **errp);
+
 void acpi_cpu_hotplug_init(MemoryRegion *parent, Object *owner,
                            AcpiCpuHotplug *gpe_cpu, uint16_t base);
 #endif
include/qom/cpu.h +9 -0
index f663199..0592b4d 100644
@@ -249,6 +249,7 @@ struct CPUState {
     bool created;
     bool stop;
     bool stopped;
+    bool exit;
     volatile sig_atomic_t exit_request;
     uint32_t interrupt_request;
     int singlestep_enabled;
@@ -613,6 +614,14 @@ void cpu_exit(CPUState *cpu);
 void cpu_resume(CPUState *cpu);
 
 /**
+ * cpu_remove:
+ * @cpu: The vCPU to remove.
+ *
+ * Requests the CPU @cpu to be removed.
+ */
+void cpu_remove(CPUState *cpu);
+
+/**
  * qemu_init_vcpu:
  * @cpu: The vCPU to initialize.
  *
Signature · 1 line
1.9.3
author
#7 lore raw

PATCH v2 7/11 acpi/piix4: add cpu hot unplug callback support

In reply to Zhu Guihua

From: Gu Zheng <[email protected]>

---
 hw/acpi/piix4.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
hw/acpi/piix4.c +8 -2
index 9b75780..2a923fc 100644
@@ -375,8 +375,14 @@ static void piix4_device_unplug_request_cb(HotplugHandler *hotplug_dev,
 static void piix4_device_unplug_cb(HotplugHandler *hotplug_dev,
                                    DeviceState *dev, Error **errp)
 {
-    error_setg(errp, "acpi: device unplug for not supported device"
-               " type: %s", object_get_typename(OBJECT(dev)));
+    PIIX4PMState *s = PIIX4_PM(hotplug_dev);
+
+    if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
+        acpi_cpu_unplug_cb(&s->ar, s->irq, &s->gpe_cpu, dev, errp);
+    } else {
+        error_setg(errp, "acpi: device unplug for not supported device"
+                   " type: %s", object_get_typename(OBJECT(dev)));
+    }
 }
 
 static void piix4_update_bus_hotplug(PCIBus *pci_bus, void *opaque)
Signature · 1 line
1.9.3
author
#8 lore raw

PATCH v2 8/11 acpi/ich9: add cpu hot unplug support

In reply to Zhu Guihua

From: Gu Zheng <[email protected]>

---
 hw/acpi/ich9.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
hw/acpi/ich9.c +6 -2
index b795f8f..a892e5d 100644
@@ -316,8 +316,12 @@ void ich9_pm_device_unplug_request_cb(ICH9LPCPMRegs *pm, DeviceState *dev,
 void ich9_pm_device_unplug_cb(ICH9LPCPMRegs *pm, DeviceState *dev,
                               Error **errp)
 {
-    error_setg(errp, "acpi: device unplug for not supported device"
-               " type: %s", object_get_typename(OBJECT(dev)));
+    if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
+        acpi_cpu_unplug_cb(&pm->acpi_regs, pm->irq, &pm->gpe_cpu, dev, errp);
+    } else {
+        error_setg(errp, "acpi: device unplug for not supported device"
+                   " type: %s", object_get_typename(OBJECT(dev)));
+    }
 }
 
 void ich9_pm_ospm_status(AcpiDeviceIf *adev, ACPIOSTInfoList ***list)
Signature · 1 line
1.9.3
author
#9 lore raw

PATCH v2 9/11 pc: add cpu hot unplug callback support

In reply to Zhu Guihua

From: Gu Zheng <[email protected]>

---
 hw/i386/pc.c | 26 ++++++++++++++++++++++++--
 1 file changed, 24 insertions(+), 2 deletions(-)
hw/i386/pc.c +24 -2
index dd6e8da..b7d5712 100644
@@ -1722,6 +1722,24 @@ out:
     error_propagate(errp, local_err);
 }
 
+static void pc_cpu_unplug(HotplugHandler *hotplug_dev,
+                          DeviceState *dev, Error **errp)
+{
+    HotplugHandlerClass *hhc;
+    Error *local_err = NULL;
+    PCMachineState *pcms = PC_MACHINE(hotplug_dev);
+
+    hhc = HOTPLUG_HANDLER_GET_CLASS(pcms->acpi_dev);
+    hhc->unplug(HOTPLUG_HANDLER(pcms->acpi_dev), dev, &local_err);
+    if (local_err) {
+        error_propagate(errp, local_err);
+        return;
+    }
+
+    /* decrement the number of CPUs */
+    rtc_set_memory(pcms->rtc, 0x5f, rtc_get_memory(pcms->rtc, 0x5f) - 1);
+}
+
 static void pc_machine_device_plug_cb(HotplugHandler *hotplug_dev,
                                       DeviceState *dev, Error **errp)
 {
@@ -1746,8 +1764,12 @@ static void pc_machine_device_unplug_request_cb(HotplugHandler *hotplug_dev,
 static void pc_machine_device_unplug_cb(HotplugHandler *hotplug_dev,
                                         DeviceState *dev, Error **errp)
 {
-    error_setg(errp, "acpi: device unplug for not supported device"
-               " type: %s", object_get_typename(OBJECT(dev)));
+    if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
+        pc_cpu_unplug(hotplug_dev, dev, errp);
+    } else {
+        error_setg(errp, "acpi: device unplug for not supported device"
+                   " type: %s", object_get_typename(OBJECT(dev)));
+    }
 }
 
 static HotplugHandler *pc_get_hotpug_handler(MachineState *machine,
Signature · 1 line
1.9.3
author
#10 lore raw

PATCH v2 10/11 cpu hotplug: implement function cpu_status_write() for vcpu ejection

In reply to Zhu Guihua

From: Chen Fan <[email protected]>

When OS ejected a vcpu (like: echo 1 > /sys/bus/acpi/devices/LNXCPUXX/eject), it would call acpi EJ0 method, the firmware need to write the new cpumap, QEMU would know which vcpu need to be ejected.

---
 hw/acpi/cpu_hotplug.c             | 42 ++++++++++++++++++++++++++++++++++++++-
 hw/core/qdev.c                    |  2 +-
 hw/i386/acpi-dsdt-cpu-hotplug.dsl |  6 +++++-
 include/hw/acpi/cpu_hotplug.h     |  1 +
 include/hw/qdev-core.h            |  1 +
 5 files changed, 49 insertions(+), 3 deletions(-)
hw/acpi/cpu_hotplug.c +41 -1
index 36feb6a..2a56df0 100644
@@ -17,6 +17,26 @@ typedef enum STS_OPT {
     CLEAR,
 } STS_OPT;
 
+static void acpi_eject_vcpu(AcpiCpuHotplug *cpus_status, int64_t cpu_id)
+{
+    CPUState *cpu;
+
+    CPU_FOREACH(cpu) {
+        CPUClass *cc = CPU_GET_CLASS(cpu);
+        int64_t id = cc->get_arch_id(cpu);
+        HotplugHandler *hotplug_ctrl;
+
+        if (cpu_id == id) {
+            cpus_status->old_sts[cpu_id / 8] &= ~(1 << (cpu_id % 8));
+            cpus_status->sts[cpu_id / 8] &= ~(1 << (cpu_id % 8));
+
+            hotplug_ctrl = qdev_get_hotplug_handler(DEVICE(cpu));
+            hotplug_handler_unplug(hotplug_ctrl, DEVICE(cpu), NULL);
+            break;
+        }
+    }
+}
+
 static uint64_t cpu_status_read(void *opaque, hwaddr addr, unsigned int size)
 {
     AcpiCpuHotplug *cpus = opaque;
@@ -28,7 +48,26 @@ static uint64_t cpu_status_read(void *opaque, hwaddr addr, unsigned int size)
 static void cpu_status_write(void *opaque, hwaddr addr, uint64_t data,
                              unsigned int size)
 {
-    /* TODO: implement VCPU removal on guest signal that CPU can be removed */
+    AcpiCpuHotplug *cpus = opaque;
+    uint8_t val;
+    int i;
+    int64_t cpu_id = -1;
+
+    val = cpus->old_sts[addr] ^ data;
+
+    if (val == 0) {
+        return;
+    }
+
+    for (i = 0; i < 8; i++) {
+        if (val & 1 << i) {
+            cpu_id = 8 * addr + i;
+        }
+    }
+
+    if (cpu_id != -1) {
+        acpi_eject_vcpu(cpus, cpu_id);
+    }
 }
 
 static const MemoryRegionOps AcpiCpuHotplug_ops = {
@@ -56,6 +95,7 @@ static void acpi_update_cpu_present_bit(AcpiCpuHotplug *g, CPUState *cpu,
     switch (opt) {
     case SET:
         g->sts[cpu_id / 8] |= (1 << (cpu_id % 8));
+        g->old_sts[cpu_id / 8] |= (1 << (cpu_id % 8));
         break;
     case CLEAR:
         g->sts[cpu_id / 8] &= ~(1 << (cpu_id % 8));
hw/core/qdev.c +1 -1
index 901f289..9f08fe6 100644
@@ -223,7 +223,7 @@ void qdev_set_legacy_instance_id(DeviceState *dev, int alias_id,
     dev->alias_required_for_version = required_for_version;
 }
 
-static HotplugHandler *qdev_get_hotplug_handler(DeviceState *dev)
+HotplugHandler *qdev_get_hotplug_handler(DeviceState *dev)
 {
     HotplugHandler *hotplug_ctrl = NULL;
 
hw/i386/acpi-dsdt-cpu-hotplug.dsl +5 -1
index 34aab5a..9485f12 100644
@@ -50,7 +50,11 @@ Scope(\_SB) {
     }
     Method(CPEJ, 2, NotSerialized) {
         // _EJ0 method - eject callback
-        Sleep(200)
+        Store(Zero, Index(CPON, ToInteger(Arg0)))
+        Store(One, Local0)
+        ShiftLeft(Local0, Arg0, Local0)
+        Not(Local0, Local0)
+        And(PRS, Local0, PRS)
     }
 
 #define CPU_STATUS_LEN ACPI_GPE_PROC_LEN
include/hw/acpi/cpu_hotplug.h +1 -0
index 0f84adb..abbb29e 100644
@@ -18,6 +18,7 @@
 typedef struct AcpiCpuHotplug {
     MemoryRegion io;
     uint8_t sts[ACPI_GPE_PROC_LEN];
+    uint8_t old_sts[ACPI_GPE_PROC_LEN];
 } AcpiCpuHotplug;
 
 void acpi_cpu_plug_cb(ACPIREGS *ar, qemu_irq irq,
include/hw/qdev-core.h +1 -0
index 589bbe7..5b28a4f 100644
@@ -263,6 +263,7 @@ void qdev_set_legacy_instance_id(DeviceState *dev, int alias_id,
 void qdev_unplug(DeviceState *dev, Error **errp);
 void qdev_simple_device_unplug_cb(HotplugHandler *hotplug_dev,
                                   DeviceState *dev, Error **errp);
+HotplugHandler *qdev_get_hotplug_handler(DeviceState *dev);
 void qdev_machine_creation_done(void);
 bool qdev_machine_modified(void);
 
Signature · 1 line
1.9.3
author
#11 lore raw

PATCH v2 11/11 cpus: reclaim allocated vCPU objects

In reply to Zhu Guihua

From: Gu Zheng <[email protected]>

After ACPI get a signal to eject a vCPU, the vCPU must be removed from CPU list,before the vCPU really removed, then release the all related vCPU objects.

In order to deal well with the kvm vcpus (which can not be removed without any protection), we do not close KVM vcpu fd, just record and mark it as stopped into a list, so that we can reuse it for the appending cpu hot-add request if possible. It is also the approach that kvm guys suggested:
https://www.mail-archive.com/[email protected]/msg102839.html

---
 cpus.c               | 37 ++++++++++++++++++++++++++++++++++
 include/sysemu/kvm.h |  1 +
 kvm-all.c            | 57 +++++++++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 94 insertions(+), 1 deletion(-)
cpus.c +37 -0
index 1c25054..b5b0c91 100644
@@ -871,6 +871,24 @@ void async_run_on_cpu(CPUState *cpu, void (*func)(void *data), void *data)
     qemu_cpu_kick(cpu);
 }
 
+static void qemu_kvm_destroy_vcpu(CPUState *cpu)
+{
+    CPU_REMOVE(cpu);
+
+    if (kvm_destroy_vcpu(cpu) < 0) {
+        error_report("kvm_destroy_vcpu failed.\n");
+        exit(EXIT_FAILURE);
+    }
+
+    object_unparent(OBJECT(cpu));
+}
+
+static void qemu_tcg_destroy_vcpu(CPUState *cpu)
+{
+    CPU_REMOVE(cpu);
+    object_unparent(OBJECT(cpu));
+}
+
 static void flush_queued_work(CPUState *cpu)
 {
     struct qemu_work_item *wi;
@@ -963,6 +981,11 @@ static void *qemu_kvm_cpu_thread_fn(void *arg)
             }
         }
         qemu_kvm_wait_io_event(cpu);
+        if (cpu->exit && !cpu_can_run(cpu)) {
+            qemu_kvm_destroy_vcpu(cpu);
+            qemu_mutex_unlock(&qemu_global_mutex);
+            return NULL;
+        }
     }
 
     return NULL;
@@ -1016,6 +1039,7 @@ static void tcg_exec_all(void);
 static void *qemu_tcg_cpu_thread_fn(void *arg)
 {
     CPUState *cpu = arg;
+    CPUState *remove_cpu = NULL;
 
     qemu_tcg_init_cpu_signals();
     qemu_thread_get_self(cpu->thread);
@@ -1049,6 +1073,16 @@ static void *qemu_tcg_cpu_thread_fn(void *arg)
             }
         }
         qemu_tcg_wait_io_event();
+        CPU_FOREACH(cpu) {
+            if (cpu->exit && !cpu_can_run(cpu)) {
+                remove_cpu = cpu;
+                break;
+            }
+        }
+        if (remove_cpu) {
+            qemu_tcg_destroy_vcpu(remove_cpu);
+            remove_cpu = NULL;
+        }
     }
 
     return NULL;
@@ -1406,6 +1440,9 @@ static void tcg_exec_all(void)
                 break;
             }
         } else if (cpu->stop || cpu->stopped) {
+            if (cpu->exit) {
+                next_cpu = CPU_NEXT(cpu);
+            }
             break;
         }
     }
include/sysemu/kvm.h +1 -0
index 30cb84d..560caef 100644
@@ -188,6 +188,7 @@ int kvm_has_intx_set_mask(void);
 
 int kvm_init_vcpu(CPUState *cpu);
 int kvm_cpu_exec(CPUState *cpu);
+int kvm_destroy_vcpu(CPUState *cpu);
 
 #ifdef NEED_CPU_H
 
kvm-all.c +56 -1

87 lines not on this page · raw message

Signature · 1 line
1.9.3

A message that is not in the archive (sent off-list or not yet received): [email protected]

#13 lore raw

Hello,

I'm currently testing the new cpu unplug features, Works fine here with debian guests and kernel 3.14.

But I have notice some small potential bugs, but I'm not sure I'm doing it right.

1)first, to unplug cpu, we need an id for cpu

The problem is that the current qemu command line
-smp 1,sockets=2,cores=1,maxcpus=2

for example, will create 1 cpu on apic-id 0 without any id, so we can't unplug it.

So, I have tried with

-smp 1,sockets=2,cores=1,maxcpus=2 -device kvm64-x86_64-cpu,apic-id=0,id=cpu0

But this give me an error:
"-device kvm64-x86_64-cpu,apic-id=0,id=cpu0: CPU with APIC ID 0 exists"

(also try to set -smp 0, but it's not working).

2) second problem, if I start with
-smp 1,sockets=2,cores=1,maxcpus=2

then hmp:
device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1

then hmp : device_del cpu1

Got an error:"
This is the last cpu, should not be removed!"

This is coming from
[PATCH 06/12] pc: add cpu hot unplug request callback support
+ if (smp_cpus == 1) {
+ error_setg(&local_err,
+ "This is the last cpu, should not be removed!");
+ goto out;
+ }

So, the only way unplug is working for me, is to start with -smp 2 minimum -smp 2,sockets=2,cores=1,maxcpus=4

Then I can hotplug|unplug cpuid >= 2

Regards,

Alexandre Derumier

----- Mail original -----
De: "Zhu Guihua" <[email protected]>
À: "qemu-devel" <[email protected]>
Cc: "Zhu Guihua" <[email protected]>, [email protected], "guz fnst" <[email protected]>, "isimatu yasuaki" <[email protected]>, "Anshul Makkar" <[email protected]>, "chen fan fnst" <[email protected]>, "Igor Mammedov" <[email protected]>, "afaerber" <[email protected]> Envoyé: Mercredi 14 Janvier 2015 08:44:53
Objet: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

This series is based on chen fan's previous i386 cpu hot remove patchset:
https://lists.nongnu.org/archive/html/qemu-devel/2013-12/msg04266.html

Via implementing ACPI standard methods _EJ0 in ACPI table, after Guest OS remove one vCPU online, the fireware will store removed bitmap to QEMU, then QEMU could know to notify the assigned vCPU of exiting. Meanwhile, intruduce the QOM command 'device_del' to remove vCPU from QEMU itself.

The whole work is based on the new hot plug/unplug framework, ,the unplug request callback does the pre-check and send the request, unplug callback does the removal handling.

This series depends on tangchen's common hot plug/unplug enhance patchset. [RESEND PATCH v1 0/5] Common unplug and unplug request cb for memory and CPU hot-unplug
https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg00429.html

The is the second half of the previous series:
[RFC V2 00/10] cpu: add device_add foo-x86_64-cpu and i386 cpu hot remove support
https://lists.nongnu.org/archive/html/qemu-devel/2014-08/msg04779.html

If you want to test the series, you need to apply the 'device_add foo-x86_64-cpu' patchset first:
[PATCH v3 0/7] cpu: add device_add foo-x86_64-cpu support
https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg01552.html

---
Changelog since v1:
-rebase on the latest version.
-delete patch i386/cpu: add instance finalize callback, and put it into patchset [PATCH v3 0/6] cpu: add device_add foo-x86_64-cpu support.

Changelog since RFC:
-splited the i386 cpu hot remove into single thread. -replaced apic_no with apic_id, so does the related stuff to make it work with arbitrary CPU hotadd.
-add the icc_device_unrealize callback to handle apic unrealize. -rework on the new hot plug/unplug platform.
---

Chen Fan (2):
x86: add x86_cpu_unrealizefn() for cpu apic remove
cpu hotplug: implement function cpu_status_write() for vcpu ejection

Gu Zheng (5):
acpi/cpu: add cpu hot unplug request callback function acpi/piix4: add cpu hot unplug callback support acpi/ich9: add cpu hot unplug support
pc: add cpu hot unplug callback support
cpus: reclaim allocated vCPU objects

Zhu Guihua (4):
acpi/piix4: add cpu hot unplug request callback support acpi/ich9: add cpu hot unplug request callback support
pc: add cpu hot unplug request callback support acpi/cpu: add cpu hot unplug callback function

cpus.c | 44 ++++++++++++++++++++
hw/acpi/cpu_hotplug.c | 88 ++++++++++++++++++++++++++++++++++++--- hw/acpi/ich9.c | 17 ++++++--
hw/acpi/piix4.c | 12 +++++-
hw/core/qdev.c | 2 +-
hw/cpu/icc_bus.c | 11 +++++ hw/i386/acpi-dsdt-cpu-hotplug.dsl | 6 ++- hw/i386/kvm/apic.c | 8 ++++
hw/i386/pc.c | 62 +++++++++++++++++++++++++-- hw/intc/apic.c | 10 +++++
hw/intc/apic_common.c | 21 ++++++---- include/hw/acpi/cpu_hotplug.h | 8 ++++ include/hw/cpu/icc_bus.h | 1 + include/hw/i386/apic_internal.h | 1 + include/hw/qdev-core.h | 1 +
include/qom/cpu.h | 9 ++++
include/sysemu/kvm.h | 1 +
kvm-all.c | 57 ++++++++++++++++++++++++- target-i386/cpu.c | 46 ++++++++++++++++++++

19 files changed, 378 insertions(+), 27 deletions(-) 
Signature · 1 line
1.9.3 
author
#14 lore raw
Quoted 5 lines · Alexandre DERUMIER

Hello,

I'm currently testing the new cpu unplug features, Works fine here with debian guests and kernel 3.14.

Thanks for your test.

Quoted 4 lines

But I have notice some small potential bugs, but I'm not sure I'm doing it right.

1)first, to unplug cpu, we need an id for cpu

Yes, if you want to unplug cpu, you must have an id for cpu.

Quoted 6 lines

The problem is that the current qemu command line
-smp 1,sockets=2,cores=1,maxcpus=2

for example, will create 1 cpu on apic-id 0 without any id, so we can't unplug it.

Quoted 7 lines

So, I have tried with

-smp 1,sockets=2,cores=1,maxcpus=2 -device kvm64-x86_64-cpu,apic-id=0,id=cpu0

But this give me an error:
"-device kvm64-x86_64-cpu,apic-id=0,id=cpu0: CPU with APIC ID 0 exists"

APIC ID 0 was used by the cpu of '-smp 1'.
So you should use apic-id=1

Quoted 7 lines

(also try to set -smp 0, but it's not working).

2) second problem, if I start with
-smp 1,sockets=2,cores=1,maxcpus=2

Quoted 9 lines

then hmp:
device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1

then hmp : device_del cpu1

Got an error:"
This is the last cpu, should not be removed!"

Oh, it's our problem, thanks for your pointing out. I will fix it in next version.

Regards,
Zhu

Quoted 21 lines

This is coming from
[PATCH 06/12] pc: add cpu hot unplug request callback support

(quoted diff) +5 -0
+    if (smp_cpus == 1) {
+        error_setg(&local_err,
+                   "This is the last cpu, should not be removed!");
+        goto out;
+    }

So, the only way unplug is working for me, is to start with -smp 2 minimum -smp 2,sockets=2,cores=1,maxcpus=4

Then I can hotplug|unplug cpuid >= 2

Regards,

Alexandre Derumier

[...]

A message that is not in the archive (sent off-list or not yet received): [email protected]

#16 lore raw

Thanks for your reply.

2 others things:

1)
on cpu unplug, I see that the cpu is correctly removed from my linux guest but not from qemu

starting with a guest with 3cpus:

guest: #ls -lah /sys/devices/system/ |grep cpu

drwxr-xr-x 6 root root    0 Jan 25 22:16 cpu0
drwxr-xr-x 6 root root    0 Jan 25 22:16 cpu1
drwxr-xr-x 6 root root    0 Jan 25 22:16 cpu2

hmp: # info cpus
* CPU #0: pc=0xffffffff81057022 (halted) thread_id=24972 CPU #1: pc=0xffffffff81057022 (halted) thread_id=24973 CPU #2: pc=0xffffffff81048bc1 (halted) thread_id=25102

then unplug cpu2
hmp : device_del cpu2

guest:

dmesg:

[  176.219754] Unregister pv shared memory for cpu 2
[  176.278881] smpboot: CPU 2 is now offline

#ls -lah /sys/devices/system/ |grep cpu
drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu0
drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu1

hmp: # info cpus
* CPU #0: pc=0xffffffff81057022 (halted) thread_id=24972 CPU #1: pc=0xffffffff81057022 (halted) thread_id=24973 CPU #2: pc=0xffffffff81048bc1 (halted) thread_id=25102

2)when numa is used, the hotplugged cpu is always on numa node 0
(cpu_add or device_add cpu)

starting a guest, with 2 sockets,1 cores

-smp 2,sockets=2,cores=1,maxcpus=2
-object memory-backend-ram,size=256M,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0 -object memory-backend-ram,size=256M,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1

hmp:

# info numa
2 nodes
node 0 cpus: 0
node 0 size: 256 MB
node 1 cpus: 1
node 1 size: 256 MB

ok

now

starting with same topology, but with 1cpu at start
-smp 2,sockets=2,cores=1,maxcpus=2
-object memory-backend-ram,size=256M,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0 -object memory-backend-ram,size=256M,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1

# info numa
2 nodes
node 0 cpus: 0
node 0 size: 256 MB
node 1 cpus: 
node 1 size: 256 MB

hotpluging a cpu

# device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1
# info numa
2 nodes
node 0 cpus: 0 1
node 0 size: 256 MB
node 1 cpus: 
node 1 size: 256 MB

cpu1 should be on node1, not node0.

Regards,

Alexandre

----- Mail original -----
De: "Zhu Guihua" <[email protected]>
À: "aderumier" <[email protected]>
Cc: "qemu-devel" <[email protected]>, [email protected], "guz fnst" <[email protected]>, "isimatu yasuaki" <[email protected]>, "Anshul Makkar" <[email protected]>, "chen fan fnst" <[email protected]>, "Igor Mammedov" <[email protected]>, "afaerber" <[email protected]> Envoyé: Lundi 26 Janvier 2015 03:01:48
Objet: Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

Quoted 5 lines · Alexandre DERUMIER

Hello,

I'm currently testing the new cpu unplug features, Works fine here with debian guests and kernel 3.14.

Thanks for your test.

Quoted 4 lines

But I have notice some small potential bugs, but I'm not sure I'm doing it right.

1)first, to unplug cpu, we need an id for cpu

Yes, if you want to unplug cpu, you must have an id for cpu.

Quoted 6 lines

The problem is that the current qemu command line
-smp 1,sockets=2,cores=1,maxcpus=2

for example, will create 1 cpu on apic-id 0 without any id, so we can't unplug it.

Quoted 7 lines

So, I have tried with

-smp 1,sockets=2,cores=1,maxcpus=2 -device kvm64-x86_64-cpu,apic-id=0,id=cpu0

But this give me an error:
"-device kvm64-x86_64-cpu,apic-id=0,id=cpu0: CPU with APIC ID 0 exists"

APIC ID 0 was used by the cpu of '-smp 1'.
So you should use apic-id=1

Quoted 7 lines

(also try to set -smp 0, but it's not working).

2) second problem, if I start with
-smp 1,sockets=2,cores=1,maxcpus=2

Quoted 9 lines

then hmp:
device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1

then hmp : device_del cpu1

Got an error:"
This is the last cpu, should not be removed!"

Oh, it's our problem, thanks for your pointing out. I will fix it in next version.

Regards,
Zhu

Quoted 21 lines

This is coming from
[PATCH 06/12] pc: add cpu hot unplug request callback support
+ if (smp_cpus == 1) {
+ error_setg(&local_err,
+ "This is the last cpu, should not be removed!");
+ goto out;
+ }

So, the only way unplug is working for me, is to start with -smp 2 minimum -smp 2,sockets=2,cores=1,maxcpus=4

Then I can hotplug|unplug cpuid >= 2

Regards,

Alexandre Derumier

[...]

#17 lore raw
Quoted 2 lines
Quoted 2 lines

2)when numa is used, the hotplugged cpu is always on numa node 0 (cpu_add or device_add cpu)

About this, it seem to be a display bug in "info numa", cpu is correctly assigned to numa node1 in the guest.

----- Mail original -----
De: "aderumier" <[email protected]>
À: "Zhu Guihua" <[email protected]>
Cc: "qemu-devel" <[email protected]>, [email protected], "guz fnst" <[email protected]>, "isimatu yasuaki" <[email protected]>, "Anshul Makkar" <[email protected]>, "chen fan fnst" <[email protected]>, "Igor Mammedov" <[email protected]>, "afaerber" <[email protected]> Envoyé: Lundi 26 Janvier 2015 04:19:52
Objet: Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

Thanks for your reply.

2 others things:

1)
on cpu unplug, I see that the cpu is correctly removed from my linux guest but not from qemu

starting with a guest with 3cpus:

guest: #ls -lah /sys/devices/system/ |grep cpu
drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu0
drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu1
drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu2

hmp: # info cpus
* CPU #0: pc=0xffffffff81057022 (halted) thread_id=24972 CPU #1: pc=0xffffffff81057022 (halted) thread_id=24973 CPU #2: pc=0xffffffff81048bc1 (halted) thread_id=25102

then unplug cpu2
hmp : device_del cpu2

guest:

dmesg:

[ 176.219754] Unregister pv shared memory for cpu 2 
[ 176.278881] smpboot: CPU 2 is now offline 

#ls -lah /sys/devices/system/ |grep cpu
drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu0
drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu1

hmp: # info cpus
* CPU #0: pc=0xffffffff81057022 (halted) thread_id=24972 CPU #1: pc=0xffffffff81057022 (halted) thread_id=24973 CPU #2: pc=0xffffffff81048bc1 (halted) thread_id=25102

2)when numa is used, the hotplugged cpu is always on numa node 0 (cpu_add or device_add cpu)

starting a guest, with 2 sockets,1 cores

-smp 2,sockets=2,cores=1,maxcpus=2
-object memory-backend-ram,size=256M,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0 -object memory-backend-ram,size=256M,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1

hmp:

# info numa 
2 nodes 
node 0 cpus: 0 
node 0 size: 256 MB 
node 1 cpus: 1 
node 1 size: 256 MB 

ok

now

starting with same topology, but with 1cpu at start
-smp 2,sockets=2,cores=1,maxcpus=2
-object memory-backend-ram,size=256M,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0 -object memory-backend-ram,size=256M,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1

# info numa 
2 nodes 
node 0 cpus: 0 
node 0 size: 256 MB 
node 1 cpus: 
node 1 size: 256 MB 

hotpluging a cpu

# device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1 
# info numa 
2 nodes 
node 0 cpus: 0 1 
node 0 size: 256 MB 
node 1 cpus: 
node 1 size: 256 MB 

cpu1 should be on node1, not node0.

Regards,

Alexandre

----- Mail original -----
De: "Zhu Guihua" <[email protected]>
À: "aderumier" <[email protected]>
Cc: "qemu-devel" <[email protected]>, [email protected], "guz fnst" <[email protected]>, "isimatu yasuaki" <[email protected]>, "Anshul Makkar" <[email protected]>, "chen fan fnst" <[email protected]>, "Igor Mammedov" <[email protected]>, "afaerber" <[email protected]> Envoyé: Lundi 26 Janvier 2015 03:01:48
Objet: Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

Quoted 5 lines · Alexandre DERUMIER

Hello,

I'm currently testing the new cpu unplug features, Works fine here with debian guests and kernel 3.14.

Thanks for your test.

Quoted 4 lines

But I have notice some small potential bugs, but I'm not sure I'm doing it right.

1)first, to unplug cpu, we need an id for cpu

Yes, if you want to unplug cpu, you must have an id for cpu.

Quoted 6 lines

The problem is that the current qemu command line
-smp 1,sockets=2,cores=1,maxcpus=2

for example, will create 1 cpu on apic-id 0 without any id, so we can't unplug it.

Quoted 7 lines

So, I have tried with

-smp 1,sockets=2,cores=1,maxcpus=2 -device kvm64-x86_64-cpu,apic-id=0,id=cpu0

But this give me an error:
"-device kvm64-x86_64-cpu,apic-id=0,id=cpu0: CPU with APIC ID 0 exists"

APIC ID 0 was used by the cpu of '-smp 1'.
So you should use apic-id=1

Quoted 7 lines

(also try to set -smp 0, but it's not working).

2) second problem, if I start with
-smp 1,sockets=2,cores=1,maxcpus=2

Quoted 9 lines

then hmp:
device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1

then hmp : device_del cpu1

Got an error:"
This is the last cpu, should not be removed!"

Oh, it's our problem, thanks for your pointing out. I will fix it in next version.

Regards,
Zhu

Quoted 21 lines

This is coming from
[PATCH 06/12] pc: add cpu hot unplug request callback support
+ if (smp_cpus == 1) {
+ error_setg(&local_err,
+ "This is the last cpu, should not be removed!");
+ goto out;
+ }

So, the only way unplug is working for me, is to start with -smp 2 minimum -smp 2,sockets=2,cores=1,maxcpus=4

Then I can hotplug|unplug cpuid >= 2

Regards,

Alexandre Derumier

[...]

author
#18 lore raw
Quoted 7 lines · Alexandre DERUMIER
Quoted 2 lines
Quoted 2 lines

2)when numa is used, the hotplugged cpu is always on numa node 0 (cpu_add or device_add cpu)

About this, it seem to be a display bug in "info numa", cpu is correctly assigned to numa node1 in the guest.

Now in the guest, VCPUs are assigned round-robin.
But in qemu, vcpus' property 'numa_node' was assigned the default value 0.

We have made a patchset to solve this successfully. But we plan to send the patchset to the community later.

Regards,
Zhu

Quoted 185 lines

----- Mail original -----
De: "aderumier" <[email protected]>
À: "Zhu Guihua" <[email protected]>
Cc: "qemu-devel" <[email protected]>, [email protected], "guz fnst" <[email protected]>, "isimatu yasuaki" <[email protected]>, "Anshul Makkar" <[email protected]>, "chen fan fnst" <[email protected]>, "Igor Mammedov" <[email protected]>, "afaerber" <[email protected]> Envoyé: Lundi 26 Janvier 2015 04:19:52
Objet: Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

Thanks for your reply.

2 others things:

1)
on cpu unplug, I see that the cpu is correctly removed from my linux guest but not from qemu

starting with a guest with 3cpus:

guest: #ls -lah /sys/devices/system/ |grep cpu
drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu0
drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu1
drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu2

hmp: # info cpus
* CPU #0: pc=0xffffffff81057022 (halted) thread_id=24972 CPU #1: pc=0xffffffff81057022 (halted) thread_id=24973 CPU #2: pc=0xffffffff81048bc1 (halted) thread_id=25102

then unplug cpu2
hmp : device_del cpu2

guest:

dmesg:

[ 176.219754] Unregister pv shared memory for cpu 2 
[ 176.278881] smpboot: CPU 2 is now offline 

#ls -lah /sys/devices/system/ |grep cpu
drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu0
drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu1

hmp: # info cpus
* CPU #0: pc=0xffffffff81057022 (halted) thread_id=24972 CPU #1: pc=0xffffffff81057022 (halted) thread_id=24973 CPU #2: pc=0xffffffff81048bc1 (halted) thread_id=25102

2)when numa is used, the hotplugged cpu is always on numa node 0 (cpu_add or device_add cpu)

starting a guest, with 2 sockets,1 cores

(quoted diff) +0 -3
-smp 2,sockets=2,cores=1,maxcpus=2 
-object memory-backend-ram,size=256M,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0 
-object memory-backend-ram,size=256M,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1 

hmp:

# info numa 
2 nodes 
node 0 cpus: 0 
node 0 size: 256 MB 
node 1 cpus: 1 
node 1 size: 256 MB 

ok

now

starting with same topology, but with 1cpu at start

(quoted diff) +0 -3
-smp 2,sockets=2,cores=1,maxcpus=2 
-object memory-backend-ram,size=256M,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0 
-object memory-backend-ram,size=256M,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1 
# info numa 
2 nodes 
node 0 cpus: 0 
node 0 size: 256 MB 
node 1 cpus: 
node 1 size: 256 MB 

hotpluging a cpu

# device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1 
# info numa 
2 nodes 
node 0 cpus: 0 1 
node 0 size: 256 MB 
node 1 cpus: 
node 1 size: 256 MB 

cpu1 should be on node1, not node0.

Regards,

Alexandre

----- Mail original -----
De: "Zhu Guihua" <[email protected]>
À: "aderumier" <[email protected]>
Cc: "qemu-devel" <[email protected]>, [email protected], "guz fnst" <[email protected]>, "isimatu yasuaki" <[email protected]>, "Anshul Makkar" <[email protected]>, "chen fan fnst" <[email protected]>, "Igor Mammedov" <[email protected]>, "afaerber" <[email protected]> Envoyé: Lundi 26 Janvier 2015 03:01:48
Objet: Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

Quoted 5 lines · Alexandre DERUMIER

Hello,

I'm currently testing the new cpu unplug features, Works fine here with debian guests and kernel 3.14.

Thanks for your test.

Quoted 4 lines

But I have notice some small potential bugs, but I'm not sure I'm doing it right.

1)first, to unplug cpu, we need an id for cpu

Yes, if you want to unplug cpu, you must have an id for cpu.

Quoted 6 lines

The problem is that the current qemu command line
-smp 1,sockets=2,cores=1,maxcpus=2

for example, will create 1 cpu on apic-id 0 without any id, so we can't unplug it.

Quoted 7 lines

So, I have tried with

-smp 1,sockets=2,cores=1,maxcpus=2 -device kvm64-x86_64-cpu,apic-id=0,id=cpu0

But this give me an error:
"-device kvm64-x86_64-cpu,apic-id=0,id=cpu0: CPU with APIC ID 0 exists"

APIC ID 0 was used by the cpu of '-smp 1'.
So you should use apic-id=1

Quoted 7 lines

(also try to set -smp 0, but it's not working).

2) second problem, if I start with
-smp 1,sockets=2,cores=1,maxcpus=2

Quoted 9 lines

then hmp:
device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1

then hmp : device_del cpu1

Got an error:"
This is the last cpu, should not be removed!"

Oh, it's our problem, thanks for your pointing out. I will fix it in next version.

Regards,
Zhu

Quoted 21 lines

This is coming from
[PATCH 06/12] pc: add cpu hot unplug request callback support
+ if (smp_cpus == 1) {
+ error_setg(&local_err,
+ "This is the last cpu, should not be removed!");
+ goto out;
+ }

So, the only way unplug is working for me, is to start with -smp 2 minimum -smp 2,sockets=2,cores=1,maxcpus=4

Then I can hotplug|unplug cpuid >= 2

Regards,

Alexandre Derumier

[...]

author
#19 lore raw
In reply to Alexandre DERUMIER
Quoted 7 lines · Alexandre DERUMIER

Thanks for your reply.

2 others things:

1)
on cpu unplug, I see that the cpu is correctly removed from my linux guest but not from qemu

About this, I can do it successfully on my qemu. So can you tell us more information about your operation?

And I found the patchset you applied is the last version in your last email. I think you'd better apply the latest version.

Regards,
Zhu

Quoted 170 lines

starting with a guest with 3cpus:

guest: #ls -lah /sys/devices/system/ |grep cpu

drwxr-xr-x 6 root root    0 Jan 25 22:16 cpu0
drwxr-xr-x 6 root root    0 Jan 25 22:16 cpu1
drwxr-xr-x 6 root root    0 Jan 25 22:16 cpu2

hmp: # info cpus
* CPU #0: pc=0xffffffff81057022 (halted) thread_id=24972 CPU #1: pc=0xffffffff81057022 (halted) thread_id=24973 CPU #2: pc=0xffffffff81048bc1 (halted) thread_id=25102

then unplug cpu2
hmp : device_del cpu2

guest:

dmesg:

[  176.219754] Unregister pv shared memory for cpu 2
[  176.278881] smpboot: CPU 2 is now offline

#ls -lah /sys/devices/system/ |grep cpu
drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu0
drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu1

hmp: # info cpus
* CPU #0: pc=0xffffffff81057022 (halted) thread_id=24972 CPU #1: pc=0xffffffff81057022 (halted) thread_id=24973 CPU #2: pc=0xffffffff81048bc1 (halted) thread_id=25102

2)when numa is used, the hotplugged cpu is always on numa node 0
(cpu_add or device_add cpu)

starting a guest, with 2 sockets,1 cores

(quoted diff) +0 -3
-smp 2,sockets=2,cores=1,maxcpus=2
-object memory-backend-ram,size=256M,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0 
-object memory-backend-ram,size=256M,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1 

hmp:

# info numa
2 nodes
node 0 cpus: 0
node 0 size: 256 MB
node 1 cpus: 1
node 1 size: 256 MB

ok

now

starting with same topology, but with 1cpu at start

(quoted diff) +0 -3
-smp 2,sockets=2,cores=1,maxcpus=2
-object memory-backend-ram,size=256M,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0 
-object memory-backend-ram,size=256M,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1 
# info numa
2 nodes
node 0 cpus: 0
node 0 size: 256 MB
node 1 cpus: 
node 1 size: 256 MB

hotpluging a cpu

# device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1
# info numa
2 nodes
node 0 cpus: 0 1
node 0 size: 256 MB
node 1 cpus: 
node 1 size: 256 MB

cpu1 should be on node1, not node0.

Regards,

Alexandre

----- Mail original -----
De: "Zhu Guihua" <[email protected]>
À: "aderumier" <[email protected]>
Cc: "qemu-devel" <[email protected]>, [email protected], "guz fnst" <[email protected]>, "isimatu yasuaki" <[email protected]>, "Anshul Makkar" <[email protected]>, "chen fan fnst" <[email protected]>, "Igor Mammedov" <[email protected]>, "afaerber" <[email protected]> Envoyé: Lundi 26 Janvier 2015 03:01:48
Objet: Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

Quoted 5 lines · Alexandre DERUMIER

Hello,

I'm currently testing the new cpu unplug features, Works fine here with debian guests and kernel 3.14.

Thanks for your test.

Quoted 4 lines

But I have notice some small potential bugs, but I'm not sure I'm doing it right.

1)first, to unplug cpu, we need an id for cpu

Yes, if you want to unplug cpu, you must have an id for cpu.

Quoted 6 lines

The problem is that the current qemu command line
-smp 1,sockets=2,cores=1,maxcpus=2

for example, will create 1 cpu on apic-id 0 without any id, so we can't unplug it.

Quoted 7 lines

So, I have tried with

-smp 1,sockets=2,cores=1,maxcpus=2 -device kvm64-x86_64-cpu,apic-id=0,id=cpu0

But this give me an error:
"-device kvm64-x86_64-cpu,apic-id=0,id=cpu0: CPU with APIC ID 0 exists"

APIC ID 0 was used by the cpu of '-smp 1'.
So you should use apic-id=1

Quoted 7 lines

(also try to set -smp 0, but it's not working).

2) second problem, if I start with
-smp 1,sockets=2,cores=1,maxcpus=2

Quoted 9 lines

then hmp:
device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1

then hmp : device_del cpu1

Got an error:"
This is the last cpu, should not be removed!"

Oh, it's our problem, thanks for your pointing out. I will fix it in next version.

Regards,
Zhu

Quoted 21 lines

This is coming from
[PATCH 06/12] pc: add cpu hot unplug request callback support
+ if (smp_cpus == 1) {
+ error_setg(&local_err,
+ "This is the last cpu, should not be removed!");
+ goto out;
+ }

So, the only way unplug is working for me, is to start with -smp 2 minimum -smp 2,sockets=2,cores=1,maxcpus=4

Then I can hotplug|unplug cpuid >= 2

Regards,

Alexandre Derumier

[...]

A message that is not in the archive (sent off-list or not yet received): [email protected]

#21 lore raw
Quoted 2 lines
Quoted 2 lines

About this, I can do it successfully on my qemu. So can you tell us more information about your operation?

simply start with

-smp 2,sockets=2,cores=2,maxcpus=4 -device kvm64-x86_64-cpu,apic-id=2,id=cpu2

then

#device_del cpu2

Guest return

[  324.195024] Unregister pv shared memory for cpu 2
[  324.250579] smpboot: CPU 2 is now offline

but cpu is not remove in qemu.

I had also try to eject manualy from guest
echo 1 > /sys/bus/acpi/devices/LNXCPU\:02/eject

But I have this error message
ACPI: \_SB_.CP02: Eject incomplete - status 0xf

(maybe is it normal ? or maybe is it a guest bug (kernel 3.16 from debian wheezy backports) ?)

Quoted 2 lines
Quoted 2 lines

And I found the patchset you applied is the last version in your last email. I think you'd better apply the latest version.

I have applied on top of git theses patches series

https://www.mail-archive.com/qemu-devel%40nongnu.org/msg272745.html
[Qemu-devel] [RESEND PATCH v1 0/5] Common unplug and unplug request cb for memory and CPU hot-unplug.
https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg01552.html
[Qemu-devel] [PATCH v3 0/7] cpu: add device_add foo-x86_64-cpu support
https://www.mail-archive.com/[email protected]/msg273870.html
[Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

----- Mail original -----
De: "Zhu Guihua" <[email protected]>
À: "aderumier" <[email protected]>
Cc: "qemu-devel" <[email protected]>, [email protected], "guz fnst" <[email protected]>, "isimatu yasuaki" <[email protected]>, "Anshul Makkar" <[email protected]>, "chen fan fnst" <[email protected]>, "Igor Mammedov" <[email protected]>, "afaerber" <[email protected]> Envoyé: Lundi 26 Janvier 2015 04:47:13
Objet: Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

Quoted 7 lines · Alexandre DERUMIER

Thanks for your reply.

2 others things:

1)
on cpu unplug, I see that the cpu is correctly removed from my linux guest but not from qemu

About this, I can do it successfully on my qemu. So can you tell us more information about your operation?

And I found the patchset you applied is the last version in your last email. I think you'd better apply the latest version.

Regards,
Zhu

Quoted 170 lines

starting with a guest with 3cpus:

guest: #ls -lah /sys/devices/system/ |grep cpu
drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu0
drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu1
drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu2

hmp: # info cpus
* CPU #0: pc=0xffffffff81057022 (halted) thread_id=24972 CPU #1: pc=0xffffffff81057022 (halted) thread_id=24973 CPU #2: pc=0xffffffff81048bc1 (halted) thread_id=25102

then unplug cpu2
hmp : device_del cpu2

guest:

dmesg:

[ 176.219754] Unregister pv shared memory for cpu 2 
[ 176.278881] smpboot: CPU 2 is now offline 

#ls -lah /sys/devices/system/ |grep cpu
drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu0
drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu1

hmp: # info cpus
* CPU #0: pc=0xffffffff81057022 (halted) thread_id=24972 CPU #1: pc=0xffffffff81057022 (halted) thread_id=24973 CPU #2: pc=0xffffffff81048bc1 (halted) thread_id=25102

2)when numa is used, the hotplugged cpu is always on numa node 0 (cpu_add or device_add cpu)

starting a guest, with 2 sockets,1 cores

(quoted diff) +0 -3
-smp 2,sockets=2,cores=1,maxcpus=2 
-object memory-backend-ram,size=256M,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0 
-object memory-backend-ram,size=256M,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1 

hmp:

# info numa 
2 nodes 
node 0 cpus: 0 
node 0 size: 256 MB 
node 1 cpus: 1 
node 1 size: 256 MB 

ok

now

starting with same topology, but with 1cpu at start

(quoted diff) +0 -3
-smp 2,sockets=2,cores=1,maxcpus=2 
-object memory-backend-ram,size=256M,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0 
-object memory-backend-ram,size=256M,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1 
# info numa 
2 nodes 
node 0 cpus: 0 
node 0 size: 256 MB 
node 1 cpus: 
node 1 size: 256 MB 

hotpluging a cpu

# device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1 
# info numa 
2 nodes 
node 0 cpus: 0 1 
node 0 size: 256 MB 
node 1 cpus: 
node 1 size: 256 MB 

cpu1 should be on node1, not node0.

Regards,

Alexandre

----- Mail original -----
De: "Zhu Guihua" <[email protected]>
À: "aderumier" <[email protected]>
Cc: "qemu-devel" <[email protected]>, [email protected], "guz fnst" <[email protected]>, "isimatu yasuaki" <[email protected]>, "Anshul Makkar" <[email protected]>, "chen fan fnst" <[email protected]>, "Igor Mammedov" <[email protected]>, "afaerber" <[email protected]> Envoyé: Lundi 26 Janvier 2015 03:01:48
Objet: Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

Quoted 5 lines · Alexandre DERUMIER

Hello,

I'm currently testing the new cpu unplug features, Works fine here with debian guests and kernel 3.14.

Thanks for your test.

Quoted 4 lines

But I have notice some small potential bugs, but I'm not sure I'm doing it right.

1)first, to unplug cpu, we need an id for cpu

Yes, if you want to unplug cpu, you must have an id for cpu.

Quoted 6 lines

The problem is that the current qemu command line
-smp 1,sockets=2,cores=1,maxcpus=2

for example, will create 1 cpu on apic-id 0 without any id, so we can't unplug it.

Quoted 7 lines

So, I have tried with

-smp 1,sockets=2,cores=1,maxcpus=2 -device kvm64-x86_64-cpu,apic-id=0,id=cpu0

But this give me an error:
"-device kvm64-x86_64-cpu,apic-id=0,id=cpu0: CPU with APIC ID 0 exists"

APIC ID 0 was used by the cpu of '-smp 1'.
So you should use apic-id=1

Quoted 7 lines

(also try to set -smp 0, but it's not working).

2) second problem, if I start with
-smp 1,sockets=2,cores=1,maxcpus=2

Quoted 9 lines

then hmp:
device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1

then hmp : device_del cpu1

Got an error:"
This is the last cpu, should not be removed!"

Oh, it's our problem, thanks for your pointing out. I will fix it in next version.

Regards,
Zhu

Quoted 21 lines

This is coming from
[PATCH 06/12] pc: add cpu hot unplug request callback support
+ if (smp_cpus == 1) {
+ error_setg(&local_err,
+ "This is the last cpu, should not be removed!");
+ goto out;
+ }

So, the only way unplug is working for me, is to start with -smp 2 minimum -smp 2,sockets=2,cores=1,maxcpus=4

Then I can hotplug|unplug cpuid >= 2

Regards,

Alexandre Derumier

[...]

author
#22 lore raw
Quoted 16 lines · Alexandre DERUMIER
Quoted 2 lines
Quoted 2 lines

About this, I can do it successfully on my qemu. So can you tell us more information about your operation?

simply start with

-smp 2,sockets=2,cores=2,maxcpus=4 -device kvm64-x86_64-cpu,apic-id=2,id=cpu2

then

#device_del cpu2

Guest return

[  324.195024] Unregister pv shared memory for cpu 2
[  324.250579] smpboot: CPU 2 is now offline

but cpu is not remove in qemu.

Quoted 8 lines

I had also try to eject manualy from guest
echo 1 > /sys/bus/acpi/devices/LNXCPU\:02/eject

But I have this error message
ACPI: \_SB_.CP02: Eject incomplete - status 0xf

(maybe is it normal ? or maybe is it a guest bug (kernel 3.16 from debian wheezy backports) ?)

According to your description, the guest has invoked _EJ0 method, but the method was executed unsuccessfully.
Thus, only hot-unplug request was invoked in qemu, but hot-unplug was not so that cpu was still in qemu.

Did you test any other kernel version? I will try kernel 3.16 to investigate this problem later.

Regards,
Zhu

Quoted 208 lines
Quoted 3 lines
Quoted 3 lines

And I found the patchset you applied is the last version in your last email. I think you'd better apply the latest version.

I have applied on top of git theses patches series

https://www.mail-archive.com/qemu-devel%40nongnu.org/msg272745.html
[Qemu-devel] [RESEND PATCH v1 0/5] Common unplug and unplug request cb for memory and CPU hot-unplug.
https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg01552.html
[Qemu-devel] [PATCH v3 0/7] cpu: add device_add foo-x86_64-cpu support
https://www.mail-archive.com/[email protected]/msg273870.html
[Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

----- Mail original -----
De: "Zhu Guihua" <[email protected]>
À: "aderumier" <[email protected]>
Cc: "qemu-devel" <[email protected]>, [email protected], "guz fnst" <[email protected]>, "isimatu yasuaki" <[email protected]>, "Anshul Makkar" <[email protected]>, "chen fan fnst" <[email protected]>, "Igor Mammedov" <[email protected]>, "afaerber" <[email protected]> Envoyé: Lundi 26 Janvier 2015 04:47:13
Objet: Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

Quoted 7 lines · Alexandre DERUMIER

Thanks for your reply.

2 others things:

1)
on cpu unplug, I see that the cpu is correctly removed from my linux guest but not from qemu

About this, I can do it successfully on my qemu. So can you tell us more information about your operation?

And I found the patchset you applied is the last version in your last email. I think you'd better apply the latest version.

Regards,
Zhu

Quoted 170 lines

starting with a guest with 3cpus:

guest: #ls -lah /sys/devices/system/ |grep cpu
drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu0
drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu1
drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu2

hmp: # info cpus
* CPU #0: pc=0xffffffff81057022 (halted) thread_id=24972 CPU #1: pc=0xffffffff81057022 (halted) thread_id=24973 CPU #2: pc=0xffffffff81048bc1 (halted) thread_id=25102

then unplug cpu2
hmp : device_del cpu2

guest:

dmesg:

[ 176.219754] Unregister pv shared memory for cpu 2 
[ 176.278881] smpboot: CPU 2 is now offline 

#ls -lah /sys/devices/system/ |grep cpu
drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu0
drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu1

hmp: # info cpus
* CPU #0: pc=0xffffffff81057022 (halted) thread_id=24972 CPU #1: pc=0xffffffff81057022 (halted) thread_id=24973 CPU #2: pc=0xffffffff81048bc1 (halted) thread_id=25102

2)when numa is used, the hotplugged cpu is always on numa node 0 (cpu_add or device_add cpu)

starting a guest, with 2 sockets,1 cores

(quoted diff) +0 -3
-smp 2,sockets=2,cores=1,maxcpus=2 
-object memory-backend-ram,size=256M,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0 
-object memory-backend-ram,size=256M,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1 

hmp:

# info numa 
2 nodes 
node 0 cpus: 0 
node 0 size: 256 MB 
node 1 cpus: 1 
node 1 size: 256 MB 

ok

now

starting with same topology, but with 1cpu at start

(quoted diff) +0 -3
-smp 2,sockets=2,cores=1,maxcpus=2 
-object memory-backend-ram,size=256M,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0 
-object memory-backend-ram,size=256M,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1 
# info numa 
2 nodes 
node 0 cpus: 0 
node 0 size: 256 MB 
node 1 cpus: 
node 1 size: 256 MB 

hotpluging a cpu

# device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1 
# info numa 
2 nodes 
node 0 cpus: 0 1 
node 0 size: 256 MB 
node 1 cpus: 
node 1 size: 256 MB 

cpu1 should be on node1, not node0.

Regards,

Alexandre

----- Mail original -----
De: "Zhu Guihua" <[email protected]>
À: "aderumier" <[email protected]>
Cc: "qemu-devel" <[email protected]>, [email protected], "guz fnst" <[email protected]>, "isimatu yasuaki" <[email protected]>, "Anshul Makkar" <[email protected]>, "chen fan fnst" <[email protected]>, "Igor Mammedov" <[email protected]>, "afaerber" <[email protected]> Envoyé: Lundi 26 Janvier 2015 03:01:48
Objet: Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

Quoted 5 lines · Alexandre DERUMIER

Hello,

I'm currently testing the new cpu unplug features, Works fine here with debian guests and kernel 3.14.

Thanks for your test.

Quoted 4 lines

But I have notice some small potential bugs, but I'm not sure I'm doing it right.

1)first, to unplug cpu, we need an id for cpu

Yes, if you want to unplug cpu, you must have an id for cpu.

Quoted 6 lines

The problem is that the current qemu command line
-smp 1,sockets=2,cores=1,maxcpus=2

for example, will create 1 cpu on apic-id 0 without any id, so we can't unplug it.

Quoted 7 lines

So, I have tried with

-smp 1,sockets=2,cores=1,maxcpus=2 -device kvm64-x86_64-cpu,apic-id=0,id=cpu0

But this give me an error:
"-device kvm64-x86_64-cpu,apic-id=0,id=cpu0: CPU with APIC ID 0 exists"

APIC ID 0 was used by the cpu of '-smp 1'.
So you should use apic-id=1

Quoted 7 lines

(also try to set -smp 0, but it's not working).

2) second problem, if I start with
-smp 1,sockets=2,cores=1,maxcpus=2

Quoted 9 lines

then hmp:
device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1

then hmp : device_del cpu1

Got an error:"
This is the last cpu, should not be removed!"

Oh, it's our problem, thanks for your pointing out. I will fix it in next version.

Regards,
Zhu

Quoted 21 lines

This is coming from
[PATCH 06/12] pc: add cpu hot unplug request callback support
+ if (smp_cpus == 1) {
+ error_setg(&local_err,
+ "This is the last cpu, should not be removed!");
+ goto out;
+ }

So, the only way unplug is working for me, is to start with -smp 2 minimum -smp 2,sockets=2,cores=1,maxcpus=4

Then I can hotplug|unplug cpuid >= 2

Regards,

Alexandre Derumier

[...]

#23 lore raw
Quoted 26 lines · Zhu Guihua
Quoted 16 lines · Alexandre DERUMIER
Quoted 2 lines
Quoted 2 lines

About this, I can do it successfully on my qemu. So can you tell us more information about your operation?

simply start with

-smp 2,sockets=2,cores=2,maxcpus=4 -device kvm64-x86_64-cpu,apic-id=2,id=cpu2

then

#device_del cpu2

Guest return

[  324.195024] Unregister pv shared memory for cpu 2
[  324.250579] smpboot: CPU 2 is now offline

but cpu is not remove in qemu.

Quoted 8 lines

I had also try to eject manualy from guest echo 1 > /sys/bus/acpi/devices/LNXCPU\:02/eject

But I have this error message
ACPI: \_SB_.CP02: Eject incomplete - status 0xf

(maybe is it normal ? or maybe is it a guest bug (kernel 3.16 from debian wheezy backports) ?)

Quoted 7 lines

According to your description, the guest has invoked _EJ0 method, but the method was executed unsuccessfully.
Thus, only hot-unplug request was invoked in qemu, but hot-unplug was not so that cpu was still in qemu.

Did you test any other kernel version? I will try kernel 3.16 to investigate this problem later.

Not sure if it is exactly same, but I had similar problem as seen by Alexandre when I had tried this patchset earlier and it was resolved by using iasl and regenerating the ACPI tables.

Ref: http://lists.nongnu.org/archive/html/qemu-devel/2014-09/msg02993.html

Regards,
Bharata.

#24 lore raw
In reply to Zhu Guihua

HI,

Can you push the patchset to a branch on github? It will be convenient for other guys to do some tests.

Quoted 82 lines · Zhu Guihua

This series is based on chen fan's previous i386 cpu hot remove patchset:
https://lists.nongnu.org/archive/html/qemu-devel/2013-12/msg04266.html

Via implementing ACPI standard methods _EJ0 in ACPI table, after Guest OS remove one vCPU online, the fireware will store removed bitmap to QEMU, then QEMU could know to notify the assigned vCPU of exiting. Meanwhile, intruduce the QOM command 'device_del' to remove vCPU from QEMU itself.

The whole work is based on the new hot plug/unplug framework, ,the unplug request callback does the pre-check and send the request, unplug callback does the removal handling.

This series depends on tangchen's common hot plug/unplug enhance patchset. [RESEND PATCH v1 0/5] Common unplug and unplug request cb for memory and CPU hot-unplug
https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg00429.html

The is the second half of the previous series:
[RFC V2 00/10] cpu: add device_add foo-x86_64-cpu and i386 cpu hot remove support
https://lists.nongnu.org/archive/html/qemu-devel/2014-08/msg04779.html

If you want to test the series, you need to apply the 'device_add foo-x86_64-cpu' patchset first:
[PATCH v3 0/7] cpu: add device_add foo-x86_64-cpu support
https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg01552.html

---
Changelog since v1:
-rebase on the latest version.
-delete patch i386/cpu: add instance finalize callback, and put it into patchset
[PATCH v3 0/6] cpu: add device_add foo-x86_64-cpu support.

Changelog since RFC:
-splited the i386 cpu hot remove into single thread. -replaced apic_no with apic_id, so does the related stuff to make it
work with arbitrary CPU hotadd.
-add the icc_device_unrealize callback to handle apic unrealize. -rework on the new hot plug/unplug platform.
---

Chen Fan (2):
x86: add x86_cpu_unrealizefn() for cpu apic remove
cpu hotplug: implement function cpu_status_write() for vcpu ejection

Gu Zheng (5):
acpi/cpu: add cpu hot unplug request callback function acpi/piix4: add cpu hot unplug callback support acpi/ich9: add cpu hot unplug support
pc: add cpu hot unplug callback support
cpus: reclaim allocated vCPU objects

Zhu Guihua (4):
acpi/piix4: add cpu hot unplug request callback support acpi/ich9: add cpu hot unplug request callback support
pc: add cpu hot unplug request callback support acpi/cpu: add cpu hot unplug callback function

 cpus.c                            | 44 ++++++++++++++++++++
 hw/acpi/cpu_hotplug.c             | 88 ++++++++++++++++++++++++++++++++++++---
 hw/acpi/ich9.c                    | 17 ++++++--
 hw/acpi/piix4.c                   | 12 +++++-
 hw/core/qdev.c                    |  2 +-
 hw/cpu/icc_bus.c                  | 11 +++++
 hw/i386/acpi-dsdt-cpu-hotplug.dsl |  6 ++-
 hw/i386/kvm/apic.c                |  8 ++++
 hw/i386/pc.c                      | 62 +++++++++++++++++++++++++--
 hw/intc/apic.c                    | 10 +++++
 hw/intc/apic_common.c             | 21 ++++++----
 include/hw/acpi/cpu_hotplug.h     |  8 ++++
 include/hw/cpu/icc_bus.h          |  1 +
 include/hw/i386/apic_internal.h   |  1 +
 include/hw/qdev-core.h            |  1 +
 include/qom/cpu.h                 |  9 ++++
 include/sysemu/kvm.h              |  1 +
 kvm-all.c                         | 57 ++++++++++++++++++++++++-
 target-i386/cpu.c                 | 46 ++++++++++++++++++++
 19 files changed, 378 insertions(+), 27 deletions(-)

--
1.9.3

Signature · 3 lines
Regards,

Zhi Yong Wu
author
#25 lore raw
Quoted 4 lines · Zhi Yong Wu

HI,

Can you push the patchset to a branch on github? It will be convenient for other guys to do some tests.

sorry for late reply.
I had pushed into https://github.com/zhugh/qemu.git
The patchset is on branch cpu-hotplug.

Welcome to test, Thanks.

Regards,
Zhu

Quoted 87 lines · Zhu Guihua
Quoted 82 lines

This series is based on chen fan's previous i386 cpu hot remove patchset:
https://lists.nongnu.org/archive/html/qemu-devel/2013-12/msg04266.html

Via implementing ACPI standard methods _EJ0 in ACPI table, after Guest OS remove one vCPU online, the fireware will store removed bitmap to QEMU, then QEMU could know to notify the assigned vCPU of exiting. Meanwhile, intruduce the QOM command 'device_del' to remove vCPU from QEMU itself.

The whole work is based on the new hot plug/unplug framework, ,the unplug request callback does the pre-check and send the request, unplug callback does the removal handling.

This series depends on tangchen's common hot plug/unplug enhance patchset. [RESEND PATCH v1 0/5] Common unplug and unplug request cb for memory and CPU hot-unplug
https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg00429.html

The is the second half of the previous series:
[RFC V2 00/10] cpu: add device_add foo-x86_64-cpu and i386 cpu hot remove support
https://lists.nongnu.org/archive/html/qemu-devel/2014-08/msg04779.html

If you want to test the series, you need to apply the 'device_add foo-x86_64-cpu' patchset first:
[PATCH v3 0/7] cpu: add device_add foo-x86_64-cpu support
https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg01552.html

---
Changelog since v1:
-rebase on the latest version.
-delete patch i386/cpu: add instance finalize callback, and put it into patchset
[PATCH v3 0/6] cpu: add device_add foo-x86_64-cpu support.

Changelog since RFC:
-splited the i386 cpu hot remove into single thread. -replaced apic_no with apic_id, so does the related stuff to make it
work with arbitrary CPU hotadd.
-add the icc_device_unrealize callback to handle apic unrealize. -rework on the new hot plug/unplug platform.
---

Chen Fan (2):
x86: add x86_cpu_unrealizefn() for cpu apic remove
cpu hotplug: implement function cpu_status_write() for vcpu ejection

Gu Zheng (5):
acpi/cpu: add cpu hot unplug request callback function acpi/piix4: add cpu hot unplug callback support acpi/ich9: add cpu hot unplug support
pc: add cpu hot unplug callback support
cpus: reclaim allocated vCPU objects

Zhu Guihua (4):
acpi/piix4: add cpu hot unplug request callback support acpi/ich9: add cpu hot unplug request callback support
pc: add cpu hot unplug request callback support acpi/cpu: add cpu hot unplug callback function

 cpus.c                            | 44 ++++++++++++++++++++
 hw/acpi/cpu_hotplug.c             | 88 ++++++++++++++++++++++++++++++++++++---
 hw/acpi/ich9.c                    | 17 ++++++--
 hw/acpi/piix4.c                   | 12 +++++-
 hw/core/qdev.c                    |  2 +-
 hw/cpu/icc_bus.c                  | 11 +++++
 hw/i386/acpi-dsdt-cpu-hotplug.dsl |  6 ++-
 hw/i386/kvm/apic.c                |  8 ++++
 hw/i386/pc.c                      | 62 +++++++++++++++++++++++++--
 hw/intc/apic.c                    | 10 +++++
 hw/intc/apic_common.c             | 21 ++++++----
 include/hw/acpi/cpu_hotplug.h     |  8 ++++
 include/hw/cpu/icc_bus.h          |  1 +
 include/hw/i386/apic_internal.h   |  1 +
 include/hw/qdev-core.h            |  1 +
 include/qom/cpu.h                 |  9 ++++
 include/sysemu/kvm.h              |  1 +
 kvm-all.c                         | 57 ++++++++++++++++++++++++-
 target-i386/cpu.c                 | 46 ++++++++++++++++++++
 19 files changed, 378 insertions(+), 27 deletions(-)

--
1.9.3

#26 lore raw
Quoted 11 lines · Zhu Guihua
Quoted 4 lines · Zhi Yong Wu

HI,

Can you push the patchset to a branch on github? It will be convenient for other guys to do some tests.

sorry for late reply.
I had pushed into https://github.com/zhugh/qemu.git
The patchset is on branch cpu-hotplug.

Welcome to test, Thanks.

HI,

Can you let me know why device_add can't support the vCPU with 'host' model?

Quoted 93 lines

Regards,
Zhu

Quoted 87 lines · Zhu Guihua
Quoted 82 lines

This series is based on chen fan's previous i386 cpu hot remove patchset:
https://lists.nongnu.org/archive/html/qemu-devel/2013-12/msg04266.html

Via implementing ACPI standard methods _EJ0 in ACPI table, after Guest OS remove one vCPU online, the fireware will store removed bitmap to QEMU, then QEMU could know to notify the assigned vCPU of exiting. Meanwhile, intruduce the QOM command 'device_del' to remove vCPU from QEMU itself.

The whole work is based on the new hot plug/unplug framework, ,the unplug request callback does the pre-check and send the request, unplug callback does the removal handling.

This series depends on tangchen's common hot plug/unplug enhance patchset. [RESEND PATCH v1 0/5] Common unplug and unplug request cb for memory and CPU hot-unplug
https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg00429.html

The is the second half of the previous series:
[RFC V2 00/10] cpu: add device_add foo-x86_64-cpu and i386 cpu hot remove support
https://lists.nongnu.org/archive/html/qemu-devel/2014-08/msg04779.html

If you want to test the series, you need to apply the 'device_add foo-x86_64-cpu' patchset first:
[PATCH v3 0/7] cpu: add device_add foo-x86_64-cpu support
https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg01552.html

---
Changelog since v1:
-rebase on the latest version.
-delete patch i386/cpu: add instance finalize callback, and put it into patchset
[PATCH v3 0/6] cpu: add device_add foo-x86_64-cpu support.

Changelog since RFC:
-splited the i386 cpu hot remove into single thread. -replaced apic_no with apic_id, so does the related stuff to make it
work with arbitrary CPU hotadd.
-add the icc_device_unrealize callback to handle apic unrealize. -rework on the new hot plug/unplug platform.
---

Chen Fan (2):
x86: add x86_cpu_unrealizefn() for cpu apic remove
cpu hotplug: implement function cpu_status_write() for vcpu ejection

Gu Zheng (5):
acpi/cpu: add cpu hot unplug request callback function acpi/piix4: add cpu hot unplug callback support acpi/ich9: add cpu hot unplug support
pc: add cpu hot unplug callback support
cpus: reclaim allocated vCPU objects

Zhu Guihua (4):
acpi/piix4: add cpu hot unplug request callback support acpi/ich9: add cpu hot unplug request callback support
pc: add cpu hot unplug request callback support acpi/cpu: add cpu hot unplug callback function

 cpus.c                            | 44 ++++++++++++++++++++
 hw/acpi/cpu_hotplug.c             | 88 ++++++++++++++++++++++++++++++++++++---
 hw/acpi/ich9.c                    | 17 ++++++--
 hw/acpi/piix4.c                   | 12 +++++-
 hw/core/qdev.c                    |  2 +-
 hw/cpu/icc_bus.c                  | 11 +++++
 hw/i386/acpi-dsdt-cpu-hotplug.dsl |  6 ++-
 hw/i386/kvm/apic.c                |  8 ++++
 hw/i386/pc.c                      | 62 +++++++++++++++++++++++++--
 hw/intc/apic.c                    | 10 +++++
 hw/intc/apic_common.c             | 21 ++++++----
 include/hw/acpi/cpu_hotplug.h     |  8 ++++
 include/hw/cpu/icc_bus.h          |  1 +
 include/hw/i386/apic_internal.h   |  1 +
 include/hw/qdev-core.h            |  1 +
 include/qom/cpu.h                 |  9 ++++
 include/sysemu/kvm.h              |  1 +
 kvm-all.c                         | 57 ++++++++++++++++++++++++-
 target-i386/cpu.c                 | 46 ++++++++++++++++++++
 19 files changed, 378 insertions(+), 27 deletions(-)

--
1.9.3

Signature · 3 lines
Regards,

Zhi Yong Wu
author
#27 lore raw
Quoted 7 lines · Zhi Yong Wu
Quoted 6 lines · Zhu Guihua
Quoted 4 lines · Zhi Yong Wu

HI,

Can you push the patchset to a branch on github? It will be convenient for other guys to do some tests.

Quoted 10 lines
Quoted 5 lines

sorry for late reply.
I had pushed into https://github.com/zhugh/qemu.git
The patchset is on branch cpu-hotplug.

Welcome to test, Thanks.

HI,

Can you let me know why device_add can't support the vCPU with 'host' model?

Quoted 1 line

I have not noticed this before, thanks for your mentioning.

host_x86_cpu does not support hotplug by default, we will fix this in next version.

Regards,
Zhu

Quoted 95 lines
Quoted 92 lines

Regards,
Zhu

Quoted 87 lines · Zhu Guihua
Quoted 82 lines

This series is based on chen fan's previous i386 cpu hot remove patchset:
https://lists.nongnu.org/archive/html/qemu-devel/2013-12/msg04266.html

Via implementing ACPI standard methods _EJ0 in ACPI table, after Guest OS remove one vCPU online, the fireware will store removed bitmap to QEMU, then QEMU could know to notify the assigned vCPU of exiting. Meanwhile, intruduce the QOM command 'device_del' to remove vCPU from QEMU itself.

The whole work is based on the new hot plug/unplug framework, ,the unplug request callback does the pre-check and send the request, unplug callback does the removal handling.

This series depends on tangchen's common hot plug/unplug enhance patchset. [RESEND PATCH v1 0/5] Common unplug and unplug request cb for memory and CPU hot-unplug
https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg00429.html

The is the second half of the previous series:
[RFC V2 00/10] cpu: add device_add foo-x86_64-cpu and i386 cpu hot remove support
https://lists.nongnu.org/archive/html/qemu-devel/2014-08/msg04779.html

If you want to test the series, you need to apply the 'device_add foo-x86_64-cpu' patchset first:
[PATCH v3 0/7] cpu: add device_add foo-x86_64-cpu support
https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg01552.html

---
Changelog since v1:
-rebase on the latest version.
-delete patch i386/cpu: add instance finalize callback, and put it into patchset
[PATCH v3 0/6] cpu: add device_add foo-x86_64-cpu support.

Changelog since RFC:
-splited the i386 cpu hot remove into single thread. -replaced apic_no with apic_id, so does the related stuff to make it
work with arbitrary CPU hotadd.
-add the icc_device_unrealize callback to handle apic unrealize. -rework on the new hot plug/unplug platform.
---

Chen Fan (2):
x86: add x86_cpu_unrealizefn() for cpu apic remove
cpu hotplug: implement function cpu_status_write() for vcpu ejection

Gu Zheng (5):
acpi/cpu: add cpu hot unplug request callback function acpi/piix4: add cpu hot unplug callback support acpi/ich9: add cpu hot unplug support
pc: add cpu hot unplug callback support
cpus: reclaim allocated vCPU objects

Zhu Guihua (4):
acpi/piix4: add cpu hot unplug request callback support acpi/ich9: add cpu hot unplug request callback support
pc: add cpu hot unplug request callback support acpi/cpu: add cpu hot unplug callback function

 cpus.c                            | 44 ++++++++++++++++++++
 hw/acpi/cpu_hotplug.c             | 88 ++++++++++++++++++++++++++++++++++++---
 hw/acpi/ich9.c                    | 17 ++++++--
 hw/acpi/piix4.c                   | 12 +++++-
 hw/core/qdev.c                    |  2 +-
 hw/cpu/icc_bus.c                  | 11 +++++
 hw/i386/acpi-dsdt-cpu-hotplug.dsl |  6 ++-
 hw/i386/kvm/apic.c                |  8 ++++
 hw/i386/pc.c                      | 62 +++++++++++++++++++++++++--
 hw/intc/apic.c                    | 10 +++++
 hw/intc/apic_common.c             | 21 ++++++----
 include/hw/acpi/cpu_hotplug.h     |  8 ++++
 include/hw/cpu/icc_bus.h          |  1 +
 include/hw/i386/apic_internal.h   |  1 +
 include/hw/qdev-core.h            |  1 +
 include/qom/cpu.h                 |  9 ++++
 include/sysemu/kvm.h              |  1 +
 kvm-all.c                         | 57 ++++++++++++++++++++++++-
 target-i386/cpu.c                 | 46 ++++++++++++++++++++
 19 files changed, 378 insertions(+), 27 deletions(-)

--
1.9.3

author
#28 lore raw
Quoted 15 lines · Zhu Guihua
Quoted 14 lines · Zhi Yong Wu
Quoted 11 lines · Zhu Guihua
Quoted 4 lines · Zhi Yong Wu

HI,

Can you push the patchset to a branch on github? It will be convenient for other guys to do some tests.

sorry for late reply.
I had pushed into https://github.com/zhugh/qemu.git
The patchset is on branch cpu-hotplug.

Welcome to test, Thanks.

HI,

Quoted 8 lines
Quoted 3 lines

Can you let me know why device_add can't support the vCPU with 'host' model?

Quoted 1 line

I have not noticed this before, thanks for your mentioning.

host_x86_cpu does not support hotplug by default, we will fix this in next version.

I underestimated the complexity of this issue, and expect to handle this in v5 for x86_64-cpu device_add. Thanks.

Regards,
Zhu

Quoted 101 lines

Regards,
Zhu

Quoted 95 lines
Quoted 92 lines

Regards,
Zhu

Quoted 87 lines · Zhu Guihua
Quoted 82 lines

This series is based on chen fan's previous i386 cpu hot remove patchset:
https://lists.nongnu.org/archive/html/qemu-devel/2013-12/msg04266.html

Via implementing ACPI standard methods _EJ0 in ACPI table, after Guest OS remove one vCPU online, the fireware will store removed bitmap to QEMU, then QEMU could know to notify the assigned vCPU of exiting. Meanwhile, intruduce the QOM command 'device_del' to remove vCPU from QEMU itself.

The whole work is based on the new hot plug/unplug framework, ,the unplug request callback does the pre-check and send the request, unplug callback does the removal handling.

This series depends on tangchen's common hot plug/unplug enhance patchset. [RESEND PATCH v1 0/5] Common unplug and unplug request cb for memory and CPU hot-unplug
https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg00429.html

The is the second half of the previous series:
[RFC V2 00/10] cpu: add device_add foo-x86_64-cpu and i386 cpu hot remove support
https://lists.nongnu.org/archive/html/qemu-devel/2014-08/msg04779.html

If you want to test the series, you need to apply the 'device_add foo-x86_64-cpu' patchset first:
[PATCH v3 0/7] cpu: add device_add foo-x86_64-cpu support
https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg01552.html

---
Changelog since v1:
-rebase on the latest version.
-delete patch i386/cpu: add instance finalize callback, and put it into patchset
[PATCH v3 0/6] cpu: add device_add foo-x86_64-cpu support.

Changelog since RFC:
-splited the i386 cpu hot remove into single thread. -replaced apic_no with apic_id, so does the related stuff to make it
work with arbitrary CPU hotadd.
-add the icc_device_unrealize callback to handle apic unrealize. -rework on the new hot plug/unplug platform.
---

Chen Fan (2):
x86: add x86_cpu_unrealizefn() for cpu apic remove
cpu hotplug: implement function cpu_status_write() for vcpu ejection

Gu Zheng (5):
acpi/cpu: add cpu hot unplug request callback function acpi/piix4: add cpu hot unplug callback support acpi/ich9: add cpu hot unplug support
pc: add cpu hot unplug callback support
cpus: reclaim allocated vCPU objects

Zhu Guihua (4):
acpi/piix4: add cpu hot unplug request callback support acpi/ich9: add cpu hot unplug request callback support
pc: add cpu hot unplug request callback support acpi/cpu: add cpu hot unplug callback function

 cpus.c                            | 44 ++++++++++++++++++++
 hw/acpi/cpu_hotplug.c             | 88 ++++++++++++++++++++++++++++++++++++---
 hw/acpi/ich9.c                    | 17 ++++++--
 hw/acpi/piix4.c                   | 12 +++++-
 hw/core/qdev.c                    |  2 +-
 hw/cpu/icc_bus.c                  | 11 +++++
 hw/i386/acpi-dsdt-cpu-hotplug.dsl |  6 ++-
 hw/i386/kvm/apic.c                |  8 ++++
 hw/i386/pc.c                      | 62 +++++++++++++++++++++++++--
 hw/intc/apic.c                    | 10 +++++
 hw/intc/apic_common.c             | 21 ++++++----
 include/hw/acpi/cpu_hotplug.h     |  8 ++++
 include/hw/cpu/icc_bus.h          |  1 +
 include/hw/i386/apic_internal.h   |  1 +
 include/hw/qdev-core.h            |  1 +
 include/qom/cpu.h                 |  9 ++++
 include/sysemu/kvm.h              |  1 +
 kvm-all.c                         | 57 ++++++++++++++++++++++++-
 target-i386/cpu.c                 | 46 ++++++++++++++++++++
 19 files changed, 378 insertions(+), 27 deletions(-)

--
1.9.3