Skip to contentSkip to conversation

target-i386: CPU hot-add with cpu-add QMP command

PATCH Applied Igor Mammedov 11 participants 85 messages Apr 23, 2013 – Apr 29, 2013 Kbuild

v5 of 9 Applied Latest is v9 →

lore
Share & tools
lore
#0 lore raw

v5 cover target-i386: CPU hot-add with cpu-add QMP command

Implements alternative way for hot-adding CPU using cpu-add QMP command, wich could be useful until it would be possible to add CPUs via device_add.

All patches except the last are also applicable to device_add aprroach.

To hot-add CPU use following command from qmp-shell:
cpu-add id=[0..max-cpus - 1)

git tree for testing: https://github.com/imammedo/qemu/tree/cpu_add.v5

based on qom-cpu-nex tree due to dependency on "mc146818rtc: QOM'ify"

v5->v4:
* style fixes
* new helper qemu_for_each_cpu()
* switch to qemu_for_each_cpu() in cpu_exists()
* "pc: update rtc ..." patch make depend it on "mc146818rtc: QOM'ify" and use QOM cast style
* call CPU added notifier right before CPU becomes runable
* s/resume_vcpu/cpu_resume/
* acpi/piix4: add spec documentation for QEMU<->Seabios CPU hotplug interface
* use error_propagate() in pc_new_cpu()
* skip cpu_exists() check in apic-id property setter if new value is the same as current
* embed icc-bus inside icc-bridge and use qbus_create_inplace()
* move include/hw/i386/icc_bus.h into include/hw/cpu/
* make missing icc-bus fatal error for softmmu target
* split "move APIC to ICC bus" and "move IOAPIC to ICC bus" on smaller patches
* use qdev_get_parent_bus() to get parent bus
* split "add cpu-add command..." on smaller patches

v4->v3:
* 'id' in cpu-add command will be a thread number instead of APIC ID
* split off resume_vcpu() into separate patch
* move notifier from rtc code into pc.c

v2->v3:
* use local error & propagate_error() instead of operating on passed in errp in several places
* replace CPUClass.get_firmware_id() with CPUClass.get_arch_id()
* leave IOAPIC creation to board and just set bus to icc-bus
* include kvm-stub.o in cpu libary if no KVM is configured
* create resume_vcpu() stub and include it in libqemustub, and use it directly instead of CPU method
* acpi_piix4: s/cpu_add_notifier/cpu_added_notifier/

v1->v2:
* generalize cpu sync to KVM, resume and hot-plug notification and invoke them form CPUClass, to make available to all targets.
* introduce cpu_exists() and CPUClass.get_firmware_id() and use the last one in acpi_piix to make code target independent.
* move IOAPIC to ICC bus, it was suggested and easy to convert.
* leave kvmvapic as SysBusDevice, it doesn't affect hot-plug and created only once for all APIC instances. I haven't found yet good/clean enough way to convert it to ICCDevice. May be follow-up though.
* split one big ICC patch into several, one per converted device
* add cpu_hot_add hook to machine and implement it for target-i386, instead of adding stabs. Could be used by other targets to implement cpu-add.
* pre-allocate links<CPU> for all possible CPUs and make them available at /machine/icc-bridge/cpu[0..N] QOM path, so users could find out possible/free CPU IDs to use in cpu-add command.

Igor Mammedov (21):
cpu: make kvm-stub.o a part of CPU library
cpu: call cpu_synchronize_post_init() from CPUClass.realize() if
hotplugged
introduce cpu_resume(), for single CPU
cpu: resume CPU from CPUClass.cpu_common_realizefn() when it is
hot-plugged
introduce CPU hot-plug notifier
target-i386: pc: update rtc_cmos on CPU hot-plug
cpu: introduce get_arch_id() method and override it for target-i386
exec: add qemu_for_each_cpu
cpu: add helper cpu_exists(), to check if CPU with specified id
exists
acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest
target-i386: introduce apic-id property
target-i386: introduce ICC bus/device/bridge
target-i386: cpu: attach ICC bus to CPU on its creation
target-i386: replace MSI_SPACE_SIZE with APIC_SPACE_SIZE
target-i386: kvmvapic: make expilict dependency on sysbus.h
target-i386: move APIC to ICC bus
introduce memory_region_get_address() and use it in kvm/ioapic
target-i386: move IOAPIC to ICC bus
add hot_add_cpu hook to QEMUMachine and export machine_args
target-i386: implement machine->hot_add_cpu hook
QMP: add cpu-add command

 MAINTAINERS                        |    6 ++
 Makefile.target                    |   14 ++--
 cpus.c                             |   24 +++++---
 default-configs/i386-softmmu.mak   |    1 +
 default-configs/x86_64-softmmu.mak |    1 +
 docs/specs/acpi_cpu_hotplug.txt    |   22 +++++++
 exec.c                             |   10 +++
 hw/acpi/piix4.c                    |  117 ++++++++++++++++++++++++++++++++++-
 hw/cpu/Makefile.objs               |    1 +
 hw/cpu/icc_bus.c                   |  120 ++++++++++++++++++++++++++++++++++++
 hw/i386/kvm/apic.c                 |    2 +-
 hw/i386/kvm/ioapic.c               |    2 +-
 hw/i386/kvmvapic.c                 |    1 +
 hw/i386/pc.c                       |   93 ++++++++++++++++++++++++++--
 hw/i386/pc_piix.c                  |    7 ++
 hw/i386/pc_q35.c                   |    7 ++
 hw/intc/apic.c                     |    2 +-
 hw/intc/apic_common.c              |   18 ++++--
 hw/intc/ioapic_common.c            |   14 +++-
 hw/timer/mc146818rtc.c             |    7 ++
 hw/xen/xen_apic.c                  |    2 +-
 include/exec/memory.h              |   10 +++
 include/hw/boards.h                |    3 +
 include/hw/cpu/icc_bus.h           |   63 +++++++++++++++++++
 include/hw/i386/apic_internal.h    |    8 +--
 include/hw/i386/ioapic_internal.h  |    6 +-
 include/hw/timer/mc146818rtc.h     |    1 +
 include/qom/cpu.h                  |   27 ++++++++
 include/sysemu/kvm.h               |   24 ++++----
 include/sysemu/sysemu.h            |    3 +
 kvm-all.c                          |    1 +
 kvm-stub.c                         |    8 ++-
 memory.c                           |   11 +++
 qapi-schema.json                   |   11 +++
 qmp-commands.hx                    |   23 +++++++
 qmp.c                              |   10 +++
 qom/cpu.c                          |   49 +++++++++++++++
 stubs/Makefile.objs                |    1 +
 stubs/cpus.c                       |    5 ++
 target-i386/cpu.c                  |   83 +++++++++++++++++++++----
 target-i386/cpu.h                  |    1 +
 vl.c                               |    7 ++-
 42 files changed, 752 insertions(+), 74 deletions(-)
 create mode 100644 docs/specs/acpi_cpu_hotplug.txt
 create mode 100644 hw/cpu/icc_bus.c
 create mode 100644 include/hw/cpu/icc_bus.h
 create mode 100644 stubs/cpus.c
#1 lore raw

PATCH v5 1/21 cpu: make kvm-stub.o a part of CPU library

---
v3:
put ifndef around msi.h in kvm-stub.c instead of in msi.h

v2:
remove unnecessary kvm-stub.o from "System emulator target" section

---
 Makefile.target      |   14 +++++++-------
 include/sysemu/kvm.h |    4 ++--
 kvm-stub.c           |    7 ++++++-
 3 files changed, 15 insertions(+), 10 deletions(-)
Makefile.target +7 -7
index 2636103..0783c13 100644
@@ -60,6 +60,12 @@ all: $(PROGS) stap
 # Dummy command so that make thinks it has done something
 	@true
 
+CONFIG_NO_PCI = $(if $(subst n,,$(CONFIG_PCI)),n,y)
+CONFIG_NO_KVM = $(if $(subst n,,$(CONFIG_KVM)),n,y)
+CONFIG_NO_XEN = $(if $(subst n,,$(CONFIG_XEN)),n,y)
+CONFIG_NO_GET_MEMORY_MAPPING = $(if $(subst n,,$(CONFIG_HAVE_GET_MEMORY_MAPPING)),n,y)
+CONFIG_NO_CORE_DUMP = $(if $(subst n,,$(CONFIG_HAVE_CORE_DUMP)),n,y)
+
 #########################################################
 # cpu emulator library
 obj-y = exec.o translate-all.o cpu-exec.o
@@ -70,6 +76,7 @@ obj-y += fpu/softfloat.o
 obj-y += target-$(TARGET_BASE_ARCH)/
 obj-y += disas.o
 obj-$(CONFIG_GDBSTUB_XML) += gdbstub-xml.o
+obj-$(CONFIG_NO_KVM) += kvm-stub.o
 
 #########################################################
 # Linux user emulator target
@@ -98,18 +105,11 @@ endif #CONFIG_BSD_USER
 #########################################################
 # System emulator target
 ifdef CONFIG_SOFTMMU
-CONFIG_NO_PCI = $(if $(subst n,,$(CONFIG_PCI)),n,y)
-CONFIG_NO_KVM = $(if $(subst n,,$(CONFIG_KVM)),n,y)
-CONFIG_NO_XEN = $(if $(subst n,,$(CONFIG_XEN)),n,y)
-CONFIG_NO_GET_MEMORY_MAPPING = $(if $(subst n,,$(CONFIG_HAVE_GET_MEMORY_MAPPING)),n,y)
-CONFIG_NO_CORE_DUMP = $(if $(subst n,,$(CONFIG_HAVE_CORE_DUMP)),n,y)
-
 obj-y += arch_init.o cpus.o monitor.o gdbstub.o balloon.o ioport.o
 obj-y += qtest.o
 obj-y += hw/
 obj-$(CONFIG_FDT) += device_tree.o
 obj-$(CONFIG_KVM) += kvm-all.o
-obj-$(CONFIG_NO_KVM) += kvm-stub.o
 obj-y += memory.o savevm.o cputlb.o
 obj-$(CONFIG_HAVE_GET_MEMORY_MAPPING) += memory_mapping.o
 obj-$(CONFIG_HAVE_CORE_DUMP) += dump.o
include/sysemu/kvm.h +2 -2
index 75bd7d9..2bc1f6b 100644
@@ -144,10 +144,10 @@ int kvm_cpu_exec(CPUArchState *env);
 #if !defined(CONFIG_USER_ONLY)
 void *kvm_vmalloc(ram_addr_t size);
 void *kvm_arch_vmalloc(ram_addr_t size);
-void kvm_setup_guest_memory(void *start, size_t size);
+#endif
 
+void kvm_setup_guest_memory(void *start, size_t size);
 void kvm_flush_coalesced_mmio_buffer(void);
-#endif
 
 int kvm_insert_breakpoint(CPUArchState *current_env, target_ulong addr,
                           target_ulong len, int type);
kvm-stub.c +6 -1
index 5f52186..b228378 100644
@@ -12,10 +12,13 @@
 
 #include "qemu-common.h"
 #include "hw/hw.h"
-#include "hw/pci/msi.h"
 #include "cpu.h"
 #include "sysemu/kvm.h"
 
+#ifndef CONFIG_USER_ONLY
+#include "hw/pci/msi.h"
+#endif
+
 KVMState *kvm_state;
 bool kvm_kernel_irqchip;
 bool kvm_async_interrupts_allowed;
@@ -111,6 +114,7 @@ int kvm_on_sigbus(int code, void *addr)
     return 1;
 }
 
+#ifndef CONFIG_USER_ONLY
 int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg)
 {
     return -ENOSYS;
@@ -134,3 +138,4 @@ int kvm_irqchip_remove_irqfd_notifier(KVMState *s, EventNotifier *n, int virq)
 {
     return -ENOSYS;
 }
+#endif
Signature · 1 line
1.7.1
#2 lore raw

Re: [Qemu-devel] [PATCH 01/21] cpu: make kvm-stub.o a part of CPU library

Am 23.04.2013 10:29, schrieb Igor Mammedov:

Quoted 8 lines

---
v3:
put ifndef around msi.h in kvm-stub.c instead of in msi.h

v2:
remove unnecessary kvm-stub.o from "System emulator target" section

Thanks, applied to qom-cpu (with jointly reworded commit message):
https://github.com/afaerber/qemu-cpu/commits/qom-cpu

Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#3 lore raw

PATCH v5 2/21 cpu: call cpu_synchronize_post_init() from CPUClass.realize() if hotplugged

In reply to Igor Mammedov

... to synchronize CPU state to KVM

---
v2:
* linking kvm-stub.o to *-user target moved in separate patch

---
 include/sysemu/kvm.h |   20 ++++++++++----------
 kvm-all.c            |    1 +
 kvm-stub.c           |    1 +
 qom/cpu.c            |    4 ++++
 vl.c                 |    1 -
 5 files changed, 16 insertions(+), 11 deletions(-)
include/sysemu/kvm.h +10 -10
index 2bc1f6b..9735c1d 100644
@@ -250,8 +250,6 @@ int kvm_check_extension(KVMState *s, unsigned int extension);
 uint32_t kvm_arch_get_supported_cpuid(KVMState *env, uint32_t function,
                                       uint32_t index, int reg);
 void kvm_cpu_synchronize_state(CPUArchState *env);
-void kvm_cpu_synchronize_post_reset(CPUState *cpu);
-void kvm_cpu_synchronize_post_init(CPUState *cpu);
 
 /* generic hooks - to be moved/refactored once there are more users */
 
@@ -262,6 +260,16 @@ static inline void cpu_synchronize_state(CPUArchState *env)
     }
 }
 
+#if !defined(CONFIG_USER_ONLY)
+int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,
+                                       hwaddr *phys_addr);
+#endif
+
+#endif /* NEED_CPU_H */
+
+void kvm_cpu_synchronize_post_reset(CPUState *cpu);
+void kvm_cpu_synchronize_post_init(CPUState *cpu);
+
 static inline void cpu_synchronize_post_reset(CPUState *cpu)
 {
     if (kvm_enabled()) {
@@ -276,14 +284,6 @@ static inline void cpu_synchronize_post_init(CPUState *cpu)
     }
 }
 
-
-#if !defined(CONFIG_USER_ONLY)
-int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,
-                                       hwaddr *phys_addr);
-#endif
-
-#endif
-
 int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg);
 int kvm_irqchip_update_msi_route(KVMState *s, int virq, MSIMessage msg);
 void kvm_irqchip_release_virq(KVMState *s, int virq);
kvm-all.c +1 -0
index 2d92721..f6c0f4a 100644
@@ -109,6 +109,7 @@ bool kvm_async_interrupts_allowed;
 bool kvm_irqfds_allowed;
 bool kvm_msi_via_irqfd_allowed;
 bool kvm_gsi_routing_allowed;
+bool kvm_allowed;
 
 static const KVMCapabilityInfo kvm_required_capabilites[] = {
     KVM_CAP_INFO(USER_MEMORY),
kvm-stub.c +1 -0
index b228378..b2c8f9b 100644
@@ -25,6 +25,7 @@ bool kvm_async_interrupts_allowed;
 bool kvm_irqfds_allowed;
 bool kvm_msi_via_irqfd_allowed;
 bool kvm_gsi_routing_allowed;
+bool kvm_allowed;
 
 int kvm_init_vcpu(CPUState *cpu)
 {
qom/cpu.c +4 -0
index e242dcb..0c76712 100644
@@ -20,6 +20,7 @@
 
 #include "qom/cpu.h"
 #include "qemu-common.h"
+#include "sysemu/kvm.h"
 
 void cpu_reset_interrupt(CPUState *cpu, int mask)
 {
@@ -57,6 +58,9 @@ static ObjectClass *cpu_common_class_by_name(const char *cpu_model)
 
 static void cpu_common_realizefn(DeviceState *dev, Error **errp)
 {
+    if (dev->hotplugged) {
+        cpu_synchronize_post_init(CPU(dev));
+    }
 }
 
 static void cpu_class_init(ObjectClass *klass, void *data)
vl.c +0 -1
index 6caa5f4..5612c33 100644
@@ -267,7 +267,6 @@ static NotifierList machine_init_done_notifiers =
     NOTIFIER_LIST_INITIALIZER(machine_init_done_notifiers);
 
 static bool tcg_allowed = true;
-bool kvm_allowed;
 bool xen_allowed;
 uint32_t xen_domid;
 enum xen_mode xen_mode = XEN_EMULATE;
Signature · 1 line
1.7.1
#4 lore raw

Re: [Qemu-devel] [PATCH 02/21] cpu: call cpu_synchronize_post_init() from CPUClass.realize() if hotplugged

Am 23.04.2013 10:29, schrieb Igor Mammedov:

Quoted 7 lines

... to synchronize CPU state to KVM

---
v2:
* linking kvm-stub.o to *-user target moved in separate patch

Thanks, applied to qom-cpu:
https://github.com/afaerber/qemu-cpu/commits/qom-cpu

Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#5 lore raw

Re: [Qemu-devel] [PATCH 02/21] cpu: call cpu_synchronize_post_init() from CPUClass.realize() if hotplugged

Am 23.04.2013 17:59, schrieb Andreas Färber:

Quoted 11 lines

Am 23.04.2013 10:29, schrieb Igor Mammedov:

Quoted 7 lines

... to synchronize CPU state to KVM

---
v2:
* linking kvm-stub.o to *-user target moved in separate patch

Thanks, applied to qom-cpu:
https://github.com/afaerber/qemu-cpu/commits/qom-cpu

To facilitate conflict resolution for future patches, I have modified this to match other functions in there and to spare refactorings:

qom/cpu.c +0 -0
index 0c76712..66f7c00 100644
@@ -58,8 +58,10 @@ static ObjectClass *cpu_common_class_by_name(const

char *cpu_model)

static void cpu_common_realizefn(DeviceState *dev, Error **errp) {
+ CPUState *cpu = CPU(dev);
+

     if (dev->hotplugged) {
-        cpu_synchronize_post_init(CPU(dev));
+        cpu_synchronize_post_init(cpu);
     }

}

Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#6 lore raw

Re: [Qemu-devel] [PATCH 02/21] cpu: call cpu_synchronize_post_init() from CPUClass.realize() if hotplugged

Quoted 23 lines · On Wed, 24 Apr 2013 14:08:45 +0200 Andreas Färber <[email protected]> wrote:

Am 23.04.2013 17:59, schrieb Andreas Färber:

Quoted 11 lines

Am 23.04.2013 10:29, schrieb Igor Mammedov:

Quoted 7 lines

... to synchronize CPU state to KVM

---
v2:
* linking kvm-stub.o to *-user target moved in separate patch

Thanks, applied to qom-cpu:
https://github.com/afaerber/qemu-cpu/commits/qom-cpu

To facilitate conflict resolution for future patches, I have modified this to match other functions in there and to spare refactorings:

qom/cpu.c +0 -0
index 0c76712..66f7c00 100644
@@ -58,8 +58,10 @@ static ObjectClass *cpu_common_class_by_name(const
 char *cpu_model)

qom/cpu.c static ObjectClass *cpu_common_class_by_name(const · line 67

Quoted 10 lines
qom/cpu.c +3 -1
 
 static void cpu_common_realizefn(DeviceState *dev, Error **errp)
 {
+    CPUState *cpu = CPU(dev);
+
     if (dev->hotplugged) {
-        cpu_synchronize_post_init(CPU(dev));
+        cpu_synchronize_post_init(cpu);
     }
 }

Fixed conflicts in 4-5/21,
available at https://github.com/imammedo/qemu/commits/cpu_set.WIP

Quoted 2 lines

Andreas

#7 lore raw

PATCH v5 3/21 introduce cpu_resume(), for single CPU

In reply to Igor Mammedov

also add stub for it, to make possible to use it in qom/cpu.c, which is shared with user emulators.

---
v2:
* rename resume_vcpu() to cpu_resume()
* remove comment about neccessity to call cpu_synchronize_post_init() which is applicable only to CPU creation. But not to generic CPU resume.

Note:
it will be re-used from qom/cpu.c by hotpluged CPU

---
 cpus.c              |   11 ++++++++---
 include/qom/cpu.h   |    7 +++++++
 stubs/Makefile.objs |    1 +
 stubs/cpus.c        |    5 +++++
 4 files changed, 21 insertions(+), 3 deletions(-)
 create mode 100644 stubs/cpus.c
cpus.c +8 -3
index 5a98a37..1d88761 100644
@@ -993,6 +993,13 @@ void pause_all_vcpus(void)
     }
 }
 
+void cpu_resume(CPUState *cpu)
+{
+    cpu->stop = false;
+    cpu->stopped = false;
+    qemu_cpu_kick(cpu);
+}
+
 void resume_all_vcpus(void)
 {
     CPUArchState *penv = first_cpu;
@@ -1000,9 +1007,7 @@ void resume_all_vcpus(void)
     qemu_clock_enable(vm_clock, true);
     while (penv) {
         CPUState *pcpu = ENV_GET_CPU(penv);
-        pcpu->stop = false;
-        pcpu->stopped = false;
-        qemu_cpu_kick(pcpu);
+        cpu_resume(pcpu);
         penv = penv->next_cpu;
     }
 }
include/qom/cpu.h +7 -0
index 3664a1b..d6a0e80 100644
@@ -256,5 +256,12 @@ void cpu_interrupt(CPUState *cpu, int mask);
  */
 void cpu_reset_interrupt(CPUState *cpu, int mask);
 
+/**
+ * cpu_resume:
+ * @cpu: The CPU to resume.
+ *
+ * Resumes CPU, i.e. puts CPU into runable state.
+ */
+void cpu_resume(CPUState *cpu);
 
 #endif
stubs/Makefile.objs +1 -0
index 9c55b34..03dff20 100644
@@ -23,3 +23,4 @@ stub-obj-y += sysbus.o
 stub-obj-y += vm-stop.o
 stub-obj-y += vmstate.o
 stub-obj-$(CONFIG_WIN32) += fd-register.o
+stub-obj-y += cpus.o
stubs/cpus.c new file +5 -0
new file mode 100644
index 0000000..37000dd
@@ -0,0 +1,5 @@
+#include "qom/cpu.h"
+
+void cpu_resume(CPUState *cpu)
+{
+}
Signature · 1 line
1.7.1
#8 lore raw

Re: [Qemu-devel] [PATCH 03/21] introduce cpu_resume(), for single CPU

Am 23.04.2013 10:29, schrieb Igor Mammedov:

Quoted 10 lines

also add stub for it, to make possible to use it in qom/cpu.c, which is shared with user emulators.

---
v2:
* rename resume_vcpu() to cpu_resume()
* remove comment about neccessity to call cpu_synchronize_post_init() which is applicable only to CPU creation. But not to generic CPU resume.

Thanks, applied to qom-cpu (with a minor change below):
https://github.com/afaerber/qemu-cpu/commits/qom-cpu

Quoted 45 lines

Note:
it will be re-used from qom/cpu.c by hotpluged CPU

---
 cpus.c              |   11 ++++++++---
 include/qom/cpu.h   |    7 +++++++
 stubs/Makefile.objs |    1 +
 stubs/cpus.c        |    5 +++++
 4 files changed, 21 insertions(+), 3 deletions(-)
 create mode 100644 stubs/cpus.c
cpus.c +8 -3
index 5a98a37..1d88761 100644
@@ -993,6 +993,13 @@ void pause_all_vcpus(void)
     }
 }
 
+void cpu_resume(CPUState *cpu)
+{
+    cpu->stop = false;
+    cpu->stopped = false;
+    qemu_cpu_kick(cpu);
+}
+
 void resume_all_vcpus(void)
 {
     CPUArchState *penv = first_cpu;
@@ -1000,9 +1007,7 @@ void resume_all_vcpus(void)
     qemu_clock_enable(vm_clock, true);
     while (penv) {
         CPUState *pcpu = ENV_GET_CPU(penv);
-        pcpu->stop = false;
-        pcpu->stopped = false;
-        qemu_cpu_kick(pcpu);
+        cpu_resume(pcpu);
         penv = penv->next_cpu;
     }
 }
include/qom/cpu.h +0 -0
index 3664a1b..d6a0e80 100644

include/qom/cpu.h void cpu_interrupt(CPUState *cpu, int mask); · line 263

Quoted 8 lines
include/qom/cpu.h +5 -0
@@ -256,5 +256,12 @@ void cpu_interrupt(CPUState *cpu, int mask);
  */
 void cpu_reset_interrupt(CPUState *cpu, int mask);
 
+/**
+ * cpu_resume:
+ * @cpu: The CPU to resume.
+ *
+ * Resumes CPU, i.e. puts CPU into runable state.

"runnable"

include/qom/cpu.h void cpu_interrupt(CPUState *cpu, int mask); · line 267

Quoted 4 lines
include/qom/cpu.h +2 -0
+ */
+void cpu_resume(CPUState *cpu);
 
 #endif

I note that you're dropping a white line, but okay.

Quoted 15 lines
stubs/Makefile.objs +1 -0
index 9c55b34..03dff20 100644
@@ -23,3 +23,4 @@ stub-obj-y += sysbus.o
 stub-obj-y += vm-stop.o
 stub-obj-y += vmstate.o
 stub-obj-$(CONFIG_WIN32) += fd-register.o
+stub-obj-y += cpus.o
stubs/cpus.c new file +0 -0
new file mode 100644
index 0000000..37000dd

stubs/cpus.c line 5

Quoted 5 lines
stubs/cpus.c new file +5 -0
@@ -0,0 +1,5 @@
+#include "qom/cpu.h"
+
+void cpu_resume(CPUState *cpu)
+{
+}

Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#9 lore raw

PATCH v5 4/21 cpu: resume CPU from CPUClass.cpu_common_realizefn() when it is hot-plugged

In reply to Igor Mammedov

---
v2:
s/resume_vcpu/cpu_resume/
v2:
* remove unnecessary now sysemu/cpus.h header
* move out introducing resume_vcpu() into a separate patch

---
 qom/cpu.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)
qom/cpu.c +1 -0
index 0c76712..abb20d0 100644
@@ -60,6 +60,7 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
 {
     if (dev->hotplugged) {
         cpu_synchronize_post_init(CPU(dev));
+        cpu_resume(CPU(dev));
     }
 }
 
Signature · 1 line
1.7.1
#10 lore raw

Re: [Qemu-devel] [PATCH 04/21] cpu: resume CPU from CPUClass.cpu_common_realizefn() when it is hot-plugged

Am 23.04.2013 10:29, schrieb Igor Mammedov:

Quoted 10 lines

---
v2:
s/resume_vcpu/cpu_resume/
v2:
* remove unnecessary now sysemu/cpus.h header
* move out introducing resume_vcpu() into a separate patch

---
 qom/cpu.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

Thanks, applied to qom-cpu (rebased on introduction of cpu variable and aligning commit messages wrt realize):

https://github.com/afaerber/qemu-cpu/commits/qom-cpu

qom/cpu.c static void cpu_common_realizefn(DeviceState *dev, Error **errp) · line 63

Quoted 9 lines
qom/cpu.c +1 -0
index 0c76712..abb20d0 100644
@@ -60,6 +60,7 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
 {
     if (dev->hotplugged) {
         cpu_synchronize_post_init(CPU(dev));
+        cpu_resume(CPU(dev));

Having introduced CPUState *cpu in a preceding patch, we can avoid the duplicate CPU() cast here.

qom/cpu.c static void cpu_common_realizefn(DeviceState *dev, Error **errp) · line 66

Quoted 3 lines
qom/cpu.c +0 -0
     }
 }
 

Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#11 lore raw

PATCH v5 5/21 introduce CPU hot-plug notifier

In reply to Igor Mammedov

hot-add CPU event will be distributed to acpi_piix4 and rtc_cmos

---
v3:
* call notifier right before CPU becomes runable
v2:
* move notifier to qom/cpu.c and call it from CPUClass.realize() on hotplug
* remove get_firmware_id() since it belong to other patch

---
 include/sysemu/sysemu.h |    3 +++
 qom/cpu.c               |   12 ++++++++++++
 2 files changed, 15 insertions(+), 0 deletions(-)
include/sysemu/sysemu.h +3 -0
index 6578782..a8c3de1 100644
@@ -152,6 +152,9 @@ void do_pci_device_hot_remove(Monitor *mon, const QDict *qdict);
 /* generic hotplug */
 void drive_hot_add(Monitor *mon, const QDict *qdict);
 
+/* CPU hotplug */
+void qemu_register_cpu_added_notifier(Notifier *notifier);
+
 /* pcie aer error injection */
 void pcie_aer_inject_error_print(Monitor *mon, const QObject *data);
 int do_pcie_aer_inject_error(Monitor *mon,
qom/cpu.c +12 -0
index abb20d0..9f78114 100644
@@ -21,6 +21,17 @@
 #include "qom/cpu.h"
 #include "qemu-common.h"
 #include "sysemu/kvm.h"
+#include "qemu/notify.h"
+#include "sysemu/sysemu.h"
+
+/* CPU hot-plug notifiers */
+static NotifierList cpu_added_notifiers =
+    NOTIFIER_LIST_INITIALIZER(cpu_add_notifiers);
+
+void qemu_register_cpu_added_notifier(Notifier *notifier)
+{
+    notifier_list_add(&cpu_added_notifiers, notifier);
+}
 
 void cpu_reset_interrupt(CPUState *cpu, int mask)
 {
@@ -60,6 +71,7 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
 {
     if (dev->hotplugged) {
         cpu_synchronize_post_init(CPU(dev));
+        notifier_list_notify(&cpu_added_notifiers, dev);
         cpu_resume(CPU(dev));
     }
 }
Signature · 1 line
1.7.1
#12 lore raw

Re: [Qemu-devel] [PATCH 05/21] introduce CPU hot-plug notifier

Am 23.04.2013 10:29, schrieb Igor Mammedov:

Quoted 9 lines

hot-add CPU event will be distributed to acpi_piix4 and rtc_cmos

---
v3:
* call notifier right before CPU becomes runable
v2:
* move notifier to qom/cpu.c and call it from CPUClass.realize() on hotplug
* remove get_firmware_id() since it belong to other patch

Quoted 4 lines
---
 include/sysemu/sysemu.h |    3 +++
 qom/cpu.c               |   12 ++++++++++++
 2 files changed, 15 insertions(+), 0 deletions(-)

Thanks, applied to qom-cpu:
https://github.com/afaerber/qemu-cpu/commits/qom-cpu

One comment below...

include/sysemu/sysemu.h void do_pci_device_hot_remove(Monitor *mon, const QDict *qdict); · line 156

Quoted 11 lines
include/sysemu/sysemu.h +2 -0
index 6578782..a8c3de1 100644
@@ -152,6 +152,9 @@ void do_pci_device_hot_remove(Monitor *mon, const QDict *qdict);
 /* generic hotplug */
 void drive_hot_add(Monitor *mon, const QDict *qdict);
 
+/* CPU hotplug */
+void qemu_register_cpu_added_notifier(Notifier *notifier);

Would be nice to turn this into a gtk-doc comment explaining when the notification is fired, as a follow-up.

Andreas

Quoted 35 lines
include/sysemu/sysemu.h +1 -0
+
 /* pcie aer error injection */
 void pcie_aer_inject_error_print(Monitor *mon, const QObject *data);
 int do_pcie_aer_inject_error(Monitor *mon,
qom/cpu.c +12 -0
index abb20d0..9f78114 100644
@@ -21,6 +21,17 @@
 #include "qom/cpu.h"
 #include "qemu-common.h"
 #include "sysemu/kvm.h"
+#include "qemu/notify.h"
+#include "sysemu/sysemu.h"
+
+/* CPU hot-plug notifiers */
+static NotifierList cpu_added_notifiers =
+    NOTIFIER_LIST_INITIALIZER(cpu_add_notifiers);
+
+void qemu_register_cpu_added_notifier(Notifier *notifier)
+{
+    notifier_list_add(&cpu_added_notifiers, notifier);
+}
 
 void cpu_reset_interrupt(CPUState *cpu, int mask)
 {
@@ -60,6 +71,7 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
 {
     if (dev->hotplugged) {
         cpu_synchronize_post_init(CPU(dev));
+        notifier_list_notify(&cpu_added_notifiers, dev);
         cpu_resume(CPU(dev));
     }
 }
Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#13 lore raw

PATCH v5 6/21 target-i386: pc: update rtc_cmos on CPU hot-plug

In reply to Igor Mammedov

it provides updated currently available CPUs count to BIOS on reboot

---
v2:
* reorder fields in rtc_cpu_hotplug_arg, make cpu_added_notifier first.
* rebase on top of "mc146818rtc: QOM'ify" and use QOM style cast

---
 hw/i386/pc.c                   |   20 ++++++++++++++++++++
 hw/timer/mc146818rtc.c         |    7 +++++++
 include/hw/timer/mc146818rtc.h |    1 +
 3 files changed, 28 insertions(+), 0 deletions(-)
hw/i386/pc.c +20 -0
index 0d6e72b..bbc3bea 100644
@@ -338,6 +338,21 @@ static void pc_cmos_init_late(void *opaque)
     qemu_unregister_reset(pc_cmos_init_late, opaque);
 }
 
+typedef struct rtc_cpu_hotplug_arg {
+    Notifier cpu_added_notifier;
+    ISADevice *rtc_state;
+} rtc_cpu_hotplug_arg;
+
+static void rtc_notify_cpu_added(Notifier *notifier, void *data)
+{
+    rtc_cpu_hotplug_arg *arg = container_of(notifier, rtc_cpu_hotplug_arg,
+                                           cpu_added_notifier);
+    ISADevice *s = arg->rtc_state;
+
+    /* increment the number of CPUs */
+    rtc_set_memory(s, 0x5f, rtc_get_memory(s, 0x5f) + 1);
+}
+
 void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
                   const char *boot_device,
                   ISADevice *floppy, BusState *idebus0, BusState *idebus1,
@@ -346,6 +361,7 @@ void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
     int val, nb, i;
     FDriveType fd_type[2] = { FDRIVE_DRV_NONE, FDRIVE_DRV_NONE };
     static pc_cmos_init_late_arg arg;
+    static rtc_cpu_hotplug_arg cpu_hotplug_cb;
 
     /* various important CMOS locations needed by PC/Bochs bios */
 
@@ -384,6 +400,10 @@ void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
 
     /* set the number of CPU */
     rtc_set_memory(s, 0x5f, smp_cpus - 1);
+    /* init CPU hotplug notifier */
+    cpu_hotplug_cb.rtc_state = s;
+    cpu_hotplug_cb.cpu_added_notifier.notify = rtc_notify_cpu_added;
+    qemu_register_cpu_added_notifier(&cpu_hotplug_cb.cpu_added_notifier);
 
     /* set boot devices, and disable floppy signature check if requested */
     if (set_boot_dev(s, boot_device, fd_bootchk)) {
hw/timer/mc146818rtc.c +7 -0
index afbd0db..481604d 100644
@@ -680,6 +680,13 @@ void rtc_set_memory(ISADevice *dev, int addr, int val)
         s->cmos_data[addr] = val;
 }
 
+int rtc_get_memory(ISADevice *dev, int addr)
+{
+    RTCState *s = MC146818_RTC(dev);
+    assert(addr >= 0 && addr <= 127);
+    return s->cmos_data[addr];
+}
+
 static void rtc_set_date_from_host(ISADevice *dev)
 {
     RTCState *s = MC146818_RTC(dev);
include/hw/timer/mc146818rtc.h +1 -0
index 425bd17..753dda6 100644
@@ -8,6 +8,7 @@
 
 ISADevice *rtc_init(ISABus *bus, int base_year, qemu_irq intercept_irq);
 void rtc_set_memory(ISADevice *dev, int addr, int val);
+int rtc_get_memory(ISADevice *dev, int addr);
 void rtc_set_date(ISADevice *dev, const struct tm *tm);
 
 #endif /* !MC146818RTC_H */
Signature · 1 line
1.7.1
#14 lore raw

Re: [Qemu-devel] [PATCH 06/21] target-i386: pc: update rtc_cmos on CPU hot-plug

Am 23.04.2013 10:29, schrieb Igor Mammedov:

Quoted 18 lines

it provides updated currently available CPUs count to BIOS on reboot

---
v2:
* reorder fields in rtc_cpu_hotplug_arg, make cpu_added_notifier first.
* rebase on top of "mc146818rtc: QOM'ify" and use QOM style cast

---
 hw/i386/pc.c                   |   20 ++++++++++++++++++++
 hw/timer/mc146818rtc.c         |    7 +++++++
 include/hw/timer/mc146818rtc.h |    1 +
 3 files changed, 28 insertions(+), 0 deletions(-)
hw/i386/pc.c +0 -0
index 0d6e72b..bbc3bea 100644

hw/i386/pc.c static void pc_cmos_init_late(void *opaque) · line 344

Quoted 7 lines
hw/i386/pc.c +4 -0
@@ -338,6 +338,21 @@ static void pc_cmos_init_late(void *opaque)
     qemu_unregister_reset(pc_cmos_init_late, opaque);
 }
 
+typedef struct rtc_cpu_hotplug_arg {
+    Notifier cpu_added_notifier;
+    ISADevice *rtc_state;
+} rtc_cpu_hotplug_arg;

RTCCPUHotplugArg?

hw/i386/pc.c static void pc_cmos_init_late(void *opaque) · line 349

Quoted 5 lines
hw/i386/pc.c +5 -0
+
+static void rtc_notify_cpu_added(Notifier *notifier, void *data)
+{
+    rtc_cpu_hotplug_arg *arg = container_of(notifier, rtc_cpu_hotplug_arg,
+                                           cpu_added_notifier);

Off-by-one (could fix myself)

Rest looks good.

Andreas

Quoted 59 lines
hw/i386/pc.c +11 -0
+    ISADevice *s = arg->rtc_state;
+
+    /* increment the number of CPUs */
+    rtc_set_memory(s, 0x5f, rtc_get_memory(s, 0x5f) + 1);
+}
+
 void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
                   const char *boot_device,
                   ISADevice *floppy, BusState *idebus0, BusState *idebus1,
@@ -346,6 +361,7 @@ void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
     int val, nb, i;
     FDriveType fd_type[2] = { FDRIVE_DRV_NONE, FDRIVE_DRV_NONE };
     static pc_cmos_init_late_arg arg;
+    static rtc_cpu_hotplug_arg cpu_hotplug_cb;
 
     /* various important CMOS locations needed by PC/Bochs bios */
 
@@ -384,6 +400,10 @@ void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
 
     /* set the number of CPU */
     rtc_set_memory(s, 0x5f, smp_cpus - 1);
+    /* init CPU hotplug notifier */
+    cpu_hotplug_cb.rtc_state = s;
+    cpu_hotplug_cb.cpu_added_notifier.notify = rtc_notify_cpu_added;
+    qemu_register_cpu_added_notifier(&cpu_hotplug_cb.cpu_added_notifier);
 
     /* set boot devices, and disable floppy signature check if requested */
     if (set_boot_dev(s, boot_device, fd_bootchk)) {
hw/timer/mc146818rtc.c +7 -0
index afbd0db..481604d 100644
@@ -680,6 +680,13 @@ void rtc_set_memory(ISADevice *dev, int addr, int val)
         s->cmos_data[addr] = val;
 }
 
+int rtc_get_memory(ISADevice *dev, int addr)
+{
+    RTCState *s = MC146818_RTC(dev);
+    assert(addr >= 0 && addr <= 127);
+    return s->cmos_data[addr];
+}
+
 static void rtc_set_date_from_host(ISADevice *dev)
 {
     RTCState *s = MC146818_RTC(dev);
include/hw/timer/mc146818rtc.h +1 -0
index 425bd17..753dda6 100644
@@ -8,6 +8,7 @@
 
 ISADevice *rtc_init(ISABus *bus, int base_year, qemu_irq intercept_irq);
 void rtc_set_memory(ISADevice *dev, int addr, int val);
+int rtc_get_memory(ISADevice *dev, int addr);
 void rtc_set_date(ISADevice *dev, const struct tm *tm);
 
 #endif /* !MC146818RTC_H */
Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#15 lore raw

Re: [Qemu-devel] [PATCH 06/21] target-i386: pc: update rtc_cmos on CPU hot-plug

Am 24.04.2013 19:03, schrieb Andreas Färber:

Quoted 22 lines

Am 23.04.2013 10:29, schrieb Igor Mammedov:

Quoted 21 lines

it provides updated currently available CPUs count to BIOS on reboot

---
v2:
* reorder fields in rtc_cpu_hotplug_arg, make cpu_added_notifier first.
* rebase on top of "mc146818rtc: QOM'ify" and use QOM style cast

---
 hw/i386/pc.c                   |   20 ++++++++++++++++++++
 hw/timer/mc146818rtc.c         |    7 +++++++
 include/hw/timer/mc146818rtc.h |    1 +
 3 files changed, 28 insertions(+), 0 deletions(-)
hw/i386/pc.c +0 -0
index 0d6e72b..bbc3bea 100644
@@ -338,6 +338,21 @@ static void pc_cmos_init_late(void *opaque)
     qemu_unregister_reset(pc_cmos_init_late, opaque);
 }
 
Quoted 6 lines

hw/i386/pc.c static void pc_cmos_init_late(void *opaque) · line 344

Quoted 4 lines
hw/i386/pc.c +4 -0
+typedef struct rtc_cpu_hotplug_arg {
+    Notifier cpu_added_notifier;
+    ISADevice *rtc_state;
+} rtc_cpu_hotplug_arg;

RTCCPUHotplugArg?

Igor agreed to this variant on IRC.

Thanks, applied to qom-cpu:
https://github.com/afaerber/qemu-cpu/commits/qom-cpu

Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#16 lore raw

PATCH v5 7/21 cpu: introduce get_arch_id() method and override it for target-i386

In reply to Igor Mammedov

get_arch_id() adds possibility for generic code to get guest visible CPU ID without accessing CPUArchState. If target doesn't override it, it will return cpu_index.

Override it on target-i386 to return APIC ID.

---
* it will be used later by new cpu_exists() generic function and acpi_piix.
* s/cpu_firmware_id/cpu_arch_id/

---
 include/qom/cpu.h |    2 ++
 qom/cpu.c         |    6 ++++++
 target-i386/cpu.c |   10 ++++++++++
 3 files changed, 18 insertions(+), 0 deletions(-)
include/qom/cpu.h +2 -0
index d6a0e80..639b436 100644
@@ -45,6 +45,7 @@ typedef struct CPUState CPUState;
  * instantiatable CPU type.
  * @reset: Callback to reset the #CPUState to its initial state.
  * @do_interrupt: Callback for interrupt handling.
+ * @get_arch_id: Callback for getting architecture depended CPU ID
  * @vmsd: State description for migration.
  *
  * Represents a CPU family or model.
@@ -58,6 +59,7 @@ typedef struct CPUClass {
 
     void (*reset)(CPUState *cpu);
     void (*do_interrupt)(CPUState *cpu);
+    int64_t (*get_arch_id)(CPUState *cpu);
 
     const struct VMStateDescription *vmsd;
 } CPUClass;
qom/cpu.c +6 -0
index 9f78114..d44f987 100644
@@ -76,6 +76,11 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
     }
 }
 
+static int64_t cpu_common_get_arch_id(CPUState *cpu)
+{
+    return cpu->cpu_index;
+}
+
 static void cpu_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
@@ -83,6 +88,7 @@ static void cpu_class_init(ObjectClass *klass, void *data)
 
     k->class_by_name = cpu_common_class_by_name;
     k->reset = cpu_common_reset;
+    k->get_arch_id = cpu_common_get_arch_id;
     dc->realize = cpu_common_realizefn;
     dc->no_user = 1;
 }
target-i386/cpu.c +10 -0
index e2302d8..a415fa3 100644
@@ -2272,6 +2272,14 @@ static void x86_cpu_initfn(Object *obj)
     }
 }
 
+static int64_t x86_cpu_get_arch_id(CPUState *cpu)
+{
+    X86CPU *x86cpu = X86_CPU(cpu);
+    CPUX86State *env = &x86cpu->env;
+
+    return env->cpuid_apic_id;
+}
+
 static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 {
     X86CPUClass *xcc = X86_CPU_CLASS(oc);
@@ -2286,6 +2294,8 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 
     cc->do_interrupt = x86_cpu_do_interrupt;
     cpu_class_set_vmsd(cc, &vmstate_x86_cpu);
+
+    cc->get_arch_id = x86_cpu_get_arch_id;
 }
 
 static const TypeInfo x86_cpu_type_info = {
Signature · 1 line
1.7.1
#17 lore raw

Re: [Qemu-devel] [PATCH 07/21] cpu: introduce get_arch_id() method and override it for target-i386

Am 23.04.2013 10:29, schrieb Igor Mammedov:

Quoted 14 lines

get_arch_id() adds possibility for generic code to get guest visible CPU ID without accessing CPUArchState. If target doesn't override it, it will return cpu_index.

Override it on target-i386 to return APIC ID.

---
* it will be used later by new cpu_exists() generic function and acpi_piix.
* s/cpu_firmware_id/cpu_arch_id/

Quoted 5 lines
---
 include/qom/cpu.h |    2 ++
 qom/cpu.c         |    6 ++++++
 target-i386/cpu.c |   10 ++++++++++
 3 files changed, 18 insertions(+), 0 deletions(-)

Thanks, applied to qom-cpu (with two minor changes below and rewording wrt target vs. QOM class):

https://github.com/afaerber/qemu-cpu/commits/qom-cpu

include/qom/cpu.h typedef struct CPUState CPUState; · line 48

Quoted 10 lines
include/qom/cpu.h +1 -0
index d6a0e80..639b436 100644
@@ -45,6 +45,7 @@ typedef struct CPUState CPUState;
  * instantiatable CPU type.
  * @reset: Callback to reset the #CPUState to its initial state.
  * @do_interrupt: Callback for interrupt handling.
+ * @get_arch_id: Callback for getting architecture depended CPU ID

"dependent"

Quoted 3 lines
include/qom/cpu.h +0 -0
  * @vmsd: State description for migration.
  *
  * Represents a CPU family or model.

[...]

Quoted 9 lines
target-i386/cpu.c +1 -0
index e2302d8..a415fa3 100644
@@ -2272,6 +2272,14 @@ static void x86_cpu_initfn(Object *obj)
     }
 }
 
+static int64_t x86_cpu_get_arch_id(CPUState *cpu)

target-i386/cpu.c static void x86_cpu_initfn(Object *obj) · line 2285

Quoted 10 lines
target-i386/cpu.c +7 -0
+{
+    X86CPU *x86cpu = X86_CPU(cpu);
+    CPUX86State *env = &x86cpu->env;
+
+    return env->cpuid_apic_id;
+}
+
 static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 {
     X86CPUClass *xcc = X86_CPU_CLASS(oc);

[snip]

To stay consistent in our X86CPU naming conventions:

target-i386/cpu.c +3 -3
index a415fa3..f34ba23 100644
@@ -2272,10 +2272,10 @@ static void x86_cpu_initfn(Object *obj)
     }
 }
 
-static int64_t x86_cpu_get_arch_id(CPUState *cpu)
+static int64_t x86_cpu_get_arch_id(CPUState *cs)
 {
-    X86CPU *x86cpu = X86_CPU(cpu);
-    CPUX86State *env = &x86cpu->env;
+    X86CPU *cpu = X86_CPU(cs);
+    CPUX86State *env = &cpu->env;
 
     return env->cpuid_apic_id;
 }

Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#18 lore raw

PATCH v5 8/21 exec: add qemu_for_each_cpu

In reply to Igor Mammedov

wrapper will help to remove open-coded loops

---
Note:
Will be used by ACPI table generation and cpu_exists()

---
 cpus.c            |   13 +++++++------
 exec.c            |   10 ++++++++++
 include/qom/cpu.h |    8 ++++++++
 3 files changed, 25 insertions(+), 6 deletions(-)
cpus.c +7 -6
index 1d88761..5850151 100644
@@ -812,6 +812,12 @@ static void *qemu_dummy_cpu_thread_fn(void *arg)
 
 static void tcg_exec_all(void);
 
+static void signal_cpu_creation(CPUState *cpu, void *data)
+{
+    cpu->thread_id = qemu_get_thread_id();
+    cpu->created = true;
+}
+
 static void *qemu_tcg_cpu_thread_fn(void *arg)
 {
     CPUState *cpu = arg;
@@ -820,13 +826,8 @@ static void *qemu_tcg_cpu_thread_fn(void *arg)
     qemu_tcg_init_cpu_signals();
     qemu_thread_get_self(cpu->thread);
 
-    /* signal CPU creation */
     qemu_mutex_lock(&qemu_global_mutex);
-    for (env = first_cpu; env != NULL; env = env->next_cpu) {
-        cpu = ENV_GET_CPU(env);
-        cpu->thread_id = qemu_get_thread_id();
-        cpu->created = true;
-    }
+    qemu_for_each_cpu(signal_cpu_creation, NULL);
     qemu_cond_signal(&qemu_cpu_cond);
 
     /* wait for initial kick-off after machine start */
exec.c +10 -0
index fa1e0c3..19725db 100644
@@ -265,6 +265,16 @@ CPUState *qemu_get_cpu(int index)
     return env ? cpu : NULL;
 }
 
+void qemu_for_each_cpu(void (*func)(CPUState *cpu, void *data), void *data)
+{
+    CPUArchState *env = first_cpu;
+
+    while (env) {
+        func(ENV_GET_CPU(env), data);
+        env = env->next_cpu;
+    }
+}
+
 void cpu_exec_init(CPUArchState *env)
 {
     CPUState *cpu = ENV_GET_CPU(env);
include/qom/cpu.h +8 -0
index 639b436..d4a21f4 100644
@@ -215,6 +215,14 @@ bool cpu_is_stopped(CPUState *cpu);
  */
 void run_on_cpu(CPUState *cpu, void (*func)(void *data), void *data);
 
+/** qemu_for_each_cpu:
+ * @func: The function to be executed.
+ * @data: Data to pass to the function.
+ *
+ * Executes @func on all CPUs
+ */
+void qemu_for_each_cpu(void (*func)(CPUState *cpu, void *data), void *data);
+
 /**
  * qemu_get_cpu:
  * @index: The CPUState@cpu_index value of the CPU to obtain.
Signature · 1 line
1.7.1
#19 lore raw

Re: [Qemu-devel] [PATCH 08/21] exec: add qemu_for_each_cpu

Am 23.04.2013 10:29, schrieb Igor Mammedov:

Quoted 4 lines

wrapper will help to remove open-coded loops

Quoted 8 lines

---
Note:
Will be used by ACPI table generation and cpu_exists()

---
 cpus.c            |   13 +++++++------
 exec.c            |   10 ++++++++++
 include/qom/cpu.h |    8 ++++++++
 3 files changed, 25 insertions(+), 6 deletions(-)

Thanks, split in two and applied to qom-cpu (with changes below):
https://github.com/afaerber/qemu-cpu/commits/qom-cpu

Quoted 23 lines
cpus.c +6 -1
index 1d88761..5850151 100644
@@ -812,6 +812,12 @@ static void *qemu_dummy_cpu_thread_fn(void *arg)
 
 static void tcg_exec_all(void);
 
+static void signal_cpu_creation(CPUState *cpu, void *data)
+{
+    cpu->thread_id = qemu_get_thread_id();
+    cpu->created = true;
+}
+
 static void *qemu_tcg_cpu_thread_fn(void *arg)
 {
     CPUState *cpu = arg;
@@ -820,13 +826,8 @@ static void *qemu_tcg_cpu_thread_fn(void *arg)
     qemu_tcg_init_cpu_signals();
     qemu_thread_get_self(cpu->thread);
 
-    /* signal CPU creation */

cpus.c static void *qemu_tcg_cpu_thread_fn(void *arg) · line 833

Quoted 10 lines
cpus.c +1 -5
     qemu_mutex_lock(&qemu_global_mutex);
-    for (env = first_cpu; env != NULL; env = env->next_cpu) {
-        cpu = ENV_GET_CPU(env);
-        cpu->thread_id = qemu_get_thread_id();
-        cpu->created = true;
-    }
+    qemu_for_each_cpu(signal_cpu_creation, NULL);
     qemu_cond_signal(&qemu_cpu_cond);
 
     /* wait for initial kick-off after machine start */

We figured on IRC that TCG was not reflected in the function name, so I changed as follows:

cpus.c +2 -2
index 5850151..2e7bbad 100644
@@ -812,7 +812,7 @@ static void *qemu_dummy_cpu_thread_fn(void *arg)
 
 static void tcg_exec_all(void);
 
-static void signal_cpu_creation(CPUState *cpu, void *data)
+static void signal_tcg_cpu_creation(CPUState *cpu, void *data)
 {
     cpu->thread_id = qemu_get_thread_id();
     cpu->created = true;
@@ -827,7 +827,7 @@ static void *qemu_tcg_cpu_thread_fn(void *arg)
     qemu_thread_get_self(cpu->thread);
 
     qemu_mutex_lock(&qemu_global_mutex);
-    qemu_for_each_cpu(signal_cpu_creation, NULL);
+    qemu_for_each_cpu(signal_tcg_cpu_creation, NULL);
     qemu_cond_signal(&qemu_cpu_cond);
 
     /* wait for initial kick-off after machine start */
Quoted 26 lines
exec.c +10 -0
index fa1e0c3..19725db 100644
@@ -265,6 +265,16 @@ CPUState *qemu_get_cpu(int index)
     return env ? cpu : NULL;
 }
 
+void qemu_for_each_cpu(void (*func)(CPUState *cpu, void *data), void *data)
+{
+    CPUArchState *env = first_cpu;
+
+    while (env) {
+        func(ENV_GET_CPU(env), data);
+        env = env->next_cpu;
+    }
+}
+
 void cpu_exec_init(CPUArchState *env)
 {
     CPUState *cpu = ENV_GET_CPU(env);
include/qom/cpu.h +0 -0
index 639b436..d4a21f4 100644

include/qom/cpu.h bool cpu_is_stopped(CPUState *cpu); · line 218

Quoted 4 lines
include/qom/cpu.h +1 -0
@@ -215,6 +215,14 @@ bool cpu_is_stopped(CPUState *cpu);
  */
 void run_on_cpu(CPUState *cpu, void (*func)(void *data), void *data);
 
+/** qemu_for_each_cpu:

Moved to next line.

include/qom/cpu.h bool cpu_is_stopped(CPUState *cpu); · line 222

Quoted 4 lines
include/qom/cpu.h +4 -0
+ * @func: The function to be executed.
+ * @data: Data to pass to the function.
+ *
+ * Executes @func on all CPUs

Changed to "for each CPU" to distinguish from run_on_cpu().

include/qom/cpu.h bool cpu_is_stopped(CPUState *cpu); · line 228

Quoted 6 lines
include/qom/cpu.h +3 -0
+ */
+void qemu_for_each_cpu(void (*func)(CPUState *cpu, void *data), void *data);
+
 /**
  * qemu_get_cpu:
  * @index: The CPUState@cpu_index value of the CPU to obtain.

Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#20 lore raw

PATCH v5 9/21 cpu: add helper cpu_exists(), to check if CPU with specified id exists

In reply to Igor Mammedov

---
v3:
* use qemu_for_each_cpu() instead of recursion
v2:
* s/get_firmware_id()/get_arch_id()/ rebase fixup
* remove check for get_arch_id being NULL, since it's always defined

---
 include/qom/cpu.h |   10 ++++++++++
 qom/cpu.c         |   26 ++++++++++++++++++++++++++
 2 files changed, 36 insertions(+), 0 deletions(-)
include/qom/cpu.h +10 -0
index d4a21f4..7ba65a5 100644
@@ -233,6 +233,16 @@ void qemu_for_each_cpu(void (*func)(CPUState *cpu, void *data), void *data);
  */
 CPUState *qemu_get_cpu(int index);
 
+/**
+ * cpu_exists:
+ * @id - guest exposed CPU ID to lookup
+ *
+ * Search for CPU with specified ID.
+ *
+ * Returns: true - CPU is found, false - CPU isn't found
+ */
+bool cpu_exists(int64_t id);
+
 #ifndef CONFIG_USER_ONLY
 
 typedef void (*CPUInterruptHandler)(CPUState *, int);
qom/cpu.c +26 -0
index d44f987..fcc34aa 100644
@@ -24,6 +24,32 @@
 #include "qemu/notify.h"
 #include "sysemu/sysemu.h"
 
+typedef struct CPU_exists_args {
+    int64_t id;
+    bool found;
+} CPU_exists_args;
+
+static void cpu_exist_cb(CPUState *cpu, void *data)
+{
+    CPUClass *klass = CPU_GET_CLASS(cpu);
+    CPU_exists_args *arg = data;
+
+    if (klass->get_arch_id(cpu) == arg->id) {
+        arg->found = true;
+    }
+}
+
+bool cpu_exists(int64_t id)
+{
+    CPU_exists_args data = {
+        .id = id,
+        .found = false,
+    };
+
+    qemu_for_each_cpu(cpu_exist_cb, &data);
+    return data.found;
+}
+
 /* CPU hot-plug notifiers */
 static NotifierList cpu_added_notifiers =
     NOTIFIER_LIST_INITIALIZER(cpu_add_notifiers);
Signature · 1 line
1.7.1
#21 lore raw

PATCH v5 10/21 acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

In reply to Igor Mammedov

* introduce processor status bitmask visible to guest at 0xaf00 addr, where ACPI asl code expects it
* set bit corresponding to APIC ID in processor status bitmask on receiving CPU hot-plug notification
* trigger CPU hot-plug SCI, to notify guest about CPU hot-plug event

---
v4:
* added spec for QEMU-Seabios interface
* added PIIX4_ prefix to PROC_ defines
v3:
* s/get_firmware_id()/get_arch_id()/ due rebase
* s/cpu_add_notifier/cpu_added_notifier/
v2:
* use CPUClass.get_firmware_id() to make code target independent
* bump up vmstate_acpi version

---
 docs/specs/acpi_cpu_hotplug.txt |   22 +++++++
 hw/acpi/piix4.c                 |  117 ++++++++++++++++++++++++++++++++++++++-
 2 files changed, 136 insertions(+), 3 deletions(-)
 create mode 100644 docs/specs/acpi_cpu_hotplug.txt
docs/specs/acpi_cpu_hotplug.txt new file +22 -0
new file mode 100644
index 0000000..5dec0c5
@@ -0,0 +1,22 @@
+QEMU<->ACPI BIOS CPU hotplug interface
+--------------------------------------
+
+QEMU supports CPU hotplug via ACPI. This document
+describes the interface between QEMU and the ACPI BIOS.
+
+ACPI GPE block (IO ports 0xafe0-0xafe3, byte access):
+-----------------------------------------
+
+Generic ACPI GPE block. Bit 2 (GPE.2) used to notify CPU
+hot-add/remove event to ACPI BIOS, via SCI interrupt.
+
+CPU present bitmap (IO port 0xaf00-0xae1f, 1-byte access):
+---------------------------------------------------------------
+One bit per CPU. Bit position reflects corresponding CPU APIC ID.
+Read-only.
+
+CPU hot-add/remove notification:
+-----------------------------------------------------
+QEMU sets/clears corresponding CPU bit on hot-add/remove event.
+CPU present map read by ACPI BIOS GPE.2 handler to notify OS of CPU
+hot-(un)plug events.
hw/acpi/piix4.c +114 -3

178 lines not on this page · raw message

Signature · 1 line
1.7.1
#22 lore raw

Re: [Qemu-devel] [PATCH 10/21] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

Quoted 6 lines · Igor Mammedov
hw/acpi/piix4.c +4 -0
+#define VMSTATE_CPU_STATUS_ARRAY(_field, _state)                             \
+ {                                                                           \
+     .name       = (stringify(_field)),                                      \
+     .version_id = 0,

\

this line should be:
.version_id = 4,

Quoted 22 lines
hw/acpi/piix4.c +9 -1
+     .num        = PIIX4_PROC_LEN,                                           \
+     .info       = &vmstate_info_uint8,                                      \
+     .size       = sizeof(uint8_t),                                          \
+     .flags      = VMS_ARRAY,                                                \
+     .offset     = vmstate_offset_array(_state, _field, uint8_t,             \
+                                        PIIX4_PROC_LEN),                     \
+ }
+
 /* qemu-kvm 1.2 uses version 3 but advertised as 2
  * To support incoming qemu-kvm 1.2 migration, change version_id
  * and minimum_version_id to 2 below (which breaks migration from
@@ -265,7 +289,7 @@ static int acpi_load_old(QEMUFile *f, void *opaque, int version_id)
  */
 static const VMStateDescription vmstate_acpi = {
     .name = "piix4_pm",
-    .version_id = 3,
+    .version_id = 4,
     .minimum_version_id = 3,
     .minimum_version_id_old = 1,
     .load_state_old = acpi_load_old,

hw/acpi/piix4.c static const VMStateDescription vmstate_acpi = { · line 308

Quoted 4 lines
hw/acpi/piix4.c +1 -0
@@ -281,6 +305,7 @@ static const VMStateDescription vmstate_acpi = {
         VMSTATE_STRUCT(ar.gpe, PIIX4PMState, 2, vmstate_gpe, ACPIGPE),
         VMSTATE_STRUCT(pci0_status, PIIX4PMState, 2, vmstate_pci_status,
                        struct pci_status),
+        VMSTATE_CPU_STATUS_ARRAY(gpe_cpu.sts, PIIX4PMState),

It is more, I think that simply:

VMSTATE_UINT8_ARRAY_V(gpu_cpu.sts, PIIX4PMstate, PIIX4_PROC_LEN, 4);

Should do the trick without the previous blob (it was needed for the old version because we have a uint32 data but we send a uint16 one).

Could you check? I don't have an easy way to test that it "receives" the right value.

Later, Juan.

Quoted 4 lines
hw/acpi/piix4.c +0 -0
         VMSTATE_END_OF_LIST()
     }
 };
@@ -585,6 +610,85 @@ static const MemoryRegionOps piix4_pci_ops = {
#23 lore raw

Re: [Qemu-devel] [PATCH 10/21] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

Quoted 51 lines · On Tue, 23 Apr 2013 13:38:10 +0200 Juan Quintela <[email protected]> wrote:
Quoted 9 lines · Igor Mammedov

+#define VMSTATE_CPU_STATUS_ARRAY(_field,
_state) \
+
{ \
+ .name =
(stringify(_field)), \
+ .version_id = 0,
\

this line should be:

   .version_id = 4, 
Quoted 28 lines

+ .num =
PIIX4_PROC_LEN, \
+ .info =
&vmstate_info_uint8, \
+ .size =
sizeof(uint8_t), \
+ .flags =
VMS_ARRAY, \
+ .offset = vmstate_offset_array(_state, _field,
uint8_t, \
+
PIIX4_PROC_LEN), \
+ }

hw/acpi/piix4.c +1 -0
+
 /* qemu-kvm 1.2 uses version 3 but advertised as 2
  * To support incoming qemu-kvm 1.2 migration, change version_id
  * and minimum_version_id to 2 below (which breaks migration from
@@ -265,7 +289,7 @@ static int acpi_load_old(QEMUFile *f, void *opaque,

int version_id) */

 static const VMStateDescription vmstate_acpi = {
     .name = "piix4_pm",
hw/acpi/piix4.c +1 -1
-    .version_id = 3,
+    .version_id = 4,
     .minimum_version_id = 3,
     .minimum_version_id_old = 1,
     .load_state_old = acpi_load_old,

hw/acpi/piix4.c static const VMStateDescription vmstate_acpi = { · line 308

Quoted 4 lines
hw/acpi/piix4.c +1 -0
@@ -281,6 +305,7 @@ static const VMStateDescription vmstate_acpi = {
         VMSTATE_STRUCT(ar.gpe, PIIX4PMState, 2, vmstate_gpe, ACPIGPE),
         VMSTATE_STRUCT(pci0_status, PIIX4PMState, 2, vmstate_pci_status,
                        struct pci_status),
+        VMSTATE_CPU_STATUS_ARRAY(gpe_cpu.sts, PIIX4PMState),

It is more, I think that simply:

Quoted 7 lines

VMSTATE_UINT8_ARRAY_V(gpu_cpu.sts, PIIX4PMstate, PIIX4_PROC_LEN, 4);

Should do the trick without the previous blob (it was needed for the old version because we have a uint32 data but we send a uint16 one).

Could you check? I don't have an easy way to test that it "receives" the right value.

Just checked, it works with VMSTATE_UINT8_ARRAY_V as well,

but I have a question, why version should be 4, looking at git history components of vmstate_acpi don't advance their version each time vmstate_acpi change, they do it only when they themselves change.

Quoted 8 lines

Later, Juan.

Quoted 4 lines
hw/acpi/piix4.c +0 -0
         VMSTATE_END_OF_LIST()
     }
 };
@@ -585,6 +610,85 @@ static const MemoryRegionOps piix4_pci_ops = {
#24 lore raw

Re: [Qemu-devel] [PATCH 10/21] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

Quoted 59 lines · Igor Mammedov
Quoted 56 lines · On Tue, 23 Apr 2013 13:38:10 +0200 Juan Quintela <[email protected]> wrote:
Quoted 9 lines · Igor Mammedov

+#define VMSTATE_CPU_STATUS_ARRAY(_field,
_state) \
+
{ \
+ .name =
(stringify(_field)), \
+ .version_id = 0,
\

this line should be:

   .version_id = 4, 
Quoted 28 lines

+ .num =
PIIX4_PROC_LEN, \
+ .info =
&vmstate_info_uint8, \
+ .size =
sizeof(uint8_t), \
+ .flags =
VMS_ARRAY, \
+ .offset = vmstate_offset_array(_state, _field,
uint8_t, \
+
PIIX4_PROC_LEN), \
+ }

hw/acpi/piix4.c +1 -0
+
 /* qemu-kvm 1.2 uses version 3 but advertised as 2
  * To support incoming qemu-kvm 1.2 migration, change version_id
  * and minimum_version_id to 2 below (which breaks migration from
@@ -265,7 +289,7 @@ static int acpi_load_old(QEMUFile *f, void *opaque,

int version_id) */

 static const VMStateDescription vmstate_acpi = {
     .name = "piix4_pm",
hw/acpi/piix4.c +1 -1
-    .version_id = 3,
+    .version_id = 4,
     .minimum_version_id = 3,
     .minimum_version_id_old = 1,
     .load_state_old = acpi_load_old,

hw/acpi/piix4.c static const VMStateDescription vmstate_acpi = { · line 308

Quoted 4 lines
hw/acpi/piix4.c +1 -0
@@ -281,6 +305,7 @@ static const VMStateDescription vmstate_acpi = {
         VMSTATE_STRUCT(ar.gpe, PIIX4PMState, 2, vmstate_gpe, ACPIGPE),
         VMSTATE_STRUCT(pci0_status, PIIX4PMState, 2, vmstate_pci_status,
                        struct pci_status),
+        VMSTATE_CPU_STATUS_ARRAY(gpe_cpu.sts, PIIX4PMState),

It is more, I think that simply:

VMSTATE_UINT8_ARRAY_V(gpu_cpu.sts, PIIX4PMstate, PIIX4_PROC_LEN, 4);

Should do the trick without the previous blob (it was needed for the old version because we have a uint32 data but we send a uint16 one).

Quoted 7 lines
Quoted 2 lines

Could you check? I don't have an easy way to test that it "receives" the right value.

Just checked, it works with VMSTATE_UINT8_ARRAY_V as well,

but I have a question, why version should be 4, looking at git history components of vmstate_acpi don't advance their version each time vmstate_acpi change, they do it only when they themselves change.

Generally changing version breaks cross version migration. So please don't do it for an optional
feature like CPU hotplug.

Quoted 8 lines
Quoted 8 lines

Later, Juan.

Quoted 4 lines
hw/acpi/piix4.c +0 -0
         VMSTATE_END_OF_LIST()
     }
 };
@@ -585,6 +610,85 @@ static const MemoryRegionOps piix4_pci_ops = {
#25 lore raw

Re: [Qemu-devel] [PATCH 10/21] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

Quoted 64 lines · On Tue, 23 Apr 2013 16:04:22 +0300 "Michael S. Tsirkin" <[email protected]> wrote:
Quoted 63 lines · Igor Mammedov
Quoted 51 lines · On Tue, 23 Apr 2013 13:38:10 +0200 Juan Quintela <[email protected]> wrote:
Quoted 9 lines · Igor Mammedov

+#define VMSTATE_CPU_STATUS_ARRAY(_field,
_state) \
+
{ \
+ .name =
(stringify(_field)), \
+ .version_id = 0,
\

this line should be:

   .version_id = 4, 
Quoted 28 lines

+ .num =
PIIX4_PROC_LEN, \
+ .info =
&vmstate_info_uint8, \
+ .size =
sizeof(uint8_t), \
+ .flags =
VMS_ARRAY, \
+ .offset = vmstate_offset_array(_state, _field,
uint8_t, \
+
PIIX4_PROC_LEN), \
+ }

hw/acpi/piix4.c +2 -1
+
 /* qemu-kvm 1.2 uses version 3 but advertised as 2
  * To support incoming qemu-kvm 1.2 migration, change version_id
  * and minimum_version_id to 2 below (which breaks migration from
@@ -265,7 +289,7 @@ static int acpi_load_old(QEMUFile *f, void
 *opaque, int version_id) */
 static const VMStateDescription vmstate_acpi = {
     .name = "piix4_pm",
-    .version_id = 3,
+    .version_id = 4,
     .minimum_version_id = 3,
     .minimum_version_id_old = 1,
     .load_state_old = acpi_load_old,

hw/acpi/piix4.c static const VMStateDescription vmstate_acpi = { · line 308

Quoted 4 lines
hw/acpi/piix4.c +1 -0
@@ -281,6 +305,7 @@ static const VMStateDescription vmstate_acpi = {
         VMSTATE_STRUCT(ar.gpe, PIIX4PMState, 2, vmstate_gpe,
 ACPIGPE), VMSTATE_STRUCT(pci0_status, PIIX4PMState, 2,
 vmstate_pci_status, struct pci_status),
+        VMSTATE_CPU_STATUS_ARRAY(gpe_cpu.sts, PIIX4PMState),

It is more, I think that simply:

Quoted 7 lines

VMSTATE_UINT8_ARRAY_V(gpu_cpu.sts, PIIX4PMstate, PIIX4_PROC_LEN, 4);

Should do the trick without the previous blob (it was needed for the old version because we have a uint32 data but we send a uint16 one).

Could you check? I don't have an easy way to test that it "receives" the right value.

Just checked, it works with VMSTATE_UINT8_ARRAY_V as well,

Quoted 7 lines
Quoted 3 lines

but I have a question, why version should be 4, looking at git history components of vmstate_acpi don't advance their version each time vmstate_acpi change, they do it only when they themselves change.

Generally changing version breaks cross version migration. So please don't do it for an optional
feature like CPU hotplug.

I probably wasn't clear enough, question was about why a new component VMSTATE_UINT8_ARRAY_V(gpu_cpu.sts, ...) should be version 4 and not 0?

As for advancing vmstate_acpi version, it was requested:
http://lists.gnu.org/archive/html/qemu-devel/2013-03/msg04775.html

Anyway why chatting on IRC with Juan, question arises:
Do we really need to save/restore gpe_cpu.sts field?

Since target has to be started with all CPUs (including hot-plugged), it will have the same gpe_cpu.sts bitmap.

Quoted 9 lines
Quoted 8 lines
Quoted 8 lines

Later, Juan.

Quoted 4 lines
hw/acpi/piix4.c +0 -0
         VMSTATE_END_OF_LIST()
     }
 };
@@ -585,6 +610,85 @@ static const MemoryRegionOps piix4_pci_ops = {
#26 lore raw

Re: [Qemu-devel] [PATCH 10/21] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

Quoted 71 lines · Igor Mammedov
Quoted 68 lines · On Tue, 23 Apr 2013 16:04:22 +0300 "Michael S. Tsirkin" <[email protected]> wrote:
Quoted 66 lines · Igor Mammedov
Quoted 51 lines · On Tue, 23 Apr 2013 13:38:10 +0200 Juan Quintela <[email protected]> wrote:
Quoted 9 lines · Igor Mammedov

+#define VMSTATE_CPU_STATUS_ARRAY(_field,
_state) \
+
{ \
+ .name =
(stringify(_field)), \
+ .version_id = 0,
\

this line should be:

   .version_id = 4, 
Quoted 28 lines

+ .num =
PIIX4_PROC_LEN, \
+ .info =
&vmstate_info_uint8, \
+ .size =
sizeof(uint8_t), \
+ .flags =
VMS_ARRAY, \
+ .offset = vmstate_offset_array(_state, _field,
uint8_t, \
+
PIIX4_PROC_LEN), \
+ }

hw/acpi/piix4.c +2 -1
+
 /* qemu-kvm 1.2 uses version 3 but advertised as 2
  * To support incoming qemu-kvm 1.2 migration, change version_id
  * and minimum_version_id to 2 below (which breaks migration from
@@ -265,7 +289,7 @@ static int acpi_load_old(QEMUFile *f, void
 *opaque, int version_id) */
 static const VMStateDescription vmstate_acpi = {
     .name = "piix4_pm",
-    .version_id = 3,
+    .version_id = 4,
     .minimum_version_id = 3,
     .minimum_version_id_old = 1,
     .load_state_old = acpi_load_old,

hw/acpi/piix4.c static const VMStateDescription vmstate_acpi = { · line 308

Quoted 4 lines
hw/acpi/piix4.c +1 -0
@@ -281,6 +305,7 @@ static const VMStateDescription vmstate_acpi = {
         VMSTATE_STRUCT(ar.gpe, PIIX4PMState, 2, vmstate_gpe,
 ACPIGPE), VMSTATE_STRUCT(pci0_status, PIIX4PMState, 2,
 vmstate_pci_status, struct pci_status),
+        VMSTATE_CPU_STATUS_ARRAY(gpe_cpu.sts, PIIX4PMState),

It is more, I think that simply:

Quoted 7 lines

VMSTATE_UINT8_ARRAY_V(gpu_cpu.sts, PIIX4PMstate, PIIX4_PROC_LEN, 4);

Should do the trick without the previous blob (it was needed for the old version because we have a uint32 data but we send a uint16 one).

Could you check? I don't have an easy way to test that it "receives" the right value.

Just checked, it works with VMSTATE_UINT8_ARRAY_V as well,

but I have a question, why version should be 4, looking at git history components of vmstate_acpi don't advance their version each time vmstate_acpi change, they do it only when they themselves change.

Quoted 8 lines
Quoted 3 lines

Generally changing version breaks cross version migration. So please don't do it for an optional
feature like CPU hotplug.

I probably wasn't clear enough, question was about why a new component VMSTATE_UINT8_ARRAY_V(gpu_cpu.sts, ...) should be version 4 and not 0?

As for advancing vmstate_acpi version, it was requested:
http://lists.gnu.org/archive/html/qemu-devel/2013-03/msg04775.html

It says
You need to either bump the version, or add a subsection. The
subsection is not needed until the first hot-(un)plug action.
please add a subsection, don't bump the version :)

Quoted 17 lines

Anyway why chatting on IRC with Juan, question arises:
Do we really need to save/restore gpe_cpu.sts field?

Since target has to be started with all CPUs (including hot-plugged), it will have the same gpe_cpu.sts bitmap.

Quoted 9 lines
Quoted 8 lines
Quoted 8 lines

Later, Juan.

Quoted 4 lines
hw/acpi/piix4.c +0 -0
         VMSTATE_END_OF_LIST()
     }
 };
@@ -585,6 +610,85 @@ static const MemoryRegionOps piix4_pci_ops = {
#27 lore raw

Re: [Qemu-devel] [PATCH 10/21] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

In reply to Igor Mammedov
Quoted 59 lines · Igor Mammedov
Quoted 56 lines · On Tue, 23 Apr 2013 13:38:10 +0200 Juan Quintela <[email protected]> wrote:
Quoted 9 lines · Igor Mammedov

+#define VMSTATE_CPU_STATUS_ARRAY(_field,
_state) \
+
{ \
+ .name =
(stringify(_field)), \
+ .version_id = 0,
\

this line should be:

   .version_id = 4, 
Quoted 28 lines

+ .num =
PIIX4_PROC_LEN, \
+ .info =
&vmstate_info_uint8, \
+ .size =
sizeof(uint8_t), \
+ .flags =
VMS_ARRAY, \
+ .offset = vmstate_offset_array(_state, _field,
uint8_t, \
+
PIIX4_PROC_LEN), \
+ }

hw/acpi/piix4.c +1 -0
+
 /* qemu-kvm 1.2 uses version 3 but advertised as 2
  * To support incoming qemu-kvm 1.2 migration, change version_id
  * and minimum_version_id to 2 below (which breaks migration from
@@ -265,7 +289,7 @@ static int acpi_load_old(QEMUFile *f, void *opaque,

int version_id) */

 static const VMStateDescription vmstate_acpi = {
     .name = "piix4_pm",
hw/acpi/piix4.c +1 -1
-    .version_id = 3,
+    .version_id = 4,
     .minimum_version_id = 3,
     .minimum_version_id_old = 1,
     .load_state_old = acpi_load_old,

hw/acpi/piix4.c static const VMStateDescription vmstate_acpi = { · line 308

Quoted 4 lines
hw/acpi/piix4.c +1 -0
@@ -281,6 +305,7 @@ static const VMStateDescription vmstate_acpi = {
         VMSTATE_STRUCT(ar.gpe, PIIX4PMState, 2, vmstate_gpe, ACPIGPE),
         VMSTATE_STRUCT(pci0_status, PIIX4PMState, 2, vmstate_pci_status,
                        struct pci_status),
+        VMSTATE_CPU_STATUS_ARRAY(gpe_cpu.sts, PIIX4PMState),

It is more, I think that simply:

VMSTATE_UINT8_ARRAY_V(gpu_cpu.sts, PIIX4PMstate, PIIX4_PROC_LEN, 4);

Should do the trick without the previous blob (it was needed for the old version because we have a uint32 data but we send a uint16 one).

Quoted 7 lines
Quoted 2 lines

Could you check? I don't have an easy way to test that it "receives" the right value.

Just checked, it works with VMSTATE_UINT8_ARRAY_V as well,

but I have a question, why version should be 4, looking at git history components of vmstate_acpi don't advance their version each time vmstate_acpi change, they do it only when they themselves change.

I didn't knew this was an optional field. Is this only needed for cpu hotplug, or said otherwise, when iti s needed? If so, I can try to put an optional section there.

Later, Juan.

Quoted 9 lines
Quoted 9 lines

Later, Juan.

Quoted 4 lines
hw/acpi/piix4.c +0 -0
         VMSTATE_END_OF_LIST()
     }
 };
@@ -585,6 +610,85 @@ static const MemoryRegionOps piix4_pci_ops = {
#28 lore raw

Re: [Qemu-devel] [PATCH 10/21] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

In reply to Igor Mammedov
Quoted 2 lines · Igor Mammedov

On Tue, 23 Apr 2013 13:38:10 +0200
Juan Quintela <[email protected]> wrote:

Something like this (on top of your series) should work.

It implements an optional subsection and is sent depending if we have used (or not) cpu hotplug. I just compiled it, haven't tested it, so it should be perfect (TM).

What do you think?

Thanks, Juan.

Quoted 1 line

From f887ce646f65a81ceff1920c7e6e2287b914b3ca Mon Sep 17 00:00:00 2001

From: Juan Quintela <[email protected]>
Date: Tue, 23 Apr 2013 17:23:29 +0200
Subject: [PATCH] vmstate subsection example

---
 hw/acpi/piix4.c | 42 +++++++++++++++++++++++++++++-------------
 1 file changed, 29 insertions(+), 13 deletions(-)
hw/acpi/piix4.c +29 -13
index b845123..14dba76 100644
@@ -60,6 +60,7 @@ struct pci_status {
 };
 
 struct cpu_status {
+    bool hotplug_happened;
     uint8_t sts[PIIX4_PROC_LEN];
 };
 
@@ -269,17 +270,23 @@ static int acpi_load_old(QEMUFile *f, void *opaque, int version_id)
     return ret;
 }
 
-#define VMSTATE_CPU_STATUS_ARRAY(_field, _state)                             \
- {                                                                           \
-     .name       = (stringify(_field)),                                      \
-     .version_id = 0,                                                        \
-     .num        = PIIX4_PROC_LEN,                                           \
-     .info       = &vmstate_info_uint8,                                      \
-     .size       = sizeof(uint8_t),                                          \
-     .flags      = VMS_ARRAY,                                                \
-     .offset     = vmstate_offset_array(_state, _field, uint8_t,             \
-                                        PIIX4_PROC_LEN),                     \
- }
+static bool vmstate_piix4_hotplug_needed(void *opaque)
+{
+    PIIX4PMState *s = opaque;
+
+    return s->gpe_cpu.hotplug_happened;
+}
+
+static const VMStateDescription vmstate_piix4_hotplug_state = {
+    .name ="piix4_pm/hotplug",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .minimum_version_id_old = 1,
+    .fields = (VMStateField []) {
+        VMSTATE_UINT8_ARRAY(gpe_cpu.sts, PIIX4PMState, PIIX4_PROC_LEN),
+        VMSTATE_END_OF_LIST()
+    }
+};
 
 /* qemu-kvm 1.2 uses version 3 but advertised as 2
  * To support incoming qemu-kvm 1.2 migration, change version_id
@@ -289,7 +296,7 @@ static int acpi_load_old(QEMUFile *f, void *opaque, int version_id)
  */
 static const VMStateDescription vmstate_acpi = {
     .name = "piix4_pm",
-    .version_id = 4,
+    .version_id = 3,
     .minimum_version_id = 3,
     .minimum_version_id_old = 1,
     .load_state_old = acpi_load_old,
@@ -305,8 +312,15 @@ static const VMStateDescription vmstate_acpi = {
         VMSTATE_STRUCT(ar.gpe, PIIX4PMState, 2, vmstate_gpe, ACPIGPE),
         VMSTATE_STRUCT(pci0_status, PIIX4PMState, 2, vmstate_pci_status,
                        struct pci_status),
-        VMSTATE_CPU_STATUS_ARRAY(gpe_cpu.sts, PIIX4PMState),
         VMSTATE_END_OF_LIST()
+    },
+    .subsections = (VMStateSubsection[]) {
+        {
+            .vmsd = &vmstate_piix4_hotplug_state,
+            .needed = vmstate_piix4_hotplug_needed,
+        }, {
+            /* empty */
+        }
     }
 };
 
@@ -664,6 +678,7 @@ static void piix4_cpu_added_req(Notifier *n, void *opaque)
 {
     PIIX4PMState *s = container_of(n, PIIX4PMState, cpu_added_notifier);
 
+    s->gpe_cpu.hotplug_happened = true;
     piix4_cpu_hotplug_req(s, CPU(opaque), PLUG);
 }
 
@@ -706,6 +721,7 @@ static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
     pci_bus_hotplug(bus, piix4_device_hotplug, &s->dev.qdev);
 
     piix4_init_cpu_status(qdev_get_machine(), &s->gpe_cpu);
+    s->gpe_cpu.hotplug_happened = false;
     memory_region_init_io(&s->io_cpu, &cpu_hotplug_ops, s, "apci-cpu-hotplug",
                           PIIX4_PROC_LEN);
     memory_region_add_subregion(parent, PIIX4_PROC_BASE, &s->io_cpu);
Signature · 1 line
1.8.1.4
#29 lore raw

Re: [Qemu-devel] [PATCH 10/21] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

Quoted 6 lines · On Tue, 23 Apr 2013 17:25:50 +0200 Juan Quintela <[email protected]> wrote:
Quoted 2 lines · Igor Mammedov

On Tue, 23 Apr 2013 13:38:10 +0200
Juan Quintela <[email protected]> wrote:

Something like this (on top of your series) should work.

Quoted 8 lines

It implements an optional subsection and is sent depending if we have used (or not) cpu hotplug. I just compiled it, haven't tested it, so it should be perfect (TM).

What do you think?

Thanks, Juan.

Just tested. it works.
I'm going to merge it in and remove unused Error* and repost patch.

Quoted 100 lines

From f887ce646f65a81ceff1920c7e6e2287b914b3ca Mon Sep 17 00:00:00 2001
From: Juan Quintela <[email protected]>
Date: Tue, 23 Apr 2013 17:23:29 +0200
Subject: [PATCH] vmstate subsection example

---
 hw/acpi/piix4.c | 42 +++++++++++++++++++++++++++++-------------
 1 file changed, 29 insertions(+), 13 deletions(-)
hw/acpi/piix4.c +29 -13
index b845123..14dba76 100644
@@ -60,6 +60,7 @@ struct pci_status {
 };
 
 struct cpu_status {
+    bool hotplug_happened;
     uint8_t sts[PIIX4_PROC_LEN];
 };
 
@@ -269,17 +270,23 @@ static int acpi_load_old(QEMUFile *f, void *opaque, int version_id)
     return ret;
 }
 
-#define VMSTATE_CPU_STATUS_ARRAY(_field, _state)                             \
- {                                                                           \
-     .name       = (stringify(_field)),                                      \
-     .version_id = 0,                                                        \
-     .num        = PIIX4_PROC_LEN,                                           \
-     .info       = &vmstate_info_uint8,                                      \
-     .size       = sizeof(uint8_t),                                          \
-     .flags      = VMS_ARRAY,                                                \
-     .offset     = vmstate_offset_array(_state, _field, uint8_t,             \
-                                        PIIX4_PROC_LEN),                     \
- }
+static bool vmstate_piix4_hotplug_needed(void *opaque)
+{
+    PIIX4PMState *s = opaque;
+
+    return s->gpe_cpu.hotplug_happened;
+}
+
+static const VMStateDescription vmstate_piix4_hotplug_state = {
+    .name ="piix4_pm/hotplug",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .minimum_version_id_old = 1,
+    .fields = (VMStateField []) {
+        VMSTATE_UINT8_ARRAY(gpe_cpu.sts, PIIX4PMState, PIIX4_PROC_LEN),
+        VMSTATE_END_OF_LIST()
+    }
+};
 
 /* qemu-kvm 1.2 uses version 3 but advertised as 2
  * To support incoming qemu-kvm 1.2 migration, change version_id
@@ -289,7 +296,7 @@ static int acpi_load_old(QEMUFile *f, void *opaque, int version_id)
  */
 static const VMStateDescription vmstate_acpi = {
     .name = "piix4_pm",
-    .version_id = 4,
+    .version_id = 3,
     .minimum_version_id = 3,
     .minimum_version_id_old = 1,
     .load_state_old = acpi_load_old,
@@ -305,8 +312,15 @@ static const VMStateDescription vmstate_acpi = {
         VMSTATE_STRUCT(ar.gpe, PIIX4PMState, 2, vmstate_gpe, ACPIGPE),
         VMSTATE_STRUCT(pci0_status, PIIX4PMState, 2, vmstate_pci_status,
                        struct pci_status),
-        VMSTATE_CPU_STATUS_ARRAY(gpe_cpu.sts, PIIX4PMState),
         VMSTATE_END_OF_LIST()
+    },
+    .subsections = (VMStateSubsection[]) {
+        {
+            .vmsd = &vmstate_piix4_hotplug_state,
+            .needed = vmstate_piix4_hotplug_needed,
+        }, {
+            /* empty */
+        }
     }
 };
 
@@ -664,6 +678,7 @@ static void piix4_cpu_added_req(Notifier *n, void *opaque)
 {
     PIIX4PMState *s = container_of(n, PIIX4PMState, cpu_added_notifier);
 
+    s->gpe_cpu.hotplug_happened = true;
     piix4_cpu_hotplug_req(s, CPU(opaque), PLUG);
 }
 
@@ -706,6 +721,7 @@ static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
     pci_bus_hotplug(bus, piix4_device_hotplug, &s->dev.qdev);
 
     piix4_init_cpu_status(qdev_get_machine(), &s->gpe_cpu);
+    s->gpe_cpu.hotplug_happened = false;
     memory_region_init_io(&s->io_cpu, &cpu_hotplug_ops, s, "apci-cpu-hotplug",
                           PIIX4_PROC_LEN);
     memory_region_add_subregion(parent, PIIX4_PROC_BASE, &s->io_cpu);
#30 lore raw

Re: [Qemu-devel] [PATCH 10/21] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

In reply to Igor Mammedov
Quoted 7 lines · Igor Mammedov

* introduce processor status bitmask visible to guest at 0xaf00 addr, where ACPI asl code expects it
* set bit corresponding to APIC ID in processor status bitmask on receiving CPU hot-plug notification
* trigger CPU hot-plug SCI, to notify guest about CPU hot-plug event

This is wrong (or at least supperfluous)

hw/acpi/piix4.c static const MemoryRegionOps piix4_pci_ops = { · line 676

Quoted 7 lines
hw/acpi/piix4.c +7 -0
+static int piix4_init_cpu_status(Object *obj, void *opaque)
+{
+    struct cpu_status *g = (struct cpu_status *)opaque;
+    Object *cpu_obj = object_dynamic_cast(obj, TYPE_CPU);
+
+    if (cpu_obj) {
+        struct Error *error = NULL;

we set error to NULL

hw/acpi/piix4.c static const MemoryRegionOps piix4_pci_ops = { · line 680

Quoted 4 lines
hw/acpi/piix4.c +4 -0
+        CPUClass *k = CPU_GET_CLASS(cpu_obj);
+        int64_t id = k->get_arch_id(CPU(cpu_obj));
+
+        if (error) {

and without touching error we test if it is != NULL.

something is missing here?

hw/acpi/piix4.c static const MemoryRegionOps piix4_pci_ops = { · line 684

Quoted 4 lines
hw/acpi/piix4.c +4 -0
+            fprintf(stderr, "failed to initilize CPU status for ACPI: %s\n",
+                    error_get_pretty(error));
+            error_free(error);
+            abort();

Can't we return an error code at this point? I guess no, but asking will not hurt.

Quoted 24 lines
hw/acpi/piix4.c +14 -0
+        }
+        g_assert((id / 8) < PIIX4_PROC_LEN);
+        g->sts[id / 8] |= (1 << (id % 8));
+    }
+    return object_child_foreach(obj, piix4_init_cpu_status, opaque);
+}
+
 static int piix4_device_hotplug(DeviceState *qdev, PCIDevice *dev,
                                 PCIHotplugState state);
@@ -600,6 +704,13 @@ static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
     memory_region_add_subregion(parent, PCI_HOTPLUG_ADDR,
                                 &s->io_pci);
     pci_bus_hotplug(bus, piix4_device_hotplug, &s->dev.qdev);
+
+    piix4_init_cpu_status(qdev_get_machine(), &s->gpe_cpu);
+    memory_region_init_io(&s->io_cpu, &cpu_hotplug_ops, s, "apci-cpu-hotplug",
+                          PIIX4_PROC_LEN);
+    memory_region_add_subregion(parent, PIIX4_PROC_BASE, &s->io_cpu);
+    s->cpu_added_notifier.notify = piix4_cpu_added_req;
+    qemu_register_cpu_added_notifier(&s->cpu_added_notifier);
 }
 
 static void enable_device(PIIX4PMState *s, int slot)
#31 lore raw

Re: [Qemu-devel] [PATCH 10/21] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

Quoted 26 lines · Juan Quintela
Quoted 7 lines · Igor Mammedov

* introduce processor status bitmask visible to guest at 0xaf00 addr, where ACPI asl code expects it
* set bit corresponding to APIC ID in processor status bitmask on receiving CPU hot-plug notification
* trigger CPU hot-plug SCI, to notify guest about CPU hot-plug event

This is wrong (or at least supperfluous)

hw/acpi/piix4.c static const MemoryRegionOps piix4_pci_ops = { · line 676

Quoted 7 lines
hw/acpi/piix4.c +7 -0
+static int piix4_init_cpu_status(Object *obj, void *opaque)
+{
+    struct cpu_status *g = (struct cpu_status *)opaque;
+    Object *cpu_obj = object_dynamic_cast(obj, TYPE_CPU);
+
+    if (cpu_obj) {
+        struct Error *error = NULL;

we set error to NULL

hw/acpi/piix4.c static const MemoryRegionOps piix4_pci_ops = { · line 680

Quoted 4 lines
hw/acpi/piix4.c +4 -0
+        CPUClass *k = CPU_GET_CLASS(cpu_obj);
+        int64_t id = k->get_arch_id(CPU(cpu_obj));
+
+        if (error) {
Quoted 4 lines

and without touching error we test if it is != NULL.

something is missing here?

Looks like a leftover from v1, that called object_property_get_int() (with an error argument) instead of k->get_arch_id().

Signature · 1 line
Eduardo
#32 lore raw

Re: [Qemu-devel] [PATCH 10/21] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

In reply to Juan Quintela
Quoted 26 lines · On Tue, 23 Apr 2013 15:43:51 +0200 Juan Quintela <[email protected]> wrote:
Quoted 7 lines · Igor Mammedov

* introduce processor status bitmask visible to guest at 0xaf00 addr, where ACPI asl code expects it
* set bit corresponding to APIC ID in processor status bitmask on receiving CPU hot-plug notification
* trigger CPU hot-plug SCI, to notify guest about CPU hot-plug event

This is wrong (or at least supperfluous)

hw/acpi/piix4.c static const MemoryRegionOps piix4_pci_ops = { · line 676

Quoted 7 lines
hw/acpi/piix4.c +7 -0
+static int piix4_init_cpu_status(Object *obj, void *opaque)
+{
+    struct cpu_status *g = (struct cpu_status *)opaque;
+    Object *cpu_obj = object_dynamic_cast(obj, TYPE_CPU);
+
+    if (cpu_obj) {
+        struct Error *error = NULL;

we set error to NULL

hw/acpi/piix4.c static const MemoryRegionOps piix4_pci_ops = { · line 680

Quoted 4 lines
hw/acpi/piix4.c +4 -0
+        CPUClass *k = CPU_GET_CLASS(cpu_obj);
+        int64_t id = k->get_arch_id(CPU(cpu_obj));
+
+        if (error) {
Quoted 4 lines

and without touching error we test if it is != NULL.

something is missing here?

It's leftover from previous versions, all error handing should in this function should be dropped, leaving only assert().

Quoted 36 lines

hw/acpi/piix4.c static const MemoryRegionOps piix4_pci_ops = { · line 684

Quoted 6 lines

+ fprintf(stderr, "failed to initilize CPU status for ACPI:
%s\n",

hw/acpi/piix4.c +3 -0
+                    error_get_pretty(error));
+            error_free(error);
+            abort();

Can't we return an error code at this point? I guess no, but asking will not hurt.

Quoted 25 lines
hw/acpi/piix4.c +14 -0
+        }
+        g_assert((id / 8) < PIIX4_PROC_LEN);
+        g->sts[id / 8] |= (1 << (id % 8));
+    }
+    return object_child_foreach(obj, piix4_init_cpu_status, opaque);
+}
+
 static int piix4_device_hotplug(DeviceState *qdev, PCIDevice *dev,
                                 PCIHotplugState state);
@@ -600,6 +704,13 @@ static void
 piix4_acpi_system_hot_add_init(MemoryRegion *parent,
 memory_region_add_subregion(parent, PCI_HOTPLUG_ADDR, &s->io_pci);
     pci_bus_hotplug(bus, piix4_device_hotplug, &s->dev.qdev);
+
+    piix4_init_cpu_status(qdev_get_machine(), &s->gpe_cpu);
+    memory_region_init_io(&s->io_cpu, &cpu_hotplug_ops, s,
 "apci-cpu-hotplug",
+                          PIIX4_PROC_LEN);
+    memory_region_add_subregion(parent, PIIX4_PROC_BASE, &s->io_cpu);
+    s->cpu_added_notifier.notify = piix4_cpu_added_req;
+    qemu_register_cpu_added_notifier(&s->cpu_added_notifier);
 }
 

static void enable_device(PIIX4PMState *s, int slot)

#33 lore raw

PATCH v6 10/21 acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

In reply to Igor Mammedov

* introduce processor status bitmask visible to guest at 0xaf00 addr, where ACPI asl code expects it
* set bit corresponding to APIC ID in processor status bitmask on receiving CPU hot-plug notification
* trigger CPU hot-plug SCI, to notify guest about CPU hot-plug event

---

v5: available at: https://github.com/imammedo/qemu/tree/cpu_set.WIP
* add optional vmstate subsection if there was CPU hotplug event
* remove unused Error*
* use qemu_for_each_cpu() instead of recursion over QOM tree
v4:
* added spec for QEMU-Seabios interface
* added PIIX4_ prefix to PROC_ defines
v3:
* s/get_firmware_id()/get_arch_id()/ due rebase
* s/cpu_add_notifier/cpu_added_notifier/
v2:
* use CPUClass.get_firmware_id() to make code target independent
* bump up vmstate_acpi version

---
 docs/specs/acpi_cpu_hotplug.txt |   22 +++++++
 hw/acpi/piix4.c                 |  119 ++++++++++++++++++++++++++++++++++++++-
 2 files changed, 139 insertions(+), 2 deletions(-)
 create mode 100644 docs/specs/acpi_cpu_hotplug.txt
docs/specs/acpi_cpu_hotplug.txt new file +22 -0
new file mode 100644
index 0000000..5dec0c5
@@ -0,0 +1,22 @@
+QEMU<->ACPI BIOS CPU hotplug interface
+--------------------------------------
+
+QEMU supports CPU hotplug via ACPI. This document
+describes the interface between QEMU and the ACPI BIOS.
+
+ACPI GPE block (IO ports 0xafe0-0xafe3, byte access):
+-----------------------------------------
+
+Generic ACPI GPE block. Bit 2 (GPE.2) used to notify CPU
+hot-add/remove event to ACPI BIOS, via SCI interrupt.
+
+CPU present bitmap (IO port 0xaf00-0xae1f, 1-byte access):
+---------------------------------------------------------------
+One bit per CPU. Bit position reflects corresponding CPU APIC ID.
+Read-only.
+
+CPU hot-add/remove notification:
+-----------------------------------------------------
+QEMU sets/clears corresponding CPU bit on hot-add/remove event.
+CPU present map read by ACPI BIOS GPE.2 handler to notify OS of CPU
+hot-(un)plug events.
hw/acpi/piix4.c +117 -2

174 lines not on this page · raw message

Signature · 1 line
1.7.1
author NAK
#34 lore raw

Re: [Qemu-devel] [PATCH 10/21 DISGISED v6] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

On Tue, 23 Apr 2013 18:27:11 +0200
Igor Mammedov <[email protected]> wrote:

NACK

gpe_cpu.sts isn't need to be migrated, since CPU hotpluging during migration just doesn't work, since destination QEMU has to be started with all present in guest CPUs (including hotplugged).
i.e. src-qemu -smp 2,max-cpus=4; cpu-add id=2; dst-qemu -smp 3,max-cpus=4 Destination QEMU will recreate the same gpe_cpu.sts=t'111' bitmap as on source by calling qemu_for_each_cpu(piix4_init_cpu_status, &s->gpe_cpu); since it has been started with 3 CPUs on command line.

tested on ping pong migration, It works as expected without gpe_cpu.sts migrated. I'll post v7 to this thread shortly.

Quoted 242 lines

* introduce processor status bitmask visible to guest at 0xaf00 addr, where ACPI asl code expects it
* set bit corresponding to APIC ID in processor status bitmask on receiving CPU hot-plug notification
* trigger CPU hot-plug SCI, to notify guest about CPU hot-plug event

---

v5: available at: https://github.com/imammedo/qemu/tree/cpu_set.WIP
* add optional vmstate subsection if there was CPU hotplug event
* remove unused Error*
* use qemu_for_each_cpu() instead of recursion over QOM tree
v4:
* added spec for QEMU-Seabios interface
* added PIIX4_ prefix to PROC_ defines
v3:
* s/get_firmware_id()/get_arch_id()/ due rebase
* s/cpu_add_notifier/cpu_added_notifier/
v2:
* use CPUClass.get_firmware_id() to make code target independent
* bump up vmstate_acpi version

---
 docs/specs/acpi_cpu_hotplug.txt |   22 +++++++
 hw/acpi/piix4.c                 |  119 ++++++++++++++++++++++++++++++++++++++-
 2 files changed, 139 insertions(+), 2 deletions(-)
 create mode 100644 docs/specs/acpi_cpu_hotplug.txt
docs/specs/acpi_cpu_hotplug.txt new file +22 -0
new file mode 100644
index 0000000..5dec0c5
@@ -0,0 +1,22 @@
+QEMU<->ACPI BIOS CPU hotplug interface
+--------------------------------------
+
+QEMU supports CPU hotplug via ACPI. This document
+describes the interface between QEMU and the ACPI BIOS.
+
+ACPI GPE block (IO ports 0xafe0-0xafe3, byte access):
+-----------------------------------------
+
+Generic ACPI GPE block. Bit 2 (GPE.2) used to notify CPU
+hot-add/remove event to ACPI BIOS, via SCI interrupt.
+
+CPU present bitmap (IO port 0xaf00-0xae1f, 1-byte access):
+---------------------------------------------------------------
+One bit per CPU. Bit position reflects corresponding CPU APIC ID.
+Read-only.
+
+CPU hot-add/remove notification:
+-----------------------------------------------------
+QEMU sets/clears corresponding CPU bit on hot-add/remove event.
+CPU present map read by ACPI BIOS GPE.2 handler to notify OS of CPU
+hot-(un)plug events.
hw/acpi/piix4.c +117 -2
index 88386d7..c29739e 100644
@@ -48,19 +48,29 @@
 #define PCI_EJ_BASE 0xae08
 #define PCI_RMV_BASE 0xae0c
 
+#define PIIX4_PROC_BASE 0xaf00
+#define PIIX4_PROC_LEN 32
+
 #define PIIX4_PCI_HOTPLUG_STATUS 2
+#define PIIX4_CPU_HOTPLUG_STATUS 4
 
 struct pci_status {
     uint32_t up; /* deprecated, maintained for migration compatibility */
     uint32_t down;
 };
 
+struct cpu_status {
+    bool hotplug_happened;
+    uint8_t sts[PIIX4_PROC_LEN];
+};
+
 typedef struct PIIX4PMState {
     PCIDevice dev;
 
     MemoryRegion io;
     MemoryRegion io_gpe;
     MemoryRegion io_pci;
+    MemoryRegion io_cpu;
     ACPIREGS ar;
 
     APMState apm;
@@ -82,6 +92,9 @@ typedef struct PIIX4PMState {
     uint8_t disable_s3;
     uint8_t disable_s4;
     uint8_t s4_val;
+
+    struct cpu_status gpe_cpu;
+    Notifier cpu_added_notifier;
 } PIIX4PMState;
 
 static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
@@ -100,8 +113,8 @@ static void pm_update_sci(PIIX4PMState *s)
                    ACPI_BITMASK_POWER_BUTTON_ENABLE |
                    ACPI_BITMASK_GLOBAL_LOCK_ENABLE |
                    ACPI_BITMASK_TIMER_ENABLE)) != 0) ||
-        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0])
-          & PIIX4_PCI_HOTPLUG_STATUS) != 0);
+        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0]) &
+          (PIIX4_PCI_HOTPLUG_STATUS | PIIX4_CPU_HOTPLUG_STATUS)) != 0);
 
     qemu_set_irq(s->irq, sci_level);
     /* schedule a timer interruption if needed */
@@ -257,6 +270,24 @@ static int acpi_load_old(QEMUFile *f, void *opaque, int version_id)
     return ret;
 }
 
+static bool vmstate_piix4_hotplug_needed(void *opaque)
+{
+    PIIX4PMState *s = opaque;
+
+    return s->gpe_cpu.hotplug_happened;
+}
+
+static const VMStateDescription vmstate_piix4_hotplug_state = {
+    .name = "piix4_pm/hotplug",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .minimum_version_id_old = 1,
+    .fields = (VMStateField[]) {
+        VMSTATE_UINT8_ARRAY(gpe_cpu.sts, PIIX4PMState, PIIX4_PROC_LEN),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 /* qemu-kvm 1.2 uses version 3 but advertised as 2
  * To support incoming qemu-kvm 1.2 migration, change version_id
  * and minimum_version_id to 2 below (which breaks migration from
@@ -282,6 +313,14 @@ static const VMStateDescription vmstate_acpi = {
         VMSTATE_STRUCT(pci0_status, PIIX4PMState, 2, vmstate_pci_status,
                        struct pci_status),
         VMSTATE_END_OF_LIST()
+    },
+    .subsections = (VMStateSubsection[]) {
+        {
+            .vmsd = &vmstate_piix4_hotplug_state,
+            .needed = vmstate_piix4_hotplug_needed,
+        }, {
+            /* empty */
+        }
     }
 };
 
@@ -585,6 +624,74 @@ static const MemoryRegionOps piix4_pci_ops = {
     },
 };
 
+static uint64_t cpu_status_read(void *opaque, hwaddr addr, unsigned width)
+{
+    PIIX4PMState *s = opaque;
+    struct cpu_status *cpus = &s->gpe_cpu;
+    uint64_t val = cpus->sts[addr];
+
+    return val;
+}
+
+static void cpu_status_write(void *opaque, hwaddr addr, uint64_t data,
+                             unsigned int size)
+{
+    /* TODO: implement VCPU removal on guest signal that CPU can be removed */
+}
+
+static const MemoryRegionOps cpu_hotplug_ops = {
+    .read = cpu_status_read,
+    .write = cpu_status_write,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+    .valid = {
+        .min_access_size = 1,
+        .max_access_size = 1,
+    },
+};
+
+typedef enum {
+    PLUG,
+    UNPLUG,
+} HotplugEventType;
+
+static void piix4_cpu_hotplug_req(PIIX4PMState *s, CPUState *cpu,
+                                  HotplugEventType action)
+{
+    struct cpu_status *g = &s->gpe_cpu;
+    ACPIGPE *gpe = &s->ar.gpe;
+    CPUClass *k = CPU_GET_CLASS(cpu);
+    int64_t cpu_id;
+
+    assert(s != NULL);
+
+    *gpe->sts = *gpe->sts | PIIX4_CPU_HOTPLUG_STATUS;
+    cpu_id = k->get_arch_id(CPU(cpu));
+    if (action == PLUG) {
+        g->sts[cpu_id / 8] |= (1 << (cpu_id % 8));
+    } else {
+        g->sts[cpu_id / 8] &= ~(1 << (cpu_id % 8));
+    }
+    pm_update_sci(s);
+}
+
+static void piix4_cpu_added_req(Notifier *n, void *opaque)
+{
+    PIIX4PMState *s = container_of(n, PIIX4PMState, cpu_added_notifier);
+
+    s->gpe_cpu.hotplug_happened = true;
+    piix4_cpu_hotplug_req(s, CPU(opaque), PLUG);
+}
+
+static void piix4_init_cpu_status(CPUState *cpu, void *data)
+{
+    struct cpu_status *g = (struct cpu_status *)data;
+    CPUClass *k = CPU_GET_CLASS(cpu);
+    int64_t id = k->get_arch_id(cpu);
+
+    g_assert((id / 8) < PIIX4_PROC_LEN);
+    g->sts[id / 8] |= (1 << (id % 8));
+}
+
 static int piix4_device_hotplug(DeviceState *qdev, PCIDevice *dev,
                                 PCIHotplugState state);
 
@@ -600,6 +707,14 @@ static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
     memory_region_add_subregion(parent, PCI_HOTPLUG_ADDR,
                                 &s->io_pci);
     pci_bus_hotplug(bus, piix4_device_hotplug, &s->dev.qdev);
+
+    qemu_for_each_cpu(piix4_init_cpu_status, &s->gpe_cpu);
+    s->gpe_cpu.hotplug_happened = false;
+    memory_region_init_io(&s->io_cpu, &cpu_hotplug_ops, s, "apci-cpu-hotplug",
+                          PIIX4_PROC_LEN);
+    memory_region_add_subregion(parent, PIIX4_PROC_BASE, &s->io_cpu);
+    s->cpu_added_notifier.notify = piix4_cpu_added_req;
+    qemu_register_cpu_added_notifier(&s->cpu_added_notifier);
 }
 
 static void enable_device(PIIX4PMState *s, int slot)
#35 lore raw

Re: [Qemu-devel] [PATCH 10/21 DISGISED v6] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

Quoted 14 lines · Igor Mammedov

On Tue, 23 Apr 2013 18:27:11 +0200
Igor Mammedov <[email protected]> wrote:

NACK

gpe_cpu.sts isn't need to be migrated, since CPU hotpluging during migration just doesn't work, since destination QEMU has to be started with all present in guest CPUs (including hotplugged).
i.e. src-qemu -smp 2,max-cpus=4; cpu-add id=2; dst-qemu -smp 3,max-cpus=4 Destination QEMU will recreate the same gpe_cpu.sts=t'111' bitmap as on source by calling qemu_for_each_cpu(piix4_init_cpu_status, &s->gpe_cpu); since it has been started with 3 CPUs on command line.

Quoted 2 lines

tested on ping pong migration, It works as expected without gpe_cpu.sts migrated. I'll post v7 to this thread shortly.

Are we going to make the cpu-add command fail in case it gets called during migration?

Signature · 1 line
Eduardo
#36 lore raw

Re: [Qemu-devel] [PATCH 10/21 DISGISED v6] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

Il 24/04/2013 18:03, Eduardo Habkost ha scritto:

Quoted 4 lines
Quoted 2 lines
Quoted 2 lines

tested on ping pong migration, It works as expected without gpe_cpu.sts migrated. I'll post v7 to this thread shortly.

Are we going to make the cpu-add command fail in case it gets called during migration?

It's the same as device-add. If the devices do not match for any reason, migration will fail when an unrecognized state is read from the source in the destination.

Paolo

#37 lore raw

Re: [Qemu-devel] [PATCH 10/21 DISGISED v6] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

In reply to Eduardo Habkost

Am 24.04.2013 18:03, schrieb Eduardo Habkost:

Quoted 15 lines · Igor Mammedov
Quoted 14 lines

On Tue, 23 Apr 2013 18:27:11 +0200
Igor Mammedov <[email protected]> wrote:

NACK

gpe_cpu.sts isn't need to be migrated, since CPU hotpluging during migration just doesn't work, since destination QEMU has to be started with all present in guest CPUs (including hotplugged).
i.e. src-qemu -smp 2,max-cpus=4; cpu-add id=2; dst-qemu -smp 3,max-cpus=4 Destination QEMU will recreate the same gpe_cpu.sts=t'111' bitmap as on source by calling qemu_for_each_cpu(piix4_init_cpu_status, &s->gpe_cpu); since it has been started with 3 CPUs on command line.

Quoted 5 lines
Quoted 2 lines

tested on ping pong migration, It works as expected without gpe_cpu.sts migrated. I'll post v7 to this thread shortly.

Are we going to make the cpu-add command fail in case it gets called during migration?

I would rather suggest to throw an Error* on realize, if this can be detected.

Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#38 lore raw

Re: [Qemu-devel] [PATCH 10/21 DISGISED v6] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

Quoted 16 lines · On Wed, 24 Apr 2013 18:09:11 +0200 Andreas Färber <[email protected]> wrote:

Am 24.04.2013 18:03, schrieb Eduardo Habkost:

Quoted 15 lines · Igor Mammedov
Quoted 14 lines

On Tue, 23 Apr 2013 18:27:11 +0200
Igor Mammedov <[email protected]> wrote:

NACK

gpe_cpu.sts isn't need to be migrated, since CPU hotpluging during migration just doesn't work, since destination QEMU has to be started with all present in guest CPUs (including hotplugged).
i.e. src-qemu -smp 2,max-cpus=4; cpu-add id=2; dst-qemu -smp 3,max-cpus=4 Destination QEMU will recreate the same gpe_cpu.sts=t'111' bitmap as on source by calling qemu_for_each_cpu(piix4_init_cpu_status, &s->gpe_cpu); since it has been started with 3 CPUs on command line.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

tested on ping pong migration, It works as expected without gpe_cpu.sts migrated. I'll post v7 to this thread shortly.

Are we going to make the cpu-add command fail in case it gets called during migration?

I would rather suggest to throw an Error* on realize, if this can be detected.

it would be better not to start/allow hot add at all if migration in progress and let it complete. it probably could be made generic???

Quoted 6 lines

Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
Signature · 2 lines
Regards,
  Igor
#39 lore raw

PATCH v7 8/19 acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

In reply to Igor Mammedov

* introduce processor status bitmask visible to guest at 0xaf00 addr, where ACPI asl code expects it
* set bit corresponding to APIC ID in processor status bitmask on receiving CPU hot-plug notification
* trigger CPU hot-plug SCI, to notify guest about CPU hot-plug event

---
Note:
gpe_cpu.sts isn't need to be migrated, since CPU hotpluging during migration just doesn't work, since destination QEMU has to be started with all present in guest CPUs (including hotplugged).
i.e. src-qemu -smp 2,max-cpus=4; cpu-add id=2; dst-qemu -smp 3,max-cpus=4 Destination QEMU will recreate the same gpe_cpu.sts=t'111' bitmap as on source by calling qemu_for_each_cpu(piix4_init_cpu_status, &s->gpe_cpu); since it has been started with 3 CPUs on command line.

v6:
* drop gpe_cpu.sts migration hunks
v5:
* add optional vmstate subsection if there was CPU hotplug event
* remove unused Error*
* use qemu_for_each_cpu() instead of recursion over QOM tree
v4:
* added spec for QEMU-Seabios interface
* added PIIX4_ prefix to PROC_ defines
v3:
* s/get_firmware_id()/get_arch_id()/ due rebase
* s/cpu_add_notifier/cpu_added_notifier/
v2:
* use CPUClass.get_firmware_id() to make code target independent
* bump up vmstate_acpi version

---
 docs/specs/acpi_cpu_hotplug.txt |   22 +++++++++
 hw/acpi/piix4.c                 |   90 ++++++++++++++++++++++++++++++++++++++-
 2 files changed, 110 insertions(+), 2 deletions(-)
 create mode 100644 docs/specs/acpi_cpu_hotplug.txt
docs/specs/acpi_cpu_hotplug.txt new file +22 -0
new file mode 100644
index 0000000..5dec0c5
@@ -0,0 +1,22 @@
+QEMU<->ACPI BIOS CPU hotplug interface
+--------------------------------------
+
+QEMU supports CPU hotplug via ACPI. This document
+describes the interface between QEMU and the ACPI BIOS.
+
+ACPI GPE block (IO ports 0xafe0-0xafe3, byte access):
+-----------------------------------------
+
+Generic ACPI GPE block. Bit 2 (GPE.2) used to notify CPU
+hot-add/remove event to ACPI BIOS, via SCI interrupt.
+
+CPU present bitmap (IO port 0xaf00-0xae1f, 1-byte access):
+---------------------------------------------------------------
+One bit per CPU. Bit position reflects corresponding CPU APIC ID.
+Read-only.
+
+CPU hot-add/remove notification:
+-----------------------------------------------------
+QEMU sets/clears corresponding CPU bit on hot-add/remove event.
+CPU present map read by ACPI BIOS GPE.2 handler to notify OS of CPU
+hot-(un)plug events.
hw/acpi/piix4.c +88 -2

133 lines not on this page · raw message

Signature · 1 line
1.7.1
#40 lore raw

Re: [Qemu-devel] [PATCH 08/19 v7] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

Am 24.04.2013 17:58, schrieb Igor Mammedov:

Quoted 38 lines

* introduce processor status bitmask visible to guest at 0xaf00 addr, where ACPI asl code expects it
* set bit corresponding to APIC ID in processor status bitmask on receiving CPU hot-plug notification
* trigger CPU hot-plug SCI, to notify guest about CPU hot-plug event

---
Note:
gpe_cpu.sts isn't need to be migrated, since CPU hotpluging during migration just doesn't work, since destination QEMU has to be started with all present in guest CPUs (including hotplugged).
i.e. src-qemu -smp 2,max-cpus=4; cpu-add id=2; dst-qemu -smp 3,max-cpus=4 Destination QEMU will recreate the same gpe_cpu.sts=t'111' bitmap as on source by calling qemu_for_each_cpu(piix4_init_cpu_status, &s->gpe_cpu); since it has been started with 3 CPUs on command line.

v6:
* drop gpe_cpu.sts migration hunks
v5:
* add optional vmstate subsection if there was CPU hotplug event
* remove unused Error*
* use qemu_for_each_cpu() instead of recursion over QOM tree
v4:
* added spec for QEMU-Seabios interface
* added PIIX4_ prefix to PROC_ defines
v3:
* s/get_firmware_id()/get_arch_id()/ due rebase
* s/cpu_add_notifier/cpu_added_notifier/
v2:
* use CPUClass.get_firmware_id() to make code target independent
* bump up vmstate_acpi version

---
 docs/specs/acpi_cpu_hotplug.txt |   22 +++++++++
 hw/acpi/piix4.c                 |   90 ++++++++++++++++++++++++++++++++++++++-
 2 files changed, 110 insertions(+), 2 deletions(-)
 create mode 100644 docs/specs/acpi_cpu_hotplug.txt

[...]

Quoted 10 lines
hw/acpi/piix4.c +2 -0
index 88386d7..b18202c 100644
@@ -48,19 +48,28 @@
 #define PCI_EJ_BASE 0xae08
 #define PCI_RMV_BASE 0xae0c
 
+#define PIIX4_PROC_BASE 0xaf00
+#define PIIX4_PROC_LEN 32

hw/acpi/piix4.c line 62

Quoted 10 lines
hw/acpi/piix4.c +3 -0
+
 #define PIIX4_PCI_HOTPLUG_STATUS 2
+#define PIIX4_CPU_HOTPLUG_STATUS 4
 
 struct pci_status {
     uint32_t up; /* deprecated, maintained for migration compatibility */
     uint32_t down;
 };
 
+struct cpu_status {

I see that you're copying pci_status above, but Coding Style asks for CamelCase and typedef, so could we change this to CPUStatus or better PIIX4CPUStatus? (I counted 5 occurrences)

Andreas

Quoted 123 lines
hw/acpi/piix4.c +83 -2
+    uint8_t sts[PIIX4_PROC_LEN];
+};
+
 typedef struct PIIX4PMState {
     PCIDevice dev;
 
     MemoryRegion io;
     MemoryRegion io_gpe;
     MemoryRegion io_pci;
+    MemoryRegion io_cpu;
     ACPIREGS ar;
 
     APMState apm;
@@ -82,6 +91,9 @@ typedef struct PIIX4PMState {
     uint8_t disable_s3;
     uint8_t disable_s4;
     uint8_t s4_val;
+
+    struct cpu_status gpe_cpu;
+    Notifier cpu_added_notifier;
 } PIIX4PMState;
 
 static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
@@ -100,8 +112,8 @@ static void pm_update_sci(PIIX4PMState *s)
                    ACPI_BITMASK_POWER_BUTTON_ENABLE |
                    ACPI_BITMASK_GLOBAL_LOCK_ENABLE |
                    ACPI_BITMASK_TIMER_ENABLE)) != 0) ||
-        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0])
-          & PIIX4_PCI_HOTPLUG_STATUS) != 0);
+        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0]) &
+          (PIIX4_PCI_HOTPLUG_STATUS | PIIX4_CPU_HOTPLUG_STATUS)) != 0);
 
     qemu_set_irq(s->irq, sci_level);
     /* schedule a timer interruption if needed */
@@ -585,6 +597,73 @@ static const MemoryRegionOps piix4_pci_ops = {
     },
 };
 
+static uint64_t cpu_status_read(void *opaque, hwaddr addr, unsigned width)
+{
+    PIIX4PMState *s = opaque;
+    struct cpu_status *cpus = &s->gpe_cpu;
+    uint64_t val = cpus->sts[addr];
+
+    return val;
+}
+
+static void cpu_status_write(void *opaque, hwaddr addr, uint64_t data,
+                             unsigned int size)
+{
+    /* TODO: implement VCPU removal on guest signal that CPU can be removed */
+}
+
+static const MemoryRegionOps cpu_hotplug_ops = {
+    .read = cpu_status_read,
+    .write = cpu_status_write,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+    .valid = {
+        .min_access_size = 1,
+        .max_access_size = 1,
+    },
+};
+
+typedef enum {
+    PLUG,
+    UNPLUG,
+} HotplugEventType;
+
+static void piix4_cpu_hotplug_req(PIIX4PMState *s, CPUState *cpu,
+                                  HotplugEventType action)
+{
+    struct cpu_status *g = &s->gpe_cpu;
+    ACPIGPE *gpe = &s->ar.gpe;
+    CPUClass *k = CPU_GET_CLASS(cpu);
+    int64_t cpu_id;
+
+    assert(s != NULL);
+
+    *gpe->sts = *gpe->sts | PIIX4_CPU_HOTPLUG_STATUS;
+    cpu_id = k->get_arch_id(CPU(cpu));
+    if (action == PLUG) {
+        g->sts[cpu_id / 8] |= (1 << (cpu_id % 8));
+    } else {
+        g->sts[cpu_id / 8] &= ~(1 << (cpu_id % 8));
+    }
+    pm_update_sci(s);
+}
+
+static void piix4_cpu_added_req(Notifier *n, void *opaque)
+{
+    PIIX4PMState *s = container_of(n, PIIX4PMState, cpu_added_notifier);
+
+    piix4_cpu_hotplug_req(s, CPU(opaque), PLUG);
+}
+
+static void piix4_init_cpu_status(CPUState *cpu, void *data)
+{
+    struct cpu_status *g = (struct cpu_status *)data;
+    CPUClass *k = CPU_GET_CLASS(cpu);
+    int64_t id = k->get_arch_id(cpu);
+
+    g_assert((id / 8) < PIIX4_PROC_LEN);
+    g->sts[id / 8] |= (1 << (id % 8));
+}
+
 static int piix4_device_hotplug(DeviceState *qdev, PCIDevice *dev,
                                 PCIHotplugState state);
 
@@ -600,6 +679,13 @@ static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
     memory_region_add_subregion(parent, PCI_HOTPLUG_ADDR,
                                 &s->io_pci);
     pci_bus_hotplug(bus, piix4_device_hotplug, &s->dev.qdev);
+
+    qemu_for_each_cpu(piix4_init_cpu_status, &s->gpe_cpu);
+    memory_region_init_io(&s->io_cpu, &cpu_hotplug_ops, s, "apci-cpu-hotplug",
+                          PIIX4_PROC_LEN);
+    memory_region_add_subregion(parent, PIIX4_PROC_BASE, &s->io_cpu);
+    s->cpu_added_notifier.notify = piix4_cpu_added_req;
+    qemu_register_cpu_added_notifier(&s->cpu_added_notifier);
 }
 
 static void enable_device(PIIX4PMState *s, int slot)
Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#41 lore raw

Re: [Qemu-devel] [PATCH 08/19 v7] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

Quoted 59 lines · On Wed, 24 Apr 2013 18:06:51 +0200 Andreas Färber <[email protected]> wrote:

Am 24.04.2013 17:58, schrieb Igor Mammedov:

Quoted 38 lines

* introduce processor status bitmask visible to guest at 0xaf00 addr, where ACPI asl code expects it
* set bit corresponding to APIC ID in processor status bitmask on receiving CPU hot-plug notification
* trigger CPU hot-plug SCI, to notify guest about CPU hot-plug event

---
Note:
gpe_cpu.sts isn't need to be migrated, since CPU hotpluging during migration just doesn't work, since destination QEMU has to be started with all present in guest CPUs (including hotplugged).
i.e. src-qemu -smp 2,max-cpus=4; cpu-add id=2; dst-qemu -smp 3,max-cpus=4 Destination QEMU will recreate the same gpe_cpu.sts=t'111' bitmap as on source by calling qemu_for_each_cpu(piix4_init_cpu_status, &s->gpe_cpu); since it has been started with 3 CPUs on command line.

v6:
* drop gpe_cpu.sts migration hunks
v5:
* add optional vmstate subsection if there was CPU hotplug event
* remove unused Error*
* use qemu_for_each_cpu() instead of recursion over QOM tree
v4:
* added spec for QEMU-Seabios interface
* added PIIX4_ prefix to PROC_ defines
v3:
* s/get_firmware_id()/get_arch_id()/ due rebase
* s/cpu_add_notifier/cpu_added_notifier/
v2:
* use CPUClass.get_firmware_id() to make code target independent
* bump up vmstate_acpi version

---
 docs/specs/acpi_cpu_hotplug.txt |   22 +++++++++
 hw/acpi/piix4.c                 |   90 ++++++++++++++++++++++++++++++++++++++-
 2 files changed, 110 insertions(+), 2 deletions(-)
 create mode 100644 docs/specs/acpi_cpu_hotplug.txt

[...]

Quoted 19 lines
hw/acpi/piix4.c +4 -0
index 88386d7..b18202c 100644
@@ -48,19 +48,28 @@
 #define PCI_EJ_BASE 0xae08
 #define PCI_RMV_BASE 0xae0c
 
+#define PIIX4_PROC_BASE 0xaf00
+#define PIIX4_PROC_LEN 32
+
 #define PIIX4_PCI_HOTPLUG_STATUS 2
+#define PIIX4_CPU_HOTPLUG_STATUS 4
 
 struct pci_status {
     uint32_t up; /* deprecated, maintained for migration compatibility */
     uint32_t down;
 };
 
Quoted 5 lines

hw/acpi/piix4.c line 62

Quoted 1 line
hw/acpi/piix4.c +1 -0
+struct cpu_status {

I see that you're copying pci_status above, but Coding Style asks for CamelCase and typedef, so could we change this to CPUStatus or better PIIX4CPUStatus? (I counted 5 occurrences)

sure, I'll resend it with CPUStatus variant.

Quoted 131 lines

Andreas

Quoted 123 lines
hw/acpi/piix4.c +83 -2
+    uint8_t sts[PIIX4_PROC_LEN];
+};
+
 typedef struct PIIX4PMState {
     PCIDevice dev;
 
     MemoryRegion io;
     MemoryRegion io_gpe;
     MemoryRegion io_pci;
+    MemoryRegion io_cpu;
     ACPIREGS ar;
 
     APMState apm;
@@ -82,6 +91,9 @@ typedef struct PIIX4PMState {
     uint8_t disable_s3;
     uint8_t disable_s4;
     uint8_t s4_val;
+
+    struct cpu_status gpe_cpu;
+    Notifier cpu_added_notifier;
 } PIIX4PMState;
 
 static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
@@ -100,8 +112,8 @@ static void pm_update_sci(PIIX4PMState *s)
                    ACPI_BITMASK_POWER_BUTTON_ENABLE |
                    ACPI_BITMASK_GLOBAL_LOCK_ENABLE |
                    ACPI_BITMASK_TIMER_ENABLE)) != 0) ||
-        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0])
-          & PIIX4_PCI_HOTPLUG_STATUS) != 0);
+        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0]) &
+          (PIIX4_PCI_HOTPLUG_STATUS | PIIX4_CPU_HOTPLUG_STATUS)) != 0);
 
     qemu_set_irq(s->irq, sci_level);
     /* schedule a timer interruption if needed */
@@ -585,6 +597,73 @@ static const MemoryRegionOps piix4_pci_ops = {
     },
 };
 
+static uint64_t cpu_status_read(void *opaque, hwaddr addr, unsigned width)
+{
+    PIIX4PMState *s = opaque;
+    struct cpu_status *cpus = &s->gpe_cpu;
+    uint64_t val = cpus->sts[addr];
+
+    return val;
+}
+
+static void cpu_status_write(void *opaque, hwaddr addr, uint64_t data,
+                             unsigned int size)
+{
+    /* TODO: implement VCPU removal on guest signal that CPU can be removed */
+}
+
+static const MemoryRegionOps cpu_hotplug_ops = {
+    .read = cpu_status_read,
+    .write = cpu_status_write,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+    .valid = {
+        .min_access_size = 1,
+        .max_access_size = 1,
+    },
+};
+
+typedef enum {
+    PLUG,
+    UNPLUG,
+} HotplugEventType;
+
+static void piix4_cpu_hotplug_req(PIIX4PMState *s, CPUState *cpu,
+                                  HotplugEventType action)
+{
+    struct cpu_status *g = &s->gpe_cpu;
+    ACPIGPE *gpe = &s->ar.gpe;
+    CPUClass *k = CPU_GET_CLASS(cpu);
+    int64_t cpu_id;
+
+    assert(s != NULL);
+
+    *gpe->sts = *gpe->sts | PIIX4_CPU_HOTPLUG_STATUS;
+    cpu_id = k->get_arch_id(CPU(cpu));
+    if (action == PLUG) {
+        g->sts[cpu_id / 8] |= (1 << (cpu_id % 8));
+    } else {
+        g->sts[cpu_id / 8] &= ~(1 << (cpu_id % 8));
+    }
+    pm_update_sci(s);
+}
+
+static void piix4_cpu_added_req(Notifier *n, void *opaque)
+{
+    PIIX4PMState *s = container_of(n, PIIX4PMState, cpu_added_notifier);
+
+    piix4_cpu_hotplug_req(s, CPU(opaque), PLUG);
+}
+
+static void piix4_init_cpu_status(CPUState *cpu, void *data)
+{
+    struct cpu_status *g = (struct cpu_status *)data;
+    CPUClass *k = CPU_GET_CLASS(cpu);
+    int64_t id = k->get_arch_id(cpu);
+
+    g_assert((id / 8) < PIIX4_PROC_LEN);
+    g->sts[id / 8] |= (1 << (id % 8));
+}
+
 static int piix4_device_hotplug(DeviceState *qdev, PCIDevice *dev,
                                 PCIHotplugState state);
 
@@ -600,6 +679,13 @@ static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
     memory_region_add_subregion(parent, PCI_HOTPLUG_ADDR,
                                 &s->io_pci);
     pci_bus_hotplug(bus, piix4_device_hotplug, &s->dev.qdev);
+
+    qemu_for_each_cpu(piix4_init_cpu_status, &s->gpe_cpu);
+    memory_region_init_io(&s->io_cpu, &cpu_hotplug_ops, s, "apci-cpu-hotplug",
+                          PIIX4_PROC_LEN);
+    memory_region_add_subregion(parent, PIIX4_PROC_BASE, &s->io_cpu);
+    s->cpu_added_notifier.notify = piix4_cpu_added_req;
+    qemu_register_cpu_added_notifier(&s->cpu_added_notifier);
 }
 
 static void enable_device(PIIX4PMState *s, int slot)
Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
Signature · 2 lines
Regards,
  Igor
#42 lore raw

PATCH v8 10/21 acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

In reply to Igor Mammedov

* introduce processor status bitmask visible to guest at 0xaf00 addr, where ACPI asl code expects it
* set bit corresponding to APIC ID in processor status bitmask on receiving CPU hot-plug notification
* trigger CPU hot-plug SCI, to notify guest about CPU hot-plug event

---
Note:
gpe_cpu.sts isn't need to be migrated, since CPU hotpluging during migration just doesn't work, since destination QEMU has to be started with all present in guest CPUs (including hotplugged).
i.e. src-qemu -smp 2,max-cpus=4; cpu-add id=2; dst-qemu -smp 3,max-cpus=4 Destination QEMU will recreate the same gpe_cpu.sts=t'111' bitmap as on source by calling qemu_for_each_cpu(piix4_init_cpu_status, &s->gpe_cpu); since it has been started with 3 CPUs on command line.

v7:
* s/struct cpu_status/struct CPUStatus/
v6:
* drop gpe_cpu.sts migration hunks
v5:
* add optional vmstate subsection if there was CPU hotplug event
* remove unused Error*
* use qemu_for_each_cpu() instead of recursion over QOM tree
v4:
* added spec for QEMU-Seabios interface
* added PIIX4_ prefix to PROC_ defines
v3:
* s/get_firmware_id()/get_arch_id()/ due rebase
* s/cpu_add_notifier/cpu_added_notifier/
v2:
* use CPUClass.get_firmware_id() to make code target independent
* bump up vmstate_acpi version

---
 docs/specs/acpi_cpu_hotplug.txt | 22 ++++++++++
 hw/acpi/piix4.c                 | 90 ++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 110 insertions(+), 2 deletions(-)
 create mode 100644 docs/specs/acpi_cpu_hotplug.txt
docs/specs/acpi_cpu_hotplug.txt new file +22 -0
new file mode 100644
index 0000000..5dec0c5
@@ -0,0 +1,22 @@
+QEMU<->ACPI BIOS CPU hotplug interface
+--------------------------------------
+
+QEMU supports CPU hotplug via ACPI. This document
+describes the interface between QEMU and the ACPI BIOS.
+
+ACPI GPE block (IO ports 0xafe0-0xafe3, byte access):
+-----------------------------------------
+
+Generic ACPI GPE block. Bit 2 (GPE.2) used to notify CPU
+hot-add/remove event to ACPI BIOS, via SCI interrupt.
+
+CPU present bitmap (IO port 0xaf00-0xae1f, 1-byte access):
+---------------------------------------------------------------
+One bit per CPU. Bit position reflects corresponding CPU APIC ID.
+Read-only.
+
+CPU hot-add/remove notification:
+-----------------------------------------------------
+QEMU sets/clears corresponding CPU bit on hot-add/remove event.
+CPU present map read by ACPI BIOS GPE.2 handler to notify OS of CPU
+hot-(un)plug events.
hw/acpi/piix4.c +88 -2

133 lines not on this page · raw message

Signature · 1 line
1.8.2
#43 lore raw

PATCH v5 11/21 target-i386: introduce apic-id property

In reply to Igor Mammedov

The property is used from board level to set APIC ID for CPUs it creates. Do so in a new pc_new_cpu() helper, to be reused for hot-plug.

---
Note:
* pc_new_cpu() function will be reused later in CPU hot-plug hook.

v4:
* after switching to qemu_for_each_cpu() in cpu_exists(), first CPU becomes visible to cpu_exists() early and setting property fails, skip APIC ID check if value to be set is the same as the current.
* use error_propagate() in pc_new_cpu()
* return CPU from pc_new_cpu(). Moved from "move APIC to ICC bus" to reduce its size.
v3:
* user error_propagate() in property setter
v2:
* use generic cpu_exists() instead of custom one
* make apic-id dynamic property, so it won't be possible to use it as global property, since it's instance specific

---
 hw/i386/pc.c      |   29 ++++++++++++++++++++++++++++-
 target-i386/cpu.c |   42 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 70 insertions(+), 1 deletions(-)
hw/i386/pc.c +28 -1
index bbc3bea..30ec338 100644
@@ -890,9 +890,33 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level)
     }
 }
 
+static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
+{
+    X86CPU *cpu;
+    Error *local_err = NULL;
+
+    cpu = cpu_x86_create(cpu_model, errp);
+    if (!cpu) {
+        return cpu;
+    }
+
+    object_property_set_int(OBJECT(cpu), apic_id, "apic-id", &local_err);
+    object_property_set_bool(OBJECT(cpu), true, "realized", &local_err);
+
+    if (local_err) {
+        if (cpu != NULL) {
+            object_unref(OBJECT(cpu));
+            cpu = NULL;
+        }
+        error_propagate(errp, local_err);
+    }
+    return cpu;
+}
+
 void pc_cpus_init(const char *cpu_model)
 {
     int i;
+    Error *error = NULL;
 
     /* init CPUs */
     if (cpu_model == NULL) {
@@ -904,7 +928,10 @@ void pc_cpus_init(const char *cpu_model)
     }
 
     for (i = 0; i < smp_cpus; i++) {
-        if (!cpu_x86_init(cpu_model)) {
+        pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
+        if (error) {
+            fprintf(stderr, "%s\n", error_get_pretty(error));
+            error_free(error);
             exit(1);
         }
     }
target-i386/cpu.c +42 -0

54 lines not on this page · raw message

Signature · 1 line
1.7.1
#44 lore raw

PATCH v5 12/21 target-i386: introduce ICC bus/device/bridge

In reply to Igor Mammedov

Provides hotplug-able bus for APIC and CPU.

* icc-bridge will serve as a parent for icc-bus and provide mmio mapping services to child icc-devices.
* icc-device will replace SysBusDevice as a parent of APIC and IOAPIC devices.

---
v3:
* mv include/hw/i386/icc_bus.h into include/hw/cpu/
* various style fixes
* embed icc-bus inside icc-bridge and use qbus_create_inplace()
* update MAINTAINERS with new files
v2:
* Rebase on top the last HW reorganization series.
* Move hw/icc_bus.c into hw/cpu/ and hw/icc_bus.h include/hw/i386/

---
 MAINTAINERS                        |    6 ++
 default-configs/i386-softmmu.mak   |    1 +
 default-configs/x86_64-softmmu.mak |    1 +
 hw/cpu/Makefile.objs               |    1 +
 hw/cpu/icc_bus.c                   |  104 ++++++++++++++++++++++++++++++++++++
 hw/i386/pc_piix.c                  |    7 +++
 hw/i386/pc_q35.c                   |    7 +++
 include/hw/cpu/icc_bus.h           |   58 ++++++++++++++++++++
 8 files changed, 185 insertions(+), 0 deletions(-)
 create mode 100644 hw/cpu/icc_bus.c
 create mode 100644 include/hw/cpu/icc_bus.h
MAINTAINERS +6 -0
index 4dfd8bf..be02724 100644
@@ -644,6 +644,12 @@ F: qom/cpu.c
 F: include/qemu/cpu.h
 F: target-i386/cpu.c
 
+ICC Bus
+M: Igor Mammedov <[email protected]>
+S: Supported
+F: include/hw/cpu/icc_bus.h
+F: hw/cpu/icc_bus.c
+
 Device Tree
 M: Peter Crosthwaite <[email protected]>
 M: Alexander Graf <[email protected]>
default-configs/i386-softmmu.mak +1 -0
index 368a776..833e722 100644
@@ -44,3 +44,4 @@ CONFIG_LPC_ICH9=y
 CONFIG_PCI_Q35=y
 CONFIG_APIC=y
 CONFIG_IOAPIC=y
+CONFIG_ICC_BUS=y
default-configs/x86_64-softmmu.mak +1 -0
index 2711b83..b749eb7 100644
@@ -44,3 +44,4 @@ CONFIG_LPC_ICH9=y
 CONFIG_PCI_Q35=y
 CONFIG_APIC=y
 CONFIG_IOAPIC=y
+CONFIG_ICC_BUS=y
hw/cpu/Makefile.objs +1 -0
index a49ca04..4461ece 100644
@@ -1,4 +1,5 @@
 obj-$(CONFIG_ARM11MPCORE) += arm11mpcore.o
 obj-$(CONFIG_ARM9MPCORE) += a9mpcore.o
 obj-$(CONFIG_ARM15MPCORE) += a15mpcore.o
+obj-$(CONFIG_ICC_BUS) += icc_bus.o
 
hw/cpu/icc_bus.c new file +104 -0

104 lines not on this page · raw message

hw/i386/pc_piix.c +7 -0
index 943758a..9a6f9fe 100644
@@ -37,6 +37,7 @@
 #include "hw/kvm/clock.h"
 #include "sysemu/sysemu.h"
 #include "hw/sysbus.h"
+#include "hw/cpu/icc_bus.h"
 #include "sysemu/arch_init.h"
 #include "sysemu/blockdev.h"
 #include "hw/i2c/smbus.h"
@@ -85,8 +86,13 @@ static void pc_init1(MemoryRegion *system_memory,
     MemoryRegion *ram_memory;
     MemoryRegion *pci_memory;
     MemoryRegion *rom_memory;
+    DeviceState *icc_bridge;
     void *fw_cfg = NULL;
 
+    icc_bridge = qdev_create(NULL, TYPE_ICC_BRIDGE);
+    object_property_add_child(qdev_get_machine(), "icc-bridge",
+                              OBJECT(icc_bridge), NULL);
+
     pc_cpus_init(cpu_model);
     pc_acpi_init("acpi-dsdt.aml");
 
@@ -161,6 +167,7 @@ static void pc_init1(MemoryRegion *system_memory,
     if (pci_enabled) {
         ioapic_init_gsi(gsi_state, "i440fx");
     }
+    qdev_init_nofail(icc_bridge);
 
     pc_register_ferr_irq(gsi[13]);
 
hw/i386/pc_q35.c +7 -0
index 6ac1a89..64c5a6c 100644
@@ -41,6 +41,7 @@
 #include "hw/ide/pci.h"
 #include "hw/ide/ahci.h"
 #include "hw/usb.h"
+#include "hw/cpu/icc_bus.h"
 
 /* ICH9 AHCI has 6 ports */
 #define MAX_SATA_PORTS     6
@@ -85,6 +86,11 @@ static void pc_q35_init(QEMUMachineInitArgs *args)
     ICH9LPCState *ich9_lpc;
     PCIDevice *ahci;
     qemu_irq *cmos_s3;
+    DeviceState *icc_bridge;
+
+    icc_bridge = qdev_create(NULL, TYPE_ICC_BRIDGE);
+    object_property_add_child(qdev_get_machine(), "icc-bridge",
+                              OBJECT(icc_bridge), NULL);
 
     pc_cpus_init(cpu_model);
     pc_acpi_init("q35-acpi-dsdt.aml");
@@ -168,6 +174,7 @@ static void pc_q35_init(QEMUMachineInitArgs *args)
     if (pci_enabled) {
         ioapic_init_gsi(gsi_state, NULL);
     }
+    qdev_init_nofail(icc_bridge);
 
     pc_register_ferr_irq(gsi[13]);
 
include/hw/cpu/icc_bus.h new file +58 -0

58 lines not on this page · raw message

Signature · 1 line
1.7.1
#45 lore raw

PATCH v5 13/21 target-i386: cpu: attach ICC bus to CPU on its creation

In reply to Igor Mammedov

X86CPU should have parent bus so it would be possible to unplug it later. Set bus_type to TYPE_ICC_BUS for X86CPU type to make device_add attach hotplugged CPU to ICC bus.

---
v2:
* make sure that missing or ambiguous icc-bus will cause error on softmmu target.

---
 target-i386/cpu.c |   15 +++++++++++++++
 1 files changed, 15 insertions(+), 0 deletions(-)
target-i386/cpu.c +15 -0
index 296ba58..8d3ad00 100644
@@ -41,6 +41,7 @@
 #endif
 
 #include "sysemu/sysemu.h"
+#include "hw/cpu/icc_bus.h"
 #ifndef CONFIG_USER_ONLY
 #include "hw/xen/xen.h"
 #include "hw/sysbus.h"
@@ -1619,6 +1620,19 @@ X86CPU *cpu_x86_create(const char *cpu_model, Error **errp)
     features = model_pieces[1];
 
     cpu = X86_CPU(object_new(TYPE_X86_CPU));
+#ifndef CONFIG_USER_ONLY
+    do {
+        bool ambiguous = false;
+        Object *icc_bus = object_resolve_path_type("icc-bus", TYPE_ICC_BUS,
+                                                   &ambiguous);
+        if ((icc_bus == NULL) || ambiguous) {
+            error_setg(&error, "Invalid icc-bus value");
+            goto out;
+        }
+        qdev_set_parent_bus(DEVICE(cpu), BUS(icc_bus));
+        object_unref(OBJECT(cpu));
+   } while (0);
+#endif
     env = &cpu->env;
     env->cpu_model_str = cpu_model;
 
@@ -2330,6 +2344,7 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 
     xcc->parent_realize = dc->realize;
     dc->realize = x86_cpu_realizefn;
+    dc->bus_type = TYPE_ICC_BUS;
 
     xcc->parent_reset = cc->reset;
     cc->reset = x86_cpu_reset;
Signature · 1 line
1.7.1
#46 lore raw

PATCH v5 14/21 target-i386: replace MSI_SPACE_SIZE with APIC_SPACE_SIZE

In reply to Igor Mammedov

... and put APIC_SPACE_SIZE in public header so that it could be reused later elsewhere.

---
 hw/i386/kvm/apic.c              |    2 +-
 hw/intc/apic.c                  |    2 +-
 hw/xen/xen_apic.c               |    2 +-
 include/hw/i386/apic_internal.h |    2 --
 target-i386/cpu.h               |    1 +
 5 files changed, 4 insertions(+), 5 deletions(-)
hw/i386/kvm/apic.c +1 -1
index c6ff982..8f80425 100644
@@ -174,7 +174,7 @@ static const MemoryRegionOps kvm_apic_io_ops = {
 static void kvm_apic_init(APICCommonState *s)
 {
     memory_region_init_io(&s->io_memory, &kvm_apic_io_ops, s, "kvm-apic-msi",
-                          MSI_SPACE_SIZE);
+                          APIC_SPACE_SIZE);
 
     if (kvm_has_gsi_routing()) {
         msi_supported = true;
hw/intc/apic.c +1 -1
index 2d79a9e..756dff0 100644
@@ -874,7 +874,7 @@ static const MemoryRegionOps apic_io_ops = {
 static void apic_init(APICCommonState *s)
 {
     memory_region_init_io(&s->io_memory, &apic_io_ops, s, "apic-msi",
-                          MSI_SPACE_SIZE);
+                          APIC_SPACE_SIZE);
 
     s->timer = qemu_new_timer_ns(vm_clock, apic_timer, s);
     local_apics[s->idx] = s;
hw/xen/xen_apic.c +1 -1
index a2eb8a1..a958782 100644
@@ -39,7 +39,7 @@ static const MemoryRegionOps xen_apic_io_ops = {
 static void xen_apic_init(APICCommonState *s)
 {
     memory_region_init_io(&s->io_memory, &xen_apic_io_ops, s, "xen-apic-msi",
-                          MSI_SPACE_SIZE);
+                          APIC_SPACE_SIZE);
 
 #if defined(CONFIG_XEN_CTRL_INTERFACE_VERSION) \
     && CONFIG_XEN_CTRL_INTERFACE_VERSION >= 420
include/hw/i386/apic_internal.h +0 -2
index 578241f..aac6290 100644
@@ -66,8 +66,6 @@
 
 #define MAX_APICS 255
 
-#define MSI_SPACE_SIZE                  0x100000
-
 typedef struct APICCommonState APICCommonState;
 
 #define TYPE_APIC_COMMON "apic-common"
target-i386/cpu.h +1 -0
index a1614e8..ab151d5 100644
@@ -1270,5 +1270,6 @@ uint32_t x86_cpu_apic_id_from_index(unsigned int cpu_index);
 void enable_compat_apic_id_mode(void);
 
 #define APIC_DEFAULT_ADDRESS 0xfee00000
+#define APIC_SPACE_SIZE      0x100000
 
 #endif /* CPU_I386_H */
Signature · 1 line
1.7.1
#47 lore raw

PATCH v5 15/21 target-i386: kvmvapic: make expilict dependency on sysbus.h

In reply to Igor Mammedov

Allows kvmvapic to compile if sysbus.h is removed from apic_internal.h, from which it is indirectly included.
sysbus.h will be removed from apic_internal.h after converting APICs to ICCDevice.

---
Note:
split it in separate patch from "move APIC to ICC bus" patch to simplify review. Feel free to squash it back.

---
 hw/i386/kvmvapic.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)
hw/i386/kvmvapic.c +1 -0
index 3a10c07..5b558aa 100644
@@ -12,6 +12,7 @@
 #include "sysemu/cpus.h"
 #include "sysemu/kvm.h"
 #include "hw/i386/apic_internal.h"
+#include "hw/sysbus.h"
 
 #define VAPIC_IO_PORT           0x7e
 
Signature · 1 line
1.7.1
#48 lore raw

PATCH v5 16/21 target-i386: move APIC to ICC bus

In reply to Igor Mammedov

It allows APIC to be hotplugged

* map APIC's mmio at board level if it is present
* do not register mmio region for each APIC, since only one is used/mapped

---
v2:
* various style fixes
* split out sysbus.h inclusion in kvmvapic.c to simplify review
* squash pc_new_cpu() hunk into "introduce apic-id property" patch to simplify review
* use qdev_get_parent_bus() to get icc-bus for apic

---
 hw/cpu/icc_bus.c                |   10 ++++++++++
 hw/i386/pc.c                    |   14 +++++++++++++-
 hw/intc/apic_common.c           |   18 ++++++++++++------
 include/hw/cpu/icc_bus.h        |    4 ++++
 include/hw/i386/apic_internal.h |    6 +++---
 target-i386/cpu.c               |   16 +++-------------
 6 files changed, 45 insertions(+), 23 deletions(-)
hw/cpu/icc_bus.c +10 -0
index f6091b9..9715e07 100644
@@ -75,6 +75,7 @@ typedef struct ICCBridgeState {
     /*< private >*/
     SysBusDevice parent_obj;
     ICCBus icc_bus;
+    MemoryRegion apic_container;
 } ICCBridgeState;
 
 #define ICC_BRIGDE(obj) OBJECT_CHECK(ICCBridgeState, (obj), TYPE_ICC_BRIDGE)
@@ -82,8 +83,17 @@ typedef struct ICCBridgeState {
 static void icc_bridge_init(Object *obj)
 {
     ICCBridgeState *s = ICC_BRIGDE(obj);
+    SysBusDevice *sb = SYS_BUS_DEVICE(obj);
 
     qbus_create_inplace(&s->icc_bus, TYPE_ICC_BUS, DEVICE(s), "icc-bus");
+
+    /* Do not change order of registering regions,
+     * APIC must be first registered region, board maps it by 0 index
+     */
+    memory_region_init(&s->apic_container, "icc-apic-container",
+                       APIC_SPACE_SIZE);
+    sysbus_init_mmio(sb, &s->apic_container);
+    s->icc_bus.apic_address_space = &s->apic_container;
 }
 
 static const TypeInfo icc_bridge_info = {
hw/i386/pc.c +13 -1
index 30ec338..a09b7ee 100644
@@ -53,6 +53,7 @@
 #include "qemu/bitmap.h"
 #include "qemu/config-file.h"
 #include "hw/acpi/acpi.h"
+#include "hw/cpu/icc_bus.h"
 
 /* debug PC/ISA interrupts */
 //#define DEBUG_IRQ
@@ -916,7 +917,9 @@ static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
 void pc_cpus_init(const char *cpu_model)
 {
     int i;
+    X86CPU *cpu = NULL;
     Error *error = NULL;
+    SysBusDevice *icc_bridge;
 
     /* init CPUs */
     if (cpu_model == NULL) {
@@ -927,14 +930,23 @@ void pc_cpus_init(const char *cpu_model)
 #endif
     }
 
+    icc_bridge = SYS_BUS_DEVICE(object_resolve_path_type("icc-bridge",
+                                                 TYPE_ICC_BRIDGE, NULL));
+
     for (i = 0; i < smp_cpus; i++) {
-        pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
+        cpu = pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
         if (error) {
             fprintf(stderr, "%s\n", error_get_pretty(error));
             error_free(error);
             exit(1);
         }
     }
+
+    /* map APIC MMIO area if CPU has APIC */
+    if (cpu && cpu->env.apic_state) {
+        /* XXX: what if the base changes? */
+        sysbus_mmio_map_overlap(icc_bridge, 0, APIC_DEFAULT_ADDRESS, 0x1000);
+    }
 }
 
 void pc_acpi_init(const char *default_dsdt)
hw/intc/apic_common.c +12 -6
index e0ae07a..b03e904 100644
@@ -21,6 +21,8 @@
 #include "hw/i386/apic_internal.h"
 #include "trace.h"
 #include "sysemu/kvm.h"
+#include "hw/qdev.h"
+#include "hw/sysbus.h"
 
 static int apic_irq_delivered;
 bool apic_report_tpr_access;
@@ -282,12 +284,13 @@ static int apic_load_old(QEMUFile *f, void *opaque, int version_id)
     return 0;
 }
 
-static int apic_init_common(SysBusDevice *dev)
+static int apic_init_common(ICCDevice *dev)
 {
     APICCommonState *s = APIC_COMMON(dev);
     APICCommonClass *info;
     static DeviceState *vapic;
     static int apic_no;
+    static bool mmio_registered;
 
     if (apic_no >= MAX_APICS) {
         return -1;
@@ -296,8 +299,11 @@ static int apic_init_common(SysBusDevice *dev)
 
     info = APIC_COMMON_GET_CLASS(s);
     info->init(s);
-
-    sysbus_init_mmio(dev, &s->io_memory);
+    if (!mmio_registered) {
+        ICCBus *b = ICC_BUS(qdev_get_parent_bus(DEVICE(dev)));
+        memory_region_add_subregion(b->apic_address_space, 0, &s->io_memory);
+        mmio_registered = true;
+    }
 
     /* Note: We need at least 1M to map the VAPIC option ROM */
     if (!vapic && s->vapic_control & VAPIC_ENABLE_MASK &&
@@ -375,19 +381,19 @@ static Property apic_properties_common[] = {
 
 static void apic_common_class_init(ObjectClass *klass, void *data)
 {
-    SysBusDeviceClass *sc = SYS_BUS_DEVICE_CLASS(klass);
+    ICCDeviceClass *idc = ICC_DEVICE_CLASS(klass);
     DeviceClass *dc = DEVICE_CLASS(klass);
 
     dc->vmsd = &vmstate_apic_common;
     dc->reset = apic_reset_common;
     dc->no_user = 1;
     dc->props = apic_properties_common;
-    sc->init = apic_init_common;
+    idc->init = apic_init_common;
 }
 
 static const TypeInfo apic_common_type = {
     .name = TYPE_APIC_COMMON,
-    .parent = TYPE_SYS_BUS_DEVICE,
+    .parent = TYPE_ICC_DEVICE,
     .instance_size = sizeof(APICCommonState),
     .class_size = sizeof(APICCommonClass),
     .class_init = apic_common_class_init,
include/hw/cpu/icc_bus.h +4 -0
index a0abc21..53ef223 100644
@@ -22,6 +22,7 @@
 #ifndef ICC_BUS_H
 #define ICC_BUS_H
 
+#include "exec/memory.h"
 #include "hw/qdev-core.h"
 
 #define TYPE_ICC_BUS "icc-bus"
@@ -31,6 +32,9 @@
 typedef struct ICCBus {
     /*< private >*/
     BusState parent_obj;
+
+    /*< public >*/
+    MemoryRegion *apic_address_space;
 } ICCBus;
 
 #define ICC_BUS(obj) OBJECT_CHECK(ICCBus, (obj), TYPE_ICC_BUS)
include/hw/i386/apic_internal.h +3 -3
index aac6290..1b0a7fb 100644
@@ -21,7 +21,7 @@
 #define QEMU_APIC_INTERNAL_H
 
 #include "exec/memory.h"
-#include "hw/sysbus.h"
+#include "hw/cpu/icc_bus.h"
 #include "qemu/timer.h"
 
 /* APIC Local Vector Table */
@@ -78,7 +78,7 @@ typedef struct APICCommonState APICCommonState;
 
 typedef struct APICCommonClass
 {
-    SysBusDeviceClass parent_class;
+    ICCDeviceClass parent_class;
 
     void (*init)(APICCommonState *s);
     void (*set_base)(APICCommonState *s, uint64_t val);
@@ -92,7 +92,7 @@ typedef struct APICCommonClass
 } APICCommonClass;
 
 struct APICCommonState {
-    SysBusDevice busdev;
+    ICCDevice busdev;
 
     MemoryRegion io_memory;
     X86CPU *cpu;
target-i386/cpu.c +3 -13
index 8d3ad00..c648ea7 100644
@@ -41,10 +41,10 @@
 #endif
 
 #include "sysemu/sysemu.h"
+#include "hw/qdev-properties.h"
 #include "hw/cpu/icc_bus.h"
 #ifndef CONFIG_USER_ONLY
 #include "hw/xen/xen.h"
-#include "hw/sysbus.h"
 #include "hw/i386/apic_internal.h"
 #endif
 
@@ -2119,6 +2119,7 @@ static void mce_init(X86CPU *cpu)
 static void x86_cpu_apic_create(X86CPU *cpu, Error **errp)
 {
     CPUX86State *env = &cpu->env;
+    DeviceState *dev = DEVICE(cpu);
     APICCommonState *apic;
     const char *apic_type = "apic";
 
@@ -2128,7 +2129,7 @@ static void x86_cpu_apic_create(X86CPU *cpu, Error **errp)
         apic_type = "xen-apic";
     }
 
-    env->apic_state = qdev_try_create(NULL, apic_type);
+    env->apic_state = qdev_try_create(qdev_get_parent_bus(dev), apic_type);
     if (env->apic_state == NULL) {
         error_setg(errp, "APIC device '%s' could not be created", apic_type);
         return;
@@ -2145,7 +2146,6 @@ static void x86_cpu_apic_create(X86CPU *cpu, Error **errp)
 static void x86_cpu_apic_realize(X86CPU *cpu, Error **errp)
 {
     CPUX86State *env = &cpu->env;
-    static int apic_mapped;
 
     if (env->apic_state == NULL) {
         return;
@@ -2156,16 +2156,6 @@ static void x86_cpu_apic_realize(X86CPU *cpu, Error **errp)
                    object_get_typename(OBJECT(env->apic_state)));
         return;
     }
-
-    /* XXX: mapping more APICs at the same memory location */
-    if (apic_mapped == 0) {
-        /* NOTE: the APIC is directly connected to the CPU - it is not
-           on the global memory bus. */
-        /* XXX: what if the base changes? */
-        sysbus_mmio_map_overlap(SYS_BUS_DEVICE(env->apic_state), 0,
-                                APIC_DEFAULT_ADDRESS, 0x1000);
-        apic_mapped = 1;
-    }
 }
 #else
 static void x86_cpu_apic_realize(X86CPU *cpu, Error **errp)
Signature · 1 line
1.7.1
#49 lore raw

PATCH v5 17/21 introduce memory_region_get_address() and use it in kvm/ioapic

In reply to Igor Mammedov

kvm/ioapic is relying on the fact that SysBus device
maps mmio regions with offset counted from start of system memory. But if ioapic's region is moved to another sub-region which doesn't start at the beginning of system memory then using offset isn't correct.

Fix kvm/ioapic by providing and using helper function that returns absolute region address in respective address space.

---
Note:
next patch "move IOAPIC to ICC bus" converts IOAPICs to ICCDevice and breaks SysBus device assumption used by kvm/ioapic.

---
 hw/i386/kvm/ioapic.c  |    2 +-
 include/exec/memory.h |   10 ++++++++++
 memory.c              |   11 +++++++++++
 3 files changed, 22 insertions(+), 1 deletions(-)
hw/i386/kvm/ioapic.c +1 -1
index a3bd519..b80d41a 100644
@@ -96,7 +96,7 @@ static void kvm_ioapic_put(IOAPICCommonState *s)
 
     kioapic->id = s->id;
     kioapic->ioregsel = s->ioregsel;
-    kioapic->base_address = s->busdev.mmio[0].addr;
+    kioapic->base_address = memory_region_get_address(&s->io_memory);
     kioapic->irr = s->irr;
     for (i = 0; i < IOAPIC_NUM_PINS; i++) {
         kioapic->redirtbl[i].bits = s->ioredtbl[i];
include/exec/memory.h +10 -0
index 9e88320..954f353 100644
@@ -706,6 +706,16 @@ void memory_region_set_enabled(MemoryRegion *mr, bool enabled);
 void memory_region_set_address(MemoryRegion *mr, hwaddr addr);
 
 /*
+ * memory_region_get_address: get current the address of a region
+ *
+ * Returns the absolute address of a region.
+ * May be used on regions that are currently part of a memory hierarchy.
+ *
+ * @mr: the region being queried
+ */
+hwaddr memory_region_get_address(MemoryRegion *mr);
+
+/*
  * memory_region_set_alias_offset: dynamically update a memory alias's offset
  *
  * Dynamically updates the offset into the target region that an alias points
memory.c +11 -0
index 75ca281..0651050 100644
@@ -1413,6 +1413,17 @@ void memory_region_set_address(MemoryRegion *mr, hwaddr addr)
     memory_region_transaction_commit();
 }
 
+hwaddr memory_region_get_address(MemoryRegion *mr)
+{
+    hwaddr addr = mr->addr;
+
+    while (mr->parent) {
+        mr = mr->parent;
+        addr += mr->addr;
+    }
+    return addr;
+}
+
 void memory_region_set_alias_offset(MemoryRegion *mr, hwaddr offset)
 {
     assert(mr->alias);
Signature · 1 line
1.7.1
#50 lore raw

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

Il 23/04/2013 10:29, Igor Mammedov ha scritto:

Quoted 9 lines

kvm/ioapic is relying on the fact that SysBus device
maps mmio regions with offset counted from start of system memory. But if ioapic's region is moved to another sub-region which doesn't start at the beginning of system memory then using offset isn't correct.

Fix kvm/ioapic by providing and using helper function that returns absolute region address in respective address space.

Looks good.

Paolo

#51 lore raw

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

In reply to Igor Mammedov
Quoted 9 lines · Igor Mammedov

/*
+ * memory_region_get_address: get current the address of a region
+ *
+ * Returns the absolute address of a region.
+ * May be used on regions that are currently part of a memory hierarchy.
+ *
+ * @mr: the region being queried
+ */
+hwaddr memory_region_get_address(MemoryRegion *mr);

This doesn't make sense as a memory API in my opinion. There's no such thing as the "absolute address of a MemoryRegion".

-- PMM

#52 lore raw

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

Il 23/04/2013 19:06, Peter Maydell ha scritto:

Quoted 11 lines
Quoted 9 lines
Quoted 9 lines

/*
+ * memory_region_get_address: get current the address of a region
+ *
+ * Returns the absolute address of a region.
+ * May be used on regions that are currently part of a memory hierarchy.
+ *
+ * @mr: the region being queried
+ */
+hwaddr memory_region_get_address(MemoryRegion *mr);

This doesn't make sense as a memory API in my opinion. There's no such thing as the "absolute address of a MemoryRegion".

What about also passing an AddressSpace and stopping at the address space's root? Would that make more sense?

Paolo

#53 lore raw

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

Quoted 10 lines · Paolo Bonzini

Il 23/04/2013 19:06, Peter Maydell ha scritto:

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

/*
+ * memory_region_get_address: get current the address of a region
+ *
+ * Returns the absolute address of a region.
+ * May be used on regions that are currently part of a memory hierarchy.
+ *
+ * @mr: the region being queried
+ */
+hwaddr memory_region_get_address(MemoryRegion *mr);

Quoted 5 lines
Quoted 2 lines

This doesn't make sense as a memory API in my opinion. There's no such thing as the "absolute address of a MemoryRegion".

What about also passing an AddressSpace and stopping at the address space's root? Would that make more sense?

That would be an improvement, but it still requires this device to know what address space it's mapped into, which is a bit ugly.

-- PMM

#54 lore raw

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

Quoted 14 lines · Peter Maydell
Quoted 13 lines · Paolo Bonzini

Il 23/04/2013 19:06, Peter Maydell ha scritto:

Quoted 11 lines
Quoted 9 lines
Quoted 9 lines

/*
+ * memory_region_get_address: get current the address of a region
+ *
+ * Returns the absolute address of a region.
+ * May be used on regions that are currently part of a memory hierarchy.
+ *
+ * @mr: the region being queried
+ */
+hwaddr memory_region_get_address(MemoryRegion *mr);

This doesn't make sense as a memory API in my opinion. There's no such thing as the "absolute address of a MemoryRegion".

Quoted 6 lines
Quoted 2 lines

What about also passing an AddressSpace and stopping at the address space's root? Would that make more sense?

That would be an improvement, but it still requires this device to know what address space it's mapped into, which is a bit ugly.

Unfortunately the KVM API mandates this knowledge (IOAPIC state contains the absolute mapping address). So we need some channel to explore where this device is mapped.

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
Corporate Competence Center Embedded Linux
#55 lore raw

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

Quoted 8 lines · Jan Kiszka
Quoted 3 lines · Peter Maydell

That would be an improvement, but it still requires this device to know what address space it's mapped into, which is a bit ugly.

Unfortunately the KVM API mandates this knowledge (IOAPIC state contains the absolute mapping address). So we need some channel to explore where this device is mapped.

For ARM we defer the mechanics of "tell the kernel where this memory region is" from the device implementation (hw/intc/arm_gic_kvm.c) to generic code in target-arm/kvm.c, which I personally think is a neat way to solve this problem. It looks like the x86 ABI has tied up one-time config stuff (like the base address) with generic state save/load though, so that doesn't quite work for the x86 case.

Anyway, I think I've previously argued on this list against having a "return address of this MR in this AddressSpace" function in the API, but I've changed my mind on that: it's kind of equivalent to the listener API functionality except it's a one-off query.

NB: I don't think we should call it memory_region_get_address() though -- that implies a non-existent parallel to the current memory_region_set_address().

thanks
-- PMM

#56 lore raw

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

Il 23/04/2013 20:00, Peter Maydell ha scritto:

Quoted 4 lines

For ARM we defer the mechanics of "tell the kernel where this memory region is" from the device implementation (hw/intc/arm_gic_kvm.c) to generic code in target-arm/kvm.c, which I personally think is a neat way to solve this problem.

Yes, it is.

Quoted 3 lines

NB: I don't think we should call it memory_region_get_address() though -- that implies a non-existent parallel to the current memory_region_set_address().

Hmm, good point. address_space_get_region_addr()?

Paolo

#57 lore raw

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

Quoted 6 lines · Paolo Bonzini

Il 23/04/2013 20:00, Peter Maydell ha scritto:

Quoted 3 lines

NB: I don't think we should call it memory_region_get_address() though -- that implies a non-existent parallel to the current memory_region_set_address().

Hmm, good point. address_space_get_region_addr()?

Maybe we should make this parallel-ish to memory_region_find() (which finds an MR within an AS by addr/size)?

Actually, I think I'd favour cleaning up some of the existing API which has parameters named "address_space" which are MemoryRegion*s. I think these probably predate the AddressSpace type and should be updated to use it (and be renamed to fit too).

So how about:
MemoryRegionSection memory_region_find(MemoryRegion *address_space,

                                       hwaddr addr, uint64_t size);
becomes
 MemoryRegionSection address_space_find_region_by_addr(
                                       AddressSpace *address_space,
                                       hwaddr addr, uint64_t size);

(bit of a mouthful, but never mind)

void memory_global_sync_dirty_bitmap(MemoryRegion *address_space);
becomes
void address_space_sync_dirty_bitmap(AddressSpace *address_space);

(in both cases the first thing the implementation does is call memory_region_to_address_space() to get the AddressSpace* anyway.)

and our new function is
MemoryRegionSection address_space_find_region(AddressSpace *as,

                                               MemoryRegion *mr);

?

(we don't have to do the tidyup first, but the new function signature makes more sense viewed in the light of the others)

thanks
-- PMM

#58 lore raw

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

Il 23/04/2013 23:39, Peter Maydell ha scritto:

Quoted 26 lines
Quoted 2 lines
Quoted 2 lines

Hmm, good point. address_space_get_region_addr()?

Maybe we should make this parallel-ish to memory_region_find() (which finds an MR within an AS by addr/size)?

Actually, I think I'd favour cleaning up some of the existing API which has parameters named "address_space" which are MemoryRegion*s. I think these probably predate the AddressSpace type and should be updated to use it (and be renamed to fit too).

So how about:
MemoryRegionSection memory_region_find(MemoryRegion *address_space,

                                       hwaddr addr, uint64_t size);
becomes
 MemoryRegionSection address_space_find_region_by_addr(
                                       AddressSpace *address_space,
                                       hwaddr addr, uint64_t size);

(bit of a mouthful, but never mind)

void memory_global_sync_dirty_bitmap(MemoryRegion *address_space);
becomes
void address_space_sync_dirty_bitmap(AddressSpace *address_space);

(in both cases the first thing the implementation does is call memory_region_to_address_space() to get the AddressSpace* anyway.)

Quoted 8 lines

and our new function is
MemoryRegionSection address_space_find_region(AddressSpace *as,

                                               MemoryRegion *mr);

?

(we don't have to do the tidyup first, but the new function signature makes more sense viewed in the light of the others)

I can look at it for 1.6, since I've already forward ported Avi's IOMMU patches and added s/DMAContext/AddressSpace/ on top.

What about 1.5 though?

Paolo

#59 lore raw

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

Quoted 8 lines · Paolo Bonzini

Il 23/04/2013 23:39, Peter Maydell ha scritto:

Quoted 6 lines

and our new function is
MemoryRegionSection address_space_find_region(AddressSpace *as,

                                               MemoryRegion *mr);

?

Quoted 7 lines
Quoted 2 lines

(we don't have to do the tidyup first, but the new function signature makes more sense viewed in the light of the others)

I can look at it for 1.6, since I've already forward ported Avi's IOMMU patches and added s/DMAContext/AddressSpace/ on top.

What about 1.5 though?

Ah, I was going to ask if this was a 1.5 series. If so, maybe just add and use that new function? Anyway, I don't want to bikeshed the function too much, as long as it takes an AddressSpace* to search and lets the caller distinguish "couldn't find the region" from valid actual addresses.

thanks
-- PMM

#60 lore raw

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

In reply to Peter Maydell
Quoted 7 lines

So how about:
MemoryRegionSection memory_region_find(MemoryRegion *address_space,

                                       hwaddr addr, uint64_t size);
becomes
 MemoryRegionSection address_space_find_region_by_addr(
                                       AddressSpace *address_space,
                                       hwaddr addr, uint64_t size);
Quoted 5 lines

(bit of a mouthful, but never mind)

void memory_global_sync_dirty_bitmap(MemoryRegion *address_space);
becomes
void address_space_sync_dirty_bitmap(AddressSpace *address_space);

I think the latter makes definite sense, I am not quite as sure about the former.

Looking at framebuffer.c's use of memory_region_find, here you really want to go from the "local" view to a global one in order to look at global data structures such as the dirty bitmap. So it is right to have memory_region_find as a MemoryRegion operation, even though right now it is always passed an AddressSpace.

Communicating the absolute address to KVM is another example of this local->global translation. Hence, my suggestion is to remove memory_region_find's limitation on the first argument, and make it work on nested regions too. With this change, memory_region_find() nicely fits Igor's use case.

Not coincidentially, the additional code in memory_region_find() is very similar to Igor's memory_region_get_address().

See the attached patch, which I tested on master. Igor, can you try it with iccbus?

Paolo

-------------- 8< --------------------

Quoted 1 line

From 953460fa8ee9f9e7243ea34eb57a901102be9307 Mon Sep 17 00:00:00 2001

From: Paolo Bonzini <[email protected]>
Date: Tue, 23 Apr 2013 10:29:51 +0200
Subject: [RFC PATCH 17/21] extend memory_region_find() and use it in kvm/ioapic

kvm/ioapic is relying on the fact that SysBus device
maps mmio regions with offset counted from start of system memory. But if ioapic's region is moved to another sub-region which doesn't start at the beginning of system memory then using offset isn't correct.

To fix kvm/ioapic, extend memory_region_find() so that it can help retrieving the absolute region address and the respective address space.

The patch is a no-op in case mr is parentless, i.e. mr->addr == 0 and mr->parent == NULL.

Based on a patch by Igor Mammedov. <[email protected]>

---
 hw/i386/kvm/ioapic.c  |    9 ++++++++-
 include/exec/memory.h |   13 +++++++------
 memory.c              |   19 ++++++++++++++-----
 3 files changed, 29 insertions(+), 12 deletions(-)
hw/i386/kvm/ioapic.c +8 -1
index a3bd519..7564d07 100644
@@ -89,14 +89,21 @@ static void kvm_ioapic_put(IOAPICCommonState *s)
 {
     struct kvm_irqchip chip;
     struct kvm_ioapic_state *kioapic;
+    MemoryRegionSection mrs;
     int ret, i;
 
+    mrs = memory_region_find(&s->io_memory, 0, 0x1000);
+    if (mrs.mr != &s->io_memory || mrs.offset_within_region != 0) {
+        fprintf(stderr, "cannot find IOAPIC base\n");
+        abort();
+    }
+
     chip.chip_id = KVM_IRQCHIP_IOAPIC;
     kioapic = &chip.chip.ioapic;
 
     kioapic->id = s->id;
     kioapic->ioregsel = s->ioregsel;
-    kioapic->base_address = s->busdev.mmio[0].addr;
+    kioapic->base_address = mrs.offset_within_address_space;
     kioapic->irr = s->irr;
     for (i = 0; i < IOAPIC_NUM_PINS; i++) {
         kioapic->redirtbl[i].bits = s->ioredtbl[i];
include/exec/memory.h +7 -6
index eb9e659..5854d19 100644
@@ -725,17 +725,18 @@ void memory_region_set_alias_offset(MemoryRegion *mr,
  *
  * Returns a #MemoryRegionSection that describes a contiguous overlap.
  * It will have the following characteristics:
- *    .@offset_within_address_space >= @addr
- *    .@offset_within_address_space + .@size <= @addr + @size
  *    .@size = 0 iff no overlap was found
  *    .@mr is non-%NULL iff an overlap was found
  *
- * @address_space: a top-level (i.e. parentless) region that contains
- *       the region to be found
- * @addr: start of the area within @address_space to be searched
+ * If @mr is parent-less,
+ *    .@offset_within_address_space >= @addr
+ *    .@offset_within_address_space + .@size <= @addr + @size
+ *
+ * @mr: a (possibly indirect) parent that contains the region to be found
+ * @addr: start of the area within @as to be searched
  * @size: size of the area to be searched
  */
-MemoryRegionSection memory_region_find(MemoryRegion *address_space,
+MemoryRegionSection memory_region_find(MemoryRegion *mr,
                                        hwaddr addr, uint64_t size);
 
 /**
memory.c +14 -5
index c82bd12..dba0a4b 100644
@@ -1451,15 +1451,24 @@ static FlatRange *address_space_lookup(AddressSpace *as, AddrRange addr)
                    sizeof(FlatRange), cmp_flatrange_addr);
 }
 
-MemoryRegionSection memory_region_find(MemoryRegion *address_space,
+MemoryRegionSection memory_region_find(MemoryRegion *mr,
                                        hwaddr addr, uint64_t size)
 {
-    AddressSpace *as = memory_region_to_address_space(address_space);
-    AddrRange range = addrrange_make(int128_make64(addr),
-                                     int128_make64(size));
-    FlatRange *fr = address_space_lookup(as, range);
     MemoryRegionSection ret = { .mr = NULL, .size = 0 };
+    MemoryRegion *root;
+    AddressSpace *as;
+    AddrRange range;
+    FlatRange *fr;
+
+    addr += mr->addr;
+    for (root = mr; root->parent; ) {
+        root = root->parent;
+        addr += root->addr;
+    }
 
+    as = memory_region_to_address_space(root);
+    range = addrrange_make(int128_make64(addr), int128_make64(size));
+    fr = address_space_lookup(as, range);
     if (!fr) {
         return ret;
     }
Signature · 1 line
1.7.1
#61 lore raw

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

Il 24/04/2013 12:22, Paolo Bonzini ha scritto:

Quoted 25 lines
memory.c +8 -5
index c82bd12..dba0a4b 100644
@@ -1451,15 +1451,24 @@ static FlatRange *address_space_lookup(AddressSpace *as, AddrRange addr)
                    sizeof(FlatRange), cmp_flatrange_addr);
 }
 
-MemoryRegionSection memory_region_find(MemoryRegion *address_space,
+MemoryRegionSection memory_region_find(MemoryRegion *mr,
                                        hwaddr addr, uint64_t size)
 {
-    AddressSpace *as = memory_region_to_address_space(address_space);
-    AddrRange range = addrrange_make(int128_make64(addr),
-                                     int128_make64(size));
-    FlatRange *fr = address_space_lookup(as, range);
     MemoryRegionSection ret = { .mr = NULL, .size = 0 };
+    MemoryRegion *root;
+    AddressSpace *as;
+    AddrRange range;
+    FlatRange *fr;
+
+    addr += mr->addr;
+    for (root = mr; root->parent; ) {

memory.c static FlatRange *address_space_lookup(AddressSpace *as, AddrRange addr) · line 1474

Quoted 10 lines
memory.c +6 -0
+        root = root->parent;
+        addr += root->addr;
+    }
 
+    as = memory_region_to_address_space(root);
+    range = addrrange_make(int128_make64(addr), int128_make64(size));
+    fr = address_space_lookup(as, range);
     if (!fr) {
         return ret;
     }

Looking at the code again, mrs.address_space is not filled in. This should be squashed in too for completeness.

memory.c +1 -0
index dba0a4b..1916937 100644
@@ -1479,6 +1479,7 @@ MemoryRegionSection memory_region_find(MemoryRegion *mr,
     }
 
     ret.mr = fr->mr;
+    ret.address_space = as;
     range = addrrange_intersection(range, fr->addr);
     ret.offset_within_region = fr->offset_in_region;
     ret.offset_within_region += int128_get64(int128_sub(range.start,

Paolo

#62 lore raw

PATCH v2 15/19 extend memory_region_find() and use it in kvm/ioapic

In reply to Igor Mammedov

From: Paolo Bonzini <[email protected]>

kvm/ioapic is relying on the fact that SysBus device
maps mmio regions with offset counted from start of system memory. But if ioapic's region is moved to another sub-region which doesn't start at the beginning of system memory then using offset isn't correct.

To fix kvm/ioapic, extend memory_region_find() so that it can help retrieving the absolute region address and the respective address space.

The patch is a no-op in case mr is parentless, i.e. mr->addr == 0 and mr->parent == NULL.

In addition fill in MemoryRegionSection.as wich was missing in original memory_region_find().

---
 hw/i386/kvm/ioapic.c  |    9 ++++++++-
 include/exec/memory.h |   13 +++++++------
 memory.c              |   20 +++++++++++++++-----
 3 files changed, 30 insertions(+), 12 deletions(-)
hw/i386/kvm/ioapic.c +8 -1
index a3bd519..dc6ddab 100644
@@ -89,14 +89,21 @@ static void kvm_ioapic_put(IOAPICCommonState *s)
 {
     struct kvm_irqchip chip;
     struct kvm_ioapic_state *kioapic;
+    MemoryRegionSection mrs;
     int ret, i;
 
+    mrs = memory_region_find(&s->io_memory, 0, 0x1000);
+    if (mrs.mr != &s->io_memory || mrs.offset_within_region != 0) {
+        fprintf(stderr, "cannot find IOAPIC base\n");
+        abort();
+    }
+
     chip.chip_id = KVM_IRQCHIP_IOAPIC;
     kioapic = &chip.chip.ioapic;
 
     kioapic->id = s->id;
     kioapic->ioregsel = s->ioregsel;
-    kioapic->base_address = s->busdev.mmio[0].addr;
+    kioapic->base_address = mrs.offset_within_address_space;
     kioapic->irr = s->irr;
     for (i = 0; i < IOAPIC_NUM_PINS; i++) {
         kioapic->redirtbl[i].bits = s->ioredtbl[i];
include/exec/memory.h +7 -6
index 9e88320..efe210b 100644
@@ -725,17 +725,18 @@ void memory_region_set_alias_offset(MemoryRegion *mr,
  *
  * Returns a #MemoryRegionSection that describes a contiguous overlap.
  * It will have the following characteristics:
- *    .@offset_within_address_space >= @addr
- *    .@offset_within_address_space + .@size <= @addr + @size
  *    .@size = 0 iff no overlap was found
  *    .@mr is non-%NULL iff an overlap was found
  *
- * @address_space: a top-level (i.e. parentless) region that contains
- *       the region to be found
- * @addr: start of the area within @address_space to be searched
+ * If @mr is parent-less,
+ *    .@offset_within_address_space >= @addr
+ *    .@offset_within_address_space + .@size <= @addr + @size
+ *
+ * @mr: a (possibly indirect) parent that contains the region to be found
+ * @addr: start of the area within @as to be searched
  * @size: size of the area to be searched
  */
-MemoryRegionSection memory_region_find(MemoryRegion *address_space,
+MemoryRegionSection memory_region_find(MemoryRegion *mr,
                                        hwaddr addr, uint64_t size);
 
 /**
memory.c +15 -5
index 75ca281..34bfb13 100644
@@ -1451,15 +1451,24 @@ static FlatRange *address_space_lookup(AddressSpace *as, AddrRange addr)
                    sizeof(FlatRange), cmp_flatrange_addr);
 }
 
-MemoryRegionSection memory_region_find(MemoryRegion *address_space,
+MemoryRegionSection memory_region_find(MemoryRegion *mr,
                                        hwaddr addr, uint64_t size)
 {
-    AddressSpace *as = memory_region_to_address_space(address_space);
-    AddrRange range = addrrange_make(int128_make64(addr),
-                                     int128_make64(size));
-    FlatRange *fr = address_space_lookup(as, range);
     MemoryRegionSection ret = { .mr = NULL, .size = 0 };
+    MemoryRegion *root;
+    AddressSpace *as;
+    AddrRange range;
+    FlatRange *fr;
+
+    addr += mr->addr;
+    for (root = mr; root->parent; ) {
+        root = root->parent;
+        addr += root->addr;
+    }
 
+    as = memory_region_to_address_space(root);
+    range = addrrange_make(int128_make64(addr), int128_make64(size));
+    fr = address_space_lookup(as, range);
     if (!fr) {
         return ret;
     }
@@ -1470,6 +1479,7 @@ MemoryRegionSection memory_region_find(MemoryRegion *address_space,
     }
 
     ret.mr = fr->mr;
+    ret.address_space = as;
     range = addrrange_intersection(range, fr->addr);
     ret.offset_within_region = fr->offset_in_region;
     ret.offset_within_region += int128_get64(int128_sub(range.start,
Signature · 1 line
1.7.1
#63 lore raw

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

In reply to Igor Mammedov
Quoted 4 lines · Igor Mammedov

kvm/ioapic is relying on the fact that SysBus device
maps mmio regions with offset counted from start of system memory. But if ioapic's region is moved to another sub-region which doesn't start at the beginning of system memory then using offset isn't correct.

But base_address in only initialized once, never changed. Does this patch matter now?

The correct solution would be to track changes to APICBASE register at PIIX3 chipset level and adjust mapping and KVM base also from there.

Quoted 74 lines

Fix kvm/ioapic by providing and using helper function that returns absolute region address in respective address space.

---
Note:
next patch "move IOAPIC to ICC bus" converts IOAPICs to ICCDevice and breaks SysBus device assumption used by kvm/ioapic.

---
 hw/i386/kvm/ioapic.c  |    2 +-
 include/exec/memory.h |   10 ++++++++++
 memory.c              |   11 +++++++++++
 3 files changed, 22 insertions(+), 1 deletions(-)
hw/i386/kvm/ioapic.c +1 -1
index a3bd519..b80d41a 100644
@@ -96,7 +96,7 @@ static void kvm_ioapic_put(IOAPICCommonState *s)
 
     kioapic->id = s->id;
     kioapic->ioregsel = s->ioregsel;
-    kioapic->base_address = s->busdev.mmio[0].addr;
+    kioapic->base_address = memory_region_get_address(&s->io_memory);
     kioapic->irr = s->irr;
     for (i = 0; i < IOAPIC_NUM_PINS; i++) {
         kioapic->redirtbl[i].bits = s->ioredtbl[i];
include/exec/memory.h +10 -0
index 9e88320..954f353 100644
@@ -706,6 +706,16 @@ void memory_region_set_enabled(MemoryRegion *mr, bool enabled);
 void memory_region_set_address(MemoryRegion *mr, hwaddr addr);
 
 /*
+ * memory_region_get_address: get current the address of a region
+ *
+ * Returns the absolute address of a region.
+ * May be used on regions that are currently part of a memory hierarchy.
+ *
+ * @mr: the region being queried
+ */
+hwaddr memory_region_get_address(MemoryRegion *mr);
+
+/*
  * memory_region_set_alias_offset: dynamically update a memory alias's offset
  *
  * Dynamically updates the offset into the target region that an alias points
memory.c +11 -1
index 75ca281..0651050 100644
@@ -1413,6 +1413,17 @@ void memory_region_set_address(MemoryRegion *mr, hwaddr addr)
     memory_region_transaction_commit();
 }
 
+hwaddr memory_region_get_address(MemoryRegion *mr)
+{
+    hwaddr addr = mr->addr;
+
+    while (mr->parent) {
+        mr = mr->parent;
+        addr += mr->addr;
+    }
+    return addr;
+}
+
 void memory_region_set_alias_offset(MemoryRegion *mr, hwaddr offset)
 {
     assert(mr->alias);
--

1.7.1

#64 lore raw

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

Quoted 8 lines · On Thu, 25 Apr 2013 18:37:19 +0000 Blue Swirl <[email protected]> wrote:
Quoted 4 lines · Igor Mammedov

kvm/ioapic is relying on the fact that SysBus device
maps mmio regions with offset counted from start of system memory. But if ioapic's region is moved to another sub-region which doesn't start at the beginning of system memory then using offset isn't correct.

But base_address in only initialized once, never changed. Does this patch matter now?

this code path is used on only at machine start-up but also on resets and post migration to initialize IO-APIC. And unfortunately KVM API takes not only base address but a bunch of other parameters in that ioctl, so splitting it doesn't look feasible for 1.5. Patch is useful in aspect that it hides direct access to parent's internals and without dangling SysBusDevice internals it allows to convert kvm/ioapic to ICCDevice [next patch in series], leaving code a bit cleaner.

BTW:
there is an improved patch:
http://permalink.gmane.org/gmane.comp.emulators.qemu/208512

that instead of introducing a new function improves current memory_region_find() API.

Quoted 2 lines

The correct solution would be to track changes to APICBASE register at PIIX3 chipset level and adjust mapping and KVM base also from there.

That we would probably need a change in KVM API first to allow set IO-APIC's base separately.

Quoted 75 lines
Quoted 75 lines

Fix kvm/ioapic by providing and using helper function that returns absolute region address in respective address space.

---
Note:
next patch "move IOAPIC to ICC bus" converts IOAPICs to ICCDevice and breaks SysBus device assumption used by kvm/ioapic.

---
 hw/i386/kvm/ioapic.c  |    2 +-
 include/exec/memory.h |   10 ++++++++++
 memory.c              |   11 +++++++++++
 3 files changed, 22 insertions(+), 1 deletions(-)
hw/i386/kvm/ioapic.c +1 -1
index a3bd519..b80d41a 100644
@@ -96,7 +96,7 @@ static void kvm_ioapic_put(IOAPICCommonState *s)
 
     kioapic->id = s->id;
     kioapic->ioregsel = s->ioregsel;
-    kioapic->base_address = s->busdev.mmio[0].addr;
+    kioapic->base_address = memory_region_get_address(&s->io_memory);
     kioapic->irr = s->irr;
     for (i = 0; i < IOAPIC_NUM_PINS; i++) {
         kioapic->redirtbl[i].bits = s->ioredtbl[i];
include/exec/memory.h +10 -0
index 9e88320..954f353 100644
@@ -706,6 +706,16 @@ void memory_region_set_enabled(MemoryRegion *mr, bool enabled);
 void memory_region_set_address(MemoryRegion *mr, hwaddr addr);
 
 /*
+ * memory_region_get_address: get current the address of a region
+ *
+ * Returns the absolute address of a region.
+ * May be used on regions that are currently part of a memory hierarchy.
+ *
+ * @mr: the region being queried
+ */
+hwaddr memory_region_get_address(MemoryRegion *mr);
+
+/*
  * memory_region_set_alias_offset: dynamically update a memory alias's offset
  *
  * Dynamically updates the offset into the target region that an alias points
memory.c +11 -1
index 75ca281..0651050 100644
@@ -1413,6 +1413,17 @@ void memory_region_set_address(MemoryRegion *mr, hwaddr addr)
     memory_region_transaction_commit();
 }
 
+hwaddr memory_region_get_address(MemoryRegion *mr)
+{
+    hwaddr addr = mr->addr;
+
+    while (mr->parent) {
+        mr = mr->parent;
+        addr += mr->addr;
+    }
+    return addr;
+}
+
 void memory_region_set_alias_offset(MemoryRegion *mr, hwaddr offset)
 {
     assert(mr->alias);
--

1.7.1

Signature · 2 lines
Regards,
  Igor
#65 lore raw

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

Quoted 11 lines · Igor Mammedov
Quoted 8 lines · On Thu, 25 Apr 2013 18:37:19 +0000 Blue Swirl <[email protected]> wrote:
Quoted 4 lines · Igor Mammedov

kvm/ioapic is relying on the fact that SysBus device
maps mmio regions with offset counted from start of system memory. But if ioapic's region is moved to another sub-region which doesn't start at the beginning of system memory then using offset isn't correct.

But base_address in only initialized once, never changed. Does this patch matter now?

Quoted 7 lines

this code path is used on only at machine start-up but also on resets and post migration to initialize IO-APIC. And unfortunately KVM API takes not only base address but a bunch of other parameters in that ioctl, so splitting it doesn't look feasible for 1.5. Patch is useful in aspect that it hides direct access to parent's internals and without dangling SysBusDevice internals it allows to convert kvm/ioapic to ICCDevice [next patch in series], leaving code a bit cleaner.

But as the address can't be changed (yet), the entire patch could be simply:
- kioapic->base_address = s->busdev.mmio[0].addr;
+ kioapic->base_address = IO_APIC_DEFAULT_ADDRESS;

Later, when it's possible to change the address via PIIX3 registers, we can adjust the base and pass that properly to kioapic and on to KVM.

Resolving the base address every time when kvm_ioapic_put() is called is also less efficient, assuming of course that the base address changes less often than the KVM ioctl is used.

Quoted 93 lines

BTW:
there is an improved patch:
http://permalink.gmane.org/gmane.comp.emulators.qemu/208512

that instead of introducing a new function improves current memory_region_find() API.

Quoted 2 lines

The correct solution would be to track changes to APICBASE register at PIIX3 chipset level and adjust mapping and KVM base also from there.

That we would probably need a change in KVM API first to allow set IO-APIC's base separately.

Quoted 67 lines
Quoted 67 lines

Fix kvm/ioapic by providing and using helper function that returns absolute region address in respective address space.

---
Note:
next patch "move IOAPIC to ICC bus" converts IOAPICs to ICCDevice and breaks SysBus device assumption used by kvm/ioapic.

---
 hw/i386/kvm/ioapic.c  |    2 +-
 include/exec/memory.h |   10 ++++++++++
 memory.c              |   11 +++++++++++
 3 files changed, 22 insertions(+), 1 deletions(-)
hw/i386/kvm/ioapic.c +1 -1
index a3bd519..b80d41a 100644
@@ -96,7 +96,7 @@ static void kvm_ioapic_put(IOAPICCommonState *s)
 
     kioapic->id = s->id;
     kioapic->ioregsel = s->ioregsel;
-    kioapic->base_address = s->busdev.mmio[0].addr;
+    kioapic->base_address = memory_region_get_address(&s->io_memory);
     kioapic->irr = s->irr;
     for (i = 0; i < IOAPIC_NUM_PINS; i++) {
         kioapic->redirtbl[i].bits = s->ioredtbl[i];
include/exec/memory.h +10 -0
index 9e88320..954f353 100644
@@ -706,6 +706,16 @@ void memory_region_set_enabled(MemoryRegion *mr, bool enabled);
 void memory_region_set_address(MemoryRegion *mr, hwaddr addr);
 
 /*
+ * memory_region_get_address: get current the address of a region
+ *
+ * Returns the absolute address of a region.
+ * May be used on regions that are currently part of a memory hierarchy.
+ *
+ * @mr: the region being queried
+ */
+hwaddr memory_region_get_address(MemoryRegion *mr);
+
+/*
  * memory_region_set_alias_offset: dynamically update a memory alias's offset
  *
  * Dynamically updates the offset into the target region that an alias points
memory.c +9 -0
index 75ca281..0651050 100644
@@ -1413,6 +1413,17 @@ void memory_region_set_address(MemoryRegion *mr, hwaddr addr)
     memory_region_transaction_commit();
 }
 
+hwaddr memory_region_get_address(MemoryRegion *mr)
+{
+    hwaddr addr = mr->addr;
+
+    while (mr->parent) {
+        mr = mr->parent;
+        addr += mr->addr;
+    }
+    return addr;
Quoted 8 lines
Quoted 8 lines
memory.c +2 -1
+}
+
 void memory_region_set_alias_offset(MemoryRegion *mr, hwaddr offset)
 {
     assert(mr->alias);
--

1.7.1

--
Regards,
Igor

#66 lore raw

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

Quoted 21 lines · On Fri, 26 Apr 2013 17:35:41 +0000 Blue Swirl <[email protected]> wrote:
Quoted 11 lines · Igor Mammedov
Quoted 8 lines · On Thu, 25 Apr 2013 18:37:19 +0000 Blue Swirl <[email protected]> wrote:
Quoted 4 lines · Igor Mammedov

kvm/ioapic is relying on the fact that SysBus device
maps mmio regions with offset counted from start of system memory. But if ioapic's region is moved to another sub-region which doesn't start at the beginning of system memory then using offset isn't correct.

But base_address in only initialized once, never changed. Does this patch matter now?

Quoted 7 lines

this code path is used on only at machine start-up but also on resets and post migration to initialize IO-APIC. And unfortunately KVM API takes not only base address but a bunch of other parameters in that ioctl, so splitting it doesn't look feasible for 1.5. Patch is useful in aspect that it hides direct access to parent's internals and without dangling SysBusDevice internals it allows to convert kvm/ioapic to ICCDevice [next patch in series], leaving code a bit cleaner.

But as the address can't be changed (yet), the entire patch could be simply:

Review

Quoted 2 lines
(quoted diff) +1 -1
-    kioapic->base_address = s->busdev.mmio[0].addr;
+    kioapic->base_address = IO_APIC_DEFAULT_ADDRESS;

It's a bit fragile, but that for sure simpler and can work.

Jan, Paolo,
Are you ok with this approach?

Quoted 22 lines

Later, when it's possible to change the address via PIIX3 registers, we can adjust the base and pass that properly to kioapic and on to KVM.

Resolving the base address every time when kvm_ioapic_put() is called is also less efficient, assuming of course that the base address changes less often than the KVM ioctl is used.

Quoted 13 lines

BTW:
there is an improved patch:
http://permalink.gmane.org/gmane.comp.emulators.qemu/208512

that instead of introducing a new function improves current memory_region_find() API.

Quoted 2 lines

The correct solution would be to track changes to APICBASE register at PIIX3 chipset level and adjust mapping and KVM base also from there.

That we would probably need a change in KVM API first to allow set IO-APIC's base separately.

Signature · 2 lines
Regards,
  Igor
#67 lore raw

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

Il 26/04/2013 19:46, Igor Mammedov ha scritto:

Quoted 8 lines
Quoted 3 lines

Review

Quoted 3 lines

But as the address can't be changed (yet), the entire patch could be simply:

(quoted diff) +1 -1
-    kioapic->base_address = s->busdev.mmio[0].addr;
+    kioapic->base_address = IO_APIC_DEFAULT_ADDRESS;

It's a bit fragile, but that for sure simpler and can work.

Jan, Paolo,
Are you ok with this approach?

I think extending memory_region_find is a good idea anyway, and at this point I don't see a reason to do the above change...

Paolo

#68 lore raw

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

Quoted 4 lines · Paolo Bonzini

Il 26/04/2013 19:46, Igor Mammedov ha scritto:

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

But as the address can't be changed (yet), the entire patch could be simply:

(quoted diff) +1 -1
-    kioapic->base_address = s->busdev.mmio[0].addr;
+    kioapic->base_address = IO_APIC_DEFAULT_ADDRESS;
Quoted 8 lines
Quoted 5 lines

It's a bit fragile, but that for sure simpler and can work.

Jan, Paolo,
Are you ok with this approach?

I think extending memory_region_find is a good idea anyway, and at this point I don't see a reason to do the above change...

The reasoning was in the part that Igor cut off:

"Later, when it's possible to change the address via PIIX3 registers, we can adjust the base and pass that properly to kioapic and on to KVM.

Resolving the base address every time when kvm_ioapic_put() is called is also less efficient, assuming of course that the base address changes less often than the KVM ioctl is used."

I think the patch is a bit flawed. If the guest maps something else on top of IOAPIC, like LAPIC (which should be in CPU specific address spaces, but for now it lives in the global system memory space), the guest could trigger the abort() by resetting the system.

Quoted 2 lines

Paolo

#69 lore raw

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

Il 27/04/2013 12:09, Blue Swirl ha scritto:

Quoted 20 lines · Paolo Bonzini
Quoted 4 lines

Il 26/04/2013 19:46, Igor Mammedov ha scritto:

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

But as the address can't be changed (yet), the entire patch could be simply:

(quoted diff) +1 -1
-    kioapic->base_address = s->busdev.mmio[0].addr;
+    kioapic->base_address = IO_APIC_DEFAULT_ADDRESS;
Quoted 8 lines
Quoted 5 lines

It's a bit fragile, but that for sure simpler and can work.

Jan, Paolo,
Are you ok with this approach?

I think extending memory_region_find is a good idea anyway, and at this point I don't see a reason to do the above change...

The reasoning was in the part that Igor cut off:

"Later, when it's possible to change the address via PIIX3 registers, we can adjust the base and pass that properly to kioapic and on to KVM.

Quoted 8 lines

Resolving the base address every time when kvm_ioapic_put() is called is also less efficient, assuming of course that the base address changes less often than the KVM ioctl is used."

I think the patch is a bit flawed. If the guest maps something else on top of IOAPIC, like LAPIC (which should be in CPU specific address spaces, but for now it lives in the global system memory space), the guest could trigger the abort() by resetting the system.

The questions are, in order of importance:

(1) what privileges would this require in the guest? Answer: a lot.

(2) is this likely to happen by chance? Answer: no, not at all.

(3) is there a workaround? Answer: yes, disable in-kernel irqchip.

Simply setting IO_APIC_DEFAULT_ADDRESS is also flawed in my opinion. I'm not sure the in-kernel irqchip handles correctly an overlap between the IOAPIC and LAPIC regions, maybe an abort is predictable after all.

Paolo

#70 lore raw

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

Quoted 30 lines · Paolo Bonzini

Il 27/04/2013 12:09, Blue Swirl ha scritto:

Quoted 20 lines · Paolo Bonzini
Quoted 4 lines

Il 26/04/2013 19:46, Igor Mammedov ha scritto:

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

But as the address can't be changed (yet), the entire patch could be simply:

(quoted diff) +1 -1
-    kioapic->base_address = s->busdev.mmio[0].addr;
+    kioapic->base_address = IO_APIC_DEFAULT_ADDRESS;
Quoted 8 lines
Quoted 5 lines

It's a bit fragile, but that for sure simpler and can work.

Jan, Paolo,
Are you ok with this approach?

I think extending memory_region_find is a good idea anyway, and at this point I don't see a reason to do the above change...

The reasoning was in the part that Igor cut off:

"Later, when it's possible to change the address via PIIX3 registers, we can adjust the base and pass that properly to kioapic and on to KVM.

Quoted 8 lines

Resolving the base address every time when kvm_ioapic_put() is called is also less efficient, assuming of course that the base address changes less often than the KVM ioctl is used."

I think the patch is a bit flawed. If the guest maps something else on top of IOAPIC, like LAPIC (which should be in CPU specific address spaces, but for now it lives in the global system memory space), the guest could trigger the abort() by resetting the system.

Quoted 7 lines

The questions are, in order of importance:

(1) what privileges would this require in the guest? Answer: a lot.

(2) is this likely to happen by chance? Answer: no, not at all.

(3) is there a workaround? Answer: yes, disable in-kernel irqchip.

These questions ask if there is a risk of benevolent guests performing these activities and I agree that the chances are close to zero.

But the interesting question is to ask if a malevolent guest can bring down a VM uncontrollably this way and I think it only needs a few elevated privileges in a guest to do this.

The fix is to avoid abort(), which is a separate issue to whether the address base should be resolved for each KVM ioctl or not.

Quoted 4 lines

Simply setting IO_APIC_DEFAULT_ADDRESS is also flawed in my opinion. I'm not sure the in-kernel irqchip handles correctly an overlap between the IOAPIC and LAPIC regions, maybe an abort is predictable after all.

At least the guest needs to be stopped. Perhaps we should have a common function which does this and logs the guest error so we can start replacing calls to abort() with it.

Quoted 2 lines

Paolo

#71 lore raw

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

Il 27/04/2013 22:57, Blue Swirl ha scritto:

Quoted 6 lines
Quoted 6 lines

The questions are, in order of importance:

(1) what privileges would this require in the guest? Answer: a lot.

(2) is this likely to happen by chance? Answer: no, not at all.

Quoted 8 lines
Quoted 1 line

(3) is there a workaround? Answer: yes, disable in-kernel irqchip.

These questions ask if there is a risk of benevolent guests performing these activities and I agree that the chances are close to zero.

But the interesting question is to ask if a malevolent guest can bring down a VM uncontrollably this way and I think it only needs a few elevated privileges in a guest to do this.

If you have them, isn't it simpler to just turn off the VM (using APM or ACPI)? Also, killing your guest is not a very interesting thing to do once you've gotten elevated privileges. ;)

Quoted 7 lines
Quoted 3 lines

Simply setting IO_APIC_DEFAULT_ADDRESS is also flawed in my opinion. I'm not sure the in-kernel irqchip handles correctly an overlap between the IOAPIC and LAPIC regions, maybe an abort is predictable after all.

At least the guest needs to be stopped. Perhaps we should have a common function which does this and logs the guest error so we can start replacing calls to abort() with it.

Yes, that's a good idea. We can reuse the internal error runstate for that.

Paolo

#72 lore raw

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

In reply to Blue Swirl
Quoted 50 lines · On Sat, 27 Apr 2013 20:57:26 +0000 Blue Swirl <[email protected]> wrote:
Quoted 30 lines · Paolo Bonzini

Il 27/04/2013 12:09, Blue Swirl ha scritto:

Quoted 20 lines · Paolo Bonzini
Quoted 4 lines

Il 26/04/2013 19:46, Igor Mammedov ha scritto:

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

But as the address can't be changed (yet), the entire patch could be simply:

(quoted diff) +1 -1
-    kioapic->base_address = s->busdev.mmio[0].addr;
+    kioapic->base_address = IO_APIC_DEFAULT_ADDRESS;
Quoted 8 lines
Quoted 5 lines

It's a bit fragile, but that for sure simpler and can work.

Jan, Paolo,
Are you ok with this approach?

I think extending memory_region_find is a good idea anyway, and at this point I don't see a reason to do the above change...

The reasoning was in the part that Igor cut off:

"Later, when it's possible to change the address via PIIX3 registers, we can adjust the base and pass that properly to kioapic and on to KVM.

Quoted 8 lines

Resolving the base address every time when kvm_ioapic_put() is called is also less efficient, assuming of course that the base address changes less often than the KVM ioctl is used."

I think the patch is a bit flawed. If the guest maps something else on top of IOAPIC, like LAPIC (which should be in CPU specific address spaces, but for now it lives in the global system memory space), the guest could trigger the abort() by resetting the system.

Quoted 7 lines

The questions are, in order of importance:

(1) what privileges would this require in the guest? Answer: a lot.

(2) is this likely to happen by chance? Answer: no, not at all.

(3) is there a workaround? Answer: yes, disable in-kernel irqchip.

These questions ask if there is a risk of benevolent guests performing these activities and I agree that the chances are close to zero.

But the interesting question is to ask if a malevolent guest can bring down a VM uncontrollably this way and I think it only needs a few elevated privileges in a guest to do this.

The fix is to avoid abort(), which is a separate issue to whether the address base should be resolved for each KVM ioctl or not.

Quoted 1 line
Quoted 11 lines
Quoted 3 lines

Simply setting IO_APIC_DEFAULT_ADDRESS is also flawed in my opinion. I'm not sure the in-kernel irqchip handles correctly an overlap between the IOAPIC and LAPIC regions, maybe an abort is predictable after all.

At least the guest needs to be stopped. Perhaps we should have a common function which does this and logs the guest error so we can start replacing calls to abort() with it.

Quoted 2 lines

Paolo

It looks like discussion got deviated from what patch does to another issue.

this patch doesn't address/change the way how/when base_address should be set/updated but it has it's benefits as well:
- removes/cleanups access to private field of parent, which allows to convert it to non SysBusDevice
- extended memory_region_find() opens venue for cleaning-up/re-factoring devices that use framebuffer which are forced currently to access system_address_space directly.

Signature · 2 lines
Regards,
  Igor
#73 lore raw

PATCH v5 18/21 target-i386: move IOAPIC to ICC bus

In reply to Igor Mammedov

* inherit IOAPICs from ICCDevice and attach them to ICC bus
* map IOAPIC's mmio at board level via indirect icc-bridge mmio region that provides address space to IOAPIC via icc-bus

---
v3:
* split out introduction of memory_region_get_address() helper
* use qdev_get_parent_bus() to get parent bus
v2:
* do not create IOAPIC in icc-bridge and do not make it QOM child of icc-bridge, just attach it to icc-bus.

---
 hw/cpu/icc_bus.c                  |    6 ++++++
 hw/i386/pc.c                      |   10 ++++++----
 hw/intc/ioapic_common.c           |   14 ++++++++++----
 include/hw/cpu/icc_bus.h          |    1 +
 include/hw/i386/ioapic_internal.h |    6 +++---
 5 files changed, 26 insertions(+), 11 deletions(-)
hw/cpu/icc_bus.c +6 -0
index 9715e07..b9e4f23 100644
@@ -76,6 +76,7 @@ typedef struct ICCBridgeState {
     SysBusDevice parent_obj;
     ICCBus icc_bus;
     MemoryRegion apic_container;
+    MemoryRegion ioapic_container;
 } ICCBridgeState;
 
 #define ICC_BRIGDE(obj) OBJECT_CHECK(ICCBridgeState, (obj), TYPE_ICC_BRIDGE)
@@ -94,6 +95,11 @@ static void icc_bridge_init(Object *obj)
                        APIC_SPACE_SIZE);
     sysbus_init_mmio(sb, &s->apic_container);
     s->icc_bus.apic_address_space = &s->apic_container;
+
+    /* must be second registered region, board maps it by 1 index */
+    memory_region_init(&s->ioapic_container, "icc-ioapic-container", 0x1000);
+    sysbus_init_mmio(sb, &s->ioapic_container);
+    s->icc_bus.ioapic_address_space = &s->ioapic_container;
 }
 
 static const TypeInfo icc_bridge_info = {
hw/i386/pc.c +6 -4
index a09b7ee..5e50127 100644
@@ -1220,20 +1220,22 @@ void ioapic_init_gsi(GSIState *gsi_state, const char *parent_name)
     DeviceState *dev;
     SysBusDevice *d;
     unsigned int i;
+    BusState *b = BUS(object_resolve_path_type("icc-bus", TYPE_ICC_BUS, NULL));
 
     if (kvm_irqchip_in_kernel()) {
-        dev = qdev_create(NULL, "kvm-ioapic");
+        dev = qdev_create(b, "kvm-ioapic");
     } else {
-        dev = qdev_create(NULL, "ioapic");
+        dev = qdev_create(b, "ioapic");
     }
     if (parent_name) {
         object_property_add_child(object_resolve_path(parent_name, NULL),
                                   "ioapic", OBJECT(dev), NULL);
     }
     qdev_init_nofail(dev);
-    d = SYS_BUS_DEVICE(dev);
-    sysbus_mmio_map(d, 0, IO_APIC_DEFAULT_ADDRESS);
 
+    d = SYS_BUS_DEVICE(object_resolve_path_type("icc-bridge", TYPE_ICC_BRIDGE,
+                                                NULL));
+    sysbus_mmio_map(d, 1, IO_APIC_DEFAULT_ADDRESS);
     for (i = 0; i < IOAPIC_NUM_PINS; i++) {
         gsi_state->ioapic_irq[i] = qdev_get_gpio_in(dev, i);
     }
hw/intc/ioapic_common.c +10 -4
index 5c5bb3c..a70d8a2 100644
@@ -57,11 +57,12 @@ static int ioapic_dispatch_post_load(void *opaque, int version_id)
     return 0;
 }
 
-static int ioapic_init_common(SysBusDevice *dev)
+static int ioapic_init_common(ICCDevice *dev)
 {
     IOAPICCommonState *s = IOAPIC_COMMON(dev);
     IOAPICCommonClass *info;
     static int ioapic_no;
+    static bool mmio_registered;
 
     if (ioapic_no >= MAX_IOAPICS) {
         return -1;
@@ -70,7 +71,12 @@ static int ioapic_init_common(SysBusDevice *dev)
     info = IOAPIC_COMMON_GET_CLASS(s);
     info->init(s, ioapic_no);
 
-    sysbus_init_mmio(&s->busdev, &s->io_memory);
+    if (!mmio_registered) {
+        ICCBus *b = ICC_BUS(qdev_get_parent_bus(DEVICE(dev)));
+        memory_region_add_subregion(b->ioapic_address_space, 0, &s->io_memory);
+        mmio_registered = true;
+    }
+
     ioapic_no++;
 
     return 0;
@@ -95,7 +101,7 @@ static const VMStateDescription vmstate_ioapic_common = {
 
 static void ioapic_common_class_init(ObjectClass *klass, void *data)
 {
-    SysBusDeviceClass *sc = SYS_BUS_DEVICE_CLASS(klass);
+    ICCDeviceClass *sc = ICC_DEVICE_CLASS(klass);
     DeviceClass *dc = DEVICE_CLASS(klass);
 
     sc->init = ioapic_init_common;
@@ -105,7 +111,7 @@ static void ioapic_common_class_init(ObjectClass *klass, void *data)
 
 static const TypeInfo ioapic_common_type = {
     .name = TYPE_IOAPIC_COMMON,
-    .parent = TYPE_SYS_BUS_DEVICE,
+    .parent = TYPE_ICC_DEVICE,
     .instance_size = sizeof(IOAPICCommonState),
     .class_size = sizeof(IOAPICCommonClass),
     .class_init = ioapic_common_class_init,
include/hw/cpu/icc_bus.h +1 -0
index 53ef223..947125d 100644
@@ -35,6 +35,7 @@ typedef struct ICCBus {
 
     /*< public >*/
     MemoryRegion *apic_address_space;
+    MemoryRegion *ioapic_address_space;
 } ICCBus;
 
 #define ICC_BUS(obj) OBJECT_CHECK(ICCBus, (obj), TYPE_ICC_BUS)
include/hw/i386/ioapic_internal.h +3 -3
index 25576c8..8d5fe3d 100644
@@ -24,7 +24,7 @@
 
 #include "hw/hw.h"
 #include "exec/memory.h"
-#include "hw/sysbus.h"
+#include "hw/cpu/icc_bus.h"
 
 #define MAX_IOAPICS                     1
 
@@ -82,14 +82,14 @@ typedef struct IOAPICCommonState IOAPICCommonState;
      OBJECT_GET_CLASS(IOAPICCommonClass, (obj), TYPE_IOAPIC_COMMON)
 
 typedef struct IOAPICCommonClass {
-    SysBusDeviceClass parent_class;
+    ICCDeviceClass parent_class;
     void (*init)(IOAPICCommonState *s, int instance_no);
     void (*pre_save)(IOAPICCommonState *s);
     void (*post_load)(IOAPICCommonState *s);
 } IOAPICCommonClass;
 
 struct IOAPICCommonState {
-    SysBusDevice busdev;
+    ICCDevice busdev;
     MemoryRegion io_memory;
     uint8_t id;
     uint8_t ioregsel;
Signature · 1 line
1.7.1
#74 lore raw

PATCH v5 19/21 add hot_add_cpu hook to QEMUMachine and export machine_args

In reply to Igor Mammedov

hot_add_cpu hook should be overriden by target that implements cpu hot-add via cpu-add QMP command.

Make machine_args available to machine init code, it allows to centralize cpu_model starage instead of adding target specific globals to keep it.

---
 include/hw/boards.h |    3 +++
 vl.c                |    6 +++++-
 2 files changed, 8 insertions(+), 1 deletions(-)
include/hw/boards.h +3 -0
index 425bdc7..de8f92a 100644
@@ -18,6 +18,8 @@ typedef struct QEMUMachineInitArgs {
     const char *cpu_model;
 } QEMUMachineInitArgs;
 
+extern QEMUMachineInitArgs *machine_args;
+
 typedef void QEMUMachineInitFunc(QEMUMachineInitArgs *args);
 
 typedef void QEMUMachineResetFunc(void);
@@ -43,6 +45,7 @@ typedef struct QEMUMachine {
     GlobalProperty *compat_props;
     struct QEMUMachine *next;
     const char *hw_version;
+    void (*hot_add_cpu)(const int64_t id, Error **errp);
 } QEMUMachine;
 
 int qemu_register_machine(QEMUMachine *m);
vl.c +5 -1
index 5612c33..6a612e6 100644
@@ -179,6 +179,8 @@ int main(int argc, char **argv)
 #define MAX_VIRTIO_CONSOLES 1
 #define MAX_SCLP_CONSOLES 1
 
+QEMUMachineInitArgs *machine_args;
+
 static const char *data_dir[16];
 static int data_dir_idx;
 const char *bios_name = NULL;
@@ -4295,13 +4297,15 @@ int main(int argc, char **argv, char **envp)
                                  .kernel_cmdline = kernel_cmdline,
                                  .initrd_filename = initrd_filename,
                                  .cpu_model = cpu_model };
+    machine_args = &args;
+    current_machine = machine;
+
     machine->init(&args);
 
     cpu_synchronize_all_post_init();
 
     set_numa_modes();
 
-    current_machine = machine;
 
     /* init USB devices */
     if (usb_enabled(false)) {
Signature · 1 line
1.7.1
#75 lore raw

Re: [Qemu-devel] [PATCH 19/21] add hot_add_cpu hook to QEMUMachine and export machine_args

Am 23.04.2013 10:29, schrieb Igor Mammedov:

Quoted 48 lines

hot_add_cpu hook should be overriden by target that implements cpu hot-add via cpu-add QMP command.

Make machine_args available to machine init code, it allows to centralize cpu_model starage instead of adding target specific globals to keep it.

---
 include/hw/boards.h |    3 +++
 vl.c                |    6 +++++-
 2 files changed, 8 insertions(+), 1 deletions(-)
include/hw/boards.h +3 -0
index 425bdc7..de8f92a 100644
@@ -18,6 +18,8 @@ typedef struct QEMUMachineInitArgs {
     const char *cpu_model;
 } QEMUMachineInitArgs;
 
+extern QEMUMachineInitArgs *machine_args;
+
 typedef void QEMUMachineInitFunc(QEMUMachineInitArgs *args);
 
 typedef void QEMUMachineResetFunc(void);
@@ -43,6 +45,7 @@ typedef struct QEMUMachine {
     GlobalProperty *compat_props;
     struct QEMUMachine *next;
     const char *hw_version;
+    void (*hot_add_cpu)(const int64_t id, Error **errp);
 } QEMUMachine;
 
 int qemu_register_machine(QEMUMachine *m);
vl.c +2 -0
index 5612c33..6a612e6 100644
@@ -179,6 +179,8 @@ int main(int argc, char **argv)
 #define MAX_VIRTIO_CONSOLES 1
 #define MAX_SCLP_CONSOLES 1
 
+QEMUMachineInitArgs *machine_args;
+
 static const char *data_dir[16];
 static int data_dir_idx;
 const char *bios_name = NULL;

vl.c int main(int argc, char **argv, char **envp) · line 4300

Quoted 4 lines
vl.c +1 -0
@@ -4295,13 +4297,15 @@ int main(int argc, char **argv, char **envp)
                                  .kernel_cmdline = kernel_cmdline,
                                  .initrd_filename = initrd_filename,
                                  .cpu_model = cpu_model };
+    machine_args = &args;

args is a stack variable, so making it global does not strike me as a good idea... (I did have a patch to split main() in two at one time)

vl.c int main(int argc, char **argv, char **envp) · line 4304

Quoted 9 lines
vl.c +2 -1
+    current_machine = machine;
+
     machine->init(&args);
 
     cpu_synchronize_all_post_init();
 
     set_numa_modes();
 
-    current_machine = machine;

Did you look up why it was placed here? Eduardo?
I'm not strongly opinionated but I wonder whether we may want to leave it after machine->init?

Andreas

Quoted 4 lines
vl.c +0 -0
     /* init USB devices */
     if (usb_enabled(false)) {
 
Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#76 lore raw

Re: [Qemu-devel] [PATCH 19/21] add hot_add_cpu hook to QEMUMachine and export machine_args

Quoted 43 lines · On Wed, 24 Apr 2013 19:25:17 +0200 Andreas Färber <[email protected]> wrote:

Am 23.04.2013 10:29, schrieb Igor Mammedov:

Quoted 42 lines

hot_add_cpu hook should be overriden by target that implements cpu hot-add via cpu-add QMP command.

Make machine_args available to machine init code, it allows to centralize cpu_model starage instead of adding target specific globals to keep it.

---
 include/hw/boards.h |    3 +++
 vl.c                |    6 +++++-
 2 files changed, 8 insertions(+), 1 deletions(-)
include/hw/boards.h +3 -0
index 425bdc7..de8f92a 100644
@@ -18,6 +18,8 @@ typedef struct QEMUMachineInitArgs {
     const char *cpu_model;
 } QEMUMachineInitArgs;
 
+extern QEMUMachineInitArgs *machine_args;
+
 typedef void QEMUMachineInitFunc(QEMUMachineInitArgs *args);
 
 typedef void QEMUMachineResetFunc(void);
@@ -43,6 +45,7 @@ typedef struct QEMUMachine {
     GlobalProperty *compat_props;
     struct QEMUMachine *next;
     const char *hw_version;
+    void (*hot_add_cpu)(const int64_t id, Error **errp);
 } QEMUMachine;
 
 int qemu_register_machine(QEMUMachine *m);
vl.c +0 -0
index 5612c33..6a612e6 100644
@@ -179,6 +179,8 @@ int main(int argc, char **argv)
 #define MAX_VIRTIO_CONSOLES 1
 #define MAX_SCLP_CONSOLES 1
 
Quoted 13 lines

vl.c int main(int argc, char **argv, char **envp) · line 4300

Quoted 10 lines
vl.c +3 -0
+QEMUMachineInitArgs *machine_args;
+
 static const char *data_dir[16];
 static int data_dir_idx;
 const char *bios_name = NULL;
@@ -4295,13 +4297,15 @@ int main(int argc, char **argv, char **envp)
                                  .kernel_cmdline = kernel_cmdline,
                                  .initrd_filename = initrd_filename,
                                  .cpu_model = cpu_model };
+    machine_args = &args;

args is a stack variable, so making it global does not strike me as a good idea... (I did have a patch to split main() in two at one time)

True, but it lives till main() exits, and by that time nothing accesses it. I can make it global though it will produce more code churn.

Quoted 7 lines
Quoted 7 lines
vl.c +2 -0
+    current_machine = machine;
+
     machine->init(&args);
 
     cpu_synchronize_all_post_init();
 
Quoted 7 lines

vl.c int main(int argc, char **argv, char **envp) · line 4304

Quoted 3 lines
vl.c +0 -1
     set_numa_modes();
 
-    current_machine = machine;

Did you look up why it was placed here? Eduardo?
I'm not strongly opinionated but I wonder whether we may want to leave it after machine->init?

it was added by 6f338c34 for the sake of device hotplug. Moving it earlier allows to use it during machine->init() as well, it's improves variable utilization. next patch sets hot_add_cpu hook from machine->init() for x86 target.

Quoted 13 lines

Andreas

Quoted 4 lines
vl.c +0 -0
     /* init USB devices */
     if (usb_enabled(false)) {
 
Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
Signature · 2 lines
Regards,
  Igor
#77 lore raw

Re: [Qemu-devel] [PATCH 19/21] add hot_add_cpu hook to QEMUMachine and export machine_args

In reply to Andreas Färber
Quoted 63 lines · Andreas Färber

Am 23.04.2013 10:29, schrieb Igor Mammedov:

Quoted 48 lines

hot_add_cpu hook should be overriden by target that implements cpu hot-add via cpu-add QMP command.

Make machine_args available to machine init code, it allows to centralize cpu_model starage instead of adding target specific globals to keep it.

---
 include/hw/boards.h |    3 +++
 vl.c                |    6 +++++-
 2 files changed, 8 insertions(+), 1 deletions(-)
include/hw/boards.h +3 -0
index 425bdc7..de8f92a 100644
@@ -18,6 +18,8 @@ typedef struct QEMUMachineInitArgs {
     const char *cpu_model;
 } QEMUMachineInitArgs;
 
+extern QEMUMachineInitArgs *machine_args;
+
 typedef void QEMUMachineInitFunc(QEMUMachineInitArgs *args);
 
 typedef void QEMUMachineResetFunc(void);
@@ -43,6 +45,7 @@ typedef struct QEMUMachine {
     GlobalProperty *compat_props;
     struct QEMUMachine *next;
     const char *hw_version;
+    void (*hot_add_cpu)(const int64_t id, Error **errp);
 } QEMUMachine;
 
 int qemu_register_machine(QEMUMachine *m);
vl.c +2 -0
index 5612c33..6a612e6 100644
@@ -179,6 +179,8 @@ int main(int argc, char **argv)
 #define MAX_VIRTIO_CONSOLES 1
 #define MAX_SCLP_CONSOLES 1
 
+QEMUMachineInitArgs *machine_args;
+
 static const char *data_dir[16];
 static int data_dir_idx;
 const char *bios_name = NULL;

vl.c int main(int argc, char **argv, char **envp) · line 4300

Quoted 4 lines
vl.c +1 -0
@@ -4295,13 +4297,15 @@ int main(int argc, char **argv, char **envp)
                                  .kernel_cmdline = kernel_cmdline,
                                  .initrd_filename = initrd_filename,
                                  .cpu_model = cpu_model };
+    machine_args = &args;

args is a stack variable, so making it global does not strike me as a good idea... (I did have a patch to split main() in two at one time)

Quoted 6 lines
vl.c +2 -0
+    current_machine = machine;
+
     machine->init(&args);
 
     cpu_synchronize_all_post_init();
 
Quoted 7 lines

vl.c int main(int argc, char **argv, char **envp) · line 4304

Quoted 3 lines
vl.c +0 -1
     set_numa_modes();
 
-    current_machine = machine;

Did you look up why it was placed here? Eduardo?
I'm not strongly opinionated but I wonder whether we may want to leave it after machine->init?

It was always there since the first time I looked at that code. But I believe it makes sense to set it earlier so initialization code can use the machine object.

On the other hand, I would really like to reduce usage of global variables during initialization, and instead pass a QEMUMachineInitArgs argument to init functions that need it. But I understand that sometimes we need to be pragmatic and a global variable is a simpler approach (if considered just a temporary solution).

Signature · 1 line
Eduardo
#78 lore raw

PATCH v5 20/21 target-i386: implement machine->hot_add_cpu hook

In reply to Igor Mammedov
---
 hw/i386/pc.c |   22 ++++++++++++++++++++++
 1 files changed, 22 insertions(+), 0 deletions(-)
hw/i386/pc.c +22 -0
index 5e50127..b649ed5 100644
@@ -54,6 +54,7 @@
 #include "qemu/config-file.h"
 #include "hw/acpi/acpi.h"
 #include "hw/cpu/icc_bus.h"
+#include "hw/boards.h"
 
 /* debug PC/ISA interrupts */
 //#define DEBUG_IRQ
@@ -914,6 +915,25 @@ static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
     return cpu;
 }
 
+static void do_cpu_hot_add(const int64_t id, Error **errp)
+{
+    int64_t apic_id = x86_cpu_apic_id_from_index(id);
+
+    if (cpu_exists(apic_id)) {
+        error_setg(errp, "Unable to add CPU: %" PRIi64
+                   ", it already exists", id);
+        return;
+    }
+
+    if (id >= max_cpus) {
+        error_setg(errp, "Unable to add CPU: %" PRIi64
+                   ", max allowed: %d", id, max_cpus - 1);
+        return;
+    }
+
+    pc_new_cpu(machine_args->cpu_model, apic_id, errp);
+}
+
 void pc_cpus_init(const char *cpu_model)
 {
     int i;
@@ -928,7 +948,9 @@ void pc_cpus_init(const char *cpu_model)
 #else
         cpu_model = "qemu32";
 #endif
+        machine_args->cpu_model = cpu_model;
     }
+    current_machine->hot_add_cpu = do_cpu_hot_add;
 
     icc_bridge = SYS_BUS_DEVICE(object_resolve_path_type("icc-bridge",
                                                  TYPE_ICC_BRIDGE, NULL));
Signature · 1 line
1.7.1
#79 lore raw

Re: [Qemu-devel] [PATCH 20/21] target-i386: implement machine->hot_add_cpu hook

Am 23.04.2013 10:29, schrieb Igor Mammedov:

Quoted 24 lines
---
 hw/i386/pc.c |   22 ++++++++++++++++++++++
 1 files changed, 22 insertions(+), 0 deletions(-)
hw/i386/pc.c +4 -0
index 5e50127..b649ed5 100644
@@ -54,6 +54,7 @@
 #include "qemu/config-file.h"
 #include "hw/acpi/acpi.h"
 #include "hw/cpu/icc_bus.h"
+#include "hw/boards.h"
 
 /* debug PC/ISA interrupts */
 //#define DEBUG_IRQ
@@ -914,6 +915,25 @@ static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
     return cpu;
 }
 
+static void do_cpu_hot_add(const int64_t id, Error **errp)
+{
+    int64_t apic_id = x86_cpu_apic_id_from_index(id);

hw/i386/pc.c static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp) · line 930

Quoted 10 lines
hw/i386/pc.c +10 -0
+
+    if (cpu_exists(apic_id)) {
+        error_setg(errp, "Unable to add CPU: %" PRIi64
+                   ", it already exists", id);
+        return;
+    }
+
+    if (id >= max_cpus) {
+        error_setg(errp, "Unable to add CPU: %" PRIi64
+                   ", max allowed: %d", id, max_cpus - 1);

Why -1?

Quoted 10 lines
hw/i386/pc.c +6 -0
+        return;
+    }
+
+    pc_new_cpu(machine_args->cpu_model, apic_id, errp);
+}
+
 void pc_cpus_init(const char *cpu_model)
 {
     int i;

hw/i386/pc.c void pc_cpus_init(const char *cpu_model) · line 951

Quoted 4 lines
hw/i386/pc.c +1 -0
@@ -928,7 +948,9 @@ void pc_cpus_init(const char *cpu_model)
 #else
         cpu_model = "qemu32";
 #endif
+        machine_args->cpu_model = cpu_model;

This could be avoided by changing argument to const char **. :) Caller will have access to QEMUMachineArgs*.

hw/i386/pc.c void pc_cpus_init(const char *cpu_model) · line 953

Quoted 2 lines
hw/i386/pc.c +1 -0
     }
+    current_machine->hot_add_cpu = do_cpu_hot_add;

That kind of answers my previous question, but why can't we have this statically in the QEMUMachine(s)?

Andreas

Quoted 4 lines
hw/i386/pc.c +0 -0
     icc_bridge = SYS_BUS_DEVICE(object_resolve_path_type("icc-bridge",
                                                  TYPE_ICC_BRIDGE, NULL));
 
Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#80 lore raw

Re: [Qemu-devel] [PATCH 20/21] target-i386: implement machine->hot_add_cpu hook

Quoted 7 lines · Andreas Färber
Quoted 7 lines
hw/i386/pc.c +7 -0
+static void do_cpu_hot_add(const int64_t id, Error **errp)
+{
+    int64_t apic_id = x86_cpu_apic_id_from_index(id);
+
+    if (cpu_exists(apic_id)) {
+        error_setg(errp, "Unable to add CPU: %" PRIi64
+                   ", it already exists", id);
Quoted 8 lines

hw/i386/pc.c static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp) · line 930

Quoted 6 lines
hw/i386/pc.c +6 -0
+        return;
+    }
+
+    if (id >= max_cpus) {
+        error_setg(errp, "Unable to add CPU: %" PRIi64
+                   ", max allowed: %d", id, max_cpus - 1);

Why -1?

max_cpus-1 makes sense to me, as the message is about "maximum ID allowed", not "maximum number of CPUs".

Quoted 10 lines
Quoted 10 lines
hw/i386/pc.c +6 -0
+        return;
+    }
+
+    pc_new_cpu(machine_args->cpu_model, apic_id, errp);
+}
+
 void pc_cpus_init(const char *cpu_model)
 {
     int i;
Quoted 8 lines

hw/i386/pc.c void pc_cpus_init(const char *cpu_model) · line 951

Quoted 5 lines
hw/i386/pc.c +1 -0
@@ -928,7 +948,9 @@ void pc_cpus_init(const char *cpu_model)
 #else
         cpu_model = "qemu32";
 #endif
+        machine_args->cpu_model = cpu_model;

This could be avoided by changing argument to const char **. :) Caller will have access to QEMUMachineArgs*.

I would prefer to simply pass a pointer to QEMUMachineArgs to pc_cpus_init(), instead of const char **.

Signature · 1 line
Eduardo
#81 lore raw

PATCH v5 21/21 QMP: add cpu-add command

In reply to Igor Mammedov

Adds "cpu-add id=xxx" QMP command.

cpu-add's "id" argument is a CPU number in a range [0..max-cpus)

Example QMP command:
-> { "execute": "cpu-add", "arguments": { "id": 2 } } <- { "return": {} }

---
v6:
* added valid values description to qapi schema
* split out cpu_hot_add hooks introduction into separate patch
* split out implementation of cpu_hot_add for target-i386
v5:
* accept id=[0..max_cpus) range in cpu-add command
v4:
* merge "qmp: add cpu-add qmp command" & "target-i386: implement CPU hot-add" patches
* move notifier call to CPUCLass.realize()
* add hook cpu_hot_add to QEMUMachine
* make QEMUMachineInitArgs global and keep default cpu_model there

v3:
* it appears that 'online/offline' in cpu-set are confusing people with what command actually does and users might have to distinguish if 'offline' is not implemented by parsing error message. To simplify things replace cpu-set with cpu-add command to show more clear what command does and just add cpu-del when CPU remove is implemented.

v2:
* s/cpu_set/cpu-set/
* qmp doc style fix
* use bool type instead of opencodding online/offline string

---
 qapi-schema.json |   11 +++++++++++
 qmp-commands.hx  |   23 +++++++++++++++++++++++
 qmp.c            |   10 ++++++++++
 3 files changed, 44 insertions(+), 0 deletions(-)
qapi-schema.json +11 -0
index 751d3c2..11a5c82 100644
@@ -1387,6 +1387,17 @@
 { 'command': 'cpu', 'data': {'index': 'int'} }
 
 ##
+# @cpu-add
+#
+# Adds CPU with specified ID
+#
+# @id: ID of CPU to be created, valid values [0..max_cpus)
+#
+# Returns: Nothing on success
+##
+{ 'command': 'cpu-add', 'data': {'id': 'int'} }
+
+##
 # @memsave:
 #
 # Save a portion of guest memory to a file.
qmp-commands.hx +23 -0
index 4d65422..1e5d299 100644
@@ -385,6 +385,29 @@ Note: CPUs' indexes are obtained with the 'query-cpus' command.
 EQMP
 
     {
+        .name       = "cpu-add",
+        .args_type  = "id:i",
+        .mhandler.cmd_new = qmp_marshal_input_cpu_add,
+    },
+
+SQMP
+cpu-add
+-------
+
+Adds virtual cpu
+
+Arguments:
+
+- "id": cpu id (json-int)
+
+Example:
+
+-> { "execute": "cpu-add", "arguments": { "id": 2 } }
+<- { "return": {} }
+
+EQMP
+
+    {
         .name       = "memsave",
         .args_type  = "val:l,size:i,filename:s,cpu:i?",
         .mhandler.cmd_new = qmp_marshal_input_memsave,
qmp.c +10 -0
index ed6c7ef..dd34be6 100644
@@ -24,6 +24,7 @@
 #include "hw/qdev.h"
 #include "sysemu/blockdev.h"
 #include "qom/qom-qobject.h"
+#include "hw/boards.h"
 
 NameInfo *qmp_query_name(Error **errp)
 {
@@ -108,6 +109,15 @@ void qmp_cpu(int64_t index, Error **errp)
     /* Just do nothing */
 }
 
+void qmp_cpu_add(int64_t id, Error **errp)
+{
+    if (current_machine->hot_add_cpu) {
+        current_machine->hot_add_cpu(id, errp);
+    } else {
+        error_setg(errp, "Not supported");
+    }
+}
+
 #ifndef CONFIG_VNC
 /* If VNC support is enabled, the "true" query-vnc command is
    defined in the VNC subsystem */
Signature · 1 line
1.7.1
Acked-by on v5 21/21
#82 lore raw

Re: [Qemu-devel] [PATCH 21/21] QMP: add cpu-add command

Quoted 9 lines · On Tue, 23 Apr 2013 10:29:55 +0200 Igor Mammedov <[email protected]> wrote:

Adds "cpu-add id=xxx" QMP command.

cpu-add's "id" argument is a CPU number in a range [0..max-cpus)

Example QMP command:
-> { "execute": "cpu-add", "arguments": { "id": 2 } } <- { "return": {} }

QMP parts (this and the previous patch) is trivial:

Just wondering, no HMP command?

Quoted 115 lines

---
v6:
* added valid values description to qapi schema
* split out cpu_hot_add hooks introduction into separate patch
* split out implementation of cpu_hot_add for target-i386
v5:
* accept id=[0..max_cpus) range in cpu-add command
v4:
* merge "qmp: add cpu-add qmp command" & "target-i386: implement CPU hot-add" patches
* move notifier call to CPUCLass.realize()
* add hook cpu_hot_add to QEMUMachine
* make QEMUMachineInitArgs global and keep default cpu_model there

v3:
* it appears that 'online/offline' in cpu-set are confusing people with what command actually does and users might have to distinguish if 'offline' is not implemented by parsing error message. To simplify things replace cpu-set with cpu-add command to show more clear what command does and just add cpu-del when CPU remove is implemented.

v2:
* s/cpu_set/cpu-set/
* qmp doc style fix
* use bool type instead of opencodding online/offline string

---
 qapi-schema.json |   11 +++++++++++
 qmp-commands.hx  |   23 +++++++++++++++++++++++
 qmp.c            |   10 ++++++++++
 3 files changed, 44 insertions(+), 0 deletions(-)
qapi-schema.json +11 -0
index 751d3c2..11a5c82 100644
@@ -1387,6 +1387,17 @@
 { 'command': 'cpu', 'data': {'index': 'int'} }
 
 ##
+# @cpu-add
+#
+# Adds CPU with specified ID
+#
+# @id: ID of CPU to be created, valid values [0..max_cpus)
+#
+# Returns: Nothing on success
+##
+{ 'command': 'cpu-add', 'data': {'id': 'int'} }
+
+##
 # @memsave:
 #
 # Save a portion of guest memory to a file.
qmp-commands.hx +23 -0
index 4d65422..1e5d299 100644
@@ -385,6 +385,29 @@ Note: CPUs' indexes are obtained with the 'query-cpus' command.
 EQMP
 
     {
+        .name       = "cpu-add",
+        .args_type  = "id:i",
+        .mhandler.cmd_new = qmp_marshal_input_cpu_add,
+    },
+
+SQMP
+cpu-add
+-------
+
+Adds virtual cpu
+
+Arguments:
+
+- "id": cpu id (json-int)
+
+Example:
+
+-> { "execute": "cpu-add", "arguments": { "id": 2 } }
+<- { "return": {} }
+
+EQMP
+
+    {
         .name       = "memsave",
         .args_type  = "val:l,size:i,filename:s,cpu:i?",
         .mhandler.cmd_new = qmp_marshal_input_memsave,
qmp.c +10 -0
index ed6c7ef..dd34be6 100644
@@ -24,6 +24,7 @@
 #include "hw/qdev.h"
 #include "sysemu/blockdev.h"
 #include "qom/qom-qobject.h"
+#include "hw/boards.h"
 
 NameInfo *qmp_query_name(Error **errp)
 {
@@ -108,6 +109,15 @@ void qmp_cpu(int64_t index, Error **errp)
     /* Just do nothing */
 }
 
+void qmp_cpu_add(int64_t id, Error **errp)
+{
+    if (current_machine->hot_add_cpu) {
+        current_machine->hot_add_cpu(id, errp);
+    } else {
+        error_setg(errp, "Not supported");
+    }
+}
+
 #ifndef CONFIG_VNC
 /* If VNC support is enabled, the "true" query-vnc command is
    defined in the VNC subsystem */
#83 lore raw

Re: [Qemu-devel] [PATCH 21/21] QMP: add cpu-add command

Quoted 11 lines · On Tue, 23 Apr 2013 09:26:22 -0400 Luiz Capitulino <[email protected]>…
Quoted 8 lines · On Tue, 23 Apr 2013 10:29:55 +0200 Igor Mammedov <[email protected]> wrote:

Adds "cpu-add id=xxx" QMP command.

cpu-add's "id" argument is a CPU number in a range [0..max-cpus)

Example QMP command:
-> { "execute": "cpu-add", "arguments": { "id": 2 } } <- { "return": {} }

Quoted 7 lines
Quoted 1 line

QMP parts (this and the previous patch) is trivial:

Just wondering, no HMP command?

Nope, libvirt does prefers QMP and humans could use QMP shell to the same effect. In shining future humans could use device_add when implemented from monitor, provided it's not deprecated by then.

Quoted 119 lines
Quoted 117 lines

---
v6:
* added valid values description to qapi schema
* split out cpu_hot_add hooks introduction into separate patch
* split out implementation of cpu_hot_add for target-i386
v5:
* accept id=[0..max_cpus) range in cpu-add command
v4:
* merge "qmp: add cpu-add qmp command" & "target-i386: implement CPU
hot-add" patches
* move notifier call to CPUCLass.realize()
* add hook cpu_hot_add to QEMUMachine
* make QEMUMachineInitArgs global and keep default cpu_model there

v3:
* it appears that 'online/offline' in cpu-set are confusing people with what command actually does and users might have to distinguish if 'offline' is not implemented by parsing error message. To simplify things replace cpu-set with cpu-add command to show more clear what command does and just add cpu-del when CPU remove is implemented.

v2:
* s/cpu_set/cpu-set/
* qmp doc style fix
* use bool type instead of opencodding online/offline string

---
 qapi-schema.json |   11 +++++++++++
 qmp-commands.hx  |   23 +++++++++++++++++++++++
 qmp.c            |   10 ++++++++++
 3 files changed, 44 insertions(+), 0 deletions(-)
qapi-schema.json +11 -0
index 751d3c2..11a5c82 100644
@@ -1387,6 +1387,17 @@
 { 'command': 'cpu', 'data': {'index': 'int'} }
 
 ##
+# @cpu-add
+#
+# Adds CPU with specified ID
+#
+# @id: ID of CPU to be created, valid values [0..max_cpus)
+#
+# Returns: Nothing on success
+##
+{ 'command': 'cpu-add', 'data': {'id': 'int'} }
+
+##
 # @memsave:
 #
 # Save a portion of guest memory to a file.
qmp-commands.hx +0 -0
index 4d65422..1e5d299 100644
@@ -385,6 +385,29 @@ Note: CPUs' indexes are obtained with the

'query-cpus' command. EQMP

     {
qmp-commands.hx +23 -0
+        .name       = "cpu-add",
+        .args_type  = "id:i",
+        .mhandler.cmd_new = qmp_marshal_input_cpu_add,
+    },
+
+SQMP
+cpu-add
+-------
+
+Adds virtual cpu
+
+Arguments:
+
+- "id": cpu id (json-int)
+
+Example:
+
+-> { "execute": "cpu-add", "arguments": { "id": 2 } }
+<- { "return": {} }
+
+EQMP
+
+    {
         .name       = "memsave",
         .args_type  = "val:l,size:i,filename:s,cpu:i?",
         .mhandler.cmd_new = qmp_marshal_input_memsave,
qmp.c +10 -0
index ed6c7ef..dd34be6 100644
@@ -24,6 +24,7 @@
 #include "hw/qdev.h"
 #include "sysemu/blockdev.h"
 #include "qom/qom-qobject.h"
+#include "hw/boards.h"
 
 NameInfo *qmp_query_name(Error **errp)
 {
@@ -108,6 +109,15 @@ void qmp_cpu(int64_t index, Error **errp)
     /* Just do nothing */
 }
 
+void qmp_cpu_add(int64_t id, Error **errp)
+{
+    if (current_machine->hot_add_cpu) {
+        current_machine->hot_add_cpu(id, errp);
+    } else {
+        error_setg(errp, "Not supported");
+    }
+}
+
 #ifndef CONFIG_VNC
 /* If VNC support is enabled, the "true" query-vnc command is
    defined in the VNC subsystem */
Reviewed-by on v5 21/21
#84 lore raw

Re: [Qemu-devel] [PATCH 21/21] QMP: add cpu-add command

In reply to Igor Mammedov
Quoted 13 lines · Igor Mammedov

Adds "cpu-add id=xxx" QMP command.

cpu-add's "id" argument is a CPU number in a range [0..max-cpus)

Example QMP command:
-> { "execute": "cpu-add", "arguments": { "id": 2 } } <- { "return": {} }

---

+++ b/qapi-schema.json

qapi-schema.json line 1396

Quoted 10 lines
qapi-schema.json +7 -0
@@ -1387,6 +1387,17 @@
 { 'command': 'cpu', 'data': {'index': 'int'} }
 
 ##
+# @cpu-add
+#
+# Adds CPU with specified ID
+#
+# @id: ID of CPU to be created, valid values [0..max_cpus)
+#
+# Returns: Nothing on success

Missing a 'Since 1.5' designation.

qapi-schema.json line 1399

Quoted 3 lines
qapi-schema.json +3 -0
+##
+{ 'command': 'cpu-add', 'data': {'id': 'int'} }
+

As that's a trivial piece of documentation to add, you can use:

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org