Skip to contentSkip to conversation

target-i386: CPU hot-add with cpu-add QMP command

PATCH Applied Igor Mammedov 5 participants 84 messages Apr 11, 2013 – Apr 15, 2013 Kbuild

v3 of 9 Applied Latest is v9 →

lore
Share & tools
lore
#0 lore raw

v3 cover target-i386: CPU hot-add with cpu-add QMP command

Implements alternative way for hot-adding CPU using cpu-add QMP command, wich could be useful until it would be possible to add CPUs via device_add.

All patches except the last are also applicable to device_add aprroach.

To hot-add CPU use following command from qmp-shell:
cpu-add id=[APIC ID]

Available APIC IDs could be looked up at following QOM path:
/machine/icc-bridge/cpu[0..N]
where number in brackets is APIC ID
Usage examples and limitations on proposed cpu-add implementation will be documented at http://wiki.qemu.org/Features/CPUHotplug

git tree for testing: https://github.com/imammedo/qemu/tree/cpu_add.v3

based on qom-cpu tree

v2->v3:
* use local error & propagate_error() instead of operating on passed in errp in several places
* replace CPUClass.get_firmware_id() with CPUClass.get_arch_id()
* leave IOAPIC creation to board and just set bus to icc-bus
* include kvm-stub.o in cpu libary if no KVM is configured
* create resume_vcpu() stub and include it in libqemustub, and use it directly instead of CPU method
* acpi_piix4: s/cpu_add_notifier/cpu_added_notifier/

v1->v2:
* generalize cpu sync to KVM, resume and hot-plug notification and invoke them form CPUClass, to make available to all targets.
* introduce cpu_exists() and CPUClass.get_firmware_id() and use the last one in acpi_piix to make code target independent.
* move IOAPIC to ICC bus, it was suggested and easy to convert.
* leave kvmvapic as SysBusDevice, it doesn't affect hot-plug and created only once for all APIC instances. I haven't found yet good/clean enough way to convert it to ICCDevice. May be follow-up though.
* split one big ICC patch into several, one per converted device
* add cpu_hot_add hook to machine and implement it for target-i386, instead of adding stabs. Could be used by other targets to implement cpu-add.
* pre-allocate links<CPU> for all possible CPUs and make them available at /machine/icc-bridge/cpu[0..N] QOM path, so users could find out possible/free CPU IDs to use in cpu-add command.

Igor Mammedov (19):
target-i386: split out CPU creation and features parsing into

    cpu_x86_create()

cpu: Pass CPUState to *cpu_synchronize_post*()
cpu: make kvm-stub.o a part of CPU library
cpu: call cpu_synchronize_post_init() from CPUClass.realize() if
hotplugged
cpu: resume CPU from CPUClass.cpu_common_realizefn() when it is
hot-plugged
introduce CPU hot-plug notifier
rtc: update rtc_cmos on CPU hot-plug
cpu: introduce get_arch_id() method and override it for target-i386
cpu: add helper cpu_exists(), to check if CPU with specified id exists
acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest
target-i386: introduce apic-id property
introduce ICC bus/device/bridge
target-i386: cpu: attach ICC bus to CPU on its creation
target-i386: replace MSI_SPACE_SIZE with APIC_SPACE_SIZE
target-i386: move APIC to ICC bus
target-i386: move IOAPIC to ICC bus
qdev: set device's parent before calling realize() down inheritance
chain.
target-i386: expose all possible CPUs as /machine/icc-bridge/cpu[0..N]
links
add cpu-add qmp command and implement CPU hot-add for target-i386

 Makefile.target                    |  13 ++--
 cpus.c                             |  15 ++--
 default-configs/i386-softmmu.mak   |   1 +
 default-configs/x86_64-softmmu.mak |   1 +
 hw/acpi/piix4.c                    | 114 +++++++++++++++++++++++++++-
 hw/core/qdev.c                     |   8 +-
 hw/cpu/Makefile.objs               |   1 +
 hw/cpu/icc_bus.c                   | 150 +++++++++++++++++++++++++++++++++++++
 hw/i386/kvm/apic.c                 |   2 +-
 hw/i386/kvm/ioapic.c               |   2 +-
 hw/i386/kvmvapic.c                 |   1 +
 hw/i386/pc.c                       |  74 ++++++++++++++++--
 hw/i386/pc_piix.c                  |   7 ++
 hw/i386/pc_q35.c                   |   7 ++
 hw/intc/apic.c                     |   2 +-
 hw/intc/apic_common.c              |  17 +++--
 hw/intc/ioapic_common.c            |  15 +++-
 hw/timer/mc146818rtc.c             |  12 +++
 hw/xen/xen_apic.c                  |   2 +-
 include/exec/memory.h              |  10 +++
 include/hw/boards.h                |   3 +
 include/hw/i386/apic_internal.h    |   8 +-
 include/hw/i386/icc_bus.h          |  55 ++++++++++++++
 include/hw/i386/ioapic_internal.h  |   6 +-
 include/hw/pci/msi.h               |   2 +
 include/qom/cpu.h                  |  12 +++
 include/sysemu/cpus.h              |   3 +
 include/sysemu/kvm.h               |  30 ++++----
 include/sysemu/sysemu.h            |   3 +
 kvm-all.c                          |   9 +--
 kvm-stub.c                         |   7 +-
 memory.c                           |  11 +++
 qapi-schema.json                   |  11 +++
 qmp-commands.hx                    |  23 ++++++
 qmp.c                              |  10 +++
 qom/cpu.c                          |  45 +++++++++++
 stubs/Makefile.objs                |   1 +
 stubs/resume_vcpu.c                |   6 ++
 target-i386/cpu.c                  |  93 +++++++++++++++++++----
 target-i386/cpu.h                  |   2 +
 vl.c                               |   7 +-
 41 files changed, 717 insertions(+), 84 deletions(-)
 create mode 100644 hw/cpu/icc_bus.c
 create mode 100644 include/hw/i386/icc_bus.h
 create mode 100644 stubs/resume_vcpu.c
Signature · 1 line
1.8.2
#1 lore raw

PATCH v3 1/19 target-i386: split out CPU creation and features parsing into cpu_x86_create()

Move CPU creation and features parsing into a separate cpu_x86_create() function, so that board would be able to set board specific CPU properties before CPU is realized.

Keep cpu_x86_init() for compatibility with the code that uses cpu_init() and doesn't need to modify CPU properties.

---
v2:
* use error_propagate() to rise error up stack

---
 target-i386/cpu.c | 18 +++++++++++++++---
 target-i386/cpu.h |  1 +
 2 files changed, 16 insertions(+), 3 deletions(-)
target-i386/cpu.c +15 -3
index f1ccc72..c28436c 100644
@@ -1564,7 +1564,7 @@ static void cpu_x86_register(X86CPU *cpu, const char *name, Error **errp)
     object_property_set_str(OBJECT(cpu), def->model_id, "model-id", errp);
 }
 
-X86CPU *cpu_x86_init(const char *cpu_model)
+X86CPU *cpu_x86_create(const char *cpu_model, Error **errp)
 {
     X86CPU *cpu = NULL;
     CPUX86State *env;
@@ -1594,13 +1594,25 @@ X86CPU *cpu_x86_init(const char *cpu_model)
         goto out;
     }
 
-    object_property_set_bool(OBJECT(cpu), true, "realized", &error);
+out:
+    error_propagate(errp, error);
+    g_strfreev(model_pieces);
+    return cpu;
+}
+
+X86CPU *cpu_x86_init(const char *cpu_model)
+{
+    Error *error = NULL;
+    X86CPU *cpu;
+
+    cpu = cpu_x86_create(cpu_model, &error);
     if (error) {
         goto out;
     }
 
+    object_property_set_bool(OBJECT(cpu), true, "realized", &error);
+
 out:
-    g_strfreev(model_pieces);
     if (error) {
         fprintf(stderr, "%s\n", error_get_pretty(error));
         error_free(error);
target-i386/cpu.h +1 -0
index 2b4e319..cf1b05c 100644
@@ -896,6 +896,7 @@ typedef struct CPUX86State {
 #include "cpu-qom.h"
 
 X86CPU *cpu_x86_init(const char *cpu_model);
+X86CPU *cpu_x86_create(const char *cpu_model, Error **errp);
 int cpu_x86_exec(CPUX86State *s);
 void x86_cpu_list(FILE *f, fprintf_function cpu_fprintf);
 void x86_cpudef_setup(void);
Signature · 1 line
1.8.2
Reviewed-by on v3 1/19
#2 lore raw

Re: [Qemu-devel] [PATCH 01/19] target-i386: split out CPU creation and features parsing into cpu_x86_create()

Quoted 8 lines · Igor Mammedov

Move CPU creation and features parsing into a separate cpu_x86_create() function, so that board would be able to set board specific CPU properties before CPU is realized.

Keep cpu_x86_init() for compatibility with the code that uses cpu_init() and doesn't need to modify CPU properties.

Nice, much simpler to review than v1. :-)

Quoted 64 lines

---
v2:
* use error_propagate() to rise error up stack

---
 target-i386/cpu.c | 18 +++++++++++++++---
 target-i386/cpu.h |  1 +
 2 files changed, 16 insertions(+), 3 deletions(-)
target-i386/cpu.c +15 -3
index f1ccc72..c28436c 100644
@@ -1564,7 +1564,7 @@ static void cpu_x86_register(X86CPU *cpu, const char *name, Error **errp)
     object_property_set_str(OBJECT(cpu), def->model_id, "model-id", errp);
 }
 
-X86CPU *cpu_x86_init(const char *cpu_model)
+X86CPU *cpu_x86_create(const char *cpu_model, Error **errp)
 {
     X86CPU *cpu = NULL;
     CPUX86State *env;
@@ -1594,13 +1594,25 @@ X86CPU *cpu_x86_init(const char *cpu_model)
         goto out;
     }
 
-    object_property_set_bool(OBJECT(cpu), true, "realized", &error);
+out:
+    error_propagate(errp, error);
+    g_strfreev(model_pieces);
+    return cpu;
+}
+
+X86CPU *cpu_x86_init(const char *cpu_model)
+{
+    Error *error = NULL;
+    X86CPU *cpu;
+
+    cpu = cpu_x86_create(cpu_model, &error);
     if (error) {
         goto out;
     }
 
+    object_property_set_bool(OBJECT(cpu), true, "realized", &error);
+
 out:
-    g_strfreev(model_pieces);
     if (error) {
         fprintf(stderr, "%s\n", error_get_pretty(error));
         error_free(error);
target-i386/cpu.h +1 -0
index 2b4e319..cf1b05c 100644
@@ -896,6 +896,7 @@ typedef struct CPUX86State {
 #include "cpu-qom.h"
 
 X86CPU *cpu_x86_init(const char *cpu_model);
+X86CPU *cpu_x86_create(const char *cpu_model, Error **errp);
 int cpu_x86_exec(CPUX86State *s);
 void x86_cpu_list(FILE *f, fprintf_function cpu_fprintf);
 void x86_cpudef_setup(void);
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
#3 lore raw

Re: [Qemu-devel] [PATCH 01/19] target-i386: split out CPU creation and features parsing into cpu_x86_create()

In reply to Igor Mammedov

Am 11.04.2013 16:51, schrieb Igor Mammedov:

Quoted 11 lines

Move CPU creation and features parsing into a separate cpu_x86_create() function, so that board would be able to set board specific CPU properties before CPU is realized.

Keep cpu_x86_init() for compatibility with the code that uses cpu_init() and doesn't need to modify CPU properties.

---
v2:
* use error_propagate() to rise error up stack

Thanks, applied to qom-cpu (shortening the subject):
https://github.com/afaerber/qemu-cpu/commits/qom-cpu

Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#4 lore raw

PATCH v3 2/19 cpu: Pass CPUState to *cpu_synchronize_post*()

In reply to Igor Mammedov

... so it could be called from without requiring CPUArchState

---
 cpus.c               |  4 ++--
 include/sysemu/kvm.h | 12 ++++++------
 kvm-all.c            |  8 ++------
 kvm-stub.c           |  4 ++--
 4 files changed, 12 insertions(+), 16 deletions(-)
cpus.c +2 -2
index e919dd7..9b9a32f 100644
@@ -419,7 +419,7 @@ void cpu_synchronize_all_post_reset(void)
     CPUArchState *cpu;
 
     for (cpu = first_cpu; cpu; cpu = cpu->next_cpu) {
-        cpu_synchronize_post_reset(cpu);
+        cpu_synchronize_post_reset(ENV_GET_CPU(cpu));
     }
 }
 
@@ -428,7 +428,7 @@ void cpu_synchronize_all_post_init(void)
     CPUArchState *cpu;
 
     for (cpu = first_cpu; cpu; cpu = cpu->next_cpu) {
-        cpu_synchronize_post_init(cpu);
+        cpu_synchronize_post_init(ENV_GET_CPU(cpu));
     }
 }
 
include/sysemu/kvm.h +6 -6
index f2d97b5..495e6f8 100644
@@ -250,8 +250,8 @@ int kvm_check_extension(KVMState *s, unsigned int extension);
 uint32_t kvm_arch_get_supported_cpuid(KVMState *env, uint32_t function,
                                       uint32_t index, int reg);
 void kvm_cpu_synchronize_state(CPUArchState *env);
-void kvm_cpu_synchronize_post_reset(CPUArchState *env);
-void kvm_cpu_synchronize_post_init(CPUArchState *env);
+void kvm_cpu_synchronize_post_reset(CPUState *cpu);
+void kvm_cpu_synchronize_post_init(CPUState *cpu);
 
 /* generic hooks - to be moved/refactored once there are more users */
 
@@ -262,17 +262,17 @@ static inline void cpu_synchronize_state(CPUArchState *env)
     }
 }
 
-static inline void cpu_synchronize_post_reset(CPUArchState *env)
+static inline void cpu_synchronize_post_reset(CPUState *cpu)
 {
     if (kvm_enabled()) {
-        kvm_cpu_synchronize_post_reset(env);
+        kvm_cpu_synchronize_post_reset(cpu);
     }
 }
 
-static inline void cpu_synchronize_post_init(CPUArchState *env)
+static inline void cpu_synchronize_post_init(CPUState *cpu)
 {
     if (kvm_enabled()) {
-        kvm_cpu_synchronize_post_init(env);
+        kvm_cpu_synchronize_post_init(cpu);
     }
 }
 
kvm-all.c +2 -6
index 9b433d3..fc4e17c 100644
@@ -1510,18 +1510,14 @@ void kvm_cpu_synchronize_state(CPUArchState *env)
     }
 }
 
-void kvm_cpu_synchronize_post_reset(CPUArchState *env)
+void kvm_cpu_synchronize_post_reset(CPUState *cpu)
 {
-    CPUState *cpu = ENV_GET_CPU(env);
-
     kvm_arch_put_registers(cpu, KVM_PUT_RESET_STATE);
     cpu->kvm_vcpu_dirty = false;
 }
 
-void kvm_cpu_synchronize_post_init(CPUArchState *env)
+void kvm_cpu_synchronize_post_init(CPUState *cpu)
 {
-    CPUState *cpu = ENV_GET_CPU(env);
-
     kvm_arch_put_registers(cpu, KVM_PUT_FULL_STATE);
     cpu->kvm_vcpu_dirty = false;
 }
kvm-stub.c +2 -2
index 760aadc..82875dd 100644
@@ -42,11 +42,11 @@ void kvm_cpu_synchronize_state(CPUArchState *env)
 {
 }
 
-void kvm_cpu_synchronize_post_reset(CPUArchState *env)
+void kvm_cpu_synchronize_post_reset(CPUState *env)
 {
 }
 
-void kvm_cpu_synchronize_post_init(CPUArchState *env)
+void kvm_cpu_synchronize_post_init(CPUState *cpu)
 {
 }
 
Signature · 1 line
1.8.2
#5 lore raw

Re: [Qemu-devel] [PATCH 02/19] cpu: Pass CPUState to *cpu_synchronize_post*()

Am 11.04.2013 16:51, schrieb Igor Mammedov:

Quoted 4 lines

... so it could be called from without requiring CPUArchState

Thanks a lot, applied to qom-cpu (with one change below):
https://github.com/afaerber/qemu-cpu/commits/qom-cpu

I believe this is the missing prerequisite to cleaning up after my monitor qemu_get_cpu() patch.

kvm-stub.c void kvm_cpu_synchronize_state(CPUArchState *env) · line 45

Quoted 10 lines
kvm-stub.c +1 -1
index 760aadc..82875dd 100644
@@ -42,11 +42,11 @@ void kvm_cpu_synchronize_state(CPUArchState *env)
 {
 }
 
-void kvm_cpu_synchronize_post_reset(CPUArchState *env)
+void kvm_cpu_synchronize_post_reset(CPUState *env)

Renamed argument to cpu for consistency.

kvm-stub.c void kvm_cpu_synchronize_state(CPUArchState *env) · line 52

Quoted 8 lines
kvm-stub.c +1 -1
 {
 }
 
-void kvm_cpu_synchronize_post_init(CPUArchState *env)
+void kvm_cpu_synchronize_post_init(CPUState *cpu)
 {
 }
 

Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#6 lore raw

PATCH v3 3/19 cpu: make kvm-stub.o a part of CPU library

In reply to Igor Mammedov
---
 Makefile.target      | 13 +++++++------
 include/hw/pci/msi.h |  2 ++
 include/sysemu/kvm.h |  4 ++--
 kvm-stub.c           |  2 ++
 4 files changed, 13 insertions(+), 8 deletions(-)
Makefile.target +7 -6
index 2bd6d14..18f348c 100644
@@ -64,6 +64,12 @@ all: $(PROGS) stap
 # Dummy command so that make thinks it has done something
 	@true
 
+CONFIG_NO_PCI = $(if $(subst n,,$(CONFIG_PCI)),n,y)
+CONFIG_NO_KVM = $(if $(subst n,,$(CONFIG_KVM)),n,y)
+CONFIG_NO_XEN = $(if $(subst n,,$(CONFIG_XEN)),n,y)
+CONFIG_NO_GET_MEMORY_MAPPING = $(if $(subst n,,$(CONFIG_HAVE_GET_MEMORY_MAPPING)),n,y)
+CONFIG_NO_CORE_DUMP = $(if $(subst n,,$(CONFIG_HAVE_CORE_DUMP)),n,y)
+
 #########################################################
 # cpu emulator library
 obj-y = exec.o translate-all.o cpu-exec.o
@@ -74,6 +80,7 @@ obj-y += fpu/softfloat.o
 obj-y += target-$(TARGET_BASE_ARCH)/
 obj-y += disas.o
 obj-$(CONFIG_GDBSTUB_XML) += gdbstub-xml.o
+obj-$(CONFIG_NO_KVM) += kvm-stub.o
 
 #########################################################
 # Linux user emulator target
@@ -102,12 +109,6 @@ endif #CONFIG_BSD_USER
 #########################################################
 # System emulator target
 ifdef CONFIG_SOFTMMU
-CONFIG_NO_PCI = $(if $(subst n,,$(CONFIG_PCI)),n,y)
-CONFIG_NO_KVM = $(if $(subst n,,$(CONFIG_KVM)),n,y)
-CONFIG_NO_XEN = $(if $(subst n,,$(CONFIG_XEN)),n,y)
-CONFIG_NO_GET_MEMORY_MAPPING = $(if $(subst n,,$(CONFIG_HAVE_GET_MEMORY_MAPPING)),n,y)
-CONFIG_NO_CORE_DUMP = $(if $(subst n,,$(CONFIG_HAVE_CORE_DUMP)),n,y)
-
 obj-y += arch_init.o cpus.o monitor.o gdbstub.o balloon.o ioport.o
 obj-y += qtest.o
 obj-y += hw/
include/hw/pci/msi.h +2 -0
index 81a3848..d4d0450 100644
@@ -21,6 +21,7 @@
 #ifndef QEMU_MSI_H
 #define QEMU_MSI_H
 
+#ifndef CONFIG_USER_ONLY
 #include "qemu-common.h"
 #include "hw/pci/pci.h"
 
@@ -47,4 +48,5 @@ static inline bool msi_present(const PCIDevice *dev)
     return dev->cap_present & QEMU_PCI_CAP_MSI;
 }
 
+#endif
 #endif /* QEMU_MSI_H */
include/sysemu/kvm.h +2 -2
index 495e6f8..93cef28 100644
@@ -144,10 +144,10 @@ int kvm_cpu_exec(CPUArchState *env);
 #if !defined(CONFIG_USER_ONLY)
 void *kvm_vmalloc(ram_addr_t size);
 void *kvm_arch_vmalloc(ram_addr_t size);
-void kvm_setup_guest_memory(void *start, size_t size);
+#endif
 
+void kvm_setup_guest_memory(void *start, size_t size);
 void kvm_flush_coalesced_mmio_buffer(void);
-#endif
 
 int kvm_insert_breakpoint(CPUArchState *current_env, target_ulong addr,
                           target_ulong len, int type);
kvm-stub.c +2 -0
index 82875dd..b34064a 100644
@@ -122,6 +122,7 @@ int kvm_on_sigbus(int code, void *addr)
     return 1;
 }
 
+#ifndef CONFIG_USER_ONLY
 int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg)
 {
     return -ENOSYS;
@@ -145,3 +146,4 @@ int kvm_irqchip_remove_irqfd_notifier(KVMState *s, EventNotifier *n, int virq)
 {
     return -ENOSYS;
 }
+#endif
Signature · 1 line
1.8.2
#7 lore raw

Re: [Qemu-devel] [PATCH 03/19] cpu: make kvm-stub.o a part of CPU library

Quoted 26 lines · Igor Mammedov
---
 Makefile.target      | 13 +++++++------
 include/hw/pci/msi.h |  2 ++
 include/sysemu/kvm.h |  4 ++--
 kvm-stub.c           |  2 ++
 4 files changed, 13 insertions(+), 8 deletions(-)
Makefile.target +6 -0
index 2bd6d14..18f348c 100644
@@ -64,6 +64,12 @@ all: $(PROGS) stap
 # Dummy command so that make thinks it has done something
 	@true
 
+CONFIG_NO_PCI = $(if $(subst n,,$(CONFIG_PCI)),n,y)
+CONFIG_NO_KVM = $(if $(subst n,,$(CONFIG_KVM)),n,y)
+CONFIG_NO_XEN = $(if $(subst n,,$(CONFIG_XEN)),n,y)
+CONFIG_NO_GET_MEMORY_MAPPING = $(if $(subst n,,$(CONFIG_HAVE_GET_MEMORY_MAPPING)),n,y)
+CONFIG_NO_CORE_DUMP = $(if $(subst n,,$(CONFIG_HAVE_CORE_DUMP)),n,y)
+
 #########################################################
 # cpu emulator library
 obj-y = exec.o translate-all.o cpu-exec.o

Makefile.target obj-y += fpu/softfloat.o · line 83

Quoted 4 lines
Makefile.target +1 -0
@@ -74,6 +80,7 @@ obj-y += fpu/softfloat.o
 obj-y += target-$(TARGET_BASE_ARCH)/
 obj-y += disas.o
 obj-$(CONFIG_GDBSTUB_XML) += gdbstub-xml.o
+obj-$(CONFIG_NO_KVM) += kvm-stub.o

The kvm-stub.o line at the "System emulator target" section is redundant now, isn't it? Any specific reason you didn't remove it?

Quoted 21 lines
Makefile.target +0 -6
 #########################################################
 # Linux user emulator target
@@ -102,12 +109,6 @@ endif #CONFIG_BSD_USER
 #########################################################
 # System emulator target
 ifdef CONFIG_SOFTMMU
-CONFIG_NO_PCI = $(if $(subst n,,$(CONFIG_PCI)),n,y)
-CONFIG_NO_KVM = $(if $(subst n,,$(CONFIG_KVM)),n,y)
-CONFIG_NO_XEN = $(if $(subst n,,$(CONFIG_XEN)),n,y)
-CONFIG_NO_GET_MEMORY_MAPPING = $(if $(subst n,,$(CONFIG_HAVE_GET_MEMORY_MAPPING)),n,y)
-CONFIG_NO_CORE_DUMP = $(if $(subst n,,$(CONFIG_HAVE_CORE_DUMP)),n,y)
-
 obj-y += arch_init.o cpus.o monitor.o gdbstub.o balloon.o ioport.o
 obj-y += qtest.o
 obj-y += hw/
include/hw/pci/msi.h +0 -0
index 81a3848..d4d0450 100644

include/hw/pci/msi.h line 24

Quoted 4 lines
include/hw/pci/msi.h +1 -0
@@ -21,6 +21,7 @@
 #ifndef QEMU_MSI_H
 #define QEMU_MSI_H
 
+#ifndef CONFIG_USER_ONLY

I would simply #ifdef the "#include pci/msi.h" line on kvm-stub.c instead. But I guess it doesn't hurt to have the #ifdef here.

Quoted 39 lines
include/hw/pci/msi.h +1 -0
 #include "qemu-common.h"
 #include "hw/pci/pci.h"
 
@@ -47,4 +48,5 @@ static inline bool msi_present(const PCIDevice *dev)
     return dev->cap_present & QEMU_PCI_CAP_MSI;
 }
 
+#endif
 #endif /* QEMU_MSI_H */
include/sysemu/kvm.h +2 -2
index 495e6f8..93cef28 100644
@@ -144,10 +144,10 @@ int kvm_cpu_exec(CPUArchState *env);
 #if !defined(CONFIG_USER_ONLY)
 void *kvm_vmalloc(ram_addr_t size);
 void *kvm_arch_vmalloc(ram_addr_t size);
-void kvm_setup_guest_memory(void *start, size_t size);
+#endif
 
+void kvm_setup_guest_memory(void *start, size_t size);
 void kvm_flush_coalesced_mmio_buffer(void);
-#endif
 
 int kvm_insert_breakpoint(CPUArchState *current_env, target_ulong addr,
                           target_ulong len, int type);
kvm-stub.c +1 -0
index 82875dd..b34064a 100644
@@ -122,6 +122,7 @@ int kvm_on_sigbus(int code, void *addr)
     return 1;
 }
 
+#ifndef CONFIG_USER_ONLY
 int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg)
 {
     return -ENOSYS;

kvm-stub.c int kvm_irqchip_remove_irqfd_notifier(KVMState *s, EventNotifier *n, int virq) · line 149

Quoted 4 lines
kvm-stub.c +1 -0
@@ -145,3 +146,4 @@ int kvm_irqchip_remove_irqfd_notifier(KVMState *s, EventNotifier *n, int virq)
 {
     return -ENOSYS;
 }
+#endif

What about making MSIMessage definition available on CONFIG_USER_ONLY, so we don't even need to #ifdef this part?

Signature · 1 line
Eduardo
#8 lore raw

Re: [Qemu-devel] [PATCH 03/19] cpu: make kvm-stub.o a part of CPU library

Quoted 26 lines · On Thu, 11 Apr 2013 14:45:42 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 25 lines · Igor Mammedov
---
 Makefile.target      | 13 +++++++------
 include/hw/pci/msi.h |  2 ++
 include/sysemu/kvm.h |  4 ++--
 kvm-stub.c           |  2 ++
 4 files changed, 13 insertions(+), 8 deletions(-)
Makefile.target +6 -0
index 2bd6d14..18f348c 100644
@@ -64,6 +64,12 @@ all: $(PROGS) stap
 # Dummy command so that make thinks it has done something
 	@true
 
+CONFIG_NO_PCI = $(if $(subst n,,$(CONFIG_PCI)),n,y)
+CONFIG_NO_KVM = $(if $(subst n,,$(CONFIG_KVM)),n,y)
+CONFIG_NO_XEN = $(if $(subst n,,$(CONFIG_XEN)),n,y)
+CONFIG_NO_GET_MEMORY_MAPPING = $(if $(subst n,,$(CONFIG_HAVE_GET_MEMORY_MAPPING)),n,y)
+CONFIG_NO_CORE_DUMP = $(if $(subst n,,$(CONFIG_HAVE_CORE_DUMP)),n,y)
+
 #########################################################
 # cpu emulator library
 obj-y = exec.o translate-all.o cpu-exec.o
Quoted 8 lines

Makefile.target obj-y += fpu/softfloat.o · line 83

Quoted 5 lines
Makefile.target +1 -0
@@ -74,6 +80,7 @@ obj-y += fpu/softfloat.o
 obj-y += target-$(TARGET_BASE_ARCH)/
 obj-y += disas.o
 obj-$(CONFIG_GDBSTUB_XML) += gdbstub-xml.o
+obj-$(CONFIG_NO_KVM) += kvm-stub.o

The kvm-stub.o line at the "System emulator target" section is redundant now, isn't it? Any specific reason you didn't remove it?

Nope, I just forgot it. Thanks, I'll fix it.

Quoted 25 lines
Quoted 25 lines
Makefile.target +0 -6
 #########################################################
 # Linux user emulator target
@@ -102,12 +109,6 @@ endif #CONFIG_BSD_USER
 #########################################################
 # System emulator target
 ifdef CONFIG_SOFTMMU
-CONFIG_NO_PCI = $(if $(subst n,,$(CONFIG_PCI)),n,y)
-CONFIG_NO_KVM = $(if $(subst n,,$(CONFIG_KVM)),n,y)
-CONFIG_NO_XEN = $(if $(subst n,,$(CONFIG_XEN)),n,y)
-CONFIG_NO_GET_MEMORY_MAPPING = $(if $(subst n,,$(CONFIG_HAVE_GET_MEMORY_MAPPING)),n,y)
-CONFIG_NO_CORE_DUMP = $(if $(subst n,,$(CONFIG_HAVE_CORE_DUMP)),n,y)
-
 obj-y += arch_init.o cpus.o monitor.o gdbstub.o balloon.o ioport.o
 obj-y += qtest.o
 obj-y += hw/
include/hw/pci/msi.h +0 -0
index 81a3848..d4d0450 100644
@@ -21,6 +21,7 @@
 #ifndef QEMU_MSI_H
 #define QEMU_MSI_H
 
Quoted 4 lines

include/hw/pci/msi.h line 24

Quoted 1 line
include/hw/pci/msi.h +1 -0
+#ifndef CONFIG_USER_ONLY

I would simply #ifdef the "#include pci/msi.h" line on kvm-stub.c instead. But I guess it doesn't hurt to have the #ifdef here.

It's more robust to not ifdef at source rather then at places where it's used.

Quoted 35 lines
Quoted 35 lines
include/hw/pci/msi.h +1 -0
 #include "qemu-common.h"
 #include "hw/pci/pci.h"
 
@@ -47,4 +48,5 @@ static inline bool msi_present(const PCIDevice *dev)
     return dev->cap_present & QEMU_PCI_CAP_MSI;
 }
 
+#endif
 #endif /* QEMU_MSI_H */
include/sysemu/kvm.h +2 -2
index 495e6f8..93cef28 100644
@@ -144,10 +144,10 @@ int kvm_cpu_exec(CPUArchState *env);
 #if !defined(CONFIG_USER_ONLY)
 void *kvm_vmalloc(ram_addr_t size);
 void *kvm_arch_vmalloc(ram_addr_t size);
-void kvm_setup_guest_memory(void *start, size_t size);
+#endif
 
+void kvm_setup_guest_memory(void *start, size_t size);
 void kvm_flush_coalesced_mmio_buffer(void);
-#endif
 
 int kvm_insert_breakpoint(CPUArchState *current_env, target_ulong addr,
                           target_ulong len, int type);
kvm-stub.c +0 -0
index 82875dd..b34064a 100644
@@ -122,6 +122,7 @@ int kvm_on_sigbus(int code, void *addr)
     return 1;
 }
 
Quoted 12 lines

kvm-stub.c int kvm_irqchip_remove_irqfd_notifier(KVMState *s, EventNotifier *n, int virq) · line 149

Quoted 9 lines
kvm-stub.c +2 -0
+#ifndef CONFIG_USER_ONLY
 int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg)
 {
     return -ENOSYS;
@@ -145,3 +146,4 @@ int kvm_irqchip_remove_irqfd_notifier(KVMState *s, EventNotifier *n, int virq)
 {
     return -ENOSYS;
 }
+#endif

What about making MSIMessage definition available on CONFIG_USER_ONLY, so we don't even need to #ifdef this part?

it might work, but it causes #includes split which I don't like much. Anyway here is patch on top of this one. If it looks more acceptable to you this way, I'll merge it.

---

include/hw/pci/msi.h +2 -2
index d4d0450..c159a15 100644
@@ -21,15 +21,15 @@
 #ifndef QEMU_MSI_H
 #define QEMU_MSI_H
 
-#ifndef CONFIG_USER_ONLY
 #include "qemu-common.h"
-#include "hw/pci/pci.h"
 
 struct MSIMessage {
     uint64_t address;
     uint32_t data;
 };
 
+#ifndef CONFIG_USER_ONLY
+#include "hw/pci/pci.h"
 extern bool msi_supported;
 
 void msi_set_message(PCIDevice *dev, MSIMessage msg);
kvm-stub.c +0 -2
index 8cb81c4..b5d4446 100644
@@ -123,7 +123,6 @@ int kvm_on_sigbus(int code, void *addr)
     return 1;
 }
 
-#ifndef CONFIG_USER_ONLY
 int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg)
 {
     return -ENOSYS;
@@ -147,4 +146,3 @@ int kvm_irqchip_remove_irqfd_notifier(KVMState *s, EventNotifier *n, int virq)
 {
     return -ENOSYS;
 }
-#endif
Quoted 3 lines
Signature · 2 lines
-- 
Eduardo
Signature · 2 lines
Regards,
  Igor
#9 lore raw

Re: [Qemu-devel] [PATCH 03/19] cpu: make kvm-stub.o a part of CPU library

On Fri, Apr 12, 2013 at 01:17:32PM +0200, Igor Mammedov wrote:
[...]

Quoted 48 lines
Quoted 35 lines
Quoted 35 lines

#include "qemu-common.h"
#include "hw/pci/pci.h"

(quoted diff) +1 -0
@@ -47,4 +48,5 @@ static inline bool msi_present(const PCIDevice *dev)
     return dev->cap_present & QEMU_PCI_CAP_MSI;
 }
 
+#endif
 #endif /* QEMU_MSI_H */
include/sysemu/kvm.h +2 -2
index 495e6f8..93cef28 100644
@@ -144,10 +144,10 @@ int kvm_cpu_exec(CPUArchState *env);
 #if !defined(CONFIG_USER_ONLY)
 void *kvm_vmalloc(ram_addr_t size);
 void *kvm_arch_vmalloc(ram_addr_t size);
-void kvm_setup_guest_memory(void *start, size_t size);
+#endif
 
+void kvm_setup_guest_memory(void *start, size_t size);
 void kvm_flush_coalesced_mmio_buffer(void);
-#endif
 
 int kvm_insert_breakpoint(CPUArchState *current_env, target_ulong addr,
                           target_ulong len, int type);
kvm-stub.c +0 -0
index 82875dd..b34064a 100644
@@ -122,6 +122,7 @@ int kvm_on_sigbus(int code, void *addr)
     return 1;
 }
 
Quoted 12 lines

kvm-stub.c int kvm_irqchip_remove_irqfd_notifier(KVMState *s, EventNotifier *n, int virq) · line 149

Quoted 9 lines
kvm-stub.c +2 -0
+#ifndef CONFIG_USER_ONLY
 int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg)
 {
     return -ENOSYS;
@@ -145,3 +146,4 @@ int kvm_irqchip_remove_irqfd_notifier(KVMState *s, EventNotifier *n, int virq)
 {
     return -ENOSYS;
 }
+#endif

What about making MSIMessage definition available on CONFIG_USER_ONLY, so we don't even need to #ifdef this part?

Quoted 3 lines

it might work, but it causes #includes split which I don't like much. Anyway here is patch on top of this one. If it looks more acceptable to you this way, I'll merge it.

OK, I believe I prefer your original patch. :-)

Having extra CONFIG_USER_ONLY #ifdefs in kvm-stub.c sounds simpler and better than reordering #include lines to appear below declarations in pci/msi.h like you did below.

include/hw/pci/msi.h line 24

Quoted 13 lines

---

include/hw/pci/msi.h +0 -1
index d4d0450..c159a15 100644
@@ -21,15 +21,15 @@
 #ifndef QEMU_MSI_H
 #define QEMU_MSI_H
 
-#ifndef CONFIG_USER_ONLY
 #include "qemu-common.h"

By the way, have you tried to eliminate the qemu-common.h include above? (not important now, but asking just in case you have tried and found issues)

Quoted 38 lines
include/hw/pci/msi.h +2 -1
-#include "hw/pci/pci.h"
 
 struct MSIMessage {
     uint64_t address;
     uint32_t data;
 };
 
+#ifndef CONFIG_USER_ONLY
+#include "hw/pci/pci.h"
 extern bool msi_supported;
 
 void msi_set_message(PCIDevice *dev, MSIMessage msg);
kvm-stub.c +0 -2
index 8cb81c4..b5d4446 100644
@@ -123,7 +123,6 @@ int kvm_on_sigbus(int code, void *addr)
     return 1;
 }
 
-#ifndef CONFIG_USER_ONLY
 int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg)
 {
     return -ENOSYS;
@@ -147,4 +146,3 @@ int kvm_irqchip_remove_irqfd_notifier(KVMState *s, EventNotifier *n, int virq)
 {
     return -ENOSYS;
 }
-#endif
Quoted 4 lines
Signature · 2 lines
-- 
Eduardo
Signature · 2 lines
Regards,
  Igor
Signature · 1 line
Eduardo
#10 lore raw

Re: [Qemu-devel] [PATCH 03/19] cpu: make kvm-stub.o a part of CPU library

Quoted 3 lines · On Fri, 12 Apr 2013 16:25:11 -0300 Eduardo Habkost <[email protected]> wrote:

On Fri, Apr 12, 2013 at 01:17:32PM +0200, Igor Mammedov wrote:
[...]

Quoted 1 line
Quoted 1 line

[...]

Quoted 9 lines
Quoted 9 lines
include/hw/pci/msi.h +0 -0
index d4d0450..c159a15 100644
@@ -21,15 +21,15 @@
 #ifndef QEMU_MSI_H
 #define QEMU_MSI_H
 
Quoted 5 lines

include/hw/pci/msi.h line 24

Quoted 2 lines
include/hw/pci/msi.h +0 -1
-#ifndef CONFIG_USER_ONLY
 #include "qemu-common.h"

By the way, have you tried to eliminate the qemu-common.h include above? (not important now, but asking just in case you have tried and found issues)

No, that would be just another bunch of not related cleanup patches.

Quoted 40 lines
Quoted 39 lines
include/hw/pci/msi.h +2 -1
-#include "hw/pci/pci.h"
 
 struct MSIMessage {
     uint64_t address;
     uint32_t data;
 };
 
+#ifndef CONFIG_USER_ONLY
+#include "hw/pci/pci.h"
 extern bool msi_supported;
 
 void msi_set_message(PCIDevice *dev, MSIMessage msg);
kvm-stub.c +0 -2
index 8cb81c4..b5d4446 100644
@@ -123,7 +123,6 @@ int kvm_on_sigbus(int code, void *addr)
     return 1;
 }
 
-#ifndef CONFIG_USER_ONLY
 int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg)
 {
     return -ENOSYS;
@@ -147,4 +146,3 @@ int kvm_irqchip_remove_irqfd_notifier(KVMState *s,
 EventNotifier *n, int virq) {
     return -ENOSYS;
 }
-#endif
Quoted 4 lines
Signature · 2 lines
-- 
Eduardo
Signature · 2 lines
Regards,
  Igor
#11 lore raw

PATCH v3 4/19 cpu: call cpu_synchronize_post_init() from CPUClass.realize() if hotplugged

In reply to Igor Mammedov

... to synchronize CPU state to KVM

---
v2:
* linking kvm-stub.o to *-user target moved in separate patch

---
 include/sysemu/kvm.h | 18 ++++++++++--------
 kvm-all.c            |  1 +
 kvm-stub.c           |  1 +
 qom/cpu.c            |  4 ++++
 vl.c                 |  1 -
 5 files changed, 16 insertions(+), 9 deletions(-)
include/sysemu/kvm.h +10 -8
index 93cef28..8fb2489 100644
@@ -250,8 +250,6 @@ int kvm_check_extension(KVMState *s, unsigned int extension);
 uint32_t kvm_arch_get_supported_cpuid(KVMState *env, uint32_t function,
                                       uint32_t index, int reg);
 void kvm_cpu_synchronize_state(CPUArchState *env);
-void kvm_cpu_synchronize_post_reset(CPUState *cpu);
-void kvm_cpu_synchronize_post_init(CPUState *cpu);
 
 /* generic hooks - to be moved/refactored once there are more users */
 
@@ -262,6 +260,16 @@ static inline void cpu_synchronize_state(CPUArchState *env)
     }
 }
 
+#if !defined(CONFIG_USER_ONLY)
+int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,
+                                       hwaddr *phys_addr);
+#endif
+
+#endif /* NEED_CPU_H */
+
+void kvm_cpu_synchronize_post_reset(CPUState *cpu);
+void kvm_cpu_synchronize_post_init(CPUState *cpu);
+
 static inline void cpu_synchronize_post_reset(CPUState *cpu)
 {
     if (kvm_enabled()) {
@@ -277,12 +285,6 @@ static inline void cpu_synchronize_post_init(CPUState *cpu)
 }
 
 
-#if !defined(CONFIG_USER_ONLY)
-int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,
-                                       hwaddr *phys_addr);
-#endif
-
-#endif
 int kvm_set_ioeventfd_mmio(int fd, uint32_t adr, uint32_t val, bool assign,
                            uint32_t size);
 
kvm-all.c +1 -0
index fc4e17c..1d17128 100644
@@ -109,6 +109,7 @@ bool kvm_async_interrupts_allowed;
 bool kvm_irqfds_allowed;
 bool kvm_msi_via_irqfd_allowed;
 bool kvm_gsi_routing_allowed;
+bool kvm_allowed;
 
 static const KVMCapabilityInfo kvm_required_capabilites[] = {
     KVM_CAP_INFO(USER_MEMORY),
kvm-stub.c +1 -0
index b34064a..8cb81c4 100644
@@ -23,6 +23,7 @@ bool kvm_async_interrupts_allowed;
 bool kvm_irqfds_allowed;
 bool kvm_msi_via_irqfd_allowed;
 bool kvm_gsi_routing_allowed;
+bool kvm_allowed;
 
 int kvm_init_vcpu(CPUState *cpu)
 {
qom/cpu.c +4 -0
index e242dcb..0c76712 100644
@@ -20,6 +20,7 @@
 
 #include "qom/cpu.h"
 #include "qemu-common.h"
+#include "sysemu/kvm.h"
 
 void cpu_reset_interrupt(CPUState *cpu, int mask)
 {
@@ -57,6 +58,9 @@ static ObjectClass *cpu_common_class_by_name(const char *cpu_model)
 
 static void cpu_common_realizefn(DeviceState *dev, Error **errp)
 {
+    if (dev->hotplugged) {
+        cpu_synchronize_post_init(CPU(dev));
+    }
 }
 
 static void cpu_class_init(ObjectClass *klass, void *data)
vl.c +0 -1
index d694a90..21952c3 100644
@@ -267,7 +267,6 @@ static NotifierList machine_init_done_notifiers =
     NOTIFIER_LIST_INITIALIZER(machine_init_done_notifiers);
 
 static bool tcg_allowed = true;
-bool kvm_allowed;
 bool xen_allowed;
 uint32_t xen_domid;
 enum xen_mode xen_mode = XEN_EMULATE;
Signature · 1 line
1.8.2
#12 lore raw

Re: [Qemu-devel] [PATCH 04/19] cpu: call cpu_synchronize_post_init() from CPUClass.realize() if hotplugged

Quoted 94 lines · Igor Mammedov

... to synchronize CPU state to KVM

---
v2:
* linking kvm-stub.o to *-user target moved in separate patch

---
 include/sysemu/kvm.h | 18 ++++++++++--------
 kvm-all.c            |  1 +
 kvm-stub.c           |  1 +
 qom/cpu.c            |  4 ++++
 vl.c                 |  1 -
 5 files changed, 16 insertions(+), 9 deletions(-)
include/sysemu/kvm.h +10 -8
index 93cef28..8fb2489 100644
@@ -250,8 +250,6 @@ int kvm_check_extension(KVMState *s, unsigned int extension);
 uint32_t kvm_arch_get_supported_cpuid(KVMState *env, uint32_t function,
                                       uint32_t index, int reg);
 void kvm_cpu_synchronize_state(CPUArchState *env);
-void kvm_cpu_synchronize_post_reset(CPUState *cpu);
-void kvm_cpu_synchronize_post_init(CPUState *cpu);
 
 /* generic hooks - to be moved/refactored once there are more users */
 
@@ -262,6 +260,16 @@ static inline void cpu_synchronize_state(CPUArchState *env)
     }
 }
 
+#if !defined(CONFIG_USER_ONLY)
+int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,
+                                       hwaddr *phys_addr);
+#endif
+
+#endif /* NEED_CPU_H */
+
+void kvm_cpu_synchronize_post_reset(CPUState *cpu);
+void kvm_cpu_synchronize_post_init(CPUState *cpu);
+
 static inline void cpu_synchronize_post_reset(CPUState *cpu)
 {
     if (kvm_enabled()) {
@@ -277,12 +285,6 @@ static inline void cpu_synchronize_post_init(CPUState *cpu)
 }
 
 
-#if !defined(CONFIG_USER_ONLY)
-int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,
-                                       hwaddr *phys_addr);
-#endif
-
-#endif
 int kvm_set_ioeventfd_mmio(int fd, uint32_t adr, uint32_t val, bool assign,
                            uint32_t size);
 
kvm-all.c +1 -0
index fc4e17c..1d17128 100644
@@ -109,6 +109,7 @@ bool kvm_async_interrupts_allowed;
 bool kvm_irqfds_allowed;
 bool kvm_msi_via_irqfd_allowed;
 bool kvm_gsi_routing_allowed;
+bool kvm_allowed;
 
 static const KVMCapabilityInfo kvm_required_capabilites[] = {
     KVM_CAP_INFO(USER_MEMORY),
kvm-stub.c +1 -0
index b34064a..8cb81c4 100644
@@ -23,6 +23,7 @@ bool kvm_async_interrupts_allowed;
 bool kvm_irqfds_allowed;
 bool kvm_msi_via_irqfd_allowed;
 bool kvm_gsi_routing_allowed;
+bool kvm_allowed;
 
 int kvm_init_vcpu(CPUState *cpu)
 {
qom/cpu.c +1 -0
index e242dcb..0c76712 100644
@@ -20,6 +20,7 @@
 
 #include "qom/cpu.h"
 #include "qemu-common.h"
+#include "sysemu/kvm.h"
 
 void cpu_reset_interrupt(CPUState *cpu, int mask)
 {

qom/cpu.c static ObjectClass *cpu_common_class_by_name(const char *cpu_model) · line 63

Quoted 6 lines
qom/cpu.c +3 -0
@@ -57,6 +58,9 @@ static ObjectClass *cpu_common_class_by_name(const char *cpu_model)
 
 static void cpu_common_realizefn(DeviceState *dev, Error **errp)
 {
+    if (dev->hotplugged) {
+        cpu_synchronize_post_init(CPU(dev));
+    }

For reference: non-hotplugged CPUs have cpu_synchronize_post_init() called from cpu_synchronize_all_post_init(), that is called from main(), just after the machine->init() call.

That said, have you considered changing the code so that's possible to call cpu_synchronize_post_init() for all CPUs, so we could kill the cpu_synchronize_all_post_init() call entirely?

Maybe the solution for this is to realize non-hotplugged CPUs later, instead of realizing them very early inside the machine init function. If doing is not possible or not enough, we could use VM state notifiers to watch for the right moment to call cpu_synchronize_post_init().

Quoted 18 lines
qom/cpu.c +0 -0
 }
 
 static void cpu_class_init(ObjectClass *klass, void *data)
vl.c +0 -1
index d694a90..21952c3 100644
@@ -267,7 +267,6 @@ static NotifierList machine_init_done_notifiers =
     NOTIFIER_LIST_INITIALIZER(machine_init_done_notifiers);
 
 static bool tcg_allowed = true;
-bool kvm_allowed;
 bool xen_allowed;
 uint32_t xen_domid;
 enum xen_mode xen_mode = XEN_EMULATE;
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
#13 lore raw

Re: [Qemu-devel] [PATCH 04/19] cpu: call cpu_synchronize_post_init() from CPUClass.realize() if hotplugged

Quoted 106 lines · On Thu, 11 Apr 2013 15:33:01 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 94 lines · Igor Mammedov

... to synchronize CPU state to KVM

---
v2:
* linking kvm-stub.o to *-user target moved in separate patch

---
 include/sysemu/kvm.h | 18 ++++++++++--------
 kvm-all.c            |  1 +
 kvm-stub.c           |  1 +
 qom/cpu.c            |  4 ++++
 vl.c                 |  1 -
 5 files changed, 16 insertions(+), 9 deletions(-)
include/sysemu/kvm.h +10 -8
index 93cef28..8fb2489 100644
@@ -250,8 +250,6 @@ int kvm_check_extension(KVMState *s, unsigned int extension);
 uint32_t kvm_arch_get_supported_cpuid(KVMState *env, uint32_t function,
                                       uint32_t index, int reg);
 void kvm_cpu_synchronize_state(CPUArchState *env);
-void kvm_cpu_synchronize_post_reset(CPUState *cpu);
-void kvm_cpu_synchronize_post_init(CPUState *cpu);
 
 /* generic hooks - to be moved/refactored once there are more users */
 
@@ -262,6 +260,16 @@ static inline void cpu_synchronize_state(CPUArchState *env)
     }
 }
 
+#if !defined(CONFIG_USER_ONLY)
+int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,
+                                       hwaddr *phys_addr);
+#endif
+
+#endif /* NEED_CPU_H */
+
+void kvm_cpu_synchronize_post_reset(CPUState *cpu);
+void kvm_cpu_synchronize_post_init(CPUState *cpu);
+
 static inline void cpu_synchronize_post_reset(CPUState *cpu)
 {
     if (kvm_enabled()) {
@@ -277,12 +285,6 @@ static inline void cpu_synchronize_post_init(CPUState *cpu)
 }
 
 
-#if !defined(CONFIG_USER_ONLY)
-int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,
-                                       hwaddr *phys_addr);
-#endif
-
-#endif
 int kvm_set_ioeventfd_mmio(int fd, uint32_t adr, uint32_t val, bool assign,
                            uint32_t size);
 
kvm-all.c +1 -0
index fc4e17c..1d17128 100644
@@ -109,6 +109,7 @@ bool kvm_async_interrupts_allowed;
 bool kvm_irqfds_allowed;
 bool kvm_msi_via_irqfd_allowed;
 bool kvm_gsi_routing_allowed;
+bool kvm_allowed;
 
 static const KVMCapabilityInfo kvm_required_capabilites[] = {
     KVM_CAP_INFO(USER_MEMORY),
kvm-stub.c +1 -0
index b34064a..8cb81c4 100644
@@ -23,6 +23,7 @@ bool kvm_async_interrupts_allowed;
 bool kvm_irqfds_allowed;
 bool kvm_msi_via_irqfd_allowed;
 bool kvm_gsi_routing_allowed;
+bool kvm_allowed;
 
 int kvm_init_vcpu(CPUState *cpu)
 {
qom/cpu.c +1 -0
index e242dcb..0c76712 100644
@@ -20,6 +20,7 @@
 
 #include "qom/cpu.h"
 #include "qemu-common.h"
+#include "sysemu/kvm.h"
 
 void cpu_reset_interrupt(CPUState *cpu, int mask)
 {

qom/cpu.c static ObjectClass *cpu_common_class_by_name(const char *cpu_model) · line 63

Quoted 6 lines
qom/cpu.c +3 -0
@@ -57,6 +58,9 @@ static ObjectClass *cpu_common_class_by_name(const char *cpu_model)
 
 static void cpu_common_realizefn(DeviceState *dev, Error **errp)
 {
+    if (dev->hotplugged) {
+        cpu_synchronize_post_init(CPU(dev));
+    }

For reference: non-hotplugged CPUs have cpu_synchronize_post_init() called from cpu_synchronize_all_post_init(), that is called from main(), just after the machine->init() call.

Quoted 8 lines

That said, have you considered changing the code so that's possible to call cpu_synchronize_post_init() for all CPUs, so we could kill the cpu_synchronize_all_post_init() call entirely?

Maybe the solution for this is to realize non-hotplugged CPUs later, instead of realizing them very early inside the machine init function. If doing is not possible or not enough, we could use VM state notifiers to watch for the right moment to call cpu_synchronize_post_init().

I have not considered it, since question is out of scope CPU hot-plug. It might be feasible to do so later in separate patch and remove "if (dev->hotplugged)" condition tough.

I think it will be more appropriate when machine will QOM-ified, then to make machine.realize() work it had to be done, but now I don't see any gain doing so (providing all the trouble it might cause).

Quoted 22 lines
Quoted 19 lines
qom/cpu.c +0 -0
 }
 
 static void cpu_class_init(ObjectClass *klass, void *data)
vl.c +0 -1
index d694a90..21952c3 100644
@@ -267,7 +267,6 @@ static NotifierList machine_init_done_notifiers =
     NOTIFIER_LIST_INITIALIZER(machine_init_done_notifiers);
 
 static bool tcg_allowed = true;
-bool kvm_allowed;
 bool xen_allowed;
 uint32_t xen_domid;
 enum xen_mode xen_mode = XEN_EMULATE;
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
Signature · 2 lines
Regards,
  Igor
#14 lore raw

Re: [Qemu-devel] [PATCH 04/19] cpu: call cpu_synchronize_post_init() from CPUClass.realize() if hotplugged

Quoted 118 lines · Igor Mammedov
Quoted 106 lines · On Thu, 11 Apr 2013 15:33:01 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 94 lines · Igor Mammedov

... to synchronize CPU state to KVM

---
v2:
* linking kvm-stub.o to *-user target moved in separate patch

---
 include/sysemu/kvm.h | 18 ++++++++++--------
 kvm-all.c            |  1 +
 kvm-stub.c           |  1 +
 qom/cpu.c            |  4 ++++
 vl.c                 |  1 -
 5 files changed, 16 insertions(+), 9 deletions(-)
include/sysemu/kvm.h +10 -8
index 93cef28..8fb2489 100644
@@ -250,8 +250,6 @@ int kvm_check_extension(KVMState *s, unsigned int extension);
 uint32_t kvm_arch_get_supported_cpuid(KVMState *env, uint32_t function,
                                       uint32_t index, int reg);
 void kvm_cpu_synchronize_state(CPUArchState *env);
-void kvm_cpu_synchronize_post_reset(CPUState *cpu);
-void kvm_cpu_synchronize_post_init(CPUState *cpu);
 
 /* generic hooks - to be moved/refactored once there are more users */
 
@@ -262,6 +260,16 @@ static inline void cpu_synchronize_state(CPUArchState *env)
     }
 }
 
+#if !defined(CONFIG_USER_ONLY)
+int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,
+                                       hwaddr *phys_addr);
+#endif
+
+#endif /* NEED_CPU_H */
+
+void kvm_cpu_synchronize_post_reset(CPUState *cpu);
+void kvm_cpu_synchronize_post_init(CPUState *cpu);
+
 static inline void cpu_synchronize_post_reset(CPUState *cpu)
 {
     if (kvm_enabled()) {
@@ -277,12 +285,6 @@ static inline void cpu_synchronize_post_init(CPUState *cpu)
 }
 
 
-#if !defined(CONFIG_USER_ONLY)
-int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,
-                                       hwaddr *phys_addr);
-#endif
-
-#endif
 int kvm_set_ioeventfd_mmio(int fd, uint32_t adr, uint32_t val, bool assign,
                            uint32_t size);
 
kvm-all.c +1 -0
index fc4e17c..1d17128 100644
@@ -109,6 +109,7 @@ bool kvm_async_interrupts_allowed;
 bool kvm_irqfds_allowed;
 bool kvm_msi_via_irqfd_allowed;
 bool kvm_gsi_routing_allowed;
+bool kvm_allowed;
 
 static const KVMCapabilityInfo kvm_required_capabilites[] = {
     KVM_CAP_INFO(USER_MEMORY),
kvm-stub.c +1 -0
index b34064a..8cb81c4 100644
@@ -23,6 +23,7 @@ bool kvm_async_interrupts_allowed;
 bool kvm_irqfds_allowed;
 bool kvm_msi_via_irqfd_allowed;
 bool kvm_gsi_routing_allowed;
+bool kvm_allowed;
 
 int kvm_init_vcpu(CPUState *cpu)
 {
qom/cpu.c +1 -0
index e242dcb..0c76712 100644
@@ -20,6 +20,7 @@
 
 #include "qom/cpu.h"
 #include "qemu-common.h"
+#include "sysemu/kvm.h"
 
 void cpu_reset_interrupt(CPUState *cpu, int mask)
 {

qom/cpu.c static ObjectClass *cpu_common_class_by_name(const char *cpu_model) · line 63

Quoted 6 lines
qom/cpu.c +3 -0
@@ -57,6 +58,9 @@ static ObjectClass *cpu_common_class_by_name(const char *cpu_model)
 
 static void cpu_common_realizefn(DeviceState *dev, Error **errp)
 {
+    if (dev->hotplugged) {
+        cpu_synchronize_post_init(CPU(dev));
+    }

For reference: non-hotplugged CPUs have cpu_synchronize_post_init() called from cpu_synchronize_all_post_init(), that is called from main(), just after the machine->init() call.

Quoted 8 lines

That said, have you considered changing the code so that's possible to call cpu_synchronize_post_init() for all CPUs, so we could kill the cpu_synchronize_all_post_init() call entirely?

Maybe the solution for this is to realize non-hotplugged CPUs later, instead of realizing them very early inside the machine init function. If doing is not possible or not enough, we could use VM state notifiers to watch for the right moment to call cpu_synchronize_post_init().

Quoted 7 lines

I have not considered it, since question is out of scope CPU hot-plug. It might be feasible to do so later in separate patch and remove "if (dev->hotplugged)" condition tough.

I think it will be more appropriate when machine will QOM-ified, then to make machine.realize() work it had to be done, but now I don't see any gain doing so (providing all the trouble it might cause).

OK, sounds like a good justification to me.

I would love to have a device model where there's no difference between hotplugged and non-hotplugged devices. But this is hard to achieve.

Signature · 1 line
Eduardo
#15 lore raw

Re: [Qemu-devel] [PATCH 04/19] cpu: call cpu_synchronize_post_init() from CPUClass.realize() if hotplugged

In reply to Igor Mammedov

On Thu, Apr 11, 2013 at 04:51:43PM +0200, Igor Mammedov wrote:
[...]

Quoted 17 lines
kvm-all.c +1 -0
index fc4e17c..1d17128 100644
@@ -109,6 +109,7 @@ bool kvm_async_interrupts_allowed;
 bool kvm_irqfds_allowed;
 bool kvm_msi_via_irqfd_allowed;
 bool kvm_gsi_routing_allowed;
+bool kvm_allowed;
 
 static const KVMCapabilityInfo kvm_required_capabilites[] = {
     KVM_CAP_INFO(USER_MEMORY),
kvm-stub.c +0 -0
index b34064a..8cb81c4 100644

kvm-stub.c bool kvm_async_interrupts_allowed; · line 26

Quoted 4 lines
kvm-stub.c +1 -0
@@ -23,6 +23,7 @@ bool kvm_async_interrupts_allowed;
 bool kvm_irqfds_allowed;
 bool kvm_msi_via_irqfd_allowed;
 bool kvm_gsi_routing_allowed;
+bool kvm_allowed;

Why do you need kvm_allowed on kvm-stub.c? Isn't simpler and more efficient to define kvm_enabled() as (0) on CONFIG_USER_ONLY?

Signature · 1 line
Eduardo
#16 lore raw

Re: [Qemu-devel] [PATCH 04/19] cpu: call cpu_synchronize_post_init() from CPUClass.realize() if hotplugged

Quoted 12 lines · On Mon, 15 Apr 2013 16:57:10 -0300 Eduardo Habkost <[email protected]> wrote:

On Thu, Apr 11, 2013 at 04:51:43PM +0200, Igor Mammedov wrote:
[...]

Quoted 10 lines
kvm-all.c +1 -0
index fc4e17c..1d17128 100644
@@ -109,6 +109,7 @@ bool kvm_async_interrupts_allowed;
 bool kvm_irqfds_allowed;
 bool kvm_msi_via_irqfd_allowed;
 bool kvm_gsi_routing_allowed;
+bool kvm_allowed;
 
Quoted 14 lines

kvm-stub.c bool kvm_async_interrupts_allowed; · line 26

Quoted 11 lines
kvm-all.c +0 -0
 static const KVMCapabilityInfo kvm_required_capabilites[] = {
     KVM_CAP_INFO(USER_MEMORY),
kvm-stub.c +1 -0
index b34064a..8cb81c4 100644
@@ -23,6 +23,7 @@ bool kvm_async_interrupts_allowed;
 bool kvm_irqfds_allowed;
 bool kvm_msi_via_irqfd_allowed;
 bool kvm_gsi_routing_allowed;
+bool kvm_allowed;

Why do you need kvm_allowed on kvm-stub.c? Isn't simpler and more efficient to define kvm_enabled() as (0) on CONFIG_USER_ONLY?

qom/cpu.o is build only one time for all targets, header ifdef won't work here.

Quoted 2 lines
Signature · 1 line
Eduardo
Signature · 2 lines
Regards,
  Igor
#17 lore raw

PATCH v3 5/19 cpu: resume CPU from CPUClass.cpu_common_realizefn() when it is hot-plugged

In reply to Igor Mammedov
---
 cpus.c                | 11 ++++++++---
 include/sysemu/cpus.h |  3 +++
 qom/cpu.c             |  2 ++
 stubs/Makefile.objs   |  1 +
 stubs/resume_vcpu.c   |  6 ++++++
 5 files changed, 20 insertions(+), 3 deletions(-)
 create mode 100644 stubs/resume_vcpu.c
cpus.c +8 -3
index 9b9a32f..f12624d 100644
@@ -973,6 +973,13 @@ void pause_all_vcpus(void)
     }
 }
 
+void resume_vcpu(CPUState *cpu)
+{
+    cpu->stop = false;
+    cpu->stopped = false;
+    qemu_cpu_kick(cpu);
+}
+
 void resume_all_vcpus(void)
 {
     CPUArchState *penv = first_cpu;
@@ -980,9 +987,7 @@ void resume_all_vcpus(void)
     qemu_clock_enable(vm_clock, true);
     while (penv) {
         CPUState *pcpu = ENV_GET_CPU(penv);
-        pcpu->stop = false;
-        pcpu->stopped = false;
-        qemu_cpu_kick(pcpu);
+        resume_vcpu(pcpu);
         penv = penv->next_cpu;
     }
 }
include/sysemu/cpus.h +3 -0
index 6502488..9437df5 100644
@@ -1,11 +1,14 @@
 #ifndef QEMU_CPUS_H
 #define QEMU_CPUS_H
 
+#include "qom/cpu.h"
+
 /* cpus.c */
 void qemu_init_cpu_loop(void);
 void resume_all_vcpus(void);
 void pause_all_vcpus(void);
 void cpu_stop_current(void);
+void resume_vcpu(CPUState *cpu);
 
 void cpu_synchronize_all_states(void);
 void cpu_synchronize_all_post_reset(void);
qom/cpu.c +2 -0
index 0c76712..2869cfe 100644
@@ -21,6 +21,7 @@
 #include "qom/cpu.h"
 #include "qemu-common.h"
 #include "sysemu/kvm.h"
+#include "sysemu/cpus.h"
 
 void cpu_reset_interrupt(CPUState *cpu, int mask)
 {
@@ -60,6 +61,7 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
 {
     if (dev->hotplugged) {
         cpu_synchronize_post_init(CPU(dev));
+        resume_vcpu(CPU(dev));
     }
 }
 
stubs/Makefile.objs +1 -0
index 9c55b34..28fb4f8 100644
@@ -23,3 +23,4 @@ stub-obj-y += sysbus.o
 stub-obj-y += vm-stop.o
 stub-obj-y += vmstate.o
 stub-obj-$(CONFIG_WIN32) += fd-register.o
+stub-obj-y += resume_vcpu.o
stubs/resume_vcpu.c new file +6 -0
new file mode 100644
index 0000000..383b71a
@@ -0,0 +1,6 @@
+#include "qemu-common.h"
+#include "sysemu/cpus.h"
+
+void resume_vcpu(CPUState *cpu)
+{
+}
Signature · 1 line
1.8.2
#18 lore raw

Re: [Qemu-devel] [PATCH 05/19] cpu: resume CPU from CPUClass.cpu_common_realizefn() when it is hot-plugged

Quoted 29 lines · Igor Mammedov
---
 cpus.c                | 11 ++++++++---
 include/sysemu/cpus.h |  3 +++
 qom/cpu.c             |  2 ++
 stubs/Makefile.objs   |  1 +
 stubs/resume_vcpu.c   |  6 ++++++
 5 files changed, 20 insertions(+), 3 deletions(-)
 create mode 100644 stubs/resume_vcpu.c
cpus.c +7 -0
index 9b9a32f..f12624d 100644
@@ -973,6 +973,13 @@ void pause_all_vcpus(void)
     }
 }
 
+void resume_vcpu(CPUState *cpu)
+{
+    cpu->stop = false;
+    cpu->stopped = false;
+    qemu_cpu_kick(cpu);
+}
+
 void resume_all_vcpus(void)
 {
     CPUArchState *penv = first_cpu;

cpus.c void resume_all_vcpus(void) · line 990

Quoted 7 lines
cpus.c +1 -3
@@ -980,9 +987,7 @@ void resume_all_vcpus(void)
     qemu_clock_enable(vm_clock, true);
     while (penv) {
         CPUState *pcpu = ENV_GET_CPU(penv);
-        pcpu->stop = false;
-        pcpu->stopped = false;
-        qemu_cpu_kick(pcpu);
+        resume_vcpu(pcpu);

If you include just the above chunks in a separate patch, I would promptly send a Reviewed-by line for that.

Quoted 35 lines
cpus.c +0 -0
         penv = penv->next_cpu;
     }
 }
include/sysemu/cpus.h +3 -0
index 6502488..9437df5 100644
@@ -1,11 +1,14 @@
 #ifndef QEMU_CPUS_H
 #define QEMU_CPUS_H
 
+#include "qom/cpu.h"
+
 /* cpus.c */
 void qemu_init_cpu_loop(void);
 void resume_all_vcpus(void);
 void pause_all_vcpus(void);
 void cpu_stop_current(void);
+void resume_vcpu(CPUState *cpu);
 
 void cpu_synchronize_all_states(void);
 void cpu_synchronize_all_post_reset(void);
qom/cpu.c +1 -0
index 0c76712..2869cfe 100644
@@ -21,6 +21,7 @@
 #include "qom/cpu.h"
 #include "qemu-common.h"
 #include "sysemu/kvm.h"
+#include "sysemu/cpus.h"
 
 void cpu_reset_interrupt(CPUState *cpu, int mask)
 {

qom/cpu.c static void cpu_common_realizefn(DeviceState *dev, Error **errp) · line 64

Quoted 4 lines
qom/cpu.c +1 -0
@@ -60,6 +61,7 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
 {
     if (dev->hotplugged) {
         cpu_synchronize_post_init(CPU(dev));
+        resume_vcpu(CPU(dev));

Simlar question to the one I had about cpu_synchronize_post_init():

What about making this not conditional on dev->hotplugged, and simply using VM state notifiers to make sure we resume the CPUs as soon as VM state is RUN_STATE_RUNNING?

Quoted 27 lines
qom/cpu.c +0 -0
     }
 }
 
stubs/Makefile.objs +1 -0
index 9c55b34..28fb4f8 100644
@@ -23,3 +23,4 @@ stub-obj-y += sysbus.o
 stub-obj-y += vm-stop.o
 stub-obj-y += vmstate.o
 stub-obj-$(CONFIG_WIN32) += fd-register.o
+stub-obj-y += resume_vcpu.o
stubs/resume_vcpu.c new file +6 -0
new file mode 100644
index 0000000..383b71a
@@ -0,0 +1,6 @@
+#include "qemu-common.h"
+#include "sysemu/cpus.h"
+
+void resume_vcpu(CPUState *cpu)
+{
+}
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
#19 lore raw

Re: [Qemu-devel] [PATCH 05/19] cpu: resume CPU from CPUClass.cpu_common_realizefn() when it is hot-plugged

Quoted 32 lines · On Thu, 11 Apr 2013 15:36:23 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 31 lines · Igor Mammedov
---
 cpus.c                | 11 ++++++++---
 include/sysemu/cpus.h |  3 +++
 qom/cpu.c             |  2 ++
 stubs/Makefile.objs   |  1 +
 stubs/resume_vcpu.c   |  6 ++++++
 5 files changed, 20 insertions(+), 3 deletions(-)
 create mode 100644 stubs/resume_vcpu.c
cpus.c +7 -0
index 9b9a32f..f12624d 100644
@@ -973,6 +973,13 @@ void pause_all_vcpus(void)
     }
 }
 
+void resume_vcpu(CPUState *cpu)
+{
+    cpu->stop = false;
+    cpu->stopped = false;
+    qemu_cpu_kick(cpu);
+}
+
 void resume_all_vcpus(void)
 {
     CPUArchState *penv = first_cpu;
@@ -980,9 +987,7 @@ void resume_all_vcpus(void)
     qemu_clock_enable(vm_clock, true);
     while (penv) {
Quoted 8 lines

cpus.c void resume_all_vcpus(void) · line 990

Quoted 5 lines
cpus.c +1 -3
         CPUState *pcpu = ENV_GET_CPU(penv);
-        pcpu->stop = false;
-        pcpu->stopped = false;
-        qemu_cpu_kick(pcpu);
+        resume_vcpu(pcpu);

If you include just the above chunks in a separate patch, I would promptly send a Reviewed-by line for that.

sure

Quoted 43 lines
Quoted 36 lines
cpus.c +0 -0
         penv = penv->next_cpu;
     }
 }
include/sysemu/cpus.h +3 -0
index 6502488..9437df5 100644
@@ -1,11 +1,14 @@
 #ifndef QEMU_CPUS_H
 #define QEMU_CPUS_H
 
+#include "qom/cpu.h"
+
 /* cpus.c */
 void qemu_init_cpu_loop(void);
 void resume_all_vcpus(void);
 void pause_all_vcpus(void);
 void cpu_stop_current(void);
+void resume_vcpu(CPUState *cpu);
 
 void cpu_synchronize_all_states(void);
 void cpu_synchronize_all_post_reset(void);
qom/cpu.c +1 -0
index 0c76712..2869cfe 100644
@@ -21,6 +21,7 @@
 #include "qom/cpu.h"
 #include "qemu-common.h"
 #include "sysemu/kvm.h"
+#include "sysemu/cpus.h"
 
 void cpu_reset_interrupt(CPUState *cpu, int mask)
 {

qom/cpu.c static void cpu_common_realizefn(DeviceState *dev, Error **errp) · line 64

Quoted 4 lines
qom/cpu.c +1 -0
@@ -60,6 +61,7 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
 {
     if (dev->hotplugged) {
         cpu_synchronize_post_init(CPU(dev));
+        resume_vcpu(CPU(dev));

Simlar question to the one I had about cpu_synchronize_post_init():

Quoted 3 lines

What about making this not conditional on dev->hotplugged, and simply using VM state notifiers to make sure we resume the CPUs as soon as VM state is RUN_STATE_RUNNING?

answered to similar question in 4/19

Quoted 31 lines
Quoted 28 lines
qom/cpu.c +0 -0
     }
 }
 
stubs/Makefile.objs +1 -0
index 9c55b34..28fb4f8 100644
@@ -23,3 +23,4 @@ stub-obj-y += sysbus.o
 stub-obj-y += vm-stop.o
 stub-obj-y += vmstate.o
 stub-obj-$(CONFIG_WIN32) += fd-register.o
+stub-obj-y += resume_vcpu.o
stubs/resume_vcpu.c new file +6 -0
new file mode 100644
index 0000000..383b71a
@@ -0,0 +1,6 @@
+#include "qemu-common.h"
+#include "sysemu/cpus.h"
+
+void resume_vcpu(CPUState *cpu)
+{
+}
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
Signature · 2 lines
Regards,
  Igor
Reviewed-by on v3 5/19
#20 lore raw

Re: [Qemu-devel] [PATCH 05/19] cpu: resume CPU from CPUClass.cpu_common_realizefn() when it is hot-plugged

In reply to Igor Mammedov
Quoted 1 line · Igor Mammedov
Quoted 102 lines
---
 cpus.c                | 11 ++++++++---
 include/sysemu/cpus.h |  3 +++
 qom/cpu.c             |  2 ++
 stubs/Makefile.objs   |  1 +
 stubs/resume_vcpu.c   |  6 ++++++
 5 files changed, 20 insertions(+), 3 deletions(-)
 create mode 100644 stubs/resume_vcpu.c
cpus.c +8 -3
index 9b9a32f..f12624d 100644
@@ -973,6 +973,13 @@ void pause_all_vcpus(void)
     }
 }
 
+void resume_vcpu(CPUState *cpu)
+{
+    cpu->stop = false;
+    cpu->stopped = false;
+    qemu_cpu_kick(cpu);
+}
+
 void resume_all_vcpus(void)
 {
     CPUArchState *penv = first_cpu;
@@ -980,9 +987,7 @@ void resume_all_vcpus(void)
     qemu_clock_enable(vm_clock, true);
     while (penv) {
         CPUState *pcpu = ENV_GET_CPU(penv);
-        pcpu->stop = false;
-        pcpu->stopped = false;
-        qemu_cpu_kick(pcpu);
+        resume_vcpu(pcpu);
         penv = penv->next_cpu;
     }
 }
include/sysemu/cpus.h +3 -0
index 6502488..9437df5 100644
@@ -1,11 +1,14 @@
 #ifndef QEMU_CPUS_H
 #define QEMU_CPUS_H
 
+#include "qom/cpu.h"
+
 /* cpus.c */
 void qemu_init_cpu_loop(void);
 void resume_all_vcpus(void);
 void pause_all_vcpus(void);
 void cpu_stop_current(void);
+void resume_vcpu(CPUState *cpu);
 
 void cpu_synchronize_all_states(void);
 void cpu_synchronize_all_post_reset(void);
qom/cpu.c +2 -0
index 0c76712..2869cfe 100644
@@ -21,6 +21,7 @@
 #include "qom/cpu.h"
 #include "qemu-common.h"
 #include "sysemu/kvm.h"
+#include "sysemu/cpus.h"
 
 void cpu_reset_interrupt(CPUState *cpu, int mask)
 {
@@ -60,6 +61,7 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
 {
     if (dev->hotplugged) {
         cpu_synchronize_post_init(CPU(dev));
+        resume_vcpu(CPU(dev));
     }
 }
 
stubs/Makefile.objs +1 -0
index 9c55b34..28fb4f8 100644
@@ -23,3 +23,4 @@ stub-obj-y += sysbus.o
 stub-obj-y += vm-stop.o
 stub-obj-y += vmstate.o
 stub-obj-$(CONFIG_WIN32) += fd-register.o
+stub-obj-y += resume_vcpu.o
stubs/resume_vcpu.c new file +6 -0
new file mode 100644
index 0000000..383b71a
@@ -0,0 +1,6 @@
+#include "qemu-common.h"
+#include "sysemu/cpus.h"
+
+void resume_vcpu(CPUState *cpu)
+{
+}
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
#21 lore raw

PATCH v3 6/19 introduce CPU hot-plug notifier

In reply to Igor Mammedov

hot-added CPU will be distributed to acpi_piix4, rtc_cmos and icc_bridge

---
v2:
* move notifier to qom/cpu.c and call it from CPUClass.realize() on hotplug
* remove get_firmware_id() since it belong to other patch

---
 include/sysemu/sysemu.h |  3 +++
 qom/cpu.c               | 12 ++++++++++++
 2 files changed, 15 insertions(+)
include/sysemu/sysemu.h +3 -0
index 6578782..a8c3de1 100644
@@ -152,6 +152,9 @@ void do_pci_device_hot_remove(Monitor *mon, const QDict *qdict);
 /* generic hotplug */
 void drive_hot_add(Monitor *mon, const QDict *qdict);
 
+/* CPU hotplug */
+void qemu_register_cpu_added_notifier(Notifier *notifier);
+
 /* pcie aer error injection */
 void pcie_aer_inject_error_print(Monitor *mon, const QObject *data);
 int do_pcie_aer_inject_error(Monitor *mon,
qom/cpu.c +12 -0
index 2869cfe..fe85960 100644
@@ -22,6 +22,17 @@
 #include "qemu-common.h"
 #include "sysemu/kvm.h"
 #include "sysemu/cpus.h"
+#include "qemu/notify.h"
+#include "sysemu/sysemu.h"
+
+/* CPU hot-plug notifiers */
+static NotifierList cpu_added_notifiers =
+    NOTIFIER_LIST_INITIALIZER(cpu_add_notifiers);
+
+void qemu_register_cpu_added_notifier(Notifier *notifier)
+{
+    notifier_list_add(&cpu_added_notifiers, notifier);
+}
 
 void cpu_reset_interrupt(CPUState *cpu, int mask)
 {
@@ -62,6 +73,7 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
     if (dev->hotplugged) {
         cpu_synchronize_post_init(CPU(dev));
         resume_vcpu(CPU(dev));
+        notifier_list_notify(&cpu_added_notifiers, dev);
     }
 }
 
Signature · 1 line
1.8.2
#22 lore raw

Re: [Qemu-devel] [PATCH 06/19] introduce CPU hot-plug notifier

Quoted 4 lines · Igor Mammedov

hot-added CPU will be distributed to acpi_piix4, rtc_cmos and icc_bridge

Have you considered making a generic device creation/realization notification mechanism, that could work with any device class?

Quoted 56 lines

---
v2:
* move notifier to qom/cpu.c and call it from CPUClass.realize() on hotplug
* remove get_firmware_id() since it belong to other patch

---
 include/sysemu/sysemu.h |  3 +++
 qom/cpu.c               | 12 ++++++++++++
 2 files changed, 15 insertions(+)
include/sysemu/sysemu.h +3 -0
index 6578782..a8c3de1 100644
@@ -152,6 +152,9 @@ void do_pci_device_hot_remove(Monitor *mon, const QDict *qdict);
 /* generic hotplug */
 void drive_hot_add(Monitor *mon, const QDict *qdict);
 
+/* CPU hotplug */
+void qemu_register_cpu_added_notifier(Notifier *notifier);
+
 /* pcie aer error injection */
 void pcie_aer_inject_error_print(Monitor *mon, const QObject *data);
 int do_pcie_aer_inject_error(Monitor *mon,
qom/cpu.c +12 -0
index 2869cfe..fe85960 100644
@@ -22,6 +22,17 @@
 #include "qemu-common.h"
 #include "sysemu/kvm.h"
 #include "sysemu/cpus.h"
+#include "qemu/notify.h"
+#include "sysemu/sysemu.h"
+
+/* CPU hot-plug notifiers */
+static NotifierList cpu_added_notifiers =
+    NOTIFIER_LIST_INITIALIZER(cpu_add_notifiers);
+
+void qemu_register_cpu_added_notifier(Notifier *notifier)
+{
+    notifier_list_add(&cpu_added_notifiers, notifier);
+}
 
 void cpu_reset_interrupt(CPUState *cpu, int mask)
 {
@@ -62,6 +73,7 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
     if (dev->hotplugged) {
         cpu_synchronize_post_init(CPU(dev));
         resume_vcpu(CPU(dev));
+        notifier_list_notify(&cpu_added_notifiers, dev);
     }
 }
 
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
#23 lore raw

Re: [Qemu-devel] [PATCH 06/19] introduce CPU hot-plug notifier

Quoted 8 lines · On Thu, 11 Apr 2013 15:46:47 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 4 lines · Igor Mammedov

hot-added CPU will be distributed to acpi_piix4, rtc_cmos and icc_bridge

Have you considered making a generic device creation/realization notification mechanism, that could work with any device class?

Not really, that would make notification handler a bit more complicated, since they will have to decide if event belong to them.
We could add it in future if there will be need for it.

Quoted 60 lines
Quoted 57 lines

---
v2:
* move notifier to qom/cpu.c and call it from CPUClass.realize() on hotplug
* remove get_firmware_id() since it belong to other patch

---
 include/sysemu/sysemu.h |  3 +++
 qom/cpu.c               | 12 ++++++++++++
 2 files changed, 15 insertions(+)
include/sysemu/sysemu.h +3 -0
index 6578782..a8c3de1 100644
@@ -152,6 +152,9 @@ void do_pci_device_hot_remove(Monitor *mon, const QDict *qdict);
 /* generic hotplug */
 void drive_hot_add(Monitor *mon, const QDict *qdict);
 
+/* CPU hotplug */
+void qemu_register_cpu_added_notifier(Notifier *notifier);
+
 /* pcie aer error injection */
 void pcie_aer_inject_error_print(Monitor *mon, const QObject *data);
 int do_pcie_aer_inject_error(Monitor *mon,
qom/cpu.c +12 -0
index 2869cfe..fe85960 100644
@@ -22,6 +22,17 @@
 #include "qemu-common.h"
 #include "sysemu/kvm.h"
 #include "sysemu/cpus.h"
+#include "qemu/notify.h"
+#include "sysemu/sysemu.h"
+
+/* CPU hot-plug notifiers */
+static NotifierList cpu_added_notifiers =
+    NOTIFIER_LIST_INITIALIZER(cpu_add_notifiers);
+
+void qemu_register_cpu_added_notifier(Notifier *notifier)
+{
+    notifier_list_add(&cpu_added_notifiers, notifier);
+}
 
 void cpu_reset_interrupt(CPUState *cpu, int mask)
 {
@@ -62,6 +73,7 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
     if (dev->hotplugged) {
         cpu_synchronize_post_init(CPU(dev));
         resume_vcpu(CPU(dev));
+        notifier_list_notify(&cpu_added_notifiers, dev);
     }
 }
 
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
Signature · 2 lines
Regards,
  Igor
Reviewed-by
#24 lore raw

Re: [Qemu-devel] [PATCH 06/19] introduce CPU hot-plug notifier

Quoted 6 lines · Igor Mammedov
Quoted 3 lines · On Thu, 11 Apr 2013 15:46:47 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 2 lines · Igor Mammedov

hot-added CPU will be distributed to acpi_piix4, rtc_cmos and icc_bridge

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

Have you considered making a generic device creation/realization notification mechanism, that could work with any device class?

Not really, that would make notification handler a bit more complicated, since they will have to decide if event belong to them.

I don't think it would be complicated. It should be as simple as:

void qdev_register_realize_notifier(DeviceClass *dc, Notifier *notifier)

{
    notifier_list_add(&dc->realize_notifiers, notifier);
}

static void notify_device_realized(ObjectClass *klass, void *opaque)

{
    DeviceState *dev = (DeviceState *)opaque;
    DeviceClass *dc = DEVICE_CLASS(klass);
    notifier_list_notify(&dc->realize_notifiers, dev);
}

static void device_realize(DeviceState *dev, Error **err)

{
    [...]
    object_class_foreach(notify_device_realized, TYPE_DEVICE, true, dev);
}
Quoted 1 line

We could add it in future if there will be need for it.

OK.

Quoted 67 lines
Quoted 61 lines
Quoted 58 lines

---
v2:
* move notifier to qom/cpu.c and call it from CPUClass.realize() on hotplug
* remove get_firmware_id() since it belong to other patch

---
 include/sysemu/sysemu.h |  3 +++
 qom/cpu.c               | 12 ++++++++++++
 2 files changed, 15 insertions(+)
include/sysemu/sysemu.h +3 -0
index 6578782..a8c3de1 100644
@@ -152,6 +152,9 @@ void do_pci_device_hot_remove(Monitor *mon, const QDict *qdict);
 /* generic hotplug */
 void drive_hot_add(Monitor *mon, const QDict *qdict);
 
+/* CPU hotplug */
+void qemu_register_cpu_added_notifier(Notifier *notifier);
+
 /* pcie aer error injection */
 void pcie_aer_inject_error_print(Monitor *mon, const QObject *data);
 int do_pcie_aer_inject_error(Monitor *mon,
qom/cpu.c +12 -0
index 2869cfe..fe85960 100644
@@ -22,6 +22,17 @@
 #include "qemu-common.h"
 #include "sysemu/kvm.h"
 #include "sysemu/cpus.h"
+#include "qemu/notify.h"
+#include "sysemu/sysemu.h"
+
+/* CPU hot-plug notifiers */
+static NotifierList cpu_added_notifiers =
+    NOTIFIER_LIST_INITIALIZER(cpu_add_notifiers);
+
+void qemu_register_cpu_added_notifier(Notifier *notifier)
+{
+    notifier_list_add(&cpu_added_notifiers, notifier);
+}
 
 void cpu_reset_interrupt(CPUState *cpu, int mask)
 {
@@ -62,6 +73,7 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
     if (dev->hotplugged) {
         cpu_synchronize_post_init(CPU(dev));
         resume_vcpu(CPU(dev));
+        notifier_list_notify(&cpu_added_notifiers, dev);
     }
 }
 
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
Signature · 2 lines
Regards,
  Igor
Signature · 1 line
Eduardo
#25 lore raw

PATCH v3 7/19 rtc: update rtc_cmos on CPU hot-plug

In reply to Igor Mammedov

... so that on reboot BIOS could read current available CPU count

v2:
* s/qemu_register_cpu_add_notifier()/qemu_register_cpu_added_notifier()/

---
 hw/timer/mc146818rtc.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
hw/timer/mc146818rtc.c +12 -0
index 69e6844..958ed6b 100644
@@ -82,6 +82,7 @@ typedef struct RTCState {
     Notifier clock_reset_notifier;
     LostTickPolicy lost_tick_policy;
     Notifier suspend_notifier;
+    Notifier cpu_added_notifier;
 } RTCState;
 
 static void rtc_set_time(RTCState *s);
@@ -759,6 +760,14 @@ static void rtc_notify_suspend(Notifier *notifier, void *data)
     rtc_set_memory(&s->dev, 0xF, 0xFE);
 }
 
+static void rtc_notify_cpu_added(Notifier *notifier, void *data)
+{
+    RTCState *s = container_of(notifier, RTCState, cpu_added_notifier);
+
+    /* increment the number of CPUs */
+    s->cmos_data[0x5f] += 1;
+}
+
 static void rtc_reset(void *opaque)
 {
     RTCState *s = opaque;
@@ -852,6 +861,9 @@ static int rtc_initfn(ISADevice *dev)
     s->suspend_notifier.notify = rtc_notify_suspend;
     qemu_register_suspend_notifier(&s->suspend_notifier);
 
+    s->cpu_added_notifier.notify = rtc_notify_cpu_added;
+    qemu_register_cpu_added_notifier(&s->cpu_added_notifier);
+
     memory_region_init_io(&s->io, &cmos_ops, s, "rtc", 2);
     isa_register_ioport(dev, &s->io, base);
 
Signature · 1 line
1.8.2
#26 lore raw

Re: [Qemu-devel] [PATCH 07/19] rtc: update rtc_cmos on CPU hot-plug

Quoted 8 lines · Igor Mammedov

... so that on reboot BIOS could read current available CPU count

v2:
* s/qemu_register_cpu_add_notifier()/qemu_register_cpu_added_notifier()/

---
 hw/timer/mc146818rtc.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

Initialization of the cmos fields (including 0x5F) is done on pc.c:pc_cmos_init(). What about making the field increment inside pc.c as well?

What happens if a CPU is hotplugged after the machine has started but before the guest OS has booted? Are we supposed to make sure the BIOS do the right thing if a CPU is hotplugged before the OS has booted, or this simply won't be supported?

Quoted 41 lines
hw/timer/mc146818rtc.c +12 -0
index 69e6844..958ed6b 100644
@@ -82,6 +82,7 @@ typedef struct RTCState {
     Notifier clock_reset_notifier;
     LostTickPolicy lost_tick_policy;
     Notifier suspend_notifier;
+    Notifier cpu_added_notifier;
 } RTCState;
 
 static void rtc_set_time(RTCState *s);
@@ -759,6 +760,14 @@ static void rtc_notify_suspend(Notifier *notifier, void *data)
     rtc_set_memory(&s->dev, 0xF, 0xFE);
 }
 
+static void rtc_notify_cpu_added(Notifier *notifier, void *data)
+{
+    RTCState *s = container_of(notifier, RTCState, cpu_added_notifier);
+
+    /* increment the number of CPUs */
+    s->cmos_data[0x5f] += 1;
+}
+
 static void rtc_reset(void *opaque)
 {
     RTCState *s = opaque;
@@ -852,6 +861,9 @@ static int rtc_initfn(ISADevice *dev)
     s->suspend_notifier.notify = rtc_notify_suspend;
     qemu_register_suspend_notifier(&s->suspend_notifier);
 
+    s->cpu_added_notifier.notify = rtc_notify_cpu_added;
+    qemu_register_cpu_added_notifier(&s->cpu_added_notifier);
+
     memory_region_init_io(&s->io, &cmos_ops, s, "rtc", 2);
     isa_register_ioport(dev, &s->io, base);
 
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
#27 lore raw

Re: [Qemu-devel] [PATCH 07/19] rtc: update rtc_cmos on CPU hot-plug

Quoted 10 lines · On Thu, 11 Apr 2013 15:59:40 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 8 lines · Igor Mammedov

... so that on reboot BIOS could read current available CPU count

v2:
* s/qemu_register_cpu_add_notifier()/qemu_register_cpu_added_notifier()/

---
 hw/timer/mc146818rtc.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
Quoted 3 lines

Initialization of the cmos fields (including 0x5F) is done on pc.c:pc_cmos_init(). What about making the field increment inside pc.c as well?

I looked at possibility but discarded it because to increment it there initial value should be -1 (field is zero based) which is not obvious, plug ugly casting to singed variable.
Result looked ugly.

Quoted 5 lines

What happens if a CPU is hotplugged after the machine has started but before the guest OS has booted? Are we supposed to make sure the BIOS do the right thing if a CPU is hotplugged before the OS has booted, or this simply won't be supported?

BIOS uses this value to set in ACPI tables what CPUs are present.
1. if hot-plug happens before BIOS reads it then OS will see all CPUs
and SCI it receives will be nop.
2. if hot-plug happens after BIOS reads it, OS will handle SCI as usual and hotplug CPU instead of initializing it smp_boot() time.
BIOS itself has nothing to do with hot-plug, it's OSPM job.

Quoted 45 lines
Quoted 42 lines
hw/timer/mc146818rtc.c +12 -0
index 69e6844..958ed6b 100644
@@ -82,6 +82,7 @@ typedef struct RTCState {
     Notifier clock_reset_notifier;
     LostTickPolicy lost_tick_policy;
     Notifier suspend_notifier;
+    Notifier cpu_added_notifier;
 } RTCState;
 
 static void rtc_set_time(RTCState *s);
@@ -759,6 +760,14 @@ static void rtc_notify_suspend(Notifier *notifier, void *data)
     rtc_set_memory(&s->dev, 0xF, 0xFE);
 }
 
+static void rtc_notify_cpu_added(Notifier *notifier, void *data)
+{
+    RTCState *s = container_of(notifier, RTCState, cpu_added_notifier);
+
+    /* increment the number of CPUs */
+    s->cmos_data[0x5f] += 1;
+}
+
 static void rtc_reset(void *opaque)
 {
     RTCState *s = opaque;
@@ -852,6 +861,9 @@ static int rtc_initfn(ISADevice *dev)
     s->suspend_notifier.notify = rtc_notify_suspend;
     qemu_register_suspend_notifier(&s->suspend_notifier);
 
+    s->cpu_added_notifier.notify = rtc_notify_cpu_added;
+    qemu_register_cpu_added_notifier(&s->cpu_added_notifier);
+
     memory_region_init_io(&s->io, &cmos_ops, s, "rtc", 2);
     isa_register_ioport(dev, &s->io, base);
 
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
Signature · 2 lines
Regards,
  Igor
#28 lore raw

Re: [Qemu-devel] [PATCH 07/19] rtc: update rtc_cmos on CPU hot-plug

Quoted 13 lines · Igor Mammedov
Quoted 10 lines · On Thu, 11 Apr 2013 15:59:40 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 8 lines · Igor Mammedov

... so that on reboot BIOS could read current available CPU count

v2:
* s/qemu_register_cpu_add_notifier()/qemu_register_cpu_added_notifier()/

---
 hw/timer/mc146818rtc.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
Quoted 7 lines
Quoted 3 lines

Initialization of the cmos fields (including 0x5F) is done on pc.c:pc_cmos_init(). What about making the field increment inside pc.c as well?

I looked at possibility but discarded it because to increment it there initial value should be -1 (field is zero based) which is not obvious, plug ugly casting to singed variable.
Result looked ugly.

I was thinking about simply adding exactly the same code with exactly the same logic, but inside pc.c instead of of mc146818rtc.c. Instead of registering the notifier inside rtc_initfn(), register exactly the same notifier with exactly the same code, but inside pc_cmos_init() (that's where 0x5f is initialized).

It would even be safer and easier review and ensure correctness: with this patch, the notifier is registered very early, even before pc_cmos_init() initializes 0x5f to smp_cpus. CPU hotplug events are unlikely to be emitted before pc_cmos_init() is called, but still: why make the initialization ordering so subtle if we don't have to?

Quoted 6 lines
Quoted 6 lines

What happens if a CPU is hotplugged after the machine has started but before the guest OS has booted? Are we supposed to make sure the BIOS do the right thing if a CPU is hotplugged before the OS has booted, or this simply won't be supported?

Quoted 6 lines

BIOS uses this value to set in ACPI tables what CPUs are present.
1. if hot-plug happens before BIOS reads it then OS will see all CPUs
and SCI it receives will be nop.
2. if hot-plug happens after BIOS reads it, OS will handle SCI as usual and hotplug CPU instead of initializing it smp_boot() time.
BIOS itself has nothing to do with hot-plug, it's OSPM job.

Makes sense, thanks.

What happens if the CPU is hotplugged after the BIOS builds the ACPI tables, but long before the OS starts handling ACPI events? Is the OS guaranteed to run the CPU hotplug ACPI method (that's \_GPE._E02 in the DSDT, right?), even if that happens?

Quoted 51 lines
Quoted 46 lines
Quoted 43 lines
hw/timer/mc146818rtc.c +12 -0
index 69e6844..958ed6b 100644
@@ -82,6 +82,7 @@ typedef struct RTCState {
     Notifier clock_reset_notifier;
     LostTickPolicy lost_tick_policy;
     Notifier suspend_notifier;
+    Notifier cpu_added_notifier;
 } RTCState;
 
 static void rtc_set_time(RTCState *s);
@@ -759,6 +760,14 @@ static void rtc_notify_suspend(Notifier *notifier, void *data)
     rtc_set_memory(&s->dev, 0xF, 0xFE);
 }
 
+static void rtc_notify_cpu_added(Notifier *notifier, void *data)
+{
+    RTCState *s = container_of(notifier, RTCState, cpu_added_notifier);
+
+    /* increment the number of CPUs */
+    s->cmos_data[0x5f] += 1;
+}
+
 static void rtc_reset(void *opaque)
 {
     RTCState *s = opaque;
@@ -852,6 +861,9 @@ static int rtc_initfn(ISADevice *dev)
     s->suspend_notifier.notify = rtc_notify_suspend;
     qemu_register_suspend_notifier(&s->suspend_notifier);
 
+    s->cpu_added_notifier.notify = rtc_notify_cpu_added;
+    qemu_register_cpu_added_notifier(&s->cpu_added_notifier);
+
     memory_region_init_io(&s->io, &cmos_ops, s, "rtc", 2);
     isa_register_ioport(dev, &s->io, base);
 
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
Signature · 2 lines
Regards,
  Igor
Signature · 1 line
Eduardo
#29 lore raw

Re: [Qemu-devel] [PATCH 07/19] rtc: update rtc_cmos on CPU hot-plug

Quoted 28 lines · On Fri, 12 Apr 2013 10:35:53 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 13 lines · Igor Mammedov
Quoted 10 lines · On Thu, 11 Apr 2013 15:59:40 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 8 lines · Igor Mammedov

... so that on reboot BIOS could read current available CPU count

v2:
* s/qemu_register_cpu_add_notifier()/qemu_register_cpu_added_notifier()/

---
 hw/timer/mc146818rtc.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
Quoted 7 lines
Quoted 3 lines

Initialization of the cmos fields (including 0x5F) is done on pc.c:pc_cmos_init(). What about making the field increment inside pc.c as well?

I looked at possibility but discarded it because to increment it there initial value should be -1 (field is zero based) which is not obvious, plug ugly casting to singed variable.
Result looked ugly.

I was thinking about simply adding exactly the same code with exactly the same logic, but inside pc.c instead of of mc146818rtc.c. Instead of registering the notifier inside rtc_initfn(), register exactly the same notifier with exactly the same code, but inside pc_cmos_init() (that's where 0x5f is initialized).

Quoted 4 lines

It would even be safer and easier review and ensure correctness: with this patch, the notifier is registered very early, even before pc_cmos_init() initializes 0x5f to smp_cpus. CPU hotplug events are unlikely to be emitted before pc_cmos_init() is called, but still: why

it isn't be called, hot-add is available only after machine initialized.

Quoted 1 line

make the initialization ordering so subtle if we don't have to?

Currently cmos init doesn't look like proper QOM object and has 3 stage
initialization: realize(), then pc_cmos_init() the last pc_cmos_init_late(). The last 2 calls are made after realize(), setting various properties. Which looks wrong from QOM perspective, so I'm against of stuffing more internal stuff in arbitrary places. We should do opposite instead.

If you look at mc146818rtc.c or hw/acpi/piix4.c, all notifiers are private to object and registered at realize() time. It looks like initialization order of mc146818rtc should be fixed, instead of adapting new code to it.

So since this patch doesn't break or violate anything in current code, I'd like to leave it as it is.

Quoted 12 lines
Quoted 11 lines
Quoted 5 lines

What happens if a CPU is hotplugged after the machine has started but before the guest OS has booted? Are we supposed to make sure the BIOS do the right thing if a CPU is hotplugged before the OS has booted, or this simply won't be supported?

BIOS uses this value to set in ACPI tables what CPUs are present.
1. if hot-plug happens before BIOS reads it then OS will see all CPUs
and SCI it receives will be nop.
2. if hot-plug happens after BIOS reads it, OS will handle SCI as usual and hotplug CPU instead of initializing it smp_boot() time.
BIOS itself has nothing to do with hot-plug, it's OSPM job.

Quoted 6 lines

Makes sense, thanks.

What happens if the CPU is hotplugged after the BIOS builds the ACPI tables, but long before the OS starts handling ACPI events? Is the OS guaranteed to run the CPU hotplug ACPI method (that's \_GPE._E02 in the DSDT, right?), even if that happens?

Theoretically interrupt should not disappear on it's own and OS should pick it up. But to say it for sure I need to test this case. If SCI will be lost for some reason, then OS won't notice new CPU until another CPU hot-plug event happens.

Quoted 54 lines
Quoted 51 lines
Quoted 46 lines
Quoted 43 lines
hw/timer/mc146818rtc.c +12 -0
index 69e6844..958ed6b 100644
@@ -82,6 +82,7 @@ typedef struct RTCState {
     Notifier clock_reset_notifier;
     LostTickPolicy lost_tick_policy;
     Notifier suspend_notifier;
+    Notifier cpu_added_notifier;
 } RTCState;
 
 static void rtc_set_time(RTCState *s);
@@ -759,6 +760,14 @@ static void rtc_notify_suspend(Notifier *notifier, void *data)
     rtc_set_memory(&s->dev, 0xF, 0xFE);
 }
 
+static void rtc_notify_cpu_added(Notifier *notifier, void *data)
+{
+    RTCState *s = container_of(notifier, RTCState, cpu_added_notifier);
+
+    /* increment the number of CPUs */
+    s->cmos_data[0x5f] += 1;
+}
+
 static void rtc_reset(void *opaque)
 {
     RTCState *s = opaque;
@@ -852,6 +861,9 @@ static int rtc_initfn(ISADevice *dev)
     s->suspend_notifier.notify = rtc_notify_suspend;
     qemu_register_suspend_notifier(&s->suspend_notifier);
 
+    s->cpu_added_notifier.notify = rtc_notify_cpu_added;
+    qemu_register_cpu_added_notifier(&s->cpu_added_notifier);
+
     memory_region_init_io(&s->io, &cmos_ops, s, "rtc", 2);
     isa_register_ioport(dev, &s->io, base);
 
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
Signature · 2 lines
Regards,
  Igor
Signature · 1 line
Eduardo
Signature · 2 lines
Regards,
  Igor
#30 lore raw

Re: [Qemu-devel] [PATCH 07/19] rtc: update rtc_cmos on CPU hot-plug

Quoted 35 lines · Igor Mammedov
Quoted 28 lines · On Fri, 12 Apr 2013 10:35:53 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 13 lines · Igor Mammedov
Quoted 10 lines · On Thu, 11 Apr 2013 15:59:40 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 8 lines · Igor Mammedov

... so that on reboot BIOS could read current available CPU count

v2:
* s/qemu_register_cpu_add_notifier()/qemu_register_cpu_added_notifier()/

---
 hw/timer/mc146818rtc.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
Quoted 7 lines
Quoted 3 lines

Initialization of the cmos fields (including 0x5F) is done on pc.c:pc_cmos_init(). What about making the field increment inside pc.c as well?

I looked at possibility but discarded it because to increment it there initial value should be -1 (field is zero based) which is not obvious, plug ugly casting to singed variable.
Result looked ugly.

I was thinking about simply adding exactly the same code with exactly the same logic, but inside pc.c instead of of mc146818rtc.c. Instead of registering the notifier inside rtc_initfn(), register exactly the same notifier with exactly the same code, but inside pc_cmos_init() (that's where 0x5f is initialized).

Quoted 4 lines

It would even be safer and easier review and ensure correctness: with this patch, the notifier is registered very early, even before pc_cmos_init() initializes 0x5f to smp_cpus. CPU hotplug events are unlikely to be emitted before pc_cmos_init() is called, but still: why

Quoted 8 lines

it isn't be called, hot-add is available only after machine initialized.

Quoted 1 line

make the initialization ordering so subtle if we don't have to?

Currently cmos init doesn't look like proper QOM object and has 3 stage
initialization: realize(), then pc_cmos_init() the last pc_cmos_init_late(). The last 2 calls are made after realize(), setting various properties. Which looks wrong from QOM perspective, so I'm against of stuffing more internal stuff in arbitrary places. We should do opposite instead.

True, but as we already have this weird 3-stage initialization process and we won't fix it really soon, I would really prefer to keep parts of the code that are closely related and depend on each other in the same part of the code.

Quoted 7 lines

If you look at mc146818rtc.c or hw/acpi/piix4.c, all notifiers are private to object and registered at realize() time. It looks like initialization order of mc146818rtc should be fixed, instead of adapting new code to it.

So since this patch doesn't break or violate anything in current code, I'd like to leave it as it is.

If you insist into making the mc146818rtc device take care of maintaining the 0x5f value by itself, why not doing:

    s->cmos_data[0x5f] = smp_cpus - 1;

inside rtc_initfn() instead of pc_cmos_init() as well?

This would be one additional step towards making pc_cmos_init() be replaced by QOM-based code (if that's what you want to do in the long term).

Quoted 15 lines
Quoted 15 lines
Quoted 6 lines
Quoted 6 lines

What happens if a CPU is hotplugged after the machine has started but before the guest OS has booted? Are we supposed to make sure the BIOS do the right thing if a CPU is hotplugged before the OS has booted, or this simply won't be supported?

Quoted 6 lines

BIOS uses this value to set in ACPI tables what CPUs are present.
1. if hot-plug happens before BIOS reads it then OS will see all CPUs
and SCI it receives will be nop.
2. if hot-plug happens after BIOS reads it, OS will handle SCI as usual and hotplug CPU instead of initializing it smp_boot() time.
BIOS itself has nothing to do with hot-plug, it's OSPM job.

Makes sense, thanks.

Quoted 8 lines
Quoted 4 lines

What happens if the CPU is hotplugged after the BIOS builds the ACPI tables, but long before the OS starts handling ACPI events? Is the OS guaranteed to run the CPU hotplug ACPI method (that's \_GPE._E02 in the DSDT, right?), even if that happens?

Theoretically interrupt should not disappear on it's own and OS should pick it up. But to say it for sure I need to test this case. If SCI will be lost for some reason, then OS won't notice new CPU until another CPU hot-plug event happens.

Makes sense, thanks.

(Anyway, if an interrupt gets lost, it's probably a BIOS or OS bug).

Quoted 1 line

[...]

Signature · 1 line
Eduardo
#31 lore raw

Re: [Qemu-devel] [PATCH 07/19] rtc: update rtc_cmos on CPU hot-plug

Quoted 58 lines · On Fri, 12 Apr 2013 12:35:14 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 35 lines · Igor Mammedov
Quoted 28 lines · On Fri, 12 Apr 2013 10:35:53 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 13 lines · Igor Mammedov
Quoted 10 lines · On Thu, 11 Apr 2013 15:59:40 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 8 lines · Igor Mammedov

... so that on reboot BIOS could read current available CPU count

v2:
* s/qemu_register_cpu_add_notifier()/qemu_register_cpu_added_notifier()/

---
 hw/timer/mc146818rtc.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
Quoted 7 lines
Quoted 3 lines

Initialization of the cmos fields (including 0x5F) is done on pc.c:pc_cmos_init(). What about making the field increment inside pc.c as well?

I looked at possibility but discarded it because to increment it there initial value should be -1 (field is zero based) which is not obvious, plug ugly casting to singed variable.
Result looked ugly.

I was thinking about simply adding exactly the same code with exactly the same logic, but inside pc.c instead of of mc146818rtc.c. Instead of registering the notifier inside rtc_initfn(), register exactly the same notifier with exactly the same code, but inside pc_cmos_init() (that's where 0x5f is initialized).

Quoted 4 lines

It would even be safer and easier review and ensure correctness: with this patch, the notifier is registered very early, even before pc_cmos_init() initializes 0x5f to smp_cpus. CPU hotplug events are unlikely to be emitted before pc_cmos_init() is called, but still: why

Quoted 8 lines

it isn't be called, hot-add is available only after machine initialized.

Quoted 1 line

make the initialization ordering so subtle if we don't have to?

Currently cmos init doesn't look like proper QOM object and has 3 stage
initialization: realize(), then pc_cmos_init() the last pc_cmos_init_late(). The last 2 calls are made after realize(), setting various properties. Which looks wrong from QOM perspective, so I'm against of stuffing more internal stuff in arbitrary places. We should do opposite instead.

True, but as we already have this weird 3-stage initialization process and we won't fix it really soon, I would really prefer to keep parts of the code that are closely related and depend on each other in the same part of the code.

Quoted 7 lines

If you look at mc146818rtc.c or hw/acpi/piix4.c, all notifiers are private to object and registered at realize() time. It looks like initialization order of mc146818rtc should be fixed, instead of adapting new code to it.

So since this patch doesn't break or violate anything in current code, I'd like to leave it as it is.

Quoted 6 lines

If you insist into making the mc146818rtc device take care of maintaining the 0x5f value by itself, why not doing:

    s->cmos_data[0x5f] = smp_cpus - 1;

inside rtc_initfn() instead of pc_cmos_init() as well?

Device is used not only by target-i386.
Right way would be to redesign rtc_init() and rtc_initfn() and it would be quite an intrusive patch.

That said it looks like current patch is incorrect if other targets are considered, where s->cmos_data[0x5f] doesn't mean smp_cpus - 1. That looks like a good reason to place notifier into pc.c and make it board specific. I'll redo it for the next respin.

Quoted 40 lines

This would be one additional step towards making pc_cmos_init() be replaced by QOM-based code (if that's what you want to do in the long term).

Quoted 15 lines
Quoted 15 lines
Quoted 6 lines
Quoted 6 lines

What happens if a CPU is hotplugged after the machine has started but before the guest OS has booted? Are we supposed to make sure the BIOS do the right thing if a CPU is hotplugged before the OS has booted, or this simply won't be supported?

Quoted 6 lines

BIOS uses this value to set in ACPI tables what CPUs are present.
1. if hot-plug happens before BIOS reads it then OS will see all CPUs
and SCI it receives will be nop.
2. if hot-plug happens after BIOS reads it, OS will handle SCI as usual and hotplug CPU instead of initializing it smp_boot() time.
BIOS itself has nothing to do with hot-plug, it's OSPM job.

Makes sense, thanks.

Quoted 8 lines
Quoted 4 lines

What happens if the CPU is hotplugged after the BIOS builds the ACPI tables, but long before the OS starts handling ACPI events? Is the OS guaranteed to run the CPU hotplug ACPI method (that's \_GPE._E02 in the DSDT, right?), even if that happens?

Theoretically interrupt should not disappear on it's own and OS should pick it up. But to say it for sure I need to test this case. If SCI will be lost for some reason, then OS won't notice new CPU until another CPU hot-plug event happens.

Makes sense, thanks.

(Anyway, if an interrupt gets lost, it's probably a BIOS or OS bug).

Quoted 1 line

[...]

Signature · 1 line
Eduardo
Signature · 2 lines
Regards,
  Igor
#32 lore raw

Re: [Qemu-devel] [PATCH 07/19] rtc: update rtc_cmos on CPU hot-plug

Quoted 70 lines · On Fri, 12 Apr 2013 18:24:23 +0200 Igor Mammedov <[email protected]> wrote:
Quoted 61 lines · On Fri, 12 Apr 2013 12:35:14 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 38 lines · Igor Mammedov
Quoted 29 lines · On Fri, 12 Apr 2013 10:35:53 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 14 lines · Igor Mammedov
Quoted 11 lines · On Thu, 11 Apr 2013 15:59:40 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 9 lines · Igor Mammedov

... so that on reboot BIOS could read current available CPU count

v2:
*
s/qemu_register_cpu_add_notifier()/qemu_register_cpu_added_notifier()/ --- hw/timer/mc146818rtc.c | 12 ++++++++++++

 1 file changed, 12 insertions(+)
Quoted 7 lines
Quoted 3 lines

Initialization of the cmos fields (including 0x5F) is done on pc.c:pc_cmos_init(). What about making the field increment inside pc.c as well?

I looked at possibility but discarded it because to increment it there initial value should be -1 (field is zero based) which is not obvious, plug ugly casting to singed variable.
Result looked ugly.

I was thinking about simply adding exactly the same code with exactly the same logic, but inside pc.c instead of of mc146818rtc.c. Instead of registering the notifier inside rtc_initfn(), register exactly the same notifier with exactly the same code, but inside pc_cmos_init() (that's where 0x5f is initialized).

Quoted 4 lines

It would even be safer and easier review and ensure correctness: with this patch, the notifier is registered very early, even before pc_cmos_init() initializes 0x5f to smp_cpus. CPU hotplug events are unlikely to be emitted before pc_cmos_init() is called, but still: why

it isn't be called, hot-add is available only after machine initialized.

Quoted 7 lines
Quoted 1 line

make the initialization ordering so subtle if we don't have to?

Currently cmos init doesn't look like proper QOM object and has 3 stage
initialization: realize(), then pc_cmos_init() the last pc_cmos_init_late(). The last 2 calls are made after realize(), setting various properties. Which looks wrong from QOM perspective, so I'm against of stuffing more internal stuff in arbitrary places. We should do opposite instead.

True, but as we already have this weird 3-stage initialization process and we won't fix it really soon, I would really prefer to keep parts of the code that are closely related and depend on each other in the same part of the code.

Quoted 8 lines

If you look at mc146818rtc.c or hw/acpi/piix4.c, all notifiers are private to object and registered at realize() time. It looks like initialization order of mc146818rtc should be fixed, instead of adapting new code to it.

So since this patch doesn't break or violate anything in current code, I'd like to leave it as it is.

Quoted 6 lines

If you insist into making the mc146818rtc device take care of maintaining the 0x5f value by itself, why not doing:

    s->cmos_data[0x5f] = smp_cpus - 1;

inside rtc_initfn() instead of pc_cmos_init() as well?

Quoted 9 lines

Device is used not only by target-i386.
Right way would be to redesign rtc_init() and rtc_initfn() and it would be quite an intrusive patch.

That said it looks like current patch is incorrect if other targets are considered, where s->cmos_data[0x5f] doesn't mean smp_cpus - 1. That looks like a good reason to place notifier into pc.c and make it board specific. I'll redo it for the next respin.

On the other hand, it probably would be better to make it a method and override it in pc.c

#33 lore raw

Re: [Qemu-devel] [PATCH 07/19] rtc: update rtc_cmos on CPU hot-plug

On Mon, Apr 15, 2013 at 11:38:45AM +0200, Igor Mammedov wrote:
[...]

Quoted 11 lines
Quoted 11 lines
Quoted 7 lines

If you insist into making the mc146818rtc device take care of maintaining the 0x5f value by itself, why not doing:

    s->cmos_data[0x5f] = smp_cpus - 1;

inside rtc_initfn() instead of pc_cmos_init() as well?

Device is used not only by target-i386.
Right way would be to redesign rtc_init() and rtc_initfn() and it would be quite an intrusive patch.

Quoted 7 lines
Quoted 5 lines

That said it looks like current patch is incorrect if other targets are considered, where s->cmos_data[0x5f] doesn't mean smp_cpus - 1. That looks like a good reason to place notifier into pc.c and make it board specific. I'll redo it for the next respin.

On the other hand, it probably would be better to make it a method and override it in pc.c

I don't see it as a feature of the mc146818rtc chip itself, but just something that the PC board does with the chip when stuff happens. So I wouldn't try to make it a method of mc146818rtc, but just something handled externally from the chip, entirely in the PC code.

Signature · 1 line
Eduardo
#34 lore raw

PATCH v3 8/19 cpu: introduce get_arch_id() method and override it for target-i386

In reply to Igor Mammedov

get_arch_id() adds possibility for generic code to get guest visible CPI id without accessing CPUArchState. If target doesn't override it, it will return cpu_index.

Override it on target-i386 to return APIC ID.

---
* it will be used later by new cpu_exists() generic function and acpi_piix.
* s/cpu_firmware_id/cpu_arch_id/

---
 include/qom/cpu.h |  2 ++
 qom/cpu.c         |  6 ++++++
 target-i386/cpu.c | 10 ++++++++++
 3 files changed, 18 insertions(+)
include/qom/cpu.h +2 -0
index 3664a1b..b376416 100644
@@ -45,6 +45,7 @@ typedef struct CPUState CPUState;
  * instantiatable CPU type.
  * @reset: Callback to reset the #CPUState to its initial state.
  * @do_interrupt: Callback for interrupt handling.
+ * @get_arch_id: Callback for getting architecture depended CPU ID
  * @vmsd: State description for migration.
  *
  * Represents a CPU family or model.
@@ -58,6 +59,7 @@ typedef struct CPUClass {
 
     void (*reset)(CPUState *cpu);
     void (*do_interrupt)(CPUState *cpu);
+    int64_t (*get_arch_id)(CPUState *cpu);
 
     const struct VMStateDescription *vmsd;
 } CPUClass;
qom/cpu.c +6 -0
index fe85960..90cbd77 100644
@@ -77,6 +77,11 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
     }
 }
 
+static int64_t cpu_common_get_arch_id(CPUState *cpu)
+{
+    return cpu->cpu_index;
+}
+
 static void cpu_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
@@ -84,6 +89,7 @@ static void cpu_class_init(ObjectClass *klass, void *data)
 
     k->class_by_name = cpu_common_class_by_name;
     k->reset = cpu_common_reset;
+    k->get_arch_id = cpu_common_get_arch_id;
     dc->realize = cpu_common_realizefn;
     dc->no_user = 1;
 }
target-i386/cpu.c +10 -0
index c28436c..9cca031 100644
@@ -2272,6 +2272,14 @@ static void x86_cpu_initfn(Object *obj)
     }
 }
 
+static int64_t x86_cpu_get_arch_id(CPUState *cpu)
+{
+    X86CPU *x86cpu = X86_CPU(cpu);
+    CPUX86State *env = &x86cpu->env;
+
+    return env->cpuid_apic_id;
+}
+
 static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 {
     X86CPUClass *xcc = X86_CPU_CLASS(oc);
@@ -2286,6 +2294,8 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 
     cc->do_interrupt = x86_cpu_do_interrupt;
     cpu_class_set_vmsd(cc, &vmstate_x86_cpu);
+
+    cc->get_arch_id = x86_cpu_get_arch_id;
 }
 
 static const TypeInfo x86_cpu_type_info = {
Signature · 1 line
1.8.2
Reviewed-by on v3 8/19
#35 lore raw

Re: [Qemu-devel] [PATCH 08/19] cpu: introduce get_arch_id() method and override it for target-i386

Quoted 5 lines · Igor Mammedov

get_arch_id() adds possibility for generic code to get guest visible CPI id without accessing CPUArchState. If target doesn't override it, it will return cpu_index.

Override it on target-i386 to return APIC ID.

The code implementing the method looks OK, so:

But I still need to understand better why exactly i386 needs to use it and make the externally-visible hotplug interface based on APIC IDs if all other architectures are going to be happy using CPU indexes to do the same thing.

Quoted 87 lines

---
* it will be used later by new cpu_exists() generic function and acpi_piix.
* s/cpu_firmware_id/cpu_arch_id/

---
 include/qom/cpu.h |  2 ++
 qom/cpu.c         |  6 ++++++
 target-i386/cpu.c | 10 ++++++++++
 3 files changed, 18 insertions(+)
include/qom/cpu.h +2 -0
index 3664a1b..b376416 100644
@@ -45,6 +45,7 @@ typedef struct CPUState CPUState;
  * instantiatable CPU type.
  * @reset: Callback to reset the #CPUState to its initial state.
  * @do_interrupt: Callback for interrupt handling.
+ * @get_arch_id: Callback for getting architecture depended CPU ID
  * @vmsd: State description for migration.
  *
  * Represents a CPU family or model.
@@ -58,6 +59,7 @@ typedef struct CPUClass {
 
     void (*reset)(CPUState *cpu);
     void (*do_interrupt)(CPUState *cpu);
+    int64_t (*get_arch_id)(CPUState *cpu);
 
     const struct VMStateDescription *vmsd;
 } CPUClass;
qom/cpu.c +6 -0
index fe85960..90cbd77 100644
@@ -77,6 +77,11 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
     }
 }
 
+static int64_t cpu_common_get_arch_id(CPUState *cpu)
+{
+    return cpu->cpu_index;
+}
+
 static void cpu_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
@@ -84,6 +89,7 @@ static void cpu_class_init(ObjectClass *klass, void *data)
 
     k->class_by_name = cpu_common_class_by_name;
     k->reset = cpu_common_reset;
+    k->get_arch_id = cpu_common_get_arch_id;
     dc->realize = cpu_common_realizefn;
     dc->no_user = 1;
 }
target-i386/cpu.c +10 -0
index c28436c..9cca031 100644
@@ -2272,6 +2272,14 @@ static void x86_cpu_initfn(Object *obj)
     }
 }
 
+static int64_t x86_cpu_get_arch_id(CPUState *cpu)
+{
+    X86CPU *x86cpu = X86_CPU(cpu);
+    CPUX86State *env = &x86cpu->env;
+
+    return env->cpuid_apic_id;
+}
+
 static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 {
     X86CPUClass *xcc = X86_CPU_CLASS(oc);
@@ -2286,6 +2294,8 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 
     cc->do_interrupt = x86_cpu_do_interrupt;
     cpu_class_set_vmsd(cc, &vmstate_x86_cpu);
+
+    cc->get_arch_id = x86_cpu_get_arch_id;
 }
 
 static const TypeInfo x86_cpu_type_info = {
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
#36 lore raw

Re: [Qemu-devel] [PATCH 08/19] cpu: introduce get_arch_id() method and override it for target-i386

Quoted 7 lines · On Thu, 11 Apr 2013 16:04:26 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 5 lines · Igor Mammedov

get_arch_id() adds possibility for generic code to get guest visible CPI id without accessing CPUArchState. If target doesn't override it, it will return cpu_index.

Override it on target-i386 to return APIC ID.

Quoted 8 lines

The code implementing the method looks OK, so:

But I still need to understand better why exactly i386 needs to use it and make the externally-visible hotplug interface based on APIC IDs if all other architectures are going to be happy using CPU indexes to do the same thing.

get_arch_id() was prompted by 10/19 which is used not only by target-i386. so it was introduced to make acpi_piix4 CPU hotplug more generic. And proved to useful in a number of other places.

Quoted 91 lines
Quoted 88 lines

---
* it will be used later by new cpu_exists() generic function and acpi_piix.
* s/cpu_firmware_id/cpu_arch_id/

---
 include/qom/cpu.h |  2 ++
 qom/cpu.c         |  6 ++++++
 target-i386/cpu.c | 10 ++++++++++
 3 files changed, 18 insertions(+)
include/qom/cpu.h +2 -0
index 3664a1b..b376416 100644
@@ -45,6 +45,7 @@ typedef struct CPUState CPUState;
  * instantiatable CPU type.
  * @reset: Callback to reset the #CPUState to its initial state.
  * @do_interrupt: Callback for interrupt handling.
+ * @get_arch_id: Callback for getting architecture depended CPU ID
  * @vmsd: State description for migration.
  *
  * Represents a CPU family or model.
@@ -58,6 +59,7 @@ typedef struct CPUClass {
 
     void (*reset)(CPUState *cpu);
     void (*do_interrupt)(CPUState *cpu);
+    int64_t (*get_arch_id)(CPUState *cpu);
 
     const struct VMStateDescription *vmsd;
 } CPUClass;
qom/cpu.c +6 -0
index fe85960..90cbd77 100644
@@ -77,6 +77,11 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
     }
 }
 
+static int64_t cpu_common_get_arch_id(CPUState *cpu)
+{
+    return cpu->cpu_index;
+}
+
 static void cpu_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
@@ -84,6 +89,7 @@ static void cpu_class_init(ObjectClass *klass, void *data)
 
     k->class_by_name = cpu_common_class_by_name;
     k->reset = cpu_common_reset;
+    k->get_arch_id = cpu_common_get_arch_id;
     dc->realize = cpu_common_realizefn;
     dc->no_user = 1;
 }
target-i386/cpu.c +10 -0
index c28436c..9cca031 100644
@@ -2272,6 +2272,14 @@ static void x86_cpu_initfn(Object *obj)
     }
 }
 
+static int64_t x86_cpu_get_arch_id(CPUState *cpu)
+{
+    X86CPU *x86cpu = X86_CPU(cpu);
+    CPUX86State *env = &x86cpu->env;
+
+    return env->cpuid_apic_id;
+}
+
 static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 {
     X86CPUClass *xcc = X86_CPU_CLASS(oc);
@@ -2286,6 +2294,8 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 
     cc->do_interrupt = x86_cpu_do_interrupt;
     cpu_class_set_vmsd(cc, &vmstate_x86_cpu);
+
+    cc->get_arch_id = x86_cpu_get_arch_id;
 }
 
 static const TypeInfo x86_cpu_type_info = {
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
Signature · 2 lines
Regards,
  Igor
#37 lore raw

Re: [Qemu-devel] [PATCH 08/19] cpu: introduce get_arch_id() method and override it for target-i386

Quoted 14 lines · Igor Mammedov
Quoted 11 lines · On Thu, 11 Apr 2013 16:04:26 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 5 lines · Igor Mammedov

get_arch_id() adds possibility for generic code to get guest visible CPI id without accessing CPUArchState. If target doesn't override it, it will return cpu_index.

Override it on target-i386 to return APIC ID.

The code implementing the method looks OK, so:

Quoted 7 lines
Quoted 4 lines

But I still need to understand better why exactly i386 needs to use it and make the externally-visible hotplug interface based on APIC IDs if all other architectures are going to be happy using CPU indexes to do the same thing.

get_arch_id() was prompted by 10/19 which is used not only by target-i386. so it was introduced to make acpi_piix4 CPU hotplug more generic. And proved to useful in a number of other places.

OK, so get_arch_id() is useful for other code too, not just for the external cpu-add interface. It makes lots of sense to use it on acpi_piix4, yes.

(My question about making the cpu-add interface based on arch IDs instead of arch-independent CPU indexes is being discussed under patch 18/19).

Quoted 97 lines
Quoted 92 lines
Quoted 89 lines

---
* it will be used later by new cpu_exists() generic function and acpi_piix.
* s/cpu_firmware_id/cpu_arch_id/

---
 include/qom/cpu.h |  2 ++
 qom/cpu.c         |  6 ++++++
 target-i386/cpu.c | 10 ++++++++++
 3 files changed, 18 insertions(+)
include/qom/cpu.h +2 -0
index 3664a1b..b376416 100644
@@ -45,6 +45,7 @@ typedef struct CPUState CPUState;
  * instantiatable CPU type.
  * @reset: Callback to reset the #CPUState to its initial state.
  * @do_interrupt: Callback for interrupt handling.
+ * @get_arch_id: Callback for getting architecture depended CPU ID
  * @vmsd: State description for migration.
  *
  * Represents a CPU family or model.
@@ -58,6 +59,7 @@ typedef struct CPUClass {
 
     void (*reset)(CPUState *cpu);
     void (*do_interrupt)(CPUState *cpu);
+    int64_t (*get_arch_id)(CPUState *cpu);
 
     const struct VMStateDescription *vmsd;
 } CPUClass;
qom/cpu.c +6 -0
index fe85960..90cbd77 100644
@@ -77,6 +77,11 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
     }
 }
 
+static int64_t cpu_common_get_arch_id(CPUState *cpu)
+{
+    return cpu->cpu_index;
+}
+
 static void cpu_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
@@ -84,6 +89,7 @@ static void cpu_class_init(ObjectClass *klass, void *data)
 
     k->class_by_name = cpu_common_class_by_name;
     k->reset = cpu_common_reset;
+    k->get_arch_id = cpu_common_get_arch_id;
     dc->realize = cpu_common_realizefn;
     dc->no_user = 1;
 }
target-i386/cpu.c +10 -0
index c28436c..9cca031 100644
@@ -2272,6 +2272,14 @@ static void x86_cpu_initfn(Object *obj)
     }
 }
 
+static int64_t x86_cpu_get_arch_id(CPUState *cpu)
+{
+    X86CPU *x86cpu = X86_CPU(cpu);
+    CPUX86State *env = &x86cpu->env;
+
+    return env->cpuid_apic_id;
+}
+
 static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 {
     X86CPUClass *xcc = X86_CPU_CLASS(oc);
@@ -2286,6 +2294,8 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 
     cc->do_interrupt = x86_cpu_do_interrupt;
     cpu_class_set_vmsd(cc, &vmstate_x86_cpu);
+
+    cc->get_arch_id = x86_cpu_get_arch_id;
 }
 
 static const TypeInfo x86_cpu_type_info = {
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
Signature · 2 lines
Regards,
  Igor
Signature · 1 line
Eduardo
#38 lore raw

Re: [Qemu-devel] [PATCH 08/19] cpu: introduce get_arch_id() method and override it for target-i386

In reply to Igor Mammedov

Am 11.04.2013 16:51, schrieb Igor Mammedov:

Quoted 11 lines

get_arch_id() adds possibility for generic code to get guest visible CPI id without accessing CPUArchState. If target doesn't override it, it will return cpu_index.

Override it on target-i386 to return APIC ID.

---
* it will be used later by new cpu_exists() generic function and acpi_piix.
* s/cpu_firmware_id/cpu_arch_id/

Quoted 5 lines
---
 include/qom/cpu.h |  2 ++
 qom/cpu.c         |  6 ++++++
 target-i386/cpu.c | 10 ++++++++++
 3 files changed, 18 insertions(+)

I'm still unclear about this ID topic, but please don't add QOM methods with arch_ in the name. The arch is expressed through subclasses.

Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#39 lore raw

Re: [Qemu-devel] [PATCH 08/19] cpu: introduce get_arch_id() method and override it for target-i386

Quoted 18 lines · On Mon, 15 Apr 2013 17:24:36 +0200 Andreas Färber <[email protected]> wrote:

Am 11.04.2013 16:51, schrieb Igor Mammedov:

Quoted 11 lines

get_arch_id() adds possibility for generic code to get guest visible CPI id without accessing CPUArchState. If target doesn't override it, it will return cpu_index.

Override it on target-i386 to return APIC ID.

---
* it will be used later by new cpu_exists() generic function and acpi_piix.
* s/cpu_firmware_id/cpu_arch_id/

Quoted 5 lines
---
 include/qom/cpu.h |  2 ++
 qom/cpu.c         |  6 ++++++
 target-i386/cpu.c | 10 ++++++++++
 3 files changed, 18 insertions(+)
Quoted 2 lines

I'm still unclear about this ID topic, but please don't add QOM methods with arch_ in the name. The arch is expressed through subclasses.

arch is there to explicitly show that returned ID is defined by target and not generic code. This method then allow to implement cpu_exists() and helps to init present cpus bitmap in acpi_piix4.

Would be cpu_id() acceptable?

Quoted 3 lines

Andreas

#40 lore raw

Re: [Qemu-devel] [PATCH 08/19] cpu: introduce get_arch_id() method and override it for target-i386

Am 15.04.2013 17:34, schrieb Igor Mammedov:

Quoted 21 lines
Quoted 18 lines · On Mon, 15 Apr 2013 17:24:36 +0200 Andreas Färber <[email protected]> wrote:

Am 11.04.2013 16:51, schrieb Igor Mammedov:

Quoted 16 lines

get_arch_id() adds possibility for generic code to get guest visible CPI id without accessing CPUArchState. If target doesn't override it, it will return cpu_index.

Override it on target-i386 to return APIC ID.

---
* it will be used later by new cpu_exists() generic function and acpi_piix.
* s/cpu_firmware_id/cpu_arch_id/

---
 include/qom/cpu.h |  2 ++
 qom/cpu.c         |  6 ++++++
 target-i386/cpu.c | 10 ++++++++++
 3 files changed, 18 insertions(+)
Quoted 8 lines
Quoted 2 lines

I'm still unclear about this ID topic, but please don't add QOM methods with arch_ in the name. The arch is expressed through subclasses.

arch is there to explicitly show that returned ID is defined by target and not generic code. This method then allow to implement cpu_exists() and helps to init present cpus bitmap in acpi_piix4.

Would be cpu_id() acceptable?

Certainly.

Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#41 lore raw

Re: [Qemu-devel] [PATCH 08/19] cpu: introduce get_arch_id() method and override it for target-i386

In reply to Andreas Färber
Quoted 18 lines · Andreas Färber

Am 11.04.2013 16:51, schrieb Igor Mammedov:

Quoted 11 lines

get_arch_id() adds possibility for generic code to get guest visible CPI id without accessing CPUArchState. If target doesn't override it, it will return cpu_index.

Override it on target-i386 to return APIC ID.

---
* it will be used later by new cpu_exists() generic function and acpi_piix.
* s/cpu_firmware_id/cpu_arch_id/

Quoted 5 lines
---
 include/qom/cpu.h |  2 ++
 qom/cpu.c         |  6 ++++++
 target-i386/cpu.c | 10 ++++++++++
 3 files changed, 18 insertions(+)
Quoted 2 lines

I'm still unclear about this ID topic, but please don't add QOM methods with arch_ in the name. The arch is expressed through subclasses.

The naming seemed appropriate to me as it is an abstract method that is expected to have arch-specific implementations and arch-specific meaning.

On the other hand, "firmware_id" looked clearer to me (as it was very specific about the meaning/usefulness of the return value), I don't know what was the original reason to change from firmware_id to arch_id.

Other than "arch_id" and "firmware_id , I don't know how we could name that method. It's not "just an ID", but a special kind of ID, so I believe just "cpu_id" wouldn't work.

Signature · 1 line
Eduardo
#42 lore raw

Re: [Qemu-devel] [PATCH 08/19] cpu: introduce get_arch_id() method and override it for target-i386

Quoted 22 lines · On Mon, 15 Apr 2013 12:47:52 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 18 lines · Andreas Färber

Am 11.04.2013 16:51, schrieb Igor Mammedov:

Quoted 11 lines

get_arch_id() adds possibility for generic code to get guest visible CPI id without accessing CPUArchState. If target doesn't override it, it will return cpu_index.

Override it on target-i386 to return APIC ID.

---
* it will be used later by new cpu_exists() generic function and acpi_piix.
* s/cpu_firmware_id/cpu_arch_id/

Quoted 5 lines
---
 include/qom/cpu.h |  2 ++
 qom/cpu.c         |  6 ++++++
 target-i386/cpu.c | 10 ++++++++++
 3 files changed, 18 insertions(+)
Quoted 2 lines

I'm still unclear about this ID topic, but please don't add QOM methods with arch_ in the name. The arch is expressed through subclasses.

Quoted 7 lines

The naming seemed appropriate to me as it is an abstract method that is expected to have arch-specific implementations and arch-specific meaning.

On the other hand, "firmware_id" looked clearer to me (as it was very specific about the meaning/usefulness of the return value), I don't know what was the original reason to change from firmware_id to arch_id.

Objection here
http://lists.nongnu.org/archive/html/qemu-devel/2013-04/msg01332.html

In case of x86, it might be firmware defined id, but in general case it might be not the case, since it could be hardwired into CPU. So get_arch_id() seems more appropriate.

Quoted 8 lines

Other than "arch_id" and "firmware_id , I don't know how we could name that method. It's not "just an ID", but a special kind of ID, so I believe just "cpu_id" wouldn't work.

Signature · 1 line
Eduardo
Signature · 2 lines
Regards,
  Igor
#43 lore raw

PATCH v3 9/19 cpu: add helper cpu_exists(), to check if CPU with specified id exists

In reply to Igor Mammedov

... it should be used only on slow path since it does recursive search
on /machine QOM tree for objects of TYPE_CPU

---
v2:
* s/get_firmware_id()/get_arch_id()/ rebase fixup
* remove check for get_arch_id being NULL, since it's always defined

---
 include/qom/cpu.h | 10 ++++++++++
 qom/cpu.c         | 21 +++++++++++++++++++++
 2 files changed, 31 insertions(+)
include/qom/cpu.h +10 -0
index b376416..cd7c4b5 100644
@@ -225,6 +225,16 @@ void run_on_cpu(CPUState *cpu, void (*func)(void *data), void *data);
  */
 CPUState *qemu_get_cpu(int index);
 
+/**
+ * cpu_exists:
+ * @id - guest exposed CPU ID to lookup
+ *
+ * Search for CPU with specified ID.
+ *
+ * Returns: true - CPU is found, false - CPU isn't found
+ */
+bool cpu_exists(int64_t id);
+
 #ifndef CONFIG_USER_ONLY
 
 typedef void (*CPUInterruptHandler)(CPUState *, int);
qom/cpu.c +21 -0
index 90cbd77..c850d27 100644
@@ -25,6 +25,27 @@
 #include "qemu/notify.h"
 #include "sysemu/sysemu.h"
 
+static int cpu_exist_cb(Object *obj, void *opaque)
+{
+    int64_t id = *(int64_t *)opaque;
+    Object *cpu_obj = object_dynamic_cast(obj, TYPE_CPU);
+
+    if (cpu_obj) {
+        CPUState *cpu = CPU(cpu_obj);
+        CPUClass *klass = CPU_GET_CLASS(cpu);
+
+        if (klass->get_arch_id(cpu) == id) {
+            return 1;
+        }
+    }
+    return object_child_foreach(obj, cpu_exist_cb, opaque);
+}
+
+bool cpu_exists(int64_t id)
+{
+   return cpu_exist_cb(qdev_get_machine(), &id) ? true : false;
+}
+
 /* CPU hot-plug notifiers */
 static NotifierList cpu_added_notifiers =
     NOTIFIER_LIST_INITIALIZER(cpu_add_notifiers);
Signature · 1 line
1.8.2
#44 lore raw

Re: [Qemu-devel] [PATCH 09/19] cpu: add helper cpu_exists(), to check if CPU with specified id exists

Quoted 5 lines · Igor Mammedov

... it should be used only on slow path since it does recursive search
on /machine QOM tree for objects of TYPE_CPU

Why not implement this only after patch 21/22, and use "/machine/icc-bridge/cpu[ID]" for the lookup?

Quoted 65 lines

---
v2:
* s/get_firmware_id()/get_arch_id()/ rebase fixup
* remove check for get_arch_id being NULL, since it's always defined

---
 include/qom/cpu.h | 10 ++++++++++
 qom/cpu.c         | 21 +++++++++++++++++++++
 2 files changed, 31 insertions(+)
include/qom/cpu.h +10 -0
index b376416..cd7c4b5 100644
@@ -225,6 +225,16 @@ void run_on_cpu(CPUState *cpu, void (*func)(void *data), void *data);
  */
 CPUState *qemu_get_cpu(int index);
 
+/**
+ * cpu_exists:
+ * @id - guest exposed CPU ID to lookup
+ *
+ * Search for CPU with specified ID.
+ *
+ * Returns: true - CPU is found, false - CPU isn't found
+ */
+bool cpu_exists(int64_t id);
+
 #ifndef CONFIG_USER_ONLY
 
 typedef void (*CPUInterruptHandler)(CPUState *, int);
qom/cpu.c +21 -0
index 90cbd77..c850d27 100644
@@ -25,6 +25,27 @@
 #include "qemu/notify.h"
 #include "sysemu/sysemu.h"
 
+static int cpu_exist_cb(Object *obj, void *opaque)
+{
+    int64_t id = *(int64_t *)opaque;
+    Object *cpu_obj = object_dynamic_cast(obj, TYPE_CPU);
+
+    if (cpu_obj) {
+        CPUState *cpu = CPU(cpu_obj);
+        CPUClass *klass = CPU_GET_CLASS(cpu);
+
+        if (klass->get_arch_id(cpu) == id) {
+            return 1;
+        }
+    }
+    return object_child_foreach(obj, cpu_exist_cb, opaque);
+}
+
+bool cpu_exists(int64_t id)
+{
+   return cpu_exist_cb(qdev_get_machine(), &id) ? true : false;
+}
+
 /* CPU hot-plug notifiers */
 static NotifierList cpu_added_notifiers =
     NOTIFIER_LIST_INITIALIZER(cpu_add_notifiers);
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
#45 lore raw

Re: [Qemu-devel] [PATCH 09/19] cpu: add helper cpu_exists(), to check if CPU with specified id exists

Quoted 9 lines · On Thu, 11 Apr 2013 16:06:58 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 5 lines · Igor Mammedov

... it should be used only on slow path since it does recursive search
on /machine QOM tree for objects of TYPE_CPU

Why not implement this only after patch 21/22, and use "/machine/icc-bridge/cpu[ID]" for the lookup?

/machine/icc-bridge/cpu[ID] is target specific and optional so far,
+ it would require parsing links names.

While looking for all CPUs, regardless of where they are located (each target could have its own notion about it) are generic.

BTW:
patches 10/19 and 11/19 use it as well.

Quoted 69 lines
Quoted 66 lines

---
v2:
* s/get_firmware_id()/get_arch_id()/ rebase fixup
* remove check for get_arch_id being NULL, since it's always defined

---
 include/qom/cpu.h | 10 ++++++++++
 qom/cpu.c         | 21 +++++++++++++++++++++
 2 files changed, 31 insertions(+)
include/qom/cpu.h +10 -0
index b376416..cd7c4b5 100644
@@ -225,6 +225,16 @@ void run_on_cpu(CPUState *cpu, void (*func)(void *data), void *data);
  */
 CPUState *qemu_get_cpu(int index);
 
+/**
+ * cpu_exists:
+ * @id - guest exposed CPU ID to lookup
+ *
+ * Search for CPU with specified ID.
+ *
+ * Returns: true - CPU is found, false - CPU isn't found
+ */
+bool cpu_exists(int64_t id);
+
 #ifndef CONFIG_USER_ONLY
 
 typedef void (*CPUInterruptHandler)(CPUState *, int);
qom/cpu.c +21 -0
index 90cbd77..c850d27 100644
@@ -25,6 +25,27 @@
 #include "qemu/notify.h"
 #include "sysemu/sysemu.h"
 
+static int cpu_exist_cb(Object *obj, void *opaque)
+{
+    int64_t id = *(int64_t *)opaque;
+    Object *cpu_obj = object_dynamic_cast(obj, TYPE_CPU);
+
+    if (cpu_obj) {
+        CPUState *cpu = CPU(cpu_obj);
+        CPUClass *klass = CPU_GET_CLASS(cpu);
+
+        if (klass->get_arch_id(cpu) == id) {
+            return 1;
+        }
+    }
+    return object_child_foreach(obj, cpu_exist_cb, opaque);
+}
+
+bool cpu_exists(int64_t id)
+{
+   return cpu_exist_cb(qdev_get_machine(), &id) ? true : false;
+}
+
 /* CPU hot-plug notifiers */
 static NotifierList cpu_added_notifiers =
     NOTIFIER_LIST_INITIALIZER(cpu_add_notifiers);
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
Signature · 2 lines
Regards,
  Igor
#46 lore raw

PATCH v3 10/19 acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

In reply to Igor Mammedov

* introduce processor status bitmask visible to guest at 0xaf00 addr, where ACPI asl code expects it
* set bit corresponding to APIC ID in processor status bitmask on receiving CPU hot-plug notification
* trigger CPU hot-plug SCI, to notify guest about CPU hot-plug event

---
v3:
* s/get_firmware_id()/get_arch_id()/ due rebase
* s/cpu_add_notifier/cpu_added_notifier/
v2:
* use CPUClass.get_firmware_id() to make code target independent
* bump up vmstate_acpi version

---
 hw/acpi/piix4.c | 114 ++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 111 insertions(+), 3 deletions(-)
hw/acpi/piix4.c +111 -3
index 88386d7..a753adf 100644
@@ -48,19 +48,28 @@
 #define PCI_EJ_BASE 0xae08
 #define PCI_RMV_BASE 0xae0c
 
+#define PROC_BASE 0xaf00
+#define PROC_LEN 32
+
 #define PIIX4_PCI_HOTPLUG_STATUS 2
+#define PIIX4_CPU_HOTPLUG_STATUS 4
 
 struct pci_status {
     uint32_t up; /* deprecated, maintained for migration compatibility */
     uint32_t down;
 };
 
+struct cpu_status {
+    uint8_t sts[PROC_LEN];
+};
+
 typedef struct PIIX4PMState {
     PCIDevice dev;
 
     MemoryRegion io;
     MemoryRegion io_gpe;
     MemoryRegion io_pci;
+    MemoryRegion io_cpu;
     ACPIREGS ar;
 
     APMState apm;
@@ -82,6 +91,9 @@ typedef struct PIIX4PMState {
     uint8_t disable_s3;
     uint8_t disable_s4;
     uint8_t s4_val;
+
+    struct cpu_status gpe_cpu;
+    Notifier cpu_added_notifier;
 } PIIX4PMState;
 
 static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
@@ -100,8 +112,8 @@ static void pm_update_sci(PIIX4PMState *s)
                    ACPI_BITMASK_POWER_BUTTON_ENABLE |
                    ACPI_BITMASK_GLOBAL_LOCK_ENABLE |
                    ACPI_BITMASK_TIMER_ENABLE)) != 0) ||
-        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0])
-          & PIIX4_PCI_HOTPLUG_STATUS) != 0);
+        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0]) &
+          (PIIX4_PCI_HOTPLUG_STATUS | PIIX4_CPU_HOTPLUG_STATUS)) != 0);
 
     qemu_set_irq(s->irq, sci_level);
     /* schedule a timer interruption if needed */
@@ -257,6 +269,17 @@ static int acpi_load_old(QEMUFile *f, void *opaque, int version_id)
     return ret;
 }
 
+#define VMSTATE_CPU_STATUS_ARRAY(_field, _state)                             \
+ {                                                                           \
+     .name       = (stringify(_field)),                                      \
+     .version_id = 0,                                                        \
+     .num        = PROC_LEN,                                                 \
+     .info       = &vmstate_info_uint8,                                      \
+     .size       = sizeof(uint8_t),                                          \
+     .flags      = VMS_ARRAY,                                                \
+     .offset     = vmstate_offset_array(_state, _field, uint8_t, PROC_LEN),  \
+ }
+
 /* qemu-kvm 1.2 uses version 3 but advertised as 2
  * To support incoming qemu-kvm 1.2 migration, change version_id
  * and minimum_version_id to 2 below (which breaks migration from
@@ -265,7 +288,7 @@ static int acpi_load_old(QEMUFile *f, void *opaque, int version_id)
  */
 static const VMStateDescription vmstate_acpi = {
     .name = "piix4_pm",
-    .version_id = 3,
+    .version_id = 4,
     .minimum_version_id = 3,
     .minimum_version_id_old = 1,
     .load_state_old = acpi_load_old,
@@ -281,6 +304,7 @@ static const VMStateDescription vmstate_acpi = {
         VMSTATE_STRUCT(ar.gpe, PIIX4PMState, 2, vmstate_gpe, ACPIGPE),
         VMSTATE_STRUCT(pci0_status, PIIX4PMState, 2, vmstate_pci_status,
                        struct pci_status),
+        VMSTATE_CPU_STATUS_ARRAY(gpe_cpu.sts, PIIX4PMState),
         VMSTATE_END_OF_LIST()
     }
 };
@@ -585,6 +609,83 @@ static const MemoryRegionOps piix4_pci_ops = {
     },
 };
 
+static uint64_t cpu_status_read(void *opaque, hwaddr addr, unsigned width)
+{
+    PIIX4PMState *s = opaque;
+    struct cpu_status *cpus = &s->gpe_cpu;
+    uint64_t val = cpus->sts[addr];
+    return val;
+}
+
+static void cpu_status_write(void *opaque, hwaddr addr, uint64_t data,
+                             unsigned int size)
+{
+    /* TODO: implement VCPU removal on guest signal that CPU can be removed */
+}
+
+static const MemoryRegionOps cpu_hotplug_ops = {
+    .read = cpu_status_read,
+    .write = cpu_status_write,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+    .valid = {
+        .min_access_size = 1,
+        .max_access_size = 1,
+    },
+};
+
+typedef enum {
+    PLUG,
+    UNPLUG,
+} HotplugEventType;
+
+static void piix4_cpu_hotplug_req(PIIX4PMState *s, CPUState *cpu,
+                                  HotplugEventType action)
+{
+    struct cpu_status *g = &s->gpe_cpu;
+    ACPIGPE *gpe = &s->ar.gpe;
+    CPUClass *k = CPU_GET_CLASS(cpu);
+    int64_t cpu_id;
+
+    assert(s != NULL);
+
+    *gpe->sts = *gpe->sts | PIIX4_CPU_HOTPLUG_STATUS;
+    cpu_id = k->get_arch_id(CPU(cpu));
+    if (action == PLUG) {
+        g->sts[cpu_id / 8] |= (1 << (cpu_id % 8));
+    } else {
+        g->sts[cpu_id / 8] &= ~(1 << (cpu_id % 8));
+    }
+    pm_update_sci(s);
+}
+
+static void piix4_cpu_added_req(Notifier *n, void *opaque)
+{
+    PIIX4PMState *s = container_of(n, PIIX4PMState, cpu_added_notifier);
+    piix4_cpu_hotplug_req(s, CPU(opaque), PLUG);
+}
+
+static int piix4_init_cpu_status(Object *obj, void *opaque)
+{
+    struct cpu_status *g = (struct cpu_status *)opaque;
+    Object *cpu_obj = object_dynamic_cast(obj, TYPE_CPU);
+
+    if (cpu_obj) {
+        struct Error *error = NULL;
+        CPUClass *k = CPU_GET_CLASS(cpu_obj);
+        int64_t id = k->get_arch_id(CPU(cpu_obj));
+
+        if (error) {
+            fprintf(stderr, "failed to initilize CPU status for ACPI: %s\n",
+                    error_get_pretty(error));
+            error_free(error);
+            abort();
+        }
+        g_assert((id / 8) < PROC_LEN);
+        g->sts[id / 8] |= (1 << (id % 8));
+    }
+    return object_child_foreach(obj, piix4_init_cpu_status, opaque);
+}
+
 static int piix4_device_hotplug(DeviceState *qdev, PCIDevice *dev,
                                 PCIHotplugState state);
 
@@ -600,6 +701,13 @@ static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
     memory_region_add_subregion(parent, PCI_HOTPLUG_ADDR,
                                 &s->io_pci);
     pci_bus_hotplug(bus, piix4_device_hotplug, &s->dev.qdev);
+
+    piix4_init_cpu_status(qdev_get_machine(), &s->gpe_cpu);
+    memory_region_init_io(&s->io_cpu, &cpu_hotplug_ops, s, "apci-cpu-hotplug",
+                          PROC_LEN);
+    memory_region_add_subregion(parent, PROC_BASE, &s->io_cpu);
+    s->cpu_added_notifier.notify = piix4_cpu_added_req;
+    qemu_register_cpu_added_notifier(&s->cpu_added_notifier);
 }
 
 static void enable_device(PIIX4PMState *s, int slot)
Signature · 1 line
1.8.2
#47 lore raw

PATCH v3 11/19 target-i386: introduce apic-id property

In reply to Igor Mammedov

... and use it from board level to set APIC ID for CPUs it creates.

---
Note:
* pc_new_cpu() function will be reused later in CPU hot-plug hook.

v3:
* user error_propagate() in property setter
v2:
* use generic cpu_exists() instead of custom one
* make apic-id dynamic property, so it won't be possible to use it as global property, since it's instance specific

---
 hw/i386/pc.c      | 25 ++++++++++++++++++++++++-
 target-i386/cpu.c | 42 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 66 insertions(+), 1 deletion(-)
hw/i386/pc.c +24 -1
index 8d75b34..3adf294 100644
@@ -869,9 +869,29 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level)
     }
 }
 
+static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
+{
+    X86CPU *cpu;
+
+    cpu = cpu_x86_create(cpu_model, errp);
+    if (!cpu) {
+        return;
+    }
+
+    object_property_set_int(OBJECT(cpu), apic_id, "apic-id", errp);
+    object_property_set_bool(OBJECT(cpu), true, "realized", errp);
+
+    if (error_is_set(errp)) {
+        if (cpu != NULL) {
+            object_unref(OBJECT(cpu));
+        }
+    }
+}
+
 void pc_cpus_init(const char *cpu_model)
 {
     int i;
+    Error *error = NULL;
 
     /* init CPUs */
     if (cpu_model == NULL) {
@@ -883,7 +903,10 @@ void pc_cpus_init(const char *cpu_model)
     }
 
     for (i = 0; i < smp_cpus; i++) {
-        if (!cpu_x86_init(cpu_model)) {
+        pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
+        if (error) {
+            fprintf(stderr, "%s\n", error_get_pretty(error));
+            error_free(error);
             exit(1);
         }
     }
target-i386/cpu.c +42 -0

54 lines not on this page · raw message

Signature · 1 line
1.8.2
#48 lore raw

Re: [Qemu-devel] [PATCH 11/19] target-i386: introduce apic-id property

Quoted 98 lines · Igor Mammedov

... and use it from board level to set APIC ID for CPUs it creates.

---
Note:
* pc_new_cpu() function will be reused later in CPU hot-plug hook.

v3:
* user error_propagate() in property setter
v2:
* use generic cpu_exists() instead of custom one
* make apic-id dynamic property, so it won't be possible to use it as global property, since it's instance specific

---
 hw/i386/pc.c      | 25 ++++++++++++++++++++++++-
 target-i386/cpu.c | 42 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 66 insertions(+), 1 deletion(-)
hw/i386/pc.c +24 -1
index 8d75b34..3adf294 100644
@@ -869,9 +869,29 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level)
     }
 }
 
+static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
+{
+    X86CPU *cpu;
+
+    cpu = cpu_x86_create(cpu_model, errp);
+    if (!cpu) {
+        return;
+    }
+
+    object_property_set_int(OBJECT(cpu), apic_id, "apic-id", errp);
+    object_property_set_bool(OBJECT(cpu), true, "realized", errp);
+
+    if (error_is_set(errp)) {
+        if (cpu != NULL) {
+            object_unref(OBJECT(cpu));
+        }
+    }
+}
+
 void pc_cpus_init(const char *cpu_model)
 {
     int i;
+    Error *error = NULL;
 
     /* init CPUs */
     if (cpu_model == NULL) {
@@ -883,7 +903,10 @@ void pc_cpus_init(const char *cpu_model)
     }
 
     for (i = 0; i < smp_cpus; i++) {
-        if (!cpu_x86_init(cpu_model)) {
+        pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
+        if (error) {
+            fprintf(stderr, "%s\n", error_get_pretty(error));
+            error_free(error);
             exit(1);
         }
     }
target-i386/cpu.c +24 -0
index 9cca031..4ddc18a 100644
@@ -1272,6 +1272,45 @@ static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, void *opaque,
     cpu->env.tsc_khz = value / 1000;
 }
 
+static void x86_cpuid_get_apic_id(Object *obj, Visitor *v, void *opaque,
+                                  const char *name, Error **errp)
+{
+    X86CPU *cpu = X86_CPU(obj);
+    int64_t value = cpu->env.cpuid_apic_id;
+
+    visit_type_int(v, &value, name, errp);
+}
+
+static void x86_cpuid_set_apic_id(Object *obj, Visitor *v, void *opaque,
+                                  const char *name, Error **errp)
+{
+    X86CPU *cpu = X86_CPU(obj);
+    const int64_t min = 0;
+    const int64_t max = UINT32_MAX;
+    Error *error = NULL;
+    int64_t value;
+
+    visit_type_int(v, &value, name, &error);
+    if (error) {
+        error_propagate(errp, error);
+        return;
+    }
+    if (value < min || value > max) {

target-i386/cpu.c static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, void *opaque, · line 1308

Quoted 10 lines
target-i386/cpu.c +10 -0
+        error_setg(&error, "Property %s.%s doesn't take value %" PRId64
+                   " (minimum: %" PRId64 ", maximum: %" PRId64 ")" ,
+                   object_get_typename(obj), name, value, min, max);
+        error_propagate(errp, error);
+        return;
+    }
+
+    if (cpu_exists(value)) {
+        error_setg(&error, "CPU with APIC ID %" PRIi64 " exists", value);
+        error_propagate(errp, error);

What about implementing this check after implementing the /machine/icc-bridge links? Then we can simply check if "/machine/icc-bridge/cpus[ID]" is already set.

(And then icc-bridge could be responsible for ensuring APIC ID uniqueness, not the CPU object itself)

Quoted 21 lines
target-i386/cpu.c +8 -0
+        return;
+    }
+    cpu->env.cpuid_apic_id = value;
+}
+
 static int cpu_x86_find_by_name(x86_def_t *x86_cpu_def, const char *name)
 {
     x86_def_t *def;
@@ -2259,6 +2298,9 @@ static void x86_cpu_initfn(Object *obj)
     object_property_add(obj, "tsc-frequency", "int",
                         x86_cpuid_get_tsc_freq,
                         x86_cpuid_set_tsc_freq, NULL, NULL, NULL);
+    object_property_add(obj, "apic-id", "int",
+                        x86_cpuid_get_apic_id,
+                        x86_cpuid_set_apic_id, NULL, NULL, NULL);
 
     env->cpuid_apic_id = x86_cpu_apic_id_from_index(cs->cpu_index);
 
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
#49 lore raw

Re: [Qemu-devel] [PATCH 11/19] target-i386: introduce apic-id property

Quoted 105 lines · On Thu, 11 Apr 2013 16:12:33 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 104 lines · Igor Mammedov

... and use it from board level to set APIC ID for CPUs it creates.

---
Note:
* pc_new_cpu() function will be reused later in CPU hot-plug hook.

v3:
* user error_propagate() in property setter
v2:
* use generic cpu_exists() instead of custom one
* make apic-id dynamic property, so it won't be possible to use it as global property, since it's instance specific

---
 hw/i386/pc.c      | 25 ++++++++++++++++++++++++-
 target-i386/cpu.c | 42 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 66 insertions(+), 1 deletion(-)
hw/i386/pc.c +24 -1
index 8d75b34..3adf294 100644
@@ -869,9 +869,29 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level)
     }
 }
 
+static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
+{
+    X86CPU *cpu;
+
+    cpu = cpu_x86_create(cpu_model, errp);
+    if (!cpu) {
+        return;
+    }
+
+    object_property_set_int(OBJECT(cpu), apic_id, "apic-id", errp);
+    object_property_set_bool(OBJECT(cpu), true, "realized", errp);
+
+    if (error_is_set(errp)) {
+        if (cpu != NULL) {
+            object_unref(OBJECT(cpu));
+        }
+    }
+}
+
 void pc_cpus_init(const char *cpu_model)
 {
     int i;
+    Error *error = NULL;
 
     /* init CPUs */
     if (cpu_model == NULL) {
@@ -883,7 +903,10 @@ void pc_cpus_init(const char *cpu_model)
     }
 
     for (i = 0; i < smp_cpus; i++) {
-        if (!cpu_x86_init(cpu_model)) {
+        pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
+        if (error) {
+            fprintf(stderr, "%s\n", error_get_pretty(error));
+            error_free(error);
             exit(1);
         }
     }
target-i386/cpu.c +30 -0
index 9cca031..4ddc18a 100644
@@ -1272,6 +1272,45 @@ static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, void *opaque,
     cpu->env.tsc_khz = value / 1000;
 }
 
+static void x86_cpuid_get_apic_id(Object *obj, Visitor *v, void *opaque,
+                                  const char *name, Error **errp)
+{
+    X86CPU *cpu = X86_CPU(obj);
+    int64_t value = cpu->env.cpuid_apic_id;
+
+    visit_type_int(v, &value, name, errp);
+}
+
+static void x86_cpuid_set_apic_id(Object *obj, Visitor *v, void *opaque,
+                                  const char *name, Error **errp)
+{
+    X86CPU *cpu = X86_CPU(obj);
+    const int64_t min = 0;
+    const int64_t max = UINT32_MAX;
+    Error *error = NULL;
+    int64_t value;
+
+    visit_type_int(v, &value, name, &error);
+    if (error) {
+        error_propagate(errp, error);
+        return;
+    }
+    if (value < min || value > max) {
+        error_setg(&error, "Property %s.%s doesn't take value %" PRId64
+                   " (minimum: %" PRId64 ", maximum: %" PRId64 ")" ,
+                   object_get_typename(obj), name, value, min, max);
+        error_propagate(errp, error);
+        return;
+    }
Quoted 8 lines

target-i386/cpu.c static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, void *opaque, · line 1308

Quoted 4 lines
target-i386/cpu.c +4 -0
+
+    if (cpu_exists(value)) {
+        error_setg(&error, "CPU with APIC ID %" PRIi64 " exists", value);
+        error_propagate(errp, error);

What about implementing this check after implementing the /machine/icc-bridge links? Then we can simply check if "/machine/icc-bridge/cpus[ID]" is already set.

because it's more generic and won't break even if link location changes

Quoted 3 lines

(And then icc-bridge could be responsible for ensuring APIC ID uniqueness, not the CPU object itself)

Yep, with cpu-add id could be/is verified before setting property, but if one considers device_add or fictional -device cpuxxx on cmd line, then won't be an external check for this. This check takes care about these use cases.

Quoted 26 lines
Quoted 22 lines
target-i386/cpu.c +8 -0
+        return;
+    }
+    cpu->env.cpuid_apic_id = value;
+}
+
 static int cpu_x86_find_by_name(x86_def_t *x86_cpu_def, const char *name)
 {
     x86_def_t *def;
@@ -2259,6 +2298,9 @@ static void x86_cpu_initfn(Object *obj)
     object_property_add(obj, "tsc-frequency", "int",
                         x86_cpuid_get_tsc_freq,
                         x86_cpuid_set_tsc_freq, NULL, NULL, NULL);
+    object_property_add(obj, "apic-id", "int",
+                        x86_cpuid_get_apic_id,
+                        x86_cpuid_set_apic_id, NULL, NULL, NULL);
 
     env->cpuid_apic_id = x86_cpu_apic_id_from_index(cs->cpu_index);
 
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
Signature · 2 lines
Regards,
  Igor
#50 lore raw

Re: [Qemu-devel] [PATCH 11/19] target-i386: introduce apic-id property

Quoted 116 lines · Igor Mammedov
Quoted 105 lines · On Thu, 11 Apr 2013 16:12:33 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 104 lines · Igor Mammedov

... and use it from board level to set APIC ID for CPUs it creates.

---
Note:
* pc_new_cpu() function will be reused later in CPU hot-plug hook.

v3:
* user error_propagate() in property setter
v2:
* use generic cpu_exists() instead of custom one
* make apic-id dynamic property, so it won't be possible to use it as global property, since it's instance specific

---
 hw/i386/pc.c      | 25 ++++++++++++++++++++++++-
 target-i386/cpu.c | 42 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 66 insertions(+), 1 deletion(-)
hw/i386/pc.c +24 -1
index 8d75b34..3adf294 100644
@@ -869,9 +869,29 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level)
     }
 }
 
+static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
+{
+    X86CPU *cpu;
+
+    cpu = cpu_x86_create(cpu_model, errp);
+    if (!cpu) {
+        return;
+    }
+
+    object_property_set_int(OBJECT(cpu), apic_id, "apic-id", errp);
+    object_property_set_bool(OBJECT(cpu), true, "realized", errp);
+
+    if (error_is_set(errp)) {
+        if (cpu != NULL) {
+            object_unref(OBJECT(cpu));
+        }
+    }
+}
+
 void pc_cpus_init(const char *cpu_model)
 {
     int i;
+    Error *error = NULL;
 
     /* init CPUs */
     if (cpu_model == NULL) {
@@ -883,7 +903,10 @@ void pc_cpus_init(const char *cpu_model)
     }
 
     for (i = 0; i < smp_cpus; i++) {
-        if (!cpu_x86_init(cpu_model)) {
+        pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
+        if (error) {
+            fprintf(stderr, "%s\n", error_get_pretty(error));
+            error_free(error);
             exit(1);
         }
     }
target-i386/cpu.c +30 -0
index 9cca031..4ddc18a 100644
@@ -1272,6 +1272,45 @@ static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, void *opaque,
     cpu->env.tsc_khz = value / 1000;
 }
 
+static void x86_cpuid_get_apic_id(Object *obj, Visitor *v, void *opaque,
+                                  const char *name, Error **errp)
+{
+    X86CPU *cpu = X86_CPU(obj);
+    int64_t value = cpu->env.cpuid_apic_id;
+
+    visit_type_int(v, &value, name, errp);
+}
+
+static void x86_cpuid_set_apic_id(Object *obj, Visitor *v, void *opaque,
+                                  const char *name, Error **errp)
+{
+    X86CPU *cpu = X86_CPU(obj);
+    const int64_t min = 0;
+    const int64_t max = UINT32_MAX;
+    Error *error = NULL;
+    int64_t value;
+
+    visit_type_int(v, &value, name, &error);
+    if (error) {
+        error_propagate(errp, error);
+        return;
+    }
+    if (value < min || value > max) {
+        error_setg(&error, "Property %s.%s doesn't take value %" PRId64
+                   " (minimum: %" PRId64 ", maximum: %" PRId64 ")" ,
+                   object_get_typename(obj), name, value, min, max);
+        error_propagate(errp, error);
+        return;
+    }
Quoted 8 lines

target-i386/cpu.c static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, void *opaque, · line 1308

Quoted 4 lines
target-i386/cpu.c +4 -0
+
+    if (cpu_exists(value)) {
+        error_setg(&error, "CPU with APIC ID %" PRIi64 " exists", value);
+        error_propagate(errp, error);

What about implementing this check after implementing the /machine/icc-bridge links? Then we can simply check if "/machine/icc-bridge/cpus[ID]" is already set.

Quoted 1 line

because it's more generic and won't break even if link location changes

Well, if link location changed, we could change the cpu_exists() implementation. We don't even need to hardcode the icc-bridge path, we could simply follow appropriate links if we set them up.

My problem with cpu_exists() is that it is based on global state, instead of simply using the links between bus/devices to find out if the IDs are being allocated correctly. We could make it work like PCI: devfn conflicts are solved simply by looking at the list of devices in the specific PCIBus where the device is being added. See do_pci_register_device().

That said, I am not against the current approach if we don't have the necessary bus/links modelled yet. But if we are going to add icc-bridge, we could benefit from it to implement cpu_exists().

Quoted 7 lines
Quoted 4 lines

(And then icc-bridge could be responsible for ensuring APIC ID uniqueness, not the CPU object itself)

Yep, with cpu-add id could be/is verified before setting property, but if one considers device_add or fictional -device cpuxxx on cmd line, then won't be an external check for this. This check takes care about these use cases.

That's why I believe the CPU has to calculate the APIC ID based on signaling coming from other devices/buses, instead of allowing the APIC ID to be directly specified in the device_add/-device options. That's how real CPUs work: as the CPU manufacturer doesn't know what will be the package ID of the CPU, the APIC IDs are not hardcoded in the CPU; they are calculated based on the CPU topology and some socket identifier signal coming from the board.

Comparing it with PCI again: devfn of PCI devices can be specified manually, but if it is not present you just need to look at the PCI bus to find out the next available devfn.

Based on that, I thought that icc-bridge was going be the component responsible for the APIC ID allocation logic, considering that it already contains APIC-ID-based links to the CPU objects.

Quoted 32 lines
Quoted 27 lines
Quoted 23 lines
target-i386/cpu.c +8 -0
+        return;
+    }
+    cpu->env.cpuid_apic_id = value;
+}
+
 static int cpu_x86_find_by_name(x86_def_t *x86_cpu_def, const char *name)
 {
     x86_def_t *def;
@@ -2259,6 +2298,9 @@ static void x86_cpu_initfn(Object *obj)
     object_property_add(obj, "tsc-frequency", "int",
                         x86_cpuid_get_tsc_freq,
                         x86_cpuid_set_tsc_freq, NULL, NULL, NULL);
+    object_property_add(obj, "apic-id", "int",
+                        x86_cpuid_get_apic_id,
+                        x86_cpuid_set_apic_id, NULL, NULL, NULL);
 
     env->cpuid_apic_id = x86_cpu_apic_id_from_index(cs->cpu_index);
 
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
Signature · 2 lines
Regards,
  Igor
Signature · 1 line
Eduardo
#51 lore raw

Re: [Qemu-devel] [PATCH 11/19] target-i386: introduce apic-id property

Quoted 130 lines · On Fri, 12 Apr 2013 10:13:13 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 116 lines · Igor Mammedov
Quoted 105 lines · On Thu, 11 Apr 2013 16:12:33 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 104 lines · Igor Mammedov

... and use it from board level to set APIC ID for CPUs it creates.

---
Note:
* pc_new_cpu() function will be reused later in CPU hot-plug hook.

v3:
* user error_propagate() in property setter
v2:
* use generic cpu_exists() instead of custom one
* make apic-id dynamic property, so it won't be possible to use it as global property, since it's instance specific

---
 hw/i386/pc.c      | 25 ++++++++++++++++++++++++-
 target-i386/cpu.c | 42 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 66 insertions(+), 1 deletion(-)
hw/i386/pc.c +24 -1
index 8d75b34..3adf294 100644
@@ -869,9 +869,29 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level)
     }
 }
 
+static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
+{
+    X86CPU *cpu;
+
+    cpu = cpu_x86_create(cpu_model, errp);
+    if (!cpu) {
+        return;
+    }
+
+    object_property_set_int(OBJECT(cpu), apic_id, "apic-id", errp);
+    object_property_set_bool(OBJECT(cpu), true, "realized", errp);
+
+    if (error_is_set(errp)) {
+        if (cpu != NULL) {
+            object_unref(OBJECT(cpu));
+        }
+    }
+}
+
 void pc_cpus_init(const char *cpu_model)
 {
     int i;
+    Error *error = NULL;
 
     /* init CPUs */
     if (cpu_model == NULL) {
@@ -883,7 +903,10 @@ void pc_cpus_init(const char *cpu_model)
     }
 
     for (i = 0; i < smp_cpus; i++) {
-        if (!cpu_x86_init(cpu_model)) {
+        pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
+        if (error) {
+            fprintf(stderr, "%s\n", error_get_pretty(error));
+            error_free(error);
             exit(1);
         }
     }
target-i386/cpu.c +30 -0
index 9cca031..4ddc18a 100644
@@ -1272,6 +1272,45 @@ static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, void *opaque,
     cpu->env.tsc_khz = value / 1000;
 }
 
+static void x86_cpuid_get_apic_id(Object *obj, Visitor *v, void *opaque,
+                                  const char *name, Error **errp)
+{
+    X86CPU *cpu = X86_CPU(obj);
+    int64_t value = cpu->env.cpuid_apic_id;
+
+    visit_type_int(v, &value, name, errp);
+}
+
+static void x86_cpuid_set_apic_id(Object *obj, Visitor *v, void *opaque,
+                                  const char *name, Error **errp)
+{
+    X86CPU *cpu = X86_CPU(obj);
+    const int64_t min = 0;
+    const int64_t max = UINT32_MAX;
+    Error *error = NULL;
+    int64_t value;
+
+    visit_type_int(v, &value, name, &error);
+    if (error) {
+        error_propagate(errp, error);
+        return;
+    }
+    if (value < min || value > max) {
+        error_setg(&error, "Property %s.%s doesn't take value %" PRId64
+                   " (minimum: %" PRId64 ", maximum: %" PRId64 ")" ,
+                   object_get_typename(obj), name, value, min, max);
+        error_propagate(errp, error);
+        return;
+    }
Quoted 8 lines

target-i386/cpu.c static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, void *opaque, · line 1308

Quoted 4 lines
target-i386/cpu.c +4 -0
+
+    if (cpu_exists(value)) {
+        error_setg(&error, "CPU with APIC ID %" PRIi64 " exists", value);
+        error_propagate(errp, error);

What about implementing this check after implementing the /machine/icc-bridge links? Then we can simply check if "/machine/icc-bridge/cpus[ID]" is already set.

Quoted 1 line

because it's more generic and won't break even if link location changes

Well, if link location changed, we could change the cpu_exists() implementation. We don't even need to hardcode the icc-bridge path, we could simply follow appropriate links if we set them up.

My problem with cpu_exists() is that it is based on global state, instead of simply using the links between bus/devices to find out if the IDs are being allocated correctly. We could make it work like PCI: devfn conflicts are solved simply by looking at the list of devices in the specific PCIBus where the device is being added. See do_pci_register_device().

Quoted 3 lines

That said, I am not against the current approach if we don't have the necessary bus/links modelled yet. But if we are going to add icc-bridge, we could benefit from it to implement cpu_exists().

Well,check has to be done here anyway, regardless how cpu_exists() implemented.

BTW:
This discussion and below belongs more to 9/19 patch that implements cpu_exists() than here.

Quoted 11 lines
Quoted 8 lines
Quoted 5 lines

(And then icc-bridge could be responsible for ensuring APIC ID uniqueness, not the CPU object itself)

Yep, with cpu-add id could be/is verified before setting property, but if one considers device_add or fictional -device cpuxxx on cmd line, then won't be an external check for this. This check takes care about these use cases.

That's why I believe the CPU has to calculate the APIC ID based on signaling coming from other devices/buses, instead of allowing the APIC

You mean by board not by CPU, I suppose.

Quoted 5 lines

ID to be directly specified in the device_add/-device options. That's how real CPUs work: as the CPU manufacturer doesn't know what will be the package ID of the CPU, the APIC IDs are not hardcoded in the CPU; they are calculated based on the CPU topology and some socket identifier signal coming from the board.

that is why apic_id has been made a property, to be set from outside.

Quoted 4 lines

Comparing it with PCI again: devfn of PCI devices can be specified manually, but if it is not present you just need to look at the PCI bus to find out the next available devfn.

18/19 gives an option to lookup available APIC IDs in a target specific way. If we come up with a better target agnostic way to lookup ID, we can always add it on top of currently proposed implementation.

Quoted 3 lines

Based on that, I thought that icc-bridge was going be the component responsible for the APIC ID allocation logic, considering that it already contains APIC-ID-based links to the CPU objects.

in 19/19 APIC IDs are allocated by board for each possible CPU, icc-bridge is just convenient place for storing links now. If machine was QOM object, I'd probably place it there.

Quoted 37 lines
Quoted 34 lines
Quoted 29 lines
Quoted 25 lines
target-i386/cpu.c +8 -0
+        return;
+    }
+    cpu->env.cpuid_apic_id = value;
+}
+
 static int cpu_x86_find_by_name(x86_def_t *x86_cpu_def, const char *name)
 {
     x86_def_t *def;
@@ -2259,6 +2298,9 @@ static void x86_cpu_initfn(Object *obj)
     object_property_add(obj, "tsc-frequency", "int",
                         x86_cpuid_get_tsc_freq,
                         x86_cpuid_set_tsc_freq, NULL, NULL, NULL);
+    object_property_add(obj, "apic-id", "int",
+                        x86_cpuid_get_apic_id,
+                        x86_cpuid_set_apic_id, NULL, NULL, NULL);
 
     env->cpuid_apic_id = x86_cpu_apic_id_from_index(cs->cpu_index);
 
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
Signature · 2 lines
Regards,
  Igor
Signature · 1 line
Eduardo
Signature · 2 lines
Regards,
  Igor
#52 lore raw

Re: [Qemu-devel] [PATCH 11/19] target-i386: introduce apic-id property

Quoted 2 lines · Igor Mammedov

On Fri, 12 Apr 2013 10:13:13 -0300
Eduardo Habkost <[email protected]> wrote:

[...]

Quoted 42 lines
Quoted 42 lines
Quoted 29 lines
Quoted 21 lines
Quoted 21 lines
target-i386/cpu.c +21 -0
+static void x86_cpuid_set_apic_id(Object *obj, Visitor *v, void *opaque,
+                                  const char *name, Error **errp)
+{
+    X86CPU *cpu = X86_CPU(obj);
+    const int64_t min = 0;
+    const int64_t max = UINT32_MAX;
+    Error *error = NULL;
+    int64_t value;
+
+    visit_type_int(v, &value, name, &error);
+    if (error) {
+        error_propagate(errp, error);
+        return;
+    }
+    if (value < min || value > max) {
+        error_setg(&error, "Property %s.%s doesn't take value %" PRId64
+                   " (minimum: %" PRId64 ", maximum: %" PRId64 ")" ,
+                   object_get_typename(obj), name, value, min, max);
+        error_propagate(errp, error);
+        return;
+    }
Quoted 8 lines

target-i386/cpu.c static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, void *opaque, · line 1308

Quoted 4 lines
target-i386/cpu.c +4 -0
+
+    if (cpu_exists(value)) {
+        error_setg(&error, "CPU with APIC ID %" PRIi64 " exists", value);
+        error_propagate(errp, error);

What about implementing this check after implementing the /machine/icc-bridge links? Then we can simply check if "/machine/icc-bridge/cpus[ID]" is already set.

Quoted 1 line

because it's more generic and won't break even if link location changes

Well, if link location changed, we could change the cpu_exists() implementation. We don't even need to hardcode the icc-bridge path, we could simply follow appropriate links if we set them up.

My problem with cpu_exists() is that it is based on global state, instead of simply using the links between bus/devices to find out if the IDs are being allocated correctly. We could make it work like PCI: devfn conflicts are solved simply by looking at the list of devices in the specific PCIBus where the device is being added. See do_pci_register_device().

Quoted 8 lines
Quoted 3 lines

That said, I am not against the current approach if we don't have the necessary bus/links modelled yet. But if we are going to add icc-bridge, we could benefit from it to implement cpu_exists().

Well,check has to be done here anyway, regardless how cpu_exists() implemented.

BTW:
This discussion and below belongs more to 9/19 patch that implements cpu_exists() than here.

OK.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

(And then icc-bridge could be responsible for ensuring APIC ID uniqueness, not the CPU object itself)

Quoted 7 lines
Quoted 6 lines
Quoted 3 lines

Yep, with cpu-add id could be/is verified before setting property, but if one considers device_add or fictional -device cpuxxx on cmd line, then won't be an external check for this. This check takes care about these use cases.

That's why I believe the CPU has to calculate the APIC ID based on signaling coming from other devices/buses, instead of allowing the APIC

You mean by board not by CPU, I suppose.

Actually I am not 100% sure. For example: we could simply provide the socket/core/thread IDs (or links to socket/core objects?) to the CPU object, and the CPU object could calculate the APIC ID itself.

Quoted 7 lines
Quoted 6 lines

ID to be directly specified in the device_add/-device options. That's how real CPUs work: as the CPU manufacturer doesn't know what will be the package ID of the CPU, the APIC IDs are not hardcoded in the CPU; they are calculated based on the CPU topology and some socket identifier signal coming from the board.

that is why apic_id has been made a property, to be set from outside.

True. I believe the conflict here is: we want other objects to set the APIC ID (be it the board, or socket/core objects), but at the same time it would be interesting to not expose the APIC ID outside QEMU. Being too flexible regarding the APIC ID is more likely to cause problems later.

That said, I don't mind having a "apic-id" property because it is easier to simply expose it directly. But do you agree that: 1) we don't really need to expose it to be set from outside QEMU; 2) we shouldn't require it to be set from outside QEMU; 3) we should recommend users to not try to fiddle it with?

Quoted 9 lines
Quoted 5 lines

Comparing it with PCI again: devfn of PCI devices can be specified manually, but if it is not present you just need to look at the PCI bus to find out the next available devfn.

18/19 gives an option to lookup available APIC IDs in a target specific way. If we come up with a better target agnostic way to lookup ID, we can always add it on top of currently proposed implementation.

Quoted 6 lines
Quoted 3 lines

Based on that, I thought that icc-bridge was going be the component responsible for the APIC ID allocation logic, considering that it already contains APIC-ID-based links to the CPU objects.

in 19/19 APIC IDs are allocated by board for each possible CPU, icc-bridge is just convenient place for storing links now. If machine was QOM object, I'd probably place it there.

I was just wondering if we could solve both problems at the same time
time: if we have a convenient place to store APIC-ID-based links, we are getting a more efficient APIC ID allocation/check system for free. Especially considering how inefficient is the method used by cpu_exists().

It looks like the problem with my approach is that the QOM APIC-ID-based links are target-specific, but cpu_exists() is target-independent. If both were target-specific or both were target-independent, we could easily reuse the same mechanism for both, but that's not the case now.

Signature · 1 line
Eduardo
#53 lore raw

Re: [Qemu-devel] [PATCH 11/19] target-i386: introduce apic-id property

Quoted 81 lines · 在 2013-04-12五的 13:29 -0300,Eduardo Habkost写道:
Quoted 2 lines · Igor Mammedov

On Fri, 12 Apr 2013 10:13:13 -0300
Eduardo Habkost <[email protected]> wrote:

[...]

Quoted 42 lines
Quoted 42 lines
Quoted 29 lines
Quoted 21 lines
Quoted 21 lines
target-i386/cpu.c +21 -0
+static void x86_cpuid_set_apic_id(Object *obj, Visitor *v, void *opaque,
+                                  const char *name, Error **errp)
+{
+    X86CPU *cpu = X86_CPU(obj);
+    const int64_t min = 0;
+    const int64_t max = UINT32_MAX;
+    Error *error = NULL;
+    int64_t value;
+
+    visit_type_int(v, &value, name, &error);
+    if (error) {
+        error_propagate(errp, error);
+        return;
+    }
+    if (value < min || value > max) {
+        error_setg(&error, "Property %s.%s doesn't take value %" PRId64
+                   " (minimum: %" PRId64 ", maximum: %" PRId64 ")" ,
+                   object_get_typename(obj), name, value, min, max);
+        error_propagate(errp, error);
+        return;
+    }
Quoted 8 lines

target-i386/cpu.c static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, void *opaque, · line 1308

Quoted 4 lines
target-i386/cpu.c +4 -0
+
+    if (cpu_exists(value)) {
+        error_setg(&error, "CPU with APIC ID %" PRIi64 " exists", value);
+        error_propagate(errp, error);

What about implementing this check after implementing the /machine/icc-bridge links? Then we can simply check if "/machine/icc-bridge/cpus[ID]" is already set.

Quoted 1 line

because it's more generic and won't break even if link location changes

Well, if link location changed, we could change the cpu_exists() implementation. We don't even need to hardcode the icc-bridge path, we could simply follow appropriate links if we set them up.

My problem with cpu_exists() is that it is based on global state, instead of simply using the links between bus/devices to find out if the IDs are being allocated correctly. We could make it work like PCI: devfn conflicts are solved simply by looking at the list of devices in the specific PCIBus where the device is being added. See do_pci_register_device().

Quoted 8 lines
Quoted 3 lines

That said, I am not against the current approach if we don't have the necessary bus/links modelled yet. But if we are going to add icc-bridge, we could benefit from it to implement cpu_exists().

Well,check has to be done here anyway, regardless how cpu_exists() implemented.

BTW:
This discussion and below belongs more to 9/19 patch that implements cpu_exists() than here.

OK.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

(And then icc-bridge could be responsible for ensuring APIC ID uniqueness, not the CPU object itself)

Quoted 7 lines
Quoted 6 lines
Quoted 3 lines

Yep, with cpu-add id could be/is verified before setting property, but if one considers device_add or fictional -device cpuxxx on cmd line, then won't be an external check for this. This check takes care about these use cases.

That's why I believe the CPU has to calculate the APIC ID based on signaling coming from other devices/buses, instead of allowing the APIC

You mean by board not by CPU, I suppose.

Actually I am not 100% sure. For example: we could simply provide the socket/core/thread IDs (or links to socket/core objects?) to the CPU object, and the CPU object could calculate the APIC ID itself.

Quoted 6 lines
Quoted 6 lines

ID to be directly specified in the device_add/-device options. That's how real CPUs work: as the CPU manufacturer doesn't know what will be the package ID of the CPU, the APIC IDs are not hardcoded in the CPU; they are calculated based on the CPU topology and some socket identifier signal coming from the board.

Quoted 4 lines
Quoted 1 line

that is why apic_id has been made a property, to be set from outside.

True. I believe the conflict here is: we want other objects to set the APIC ID (be it the board, or socket/core objects),

ideally, we really prefer that,
but, you know, QEMU machine implementation seems can't feed us enough to let us decide CPU ID as real hardware board, and neither in future for QEMU doesn't like to emulate many complex signals that connect devices.
so, I think, this property is just as well as we can do, maybe we can't expect too much :-).

Quoted 39 lines

but at the same time
it would be interesting to not expose the APIC ID outside QEMU. Being too flexible regarding the APIC ID is more likely to cause problems later.

That said, I don't mind having a "apic-id" property because it is easier to simply expose it directly. But do you agree that: 1) we don't really need to expose it to be set from outside QEMU; 2) we shouldn't require it to be set from outside QEMU; 3) we should recommend users to not try to fiddle it with?

Quoted 9 lines
Quoted 5 lines

Comparing it with PCI again: devfn of PCI devices can be specified manually, but if it is not present you just need to look at the PCI bus to find out the next available devfn.

18/19 gives an option to lookup available APIC IDs in a target specific way. If we come up with a better target agnostic way to lookup ID, we can always add it on top of currently proposed implementation.

Quoted 6 lines
Quoted 3 lines

Based on that, I thought that icc-bridge was going be the component responsible for the APIC ID allocation logic, considering that it already contains APIC-ID-based links to the CPU objects.

in 19/19 APIC IDs are allocated by board for each possible CPU, icc-bridge is just convenient place for storing links now. If machine was QOM object, I'd probably place it there.

I was just wondering if we could solve both problems at the same time
time: if we have a convenient place to store APIC-ID-based links, we are getting a more efficient APIC ID allocation/check system for free. Especially considering how inefficient is the method used by cpu_exists().

It looks like the problem with my approach is that the QOM APIC-ID-based links are target-specific, but cpu_exists() is target-independent. If both were target-specific or both were target-independent, we could easily reuse the same mechanism for both, but that's not the case now.

#54 lore raw

Re: [Qemu-devel] [PATCH 11/19] target-i386: introduce apic-id property

In reply to Eduardo Habkost
Quoted 97 lines · On Fri, 12 Apr 2013 13:29:40 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 2 lines · Igor Mammedov

On Fri, 12 Apr 2013 10:13:13 -0300
Eduardo Habkost <[email protected]> wrote:

[...]

Quoted 51 lines
Quoted 48 lines
Quoted 34 lines
Quoted 26 lines
Quoted 26 lines

+static void x86_cpuid_set_apic_id(Object *obj, Visitor *v, void *opaque,

target-i386/cpu.c +15 -0
+                                  const char *name, Error **errp)
+{
+    X86CPU *cpu = X86_CPU(obj);
+    const int64_t min = 0;
+    const int64_t max = UINT32_MAX;
+    Error *error = NULL;
+    int64_t value;
+
+    visit_type_int(v, &value, name, &error);
+    if (error) {
+        error_propagate(errp, error);
+        return;
+    }
+    if (value < min || value > max) {
+        error_setg(&error, "Property %s.%s doesn't take value %"

PRId64
+ " (minimum: %" PRId64 ", maximum: %" PRId64
")" ,
+ object_get_typename(obj), name, value, min,
max);

target-i386/cpu.c +4 -0
+        error_propagate(errp, error);
+        return;
+    }
+
Quoted 8 lines
Quoted 4 lines
(quoted diff) +2 -0
+    if (cpu_exists(value)) {
+        error_setg(&error, "CPU with APIC ID %" PRIi64 "

exists", value);
+ error_propagate(errp, error);

What about implementing this check after implementing the /machine/icc-bridge links? Then we can simply check if "/machine/icc-bridge/cpus[ID]" is already set.

Quoted 2 lines

because it's more generic and won't break even if link location changes

Well, if link location changed, we could change the cpu_exists() implementation. We don't even need to hardcode the icc-bridge path, we could simply follow appropriate links if we set them up.

My problem with cpu_exists() is that it is based on global state, instead of simply using the links between bus/devices to find out if the IDs are being allocated correctly. We could make it work like PCI: devfn conflicts are solved simply by looking at the list of devices in the specific PCIBus where the device is being added. See do_pci_register_device().

Quoted 3 lines

That said, I am not against the current approach if we don't have the necessary bus/links modelled yet. But if we are going to add icc-bridge, we could benefit from it to implement cpu_exists().

Quoted 6 lines

Well,check has to be done here anyway, regardless how cpu_exists() implemented.

BTW:
This discussion and below belongs more to 9/19 patch that implements cpu_exists() than here.

OK.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

(And then icc-bridge could be responsible for ensuring APIC ID uniqueness, not the CPU object itself)

Quoted 8 lines
Quoted 7 lines
Quoted 4 lines

Yep, with cpu-add id could be/is verified before setting property, but if one considers device_add or fictional -device cpuxxx on cmd line, then won't be an external check for this. This check takes care about these use cases.

That's why I believe the CPU has to calculate the APIC ID based on signaling coming from other devices/buses, instead of allowing the APIC

You mean by board not by CPU, I suppose.

Actually I am not 100% sure. For example: we could simply provide the socket/core/thread IDs (or links to socket/core objects?) to the CPU object, and the CPU object could calculate the APIC ID itself.

Quoted 7 lines
Quoted 6 lines

ID to be directly specified in the device_add/-device options. That's how real CPUs work: as the CPU manufacturer doesn't know what will be the package ID of the CPU, the APIC IDs are not hardcoded in the CPU; they are calculated based on the CPU topology and some socket identifier signal coming from the board.

that is why apic_id has been made a property, to be set from outside.

True. I believe the conflict here is: we want other objects to set the APIC ID (be it the board, or socket/core objects), but at the same time it would be interesting to not expose the APIC ID outside QEMU. Being too flexible regarding the APIC ID is more likely to cause problems later.

Quoted 5 lines

That said, I don't mind having a "apic-id" property because it is easier to simply expose it directly. But do you agree that: 1) we don't really need to expose it to be set from outside QEMU; 2) we shouldn't require it to be set from outside QEMU; 3) we should recommend users to not try to fiddle it with?

Due to nature of per thread CPU hotplug, management will have to specify some kind of ID to specify which CPU is being plugged. Management really doesn't/shouldn't care what this ID is.

Quoted 17 lines
Quoted 10 lines
Quoted 6 lines

Comparing it with PCI again: devfn of PCI devices can be specified manually, but if it is not present you just need to look at the PCI bus to find out the next available devfn.

18/19 gives an option to lookup available APIC IDs in a target specific way. If we come up with a better target agnostic way to lookup ID, we can always add it on top of currently proposed implementation.

Quoted 6 lines
Quoted 3 lines

Based on that, I thought that icc-bridge was going be the component responsible for the APIC ID allocation logic, considering that it already contains APIC-ID-based links to the CPU objects.

in 19/19 APIC IDs are allocated by board for each possible CPU, icc-bridge is just convenient place for storing links now. If machine was QOM object, I'd probably place it there.

Quoted 5 lines

I was just wondering if we could solve both problems at the same time
time: if we have a convenient place to store APIC-ID-based links, we are getting a more efficient APIC ID allocation/check system for free. Especially considering how inefficient is the method used by cpu_exists().

Like I said, when we come up with target-independent links interface, we can easily switch implementation of cpu_exists(). Right now inefficiency here isn't issue since it's used on slow path only.

Quoted 6 lines

It looks like the problem with my approach is that the QOM APIC-ID-based links are target-specific, but cpu_exists() is target-independent. If both were target-specific or both were target-independent, we could easily reuse the same mechanism for both, but that's not the case now.

cpu_exists() is target-independent but it look-ups CPU by target specific ID, that target provides via get_arch_id() hook. So for target-i386 it will be APIC ID.

#55 lore raw

Re: [Qemu-devel] [PATCH 11/19] target-i386: introduce apic-id property

On Mon, Apr 15, 2013 at 03:37:00PM +0200, Igor Mammedov wrote:
[...]

Quoted 13 lines
Quoted 13 lines
Quoted 6 lines
Quoted 5 lines

ID to be directly specified in the device_add/-device options. That's how real CPUs work: as the CPU manufacturer doesn't know what will be the package ID of the CPU, the APIC IDs are not hardcoded in the CPU; they are calculated based on the CPU topology and some socket identifier signal coming from the board.

that is why apic_id has been made a property, to be set from outside.

True. I believe the conflict here is: we want other objects to set the APIC ID (be it the board, or socket/core objects), but at the same time it would be interesting to not expose the APIC ID outside QEMU. Being too flexible regarding the APIC ID is more likely to cause problems later.

Quoted 8 lines
Quoted 5 lines

That said, I don't mind having a "apic-id" property because it is easier to simply expose it directly. But do you agree that: 1) we don't really need to expose it to be set from outside QEMU; 2) we shouldn't require it to be set from outside QEMU; 3) we should recommend users to not try to fiddle it with?

Due to nature of per thread CPU hotplug, management will have to specify some kind of ID to specify which CPU is being plugged. Management really doesn't/shouldn't care what this ID is.

As long as management really doesn't/shouldn't care what the ID is, exposing the APIC ID in the form of an opaque CPU identifier wouldn't be a problem to me. I just wanted to make clarify if we agree that messing with the APIC ID directly won't be recommended and that the "apic-id" property will be for QEMU internal use only.

Signature · 1 line
Eduardo
#56 lore raw

Re: [Qemu-devel] [PATCH 11/19] target-i386: introduce apic-id property

Quoted 24 lines · On Mon, 15 Apr 2013 10:45:24 -0300 Eduardo Habkost <[email protected]> wrote:

On Mon, Apr 15, 2013 at 03:37:00PM +0200, Igor Mammedov wrote:
[...]

Quoted 13 lines
Quoted 13 lines
Quoted 6 lines
Quoted 5 lines

ID to be directly specified in the device_add/-device options. That's how real CPUs work: as the CPU manufacturer doesn't know what will be the package ID of the CPU, the APIC IDs are not hardcoded in the CPU; they are calculated based on the CPU topology and some socket identifier signal coming from the board.

that is why apic_id has been made a property, to be set from outside.

True. I believe the conflict here is: we want other objects to set the APIC ID (be it the board, or socket/core objects), but at the same time it would be interesting to not expose the APIC ID outside QEMU. Being too flexible regarding the APIC ID is more likely to cause problems later.

Quoted 8 lines
Quoted 5 lines

That said, I don't mind having a "apic-id" property because it is easier to simply expose it directly. But do you agree that: 1) we don't really need to expose it to be set from outside QEMU; 2) we shouldn't require it to be set from outside QEMU; 3) we should recommend users to not try to fiddle it with?

Due to nature of per thread CPU hotplug, management will have to specify some kind of ID to specify which CPU is being plugged. Management really doesn't/shouldn't care what this ID is.

Quoted 5 lines

As long as management really doesn't/shouldn't care what the ID is, exposing the APIC ID in the form of an opaque CPU identifier wouldn't be a problem to me. I just wanted to make clarify if we agree that messing with the APIC ID directly won't be recommended and that the "apic-id" property will be for QEMU internal use only.

On contrary, it's useful external feature, x86 guests see only APIC ID, since it's the only ID they [should] know about. So guest aware mgmt could definitely use apic_id propery to correlate CPU in guest with QEMU view of them.

#57 lore raw

Re: [Qemu-devel] [PATCH 11/19] target-i386: introduce apic-id property

Quoted 32 lines · Igor Mammedov
Quoted 24 lines · On Mon, 15 Apr 2013 10:45:24 -0300 Eduardo Habkost <[email protected]> wrote:

On Mon, Apr 15, 2013 at 03:37:00PM +0200, Igor Mammedov wrote:
[...]

Quoted 13 lines
Quoted 13 lines
Quoted 6 lines
Quoted 5 lines

ID to be directly specified in the device_add/-device options. That's how real CPUs work: as the CPU manufacturer doesn't know what will be the package ID of the CPU, the APIC IDs are not hardcoded in the CPU; they are calculated based on the CPU topology and some socket identifier signal coming from the board.

that is why apic_id has been made a property, to be set from outside.

True. I believe the conflict here is: we want other objects to set the APIC ID (be it the board, or socket/core objects), but at the same time it would be interesting to not expose the APIC ID outside QEMU. Being too flexible regarding the APIC ID is more likely to cause problems later.

Quoted 8 lines
Quoted 5 lines

That said, I don't mind having a "apic-id" property because it is easier to simply expose it directly. But do you agree that: 1) we don't really need to expose it to be set from outside QEMU; 2) we shouldn't require it to be set from outside QEMU; 3) we should recommend users to not try to fiddle it with?

Due to nature of per thread CPU hotplug, management will have to specify some kind of ID to specify which CPU is being plugged. Management really doesn't/shouldn't care what this ID is.

Quoted 5 lines

As long as management really doesn't/shouldn't care what the ID is, exposing the APIC ID in the form of an opaque CPU identifier wouldn't be a problem to me. I just wanted to make clarify if we agree that messing with the APIC ID directly won't be recommended and that the "apic-id" property will be for QEMU internal use only.

Quoted 4 lines

On contrary, it's useful external feature, x86 guests see only APIC ID, since it's the only ID they [should] know about. So guest aware mgmt could definitely use apic_id propery to correlate CPU in guest with QEMU view of them.

You're right, _reading_ the APIC ID is very useful. I am worried about _setting_ it from external code.

Signature · 1 line
Eduardo
#58 lore raw

Re: [Qemu-devel] [PATCH 11/19] target-i386: introduce apic-id property

Quoted 33 lines · On Mon, 15 Apr 2013 11:49:40 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 32 lines · Igor Mammedov
Quoted 29 lines · On Mon, 15 Apr 2013 10:45:24 -0300 Eduardo Habkost <[email protected]> wrote:

On Mon, Apr 15, 2013 at 03:37:00PM +0200, Igor Mammedov wrote:
[...]

Quoted 13 lines
Quoted 13 lines
Quoted 6 lines
Quoted 5 lines

ID to be directly specified in the device_add/-device options. That's how real CPUs work: as the CPU manufacturer doesn't know what will be the package ID of the CPU, the APIC IDs are not hardcoded in the CPU; they are calculated based on the CPU topology and some socket identifier signal coming from the board.

that is why apic_id has been made a property, to be set from outside.

True. I believe the conflict here is: we want other objects to set the APIC ID (be it the board, or socket/core objects), but at the same time it would be interesting to not expose the APIC ID outside QEMU. Being too flexible regarding the APIC ID is more likely to cause problems later.

Quoted 8 lines
Quoted 5 lines

That said, I don't mind having a "apic-id" property because it is easier to simply expose it directly. But do you agree that: 1) we don't really need to expose it to be set from outside QEMU; 2) we shouldn't require it to be set from outside QEMU; 3) we should recommend users to not try to fiddle it with?

Due to nature of per thread CPU hotplug, management will have to specify some kind of ID to specify which CPU is being plugged. Management really doesn't/shouldn't care what this ID is.

As long as management really doesn't/shouldn't care what the ID is, exposing the APIC ID in the form of an opaque CPU identifier wouldn't be a problem to me. I just wanted to make clarify if we agree that messing with the APIC ID directly won't be recommended and that the "apic-id" property will be for QEMU internal use only.

Quoted 7 lines
Quoted 4 lines

On contrary, it's useful external feature, x86 guests see only APIC ID, since it's the only ID they [should] know about. So guest aware mgmt could definitely use apic_id propery to correlate CPU in guest with QEMU view of them.

You're right, _reading_ the APIC ID is very useful. I am worried about _setting_ it from external code.

currently it's not possible since cpu-add doesn't allow to set any properties.

We will need setting it for device_add though.
By then, I guess some way to check that it's valid would be enough, otherwise hot-plugged CPU will be out of scope of MADT and guest would ignore it or through an error.

Quoted 4 lines
Signature · 2 lines
-- 
Eduardo
Signature · 2 lines
Regards,
  Igor
#59 lore raw

Re: [Qemu-devel] [PATCH 11/19] target-i386: introduce apic-id property

Quoted 41 lines · Igor Mammedov
Quoted 38 lines · On Mon, 15 Apr 2013 11:49:40 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 36 lines · Igor Mammedov
Quoted 24 lines · On Mon, 15 Apr 2013 10:45:24 -0300 Eduardo Habkost <[email protected]> wrote:

On Mon, Apr 15, 2013 at 03:37:00PM +0200, Igor Mammedov wrote:
[...]

Quoted 13 lines
Quoted 13 lines
Quoted 6 lines
Quoted 5 lines

ID to be directly specified in the device_add/-device options. That's how real CPUs work: as the CPU manufacturer doesn't know what will be the package ID of the CPU, the APIC IDs are not hardcoded in the CPU; they are calculated based on the CPU topology and some socket identifier signal coming from the board.

that is why apic_id has been made a property, to be set from outside.

True. I believe the conflict here is: we want other objects to set the APIC ID (be it the board, or socket/core objects), but at the same time it would be interesting to not expose the APIC ID outside QEMU. Being too flexible regarding the APIC ID is more likely to cause problems later.

Quoted 8 lines
Quoted 5 lines

That said, I don't mind having a "apic-id" property because it is easier to simply expose it directly. But do you agree that: 1) we don't really need to expose it to be set from outside QEMU; 2) we shouldn't require it to be set from outside QEMU; 3) we should recommend users to not try to fiddle it with?

Due to nature of per thread CPU hotplug, management will have to specify some kind of ID to specify which CPU is being plugged. Management really doesn't/shouldn't care what this ID is.

Quoted 5 lines

As long as management really doesn't/shouldn't care what the ID is, exposing the APIC ID in the form of an opaque CPU identifier wouldn't be a problem to me. I just wanted to make clarify if we agree that messing with the APIC ID directly won't be recommended and that the "apic-id" property will be for QEMU internal use only.

On contrary, it's useful external feature, x86 guests see only APIC ID, since it's the only ID they [should] know about. So guest aware mgmt could definitely use apic_id propery to correlate CPU in guest with QEMU view of them.

Quoted 6 lines
Quoted 2 lines

You're right, _reading_ the APIC ID is very useful. I am worried about _setting_ it from external code.

currently it's not possible since cpu-add doesn't allow to set any properties.

We will need setting it for device_add though.

Not necessarily. That's why I am insisting on an interface based on links/topology, not based on a raw "apic-id" property: instead of setting apic-id directly, we could just require that the CPU be attached to the right socket/core objects, and the APIC ID would be magically calculated correctly.

Or we could just let the right socket/core/thread IDs to be set as properties, and apic-id could be calculated based on that. There are many ways to expose an abstraction that's simpler to use and less likely to cause problems.

Quoted 4 lines

By then, I guess some way to check that it's valid would be enough, otherwise hot-plugged CPU will be out of scope of MADT and guest would ignore it or through an error.

Signature · 1 line
Eduardo
#60 lore raw

Re: [Qemu-devel] [PATCH 11/19] target-i386: introduce apic-id property

Quoted 55 lines · On Mon, 15 Apr 2013 17:49:20 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 41 lines · Igor Mammedov
Quoted 38 lines · On Mon, 15 Apr 2013 11:49:40 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 36 lines · Igor Mammedov
Quoted 24 lines · On Mon, 15 Apr 2013 10:45:24 -0300 Eduardo Habkost <[email protected]> wrote:

On Mon, Apr 15, 2013 at 03:37:00PM +0200, Igor Mammedov wrote:
[...]

Quoted 13 lines
Quoted 13 lines
Quoted 6 lines
Quoted 5 lines

ID to be directly specified in the device_add/-device options. That's how real CPUs work: as the CPU manufacturer doesn't know what will be the package ID of the CPU, the APIC IDs are not hardcoded in the CPU; they are calculated based on the CPU topology and some socket identifier signal coming from the board.

that is why apic_id has been made a property, to be set from outside.

True. I believe the conflict here is: we want other objects to set the APIC ID (be it the board, or socket/core objects), but at the same time it would be interesting to not expose the APIC ID outside QEMU. Being too flexible regarding the APIC ID is more likely to cause problems later.

Quoted 8 lines
Quoted 5 lines

That said, I don't mind having a "apic-id" property because it is easier to simply expose it directly. But do you agree that: 1) we don't really need to expose it to be set from outside QEMU; 2) we shouldn't require it to be set from outside QEMU; 3) we should recommend users to not try to fiddle it with?

Due to nature of per thread CPU hotplug, management will have to specify some kind of ID to specify which CPU is being plugged. Management really doesn't/shouldn't care what this ID is.

Quoted 5 lines

As long as management really doesn't/shouldn't care what the ID is, exposing the APIC ID in the form of an opaque CPU identifier wouldn't be a problem to me. I just wanted to make clarify if we agree that messing with the APIC ID directly won't be recommended and that the "apic-id" property will be for QEMU internal use only.

On contrary, it's useful external feature, x86 guests see only APIC ID, since it's the only ID they [should] know about. So guest aware mgmt could definitely use apic_id propery to correlate CPU in guest with QEMU view of them.

Quoted 6 lines
Quoted 2 lines

You're right, _reading_ the APIC ID is very useful. I am worried about _setting_ it from external code.

currently it's not possible since cpu-add doesn't allow to set any properties.

We will need setting it for device_add though.

Not necessarily. That's why I am insisting on an interface based on links/topology, not based on a raw "apic-id" property: instead of setting apic-id directly, we could just require that the CPU be attached to the right socket/core objects, and the APIC ID would be magically calculated correctly.

Quoted 4 lines

Or we could just let the right socket/core/thread IDs to be set as properties, and apic-id could be calculated based on that. There are many ways to expose an abstraction that's simpler to use and less likely to cause problems.

if we have tree like this:

/nodes/0../sockets/0../cores/0../threads/0..
user will have to parse all branches IDs to collect node/socket/core/thread IDs before adding CPU.
If use APIC ID on thread level then user is required to get only it and no magic would be required to calculate it, because board has already calculated it in advance.

BTW:
it'd be nicer to refuse wrong id earlier at property setting time then postponing it till realize when it would be possible to check that node/socket/core/thread info provided earlier was correct.

Quoted 9 lines
Quoted 6 lines

By then, I guess some way to check that it's valid would be enough, otherwise hot-plugged CPU will be out of scope of MADT and guest would ignore it or through an error.

Signature · 1 line
Eduardo
Signature · 2 lines
Regards,
  Igor
#61 lore raw

PATCH v3 12/19 introduce ICC bus/device/bridge

In reply to Igor Mammedov

... to provide hotplug-able bus.

* icc-bridge will serve as a parent for icc-bus, provide mmio mapping services to child icc-device and create IOAPIC when requested.
* icc-device will replace SysBusDevice as a parent of APIC and IOAPIC devices.

---
v2:
* Rebase on top the last HW reorganization series.
* Move hw/icc_bus.c into hw/cpu/ and hw/icc_bus.h include/hw/i386/

mv icc to hw/cpu

---
 default-configs/i386-softmmu.mak   |  1 +
 default-configs/x86_64-softmmu.mak |  1 +
 hw/cpu/Makefile.objs               |  1 +
 hw/cpu/icc_bus.c                   | 85 ++++++++++++++++++++++++++++++++++++++
 hw/i386/pc_piix.c                  |  7 ++++
 hw/i386/pc_q35.c                   |  7 ++++
 include/hw/i386/icc_bus.h          | 50 ++++++++++++++++++++++
 7 files changed, 152 insertions(+)
 create mode 100644 hw/cpu/icc_bus.c
 create mode 100644 include/hw/i386/icc_bus.h
default-configs/i386-softmmu.mak +1 -0
index 4ac0694..0789303 100644
@@ -45,3 +45,4 @@ CONFIG_LPC_ICH9=y
 CONFIG_Q35=y
 CONFIG_APIC=y
 CONFIG_IOAPIC=y
+CONFIG_ICC_BUS=y
default-configs/x86_64-softmmu.mak +1 -0
index bf4091c..8d03e9a 100644
@@ -45,3 +45,4 @@ CONFIG_LPC_ICH9=y
 CONFIG_Q35=y
 CONFIG_APIC=y
 CONFIG_IOAPIC=y
+CONFIG_ICC_BUS=y
hw/cpu/Makefile.objs +1 -0
index a49ca04..4461ece 100644
@@ -1,4 +1,5 @@
 obj-$(CONFIG_ARM11MPCORE) += arm11mpcore.o
 obj-$(CONFIG_ARM9MPCORE) += a9mpcore.o
 obj-$(CONFIG_ARM15MPCORE) += a15mpcore.o
+obj-$(CONFIG_ICC_BUS) += icc_bus.o
 
hw/cpu/icc_bus.c new file +85 -0

85 lines not on this page · raw message

hw/i386/pc_piix.c +7 -0
index cff8013..d8c3e1a 100644
@@ -37,6 +37,7 @@
 #include "hw/kvm/clock.h"
 #include "sysemu/sysemu.h"
 #include "hw/sysbus.h"
+#include "hw/i386/icc_bus.h"
 #include "sysemu/arch_init.h"
 #include "sysemu/blockdev.h"
 #include "hw/i2c/smbus.h"
@@ -84,8 +85,13 @@ static void pc_init1(MemoryRegion *system_memory,
     MemoryRegion *ram_memory;
     MemoryRegion *pci_memory;
     MemoryRegion *rom_memory;
+    DeviceState *icc_bridge;
     void *fw_cfg = NULL;
 
+    icc_bridge = qdev_create(NULL, TYPE_ICC_BRIDGE);
+    object_property_add_child(qdev_get_machine(), "icc-bridge",
+                              OBJECT(icc_bridge), NULL);
+
     pc_cpus_init(cpu_model);
     pc_acpi_init("acpi-dsdt.aml");
 
@@ -160,6 +166,7 @@ static void pc_init1(MemoryRegion *system_memory,
     if (pci_enabled) {
         ioapic_init_gsi(gsi_state, "i440fx");
     }
+    qdev_init_nofail(icc_bridge);
 
     pc_register_ferr_irq(gsi[13]);
 
hw/i386/pc_q35.c +7 -0
index 6ac1a89..9055145 100644
@@ -41,6 +41,7 @@
 #include "hw/ide/pci.h"
 #include "hw/ide/ahci.h"
 #include "hw/usb.h"
+#include "hw/i386/icc_bus.h"
 
 /* ICH9 AHCI has 6 ports */
 #define MAX_SATA_PORTS     6
@@ -85,6 +86,11 @@ static void pc_q35_init(QEMUMachineInitArgs *args)
     ICH9LPCState *ich9_lpc;
     PCIDevice *ahci;
     qemu_irq *cmos_s3;
+    DeviceState *icc_bridge;
+
+    icc_bridge = qdev_create(NULL, TYPE_ICC_BRIDGE);
+    object_property_add_child(qdev_get_machine(), "icc-bridge",
+                              OBJECT(icc_bridge), NULL);
 
     pc_cpus_init(cpu_model);
     pc_acpi_init("q35-acpi-dsdt.aml");
@@ -168,6 +174,7 @@ static void pc_q35_init(QEMUMachineInitArgs *args)
     if (pci_enabled) {
         ioapic_init_gsi(gsi_state, NULL);
     }
+    qdev_init_nofail(icc_bridge);
 
     pc_register_ferr_irq(gsi[13]);
 
include/hw/i386/icc_bus.h new file +50 -0

50 lines not on this page · raw message

Signature · 1 line
1.8.2
#62 lore raw

PATCH v3 13/19 target-i386: cpu: attach ICC bus to CPU on its creation

In reply to Igor Mammedov

... during startup, so it would be possible to uplug it later and set bus_type to TYPE_ICC_BUS for X86CPU type to make device_add attach hotplugged CPU to ICC bus.

---
 target-i386/cpu.c | 7 +++++++
 1 file changed, 7 insertions(+)
target-i386/cpu.c +7 -0
index 4ddc18a..2bb38d6 100644
@@ -41,6 +41,7 @@
 #endif
 
 #include "sysemu/sysemu.h"
+#include "hw/i386/icc_bus.h"
 #ifndef CONFIG_USER_ONLY
 #include "hw/xen/xen.h"
 #include "hw/sysbus.h"
@@ -1610,6 +1611,7 @@ X86CPU *cpu_x86_create(const char *cpu_model, Error **errp)
     gchar **model_pieces;
     char *name, *features;
     Error *error = NULL;
+    Object *icc_bus = object_resolve_path_type("icc-bus", TYPE_ICC_BUS, NULL);
 
     model_pieces = g_strsplit(cpu_model, ",", 2);
     if (!model_pieces[0]) {
@@ -1620,6 +1622,10 @@ X86CPU *cpu_x86_create(const char *cpu_model, Error **errp)
     features = model_pieces[1];
 
     cpu = X86_CPU(object_new(TYPE_X86_CPU));
+    if (icc_bus) {
+        qdev_set_parent_bus(DEVICE(cpu), BUS(icc_bus));
+        object_unref(OBJECT(cpu));
+    }
     env = &cpu->env;
     env->cpu_model_str = cpu_model;
 
@@ -2330,6 +2336,7 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 
     xcc->parent_realize = dc->realize;
     dc->realize = x86_cpu_realizefn;
+    dc->bus_type = TYPE_ICC_BUS;
 
     xcc->parent_reset = cc->reset;
     cc->reset = x86_cpu_reset;
Signature · 1 line
1.8.2
#63 lore raw

Re: [Qemu-devel] [PATCH 13/19] target-i386: cpu: attach ICC bus to CPU on its creation

Am 11.04.2013 16:51, schrieb Igor Mammedov:

Quoted 1 line

... during startup, so it would be possible to uplug it later

unplug?

Quoted 29 lines

and set bus_type to TYPE_ICC_BUS for X86CPU type to make device_add attach hotplugged CPU to ICC bus.

---
 target-i386/cpu.c | 7 +++++++
 1 file changed, 7 insertions(+)
target-i386/cpu.c +2 -0
index 4ddc18a..2bb38d6 100644
@@ -41,6 +41,7 @@
 #endif
 
 #include "sysemu/sysemu.h"
+#include "hw/i386/icc_bus.h"
 #ifndef CONFIG_USER_ONLY
 #include "hw/xen/xen.h"
 #include "hw/sysbus.h"
@@ -1610,6 +1611,7 @@ X86CPU *cpu_x86_create(const char *cpu_model, Error **errp)
     gchar **model_pieces;
     char *name, *features;
     Error *error = NULL;
+    Object *icc_bus = object_resolve_path_type("icc-bus", TYPE_ICC_BUS, NULL);
 
     model_pieces = g_strsplit(cpu_model, ",", 2);
     if (!model_pieces[0]) {

target-i386/cpu.c X86CPU *cpu_x86_create(const char *cpu_model, Error **errp) · line 1627

Quoted 6 lines
target-i386/cpu.c +3 -0
@@ -1620,6 +1622,10 @@ X86CPU *cpu_x86_create(const char *cpu_model, Error **errp)
     features = model_pieces[1];
 
     cpu = X86_CPU(object_new(TYPE_X86_CPU));
+    if (icc_bus) {
+        qdev_set_parent_bus(DEVICE(cpu), BUS(icc_bus));
+        object_unref(OBJECT(cpu));

Why are you unref'ing here? (Paolo?)

target-i386/cpu.c static void x86_cpu_common_class_init(ObjectClass *oc, void *data) · line 2342

Quoted 13 lines
target-i386/cpu.c +2 -0
+    }
     env = &cpu->env;
     env->cpu_model_str = cpu_model;
 
@@ -2330,6 +2336,7 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 
     xcc->parent_realize = dc->realize;
     dc->realize = x86_cpu_realizefn;
+    dc->bus_type = TYPE_ICC_BUS;
 
     xcc->parent_reset = cc->reset;
     cc->reset = x86_cpu_reset;

I was relieved when I recently saw this was all the magic required to put it onto the new ICC bus. I thought we'd need to mess with ICCDevice base classes, but that seems to be used for APIC and IOAPIC only.

Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#64 lore raw

Re: [Qemu-devel] [PATCH 13/19] target-i386: cpu: attach ICC bus to CPU on its creation

Quoted 4 lines · On Mon, 15 Apr 2013 17:38:51 +0200 Andreas Färber <[email protected]> wrote:

Am 11.04.2013 16:51, schrieb Igor Mammedov:

Quoted 1 line

... during startup, so it would be possible to uplug it later

unplug?

CPU won't be unplug-able if it's not on ICC bus.

Quoted 32 lines
Quoted 32 lines

and set bus_type to TYPE_ICC_BUS for X86CPU type to make device_add attach hotplugged CPU to ICC bus.

---
 target-i386/cpu.c | 7 +++++++
 1 file changed, 7 insertions(+)
target-i386/cpu.c +2 -0
index 4ddc18a..2bb38d6 100644
@@ -41,6 +41,7 @@
 #endif
 
 #include "sysemu/sysemu.h"
+#include "hw/i386/icc_bus.h"
 #ifndef CONFIG_USER_ONLY
 #include "hw/xen/xen.h"
 #include "hw/sysbus.h"
@@ -1610,6 +1611,7 @@ X86CPU *cpu_x86_create(const char *cpu_model, Error
 **errp) gchar **model_pieces;
     char *name, *features;
     Error *error = NULL;
+    Object *icc_bus = object_resolve_path_type("icc-bus", TYPE_ICC_BUS,
 NULL); 
     model_pieces = g_strsplit(cpu_model, ",", 2);
     if (!model_pieces[0]) {
@@ -1620,6 +1622,10 @@ X86CPU *cpu_x86_create(const char *cpu_model,
 Error **errp) features = model_pieces[1];
 
Quoted 6 lines

target-i386/cpu.c X86CPU *cpu_x86_create(const char *cpu_model, Error **errp) · line 1627

Quoted 4 lines
target-i386/cpu.c +3 -0
     cpu = X86_CPU(object_new(TYPE_X86_CPU));
+    if (icc_bus) {
+        qdev_set_parent_bus(DEVICE(cpu), BUS(icc_bus));
+        object_unref(OBJECT(cpu));

Why are you unref'ing here? (Paolo?)

To leave only one reference to it since qdev_set_parent_bus() just incremented it. Like it's done in qdev_try_create().

Quoted 21 lines
Quoted 7 lines
target-i386/cpu.c +1 -0
+    }
     env = &cpu->env;
     env->cpu_model_str = cpu_model;
 

target-i386/cpu.c static void x86_cpu_common_class_init(ObjectClass · line 2342

Quoted 7 lines
target-i386/cpu.c +1 -0
@@ -2330,6 +2336,7 @@ static void x86_cpu_common_class_init(ObjectClass
 *oc, void *data) 
     xcc->parent_realize = dc->realize;
     dc->realize = x86_cpu_realizefn;
+    dc->bus_type = TYPE_ICC_BUS;
 
     xcc->parent_reset = cc->reset;
     cc->reset = x86_cpu_reset;

I was relieved when I recently saw this was all the magic required to put it onto the new ICC bus. I thought we'd need to mess with ICCDevice base classes, but that seems to be used for APIC and IOAPIC only.

Andreas

#65 lore raw

Re: [Qemu-devel] [PATCH 13/19] target-i386: cpu: attach ICC bus to CPU on its creation

Am 15.04.2013 17:49, schrieb Igor Mammedov:

Quoted 8 lines
Quoted 4 lines · On Mon, 15 Apr 2013 17:38:51 +0200 Andreas Färber <[email protected]> wrote:

Am 11.04.2013 16:51, schrieb Igor Mammedov:

Quoted 1 line

... during startup, so it would be possible to uplug it later

unplug?

CPU won't be unplug-able if it's not on ICC bus.

Sorry? Question was whether "uplug" is a typo and should read "unplug"? On openSUSE 12.3 there is no command called uplug. :)

Quoted 12 lines
Quoted 12 lines
Quoted 12 lines

and set bus_type to TYPE_ICC_BUS for X86CPU type to make device_add attach hotplugged CPU to ICC bus.

---
 target-i386/cpu.c | 7 +++++++
 1 file changed, 7 insertions(+)
target-i386/cpu.c +0 -0
index 4ddc18a..2bb38d6 100644

[...]

Quoted 10 lines
Quoted 9 lines

target-i386/cpu.c X86CPU *cpu_x86_create(const char *cpu_model, · line 1627

Quoted 7 lines
target-i386/cpu.c +3 -0
@@ -1620,6 +1622,10 @@ X86CPU *cpu_x86_create(const char *cpu_model,
 Error **errp) features = model_pieces[1];
 
     cpu = X86_CPU(object_new(TYPE_X86_CPU));
+    if (icc_bus) {
+        qdev_set_parent_bus(DEVICE(cpu), BUS(icc_bus));
+        object_unref(OBJECT(cpu));

Why are you unref'ing here? (Paolo?)

Quoted 2 lines

To leave only one reference to it since qdev_set_parent_bus() just incremented it. Like it's done in qdev_try_create().

Ah, I see, thanks.

Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#66 lore raw

Re: [Qemu-devel] [PATCH 13/19] target-i386: cpu: attach ICC bus to CPU on its creation

Quoted 4 lines · On Mon, 15 Apr 2013 18:06:40 +0200 Andreas Färber <[email protected]> wrote:

Am 15.04.2013 17:49, schrieb Igor Mammedov:

Quoted 3 lines

On Mon, 15 Apr 2013 17:38:51 +0200
Andreas Färber <[email protected]> wrote:

Quoted 8 lines
Quoted 5 lines
Quoted 4 lines

Am 11.04.2013 16:51, schrieb Igor Mammedov:

Quoted 1 line

... during startup, so it would be possible to uplug it later

unplug?

CPU won't be unplug-able if it's not on ICC bus.

Sorry? Question was whether "uplug" is a typo and should read "unplug"? On openSUSE 12.3 there is no command called uplug. :)

Thanks, I'll fix it.

Quoted 31 lines
Quoted 13 lines
Quoted 13 lines
Quoted 13 lines

and set bus_type to TYPE_ICC_BUS for X86CPU type to make device_add attach hotplugged CPU to ICC bus.

---
 target-i386/cpu.c | 7 +++++++
 1 file changed, 7 insertions(+)
target-i386/cpu.c +0 -0
index 4ddc18a..2bb38d6 100644

[...]

Quoted 10 lines
Quoted 9 lines

target-i386/cpu.c X86CPU *cpu_x86_create(const char *cpu_model, · line 1627

Quoted 7 lines
target-i386/cpu.c +3 -0
@@ -1620,6 +1622,10 @@ X86CPU *cpu_x86_create(const char *cpu_model,
 Error **errp) features = model_pieces[1];
 
     cpu = X86_CPU(object_new(TYPE_X86_CPU));
+    if (icc_bus) {
+        qdev_set_parent_bus(DEVICE(cpu), BUS(icc_bus));
+        object_unref(OBJECT(cpu));

Why are you unref'ing here? (Paolo?)

Quoted 2 lines

To leave only one reference to it since qdev_set_parent_bus() just incremented it. Like it's done in qdev_try_create().

Ah, I see, thanks.

Andreas

#67 lore raw

PATCH v3 14/19 target-i386: replace MSI_SPACE_SIZE with APIC_SPACE_SIZE

In reply to Igor Mammedov

... and put APIC_SPACE_SIZE in public header so that it could be reused later elsewhere.

---
 hw/i386/kvm/apic.c              | 2 +-
 hw/intc/apic.c                  | 2 +-
 hw/xen/xen_apic.c               | 2 +-
 include/hw/i386/apic_internal.h | 2 --
 target-i386/cpu.h               | 1 +
 5 files changed, 4 insertions(+), 5 deletions(-)
hw/i386/kvm/apic.c +1 -1
index c6ff982..8f80425 100644
@@ -174,7 +174,7 @@ static const MemoryRegionOps kvm_apic_io_ops = {
 static void kvm_apic_init(APICCommonState *s)
 {
     memory_region_init_io(&s->io_memory, &kvm_apic_io_ops, s, "kvm-apic-msi",
-                          MSI_SPACE_SIZE);
+                          APIC_SPACE_SIZE);
 
     if (kvm_has_gsi_routing()) {
         msi_supported = true;
hw/intc/apic.c +1 -1
index 2d79a9e..756dff0 100644
@@ -874,7 +874,7 @@ static const MemoryRegionOps apic_io_ops = {
 static void apic_init(APICCommonState *s)
 {
     memory_region_init_io(&s->io_memory, &apic_io_ops, s, "apic-msi",
-                          MSI_SPACE_SIZE);
+                          APIC_SPACE_SIZE);
 
     s->timer = qemu_new_timer_ns(vm_clock, apic_timer, s);
     local_apics[s->idx] = s;
hw/xen/xen_apic.c +1 -1
index a2eb8a1..a958782 100644
@@ -39,7 +39,7 @@ static const MemoryRegionOps xen_apic_io_ops = {
 static void xen_apic_init(APICCommonState *s)
 {
     memory_region_init_io(&s->io_memory, &xen_apic_io_ops, s, "xen-apic-msi",
-                          MSI_SPACE_SIZE);
+                          APIC_SPACE_SIZE);
 
 #if defined(CONFIG_XEN_CTRL_INTERFACE_VERSION) \
     && CONFIG_XEN_CTRL_INTERFACE_VERSION >= 420
include/hw/i386/apic_internal.h +0 -2
index 578241f..aac6290 100644
@@ -66,8 +66,6 @@
 
 #define MAX_APICS 255
 
-#define MSI_SPACE_SIZE                  0x100000
-
 typedef struct APICCommonState APICCommonState;
 
 #define TYPE_APIC_COMMON "apic-common"
target-i386/cpu.h +1 -0
index cf1b05c..65cf97d 100644
@@ -1269,5 +1269,6 @@ uint32_t x86_cpu_apic_id_from_index(unsigned int cpu_index);
 void enable_compat_apic_id_mode(void);
 
 #define APIC_DEFAULT_ADDRESS 0xfee00000
+#define APIC_SPACE_SIZE      0x100000
 
 #endif /* CPU_I386_H */
Signature · 1 line
1.8.2
#68 lore raw

Re: [Qemu-devel] [PATCH 14/19] target-i386: replace MSI_SPACE_SIZE with APIC_SPACE_SIZE

Am 11.04.2013 16:51, schrieb Igor Mammedov:

Quoted 4 lines

... and put APIC_SPACE_SIZE in public header so that it could be reused later elsewhere.

mst or Jan, can one of you please ack or nack?

Thanks,
Andreas

Quoted 72 lines
---
 hw/i386/kvm/apic.c              | 2 +-
 hw/intc/apic.c                  | 2 +-
 hw/xen/xen_apic.c               | 2 +-
 include/hw/i386/apic_internal.h | 2 --
 target-i386/cpu.h               | 1 +
 5 files changed, 4 insertions(+), 5 deletions(-)
hw/i386/kvm/apic.c +1 -1
index c6ff982..8f80425 100644
@@ -174,7 +174,7 @@ static const MemoryRegionOps kvm_apic_io_ops = {
 static void kvm_apic_init(APICCommonState *s)
 {
     memory_region_init_io(&s->io_memory, &kvm_apic_io_ops, s, "kvm-apic-msi",
-                          MSI_SPACE_SIZE);
+                          APIC_SPACE_SIZE);
 
     if (kvm_has_gsi_routing()) {
         msi_supported = true;
hw/intc/apic.c +1 -1
index 2d79a9e..756dff0 100644
@@ -874,7 +874,7 @@ static const MemoryRegionOps apic_io_ops = {
 static void apic_init(APICCommonState *s)
 {
     memory_region_init_io(&s->io_memory, &apic_io_ops, s, "apic-msi",
-                          MSI_SPACE_SIZE);
+                          APIC_SPACE_SIZE);
 
     s->timer = qemu_new_timer_ns(vm_clock, apic_timer, s);
     local_apics[s->idx] = s;
hw/xen/xen_apic.c +1 -1
index a2eb8a1..a958782 100644
@@ -39,7 +39,7 @@ static const MemoryRegionOps xen_apic_io_ops = {
 static void xen_apic_init(APICCommonState *s)
 {
     memory_region_init_io(&s->io_memory, &xen_apic_io_ops, s, "xen-apic-msi",
-                          MSI_SPACE_SIZE);
+                          APIC_SPACE_SIZE);
 
 #if defined(CONFIG_XEN_CTRL_INTERFACE_VERSION) \
     && CONFIG_XEN_CTRL_INTERFACE_VERSION >= 420
include/hw/i386/apic_internal.h +0 -2
index 578241f..aac6290 100644
@@ -66,8 +66,6 @@
 
 #define MAX_APICS 255
 
-#define MSI_SPACE_SIZE                  0x100000
-
 typedef struct APICCommonState APICCommonState;
 
 #define TYPE_APIC_COMMON "apic-common"
target-i386/cpu.h +1 -0
index cf1b05c..65cf97d 100644
@@ -1269,5 +1269,6 @@ uint32_t x86_cpu_apic_id_from_index(unsigned int cpu_index);
 void enable_compat_apic_id_mode(void);
 
 #define APIC_DEFAULT_ADDRESS 0xfee00000
+#define APIC_SPACE_SIZE      0x100000
 
 #endif /* CPU_I386_H */
Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#69 lore raw

Re: [Qemu-devel] [PATCH 14/19] target-i386: replace MSI_SPACE_SIZE with APIC_SPACE_SIZE

Quoted 7 lines · Andreas Färber

Am 11.04.2013 16:51, schrieb Igor Mammedov:

Quoted 4 lines

... and put APIC_SPACE_SIZE in public header so that it could be reused later elsewhere.

mst or Jan, can one of you please ack or nack?

Well, the problem remains that we are mixing up two distinct address spaces in one mapping. But that's not the fault of the patch and nothing this series can change (we need CPU-local MMIO dispatching to fix it).

IOW, renaming and relocation does not make things worse, so I'm fine with it if it helps.

Jan

Quoted 72 lines
Quoted 72 lines
---
 hw/i386/kvm/apic.c              | 2 +-
 hw/intc/apic.c                  | 2 +-
 hw/xen/xen_apic.c               | 2 +-
 include/hw/i386/apic_internal.h | 2 --
 target-i386/cpu.h               | 1 +
 5 files changed, 4 insertions(+), 5 deletions(-)
hw/i386/kvm/apic.c +1 -1
index c6ff982..8f80425 100644
@@ -174,7 +174,7 @@ static const MemoryRegionOps kvm_apic_io_ops = {
 static void kvm_apic_init(APICCommonState *s)
 {
     memory_region_init_io(&s->io_memory, &kvm_apic_io_ops, s, "kvm-apic-msi",
-                          MSI_SPACE_SIZE);
+                          APIC_SPACE_SIZE);
 
     if (kvm_has_gsi_routing()) {
         msi_supported = true;
hw/intc/apic.c +1 -1
index 2d79a9e..756dff0 100644
@@ -874,7 +874,7 @@ static const MemoryRegionOps apic_io_ops = {
 static void apic_init(APICCommonState *s)
 {
     memory_region_init_io(&s->io_memory, &apic_io_ops, s, "apic-msi",
-                          MSI_SPACE_SIZE);
+                          APIC_SPACE_SIZE);
 
     s->timer = qemu_new_timer_ns(vm_clock, apic_timer, s);
     local_apics[s->idx] = s;
hw/xen/xen_apic.c +1 -1
index a2eb8a1..a958782 100644
@@ -39,7 +39,7 @@ static const MemoryRegionOps xen_apic_io_ops = {
 static void xen_apic_init(APICCommonState *s)
 {
     memory_region_init_io(&s->io_memory, &xen_apic_io_ops, s, "xen-apic-msi",
-                          MSI_SPACE_SIZE);
+                          APIC_SPACE_SIZE);
 
 #if defined(CONFIG_XEN_CTRL_INTERFACE_VERSION) \
     && CONFIG_XEN_CTRL_INTERFACE_VERSION >= 420
include/hw/i386/apic_internal.h +0 -2
index 578241f..aac6290 100644
@@ -66,8 +66,6 @@
 
 #define MAX_APICS 255
 
-#define MSI_SPACE_SIZE                  0x100000
-
 typedef struct APICCommonState APICCommonState;
 
 #define TYPE_APIC_COMMON "apic-common"
target-i386/cpu.h +1 -0
index cf1b05c..65cf97d 100644
@@ -1269,5 +1269,6 @@ uint32_t x86_cpu_apic_id_from_index(unsigned int cpu_index);
 void enable_compat_apic_id_mode(void);
 
 #define APIC_DEFAULT_ADDRESS 0xfee00000
+#define APIC_SPACE_SIZE      0x100000
 
 #endif /* CPU_I386_H */
Signature · 2 lines
Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
Corporate Competence Center Embedded Linux
#70 lore raw

PATCH v3 15/19 target-i386: move APIC to ICC bus

In reply to Igor Mammedov

... to allow it to be hotplugged

* map APIC's mmio at board level if it is present
* do not register mmio region for each APIC, since only one is used/mapped

---
 hw/cpu/icc_bus.c                | 15 ++++++++++++++-
 hw/i386/kvmvapic.c              |  1 +
 hw/i386/pc.c                    | 20 +++++++++++++++++---
 hw/intc/apic_common.c           | 17 ++++++++++++-----
 include/hw/i386/apic_internal.h |  6 +++---
 include/hw/i386/icc_bus.h       |  2 ++
 target-i386/cpu.c               | 16 +++-------------
 7 files changed, 52 insertions(+), 25 deletions(-)
hw/cpu/icc_bus.c +14 -1
index 3824968..c499f09 100644
@@ -59,13 +59,26 @@ static const TypeInfo icc_device_info = {
 
 typedef struct ICCBridgeState {
     SysBusDevice busdev;
+    MemoryRegion apic_container;
 } ICCBridgeState;
 #define ICC_BRIGDE(obj) OBJECT_CHECK(ICCBridgeState, (obj), TYPE_ICC_BRIDGE)
 
 
 static void icc_bridge_initfn(Object *obj)
 {
-    qbus_create(TYPE_ICC_BUS, DEVICE(obj), "icc-bus");
+    ICCBridgeState *s = ICC_BRIGDE(obj);
+    SysBusDevice *sb = SYS_BUS_DEVICE(obj);
+    ICCBus *ibus;
+
+    ibus = ICC_BUS(qbus_create(TYPE_ICC_BUS, DEVICE(obj), "icc-bus"));
+
+    /* Do not change order of registering regions,
+     * APIC must be first registered region, board maps it by 0 index
+     */
+    memory_region_init(&s->apic_container, "icc-apic-container",
+                       APIC_SPACE_SIZE);
+    sysbus_init_mmio(sb, &s->apic_container);
+    ibus->apic_address_space = &s->apic_container;
 }
 
 static const TypeInfo icc_bridge_info = {
hw/i386/kvmvapic.c +1 -0
index 3a10c07..5b558aa 100644
@@ -12,6 +12,7 @@
 #include "sysemu/cpus.h"
 #include "sysemu/kvm.h"
 #include "hw/i386/apic_internal.h"
+#include "hw/sysbus.h"
 
 #define VAPIC_IO_PORT           0x7e
 
hw/i386/pc.c +17 -3
index 3adf294..1c7b3c2 100644
@@ -52,6 +52,7 @@
 #include "sysemu/arch_init.h"
 #include "qemu/bitmap.h"
 #include "qemu/config-file.h"
+#include "hw/i386/icc_bus.h"
 
 /* debug PC/ISA interrupts */
 //#define DEBUG_IRQ
@@ -869,13 +870,13 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level)
     }
 }
 
-static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
+static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
 {
     X86CPU *cpu;
 
     cpu = cpu_x86_create(cpu_model, errp);
     if (!cpu) {
-        return;
+        return cpu;
     }
 
     object_property_set_int(OBJECT(cpu), apic_id, "apic-id", errp);
@@ -884,14 +885,18 @@ static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
     if (error_is_set(errp)) {
         if (cpu != NULL) {
             object_unref(OBJECT(cpu));
+            cpu = NULL;
         }
     }
+    return cpu;
 }
 
 void pc_cpus_init(const char *cpu_model)
 {
     int i;
+    X86CPU *cpu = NULL;
     Error *error = NULL;
+    SysBusDevice *ib;
 
     /* init CPUs */
     if (cpu_model == NULL) {
@@ -902,14 +907,23 @@ void pc_cpus_init(const char *cpu_model)
 #endif
     }
 
+    ib = SYS_BUS_DEVICE(object_resolve_path_type("icc-bridge",
+                                                 TYPE_ICC_BRIDGE, NULL));
+
     for (i = 0; i < smp_cpus; i++) {
-        pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
+        cpu = pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
         if (error) {
             fprintf(stderr, "%s\n", error_get_pretty(error));
             error_free(error);
             exit(1);
         }
     }
+
+    /* map APIC MMIO area if CPU has APIC */
+    if (cpu && cpu->env.apic_state) {
+        /* XXX: what if the base changes? */
+        sysbus_mmio_map_overlap(ib, 0, APIC_DEFAULT_ADDRESS, 0x1000);
+    }
 }
 
 void pc_acpi_init(const char *default_dsdt)
hw/intc/apic_common.c +12 -5
index e0ae07a..39396f1 100644
@@ -21,6 +21,8 @@
 #include "hw/i386/apic_internal.h"
 #include "trace.h"
 #include "sysemu/kvm.h"
+#include "hw/qdev.h"
+#include "hw/sysbus.h"
 
 static int apic_irq_delivered;
 bool apic_report_tpr_access;
@@ -282,12 +284,14 @@ static int apic_load_old(QEMUFile *f, void *opaque, int version_id)
     return 0;
 }
 
-static int apic_init_common(SysBusDevice *dev)
+static int apic_init_common(ICCDevice *dev)
 {
     APICCommonState *s = APIC_COMMON(dev);
+    DeviceState *d = DEVICE(dev);
     APICCommonClass *info;
     static DeviceState *vapic;
     static int apic_no;
+    static bool mmio_registered;
 
     if (apic_no >= MAX_APICS) {
         return -1;
@@ -296,8 +300,11 @@ static int apic_init_common(SysBusDevice *dev)
 
     info = APIC_COMMON_GET_CLASS(s);
     info->init(s);
-
-    sysbus_init_mmio(dev, &s->io_memory);
+    if (!mmio_registered) {
+        MemoryRegion *as = ICC_BUS(d->parent_bus)->apic_address_space;
+        memory_region_add_subregion(as, 0, &s->io_memory);
+        mmio_registered = true;
+    }
 
     /* Note: We need at least 1M to map the VAPIC option ROM */
     if (!vapic && s->vapic_control & VAPIC_ENABLE_MASK &&
@@ -375,7 +382,7 @@ static Property apic_properties_common[] = {
 
 static void apic_common_class_init(ObjectClass *klass, void *data)
 {
-    SysBusDeviceClass *sc = SYS_BUS_DEVICE_CLASS(klass);
+    ICCDeviceClass *sc = ICC_DEVICE_CLASS(klass);
     DeviceClass *dc = DEVICE_CLASS(klass);
 
     dc->vmsd = &vmstate_apic_common;
@@ -387,7 +394,7 @@ static void apic_common_class_init(ObjectClass *klass, void *data)
 
 static const TypeInfo apic_common_type = {
     .name = TYPE_APIC_COMMON,
-    .parent = TYPE_SYS_BUS_DEVICE,
+    .parent = TYPE_ICC_DEVICE,
     .instance_size = sizeof(APICCommonState),
     .class_size = sizeof(APICCommonClass),
     .class_init = apic_common_class_init,
include/hw/i386/apic_internal.h +3 -3
index aac6290..05acf4b 100644
@@ -21,7 +21,7 @@
 #define QEMU_APIC_INTERNAL_H
 
 #include "exec/memory.h"
-#include "hw/sysbus.h"
+#include "hw/i386/icc_bus.h"
 #include "qemu/timer.h"
 
 /* APIC Local Vector Table */
@@ -78,7 +78,7 @@ typedef struct APICCommonState APICCommonState;
 
 typedef struct APICCommonClass
 {
-    SysBusDeviceClass parent_class;
+    ICCDeviceClass parent_class;
 
     void (*init)(APICCommonState *s);
     void (*set_base)(APICCommonState *s, uint64_t val);
@@ -92,7 +92,7 @@ typedef struct APICCommonClass
 } APICCommonClass;
 
 struct APICCommonState {
-    SysBusDevice busdev;
+    ICCDevice busdev;
 
     MemoryRegion io_memory;
     X86CPU *cpu;
include/hw/i386/icc_bus.h +2 -0
index ae1866c..2c8f78f 100644
@@ -19,6 +19,7 @@
 #ifndef ICC_BUS_H
 #define ICC_BUS_H
 
+#include "exec/memory.h"
 #include "hw/qdev-core.h"
 
 #define TYPE_ICC_BUS "icc-bus"
@@ -26,6 +27,7 @@
 #ifndef CONFIG_USER_ONLY
 typedef struct ICCBus {
     BusState qbus;
+    MemoryRegion *apic_address_space;
 } ICCBus;
 #define ICC_BUS(obj) OBJECT_CHECK(ICCBus, (obj), TYPE_ICC_BUS)
 
target-i386/cpu.c +3 -13
index 2bb38d6..f77c3bb 100644
@@ -41,10 +41,10 @@
 #endif
 
 #include "sysemu/sysemu.h"
+#include "hw/qdev-properties.h"
 #include "hw/i386/icc_bus.h"
 #ifndef CONFIG_USER_ONLY
 #include "hw/xen/xen.h"
-#include "hw/sysbus.h"
 #include "hw/i386/apic_internal.h"
 #endif
 
@@ -2111,6 +2111,7 @@ static void mce_init(X86CPU *cpu)
 static void x86_cpu_apic_create(X86CPU *cpu, Error **errp)
 {
     CPUX86State *env = &cpu->env;
+    DeviceState *dev = DEVICE(cpu);
     APICCommonState *apic;
     const char *apic_type = "apic";
 
@@ -2120,7 +2121,7 @@ static void x86_cpu_apic_create(X86CPU *cpu, Error **errp)
         apic_type = "xen-apic";
     }
 
-    env->apic_state = qdev_try_create(NULL, apic_type);
+    env->apic_state = qdev_try_create(dev->parent_bus, apic_type);
     if (env->apic_state == NULL) {
         error_setg(errp, "APIC device '%s' could not be created", apic_type);
         return;
@@ -2137,7 +2138,6 @@ static void x86_cpu_apic_create(X86CPU *cpu, Error **errp)
 static void x86_cpu_apic_realize(X86CPU *cpu, Error **errp)
 {
     CPUX86State *env = &cpu->env;
-    static int apic_mapped;
 
     if (env->apic_state == NULL) {
         return;
@@ -2148,16 +2148,6 @@ static void x86_cpu_apic_realize(X86CPU *cpu, Error **errp)
                    object_get_typename(OBJECT(env->apic_state)));
         return;
     }
-
-    /* XXX: mapping more APICs at the same memory location */
-    if (apic_mapped == 0) {
-        /* NOTE: the APIC is directly connected to the CPU - it is not
-           on the global memory bus. */
-        /* XXX: what if the base changes? */
-        sysbus_mmio_map_overlap(SYS_BUS_DEVICE(env->apic_state), 0,
-                                APIC_DEFAULT_ADDRESS, 0x1000);
-        apic_mapped = 1;
-    }
 }
 #else
 static void x86_cpu_apic_realize(X86CPU *cpu, Error **errp)
Signature · 1 line
1.8.2
#71 lore raw

PATCH v3 16/19 target-i386: move IOAPIC to ICC bus

In reply to Igor Mammedov

* inherit IOAPICs from ICCDevice and attach them to ICC bus
* map IOAPIC's mmio at board level via indirect icc-bridge mmio region that provided address space to IOAPIC via icc-bus

---
* do not create IOAPIC in icc-bridge and do not make it QOM child of icc-bridge, just attach it to icc-bus.

---
 hw/cpu/icc_bus.c                  |  6 ++++++
 hw/i386/kvm/ioapic.c              |  2 +-
 hw/i386/pc.c                      | 10 ++++++----
 hw/intc/ioapic_common.c           | 15 +++++++++++----
 include/exec/memory.h             | 10 ++++++++++
 include/hw/i386/icc_bus.h         |  1 +
 include/hw/i386/ioapic_internal.h |  6 +++---
 memory.c                          | 11 +++++++++++
 8 files changed, 49 insertions(+), 12 deletions(-)
hw/cpu/icc_bus.c +6 -0
index c499f09..ab9623d 100644
@@ -60,6 +60,7 @@ static const TypeInfo icc_device_info = {
 typedef struct ICCBridgeState {
     SysBusDevice busdev;
     MemoryRegion apic_container;
+    MemoryRegion ioapic_container;
 } ICCBridgeState;
 #define ICC_BRIGDE(obj) OBJECT_CHECK(ICCBridgeState, (obj), TYPE_ICC_BRIDGE)
 
@@ -79,6 +80,11 @@ static void icc_bridge_initfn(Object *obj)
                        APIC_SPACE_SIZE);
     sysbus_init_mmio(sb, &s->apic_container);
     ibus->apic_address_space = &s->apic_container;
+
+    /* must be second registered region, board maps it by 1 index */
+    memory_region_init(&s->ioapic_container, "icc-ioapic-container", 0x1000);
+    sysbus_init_mmio(sb, &s->ioapic_container);
+    ibus->ioapic_address_space = &s->ioapic_container;
 }
 
 static const TypeInfo icc_bridge_info = {
hw/i386/kvm/ioapic.c +1 -1
index a3bd519..b80d41a 100644
@@ -96,7 +96,7 @@ static void kvm_ioapic_put(IOAPICCommonState *s)
 
     kioapic->id = s->id;
     kioapic->ioregsel = s->ioregsel;
-    kioapic->base_address = s->busdev.mmio[0].addr;
+    kioapic->base_address = memory_region_get_address(&s->io_memory);
     kioapic->irr = s->irr;
     for (i = 0; i < IOAPIC_NUM_PINS; i++) {
         kioapic->redirtbl[i].bits = s->ioredtbl[i];
hw/i386/pc.c +6 -4
index 1c7b3c2..6d5e164 100644
@@ -1197,20 +1197,22 @@ void ioapic_init_gsi(GSIState *gsi_state, const char *parent_name)
     DeviceState *dev;
     SysBusDevice *d;
     unsigned int i;
+    BusState *b = BUS(object_resolve_path_type("icc-bus", TYPE_ICC_BUS, NULL));
 
     if (kvm_irqchip_in_kernel()) {
-        dev = qdev_create(NULL, "kvm-ioapic");
+        dev = qdev_create(b, "kvm-ioapic");
     } else {
-        dev = qdev_create(NULL, "ioapic");
+        dev = qdev_create(b, "ioapic");
     }
     if (parent_name) {
         object_property_add_child(object_resolve_path(parent_name, NULL),
                                   "ioapic", OBJECT(dev), NULL);
     }
     qdev_init_nofail(dev);
-    d = SYS_BUS_DEVICE(dev);
-    sysbus_mmio_map(d, 0, IO_APIC_DEFAULT_ADDRESS);
 
+    d = SYS_BUS_DEVICE(object_resolve_path_type("icc-bridge", TYPE_ICC_BRIDGE,
+                                                NULL));
+    sysbus_mmio_map(d, 1, IO_APIC_DEFAULT_ADDRESS);
     for (i = 0; i < IOAPIC_NUM_PINS; i++) {
         gsi_state->ioapic_irq[i] = qdev_get_gpio_in(dev, i);
     }
hw/intc/ioapic_common.c +11 -4
index 5c5bb3c..72c48a8 100644
@@ -57,11 +57,13 @@ static int ioapic_dispatch_post_load(void *opaque, int version_id)
     return 0;
 }
 
-static int ioapic_init_common(SysBusDevice *dev)
+static int ioapic_init_common(ICCDevice *dev)
 {
     IOAPICCommonState *s = IOAPIC_COMMON(dev);
+    DeviceState *d = DEVICE(dev);
     IOAPICCommonClass *info;
     static int ioapic_no;
+    static bool mmio_registered;
 
     if (ioapic_no >= MAX_IOAPICS) {
         return -1;
@@ -70,7 +72,12 @@ static int ioapic_init_common(SysBusDevice *dev)
     info = IOAPIC_COMMON_GET_CLASS(s);
     info->init(s, ioapic_no);
 
-    sysbus_init_mmio(&s->busdev, &s->io_memory);
+    if (!mmio_registered) {
+        MemoryRegion *as = ICC_BUS(d->parent_bus)->ioapic_address_space;
+        memory_region_add_subregion(as, 0, &s->io_memory);
+        mmio_registered = true;
+    }
+
     ioapic_no++;
 
     return 0;
@@ -95,7 +102,7 @@ static const VMStateDescription vmstate_ioapic_common = {
 
 static void ioapic_common_class_init(ObjectClass *klass, void *data)
 {
-    SysBusDeviceClass *sc = SYS_BUS_DEVICE_CLASS(klass);
+    ICCDeviceClass *sc = ICC_DEVICE_CLASS(klass);
     DeviceClass *dc = DEVICE_CLASS(klass);
 
     sc->init = ioapic_init_common;
@@ -105,7 +112,7 @@ static void ioapic_common_class_init(ObjectClass *klass, void *data)
 
 static const TypeInfo ioapic_common_type = {
     .name = TYPE_IOAPIC_COMMON,
-    .parent = TYPE_SYS_BUS_DEVICE,
+    .parent = TYPE_ICC_DEVICE,
     .instance_size = sizeof(IOAPICCommonState),
     .class_size = sizeof(IOAPICCommonClass),
     .class_init = ioapic_common_class_init,
include/exec/memory.h +10 -0
index 2322732..826e401 100644
@@ -711,6 +711,16 @@ void memory_region_set_enabled(MemoryRegion *mr, bool enabled);
 void memory_region_set_address(MemoryRegion *mr, hwaddr addr);
 
 /*
+ * memory_region_get_address: get current the address of a region
+ *
+ * Returns the absolute address of a region.
+ * May be used on regions that are currently part of a memory hierarchy.
+ *
+ * @mr: the region being queried
+ */
+hwaddr memory_region_get_address(MemoryRegion *mr);
+
+/*
  * memory_region_set_alias_offset: dynamically update a memory alias's offset
  *
  * Dynamically updates the offset into the target region that an alias points
include/hw/i386/icc_bus.h +1 -0
index 2c8f78f..69a0278 100644
@@ -28,6 +28,7 @@
 typedef struct ICCBus {
     BusState qbus;
     MemoryRegion *apic_address_space;
+    MemoryRegion *ioapic_address_space;
 } ICCBus;
 #define ICC_BUS(obj) OBJECT_CHECK(ICCBus, (obj), TYPE_ICC_BUS)
 
include/hw/i386/ioapic_internal.h +3 -3
index 25576c8..4768c5b 100644
@@ -24,7 +24,7 @@
 
 #include "hw/hw.h"
 #include "exec/memory.h"
-#include "hw/sysbus.h"
+#include "hw/i386/icc_bus.h"
 
 #define MAX_IOAPICS                     1
 
@@ -82,14 +82,14 @@ typedef struct IOAPICCommonState IOAPICCommonState;
      OBJECT_GET_CLASS(IOAPICCommonClass, (obj), TYPE_IOAPIC_COMMON)
 
 typedef struct IOAPICCommonClass {
-    SysBusDeviceClass parent_class;
+    ICCDeviceClass parent_class;
     void (*init)(IOAPICCommonState *s, int instance_no);
     void (*pre_save)(IOAPICCommonState *s);
     void (*post_load)(IOAPICCommonState *s);
 } IOAPICCommonClass;
 
 struct IOAPICCommonState {
-    SysBusDevice busdev;
+    ICCDevice busdev;
     MemoryRegion io_memory;
     uint8_t id;
     uint8_t ioregsel;
memory.c +11 -0
index 75ca281..0651050 100644
@@ -1413,6 +1413,17 @@ void memory_region_set_address(MemoryRegion *mr, hwaddr addr)
     memory_region_transaction_commit();
 }
 
+hwaddr memory_region_get_address(MemoryRegion *mr)
+{
+    hwaddr addr = mr->addr;
+
+    while (mr->parent) {
+        mr = mr->parent;
+        addr += mr->addr;
+    }
+    return addr;
+}
+
 void memory_region_set_alias_offset(MemoryRegion *mr, hwaddr offset)
 {
     assert(mr->alias);
Signature · 1 line
1.8.2
#72 lore raw

PATCH v3 17/19 qdev: set device's parent before calling realize() down inheritance chain.

In reply to Igor Mammedov

Currently device_set_realized() sets parent only after device was realized, But qdev_device_add() sets it before device is realized.
Make behavior consistent and alter device_set_realized() to behave like qdev_device_add().

It will allow to set link<> properties in realize() method in classes inherited from DEVICE.

---
* Usage examples
- conversion to CPU back-link property in APIC
- pre-allocate link<CPU[n]> at board level, and populate them at realize time when CPUs are created.

---
 hw/core/qdev.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
hw/core/qdev.c +4 -4
index e2bb37d..b9bd6f5 100644
@@ -671,10 +671,6 @@ static void device_set_realized(Object *obj, bool value, Error **err)
     Error *local_err = NULL;
 
     if (value && !dev->realized) {
-        if (dc->realize) {
-            dc->realize(dev, &local_err);
-        }
-
         if (!obj->parent && local_err == NULL) {
             static int unattached_count;
             gchar *name = g_strdup_printf("device[%d]", unattached_count++);
@@ -685,6 +681,10 @@ static void device_set_realized(Object *obj, bool value, Error **err)
             g_free(name);
         }
 
+        if (dc->realize) {
+            dc->realize(dev, &local_err);
+        }
+
         if (qdev_get_vmsd(dev) && local_err == NULL) {
             vmstate_register_with_alias_id(dev, -1, qdev_get_vmsd(dev), dev,
                                            dev->instance_id_alias,
Signature · 1 line
1.8.2
#73 lore raw

Re: [Qemu-devel] [PATCH 17/19] qdev: set device's parent before calling realize() down inheritance chain.

Am 11.04.2013 16:51, schrieb Igor Mammedov:

Quoted 10 lines

Currently device_set_realized() sets parent only after device was realized, But qdev_device_add() sets it before device is realized.
Make behavior consistent and alter device_set_realized() to behave like qdev_device_add().

It will allow to set link<> properties in realize() method in classes inherited from DEVICE.

Quoted 8 lines

---
* Usage examples
- conversion to CPU back-link property in APIC
- pre-allocate link<CPU[n]> at board level, and populate them at realize time when CPUs are created.

---
 hw/core/qdev.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

Thanks, applied to qom-cpu:
https://github.com/afaerber/qemu-cpu/commits/qom-cpu

Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#74 lore raw

PATCH v3 18/19 target-i386: expose all possible CPUs as /machine/icc-bridge/cpu[0..N] links

In reply to Igor Mammedov

... and leave links for not present CPUs empty.

It will allow users to query for possible APIC IDs and use them with cpu-add QMP command.

---
v2:
* s/get_firmware_id/get_arch_id/ due to rebase
* rename cpu_add_notifier to cpu_added_notifier & icc_bridge_cpu_add_req -> icc_bridge_cpued_add_req

---
 hw/cpu/icc_bus.c          | 46 ++++++++++++++++++++++++++++++++++++++++++++++
 hw/i386/pc.c              |  9 +++++++--
 include/hw/i386/icc_bus.h |  2 ++
 3 files changed, 55 insertions(+), 2 deletions(-)
hw/cpu/icc_bus.c +46 -0
index ab9623d..5c0b9d4 100644
@@ -18,6 +18,7 @@
  */
 #include "hw/i386/icc_bus.h"
 #include "hw/sysbus.h"
+#include "sysemu/sysemu.h"
 
 static void icc_bus_initfn(Object *obj)
 {
@@ -61,15 +62,39 @@ typedef struct ICCBridgeState {
     SysBusDevice busdev;
     MemoryRegion apic_container;
     MemoryRegion ioapic_container;
+    Notifier cpu_added_notifier;
+    Object **links;
 } ICCBridgeState;
 #define ICC_BRIGDE(obj) OBJECT_CHECK(ICCBridgeState, (obj), TYPE_ICC_BRIDGE)
 
 
+void icc_bridge_set_cpu_link(Object *bridge, Object *cpu_obj)
+{
+    gchar *name;
+    Error *error = NULL;
+    CPUState *cpu = CPU(cpu_obj);
+    int64_t id = CPU_GET_CLASS(cpu)->get_arch_id(cpu);
+
+    name = g_strdup_printf("cpu[%" PRIu32 "]", x86_cpu_apic_id_from_index(id));
+    object_property_set_link(bridge, cpu_obj, name, &error);
+    g_free(name);
+
+    g_assert(error == NULL);
+}
+
+static void icc_bridge_cpu_added_req(Notifier *n, void *opaque)
+{
+    ICCBridgeState *s = container_of(n, ICCBridgeState, cpu_added_notifier);
+
+    icc_bridge_set_cpu_link(OBJECT(s), OBJECT(opaque));
+}
+
 static void icc_bridge_initfn(Object *obj)
 {
     ICCBridgeState *s = ICC_BRIGDE(obj);
     SysBusDevice *sb = SYS_BUS_DEVICE(obj);
     ICCBus *ibus;
+    int i;
 
     ibus = ICC_BUS(qbus_create(TYPE_ICC_BUS, DEVICE(obj), "icc-bus"));
 
@@ -85,12 +110,33 @@ static void icc_bridge_initfn(Object *obj)
     memory_region_init(&s->ioapic_container, "icc-ioapic-container", 0x1000);
     sysbus_init_mmio(sb, &s->ioapic_container);
     ibus->ioapic_address_space = &s->ioapic_container;
+
+    s->links = g_malloc0(sizeof(Object *) * max_cpus);
+    for (i = 0; i < max_cpus; i++) {
+        gchar *cpu_name;
+
+        cpu_name = g_strdup_printf("cpu[%" PRIu32 "]",
+                                   x86_cpu_apic_id_from_index(i));
+        object_property_add_link(obj, cpu_name, TYPE_CPU, &s->links[i], NULL);
+        g_free(cpu_name);
+    }
+
+    s->cpu_added_notifier.notify = icc_bridge_cpu_added_req;
+    qemu_register_cpu_added_notifier(&s->cpu_added_notifier);
+}
+
+static void icc_bridge_fini(Object *obj)
+{
+    ICCBridgeState *s = ICC_BRIGDE(obj);
+
+    g_free(s->links);
 }
 
 static const TypeInfo icc_bridge_info = {
     .name  = "icc-bridge",
     .parent = TYPE_SYS_BUS_DEVICE,
     .instance_init  = icc_bridge_initfn,
+    .instance_finalize  = icc_bridge_fini,
     .instance_size  = sizeof(ICCBridgeState),
 };
 
hw/i386/pc.c +7 -2
index 6d5e164..ada235c 100644
@@ -870,7 +870,8 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level)
     }
 }
 
-static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
+static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id,
+                          SysBusDevice *icc_bridge, Error **errp)
 {
     X86CPU *cpu;
 
@@ -882,6 +883,10 @@ static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
     object_property_set_int(OBJECT(cpu), apic_id, "apic-id", errp);
     object_property_set_bool(OBJECT(cpu), true, "realized", errp);
 
+    if (icc_bridge != NULL) {
+        icc_bridge_set_cpu_link(OBJECT(icc_bridge), OBJECT(cpu));
+    }
+
     if (error_is_set(errp)) {
         if (cpu != NULL) {
             object_unref(OBJECT(cpu));
@@ -911,7 +916,7 @@ void pc_cpus_init(const char *cpu_model)
                                                  TYPE_ICC_BRIDGE, NULL));
 
     for (i = 0; i < smp_cpus; i++) {
-        cpu = pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
+        cpu = pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), ib, &error);
         if (error) {
             fprintf(stderr, "%s\n", error_get_pretty(error));
             error_free(error);
include/hw/i386/icc_bus.h +2 -0
index 69a0278..bc31cd9 100644
@@ -49,5 +49,7 @@ typedef struct ICCDeviceClass {
 
 #define TYPE_ICC_BRIDGE "icc-bridge"
 
+void icc_bridge_set_cpu_link(Object *bridge, Object *cpu);
+
 #endif /* CONFIG_USER_ONLY */
 #endif
Signature · 1 line
1.8.2
#75 lore raw

Re: [Qemu-devel] [PATCH 18/19] target-i386: expose all possible CPUs as /machine/icc-bridge/cpu[0..N] links

Quoted 6 lines · Igor Mammedov

... and leave links for not present CPUs empty.

It will allow users to query for possible APIC IDs and use them with cpu-add QMP command.

I don't see anything wrong with having icc-bridge links as well, but I would really like to have a target-independent namespace with links, that could be used to query for the available/valid CPU IDs for cpu-add commands instead of icc-bridge. The IDs on that namespace could be considered completely opaque.

Quoted 146 lines

---
v2:
* s/get_firmware_id/get_arch_id/ due to rebase
* rename cpu_add_notifier to cpu_added_notifier & icc_bridge_cpu_add_req -> icc_bridge_cpued_add_req

---
 hw/cpu/icc_bus.c          | 46 ++++++++++++++++++++++++++++++++++++++++++++++
 hw/i386/pc.c              |  9 +++++++--
 include/hw/i386/icc_bus.h |  2 ++
 3 files changed, 55 insertions(+), 2 deletions(-)
hw/cpu/icc_bus.c +46 -0
index ab9623d..5c0b9d4 100644
@@ -18,6 +18,7 @@
  */
 #include "hw/i386/icc_bus.h"
 #include "hw/sysbus.h"
+#include "sysemu/sysemu.h"
 
 static void icc_bus_initfn(Object *obj)
 {
@@ -61,15 +62,39 @@ typedef struct ICCBridgeState {
     SysBusDevice busdev;
     MemoryRegion apic_container;
     MemoryRegion ioapic_container;
+    Notifier cpu_added_notifier;
+    Object **links;
 } ICCBridgeState;
 #define ICC_BRIGDE(obj) OBJECT_CHECK(ICCBridgeState, (obj), TYPE_ICC_BRIDGE)
 
 
+void icc_bridge_set_cpu_link(Object *bridge, Object *cpu_obj)
+{
+    gchar *name;
+    Error *error = NULL;
+    CPUState *cpu = CPU(cpu_obj);
+    int64_t id = CPU_GET_CLASS(cpu)->get_arch_id(cpu);
+
+    name = g_strdup_printf("cpu[%" PRIu32 "]", x86_cpu_apic_id_from_index(id));
+    object_property_set_link(bridge, cpu_obj, name, &error);
+    g_free(name);
+
+    g_assert(error == NULL);
+}
+
+static void icc_bridge_cpu_added_req(Notifier *n, void *opaque)
+{
+    ICCBridgeState *s = container_of(n, ICCBridgeState, cpu_added_notifier);
+
+    icc_bridge_set_cpu_link(OBJECT(s), OBJECT(opaque));
+}
+
 static void icc_bridge_initfn(Object *obj)
 {
     ICCBridgeState *s = ICC_BRIGDE(obj);
     SysBusDevice *sb = SYS_BUS_DEVICE(obj);
     ICCBus *ibus;
+    int i;
 
     ibus = ICC_BUS(qbus_create(TYPE_ICC_BUS, DEVICE(obj), "icc-bus"));
 
@@ -85,12 +110,33 @@ static void icc_bridge_initfn(Object *obj)
     memory_region_init(&s->ioapic_container, "icc-ioapic-container", 0x1000);
     sysbus_init_mmio(sb, &s->ioapic_container);
     ibus->ioapic_address_space = &s->ioapic_container;
+
+    s->links = g_malloc0(sizeof(Object *) * max_cpus);
+    for (i = 0; i < max_cpus; i++) {
+        gchar *cpu_name;
+
+        cpu_name = g_strdup_printf("cpu[%" PRIu32 "]",
+                                   x86_cpu_apic_id_from_index(i));
+        object_property_add_link(obj, cpu_name, TYPE_CPU, &s->links[i], NULL);
+        g_free(cpu_name);
+    }
+
+    s->cpu_added_notifier.notify = icc_bridge_cpu_added_req;
+    qemu_register_cpu_added_notifier(&s->cpu_added_notifier);
+}
+
+static void icc_bridge_fini(Object *obj)
+{
+    ICCBridgeState *s = ICC_BRIGDE(obj);
+
+    g_free(s->links);
 }
 
 static const TypeInfo icc_bridge_info = {
     .name  = "icc-bridge",
     .parent = TYPE_SYS_BUS_DEVICE,
     .instance_init  = icc_bridge_initfn,
+    .instance_finalize  = icc_bridge_fini,
     .instance_size  = sizeof(ICCBridgeState),
 };
 
hw/i386/pc.c +7 -2
index 6d5e164..ada235c 100644
@@ -870,7 +870,8 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level)
     }
 }
 
-static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
+static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id,
+                          SysBusDevice *icc_bridge, Error **errp)
 {
     X86CPU *cpu;
 
@@ -882,6 +883,10 @@ static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
     object_property_set_int(OBJECT(cpu), apic_id, "apic-id", errp);
     object_property_set_bool(OBJECT(cpu), true, "realized", errp);
 
+    if (icc_bridge != NULL) {
+        icc_bridge_set_cpu_link(OBJECT(icc_bridge), OBJECT(cpu));
+    }
+
     if (error_is_set(errp)) {
         if (cpu != NULL) {
             object_unref(OBJECT(cpu));
@@ -911,7 +916,7 @@ void pc_cpus_init(const char *cpu_model)
                                                  TYPE_ICC_BRIDGE, NULL));
 
     for (i = 0; i < smp_cpus; i++) {
-        cpu = pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
+        cpu = pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), ib, &error);
         if (error) {
             fprintf(stderr, "%s\n", error_get_pretty(error));
             error_free(error);
include/hw/i386/icc_bus.h +2 -0
index 69a0278..bc31cd9 100644
@@ -49,5 +49,7 @@ typedef struct ICCDeviceClass {
 
 #define TYPE_ICC_BRIDGE "icc-bridge"
 
+void icc_bridge_set_cpu_link(Object *bridge, Object *cpu);
+
 #endif /* CONFIG_USER_ONLY */
 #endif
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
#76 lore raw

Re: [Qemu-devel] [PATCH 18/19] target-i386: expose all possible CPUs as /machine/icc-bridge/cpu[0..N] links

Quoted 6 lines · On Thu, 11 Apr 2013 14:19:37 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 5 lines · Igor Mammedov

... and leave links for not present CPUs empty.

It will allow users to query for possible APIC IDs and use them with cpu-add QMP command.

Quoted 7 lines
Quoted 1 line

I don't see anything wrong with having icc-bridge links as well, but I would really like to have a target-independent namespace with links, that could be used to query for the available/valid CPU IDs for cpu-add commands instead of icc-bridge. The IDs on that namespace could be considered completely opaque.

Considering that -numa in present state is not compatible with cpu-add and that all CPU ID in this case are are sequence [0..maxcpus-1], this patch could be dropped without any harm. libvirt could just use numbers from this sequence like it's doing with current cpu_set without any ID discovery.

So, I've postponed target independent until we have -numa reworked, then we could have /machine/node/socket/cpu containers with links. The problem that needs to be solved, is the links storage ownership. Who should allocate and own it? If machine was QOM object already, I'd go with machine but it's not yet.

Quoted 150 lines
Quoted 147 lines

---
v2:
* s/get_firmware_id/get_arch_id/ due to rebase
* rename cpu_add_notifier to cpu_added_notifier & icc_bridge_cpu_add_req -> icc_bridge_cpued_add_req

---
 hw/cpu/icc_bus.c          | 46 ++++++++++++++++++++++++++++++++++++++++++++++
 hw/i386/pc.c              |  9 +++++++--
 include/hw/i386/icc_bus.h |  2 ++
 3 files changed, 55 insertions(+), 2 deletions(-)
hw/cpu/icc_bus.c +46 -0
index ab9623d..5c0b9d4 100644
@@ -18,6 +18,7 @@
  */
 #include "hw/i386/icc_bus.h"
 #include "hw/sysbus.h"
+#include "sysemu/sysemu.h"
 
 static void icc_bus_initfn(Object *obj)
 {
@@ -61,15 +62,39 @@ typedef struct ICCBridgeState {
     SysBusDevice busdev;
     MemoryRegion apic_container;
     MemoryRegion ioapic_container;
+    Notifier cpu_added_notifier;
+    Object **links;
 } ICCBridgeState;
 #define ICC_BRIGDE(obj) OBJECT_CHECK(ICCBridgeState, (obj), TYPE_ICC_BRIDGE)
 
 
+void icc_bridge_set_cpu_link(Object *bridge, Object *cpu_obj)
+{
+    gchar *name;
+    Error *error = NULL;
+    CPUState *cpu = CPU(cpu_obj);
+    int64_t id = CPU_GET_CLASS(cpu)->get_arch_id(cpu);
+
+    name = g_strdup_printf("cpu[%" PRIu32 "]", x86_cpu_apic_id_from_index(id));
+    object_property_set_link(bridge, cpu_obj, name, &error);
+    g_free(name);
+
+    g_assert(error == NULL);
+}
+
+static void icc_bridge_cpu_added_req(Notifier *n, void *opaque)
+{
+    ICCBridgeState *s = container_of(n, ICCBridgeState, cpu_added_notifier);
+
+    icc_bridge_set_cpu_link(OBJECT(s), OBJECT(opaque));
+}
+
 static void icc_bridge_initfn(Object *obj)
 {
     ICCBridgeState *s = ICC_BRIGDE(obj);
     SysBusDevice *sb = SYS_BUS_DEVICE(obj);
     ICCBus *ibus;
+    int i;
 
     ibus = ICC_BUS(qbus_create(TYPE_ICC_BUS, DEVICE(obj), "icc-bus"));
 
@@ -85,12 +110,33 @@ static void icc_bridge_initfn(Object *obj)
     memory_region_init(&s->ioapic_container, "icc-ioapic-container", 0x1000);
     sysbus_init_mmio(sb, &s->ioapic_container);
     ibus->ioapic_address_space = &s->ioapic_container;
+
+    s->links = g_malloc0(sizeof(Object *) * max_cpus);
+    for (i = 0; i < max_cpus; i++) {
+        gchar *cpu_name;
+
+        cpu_name = g_strdup_printf("cpu[%" PRIu32 "]",
+                                   x86_cpu_apic_id_from_index(i));
+        object_property_add_link(obj, cpu_name, TYPE_CPU, &s->links[i], NULL);
+        g_free(cpu_name);
+    }
+
+    s->cpu_added_notifier.notify = icc_bridge_cpu_added_req;
+    qemu_register_cpu_added_notifier(&s->cpu_added_notifier);
+}
+
+static void icc_bridge_fini(Object *obj)
+{
+    ICCBridgeState *s = ICC_BRIGDE(obj);
+
+    g_free(s->links);
 }
 
 static const TypeInfo icc_bridge_info = {
     .name  = "icc-bridge",
     .parent = TYPE_SYS_BUS_DEVICE,
     .instance_init  = icc_bridge_initfn,
+    .instance_finalize  = icc_bridge_fini,
     .instance_size  = sizeof(ICCBridgeState),
 };
 
hw/i386/pc.c +7 -2
index 6d5e164..ada235c 100644
@@ -870,7 +870,8 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level)
     }
 }
 
-static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
+static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id,
+                          SysBusDevice *icc_bridge, Error **errp)
 {
     X86CPU *cpu;
 
@@ -882,6 +883,10 @@ static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
     object_property_set_int(OBJECT(cpu), apic_id, "apic-id", errp);
     object_property_set_bool(OBJECT(cpu), true, "realized", errp);
 
+    if (icc_bridge != NULL) {
+        icc_bridge_set_cpu_link(OBJECT(icc_bridge), OBJECT(cpu));
+    }
+
     if (error_is_set(errp)) {
         if (cpu != NULL) {
             object_unref(OBJECT(cpu));
@@ -911,7 +916,7 @@ void pc_cpus_init(const char *cpu_model)
                                                  TYPE_ICC_BRIDGE, NULL));
 
     for (i = 0; i < smp_cpus; i++) {
-        cpu = pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
+        cpu = pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), ib, &error);
         if (error) {
             fprintf(stderr, "%s\n", error_get_pretty(error));
             error_free(error);
include/hw/i386/icc_bus.h +2 -0
index 69a0278..bc31cd9 100644
@@ -49,5 +49,7 @@ typedef struct ICCDeviceClass {
 
 #define TYPE_ICC_BRIDGE "icc-bridge"
 
+void icc_bridge_set_cpu_link(Object *bridge, Object *cpu);
+
 #endif /* CONFIG_USER_ONLY */
 #endif
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
Signature · 2 lines
Regards,
  Igor
#77 lore raw

Re: [Qemu-devel] [PATCH 18/19] target-i386: expose all possible CPUs as /machine/icc-bridge/cpu[0..N] links

Quoted 17 lines · Igor Mammedov
Quoted 6 lines · On Thu, 11 Apr 2013 14:19:37 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 5 lines · Igor Mammedov

... and leave links for not present CPUs empty.

It will allow users to query for possible APIC IDs and use them with cpu-add QMP command.

Quoted 7 lines
Quoted 1 line

I don't see anything wrong with having icc-bridge links as well, but I would really like to have a target-independent namespace with links, that could be used to query for the available/valid CPU IDs for cpu-add commands instead of icc-bridge. The IDs on that namespace could be considered completely opaque.

Quoted 5 lines

Considering that -numa in present state is not compatible with cpu-add and that all CPU ID in this case are are sequence [0..maxcpus-1], this patch could be dropped without any harm. libvirt could just use numbers from this sequence like it's doing with current cpu_set without any ID discovery.

But it's not -numa that makes APIC ID probing necessary, it's non-power-of-2 core/thread counts on -smp (that make APIC IDs not match CPU indexes).

"Don't use CPU hotplug with -numa" is easy to be understood by users and by libvirt, but "don't use CPU hotplug with non-power-of-2 cores/threads counts" is harder to explain.

Quoted 6 lines

So, I've postponed target independent until we have -numa reworked, then we could have /machine/node/socket/cpu containers with links. The problem that needs to be solved, is the links storage ownership. Who should allocate and own it? If machine was QOM object already, I'd go with machine but it's not yet.

If we use CPU index as argument to cpu-add, we don't need to handle all those problems right now, we don't need to expose an APIC ID discovery interface, we make it work even with non-power-of-2 cores/threads counts, and we make it work with -numa.

So, my big question is: why are we trying so hard to avoid using CPU indexes as argument to cpu-add, if it's so much easier, and it is an obvious solution that makes the interface target-independent without any extra effort?

Quoted 156 lines
Quoted 151 lines
Quoted 148 lines

---
v2:
* s/get_firmware_id/get_arch_id/ due to rebase
* rename cpu_add_notifier to cpu_added_notifier & icc_bridge_cpu_add_req -> icc_bridge_cpued_add_req

---
 hw/cpu/icc_bus.c          | 46 ++++++++++++++++++++++++++++++++++++++++++++++
 hw/i386/pc.c              |  9 +++++++--
 include/hw/i386/icc_bus.h |  2 ++
 3 files changed, 55 insertions(+), 2 deletions(-)
hw/cpu/icc_bus.c +46 -0
index ab9623d..5c0b9d4 100644
@@ -18,6 +18,7 @@
  */
 #include "hw/i386/icc_bus.h"
 #include "hw/sysbus.h"
+#include "sysemu/sysemu.h"
 
 static void icc_bus_initfn(Object *obj)
 {
@@ -61,15 +62,39 @@ typedef struct ICCBridgeState {
     SysBusDevice busdev;
     MemoryRegion apic_container;
     MemoryRegion ioapic_container;
+    Notifier cpu_added_notifier;
+    Object **links;
 } ICCBridgeState;
 #define ICC_BRIGDE(obj) OBJECT_CHECK(ICCBridgeState, (obj), TYPE_ICC_BRIDGE)
 
 
+void icc_bridge_set_cpu_link(Object *bridge, Object *cpu_obj)
+{
+    gchar *name;
+    Error *error = NULL;
+    CPUState *cpu = CPU(cpu_obj);
+    int64_t id = CPU_GET_CLASS(cpu)->get_arch_id(cpu);
+
+    name = g_strdup_printf("cpu[%" PRIu32 "]", x86_cpu_apic_id_from_index(id));
+    object_property_set_link(bridge, cpu_obj, name, &error);
+    g_free(name);
+
+    g_assert(error == NULL);
+}
+
+static void icc_bridge_cpu_added_req(Notifier *n, void *opaque)
+{
+    ICCBridgeState *s = container_of(n, ICCBridgeState, cpu_added_notifier);
+
+    icc_bridge_set_cpu_link(OBJECT(s), OBJECT(opaque));
+}
+
 static void icc_bridge_initfn(Object *obj)
 {
     ICCBridgeState *s = ICC_BRIGDE(obj);
     SysBusDevice *sb = SYS_BUS_DEVICE(obj);
     ICCBus *ibus;
+    int i;
 
     ibus = ICC_BUS(qbus_create(TYPE_ICC_BUS, DEVICE(obj), "icc-bus"));
 
@@ -85,12 +110,33 @@ static void icc_bridge_initfn(Object *obj)
     memory_region_init(&s->ioapic_container, "icc-ioapic-container", 0x1000);
     sysbus_init_mmio(sb, &s->ioapic_container);
     ibus->ioapic_address_space = &s->ioapic_container;
+
+    s->links = g_malloc0(sizeof(Object *) * max_cpus);
+    for (i = 0; i < max_cpus; i++) {
+        gchar *cpu_name;
+
+        cpu_name = g_strdup_printf("cpu[%" PRIu32 "]",
+                                   x86_cpu_apic_id_from_index(i));
+        object_property_add_link(obj, cpu_name, TYPE_CPU, &s->links[i], NULL);
+        g_free(cpu_name);
+    }
+
+    s->cpu_added_notifier.notify = icc_bridge_cpu_added_req;
+    qemu_register_cpu_added_notifier(&s->cpu_added_notifier);
+}
+
+static void icc_bridge_fini(Object *obj)
+{
+    ICCBridgeState *s = ICC_BRIGDE(obj);
+
+    g_free(s->links);
 }
 
 static const TypeInfo icc_bridge_info = {
     .name  = "icc-bridge",
     .parent = TYPE_SYS_BUS_DEVICE,
     .instance_init  = icc_bridge_initfn,
+    .instance_finalize  = icc_bridge_fini,
     .instance_size  = sizeof(ICCBridgeState),
 };
 
hw/i386/pc.c +7 -2
index 6d5e164..ada235c 100644
@@ -870,7 +870,8 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level)
     }
 }
 
-static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
+static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id,
+                          SysBusDevice *icc_bridge, Error **errp)
 {
     X86CPU *cpu;
 
@@ -882,6 +883,10 @@ static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
     object_property_set_int(OBJECT(cpu), apic_id, "apic-id", errp);
     object_property_set_bool(OBJECT(cpu), true, "realized", errp);
 
+    if (icc_bridge != NULL) {
+        icc_bridge_set_cpu_link(OBJECT(icc_bridge), OBJECT(cpu));
+    }
+
     if (error_is_set(errp)) {
         if (cpu != NULL) {
             object_unref(OBJECT(cpu));
@@ -911,7 +916,7 @@ void pc_cpus_init(const char *cpu_model)
                                                  TYPE_ICC_BRIDGE, NULL));
 
     for (i = 0; i < smp_cpus; i++) {
-        cpu = pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
+        cpu = pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), ib, &error);
         if (error) {
             fprintf(stderr, "%s\n", error_get_pretty(error));
             error_free(error);
include/hw/i386/icc_bus.h +2 -0
index 69a0278..bc31cd9 100644
@@ -49,5 +49,7 @@ typedef struct ICCDeviceClass {
 
 #define TYPE_ICC_BRIDGE "icc-bridge"
 
+void icc_bridge_set_cpu_link(Object *bridge, Object *cpu);
+
 #endif /* CONFIG_USER_ONLY */
 #endif
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
Signature · 2 lines
Regards,
  Igor
Signature · 1 line
Eduardo
#78 lore raw

Re: [Qemu-devel] [PATCH 18/19] target-i386: expose all possible CPUs as /machine/icc-bridge/cpu[0..N] links

Quoted 40 lines · On Fri, 12 Apr 2013 09:44:09 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 17 lines · Igor Mammedov
Quoted 6 lines · On Thu, 11 Apr 2013 14:19:37 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 5 lines · Igor Mammedov

... and leave links for not present CPUs empty.

It will allow users to query for possible APIC IDs and use them with cpu-add QMP command.

Quoted 7 lines
Quoted 1 line

I don't see anything wrong with having icc-bridge links as well, but I would really like to have a target-independent namespace with links, that could be used to query for the available/valid CPU IDs for cpu-add commands instead of icc-bridge. The IDs on that namespace could be considered completely opaque.

Quoted 5 lines

Considering that -numa in present state is not compatible with cpu-add and that all CPU ID in this case are are sequence [0..maxcpus-1], this patch could be dropped without any harm. libvirt could just use numbers from this sequence like it's doing with current cpu_set without any ID discovery.

But it's not -numa that makes APIC ID probing necessary, it's non-power-of-2 core/thread counts on -smp (that make APIC IDs not match CPU indexes).

"Don't use CPU hotplug with -numa" is easy to be understood by users and by libvirt, but "don't use CPU hotplug with non-power-of-2 cores/threads counts" is harder to explain.

Quoted 6 lines

So, I've postponed target independent until we have -numa reworked, then we could have /machine/node/socket/cpu containers with links. The problem that needs to be solved, is the links storage ownership. Who should allocate and own it? If machine was QOM object already, I'd go with machine but it's not yet.

Quoted 3 lines

If we use CPU index as argument to cpu-add, we don't need to handle all those problems right now, we don't need to expose an APIC ID discovery interface, we make it work even with non-power-of-2 cores/threads

yes, you will get non-power-of-2 working without ID look-up.

Quoted 1 line

counts, and we make it work with -numa.

But you won't get this since, only next non-plugged ID will work, due to how cpu_index is allocated. You can't just overwrite it with new value without breaking current code.

Quoted 5 lines

So, my big question is: why are we trying so hard to avoid using CPU indexes as argument to cpu-add, if it's so much easier, and it is an obvious solution that makes the interface target-independent without any extra effort?

Using cpu_index instead of APIC ID definitely is not effort free and requires quite a bit of rewrite how its used currently, APIC ID is much much easier and less risky choice in this regard.

As for target-independence, any kind of ID is target-independent if treated as opaque. Given that with unplug should come not-contiguous ID usage, the interface to track which CPUs are plugged would be needed anyway. So it could be introduced with this series and provide ID look-up meanwhile. That would give libvirt time actually to start using it, and just remove not-contiguous ID restriction when unplug is ready with all necessary infrastructure already around.

Quoted 162 lines
Quoted 161 lines
Quoted 156 lines
Quoted 153 lines

---
v2:
* s/get_firmware_id/get_arch_id/ due to rebase
* rename cpu_add_notifier to cpu_added_notifier & icc_bridge_cpu_add_req -> icc_bridge_cpued_add_req

---
 hw/cpu/icc_bus.c          | 46

++++++++++++++++++++++++++++++++++++++++++++++
hw/i386/pc.c | 9 +++++++-- include/hw/i386/icc_bus.h |
2 ++ 3 files changed, 55 insertions(+), 2 deletions(-)

hw/cpu/icc_bus.c +20 -0
index ab9623d..5c0b9d4 100644
@@ -18,6 +18,7 @@
  */
 #include "hw/i386/icc_bus.h"
 #include "hw/sysbus.h"
+#include "sysemu/sysemu.h"
 
 static void icc_bus_initfn(Object *obj)
 {
@@ -61,15 +62,39 @@ typedef struct ICCBridgeState {
     SysBusDevice busdev;
     MemoryRegion apic_container;
     MemoryRegion ioapic_container;
+    Notifier cpu_added_notifier;
+    Object **links;
 } ICCBridgeState;
 #define ICC_BRIGDE(obj) OBJECT_CHECK(ICCBridgeState, (obj),
 TYPE_ICC_BRIDGE) 
 
+void icc_bridge_set_cpu_link(Object *bridge, Object *cpu_obj)
+{
+    gchar *name;
+    Error *error = NULL;
+    CPUState *cpu = CPU(cpu_obj);
+    int64_t id = CPU_GET_CLASS(cpu)->get_arch_id(cpu);
+
+    name = g_strdup_printf("cpu[%" PRIu32 "]",
 x86_cpu_apic_id_from_index(id));
+    object_property_set_link(bridge, cpu_obj, name, &error);
+    g_free(name);
+
+    g_assert(error == NULL);
+}
+
+static void icc_bridge_cpu_added_req(Notifier *n, void *opaque)
+{
+    ICCBridgeState *s = container_of(n, ICCBridgeState,

cpu_added_notifier); +

hw/cpu/icc_bus.c +4 -0
+    icc_bridge_set_cpu_link(OBJECT(s), OBJECT(opaque));
+}
+
 static void icc_bridge_initfn(Object *obj)
 {
     ICCBridgeState *s = ICC_BRIGDE(obj);
     SysBusDevice *sb = SYS_BUS_DEVICE(obj);
     ICCBus *ibus;
+    int i;
 
     ibus = ICC_BUS(qbus_create(TYPE_ICC_BUS, DEVICE(obj),

"icc-bus"));

hw/cpu/icc_bus.c +21 -0
@@ -85,12 +110,33 @@ static void icc_bridge_initfn(Object *obj)
     memory_region_init(&s->ioapic_container, "icc-ioapic-container",
 0x1000); sysbus_init_mmio(sb, &s->ioapic_container);
     ibus->ioapic_address_space = &s->ioapic_container;
+
+    s->links = g_malloc0(sizeof(Object *) * max_cpus);
+    for (i = 0; i < max_cpus; i++) {
+        gchar *cpu_name;
+
+        cpu_name = g_strdup_printf("cpu[%" PRIu32 "]",
+                                   x86_cpu_apic_id_from_index(i));
+        object_property_add_link(obj, cpu_name, TYPE_CPU,
 &s->links[i], NULL);
+        g_free(cpu_name);
+    }
+
+    s->cpu_added_notifier.notify = icc_bridge_cpu_added_req;
+    qemu_register_cpu_added_notifier(&s->cpu_added_notifier);
+}
+
+static void icc_bridge_fini(Object *obj)
+{
+    ICCBridgeState *s = ICC_BRIGDE(obj);
+
+    g_free(s->links);
 }
 
 static const TypeInfo icc_bridge_info = {
     .name  = "icc-bridge",
     .parent = TYPE_SYS_BUS_DEVICE,
     .instance_init  = icc_bridge_initfn,
+    .instance_finalize  = icc_bridge_fini,
     .instance_size  = sizeof(ICCBridgeState),
 };
hw/i386/pc.c +6 -2
index 6d5e164..ada235c 100644
@@ -870,7 +870,8 @@ void pc_acpi_smi_interrupt(void *opaque, int irq,
 int level) }
 }
 
-static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id,
 Error **errp) +static X86CPU *pc_new_cpu(const char *cpu_model,
 int64_t apic_id,
+                          SysBusDevice *icc_bridge, Error **errp)
 {
     X86CPU *cpu;
@@ -882,6 +883,10 @@ static X86CPU *pc_new_cpu(const char *cpu_model,
 int64_t apic_id, Error **errp) object_property_set_int(OBJECT(cpu),
 apic_id, "apic-id", errp); object_property_set_bool(OBJECT(cpu),
 true, "realized", errp); 
+    if (icc_bridge != NULL) {
+        icc_bridge_set_cpu_link(OBJECT(icc_bridge), OBJECT(cpu));
+    }
+
     if (error_is_set(errp)) {
         if (cpu != NULL) {
             object_unref(OBJECT(cpu));
@@ -911,7 +916,7 @@ void pc_cpus_init(const char *cpu_model)
                                                  TYPE_ICC_BRIDGE,
 NULL)); 
     for (i = 0; i < smp_cpus; i++) {
-        cpu = pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i),
 &error);
+        cpu = pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i),
 ib, &error); if (error) {
             fprintf(stderr, "%s\n", error_get_pretty(error));
             error_free(error);
include/hw/i386/icc_bus.h +2 -0
index 69a0278..bc31cd9 100644
@@ -49,5 +49,7 @@ typedef struct ICCDeviceClass {
 
 #define TYPE_ICC_BRIDGE "icc-bridge"
 
+void icc_bridge_set_cpu_link(Object *bridge, Object *cpu);
+
 #endif /* CONFIG_USER_ONLY */
 #endif
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
Signature · 2 lines
Regards,
  Igor
#79 lore raw

Re: [Qemu-devel] [PATCH 18/19] target-i386: expose all possible CPUs as /machine/icc-bridge/cpu[0..N] links

Quoted 46 lines · Igor Mammedov
Quoted 40 lines · On Fri, 12 Apr 2013 09:44:09 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 17 lines · Igor Mammedov
Quoted 6 lines · On Thu, 11 Apr 2013 14:19:37 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 5 lines · Igor Mammedov

... and leave links for not present CPUs empty.

It will allow users to query for possible APIC IDs and use them with cpu-add QMP command.

Quoted 7 lines
Quoted 1 line

I don't see anything wrong with having icc-bridge links as well, but I would really like to have a target-independent namespace with links, that could be used to query for the available/valid CPU IDs for cpu-add commands instead of icc-bridge. The IDs on that namespace could be considered completely opaque.

Quoted 5 lines

Considering that -numa in present state is not compatible with cpu-add and that all CPU ID in this case are are sequence [0..maxcpus-1], this patch could be dropped without any harm. libvirt could just use numbers from this sequence like it's doing with current cpu_set without any ID discovery.

But it's not -numa that makes APIC ID probing necessary, it's non-power-of-2 core/thread counts on -smp (that make APIC IDs not match CPU indexes).

"Don't use CPU hotplug with -numa" is easy to be understood by users and by libvirt, but "don't use CPU hotplug with non-power-of-2 cores/threads counts" is harder to explain.

Quoted 6 lines

So, I've postponed target independent until we have -numa reworked, then we could have /machine/node/socket/cpu containers with links. The problem that needs to be solved, is the links storage ownership. Who should allocate and own it? If machine was QOM object already, I'd go with machine but it's not yet.

Quoted 3 lines

If we use CPU index as argument to cpu-add, we don't need to handle all those problems right now, we don't need to expose an APIC ID discovery interface, we make it work even with non-power-of-2 cores/threads

Quoted 6 lines

yes, you will get non-power-of-2 working without ID look-up.

Quoted 1 line

counts, and we make it work with -numa.

But you won't get this since, only next non-plugged ID will work, due to how cpu_index is allocated. You can't just overwrite it with new value without breaking current code.

OK, I think I get it: the problem is the cpu_index field specifically, because it is set automatically, and there may be lots of assumptions in the code about it. Maybe I should rephrase my suggestion: let's replace "CPU index" above with "a predictable ID allocation system where IDs will be in the range 0..(max_cpus-1)". We don't even need to touch the cpu_index field in the CPU objects if we think this is too risky.

I am just proposing that we use IDs so that we just need to add a
apic_id = apic_id_for_cpu_index(id)
line at the beginning of the cpu-add implementation, and everything else would look exactly the same.

Nothing else would change in your implementation, except that now we won't need a ID lookup system for 1.5 because libvirt can assume that the next available CPU ID will be smp_cpus+1.

There would be no requirement to make the index-based IDs contiguous, just like there's no requirement to make APIC IDs contiguous. The only difference is that CPU indexes will predictable IDs, that will be always in the range 0..(max_cpus-1) and won't require an ID lookup mechanism. And it will work with non-power-of-2 threads/counts, and it will work with -numa.

Quoted 6 lines
Quoted 6 lines

So, my big question is: why are we trying so hard to avoid using CPU indexes as argument to cpu-add, if it's so much easier, and it is an obvious solution that makes the interface target-independent without any extra effort?

Quoted 6 lines

Using cpu_index instead of APIC ID definitely is not effort free and requires quite a bit of rewrite how its used currently, APIC ID is much much easier and less risky choice in this regard.

As for target-independence, any kind of ID is target-independent if treated as opaque.

True, as long as we have a target-independent ID lookup system, which we don't have today. That's the part where we would need less effort.

Quoted 6 lines

Given that with unplug should come not-contiguous ID usage, the interface to track which CPUs are plugged would be needed anyway. So it could be introduced with this series and provide ID look-up meanwhile. That would give libvirt time actually to start using it, and just remove not-contiguous ID restriction when unplug is ready with all necessary infrastructure already around.

Let's rephrase my suggestion: I don't think we should use the internal "cpu->cpu_index" field as ID, necessarily. I only suggest that we make the IDs predictably in the range 0..(max_cpus-1) so we:
1) don't need an ID lookup mechanism;
2) keep it compatible with the existing -numa options.

If we really want to use APIC ID one day, we may implement an lookup mechanism that will provide IDs to libvirt, and declare the "IDs between 0..(max_cpus-1)" assumption as deprecated, so we can start using APIC IDs as the (opaque) CPU IDs in the future. But before we do that, we would implement a better "-numa" interface first.

You even suggested that we did something similar, above:

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Considering that -numa in present state is not compatible with cpu-add and that all CPU ID in this case are are sequence [0..maxcpus-1], this patch could be dropped without any harm. libvirt could just use numbers from this sequence like it's doing with current cpu_set without any ID discovery.

What I suggest is that we let libvirt make exactly the same assumptions you suggest, but if we add a single line to the code:
apic_id = apic_id_for_cpu_index(id)
we will make this intermediate solution work with NUMA _and_ work with non-power-of-2 cores/threads counts.

The other cases (non-numa with power-of-2 cores/threads counts) would work exactly the same, because in those cases apic_id_for_cpu_index(id) == id.

Signature · 1 line
Eduardo
#80 lore raw

Re: [Qemu-devel] [PATCH 18/19] target-i386: expose all possible CPUs as /machine/icc-bridge/cpu[0..N] links

Quoted 127 lines · On Mon, 15 Apr 2013 11:48:45 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 46 lines · Igor Mammedov
Quoted 40 lines · On Fri, 12 Apr 2013 09:44:09 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 17 lines · Igor Mammedov
Quoted 6 lines · On Thu, 11 Apr 2013 14:19:37 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 5 lines · Igor Mammedov

... and leave links for not present CPUs empty.

It will allow users to query for possible APIC IDs and use them with cpu-add QMP command.

Quoted 7 lines
Quoted 1 line

I don't see anything wrong with having icc-bridge links as well, but I would really like to have a target-independent namespace with links, that could be used to query for the available/valid CPU IDs for cpu-add commands instead of icc-bridge. The IDs on that namespace could be considered completely opaque.

Quoted 5 lines

Considering that -numa in present state is not compatible with cpu-add and that all CPU ID in this case are are sequence [0..maxcpus-1], this patch could be dropped without any harm. libvirt could just use numbers from this sequence like it's doing with current cpu_set without any ID discovery.

But it's not -numa that makes APIC ID probing necessary, it's non-power-of-2 core/thread counts on -smp (that make APIC IDs not match CPU indexes).

"Don't use CPU hotplug with -numa" is easy to be understood by users and by libvirt, but "don't use CPU hotplug with non-power-of-2 cores/threads counts" is harder to explain.

Quoted 6 lines

So, I've postponed target independent until we have -numa reworked, then we could have /machine/node/socket/cpu containers with links. The problem that needs to be solved, is the links storage ownership. Who should allocate and own it? If machine was QOM object already, I'd go with machine but it's not yet.

Quoted 3 lines

If we use CPU index as argument to cpu-add, we don't need to handle all those problems right now, we don't need to expose an APIC ID discovery interface, we make it work even with non-power-of-2 cores/threads

Quoted 6 lines

yes, you will get non-power-of-2 working without ID look-up.

Quoted 1 line

counts, and we make it work with -numa.

But you won't get this since, only next non-plugged ID will work, due to how cpu_index is allocated. You can't just overwrite it with new value without breaking current code.

OK, I think I get it: the problem is the cpu_index field specifically, because it is set automatically, and there may be lots of assumptions in the code about it. Maybe I should rephrase my suggestion: let's replace "CPU index" above with "a predictable ID allocation system where IDs will be in the range 0..(max_cpus-1)". We don't even need to touch the cpu_index field in the CPU objects if we think this is too risky.

I am just proposing that we use IDs so that we just need to add a
apic_id = apic_id_for_cpu_index(id)
line at the beginning of the cpu-add implementation, and everything else would look exactly the same.

Nothing else would change in your implementation, except that now we won't need a ID lookup system for 1.5 because libvirt can assume that the next available CPU ID will be smp_cpus+1.

There would be no requirement to make the index-based IDs contiguous, just like there's no requirement to make APIC IDs contiguous. The only difference is that CPU indexes will predictable IDs, that will be always in the range 0..(max_cpus-1) and won't require an ID lookup mechanism. And it will work with non-power-of-2 threads/counts, and it will work with -numa.

Quoted 6 lines
Quoted 6 lines

So, my big question is: why are we trying so hard to avoid using CPU indexes as argument to cpu-add, if it's so much easier, and it is an obvious solution that makes the interface target-independent without any extra effort?

Quoted 6 lines

Using cpu_index instead of APIC ID definitely is not effort free and requires quite a bit of rewrite how its used currently, APIC ID is much much easier and less risky choice in this regard.

As for target-independence, any kind of ID is target-independent if treated as opaque.

True, as long as we have a target-independent ID lookup system, which we don't have today. That's the part where we would need less effort.

Quoted 6 lines

Given that with unplug should come not-contiguous ID usage, the interface to track which CPUs are plugged would be needed anyway. So it could be introduced with this series and provide ID look-up meanwhile. That would give libvirt time actually to start using it, and just remove not-contiguous ID restriction when unplug is ready with all necessary infrastructure already around.

Let's rephrase my suggestion: I don't think we should use the internal "cpu->cpu_index" field as ID, necessarily. I only suggest that we make the IDs predictably in the range 0..(max_cpus-1) so we:
1) don't need an ID lookup mechanism;
2) keep it compatible with the existing -numa options.

If we really want to use APIC ID one day, we may implement an lookup mechanism that will provide IDs to libvirt, and declare the "IDs between 0..(max_cpus-1)" assumption as deprecated, so we can start using APIC IDs as the (opaque) CPU IDs in the future. But before we do that, we would implement a better "-numa" interface first.

You even suggested that we did something similar, above:

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Considering that -numa in present state is not compatible with cpu-add and that all CPU ID in this case are are sequence [0..maxcpus-1], this patch could be dropped without any harm. libvirt could just use numbers from this sequence like it's doing with current cpu_set without any ID discovery.

What I suggest is that we let libvirt make exactly the same assumptions you suggest, but if we add a single line to the code:
apic_id = apic_id_for_cpu_index(id)
we will make this intermediate solution work with NUMA _and_ work with non-power-of-2 cores/threads counts.

Quoted 4 lines

The other cases (non-numa with power-of-2 cores/threads counts) would work exactly the same, because in those cases apic_id_for_cpu_index(id) == id.

Rephrasing suggestion:
drop 18/19 and use apic_id = apic_id_for_cpu_index(id) in 19/19 Am I correct?

#81 lore raw

Re: [Qemu-devel] [PATCH 18/19] target-i386: expose all possible CPUs as /machine/icc-bridge/cpu[0..N] links

Quoted 130 lines · Igor Mammedov
Quoted 127 lines · On Mon, 15 Apr 2013 11:48:45 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 46 lines · Igor Mammedov
Quoted 40 lines · On Fri, 12 Apr 2013 09:44:09 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 17 lines · Igor Mammedov
Quoted 6 lines · On Thu, 11 Apr 2013 14:19:37 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 5 lines · Igor Mammedov

... and leave links for not present CPUs empty.

It will allow users to query for possible APIC IDs and use them with cpu-add QMP command.

Quoted 7 lines
Quoted 1 line

I don't see anything wrong with having icc-bridge links as well, but I would really like to have a target-independent namespace with links, that could be used to query for the available/valid CPU IDs for cpu-add commands instead of icc-bridge. The IDs on that namespace could be considered completely opaque.

Quoted 5 lines

Considering that -numa in present state is not compatible with cpu-add and that all CPU ID in this case are are sequence [0..maxcpus-1], this patch could be dropped without any harm. libvirt could just use numbers from this sequence like it's doing with current cpu_set without any ID discovery.

But it's not -numa that makes APIC ID probing necessary, it's non-power-of-2 core/thread counts on -smp (that make APIC IDs not match CPU indexes).

"Don't use CPU hotplug with -numa" is easy to be understood by users and by libvirt, but "don't use CPU hotplug with non-power-of-2 cores/threads counts" is harder to explain.

Quoted 6 lines

So, I've postponed target independent until we have -numa reworked, then we could have /machine/node/socket/cpu containers with links. The problem that needs to be solved, is the links storage ownership. Who should allocate and own it? If machine was QOM object already, I'd go with machine but it's not yet.

Quoted 3 lines

If we use CPU index as argument to cpu-add, we don't need to handle all those problems right now, we don't need to expose an APIC ID discovery interface, we make it work even with non-power-of-2 cores/threads

Quoted 6 lines

yes, you will get non-power-of-2 working without ID look-up.

Quoted 1 line

counts, and we make it work with -numa.

But you won't get this since, only next non-plugged ID will work, due to how cpu_index is allocated. You can't just overwrite it with new value without breaking current code.

OK, I think I get it: the problem is the cpu_index field specifically, because it is set automatically, and there may be lots of assumptions in the code about it. Maybe I should rephrase my suggestion: let's replace "CPU index" above with "a predictable ID allocation system where IDs will be in the range 0..(max_cpus-1)". We don't even need to touch the cpu_index field in the CPU objects if we think this is too risky.

I am just proposing that we use IDs so that we just need to add a
apic_id = apic_id_for_cpu_index(id)
line at the beginning of the cpu-add implementation, and everything else would look exactly the same.

Nothing else would change in your implementation, except that now we won't need a ID lookup system for 1.5 because libvirt can assume that the next available CPU ID will be smp_cpus+1.

There would be no requirement to make the index-based IDs contiguous, just like there's no requirement to make APIC IDs contiguous. The only difference is that CPU indexes will predictable IDs, that will be always in the range 0..(max_cpus-1) and won't require an ID lookup mechanism. And it will work with non-power-of-2 threads/counts, and it will work with -numa.

Quoted 6 lines
Quoted 6 lines

So, my big question is: why are we trying so hard to avoid using CPU indexes as argument to cpu-add, if it's so much easier, and it is an obvious solution that makes the interface target-independent without any extra effort?

Quoted 6 lines

Using cpu_index instead of APIC ID definitely is not effort free and requires quite a bit of rewrite how its used currently, APIC ID is much much easier and less risky choice in this regard.

As for target-independence, any kind of ID is target-independent if treated as opaque.

True, as long as we have a target-independent ID lookup system, which we don't have today. That's the part where we would need less effort.

Quoted 6 lines

Given that with unplug should come not-contiguous ID usage, the interface to track which CPUs are plugged would be needed anyway. So it could be introduced with this series and provide ID look-up meanwhile. That would give libvirt time actually to start using it, and just remove not-contiguous ID restriction when unplug is ready with all necessary infrastructure already around.

Let's rephrase my suggestion: I don't think we should use the internal "cpu->cpu_index" field as ID, necessarily. I only suggest that we make the IDs predictably in the range 0..(max_cpus-1) so we:
1) don't need an ID lookup mechanism;
2) keep it compatible with the existing -numa options.

If we really want to use APIC ID one day, we may implement an lookup mechanism that will provide IDs to libvirt, and declare the "IDs between 0..(max_cpus-1)" assumption as deprecated, so we can start using APIC IDs as the (opaque) CPU IDs in the future. But before we do that, we would implement a better "-numa" interface first.

You even suggested that we did something similar, above:

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Considering that -numa in present state is not compatible with cpu-add and that all CPU ID in this case are are sequence [0..maxcpus-1], this patch could be dropped without any harm. libvirt could just use numbers from this sequence like it's doing with current cpu_set without any ID discovery.

What I suggest is that we let libvirt make exactly the same assumptions you suggest, but if we add a single line to the code:
apic_id = apic_id_for_cpu_index(id)
we will make this intermediate solution work with NUMA _and_ work with non-power-of-2 cores/threads counts.

Quoted 8 lines
Quoted 4 lines

The other cases (non-numa with power-of-2 cores/threads counts) would work exactly the same, because in those cases apic_id_for_cpu_index(id) == id.

Rephrasing suggestion:
drop 18/19 and use apic_id = apic_id_for_cpu_index(id) in 19/19 Am I correct?

Exactly. I mean: I expect that to be enough and not require other changes, but I didn't look at all patches line-by-line yet.

Signature · 1 line
Eduardo
#82 lore raw

Re: [Qemu-devel] [PATCH 18/19] target-i386: expose all possible CPUs as /machine/icc-bridge/cpu[0..N] links

Quoted 136 lines · On Mon, 15 Apr 2013 12:26:07 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 135 lines · Igor Mammedov
Quoted 131 lines · On Mon, 15 Apr 2013 11:48:45 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 46 lines · Igor Mammedov
Quoted 40 lines · On Fri, 12 Apr 2013 09:44:09 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 17 lines · Igor Mammedov
Quoted 6 lines · On Thu, 11 Apr 2013 14:19:37 -0300 Eduardo Habkost <[email protected]> wrote:
Quoted 5 lines · Igor Mammedov

... and leave links for not present CPUs empty.

It will allow users to query for possible APIC IDs and use them with cpu-add QMP command.

Quoted 7 lines
Quoted 1 line

I don't see anything wrong with having icc-bridge links as well, but I would really like to have a target-independent namespace with links, that could be used to query for the available/valid CPU IDs for cpu-add commands instead of icc-bridge. The IDs on that namespace could be considered completely opaque.

Quoted 5 lines

Considering that -numa in present state is not compatible with cpu-add and that all CPU ID in this case are are sequence [0..maxcpus-1], this patch could be dropped without any harm. libvirt could just use numbers from this sequence like it's doing with current cpu_set without any ID discovery.

But it's not -numa that makes APIC ID probing necessary, it's non-power-of-2 core/thread counts on -smp (that make APIC IDs not match CPU indexes).

"Don't use CPU hotplug with -numa" is easy to be understood by users and by libvirt, but "don't use CPU hotplug with non-power-of-2 cores/threads counts" is harder to explain.

Quoted 6 lines

So, I've postponed target independent until we have -numa reworked, then we could have /machine/node/socket/cpu containers with links. The problem that needs to be solved, is the links storage ownership. Who should allocate and own it? If machine was QOM object already, I'd go with machine but it's not yet.

Quoted 3 lines

If we use CPU index as argument to cpu-add, we don't need to handle all those problems right now, we don't need to expose an APIC ID discovery interface, we make it work even with non-power-of-2 cores/threads

Quoted 6 lines

yes, you will get non-power-of-2 working without ID look-up.

Quoted 1 line

counts, and we make it work with -numa.

But you won't get this since, only next non-plugged ID will work, due to how cpu_index is allocated. You can't just overwrite it with new value without breaking current code.

OK, I think I get it: the problem is the cpu_index field specifically, because it is set automatically, and there may be lots of assumptions in the code about it. Maybe I should rephrase my suggestion: let's replace "CPU index" above with "a predictable ID allocation system where IDs will be in the range 0..(max_cpus-1)". We don't even need to touch the cpu_index field in the CPU objects if we think this is too risky.

I am just proposing that we use IDs so that we just need to add a
apic_id = apic_id_for_cpu_index(id)
line at the beginning of the cpu-add implementation, and everything else would look exactly the same.

Nothing else would change in your implementation, except that now we won't need a ID lookup system for 1.5 because libvirt can assume that the next available CPU ID will be smp_cpus+1.

There would be no requirement to make the index-based IDs contiguous, just like there's no requirement to make APIC IDs contiguous. The only difference is that CPU indexes will predictable IDs, that will be always in the range 0..(max_cpus-1) and won't require an ID lookup mechanism. And it will work with non-power-of-2 threads/counts, and it will work with -numa.

Quoted 6 lines
Quoted 6 lines

So, my big question is: why are we trying so hard to avoid using CPU indexes as argument to cpu-add, if it's so much easier, and it is an obvious solution that makes the interface target-independent without any extra effort?

Quoted 6 lines

Using cpu_index instead of APIC ID definitely is not effort free and requires quite a bit of rewrite how its used currently, APIC ID is much much easier and less risky choice in this regard.

As for target-independence, any kind of ID is target-independent if treated as opaque.

True, as long as we have a target-independent ID lookup system, which we don't have today. That's the part where we would need less effort.

Quoted 6 lines

Given that with unplug should come not-contiguous ID usage, the interface to track which CPUs are plugged would be needed anyway. So it could be introduced with this series and provide ID look-up meanwhile. That would give libvirt time actually to start using it, and just remove not-contiguous ID restriction when unplug is ready with all necessary infrastructure already around.

Let's rephrase my suggestion: I don't think we should use the internal "cpu->cpu_index" field as ID, necessarily. I only suggest that we make the IDs predictably in the range 0..(max_cpus-1) so we:
1) don't need an ID lookup mechanism;
2) keep it compatible with the existing -numa options.

If we really want to use APIC ID one day, we may implement an lookup mechanism that will provide IDs to libvirt, and declare the "IDs between 0..(max_cpus-1)" assumption as deprecated, so we can start using APIC IDs as the (opaque) CPU IDs in the future. But before we do that, we would implement a better "-numa" interface first.

You even suggested that we did something similar, above:

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Considering that -numa in present state is not compatible with cpu-add and that all CPU ID in this case are are sequence [0..maxcpus-1], this patch could be dropped without any harm. libvirt could just use numbers from this sequence like it's doing with current cpu_set without any ID discovery.

What I suggest is that we let libvirt make exactly the same assumptions you suggest, but if we add a single line to the code:
apic_id = apic_id_for_cpu_index(id)
we will make this intermediate solution work with NUMA _and_ work with non-power-of-2 cores/threads counts.

The other cases (non-numa with power-of-2 cores/threads counts) would work exactly the same, because in those cases apic_id_for_cpu_index(id) == id.

Quoted 6 lines
Quoted 3 lines

Rephrasing suggestion:
drop 18/19 and use apic_id = apic_id_for_cpu_index(id) in 19/19 Am I correct?

Exactly. I mean: I expect that to be enough and not require other changes, but I didn't look at all patches line-by-line yet.

Ok, I'm dropping it and submitting amended series,

Thanks for review.

Quoted 4 lines
Signature · 2 lines
-- 
Eduardo
Signature · 2 lines
Regards,
  Igor
#83 lore raw

PATCH v3 19/19 add cpu-add qmp command and implement CPU hot-add for target-i386

In reply to Igor Mammedov

... via current_machine->cpu_hot_add() hook called by cpu-set QMP command, for x86 target.

cpu-add command usage will be descrbed at:
http://wiki.qemu.org/Features/CPUHotplug

---
v4:
* merge "qmp: add cpu-add qmp command" & "target-i386: implement CPU hot-add" patches
* move notifier call to CPUCLass.realize()
* add hook cpu_hot_add to QEMUMachine
* make QEMUMachineInitArgs global and keep default cpu_model there

v3:
* it appears that 'online/offline' in cpu-set are confusing people with what command actually does and users might have to distinguish if 'offline' is not implemented by parsing error message. To simplify things replace cpu-set with cpu-add command to show more clear what command does and just add cpu-del when CPU remove is implemented.

v2:
* s/cpu_set/cpu-set/
* qmp doc style fix
* use bool type instead of opencodding online/offline string

---
 hw/i386/pc.c        | 20 ++++++++++++++++++++
 include/hw/boards.h |  3 +++
 qapi-schema.json    | 11 +++++++++++
 qmp-commands.hx     | 23 +++++++++++++++++++++++
 qmp.c               | 10 ++++++++++
 vl.c                |  6 +++++-
 6 files changed, 72 insertions(+), 1 deletion(-)
hw/i386/pc.c +20 -0
index ada235c..ba93180 100644
@@ -53,6 +53,7 @@
 #include "qemu/bitmap.h"
 #include "qemu/config-file.h"
 #include "hw/i386/icc_bus.h"
+#include "hw/boards.h"
 
 /* debug PC/ISA interrupts */
 //#define DEBUG_IRQ
@@ -896,6 +897,23 @@ static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id,
     return cpu;
 }
 
+static void do_cpu_hot_add(const int64_t id, Error **errp)
+{
+    if (cpu_exists(id)) {
+        error_setg(errp, "Unable to add CPU with APIC ID: 0x%" PRIx64
+                   ", it already exists", id);
+        return;
+    }
+
+    if (id >= pc_apic_id_limit(max_cpus)) {
+        error_setg(errp, "Unable to add CPU with APIC ID: 0x%" PRIx64
+                   ", max allowed: 0x%x", id, pc_apic_id_limit(max_cpus) - 1);
+        return;
+    }
+
+    pc_new_cpu(machine_args->cpu_model, id, NULL, errp);
+}
+
 void pc_cpus_init(const char *cpu_model)
 {
     int i;
@@ -910,7 +928,9 @@ void pc_cpus_init(const char *cpu_model)
 #else
         cpu_model = "qemu32";
 #endif
+        machine_args->cpu_model = cpu_model;
     }
+    current_machine->hot_add_cpu = do_cpu_hot_add;
 
     ib = SYS_BUS_DEVICE(object_resolve_path_type("icc-bridge",
                                                  TYPE_ICC_BRIDGE, NULL));
include/hw/boards.h +3 -0
index 425bdc7..de8f92a 100644
@@ -18,6 +18,8 @@ typedef struct QEMUMachineInitArgs {
     const char *cpu_model;
 } QEMUMachineInitArgs;
 
+extern QEMUMachineInitArgs *machine_args;
+
 typedef void QEMUMachineInitFunc(QEMUMachineInitArgs *args);
 
 typedef void QEMUMachineResetFunc(void);
@@ -43,6 +45,7 @@ typedef struct QEMUMachine {
     GlobalProperty *compat_props;
     struct QEMUMachine *next;
     const char *hw_version;
+    void (*hot_add_cpu)(const int64_t id, Error **errp);
 } QEMUMachine;
 
 int qemu_register_machine(QEMUMachine *m);
qapi-schema.json +11 -0
index db542f6..a760ed5 100644
@@ -1387,6 +1387,17 @@
 { 'command': 'cpu', 'data': {'index': 'int'} }
 
 ##
+# @cpu-add
+#
+# Adds CPU with specified id
+#
+# @id: cpu id of CPU to be created
+#
+# Returns: Nothing on success
+##
+{ 'command': 'cpu-add', 'data': {'id': 'int'} }
+
+##
 # @memsave:
 #
 # Save a portion of guest memory to a file.
qmp-commands.hx +23 -0
index 1e0e11e..79b2ba7 100644
@@ -407,6 +407,29 @@ Example:
 EQMP
 
     {
+        .name       = "cpu-add",
+        .args_type  = "id:i",
+        .mhandler.cmd_new = qmp_marshal_input_cpu_add,
+    },
+
+SQMP
+cpu-add
+-------
+
+Adds virtual cpu
+
+Arguments:
+
+- "id": cpu id (json-int)
+
+Example:
+
+-> { "execute": "cpu-add", "arguments": { "id": 2 } }
+<- { "return": {} }
+
+EQMP
+
+    {
         .name       = "memsave",
         .args_type  = "val:l,size:i,filename:s,cpu:i?",
         .mhandler.cmd_new = qmp_marshal_input_memsave,
qmp.c +10 -0
index 55b056b..8a9fd9e 100644
@@ -24,6 +24,7 @@
 #include "hw/qdev.h"
 #include "sysemu/blockdev.h"
 #include "qom/qom-qobject.h"
+#include "hw/boards.h"
 
 NameInfo *qmp_query_name(Error **errp)
 {
@@ -108,6 +109,15 @@ void qmp_cpu(int64_t index, Error **errp)
     /* Just do nothing */
 }
 
+void qmp_cpu_add(int64_t id, Error **errp)
+{
+    if (current_machine->hot_add_cpu) {
+        current_machine->hot_add_cpu(id, errp);
+    } else {
+        error_setg(errp, "Not supported");
+    }
+}
+
 #ifndef CONFIG_VNC
 /* If VNC support is enabled, the "true" query-vnc command is
    defined in the VNC subsystem */
vl.c +5 -1
index 21952c3..cdce08d 100644
@@ -179,6 +179,8 @@ int main(int argc, char **argv)
 #define MAX_VIRTIO_CONSOLES 1
 #define MAX_SCLP_CONSOLES 1
 
+QEMUMachineInitArgs *machine_args;
+
 static const char *data_dir[16];
 static int data_dir_idx;
 const char *bios_name = NULL;
@@ -4310,13 +4312,15 @@ int main(int argc, char **argv, char **envp)
                                  .kernel_cmdline = kernel_cmdline,
                                  .initrd_filename = initrd_filename,
                                  .cpu_model = cpu_model };
+    machine_args = &args;
+    current_machine = machine;
+
     machine->init(&args);
 
     cpu_synchronize_all_post_init();
 
     set_numa_modes();
 
-    current_machine = machine;
 
     /* init USB devices */
     if (usb_enabled(false)) {
Signature · 1 line
1.8.2