Skip to contentSkip to conversation

target-i386: CPU hot-add with cpu-add QMP command

PATCH Superseded Igor Mammedov 10 participants 73 messages Apr 15, 2013 – Apr 26, 2013 1 message not in the archive Kbuild

v4 of 9 Superseded Latest is v9 →

lore
Share & tools
lore
#0 lore raw

v4 cover target-i386: CPU hot-add with cpu-add QMP command

Implements alternative way for hot-adding CPU using cpu-add QMP command, wich could be useful until it would be possible to add CPUs via device_add.

All patches except the last are also applicable to device_add aprroach.

To hot-add CPU use following command from qmp-shell:
cpu-add id=[0..max-cpus - 1)

git tree for testing: https://github.com/imammedo/qemu/tree/cpu_add.v4

based on qom-cpu tree

v4->v3:
* 'id' in cpu-add command will be a thread number instead of APIC ID
* split off resume_vcpu() into separate patch
* move notifier from rtc code into pc.c

v2->v3:
* use local error & propagate_error() instead of operating on passed in errp in several places
* replace CPUClass.get_firmware_id() with CPUClass.get_arch_id()
* leave IOAPIC creation to board and just set bus to icc-bus
* include kvm-stub.o in cpu libary if no KVM is configured
* create resume_vcpu() stub and include it in libqemustub, and use it directly instead of CPU method
* acpi_piix4: s/cpu_add_notifier/cpu_added_notifier/

v1->v2:
* generalize cpu sync to KVM, resume and hot-plug notification and invoke them form CPUClass, to make available to all targets.
* introduce cpu_exists() and CPUClass.get_firmware_id() and use the last one in acpi_piix to make code target independent.
* move IOAPIC to ICC bus, it was suggested and easy to convert.
* leave kvmvapic as SysBusDevice, it doesn't affect hot-plug and created only once for all APIC instances. I haven't found yet good/clean enough way to convert it to ICCDevice. May be follow-up though.
* split one big ICC patch into several, one per converted device
* add cpu_hot_add hook to machine and implement it for target-i386, instead of adding stabs. Could be used by other targets to implement cpu-add.
* pre-allocate links<CPU> for all possible CPUs and make them available at /machine/icc-bridge/cpu[0..N] QOM path, so users could find out possible/free CPU IDs to use in cpu-add command.

Igor Mammedov (16):
cpu: make kvm-stub.o a part of CPU library
cpu: call cpu_synchronize_post_init() from CPUClass.realize() if
hotplugged
introduce resume_vcpu(), for single CPU
cpu: resume CPU from CPUClass.cpu_common_realizefn() when it is
hot-plugged
introduce CPU hot-plug notifier
target-i386: pc: update rtc_cmos on CPU hot-plug
cpu: introduce get_arch_id() method and override it for target-i386
cpu: add helper cpu_exists(), to check if CPU with specified id exists
acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest
target-i386: introduce apic-id property
introduce ICC bus/device/bridge
target-i386: cpu: attach ICC bus to CPU on its creation
target-i386: replace MSI_SPACE_SIZE with APIC_SPACE_SIZE
target-i386: move APIC to ICC bus
target-i386: move IOAPIC to ICC bus
add cpu-add qmp command and implement CPU hot-add for target-i386

 Makefile.target                    |  14 ++---
 cpus.c                             |  11 +++-
 default-configs/i386-softmmu.mak   |   1 +
 default-configs/x86_64-softmmu.mak |   1 +
 hw/acpi/piix4.c                    | 114 ++++++++++++++++++++++++++++++++++++-
 hw/cpu/Makefile.objs               |   1 +
 hw/cpu/icc_bus.c                   | 104 +++++++++++++++++++++++++++++++++
 hw/i386/kvm/apic.c                 |   2 +-
 hw/i386/kvm/ioapic.c               |   2 +-
 hw/i386/kvmvapic.c                 |   1 +
 hw/i386/pc.c                       |  91 +++++++++++++++++++++++++++--
 hw/i386/pc_piix.c                  |   7 +++
 hw/i386/pc_q35.c                   |   7 +++
 hw/intc/apic.c                     |   2 +-
 hw/intc/apic_common.c              |  17 ++++--
 hw/intc/ioapic_common.c            |  15 +++--
 hw/timer/mc146818rtc.c             |   7 +++
 hw/xen/xen_apic.c                  |   2 +-
 include/exec/memory.h              |  10 ++++
 include/hw/boards.h                |   3 +
 include/hw/i386/apic_internal.h    |   8 +--
 include/hw/i386/icc_bus.h          |  53 +++++++++++++++++
 include/hw/i386/ioapic_internal.h  |   6 +-
 include/hw/pci/msi.h               |   2 +
 include/hw/timer/mc146818rtc.h     |   1 +
 include/qom/cpu.h                  |  21 +++++++
 include/sysemu/kvm.h               |  22 +++----
 include/sysemu/sysemu.h            |   3 +
 kvm-all.c                          |   1 +
 kvm-stub.c                         |   3 +
 memory.c                           |  11 ++++
 qapi-schema.json                   |  11 ++++
 qmp-commands.hx                    |  23 ++++++++
 qmp.c                              |  10 ++++
 qom/cpu.c                          |  44 ++++++++++++++
 stubs/Makefile.objs                |   1 +
 stubs/cpus.c                       |   5 ++
 target-i386/cpu.c                  |  75 +++++++++++++++++++-----
 target-i386/cpu.h                  |   1 +
 vl.c                               |   7 ++-
 40 files changed, 656 insertions(+), 64 deletions(-)
 create mode 100644 hw/cpu/icc_bus.c
 create mode 100644 include/hw/i386/icc_bus.h
 create mode 100644 stubs/cpus.c
Signature · 1 line
1.8.2
#1 lore raw

PATCH v4 1/16 cpu: make kvm-stub.o a part of CPU library

---
v2:
remove unnecessary kvm-stub.o from "System emulator target" section

---
 Makefile.target      | 14 +++++++-------
 include/hw/pci/msi.h |  2 ++
 include/sysemu/kvm.h |  4 ++--
 kvm-stub.c           |  2 ++
 4 files changed, 13 insertions(+), 9 deletions(-)
Makefile.target +7 -7
index 2bd6d14..9bde8e5 100644
@@ -64,6 +64,12 @@ all: $(PROGS) stap
 # Dummy command so that make thinks it has done something
 	@true
 
+CONFIG_NO_PCI = $(if $(subst n,,$(CONFIG_PCI)),n,y)
+CONFIG_NO_KVM = $(if $(subst n,,$(CONFIG_KVM)),n,y)
+CONFIG_NO_XEN = $(if $(subst n,,$(CONFIG_XEN)),n,y)
+CONFIG_NO_GET_MEMORY_MAPPING = $(if $(subst n,,$(CONFIG_HAVE_GET_MEMORY_MAPPING)),n,y)
+CONFIG_NO_CORE_DUMP = $(if $(subst n,,$(CONFIG_HAVE_CORE_DUMP)),n,y)
+
 #########################################################
 # cpu emulator library
 obj-y = exec.o translate-all.o cpu-exec.o
@@ -74,6 +80,7 @@ obj-y += fpu/softfloat.o
 obj-y += target-$(TARGET_BASE_ARCH)/
 obj-y += disas.o
 obj-$(CONFIG_GDBSTUB_XML) += gdbstub-xml.o
+obj-$(CONFIG_NO_KVM) += kvm-stub.o
 
 #########################################################
 # Linux user emulator target
@@ -102,18 +109,11 @@ endif #CONFIG_BSD_USER
 #########################################################
 # System emulator target
 ifdef CONFIG_SOFTMMU
-CONFIG_NO_PCI = $(if $(subst n,,$(CONFIG_PCI)),n,y)
-CONFIG_NO_KVM = $(if $(subst n,,$(CONFIG_KVM)),n,y)
-CONFIG_NO_XEN = $(if $(subst n,,$(CONFIG_XEN)),n,y)
-CONFIG_NO_GET_MEMORY_MAPPING = $(if $(subst n,,$(CONFIG_HAVE_GET_MEMORY_MAPPING)),n,y)
-CONFIG_NO_CORE_DUMP = $(if $(subst n,,$(CONFIG_HAVE_CORE_DUMP)),n,y)
-
 obj-y += arch_init.o cpus.o monitor.o gdbstub.o balloon.o ioport.o
 obj-y += qtest.o
 obj-y += hw/
 obj-$(CONFIG_FDT) += device_tree.o
 obj-$(CONFIG_KVM) += kvm-all.o
-obj-$(CONFIG_NO_KVM) += kvm-stub.o
 obj-y += memory.o savevm.o cputlb.o
 obj-$(CONFIG_HAVE_GET_MEMORY_MAPPING) += memory_mapping.o
 obj-$(CONFIG_HAVE_CORE_DUMP) += dump.o
include/hw/pci/msi.h +2 -0
index 81a3848..d4d0450 100644
@@ -21,6 +21,7 @@
 #ifndef QEMU_MSI_H
 #define QEMU_MSI_H
 
+#ifndef CONFIG_USER_ONLY
 #include "qemu-common.h"
 #include "hw/pci/pci.h"
 
@@ -47,4 +48,5 @@ static inline bool msi_present(const PCIDevice *dev)
     return dev->cap_present & QEMU_PCI_CAP_MSI;
 }
 
+#endif
 #endif /* QEMU_MSI_H */
include/sysemu/kvm.h +2 -2
index 495e6f8..93cef28 100644
@@ -144,10 +144,10 @@ int kvm_cpu_exec(CPUArchState *env);
 #if !defined(CONFIG_USER_ONLY)
 void *kvm_vmalloc(ram_addr_t size);
 void *kvm_arch_vmalloc(ram_addr_t size);
-void kvm_setup_guest_memory(void *start, size_t size);
+#endif
 
+void kvm_setup_guest_memory(void *start, size_t size);
 void kvm_flush_coalesced_mmio_buffer(void);
-#endif
 
 int kvm_insert_breakpoint(CPUArchState *current_env, target_ulong addr,
                           target_ulong len, int type);
kvm-stub.c +2 -0
index 3e5c201..a131305 100644
@@ -122,6 +122,7 @@ int kvm_on_sigbus(int code, void *addr)
     return 1;
 }
 
+#ifndef CONFIG_USER_ONLY
 int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg)
 {
     return -ENOSYS;
@@ -145,3 +146,4 @@ int kvm_irqchip_remove_irqfd_notifier(KVMState *s, EventNotifier *n, int virq)
 {
     return -ENOSYS;
 }
+#endif
Signature · 1 line
1.8.2

A message that is not in the archive (sent off-list or not yet received): [email protected]

#3 lore raw

Re: [Qemu-devel] [PATCH 01/16] cpu: make kvm-stub.o a part of CPU library

Quoted 10 lines

include/hw/pci/msi.h static inline bool msi_present(const PCIDevice *dev) · line 51

Quoted 10 lines
include/hw/pci/msi.h +2 -0
+#ifndef CONFIG_USER_ONLY
 #include "qemu-common.h"
 #include "hw/pci/pci.h"
@@ -47,4 +48,5 @@ static inline bool msi_present(const PCIDevice *dev)
     return dev->cap_present & QEMU_PCI_CAP_MSI;
 }
 
+#endif

Please put the #ifndef in the .c file instead.

Paolo

Quoted 44 lines
Quoted 37 lines
include/hw/pci/msi.h +0 -0
 #endif /* QEMU_MSI_H */
include/sysemu/kvm.h +2 -2
index 495e6f8..93cef28 100644
@@ -144,10 +144,10 @@ int kvm_cpu_exec(CPUArchState *env);
 #if !defined(CONFIG_USER_ONLY)
 void *kvm_vmalloc(ram_addr_t size);
 void *kvm_arch_vmalloc(ram_addr_t size);
-void kvm_setup_guest_memory(void *start, size_t size);
+#endif
 
+void kvm_setup_guest_memory(void *start, size_t size);
 void kvm_flush_coalesced_mmio_buffer(void);
-#endif
 
 int kvm_insert_breakpoint(CPUArchState *current_env, target_ulong addr,
                           target_ulong len, int type);
kvm-stub.c +2 -1
index 3e5c201..a131305 100644
@@ -122,6 +122,7 @@ int kvm_on_sigbus(int code, void *addr)
     return 1;
 }
 
+#ifndef CONFIG_USER_ONLY
 int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg)
 {
     return -ENOSYS;
@@ -145,3 +146,4 @@ int kvm_irqchip_remove_irqfd_notifier(KVMState *s,
 EventNotifier *n, int virq)
 {
     return -ENOSYS;
 }
+#endif
--
Quoted 3 lines

1.8.2

--
Eduardo

#4 lore raw

PATCH v2 1/16 cpu: make kvm-stub.o a part of CPU library

---
v3:
put ifndef around msi.h in kvm-stub.c instead of in msi.h

v2:
remove unnecessary kvm-stub.o from "System emulator target" section

---
 Makefile.target      | 14 +++++++-------
 include/sysemu/kvm.h |  4 ++--
 kvm-stub.c           |  7 ++++++-
 3 files changed, 15 insertions(+), 10 deletions(-)
Makefile.target +7 -7
index 2bd6d14..9bde8e5 100644
@@ -64,6 +64,12 @@ all: $(PROGS) stap
 # Dummy command so that make thinks it has done something
 	@true
 
+CONFIG_NO_PCI = $(if $(subst n,,$(CONFIG_PCI)),n,y)
+CONFIG_NO_KVM = $(if $(subst n,,$(CONFIG_KVM)),n,y)
+CONFIG_NO_XEN = $(if $(subst n,,$(CONFIG_XEN)),n,y)
+CONFIG_NO_GET_MEMORY_MAPPING = $(if $(subst n,,$(CONFIG_HAVE_GET_MEMORY_MAPPING)),n,y)
+CONFIG_NO_CORE_DUMP = $(if $(subst n,,$(CONFIG_HAVE_CORE_DUMP)),n,y)
+
 #########################################################
 # cpu emulator library
 obj-y = exec.o translate-all.o cpu-exec.o
@@ -74,6 +80,7 @@ obj-y += fpu/softfloat.o
 obj-y += target-$(TARGET_BASE_ARCH)/
 obj-y += disas.o
 obj-$(CONFIG_GDBSTUB_XML) += gdbstub-xml.o
+obj-$(CONFIG_NO_KVM) += kvm-stub.o
 
 #########################################################
 # Linux user emulator target
@@ -102,18 +109,11 @@ endif #CONFIG_BSD_USER
 #########################################################
 # System emulator target
 ifdef CONFIG_SOFTMMU
-CONFIG_NO_PCI = $(if $(subst n,,$(CONFIG_PCI)),n,y)
-CONFIG_NO_KVM = $(if $(subst n,,$(CONFIG_KVM)),n,y)
-CONFIG_NO_XEN = $(if $(subst n,,$(CONFIG_XEN)),n,y)
-CONFIG_NO_GET_MEMORY_MAPPING = $(if $(subst n,,$(CONFIG_HAVE_GET_MEMORY_MAPPING)),n,y)
-CONFIG_NO_CORE_DUMP = $(if $(subst n,,$(CONFIG_HAVE_CORE_DUMP)),n,y)
-
 obj-y += arch_init.o cpus.o monitor.o gdbstub.o balloon.o ioport.o
 obj-y += qtest.o
 obj-y += hw/
 obj-$(CONFIG_FDT) += device_tree.o
 obj-$(CONFIG_KVM) += kvm-all.o
-obj-$(CONFIG_NO_KVM) += kvm-stub.o
 obj-y += memory.o savevm.o cputlb.o
 obj-$(CONFIG_HAVE_GET_MEMORY_MAPPING) += memory_mapping.o
 obj-$(CONFIG_HAVE_CORE_DUMP) += dump.o
include/sysemu/kvm.h +2 -2
index 495e6f8..93cef28 100644
@@ -144,10 +144,10 @@ int kvm_cpu_exec(CPUArchState *env);
 #if !defined(CONFIG_USER_ONLY)
 void *kvm_vmalloc(ram_addr_t size);
 void *kvm_arch_vmalloc(ram_addr_t size);
-void kvm_setup_guest_memory(void *start, size_t size);
+#endif
 
+void kvm_setup_guest_memory(void *start, size_t size);
 void kvm_flush_coalesced_mmio_buffer(void);
-#endif
 
 int kvm_insert_breakpoint(CPUArchState *current_env, target_ulong addr,
                           target_ulong len, int type);
kvm-stub.c +6 -1
index 3e5c201..e5d2906 100644
@@ -12,11 +12,14 @@
 
 #include "qemu-common.h"
 #include "hw/hw.h"
-#include "hw/pci/msi.h"
 #include "cpu.h"
 #include "exec/gdbstub.h"
 #include "sysemu/kvm.h"
 
+#ifndef CONFIG_USER_ONLY
+#include "hw/pci/msi.h"
+#endif
+
 KVMState *kvm_state;
 bool kvm_kernel_irqchip;
 bool kvm_async_interrupts_allowed;
@@ -122,6 +125,7 @@ int kvm_on_sigbus(int code, void *addr)
     return 1;
 }
 
+#ifndef CONFIG_USER_ONLY
 int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg)
 {
     return -ENOSYS;
@@ -145,3 +149,4 @@ int kvm_irqchip_remove_irqfd_notifier(KVMState *s, EventNotifier *n, int virq)
 {
     return -ENOSYS;
 }
+#endif
Signature · 1 line
1.8.2
Reviewed-by on v2 1/16
#5 lore raw

Re: [Qemu-devel] [PATCH 01/16 v2] cpu: make kvm-stub.o a part of CPU library

Quoted 1 line · Igor Mammedov
Quoted 109 lines

---
v3:
put ifndef around msi.h in kvm-stub.c instead of in msi.h

v2:
remove unnecessary kvm-stub.o from "System emulator target" section

---
 Makefile.target      | 14 +++++++-------
 include/sysemu/kvm.h |  4 ++--
 kvm-stub.c           |  7 ++++++-
 3 files changed, 15 insertions(+), 10 deletions(-)
Makefile.target +7 -7
index 2bd6d14..9bde8e5 100644
@@ -64,6 +64,12 @@ all: $(PROGS) stap
 # Dummy command so that make thinks it has done something
 	@true
 
+CONFIG_NO_PCI = $(if $(subst n,,$(CONFIG_PCI)),n,y)
+CONFIG_NO_KVM = $(if $(subst n,,$(CONFIG_KVM)),n,y)
+CONFIG_NO_XEN = $(if $(subst n,,$(CONFIG_XEN)),n,y)
+CONFIG_NO_GET_MEMORY_MAPPING = $(if $(subst n,,$(CONFIG_HAVE_GET_MEMORY_MAPPING)),n,y)
+CONFIG_NO_CORE_DUMP = $(if $(subst n,,$(CONFIG_HAVE_CORE_DUMP)),n,y)
+
 #########################################################
 # cpu emulator library
 obj-y = exec.o translate-all.o cpu-exec.o
@@ -74,6 +80,7 @@ obj-y += fpu/softfloat.o
 obj-y += target-$(TARGET_BASE_ARCH)/
 obj-y += disas.o
 obj-$(CONFIG_GDBSTUB_XML) += gdbstub-xml.o
+obj-$(CONFIG_NO_KVM) += kvm-stub.o
 
 #########################################################
 # Linux user emulator target
@@ -102,18 +109,11 @@ endif #CONFIG_BSD_USER
 #########################################################
 # System emulator target
 ifdef CONFIG_SOFTMMU
-CONFIG_NO_PCI = $(if $(subst n,,$(CONFIG_PCI)),n,y)
-CONFIG_NO_KVM = $(if $(subst n,,$(CONFIG_KVM)),n,y)
-CONFIG_NO_XEN = $(if $(subst n,,$(CONFIG_XEN)),n,y)
-CONFIG_NO_GET_MEMORY_MAPPING = $(if $(subst n,,$(CONFIG_HAVE_GET_MEMORY_MAPPING)),n,y)
-CONFIG_NO_CORE_DUMP = $(if $(subst n,,$(CONFIG_HAVE_CORE_DUMP)),n,y)
-
 obj-y += arch_init.o cpus.o monitor.o gdbstub.o balloon.o ioport.o
 obj-y += qtest.o
 obj-y += hw/
 obj-$(CONFIG_FDT) += device_tree.o
 obj-$(CONFIG_KVM) += kvm-all.o
-obj-$(CONFIG_NO_KVM) += kvm-stub.o
 obj-y += memory.o savevm.o cputlb.o
 obj-$(CONFIG_HAVE_GET_MEMORY_MAPPING) += memory_mapping.o
 obj-$(CONFIG_HAVE_CORE_DUMP) += dump.o
include/sysemu/kvm.h +2 -2
index 495e6f8..93cef28 100644
@@ -144,10 +144,10 @@ int kvm_cpu_exec(CPUArchState *env);
 #if !defined(CONFIG_USER_ONLY)
 void *kvm_vmalloc(ram_addr_t size);
 void *kvm_arch_vmalloc(ram_addr_t size);
-void kvm_setup_guest_memory(void *start, size_t size);
+#endif
 
+void kvm_setup_guest_memory(void *start, size_t size);
 void kvm_flush_coalesced_mmio_buffer(void);
-#endif
 
 int kvm_insert_breakpoint(CPUArchState *current_env, target_ulong addr,
                           target_ulong len, int type);
kvm-stub.c +6 -1
index 3e5c201..e5d2906 100644
@@ -12,11 +12,14 @@
 
 #include "qemu-common.h"
 #include "hw/hw.h"
-#include "hw/pci/msi.h"
 #include "cpu.h"
 #include "exec/gdbstub.h"
 #include "sysemu/kvm.h"
 
+#ifndef CONFIG_USER_ONLY
+#include "hw/pci/msi.h"
+#endif
+
 KVMState *kvm_state;
 bool kvm_kernel_irqchip;
 bool kvm_async_interrupts_allowed;
@@ -122,6 +125,7 @@ int kvm_on_sigbus(int code, void *addr)
     return 1;
 }
 
+#ifndef CONFIG_USER_ONLY
 int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg)
 {
     return -ENOSYS;
@@ -145,3 +149,4 @@ int kvm_irqchip_remove_irqfd_notifier(KVMState *s, EventNotifier *n, int virq)
 {
     return -ENOSYS;
 }
+#endif
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
#6 lore raw

Re: [Qemu-devel] [PATCH 01/16 v2] cpu: make kvm-stub.o a part of CPU library

Am 18.04.2013 16:24, schrieb Eduardo Habkost:

Quoted 4 lines · Igor Mammedov
Quoted 1 line

Looks good to me, too.

Paolo and Gleb, are you okay with this disguised v5?

Andreas

Quoted 110 lines
Quoted 109 lines

---
v3:
put ifndef around msi.h in kvm-stub.c instead of in msi.h

v2:
remove unnecessary kvm-stub.o from "System emulator target" section

---
 Makefile.target      | 14 +++++++-------
 include/sysemu/kvm.h |  4 ++--
 kvm-stub.c           |  7 ++++++-
 3 files changed, 15 insertions(+), 10 deletions(-)
Makefile.target +7 -7
index 2bd6d14..9bde8e5 100644
@@ -64,6 +64,12 @@ all: $(PROGS) stap
 # Dummy command so that make thinks it has done something
 	@true
 
+CONFIG_NO_PCI = $(if $(subst n,,$(CONFIG_PCI)),n,y)
+CONFIG_NO_KVM = $(if $(subst n,,$(CONFIG_KVM)),n,y)
+CONFIG_NO_XEN = $(if $(subst n,,$(CONFIG_XEN)),n,y)
+CONFIG_NO_GET_MEMORY_MAPPING = $(if $(subst n,,$(CONFIG_HAVE_GET_MEMORY_MAPPING)),n,y)
+CONFIG_NO_CORE_DUMP = $(if $(subst n,,$(CONFIG_HAVE_CORE_DUMP)),n,y)
+
 #########################################################
 # cpu emulator library
 obj-y = exec.o translate-all.o cpu-exec.o
@@ -74,6 +80,7 @@ obj-y += fpu/softfloat.o
 obj-y += target-$(TARGET_BASE_ARCH)/
 obj-y += disas.o
 obj-$(CONFIG_GDBSTUB_XML) += gdbstub-xml.o
+obj-$(CONFIG_NO_KVM) += kvm-stub.o
 
 #########################################################
 # Linux user emulator target
@@ -102,18 +109,11 @@ endif #CONFIG_BSD_USER
 #########################################################
 # System emulator target
 ifdef CONFIG_SOFTMMU
-CONFIG_NO_PCI = $(if $(subst n,,$(CONFIG_PCI)),n,y)
-CONFIG_NO_KVM = $(if $(subst n,,$(CONFIG_KVM)),n,y)
-CONFIG_NO_XEN = $(if $(subst n,,$(CONFIG_XEN)),n,y)
-CONFIG_NO_GET_MEMORY_MAPPING = $(if $(subst n,,$(CONFIG_HAVE_GET_MEMORY_MAPPING)),n,y)
-CONFIG_NO_CORE_DUMP = $(if $(subst n,,$(CONFIG_HAVE_CORE_DUMP)),n,y)
-
 obj-y += arch_init.o cpus.o monitor.o gdbstub.o balloon.o ioport.o
 obj-y += qtest.o
 obj-y += hw/
 obj-$(CONFIG_FDT) += device_tree.o
 obj-$(CONFIG_KVM) += kvm-all.o
-obj-$(CONFIG_NO_KVM) += kvm-stub.o
 obj-y += memory.o savevm.o cputlb.o
 obj-$(CONFIG_HAVE_GET_MEMORY_MAPPING) += memory_mapping.o
 obj-$(CONFIG_HAVE_CORE_DUMP) += dump.o
include/sysemu/kvm.h +2 -2
index 495e6f8..93cef28 100644
@@ -144,10 +144,10 @@ int kvm_cpu_exec(CPUArchState *env);
 #if !defined(CONFIG_USER_ONLY)
 void *kvm_vmalloc(ram_addr_t size);
 void *kvm_arch_vmalloc(ram_addr_t size);
-void kvm_setup_guest_memory(void *start, size_t size);
+#endif
 
+void kvm_setup_guest_memory(void *start, size_t size);
 void kvm_flush_coalesced_mmio_buffer(void);
-#endif
 
 int kvm_insert_breakpoint(CPUArchState *current_env, target_ulong addr,
                           target_ulong len, int type);
kvm-stub.c +6 -1
index 3e5c201..e5d2906 100644
@@ -12,11 +12,14 @@
 
 #include "qemu-common.h"
 #include "hw/hw.h"
-#include "hw/pci/msi.h"
 #include "cpu.h"
 #include "exec/gdbstub.h"
 #include "sysemu/kvm.h"
 
+#ifndef CONFIG_USER_ONLY
+#include "hw/pci/msi.h"
+#endif
+
 KVMState *kvm_state;
 bool kvm_kernel_irqchip;
 bool kvm_async_interrupts_allowed;
@@ -122,6 +125,7 @@ int kvm_on_sigbus(int code, void *addr)
     return 1;
 }
 
+#ifndef CONFIG_USER_ONLY
 int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg)
 {
     return -ENOSYS;
@@ -145,3 +149,4 @@ int kvm_irqchip_remove_irqfd_notifier(KVMState *s, EventNotifier *n, int virq)
 {
     return -ENOSYS;
 }
+#endif
Signature · 1 line
1.8.2
Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#7 lore raw

Re: [Qemu-devel] [PATCH 01/16 v2] cpu: make kvm-stub.o a part of CPU library

Il 22/04/2013 15:45, Andreas Färber ha scritto:

Quoted 8 lines

Am 18.04.2013 16:24, schrieb Eduardo Habkost:

Quoted 4 lines
Quoted 4 lines · Igor Mammedov
Quoted 1 line
Quoted 1 line

Looks good to me, too.

Paolo and Gleb, are you okay with this disguised v5?

I am fine with it too.

Paolo

#8 lore raw

Re: [Qemu-devel] [PATCH 01/16 v2] cpu: make kvm-stub.o a part of CPU library

Quoted 12 lines · Paolo Bonzini

Il 22/04/2013 15:45, Andreas Färber ha scritto:

Quoted 8 lines

Am 18.04.2013 16:24, schrieb Eduardo Habkost:

Quoted 4 lines
Quoted 4 lines · Igor Mammedov
Quoted 1 line
Quoted 1 line

Looks good to me, too.

Paolo and Gleb, are you okay with this disguised v5?

I am fine with it too.

Same here.

--
Gleb.

#9 lore raw

PATCH v4 2/16 cpu: call cpu_synchronize_post_init() from CPUClass.realize() if hotplugged

In reply to Igor Mammedov

... to synchronize CPU state to KVM

---
v2:
* linking kvm-stub.o to *-user target moved in separate patch

---
 include/sysemu/kvm.h | 18 ++++++++++--------
 kvm-all.c            |  1 +
 kvm-stub.c           |  1 +
 qom/cpu.c            |  4 ++++
 vl.c                 |  1 -
 5 files changed, 16 insertions(+), 9 deletions(-)
include/sysemu/kvm.h +10 -8
index 93cef28..8fb2489 100644
@@ -250,8 +250,6 @@ int kvm_check_extension(KVMState *s, unsigned int extension);
 uint32_t kvm_arch_get_supported_cpuid(KVMState *env, uint32_t function,
                                       uint32_t index, int reg);
 void kvm_cpu_synchronize_state(CPUArchState *env);
-void kvm_cpu_synchronize_post_reset(CPUState *cpu);
-void kvm_cpu_synchronize_post_init(CPUState *cpu);
 
 /* generic hooks - to be moved/refactored once there are more users */
 
@@ -262,6 +260,16 @@ static inline void cpu_synchronize_state(CPUArchState *env)
     }
 }
 
+#if !defined(CONFIG_USER_ONLY)
+int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,
+                                       hwaddr *phys_addr);
+#endif
+
+#endif /* NEED_CPU_H */
+
+void kvm_cpu_synchronize_post_reset(CPUState *cpu);
+void kvm_cpu_synchronize_post_init(CPUState *cpu);
+
 static inline void cpu_synchronize_post_reset(CPUState *cpu)
 {
     if (kvm_enabled()) {
@@ -277,12 +285,6 @@ static inline void cpu_synchronize_post_init(CPUState *cpu)
 }
 
 
-#if !defined(CONFIG_USER_ONLY)
-int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,
-                                       hwaddr *phys_addr);
-#endif
-
-#endif
 int kvm_set_ioeventfd_mmio(int fd, uint32_t adr, uint32_t val, bool assign,
                            uint32_t size);
 
kvm-all.c +1 -0
index fc4e17c..1d17128 100644
@@ -109,6 +109,7 @@ bool kvm_async_interrupts_allowed;
 bool kvm_irqfds_allowed;
 bool kvm_msi_via_irqfd_allowed;
 bool kvm_gsi_routing_allowed;
+bool kvm_allowed;
 
 static const KVMCapabilityInfo kvm_required_capabilites[] = {
     KVM_CAP_INFO(USER_MEMORY),
kvm-stub.c +1 -0
index a131305..4282e09 100644
@@ -23,6 +23,7 @@ bool kvm_async_interrupts_allowed;
 bool kvm_irqfds_allowed;
 bool kvm_msi_via_irqfd_allowed;
 bool kvm_gsi_routing_allowed;
+bool kvm_allowed;
 
 int kvm_init_vcpu(CPUState *cpu)
 {
qom/cpu.c +4 -0
index e242dcb..0c76712 100644
@@ -20,6 +20,7 @@
 
 #include "qom/cpu.h"
 #include "qemu-common.h"
+#include "sysemu/kvm.h"
 
 void cpu_reset_interrupt(CPUState *cpu, int mask)
 {
@@ -57,6 +58,9 @@ static ObjectClass *cpu_common_class_by_name(const char *cpu_model)
 
 static void cpu_common_realizefn(DeviceState *dev, Error **errp)
 {
+    if (dev->hotplugged) {
+        cpu_synchronize_post_init(CPU(dev));
+    }
 }
 
 static void cpu_class_init(ObjectClass *klass, void *data)
vl.c +0 -1
index dd94549..bc9c016 100644
@@ -267,7 +267,6 @@ static NotifierList machine_init_done_notifiers =
     NOTIFIER_LIST_INITIALIZER(machine_init_done_notifiers);
 
 static bool tcg_allowed = true;
-bool kvm_allowed;
 bool xen_allowed;
 uint32_t xen_domid;
 enum xen_mode xen_mode = XEN_EMULATE;
Signature · 1 line
1.8.2
Reviewed-by on v4 2/16
#10 lore raw

Re: [Qemu-devel] [PATCH 02/16] cpu: call cpu_synchronize_post_init() from CPUClass.realize() if hotplugged

Quoted 3 lines · Igor Mammedov

... to synchronize CPU state to KVM

Quoted 116 lines

---
v2:
* linking kvm-stub.o to *-user target moved in separate patch

---
 include/sysemu/kvm.h | 18 ++++++++++--------
 kvm-all.c            |  1 +
 kvm-stub.c           |  1 +
 qom/cpu.c            |  4 ++++
 vl.c                 |  1 -
 5 files changed, 16 insertions(+), 9 deletions(-)
include/sysemu/kvm.h +10 -8
index 93cef28..8fb2489 100644
@@ -250,8 +250,6 @@ int kvm_check_extension(KVMState *s, unsigned int extension);
 uint32_t kvm_arch_get_supported_cpuid(KVMState *env, uint32_t function,
                                       uint32_t index, int reg);
 void kvm_cpu_synchronize_state(CPUArchState *env);
-void kvm_cpu_synchronize_post_reset(CPUState *cpu);
-void kvm_cpu_synchronize_post_init(CPUState *cpu);
 
 /* generic hooks - to be moved/refactored once there are more users */
 
@@ -262,6 +260,16 @@ static inline void cpu_synchronize_state(CPUArchState *env)
     }
 }
 
+#if !defined(CONFIG_USER_ONLY)
+int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,
+                                       hwaddr *phys_addr);
+#endif
+
+#endif /* NEED_CPU_H */
+
+void kvm_cpu_synchronize_post_reset(CPUState *cpu);
+void kvm_cpu_synchronize_post_init(CPUState *cpu);
+
 static inline void cpu_synchronize_post_reset(CPUState *cpu)
 {
     if (kvm_enabled()) {
@@ -277,12 +285,6 @@ static inline void cpu_synchronize_post_init(CPUState *cpu)
 }
 
 
-#if !defined(CONFIG_USER_ONLY)
-int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,
-                                       hwaddr *phys_addr);
-#endif
-
-#endif
 int kvm_set_ioeventfd_mmio(int fd, uint32_t adr, uint32_t val, bool assign,
                            uint32_t size);
 
kvm-all.c +1 -0
index fc4e17c..1d17128 100644
@@ -109,6 +109,7 @@ bool kvm_async_interrupts_allowed;
 bool kvm_irqfds_allowed;
 bool kvm_msi_via_irqfd_allowed;
 bool kvm_gsi_routing_allowed;
+bool kvm_allowed;
 
 static const KVMCapabilityInfo kvm_required_capabilites[] = {
     KVM_CAP_INFO(USER_MEMORY),
kvm-stub.c +1 -0
index a131305..4282e09 100644
@@ -23,6 +23,7 @@ bool kvm_async_interrupts_allowed;
 bool kvm_irqfds_allowed;
 bool kvm_msi_via_irqfd_allowed;
 bool kvm_gsi_routing_allowed;
+bool kvm_allowed;
 
 int kvm_init_vcpu(CPUState *cpu)
 {
qom/cpu.c +4 -0
index e242dcb..0c76712 100644
@@ -20,6 +20,7 @@
 
 #include "qom/cpu.h"
 #include "qemu-common.h"
+#include "sysemu/kvm.h"
 
 void cpu_reset_interrupt(CPUState *cpu, int mask)
 {
@@ -57,6 +58,9 @@ static ObjectClass *cpu_common_class_by_name(const char *cpu_model)
 
 static void cpu_common_realizefn(DeviceState *dev, Error **errp)
 {
+    if (dev->hotplugged) {
+        cpu_synchronize_post_init(CPU(dev));
+    }
 }
 
 static void cpu_class_init(ObjectClass *klass, void *data)
vl.c +0 -1
index dd94549..bc9c016 100644
@@ -267,7 +267,6 @@ static NotifierList machine_init_done_notifiers =
     NOTIFIER_LIST_INITIALIZER(machine_init_done_notifiers);
 
 static bool tcg_allowed = true;
-bool kvm_allowed;
 bool xen_allowed;
 uint32_t xen_domid;
 enum xen_mode xen_mode = XEN_EMULATE;
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
#11 lore raw

Re: [Qemu-devel] [PATCH 02/16] cpu: call cpu_synchronize_post_init() from CPUClass.realize() if hotplugged

In reply to Igor Mammedov

Am 16.04.2013 00:12, schrieb Igor Mammedov:

Quoted 6 lines

... to synchronize CPU state to KVM

---
v2:
* linking kvm-stub.o to *-user target moved in separate patch

Quoted 7 lines
---
 include/sysemu/kvm.h | 18 ++++++++++--------
 kvm-all.c            |  1 +
 kvm-stub.c           |  1 +
 qom/cpu.c            |  4 ++++
 vl.c                 |  1 -
 5 files changed, 16 insertions(+), 9 deletions(-)

Looks okay to me. Gleb, can you ack?

Andreas

Quoted 102 lines
include/sysemu/kvm.h +10 -8
index 93cef28..8fb2489 100644
@@ -250,8 +250,6 @@ int kvm_check_extension(KVMState *s, unsigned int extension);
 uint32_t kvm_arch_get_supported_cpuid(KVMState *env, uint32_t function,
                                       uint32_t index, int reg);
 void kvm_cpu_synchronize_state(CPUArchState *env);
-void kvm_cpu_synchronize_post_reset(CPUState *cpu);
-void kvm_cpu_synchronize_post_init(CPUState *cpu);
 
 /* generic hooks - to be moved/refactored once there are more users */
 
@@ -262,6 +260,16 @@ static inline void cpu_synchronize_state(CPUArchState *env)
     }
 }
 
+#if !defined(CONFIG_USER_ONLY)
+int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,
+                                       hwaddr *phys_addr);
+#endif
+
+#endif /* NEED_CPU_H */
+
+void kvm_cpu_synchronize_post_reset(CPUState *cpu);
+void kvm_cpu_synchronize_post_init(CPUState *cpu);
+
 static inline void cpu_synchronize_post_reset(CPUState *cpu)
 {
     if (kvm_enabled()) {
@@ -277,12 +285,6 @@ static inline void cpu_synchronize_post_init(CPUState *cpu)
 }
 
 
-#if !defined(CONFIG_USER_ONLY)
-int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,
-                                       hwaddr *phys_addr);
-#endif
-
-#endif
 int kvm_set_ioeventfd_mmio(int fd, uint32_t adr, uint32_t val, bool assign,
                            uint32_t size);
 
kvm-all.c +1 -0
index fc4e17c..1d17128 100644
@@ -109,6 +109,7 @@ bool kvm_async_interrupts_allowed;
 bool kvm_irqfds_allowed;
 bool kvm_msi_via_irqfd_allowed;
 bool kvm_gsi_routing_allowed;
+bool kvm_allowed;
 
 static const KVMCapabilityInfo kvm_required_capabilites[] = {
     KVM_CAP_INFO(USER_MEMORY),
kvm-stub.c +1 -0
index a131305..4282e09 100644
@@ -23,6 +23,7 @@ bool kvm_async_interrupts_allowed;
 bool kvm_irqfds_allowed;
 bool kvm_msi_via_irqfd_allowed;
 bool kvm_gsi_routing_allowed;
+bool kvm_allowed;
 
 int kvm_init_vcpu(CPUState *cpu)
 {
qom/cpu.c +4 -0
index e242dcb..0c76712 100644
@@ -20,6 +20,7 @@
 
 #include "qom/cpu.h"
 #include "qemu-common.h"
+#include "sysemu/kvm.h"
 
 void cpu_reset_interrupt(CPUState *cpu, int mask)
 {
@@ -57,6 +58,9 @@ static ObjectClass *cpu_common_class_by_name(const char *cpu_model)
 
 static void cpu_common_realizefn(DeviceState *dev, Error **errp)
 {
+    if (dev->hotplugged) {
+        cpu_synchronize_post_init(CPU(dev));
+    }
 }
 
 static void cpu_class_init(ObjectClass *klass, void *data)
vl.c +0 -1
index dd94549..bc9c016 100644
@@ -267,7 +267,6 @@ static NotifierList machine_init_done_notifiers =
     NOTIFIER_LIST_INITIALIZER(machine_init_done_notifiers);
 
 static bool tcg_allowed = true;
-bool kvm_allowed;
 bool xen_allowed;
 uint32_t xen_domid;
 enum xen_mode xen_mode = XEN_EMULATE;
Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#12 lore raw

Re: [Qemu-devel] [PATCH 02/16] cpu: call cpu_synchronize_post_init() from CPUClass.realize() if hotplugged

Quoted 15 lines · Andreas Färber

Am 16.04.2013 00:12, schrieb Igor Mammedov:

Quoted 6 lines

... to synchronize CPU state to KVM

---
v2:
* linking kvm-stub.o to *-user target moved in separate patch

Quoted 7 lines
---
 include/sysemu/kvm.h | 18 ++++++++++--------
 kvm-all.c            |  1 +
 kvm-stub.c           |  1 +
 qom/cpu.c            |  4 ++++
 vl.c                 |  1 -
 5 files changed, 16 insertions(+), 9 deletions(-)
Quoted 2 lines

Looks okay to me. Gleb, can you ack?

Yes, looks good to me.

Quoted 98 lines

Andreas

Quoted 96 lines
include/sysemu/kvm.h +10 -8
index 93cef28..8fb2489 100644
@@ -250,8 +250,6 @@ int kvm_check_extension(KVMState *s, unsigned int extension);
 uint32_t kvm_arch_get_supported_cpuid(KVMState *env, uint32_t function,
                                       uint32_t index, int reg);
 void kvm_cpu_synchronize_state(CPUArchState *env);
-void kvm_cpu_synchronize_post_reset(CPUState *cpu);
-void kvm_cpu_synchronize_post_init(CPUState *cpu);
 
 /* generic hooks - to be moved/refactored once there are more users */
 
@@ -262,6 +260,16 @@ static inline void cpu_synchronize_state(CPUArchState *env)
     }
 }
 
+#if !defined(CONFIG_USER_ONLY)
+int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,
+                                       hwaddr *phys_addr);
+#endif
+
+#endif /* NEED_CPU_H */
+
+void kvm_cpu_synchronize_post_reset(CPUState *cpu);
+void kvm_cpu_synchronize_post_init(CPUState *cpu);
+
 static inline void cpu_synchronize_post_reset(CPUState *cpu)
 {
     if (kvm_enabled()) {
@@ -277,12 +285,6 @@ static inline void cpu_synchronize_post_init(CPUState *cpu)
 }
 
 
-#if !defined(CONFIG_USER_ONLY)
-int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,
-                                       hwaddr *phys_addr);
-#endif
-
-#endif
 int kvm_set_ioeventfd_mmio(int fd, uint32_t adr, uint32_t val, bool assign,
                            uint32_t size);
 
kvm-all.c +1 -0
index fc4e17c..1d17128 100644
@@ -109,6 +109,7 @@ bool kvm_async_interrupts_allowed;
 bool kvm_irqfds_allowed;
 bool kvm_msi_via_irqfd_allowed;
 bool kvm_gsi_routing_allowed;
+bool kvm_allowed;
 
 static const KVMCapabilityInfo kvm_required_capabilites[] = {
     KVM_CAP_INFO(USER_MEMORY),
kvm-stub.c +1 -0
index a131305..4282e09 100644
@@ -23,6 +23,7 @@ bool kvm_async_interrupts_allowed;
 bool kvm_irqfds_allowed;
 bool kvm_msi_via_irqfd_allowed;
 bool kvm_gsi_routing_allowed;
+bool kvm_allowed;
 
 int kvm_init_vcpu(CPUState *cpu)
 {
qom/cpu.c +4 -0
index e242dcb..0c76712 100644
@@ -20,6 +20,7 @@
 
 #include "qom/cpu.h"
 #include "qemu-common.h"
+#include "sysemu/kvm.h"
 
 void cpu_reset_interrupt(CPUState *cpu, int mask)
 {
@@ -57,6 +58,9 @@ static ObjectClass *cpu_common_class_by_name(const char *cpu_model)
 
 static void cpu_common_realizefn(DeviceState *dev, Error **errp)
 {
+    if (dev->hotplugged) {
+        cpu_synchronize_post_init(CPU(dev));
+    }
 }
 
 static void cpu_class_init(ObjectClass *klass, void *data)
vl.c +0 -0
index dd94549..bc9c016 100644
@@ -267,7 +267,6 @@ static NotifierList machine_init_done_notifiers =
     NOTIFIER_LIST_INITIALIZER(machine_init_done_notifiers);
 
Quoted 11 lines
Quoted 6 lines
vl.c +0 -1
 static bool tcg_allowed = true;
-bool kvm_allowed;
 bool xen_allowed;
 uint32_t xen_domid;
 enum xen_mode xen_mode = XEN_EMULATE;
Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

--
Gleb.

#13 lore raw

PATCH v4 3/16 introduce resume_vcpu(), for single CPU

In reply to Igor Mammedov

... also add stub for it, to make possible to use it in qom/cpu.c, which is shared with user emulators.

---
Note:
it will be re-used from qom/cpu.c by hotpluged CPU

---
 cpus.c              | 11 ++++++++---
 include/qom/cpu.h   |  9 +++++++++
 stubs/Makefile.objs |  1 +
 stubs/cpus.c        |  5 +++++
 4 files changed, 23 insertions(+), 3 deletions(-)
 create mode 100644 stubs/cpus.c
cpus.c +8 -3
index c15ff6c..c347db5 100644
@@ -993,6 +993,13 @@ void pause_all_vcpus(void)
     }
 }
 
+void resume_vcpu(CPUState *cpu)
+{
+    cpu->stop = false;
+    cpu->stopped = false;
+    qemu_cpu_kick(cpu);
+}
+
 void resume_all_vcpus(void)
 {
     CPUArchState *penv = first_cpu;
@@ -1000,9 +1007,7 @@ void resume_all_vcpus(void)
     qemu_clock_enable(vm_clock, true);
     while (penv) {
         CPUState *pcpu = ENV_GET_CPU(penv);
-        pcpu->stop = false;
-        pcpu->stopped = false;
-        qemu_cpu_kick(pcpu);
+        resume_vcpu(pcpu);
         penv = penv->next_cpu;
     }
 }
include/qom/cpu.h +9 -0
index 3664a1b..6ce6f10 100644
@@ -256,5 +256,14 @@ void cpu_interrupt(CPUState *cpu, int mask);
  */
 void cpu_reset_interrupt(CPUState *cpu, int mask);
 
+/**
+ * resume_vcpu:
+ * @cpu: The CPU to resume.
+ *
+ * Resumes CPU, i.e. puts CPU into runable state.
+ * KVM requires extra call cpu_synchronize_post_init() to make
+ * KVM's CPU instance runable.
+ */
+void resume_vcpu(CPUState *cpu);
 
 #endif
stubs/Makefile.objs +1 -0
index 9c55b34..03dff20 100644
@@ -23,3 +23,4 @@ stub-obj-y += sysbus.o
 stub-obj-y += vm-stop.o
 stub-obj-y += vmstate.o
 stub-obj-$(CONFIG_WIN32) += fd-register.o
+stub-obj-y += cpus.o
stubs/cpus.c new file +5 -0
new file mode 100644
index 0000000..3d17451
@@ -0,0 +1,5 @@
+#include "qom/cpu.h"
+
+void resume_vcpu(CPUState *cpu)
+{
+}
Signature · 1 line
1.8.2
Reviewed-by on v4 3/16
#14 lore raw

Re: [Qemu-devel] [PATCH 03/16] introduce resume_vcpu(), for single CPU

Quoted 4 lines · Igor Mammedov

... also add stub for it, to make possible to use it in qom/cpu.c, which is shared with user emulators.

Quoted 83 lines

---
Note:
it will be re-used from qom/cpu.c by hotpluged CPU

---
 cpus.c              | 11 ++++++++---
 include/qom/cpu.h   |  9 +++++++++
 stubs/Makefile.objs |  1 +
 stubs/cpus.c        |  5 +++++
 4 files changed, 23 insertions(+), 3 deletions(-)
 create mode 100644 stubs/cpus.c
cpus.c +8 -3
index c15ff6c..c347db5 100644
@@ -993,6 +993,13 @@ void pause_all_vcpus(void)
     }
 }
 
+void resume_vcpu(CPUState *cpu)
+{
+    cpu->stop = false;
+    cpu->stopped = false;
+    qemu_cpu_kick(cpu);
+}
+
 void resume_all_vcpus(void)
 {
     CPUArchState *penv = first_cpu;
@@ -1000,9 +1007,7 @@ void resume_all_vcpus(void)
     qemu_clock_enable(vm_clock, true);
     while (penv) {
         CPUState *pcpu = ENV_GET_CPU(penv);
-        pcpu->stop = false;
-        pcpu->stopped = false;
-        qemu_cpu_kick(pcpu);
+        resume_vcpu(pcpu);
         penv = penv->next_cpu;
     }
 }
include/qom/cpu.h +9 -0
index 3664a1b..6ce6f10 100644
@@ -256,5 +256,14 @@ void cpu_interrupt(CPUState *cpu, int mask);
  */
 void cpu_reset_interrupt(CPUState *cpu, int mask);
 
+/**
+ * resume_vcpu:
+ * @cpu: The CPU to resume.
+ *
+ * Resumes CPU, i.e. puts CPU into runable state.
+ * KVM requires extra call cpu_synchronize_post_init() to make
+ * KVM's CPU instance runable.
+ */
+void resume_vcpu(CPUState *cpu);
 
 #endif
stubs/Makefile.objs +1 -0
index 9c55b34..03dff20 100644
@@ -23,3 +23,4 @@ stub-obj-y += sysbus.o
 stub-obj-y += vm-stop.o
 stub-obj-y += vmstate.o
 stub-obj-$(CONFIG_WIN32) += fd-register.o
+stub-obj-y += cpus.o
stubs/cpus.c new file +5 -0
new file mode 100644
index 0000000..3d17451
@@ -0,0 +1,5 @@
+#include "qom/cpu.h"
+
+void resume_vcpu(CPUState *cpu)
+{
+}
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
#15 lore raw

Re: [Qemu-devel] [PATCH 03/16] introduce resume_vcpu(), for single CPU

In reply to Igor Mammedov
Quoted 50 lines · Igor Mammedov

... also add stub for it, to make possible to use it in qom/cpu.c, which is shared with user emulators.

---
Note:
it will be re-used from qom/cpu.c by hotpluged CPU

---
 cpus.c              | 11 ++++++++---
 include/qom/cpu.h   |  9 +++++++++
 stubs/Makefile.objs |  1 +
 stubs/cpus.c        |  5 +++++
 4 files changed, 23 insertions(+), 3 deletions(-)
 create mode 100644 stubs/cpus.c
cpus.c +8 -3
index c15ff6c..c347db5 100644
@@ -993,6 +993,13 @@ void pause_all_vcpus(void)
     }
 }
 
+void resume_vcpu(CPUState *cpu)
+{
+    cpu->stop = false;
+    cpu->stopped = false;
+    qemu_cpu_kick(cpu);
+}
+
 void resume_all_vcpus(void)
 {
     CPUArchState *penv = first_cpu;
@@ -1000,9 +1007,7 @@ void resume_all_vcpus(void)
     qemu_clock_enable(vm_clock, true);
     while (penv) {
         CPUState *pcpu = ENV_GET_CPU(penv);
-        pcpu->stop = false;
-        pcpu->stopped = false;
-        qemu_cpu_kick(pcpu);
+        resume_vcpu(pcpu);
         penv = penv->next_cpu;
     }
 }
include/qom/cpu.h +0 -0
index 3664a1b..6ce6f10 100644
@@ -256,5 +256,14 @@ void cpu_interrupt(CPUState *cpu, int mask);
  */

include/qom/cpu.h void cpu_interrupt(CPUState *cpu, int mask); · line 266

Quoted 10 lines
include/qom/cpu.h +8 -0
 void cpu_reset_interrupt(CPUState *cpu, int mask);
 
+/**
+ * resume_vcpu:
+ * @cpu: The CPU to resume.
+ *
+ * Resumes CPU, i.e. puts CPU into runable state.
+ * KVM requires extra call cpu_synchronize_post_init() to make
+ * KVM's CPU instance runable.
+ */

Can you elaborate on this a bit. Why is it needed and why is it relevant for this function?

Quoted 27 lines
include/qom/cpu.h +1 -0
+void resume_vcpu(CPUState *cpu);
 
 #endif
stubs/Makefile.objs +1 -0
index 9c55b34..03dff20 100644
@@ -23,3 +23,4 @@ stub-obj-y += sysbus.o
 stub-obj-y += vm-stop.o
 stub-obj-y += vmstate.o
 stub-obj-$(CONFIG_WIN32) += fd-register.o
+stub-obj-y += cpus.o
stubs/cpus.c new file +5 -0
new file mode 100644
index 0000000..3d17451
@@ -0,0 +1,5 @@
+#include "qom/cpu.h"
+
+void resume_vcpu(CPUState *cpu)
+{
+}
Signature · 1 line
1.8.2

--
Gleb.

#16 lore raw

Re: [Qemu-devel] [PATCH 03/16] introduce resume_vcpu(), for single CPU

Quoted 53 lines · On Mon, 22 Apr 2013 13:40:02 +0300 Gleb Natapov <[email protected]> wrote:
Quoted 52 lines · Igor Mammedov

... also add stub for it, to make possible to use it in qom/cpu.c, which is shared with user emulators.

---
Note:
it will be re-used from qom/cpu.c by hotpluged CPU

---
 cpus.c              | 11 ++++++++---
 include/qom/cpu.h   |  9 +++++++++
 stubs/Makefile.objs |  1 +
 stubs/cpus.c        |  5 +++++
 4 files changed, 23 insertions(+), 3 deletions(-)
 create mode 100644 stubs/cpus.c
cpus.c +8 -3
index c15ff6c..c347db5 100644
@@ -993,6 +993,13 @@ void pause_all_vcpus(void)
     }
 }
 
+void resume_vcpu(CPUState *cpu)
+{
+    cpu->stop = false;
+    cpu->stopped = false;
+    qemu_cpu_kick(cpu);
+}
+
 void resume_all_vcpus(void)
 {
     CPUArchState *penv = first_cpu;
@@ -1000,9 +1007,7 @@ void resume_all_vcpus(void)
     qemu_clock_enable(vm_clock, true);
     while (penv) {
         CPUState *pcpu = ENV_GET_CPU(penv);
-        pcpu->stop = false;
-        pcpu->stopped = false;
-        qemu_cpu_kick(pcpu);
+        resume_vcpu(pcpu);
         penv = penv->next_cpu;
     }
 }
include/qom/cpu.h +0 -0
index 3664a1b..6ce6f10 100644
@@ -256,5 +256,14 @@ void cpu_interrupt(CPUState *cpu, int mask);
  */
 void cpu_reset_interrupt(CPUState *cpu, int mask);
 
Quoted 10 lines

include/qom/cpu.h void cpu_interrupt(CPUState *cpu, int mask); · line 266

Quoted 8 lines
include/qom/cpu.h +8 -0
+/**
+ * resume_vcpu:
+ * @cpu: The CPU to resume.
+ *
+ * Resumes CPU, i.e. puts CPU into runable state.
+ * KVM requires extra call cpu_synchronize_post_init() to make
+ * KVM's CPU instance runable.
+ */

Can you elaborate on this a bit. Why is it needed and why is it relevant for this function?

If running in KVM mode and resume_vcpu() is called without cpu_synchronize_post_init(), then CPU instance in KVM will be left in UNCONFIGURED state. I've added comment here to document that resume_vcpu() isn't enough to make CPU run-able in case of KVM.

Quoted 31 lines
Quoted 28 lines
include/qom/cpu.h +1 -0
+void resume_vcpu(CPUState *cpu);
 
 #endif
stubs/Makefile.objs +1 -0
index 9c55b34..03dff20 100644
@@ -23,3 +23,4 @@ stub-obj-y += sysbus.o
 stub-obj-y += vm-stop.o
 stub-obj-y += vmstate.o
 stub-obj-$(CONFIG_WIN32) += fd-register.o
+stub-obj-y += cpus.o
stubs/cpus.c new file +5 -0
new file mode 100644
index 0000000..3d17451
@@ -0,0 +1,5 @@
+#include "qom/cpu.h"
+
+void resume_vcpu(CPUState *cpu)
+{
+}
Signature · 1 line
1.8.2

--
Gleb.

#17 lore raw

Re: [Qemu-devel] [PATCH 03/16] introduce resume_vcpu(), for single CPU

Quoted 66 lines · Igor Mammedov
Quoted 53 lines · On Mon, 22 Apr 2013 13:40:02 +0300 Gleb Natapov <[email protected]> wrote:
Quoted 52 lines · Igor Mammedov

... also add stub for it, to make possible to use it in qom/cpu.c, which is shared with user emulators.

---
Note:
it will be re-used from qom/cpu.c by hotpluged CPU

---
 cpus.c              | 11 ++++++++---
 include/qom/cpu.h   |  9 +++++++++
 stubs/Makefile.objs |  1 +
 stubs/cpus.c        |  5 +++++
 4 files changed, 23 insertions(+), 3 deletions(-)
 create mode 100644 stubs/cpus.c
cpus.c +8 -3
index c15ff6c..c347db5 100644
@@ -993,6 +993,13 @@ void pause_all_vcpus(void)
     }
 }
 
+void resume_vcpu(CPUState *cpu)
+{
+    cpu->stop = false;
+    cpu->stopped = false;
+    qemu_cpu_kick(cpu);
+}
+
 void resume_all_vcpus(void)
 {
     CPUArchState *penv = first_cpu;
@@ -1000,9 +1007,7 @@ void resume_all_vcpus(void)
     qemu_clock_enable(vm_clock, true);
     while (penv) {
         CPUState *pcpu = ENV_GET_CPU(penv);
-        pcpu->stop = false;
-        pcpu->stopped = false;
-        qemu_cpu_kick(pcpu);
+        resume_vcpu(pcpu);
         penv = penv->next_cpu;
     }
 }
include/qom/cpu.h +0 -0
index 3664a1b..6ce6f10 100644
@@ -256,5 +256,14 @@ void cpu_interrupt(CPUState *cpu, int mask);
  */
 void cpu_reset_interrupt(CPUState *cpu, int mask);
 
Quoted 10 lines

include/qom/cpu.h void cpu_interrupt(CPUState *cpu, int mask); · line 266

Quoted 8 lines
include/qom/cpu.h +8 -0
+/**
+ * resume_vcpu:
+ * @cpu: The CPU to resume.
+ *
+ * Resumes CPU, i.e. puts CPU into runable state.
+ * KVM requires extra call cpu_synchronize_post_init() to make
+ * KVM's CPU instance runable.
+ */

Can you elaborate on this a bit. Why is it needed and why is it relevant for this function?

Quoted 5 lines

If running in KVM mode and resume_vcpu() is called without cpu_synchronize_post_init(), then CPU instance in KVM will be left in UNCONFIGURED state. I've added comment here to document that resume_vcpu() isn't enough to make CPU run-able in case of KVM.

But this really relevant only during vcpu creation after that resume_vcpu() should just work. My be have launch_vcpu() function that does the right thing and add assert to resume_vcpu() if it is called on vcpu that is not launched yet. In other words made the dependency clear in the code, not comment.

--
Gleb.

#18 lore raw

Re: [Qemu-devel] [PATCH 03/16] introduce resume_vcpu(), for single CPU

Quoted 67 lines · On Mon, 22 Apr 2013 13:59:17 +0300 Gleb Natapov <[email protected]> wrote:
Quoted 66 lines · Igor Mammedov
Quoted 63 lines · On Mon, 22 Apr 2013 13:40:02 +0300 Gleb Natapov <[email protected]> wrote:
Quoted 50 lines · Igor Mammedov

... also add stub for it, to make possible to use it in qom/cpu.c, which is shared with user emulators.

---
Note:
it will be re-used from qom/cpu.c by hotpluged CPU

---
 cpus.c              | 11 ++++++++---
 include/qom/cpu.h   |  9 +++++++++
 stubs/Makefile.objs |  1 +
 stubs/cpus.c        |  5 +++++
 4 files changed, 23 insertions(+), 3 deletions(-)
 create mode 100644 stubs/cpus.c
cpus.c +8 -3
index c15ff6c..c347db5 100644
@@ -993,6 +993,13 @@ void pause_all_vcpus(void)
     }
 }
 
+void resume_vcpu(CPUState *cpu)
+{
+    cpu->stop = false;
+    cpu->stopped = false;
+    qemu_cpu_kick(cpu);
+}
+
 void resume_all_vcpus(void)
 {
     CPUArchState *penv = first_cpu;
@@ -1000,9 +1007,7 @@ void resume_all_vcpus(void)
     qemu_clock_enable(vm_clock, true);
     while (penv) {
         CPUState *pcpu = ENV_GET_CPU(penv);
-        pcpu->stop = false;
-        pcpu->stopped = false;
-        qemu_cpu_kick(pcpu);
+        resume_vcpu(pcpu);
         penv = penv->next_cpu;
     }
 }
include/qom/cpu.h +0 -0
index 3664a1b..6ce6f10 100644
@@ -256,5 +256,14 @@ void cpu_interrupt(CPUState *cpu, int mask);
  */

include/qom/cpu.h void cpu_interrupt(CPUState *cpu, int mask); · line 266

Quoted 10 lines
include/qom/cpu.h +8 -0
 void cpu_reset_interrupt(CPUState *cpu, int mask);
 
+/**
+ * resume_vcpu:
+ * @cpu: The CPU to resume.
+ *
+ * Resumes CPU, i.e. puts CPU into runable state.
+ * KVM requires extra call cpu_synchronize_post_init() to make
+ * KVM's CPU instance runable.
+ */

Can you elaborate on this a bit. Why is it needed and why is it relevant for this function?

Quoted 7 lines
Quoted 5 lines

If running in KVM mode and resume_vcpu() is called without cpu_synchronize_post_init(), then CPU instance in KVM will be left in UNCONFIGURED state. I've added comment here to document that resume_vcpu() isn't enough to make CPU run-able in case of KVM.

But this really relevant only during vcpu creation after that resume_vcpu() should just work. My be have launch_vcpu() function that

Agreed, it's applicable only during CPU creation.

Quoted 3 lines

does the right thing and add assert to resume_vcpu() if it is called on vcpu that is not launched yet. In other words made the dependency clear in the code, not comment.

Would dropping this comment, suffice
and re-factoring to use launch_vcpu() done later?

Quoted 2 lines

--
Gleb.

#19 lore raw

Re: [Qemu-devel] [PATCH 03/16] introduce resume_vcpu(), for single CPU

Quoted 77 lines · Igor Mammedov
Quoted 67 lines · On Mon, 22 Apr 2013 13:59:17 +0300 Gleb Natapov <[email protected]> wrote:
Quoted 66 lines · Igor Mammedov
Quoted 63 lines · On Mon, 22 Apr 2013 13:40:02 +0300 Gleb Natapov <[email protected]> wrote:
Quoted 50 lines · Igor Mammedov

... also add stub for it, to make possible to use it in qom/cpu.c, which is shared with user emulators.

---
Note:
it will be re-used from qom/cpu.c by hotpluged CPU

---
 cpus.c              | 11 ++++++++---
 include/qom/cpu.h   |  9 +++++++++
 stubs/Makefile.objs |  1 +
 stubs/cpus.c        |  5 +++++
 4 files changed, 23 insertions(+), 3 deletions(-)
 create mode 100644 stubs/cpus.c
cpus.c +8 -3
index c15ff6c..c347db5 100644
@@ -993,6 +993,13 @@ void pause_all_vcpus(void)
     }
 }
 
+void resume_vcpu(CPUState *cpu)
+{
+    cpu->stop = false;
+    cpu->stopped = false;
+    qemu_cpu_kick(cpu);
+}
+
 void resume_all_vcpus(void)
 {
     CPUArchState *penv = first_cpu;
@@ -1000,9 +1007,7 @@ void resume_all_vcpus(void)
     qemu_clock_enable(vm_clock, true);
     while (penv) {
         CPUState *pcpu = ENV_GET_CPU(penv);
-        pcpu->stop = false;
-        pcpu->stopped = false;
-        qemu_cpu_kick(pcpu);
+        resume_vcpu(pcpu);
         penv = penv->next_cpu;
     }
 }
include/qom/cpu.h +0 -0
index 3664a1b..6ce6f10 100644
@@ -256,5 +256,14 @@ void cpu_interrupt(CPUState *cpu, int mask);
  */

include/qom/cpu.h void cpu_interrupt(CPUState *cpu, int mask); · line 266

Quoted 10 lines
include/qom/cpu.h +8 -0
 void cpu_reset_interrupt(CPUState *cpu, int mask);
 
+/**
+ * resume_vcpu:
+ * @cpu: The CPU to resume.
+ *
+ * Resumes CPU, i.e. puts CPU into runable state.
+ * KVM requires extra call cpu_synchronize_post_init() to make
+ * KVM's CPU instance runable.
+ */

Can you elaborate on this a bit. Why is it needed and why is it relevant for this function?

Quoted 7 lines
Quoted 5 lines

If running in KVM mode and resume_vcpu() is called without cpu_synchronize_post_init(), then CPU instance in KVM will be left in UNCONFIGURED state. I've added comment here to document that resume_vcpu() isn't enough to make CPU run-able in case of KVM.

But this really relevant only during vcpu creation after that resume_vcpu() should just work. My be have launch_vcpu() function that

Quoted 8 lines

Agreed, it's applicable only during CPU creation.

Quoted 3 lines

does the right thing and add assert to resume_vcpu() if it is called on vcpu that is not launched yet. In other words made the dependency clear in the code, not comment.

Would dropping this comment, suffice
and re-factoring to use launch_vcpu() done later?

Yes.

--
Gleb.

#20 lore raw

Re: [Qemu-devel] [PATCH 03/16] introduce resume_vcpu(), for single CPU

In reply to Igor Mammedov

Am 22.04.2013 13:45, schrieb Igor Mammedov:

Quoted 77 lines
Quoted 67 lines · On Mon, 22 Apr 2013 13:59:17 +0300 Gleb Natapov <[email protected]> wrote:
Quoted 66 lines · Igor Mammedov
Quoted 63 lines · On Mon, 22 Apr 2013 13:40:02 +0300 Gleb Natapov <[email protected]> wrote:
Quoted 50 lines · Igor Mammedov

... also add stub for it, to make possible to use it in qom/cpu.c, which is shared with user emulators.

---
Note:
it will be re-used from qom/cpu.c by hotpluged CPU

---
 cpus.c              | 11 ++++++++---
 include/qom/cpu.h   |  9 +++++++++
 stubs/Makefile.objs |  1 +
 stubs/cpus.c        |  5 +++++
 4 files changed, 23 insertions(+), 3 deletions(-)
 create mode 100644 stubs/cpus.c
cpus.c +8 -3
index c15ff6c..c347db5 100644
@@ -993,6 +993,13 @@ void pause_all_vcpus(void)
     }
 }
 
+void resume_vcpu(CPUState *cpu)
+{
+    cpu->stop = false;
+    cpu->stopped = false;
+    qemu_cpu_kick(cpu);
+}
+
 void resume_all_vcpus(void)
 {
     CPUArchState *penv = first_cpu;
@@ -1000,9 +1007,7 @@ void resume_all_vcpus(void)
     qemu_clock_enable(vm_clock, true);
     while (penv) {
         CPUState *pcpu = ENV_GET_CPU(penv);
-        pcpu->stop = false;
-        pcpu->stopped = false;
-        qemu_cpu_kick(pcpu);
+        resume_vcpu(pcpu);
         penv = penv->next_cpu;
     }
 }
include/qom/cpu.h +0 -0
index 3664a1b..6ce6f10 100644
@@ -256,5 +256,14 @@ void cpu_interrupt(CPUState *cpu, int mask);
  */

include/qom/cpu.h void cpu_interrupt(CPUState *cpu, int mask); · line 266

Quoted 10 lines
include/qom/cpu.h +8 -0
 void cpu_reset_interrupt(CPUState *cpu, int mask);
 
+/**
+ * resume_vcpu:
+ * @cpu: The CPU to resume.
+ *
+ * Resumes CPU, i.e. puts CPU into runable state.
+ * KVM requires extra call cpu_synchronize_post_init() to make
+ * KVM's CPU instance runable.
+ */

Can you elaborate on this a bit. Why is it needed and why is it relevant for this function?

Quoted 7 lines
Quoted 5 lines

If running in KVM mode and resume_vcpu() is called without cpu_synchronize_post_init(), then CPU instance in KVM will be left in UNCONFIGURED state. I've added comment here to document that resume_vcpu() isn't enough to make CPU run-able in case of KVM.

But this really relevant only during vcpu creation after that resume_vcpu() should just work. My be have launch_vcpu() function that

Quoted 7 lines

Agreed, it's applicable only during CPU creation.

Quoted 3 lines

does the right thing and add assert to resume_vcpu() if it is called on vcpu that is not launched yet. In other words made the dependency clear in the code, not comment.

Would dropping this comment, suffice
and re-factoring to use launch_vcpu() done later?

Either way I had request cpu_... naming for CPUState functions on IRC. Could be a follow-up cleanup.

Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#21 lore raw

Re: [Qemu-devel] [PATCH 03/16] introduce resume_vcpu(), for single CPU

Quoted 80 lines · On Mon, 22 Apr 2013 15:42:09 +0200 Andreas Färber <[email protected]> wrote:

Am 22.04.2013 13:45, schrieb Igor Mammedov:

Quoted 79 lines
Quoted 67 lines · On Mon, 22 Apr 2013 13:59:17 +0300 Gleb Natapov <[email protected]> wrote:
Quoted 66 lines · Igor Mammedov
Quoted 63 lines · On Mon, 22 Apr 2013 13:40:02 +0300 Gleb Natapov <[email protected]> wrote:
Quoted 50 lines · Igor Mammedov

... also add stub for it, to make possible to use it in qom/cpu.c, which is shared with user emulators.

---
Note:
it will be re-used from qom/cpu.c by hotpluged CPU

---
 cpus.c              | 11 ++++++++---
 include/qom/cpu.h   |  9 +++++++++
 stubs/Makefile.objs |  1 +
 stubs/cpus.c        |  5 +++++
 4 files changed, 23 insertions(+), 3 deletions(-)
 create mode 100644 stubs/cpus.c
cpus.c +8 -3
index c15ff6c..c347db5 100644
@@ -993,6 +993,13 @@ void pause_all_vcpus(void)
     }
 }
 
+void resume_vcpu(CPUState *cpu)
+{
+    cpu->stop = false;
+    cpu->stopped = false;
+    qemu_cpu_kick(cpu);
+}
+
 void resume_all_vcpus(void)
 {
     CPUArchState *penv = first_cpu;
@@ -1000,9 +1007,7 @@ void resume_all_vcpus(void)
     qemu_clock_enable(vm_clock, true);
     while (penv) {
         CPUState *pcpu = ENV_GET_CPU(penv);
-        pcpu->stop = false;
-        pcpu->stopped = false;
-        qemu_cpu_kick(pcpu);
+        resume_vcpu(pcpu);
         penv = penv->next_cpu;
     }
 }
include/qom/cpu.h +0 -0
index 3664a1b..6ce6f10 100644
@@ -256,5 +256,14 @@ void cpu_interrupt(CPUState *cpu, int mask);
  */

include/qom/cpu.h void cpu_interrupt(CPUState *cpu, int mask); · line 266

Quoted 10 lines
include/qom/cpu.h +8 -0
 void cpu_reset_interrupt(CPUState *cpu, int mask);
 
+/**
+ * resume_vcpu:
+ * @cpu: The CPU to resume.
+ *
+ * Resumes CPU, i.e. puts CPU into runable state.
+ * KVM requires extra call cpu_synchronize_post_init() to make
+ * KVM's CPU instance runable.
+ */

Can you elaborate on this a bit. Why is it needed and why is it relevant for this function?

Quoted 7 lines
Quoted 5 lines

If running in KVM mode and resume_vcpu() is called without cpu_synchronize_post_init(), then CPU instance in KVM will be left in UNCONFIGURED state. I've added comment here to document that resume_vcpu() isn't enough to make CPU run-able in case of KVM.

But this really relevant only during vcpu creation after that resume_vcpu() should just work. My be have launch_vcpu() function that

Agreed, it's applicable only during CPU creation.

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

does the right thing and add assert to resume_vcpu() if it is called on vcpu that is not launched yet. In other words made the dependency clear in the code, not comment.

Would dropping this comment, suffice
and re-factoring to use launch_vcpu() done later?

Either way I had request cpu_... naming for CPUState functions on IRC. Could be a follow-up cleanup.

How about cpu_resume(), is it suitable?

Quoted 3 lines

Andreas

#22 lore raw

Re: [Qemu-devel] [PATCH 03/16] introduce resume_vcpu(), for single CPU

Am 22.04.2013 16:50, schrieb Igor Mammedov:

Quoted 86 lines
Quoted 83 lines · On Mon, 22 Apr 2013 15:42:09 +0200 Andreas Färber <[email protected]> wrote:

Am 22.04.2013 13:45, schrieb Igor Mammedov:

Quoted 82 lines
Quoted 67 lines · On Mon, 22 Apr 2013 13:59:17 +0300 Gleb Natapov <[email protected]> wrote:
Quoted 66 lines · Igor Mammedov
Quoted 63 lines · On Mon, 22 Apr 2013 13:40:02 +0300 Gleb Natapov <[email protected]> wrote:
Quoted 50 lines · Igor Mammedov

... also add stub for it, to make possible to use it in qom/cpu.c, which is shared with user emulators.

---
Note:
it will be re-used from qom/cpu.c by hotpluged CPU

---
 cpus.c              | 11 ++++++++---
 include/qom/cpu.h   |  9 +++++++++
 stubs/Makefile.objs |  1 +
 stubs/cpus.c        |  5 +++++
 4 files changed, 23 insertions(+), 3 deletions(-)
 create mode 100644 stubs/cpus.c
cpus.c +8 -3
index c15ff6c..c347db5 100644
@@ -993,6 +993,13 @@ void pause_all_vcpus(void)
     }
 }
 
+void resume_vcpu(CPUState *cpu)
+{
+    cpu->stop = false;
+    cpu->stopped = false;
+    qemu_cpu_kick(cpu);
+}
+
 void resume_all_vcpus(void)
 {
     CPUArchState *penv = first_cpu;
@@ -1000,9 +1007,7 @@ void resume_all_vcpus(void)
     qemu_clock_enable(vm_clock, true);
     while (penv) {
         CPUState *pcpu = ENV_GET_CPU(penv);
-        pcpu->stop = false;
-        pcpu->stopped = false;
-        qemu_cpu_kick(pcpu);
+        resume_vcpu(pcpu);
         penv = penv->next_cpu;
     }
 }
include/qom/cpu.h +0 -0
index 3664a1b..6ce6f10 100644
@@ -256,5 +256,14 @@ void cpu_interrupt(CPUState *cpu, int mask);
  */

include/qom/cpu.h void cpu_interrupt(CPUState *cpu, int mask); · line 266

Quoted 10 lines
include/qom/cpu.h +8 -0
 void cpu_reset_interrupt(CPUState *cpu, int mask);
 
+/**
+ * resume_vcpu:
+ * @cpu: The CPU to resume.
+ *
+ * Resumes CPU, i.e. puts CPU into runable state.
+ * KVM requires extra call cpu_synchronize_post_init() to make
+ * KVM's CPU instance runable.
+ */

Can you elaborate on this a bit. Why is it needed and why is it relevant for this function?

Quoted 7 lines
Quoted 5 lines

If running in KVM mode and resume_vcpu() is called without cpu_synchronize_post_init(), then CPU instance in KVM will be left in UNCONFIGURED state. I've added comment here to document that resume_vcpu() isn't enough to make CPU run-able in case of KVM.

But this really relevant only during vcpu creation after that resume_vcpu() should just work. My be have launch_vcpu() function that

Agreed, it's applicable only during CPU creation.

Quoted 3 lines

does the right thing and add assert to resume_vcpu() if it is called on vcpu that is not launched yet. In other words made the dependency clear in the code, not comment.

Quoted 6 lines
Quoted 5 lines
Quoted 2 lines

Would dropping this comment, suffice
and re-factoring to use launch_vcpu() done later?

Either way I had request cpu_... naming for CPUState functions on IRC. Could be a follow-up cleanup.

How about cpu_resume(), is it suitable?

Sure, that matches the scheme.

Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#23 lore raw

PATCH v4 4/16 cpu: resume CPU from CPUClass.cpu_common_realizefn() when it is hot-plugged

In reply to Igor Mammedov

---
v2:
* remove unnecessary now sysemu/cpus.h header
* move out introducing resume_vcpu() into a separate patch

---
 qom/cpu.c | 1 +
 1 file changed, 1 insertion(+)
qom/cpu.c +1 -0
index 0c76712..40a4259 100644
@@ -60,6 +60,7 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
 {
     if (dev->hotplugged) {
         cpu_synchronize_post_init(CPU(dev));
+        resume_vcpu(CPU(dev));
     }
 }
 
Signature · 1 line
1.8.2
Reviewed-by on v4 4/16
#24 lore raw

Re: [Qemu-devel] [PATCH 04/16] cpu: resume CPU from CPUClass.cpu_common_realizefn() when it is hot-plugged

Quoted 1 line · Igor Mammedov
Quoted 24 lines

---
v2:
* remove unnecessary now sysemu/cpus.h header
* move out introducing resume_vcpu() into a separate patch

---
 qom/cpu.c | 1 +
 1 file changed, 1 insertion(+)
qom/cpu.c +1 -0
index 0c76712..40a4259 100644
@@ -60,6 +60,7 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
 {
     if (dev->hotplugged) {
         cpu_synchronize_post_init(CPU(dev));
+        resume_vcpu(CPU(dev));
     }
 }
 
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
#25 lore raw

PATCH v4 5/16 introduce CPU hot-plug notifier

In reply to Igor Mammedov

hot-added CPU will be distributed to acpi_piix4 and rtc_cmos

---
v2:
* move notifier to qom/cpu.c and call it from CPUClass.realize() on hotplug
* remove get_firmware_id() since it belong to other patch

---
 include/sysemu/sysemu.h |  3 +++
 qom/cpu.c               | 12 ++++++++++++
 2 files changed, 15 insertions(+)
include/sysemu/sysemu.h +3 -0
index 6578782..a8c3de1 100644
@@ -152,6 +152,9 @@ void do_pci_device_hot_remove(Monitor *mon, const QDict *qdict);
 /* generic hotplug */
 void drive_hot_add(Monitor *mon, const QDict *qdict);
 
+/* CPU hotplug */
+void qemu_register_cpu_added_notifier(Notifier *notifier);
+
 /* pcie aer error injection */
 void pcie_aer_inject_error_print(Monitor *mon, const QObject *data);
 int do_pcie_aer_inject_error(Monitor *mon,
qom/cpu.c +12 -0
index 40a4259..819986e 100644
@@ -21,6 +21,17 @@
 #include "qom/cpu.h"
 #include "qemu-common.h"
 #include "sysemu/kvm.h"
+#include "qemu/notify.h"
+#include "sysemu/sysemu.h"
+
+/* CPU hot-plug notifiers */
+static NotifierList cpu_added_notifiers =
+    NOTIFIER_LIST_INITIALIZER(cpu_add_notifiers);
+
+void qemu_register_cpu_added_notifier(Notifier *notifier)
+{
+    notifier_list_add(&cpu_added_notifiers, notifier);
+}
 
 void cpu_reset_interrupt(CPUState *cpu, int mask)
 {
@@ -61,6 +72,7 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
     if (dev->hotplugged) {
         cpu_synchronize_post_init(CPU(dev));
         resume_vcpu(CPU(dev));
+        notifier_list_notify(&cpu_added_notifiers, dev);
     }
 }
 
Signature · 1 line
1.8.2
#26 lore raw

Re: [Qemu-devel] [PATCH 05/16] introduce CPU hot-plug notifier

Quoted 51 lines · Igor Mammedov

hot-added CPU will be distributed to acpi_piix4 and rtc_cmos

---
v2:
* move notifier to qom/cpu.c and call it from CPUClass.realize() on hotplug
* remove get_firmware_id() since it belong to other patch

---
 include/sysemu/sysemu.h |  3 +++
 qom/cpu.c               | 12 ++++++++++++
 2 files changed, 15 insertions(+)
include/sysemu/sysemu.h +3 -0
index 6578782..a8c3de1 100644
@@ -152,6 +152,9 @@ void do_pci_device_hot_remove(Monitor *mon, const QDict *qdict);
 /* generic hotplug */
 void drive_hot_add(Monitor *mon, const QDict *qdict);
 
+/* CPU hotplug */
+void qemu_register_cpu_added_notifier(Notifier *notifier);
+
 /* pcie aer error injection */
 void pcie_aer_inject_error_print(Monitor *mon, const QObject *data);
 int do_pcie_aer_inject_error(Monitor *mon,
qom/cpu.c +11 -0
index 40a4259..819986e 100644
@@ -21,6 +21,17 @@
 #include "qom/cpu.h"
 #include "qemu-common.h"
 #include "sysemu/kvm.h"
+#include "qemu/notify.h"
+#include "sysemu/sysemu.h"
+
+/* CPU hot-plug notifiers */
+static NotifierList cpu_added_notifiers =
+    NOTIFIER_LIST_INITIALIZER(cpu_add_notifiers);
+
+void qemu_register_cpu_added_notifier(Notifier *notifier)
+{
+    notifier_list_add(&cpu_added_notifiers, notifier);
+}
 
 void cpu_reset_interrupt(CPUState *cpu, int mask)
 {

qom/cpu.c static void cpu_common_realizefn(DeviceState *dev, Error **errp) · line 75

Quoted 4 lines
qom/cpu.c +1 -0
@@ -61,6 +72,7 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
     if (dev->hotplugged) {
         cpu_synchronize_post_init(CPU(dev));
         resume_vcpu(CPU(dev));
+        notifier_list_notify(&cpu_added_notifiers, dev);

Why do it after vcpu is running? May be notifier want to do something that should be done before vcpu is running.

Quoted 7 lines
qom/cpu.c +0 -0
     }
 }
 
Signature · 1 line
1.8.2

--
Gleb.

#27 lore raw

Re: [Qemu-devel] [PATCH 05/16] introduce CPU hot-plug notifier

Quoted 50 lines · On Mon, 22 Apr 2013 14:00:01 +0300 Gleb Natapov <[email protected]> wrote:
Quoted 49 lines · Igor Mammedov

hot-added CPU will be distributed to acpi_piix4 and rtc_cmos

---
v2:
* move notifier to qom/cpu.c and call it from CPUClass.realize() on
hotplug
* remove get_firmware_id() since it belong to other patch

---
 include/sysemu/sysemu.h |  3 +++
 qom/cpu.c               | 12 ++++++++++++
 2 files changed, 15 insertions(+)
include/sysemu/sysemu.h +0 -0
index 6578782..a8c3de1 100644
@@ -152,6 +152,9 @@ void do_pci_device_hot_remove(Monitor *mon, const

QDict *qdict); /* generic hotplug */
void drive_hot_add(Monitor *mon, const QDict *qdict);

include/sysemu/sysemu.h +3 -0
+/* CPU hotplug */
+void qemu_register_cpu_added_notifier(Notifier *notifier);
+
 /* pcie aer error injection */
 void pcie_aer_inject_error_print(Monitor *mon, const QObject *data);
 int do_pcie_aer_inject_error(Monitor *mon,
qom/cpu.c +11 -0
index 40a4259..819986e 100644
@@ -21,6 +21,17 @@
 #include "qom/cpu.h"
 #include "qemu-common.h"
 #include "sysemu/kvm.h"
+#include "qemu/notify.h"
+#include "sysemu/sysemu.h"
+
+/* CPU hot-plug notifiers */
+static NotifierList cpu_added_notifiers =
+    NOTIFIER_LIST_INITIALIZER(cpu_add_notifiers);
+
+void qemu_register_cpu_added_notifier(Notifier *notifier)
+{
+    notifier_list_add(&cpu_added_notifiers, notifier);
+}
 
Quoted 9 lines

qom/cpu.c static void cpu_common_realizefn(DeviceState *dev, · line 75

Quoted 7 lines
qom/cpu.c +1 -0
 void cpu_reset_interrupt(CPUState *cpu, int mask)
 {
@@ -61,6 +72,7 @@ static void cpu_common_realizefn(DeviceState *dev,
 Error **errp) if (dev->hotplugged) {
         cpu_synchronize_post_init(CPU(dev));
         resume_vcpu(CPU(dev));
+        notifier_list_notify(&cpu_added_notifiers, dev);

Why do it after vcpu is running? May be notifier want to do something that should be done before vcpu is running.

currently there is not need for adding_cpu notifier.
This one signals that there is a new fully functional CPU available.

Quoted 10 lines
Quoted 7 lines
qom/cpu.c +0 -0
     }
 }
 
Signature · 1 line
1.8.2

--
Gleb.

#28 lore raw

Re: [Qemu-devel] [PATCH 05/16] introduce CPU hot-plug notifier

Quoted 62 lines · Igor Mammedov
Quoted 50 lines · On Mon, 22 Apr 2013 14:00:01 +0300 Gleb Natapov <[email protected]> wrote:
Quoted 49 lines · Igor Mammedov

hot-added CPU will be distributed to acpi_piix4 and rtc_cmos

---
v2:
* move notifier to qom/cpu.c and call it from CPUClass.realize() on
hotplug
* remove get_firmware_id() since it belong to other patch

---
 include/sysemu/sysemu.h |  3 +++
 qom/cpu.c               | 12 ++++++++++++
 2 files changed, 15 insertions(+)
include/sysemu/sysemu.h +0 -0
index 6578782..a8c3de1 100644
@@ -152,6 +152,9 @@ void do_pci_device_hot_remove(Monitor *mon, const

QDict *qdict); /* generic hotplug */
void drive_hot_add(Monitor *mon, const QDict *qdict);

include/sysemu/sysemu.h +3 -0
+/* CPU hotplug */
+void qemu_register_cpu_added_notifier(Notifier *notifier);
+
 /* pcie aer error injection */
 void pcie_aer_inject_error_print(Monitor *mon, const QObject *data);
 int do_pcie_aer_inject_error(Monitor *mon,
qom/cpu.c +11 -0
index 40a4259..819986e 100644
@@ -21,6 +21,17 @@
 #include "qom/cpu.h"
 #include "qemu-common.h"
 #include "sysemu/kvm.h"
+#include "qemu/notify.h"
+#include "sysemu/sysemu.h"
+
+/* CPU hot-plug notifiers */
+static NotifierList cpu_added_notifiers =
+    NOTIFIER_LIST_INITIALIZER(cpu_add_notifiers);
+
+void qemu_register_cpu_added_notifier(Notifier *notifier)
+{
+    notifier_list_add(&cpu_added_notifiers, notifier);
+}
 
Quoted 9 lines

qom/cpu.c static void cpu_common_realizefn(DeviceState *dev, · line 75

Quoted 7 lines
qom/cpu.c +1 -0
 void cpu_reset_interrupt(CPUState *cpu, int mask)
 {
@@ -61,6 +72,7 @@ static void cpu_common_realizefn(DeviceState *dev,
 Error **errp) if (dev->hotplugged) {
         cpu_synchronize_post_init(CPU(dev));
         resume_vcpu(CPU(dev));
+        notifier_list_notify(&cpu_added_notifiers, dev);

Why do it after vcpu is running? May be notifier want to do something that should be done before vcpu is running.

Quoted 3 lines

currently there is not need for adding_cpu notifier.
This one signals that there is a new fully functional CPU available.

You are using it to update cmos information for BIOS. It would be strange for newly added vcpu to see incorrect number of cpus in cmos if, for some reason, it will check it. The "new cpu is added and ready to tun, do your last things before it is launched" callback seams more reasonable then "new cpu is running wild" one.

--
Gleb.

#29 lore raw

Re: [Qemu-devel] [PATCH 05/16] introduce CPU hot-plug notifier

Quoted 63 lines · On Mon, 22 Apr 2013 14:24:00 +0300 Gleb Natapov <[email protected]> wrote:
Quoted 62 lines · Igor Mammedov
Quoted 59 lines · On Mon, 22 Apr 2013 14:00:01 +0300 Gleb Natapov <[email protected]> wrote:
Quoted 52 lines · Igor Mammedov

hot-added CPU will be distributed to acpi_piix4 and rtc_cmos

---
v2:
* move notifier to qom/cpu.c and call it from CPUClass.realize() on
hotplug
* remove get_firmware_id() since it belong to other patch

---
 include/sysemu/sysemu.h |  3 +++
 qom/cpu.c               | 12 ++++++++++++
 2 files changed, 15 insertions(+)
include/sysemu/sysemu.h +0 -0
index 6578782..a8c3de1 100644
@@ -152,6 +152,9 @@ void do_pci_device_hot_remove(Monitor *mon, const

QDict *qdict); /* generic hotplug */
void drive_hot_add(Monitor *mon, const QDict *qdict);

include/sysemu/sysemu.h +3 -0
+/* CPU hotplug */
+void qemu_register_cpu_added_notifier(Notifier *notifier);
+
 /* pcie aer error injection */
 void pcie_aer_inject_error_print(Monitor *mon, const QObject *data);
 int do_pcie_aer_inject_error(Monitor *mon,
qom/cpu.c +11 -0
index 40a4259..819986e 100644
@@ -21,6 +21,17 @@
 #include "qom/cpu.h"
 #include "qemu-common.h"
 #include "sysemu/kvm.h"
+#include "qemu/notify.h"
+#include "sysemu/sysemu.h"
+
+/* CPU hot-plug notifiers */
+static NotifierList cpu_added_notifiers =
+    NOTIFIER_LIST_INITIALIZER(cpu_add_notifiers);
+
+void qemu_register_cpu_added_notifier(Notifier *notifier)
+{
+    notifier_list_add(&cpu_added_notifiers, notifier);
+}
 
 void cpu_reset_interrupt(CPUState *cpu, int mask)
 {

qom/cpu.c static void cpu_common_realizefn(DeviceState *dev, · line 75

Quoted 4 lines
qom/cpu.c +1 -0
@@ -61,6 +72,7 @@ static void cpu_common_realizefn(DeviceState *dev,
 Error **errp) if (dev->hotplugged) {
         cpu_synchronize_post_init(CPU(dev));
         resume_vcpu(CPU(dev));
+        notifier_list_notify(&cpu_added_notifiers, dev);

Why do it after vcpu is running? May be notifier want to do something that should be done before vcpu is running.

Quoted 8 lines
Quoted 3 lines

currently there is not need for adding_cpu notifier.
This one signals that there is a new fully functional CPU available.

You are using it to update cmos information for BIOS. It would be strange for newly added vcpu to see incorrect number of cpus in cmos if, for some reason, it will check it. The "new cpu is added and ready to tun, do your last things before it is launched" callback seams more reasonable then "new cpu is running wild" one.

IRC,
I'm moving it between cpu_synchronize_post_init() and resume_vcpu(), since cpu_synchronize_post_init() pushes apic_state into kvm.

Quoted 2 lines

--
Gleb.

#30 lore raw

PATCH v4 6/16 target-i386: pc: update rtc_cmos on CPU hot-plug

In reply to Igor Mammedov

... so that on reboot BIOS could read current available CPU count

---
 hw/i386/pc.c                   | 20 ++++++++++++++++++++
 hw/timer/mc146818rtc.c         |  7 +++++++
 include/hw/timer/mc146818rtc.h |  1 +
 3 files changed, 28 insertions(+)
hw/i386/pc.c +20 -0
index 8d75b34..dc1a78b 100644
@@ -337,6 +337,21 @@ static void pc_cmos_init_late(void *opaque)
     qemu_unregister_reset(pc_cmos_init_late, opaque);
 }
 
+typedef struct rtc_cpu_hotplug_arg {
+    ISADevice *rtc_state;
+    Notifier cpu_added_notifier;
+} rtc_cpu_hotplug_arg;
+
+static void rtc_notify_cpu_added(Notifier *notifier, void *data)
+{
+    rtc_cpu_hotplug_arg *arg = container_of(notifier, rtc_cpu_hotplug_arg,
+                                           cpu_added_notifier);
+    ISADevice *s = arg->rtc_state;
+
+    /* increment the number of CPUs */
+    rtc_set_memory(s, 0x5f, rtc_get_memory(s, 0x5f) + 1);
+}
+
 void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
                   const char *boot_device,
                   ISADevice *floppy, BusState *idebus0, BusState *idebus1,
@@ -345,6 +360,7 @@ void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
     int val, nb, i;
     FDriveType fd_type[2] = { FDRIVE_DRV_NONE, FDRIVE_DRV_NONE };
     static pc_cmos_init_late_arg arg;
+    static rtc_cpu_hotplug_arg cpu_hotplug_cb;
 
     /* various important CMOS locations needed by PC/Bochs bios */
 
@@ -383,6 +399,10 @@ void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
 
     /* set the number of CPU */
     rtc_set_memory(s, 0x5f, smp_cpus - 1);
+    /* init CPU hotplug notifier */
+    cpu_hotplug_cb.rtc_state = s;
+    cpu_hotplug_cb.cpu_added_notifier.notify = rtc_notify_cpu_added;
+    qemu_register_cpu_added_notifier(&cpu_hotplug_cb.cpu_added_notifier);
 
     /* set boot devices, and disable floppy signature check if requested */
     if (set_boot_dev(s, boot_device, fd_bootchk)) {
hw/timer/mc146818rtc.c +7 -0
index 69e6844..e639942 100644
@@ -677,6 +677,13 @@ void rtc_set_memory(ISADevice *dev, int addr, int val)
         s->cmos_data[addr] = val;
 }
 
+int rtc_get_memory(ISADevice *dev, int addr)
+{
+    RTCState *s = DO_UPCAST(RTCState, dev, dev);
+    assert(addr >= 0 && addr <= 127);
+    return s->cmos_data[addr];
+}
+
 static void rtc_set_date_from_host(ISADevice *dev)
 {
     RTCState *s = DO_UPCAST(RTCState, dev, dev);
include/hw/timer/mc146818rtc.h +1 -0
index 854ea3f..09f37b7 100644
@@ -6,6 +6,7 @@
 
 ISADevice *rtc_init(ISABus *bus, int base_year, qemu_irq intercept_irq);
 void rtc_set_memory(ISADevice *dev, int addr, int val);
+int rtc_get_memory(ISADevice *dev, int addr);
 void rtc_set_date(ISADevice *dev, const struct tm *tm);
 
 #endif /* !MC146818RTC_H */
Signature · 1 line
1.8.2
Reviewed-by on v4 6/16
#31 lore raw

Re: [Qemu-devel] [PATCH 06/16] target-i386: pc: update rtc_cmos on CPU hot-plug

Quoted 3 lines · Igor Mammedov

... so that on reboot BIOS could read current available CPU count

Quoted 85 lines
---
 hw/i386/pc.c                   | 20 ++++++++++++++++++++
 hw/timer/mc146818rtc.c         |  7 +++++++
 include/hw/timer/mc146818rtc.h |  1 +
 3 files changed, 28 insertions(+)
hw/i386/pc.c +20 -0
index 8d75b34..dc1a78b 100644
@@ -337,6 +337,21 @@ static void pc_cmos_init_late(void *opaque)
     qemu_unregister_reset(pc_cmos_init_late, opaque);
 }
 
+typedef struct rtc_cpu_hotplug_arg {
+    ISADevice *rtc_state;
+    Notifier cpu_added_notifier;
+} rtc_cpu_hotplug_arg;
+
+static void rtc_notify_cpu_added(Notifier *notifier, void *data)
+{
+    rtc_cpu_hotplug_arg *arg = container_of(notifier, rtc_cpu_hotplug_arg,
+                                           cpu_added_notifier);
+    ISADevice *s = arg->rtc_state;
+
+    /* increment the number of CPUs */
+    rtc_set_memory(s, 0x5f, rtc_get_memory(s, 0x5f) + 1);
+}
+
 void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
                   const char *boot_device,
                   ISADevice *floppy, BusState *idebus0, BusState *idebus1,
@@ -345,6 +360,7 @@ void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
     int val, nb, i;
     FDriveType fd_type[2] = { FDRIVE_DRV_NONE, FDRIVE_DRV_NONE };
     static pc_cmos_init_late_arg arg;
+    static rtc_cpu_hotplug_arg cpu_hotplug_cb;
 
     /* various important CMOS locations needed by PC/Bochs bios */
 
@@ -383,6 +399,10 @@ void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
 
     /* set the number of CPU */
     rtc_set_memory(s, 0x5f, smp_cpus - 1);
+    /* init CPU hotplug notifier */
+    cpu_hotplug_cb.rtc_state = s;
+    cpu_hotplug_cb.cpu_added_notifier.notify = rtc_notify_cpu_added;
+    qemu_register_cpu_added_notifier(&cpu_hotplug_cb.cpu_added_notifier);
 
     /* set boot devices, and disable floppy signature check if requested */
     if (set_boot_dev(s, boot_device, fd_bootchk)) {
hw/timer/mc146818rtc.c +7 -0
index 69e6844..e639942 100644
@@ -677,6 +677,13 @@ void rtc_set_memory(ISADevice *dev, int addr, int val)
         s->cmos_data[addr] = val;
 }
 
+int rtc_get_memory(ISADevice *dev, int addr)
+{
+    RTCState *s = DO_UPCAST(RTCState, dev, dev);
+    assert(addr >= 0 && addr <= 127);
+    return s->cmos_data[addr];
+}
+
 static void rtc_set_date_from_host(ISADevice *dev)
 {
     RTCState *s = DO_UPCAST(RTCState, dev, dev);
include/hw/timer/mc146818rtc.h +1 -0
index 854ea3f..09f37b7 100644
@@ -6,6 +6,7 @@
 
 ISADevice *rtc_init(ISABus *bus, int base_year, qemu_irq intercept_irq);
 void rtc_set_memory(ISADevice *dev, int addr, int val);
+int rtc_get_memory(ISADevice *dev, int addr);
 void rtc_set_date(ISADevice *dev, const struct tm *tm);
 
 #endif /* !MC146818RTC_H */
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
#32 lore raw

Re: [Qemu-devel] [PATCH 06/16] target-i386: pc: update rtc_cmos on CPU hot-plug

In reply to Igor Mammedov

Am 16.04.2013 00:12, schrieb Igor Mammedov:

Quoted 14 lines

... so that on reboot BIOS could read current available CPU count

---
 hw/i386/pc.c                   | 20 ++++++++++++++++++++
 hw/timer/mc146818rtc.c         |  7 +++++++
 include/hw/timer/mc146818rtc.h |  1 +
 3 files changed, 28 insertions(+)
hw/i386/pc.c +0 -0
index 8d75b34..dc1a78b 100644

hw/i386/pc.c static void pc_cmos_init_late(void *opaque) · line 343

Quoted 7 lines
hw/i386/pc.c +4 -0
@@ -337,6 +337,21 @@ static void pc_cmos_init_late(void *opaque)
     qemu_unregister_reset(pc_cmos_init_late, opaque);
 }
 
+typedef struct rtc_cpu_hotplug_arg {
+    ISADevice *rtc_state;
+    Notifier cpu_added_notifier;
+} rtc_cpu_hotplug_arg;

Are the arguments (rtc_state) intentionally placed before the common part?

hw/i386/pc.c static void pc_cmos_init_late(void *opaque) · line 353

Quoted 10 lines
hw/i386/pc.c +10 -0
+
+static void rtc_notify_cpu_added(Notifier *notifier, void *data)
+{
+    rtc_cpu_hotplug_arg *arg = container_of(notifier, rtc_cpu_hotplug_arg,
+                                           cpu_added_notifier);
+    ISADevice *s = arg->rtc_state;
+
+    /* increment the number of CPUs */
+    rtc_set_memory(s, 0x5f, rtc_get_memory(s, 0x5f) + 1);
+}

I think this proves Gleb's point of reordering the notifier and CPU resumption since the first instruction executed might theoretically be reading the RTC memory.

Otherwise looks good with one minor nit...

Quoted 28 lines
hw/i386/pc.c +6 -0
+
 void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
                   const char *boot_device,
                   ISADevice *floppy, BusState *idebus0, BusState *idebus1,
@@ -345,6 +360,7 @@ void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
     int val, nb, i;
     FDriveType fd_type[2] = { FDRIVE_DRV_NONE, FDRIVE_DRV_NONE };
     static pc_cmos_init_late_arg arg;
+    static rtc_cpu_hotplug_arg cpu_hotplug_cb;
 
     /* various important CMOS locations needed by PC/Bochs bios */
 
@@ -383,6 +399,10 @@ void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
 
     /* set the number of CPU */
     rtc_set_memory(s, 0x5f, smp_cpus - 1);
+    /* init CPU hotplug notifier */
+    cpu_hotplug_cb.rtc_state = s;
+    cpu_hotplug_cb.cpu_added_notifier.notify = rtc_notify_cpu_added;
+    qemu_register_cpu_added_notifier(&cpu_hotplug_cb.cpu_added_notifier);
 
     /* set boot devices, and disable floppy signature check if requested */
     if (set_boot_dev(s, boot_device, fd_bootchk)) {
hw/timer/mc146818rtc.c +0 -0
index 69e6844..e639942 100644

hw/timer/mc146818rtc.c void rtc_set_memory(ISADevice *dev, int addr, int val) · line 682

Quoted 6 lines
hw/timer/mc146818rtc.c +3 -0
@@ -677,6 +677,13 @@ void rtc_set_memory(ISADevice *dev, int addr, int val)
         s->cmos_data[addr] = val;
 }
 
+int rtc_get_memory(ISADevice *dev, int addr)
+{
+    RTCState *s = DO_UPCAST(RTCState, dev, dev);

If we apply my mc146818rtc.c QOM'ification patch to qom-cpu first, this can become MC146818_RTC(dev). Anthony just ack'ed that, but qemu.git is currently broken for testing, so I'm still holding off changes.

Andreas

Quoted 19 lines
hw/timer/mc146818rtc.c +4 -0
+    assert(addr >= 0 && addr <= 127);
+    return s->cmos_data[addr];
+}
+
 static void rtc_set_date_from_host(ISADevice *dev)
 {
     RTCState *s = DO_UPCAST(RTCState, dev, dev);
include/hw/timer/mc146818rtc.h +1 -0
index 854ea3f..09f37b7 100644
@@ -6,6 +6,7 @@
 
 ISADevice *rtc_init(ISABus *bus, int base_year, qemu_irq intercept_irq);
 void rtc_set_memory(ISADevice *dev, int addr, int val);
+int rtc_get_memory(ISADevice *dev, int addr);
 void rtc_set_date(ISADevice *dev, const struct tm *tm);
 
 #endif /* !MC146818RTC_H */
Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#33 lore raw

Re: [Qemu-devel] [PATCH 06/16] target-i386: pc: update rtc_cmos on CPU hot-plug

Quoted 18 lines · On Mon, 22 Apr 2013 16:56:59 +0200 Andreas Färber <[email protected]> wrote:

Am 16.04.2013 00:12, schrieb Igor Mammedov:

Quoted 17 lines

... so that on reboot BIOS could read current available CPU count

---
 hw/i386/pc.c                   | 20 ++++++++++++++++++++
 hw/timer/mc146818rtc.c         |  7 +++++++
 include/hw/timer/mc146818rtc.h |  1 +
 3 files changed, 28 insertions(+)
hw/i386/pc.c +0 -0
index 8d75b34..dc1a78b 100644
@@ -337,6 +337,21 @@ static void pc_cmos_init_late(void *opaque)
     qemu_unregister_reset(pc_cmos_init_late, opaque);
 }
 
Quoted 6 lines

hw/i386/pc.c static void pc_cmos_init_late(void *opaque) · line 343

Quoted 4 lines
hw/i386/pc.c +4 -0
+typedef struct rtc_cpu_hotplug_arg {
+    ISADevice *rtc_state;
+    Notifier cpu_added_notifier;
+} rtc_cpu_hotplug_arg;

Are the arguments (rtc_state) intentionally placed before the common part?

no, but it's not QOM type so it doesn't matter.
I'll reorder it on next respin.

Quoted 16 lines

hw/i386/pc.c static void pc_cmos_init_late(void *opaque) · line 353

Quoted 12 lines
hw/i386/pc.c +10 -0
+
+static void rtc_notify_cpu_added(Notifier *notifier, void *data)
+{
+    rtc_cpu_hotplug_arg *arg = container_of(notifier,
 rtc_cpu_hotplug_arg,
+                                           cpu_added_notifier);
+    ISADevice *s = arg->rtc_state;
+
+    /* increment the number of CPUs */
+    rtc_set_memory(s, 0x5f, rtc_get_memory(s, 0x5f) + 1);
+}

I think this proves Gleb's point of reordering the notifier and CPU resumption since the first instruction executed might theoretically be reading the RTC memory.

I've already changed notifier clall place according to Gleb's suggestions.

Quoted 34 lines

Otherwise looks good with one minor nit...

Quoted 31 lines
hw/i386/pc.c +1 -0
+
 void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
                   const char *boot_device,

ISADevice *floppy, BusState *idebus0, BusState
*idebus1, @@ -345,6 +360,7 @@ void pc_cmos_init(ram_addr_t ram_size,

ram_addr_t above_4g_mem_size, int val, nb, i;
     FDriveType fd_type[2] = { FDRIVE_DRV_NONE, FDRIVE_DRV_NONE };
     static pc_cmos_init_late_arg arg;
hw/i386/pc.c +5 -0
+    static rtc_cpu_hotplug_arg cpu_hotplug_cb;
 
     /* various important CMOS locations needed by PC/Bochs bios */
@@ -383,6 +399,10 @@ void pc_cmos_init(ram_addr_t ram_size, ram_addr_t
 above_4g_mem_size, 
     /* set the number of CPU */
     rtc_set_memory(s, 0x5f, smp_cpus - 1);
+    /* init CPU hotplug notifier */
+    cpu_hotplug_cb.rtc_state = s;
+    cpu_hotplug_cb.cpu_added_notifier.notify = rtc_notify_cpu_added;
+    qemu_register_cpu_added_notifier(&cpu_hotplug_cb.cpu_added_notifier);
 
     /* set boot devices, and disable floppy signature check if requested
 */ if (set_boot_dev(s, boot_device, fd_bootchk)) {
hw/timer/mc146818rtc.c +0 -0
index 69e6844..e639942 100644
@@ -677,6 +677,13 @@ void rtc_set_memory(ISADevice *dev, int addr, int
 val) s->cmos_data[addr] = val;
 }
 
Quoted 7 lines

hw/timer/mc146818rtc.c void rtc_set_memory(ISADevice *dev, int addr, int val) · line 682

Quoted 3 lines
hw/timer/mc146818rtc.c +3 -0
+int rtc_get_memory(ISADevice *dev, int addr)
+{
+    RTCState *s = DO_UPCAST(RTCState, dev, dev);

If we apply my mc146818rtc.c QOM'ification patch to qom-cpu first, this can become MC146818_RTC(dev). Anthony just ack'ed that, but qemu.git is currently broken for testing, so I'm still holding off changes.

if you push it into your qom-cpu tree now, I'll rebase on top of it.

Quoted 23 lines

Andreas

Quoted 19 lines
hw/timer/mc146818rtc.c +4 -0
+    assert(addr >= 0 && addr <= 127);
+    return s->cmos_data[addr];
+}
+
 static void rtc_set_date_from_host(ISADevice *dev)
 {
     RTCState *s = DO_UPCAST(RTCState, dev, dev);
(quoted diff) +0 -0

b/include/hw/timer/mc146818rtc.h index 854ea3f..09f37b7 100644

include/hw/timer/mc146818rtc.h +1 -0
@@ -6,6 +6,7 @@
 
 ISADevice *rtc_init(ISABus *bus, int base_year, qemu_irq intercept_irq);
 void rtc_set_memory(ISADevice *dev, int addr, int val);
+int rtc_get_memory(ISADevice *dev, int addr);
 void rtc_set_date(ISADevice *dev, const struct tm *tm);
 
 #endif /* !MC146818RTC_H */
#34 lore raw

PATCH v4 7/16 cpu: introduce get_arch_id() method and override it for target-i386

In reply to Igor Mammedov

get_arch_id() adds possibility for generic code to get guest visible CPU ID without accessing CPUArchState. If target doesn't override it, it will return cpu_index.

Override it on target-i386 to return APIC ID.

---
* it will be used later by new cpu_exists() generic function and acpi_piix.
* s/cpu_firmware_id/cpu_arch_id/

---
 include/qom/cpu.h |  2 ++
 qom/cpu.c         |  6 ++++++
 target-i386/cpu.c | 10 ++++++++++
 3 files changed, 18 insertions(+)
include/qom/cpu.h +2 -0
index 6ce6f10..428aaf0 100644
@@ -45,6 +45,7 @@ typedef struct CPUState CPUState;
  * instantiatable CPU type.
  * @reset: Callback to reset the #CPUState to its initial state.
  * @do_interrupt: Callback for interrupt handling.
+ * @get_arch_id: Callback for getting architecture depended CPU ID
  * @vmsd: State description for migration.
  *
  * Represents a CPU family or model.
@@ -58,6 +59,7 @@ typedef struct CPUClass {
 
     void (*reset)(CPUState *cpu);
     void (*do_interrupt)(CPUState *cpu);
+    int64_t (*get_arch_id)(CPUState *cpu);
 
     const struct VMStateDescription *vmsd;
 } CPUClass;
qom/cpu.c +6 -0
index 819986e..a13ddda 100644
@@ -76,6 +76,11 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
     }
 }
 
+static int64_t cpu_common_get_arch_id(CPUState *cpu)
+{
+    return cpu->cpu_index;
+}
+
 static void cpu_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
@@ -83,6 +88,7 @@ static void cpu_class_init(ObjectClass *klass, void *data)
 
     k->class_by_name = cpu_common_class_by_name;
     k->reset = cpu_common_reset;
+    k->get_arch_id = cpu_common_get_arch_id;
     dc->realize = cpu_common_realizefn;
     dc->no_user = 1;
 }
target-i386/cpu.c +10 -0
index e2302d8..a415fa3 100644
@@ -2272,6 +2272,14 @@ static void x86_cpu_initfn(Object *obj)
     }
 }
 
+static int64_t x86_cpu_get_arch_id(CPUState *cpu)
+{
+    X86CPU *x86cpu = X86_CPU(cpu);
+    CPUX86State *env = &x86cpu->env;
+
+    return env->cpuid_apic_id;
+}
+
 static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 {
     X86CPUClass *xcc = X86_CPU_CLASS(oc);
@@ -2286,6 +2294,8 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 
     cc->do_interrupt = x86_cpu_do_interrupt;
     cpu_class_set_vmsd(cc, &vmstate_x86_cpu);
+
+    cc->get_arch_id = x86_cpu_get_arch_id;
 }
 
 static const TypeInfo x86_cpu_type_info = {
Signature · 1 line
1.8.2
Reviewed-by on v4 7/16
#35 lore raw

Re: [Qemu-devel] [PATCH 07/16] cpu: introduce get_arch_id() method and override it for target-i386

Quoted 7 lines · Igor Mammedov

get_arch_id() adds possibility for generic code to get guest visible CPU ID without accessing CPUArchState. If target doesn't override it, it will return cpu_index.

Override it on target-i386 to return APIC ID.

Quoted 86 lines

---
* it will be used later by new cpu_exists() generic function and acpi_piix.
* s/cpu_firmware_id/cpu_arch_id/

---
 include/qom/cpu.h |  2 ++
 qom/cpu.c         |  6 ++++++
 target-i386/cpu.c | 10 ++++++++++
 3 files changed, 18 insertions(+)
include/qom/cpu.h +2 -0
index 6ce6f10..428aaf0 100644
@@ -45,6 +45,7 @@ typedef struct CPUState CPUState;
  * instantiatable CPU type.
  * @reset: Callback to reset the #CPUState to its initial state.
  * @do_interrupt: Callback for interrupt handling.
+ * @get_arch_id: Callback for getting architecture depended CPU ID
  * @vmsd: State description for migration.
  *
  * Represents a CPU family or model.
@@ -58,6 +59,7 @@ typedef struct CPUClass {
 
     void (*reset)(CPUState *cpu);
     void (*do_interrupt)(CPUState *cpu);
+    int64_t (*get_arch_id)(CPUState *cpu);
 
     const struct VMStateDescription *vmsd;
 } CPUClass;
qom/cpu.c +6 -0
index 819986e..a13ddda 100644
@@ -76,6 +76,11 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
     }
 }
 
+static int64_t cpu_common_get_arch_id(CPUState *cpu)
+{
+    return cpu->cpu_index;
+}
+
 static void cpu_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
@@ -83,6 +88,7 @@ static void cpu_class_init(ObjectClass *klass, void *data)
 
     k->class_by_name = cpu_common_class_by_name;
     k->reset = cpu_common_reset;
+    k->get_arch_id = cpu_common_get_arch_id;
     dc->realize = cpu_common_realizefn;
     dc->no_user = 1;
 }
target-i386/cpu.c +10 -0
index e2302d8..a415fa3 100644
@@ -2272,6 +2272,14 @@ static void x86_cpu_initfn(Object *obj)
     }
 }
 
+static int64_t x86_cpu_get_arch_id(CPUState *cpu)
+{
+    X86CPU *x86cpu = X86_CPU(cpu);
+    CPUX86State *env = &x86cpu->env;
+
+    return env->cpuid_apic_id;
+}
+
 static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 {
     X86CPUClass *xcc = X86_CPU_CLASS(oc);
@@ -2286,6 +2294,8 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 
     cc->do_interrupt = x86_cpu_do_interrupt;
     cpu_class_set_vmsd(cc, &vmstate_x86_cpu);
+
+    cc->get_arch_id = x86_cpu_get_arch_id;
 }
 
 static const TypeInfo x86_cpu_type_info = {
Signature · 1 line
1.8.2
Signature · 1 line
Eduardo
Reviewed-by on v4 7/16
#36 lore raw

Re: [Qemu-devel] [PATCH 07/16] cpu: introduce get_arch_id() method and override it for target-i386

In reply to Igor Mammedov
Quoted 7 lines · 在 2013-04-16二的 00:12 +0200,Igor Mammedov写道:

get_arch_id() adds possibility for generic code to get guest visible CPU ID without accessing CPUArchState. If target doesn't override it, it will return cpu_index.

Override it on target-i386 to return APIC ID.

Quoted 82 lines

---
* it will be used later by new cpu_exists() generic function and acpi_piix.
* s/cpu_firmware_id/cpu_arch_id/

---
 include/qom/cpu.h |  2 ++
 qom/cpu.c         |  6 ++++++
 target-i386/cpu.c | 10 ++++++++++
 3 files changed, 18 insertions(+)
include/qom/cpu.h +2 -0
index 6ce6f10..428aaf0 100644
@@ -45,6 +45,7 @@ typedef struct CPUState CPUState;
  * instantiatable CPU type.
  * @reset: Callback to reset the #CPUState to its initial state.
  * @do_interrupt: Callback for interrupt handling.
+ * @get_arch_id: Callback for getting architecture depended CPU ID
  * @vmsd: State description for migration.
  *
  * Represents a CPU family or model.
@@ -58,6 +59,7 @@ typedef struct CPUClass {
 
     void (*reset)(CPUState *cpu);
     void (*do_interrupt)(CPUState *cpu);
+    int64_t (*get_arch_id)(CPUState *cpu);
 
     const struct VMStateDescription *vmsd;
 } CPUClass;
qom/cpu.c +6 -0
index 819986e..a13ddda 100644
@@ -76,6 +76,11 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
     }
 }
 
+static int64_t cpu_common_get_arch_id(CPUState *cpu)
+{
+    return cpu->cpu_index;
+}
+
 static void cpu_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
@@ -83,6 +88,7 @@ static void cpu_class_init(ObjectClass *klass, void *data)
 
     k->class_by_name = cpu_common_class_by_name;
     k->reset = cpu_common_reset;
+    k->get_arch_id = cpu_common_get_arch_id;
     dc->realize = cpu_common_realizefn;
     dc->no_user = 1;
 }
target-i386/cpu.c +10 -0
index e2302d8..a415fa3 100644
@@ -2272,6 +2272,14 @@ static void x86_cpu_initfn(Object *obj)
     }
 }
 
+static int64_t x86_cpu_get_arch_id(CPUState *cpu)
+{
+    X86CPU *x86cpu = X86_CPU(cpu);
+    CPUX86State *env = &x86cpu->env;
+
+    return env->cpuid_apic_id;
+}
+
 static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 {
     X86CPUClass *xcc = X86_CPU_CLASS(oc);
@@ -2286,6 +2294,8 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 
     cc->do_interrupt = x86_cpu_do_interrupt;
     cpu_class_set_vmsd(cc, &vmstate_x86_cpu);
+
+    cc->get_arch_id = x86_cpu_get_arch_id;
 }
 
 static const TypeInfo x86_cpu_type_info = {
Acked-by on v4 7/16
#37 lore raw

Re: [Qemu-devel] [PATCH 07/16] cpu: introduce get_arch_id() method and override it for target-i386

In reply to Igor Mammedov
Quoted 7 lines · Igor Mammedov

get_arch_id() adds possibility for generic code to get guest visible CPU ID without accessing CPUArchState. If target doesn't override it, it will return cpu_index.

Override it on target-i386 to return APIC ID.

Will also be helpful for dynamic ACPI Laszlo and me are working on.

Quoted 85 lines

---
* it will be used later by new cpu_exists() generic function and acpi_piix.
* s/cpu_firmware_id/cpu_arch_id/

---
 include/qom/cpu.h |  2 ++
 qom/cpu.c         |  6 ++++++
 target-i386/cpu.c | 10 ++++++++++
 3 files changed, 18 insertions(+)
include/qom/cpu.h +2 -0
index 6ce6f10..428aaf0 100644
@@ -45,6 +45,7 @@ typedef struct CPUState CPUState;
  * instantiatable CPU type.
  * @reset: Callback to reset the #CPUState to its initial state.
  * @do_interrupt: Callback for interrupt handling.
+ * @get_arch_id: Callback for getting architecture depended CPU ID
  * @vmsd: State description for migration.
  *
  * Represents a CPU family or model.
@@ -58,6 +59,7 @@ typedef struct CPUClass {
 
     void (*reset)(CPUState *cpu);
     void (*do_interrupt)(CPUState *cpu);
+    int64_t (*get_arch_id)(CPUState *cpu);
 
     const struct VMStateDescription *vmsd;
 } CPUClass;
qom/cpu.c +6 -0
index 819986e..a13ddda 100644
@@ -76,6 +76,11 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
     }
 }
 
+static int64_t cpu_common_get_arch_id(CPUState *cpu)
+{
+    return cpu->cpu_index;
+}
+
 static void cpu_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
@@ -83,6 +88,7 @@ static void cpu_class_init(ObjectClass *klass, void *data)
 
     k->class_by_name = cpu_common_class_by_name;
     k->reset = cpu_common_reset;
+    k->get_arch_id = cpu_common_get_arch_id;
     dc->realize = cpu_common_realizefn;
     dc->no_user = 1;
 }
target-i386/cpu.c +10 -0
index e2302d8..a415fa3 100644
@@ -2272,6 +2272,14 @@ static void x86_cpu_initfn(Object *obj)
     }
 }
 
+static int64_t x86_cpu_get_arch_id(CPUState *cpu)
+{
+    X86CPU *x86cpu = X86_CPU(cpu);
+    CPUX86State *env = &x86cpu->env;
+
+    return env->cpuid_apic_id;
+}
+
 static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 {
     X86CPUClass *xcc = X86_CPU_CLASS(oc);
@@ -2286,6 +2294,8 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 
     cc->do_interrupt = x86_cpu_do_interrupt;
     cpu_class_set_vmsd(cc, &vmstate_x86_cpu);
+
+    cc->get_arch_id = x86_cpu_get_arch_id;
 }
 
 static const TypeInfo x86_cpu_type_info = {
Signature · 1 line
1.8.2
#38 lore raw

Re: [Qemu-devel] [PATCH 07/16] cpu: introduce get_arch_id() method and override it for target-i386

In reply to Igor Mammedov

Am 16.04.2013 00:12, schrieb Igor Mammedov:

Quoted 11 lines

get_arch_id() adds possibility for generic code to get guest visible CPU ID without accessing CPUArchState. If target doesn't override it, it will return cpu_index.

Override it on target-i386 to return APIC ID.

---
* it will be used later by new cpu_exists() generic function and acpi_piix.
* s/cpu_firmware_id/cpu_arch_id/

Quoted 5 lines
---
 include/qom/cpu.h |  2 ++
 qom/cpu.c         |  6 ++++++
 target-i386/cpu.c | 10 ++++++++++
 3 files changed, 18 insertions(+)

Eduardo and me had discussed on IRC that apparently the only reason we need this generalized "arch_id" is that acpi_piix4 is shared between x86 and mips. Did you investigate whether we can just access the x86 apic-id property instead for x86 and leave mips as-is without hotplug support?

Andreas

Quoted 74 lines
include/qom/cpu.h +2 -0
index 6ce6f10..428aaf0 100644
@@ -45,6 +45,7 @@ typedef struct CPUState CPUState;
  * instantiatable CPU type.
  * @reset: Callback to reset the #CPUState to its initial state.
  * @do_interrupt: Callback for interrupt handling.
+ * @get_arch_id: Callback for getting architecture depended CPU ID
  * @vmsd: State description for migration.
  *
  * Represents a CPU family or model.
@@ -58,6 +59,7 @@ typedef struct CPUClass {
 
     void (*reset)(CPUState *cpu);
     void (*do_interrupt)(CPUState *cpu);
+    int64_t (*get_arch_id)(CPUState *cpu);
 
     const struct VMStateDescription *vmsd;
 } CPUClass;
qom/cpu.c +6 -0
index 819986e..a13ddda 100644
@@ -76,6 +76,11 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
     }
 }
 
+static int64_t cpu_common_get_arch_id(CPUState *cpu)
+{
+    return cpu->cpu_index;
+}
+
 static void cpu_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
@@ -83,6 +88,7 @@ static void cpu_class_init(ObjectClass *klass, void *data)
 
     k->class_by_name = cpu_common_class_by_name;
     k->reset = cpu_common_reset;
+    k->get_arch_id = cpu_common_get_arch_id;
     dc->realize = cpu_common_realizefn;
     dc->no_user = 1;
 }
target-i386/cpu.c +10 -0
index e2302d8..a415fa3 100644
@@ -2272,6 +2272,14 @@ static void x86_cpu_initfn(Object *obj)
     }
 }
 
+static int64_t x86_cpu_get_arch_id(CPUState *cpu)
+{
+    X86CPU *x86cpu = X86_CPU(cpu);
+    CPUX86State *env = &x86cpu->env;
+
+    return env->cpuid_apic_id;
+}
+
 static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 {
     X86CPUClass *xcc = X86_CPU_CLASS(oc);
@@ -2286,6 +2294,8 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 
     cc->do_interrupt = x86_cpu_do_interrupt;
     cpu_class_set_vmsd(cc, &vmstate_x86_cpu);
+
+    cc->get_arch_id = x86_cpu_get_arch_id;
 }
 
 static const TypeInfo x86_cpu_type_info = {
Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#39 lore raw

Re: [Qemu-devel] [PATCH 07/16] cpu: introduce get_arch_id() method and override it for target-i386

Quoted 18 lines · On Mon, 22 Apr 2013 18:33:48 +0200 Andreas Färber <[email protected]> wrote:

Am 16.04.2013 00:12, schrieb Igor Mammedov:

Quoted 11 lines

get_arch_id() adds possibility for generic code to get guest visible CPU ID without accessing CPUArchState. If target doesn't override it, it will return cpu_index.

Override it on target-i386 to return APIC ID.

---
* it will be used later by new cpu_exists() generic function and acpi_piix.
* s/cpu_firmware_id/cpu_arch_id/

Quoted 5 lines
---
 include/qom/cpu.h |  2 ++
 qom/cpu.c         |  6 ++++++
 target-i386/cpu.c | 10 ++++++++++
 3 files changed, 18 insertions(+)
Quoted 4 lines

Eduardo and me had discussed on IRC that apparently the only reason we need this generalized "arch_id" is that acpi_piix4 is shared between x86 and mips. Did you investigate whether we can just access the x86 apic-id property instead for x86 and leave mips as-is without hotplug support?

Yep, one possible way is to used so unloved ifdefs in piix4.c and inclusion of cpu.h in it, which is not nice as well.

BTW:
* if ACPI will be someday used for on ARM then using cpu_arch_id() when building ACPI tables is also justified.
* it could reused in s360 hotplug patches as well, if you looked at my review comment on its CPU hot-plug patch.

Quoted 78 lines

Andreas

Quoted 74 lines
include/qom/cpu.h +2 -0
index 6ce6f10..428aaf0 100644
@@ -45,6 +45,7 @@ typedef struct CPUState CPUState;
  * instantiatable CPU type.
  * @reset: Callback to reset the #CPUState to its initial state.
  * @do_interrupt: Callback for interrupt handling.
+ * @get_arch_id: Callback for getting architecture depended CPU ID
  * @vmsd: State description for migration.
  *
  * Represents a CPU family or model.
@@ -58,6 +59,7 @@ typedef struct CPUClass {
 
     void (*reset)(CPUState *cpu);
     void (*do_interrupt)(CPUState *cpu);
+    int64_t (*get_arch_id)(CPUState *cpu);
 
     const struct VMStateDescription *vmsd;
 } CPUClass;
qom/cpu.c +6 -0
index 819986e..a13ddda 100644
@@ -76,6 +76,11 @@ static void cpu_common_realizefn(DeviceState *dev,
 Error **errp) }
 }
 
+static int64_t cpu_common_get_arch_id(CPUState *cpu)
+{
+    return cpu->cpu_index;
+}
+
 static void cpu_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
@@ -83,6 +88,7 @@ static void cpu_class_init(ObjectClass *klass, void
 *data) 
     k->class_by_name = cpu_common_class_by_name;
     k->reset = cpu_common_reset;
+    k->get_arch_id = cpu_common_get_arch_id;
     dc->realize = cpu_common_realizefn;
     dc->no_user = 1;
 }
target-i386/cpu.c +10 -0
index e2302d8..a415fa3 100644
@@ -2272,6 +2272,14 @@ static void x86_cpu_initfn(Object *obj)
     }
 }
 
+static int64_t x86_cpu_get_arch_id(CPUState *cpu)
+{
+    X86CPU *x86cpu = X86_CPU(cpu);
+    CPUX86State *env = &x86cpu->env;
+
+    return env->cpuid_apic_id;
+}
+
 static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 {
     X86CPUClass *xcc = X86_CPU_CLASS(oc);
@@ -2286,6 +2294,8 @@ static void x86_cpu_common_class_init(ObjectClass
 *oc, void *data) 
     cc->do_interrupt = x86_cpu_do_interrupt;
     cpu_class_set_vmsd(cc, &vmstate_x86_cpu);
+
+    cc->get_arch_id = x86_cpu_get_arch_id;
 }
 
 static const TypeInfo x86_cpu_type_info = {
#40 lore raw

PATCH v4 8/16 cpu: add helper cpu_exists(), to check if CPU with specified id exists

In reply to Igor Mammedov

... it should be used only on slow path since it does recursive search
on /machine QOM tree for objects of TYPE_CPU

---
v2:
* s/get_firmware_id()/get_arch_id()/ rebase fixup
* remove check for get_arch_id being NULL, since it's always defined

---
 include/qom/cpu.h | 10 ++++++++++
 qom/cpu.c         | 21 +++++++++++++++++++++
 2 files changed, 31 insertions(+)
include/qom/cpu.h +10 -0
index 428aaf0..33d5e91 100644
@@ -225,6 +225,16 @@ void run_on_cpu(CPUState *cpu, void (*func)(void *data), void *data);
  */
 CPUState *qemu_get_cpu(int index);
 
+/**
+ * cpu_exists:
+ * @id - guest exposed CPU ID to lookup
+ *
+ * Search for CPU with specified ID.
+ *
+ * Returns: true - CPU is found, false - CPU isn't found
+ */
+bool cpu_exists(int64_t id);
+
 #ifndef CONFIG_USER_ONLY
 
 typedef void (*CPUInterruptHandler)(CPUState *, int);
qom/cpu.c +21 -0
index a13ddda..02fab9a 100644
@@ -24,6 +24,27 @@
 #include "qemu/notify.h"
 #include "sysemu/sysemu.h"
 
+static int cpu_exist_cb(Object *obj, void *opaque)
+{
+    int64_t id = *(int64_t *)opaque;
+    Object *cpu_obj = object_dynamic_cast(obj, TYPE_CPU);
+
+    if (cpu_obj) {
+        CPUState *cpu = CPU(cpu_obj);
+        CPUClass *klass = CPU_GET_CLASS(cpu);
+
+        if (klass->get_arch_id(cpu) == id) {
+            return 1;
+        }
+    }
+    return object_child_foreach(obj, cpu_exist_cb, opaque);
+}
+
+bool cpu_exists(int64_t id)
+{
+   return cpu_exist_cb(qdev_get_machine(), &id) ? true : false;
+}
+
 /* CPU hot-plug notifiers */
 static NotifierList cpu_added_notifiers =
     NOTIFIER_LIST_INITIALIZER(cpu_add_notifiers);
Signature · 1 line
1.8.2
#41 lore raw

Re: [Qemu-devel] [PATCH 08/16] cpu: add helper cpu_exists(), to check if CPU with specified id exists

Quoted 5 lines · Igor Mammedov

... it should be used only on slow path since it does recursive search
on /machine QOM tree for objects of TYPE_CPU

I would prefer qemu_for_each_cpu, that is also useful for ACPI. And do we need to scan all QOM? This will take a while to repeat if there are many devices.
How about the below? You can add a wrapper that sets
a bit if CPU exists.

commit 4ff1332ff56069574f450bb44819156bd91bd105

Author: Michael S. Tsirkin <[email protected]>
Date:   Sun Apr 21 15:44:47 2013 +0300

exec: add qemu_for_each_cpu

Will be used by ACPI table generation.

exec.c +13 -0
index fa1e0c3..2b44cbe 100644
@@ -265,6 +265,19 @@ CPUState *qemu_get_cpu(int index)
     return env ? cpu : NULL;
 }
 
+void qemu_for_each_cpu( void (*func)( CPUState *c, void *arg), void *arg)
+{
+    CPUArchState *env = first_cpu;
+    CPUState *cpu;
+
+    while (env) {
+        cpu = ENV_GET_CPU(env);
+        func(cpu, arg);
+        env = env->next_cpu;
+    }
+}
+
+
 void cpu_exec_init(CPUArchState *env)
 {
     CPUState *cpu = ENV_GET_CPU(env);
include/qom/cpu.h +2 -0
index 3664a1b..db857e3 100644
@@ -223,6 +223,8 @@ void run_on_cpu(CPUState *cpu, void (*func)(void *data), void *data);
  */
 CPUState *qemu_get_cpu(int index);
 
+void qemu_for_each_cpu( void (*func)( CPUState *c, void *arg), void *arg);
+
 #ifndef CONFIG_USER_ONLY
 
 typedef void (*CPUInterruptHandler)(CPUState *, int);
#42 lore raw

Re: [Qemu-devel] [PATCH 08/16] cpu: add helper cpu_exists(), to check if CPU with specified id exists

Quoted 4 lines · On Mon, 22 Apr 2013 13:28:12 +0300 "Michael S. Tsirkin" <[email protected]> wrote:
Quoted 3 lines · Igor Mammedov

... it should be used only on slow path since it does recursive search
on /machine QOM tree for objects of TYPE_CPU

Quoted 8 lines
Quoted 2 lines

I would prefer qemu_for_each_cpu, that is also useful for ACPI. And do we need to scan all QOM? This will take a while to repeat if there are many devices.
How about the below? You can add a wrapper that sets
a bit if CPU exists.

qemu_for_each_cpu looks usefull, it could replace a lot of place that do open-coded exactly this operation and hide open-codded access to next_cpu pointer.

I can easily change to using it, and take it this patch into this series.

Quoted 50 lines

commit 4ff1332ff56069574f450bb44819156bd91bd105

Author: Michael S. Tsirkin <[email protected]>
Date:   Sun Apr 21 15:44:47 2013 +0300

exec: add qemu_for_each_cpu

Will be used by ACPI table generation.

exec.c +13 -0
index fa1e0c3..2b44cbe 100644
@@ -265,6 +265,19 @@ CPUState *qemu_get_cpu(int index)
     return env ? cpu : NULL;
 }
 
+void qemu_for_each_cpu( void (*func)( CPUState *c, void *arg), void *arg)
+{
+    CPUArchState *env = first_cpu;
+    CPUState *cpu;
+
+    while (env) {
+        cpu = ENV_GET_CPU(env);
+        func(cpu, arg);
+        env = env->next_cpu;
+    }
+}
+
+
 void cpu_exec_init(CPUArchState *env)
 {
     CPUState *cpu = ENV_GET_CPU(env);
include/qom/cpu.h +2 -0
index 3664a1b..db857e3 100644
@@ -223,6 +223,8 @@ void run_on_cpu(CPUState *cpu, void (*func)(void
 *data), void *data); */
 CPUState *qemu_get_cpu(int index);
 
+void qemu_for_each_cpu( void (*func)( CPUState *c, void *arg), void *arg);
+
 #ifndef CONFIG_USER_ONLY
 
 typedef void (*CPUInterruptHandler)(CPUState *, int);
#43 lore raw

Re: [Qemu-devel] [PATCH 08/16] cpu: add helper cpu_exists(), to check if CPU with specified id exists

In reply to Michael S. Tsirkin

Am 22.04.2013 12:28, schrieb Michael S. Tsirkin:

Quoted 12 lines · Igor Mammedov
Quoted 5 lines

... it should be used only on slow path since it does recursive search
on /machine QOM tree for objects of TYPE_CPU

I would prefer qemu_for_each_cpu, that is also useful for ACPI. And do we need to scan all QOM? This will take a while to repeat if there are many devices.
How about the below? You can add a wrapper that sets
a bit if CPU exists.

I really like it! Didn't get around to writing such a helper myself yet
- this will allow us to drop quite some CPUArchState usages in the tree. If you can submit this as a proper patch (git-send-email, with a gtk-doc comment, is the spacing intentional?) along with a trivial loop -> helper conversion as proof of concept I'll queue it immediately.

Toda,
Andreas

Quoted 48 lines

commit 4ff1332ff56069574f450bb44819156bd91bd105

Author: Michael S. Tsirkin <[email protected]>
Date:   Sun Apr 21 15:44:47 2013 +0300

exec: add qemu_for_each_cpu

Will be used by ACPI table generation.

exec.c +13 -0
index fa1e0c3..2b44cbe 100644
@@ -265,6 +265,19 @@ CPUState *qemu_get_cpu(int index)
     return env ? cpu : NULL;
 }
 
+void qemu_for_each_cpu( void (*func)( CPUState *c, void *arg), void *arg)
+{
+    CPUArchState *env = first_cpu;
+    CPUState *cpu;
+
+    while (env) {
+        cpu = ENV_GET_CPU(env);
+        func(cpu, arg);
+        env = env->next_cpu;
+    }
+}
+
+
 void cpu_exec_init(CPUArchState *env)
 {
     CPUState *cpu = ENV_GET_CPU(env);
include/qom/cpu.h +2 -0
index 3664a1b..db857e3 100644
@@ -223,6 +223,8 @@ void run_on_cpu(CPUState *cpu, void (*func)(void *data), void *data);
  */
 CPUState *qemu_get_cpu(int index);
 
+void qemu_for_each_cpu( void (*func)( CPUState *c, void *arg), void *arg);
+
 #ifndef CONFIG_USER_ONLY
 
 typedef void (*CPUInterruptHandler)(CPUState *, int);
Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#44 lore raw

PATCH v4 9/16 acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

In reply to Igor Mammedov

* introduce processor status bitmask visible to guest at 0xaf00 addr, where ACPI asl code expects it
* set bit corresponding to APIC ID in processor status bitmask on receiving CPU hot-plug notification
* trigger CPU hot-plug SCI, to notify guest about CPU hot-plug event

---
v3:
* s/get_firmware_id()/get_arch_id()/ due rebase
* s/cpu_add_notifier/cpu_added_notifier/
v2:
* use CPUClass.get_firmware_id() to make code target independent
* bump up vmstate_acpi version

---
 hw/acpi/piix4.c | 116 ++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 113 insertions(+), 3 deletions(-)
hw/acpi/piix4.c +113 -3
index 88386d7..eecdbb0 100644
@@ -48,19 +48,28 @@
 #define PCI_EJ_BASE 0xae08
 #define PCI_RMV_BASE 0xae0c
 
+#define PROC_BASE 0xaf00
+#define PROC_LEN 32
+
 #define PIIX4_PCI_HOTPLUG_STATUS 2
+#define PIIX4_CPU_HOTPLUG_STATUS 4
 
 struct pci_status {
     uint32_t up; /* deprecated, maintained for migration compatibility */
     uint32_t down;
 };
 
+struct cpu_status {
+    uint8_t sts[PROC_LEN];
+};
+
 typedef struct PIIX4PMState {
     PCIDevice dev;
 
     MemoryRegion io;
     MemoryRegion io_gpe;
     MemoryRegion io_pci;
+    MemoryRegion io_cpu;
     ACPIREGS ar;
 
     APMState apm;
@@ -82,6 +91,9 @@ typedef struct PIIX4PMState {
     uint8_t disable_s3;
     uint8_t disable_s4;
     uint8_t s4_val;
+
+    struct cpu_status gpe_cpu;
+    Notifier cpu_added_notifier;
 } PIIX4PMState;
 
 static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
@@ -100,8 +112,8 @@ static void pm_update_sci(PIIX4PMState *s)
                    ACPI_BITMASK_POWER_BUTTON_ENABLE |
                    ACPI_BITMASK_GLOBAL_LOCK_ENABLE |
                    ACPI_BITMASK_TIMER_ENABLE)) != 0) ||
-        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0])
-          & PIIX4_PCI_HOTPLUG_STATUS) != 0);
+        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0]) &
+          (PIIX4_PCI_HOTPLUG_STATUS | PIIX4_CPU_HOTPLUG_STATUS)) != 0);
 
     qemu_set_irq(s->irq, sci_level);
     /* schedule a timer interruption if needed */
@@ -257,6 +269,17 @@ static int acpi_load_old(QEMUFile *f, void *opaque, int version_id)
     return ret;
 }
 
+#define VMSTATE_CPU_STATUS_ARRAY(_field, _state)                             \
+ {                                                                           \
+     .name       = (stringify(_field)),                                      \
+     .version_id = 0,                                                        \
+     .num        = PROC_LEN,                                                 \
+     .info       = &vmstate_info_uint8,                                      \
+     .size       = sizeof(uint8_t),                                          \
+     .flags      = VMS_ARRAY,                                                \
+     .offset     = vmstate_offset_array(_state, _field, uint8_t, PROC_LEN),  \
+ }
+
 /* qemu-kvm 1.2 uses version 3 but advertised as 2
  * To support incoming qemu-kvm 1.2 migration, change version_id
  * and minimum_version_id to 2 below (which breaks migration from
@@ -265,7 +288,7 @@ static int acpi_load_old(QEMUFile *f, void *opaque, int version_id)
  */
 static const VMStateDescription vmstate_acpi = {
     .name = "piix4_pm",
-    .version_id = 3,
+    .version_id = 4,
     .minimum_version_id = 3,
     .minimum_version_id_old = 1,
     .load_state_old = acpi_load_old,
@@ -281,6 +304,7 @@ static const VMStateDescription vmstate_acpi = {
         VMSTATE_STRUCT(ar.gpe, PIIX4PMState, 2, vmstate_gpe, ACPIGPE),
         VMSTATE_STRUCT(pci0_status, PIIX4PMState, 2, vmstate_pci_status,
                        struct pci_status),
+        VMSTATE_CPU_STATUS_ARRAY(gpe_cpu.sts, PIIX4PMState),
         VMSTATE_END_OF_LIST()
     }
 };
@@ -585,6 +609,85 @@ static const MemoryRegionOps piix4_pci_ops = {
     },
 };
 
+static uint64_t cpu_status_read(void *opaque, hwaddr addr, unsigned width)
+{
+    PIIX4PMState *s = opaque;
+    struct cpu_status *cpus = &s->gpe_cpu;
+    uint64_t val = cpus->sts[addr];
+
+    return val;
+}
+
+static void cpu_status_write(void *opaque, hwaddr addr, uint64_t data,
+                             unsigned int size)
+{
+    /* TODO: implement VCPU removal on guest signal that CPU can be removed */
+}
+
+static const MemoryRegionOps cpu_hotplug_ops = {
+    .read = cpu_status_read,
+    .write = cpu_status_write,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+    .valid = {
+        .min_access_size = 1,
+        .max_access_size = 1,
+    },
+};
+
+typedef enum {
+    PLUG,
+    UNPLUG,
+} HotplugEventType;
+
+static void piix4_cpu_hotplug_req(PIIX4PMState *s, CPUState *cpu,
+                                  HotplugEventType action)
+{
+    struct cpu_status *g = &s->gpe_cpu;
+    ACPIGPE *gpe = &s->ar.gpe;
+    CPUClass *k = CPU_GET_CLASS(cpu);
+    int64_t cpu_id;
+
+    assert(s != NULL);
+
+    *gpe->sts = *gpe->sts | PIIX4_CPU_HOTPLUG_STATUS;
+    cpu_id = k->get_arch_id(CPU(cpu));
+    if (action == PLUG) {
+        g->sts[cpu_id / 8] |= (1 << (cpu_id % 8));
+    } else {
+        g->sts[cpu_id / 8] &= ~(1 << (cpu_id % 8));
+    }
+    pm_update_sci(s);
+}
+
+static void piix4_cpu_added_req(Notifier *n, void *opaque)
+{
+    PIIX4PMState *s = container_of(n, PIIX4PMState, cpu_added_notifier);
+
+    piix4_cpu_hotplug_req(s, CPU(opaque), PLUG);
+}
+
+static int piix4_init_cpu_status(Object *obj, void *opaque)
+{
+    struct cpu_status *g = (struct cpu_status *)opaque;
+    Object *cpu_obj = object_dynamic_cast(obj, TYPE_CPU);
+
+    if (cpu_obj) {
+        struct Error *error = NULL;
+        CPUClass *k = CPU_GET_CLASS(cpu_obj);
+        int64_t id = k->get_arch_id(CPU(cpu_obj));
+
+        if (error) {
+            fprintf(stderr, "failed to initilize CPU status for ACPI: %s\n",
+                    error_get_pretty(error));
+            error_free(error);
+            abort();
+        }
+        g_assert((id / 8) < PROC_LEN);
+        g->sts[id / 8] |= (1 << (id % 8));
+    }
+    return object_child_foreach(obj, piix4_init_cpu_status, opaque);
+}
+
 static int piix4_device_hotplug(DeviceState *qdev, PCIDevice *dev,
                                 PCIHotplugState state);
 
@@ -600,6 +703,13 @@ static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
     memory_region_add_subregion(parent, PCI_HOTPLUG_ADDR,
                                 &s->io_pci);
     pci_bus_hotplug(bus, piix4_device_hotplug, &s->dev.qdev);
+
+    piix4_init_cpu_status(qdev_get_machine(), &s->gpe_cpu);
+    memory_region_init_io(&s->io_cpu, &cpu_hotplug_ops, s, "apci-cpu-hotplug",
+                          PROC_LEN);
+    memory_region_add_subregion(parent, PROC_BASE, &s->io_cpu);
+    s->cpu_added_notifier.notify = piix4_cpu_added_req;
+    qemu_register_cpu_added_notifier(&s->cpu_added_notifier);
 }
 
 static void enable_device(PIIX4PMState *s, int slot)
Signature · 1 line
1.8.2
#45 lore raw

Re: [Qemu-devel] [PATCH 09/16] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

Quoted 23 lines · Igor Mammedov

* introduce processor status bitmask visible to guest at 0xaf00 addr, where ACPI asl code expects it
* set bit corresponding to APIC ID in processor status bitmask on receiving CPU hot-plug notification
* trigger CPU hot-plug SCI, to notify guest about CPU hot-plug event

---
v3:
* s/get_firmware_id()/get_arch_id()/ due rebase
* s/cpu_add_notifier/cpu_added_notifier/
v2:
* use CPUClass.get_firmware_id() to make code target independent
* bump up vmstate_acpi version

---
 hw/acpi/piix4.c | 116 ++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 113 insertions(+), 3 deletions(-)
hw/acpi/piix4.c +0 -0
index 88386d7..eecdbb0 100644

hw/acpi/piix4.c line 52

Quoted 5 lines
hw/acpi/piix4.c +2 -0
@@ -48,19 +48,28 @@
 #define PCI_EJ_BASE 0xae08
 #define PCI_RMV_BASE 0xae0c
 
+#define PROC_BASE 0xaf00
+#define PROC_LEN 32

Better prefix macros with PIIX4 or something.

Quoted 166 lines
hw/acpi/piix4.c +104 -3
+
 #define PIIX4_PCI_HOTPLUG_STATUS 2
+#define PIIX4_CPU_HOTPLUG_STATUS 4
 
 struct pci_status {
     uint32_t up; /* deprecated, maintained for migration compatibility */
     uint32_t down;
 };
 
+struct cpu_status {
+    uint8_t sts[PROC_LEN];
+};
+
 typedef struct PIIX4PMState {
     PCIDevice dev;
 
     MemoryRegion io;
     MemoryRegion io_gpe;
     MemoryRegion io_pci;
+    MemoryRegion io_cpu;
     ACPIREGS ar;
 
     APMState apm;
@@ -82,6 +91,9 @@ typedef struct PIIX4PMState {
     uint8_t disable_s3;
     uint8_t disable_s4;
     uint8_t s4_val;
+
+    struct cpu_status gpe_cpu;
+    Notifier cpu_added_notifier;
 } PIIX4PMState;
 
 static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
@@ -100,8 +112,8 @@ static void pm_update_sci(PIIX4PMState *s)
                    ACPI_BITMASK_POWER_BUTTON_ENABLE |
                    ACPI_BITMASK_GLOBAL_LOCK_ENABLE |
                    ACPI_BITMASK_TIMER_ENABLE)) != 0) ||
-        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0])
-          & PIIX4_PCI_HOTPLUG_STATUS) != 0);
+        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0]) &
+          (PIIX4_PCI_HOTPLUG_STATUS | PIIX4_CPU_HOTPLUG_STATUS)) != 0);
 
     qemu_set_irq(s->irq, sci_level);
     /* schedule a timer interruption if needed */
@@ -257,6 +269,17 @@ static int acpi_load_old(QEMUFile *f, void *opaque, int version_id)
     return ret;
 }
 
+#define VMSTATE_CPU_STATUS_ARRAY(_field, _state)                             \
+ {                                                                           \
+     .name       = (stringify(_field)),                                      \
+     .version_id = 0,                                                        \
+     .num        = PROC_LEN,                                                 \
+     .info       = &vmstate_info_uint8,                                      \
+     .size       = sizeof(uint8_t),                                          \
+     .flags      = VMS_ARRAY,                                                \
+     .offset     = vmstate_offset_array(_state, _field, uint8_t, PROC_LEN),  \
+ }
+
 /* qemu-kvm 1.2 uses version 3 but advertised as 2
  * To support incoming qemu-kvm 1.2 migration, change version_id
  * and minimum_version_id to 2 below (which breaks migration from
@@ -265,7 +288,7 @@ static int acpi_load_old(QEMUFile *f, void *opaque, int version_id)
  */
 static const VMStateDescription vmstate_acpi = {
     .name = "piix4_pm",
-    .version_id = 3,
+    .version_id = 4,
     .minimum_version_id = 3,
     .minimum_version_id_old = 1,
     .load_state_old = acpi_load_old,
@@ -281,6 +304,7 @@ static const VMStateDescription vmstate_acpi = {
         VMSTATE_STRUCT(ar.gpe, PIIX4PMState, 2, vmstate_gpe, ACPIGPE),
         VMSTATE_STRUCT(pci0_status, PIIX4PMState, 2, vmstate_pci_status,
                        struct pci_status),
+        VMSTATE_CPU_STATUS_ARRAY(gpe_cpu.sts, PIIX4PMState),
         VMSTATE_END_OF_LIST()
     }
 };
@@ -585,6 +609,85 @@ static const MemoryRegionOps piix4_pci_ops = {
     },
 };
 
+static uint64_t cpu_status_read(void *opaque, hwaddr addr, unsigned width)
+{
+    PIIX4PMState *s = opaque;
+    struct cpu_status *cpus = &s->gpe_cpu;
+    uint64_t val = cpus->sts[addr];
+
+    return val;
+}
+
+static void cpu_status_write(void *opaque, hwaddr addr, uint64_t data,
+                             unsigned int size)
+{
+    /* TODO: implement VCPU removal on guest signal that CPU can be removed */
+}
+
+static const MemoryRegionOps cpu_hotplug_ops = {
+    .read = cpu_status_read,
+    .write = cpu_status_write,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+    .valid = {
+        .min_access_size = 1,
+        .max_access_size = 1,
+    },
+};
+
+typedef enum {
+    PLUG,
+    UNPLUG,
+} HotplugEventType;
+
+static void piix4_cpu_hotplug_req(PIIX4PMState *s, CPUState *cpu,
+                                  HotplugEventType action)
+{
+    struct cpu_status *g = &s->gpe_cpu;
+    ACPIGPE *gpe = &s->ar.gpe;
+    CPUClass *k = CPU_GET_CLASS(cpu);
+    int64_t cpu_id;
+
+    assert(s != NULL);
+
+    *gpe->sts = *gpe->sts | PIIX4_CPU_HOTPLUG_STATUS;
+    cpu_id = k->get_arch_id(CPU(cpu));
+    if (action == PLUG) {
+        g->sts[cpu_id / 8] |= (1 << (cpu_id % 8));
+    } else {
+        g->sts[cpu_id / 8] &= ~(1 << (cpu_id % 8));
+    }
+    pm_update_sci(s);
+}
+
+static void piix4_cpu_added_req(Notifier *n, void *opaque)
+{
+    PIIX4PMState *s = container_of(n, PIIX4PMState, cpu_added_notifier);
+
+    piix4_cpu_hotplug_req(s, CPU(opaque), PLUG);
+}
+
+static int piix4_init_cpu_status(Object *obj, void *opaque)
+{
+    struct cpu_status *g = (struct cpu_status *)opaque;
+    Object *cpu_obj = object_dynamic_cast(obj, TYPE_CPU);
+
+    if (cpu_obj) {
+        struct Error *error = NULL;
+        CPUClass *k = CPU_GET_CLASS(cpu_obj);
+        int64_t id = k->get_arch_id(CPU(cpu_obj));
+
+        if (error) {
+            fprintf(stderr, "failed to initilize CPU status for ACPI: %s\n",
+                    error_get_pretty(error));
+            error_free(error);
+            abort();
+        }
+        g_assert((id / 8) < PROC_LEN);
+        g->sts[id / 8] |= (1 << (id % 8));
+    }
+    return object_child_foreach(obj, piix4_init_cpu_status, opaque);
+}
+
 static int piix4_device_hotplug(DeviceState *qdev, PCIDevice *dev,
                                 PCIHotplugState state);
 

hw/acpi/piix4.c static void piix4_acpi_system_hot_add_init(MemoryRegion *parent, · line 710

Quoted 8 lines
hw/acpi/piix4.c +5 -0
@@ -600,6 +703,13 @@ static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
     memory_region_add_subregion(parent, PCI_HOTPLUG_ADDR,
                                 &s->io_pci);
     pci_bus_hotplug(bus, piix4_device_hotplug, &s->dev.qdev);
+
+    piix4_init_cpu_status(qdev_get_machine(), &s->gpe_cpu);
+    memory_region_init_io(&s->io_cpu, &cpu_hotplug_ops, s, "apci-cpu-hotplug",
+                          PROC_LEN);
+    memory_region_add_subregion(parent, PROC_BASE, &s->io_cpu);

Is there documentation for this new interface?

Quoted 8 lines
hw/acpi/piix4.c +2 -0
+    s->cpu_added_notifier.notify = piix4_cpu_added_req;
+    qemu_register_cpu_added_notifier(&s->cpu_added_notifier);
 }
 
 static void enable_device(PIIX4PMState *s, int slot)
Signature · 1 line
1.8.2
#46 lore raw

PATCH v4 10/16 target-i386: introduce apic-id property

In reply to Igor Mammedov

... and use it from board level to set APIC ID for CPUs it creates.

---
Note:
* pc_new_cpu() function will be reused later in CPU hot-plug hook.

v3:
* user error_propagate() in property setter
v2:
* use generic cpu_exists() instead of custom one
* make apic-id dynamic property, so it won't be possible to use it as global property, since it's instance specific

---
 hw/i386/pc.c      | 25 ++++++++++++++++++++++++-
 target-i386/cpu.c | 42 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 66 insertions(+), 1 deletion(-)
hw/i386/pc.c +24 -1
index dc1a78b..cb57878 100644
@@ -889,9 +889,29 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level)
     }
 }
 
+static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
+{
+    X86CPU *cpu;
+
+    cpu = cpu_x86_create(cpu_model, errp);
+    if (!cpu) {
+        return;
+    }
+
+    object_property_set_int(OBJECT(cpu), apic_id, "apic-id", errp);
+    object_property_set_bool(OBJECT(cpu), true, "realized", errp);
+
+    if (error_is_set(errp)) {
+        if (cpu != NULL) {
+            object_unref(OBJECT(cpu));
+        }
+    }
+}
+
 void pc_cpus_init(const char *cpu_model)
 {
     int i;
+    Error *error = NULL;
 
     /* init CPUs */
     if (cpu_model == NULL) {
@@ -903,7 +923,10 @@ void pc_cpus_init(const char *cpu_model)
     }
 
     for (i = 0; i < smp_cpus; i++) {
-        if (!cpu_x86_init(cpu_model)) {
+        pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
+        if (error) {
+            fprintf(stderr, "%s\n", error_get_pretty(error));
+            error_free(error);
             exit(1);
         }
     }
target-i386/cpu.c +42 -0

54 lines not on this page · raw message

Signature · 1 line
1.8.2
Reviewed-by on v4 10/16
#47 lore raw

Re: [Qemu-devel] [PATCH 10/16] target-i386: introduce apic-id property

Quoted 5 lines · Igor Mammedov

... and use it from board level to set APIC ID for CPUs it creates.

Quoted 124 lines

---
Note:
* pc_new_cpu() function will be reused later in CPU hot-plug hook.

v3:
* user error_propagate() in property setter
v2:
* use generic cpu_exists() instead of custom one
* make apic-id dynamic property, so it won't be possible to use it as global property, since it's instance specific

---
 hw/i386/pc.c      | 25 ++++++++++++++++++++++++-
 target-i386/cpu.c | 42 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 66 insertions(+), 1 deletion(-)
hw/i386/pc.c +24 -1
index dc1a78b..cb57878 100644
@@ -889,9 +889,29 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level)
     }
 }
 
+static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
+{
+    X86CPU *cpu;
+
+    cpu = cpu_x86_create(cpu_model, errp);
+    if (!cpu) {
+        return;
+    }
+
+    object_property_set_int(OBJECT(cpu), apic_id, "apic-id", errp);
+    object_property_set_bool(OBJECT(cpu), true, "realized", errp);
+
+    if (error_is_set(errp)) {
+        if (cpu != NULL) {
+            object_unref(OBJECT(cpu));
+        }
+    }
+}
+
 void pc_cpus_init(const char *cpu_model)
 {
     int i;
+    Error *error = NULL;
 
     /* init CPUs */
     if (cpu_model == NULL) {
@@ -903,7 +923,10 @@ void pc_cpus_init(const char *cpu_model)
     }
 
     for (i = 0; i < smp_cpus; i++) {
-        if (!cpu_x86_init(cpu_model)) {
+        pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
+        if (error) {
+            fprintf(stderr, "%s\n", error_get_pretty(error));
+            error_free(error);
             exit(1);
         }
     }
target-i386/cpu.c +42 -0
index a415fa3..6d6c527 100644
@@ -1271,6 +1271,45 @@ static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, void *opaque,
     cpu->env.tsc_khz = value / 1000;
 }
 
+static void x86_cpuid_get_apic_id(Object *obj, Visitor *v, void *opaque,
+                                  const char *name, Error **errp)
+{
+    X86CPU *cpu = X86_CPU(obj);
+    int64_t value = cpu->env.cpuid_apic_id;
+
+    visit_type_int(v, &value, name, errp);
+}
+
+static void x86_cpuid_set_apic_id(Object *obj, Visitor *v, void *opaque,
+                                  const char *name, Error **errp)
+{
+    X86CPU *cpu = X86_CPU(obj);
+    const int64_t min = 0;
+    const int64_t max = UINT32_MAX;
+    Error *error = NULL;
+    int64_t value;
+
+    visit_type_int(v, &value, name, &error);
+    if (error) {
+        error_propagate(errp, error);
+        return;
+    }
+    if (value < min || value > max) {
+        error_setg(&error, "Property %s.%s doesn't take value %" PRId64
+                   " (minimum: %" PRId64 ", maximum: %" PRId64 ")" ,
+                   object_get_typename(obj), name, value, min, max);
+        error_propagate(errp, error);
+        return;
+    }
+
+    if (cpu_exists(value)) {
+        error_setg(&error, "CPU with APIC ID %" PRIi64 " exists", value);
+        error_propagate(errp, error);
+        return;
+    }
+    cpu->env.cpuid_apic_id = value;
+}
+
 static int cpu_x86_find_by_name(x86_def_t *x86_cpu_def, const char *name)
 {
     x86_def_t *def;
@@ -2259,6 +2298,9 @@ static void x86_cpu_initfn(Object *obj)
     object_property_add(obj, "tsc-frequency", "int",
                         x86_cpuid_get_tsc_freq,
                         x86_cpuid_set_tsc_freq, NULL, NULL, NULL);
+    object_property_add(obj, "apic-id", "int",
+                        x86_cpuid_get_apic_id,
+                        x86_cpuid_set_apic_id, NULL, NULL, NULL);
 
     env->cpuid_apic_id = x86_cpu_apic_id_from_index(cs->cpu_index);
 
Signature · 1 line
1.8.2
#48 lore raw

Re: [Qemu-devel] [PATCH 10/16] target-i386: introduce apic-id property

In reply to Igor Mammedov

Am 16.04.2013 00:12, schrieb Igor Mammedov:

Quoted 8 lines

... and use it from board level to set APIC ID for CPUs it creates.

---
Note:
* pc_new_cpu() function will be reused later in CPU hot-plug hook.

Suggest as main commit message to avoid the "...":

The property is used from board level to set APIC ID for CPUs it creates. Do so in a new pc_new_cpu() helper, to be reused for hot-plug.

Quoted 23 lines

v3:
* user error_propagate() in property setter
v2:
* use generic cpu_exists() instead of custom one
* make apic-id dynamic property, so it won't be possible to use it as global property, since it's instance specific

---
 hw/i386/pc.c      | 25 ++++++++++++++++++++++++-
 target-i386/cpu.c | 42 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 66 insertions(+), 1 deletion(-)
hw/i386/pc.c +3 -0
index dc1a78b..cb57878 100644
@@ -889,9 +889,29 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level)
     }
 }
 
+static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
+{
+    X86CPU *cpu;

hw/i386/pc.c void pc_acpi_smi_interrupt(void *opaque, int irq, int level) · line 904

Quoted 10 lines
hw/i386/pc.c +10 -0
+
+    cpu = cpu_x86_create(cpu_model, errp);
+    if (!cpu) {
+        return;
+    }
+
+    object_property_set_int(OBJECT(cpu), apic_id, "apic-id", errp);
+    object_property_set_bool(OBJECT(cpu), true, "realized", errp);
+
+    if (error_is_set(errp)) {

Please use a local Error* variable with error_propagate, otherwise this is fragile.

Quoted 58 lines
hw/i386/pc.c +11 -1
+        if (cpu != NULL) {
+            object_unref(OBJECT(cpu));
+        }
+    }
+}
+
 void pc_cpus_init(const char *cpu_model)
 {
     int i;
+    Error *error = NULL;
 
     /* init CPUs */
     if (cpu_model == NULL) {
@@ -903,7 +923,10 @@ void pc_cpus_init(const char *cpu_model)
     }
 
     for (i = 0; i < smp_cpus; i++) {
-        if (!cpu_x86_init(cpu_model)) {
+        pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
+        if (error) {
+            fprintf(stderr, "%s\n", error_get_pretty(error));
+            error_free(error);
             exit(1);
         }
     }
target-i386/cpu.c +24 -0
index a415fa3..6d6c527 100644
@@ -1271,6 +1271,45 @@ static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, void *opaque,
     cpu->env.tsc_khz = value / 1000;
 }
 
+static void x86_cpuid_get_apic_id(Object *obj, Visitor *v, void *opaque,
+                                  const char *name, Error **errp)
+{
+    X86CPU *cpu = X86_CPU(obj);
+    int64_t value = cpu->env.cpuid_apic_id;
+
+    visit_type_int(v, &value, name, errp);
+}
+
+static void x86_cpuid_set_apic_id(Object *obj, Visitor *v, void *opaque,
+                                  const char *name, Error **errp)
+{
+    X86CPU *cpu = X86_CPU(obj);
+    const int64_t min = 0;
+    const int64_t max = UINT32_MAX;
+    Error *error = NULL;
+    int64_t value;
+
+    visit_type_int(v, &value, name, &error);
+    if (error) {
+        error_propagate(errp, error);
+        return;
+    }
 
+    if (value < min || value > max) {

target-i386/cpu.c static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, void *opaque, · line 1308

Quoted 10 lines
target-i386/cpu.c +10 -0
+        error_setg(&error, "Property %s.%s doesn't take value %" PRId64
+                   " (minimum: %" PRId64 ", maximum: %" PRId64 ")" ,
+                   object_get_typename(obj), name, value, min, max);
+        error_propagate(errp, error);
+        return;
+    }
+
+    if (cpu_exists(value)) {
+        error_setg(&error, "CPU with APIC ID %" PRIi64 " exists", value);
+        error_propagate(errp, error);

You could do both error_setg()s directly on errp.

Quoted 9 lines
target-i386/cpu.c +5 -0
+        return;
+    }
+    cpu->env.cpuid_apic_id = value;
+}
+
 static int cpu_x86_find_by_name(x86_def_t *x86_cpu_def, const char *name)
 {
     x86_def_t *def;

target-i386/cpu.c static void x86_cpu_initfn(Object *obj) · line 2306

Quoted 9 lines
target-i386/cpu.c +3 -0
@@ -2259,6 +2298,9 @@ static void x86_cpu_initfn(Object *obj)
     object_property_add(obj, "tsc-frequency", "int",
                         x86_cpuid_get_tsc_freq,
                         x86_cpuid_set_tsc_freq, NULL, NULL, NULL);
+    object_property_add(obj, "apic-id", "int",
+                        x86_cpuid_get_apic_id,
+                        x86_cpuid_set_apic_id, NULL, NULL, NULL);
 
     env->cpuid_apic_id = x86_cpu_apic_id_from_index(cs->cpu_index);
 

Otherwise I like this x86-specific version now!

Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#49 lore raw

Re: [Qemu-devel] [PATCH 10/16] target-i386: introduce apic-id property

Quoted 10 lines · On Mon, 22 Apr 2013 16:05:34 +0200 Andreas Färber <[email protected]> wrote:

Am 16.04.2013 00:12, schrieb Igor Mammedov:

Quoted 8 lines

... and use it from board level to set APIC ID for CPUs it creates.

---
Note:
* pc_new_cpu() function will be reused later in CPU hot-plug hook.

Quoted 4 lines

Suggest as main commit message to avoid the "...":

The property is used from board level to set APIC ID for CPUs it creates. Do so in a new pc_new_cpu() helper, to be reused for hot-plug.

I'll do on next respin.

Quoted 20 lines
Quoted 20 lines

v3:
* user error_propagate() in property setter
v2:
* use generic cpu_exists() instead of custom one
* make apic-id dynamic property, so it won't be possible to use it as global property, since it's instance specific

---
 hw/i386/pc.c      | 25 ++++++++++++++++++++++++-
 target-i386/cpu.c | 42 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 66 insertions(+), 1 deletion(-)
hw/i386/pc.c +0 -0
index dc1a78b..cb57878 100644
@@ -889,9 +889,29 @@ void pc_acpi_smi_interrupt(void *opaque, int irq,
 int level) }
 }
 
Quoted 16 lines

hw/i386/pc.c void pc_acpi_smi_interrupt(void *opaque, int irq, int level) · line 904

Quoted 13 lines
hw/i386/pc.c +12 -0
+static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error
 **errp) +{
+    X86CPU *cpu;
+
+    cpu = cpu_x86_create(cpu_model, errp);
+    if (!cpu) {
+        return;
+    }
+
+    object_property_set_int(OBJECT(cpu), apic_id, "apic-id", errp);
+    object_property_set_bool(OBJECT(cpu), true, "realized", errp);
+
+    if (error_is_set(errp)) {

Please use a local Error* variable with error_propagate, otherwise this is fragile.

done

Quoted 70 lines
Quoted 59 lines
hw/i386/pc.c +11 -1
+        if (cpu != NULL) {
+            object_unref(OBJECT(cpu));
+        }
+    }
+}
+
 void pc_cpus_init(const char *cpu_model)
 {
     int i;
+    Error *error = NULL;
 
     /* init CPUs */
     if (cpu_model == NULL) {
@@ -903,7 +923,10 @@ void pc_cpus_init(const char *cpu_model)
     }
 
     for (i = 0; i < smp_cpus; i++) {
-        if (!cpu_x86_init(cpu_model)) {
+        pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
+        if (error) {
+            fprintf(stderr, "%s\n", error_get_pretty(error));
+            error_free(error);
             exit(1);
         }
     }
target-i386/cpu.c +24 -0
index a415fa3..6d6c527 100644
@@ -1271,6 +1271,45 @@ static void x86_cpuid_set_tsc_freq(Object *obj,
 Visitor *v, void *opaque, cpu->env.tsc_khz = value / 1000;
 }
 
+static void x86_cpuid_get_apic_id(Object *obj, Visitor *v, void *opaque,
+                                  const char *name, Error **errp)
+{
+    X86CPU *cpu = X86_CPU(obj);
+    int64_t value = cpu->env.cpuid_apic_id;
+
+    visit_type_int(v, &value, name, errp);
+}
+
+static void x86_cpuid_set_apic_id(Object *obj, Visitor *v, void *opaque,
+                                  const char *name, Error **errp)
+{
+    X86CPU *cpu = X86_CPU(obj);
+    const int64_t min = 0;
+    const int64_t max = UINT32_MAX;
+    Error *error = NULL;
+    int64_t value;
+
+    visit_type_int(v, &value, name, &error);
+    if (error) {
+        error_propagate(errp, error);
+        return;
+    }
 
+    if (value < min || value > max) {

target-i386/cpu.c static void x86_cpuid_set_tsc_freq(Object *obj, · line 1308

Quoted 10 lines
target-i386/cpu.c +10 -0
+        error_setg(&error, "Property %s.%s doesn't take value %" PRId64
+                   " (minimum: %" PRId64 ", maximum: %" PRId64 ")" ,
+                   object_get_typename(obj), name, value, min, max);
+        error_propagate(errp, error);
+        return;
+    }
+
+    if (cpu_exists(value)) {
+        error_setg(&error, "CPU with APIC ID %" PRIi64 " exists", value);
+        error_propagate(errp, error);
Quoted 1 line

You could do both error_setg()s directly on errp.

it was so before, but I was requested to use local error here:
http://lists.gnu.org/archive/html/qemu-devel/2013-04/msg01606.html

Quoted 24 lines
Quoted 10 lines
target-i386/cpu.c +5 -0
+        return;
+    }
+    cpu->env.cpuid_apic_id = value;
+}
+
 static int cpu_x86_find_by_name(x86_def_t *x86_cpu_def, const char *name)
 {
     x86_def_t *def;

target-i386/cpu.c static void x86_cpu_initfn(Object *obj) · line 2306

Quoted 9 lines
target-i386/cpu.c +3 -0
@@ -2259,6 +2298,9 @@ static void x86_cpu_initfn(Object *obj)
     object_property_add(obj, "tsc-frequency", "int",
                         x86_cpuid_get_tsc_freq,
                         x86_cpuid_set_tsc_freq, NULL, NULL, NULL);
+    object_property_add(obj, "apic-id", "int",
+                        x86_cpuid_get_apic_id,
+                        x86_cpuid_set_apic_id, NULL, NULL, NULL);
 
     env->cpuid_apic_id = x86_cpu_apic_id_from_index(cs->cpu_index);
 

Otherwise I like this x86-specific version now!

Andreas

#50 lore raw

Re: [Qemu-devel] [PATCH 10/16] target-i386: introduce apic-id property

Am 22.04.2013 18:30, schrieb Igor Mammedov:

Quoted 127 lines
Quoted 10 lines · On Mon, 22 Apr 2013 16:05:34 +0200 Andreas Färber <[email protected]> wrote:

Am 16.04.2013 00:12, schrieb Igor Mammedov:

Quoted 8 lines

... and use it from board level to set APIC ID for CPUs it creates.

---
Note:
* pc_new_cpu() function will be reused later in CPU hot-plug hook.

Quoted 4 lines

Suggest as main commit message to avoid the "...":

The property is used from board level to set APIC ID for CPUs it creates. Do so in a new pc_new_cpu() helper, to be reused for hot-plug.

I'll do on next respin.

Quoted 20 lines
Quoted 20 lines

v3:
* user error_propagate() in property setter
v2:
* use generic cpu_exists() instead of custom one
* make apic-id dynamic property, so it won't be possible to use it as global property, since it's instance specific

---
 hw/i386/pc.c      | 25 ++++++++++++++++++++++++-
 target-i386/cpu.c | 42 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 66 insertions(+), 1 deletion(-)
hw/i386/pc.c +0 -0
index dc1a78b..cb57878 100644
@@ -889,9 +889,29 @@ void pc_acpi_smi_interrupt(void *opaque, int irq,
 int level) }
 }
 
Quoted 16 lines

hw/i386/pc.c void pc_acpi_smi_interrupt(void *opaque, int irq, int level) · line 904

Quoted 13 lines
hw/i386/pc.c +12 -0
+static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error
 **errp) +{
+    X86CPU *cpu;
+
+    cpu = cpu_x86_create(cpu_model, errp);
+    if (!cpu) {
+        return;
+    }
+
+    object_property_set_int(OBJECT(cpu), apic_id, "apic-id", errp);
+    object_property_set_bool(OBJECT(cpu), true, "realized", errp);
+
+    if (error_is_set(errp)) {

Please use a local Error* variable with error_propagate, otherwise this is fragile.

done

Quoted 70 lines
Quoted 69 lines
hw/i386/pc.c +11 -1
+        if (cpu != NULL) {
+            object_unref(OBJECT(cpu));
+        }
+    }
+}
+
 void pc_cpus_init(const char *cpu_model)
 {
     int i;
+    Error *error = NULL;
 
     /* init CPUs */
     if (cpu_model == NULL) {
@@ -903,7 +923,10 @@ void pc_cpus_init(const char *cpu_model)
     }
 
     for (i = 0; i < smp_cpus; i++) {
-        if (!cpu_x86_init(cpu_model)) {
+        pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
+        if (error) {
+            fprintf(stderr, "%s\n", error_get_pretty(error));
+            error_free(error);
             exit(1);
         }
     }
target-i386/cpu.c +34 -0
index a415fa3..6d6c527 100644
@@ -1271,6 +1271,45 @@ static void x86_cpuid_set_tsc_freq(Object *obj,
 Visitor *v, void *opaque, cpu->env.tsc_khz = value / 1000;
 }
 
+static void x86_cpuid_get_apic_id(Object *obj, Visitor *v, void *opaque,
+                                  const char *name, Error **errp)
+{
+    X86CPU *cpu = X86_CPU(obj);
+    int64_t value = cpu->env.cpuid_apic_id;
+
+    visit_type_int(v, &value, name, errp);
+}
+
+static void x86_cpuid_set_apic_id(Object *obj, Visitor *v, void *opaque,
+                                  const char *name, Error **errp)
+{
+    X86CPU *cpu = X86_CPU(obj);
+    const int64_t min = 0;
+    const int64_t max = UINT32_MAX;
+    Error *error = NULL;
+    int64_t value;
+
+    visit_type_int(v, &value, name, &error);
+    if (error) {
+        error_propagate(errp, error);
+        return;
+    }
 
+    if (value < min || value > max) {
+        error_setg(&error, "Property %s.%s doesn't take value %" PRId64
+                   " (minimum: %" PRId64 ", maximum: %" PRId64 ")" ,
+                   object_get_typename(obj), name, value, min, max);
+        error_propagate(errp, error);
+        return;
+    }
+
+    if (cpu_exists(value)) {
+        error_setg(&error, "CPU with APIC ID %" PRIi64 " exists", value);
+        error_propagate(errp, error);
Quoted 3 lines
Quoted 1 line

You could do both error_setg()s directly on errp.

it was so before, but I was requested to use local error here:
http://lists.gnu.org/archive/html/qemu-devel/2013-04/msg01606.html

The only thing Paolo requested there (and which I +1) is not to do error_is_set(errp). That is independent of error_setg() though, which is followed by a return, so has no functional difference.

Andreas

Quoted 27 lines
Quoted 25 lines
Quoted 11 lines
target-i386/cpu.c +5 -0
+        return;
+    }
+    cpu->env.cpuid_apic_id = value;
+}
+
 static int cpu_x86_find_by_name(x86_def_t *x86_cpu_def, const char *name)
 {
     x86_def_t *def;

target-i386/cpu.c static void x86_cpu_initfn(Object *obj) · line 2306

Quoted 9 lines
target-i386/cpu.c +3 -0
@@ -2259,6 +2298,9 @@ static void x86_cpu_initfn(Object *obj)
     object_property_add(obj, "tsc-frequency", "int",
                         x86_cpuid_get_tsc_freq,
                         x86_cpuid_set_tsc_freq, NULL, NULL, NULL);
+    object_property_add(obj, "apic-id", "int",
+                        x86_cpuid_get_apic_id,
+                        x86_cpuid_set_apic_id, NULL, NULL, NULL);
 
     env->cpuid_apic_id = x86_cpu_apic_id_from_index(cs->cpu_index);
 

Otherwise I like this x86-specific version now!

Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#51 lore raw

PATCH v4 11/16 introduce ICC bus/device/bridge

In reply to Igor Mammedov

... to provide hotplug-able bus.

* icc-bridge will serve as a parent for icc-bus and provide mmio mapping services to child icc-devices.
* icc-device will replace SysBusDevice as a parent of APIC and IOAPIC devices.

---
v2:
* Rebase on top the last HW reorganization series.
* Move hw/icc_bus.c into hw/cpu/ and hw/icc_bus.h include/hw/i386/

---
 default-configs/i386-softmmu.mak   |  1 +
 default-configs/x86_64-softmmu.mak |  1 +
 hw/cpu/Makefile.objs               |  1 +
 hw/cpu/icc_bus.c                   | 88 ++++++++++++++++++++++++++++++++++++++
 hw/i386/pc_piix.c                  |  7 +++
 hw/i386/pc_q35.c                   |  7 +++
 include/hw/i386/icc_bus.h          | 53 +++++++++++++++++++++++
 7 files changed, 158 insertions(+)
 create mode 100644 hw/cpu/icc_bus.c
 create mode 100644 include/hw/i386/icc_bus.h
default-configs/i386-softmmu.mak +1 -0
index 4ac0694..0789303 100644
@@ -45,3 +45,4 @@ CONFIG_LPC_ICH9=y
 CONFIG_Q35=y
 CONFIG_APIC=y
 CONFIG_IOAPIC=y
+CONFIG_ICC_BUS=y
default-configs/x86_64-softmmu.mak +1 -0
index bf4091c..8d03e9a 100644
@@ -45,3 +45,4 @@ CONFIG_LPC_ICH9=y
 CONFIG_Q35=y
 CONFIG_APIC=y
 CONFIG_IOAPIC=y
+CONFIG_ICC_BUS=y
hw/cpu/Makefile.objs +1 -0
index a49ca04..4461ece 100644
@@ -1,4 +1,5 @@
 obj-$(CONFIG_ARM11MPCORE) += arm11mpcore.o
 obj-$(CONFIG_ARM9MPCORE) += a9mpcore.o
 obj-$(CONFIG_ARM15MPCORE) += a15mpcore.o
+obj-$(CONFIG_ICC_BUS) += icc_bus.o
 
hw/cpu/icc_bus.c new file +88 -0

88 lines not on this page · raw message

hw/i386/pc_piix.c +7 -0
index cff8013..d8c3e1a 100644
@@ -37,6 +37,7 @@
 #include "hw/kvm/clock.h"
 #include "sysemu/sysemu.h"
 #include "hw/sysbus.h"
+#include "hw/i386/icc_bus.h"
 #include "sysemu/arch_init.h"
 #include "sysemu/blockdev.h"
 #include "hw/i2c/smbus.h"
@@ -84,8 +85,13 @@ static void pc_init1(MemoryRegion *system_memory,
     MemoryRegion *ram_memory;
     MemoryRegion *pci_memory;
     MemoryRegion *rom_memory;
+    DeviceState *icc_bridge;
     void *fw_cfg = NULL;
 
+    icc_bridge = qdev_create(NULL, TYPE_ICC_BRIDGE);
+    object_property_add_child(qdev_get_machine(), "icc-bridge",
+                              OBJECT(icc_bridge), NULL);
+
     pc_cpus_init(cpu_model);
     pc_acpi_init("acpi-dsdt.aml");
 
@@ -160,6 +166,7 @@ static void pc_init1(MemoryRegion *system_memory,
     if (pci_enabled) {
         ioapic_init_gsi(gsi_state, "i440fx");
     }
+    qdev_init_nofail(icc_bridge);
 
     pc_register_ferr_irq(gsi[13]);
 
hw/i386/pc_q35.c +7 -0
index 6ac1a89..9055145 100644
@@ -41,6 +41,7 @@
 #include "hw/ide/pci.h"
 #include "hw/ide/ahci.h"
 #include "hw/usb.h"
+#include "hw/i386/icc_bus.h"
 
 /* ICH9 AHCI has 6 ports */
 #define MAX_SATA_PORTS     6
@@ -85,6 +86,11 @@ static void pc_q35_init(QEMUMachineInitArgs *args)
     ICH9LPCState *ich9_lpc;
     PCIDevice *ahci;
     qemu_irq *cmos_s3;
+    DeviceState *icc_bridge;
+
+    icc_bridge = qdev_create(NULL, TYPE_ICC_BRIDGE);
+    object_property_add_child(qdev_get_machine(), "icc-bridge",
+                              OBJECT(icc_bridge), NULL);
 
     pc_cpus_init(cpu_model);
     pc_acpi_init("q35-acpi-dsdt.aml");
@@ -168,6 +174,7 @@ static void pc_q35_init(QEMUMachineInitArgs *args)
     if (pci_enabled) {
         ioapic_init_gsi(gsi_state, NULL);
     }
+    qdev_init_nofail(icc_bridge);
 
     pc_register_ferr_irq(gsi[13]);
 
include/hw/i386/icc_bus.h new file +53 -0

53 lines not on this page · raw message

Signature · 1 line
1.8.2
#52 lore raw

Re: [Qemu-devel] [PATCH 11/16] introduce ICC bus/device/bridge

Quoted 7 lines · Igor Mammedov

... to provide hotplug-able bus.

* icc-bridge will serve as a parent for icc-bus and provide mmio mapping services to child icc-devices.
* icc-device will replace SysBusDevice as a parent of APIC and IOAPIC devices.

Quoted 5 lines

---
v2:
* Rebase on top the last HW reorganization series.
* Move hw/icc_bus.c into hw/cpu/ and hw/icc_bus.h include/hw/i386/

This seems inconsistent. I think the header should be in include/hw/cpu/ to match the location of the .c file.

thanks
-- PMM

#53 lore raw

Re: [Qemu-devel] [PATCH 11/16] introduce ICC bus/device/bridge

Il 22/04/2013 13:39, Peter Maydell ha scritto:

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

... to provide hotplug-able bus.

* icc-bridge will serve as a parent for icc-bus and provide mmio mapping services to child icc-devices.
* icc-device will replace SysBusDevice as a parent of APIC and IOAPIC devices.

Quoted 7 lines
Quoted 5 lines
Quoted 5 lines

---
v2:
* Rebase on top the last HW reorganization series.
* Move hw/icc_bus.c into hw/cpu/ and hw/icc_bus.h include/hw/i386/

This seems inconsistent. I think the header should be in include/hw/cpu/ to match the location of the .c file.

Agreed.

Paolo

#54 lore raw

Re: [Qemu-devel] [PATCH 11/16] introduce ICC bus/device/bridge

In reply to Igor Mammedov

Am 16.04.2013 00:12, schrieb Igor Mammedov:

Quoted 7 lines

... to provide hotplug-able bus.

* icc-bridge will serve as a parent for icc-bus and provide mmio mapping services to child icc-devices.
* icc-device will replace SysBusDevice as a parent of APIC and IOAPIC devices.

Quoted 5 lines

---
v2:
* Rebase on top the last HW reorganization series.
* Move hw/icc_bus.c into hw/cpu/ and hw/icc_bus.h include/hw/i386/

http://wiki.qemu.org/QOMConventions

Comments below...

Quoted 46 lines
---
 default-configs/i386-softmmu.mak   |  1 +
 default-configs/x86_64-softmmu.mak |  1 +
 hw/cpu/Makefile.objs               |  1 +
 hw/cpu/icc_bus.c                   | 88 ++++++++++++++++++++++++++++++++++++++
 hw/i386/pc_piix.c                  |  7 +++
 hw/i386/pc_q35.c                   |  7 +++
 include/hw/i386/icc_bus.h          | 53 +++++++++++++++++++++++
 7 files changed, 158 insertions(+)
 create mode 100644 hw/cpu/icc_bus.c
 create mode 100644 include/hw/i386/icc_bus.h
default-configs/i386-softmmu.mak +1 -0
index 4ac0694..0789303 100644
@@ -45,3 +45,4 @@ CONFIG_LPC_ICH9=y
 CONFIG_Q35=y
 CONFIG_APIC=y
 CONFIG_IOAPIC=y
+CONFIG_ICC_BUS=y
default-configs/x86_64-softmmu.mak +1 -0
index bf4091c..8d03e9a 100644
@@ -45,3 +45,4 @@ CONFIG_LPC_ICH9=y
 CONFIG_Q35=y
 CONFIG_APIC=y
 CONFIG_IOAPIC=y
+CONFIG_ICC_BUS=y
hw/cpu/Makefile.objs +1 -0
index a49ca04..4461ece 100644
@@ -1,4 +1,5 @@
 obj-$(CONFIG_ARM11MPCORE) += arm11mpcore.o
 obj-$(CONFIG_ARM9MPCORE) += a9mpcore.o
 obj-$(CONFIG_ARM15MPCORE) += a15mpcore.o
+obj-$(CONFIG_ICC_BUS) += icc_bus.o
 
hw/cpu/icc_bus.c new file +0 -0
new file mode 100644
index 0000000..00b9be3

hw/cpu/icc_bus.c line 2

Quoted 2 lines
hw/cpu/icc_bus.c new file +2 -0
@@ -0,0 +1,88 @@
+/* icc_bus.c
+ * emulate x86 ICC(INTERRUPT CONTROLLER COMMUNICATIONS) bus

Does this need to be all-capitals? :)

Quoted 15 lines
hw/cpu/icc_bus.c new file +15 -0
+ *
+ * Copyright (c) 2013 Red Hat, Inc
+ *
+ * Authors:
+ *     Igor Mammedov <[email protected]>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.

hw/cpu/icc_bus.c line 27

Quoted 10 lines
hw/cpu/icc_bus.c new file +10 -0
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>
+ */
+#include "hw/i386/icc_bus.h"
+#include "hw/sysbus.h"
+
+static void icc_bus_initfn(Object *obj)
+{
+    BusState *b = BUS(obj);

Whiteline as below?

Quoted 13 lines
hw/cpu/icc_bus.c new file +13 -0
+    b->allow_hotplug = true;
+}
+
+static const TypeInfo icc_bus_info = {
+    .name = TYPE_ICC_BUS,
+    .parent = TYPE_BUS,
+    .instance_size = sizeof(ICCBus),
+    .instance_init = icc_bus_initfn,
+};
+
+static int icc_device_init(DeviceState *dev)
+{
+    ICCDevice *id = ICC_DEVICE(dev);

hw/cpu/icc_bus.c line 50

Quoted 10 lines
hw/cpu/icc_bus.c new file +10 -0
+    ICCDeviceClass *k = ICC_DEVICE_GET_CLASS(id);
+
+    return k->init(id);
+}
+
+static void icc_device_class_init(ObjectClass *klass, void *data)
+{
+    DeviceClass *k = DEVICE_CLASS(klass);
+
+    k->init = icc_device_init;

Please use k->realize for any new base device type or type derived from SysBusDevice.

If there's nothing to do at this level, you can just provide a dummy realize function here to override Device's and override it (without saving the parent's dummy function) in IOAPIC or wherever you need it.

Quoted 4 lines
hw/cpu/icc_bus.c new file +4 -0
+    k->bus_type = TYPE_ICC_BUS;
+}
+
+static const TypeInfo icc_device_info = {

hw/cpu/icc_bus.c line 64

Quoted 10 lines
hw/cpu/icc_bus.c new file +10 -0
+    .name = TYPE_ICC_DEVICE,
+    .parent = TYPE_DEVICE,
+    .abstract = true,
+    .instance_size = sizeof(ICCDevice),
+    .class_size = sizeof(ICCDeviceClass),
+    .class_init = icc_device_class_init,
+};
+
+typedef struct ICCBridgeState {
+    SysBusDevice busdev;

parent_obj please - makes it clear that it is a QOM struct and flags accidental uses of obsolete SysBus macros.

hw/cpu/icc_bus.c line 71

Quoted 7 lines
hw/cpu/icc_bus.c new file +7 -0
+} ICCBridgeState;
+#define ICC_BRIGDE(obj) OBJECT_CHECK(ICCBridgeState, (obj), TYPE_ICC_BRIDGE)
+
+
+static void icc_bridge_initfn(Object *obj)
+{
+    qbus_create(TYPE_ICC_BUS, DEVICE(obj), "icc-bus");

qbus_create_inplace()?

hw/cpu/icc_bus.c line 75

Quoted 4 lines
hw/cpu/icc_bus.c new file +4 -0
+}
+
+static const TypeInfo icc_bridge_info = {
+    .name  = "icc-bridge",

TYPE_ICC_BRIDGE?

hw/cpu/icc_bus.c line 88

Quoted 13 lines
hw/cpu/icc_bus.c new file +13 -0
+    .parent = TYPE_SYS_BUS_DEVICE,
+    .instance_init  = icc_bridge_initfn,
+    .instance_size  = sizeof(ICCBridgeState),
+};
+
+static void icc_bus_register_types(void)
+{
+    type_register_static(&icc_bus_info);
+    type_register_static(&icc_device_info);
+    type_register_static(&icc_bridge_info);
+}
+
+type_init(icc_bus_register_types)

BTW Anthony recently suggested to drop the "fn" where there's no init vs. initfn name conflicts; I don't mind either way.

Quoted 92 lines
hw/i386/pc_piix.c +7 -0
index cff8013..d8c3e1a 100644
@@ -37,6 +37,7 @@
 #include "hw/kvm/clock.h"
 #include "sysemu/sysemu.h"
 #include "hw/sysbus.h"
+#include "hw/i386/icc_bus.h"
 #include "sysemu/arch_init.h"
 #include "sysemu/blockdev.h"
 #include "hw/i2c/smbus.h"
@@ -84,8 +85,13 @@ static void pc_init1(MemoryRegion *system_memory,
     MemoryRegion *ram_memory;
     MemoryRegion *pci_memory;
     MemoryRegion *rom_memory;
+    DeviceState *icc_bridge;
     void *fw_cfg = NULL;
 
+    icc_bridge = qdev_create(NULL, TYPE_ICC_BRIDGE);
+    object_property_add_child(qdev_get_machine(), "icc-bridge",
+                              OBJECT(icc_bridge), NULL);
+
     pc_cpus_init(cpu_model);
     pc_acpi_init("acpi-dsdt.aml");
 
@@ -160,6 +166,7 @@ static void pc_init1(MemoryRegion *system_memory,
     if (pci_enabled) {
         ioapic_init_gsi(gsi_state, "i440fx");
     }
+    qdev_init_nofail(icc_bridge);
 
     pc_register_ferr_irq(gsi[13]);
 
hw/i386/pc_q35.c +7 -0
index 6ac1a89..9055145 100644
@@ -41,6 +41,7 @@
 #include "hw/ide/pci.h"
 #include "hw/ide/ahci.h"
 #include "hw/usb.h"
+#include "hw/i386/icc_bus.h"
 
 /* ICH9 AHCI has 6 ports */
 #define MAX_SATA_PORTS     6
@@ -85,6 +86,11 @@ static void pc_q35_init(QEMUMachineInitArgs *args)
     ICH9LPCState *ich9_lpc;
     PCIDevice *ahci;
     qemu_irq *cmos_s3;
+    DeviceState *icc_bridge;
+
+    icc_bridge = qdev_create(NULL, TYPE_ICC_BRIDGE);
+    object_property_add_child(qdev_get_machine(), "icc-bridge",
+                              OBJECT(icc_bridge), NULL);
 
     pc_cpus_init(cpu_model);
     pc_acpi_init("q35-acpi-dsdt.aml");
@@ -168,6 +174,7 @@ static void pc_q35_init(QEMUMachineInitArgs *args)
     if (pci_enabled) {
         ioapic_init_gsi(gsi_state, NULL);
     }
+    qdev_init_nofail(icc_bridge);
 
     pc_register_ferr_irq(gsi[13]);
 
include/hw/i386/icc_bus.h new file +20 -0
new file mode 100644
index 0000000..aa30e0c
@@ -0,0 +1,53 @@
+/* icc_bus.h
+ * emulate x86 ICC(INTERRUPT CONTROLLER COMMUNICATIONS) bus
+ *
+ * Copyright (c) 2013 Red Hat, Inc
+ *
+ * Authors:
+ *     Igor Mammedov <[email protected]>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>

include/hw/i386/icc_bus.h line 30

Quoted 10 lines
include/hw/i386/icc_bus.h new file +10 -0
+ */
+#ifndef ICC_BUS_H
+#define ICC_BUS_H
+
+#include "hw/qdev-core.h"
+
+#define TYPE_ICC_BUS "icc-bus"
+
+#ifndef CONFIG_USER_ONLY
+typedef struct ICCBus {

/*< private >*/

include/hw/i386/icc_bus.h line 31

Quoted 1 line
include/hw/i386/icc_bus.h new file +1 -0
+    BusState qbus;

parent_obj

/*< public >*/

...since it is in a header.

include/hw/i386/icc_bus.h line 36

Quoted 5 lines
include/hw/i386/icc_bus.h new file +5 -0
+} ICCBus;
+#define ICC_BUS(obj) OBJECT_CHECK(ICCBus, (obj), TYPE_ICC_BUS)
+
+typedef struct ICCDevice {
+    DeviceState qdev;

dito

include/hw/i386/icc_bus.h line 41

Quoted 5 lines
include/hw/i386/icc_bus.h new file +5 -0
+} ICCDevice;
+
+typedef struct ICCDeviceClass {
+    DeviceClass parent_class;
+    int (*init)(ICCDevice *dev);

Please read the realize documentation in qdev-core.h - instead of having the device provide a specialized init hook, it should override (and if necessary, save) the parent's realize callback.

include/hw/i386/icc_bus.h line 53

Quoted 12 lines
include/hw/i386/icc_bus.h new file +12 -0
+} ICCDeviceClass;
+#define TYPE_ICC_DEVICE "icc-device"
+#define ICC_DEVICE(obj) OBJECT_CHECK(ICCDevice, (obj), TYPE_ICC_DEVICE)
+#define ICC_DEVICE_CLASS(klass) \
+     OBJECT_CLASS_CHECK(ICCDeviceClass, (klass), TYPE_ICC_DEVICE)
+#define ICC_DEVICE_GET_CLASS(obj) \
+     OBJECT_GET_CLASS(ICCDeviceClass, (obj), TYPE_ICC_DEVICE)
+
+#define TYPE_ICC_BRIDGE "icc-bridge"
+
+#endif /* CONFIG_USER_ONLY */
+#endif

General concept looks fine to me. Maybe add a bit more whiteline spacing between struct and macros and between different QOM types in .c.

Who becomes the MAINTAINER of ICC?

Regards,
Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#55 lore raw

Re: [Qemu-devel] [PATCH 11/16] introduce ICC bus/device/bridge

Quoted 18 lines · On Mon, 22 Apr 2013 15:22:30 +0200 Andreas Färber <[email protected]> wrote:

Am 16.04.2013 00:12, schrieb Igor Mammedov:

Quoted 7 lines

... to provide hotplug-able bus.

* icc-bridge will serve as a parent for icc-bus and provide mmio mapping services to child icc-devices.
* icc-device will replace SysBusDevice as a parent of APIC and IOAPIC devices.

Quoted 5 lines

---
v2:
* Rebase on top the last HW reorganization series.
* Move hw/icc_bus.c into hw/cpu/ and hw/icc_bus.h include/hw/i386/

http://wiki.qemu.org/QOMConventions

Comments below...

[...]

Quoted 10 lines

hw/cpu/icc_bus.c line 50

Quoted 6 lines
hw/cpu/icc_bus.c new file +6 -0
+
+static void icc_device_class_init(ObjectClass *klass, void *data)
+{
+    DeviceClass *k = DEVICE_CLASS(klass);
+
+    k->init = icc_device_init;

Please use k->realize for any new base device type or type derived from SysBusDevice.

Quoted 4 lines

If there's nothing to do at this level, you can just provide a dummy realize function here to override Device's and override it (without saving the parent's dummy function) in IOAPIC or wherever you need it.

I'll try it, and if changes isn't big, I'll merge into this patch.

Quoted 9 lines
Quoted 9 lines
hw/cpu/icc_bus.c new file +9 -0
+    k->bus_type = TYPE_ICC_BUS;
+}
+
+static const TypeInfo icc_device_info = {
+    .name = TYPE_ICC_DEVICE,
+    .parent = TYPE_DEVICE,
+    .abstract = true,
+    .instance_size = sizeof(ICCDevice),
+    .class_size = sizeof(ICCDeviceClass),
Quoted 8 lines

hw/cpu/icc_bus.c line 64

Quoted 5 lines
hw/cpu/icc_bus.c new file +5 -0
+    .class_init = icc_device_class_init,
+};
+
+typedef struct ICCBridgeState {
+    SysBusDevice busdev;

parent_obj please - makes it clear that it is a QOM struct and flags accidental uses of obsolete SysBus macros.

done

Quoted 10 lines

hw/cpu/icc_bus.c line 71

Quoted 8 lines
(quoted diff) +2 -0
+} ICCBridgeState;
+#define ICC_BRIGDE(obj) OBJECT_CHECK(ICCBridgeState, (obj),

TYPE_ICC_BRIDGE) +

hw/cpu/icc_bus.c new file +4 -0
+
+static void icc_bridge_initfn(Object *obj)
+{
+    qbus_create(TYPE_ICC_BUS, DEVICE(obj), "icc-bus");

qbus_create_inplace()?

done

[...]

Quoted 3 lines

BTW Anthony recently suggested to drop the "fn" where there's no init vs. initfn name conflicts; I don't mind either way.

changed to init

[...]

Quoted 6 lines

/*< private >*/

Quoted 1 line

+ BusState qbus;

parent_obj

done

Quoted 5 lines

/*< public >*/

...since it is in a header.

Quoted 7 lines

include/hw/i386/icc_bus.h line 36

Quoted 5 lines
include/hw/i386/icc_bus.h new file +5 -0
+} ICCBus;
+#define ICC_BUS(obj) OBJECT_CHECK(ICCBus, (obj), TYPE_ICC_BUS)
+
+typedef struct ICCDevice {
+    DeviceState qdev;

dito

s/qdev/parent_obj/ will affect APIC and IOAPIC patches make-ing them even bigger and more difficult to review distracting from real changes.

it would be better to send follow-up patch, that does trivial QOM conversions afterwards.

I'm not sure that making ICCDevice.qdev private would be good performance wise since it would require using DEVICE() dynamic cast in apic_check_pic().

I'd like to keep this as is for now and put cleanup patch on TODO list for 1.6 to get rid of DO_UPCAST() macro in apic/ioapic files.

Quoted 2 lines

Who becomes the MAINTAINER of ICC?

Perhaps me??

Quoted 4 lines

Regards,
Andreas

#56 lore raw

PATCH v4 12/16 target-i386: cpu: attach ICC bus to CPU on its creation

In reply to Igor Mammedov

... during startup, so it would be possible to unplug it later and set bus_type to TYPE_ICC_BUS for X86CPU type to make device_add attach hotplugged CPU to ICC bus.

---
 target-i386/cpu.c | 7 +++++++
 1 file changed, 7 insertions(+)
target-i386/cpu.c +7 -0
index 6d6c527..3b5f90b 100644
@@ -41,6 +41,7 @@
 #endif
 
 #include "sysemu/sysemu.h"
+#include "hw/i386/icc_bus.h"
 #ifndef CONFIG_USER_ONLY
 #include "hw/xen/xen.h"
 #include "hw/sysbus.h"
@@ -1609,6 +1610,7 @@ X86CPU *cpu_x86_create(const char *cpu_model, Error **errp)
     gchar **model_pieces;
     char *name, *features;
     Error *error = NULL;
+    Object *icc_bus = object_resolve_path_type("icc-bus", TYPE_ICC_BUS, NULL);
 
     model_pieces = g_strsplit(cpu_model, ",", 2);
     if (!model_pieces[0]) {
@@ -1619,6 +1621,10 @@ X86CPU *cpu_x86_create(const char *cpu_model, Error **errp)
     features = model_pieces[1];
 
     cpu = X86_CPU(object_new(TYPE_X86_CPU));
+    if (icc_bus) {
+        qdev_set_parent_bus(DEVICE(cpu), BUS(icc_bus));
+        object_unref(OBJECT(cpu));
+    }
     env = &cpu->env;
     env->cpu_model_str = cpu_model;
 
@@ -2330,6 +2336,7 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 
     xcc->parent_realize = dc->realize;
     dc->realize = x86_cpu_realizefn;
+    dc->bus_type = TYPE_ICC_BUS;
 
     xcc->parent_reset = cc->reset;
     cc->reset = x86_cpu_reset;
Signature · 1 line
1.8.2
Reviewed-by on v4 12/16
#57 lore raw

Re: [Qemu-devel] [PATCH 12/16] target-i386: cpu: attach ICC bus to CPU on its creation

Am 16.04.2013 00:12, schrieb Igor Mammedov:

Quoted 5 lines

... during startup, so it would be possible to unplug it later and set bus_type to TYPE_ICC_BUS for X86CPU type to make device_add attach hotplugged CPU to ICC bus.

But still one question...

Quoted 25 lines
---
 target-i386/cpu.c | 7 +++++++
 1 file changed, 7 insertions(+)
target-i386/cpu.c +2 -0
index 6d6c527..3b5f90b 100644
@@ -41,6 +41,7 @@
 #endif
 
 #include "sysemu/sysemu.h"
+#include "hw/i386/icc_bus.h"
 #ifndef CONFIG_USER_ONLY
 #include "hw/xen/xen.h"
 #include "hw/sysbus.h"
@@ -1609,6 +1610,7 @@ X86CPU *cpu_x86_create(const char *cpu_model, Error **errp)
     gchar **model_pieces;
     char *name, *features;
     Error *error = NULL;
+    Object *icc_bus = object_resolve_path_type("icc-bus", TYPE_ICC_BUS, NULL);
 
     model_pieces = g_strsplit(cpu_model, ",", 2);
     if (!model_pieces[0]) {

target-i386/cpu.c X86CPU *cpu_x86_create(const char *cpu_model, Error **errp) · line 1630

Quoted 10 lines
target-i386/cpu.c +4 -0
@@ -1619,6 +1621,10 @@ X86CPU *cpu_x86_create(const char *cpu_model, Error **errp)
     features = model_pieces[1];
 
     cpu = X86_CPU(object_new(TYPE_X86_CPU));
+    if (icc_bus) {
+        qdev_set_parent_bus(DEVICE(cpu), BUS(icc_bus));
+        object_unref(OBJECT(cpu));
+    }
     env = &cpu->env;
     env->cpu_model_str = cpu_model;
 

You seem to be avoiding making lack of icc-bus an error although you add it for both PC and q35 PC - which non-ICC use cases are you thinking of?

Andreas

Quoted 9 lines
target-i386/cpu.c +1 -0
@@ -2330,6 +2336,7 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 
     xcc->parent_realize = dc->realize;
     dc->realize = x86_cpu_realizefn;
+    dc->bus_type = TYPE_ICC_BUS;
 
     xcc->parent_reset = cc->reset;
     cc->reset = x86_cpu_reset;
Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#58 lore raw

Re: [Qemu-devel] [PATCH 12/16] target-i386: cpu: attach ICC bus to CPU on its creation

Quoted 38 lines · On Mon, 22 Apr 2013 17:02:33 +0200 Andreas Färber <[email protected]> wrote:

Am 16.04.2013 00:12, schrieb Igor Mammedov:

Quoted 5 lines

... during startup, so it would be possible to unplug it later and set bus_type to TYPE_ICC_BUS for X86CPU type to make device_add attach hotplugged CPU to ICC bus.

But still one question...

Quoted 27 lines
---
 target-i386/cpu.c | 7 +++++++
 1 file changed, 7 insertions(+)
target-i386/cpu.c +2 -0
index 6d6c527..3b5f90b 100644
@@ -41,6 +41,7 @@
 #endif
 
 #include "sysemu/sysemu.h"
+#include "hw/i386/icc_bus.h"
 #ifndef CONFIG_USER_ONLY
 #include "hw/xen/xen.h"
 #include "hw/sysbus.h"
@@ -1609,6 +1610,7 @@ X86CPU *cpu_x86_create(const char *cpu_model, Error
 **errp) gchar **model_pieces;
     char *name, *features;
     Error *error = NULL;
+    Object *icc_bus = object_resolve_path_type("icc-bus", TYPE_ICC_BUS,
 NULL); 
     model_pieces = g_strsplit(cpu_model, ",", 2);
     if (!model_pieces[0]) {
@@ -1619,6 +1621,10 @@ X86CPU *cpu_x86_create(const char *cpu_model,
 Error **errp) features = model_pieces[1];
 
Quoted 11 lines

target-i386/cpu.c X86CPU *cpu_x86_create(const char *cpu_model, Error **errp) · line 1630

Quoted 8 lines
target-i386/cpu.c +4 -0
     cpu = X86_CPU(object_new(TYPE_X86_CPU));
+    if (icc_bus) {
+        qdev_set_parent_bus(DEVICE(cpu), BUS(icc_bus));
+        object_unref(OBJECT(cpu));
+    }
     env = &cpu->env;
     env->cpu_model_str = cpu_model;
 

You seem to be avoiding making lack of icc-bus an error although you add it for both PC and q35 PC - which non-ICC use cases are you thinking of?

*-user targets, CPU is bus-less there.

Quoted 14 lines

Andreas

Quoted 9 lines
target-i386/cpu.c +1 -0
@@ -2330,6 +2336,7 @@ static void x86_cpu_common_class_init(ObjectClass
 *oc, void *data) 
     xcc->parent_realize = dc->realize;
     dc->realize = x86_cpu_realizefn;
+    dc->bus_type = TYPE_ICC_BUS;
 
     xcc->parent_reset = cc->reset;
     cc->reset = x86_cpu_reset;
#59 lore raw

Re: [Qemu-devel] [PATCH 12/16] target-i386: cpu: attach ICC bus to CPU on its creation

Am 22.04.2013 17:20, schrieb Igor Mammedov:

Quoted 52 lines
Quoted 38 lines · On Mon, 22 Apr 2013 17:02:33 +0200 Andreas Färber <[email protected]> wrote:

Am 16.04.2013 00:12, schrieb Igor Mammedov:

Quoted 5 lines

... during startup, so it would be possible to unplug it later and set bus_type to TYPE_ICC_BUS for X86CPU type to make device_add attach hotplugged CPU to ICC bus.

But still one question...

Quoted 27 lines
---
 target-i386/cpu.c | 7 +++++++
 1 file changed, 7 insertions(+)
target-i386/cpu.c +2 -0
index 6d6c527..3b5f90b 100644
@@ -41,6 +41,7 @@
 #endif
 
 #include "sysemu/sysemu.h"
+#include "hw/i386/icc_bus.h"
 #ifndef CONFIG_USER_ONLY
 #include "hw/xen/xen.h"
 #include "hw/sysbus.h"
@@ -1609,6 +1610,7 @@ X86CPU *cpu_x86_create(const char *cpu_model, Error
 **errp) gchar **model_pieces;
     char *name, *features;
     Error *error = NULL;
+    Object *icc_bus = object_resolve_path_type("icc-bus", TYPE_ICC_BUS,
 NULL); 
     model_pieces = g_strsplit(cpu_model, ",", 2);
     if (!model_pieces[0]) {
@@ -1619,6 +1621,10 @@ X86CPU *cpu_x86_create(const char *cpu_model,
 Error **errp) features = model_pieces[1];
 
Quoted 11 lines

target-i386/cpu.c X86CPU *cpu_x86_create(const char *cpu_model, Error **errp) · line 1630

Quoted 8 lines
target-i386/cpu.c +4 -0
     cpu = X86_CPU(object_new(TYPE_X86_CPU));
+    if (icc_bus) {
+        qdev_set_parent_bus(DEVICE(cpu), BUS(icc_bus));
+        object_unref(OBJECT(cpu));
+    }
     env = &cpu->env;
     env->cpu_model_str = cpu_model;
 

You seem to be avoiding making lack of icc-bus an error although you add it for both PC and q35 PC - which non-ICC use cases are you thinking of?

Quoted 1 line

*-user targets, CPU is bus-less there.

Would you be opposed to #ifdef'ing it then? In particular I am asking because of the NULL error argument to the resolve function - for softmmu I would prefer to make lack of icc-bus a fatal error.

Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#60 lore raw

Re: [Qemu-devel] [PATCH 12/16] target-i386: cpu: attach ICC bus to CPU on its creation

Quoted 53 lines · On Mon, 22 Apr 2013 17:27:29 +0200 Andreas Färber <[email protected]> wrote:

Am 22.04.2013 17:20, schrieb Igor Mammedov:

Quoted 52 lines
Quoted 49 lines · On Mon, 22 Apr 2013 17:02:33 +0200 Andreas Färber <[email protected]> wrote:

Am 16.04.2013 00:12, schrieb Igor Mammedov:

Quoted 5 lines

... during startup, so it would be possible to unplug it later and set bus_type to TYPE_ICC_BUS for X86CPU type to make device_add attach hotplugged CPU to ICC bus.

But still one question...

Quoted 25 lines
---
 target-i386/cpu.c | 7 +++++++
 1 file changed, 7 insertions(+)
target-i386/cpu.c +2 -0
index 6d6c527..3b5f90b 100644
@@ -41,6 +41,7 @@
 #endif
 
 #include "sysemu/sysemu.h"
+#include "hw/i386/icc_bus.h"
 #ifndef CONFIG_USER_ONLY
 #include "hw/xen/xen.h"
 #include "hw/sysbus.h"
@@ -1609,6 +1610,7 @@ X86CPU *cpu_x86_create(const char *cpu_model,
 Error **errp) gchar **model_pieces;
     char *name, *features;
     Error *error = NULL;
+    Object *icc_bus = object_resolve_path_type("icc-bus", TYPE_ICC_BUS,
 NULL); 
     model_pieces = g_strsplit(cpu_model, ",", 2);
     if (!model_pieces[0]) {

target-i386/cpu.c X86CPU *cpu_x86_create(const char *cpu_model, · line 1630

Quoted 10 lines
target-i386/cpu.c +4 -0
@@ -1619,6 +1621,10 @@ X86CPU *cpu_x86_create(const char *cpu_model,
 Error **errp) features = model_pieces[1];
 
     cpu = X86_CPU(object_new(TYPE_X86_CPU));
+    if (icc_bus) {
+        qdev_set_parent_bus(DEVICE(cpu), BUS(icc_bus));
+        object_unref(OBJECT(cpu));
+    }
     env = &cpu->env;
     env->cpu_model_str = cpu_model;
 

You seem to be avoiding making lack of icc-bus an error although you add it for both PC and q35 PC - which non-ICC use cases are you thinking of?

Quoted 5 lines
Quoted 1 line

*-user targets, CPU is bus-less there.

Would you be opposed to #ifdef'ing it then? In particular I am asking because of the NULL error argument to the resolve function - for softmmu I would prefer to make lack of icc-bus a fatal error.

I'm fine with ifdef-ing there, if there is not objections. I'll look at making it fatal error on softmmu

Quoted 3 lines

Andreas

#61 lore raw

PATCH v4 13/16 target-i386: replace MSI_SPACE_SIZE with APIC_SPACE_SIZE

In reply to Igor Mammedov

... and put APIC_SPACE_SIZE in public header so that it could be reused later elsewhere.

---
 hw/i386/kvm/apic.c              | 2 +-
 hw/intc/apic.c                  | 2 +-
 hw/xen/xen_apic.c               | 2 +-
 include/hw/i386/apic_internal.h | 2 --
 target-i386/cpu.h               | 1 +
 5 files changed, 4 insertions(+), 5 deletions(-)
hw/i386/kvm/apic.c +1 -1
index c6ff982..8f80425 100644
@@ -174,7 +174,7 @@ static const MemoryRegionOps kvm_apic_io_ops = {
 static void kvm_apic_init(APICCommonState *s)
 {
     memory_region_init_io(&s->io_memory, &kvm_apic_io_ops, s, "kvm-apic-msi",
-                          MSI_SPACE_SIZE);
+                          APIC_SPACE_SIZE);
 
     if (kvm_has_gsi_routing()) {
         msi_supported = true;
hw/intc/apic.c +1 -1
index 2d79a9e..756dff0 100644
@@ -874,7 +874,7 @@ static const MemoryRegionOps apic_io_ops = {
 static void apic_init(APICCommonState *s)
 {
     memory_region_init_io(&s->io_memory, &apic_io_ops, s, "apic-msi",
-                          MSI_SPACE_SIZE);
+                          APIC_SPACE_SIZE);
 
     s->timer = qemu_new_timer_ns(vm_clock, apic_timer, s);
     local_apics[s->idx] = s;
hw/xen/xen_apic.c +1 -1
index a2eb8a1..a958782 100644
@@ -39,7 +39,7 @@ static const MemoryRegionOps xen_apic_io_ops = {
 static void xen_apic_init(APICCommonState *s)
 {
     memory_region_init_io(&s->io_memory, &xen_apic_io_ops, s, "xen-apic-msi",
-                          MSI_SPACE_SIZE);
+                          APIC_SPACE_SIZE);
 
 #if defined(CONFIG_XEN_CTRL_INTERFACE_VERSION) \
     && CONFIG_XEN_CTRL_INTERFACE_VERSION >= 420
include/hw/i386/apic_internal.h +0 -2
index 578241f..aac6290 100644
@@ -66,8 +66,6 @@
 
 #define MAX_APICS 255
 
-#define MSI_SPACE_SIZE                  0x100000
-
 typedef struct APICCommonState APICCommonState;
 
 #define TYPE_APIC_COMMON "apic-common"
target-i386/cpu.h +1 -0
index cf1b05c..65cf97d 100644
@@ -1269,5 +1269,6 @@ uint32_t x86_cpu_apic_id_from_index(unsigned int cpu_index);
 void enable_compat_apic_id_mode(void);
 
 #define APIC_DEFAULT_ADDRESS 0xfee00000
+#define APIC_SPACE_SIZE      0x100000
 
 #endif /* CPU_I386_H */
Signature · 1 line
1.8.2
Reviewed-by on v4 13/16
#62 lore raw

Re: [Qemu-devel] [PATCH 13/16] target-i386: replace MSI_SPACE_SIZE with APIC_SPACE_SIZE

Am 16.04.2013 00:12, schrieb Igor Mammedov:

Quoted 4 lines

... and put APIC_SPACE_SIZE in public header so that it could be reused later elsewhere.

Andreas

Quoted 72 lines
---
 hw/i386/kvm/apic.c              | 2 +-
 hw/intc/apic.c                  | 2 +-
 hw/xen/xen_apic.c               | 2 +-
 include/hw/i386/apic_internal.h | 2 --
 target-i386/cpu.h               | 1 +
 5 files changed, 4 insertions(+), 5 deletions(-)
hw/i386/kvm/apic.c +1 -1
index c6ff982..8f80425 100644
@@ -174,7 +174,7 @@ static const MemoryRegionOps kvm_apic_io_ops = {
 static void kvm_apic_init(APICCommonState *s)
 {
     memory_region_init_io(&s->io_memory, &kvm_apic_io_ops, s, "kvm-apic-msi",
-                          MSI_SPACE_SIZE);
+                          APIC_SPACE_SIZE);
 
     if (kvm_has_gsi_routing()) {
         msi_supported = true;
hw/intc/apic.c +1 -1
index 2d79a9e..756dff0 100644
@@ -874,7 +874,7 @@ static const MemoryRegionOps apic_io_ops = {
 static void apic_init(APICCommonState *s)
 {
     memory_region_init_io(&s->io_memory, &apic_io_ops, s, "apic-msi",
-                          MSI_SPACE_SIZE);
+                          APIC_SPACE_SIZE);
 
     s->timer = qemu_new_timer_ns(vm_clock, apic_timer, s);
     local_apics[s->idx] = s;
hw/xen/xen_apic.c +1 -1
index a2eb8a1..a958782 100644
@@ -39,7 +39,7 @@ static const MemoryRegionOps xen_apic_io_ops = {
 static void xen_apic_init(APICCommonState *s)
 {
     memory_region_init_io(&s->io_memory, &xen_apic_io_ops, s, "xen-apic-msi",
-                          MSI_SPACE_SIZE);
+                          APIC_SPACE_SIZE);
 
 #if defined(CONFIG_XEN_CTRL_INTERFACE_VERSION) \
     && CONFIG_XEN_CTRL_INTERFACE_VERSION >= 420
include/hw/i386/apic_internal.h +0 -2
index 578241f..aac6290 100644
@@ -66,8 +66,6 @@
 
 #define MAX_APICS 255
 
-#define MSI_SPACE_SIZE                  0x100000
-
 typedef struct APICCommonState APICCommonState;
 
 #define TYPE_APIC_COMMON "apic-common"
target-i386/cpu.h +1 -0
index cf1b05c..65cf97d 100644
@@ -1269,5 +1269,6 @@ uint32_t x86_cpu_apic_id_from_index(unsigned int cpu_index);
 void enable_compat_apic_id_mode(void);
 
 #define APIC_DEFAULT_ADDRESS 0xfee00000
+#define APIC_SPACE_SIZE      0x100000
 
 #endif /* CPU_I386_H */
Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#63 lore raw

PATCH v4 14/16 target-i386: move APIC to ICC bus

In reply to Igor Mammedov

... to allow it to be hotplugged

* map APIC's mmio at board level if it is present
* do not register mmio region for each APIC, since only one is used/mapped

---
 hw/cpu/icc_bus.c                | 15 ++++++++++++++-
 hw/i386/kvmvapic.c              |  1 +
 hw/i386/pc.c                    | 20 +++++++++++++++++---
 hw/intc/apic_common.c           | 17 ++++++++++++-----
 include/hw/i386/apic_internal.h |  6 +++---
 include/hw/i386/icc_bus.h       |  2 ++
 target-i386/cpu.c               | 16 +++-------------
 7 files changed, 52 insertions(+), 25 deletions(-)
hw/cpu/icc_bus.c +14 -1
index 00b9be3..5078c38 100644
@@ -62,13 +62,26 @@ static const TypeInfo icc_device_info = {
 
 typedef struct ICCBridgeState {
     SysBusDevice busdev;
+    MemoryRegion apic_container;
 } ICCBridgeState;
 #define ICC_BRIGDE(obj) OBJECT_CHECK(ICCBridgeState, (obj), TYPE_ICC_BRIDGE)
 
 
 static void icc_bridge_initfn(Object *obj)
 {
-    qbus_create(TYPE_ICC_BUS, DEVICE(obj), "icc-bus");
+    ICCBridgeState *s = ICC_BRIGDE(obj);
+    SysBusDevice *sb = SYS_BUS_DEVICE(obj);
+    ICCBus *ibus;
+
+    ibus = ICC_BUS(qbus_create(TYPE_ICC_BUS, DEVICE(obj), "icc-bus"));
+
+    /* Do not change order of registering regions,
+     * APIC must be first registered region, board maps it by 0 index
+     */
+    memory_region_init(&s->apic_container, "icc-apic-container",
+                       APIC_SPACE_SIZE);
+    sysbus_init_mmio(sb, &s->apic_container);
+    ibus->apic_address_space = &s->apic_container;
 }
 
 static const TypeInfo icc_bridge_info = {
hw/i386/kvmvapic.c +1 -0
index 3a10c07..5b558aa 100644
@@ -12,6 +12,7 @@
 #include "sysemu/cpus.h"
 #include "sysemu/kvm.h"
 #include "hw/i386/apic_internal.h"
+#include "hw/sysbus.h"
 
 #define VAPIC_IO_PORT           0x7e
 
hw/i386/pc.c +17 -3
index cb57878..31b5294 100644
@@ -52,6 +52,7 @@
 #include "sysemu/arch_init.h"
 #include "qemu/bitmap.h"
 #include "qemu/config-file.h"
+#include "hw/i386/icc_bus.h"
 
 /* debug PC/ISA interrupts */
 //#define DEBUG_IRQ
@@ -889,13 +890,13 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level)
     }
 }
 
-static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
+static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
 {
     X86CPU *cpu;
 
     cpu = cpu_x86_create(cpu_model, errp);
     if (!cpu) {
-        return;
+        return cpu;
     }
 
     object_property_set_int(OBJECT(cpu), apic_id, "apic-id", errp);
@@ -904,14 +905,18 @@ static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
     if (error_is_set(errp)) {
         if (cpu != NULL) {
             object_unref(OBJECT(cpu));
+            cpu = NULL;
         }
     }
+    return cpu;
 }
 
 void pc_cpus_init(const char *cpu_model)
 {
     int i;
+    X86CPU *cpu = NULL;
     Error *error = NULL;
+    SysBusDevice *ib;
 
     /* init CPUs */
     if (cpu_model == NULL) {
@@ -922,14 +927,23 @@ void pc_cpus_init(const char *cpu_model)
 #endif
     }
 
+    ib = SYS_BUS_DEVICE(object_resolve_path_type("icc-bridge",
+                                                 TYPE_ICC_BRIDGE, NULL));
+
     for (i = 0; i < smp_cpus; i++) {
-        pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
+        cpu = pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
         if (error) {
             fprintf(stderr, "%s\n", error_get_pretty(error));
             error_free(error);
             exit(1);
         }
     }
+
+    /* map APIC MMIO area if CPU has APIC */
+    if (cpu && cpu->env.apic_state) {
+        /* XXX: what if the base changes? */
+        sysbus_mmio_map_overlap(ib, 0, APIC_DEFAULT_ADDRESS, 0x1000);
+    }
 }
 
 void pc_acpi_init(const char *default_dsdt)
hw/intc/apic_common.c +12 -5
index e0ae07a..39396f1 100644
@@ -21,6 +21,8 @@
 #include "hw/i386/apic_internal.h"
 #include "trace.h"
 #include "sysemu/kvm.h"
+#include "hw/qdev.h"
+#include "hw/sysbus.h"
 
 static int apic_irq_delivered;
 bool apic_report_tpr_access;
@@ -282,12 +284,14 @@ static int apic_load_old(QEMUFile *f, void *opaque, int version_id)
     return 0;
 }
 
-static int apic_init_common(SysBusDevice *dev)
+static int apic_init_common(ICCDevice *dev)
 {
     APICCommonState *s = APIC_COMMON(dev);
+    DeviceState *d = DEVICE(dev);
     APICCommonClass *info;
     static DeviceState *vapic;
     static int apic_no;
+    static bool mmio_registered;
 
     if (apic_no >= MAX_APICS) {
         return -1;
@@ -296,8 +300,11 @@ static int apic_init_common(SysBusDevice *dev)
 
     info = APIC_COMMON_GET_CLASS(s);
     info->init(s);
-
-    sysbus_init_mmio(dev, &s->io_memory);
+    if (!mmio_registered) {
+        MemoryRegion *as = ICC_BUS(d->parent_bus)->apic_address_space;
+        memory_region_add_subregion(as, 0, &s->io_memory);
+        mmio_registered = true;
+    }
 
     /* Note: We need at least 1M to map the VAPIC option ROM */
     if (!vapic && s->vapic_control & VAPIC_ENABLE_MASK &&
@@ -375,7 +382,7 @@ static Property apic_properties_common[] = {
 
 static void apic_common_class_init(ObjectClass *klass, void *data)
 {
-    SysBusDeviceClass *sc = SYS_BUS_DEVICE_CLASS(klass);
+    ICCDeviceClass *sc = ICC_DEVICE_CLASS(klass);
     DeviceClass *dc = DEVICE_CLASS(klass);
 
     dc->vmsd = &vmstate_apic_common;
@@ -387,7 +394,7 @@ static void apic_common_class_init(ObjectClass *klass, void *data)
 
 static const TypeInfo apic_common_type = {
     .name = TYPE_APIC_COMMON,
-    .parent = TYPE_SYS_BUS_DEVICE,
+    .parent = TYPE_ICC_DEVICE,
     .instance_size = sizeof(APICCommonState),
     .class_size = sizeof(APICCommonClass),
     .class_init = apic_common_class_init,
include/hw/i386/apic_internal.h +3 -3
index aac6290..05acf4b 100644
@@ -21,7 +21,7 @@
 #define QEMU_APIC_INTERNAL_H
 
 #include "exec/memory.h"
-#include "hw/sysbus.h"
+#include "hw/i386/icc_bus.h"
 #include "qemu/timer.h"
 
 /* APIC Local Vector Table */
@@ -78,7 +78,7 @@ typedef struct APICCommonState APICCommonState;
 
 typedef struct APICCommonClass
 {
-    SysBusDeviceClass parent_class;
+    ICCDeviceClass parent_class;
 
     void (*init)(APICCommonState *s);
     void (*set_base)(APICCommonState *s, uint64_t val);
@@ -92,7 +92,7 @@ typedef struct APICCommonClass
 } APICCommonClass;
 
 struct APICCommonState {
-    SysBusDevice busdev;
+    ICCDevice busdev;
 
     MemoryRegion io_memory;
     X86CPU *cpu;
include/hw/i386/icc_bus.h +2 -0
index aa30e0c..1b07619 100644
@@ -22,6 +22,7 @@
 #ifndef ICC_BUS_H
 #define ICC_BUS_H
 
+#include "exec/memory.h"
 #include "hw/qdev-core.h"
 
 #define TYPE_ICC_BUS "icc-bus"
@@ -29,6 +30,7 @@
 #ifndef CONFIG_USER_ONLY
 typedef struct ICCBus {
     BusState qbus;
+    MemoryRegion *apic_address_space;
 } ICCBus;
 #define ICC_BUS(obj) OBJECT_CHECK(ICCBus, (obj), TYPE_ICC_BUS)
 
target-i386/cpu.c +3 -13
index 3b5f90b..bdac6ea 100644
@@ -41,10 +41,10 @@
 #endif
 
 #include "sysemu/sysemu.h"
+#include "hw/qdev-properties.h"
 #include "hw/i386/icc_bus.h"
 #ifndef CONFIG_USER_ONLY
 #include "hw/xen/xen.h"
-#include "hw/sysbus.h"
 #include "hw/i386/apic_internal.h"
 #endif
 
@@ -2111,6 +2111,7 @@ static void mce_init(X86CPU *cpu)
 static void x86_cpu_apic_create(X86CPU *cpu, Error **errp)
 {
     CPUX86State *env = &cpu->env;
+    DeviceState *dev = DEVICE(cpu);
     APICCommonState *apic;
     const char *apic_type = "apic";
 
@@ -2120,7 +2121,7 @@ static void x86_cpu_apic_create(X86CPU *cpu, Error **errp)
         apic_type = "xen-apic";
     }
 
-    env->apic_state = qdev_try_create(NULL, apic_type);
+    env->apic_state = qdev_try_create(dev->parent_bus, apic_type);
     if (env->apic_state == NULL) {
         error_setg(errp, "APIC device '%s' could not be created", apic_type);
         return;
@@ -2137,7 +2138,6 @@ static void x86_cpu_apic_create(X86CPU *cpu, Error **errp)
 static void x86_cpu_apic_realize(X86CPU *cpu, Error **errp)
 {
     CPUX86State *env = &cpu->env;
-    static int apic_mapped;
 
     if (env->apic_state == NULL) {
         return;
@@ -2148,16 +2148,6 @@ static void x86_cpu_apic_realize(X86CPU *cpu, Error **errp)
                    object_get_typename(OBJECT(env->apic_state)));
         return;
     }
-
-    /* XXX: mapping more APICs at the same memory location */
-    if (apic_mapped == 0) {
-        /* NOTE: the APIC is directly connected to the CPU - it is not
-           on the global memory bus. */
-        /* XXX: what if the base changes? */
-        sysbus_mmio_map_overlap(SYS_BUS_DEVICE(env->apic_state), 0,
-                                APIC_DEFAULT_ADDRESS, 0x1000);
-        apic_mapped = 1;
-    }
 }
 #else
 static void x86_cpu_apic_realize(X86CPU *cpu, Error **errp)
Signature · 1 line
1.8.2
#64 lore raw

Re: [Qemu-devel] [PATCH 14/16] target-i386: move APIC to ICC bus

Am 16.04.2013 00:12, schrieb Igor Mammedov:

Quoted 28 lines

... to allow it to be hotplugged

* map APIC's mmio at board level if it is present
* do not register mmio region for each APIC, since only one is used/mapped

---
 hw/cpu/icc_bus.c                | 15 ++++++++++++++-
 hw/i386/kvmvapic.c              |  1 +
 hw/i386/pc.c                    | 20 +++++++++++++++++---
 hw/intc/apic_common.c           | 17 ++++++++++++-----
 include/hw/i386/apic_internal.h |  6 +++---
 include/hw/i386/icc_bus.h       |  2 ++
 target-i386/cpu.c               | 16 +++-------------
 7 files changed, 52 insertions(+), 25 deletions(-)
hw/cpu/icc_bus.c +1 -0
index 00b9be3..5078c38 100644
@@ -62,13 +62,26 @@ static const TypeInfo icc_device_info = {
 
 typedef struct ICCBridgeState {
     SysBusDevice busdev;
+    MemoryRegion apic_container;
 } ICCBridgeState;
 #define ICC_BRIGDE(obj) OBJECT_CHECK(ICCBridgeState, (obj), TYPE_ICC_BRIDGE)

hw/cpu/icc_bus.c static const TypeInfo icc_device_info = { · line 76

Quoted 10 lines
hw/cpu/icc_bus.c +5 -1
 
 
 static void icc_bridge_initfn(Object *obj)
 {
-    qbus_create(TYPE_ICC_BUS, DEVICE(obj), "icc-bus");
+    ICCBridgeState *s = ICC_BRIGDE(obj);
+    SysBusDevice *sb = SYS_BUS_DEVICE(obj);
+    ICCBus *ibus;
+
+    ibus = ICC_BUS(qbus_create(TYPE_ICC_BUS, DEVICE(obj), "icc-bus"));

qbus_create_inplace() as suggested on ICC patch would allow to access it through state field.

Quoted 16 lines
hw/cpu/icc_bus.c +8 -0
+
+    /* Do not change order of registering regions,
+     * APIC must be first registered region, board maps it by 0 index
+     */
+    memory_region_init(&s->apic_container, "icc-apic-container",
+                       APIC_SPACE_SIZE);
+    sysbus_init_mmio(sb, &s->apic_container);
+    ibus->apic_address_space = &s->apic_container;
 }
 
 static const TypeInfo icc_bridge_info = {
hw/i386/kvmvapic.c +0 -0
index 3a10c07..5b558aa 100644

hw/i386/kvmvapic.c line 18

Quoted 7 lines
hw/i386/kvmvapic.c +1 -0
@@ -12,6 +12,7 @@
 #include "sysemu/cpus.h"
 #include "sysemu/kvm.h"
 #include "hw/i386/apic_internal.h"
+#include "hw/sysbus.h"
 
 #define VAPIC_IO_PORT           0x7e
 

I take it, this is necessary due to dropping hw/sysbus.h include in cpu.c? Might be split out then since patch is rather large already.

Quoted 30 lines
hw/i386/pc.c +3 -2
index cb57878..31b5294 100644
@@ -52,6 +52,7 @@
 #include "sysemu/arch_init.h"
 #include "qemu/bitmap.h"
 #include "qemu/config-file.h"
+#include "hw/i386/icc_bus.h"
 
 /* debug PC/ISA interrupts */
 //#define DEBUG_IRQ
@@ -889,13 +890,13 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level)
     }
 }
 
-static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
+static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
 {
     X86CPU *cpu;
 
     cpu = cpu_x86_create(cpu_model, errp);
     if (!cpu) {
-        return;
+        return cpu;
     }
 
     object_property_set_int(OBJECT(cpu), apic_id, "apic-id", errp);

hw/i386/pc.c static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp) · line 913

Quoted 9 lines
hw/i386/pc.c +2 -0
@@ -904,14 +905,18 @@ static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
     if (error_is_set(errp)) {
         if (cpu != NULL) {
             object_unref(OBJECT(cpu));
+            cpu = NULL;
         }
     }
+    return cpu;
 }
 

Squash this into the pc_new_cpu() introduction?

hw/i386/pc.c static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp) · line 919

Quoted 6 lines
hw/i386/pc.c +2 -0
 void pc_cpus_init(const char *cpu_model)
 {
     int i;
+    X86CPU *cpu = NULL;
     Error *error = NULL;
+    SysBusDevice *ib;

icc maybe? Anything that more's comprehensible. ;)

hw/i386/pc.c void pc_cpus_init(const char *cpu_model) · line 934

Quoted 13 lines
hw/i386/pc.c +4 -1
     /* init CPUs */
     if (cpu_model == NULL) {
@@ -922,14 +927,23 @@ void pc_cpus_init(const char *cpu_model)
 #endif
     }
 
+    ib = SYS_BUS_DEVICE(object_resolve_path_type("icc-bridge",
+                                                 TYPE_ICC_BRIDGE, NULL));
+
     for (i = 0; i < smp_cpus; i++) {
-        pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
+        cpu = pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);

(Here it would've been nice too but would lead to assigned-but-not-used warnings.)

Quoted 29 lines
hw/i386/pc.c +6 -0
         if (error) {
             fprintf(stderr, "%s\n", error_get_pretty(error));
             error_free(error);
             exit(1);
         }
     }
+
+    /* map APIC MMIO area if CPU has APIC */
+    if (cpu && cpu->env.apic_state) {
+        /* XXX: what if the base changes? */
+        sysbus_mmio_map_overlap(ib, 0, APIC_DEFAULT_ADDRESS, 0x1000);
+    }
 }
 
 void pc_acpi_init(const char *default_dsdt)
hw/intc/apic_common.c +2 -0
index e0ae07a..39396f1 100644
@@ -21,6 +21,8 @@
 #include "hw/i386/apic_internal.h"
 #include "trace.h"
 #include "sysemu/kvm.h"
+#include "hw/qdev.h"
+#include "hw/sysbus.h"
 
 static int apic_irq_delivered;
 bool apic_report_tpr_access;

hw/intc/apic_common.c static int apic_load_old(QEMUFile *f, void *opaque, int version_id) · line 287

Quoted 5 lines
hw/intc/apic_common.c +1 -1
@@ -282,12 +284,14 @@ static int apic_load_old(QEMUFile *f, void *opaque, int version_id)
     return 0;
 }
 
-static int apic_init_common(SysBusDevice *dev)
+static int apic_init_common(ICCDevice *dev)

Instead of mechanically replacing SysBusDevice with ICCDevice in qdev init functions, please change to realize. Could be its own prepended patch if you prefer.

Quoted 25 lines
hw/intc/apic_common.c +7 -2
 {
     APICCommonState *s = APIC_COMMON(dev);
+    DeviceState *d = DEVICE(dev);
     APICCommonClass *info;
     static DeviceState *vapic;
     static int apic_no;
+    static bool mmio_registered;
 
     if (apic_no >= MAX_APICS) {
         return -1;
@@ -296,8 +300,11 @@ static int apic_init_common(SysBusDevice *dev)
 
     info = APIC_COMMON_GET_CLASS(s);
     info->init(s);
-
-    sysbus_init_mmio(dev, &s->io_memory);
+    if (!mmio_registered) {
+        MemoryRegion *as = ICC_BUS(d->parent_bus)->apic_address_space;
+        memory_region_add_subregion(as, 0, &s->io_memory);
+        mmio_registered = true;
+    }
 
     /* Note: We need at least 1M to map the VAPIC option ROM */
     if (!vapic && s->vapic_control & VAPIC_ENABLE_MASK &&

hw/intc/apic_common.c static Property apic_properties_common[] = { · line 385

Quoted 5 lines
hw/intc/apic_common.c +1 -1
@@ -375,7 +382,7 @@ static Property apic_properties_common[] = {
 
 static void apic_common_class_init(ObjectClass *klass, void *data)
 {
-    SysBusDeviceClass *sc = SYS_BUS_DEVICE_CLASS(klass);
+    ICCDeviceClass *sc = ICC_DEVICE_CLASS(klass);

idc please. :)

Quoted 35 lines
hw/intc/apic_common.c +1 -1
     DeviceClass *dc = DEVICE_CLASS(klass);
 
     dc->vmsd = &vmstate_apic_common;
@@ -387,7 +394,7 @@ static void apic_common_class_init(ObjectClass *klass, void *data)
 
 static const TypeInfo apic_common_type = {
     .name = TYPE_APIC_COMMON,
-    .parent = TYPE_SYS_BUS_DEVICE,
+    .parent = TYPE_ICC_DEVICE,
     .instance_size = sizeof(APICCommonState),
     .class_size = sizeof(APICCommonClass),
     .class_init = apic_common_class_init,
include/hw/i386/apic_internal.h +2 -2
index aac6290..05acf4b 100644
@@ -21,7 +21,7 @@
 #define QEMU_APIC_INTERNAL_H
 
 #include "exec/memory.h"
-#include "hw/sysbus.h"
+#include "hw/i386/icc_bus.h"
 #include "qemu/timer.h"
 
 /* APIC Local Vector Table */
@@ -78,7 +78,7 @@ typedef struct APICCommonState APICCommonState;
 
 typedef struct APICCommonClass
 {
-    SysBusDeviceClass parent_class;
+    ICCDeviceClass parent_class;
 
     void (*init)(APICCommonState *s);
     void (*set_base)(APICCommonState *s, uint64_t val);

include/hw/i386/apic_internal.h typedef struct APICCommonClass · line 95

Quoted 5 lines
include/hw/i386/apic_internal.h +1 -1
@@ -92,7 +92,7 @@ typedef struct APICCommonClass
 } APICCommonClass;
 
 struct APICCommonState {
-    SysBusDevice busdev;
+    ICCDevice busdev;

parent_obj please, while at it.

Quoted 48 lines
include/hw/i386/apic_internal.h +0 -0
     MemoryRegion io_memory;
     X86CPU *cpu;
include/hw/i386/icc_bus.h +2 -0
index aa30e0c..1b07619 100644
@@ -22,6 +22,7 @@
 #ifndef ICC_BUS_H
 #define ICC_BUS_H
 
+#include "exec/memory.h"
 #include "hw/qdev-core.h"
 
 #define TYPE_ICC_BUS "icc-bus"
@@ -29,6 +30,7 @@
 #ifndef CONFIG_USER_ONLY
 typedef struct ICCBus {
     BusState qbus;
+    MemoryRegion *apic_address_space;
 } ICCBus;
 #define ICC_BUS(obj) OBJECT_CHECK(ICCBus, (obj), TYPE_ICC_BUS)
 
target-i386/cpu.c +2 -1
index 3b5f90b..bdac6ea 100644
@@ -41,10 +41,10 @@
 #endif
 
 #include "sysemu/sysemu.h"
+#include "hw/qdev-properties.h"
 #include "hw/i386/icc_bus.h"
 #ifndef CONFIG_USER_ONLY
 #include "hw/xen/xen.h"
-#include "hw/sysbus.h"
 #include "hw/i386/apic_internal.h"
 #endif
 
@@ -2111,6 +2111,7 @@ static void mce_init(X86CPU *cpu)
 static void x86_cpu_apic_create(X86CPU *cpu, Error **errp)
 {
     CPUX86State *env = &cpu->env;
+    DeviceState *dev = DEVICE(cpu);
     APICCommonState *apic;
     const char *apic_type = "apic";
 

target-i386/cpu.c static void x86_cpu_apic_create(X86CPU *cpu, Error **errp) · line 2124

Quoted 5 lines
target-i386/cpu.c +1 -1
@@ -2120,7 +2121,7 @@ static void x86_cpu_apic_create(X86CPU *cpu, Error **errp)
         apic_type = "xen-apic";
     }
 
-    env->apic_state = qdev_try_create(NULL, apic_type);
+    env->apic_state = qdev_try_create(dev->parent_bus, apic_type);

qdev_get_parent_bus() according to header comment.

Quoted 19 lines
target-i386/cpu.c +0 -4
     if (env->apic_state == NULL) {
         error_setg(errp, "APIC device '%s' could not be created", apic_type);
         return;
@@ -2137,7 +2138,6 @@ static void x86_cpu_apic_create(X86CPU *cpu, Error **errp)
 static void x86_cpu_apic_realize(X86CPU *cpu, Error **errp)
 {
     CPUX86State *env = &cpu->env;
-    static int apic_mapped;
 
     if (env->apic_state == NULL) {
         return;
@@ -2148,16 +2148,6 @@ static void x86_cpu_apic_realize(X86CPU *cpu, Error **errp)
                    object_get_typename(OBJECT(env->apic_state)));
         return;
     }
-
-    /* XXX: mapping more APICs at the same memory location */
-    if (apic_mapped == 0) {

target-i386/cpu.c static void x86_cpu_apic_realize(X86CPU *cpu, Error **errp) · line 2153

Quoted 10 lines
target-i386/cpu.c +0 -7
-        /* NOTE: the APIC is directly connected to the CPU - it is not
-           on the global memory bus. */
-        /* XXX: what if the base changes? */
-        sysbus_mmio_map_overlap(SYS_BUS_DEVICE(env->apic_state), 0,
-                                APIC_DEFAULT_ADDRESS, 0x1000);
-        apic_mapped = 1;
-    }
 }
 #else
 static void x86_cpu_apic_realize(X86CPU *cpu, Error **errp)

Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#65 lore raw

Re: [Qemu-devel] [PATCH 14/16] target-i386: move APIC to ICC bus

Quoted 1 line · On Mon, 22 Apr 2013 17:18:58 +0200 Andreas Färber <[email protected]> wrote:

Am 16.04.2013 00:12, schrieb Igor Mammedov:

[...]

Quoted 11 lines

hw/cpu/icc_bus.c static const TypeInfo icc_device_info = { · line 76

Quoted 8 lines

static void icc_bridge_initfn(Object *obj)
{

hw/cpu/icc_bus.c +5 -1
-    qbus_create(TYPE_ICC_BUS, DEVICE(obj), "icc-bus");
+    ICCBridgeState *s = ICC_BRIGDE(obj);
+    SysBusDevice *sb = SYS_BUS_DEVICE(obj);
+    ICCBus *ibus;
+
+    ibus = ICC_BUS(qbus_create(TYPE_ICC_BUS, DEVICE(obj), "icc-bus"));

qbus_create_inplace() as suggested on ICC patch would allow to access it through state field.

done

Quoted 22 lines
Quoted 22 lines
hw/cpu/icc_bus.c +8 -0
+
+    /* Do not change order of registering regions,
+     * APIC must be first registered region, board maps it by 0 index
+     */
+    memory_region_init(&s->apic_container, "icc-apic-container",
+                       APIC_SPACE_SIZE);
+    sysbus_init_mmio(sb, &s->apic_container);
+    ibus->apic_address_space = &s->apic_container;
 }
 
 static const TypeInfo icc_bridge_info = {
hw/i386/kvmvapic.c +1 -0
index 3a10c07..5b558aa 100644
@@ -12,6 +12,7 @@
 #include "sysemu/cpus.h"
 #include "sysemu/kvm.h"
 #include "hw/i386/apic_internal.h"
+#include "hw/sysbus.h"
 
Quoted 5 lines

hw/i386/kvmvapic.c line 18

Quoted 2 lines
hw/i386/kvmvapic.c +0 -0
 #define VAPIC_IO_PORT           0x7e
 

I take it, this is necessary due to dropping hw/sysbus.h include in cpu.c? Might be split out then since patch is rather large already.

It's due to dropping hw/sysbus.h from include/hw/i386/apic_internal.h It's hard to split in meaningful separate patch.

Quoted 29 lines
Quoted 29 lines
hw/i386/pc.c +2 -2
index cb57878..31b5294 100644
@@ -52,6 +52,7 @@
 #include "sysemu/arch_init.h"
 #include "qemu/bitmap.h"
 #include "qemu/config-file.h"
+#include "hw/i386/icc_bus.h"
 
 /* debug PC/ISA interrupts */
 //#define DEBUG_IRQ
@@ -889,13 +890,13 @@ void pc_acpi_smi_interrupt(void *opaque, int irq,
 int level) }
 }
 
-static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error
 **errp) +static X86CPU *pc_new_cpu(const char *cpu_model, int64_t
 apic_id, Error **errp) {
     X86CPU *cpu;
 
     cpu = cpu_x86_create(cpu_model, errp);
     if (!cpu) {
-        return;
+        return cpu;
     }
 
Quoted 13 lines

hw/i386/pc.c static void pc_new_cpu(const char *cpu_model, · line 913

Quoted 11 lines
hw/i386/pc.c +2 -0
     object_property_set_int(OBJECT(cpu), apic_id, "apic-id", errp);
@@ -904,14 +905,18 @@ static void pc_new_cpu(const char *cpu_model,
 int64_t apic_id, Error **errp) if (error_is_set(errp)) {
         if (cpu != NULL) {
             object_unref(OBJECT(cpu));
+            cpu = NULL;
         }
     }
+    return cpu;
 }
 

Squash this into the pc_new_cpu() introduction?

sure, it's less obvious why it would be needed but makes this patch smaller.

Quoted 9 lines

hw/i386/pc.c static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp) · line 919

Quoted 7 lines
hw/i386/pc.c +2 -0
 void pc_cpus_init(const char *cpu_model)
 {
     int i;
+    X86CPU *cpu = NULL;
     Error *error = NULL;
+    SysBusDevice *ib;

icc maybe? Anything that more's comprehensible. ;)

icc_bridge then.

[...]

Quoted 17 lines
Quoted 17 lines
hw/intc/apic_common.c +2 -0
index e0ae07a..39396f1 100644
@@ -21,6 +21,8 @@
 #include "hw/i386/apic_internal.h"
 #include "trace.h"
 #include "sysemu/kvm.h"
+#include "hw/qdev.h"
+#include "hw/sysbus.h"
 
 static int apic_irq_delivered;
 bool apic_report_tpr_access;
@@ -282,12 +284,14 @@ static int apic_load_old(QEMUFile *f, void *opaque,
 int version_id) return 0;
 }
 
Quoted 6 lines

hw/intc/apic_common.c static int apic_load_old(QEMUFile *f, void *opaque, int version_id) · line 287

Quoted 2 lines
hw/intc/apic_common.c +1 -1
-static int apic_init_common(SysBusDevice *dev)
+static int apic_init_common(ICCDevice *dev)

Instead of mechanically replacing SysBusDevice with ICCDevice in qdev init functions, please change to realize. Could be its own prepended patch if you prefer.

IRC with Andreas, agreed to do it only for ICCDevice. And do re-factoring of init() in APICs/IOAPIC later.

Quoted 11 lines
Quoted 11 lines
hw/intc/apic_common.c +2 -0
 {
     APICCommonState *s = APIC_COMMON(dev);
+    DeviceState *d = DEVICE(dev);
     APICCommonClass *info;
     static DeviceState *vapic;
     static int apic_no;
+    static bool mmio_registered;
 
     if (apic_no >= MAX_APICS) {
         return -1;

[...]

Quoted 8 lines

hw/intc/apic_common.c static Property apic_properties_common[] = { · line 385

Quoted 6 lines
hw/intc/apic_common.c +1 -1
@@ -375,7 +382,7 @@ static Property apic_properties_common[] = {
 
 static void apic_common_class_init(ObjectClass *klass, void *data)
 {
-    SysBusDeviceClass *sc = SYS_BUS_DEVICE_CLASS(klass);
+    ICCDeviceClass *sc = ICC_DEVICE_CLASS(klass);

idc please. :)

done

Quoted 7 lines
Quoted 7 lines
hw/intc/apic_common.c +0 -0
     DeviceClass *dc = DEVICE_CLASS(klass);
 
     dc->vmsd = &vmstate_apic_common;
@@ -387,7 +394,7 @@ static void apic_common_class_init(ObjectClass
 *klass, void *data) 
 static const TypeInfo apic_common_type = {

[...]

Quoted 8 lines

hw/intc/apic_common.c static void x86_cpu_apic_create(X86CPU *cpu, Error · line 2124

Quoted 6 lines
hw/intc/apic_common.c +1 -1
@@ -2120,7 +2121,7 @@ static void x86_cpu_apic_create(X86CPU *cpu, Error
 **errp) apic_type = "xen-apic";
     }
 
-    env->apic_state = qdev_try_create(NULL, apic_type);
+    env->apic_state = qdev_try_create(dev->parent_bus, apic_type);

qdev_get_parent_bus() according to header comment.

done

Quoted 2 lines
Quoted 2 lines
hw/intc/apic_common.c +0 -0
     if (env->apic_state == NULL) {

[...]

Quoted 2 lines

Andreas

#66 lore raw

PATCH v4 15/16 target-i386: move IOAPIC to ICC bus

In reply to Igor Mammedov

* inherit IOAPICs from ICCDevice and attach them to ICC bus
* map IOAPIC's mmio at board level via indirect icc-bridge mmio region that provides address space to IOAPIC via icc-bus

---
* do not create IOAPIC in icc-bridge and do not make it QOM child of icc-bridge, just attach it to icc-bus.

---
 hw/cpu/icc_bus.c                  |  6 ++++++
 hw/i386/kvm/ioapic.c              |  2 +-
 hw/i386/pc.c                      | 10 ++++++----
 hw/intc/ioapic_common.c           | 15 +++++++++++----
 include/exec/memory.h             | 10 ++++++++++
 include/hw/i386/icc_bus.h         |  1 +
 include/hw/i386/ioapic_internal.h |  6 +++---
 memory.c                          | 11 +++++++++++
 8 files changed, 49 insertions(+), 12 deletions(-)
hw/cpu/icc_bus.c +6 -0
index 5078c38..d241438 100644
@@ -63,6 +63,7 @@ static const TypeInfo icc_device_info = {
 typedef struct ICCBridgeState {
     SysBusDevice busdev;
     MemoryRegion apic_container;
+    MemoryRegion ioapic_container;
 } ICCBridgeState;
 #define ICC_BRIGDE(obj) OBJECT_CHECK(ICCBridgeState, (obj), TYPE_ICC_BRIDGE)
 
@@ -82,6 +83,11 @@ static void icc_bridge_initfn(Object *obj)
                        APIC_SPACE_SIZE);
     sysbus_init_mmio(sb, &s->apic_container);
     ibus->apic_address_space = &s->apic_container;
+
+    /* must be second registered region, board maps it by 1 index */
+    memory_region_init(&s->ioapic_container, "icc-ioapic-container", 0x1000);
+    sysbus_init_mmio(sb, &s->ioapic_container);
+    ibus->ioapic_address_space = &s->ioapic_container;
 }
 
 static const TypeInfo icc_bridge_info = {
hw/i386/kvm/ioapic.c +1 -1
index a3bd519..b80d41a 100644
@@ -96,7 +96,7 @@ static void kvm_ioapic_put(IOAPICCommonState *s)
 
     kioapic->id = s->id;
     kioapic->ioregsel = s->ioregsel;
-    kioapic->base_address = s->busdev.mmio[0].addr;
+    kioapic->base_address = memory_region_get_address(&s->io_memory);
     kioapic->irr = s->irr;
     for (i = 0; i < IOAPIC_NUM_PINS; i++) {
         kioapic->redirtbl[i].bits = s->ioredtbl[i];
hw/i386/pc.c +6 -4
index 31b5294..a5c19a2 100644
@@ -1217,20 +1217,22 @@ void ioapic_init_gsi(GSIState *gsi_state, const char *parent_name)
     DeviceState *dev;
     SysBusDevice *d;
     unsigned int i;
+    BusState *b = BUS(object_resolve_path_type("icc-bus", TYPE_ICC_BUS, NULL));
 
     if (kvm_irqchip_in_kernel()) {
-        dev = qdev_create(NULL, "kvm-ioapic");
+        dev = qdev_create(b, "kvm-ioapic");
     } else {
-        dev = qdev_create(NULL, "ioapic");
+        dev = qdev_create(b, "ioapic");
     }
     if (parent_name) {
         object_property_add_child(object_resolve_path(parent_name, NULL),
                                   "ioapic", OBJECT(dev), NULL);
     }
     qdev_init_nofail(dev);
-    d = SYS_BUS_DEVICE(dev);
-    sysbus_mmio_map(d, 0, IO_APIC_DEFAULT_ADDRESS);
 
+    d = SYS_BUS_DEVICE(object_resolve_path_type("icc-bridge", TYPE_ICC_BRIDGE,
+                                                NULL));
+    sysbus_mmio_map(d, 1, IO_APIC_DEFAULT_ADDRESS);
     for (i = 0; i < IOAPIC_NUM_PINS; i++) {
         gsi_state->ioapic_irq[i] = qdev_get_gpio_in(dev, i);
     }
hw/intc/ioapic_common.c +11 -4
index 5c5bb3c..72c48a8 100644
@@ -57,11 +57,13 @@ static int ioapic_dispatch_post_load(void *opaque, int version_id)
     return 0;
 }
 
-static int ioapic_init_common(SysBusDevice *dev)
+static int ioapic_init_common(ICCDevice *dev)
 {
     IOAPICCommonState *s = IOAPIC_COMMON(dev);
+    DeviceState *d = DEVICE(dev);
     IOAPICCommonClass *info;
     static int ioapic_no;
+    static bool mmio_registered;
 
     if (ioapic_no >= MAX_IOAPICS) {
         return -1;
@@ -70,7 +72,12 @@ static int ioapic_init_common(SysBusDevice *dev)
     info = IOAPIC_COMMON_GET_CLASS(s);
     info->init(s, ioapic_no);
 
-    sysbus_init_mmio(&s->busdev, &s->io_memory);
+    if (!mmio_registered) {
+        MemoryRegion *as = ICC_BUS(d->parent_bus)->ioapic_address_space;
+        memory_region_add_subregion(as, 0, &s->io_memory);
+        mmio_registered = true;
+    }
+
     ioapic_no++;
 
     return 0;
@@ -95,7 +102,7 @@ static const VMStateDescription vmstate_ioapic_common = {
 
 static void ioapic_common_class_init(ObjectClass *klass, void *data)
 {
-    SysBusDeviceClass *sc = SYS_BUS_DEVICE_CLASS(klass);
+    ICCDeviceClass *sc = ICC_DEVICE_CLASS(klass);
     DeviceClass *dc = DEVICE_CLASS(klass);
 
     sc->init = ioapic_init_common;
@@ -105,7 +112,7 @@ static void ioapic_common_class_init(ObjectClass *klass, void *data)
 
 static const TypeInfo ioapic_common_type = {
     .name = TYPE_IOAPIC_COMMON,
-    .parent = TYPE_SYS_BUS_DEVICE,
+    .parent = TYPE_ICC_DEVICE,
     .instance_size = sizeof(IOAPICCommonState),
     .class_size = sizeof(IOAPICCommonClass),
     .class_init = ioapic_common_class_init,
include/exec/memory.h +10 -0
index 2322732..826e401 100644
@@ -711,6 +711,16 @@ void memory_region_set_enabled(MemoryRegion *mr, bool enabled);
 void memory_region_set_address(MemoryRegion *mr, hwaddr addr);
 
 /*
+ * memory_region_get_address: get current the address of a region
+ *
+ * Returns the absolute address of a region.
+ * May be used on regions that are currently part of a memory hierarchy.
+ *
+ * @mr: the region being queried
+ */
+hwaddr memory_region_get_address(MemoryRegion *mr);
+
+/*
  * memory_region_set_alias_offset: dynamically update a memory alias's offset
  *
  * Dynamically updates the offset into the target region that an alias points
include/hw/i386/icc_bus.h +1 -0
index 1b07619..d92c6ab 100644
@@ -31,6 +31,7 @@
 typedef struct ICCBus {
     BusState qbus;
     MemoryRegion *apic_address_space;
+    MemoryRegion *ioapic_address_space;
 } ICCBus;
 #define ICC_BUS(obj) OBJECT_CHECK(ICCBus, (obj), TYPE_ICC_BUS)
 
include/hw/i386/ioapic_internal.h +3 -3
index 25576c8..4768c5b 100644
@@ -24,7 +24,7 @@
 
 #include "hw/hw.h"
 #include "exec/memory.h"
-#include "hw/sysbus.h"
+#include "hw/i386/icc_bus.h"
 
 #define MAX_IOAPICS                     1
 
@@ -82,14 +82,14 @@ typedef struct IOAPICCommonState IOAPICCommonState;
      OBJECT_GET_CLASS(IOAPICCommonClass, (obj), TYPE_IOAPIC_COMMON)
 
 typedef struct IOAPICCommonClass {
-    SysBusDeviceClass parent_class;
+    ICCDeviceClass parent_class;
     void (*init)(IOAPICCommonState *s, int instance_no);
     void (*pre_save)(IOAPICCommonState *s);
     void (*post_load)(IOAPICCommonState *s);
 } IOAPICCommonClass;
 
 struct IOAPICCommonState {
-    SysBusDevice busdev;
+    ICCDevice busdev;
     MemoryRegion io_memory;
     uint8_t id;
     uint8_t ioregsel;
memory.c +11 -0
index 75ca281..0651050 100644
@@ -1413,6 +1413,17 @@ void memory_region_set_address(MemoryRegion *mr, hwaddr addr)
     memory_region_transaction_commit();
 }
 
+hwaddr memory_region_get_address(MemoryRegion *mr)
+{
+    hwaddr addr = mr->addr;
+
+    while (mr->parent) {
+        mr = mr->parent;
+        addr += mr->addr;
+    }
+    return addr;
+}
+
 void memory_region_set_alias_offset(MemoryRegion *mr, hwaddr offset)
 {
     assert(mr->alias);
Signature · 1 line
1.8.2
#67 lore raw

PATCH v4 16/16 add cpu-add qmp command and implement CPU hot-add for target-i386

In reply to Igor Mammedov

... via current_machine->cpu_hot_add() hook called by cpu-set QMP command, for x86 target.

cpu-add's "id" argument is a CPU thread number in a range [0..max-cpus - 1)

---
v5:
* accept id=[0..max_cpus) range in cpu-add command
v4:
* merge "qmp: add cpu-add qmp command" & "target-i386: implement CPU hot-add" patches
* move notifier call to CPUCLass.realize()
* add hook cpu_hot_add to QEMUMachine
* make QEMUMachineInitArgs global and keep default cpu_model there

v3:
* it appears that 'online/offline' in cpu-set are confusing people with what command actually does and users might have to distinguish if 'offline' is not implemented by parsing error message. To simplify things replace cpu-set with cpu-add command to show more clear what command does and just add cpu-del when CPU remove is implemented.

v2:
* s/cpu_set/cpu-set/
* qmp doc style fix
* use bool type instead of opencodding online/offline string

---
 hw/i386/pc.c        | 22 ++++++++++++++++++++++
 include/hw/boards.h |  3 +++
 qapi-schema.json    | 11 +++++++++++
 qmp-commands.hx     | 23 +++++++++++++++++++++++
 qmp.c               | 10 ++++++++++
 vl.c                |  6 +++++-
 6 files changed, 74 insertions(+), 1 deletion(-)
hw/i386/pc.c +22 -0
index a5c19a2..f09e44b 100644
@@ -53,6 +53,7 @@
 #include "qemu/bitmap.h"
 #include "qemu/config-file.h"
 #include "hw/i386/icc_bus.h"
+#include "hw/boards.h"
 
 /* debug PC/ISA interrupts */
 //#define DEBUG_IRQ
@@ -911,6 +912,25 @@ static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
     return cpu;
 }
 
+static void do_cpu_hot_add(const int64_t id, Error **errp)
+{
+    int64_t apic_id = x86_cpu_apic_id_from_index(id);
+
+    if (cpu_exists(apic_id)) {
+        error_setg(errp, "Unable to add CPU: %" PRIi64
+                   ", it already exists", id);
+        return;
+    }
+
+    if (id >= max_cpus) {
+        error_setg(errp, "Unable to add CPU: %" PRIi64
+                   ", max allowed: %d", id, max_cpus - 1);
+        return;
+    }
+
+    pc_new_cpu(machine_args->cpu_model, apic_id, errp);
+}
+
 void pc_cpus_init(const char *cpu_model)
 {
     int i;
@@ -925,7 +945,9 @@ void pc_cpus_init(const char *cpu_model)
 #else
         cpu_model = "qemu32";
 #endif
+        machine_args->cpu_model = cpu_model;
     }
+    current_machine->hot_add_cpu = do_cpu_hot_add;
 
     ib = SYS_BUS_DEVICE(object_resolve_path_type("icc-bridge",
                                                  TYPE_ICC_BRIDGE, NULL));
include/hw/boards.h +3 -0
index 425bdc7..de8f92a 100644
@@ -18,6 +18,8 @@ typedef struct QEMUMachineInitArgs {
     const char *cpu_model;
 } QEMUMachineInitArgs;
 
+extern QEMUMachineInitArgs *machine_args;
+
 typedef void QEMUMachineInitFunc(QEMUMachineInitArgs *args);
 
 typedef void QEMUMachineResetFunc(void);
@@ -43,6 +45,7 @@ typedef struct QEMUMachine {
     GlobalProperty *compat_props;
     struct QEMUMachine *next;
     const char *hw_version;
+    void (*hot_add_cpu)(const int64_t id, Error **errp);
 } QEMUMachine;
 
 int qemu_register_machine(QEMUMachine *m);
qapi-schema.json +11 -0
index 751d3c2..34d3e84 100644
@@ -1387,6 +1387,17 @@
 { 'command': 'cpu', 'data': {'index': 'int'} }
 
 ##
+# @cpu-add
+#
+# Adds CPU with specified id
+#
+# @id: cpu id of CPU to be created
+#
+# Returns: Nothing on success
+##
+{ 'command': 'cpu-add', 'data': {'id': 'int'} }
+
+##
 # @memsave:
 #
 # Save a portion of guest memory to a file.
qmp-commands.hx +23 -0
index 4d65422..1e5d299 100644
@@ -385,6 +385,29 @@ Note: CPUs' indexes are obtained with the 'query-cpus' command.
 EQMP
 
     {
+        .name       = "cpu-add",
+        .args_type  = "id:i",
+        .mhandler.cmd_new = qmp_marshal_input_cpu_add,
+    },
+
+SQMP
+cpu-add
+-------
+
+Adds virtual cpu
+
+Arguments:
+
+- "id": cpu id (json-int)
+
+Example:
+
+-> { "execute": "cpu-add", "arguments": { "id": 2 } }
+<- { "return": {} }
+
+EQMP
+
+    {
         .name       = "memsave",
         .args_type  = "val:l,size:i,filename:s,cpu:i?",
         .mhandler.cmd_new = qmp_marshal_input_memsave,
qmp.c +10 -0
index 55b056b..8a9fd9e 100644
@@ -24,6 +24,7 @@
 #include "hw/qdev.h"
 #include "sysemu/blockdev.h"
 #include "qom/qom-qobject.h"
+#include "hw/boards.h"
 
 NameInfo *qmp_query_name(Error **errp)
 {
@@ -108,6 +109,15 @@ void qmp_cpu(int64_t index, Error **errp)
     /* Just do nothing */
 }
 
+void qmp_cpu_add(int64_t id, Error **errp)
+{
+    if (current_machine->hot_add_cpu) {
+        current_machine->hot_add_cpu(id, errp);
+    } else {
+        error_setg(errp, "Not supported");
+    }
+}
+
 #ifndef CONFIG_VNC
 /* If VNC support is enabled, the "true" query-vnc command is
    defined in the VNC subsystem */
vl.c +5 -1
index bc9c016..8d87d97 100644
@@ -179,6 +179,8 @@ int main(int argc, char **argv)
 #define MAX_VIRTIO_CONSOLES 1
 #define MAX_SCLP_CONSOLES 1
 
+QEMUMachineInitArgs *machine_args;
+
 static const char *data_dir[16];
 static int data_dir_idx;
 const char *bios_name = NULL;
@@ -4306,13 +4308,15 @@ int main(int argc, char **argv, char **envp)
                                  .kernel_cmdline = kernel_cmdline,
                                  .initrd_filename = initrd_filename,
                                  .cpu_model = cpu_model };
+    machine_args = &args;
+    current_machine = machine;
+
     machine->init(&args);
 
     cpu_synchronize_all_post_init();
 
     set_numa_modes();
 
-    current_machine = machine;
 
     /* init USB devices */
     if (usb_enabled(false)) {
Signature · 1 line
1.8.2
#68 lore raw

Re: [Qemu-devel] [PATCH 16/16] add cpu-add qmp command and implement CPU hot-add for target-i386

Quoted 4 lines · Igor Mammedov

... via current_machine->cpu_hot_add() hook called by cpu-set QMP command, for x86 target.

cpu-add's "id" argument is a CPU thread number in a range [0..max-cpus - 1)

Off by one. It's either [0..max-cpus) or [0..max-cpus - 1] (they mean the same thing).

It might be worth including a sample QMP command in the commit message.

Quoted 5 lines

---
v5:
* accept id=[0..max_cpus) range in cpu-add command

This notation is right, unlike the commit message.

Reviewing just the QMP portion:

qapi-schema.json line 1394

Quoted 10 lines

+++ b/qapi-schema.json

qapi-schema.json +5 -0
@@ -1387,6 +1387,17 @@
 { 'command': 'cpu', 'data': {'index': 'int'} }
 
 ##
+# @cpu-add
+#
+# Adds CPU with specified id
+#
+# @id: cpu id of CPU to be created

Here it would be helpful to mention what forms a valid id (your [0..max-cpus) notation from the commit message, for example).

qapi-schema.json line 1399

Quoted 5 lines
qapi-schema.json +5 -0
+#
+# Returns: Nothing on success
+##
+{ 'command': 'cpu-add', 'data': {'id': 'int'} }
+

Should be usable from libvirt's perspective, even if hot-plugging more than one cpu requires more than one QMP call. Do we have a counterpart QMP call to easily determine which cpu ids can still be hotplugged? If so, should we mention that in the documentation of this command?

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#69 lore raw

Re: [Qemu-devel] [PATCH 16/16] add cpu-add qmp command and implement CPU hot-add for target-i386

Quoted 10 lines · On Mon, 15 Apr 2013 16:20:15 -0600 Eric Blake <[email protected]> wrote:
Quoted 4 lines · Igor Mammedov

... via current_machine->cpu_hot_add() hook called by cpu-set QMP command, for x86 target.

cpu-add's "id" argument is a CPU thread number in a range [0..max-cpus - 1)

Off by one. It's either [0..max-cpus) or [0..max-cpus - 1] (they mean the same thing).

It might be worth including a sample QMP command in the commit message.

Ok, will ammend it on next respin.

Quoted 15 lines
Quoted 6 lines

---
v5:
* accept id=[0..max_cpus) range in cpu-add command

This notation is right, unlike the commit message.

Reviewing just the QMP portion:

Quoted 4 lines

+++ b/qapi-schema.json

qapi-schema.json +0 -0
@@ -1387,6 +1387,17 @@
 { 'command': 'cpu', 'data': {'index': 'int'} }
 
Quoted 9 lines

qapi-schema.json line 1394

Quoted 6 lines
qapi-schema.json +5 -0
 ##
+# @cpu-add
+#
+# Adds CPU with specified id
+#
+# @id: cpu id of CPU to be created

Here it would be helpful to mention what forms a valid id (your [0..max-cpus) notation from the commit message, for example).

ditto

Quoted 11 lines

qapi-schema.json line 1399

Quoted 6 lines
qapi-schema.json +5 -0
+#
+# Returns: Nothing on success
+##
+{ 'command': 'cpu-add', 'data': {'id': 'int'} }
+

Should be usable from libvirt's perspective, even if hot-plugging more than one cpu requires more than one QMP call. Do we have a counterpart QMP call to easily determine which cpu ids can still be hotplugged? If so, should we mention that in the documentation of this command?

We do not have it yet. Despite interface allowing to plug arbitrary CPU, libvirt shouldn't do it in order not to break migration support. Since migration target should be started with all CPU from source (including hot-plugged ones). And current command line doesn't have means for this.

I'd propose do implement in libvirt something like:

hotplug_id = current_cpu_count
{ "execute": "cpu-add", "arguments": { "id": hotplug_id } } current_cpu_count += 1

until arbitrary CPU hotplug and interface for enumerating possible CPUs settle down.

There was a proposal for enumerating possible CPUs in previous version, but it was target specific, and I was convinced to drop it for 1.5 and aim for generic way to expose this information later.
http://lists.gnu.org/archive/html/qemu-devel/2013-04/msg02286.html
Opinion from libvirt POV would be nice to have though.

Quoted 5 lines
Signature · 3 lines
-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
Signature · 2 lines
Regards,
  Igor
#70 lore raw

Re: [Qemu-devel] [PATCH 16/16] add cpu-add qmp command and implement CPU hot-add for target-i386

Quoted 15 lines · Igor Mammedov
Quoted 10 lines

qapi-schema.json line 1399

Quoted 5 lines
qapi-schema.json +5 -0
+#
+# Returns: Nothing on success
+##
+{ 'command': 'cpu-add', 'data': {'id': 'int'} }
+

Should be usable from libvirt's perspective, even if hot-plugging more than one cpu requires more than one QMP call. Do we have a counterpart QMP call to easily determine which cpu ids can still be hotplugged? If so, should we mention that in the documentation of this command?

We do not have it yet. Despite interface allowing to plug arbitrary CPU, libvirt shouldn't do it in order not to break migration support. Since migration target should be started with all CPU from source (including hot-plugged ones). And current command line doesn't have means for this.

Quoted 8 lines

I'd propose do implement in libvirt something like:

hotplug_id = current_cpu_count
{ "execute": "cpu-add", "arguments": { "id": hotplug_id } } current_cpu_count += 1

until arbitrary CPU hotplug and interface for enumerating possible CPUs settle down.

Yes, that's pretty much how libvirt already handles hotplug for xen:
libvirt currently maintains its own count of plugged cpus, and hot[un]plugs the next id so that the online cpus are always contiguous ids starting from 0. We are discussing on the libvirt lists ways of improving libvirt API to expose hotplug of arbitrary ids (after all, a 4-cpu bare metal machine can hotunplug cpu 1 while still leaving 2 and 3 online) - but that can come later, perhaps as qemu 1.6 gains better support for NUMA layout with better control of what ids should be used.

Quoted 6 lines

There was a proposal for enumerating possible CPUs in previous version, but it was target specific, and I was convinced to drop it for 1.5 and aim for generic way to expose this information later.
http://lists.gnu.org/archive/html/qemu-devel/2013-04/msg02286.html
Opinion from libvirt POV would be nice to have though.

Agreed that deferring more powerful id enumeration for a later patch series is fine, and shouldn't hold up getting basic hotplug into 1.5.

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#71 lore raw
In reply to Igor Mammedov
Quoted 2 lines · Igor Mammedov

Implements alternative way for hot-adding CPU using cpu-add QMP command, wich could be useful until it would be possible to add CPUs via device_add.

Didn't track the full story: What prevents currently a device_add approach? And that so effectively that we have to create a to-be-deprecated-again QMP API first?

Thanks,
Jan

Quoted 112 lines

All patches except the last are also applicable to device_add aprroach.

To hot-add CPU use following command from qmp-shell:
cpu-add id=[0..max-cpus - 1)

git tree for testing: https://github.com/imammedo/qemu/tree/cpu_add.v4

based on qom-cpu tree

v4->v3:
* 'id' in cpu-add command will be a thread number instead of APIC ID
* split off resume_vcpu() into separate patch
* move notifier from rtc code into pc.c

v2->v3:
* use local error & propagate_error() instead of operating on passed in errp in several places
* replace CPUClass.get_firmware_id() with CPUClass.get_arch_id()
* leave IOAPIC creation to board and just set bus to icc-bus
* include kvm-stub.o in cpu libary if no KVM is configured
* create resume_vcpu() stub and include it in libqemustub, and use it directly instead of CPU method
* acpi_piix4: s/cpu_add_notifier/cpu_added_notifier/

v1->v2:
* generalize cpu sync to KVM, resume and hot-plug notification and invoke them form CPUClass, to make available to all targets.
* introduce cpu_exists() and CPUClass.get_firmware_id() and use the last one in acpi_piix to make code target independent.
* move IOAPIC to ICC bus, it was suggested and easy to convert.
* leave kvmvapic as SysBusDevice, it doesn't affect hot-plug and created only once for all APIC instances. I haven't found yet good/clean enough way to convert it to ICCDevice. May be follow-up though.
* split one big ICC patch into several, one per converted device
* add cpu_hot_add hook to machine and implement it for target-i386, instead of adding stabs. Could be used by other targets to implement cpu-add.
* pre-allocate links<CPU> for all possible CPUs and make them available at /machine/icc-bridge/cpu[0..N] QOM path, so users could find out possible/free CPU IDs to use in cpu-add command.

Igor Mammedov (16):
cpu: make kvm-stub.o a part of CPU library
cpu: call cpu_synchronize_post_init() from CPUClass.realize() if
hotplugged
introduce resume_vcpu(), for single CPU
cpu: resume CPU from CPUClass.cpu_common_realizefn() when it is
hot-plugged
introduce CPU hot-plug notifier
target-i386: pc: update rtc_cmos on CPU hot-plug
cpu: introduce get_arch_id() method and override it for target-i386
cpu: add helper cpu_exists(), to check if CPU with specified id exists
acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest
target-i386: introduce apic-id property
introduce ICC bus/device/bridge
target-i386: cpu: attach ICC bus to CPU on its creation
target-i386: replace MSI_SPACE_SIZE with APIC_SPACE_SIZE
target-i386: move APIC to ICC bus
target-i386: move IOAPIC to ICC bus
add cpu-add qmp command and implement CPU hot-add for target-i386

 Makefile.target                    |  14 ++---
 cpus.c                             |  11 +++-
 default-configs/i386-softmmu.mak   |   1 +
 default-configs/x86_64-softmmu.mak |   1 +
 hw/acpi/piix4.c                    | 114 ++++++++++++++++++++++++++++++++++++-
 hw/cpu/Makefile.objs               |   1 +
 hw/cpu/icc_bus.c                   | 104 +++++++++++++++++++++++++++++++++
 hw/i386/kvm/apic.c                 |   2 +-
 hw/i386/kvm/ioapic.c               |   2 +-
 hw/i386/kvmvapic.c                 |   1 +
 hw/i386/pc.c                       |  91 +++++++++++++++++++++++++++--
 hw/i386/pc_piix.c                  |   7 +++
 hw/i386/pc_q35.c                   |   7 +++
 hw/intc/apic.c                     |   2 +-
 hw/intc/apic_common.c              |  17 ++++--
 hw/intc/ioapic_common.c            |  15 +++--
 hw/timer/mc146818rtc.c             |   7 +++
 hw/xen/xen_apic.c                  |   2 +-
 include/exec/memory.h              |  10 ++++
 include/hw/boards.h                |   3 +
 include/hw/i386/apic_internal.h    |   8 +--
 include/hw/i386/icc_bus.h          |  53 +++++++++++++++++
 include/hw/i386/ioapic_internal.h  |   6 +-
 include/hw/pci/msi.h               |   2 +
 include/hw/timer/mc146818rtc.h     |   1 +
 include/qom/cpu.h                  |  21 +++++++
 include/sysemu/kvm.h               |  22 +++----
 include/sysemu/sysemu.h            |   3 +
 kvm-all.c                          |   1 +
 kvm-stub.c                         |   3 +
 memory.c                           |  11 ++++
 qapi-schema.json                   |  11 ++++
 qmp-commands.hx                    |  23 ++++++++
 qmp.c                              |  10 ++++
 qom/cpu.c                          |  44 ++++++++++++++
 stubs/Makefile.objs                |   1 +
 stubs/cpus.c                       |   5 ++
 target-i386/cpu.c                  |  75 +++++++++++++++++++-----
 target-i386/cpu.h                  |   1 +
 vl.c                               |   7 ++-
 40 files changed, 656 insertions(+), 64 deletions(-)
 create mode 100644 hw/cpu/icc_bus.c
 create mode 100644 include/hw/i386/icc_bus.h
 create mode 100644 stubs/cpus.c
Signature · 2 lines
Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
Corporate Competence Center Embedded Linux
#72 lore raw
Quoted 7 lines · On Tue, 16 Apr 2013 10:30:55 +0200 Jan Kiszka <[email protected]> wrote:
Quoted 2 lines · Igor Mammedov

Implements alternative way for hot-adding CPU using cpu-add QMP command, wich could be useful until it would be possible to add CPUs via device_add.

Didn't track the full story: What prevents currently a device_add approach? And that so effectively that we have to create a to-be-deprecated-again QMP API first?

I guess mostly it's review bandwidth and time concerns, it moves but quite slow and won't be able to make into 1.5 and possibly miss 1.6 with current speed.

In this series all patches except the last one applicable to device_add as well so any feedback is appreciated.

To make device_add usable for CPU, I have on my TODO list following series/topics:

1. convert cpuid features to static properties.
http://lists.gnu.org/archive/html/qemu-devel/2013-02/msg04426.html

2. x86 CPU subclasses
http://lists.gnu.org/archive/html/qemu-devel/2013-02/msg00673.html

Summarized issues/ways to go with CPU subclasses could be found here:
http://wiki.qemu.org/Features/CPUHotplug section "CPU models as CPU subclasses"

[optional]
3. nice to have, unify cpu_model handling and convert it to utilizing global properties. With it hotplug could look as simple as:
device_add apic_id=xxx
without specifying all flags that were on -cpu command line.

BTW:
You were advocating using APIC ID on device_add to identify CPU but Eduardo would like avoid its usage on external interfaces.
It would be nice to have your opinion on subject.
relevant discussion threads are here:
1. http://lists.gnu.org/archive/html/qemu-devel/2013-04/msg02205.html
2. http://lists.gnu.org/archive/html/qemu-devel/2013-04/msg02274.html

Quoted 120 lines

Thanks,
Jan

Quoted 112 lines

All patches except the last are also applicable to device_add aprroach.

To hot-add CPU use following command from qmp-shell:
cpu-add id=[0..max-cpus - 1)

git tree for testing: https://github.com/imammedo/qemu/tree/cpu_add.v4

based on qom-cpu tree

v4->v3:
* 'id' in cpu-add command will be a thread number instead of APIC ID
* split off resume_vcpu() into separate patch
* move notifier from rtc code into pc.c

v2->v3:
* use local error & propagate_error() instead of operating on passed in errp in several places
* replace CPUClass.get_firmware_id() with CPUClass.get_arch_id()
* leave IOAPIC creation to board and just set bus to icc-bus
* include kvm-stub.o in cpu libary if no KVM is configured
* create resume_vcpu() stub and include it in libqemustub, and use it directly instead of CPU method
* acpi_piix4: s/cpu_add_notifier/cpu_added_notifier/

v1->v2:
* generalize cpu sync to KVM, resume and hot-plug notification and invoke them form CPUClass, to make available to all targets.
* introduce cpu_exists() and CPUClass.get_firmware_id() and use the last one in acpi_piix to make code target independent.
* move IOAPIC to ICC bus, it was suggested and easy to convert.
* leave kvmvapic as SysBusDevice, it doesn't affect hot-plug and created only once for all APIC instances. I haven't found yet good/clean enough way to convert it to ICCDevice. May be follow-up though.
* split one big ICC patch into several, one per converted device
* add cpu_hot_add hook to machine and implement it for target-i386, instead of adding stabs. Could be used by other targets to implement cpu-add.
* pre-allocate links<CPU> for all possible CPUs and make them available at /machine/icc-bridge/cpu[0..N] QOM path, so users could find out possible/free CPU IDs to use in cpu-add command.

Igor Mammedov (16):
cpu: make kvm-stub.o a part of CPU library
cpu: call cpu_synchronize_post_init() from CPUClass.realize() if
hotplugged
introduce resume_vcpu(), for single CPU
cpu: resume CPU from CPUClass.cpu_common_realizefn() when it is
hot-plugged
introduce CPU hot-plug notifier
target-i386: pc: update rtc_cmos on CPU hot-plug
cpu: introduce get_arch_id() method and override it for target-i386
cpu: add helper cpu_exists(), to check if CPU with specified id exists
acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest
target-i386: introduce apic-id property
introduce ICC bus/device/bridge
target-i386: cpu: attach ICC bus to CPU on its creation
target-i386: replace MSI_SPACE_SIZE with APIC_SPACE_SIZE
target-i386: move APIC to ICC bus
target-i386: move IOAPIC to ICC bus
add cpu-add qmp command and implement CPU hot-add for target-i386

 Makefile.target                    |  14 ++---
 cpus.c                             |  11 +++-
 default-configs/i386-softmmu.mak   |   1 +
 default-configs/x86_64-softmmu.mak |   1 +
 hw/acpi/piix4.c                    | 114 ++++++++++++++++++++++++++++++++++++-
 hw/cpu/Makefile.objs               |   1 +
 hw/cpu/icc_bus.c                   | 104 +++++++++++++++++++++++++++++++++
 hw/i386/kvm/apic.c                 |   2 +-
 hw/i386/kvm/ioapic.c               |   2 +-
 hw/i386/kvmvapic.c                 |   1 +
 hw/i386/pc.c                       |  91 +++++++++++++++++++++++++++--
 hw/i386/pc_piix.c                  |   7 +++
 hw/i386/pc_q35.c                   |   7 +++
 hw/intc/apic.c                     |   2 +-
 hw/intc/apic_common.c              |  17 ++++--
 hw/intc/ioapic_common.c            |  15 +++--
 hw/timer/mc146818rtc.c             |   7 +++
 hw/xen/xen_apic.c                  |   2 +-
 include/exec/memory.h              |  10 ++++
 include/hw/boards.h                |   3 +
 include/hw/i386/apic_internal.h    |   8 +--
 include/hw/i386/icc_bus.h          |  53 +++++++++++++++++
 include/hw/i386/ioapic_internal.h  |   6 +-
 include/hw/pci/msi.h               |   2 +
 include/hw/timer/mc146818rtc.h     |   1 +
 include/qom/cpu.h                  |  21 +++++++
 include/sysemu/kvm.h               |  22 +++----
 include/sysemu/sysemu.h            |   3 +
 kvm-all.c                          |   1 +
 kvm-stub.c                         |   3 +
 memory.c                           |  11 ++++
 qapi-schema.json                   |  11 ++++
 qmp-commands.hx                    |  23 ++++++++
 qmp.c                              |  10 ++++
 qom/cpu.c                          |  44 ++++++++++++++
 stubs/Makefile.objs                |   1 +
 stubs/cpus.c                       |   5 ++
 target-i386/cpu.c                  |  75 +++++++++++++++++++-----
 target-i386/cpu.h                  |   1 +
 vl.c                               |   7 ++-
 40 files changed, 656 insertions(+), 64 deletions(-)
 create mode 100644 hw/cpu/icc_bus.c
 create mode 100644 include/hw/i386/icc_bus.h
 create mode 100644 stubs/cpus.c
Signature · 2 lines
Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
Corporate Competence Center Embedded Linux
Signature · 2 lines
Regards,
  Igor
#73 lore raw

On Tue, Apr 16, 2013 at 11:43:42AM +0200, Igor Mammedov wrote:
[...]

Quoted 7 lines

BTW:
You were advocating using APIC ID on device_add to identify CPU but Eduardo would like avoid its usage on external interfaces.
It would be nice to have your opinion on subject.
relevant discussion threads are here:
1. http://lists.gnu.org/archive/html/qemu-devel/2013-04/msg02205.html
2. http://lists.gnu.org/archive/html/qemu-devel/2013-04/msg02274.html

Trying to summarize my concerns quickly:

0) APIC IDs must be calculated by QEMU based on CPU topology and can't be chosen arbitrarily.
1) Requiring APIC ID as the CPU ID requires a target-independent probing interface to find out which IDs are valid, and how they correspond to locations in the CPU topology. We don't have such interface yet.
* This makes it impossible to use APIC IDs to identify CPUs in the command-line (before QMP is available), so we still need another kind of identifier (topology-based, probably).
2) apic-id is not compatible with the existing IDs used for the "-numa" command-line option.

So, using APIC ID is possible, but I believe it introduces too much unnecessary complexity in the external interface. The day we introduce a full socket/core/thread-based device_add interface, setting APIC ID directly externally won't be necessary anymore (QEMU could calculate it based on the object topology), but while we don't have the full-featured core/socket/thread-based interface we can simply use IDs in the range [0..(max_cpus-1)], that are already used by the "-numa" command-line option.

Signature · 1 line
Eduardo