target-i386: CPU hot-add with cpu-add QMP command
v8 cover target-i386: CPU hot-add with cpu-add QMP command
Implements alternative way for hot-adding CPU using cpu-add QMP command, which could be useful until it would be possible to add CPUs via device_add.
To hot-add CPU use following command from qmp-shell:
cpu-add id=[0..max-cpus - 1)
git tree for testing: https://github.com/imammedo/qemu/tree/cpu_add.v8
based on qom-cpu tree
v8->v7:
* skip already applied patches
* split adding hot_add_cpu hook into separate function
* add cpu-model machine option and use QemuOpts for getting it during CPU hotplug.
v7->v6:
* skip already applied patches
* rename icc-bus instance name to "icc"
* pass icc_bridge from board as argument down CPU creation call chain, instead of dynamically resolving it for each CPU.
v6->v5:
* override hot_add_cpu hook statically
* extend and use memory_region_find() in IOAPIC
* s/signal_cpu_creation/tcg_signal_cpu_creation/
* add "since 1.5 to cpu-addQAPI schema description
v5->v4:
* style fixes
* new helper qemu_for_each_cpu()
* switch to qemu_for_each_cpu() in cpu_exists()
* "pc: update rtc ..." patch make depend it on "mc146818rtc: QOM'ify" and use QOM cast style
* call CPU added notifier right before CPU becomes runable
* s/resume_vcpu/cpu_resume/
* acpi/piix4: add spec documentation for QEMU<->Seabios CPU hotplug interface
* use error_propagate() in pc_new_cpu()
* skip cpu_exists() check in apic-id property setter if new value is the same as current
* embed icc-bus inside icc-bridge and use qbus_create_inplace()
* move include/hw/i386/icc_bus.h into include/hw/cpu/
* make missing icc-bus fatal error for softmmu target
* split "move APIC to ICC bus" and "move IOAPIC to ICC bus" on smaller patches
* use qdev_get_parent_bus() to get parent bus
* split "add cpu-add command..." on smaller patches
v4->v3:
* 'id' in cpu-add command will be a thread number instead of APIC ID
* split off resume_vcpu() into separate patch
* move notifier from rtc code into pc.c
v2->v3:
* use local error & propagate_error() instead of operating on passed in errp in several places
* replace CPUClass.get_firmware_id() with CPUClass.get_arch_id()
* leave IOAPIC creation to board and just set bus to icc-bus
* include kvm-stub.o in cpu libary if no KVM is configured
* create resume_vcpu() stub and include it in libqemustub, and use it directly instead of CPU method
* acpi_piix4: s/cpu_add_notifier/cpu_added_notifier/
v1->v2:
* generalize cpu sync to KVM, resume and hot-plug notification and invoke them form CPUClass, to make available to all targets.
* introduce cpu_exists() and CPUClass.get_firmware_id() and use the last one in acpi_piix to make code target independent.
* move IOAPIC to ICC bus, it was suggested and easy to convert.
* leave kvmvapic as SysBusDevice, it doesn't affect hot-plug and created only once for all APIC instances. I haven't found yet good/clean enough way to convert it to ICCDevice. May be follow-up though.
* split one big ICC patch into several, one per converted device
* add cpu_hot_add hook to machine and implement it for target-i386, instead of adding stabs. Could be used by other targets to implement cpu-add.
* pre-allocate links<CPU> for all possible CPUs and make them available at /machine/icc-bridge/cpu[0..N] QOM path, so users could find out possible/free CPU IDs to use in cpu-add command.
Igor Mammedov (4):
add hot_add_cpu hook to QEMUMachine
QMP: add cpu-add command
add cpu-model option to -machine
target-i386: implement machine->hot_add_cpu hook
hw/i386/pc.c | 29 +++++++++++++++++++++++++++++ hw/i386/pc_piix.c | 1 + hw/i386/pc_q35.c | 1 + include/hw/boards.h | 1 + include/hw/i386/pc.h | 1 + qapi-schema.json | 13 +++++++++++++ qmp-commands.hx | 23 +++++++++++++++++++++++ qmp.c | 10 ++++++++++ vl.c | 12 +++++++++++- 9 files changed, 90 insertions(+), 1 deletion(-)
Signature · 1 line
1.8.2.1
PATCH v8 1/4 add hot_add_cpu hook to QEMUMachine
hot_add_cpu hook should be overriden by target that implements cpu hot-add via cpu-add QMP command.
---
Note:
hook will be used in next patch for target-i386 cp-add implementation.
--- include/hw/boards.h | 1 + 1 file changed, 1 insertion(+)
include/hw/boards.h +1 -0
@@ -43,6 +43,7 @@ typedef struct QEMUMachine {GlobalProperty *compat_props;struct QEMUMachine *next;const char *hw_version; +void (*hot_add_cpu)(const int64_t id,Error **errp); } QEMUMachine;int qemu_register_machine (QEMUMachine *m);
Signature · 1 line
1.8.2.1
PATCH v8 2/4 QMP: add cpu-add command
In reply to Igor MammedovAdds "cpu-add id=xxx" QMP command.
cpu-add's "id" argument is a CPU number in a range [0..max-cpus)
Example QMP command:
-> { "execute": "cpu-add", "arguments": { "id": 2 } } <- { "return": {} }
---
v7:
* added "Since 1.5" to cpu-add qapi schema definition
v6:
* added valid values description to qapi schema
* split out cpu_hot_add hooks introduction into separate patch
* split out implementation of cpu_hot_add for target-i386
v5:
* accept id=[0..max_cpus) range in cpu-add command
v4:
* merge "qmp: add cpu-add qmp command" & "target-i386: implement CPU hot-add" patches
* move notifier call to CPUCLass.realize()
* add hook cpu_hot_add to QEMUMachine
* make QEMUMachineInitArgs global and keep default cpu_model there
v3:
* it appears that 'online/offline' in cpu-set are confusing people with what command actually does and users might have to distinguish if 'offline' is not implemented by parsing error message. To simplify things replace cpu-set with cpu-add command to show more clear what command does and just add cpu-del when CPU remove is implemented.
v2:
* s/cpu_set/cpu-set/
* qmp doc style fix
* use bool type instead of opencodding online/offline string
--- qapi-schema.json | 13 +++++++++++++ qmp-commands.hx | 23 +++++++++++++++++++++++ qmp.c | 10 ++++++++++ 3 files changed, 46 insertions(+)
qapi-schema.json +13 -0
@@ -1387,6 +1387,19 @@ { 'command': 'cpu', 'data': {'index': 'int'} } ## +# @cpu-add +# +# Adds CPU with specified ID +# +# @id: ID of CPU to be created, valid values [0..max_cpus) +# +# Returns: Nothing on success +# +# Since 1.5 +## +{ 'command': 'cpu-add', 'data': {'id': 'int'} } + +## # @memsave: # # Save a portion of guest memory to a file.
qmp-commands.hx +23 -0
@@ -385,6 +385,29 @@ Note: CPUs' indexes are obtained with the 'query-cpus' command. EQMP { + .name = "cpu-add", + .args_type = "id:i", + .mhandler.cmd_new = qmp_marshal_input_cpu_add, + }, + +SQMP +cpu-add +------- + +Adds virtual cpu + +Arguments: + +- "id": cpu id (json-int) + +Example: + +-> { "execute": "cpu-add", "arguments": { "id": 2 } } +<- { "return": {} } + +EQMP + + { .name = "memsave", .args_type = "val:l,size:i,filename:s,cpu:i?", .mhandler.cmd_new = qmp_marshal_input_memsave,
qmp.c +10 -0
@@ -24,6 +24,7 @@#include "hw/qdev.h" #include "sysemu/blockdev.h" #include "qom/qom-qobject.h" +#include "hw/boards.h" NameInfo *qmp_query_name (Error **errp) {
@@ -108,6 +109,15 @@ void qmp_cpu(int64_t index, Error **errp)/* Just do nothing */ } +void qmp_cpu_add (int64_t id,Error **errp) +{ +if (current_machine->hot_add_cpu) { + current_machine->hot_add_cpu (id, errp); + }else { +error_setg (errp,"Not supported" ); + } +} +#ifndef CONFIG_VNC /* If VNC support is enabled, the "true" query-vnc command is defined in the VNC subsystem */
Signature · 1 line
1.8.2.1
Re: [Qemu-devel] [PATCH 2/4] QMP: add cpu-add command
(CCing libvir-list)
Quoted 11 lines · Igor Mammedov
Adds "cpu-add id=xxx" QMP command.
cpu-add's "id" argument is a CPU number in a range [0..max-cpus)
Example QMP command:
-> { "execute": "cpu-add", "arguments": { "id": 2 } } <- { "return": {} }Signed-off-by Igor Mammedov <[email protected]>Acked-by Luiz Capitulino <[email protected]>Reviewed-by Eric Blake <[email protected]>
The only way to find out if CPU hotplug is really available on a given machine-type is by actually trying to run cpu-add, right? Is this sufficient for libvirt requirements?
Quoted 122 lines
---
v7:
* added "Since 1.5" to cpu-add qapi schema definition
v6:
* added valid values description to qapi schema
* split outcpu_hot_addhooks introduction into separate patch
* split out implementation ofcpu_hot_addfor target-i386
v5:
* accept id=[0..max_cpus) range in cpu-add command
v4:
* merge "qmp: add cpu-add qmp command" & "target-i386: implement CPU hot-add" patches
* move notifier call toCPUCLass.realize()
* add hookcpu_hot_addto QEMUMachine
* make QEMUMachineInitArgs global and keep defaultcpu_modeltherev3:
* it appears that 'online/offline' in cpu-set are confusing people with what command actually does and users might have to distinguish if 'offline' is not implemented by parsing error message. To simplify things replace cpu-set with cpu-add command to show more clear what command does and just add cpu-del when CPU remove is implemented.v2:
* s/cpu_set/cpu-set/
* qmp doc style fix
* use bool type instead of opencodding online/offline stringsuggested-by Eric Blake <[email protected]>--- qapi-schema.json | 13 +++++++++++++ qmp-commands.hx | 23 +++++++++++++++++++++++ qmp.c | 10 ++++++++++ 3 files changed, 46 insertions(+)qapi-schema.json +13
-0@@ -1387,6 +1387,19 @@ { 'command': 'cpu', 'data': {'index': 'int'} } ## +# @cpu-add +# +# Adds CPU with specified ID +# +# @id: ID of CPU to be created, valid values [0..max_cpus) +# +# Returns: Nothing on success +# +# Since 1.5 +## +{ 'command': 'cpu-add', 'data': {'id': 'int'} } + +## # @memsave: # # Save a portion of guest memory to a file.qmp-commands.hx +23
-0@@ -385,6 +385,29 @@ Note: CPUs' indexes are obtained with the 'query-cpus' command. EQMP { + .name = "cpu-add", + .args_type = "id:i", + .mhandler.cmd_new = qmp_marshal_input_cpu_add, + }, + +SQMP +cpu-add +------- + +Adds virtual cpu + +Arguments: + +- "id": cpu id (json-int) + +Example: + +-> { "execute": "cpu-add", "arguments": { "id": 2 } } +<- { "return": {} } + +EQMP + + { .name = "memsave", .args_type = "val:l,size:i,filename:s,cpu:i?", .mhandler.cmd_new = qmp_marshal_input_memsave,qmp.c +10
-0@@ -24,6 +24,7 @@#include "hw/qdev.h" #include "sysemu/blockdev.h" #include "qom/qom-qobject.h" +#include "hw/boards.h" NameInfo *qmp_query_name (Error **errp) {@@ -108,6 +109,15 @@ void qmp_cpu(int64_t index, Error **errp)/* Just do nothing */ } +void qmp_cpu_add (int64_t id,Error **errp) +{ +if (current_machine->hot_add_cpu) { + current_machine->hot_add_cpu (id, errp); + }else { +error_setg (errp,"Not supported" ); + } +} +#ifndef CONFIG_VNC /* If VNC support is enabled, the "true" query-vnc command is defined in the VNC subsystem */ Signature · 1 line
1.8.2.1
Signature · 1 line
Eduardo
Re: [Qemu-devel] [libvirt] [PATCH 2/4] QMP: add cpu-add command
Quoted 11 lines · Eduardo Habkost
(CCing libvir-list)
Quoted 8 lines · Igor Mammedov
Adds "cpu-add id=xxx" QMP command.
cpu-add's "id" argument is a CPU number in a range [0..max-cpus)
Example QMP command:
-> { "execute": "cpu-add", "arguments": { "id": 2 } } <- { "return": {} }
Quoted 9 lines
Quoted 3 lines
Signed-off-by Igor Mammedov <[email protected]>Acked-by Luiz Capitulino <[email protected]>Reviewed-by Eric Blake <[email protected]>The only way to find out if CPU hotplug is really available on a given machine-type is by actually trying to run cpu-add, right? Is this sufficient for libvirt requirements?
As long as the command fails when it's not supported it's okay.
(cpu_set HMP command does not fail when offlining a cpu even if it isn't supported and that's real pain to use)
Peter
Re: [Qemu-devel] [PATCH 2/4] QMP: add cpu-add command
In reply to Eduardo HabkostQuoted 11 lines · On Tue, 30 Apr 2013 10:46:20 -0300 Eduardo Habkost <[email protected]> wrote:
(CCing libvir-list)
Quoted 8 lines · Igor Mammedov
Adds "cpu-add id=xxx" QMP command.
cpu-add's "id" argument is a CPU number in a range [0..max-cpus)
Example QMP command:
-> { "execute": "cpu-add", "arguments": { "id": 2 } } <- { "return": {} }
Quoted 7 lines
Quoted 3 lines
Signed-off-by Igor Mammedov <[email protected]>Acked-by Luiz Capitulino <[email protected]>Reviewed-by Eric Blake <[email protected]>The only way to find out if CPU hotplug is really available on a given machine-type is by actually trying to run cpu-add, right? Is this sufficient for libvirt requirements?
yes, for now it is the only way.it was acked from their side.
Quoted 125 lines
Quoted 124 lines
---
v7:
* added "Since 1.5" to cpu-add qapi schema definition
v6:
* added valid values description to qapi schema
* split outcpu_hot_addhooks introduction into separate patch
* split out implementation ofcpu_hot_addfor target-i386
v5:
* accept id=[0..max_cpus) range in cpu-add command
v4:
* merge "qmp: add cpu-add qmp command" & "target-i386: implement CPU hot-add" patches
* move notifier call toCPUCLass.realize()
* add hookcpu_hot_addto QEMUMachine
* make QEMUMachineInitArgs global and keep defaultcpu_modeltherev3:
* it appears that 'online/offline' in cpu-set are confusing people with what command actually does and users might have to distinguish if 'offline' is not implemented by parsing error message. To simplify things replace cpu-set with cpu-add command to show more clear what command does and just add cpu-del when CPU remove is implemented.v2:
* s/cpu_set/cpu-set/
* qmp doc style fix
* use bool type instead of opencodding online/offline stringsuggested-by Eric Blake <[email protected]>--- qapi-schema.json | 13 +++++++++++++ qmp-commands.hx | 23 +++++++++++++++++++++++ qmp.c | 10 ++++++++++ 3 files changed, 46 insertions(+)qapi-schema.json +13
-0@@ -1387,6 +1387,19 @@ { 'command': 'cpu', 'data': {'index': 'int'} } ## +# @cpu-add +# +# Adds CPU with specified ID +# +# @id: ID of CPU to be created, valid values [0..max_cpus) +# +# Returns: Nothing on success +# +# Since 1.5 +## +{ 'command': 'cpu-add', 'data': {'id': 'int'} } + +## # @memsave: # # Save a portion of guest memory to a file.qmp-commands.hx +23
-0@@ -385,6 +385,29 @@ Note: CPUs' indexes are obtained with the 'query-cpus' command. EQMP { + .name = "cpu-add", + .args_type = "id:i", + .mhandler.cmd_new = qmp_marshal_input_cpu_add, + }, + +SQMP +cpu-add +------- + +Adds virtual cpu + +Arguments: + +- "id": cpu id (json-int) + +Example: + +-> { "execute": "cpu-add", "arguments": { "id": 2 } } +<- { "return": {} } + +EQMP + + { .name = "memsave", .args_type = "val:l,size:i,filename:s,cpu:i?", .mhandler.cmd_new = qmp_marshal_input_memsave,qmp.c +10
-0@@ -24,6 +24,7 @@#include "hw/qdev.h" #include "sysemu/blockdev.h" #include "qom/qom-qobject.h" +#include "hw/boards.h" NameInfo *qmp_query_name (Error **errp) {@@ -108,6 +109,15 @@ void qmp_cpu(int64_t index, Error **errp)/* Just do nothing */ } +void qmp_cpu_add (int64_t id,Error **errp) +{ +if (current_machine->hot_add_cpu) { + current_machine->hot_add_cpu (id, errp); + }else { +error_setg (errp,"Not supported" ); + } +} +#ifndef CONFIG_VNC /* If VNC support is enabled, the "true" query-vnc command is defined in the VNC subsystem */ Signature · 1 line
1.8.2.1
PATCH v8 3/4 add cpu-model option to -machine
In reply to Igor MammedovProvides globally accessible cpu-model via machine opts.
---
Note:
- it will be used in cpu-add hook on tartget-i386.
--- vl.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)
vl.c +11 -1
@@ -429,6 +429,10 @@ static QemuOptsList qemu_machine_opts = { .name = "usb", .type = QEMU_OPT_BOOL, .help = "Set on/off to enable/disable usb", + }, { + .name = "cpu-model", + .type = QEMU_OPT_STRING, + .help = "alias for \"-cpu\" CPU model definition", }, {/* End of list */ } },
@@ -2979,7 +2983,7 @@ int main(int argc, char **argv, char **envp) }case QEMU_OPTION_cpu:/* hw initialization will check this */ - cpu_model = optarg; +qemu_opts_set (qemu_find_opts ("machine" ),0 ,"cpu-model" , optarg);break ;case QEMU_OPTION_hda: {
@@ -3919,6 +3923,11 @@ int main(int argc, char **argv, char **envp)*/ cpudef_init (); + machine_opts =qemu_opts_find (qemu_find_opts ("machine" ),0 ); +if (machine_opts) { + cpu_model =qemu_opt_get (machine_opts,"cpu-model" ); + } +if (cpu_model &&is_help_option (cpu_model)) {list_cpus (stdout, &fprintf, cpu_model);exit (0 );
@@ -4124,6 +4133,7 @@ int main(int argc, char **argv, char **envp) kernel_filename =qemu_opt_get (machine_opts,"kernel" ); initrd_filename =qemu_opt_get (machine_opts,"initrd" ); kernel_cmdline =qemu_opt_get (machine_opts,"append" ); + cpu_model =qemu_opt_get (machine_opts,"cpu-model" ); }else { kernel_filename = initrd_filename = kernel_cmdline =NULL ; }
Signature · 1 line
1.8.2.1
Re: [Qemu-devel] [PATCH 3/4] add cpu-model option to -machine
Quoted 3 lines · Igor Mammedov
Provides globally accessible cpu-model via machine opts.
Signed-off-by Igor Mammedov <[email protected]>
It looks like most data on QEMUMachineInitArgs is already present on machine_opts. Maybe we could add machine_opts to QEMUMachineInitArgs and gradually remove existing QEMUMachineInitArgs fields?
Quoted 54 lines
---
Note:
- it will be used in cpu-add hook on tartget-i386.--- vl.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)vl.c +11
-1@@ -429,6 +429,10 @@ static QemuOptsList qemu_machine_opts = { .name = "usb", .type = QEMU_OPT_BOOL, .help = "Set on/off to enable/disable usb", + }, { + .name = "cpu-model", + .type = QEMU_OPT_STRING, + .help = "alias for \"-cpu\" CPU model definition", }, {/* End of list */ } },@@ -2979,7 +2983,7 @@ int main(int argc, char **argv, char **envp) }case QEMU_OPTION_cpu:/* hw initialization will check this */ - cpu_model = optarg; +qemu_opts_set (qemu_find_opts ("machine" ),0 ,"cpu-model" , optarg);break ;case QEMU_OPTION_hda: {@@ -3919,6 +3923,11 @@ int main(int argc, char **argv, char **envp)*/ cpudef_init (); + machine_opts =qemu_opts_find (qemu_find_opts ("machine" ),0 ); +if (machine_opts) { + cpu_model =qemu_opt_get (machine_opts,"cpu-model" ); + } +if (cpu_model &&is_help_option (cpu_model)) {list_cpus (stdout, &fprintf, cpu_model);exit (0 );@@ -4124,6 +4133,7 @@ int main(int argc, char **argv, char **envp) kernel_filename =qemu_opt_get (machine_opts,"kernel" ); initrd_filename =qemu_opt_get (machine_opts,"initrd" ); kernel_cmdline =qemu_opt_get (machine_opts,"append" ); + cpu_model =qemu_opt_get (machine_opts,"cpu-model" ); }else { kernel_filename = initrd_filename = kernel_cmdline =NULL ; }Signature · 1 line
1.8.2.1
Signature · 1 line
Eduardo
PATCH v8 4/4 target-i386: implement machine->hot_add_cpu hook
In reply to Igor Mammedov---
v3:
* use machine.cpu-model option to store default value and to get current value when hot-adding CPU
v2:
* override .hot_add_cpu staticaly starting with 1.5 machine
--- hw/i386/pc.c | 29 +++++++++++++++++++++++++++++ hw/i386/pc_piix.c | 1 + hw/i386/pc_q35.c | 1 + include/hw/i386/pc.h | 1 + 4 files changed, 32 insertions(+)
hw/i386/pc.c +29 -0
@@ -54,6 +54,7 @@#include "qemu/config-file.h" #include "hw/acpi/acpi.h" #include "hw/cpu/icc_bus.h" +#include "hw/boards.h" /* debug PC/ISA interrupts */ //#define DEBUG_IRQ
@@ -915,6 +916,33 @@ static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id,return cpu; } +void pc_hot_add_cpu (const int64_t id,Error **errp) +{ +DeviceState *icc_bridge; +const char *cpu_model; +QemuOpts *machine_opts; +int64_t apic_id =x86_cpu_apic_id_from_index (id); + +if (cpu_exists (apic_id)) { +error_setg (errp,"Unable to add CPU: %" PRIi64 +", it already exists" , id); +return ; + } + +if (id >= max_cpus) { +error_setg (errp,"Unable to add CPU: %" PRIi64 +", max allowed: %d" , id, max_cpus -1 ); +return ; + } + + machine_opts =qemu_opts_find (qemu_find_opts ("machine" ),0 ); + cpu_model =qemu_opt_get (machine_opts,"cpu-model" ); + + icc_bridge =DEVICE (object_resolve_path_type ("icc-bridge" , +TYPE_ICC_BRIDGE ,NULL )); +pc_new_cpu (cpu_model, apic_id, icc_bridge, errp); +} +void pc_cpus_init (const char *cpu_model,DeviceState *icc_bridge) {int i;
@@ -928,6 +956,7 @@ void pc_cpus_init(const char *cpu_model, DeviceState *icc_bridge)#else cpu_model ="qemu32" ;#endif +qemu_opts_set (qemu_find_opts ("machine" ),0 ,"cpu-model" , cpu_model); }for (i =0 ; i < smp_cpus; i++) {
hw/i386/pc_piix.c +1 -0
@@ -325,6 +325,7 @@ static QEMUMachine pc_i440fx_machine_v1_5 = { .alias = "pc", .desc = "Standard PC (i440FX + PIIX, 1996)", .init = pc_init_pci, + .hot_add_cpu = pc_hot_add_cpu, .max_cpus = 255, .is_default =1 , DEFAULT_MACHINE_OPTIONS,
hw/i386/pc_q35.c +1 -0
@@ -214,6 +214,7 @@ static QEMUMachine pc_q35_machine_v1_5 = { .alias = "q35", .desc = "Standard PC (Q35 + ICH9, 2009)", .init = pc_q35_init, + .hot_add_cpu = pc_hot_add_cpu, .max_cpus = 255, DEFAULT_MACHINE_OPTIONS, };
include/hw/i386/pc.h +1 -0
@@ -79,6 +79,7 @@ void pc_register_ferr_irq(qemu_irq irq);void pc_acpi_smi_interrupt (void *opaque,int irq,int level);void pc_cpus_init (const char *cpu_model,DeviceState *icc_bridge); +void pc_hot_add_cpu (const int64_t id,Error **errp);void pc_acpi_init (const char *default_dsdt); void *pc_memory_init(MemoryRegion *system_memory, const char *kernel_filename,
Signature · 1 line
1.8.2.1