xen: pvhvm: support bootup on secondary vCPUs
Subsystems (6)
v2 cover xen: pvhvm: support bootup on secondary vCPUs
It may happen that Xen's and Linux's ideas of vCPU id diverge. In particular, when we crash on a secondary vCPU we may want to do kdump and unlike plain kexec where we do migrate_to_reboot_cpu() we try booting on the vCPU which crashed. This doesn't work very well for PVHVM guests as we have a number of hypercalls where we pass vCPU id as a parameter. These hypercalls either fail or do something unexpected. To solve the issue we need to have a mapping between Linux's and Xen's vCPU ids.
This series solves the issue for x86 PVHVM guests. PV guests don't (and probably won't) support kdump so I always assume Xen's vCPU id == Linux's vCPU id. ARM guests will probably need to get proper mapping once we start supporting kexec/kdump there.
Changes since v1:
- "x86/acpi: store ACPI ids from MADT for future usage" patch added.
- Use ACPI ids instead of vLAPIC ids/2 [Andrew Cooper, Jan Beulich]
- Introduce xen_vcpu_nr() helper [David Vrabel].
- Modify all callers of HYPERVISOR_vcpu_op() instead of modifying HYPERVISOR_vcpu_op() [David Vrabel]
Vitaly Kuznetsov (9):
x86/xen: update cpuid.h from Xen-4.7
x86/acpi: store ACPI ids from MADT for future usage
xen: introduce xen_vcpu_id mapping
x86/xen: use xen_vcpu_id mapping for HYPERVISOR_vcpu_op x86/xen: use xen_vcpu_id mapping when pointing vcpu_info to the
shared_info page
xen/events: use xen_vcpu_id mapping in events_base xen/events: fifo: use xen_vcpu_id mapping
xen/evtchn: use xen_vcpu_id mapping
xen/pvhvm: run xen_vcpu_setup() for the boot CPU
arch/arm/xen/enlighten.c | 13 +++++++++++- arch/x86/include/asm/cpu.h | 1 + arch/x86/include/asm/smp.h | 2 ++ arch/x86/include/asm/xen/cpuid.h | 5 ++++- arch/x86/kernel/acpi/boot.c | 16 ++++++++++---- arch/x86/kernel/apic/apic.c | 2 ++ arch/x86/kernel/setup_percpu.c | 3 +++ arch/x86/xen/enlighten.c | 45 +++++++++++++++++++++++++++++++--------- arch/x86/xen/irq.c | 3 ++- arch/x86/xen/smp.c | 18 +++++++++++----- arch/x86/xen/time.c | 18 ++++++++++------ arch/x86/xen/xen-ops.h | 1 + drivers/xen/events/events_base.c | 13 ++++++------ drivers/xen/events/events_fifo.c | 2 +- drivers/xen/evtchn.c | 3 ++- drivers/xen/time.c | 2 +- include/xen/xen-ops.h | 6 ++++++ 17 files changed, 116 insertions(+), 37 deletions(-)
Signature · 1 line
2.5.5
PATCH v2 1/9 x86/xen: update cpuid.h from Xen-4.7
Update cpuid.h header from xen hypervisor tree to get XEN_HVM_CPUID_VCPU_ID_PRESENT definition.
--- arch/x86/include/asm/xen/cpuid.h | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
arch/x86/include/asm/xen/cpuid.h +4 -1
@@ -76,15 +76,18 @@/* * Leaf 5 (0x40000x04) * HVM-specific features +* EAX: Features +* EBX: vcpu id (iff EAX has XEN_HVM_CPUID_VCPU_ID_PRESENT flag) */ -/* EAX Features */ /* Virtualized APIC registers */ #define XEN_HVM_CPUID_APIC_ACCESS_VIRT (1u << 0)/* Virtualized x2APIC accesses */ #define XEN_HVM_CPUID_X2APIC_VIRT (1u << 1)/* Memory mapped from other domains has valid IOMMU entries */ #define XEN_HVM_CPUID_IOMMU_MAPPINGS (1u << 2) +/* vcpu id is present in EBX */ +#define XEN_HVM_CPUID_VCPU_ID_PRESENT (1u << 3)#define XEN_CPUID_MAX_NUM_LEAVES 4
Signature · 7 lines
2.5.5 _______________________________________________ Xen-devel mailing list [email protected] http://lists.xen.org/xen-devel
PATCH v2 1/9 x86/xen: update cpuid.h from Xen-4.7
In reply to Vitaly KuznetsovUpdate cpuid.h header from xen hypervisor tree to get XEN_HVM_CPUID_VCPU_ID_PRESENT definition.
--- arch/x86/include/asm/xen/cpuid.h | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
arch/x86/include/asm/xen/cpuid.h +4 -1
@@ -76,15 +76,18 @@/* * Leaf 5 (0x40000x04) * HVM-specific features +* EAX: Features +* EBX: vcpu id (iff EAX has XEN_HVM_CPUID_VCPU_ID_PRESENT flag) */ -/* EAX Features */ /* Virtualized APIC registers */ #define XEN_HVM_CPUID_APIC_ACCESS_VIRT (1u << 0)/* Virtualized x2APIC accesses */ #define XEN_HVM_CPUID_X2APIC_VIRT (1u << 1)/* Memory mapped from other domains has valid IOMMU entries */ #define XEN_HVM_CPUID_IOMMU_MAPPINGS (1u << 2) +/* vcpu id is present in EBX */ +#define XEN_HVM_CPUID_VCPU_ID_PRESENT (1u << 3)#define XEN_CPUID_MAX_NUM_LEAVES 4
Signature · 1 line
2.5.5
PATCH v2 2/9 x86/acpi: store ACPI ids from MADT for future usage
In reply to Vitaly KuznetsovCurrently we don't save ACPI ids (unlike LAPIC ids which go to x86_cpu_to_apicid) from MADT and we may need this information later. Particularly, ACPI ids is the only existent way for a PVHVM Xen guest to figure out Xen's idea of its vCPUs ids before these CPUs boot and in some cases these ids diverge from Linux's cpu ids.
--- arch/x86/include/asm/cpu.h | 1 + arch/x86/include/asm/smp.h | 2 ++ arch/x86/kernel/acpi/boot.c | 16 ++++++++++++---- arch/x86/kernel/apic/apic.c | 2 ++ arch/x86/kernel/setup_percpu.c | 3 +++ 5 files changed, 20 insertions(+), 4 deletions(-)
arch/x86/include/asm/cpu.h +1 -0
@@ -16,6 +16,7 @@ extern void prefill_possible_map(void);static inline void prefill_possible_map (void ) {}#define cpu_physical_id (cpu) boot_cpu_physical_apicid +#define cpu_acpi_id (cpu) 0#define safe_smp_processor_id () 0#define stack_smp_processor_id () 0
arch/x86/include/asm/smp.h +2 -0
@@ -33,6 +33,7 @@ static inline struct cpumask *cpu_llc_shared_mask(int cpu) }DECLARE_EARLY_PER_CPU_READ_MOSTLY (u16, x86_cpu_to_apicid); +DECLARE_EARLY_PER_CPU_READ_MOSTLY (u32, x86_cpu_to_acpiid);DECLARE_EARLY_PER_CPU_READ_MOSTLY (u16, x86_bios_cpu_apicid);#if defined(CONFIG_X86_LOCAL_APIC ) && defined(CONFIG_X86_32 )DECLARE_EARLY_PER_CPU_READ_MOSTLY (int, x86_cpu_to_logical_apicid);
@@ -147,6 +148,7 @@ void x86_idle_thread_init(unsigned int cpu, struct task_struct *idle);void smp_store_boot_cpu_info (void );void smp_store_cpu_info (int id);#define cpu_physical_id (cpu) per_cpu(x86_cpu_to_apicid, cpu) +#define cpu_acpi_id (cpu) per_cpu(x86_cpu_to_acpiid, cpu)#else /* !CONFIG_SMP */ #define wbinvd_on_cpu (cpu) wbinvd()
arch/x86/kernel/acpi/boot.c +12 -4
@@ -161,13 +161,15 @@ static int __init acpi_parse_madt(struct acpi_table_header *table)/** * acpi_register_lapic - register a local apic and generates a logic cpu number * @id: local apic id to register +* @acpiid: ACPI id to register * @enabled: this cpu is enabled or not * * Returns the logic cpu number which maps to the local apic */ -static int acpi_register_lapic (int id,u8 enabled) +static int acpi_register_lapic (int id,u32 acpiid,u8 enabled) {unsigned int ver =0 ; +int cpu;if (id >=MAX_LOCAL_APIC ) { printk(KERN_INFO PREFIX "skipped apicid that is too big\n");
@@ -182,7 +184,11 @@ static int acpi_register_lapic(int id, u8 enabled)if (boot_cpu_physical_apicid !=-1U ) ver = apic_version[boot_cpu_physical_apicid]; -return generic_processor_info (id, ver); + cpu =generic_processor_info (id, ver); +if (cpu >=0 ) +early_per_cpu (x86_cpu_to_acpiid, cpu) = acpiid; + +return cpu; } static int __init
@@ -212,7 +218,7 @@ acpi_parse_x2apic(struct acpi_subtable_header *header, const unsigned long end)if (!apic->apic_id_valid (apic_id) && enabled) printk(KERN_WARNING PREFIX "x2apic entry ignored\n");else -acpi_register_lapic (apic_id, enabled); +acpi_register_lapic (apic_id, processor->uid, enabled);#else printk(KERN_WARNING PREFIX "x2apic entry ignored\n");#endif
@@ -240,6 +246,7 @@ acpi_parse_lapic(struct acpi_subtable_header * header, const unsigned long end)* when we use CPU hotplug. */ acpi_register_lapic (processor->id,/* APIC ID */ + processor->processor_id,/* ACPI ID */ processor->lapic_flags &ACPI_MADT_ENABLED );return 0 ;
@@ -258,6 +265,7 @@ acpi_parse_sapic(struct acpi_subtable_header *header, const unsigned long end)acpi_table_print_madt_entry (header);acpi_register_lapic ((processor->id <<8 ) | processor->eid,/* APIC ID */ + processor->processor_id,/* ACPI ID */ processor->lapic_flags &ACPI_MADT_ENABLED );return 0 ;
@@ -714,7 +722,7 @@ int acpi_map_cpu(acpi_handle handle, phys_cpuid_t physid, int *pcpu) {int cpu; - cpu =acpi_register_lapic (physid,ACPI_MADT_ENABLED ); + cpu =acpi_register_lapic (physid,U32_MAX ,ACPI_MADT_ENABLED );if (cpu <0 ) {pr_info (PREFIX "Unable to map lapic to logical cpu number\n" );return cpu;
arch/x86/kernel/apic/apic.c +2 -0
@@ -92,8 +92,10 @@ static int apic_extnmi = APIC_EXTNMI_BSP;*/ DEFINE_EARLY_PER_CPU_READ_MOSTLY (u16, x86_cpu_to_apicid,BAD_APICID );DEFINE_EARLY_PER_CPU_READ_MOSTLY (u16, x86_bios_cpu_apicid,BAD_APICID ); +DEFINE_EARLY_PER_CPU_READ_MOSTLY (u32, x86_cpu_to_acpiid,U32_MAX );EXPORT_EARLY_PER_CPU_SYMBOL (x86_cpu_to_apicid);EXPORT_EARLY_PER_CPU_SYMBOL (x86_bios_cpu_apicid); +EXPORT_EARLY_PER_CPU_SYMBOL (x86_cpu_to_acpiid);#ifdef CONFIG_X86_32
arch/x86/kernel/setup_percpu.c +3 -0
@@ -236,6 +236,8 @@ void __init setup_per_cpu_areas(void)early_per_cpu_map (x86_cpu_to_apicid, cpu);per_cpu (x86_bios_cpu_apicid, cpu) =early_per_cpu_map (x86_bios_cpu_apicid, cpu); +per_cpu (x86_cpu_to_acpiid, cpu) = +early_per_cpu_map (x86_cpu_to_acpiid, cpu);#endif #ifdef CONFIG_X86_32 per_cpu(x86_cpu_to_logical_apicid, cpu) =
@@ -271,6 +273,7 @@ void __init setup_per_cpu_areas(void)#ifdef CONFIG_X86_LOCAL_APIC early_per_cpu_ptr (x86_cpu_to_apicid) =NULL ;early_per_cpu_ptr (x86_bios_cpu_apicid) =NULL ; +early_per_cpu_ptr (x86_cpu_to_acpiid) =NULL ;#endif #ifdef CONFIG_X86_32 early_per_cpu_ptr (x86_cpu_to_logical_apicid) =NULL ;
Signature · 1 line
2.5.5
PATCH v2 3/9 xen: introduce xen_vcpu_id mapping
In reply to Vitaly KuznetsovIt may happen that Xen's and Linux's ideas of vCPU id diverge. In particular, when we crash on a secondary vCPU we may want to do kdump and unlike plain kexec where we do migrate_to_reboot_cpu() we try booting on the vCPU which crashed. This doesn't work very well for PVHVM guests as we have a number of hypercalls where we pass vCPU id as a parameter. These hypercalls either fail or do something unexpected. To solve the issue introduce percpu xen_vcpu_id mapping. ARM and PV guests get direct mapping for now. Boot CPU for PVHVM guest gets its id from CPUID. With secondary CPUs it is a bit more trickier. Currently, we initialize IPI vectors before these CPUs boot so we can't use CPUID. Use ACPI ids from MADT instead.
---
Changes since v1:
- Introduce xen_vcpu_nr() helper [David Vrabel]
- Use ACPI ids instead of vLAPIC ids /2 [Andrew Cooper, Jan Beulich]
--- arch/arm/xen/enlighten.c | 10 ++++++++++ arch/x86/xen/enlighten.c | 23 ++++++++++++++++++++++- include/xen/xen-ops.h | 6 ++++++ 3 files changed, 38 insertions(+), 1 deletion(-)
arch/arm/xen/enlighten.c +10 -0
@@ -46,6 +46,10 @@ struct shared_info *HYPERVISOR_shared_info = (void *)&xen_dummy_shared_info; DEFINE_PER_CPU(struct vcpu_info *, xen_vcpu);static struct vcpu_info __percpu *xen_vcpu_info; +/* Linux <-> Xen vCPU id mapping */ +DEFINE_PER_CPU (int, xen_vcpu_id) =-1 ; +EXPORT_PER_CPU_SYMBOL (xen_vcpu_id); +/* These are unused until we support booting "pre-ballooned" */ unsigned long xen_released_pages;struct xen_memory_region xen_extra_mem[XEN_EXTRA_MEM_MAX_REGIONS ]__initdata ;
@@ -179,6 +183,9 @@ static void xen_percpu_init(void)pr_info ("Xen: initializing cpu%d\n" , cpu); vcpup =per_cpu_ptr (xen_vcpu_info, cpu); +/* Direct vCPU id mapping for ARM guests. */ +per_cpu (xen_vcpu_id, cpu) = cpu; + info.mfn =virt_to_gfn (vcpup); info.offset =xen_offset_in_page (vcpup);
@@ -328,6 +335,9 @@ static int __init xen_guest_init(void)if (xen_vcpu_info ==NULL )return -ENOMEM ; +/* Direct vCPU id mapping for ARM guests. */ +per_cpu (xen_vcpu_id,0 ) =0 ; +if (gnttab_setup_auto_xlat_frames (grant_frames)) {free_percpu (xen_vcpu_info);return -ENOMEM ;
arch/x86/xen/enlighten.c +22 -1
@@ -59,6 +59,7 @@#include <asm/xen/pci.h> #include <asm/xen/hypercall.h> #include <asm/xen/hypervisor.h> +#include <asm/xen/cpuid.h> #include <asm/fixmap.h> #include <asm/processor.h> #include <asm/proto.h>
@@ -118,6 +119,10 @@ DEFINE_PER_CPU(struct vcpu_info *, xen_vcpu);*/ DEFINE_PER_CPU(struct vcpu_info, xen_vcpu_info); +/* Linux <-> Xen vCPU id mapping */ +DEFINE_PER_CPU (int, xen_vcpu_id) =-1 ; +EXPORT_PER_CPU_SYMBOL (xen_vcpu_id); +enum xen_domain_type xen_domain_type =XEN_NATIVE ;EXPORT_SYMBOL_GPL (xen_domain_type);
@@ -1137,8 +1142,11 @@ void xen_setup_vcpu_info_placement(void) {int cpu; -for_each_possible_cpu (cpu) +for_each_possible_cpu (cpu) { +/* Set up direct vCPU id mapping for PV guests. */ +per_cpu (xen_vcpu_id, cpu) = cpu;xen_vcpu_setup (cpu); + }/* xen_vcpu_setup managed to place the vcpu_info within the * percpu area for all cpus, so make use of it. Note that for
@@ -1729,6 +1737,9 @@ asmlinkage __visible void __init xen_start_kernel(void)#endif xen_raw_console_write ("about to get started...\n" ); +/* Let's presume PV guests always boot on vCPU with id 0. */ +per_cpu (xen_vcpu_id,0 ) =0 ; +xen_setup_runstate_info (0 );xen_efi_init ();
@@ -1797,6 +1808,12 @@ static void __init init_hvm_pv_info(void)xen_setup_features (); +cpuid (base +4 , &eax, &ebx, &ecx, &edx); +if (eax &XEN_HVM_CPUID_VCPU_ID_PRESENT ) +this_cpu_write (xen_vcpu_id, ebx); +else +this_cpu_write (xen_vcpu_id,smp_processor_id ()); + pv_info.name ="Xen HVM" ; xen_domain_type =XEN_HVM_DOMAIN ;
@@ -1808,6 +1825,10 @@ static int xen_hvm_cpu_notify(struct notifier_block *self, unsigned long action,int cpu = (long )hcpu;switch (action) {case CPU_UP_PREPARE : +if (cpu_acpi_id (cpu) !=U32_MAX ) +per_cpu (xen_vcpu_id, cpu) =cpu_acpi_id (cpu); +else +per_cpu (xen_vcpu_id, cpu) = cpu;xen_vcpu_setup (cpu);if (xen_have_vector_callback) {if (xen_feature (XENFEAT_hvm_safe_pvclock))
include/xen/xen-ops.h +6 -0
@@ -9,6 +9,12 @@ DECLARE_PER_CPU(struct vcpu_info *, xen_vcpu); +DECLARE_PER_CPU (int, xen_vcpu_id); +static inline int xen_vcpu_nr (int cpu) +{ +return per_cpu (xen_vcpu_id, cpu); +} +void xen_arch_pre_suspend (void );void xen_arch_post_suspend (int suspend_cancelled);
Signature · 7 lines
2.5.5 _______________________________________________ Xen-devel mailing list [email protected] http://lists.xen.org/xen-devel
PATCH v2 4/9 x86/xen: use xen_vcpu_id mapping for HYPERVISOR_vcpu_op
In reply to Vitaly KuznetsovHYPERVISOR_vcpu_op() passes Linux's idea of vCPU id as a parameter while Xen's idea is expected. In some cases these ideas diverge so we need to do remapping. Convert all callers of HYPERVISOR_vcpu_op() to use xen_vcpu_nr(). Leave xen_fill_possible_map() and xen_filter_cpu_maps() intact as they're only being called by PV guests before perpu areas are initialized. While the issue could be solved by switching to early_percpu for xen_vcpu_id I think it's not worth it: PV guests will probably never get to the point where their idea of vCPU id diverges from Xen's.
---
Changes since v1:
- Modify all callers of HYPERVISOR_vcpu_op() instead of modifying HYPERVISOR_vcpu_op() [David Vrabel]
--- arch/arm/xen/enlighten.c | 3 ++- arch/x86/xen/enlighten.c | 10 ++++++---- arch/x86/xen/irq.c | 3 ++- arch/x86/xen/smp.c | 11 ++++++----- arch/x86/xen/time.c | 18 ++++++++++++------ drivers/xen/events/events_base.c | 3 ++- drivers/xen/time.c | 2 +- 7 files changed, 31 insertions(+), 19 deletions(-)
arch/arm/xen/enlighten.c +2 -1
@@ -189,7 +189,8 @@ static void xen_percpu_init(void) info.mfn =virt_to_gfn (vcpup); info.offset =xen_offset_in_page (vcpup); - err =HYPERVISOR_vcpu_op (VCPUOP_register_vcpu_info, cpu, &info); + err =HYPERVISOR_vcpu_op (VCPUOP_register_vcpu_info,xen_vcpu_nr (cpu), + &info);BUG_ON (err);per_cpu (xen_vcpu, cpu) = vcpup;
arch/x86/xen/enlighten.c +6 -4
@@ -228,7 +228,8 @@ static void xen_vcpu_setup(int cpu)hypervisor has no unregister variant and this hypercall does not allow to over-write info.mfn and info.offset. */ - err =HYPERVISOR_vcpu_op (VCPUOP_register_vcpu_info, cpu, &info); + err =HYPERVISOR_vcpu_op (VCPUOP_register_vcpu_info,xen_vcpu_nr (cpu), + &info);if (err) {printk (KERN_DEBUG "register_vcpu_info failed: err=%d\n" , err);
@@ -252,10 +253,11 @@ void xen_vcpu_restore(void)for_each_possible_cpu (cpu) {bool other_cpu = (cpu !=smp_processor_id ()); -bool is_up =HYPERVISOR_vcpu_op (VCPUOP_is_up, cpu,NULL ); +bool is_up =HYPERVISOR_vcpu_op (VCPUOP_is_up,xen_vcpu_nr (cpu), +NULL );if (other_cpu && is_up && -HYPERVISOR_vcpu_op (VCPUOP_down, cpu,NULL )) +HYPERVISOR_vcpu_op (VCPUOP_down,xen_vcpu_nr (cpu),NULL ))BUG ();xen_setup_runstate_info (cpu);
@@ -264,7 +266,7 @@ void xen_vcpu_restore(void)xen_vcpu_setup (cpu);if (other_cpu && is_up && -HYPERVISOR_vcpu_op (VCPUOP_up, cpu,NULL )) +HYPERVISOR_vcpu_op (VCPUOP_up,xen_vcpu_nr (cpu),NULL ))BUG (); } }
arch/x86/xen/irq.c +2 -1
@@ -109,7 +109,8 @@ static void xen_safe_halt(void)static void xen_halt (void ) {if (irqs_disabled ()) -HYPERVISOR_vcpu_op (VCPUOP_down,smp_processor_id (),NULL ); +HYPERVISOR_vcpu_op (VCPUOP_down, +xen_vcpu_nr (smp_processor_id ()),NULL );else xen_safe_halt (); }
arch/x86/xen/smp.c +6 -5
@@ -454,7 +454,7 @@ cpu_initialize_context(unsigned int cpu, struct task_struct *idle)#endif ctxt->user_regs.esp = idle->thread.sp0 -sizeof (struct pt_regs ); ctxt->ctrlreg[3 ] =xen_pfn_to_cr3 (virt_to_gfn (swapper_pg_dir)); -if (HYPERVISOR_vcpu_op (VCPUOP_initialise, cpu, ctxt)) +if (HYPERVISOR_vcpu_op (VCPUOP_initialise,xen_vcpu_nr (cpu), ctxt))BUG ();kfree (ctxt);
@@ -492,7 +492,7 @@ static int xen_cpu_up(unsigned int cpu, struct task_struct *idle)if (rc)return rc; - rc =HYPERVISOR_vcpu_op (VCPUOP_up, cpu,NULL ); + rc =HYPERVISOR_vcpu_op (VCPUOP_up,xen_vcpu_nr (cpu),NULL );BUG_ON (rc);while (cpu_report_state (cpu) !=CPU_ONLINE )
@@ -520,7 +520,8 @@ static int xen_cpu_disable(void)static void xen_cpu_die (unsigned int cpu) { -while (xen_pv_domain () &&HYPERVISOR_vcpu_op (VCPUOP_is_up, cpu,NULL )) { +while (xen_pv_domain () &&HYPERVISOR_vcpu_op (VCPUOP_is_up, +xen_vcpu_nr (cpu),NULL )) {__set_current_state (TASK_UNINTERRUPTIBLE );schedule_timeout (HZ /10 ); }
@@ -536,7 +537,7 @@ static void xen_cpu_die(unsigned int cpu)static void xen_play_dead (void )/* used only with HOTPLUG_CPU */ {play_dead_common (); -HYPERVISOR_vcpu_op (VCPUOP_down,smp_processor_id (),NULL ); +HYPERVISOR_vcpu_op (VCPUOP_down,xen_vcpu_nr (smp_processor_id ()),NULL );cpu_bringup ();/* * commit 4b0c0f294 (tick: Cleanup NOHZ per cpu data on cpu down)
@@ -576,7 +577,7 @@ static void stop_self(void *v)set_cpu_online (cpu, false); -HYPERVISOR_vcpu_op (VCPUOP_down, cpu,NULL ); +HYPERVISOR_vcpu_op (VCPUOP_down,xen_vcpu_nr (cpu),NULL );BUG (); }
arch/x86/xen/time.c +12 -6
@@ -263,8 +263,10 @@ static int xen_vcpuop_shutdown(struct clock_event_device *evt) {int cpu =smp_processor_id (); -if (HYPERVISOR_vcpu_op (VCPUOP_stop_singleshot_timer, cpu,NULL ) || -HYPERVISOR_vcpu_op (VCPUOP_stop_periodic_timer, cpu,NULL )) +if (HYPERVISOR_vcpu_op (VCPUOP_stop_singleshot_timer,xen_vcpu_nr (cpu), +NULL ) || +HYPERVISOR_vcpu_op (VCPUOP_stop_periodic_timer,xen_vcpu_nr (cpu), +NULL ))BUG ();return 0 ;
@@ -274,7 +276,8 @@ static int xen_vcpuop_set_oneshot(struct clock_event_device *evt) {int cpu =smp_processor_id (); -if (HYPERVISOR_vcpu_op (VCPUOP_stop_periodic_timer, cpu,NULL )) +if (HYPERVISOR_vcpu_op (VCPUOP_stop_periodic_timer,xen_vcpu_nr (cpu), +NULL ))BUG ();return 0 ;
@@ -293,7 +296,8 @@ static int xen_vcpuop_set_next_event(unsigned long delta,/* Get an event anyway, even if the timeout is already expired */ single.flags =0 ; - ret =HYPERVISOR_vcpu_op (VCPUOP_set_singleshot_timer, cpu, &single); + ret =HYPERVISOR_vcpu_op (VCPUOP_set_singleshot_timer,xen_vcpu_nr (cpu), + &single);BUG_ON (ret !=0 );return ret;
@@ -394,7 +398,8 @@ void xen_timer_resume(void)return ;for_each_online_cpu (cpu) { -if (HYPERVISOR_vcpu_op (VCPUOP_stop_periodic_timer, cpu,NULL )) +if (HYPERVISOR_vcpu_op (VCPUOP_stop_periodic_timer, +xen_vcpu_nr (cpu),NULL ))BUG (); } }
@@ -414,7 +419,8 @@ static void __init xen_time_init(void)clocksource_register_hz (&xen_clocksource,NSEC_PER_SEC ); -if (HYPERVISOR_vcpu_op (VCPUOP_stop_periodic_timer, cpu,NULL ) ==0 ) { +if (HYPERVISOR_vcpu_op (VCPUOP_stop_periodic_timer,xen_vcpu_nr (cpu), +NULL ) ==0 ) {/* Successfully turned off 100Hz tick, so we have the vcpuop-based timer interface */ printk (KERN_DEBUG "Xen: using vcpuop timer interface\n" );
drivers/xen/events/events_base.c +2 -1
@@ -1211,7 +1211,8 @@ void xen_send_IPI_one(unsigned int cpu, enum ipi_vector vector)#ifdef CONFIG_X86 if (unlikely (vector ==XEN_NMI_VECTOR )) { -int rc =HYPERVISOR_vcpu_op (VCPUOP_send_nmi, cpu,NULL ); +int rc =HYPERVISOR_vcpu_op (VCPUOP_send_nmi,xen_vcpu_nr (cpu), +NULL );if (rc <0 )printk (KERN_WARNING "Sending nmi to CPU%d failed (rc:%d)\n" , cpu, rc);return ;
drivers/xen/time.c +1 -1
@@ -82,7 +82,7 @@ void xen_setup_runstate_info(int cpu) area.addr.v = &per_cpu (xen_runstate, cpu);if (HYPERVISOR_vcpu_op (VCPUOP_register_runstate_memory_area, - cpu, &area)) +xen_vcpu_nr (cpu), &area))BUG (); }
Signature · 7 lines
2.5.5 _______________________________________________ Xen-devel mailing list [email protected] http://lists.xen.org/xen-devel
PATCH v2 4/9 x86/xen: use xen_vcpu_id mapping for HYPERVISOR_vcpu_op
In reply to Vitaly KuznetsovHYPERVISOR_vcpu_op() passes Linux's idea of vCPU id as a parameter while Xen's idea is expected. In some cases these ideas diverge so we need to do remapping. Convert all callers of HYPERVISOR_vcpu_op() to use xen_vcpu_nr(). Leave xen_fill_possible_map() and xen_filter_cpu_maps() intact as they're only being called by PV guests before perpu areas are initialized. While the issue could be solved by switching to early_percpu for xen_vcpu_id I think it's not worth it: PV guests will probably never get to the point where their idea of vCPU id diverges from Xen's.
---
Changes since v1:
- Modify all callers of HYPERVISOR_vcpu_op() instead of modifying HYPERVISOR_vcpu_op() [David Vrabel]
--- arch/arm/xen/enlighten.c | 3 ++- arch/x86/xen/enlighten.c | 10 ++++++---- arch/x86/xen/irq.c | 3 ++- arch/x86/xen/smp.c | 11 ++++++----- arch/x86/xen/time.c | 18 ++++++++++++------ drivers/xen/events/events_base.c | 3 ++- drivers/xen/time.c | 2 +- 7 files changed, 31 insertions(+), 19 deletions(-)
arch/arm/xen/enlighten.c +2 -1
@@ -189,7 +189,8 @@ static void xen_percpu_init(void) info.mfn =virt_to_gfn (vcpup); info.offset =xen_offset_in_page (vcpup); - err =HYPERVISOR_vcpu_op (VCPUOP_register_vcpu_info, cpu, &info); + err =HYPERVISOR_vcpu_op (VCPUOP_register_vcpu_info,xen_vcpu_nr (cpu), + &info);BUG_ON (err);per_cpu (xen_vcpu, cpu) = vcpup;
arch/x86/xen/enlighten.c +6 -4
@@ -228,7 +228,8 @@ static void xen_vcpu_setup(int cpu)hypervisor has no unregister variant and this hypercall does not allow to over-write info.mfn and info.offset. */ - err =HYPERVISOR_vcpu_op (VCPUOP_register_vcpu_info, cpu, &info); + err =HYPERVISOR_vcpu_op (VCPUOP_register_vcpu_info,xen_vcpu_nr (cpu), + &info);if (err) {printk (KERN_DEBUG "register_vcpu_info failed: err=%d\n" , err);
@@ -252,10 +253,11 @@ void xen_vcpu_restore(void)for_each_possible_cpu (cpu) {bool other_cpu = (cpu !=smp_processor_id ()); -bool is_up =HYPERVISOR_vcpu_op (VCPUOP_is_up, cpu,NULL ); +bool is_up =HYPERVISOR_vcpu_op (VCPUOP_is_up,xen_vcpu_nr (cpu), +NULL );if (other_cpu && is_up && -HYPERVISOR_vcpu_op (VCPUOP_down, cpu,NULL )) +HYPERVISOR_vcpu_op (VCPUOP_down,xen_vcpu_nr (cpu),NULL ))BUG ();xen_setup_runstate_info (cpu);
@@ -264,7 +266,7 @@ void xen_vcpu_restore(void)xen_vcpu_setup (cpu);if (other_cpu && is_up && -HYPERVISOR_vcpu_op (VCPUOP_up, cpu,NULL )) +HYPERVISOR_vcpu_op (VCPUOP_up,xen_vcpu_nr (cpu),NULL ))BUG (); } }
arch/x86/xen/irq.c +2 -1
@@ -109,7 +109,8 @@ static void xen_safe_halt(void)static void xen_halt (void ) {if (irqs_disabled ()) -HYPERVISOR_vcpu_op (VCPUOP_down,smp_processor_id (),NULL ); +HYPERVISOR_vcpu_op (VCPUOP_down, +xen_vcpu_nr (smp_processor_id ()),NULL );else xen_safe_halt (); }
arch/x86/xen/smp.c +6 -5
@@ -454,7 +454,7 @@ cpu_initialize_context(unsigned int cpu, struct task_struct *idle)#endif ctxt->user_regs.esp = idle->thread.sp0 -sizeof (struct pt_regs ); ctxt->ctrlreg[3 ] =xen_pfn_to_cr3 (virt_to_gfn (swapper_pg_dir)); -if (HYPERVISOR_vcpu_op (VCPUOP_initialise, cpu, ctxt)) +if (HYPERVISOR_vcpu_op (VCPUOP_initialise,xen_vcpu_nr (cpu), ctxt))BUG ();kfree (ctxt);
@@ -492,7 +492,7 @@ static int xen_cpu_up(unsigned int cpu, struct task_struct *idle)if (rc)return rc; - rc =HYPERVISOR_vcpu_op (VCPUOP_up, cpu,NULL ); + rc =HYPERVISOR_vcpu_op (VCPUOP_up,xen_vcpu_nr (cpu),NULL );BUG_ON (rc);while (cpu_report_state (cpu) !=CPU_ONLINE )
@@ -520,7 +520,8 @@ static int xen_cpu_disable(void)static void xen_cpu_die (unsigned int cpu) { -while (xen_pv_domain () &&HYPERVISOR_vcpu_op (VCPUOP_is_up, cpu,NULL )) { +while (xen_pv_domain () &&HYPERVISOR_vcpu_op (VCPUOP_is_up, +xen_vcpu_nr (cpu),NULL )) {__set_current_state (TASK_UNINTERRUPTIBLE );schedule_timeout (HZ /10 ); }
@@ -536,7 +537,7 @@ static void xen_cpu_die(unsigned int cpu)static void xen_play_dead (void )/* used only with HOTPLUG_CPU */ {play_dead_common (); -HYPERVISOR_vcpu_op (VCPUOP_down,smp_processor_id (),NULL ); +HYPERVISOR_vcpu_op (VCPUOP_down,xen_vcpu_nr (smp_processor_id ()),NULL );cpu_bringup ();/* * commit 4b0c0f294 (tick: Cleanup NOHZ per cpu data on cpu down)
@@ -576,7 +577,7 @@ static void stop_self(void *v)set_cpu_online (cpu, false); -HYPERVISOR_vcpu_op (VCPUOP_down, cpu,NULL ); +HYPERVISOR_vcpu_op (VCPUOP_down,xen_vcpu_nr (cpu),NULL );BUG (); }
arch/x86/xen/time.c +12 -6
@@ -263,8 +263,10 @@ static int xen_vcpuop_shutdown(struct clock_event_device *evt) {int cpu =smp_processor_id (); -if (HYPERVISOR_vcpu_op (VCPUOP_stop_singleshot_timer, cpu,NULL ) || -HYPERVISOR_vcpu_op (VCPUOP_stop_periodic_timer, cpu,NULL )) +if (HYPERVISOR_vcpu_op (VCPUOP_stop_singleshot_timer,xen_vcpu_nr (cpu), +NULL ) || +HYPERVISOR_vcpu_op (VCPUOP_stop_periodic_timer,xen_vcpu_nr (cpu), +NULL ))BUG ();return 0 ;
@@ -274,7 +276,8 @@ static int xen_vcpuop_set_oneshot(struct clock_event_device *evt) {int cpu =smp_processor_id (); -if (HYPERVISOR_vcpu_op (VCPUOP_stop_periodic_timer, cpu,NULL )) +if (HYPERVISOR_vcpu_op (VCPUOP_stop_periodic_timer,xen_vcpu_nr (cpu), +NULL ))BUG ();return 0 ;
@@ -293,7 +296,8 @@ static int xen_vcpuop_set_next_event(unsigned long delta,/* Get an event anyway, even if the timeout is already expired */ single.flags =0 ; - ret =HYPERVISOR_vcpu_op (VCPUOP_set_singleshot_timer, cpu, &single); + ret =HYPERVISOR_vcpu_op (VCPUOP_set_singleshot_timer,xen_vcpu_nr (cpu), + &single);BUG_ON (ret !=0 );return ret;
@@ -394,7 +398,8 @@ void xen_timer_resume(void)return ;for_each_online_cpu (cpu) { -if (HYPERVISOR_vcpu_op (VCPUOP_stop_periodic_timer, cpu,NULL )) +if (HYPERVISOR_vcpu_op (VCPUOP_stop_periodic_timer, +xen_vcpu_nr (cpu),NULL ))BUG (); } }
@@ -414,7 +419,8 @@ static void __init xen_time_init(void)clocksource_register_hz (&xen_clocksource,NSEC_PER_SEC ); -if (HYPERVISOR_vcpu_op (VCPUOP_stop_periodic_timer, cpu,NULL ) ==0 ) { +if (HYPERVISOR_vcpu_op (VCPUOP_stop_periodic_timer,xen_vcpu_nr (cpu), +NULL ) ==0 ) {/* Successfully turned off 100Hz tick, so we have the vcpuop-based timer interface */ printk (KERN_DEBUG "Xen: using vcpuop timer interface\n" );
drivers/xen/events/events_base.c +2 -1
@@ -1211,7 +1211,8 @@ void xen_send_IPI_one(unsigned int cpu, enum ipi_vector vector)#ifdef CONFIG_X86 if (unlikely (vector ==XEN_NMI_VECTOR )) { -int rc =HYPERVISOR_vcpu_op (VCPUOP_send_nmi, cpu,NULL ); +int rc =HYPERVISOR_vcpu_op (VCPUOP_send_nmi,xen_vcpu_nr (cpu), +NULL );if (rc <0 )printk (KERN_WARNING "Sending nmi to CPU%d failed (rc:%d)\n" , cpu, rc);return ;
drivers/xen/time.c +1 -1
@@ -82,7 +82,7 @@ void xen_setup_runstate_info(int cpu) area.addr.v = &per_cpu (xen_runstate, cpu);if (HYPERVISOR_vcpu_op (VCPUOP_register_runstate_memory_area, - cpu, &area)) +xen_vcpu_nr (cpu), &area))BUG (); }
Signature · 1 line
2.5.5
PATCH v2 5/9 x86/xen: use xen_vcpu_id mapping when pointing vcpu_info to the shared_info page
In reply to Vitaly Kuznetsovshared_info page has space for 32 vcpu info slots for first 32 vCPUs but these are the first 32 vCPUs from Xen's perspective and we should map them accordingly with the newly introduced xen_vcpu_id mapping.
---
Changes since v1:
- Use xen_vcpu_nr() helper [David Vrabel]
--- arch/x86/xen/enlighten.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
arch/x86/xen/enlighten.c +6 -4
@@ -207,8 +207,9 @@ static void xen_vcpu_setup(int cpu)if (per_cpu (xen_vcpu, cpu) == &per_cpu (xen_vcpu_info, cpu))return ; } -if (cpu <MAX_VIRT_CPUS ) -per_cpu (xen_vcpu,cpu) = &HYPERVISOR_shared_info->vcpu_info[cpu]; +if (xen_vcpu_nr (cpu) <MAX_VIRT_CPUS ) +per_cpu (xen_vcpu, cpu) = + &HYPERVISOR_shared_info->vcpu_info[xen_vcpu_nr (cpu)];if (!have_vcpu_info_placement) {if (cpu >=MAX_VIRT_CPUS )
@@ -1783,9 +1784,10 @@ void __ref xen_hvm_init_shared_info(void)* in that case multiple vcpus might be online. */ for_each_online_cpu (cpu) {/* Leave it to be NULL. */ -if (cpu >=MAX_VIRT_CPUS ) +if (xen_vcpu_nr (cpu) >=MAX_VIRT_CPUS )continue ; -per_cpu (xen_vcpu, cpu) = &HYPERVISOR_shared_info->vcpu_info[cpu]; +per_cpu (xen_vcpu, cpu) = + &HYPERVISOR_shared_info->vcpu_info[xen_vcpu_nr (cpu)]; } }
Signature · 7 lines
2.5.5 _______________________________________________ Xen-devel mailing list [email protected] http://lists.xen.org/xen-devel
PATCH v2 6/9 xen/events: use xen_vcpu_id mapping in events_base
In reply to Vitaly KuznetsovEVTCHNOP_bind_ipi and EVTCHNOP_bind_virq pass vCPU id as a parameter and Xen's idea of vCPU id should be used. Use the newly introduced xen_vcpu_id mapping to convert it from Linux's id.
---
Changes since v1:
- Use xen_vcpu_nr() helper [David Vrabel]
--- drivers/xen/events/events_base.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
drivers/xen/events/events_base.c +5 -5
@@ -895,7 +895,7 @@ static int bind_ipi_to_irq(unsigned int ipi, unsigned int cpu)irq_set_chip_and_handler_name (irq, &xen_percpu_chip, handle_percpu_irq,"ipi" ); - bind_ipi.vcpu = cpu; + bind_ipi.vcpu =xen_vcpu_nr (cpu);if (HYPERVISOR_event_channel_op (EVTCHNOP_bind_ipi, &bind_ipi) !=0 )BUG ();
@@ -991,7 +991,7 @@ int bind_virq_to_irq(unsigned int virq, unsigned int cpu, bool percpu) handle_edge_irq, "virq"); bind_virq.virq = virq; - bind_virq.vcpu = cpu; + bind_virq.vcpu =xen_vcpu_nr (cpu); ret =HYPERVISOR_event_channel_op (EVTCHNOP_bind_virq, &bind_virq);if (ret ==0 )
@@ -1319,7 +1319,7 @@ static int rebind_irq_to_cpu(unsigned irq, unsigned tcpu)/* Send future instances of this interrupt to other vcpu. */ bind_vcpu.port = evtchn; - bind_vcpu.vcpu = tcpu; + bind_vcpu.vcpu =xen_vcpu_nr (tcpu);/* * Mask the event while changing the VCPU binding to prevent
@@ -1459,7 +1459,7 @@ static void restore_cpu_virqs(unsigned int cpu)/* Get a new binding from Xen. */ bind_virq.virq = virq; - bind_virq.vcpu = cpu; + bind_virq.vcpu =xen_vcpu_nr (cpu);if (HYPERVISOR_event_channel_op (EVTCHNOP_bind_virq, &bind_virq) !=0 )BUG ();
@@ -1483,7 +1483,7 @@ static void restore_cpu_ipis(unsigned int cpu)BUG_ON (ipi_from_irq (irq) != ipi);/* Get a new binding from Xen. */ - bind_ipi.vcpu = cpu; + bind_ipi.vcpu =xen_vcpu_nr (cpu);if (HYPERVISOR_event_channel_op (EVTCHNOP_bind_ipi, &bind_ipi) !=0 )BUG ();
Signature · 1 line
2.5.5
PATCH v2 6/9 xen/events: use xen_vcpu_id mapping in events_base
In reply to Vitaly KuznetsovEVTCHNOP_bind_ipi and EVTCHNOP_bind_virq pass vCPU id as a parameter and Xen's idea of vCPU id should be used. Use the newly introduced xen_vcpu_id mapping to convert it from Linux's id.
---
Changes since v1:
- Use xen_vcpu_nr() helper [David Vrabel]
--- drivers/xen/events/events_base.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
drivers/xen/events/events_base.c +5 -5
@@ -895,7 +895,7 @@ static int bind_ipi_to_irq(unsigned int ipi, unsigned int cpu)irq_set_chip_and_handler_name (irq, &xen_percpu_chip, handle_percpu_irq,"ipi" ); - bind_ipi.vcpu = cpu; + bind_ipi.vcpu =xen_vcpu_nr (cpu);if (HYPERVISOR_event_channel_op (EVTCHNOP_bind_ipi, &bind_ipi) !=0 )BUG ();
@@ -991,7 +991,7 @@ int bind_virq_to_irq(unsigned int virq, unsigned int cpu, bool percpu) handle_edge_irq, "virq"); bind_virq.virq = virq; - bind_virq.vcpu = cpu; + bind_virq.vcpu =xen_vcpu_nr (cpu); ret =HYPERVISOR_event_channel_op (EVTCHNOP_bind_virq, &bind_virq);if (ret ==0 )
@@ -1319,7 +1319,7 @@ static int rebind_irq_to_cpu(unsigned irq, unsigned tcpu)/* Send future instances of this interrupt to other vcpu. */ bind_vcpu.port = evtchn; - bind_vcpu.vcpu = tcpu; + bind_vcpu.vcpu =xen_vcpu_nr (tcpu);/* * Mask the event while changing the VCPU binding to prevent
@@ -1459,7 +1459,7 @@ static void restore_cpu_virqs(unsigned int cpu)/* Get a new binding from Xen. */ bind_virq.virq = virq; - bind_virq.vcpu = cpu; + bind_virq.vcpu =xen_vcpu_nr (cpu);if (HYPERVISOR_event_channel_op (EVTCHNOP_bind_virq, &bind_virq) !=0 )BUG ();
@@ -1483,7 +1483,7 @@ static void restore_cpu_ipis(unsigned int cpu)BUG_ON (ipi_from_irq (irq) != ipi);/* Get a new binding from Xen. */ - bind_ipi.vcpu = cpu; + bind_ipi.vcpu =xen_vcpu_nr (cpu);if (HYPERVISOR_event_channel_op (EVTCHNOP_bind_ipi, &bind_ipi) !=0 )BUG ();
Signature · 7 lines
2.5.5 _______________________________________________ Xen-devel mailing list [email protected] http://lists.xen.org/xen-devel
PATCH v2 7/9 xen/events: fifo: use xen_vcpu_id mapping
In reply to Vitaly KuznetsovEVTCHNOP_init_control has vCPU id as a parameter and Xen's idea of vCPU id should be used. Use the newly introduced xen_vcpu_id mapping to convert it from Linux's id.
---
Changes since v1:
- Use xen_vcpu_nr() helper [David Vrabel]
--- drivers/xen/events/events_fifo.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/xen/events/events_fifo.c +1 -1
@@ -113,7 +113,7 @@ static int init_control_block(int cpu, init_control.control_gfn =virt_to_gfn (control_block); init_control.offset =0 ; - init_control.vcpu = cpu; + init_control.vcpu =xen_vcpu_nr (cpu);return HYPERVISOR_event_channel_op (EVTCHNOP_init_control, &init_control); }
Signature · 7 lines
2.5.5 _______________________________________________ Xen-devel mailing list [email protected] http://lists.xen.org/xen-devel
PATCH v2 8/9 xen/evtchn: use xen_vcpu_id mapping
In reply to Vitaly KuznetsovUse the newly introduced xen_vcpu_id mapping to get Xen's idea of vCPU id for CPU0.
---
Changes since v1:
- Use xen_vcpu_nr() helper [David Vrabel]
--- drivers/xen/evtchn.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/xen/evtchn.c +2 -1
@@ -55,6 +55,7 @@#include <xen/xen.h> #include <xen/events.h> #include <xen/evtchn.h> +#include <xen/xen-ops.h> #include <asm/xen/hypervisor.h> struct per_user_data {
@@ -448,7 +449,7 @@ static long evtchn_ioctl(struct file *file,break ; bind_virq.virq = bind.virq; - bind_virq.vcpu =0 ; + bind_virq.vcpu =xen_vcpu_nr (0 ); rc =HYPERVISOR_event_channel_op (EVTCHNOP_bind_virq, &bind_virq);if (rc !=0 )
Signature · 7 lines
2.5.5 _______________________________________________ Xen-devel mailing list [email protected] http://lists.xen.org/xen-devel
PATCH v2 8/9 xen/evtchn: use xen_vcpu_id mapping
In reply to Vitaly KuznetsovUse the newly introduced xen_vcpu_id mapping to get Xen's idea of vCPU id for CPU0.
---
Changes since v1:
- Use xen_vcpu_nr() helper [David Vrabel]
--- drivers/xen/evtchn.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/xen/evtchn.c +2 -1
@@ -55,6 +55,7 @@#include <xen/xen.h> #include <xen/events.h> #include <xen/evtchn.h> +#include <xen/xen-ops.h> #include <asm/xen/hypervisor.h> struct per_user_data {
@@ -448,7 +449,7 @@ static long evtchn_ioctl(struct file *file,break ; bind_virq.virq = bind.virq; - bind_virq.vcpu =0 ; + bind_virq.vcpu =xen_vcpu_nr (0 ); rc =HYPERVISOR_event_channel_op (EVTCHNOP_bind_virq, &bind_virq);if (rc !=0 )
Signature · 1 line
2.5.5
PATCH v2 9/9 xen/pvhvm: run xen_vcpu_setup() for the boot CPU
In reply to Vitaly KuznetsovHistorically we didn't call VCPUOP_register_vcpu_info for CPU0 for PVHVM guests (while we had it for PV and ARM guests). This is usually fine as we can use vcpu info in the sared_info page but when we try booting on a vCPU with Xen's vCPU id > 31 (e.g. when we try to kdump after crashing on this CPU) we're not able to boot. Switch to always doing VCPUOP_register_vcpu_info for the boot CPU.
--- arch/x86/xen/enlighten.c | 2 +- arch/x86/xen/smp.c | 7 +++++++ arch/x86/xen/xen-ops.h | 1 + 3 files changed, 9 insertions(+), 1 deletion(-)
arch/x86/xen/enlighten.c +1 -1
@@ -184,7 +184,7 @@ static void clamp_max_cpus(void)#endif } -static void xen_vcpu_setup (int cpu) +void xen_vcpu_setup (int cpu) {struct vcpu_register_vcpu_info info;int err;
arch/x86/xen/smp.c +7 -0
@@ -322,6 +322,13 @@ static void __init xen_smp_prepare_boot_cpu(void)xen_filter_cpu_maps ();xen_setup_vcpu_info_placement (); } + +/* +* Setup vcpu_info for boot CPU. +*/ +if (xen_hvm_domain ()) +xen_vcpu_setup (0 ); +/* * The alternative logic (which patches the unlock/lock) runs before * the smp bootup up code is activated. Hence we need to set this up
arch/x86/xen/xen-ops.h +1 -0
@@ -76,6 +76,7 @@ irqreturn_t xen_debug_interrupt(int irq, void *dev_id);bool xen_vcpu_stolen (int vcpu); +void xen_vcpu_setup (int cpu);void xen_setup_vcpu_info_placement (void );#ifdef CONFIG_SMP
Signature · 7 lines
2.5.5 _______________________________________________ Xen-devel mailing list [email protected] http://lists.xen.org/xen-devel
PATCH v2 9/9 xen/pvhvm: run xen_vcpu_setup() for the boot CPU
In reply to Vitaly KuznetsovHistorically we didn't call VCPUOP_register_vcpu_info for CPU0 for PVHVM guests (while we had it for PV and ARM guests). This is usually fine as we can use vcpu info in the sared_info page but when we try booting on a vCPU with Xen's vCPU id > 31 (e.g. when we try to kdump after crashing on this CPU) we're not able to boot. Switch to always doing VCPUOP_register_vcpu_info for the boot CPU.
--- arch/x86/xen/enlighten.c | 2 +- arch/x86/xen/smp.c | 7 +++++++ arch/x86/xen/xen-ops.h | 1 + 3 files changed, 9 insertions(+), 1 deletion(-)
arch/x86/xen/enlighten.c +1 -1
@@ -184,7 +184,7 @@ static void clamp_max_cpus(void)#endif } -static void xen_vcpu_setup (int cpu) +void xen_vcpu_setup (int cpu) {struct vcpu_register_vcpu_info info;int err;
arch/x86/xen/smp.c +7 -0
@@ -322,6 +322,13 @@ static void __init xen_smp_prepare_boot_cpu(void)xen_filter_cpu_maps ();xen_setup_vcpu_info_placement (); } + +/* +* Setup vcpu_info for boot CPU. +*/ +if (xen_hvm_domain ()) +xen_vcpu_setup (0 ); +/* * The alternative logic (which patches the unlock/lock) runs before * the smp bootup up code is activated. Hence we need to set this up
arch/x86/xen/xen-ops.h +1 -0
@@ -76,6 +76,7 @@ irqreturn_t xen_debug_interrupt(int irq, void *dev_id);bool xen_vcpu_stolen (int vcpu); +void xen_vcpu_setup (int cpu);void xen_setup_vcpu_info_placement (void );#ifdef CONFIG_SMP
Signature · 1 line
2.5.5
Quoted 13 lines · Vitaly Kuznetsov
It may happen that Xen's and Linux's ideas of vCPU id diverge. In particular, when we crash on a secondary vCPU we may want to do kdump and unlike plain kexec where we do
migrate_to_reboot_cpu()we try booting on the vCPU which crashed. This doesn't work very well for PVHVM guests as we have a number of hypercalls where we pass vCPU id as a parameter. These hypercalls either fail or do something unexpected. To solve the issue we need to have a mapping between Linux's and Xen's vCPU ids.This series solves the issue for x86 PVHVM guests. PV guests don't (and probably won't) support kdump so I always assume Xen's vCPU id == Linux's vCPU id. ARM guests will probably need to get proper mapping once we start supporting kexec/kdump there.
Quoted 6 lines
Changes since v1:
- "x86/acpi: store ACPI ids from MADT for future usage" patch added.
- Use ACPI ids instead of vLAPIC ids/2 [Andrew Cooper, Jan Beulich]
- Introducexen_vcpu_nr()helper [David Vrabel].
- Modify all callers ofHYPERVISOR_vcpu_op()instead of modifyingHYPERVISOR_vcpu_op()[David Vrabel]
David, Boris,
I didn't get any feedback for the v2 of this patchset, is it ready to go in for 4.8 or is there anything else I need to do to make this happen?
BTW, the patch describing the de facto "ACPI id == Xen's VCPU id" policy is in xen.git:
commit ea210c52abb6458e39f5365f7f2c3abb9c191c47
Author: Vitaly Kuznetsov <[email protected]> Date: Tue Jul 12 13:44:42 2016 +0200
x86, hvm: document the de facto policy for vCPU ids
Thanks,
Signature · 6 lines
Vitaly _______________________________________________ Xen-devel mailing list [email protected] https://lists.xen.org/xen-devel
Quoted 13 lines · Vitaly Kuznetsov
It may happen that Xen's and Linux's ideas of vCPU id diverge. In particular, when we crash on a secondary vCPU we may want to do kdump and unlike plain kexec where we do
migrate_to_reboot_cpu()we try booting on the vCPU which crashed. This doesn't work very well for PVHVM guests as we have a number of hypercalls where we pass vCPU id as a parameter. These hypercalls either fail or do something unexpected. To solve the issue we need to have a mapping between Linux's and Xen's vCPU ids.This series solves the issue for x86 PVHVM guests. PV guests don't (and probably won't) support kdump so I always assume Xen's vCPU id == Linux's vCPU id. ARM guests will probably need to get proper mapping once we start supporting kexec/kdump there.
Quoted 6 lines
Changes since v1:
- "x86/acpi: store ACPI ids from MADT for future usage" patch added.
- Use ACPI ids instead of vLAPIC ids/2 [Andrew Cooper, Jan Beulich]
- Introducexen_vcpu_nr()helper [David Vrabel].
- Modify all callers ofHYPERVISOR_vcpu_op()instead of modifyingHYPERVISOR_vcpu_op()[David Vrabel]
David, Boris,
I didn't get any feedback for the v2 of this patchset, is it ready to go in for 4.8 or is there anything else I need to do to make this happen?
BTW, the patch describing the de facto "ACPI id == Xen's VCPU id" policy is in xen.git:
commit ea210c52abb6458e39f5365f7f2c3abb9c191c47
Author: Vitaly Kuznetsov <[email protected]> Date: Tue Jul 12 13:44:42 2016 +0200
x86, hvm: document the de facto policy for vCPU ids
Thanks,
Signature · 1 line
Vitaly
David Vrabel
AppliedQuoted 8 lines · Vitaly Kuznetsov
It may happen that Xen's and Linux's ideas of vCPU id diverge. In particular, when we crash on a secondary vCPU we may want to do kdump and unlike plain kexec where we do
migrate_to_reboot_cpu()we try booting on the vCPU which crashed. This doesn't work very well for PVHVM guests as we have a number of hypercalls where we pass vCPU id as a parameter. These hypercalls either fail or do something unexpected. To solve the issue we need to have a mapping between Linux's and Xen's vCPU ids.
Quoted 4 lines
This series solves the issue for x86 PVHVM guests. PV guests don't (and probably won't) support kdump so I always assume Xen's vCPU id == Linux's vCPU id. ARM guests will probably need to get proper mapping once we start supporting kexec/kdump there.
Applied to for-linus-4.8, thanks.
David
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xen.org/xen-devel
David Vrabel
AppliedQuoted 8 lines · Vitaly Kuznetsov
It may happen that Xen's and Linux's ideas of vCPU id diverge. In particular, when we crash on a secondary vCPU we may want to do kdump and unlike plain kexec where we do
migrate_to_reboot_cpu()we try booting on the vCPU which crashed. This doesn't work very well for PVHVM guests as we have a number of hypercalls where we pass vCPU id as a parameter. These hypercalls either fail or do something unexpected. To solve the issue we need to have a mapping between Linux's and Xen's vCPU ids.
Quoted 4 lines
This series solves the issue for x86 PVHVM guests. PV guests don't (and probably won't) support kdump so I always assume Xen's vCPU id == Linux's vCPU id. ARM guests will probably need to get proper mapping once we start supporting kexec/kdump there.
Applied to for-linus-4.8, thanks.
David
Hi David,
Quoted 9 lines · David Vrabel
Quoted 8 lines · Vitaly Kuznetsov
It may happen that Xen's and Linux's ideas of vCPU id diverge. In particular, when we crash on a secondary vCPU we may want to do kdump and unlike plain kexec where we do
migrate_to_reboot_cpu()we try booting on the vCPU which crashed. This doesn't work very well for PVHVM guests as we have a number of hypercalls where we pass vCPU id as a parameter. These hypercalls either fail or do something unexpected. To solve the issue we need to have a mapping between Linux's and Xen's vCPU ids.
Quoted 6 lines
Quoted 4 lines
This series solves the issue for x86 PVHVM guests. PV guests don't (and probably won't) support kdump so I always assume Xen's vCPU id == Linux's vCPU id. ARM guests will probably need to get proper mapping once we start supporting kexec/kdump there.
Applied to for-linus-4.8, thanks.
It would have been nice to send a ping before applying. This patch series is containing Xen ARM code which has not been acked by Stefano, nor had feedback from ARM side.
For instance given that all the hypercalls are representing a "vcpu id" using "uint32_t" it is a bit weird to use "int" to define xen_vcpu_id (see patch #3).
Regards,
Signature · 6 lines
Julien Grall _______________________________________________ Xen-devel mailing list [email protected] https://lists.xen.org/xen-devel
Hi David,
Quoted 9 lines · David Vrabel
Quoted 8 lines · Vitaly Kuznetsov
It may happen that Xen's and Linux's ideas of vCPU id diverge. In particular, when we crash on a secondary vCPU we may want to do kdump and unlike plain kexec where we do
migrate_to_reboot_cpu()we try booting on the vCPU which crashed. This doesn't work very well for PVHVM guests as we have a number of hypercalls where we pass vCPU id as a parameter. These hypercalls either fail or do something unexpected. To solve the issue we need to have a mapping between Linux's and Xen's vCPU ids.
Quoted 6 lines
Quoted 4 lines
This series solves the issue for x86 PVHVM guests. PV guests don't (and probably won't) support kdump so I always assume Xen's vCPU id == Linux's vCPU id. ARM guests will probably need to get proper mapping once we start supporting kexec/kdump there.
Applied to for-linus-4.8, thanks.
It would have been nice to send a ping before applying. This patch series is containing Xen ARM code which has not been acked by Stefano, nor had feedback from ARM side.
For instance given that all the hypercalls are representing a "vcpu id" using "uint32_t" it is a bit weird to use "int" to define xen_vcpu_id (see patch #3).
Regards,
Signature · 1 line
Julien Grall
Quoted 19 lines · Julien Grall
Hi David,
Quoted 9 lines · David Vrabel
Quoted 8 lines · Vitaly Kuznetsov
It may happen that Xen's and Linux's ideas of vCPU id diverge. In particular, when we crash on a secondary vCPU we may want to do kdump and unlike plain kexec where we do
migrate_to_reboot_cpu()we try booting on the vCPU which crashed. This doesn't work very well for PVHVM guests as we have a number of hypercalls where we pass vCPU id as a parameter. These hypercalls either fail or do something unexpected. To solve the issue we need to have a mapping between Linux's and Xen's vCPU ids.Quoted 6 lines
Quoted 4 lines
This series solves the issue for x86 PVHVM guests. PV guests don't (and probably won't) support kdump so I always assume Xen's vCPU id == Linux's vCPU id. ARM guests will probably need to get proper mapping once we start supporting kexec/kdump there.
Applied to for-linus-4.8, thanks.
Quoted 7 lines
It would have been nice to send a ping before applying. This patch series is containing Xen ARM code which has not been acked by Stefano, nor had feedback from ARM side.
For instance given that all the hypercalls are representing a "vcpu id" using "
uint32_t" it is a bit weird to use "int" to definexen_vcpu_id(see patch #3).
CPU id is usually 'int' in linux and now we pass it to all hypercalls as it is. It is a bit more convenient in the mapping I introduce as we can set it to a negative value to indicate there is no mapping available. I can definitely change that and use something like U32_MAX-1 to instead but I'm not sure it is worth it...
Signature · 6 lines
Vitaly _______________________________________________ Xen-devel mailing list [email protected] https://lists.xen.org/xen-devel
Quoted 19 lines · Julien Grall
Hi David,
Quoted 9 lines · David Vrabel
Quoted 8 lines · Vitaly Kuznetsov
It may happen that Xen's and Linux's ideas of vCPU id diverge. In particular, when we crash on a secondary vCPU we may want to do kdump and unlike plain kexec where we do
migrate_to_reboot_cpu()we try booting on the vCPU which crashed. This doesn't work very well for PVHVM guests as we have a number of hypercalls where we pass vCPU id as a parameter. These hypercalls either fail or do something unexpected. To solve the issue we need to have a mapping between Linux's and Xen's vCPU ids.Quoted 6 lines
Quoted 4 lines
This series solves the issue for x86 PVHVM guests. PV guests don't (and probably won't) support kdump so I always assume Xen's vCPU id == Linux's vCPU id. ARM guests will probably need to get proper mapping once we start supporting kexec/kdump there.
Applied to for-linus-4.8, thanks.
Quoted 7 lines
It would have been nice to send a ping before applying. This patch series is containing Xen ARM code which has not been acked by Stefano, nor had feedback from ARM side.
For instance given that all the hypercalls are representing a "vcpu id" using "
uint32_t" it is a bit weird to use "int" to definexen_vcpu_id(see patch #3).
CPU id is usually 'int' in linux and now we pass it to all hypercalls as it is. It is a bit more convenient in the mapping I introduce as we can set it to a negative value to indicate there is no mapping available. I can definitely change that and use something like U32_MAX-1 to instead but I'm not sure it is worth it...
Signature · 1 line
Vitaly
Hello,
Quoted 25 lines · Vitaly Kuznetsov
Quoted 23 lines · Julien Grall
Hi David,
Quoted 9 lines · David Vrabel
Quoted 8 lines · Vitaly Kuznetsov
It may happen that Xen's and Linux's ideas of vCPU id diverge. In particular, when we crash on a secondary vCPU we may want to do kdump and unlike plain kexec where we do
migrate_to_reboot_cpu()we try booting on the vCPU which crashed. This doesn't work very well for PVHVM guests as we have a number of hypercalls where we pass vCPU id as a parameter. These hypercalls either fail or do something unexpected. To solve the issue we need to have a mapping between Linux's and Xen's vCPU ids.Quoted 6 lines
Quoted 4 lines
This series solves the issue for x86 PVHVM guests. PV guests don't (and probably won't) support kdump so I always assume Xen's vCPU id == Linux's vCPU id. ARM guests will probably need to get proper mapping once we start supporting kexec/kdump there.
Applied to for-linus-4.8, thanks.
It would have been nice to send a ping before applying. This patch series is containing Xen ARM code which has not been acked by Stefano, nor had feedback from ARM side.
Quoted 6 lines
Quoted 3 lines
For instance given that all the hypercalls are representing a "vcpu id" using "
uint32_t" it is a bit weird to use "int" to definexen_vcpu_id(see patch #3).CPU id is usually 'int' in linux and now we pass it to all hypercalls as it is.
Well, we need to differentiate between the internal representation of the CPU which is based on the boot order and the logical CPU ID. For instance on ARM, the logical CPU ID may not be contiguous nor 0 for the first CPU.
From my understanding, the macros in patch #3 will be used at the last minute when prepare the hypercall data. IHMO this is very similar to a logical ID and defined as uint32_t by the hypercall ABI.
Although, I agree that currently we use the internal CPU id on ARM which is very unfortunate because this value is based on the order of the nodes in the device tree.
One way to abolish it on ARM would be to use the MPIDR (or at least a part) for the VCPU ID.
Quoted 4 lines
It is a bit more convenient in the mapping I
introduce as we can set it to a negative value to indicate there is no mapping available. I can definitely change that and use something like U32_MAX-1 to instead but I'm not sure it is worth it...
I looked at the definition of cpu_acpi_id on x86 which return x86_cpu_to_acpiid that has been defined to an uint32_t.
So you are assuming that it will never be possible to have an ID > 0x80000000.
Also, this may not be true on ARM depending how we define the VCPU mapping. We could decide to use the MPIDR which is in this case may be considered as "negative".
Regards,
Signature · 6 lines
Julien Grall _______________________________________________ Xen-devel mailing list [email protected] https://lists.xen.org/xen-devel
Hello,
Quoted 25 lines · Vitaly Kuznetsov
Quoted 23 lines · Julien Grall
Hi David,
Quoted 9 lines · David Vrabel
Quoted 8 lines · Vitaly Kuznetsov
It may happen that Xen's and Linux's ideas of vCPU id diverge. In particular, when we crash on a secondary vCPU we may want to do kdump and unlike plain kexec where we do
migrate_to_reboot_cpu()we try booting on the vCPU which crashed. This doesn't work very well for PVHVM guests as we have a number of hypercalls where we pass vCPU id as a parameter. These hypercalls either fail or do something unexpected. To solve the issue we need to have a mapping between Linux's and Xen's vCPU ids.Quoted 6 lines
Quoted 4 lines
This series solves the issue for x86 PVHVM guests. PV guests don't (and probably won't) support kdump so I always assume Xen's vCPU id == Linux's vCPU id. ARM guests will probably need to get proper mapping once we start supporting kexec/kdump there.
Applied to for-linus-4.8, thanks.
It would have been nice to send a ping before applying. This patch series is containing Xen ARM code which has not been acked by Stefano, nor had feedback from ARM side.
Quoted 6 lines
Quoted 3 lines
For instance given that all the hypercalls are representing a "vcpu id" using "
uint32_t" it is a bit weird to use "int" to definexen_vcpu_id(see patch #3).CPU id is usually 'int' in linux and now we pass it to all hypercalls as it is.
Well, we need to differentiate between the internal representation of the CPU which is based on the boot order and the logical CPU ID. For instance on ARM, the logical CPU ID may not be contiguous nor 0 for the first CPU.
From my understanding, the macros in patch #3 will be used at the last minute when prepare the hypercall data. IHMO this is very similar to a logical ID and defined as uint32_t by the hypercall ABI.
Although, I agree that currently we use the internal CPU id on ARM which is very unfortunate because this value is based on the order of the nodes in the device tree.
One way to abolish it on ARM would be to use the MPIDR (or at least a part) for the VCPU ID.
Quoted 4 lines
It is a bit more convenient in the mapping I
introduce as we can set it to a negative value to indicate there is no mapping available. I can definitely change that and use something like U32_MAX-1 to instead but I'm not sure it is worth it...
I looked at the definition of cpu_acpi_id on x86 which return x86_cpu_to_acpiid that has been defined to an uint32_t.
So you are assuming that it will never be possible to have an ID > 0x80000000.
Also, this may not be true on ARM depending how we define the VCPU mapping. We could decide to use the MPIDR which is in this case may be considered as "negative".
Regards,
Signature · 1 line
Julien Grall
Quoted 66 lines · Julien Grall
Hello,
Quoted 32 lines · Vitaly Kuznetsov
Quoted 30 lines · Julien Grall
Hi David,
Quoted 21 lines · David Vrabel
Quoted 12 lines · Vitaly Kuznetsov
It may happen that Xen's and Linux's ideas of vCPU id diverge. In particular, when we crash on a secondary vCPU we may want to do kdump and unlike plain kexec where we do
migrate_to_reboot_cpu()we try booting
on the vCPU which crashed. This doesn't work very well for PVHVM guests as
we have a number of hypercalls where we pass vCPU id as a parameter. These
hypercalls either fail or do something unexpected. To solve the issue we
need to have a mapping between Linux's and Xen's vCPU ids.Quoted 7 lines
This series solves the issue for x86 PVHVM guests. PV guests don't (and
probably won't) support kdump so I always assume Xen's vCPU id == Linux's
vCPU id. ARM guests will probably need to get proper mapping once we start
supporting kexec/kdump there.Quoted 1 line
Applied to for-linus-4.8, thanks.
It would have been nice to send a ping before applying. This patch series is containing Xen ARM code which has not been acked by Stefano, nor had feedback from ARM side.
Quoted 6 lines
Quoted 3 lines
For instance given that all the hypercalls are representing a "vcpu id" using "
uint32_t" it is a bit weird to use "int" to definexen_vcpu_id(see patch #3).CPU id is usually 'int' in linux and now we pass it to all hypercalls as it is.
Well, we need to differentiate between the internal representation of the CPU which is based on the boot order and the logical CPU ID. For instance on ARM, the logical CPU ID may not be contiguous nor 0 for the first CPU.
From my understanding, the macros in patch #3 will be used at the last minute when prepare the hypercall data. IHMO this is very similar to a logical ID and defined as
uint32_tby the hypercall ABI.Although, I agree that currently we use the internal CPU id on ARM which is very unfortunate because this value is based on the order of the nodes in the device tree.
One way to abolish it on ARM would be to use the MPIDR (or at least a part) for the VCPU ID.
Quoted 4 lines
It is a bit more convenient in the mapping I
introduce as we can set it to a negative value to indicate there is no mapping available. I can definitely change that and use something like U32_MAX-1 to instead but I'm not sure it is worth it...I looked at the definition of
cpu_acpi_idon x86 which returnx86_cpu_to_acpiidthat has been defined to anuint32_t.
Quoted 6 lines
So you are assuming that it will never be possible to have an ID >
0x80000000.Also, this may not be true on ARM depending how we define the VCPU mapping. We could decide to use the MPIDR which is in this case may be considered as "negative".
If you want to change how you number vCPUs on ARM in the future, you can (if necessary) change the type of this per-cpu variable as well.
David
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xen.org/xen-devel
Quoted 66 lines · Julien Grall
Hello,
Quoted 32 lines · Vitaly Kuznetsov
Quoted 30 lines · Julien Grall
Hi David,
Quoted 21 lines · David Vrabel
Quoted 12 lines · Vitaly Kuznetsov
It may happen that Xen's and Linux's ideas of vCPU id diverge. In particular, when we crash on a secondary vCPU we may want to do kdump and unlike plain kexec where we do
migrate_to_reboot_cpu()we try booting
on the vCPU which crashed. This doesn't work very well for PVHVM guests as
we have a number of hypercalls where we pass vCPU id as a parameter. These
hypercalls either fail or do something unexpected. To solve the issue we
need to have a mapping between Linux's and Xen's vCPU ids.Quoted 7 lines
This series solves the issue for x86 PVHVM guests. PV guests don't (and
probably won't) support kdump so I always assume Xen's vCPU id == Linux's
vCPU id. ARM guests will probably need to get proper mapping once we start
supporting kexec/kdump there.Quoted 1 line
Applied to for-linus-4.8, thanks.
It would have been nice to send a ping before applying. This patch series is containing Xen ARM code which has not been acked by Stefano, nor had feedback from ARM side.
Quoted 6 lines
Quoted 3 lines
For instance given that all the hypercalls are representing a "vcpu id" using "
uint32_t" it is a bit weird to use "int" to definexen_vcpu_id(see patch #3).CPU id is usually 'int' in linux and now we pass it to all hypercalls as it is.
Well, we need to differentiate between the internal representation of the CPU which is based on the boot order and the logical CPU ID. For instance on ARM, the logical CPU ID may not be contiguous nor 0 for the first CPU.
From my understanding, the macros in patch #3 will be used at the last minute when prepare the hypercall data. IHMO this is very similar to a logical ID and defined as
uint32_tby the hypercall ABI.Although, I agree that currently we use the internal CPU id on ARM which is very unfortunate because this value is based on the order of the nodes in the device tree.
One way to abolish it on ARM would be to use the MPIDR (or at least a part) for the VCPU ID.
Quoted 4 lines
It is a bit more convenient in the mapping I
introduce as we can set it to a negative value to indicate there is no mapping available. I can definitely change that and use something like U32_MAX-1 to instead but I'm not sure it is worth it...I looked at the definition of
cpu_acpi_idon x86 which returnx86_cpu_to_acpiidthat has been defined to anuint32_t.
Quoted 6 lines
So you are assuming that it will never be possible to have an ID >
0x80000000.Also, this may not be true on ARM depending how we define the VCPU mapping. We could decide to use the MPIDR which is in this case may be considered as "negative".
If you want to change how you number vCPUs on ARM in the future, you can (if necessary) change the type of this per-cpu variable as well.
David
Quoted 44 lines · Julien Grall
Hello,
Quoted 25 lines · Vitaly Kuznetsov
Quoted 23 lines · Julien Grall
Hi David,
Quoted 9 lines · David Vrabel
Quoted 8 lines · Vitaly Kuznetsov
It may happen that Xen's and Linux's ideas of vCPU id diverge. In particular, when we crash on a secondary vCPU we may want to do kdump and unlike plain kexec where we do
migrate_to_reboot_cpu()we try booting on the vCPU which crashed. This doesn't work very well for PVHVM guests as we have a number of hypercalls where we pass vCPU id as a parameter. These hypercalls either fail or do something unexpected. To solve the issue we need to have a mapping between Linux's and Xen's vCPU ids.Quoted 6 lines
Quoted 4 lines
This series solves the issue for x86 PVHVM guests. PV guests don't (and probably won't) support kdump so I always assume Xen's vCPU id == Linux's vCPU id. ARM guests will probably need to get proper mapping once we start supporting kexec/kdump there.
Applied to for-linus-4.8, thanks.
It would have been nice to send a ping before applying. This patch series is containing Xen ARM code which has not been acked by Stefano, nor had feedback from ARM side.
Quoted 6 lines
Quoted 3 lines
For instance given that all the hypercalls are representing a "vcpu id" using "
uint32_t" it is a bit weird to use "int" to definexen_vcpu_id(see patch #3).CPU id is usually 'int' in linux and now we pass it to all hypercalls as it is.
Well, we need to differentiate between the internal representation of the CPU which is based on the boot order and the logical CPU ID. For instance on ARM, the logical CPU ID may not be contiguous nor 0 for the first CPU.
From my understanding, the macros in patch #3 will be used at the last minute when prepare the hypercall data. IHMO this is very similar to a logical ID and defined as
uint32_tby the hypercall ABI.
Quoted 7 lines
Although, I agree that currently we use the internal CPU id on ARM which is very unfortunate because this value is based on the order of the nodes in the device tree.
One way to abolish it on ARM would be to use the MPIDR (or at least a part) for the VCPU ID.
I probably know too little about ARM but it seems that unlike x86 we don't need the knowledge of _other_ vCPU ids before we start them so MPIDR looks very suitable.
Quoted 8 lines
Quoted 4 lines
It is a bit more convenient in the mapping I
introduce as we can set it to a negative value to indicate there is no mapping available. I can definitely change that and use something like U32_MAX-1 to instead but I'm not sure it is worth it...I looked at the definition of
cpu_acpi_idon x86 which returnx86_cpu_to_acpiidthat has been defined to anuint32_t.
Quoted 6 lines
So you are assuming that it will never be possible to have an ID >
0x80000000.Also, this may not be true on ARM depending how we define the VCPU mapping. We could decide to use the MPIDR which is in this case may be considered as "negative".
While we're not obliged to have the same type for xen_vcpu_id on all arches I see no point in diverging without a reason. I can do v3 making the mapping uint32 and indicating the missing value as U32_MAX-1 if nobody is against the idea.
Signature · 6 lines
Vitaly _______________________________________________ Xen-devel mailing list [email protected] https://lists.xen.org/xen-devel
Quoted 44 lines · Julien Grall
Hello,
Quoted 25 lines · Vitaly Kuznetsov
Quoted 23 lines · Julien Grall
Hi David,
Quoted 9 lines · David Vrabel
Quoted 8 lines · Vitaly Kuznetsov
It may happen that Xen's and Linux's ideas of vCPU id diverge. In particular, when we crash on a secondary vCPU we may want to do kdump and unlike plain kexec where we do
migrate_to_reboot_cpu()we try booting on the vCPU which crashed. This doesn't work very well for PVHVM guests as we have a number of hypercalls where we pass vCPU id as a parameter. These hypercalls either fail or do something unexpected. To solve the issue we need to have a mapping between Linux's and Xen's vCPU ids.Quoted 6 lines
Quoted 4 lines
This series solves the issue for x86 PVHVM guests. PV guests don't (and probably won't) support kdump so I always assume Xen's vCPU id == Linux's vCPU id. ARM guests will probably need to get proper mapping once we start supporting kexec/kdump there.
Applied to for-linus-4.8, thanks.
It would have been nice to send a ping before applying. This patch series is containing Xen ARM code which has not been acked by Stefano, nor had feedback from ARM side.
Quoted 6 lines
Quoted 3 lines
For instance given that all the hypercalls are representing a "vcpu id" using "
uint32_t" it is a bit weird to use "int" to definexen_vcpu_id(see patch #3).CPU id is usually 'int' in linux and now we pass it to all hypercalls as it is.
Well, we need to differentiate between the internal representation of the CPU which is based on the boot order and the logical CPU ID. For instance on ARM, the logical CPU ID may not be contiguous nor 0 for the first CPU.
From my understanding, the macros in patch #3 will be used at the last minute when prepare the hypercall data. IHMO this is very similar to a logical ID and defined as
uint32_tby the hypercall ABI.
Quoted 7 lines
Although, I agree that currently we use the internal CPU id on ARM which is very unfortunate because this value is based on the order of the nodes in the device tree.
One way to abolish it on ARM would be to use the MPIDR (or at least a part) for the VCPU ID.
I probably know too little about ARM but it seems that unlike x86 we don't need the knowledge of _other_ vCPU ids before we start them so MPIDR looks very suitable.
Quoted 8 lines
Quoted 4 lines
It is a bit more convenient in the mapping I
introduce as we can set it to a negative value to indicate there is no mapping available. I can definitely change that and use something like U32_MAX-1 to instead but I'm not sure it is worth it...I looked at the definition of
cpu_acpi_idon x86 which returnx86_cpu_to_acpiidthat has been defined to anuint32_t.
Quoted 6 lines
So you are assuming that it will never be possible to have an ID >
0x80000000.Also, this may not be true on ARM depending how we define the VCPU mapping. We could decide to use the MPIDR which is in this case may be considered as "negative".
While we're not obliged to have the same type for xen_vcpu_id on all arches I see no point in diverging without a reason. I can do v3 making the mapping uint32 and indicating the missing value as U32_MAX-1 if nobody is against the idea.
Signature · 1 line
Vitaly
Stefano Stabellini
maintainerQuoted 69 lines · Vitaly Kuznetsov
Quoted 44 lines · Julien Grall
Hello,
Quoted 25 lines · Vitaly Kuznetsov
Quoted 23 lines · Julien Grall
Hi David,
Quoted 9 lines · David Vrabel
Quoted 8 lines · Vitaly Kuznetsov
It may happen that Xen's and Linux's ideas of vCPU id diverge. In particular, when we crash on a secondary vCPU we may want to do kdump and unlike plain kexec where we do
migrate_to_reboot_cpu()we try booting on the vCPU which crashed. This doesn't work very well for PVHVM guests as we have a number of hypercalls where we pass vCPU id as a parameter. These hypercalls either fail or do something unexpected. To solve the issue we need to have a mapping between Linux's and Xen's vCPU ids.Quoted 6 lines
Quoted 4 lines
This series solves the issue for x86 PVHVM guests. PV guests don't (and probably won't) support kdump so I always assume Xen's vCPU id == Linux's vCPU id. ARM guests will probably need to get proper mapping once we start supporting kexec/kdump there.
Applied to for-linus-4.8, thanks.
It would have been nice to send a ping before applying. This patch series is containing Xen ARM code which has not been acked by Stefano, nor had feedback from ARM side.
Quoted 6 lines
Quoted 3 lines
For instance given that all the hypercalls are representing a "vcpu id" using "
uint32_t" it is a bit weird to use "int" to definexen_vcpu_id(see patch #3).CPU id is usually 'int' in linux and now we pass it to all hypercalls as it is.
Well, we need to differentiate between the internal representation of the CPU which is based on the boot order and the logical CPU ID. For instance on ARM, the logical CPU ID may not be contiguous nor 0 for the first CPU.
From my understanding, the macros in patch #3 will be used at the last minute when prepare the hypercall data. IHMO this is very similar to a logical ID and defined as
uint32_tby the hypercall ABI.Quoted 7 lines
Although, I agree that currently we use the internal CPU id on ARM which is very unfortunate because this value is based on the order of the nodes in the device tree.
One way to abolish it on ARM would be to use the MPIDR (or at least a part) for the VCPU ID.
I probably know too little about ARM but it seems that unlike x86 we don't need the knowledge of _other_ vCPU ids before we start them so MPIDR looks very suitable.
Quoted 11 lines
Quoted 4 lines
It is a bit more convenient in the mapping I
introduce as we can set it to a negative value to indicate there is no mapping available. I can definitely change that and use something like U32_MAX-1 to instead but I'm not sure it is worth it...I looked at the definition of
cpu_acpi_idon x86 which returnx86_cpu_to_acpiidthat has been defined to anuint32_t.So you are assuming that it will never be possible to have an ID >
0x80000000.
Quoted 7 lines
Quoted 3 lines
Also, this may not be true on ARM depending how we define the VCPU mapping. We could decide to use the MPIDR which is in this case may be considered as "negative".
While we're not obliged to have the same type for
xen_vcpu_idon all arches I see no point in diverging without a reason. I can do v3 making the mapping uint32
I agree that making the mapping uint32_t would be desirable. It would even make sense from the int types point of view in Linux.
Quoted 2 lines
and indicating the missing value as U32_MAX-1 if nobody is against the idea.
Why U32_MAX-1? (int)-1 is (unsigned)U32_MAX. EvenXEN_INVALID_MAX_VCPU_ID is defined as (~0U).
Quoted 6 lines · Stefano Stabellini
Quoted 4 lines · Vitaly Kuznetsov
While we're not obliged to have the same type for
xen_vcpu_idon all arches I see no point in diverging without a reason. I can do v3 making the mapping uint32
Quoted 8 lines
I agree that making the mapping
uint32_twould be desirable. It would even make sense from the int types point of view in Linux.Quoted 2 lines
and indicating the missing value as U32_MAX-1 if nobody is against the idea.
Why U32_MAX-1? (int)-1 is (unsigned)U32_MAX. Even
XEN_INVALID_MAX_VCPU_IDis defined as (~0U).
Yes, my bad, of course it should be U32_MAX.
Signature · 1 line
Vitaly
Quoted 6 lines · Stefano Stabellini
Quoted 4 lines · Vitaly Kuznetsov
While we're not obliged to have the same type for
xen_vcpu_idon all arches I see no point in diverging without a reason. I can do v3 making the mapping uint32
Quoted 8 lines
I agree that making the mapping
uint32_twould be desirable. It would even make sense from the int types point of view in Linux.Quoted 2 lines
and indicating the missing value as U32_MAX-1 if nobody is against the idea.
Why U32_MAX-1? (int)-1 is (unsigned)U32_MAX. Even
XEN_INVALID_MAX_VCPU_IDis defined as (~0U).
Yes, my bad, of course it should be U32_MAX.
Signature · 6 lines
Vitaly _______________________________________________ Xen-devel mailing list [email protected] https://lists.xen.org/xen-devel
Stefano Stabellini
maintainerQuoted 69 lines · Vitaly Kuznetsov
Quoted 44 lines · Julien Grall
Hello,
Quoted 25 lines · Vitaly Kuznetsov
Quoted 23 lines · Julien Grall
Hi David,
Quoted 9 lines · David Vrabel
Quoted 8 lines · Vitaly Kuznetsov
It may happen that Xen's and Linux's ideas of vCPU id diverge. In particular, when we crash on a secondary vCPU we may want to do kdump and unlike plain kexec where we do
migrate_to_reboot_cpu()we try booting on the vCPU which crashed. This doesn't work very well for PVHVM guests as we have a number of hypercalls where we pass vCPU id as a parameter. These hypercalls either fail or do something unexpected. To solve the issue we need to have a mapping between Linux's and Xen's vCPU ids.Quoted 6 lines
Quoted 4 lines
This series solves the issue for x86 PVHVM guests. PV guests don't (and probably won't) support kdump so I always assume Xen's vCPU id == Linux's vCPU id. ARM guests will probably need to get proper mapping once we start supporting kexec/kdump there.
Applied to for-linus-4.8, thanks.
It would have been nice to send a ping before applying. This patch series is containing Xen ARM code which has not been acked by Stefano, nor had feedback from ARM side.
Quoted 6 lines
Quoted 3 lines
For instance given that all the hypercalls are representing a "vcpu id" using "
uint32_t" it is a bit weird to use "int" to definexen_vcpu_id(see patch #3).CPU id is usually 'int' in linux and now we pass it to all hypercalls as it is.
Well, we need to differentiate between the internal representation of the CPU which is based on the boot order and the logical CPU ID. For instance on ARM, the logical CPU ID may not be contiguous nor 0 for the first CPU.
From my understanding, the macros in patch #3 will be used at the last minute when prepare the hypercall data. IHMO this is very similar to a logical ID and defined as
uint32_tby the hypercall ABI.Quoted 7 lines
Although, I agree that currently we use the internal CPU id on ARM which is very unfortunate because this value is based on the order of the nodes in the device tree.
One way to abolish it on ARM would be to use the MPIDR (or at least a part) for the VCPU ID.
I probably know too little about ARM but it seems that unlike x86 we don't need the knowledge of _other_ vCPU ids before we start them so MPIDR looks very suitable.
Quoted 11 lines
Quoted 4 lines
It is a bit more convenient in the mapping I
introduce as we can set it to a negative value to indicate there is no mapping available. I can definitely change that and use something like U32_MAX-1 to instead but I'm not sure it is worth it...I looked at the definition of
cpu_acpi_idon x86 which returnx86_cpu_to_acpiidthat has been defined to anuint32_t.So you are assuming that it will never be possible to have an ID >
0x80000000.
Quoted 7 lines
Quoted 3 lines
Also, this may not be true on ARM depending how we define the VCPU mapping. We could decide to use the MPIDR which is in this case may be considered as "negative".
While we're not obliged to have the same type for
xen_vcpu_idon all arches I see no point in diverging without a reason. I can do v3 making the mapping uint32
I agree that making the mapping uint32_t would be desirable. It would even make sense from the int types point of view in Linux.
Quoted 2 lines
and indicating the missing value as U32_MAX-1 if nobody is against the idea.
Why U32_MAX-1? (int)-1 is (unsigned)U32_MAX. EvenXEN_INVALID_MAX_VCPU_ID is defined as (~0U).
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xen.org/xen-devel
Stefano Stabellini
maintainerQuoted 48 lines · Julien Grall
Quoted 32 lines · Vitaly Kuznetsov
Quoted 30 lines · Julien Grall
Hi David,
Quoted 21 lines · David Vrabel
Quoted 12 lines · Vitaly Kuznetsov
It may happen that Xen's and Linux's ideas of vCPU id diverge. In particular, when we crash on a secondary vCPU we may want to do kdump and unlike plain kexec where we do
migrate_to_reboot_cpu()we try booting
on the vCPU which crashed. This doesn't work very well for PVHVM guests as
we have a number of hypercalls where we pass vCPU id as a parameter. These
hypercalls either fail or do something unexpected. To solve the issue we
need to have a mapping between Linux's and Xen's vCPU ids.Quoted 7 lines
This series solves the issue for x86 PVHVM guests. PV guests don't (and
probably won't) support kdump so I always assume Xen's vCPU id == Linux's
vCPU id. ARM guests will probably need to get proper mapping once we start
supporting kexec/kdump there.Quoted 1 line
Applied to for-linus-4.8, thanks.
It would have been nice to send a ping before applying. This patch series is containing Xen ARM code which has not been acked by Stefano, nor had feedback from ARM side.
Quoted 6 lines
Quoted 3 lines
For instance given that all the hypercalls are representing a "vcpu id" using "
uint32_t" it is a bit weird to use "int" to definexen_vcpu_id(see patch #3).CPU id is usually 'int' in linux and now we pass it to all hypercalls as it is.
Well, we need to differentiate between the internal representation of the CPU which is based on the boot order and the logical CPU ID. For instance on ARM, the logical CPU ID may not be contiguous nor 0 for the first CPU.
From my understanding, the macros in patch #3 will be used at the last minute when prepare the hypercall data. IHMO this is very similar to a logical ID and defined as
uint32_tby the hypercall ABI.
Quoted 6 lines
Although, I agree that currently we use the internal CPU id on ARM which is very unfortunate because this value is based on the order of the nodes in the device tree.
One way to abolish it on ARM would be to use the MPIDR (or at least a part) for the VCPU ID.
That's a good idea -- much more reliable than the device tree cpu order.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xen.org/xen-devel
Stefano Stabellini
maintainerQuoted 48 lines · Julien Grall
Quoted 32 lines · Vitaly Kuznetsov
Quoted 30 lines · Julien Grall
Hi David,
Quoted 21 lines · David Vrabel
Quoted 12 lines · Vitaly Kuznetsov
It may happen that Xen's and Linux's ideas of vCPU id diverge. In particular, when we crash on a secondary vCPU we may want to do kdump and unlike plain kexec where we do
migrate_to_reboot_cpu()we try booting
on the vCPU which crashed. This doesn't work very well for PVHVM guests as
we have a number of hypercalls where we pass vCPU id as a parameter. These
hypercalls either fail or do something unexpected. To solve the issue we
need to have a mapping between Linux's and Xen's vCPU ids.Quoted 7 lines
This series solves the issue for x86 PVHVM guests. PV guests don't (and
probably won't) support kdump so I always assume Xen's vCPU id == Linux's
vCPU id. ARM guests will probably need to get proper mapping once we start
supporting kexec/kdump there.Quoted 1 line
Applied to for-linus-4.8, thanks.
It would have been nice to send a ping before applying. This patch series is containing Xen ARM code which has not been acked by Stefano, nor had feedback from ARM side.
Quoted 6 lines
Quoted 3 lines
For instance given that all the hypercalls are representing a "vcpu id" using "
uint32_t" it is a bit weird to use "int" to definexen_vcpu_id(see patch #3).CPU id is usually 'int' in linux and now we pass it to all hypercalls as it is.
Well, we need to differentiate between the internal representation of the CPU which is based on the boot order and the logical CPU ID. For instance on ARM, the logical CPU ID may not be contiguous nor 0 for the first CPU.
From my understanding, the macros in patch #3 will be used at the last minute when prepare the hypercall data. IHMO this is very similar to a logical ID and defined as
uint32_tby the hypercall ABI.
Quoted 6 lines
Although, I agree that currently we use the internal CPU id on ARM which is very unfortunate because this value is based on the order of the nodes in the device tree.
One way to abolish it on ARM would be to use the MPIDR (or at least a part) for the VCPU ID.
That's a good idea -- much more reliable than the device tree cpu order.
Quoted 22 lines · Julien Grall
Hi David,
Quoted 19 lines · David Vrabel
Quoted 17 lines · Vitaly Kuznetsov
It may happen that Xen's and Linux's ideas of vCPU id diverge. In particular, when we crash on a secondary vCPU we may want to do kdump and unlike plain kexec where we do
migrate_to_reboot_cpu()we try booting
on the vCPU which crashed. This doesn't work very well for PVHVM guests as
we have a number of hypercalls where we pass vCPU id as a parameter. These
hypercalls either fail or do something unexpected. To solve the issue we need to have a mapping between Linux's and Xen's vCPU ids.This series solves the issue for x86 PVHVM guests. PV guests don't (and probably won't) support kdump so I always assume Xen's vCPU id == Linux's
vCPU id. ARM guests will probably need to get proper mapping once we start
supporting kexec/kdump there.
Quoted 5 lines
Quoted 1 line
Applied to for-linus-4.8, thanks.
It would have been nice to send a ping before applying. This patch series is containing Xen ARM code which has not been acked by Stefano, nor had feedback from ARM side.
The ARM parts are trivial and such a small part of this series, I saw no need to wait.
David
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xen.org/xen-devel
Quoted 22 lines · Julien Grall
Hi David,
Quoted 19 lines · David Vrabel
Quoted 17 lines · Vitaly Kuznetsov
It may happen that Xen's and Linux's ideas of vCPU id diverge. In particular, when we crash on a secondary vCPU we may want to do kdump and unlike plain kexec where we do
migrate_to_reboot_cpu()we try booting
on the vCPU which crashed. This doesn't work very well for PVHVM guests as
we have a number of hypercalls where we pass vCPU id as a parameter. These
hypercalls either fail or do something unexpected. To solve the issue we need to have a mapping between Linux's and Xen's vCPU ids.This series solves the issue for x86 PVHVM guests. PV guests don't (and probably won't) support kdump so I always assume Xen's vCPU id == Linux's
vCPU id. ARM guests will probably need to get proper mapping once we start
supporting kexec/kdump there.
Quoted 5 lines
Quoted 1 line
Applied to for-linus-4.8, thanks.
It would have been nice to send a ping before applying. This patch series is containing Xen ARM code which has not been acked by Stefano, nor had feedback from ARM side.
The ARM parts are trivial and such a small part of this series, I saw no need to wait.
David
Stefano Stabellini
maintainerQuoted 23 lines · David Vrabel
Quoted 22 lines · Julien Grall
Hi David,
Quoted 19 lines · David Vrabel
Quoted 17 lines · Vitaly Kuznetsov
It may happen that Xen's and Linux's ideas of vCPU id diverge. In particular, when we crash on a secondary vCPU we may want to do kdump and unlike plain kexec where we do
migrate_to_reboot_cpu()we try booting
on the vCPU which crashed. This doesn't work very well for PVHVM guests as
we have a number of hypercalls where we pass vCPU id as a parameter. These
hypercalls either fail or do something unexpected. To solve the issue we need to have a mapping between Linux's and Xen's vCPU ids.This series solves the issue for x86 PVHVM guests. PV guests don't (and probably won't) support kdump so I always assume Xen's vCPU id == Linux's
vCPU id. ARM guests will probably need to get proper mapping once we start
supporting kexec/kdump there.
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
Applied to for-linus-4.8, thanks.
It would have been nice to send a ping before applying. This patch series is containing Xen ARM code which has not been acked by Stefano, nor had feedback from ARM side.
The ARM parts are trivial and such a small part of this series, I saw no need to wait.
That's OK, no worries. In the future it would be nice to wait a few more hours, until morning PST.
Cheers,
Stefano
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xen.org/xen-devel
Stefano Stabellini
maintainerQuoted 23 lines · David Vrabel
Quoted 22 lines · Julien Grall
Hi David,
Quoted 19 lines · David Vrabel
Quoted 17 lines · Vitaly Kuznetsov
It may happen that Xen's and Linux's ideas of vCPU id diverge. In particular, when we crash on a secondary vCPU we may want to do kdump and unlike plain kexec where we do
migrate_to_reboot_cpu()we try booting
on the vCPU which crashed. This doesn't work very well for PVHVM guests as
we have a number of hypercalls where we pass vCPU id as a parameter. These
hypercalls either fail or do something unexpected. To solve the issue we need to have a mapping between Linux's and Xen's vCPU ids.This series solves the issue for x86 PVHVM guests. PV guests don't (and probably won't) support kdump so I always assume Xen's vCPU id == Linux's
vCPU id. ARM guests will probably need to get proper mapping once we start
supporting kexec/kdump there.
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
Applied to for-linus-4.8, thanks.
It would have been nice to send a ping before applying. This patch series is containing Xen ARM code which has not been acked by Stefano, nor had feedback from ARM side.
The ARM parts are trivial and such a small part of this series, I saw no need to wait.
That's OK, no worries. In the future it would be nice to wait a few more hours, until morning PST.
Cheers,
Stefano