KVM: x86: CPU isolation and direct interrupts delivery to guests

RFC PATCH In review Tomoki Sekiyama 4 participants 29 messages Sep 6, 2012 – Sep 28, 2012

Subsystems (8)

x86 KVM KVM x86 IOMMU +4 more subsystems

lore

Share & tools

lore

Tomoki Sekiyama

author

2012-09-06 11:27 UTC #0 lore raw

v2 cover KVM: x86: CPU isolation and direct interrupts delivery to guests

This RFC patch series provides facility to dedicate CPUs to KVM guests and enable the guests to handle interrupts from passed-through PCI devices directly (without VM exit and relay by the host).

With this feature, we can improve throughput and response time of the device and the host's CPU usage by reducing the overhead of interrupt handling. This is good for the application using very high throughput/frequent interrupt device (e.g. 10GbE NIC).
Real-time applicatoins also gets benefit from CPU isolation feature, which reduces interfare from host kernel tasks and scheduling delay.

The overview of this patch series is presented in CloudOpen 2012. The slides are available at:
http://events.linuxfoundation.org/images/stories/pdf/lcna_co2012_sekiyama.pdf

* Changes from v1 ( https://lkml.org/lkml/2012/6/28/30 )
- SMP guest is supported
- Direct EOI is added, that eliminate VM exit on EOI
- Direct local APIC timer access from guests is added, which pass-through the physical timer of a dedicated CPU to the guest.
- Rebased on v3.6-rc4

* How to test
- Create a guest VM with 1 CPU and some PCI passthrough devices (which supports MSI/MSI-X).
No VGA display will be better...
- Apply the patch at the end of this mail to qemu-kvm. (This patch is just for simple testing, and dedicated CPU ID for the guest is hard-coded.)
- Run the guest once to ensure the PCI passthrough works correctly.
- Make the specified CPU offline.

     # echo 0 > /sys/devices/system/cpu/cpu3/online
 - Launch qemu-kvm with -no-kvm-pit option.
   The offlined CPU is booted as a slave CPU and guest is runs on that CPU.

* To-do
- Enable slave CPUs to handle access fault
- Support AMD SVM
- Support non-Linux guests

---

Tomoki Sekiyama (21):
x86: request TLB flush to slave CPU using NMI
KVM: Pass-through local APIC timer of on slave CPUs to guest VM
KVM: Enable direct EOI for directly routed interrupts to guests
KVM: route assigned devices' MSI/MSI-X directly to guests on slave CPUs
KVM: add kvm_arch_vcpu_prevent_run to prevent VM ENTER when NMI is received
KVM: vmx: Add definitions PIN_BASED_PREEMPTION_TIMER
KVM: add tracepoint on enabling/disabling direct interrupt delivery
KVM: Directly handle interrupts by guests without VM EXIT on slave CPUs x86/apic: IRQ vector remapping on slave for slave CPUs x86/apic: Enable external interrupt routing to slave CPUs
KVM: no exiting from guest when slave CPU halted
KVM: proxy slab operations for slave CPUs on online CPUs
KVM: Go back to online CPU on VM exit by external interrupt
KVM: Add KVM_GET_SLAVE_CPU and KVM_SET_SLAVE_CPU to vCPU ioctl
KVM: handle page faults of slave guests on online CPUs
KVM: Add facility to run guests on slave CPUs
KVM: Enable/Disable virtualization on slave CPUs are activated/dying
x86: Avoid RCU warnings on slave CPUs
x86: Support hrtimer on slave CPUs
x86: Add a facility to use offlined CPUs as slave CPUs
x86: Split memory hotplug function from cpu_up() as cpu_memory_up()

 arch/x86/Kconfig                      |   10 +
 arch/x86/include/asm/apic.h           |   10 +
 arch/x86/include/asm/irq.h            |   15 +
 arch/x86/include/asm/kvm_host.h       |   59 +++++
 arch/x86/include/asm/tlbflush.h       |    5 
 arch/x86/include/asm/vmx.h            |    3 
 arch/x86/kernel/apic/apic.c           |   11 +
 arch/x86/kernel/apic/io_apic.c        |  111 ++++++++-
 arch/x86/kernel/apic/x2apic_cluster.c |    8 -
 arch/x86/kernel/cpu/common.c          |    5 
 arch/x86/kernel/smp.c                 |    2 
 arch/x86/kernel/smpboot.c             |  264 ++++++++++++++++++++++-
 arch/x86/kvm/irq.c                    |  136 ++++++++++++
 arch/x86/kvm/lapic.c                  |   56 +++++
 arch/x86/kvm/lapic.h                  |    2 
 arch/x86/kvm/mmu.c                    |   63 ++++-
 arch/x86/kvm/mmu.h                    |    4 
 arch/x86/kvm/trace.h                  |   19 ++
 arch/x86/kvm/vmx.c                    |  180 +++++++++++++++
 arch/x86/kvm/x86.c                    |  387 +++++++++++++++++++++++++++++++--
 arch/x86/kvm/x86.h                    |    9 +
 arch/x86/mm/tlb.c                     |   94 ++++++++
 drivers/iommu/intel_irq_remapping.c   |   32 ++-
 include/linux/cpu.h                   |   36 +++
 include/linux/cpumask.h               |   26 ++
 include/linux/kvm.h                   |    4 
 include/linux/kvm_host.h              |    2 
 kernel/cpu.c                          |   83 +++++--
 kernel/hrtimer.c                      |   14 +
 kernel/irq/manage.c                   |    4 
 kernel/irq/migration.c                |    2 
 kernel/irq/proc.c                     |    2 
 kernel/rcutree.c                      |   14 +
 kernel/smp.c                          |    9 +
 virt/kvm/assigned-dev.c               |    8 +
 virt/kvm/async_pf.c                   |   17 +
 virt/kvm/kvm_main.c                   |   32 +++
 37 files changed, 1629 insertions(+), 109 deletions(-)

* Patch for qemu-kvm-1.0

qemu-kvm-1.0/linux-headers/linux/kvm.h +5 -0

diff -Narup a/qemu-kvm-1.0/linux-headers/linux/kvm.h b/qemu-kvm-1.0/linux-headers/linux/kvm.h
@@ -558,6 +558,7 @@ struct kvm_ppc_pvinfo {
 #define KVM_CAP_PPC_PAPR 68
 #define KVM_CAP_SW_TLB 69
 #define KVM_CAP_ONE_REG 70
+#define KVM_CAP_SLAVE_CPU 81
 
 #ifdef KVM_CAP_IRQ_ROUTING

@@ -811,6 +812,10 @@ struct kvm_one_reg {
 /* Available with KVM_CAP_ONE_REG */
 #define KVM_GET_ONE_REG		  _IOWR(KVMIO, 0xab, struct kvm_one_reg)
 #define KVM_SET_ONE_REG		  _IOW(KVMIO,  0xac, struct kvm_one_reg)
+/* Available with KVM_CAP_SLAVE_CPU */
+#define KVM_GET_SLAVE_CPU	  _IO(KVMIO,  0xae)
+#define KVM_SET_SLAVE_CPU	  _IO(KVMIO,  0xaf)
+
 
 #define KVM_DEV_ASSIGN_ENABLE_IOMMU	(1 << 0)

qemu-kvm-1.0/qemu-kvm-x86.c +16 -0

diff -Narup a/qemu-kvm-1.0/qemu-kvm-x86.c b/qemu-kvm-1.0/qemu-kvm-x86.c
@@ -139,12 +139,28 @@ static int kvm_enable_tpr_access_reporti
     return kvm_vcpu_ioctl(env, KVM_TPR_ACCESS_REPORTING, &tac);
 }
 
+static int kvm_set_slave_cpu(CPUState *env)
+{
+    int r, slave = env->cpu_index == 0 ? 2 : env->cpu_index == 1 ? 3 : -1;
+
+    r = kvm_ioctl(env->kvm_state, KVM_CHECK_EXTENSION, KVM_CAP_SLAVE_CPU);
+    if (r <= 0) {
+        return -ENOSYS;
+    }
+    r = kvm_vcpu_ioctl(env, KVM_SET_SLAVE_CPU, slave);
+    if (r < 0)
+        perror("kvm_set_slave_cpu");
+    return r;
+}
+
 static int _kvm_arch_init_vcpu(CPUState *env)
 {
     kvm_arch_reset_vcpu(env);
 
     kvm_enable_tpr_access_reporting(env);
 
+    kvm_set_slave_cpu(env);
+
     return kvm_update_ioport_access(env);
 }

Tomoki Sekiyama

author

2012-09-06 11:27 UTC #1 lore raw

RFC PATCH v2 1/21 x86: Split memory hotplug function from cpu_up() as cpu_memory_up()

Split memory hotplug function from cpu_up() as cpu_memory_up(), which will be used for assigning memory area to off-lined cpus at following patch in this series.

Signed-off-by Tomoki Sekiyama <[email protected]>

Cc Avi Kivity <[email protected]>

Cc Marcelo Tosatti <[email protected]>

Cc Thomas Gleixner <[email protected]>

Cc Ingo Molnar <[email protected]>

Cc "H. Peter Anvin" <[email protected]>

---

 include/linux/cpu.h |    9 +++++++++
 kernel/cpu.c        |   46 +++++++++++++++++++++++++++-------------------
 2 files changed, 36 insertions(+), 19 deletions(-)

include/linux/cpu.h +9 -0

index ce7a074..8395ac9 100644
@@ -146,6 +146,15 @@ void notify_cpu_starting(unsigned int cpu);
 extern void cpu_maps_update_begin(void);
 extern void cpu_maps_update_done(void);
 
+#ifdef	CONFIG_MEMORY_HOTPLUG
+extern int cpu_memory_up(unsigned int cpu);
+#else
+static inline int cpu_memory_up(unsigned int cpu)
+{
+	return 0;
+}
+#endif
+
 #else	/* CONFIG_SMP */
 
 #define cpu_notifier(fn, pri)	do { (void)(fn); } while (0)

kernel/cpu.c +27 ~~-19~~

67 lines not on this page · raw message

Avi Kivity

2012-09-06 11:32 UTC #2 lore raw

Re: [RFC v2 PATCH 01/21] x86: Split memory hotplug function from cpu_up() as cpu_memory_up()

Quoted 4 lines · Tomoki Sekiyama

Split memory hotplug function from cpu_up() as cpu_memory_up(), which will be used for assigning memory area to off-lined cpus at following patch in this series.

Can post a summary containing both the general outline for people reading this for the first time, or who have forgotten it, and the list of changes from v1?

Signature · 1 line

error compiling committee.c: too many arguments to function

Avi Kivity

2012-09-06 11:32 UTC #3 lore raw

Re: [RFC v2 PATCH 01/21] x86: Split memory hotplug function from cpu_up() as cpu_memory_up()

Quoted 10 lines · Avi Kivity

Quoted 4 lines · Tomoki Sekiyama
Split memory hotplug function from cpu_up() as cpu_memory_up(), which will be used for assigning memory area to off-lined cpus at following patch in this series.
Can post a summary containing both the general outline for people reading this for the first time, or who have forgotten it, and the list of changes from v1?

Never mind, I see it was posted, just that I wasn't copied on it.

Signature · 1 line

error compiling committee.c: too many arguments to function

Tomoki Sekiyama

author

2012-09-06 11:27 UTC #4 lore raw

RFC PATCH v2 2/21 x86: Add a facility to use offlined CPUs as slave CPUs

In reply to Tomoki Sekiyama

Add a facility of using offlined CPUs as slave CPUs. Slave CPUs are specialized to exclusively run functions specified by online CPUs, which do not run user processes.

To use this feature, build the kernel with CONFIG_SLAVE_CPU=y.

A slave CPU is launched by calling cpu_slave_up() when the CPU is offlined. Once launched, the slave CPU waits for IPI in the idle thread context. Users of the slave CPU can run specific kernel function by sending IPI using slave_cpu_call_function().

When cpu_slave_down() is called, the slave cpu will go offline state again.

Cpumask cpu_slave_mask' is provided to manage whether CPU is slave. In addition, cpu_online_or_slave_mask' is also provided for convenence of APIC handling, etc.

Signed-off-by Tomoki Sekiyama <[email protected]>

Cc Avi Kivity <[email protected]>

Cc Marcelo Tosatti <[email protected]>

Cc Thomas Gleixner <[email protected]>

Cc Ingo Molnar <[email protected]>

Cc "H. Peter Anvin" <[email protected]>

---

 arch/x86/Kconfig             |   10 ++
 arch/x86/include/asm/cpu.h   |   11 ++
 arch/x86/kernel/cpu/common.c |    5 +
 arch/x86/kernel/smpboot.c    |  190 ++++++++++++++++++++++++++++++++++++++++--
 include/linux/cpumask.h      |   26 ++++++
 kernel/cpu.c                 |   37 ++++++++
 kernel/smp.c                 |    8 +-
 7 files changed, 275 insertions(+), 12 deletions(-)

arch/x86/Kconfig +10 -0

index 8ec3a1a..106c958 100644
@@ -1678,6 +1678,16 @@ config HOTPLUG_CPU
 	    automatically on SMP systems. )
 	  Say N if you want to disable CPU hotplug.
 
+config SLAVE_CPU
+	bool "Support for slave CPUs (EXPERIMENTAL)"
+	depends on EXPERIMENTAL && HOTPLUG_CPU
+	---help---
+	  Say Y here to allow use some of CPUs as slave processors.
+	  Slave CPUs are controlled from another CPU and do some tasks
+	  and cannot run user processes. Slave processors can be
+	  specified through /sys/devices/system/cpu.
+	  Say N if you want to disable slave CPU support.
+
 config COMPAT_VDSO
 	def_bool y
 	prompt "Compat VDSO support"

arch/x86/include/asm/cpu.h +11 -0

index 4564c8e..b7ace52 100644
@@ -30,6 +30,17 @@ extern int arch_register_cpu(int num);
 extern void arch_unregister_cpu(int);
 #endif
 
+#ifdef CONFIG_SLAVE_CPU
+#define CPU_SLAVE_UP_PREPARE	0xff00
+#define CPU_SLAVE_UP		0xff01
+#define CPU_SLAVE_DEAD		0xff02
+
+extern int slave_cpu_up(unsigned int cpu);
+extern int slave_cpu_down(unsigned int cpu);
+extern void slave_cpu_call_function(unsigned int cpu,
+				    void (*f)(void *), void *arg);
+#endif
+
 DECLARE_PER_CPU(int, cpu_state);
 
 int mwait_usable(const struct cpuinfo_x86 *);

arch/x86/kernel/cpu/common.c +4 -1

index a5fbc3c..ab7f9a7 100644
@@ -913,7 +913,10 @@ static void __cpuinit identify_cpu(struct cpuinfo_x86 *c)
 	}
 
 	/* Init Machine Check Exception if available. */
-	mcheck_cpu_init(c);
+#ifdef CONFIG_SLAVE_CPU
+	if (per_cpu(cpu_state, smp_processor_id()) != CPU_SLAVE_UP_PREPARE)
+#endif
+		mcheck_cpu_init(c);
 
 	select_idle_routine(c);

arch/x86/kernel/smpboot.c +183 -7

258 lines not on this page · raw message

include/linux/cpumask.h +26 -0

index 0325602..f5fec4f 100644
@@ -81,6 +81,19 @@ extern const struct cpumask *const cpu_online_mask;
 extern const struct cpumask *const cpu_present_mask;
 extern const struct cpumask *const cpu_active_mask;
 
+/*
+ * cpu_online_or_slave_mask represents cpu_online_mask | cpu_slave_mask.
+ * This mask indicates the cpu can hanlde irq.
+ * if CONFIG_SLAVE_CPU is not defined, cpu_slave is defined as 0,
+ * and cpu_online_or_slave_mask is equals to cpu_online_mask.
+ */
+#ifdef CONFIG_SLAVE_CPU
+extern const struct cpumask *const cpu_slave_mask;
+extern const struct cpumask *const cpu_online_or_slave_mask;
+#else
+#define cpu_online_or_slave_mask cpu_online_mask
+#endif
+
 #if NR_CPUS > 1
 #define num_online_cpus()	cpumask_weight(cpu_online_mask)
 #define num_possible_cpus()	cpumask_weight(cpu_possible_mask)

@@ -101,6 +114,12 @@ extern const struct cpumask *const cpu_active_mask;
 #define cpu_active(cpu)		((cpu) == 0)
 #endif
 
+#if defined(CONFIG_SLAVE_CPU) && NR_CPUS > 1
+#define cpu_slave(cpu)		cpumask_test_cpu((cpu), cpu_slave_mask)
+#else
+#define cpu_slave(cpu)		0
+#endif
+
 /* verify cpu argument to cpumask_* operators */
 static inline unsigned int cpumask_check(unsigned int cpu)
 {

@@ -716,6 +735,13 @@ void init_cpu_present(const struct cpumask *src);
 void init_cpu_possible(const struct cpumask *src);
 void init_cpu_online(const struct cpumask *src);
 
+#ifdef CONFIG_SLAVE_CPU
+#define for_each_slave_cpu(cpu)  for_each_cpu((cpu), cpu_slave_mask)
+void set_cpu_slave(unsigned int cpu, bool slave);
+#else
+#define for_each_slave_cpu(cpu)  while (0)
+#endif
+
 /**
  * to_cpumask - convert an NR_CPUS bitmap to a struct cpumask *
  * @bitmap: the bitmap

kernel/cpu.c +36 -1

index 5df8f36..fecb394 100644
@@ -343,7 +343,7 @@ static int __cpuinit _cpu_up(unsigned int cpu, int tasks_frozen)
 	unsigned long mod = tasks_frozen ? CPU_TASKS_FROZEN : 0;
 	struct task_struct *idle;
 
-	if (cpu_online(cpu) || !cpu_present(cpu))
+	if (cpu_online(cpu) || cpu_slave(cpu) || !cpu_present(cpu))
 		return -EINVAL;
 
 	cpu_hotplug_begin();

@@ -685,6 +685,17 @@ static DECLARE_BITMAP(cpu_active_bits, CONFIG_NR_CPUS) __read_mostly;
 const struct cpumask *const cpu_active_mask = to_cpumask(cpu_active_bits);
 EXPORT_SYMBOL(cpu_active_mask);
 
+#ifdef CONFIG_SLAVE_CPU
+static DECLARE_BITMAP(cpu_slave_bits, CONFIG_NR_CPUS) __read_mostly;
+const struct cpumask *const cpu_slave_mask = to_cpumask(cpu_slave_bits);
+EXPORT_SYMBOL(cpu_slave_mask);
+
+static DECLARE_BITMAP(cpu_online_or_slave_bits, CONFIG_NR_CPUS) __read_mostly;
+const struct cpumask *const cpu_online_or_slave_mask =
+	to_cpumask(cpu_online_or_slave_bits);
+EXPORT_SYMBOL(cpu_online_or_slave_mask);
+#endif
+
 void set_cpu_possible(unsigned int cpu, bool possible)
 {
 	if (possible)

@@ -707,6 +718,13 @@ void set_cpu_online(unsigned int cpu, bool online)
 		cpumask_set_cpu(cpu, to_cpumask(cpu_online_bits));
 	else
 		cpumask_clear_cpu(cpu, to_cpumask(cpu_online_bits));
+
+#ifdef CONFIG_SLAVE_CPU
+	if (online)
+		cpumask_set_cpu(cpu, to_cpumask(cpu_online_or_slave_bits));
+	else
+		cpumask_clear_cpu(cpu, to_cpumask(cpu_online_or_slave_bits));
+#endif
 }
 
 void set_cpu_active(unsigned int cpu, bool active)

@@ -717,6 +735,19 @@ void set_cpu_active(unsigned int cpu, bool active)
 		cpumask_clear_cpu(cpu, to_cpumask(cpu_active_bits));
 }
 
+#ifdef CONFIG_SLAVE_CPU
+void set_cpu_slave(unsigned int cpu, bool slave)
+{
+	if (slave) {
+		cpumask_set_cpu(cpu, to_cpumask(cpu_slave_bits));
+		cpumask_set_cpu(cpu, to_cpumask(cpu_online_or_slave_bits));
+	} else {
+		cpumask_clear_cpu(cpu, to_cpumask(cpu_slave_bits));
+		cpumask_clear_cpu(cpu, to_cpumask(cpu_online_or_slave_bits));
+	}
+}
+#endif
+
 void init_cpu_present(const struct cpumask *src)
 {
 	cpumask_copy(to_cpumask(cpu_present_bits), src);

@@ -730,4 +761,8 @@ void init_cpu_possible(const struct cpumask *src)
 void init_cpu_online(const struct cpumask *src)
 {
 	cpumask_copy(to_cpumask(cpu_online_bits), src);
+
+#ifdef CONFIG_SLAVE_CPU
+	cpumask_copy(to_cpumask(cpu_online_or_slave_bits), src);
+#endif
 }

kernel/smp.c +5 -3

index 29dd40a..fda7a8d 100644
@@ -177,7 +177,7 @@ void generic_smp_call_function_interrupt(void)
 	/*
 	 * Shouldn't receive this interrupt on a cpu that is not yet online.
 	 */
-	WARN_ON_ONCE(!cpu_online(cpu));
+	WARN_ON_ONCE(!cpu_online(cpu) && !cpu_slave(cpu));
 
 	/*
 	 * Ensure entry is visible on call_function_queue after we have

@@ -257,7 +257,8 @@ void generic_smp_call_function_single_interrupt(void)
 	/*
 	 * Shouldn't receive this interrupt on a cpu that is not yet online.
 	 */
-	WARN_ON_ONCE(!cpu_online(smp_processor_id()));
+	WARN_ON_ONCE(!cpu_online(smp_processor_id()) &&
+		     !cpu_slave(smp_processor_id()));
 
 	raw_spin_lock(&q->lock);
 	list_replace_init(&q->list, &list);

@@ -326,7 +327,8 @@ int smp_call_function_single(int cpu, smp_call_func_t func, void *info,
 		func(info);
 		local_irq_restore(flags);
 	} else {
-		if ((unsigned)cpu < nr_cpu_ids && cpu_online(cpu)) {
+		if ((unsigned)cpu < nr_cpu_ids &&
+		    (cpu_online(cpu) || cpu_slave(cpu))) {
 			struct call_single_data *data = &d;
 
 			if (!wait)

Tomoki Sekiyama

author

2012-09-06 11:27 UTC #5 lore raw

RFC PATCH v2 3/21 x86: Support hrtimer on slave CPUs

In reply to Tomoki Sekiyama

Adds a facility to use hrtimer on slave CPUs.

To initialize hrtimer when slave CPUs are activated, and to shutdown hrtimer when slave CPUs are stopped, this patch adds the slave cpu notifier chain, which call registered callbacks when slave CPUs are up, dying, and died.

The registered callbacks are called with CPU_SLAVE_UP when a slave CPU becomes active. When the slave CPU is stopped, callbacks are called with CPU_SLAVE_DYING on slave CPUs, and with CPU_SLAVE_DEAD on online CPUs.

Signed-off-by Tomoki Sekiyama <[email protected]>

Cc Avi Kivity <[email protected]>

Cc Marcelo Tosatti <[email protected]>

Cc Thomas Gleixner <[email protected]>

Cc Ingo Molnar <[email protected]>

Cc "H. Peter Anvin" <[email protected]>

---

 arch/x86/include/asm/cpu.h |   11 -----------
 arch/x86/kernel/smpboot.c  |   37 +++++++++++++++++++++++++++++++++++++
 include/linux/cpu.h        |   22 ++++++++++++++++++++++
 kernel/hrtimer.c           |   14 ++++++++++++++
 4 files changed, 73 insertions(+), 11 deletions(-)

arch/x86/include/asm/cpu.h +0 ~~-11~~

index b7ace52..4564c8e 100644
@@ -30,17 +30,6 @@ extern int arch_register_cpu(int num);
 extern void arch_unregister_cpu(int);
 #endif
 
-#ifdef CONFIG_SLAVE_CPU
-#define CPU_SLAVE_UP_PREPARE	0xff00
-#define CPU_SLAVE_UP		0xff01
-#define CPU_SLAVE_DEAD		0xff02
-
-extern int slave_cpu_up(unsigned int cpu);
-extern int slave_cpu_down(unsigned int cpu);
-extern void slave_cpu_call_function(unsigned int cpu,
-				    void (*f)(void *), void *arg);
-#endif
-
 DECLARE_PER_CPU(int, cpu_state);
 
 int mwait_usable(const struct cpuinfo_x86 *);

arch/x86/kernel/smpboot.c +37 -0

index b9e1297..e8cfe377 100644
@@ -127,6 +127,36 @@ EXPORT_PER_CPU_SYMBOL(cpu_info);
 
 atomic_t init_deasserted;
 
+static void __ref remove_cpu_from_maps(int cpu);
+
+
+#ifdef CONFIG_SLAVE_CPU
+/* Notify slave cpu up and down */
+static RAW_NOTIFIER_HEAD(slave_cpu_chain);
+
+int register_slave_cpu_notifier(struct notifier_block *nb)
+{
+	return raw_notifier_chain_register(&slave_cpu_chain, nb);
+}
+EXPORT_SYMBOL(register_slave_cpu_notifier);
+
+void unregister_slave_cpu_notifier(struct notifier_block *nb)
+{
+	raw_notifier_chain_unregister(&slave_cpu_chain, nb);
+}
+EXPORT_SYMBOL(unregister_slave_cpu_notifier);
+
+static int slave_cpu_notify(unsigned long val, int cpu)
+{
+	int ret;
+
+	ret = __raw_notifier_call_chain(&slave_cpu_chain, val,
+					(void *)(long)cpu, -1, NULL);
+
+	return notifier_to_errno(ret);
+}
+#endif
+
 /*
  * Report back to the Boot Processor.
  * Running on AP.

@@ -307,6 +337,7 @@ notrace static void __cpuinit start_slave_cpu(void *unused)
 	 * most necessary things.
 	 */
 	cpu_init();
+	x86_cpuinit.early_percpu_clock_init();
 	preempt_disable();
 	smp_callin(0);

@@ -333,10 +364,14 @@ notrace static void __cpuinit start_slave_cpu(void *unused)
 	/* to prevent fake stack check failure */
 	boot_init_stack_canary();
 
+	x86_cpuinit.setup_percpu_clockev();
+	tick_nohz_idle_enter();
+
 	/* announce slave CPU started */
 	pr_info("Slave CPU %d is up\n", cpu);
 	__this_cpu_write(cpu_state, CPU_SLAVE_UP);
 	set_cpu_slave(cpu, true);
+	slave_cpu_notify(CPU_SLAVE_UP, cpu);
 	wmb();
 
 	/* wait for slave_cpu_call_function or slave_cpu_down */

@@ -363,6 +398,7 @@ notrace static void __cpuinit start_slave_cpu(void *unused)
 	local_irq_disable();
 	native_cpu_disable();
 	set_cpu_slave(cpu, false);
+	slave_cpu_notify(CPU_SLAVE_DYING, cpu);
 	native_play_dead();
 }
 #endif

@@ -995,6 +1031,7 @@ int slave_cpu_down(unsigned int cpu)
 		return -EBUSY;
 	}
 
+	slave_cpu_notify(CPU_SLAVE_DEAD, cpu);
 	return 0;
 }
 EXPORT_SYMBOL(slave_cpu_down);

include/linux/cpu.h +22 -0

index 8395ac9..f1aa3cc 100644
@@ -221,4 +221,26 @@ static inline int disable_nonboot_cpus(void) { return 0; }
 static inline void enable_nonboot_cpus(void) {}
 #endif /* !CONFIG_PM_SLEEP_SMP */
 
+#ifdef CONFIG_SLAVE_CPU
+int register_slave_cpu_notifier(struct notifier_block *nb);
+void unregister_slave_cpu_notifier(struct notifier_block *nb);
+
+/* CPU notifier constants for slave processors */
+#define CPU_SLAVE_UP_PREPARE	0xff00
+#define CPU_SLAVE_UP		0xff01
+#define CPU_SLAVE_DEAD		0xff02
+#define CPU_SLAVE_DYING		0xff03
+
+extern int slave_cpu_up(unsigned int cpu);
+extern int slave_cpu_down(unsigned int cpu);
+extern void slave_cpu_call_function(unsigned int cpu,
+				    void (*f)(void *), void *arg);
+#else
+static inline int register_slave_cpu_notifier(struct notifier_block *nb)
+{
+	return 0;
+}
+static inline void unregister_slave_cpu_notifier(struct notifier_block *nb) {}
+#endif
+
 #endif /* _LINUX_CPU_H_ */

kernel/hrtimer.c +14 -0

index 6db7a5e..e899a2c 100644
@@ -1727,16 +1727,25 @@ static int __cpuinit hrtimer_cpu_notify(struct notifier_block *self,
 
 	case CPU_UP_PREPARE:
 	case CPU_UP_PREPARE_FROZEN:
+#ifdef CONFIG_SLAVE_CPU
+	case CPU_SLAVE_UP:
+#endif
 		init_hrtimers_cpu(scpu);
 		break;
 
 #ifdef CONFIG_HOTPLUG_CPU
 	case CPU_DYING:
 	case CPU_DYING_FROZEN:
+#ifdef CONFIG_SLAVE_CPU
+	case CPU_SLAVE_DYING:
+#endif
 		clockevents_notify(CLOCK_EVT_NOTIFY_CPU_DYING, &scpu);
 		break;
 	case CPU_DEAD:
 	case CPU_DEAD_FROZEN:
+#ifdef CONFIG_SLAVE_CPU
+	case CPU_SLAVE_DEAD:
+#endif
 	{
 		clockevents_notify(CLOCK_EVT_NOTIFY_CPU_DEAD, &scpu);
 		migrate_hrtimers(scpu);

@@ -1755,11 +1764,16 @@ static struct notifier_block __cpuinitdata hrtimers_nb = {
 	.notifier_call = hrtimer_cpu_notify,
 };
 
+static struct notifier_block __cpuinitdata hrtimers_slave_nb = {
+	.notifier_call = hrtimer_cpu_notify,
+};
+
 void __init hrtimers_init(void)
 {
 	hrtimer_cpu_notify(&hrtimers_nb, (unsigned long)CPU_UP_PREPARE,
 			  (void *)(long)smp_processor_id());
 	register_cpu_notifier(&hrtimers_nb);
+	register_slave_cpu_notifier(&hrtimers_slave_nb);
 #ifdef CONFIG_HIGH_RES_TIMERS
 	open_softirq(HRTIMER_SOFTIRQ, run_hrtimer_softirq);
 #endif

Tomoki Sekiyama

author

2012-09-06 11:27 UTC #6 lore raw

RFC PATCH v2 4/21 x86: Avoid RCU warnings on slave CPUs

In reply to Tomoki Sekiyama

Initialize rcu related variables to avoid warnings about RCU usage while slave CPUs is running specified functions. Also notify RCU subsystem before the slave CPU is entered into idle state.

Signed-off-by Tomoki Sekiyama <[email protected]>

Cc Avi Kivity <[email protected]>

Cc Marcelo Tosatti <[email protected]>

Cc Thomas Gleixner <[email protected]>

Cc Ingo Molnar <[email protected]>

Cc "H. Peter Anvin" <[email protected]>

---

 arch/x86/kernel/smpboot.c |    4 ++++
 kernel/rcutree.c          |   14 ++++++++++++++
 2 files changed, 18 insertions(+), 0 deletions(-)

arch/x86/kernel/smpboot.c +4 -0

index e8cfe377..45dfc1d 100644
@@ -382,6 +382,8 @@ notrace static void __cpuinit start_slave_cpu(void *unused)
 		f = per_cpu(slave_cpu_func, cpu);
 		per_cpu(slave_cpu_func, cpu).func = NULL;
 
+		rcu_note_context_switch(cpu);
+
 		if (!f.func) {
 			native_safe_halt();
 			continue;

@@ -1005,6 +1007,8 @@ int __cpuinit slave_cpu_up(unsigned int cpu)
 	if (IS_ERR(idle))
 		return PTR_ERR(idle);
 
+	slave_cpu_notify(CPU_SLAVE_UP_PREPARE, cpu);
+
 	ret = __native_cpu_up(cpu, idle, 1);
 
 	cpu_maps_update_done();

kernel/rcutree.c +14 -0

index f280e54..31a7c8c 100644
@@ -2589,6 +2589,9 @@ static int __cpuinit rcu_cpu_notify(struct notifier_block *self,
 	switch (action) {
 	case CPU_UP_PREPARE:
 	case CPU_UP_PREPARE_FROZEN:
+#ifdef CONFIG_SLAVE_CPU
+	case CPU_SLAVE_UP_PREPARE:
+#endif
 		rcu_prepare_cpu(cpu);
 		rcu_prepare_kthreads(cpu);
 		break;

@@ -2603,6 +2606,9 @@ static int __cpuinit rcu_cpu_notify(struct notifier_block *self,
 		break;
 	case CPU_DYING:
 	case CPU_DYING_FROZEN:
+#ifdef CONFIG_SLAVE_CPU
+	case CPU_SLAVE_DYING:
+#endif
 		/*
 		 * The whole machine is "stopped" except this CPU, so we can
 		 * touch any data without introducing corruption. We send the

@@ -2616,6 +2622,9 @@ static int __cpuinit rcu_cpu_notify(struct notifier_block *self,
 	case CPU_DEAD_FROZEN:
 	case CPU_UP_CANCELED:
 	case CPU_UP_CANCELED_FROZEN:
+#ifdef CONFIG_SLAVE_CPU
+	case CPU_SLAVE_DEAD:
+#endif
 		for_each_rcu_flavor(rsp)
 			rcu_cleanup_dead_cpu(cpu, rsp);
 		break;

@@ -2797,6 +2806,10 @@ static void __init rcu_init_geometry(void)
 	rcu_num_nodes -= n;
 }
 
+static struct notifier_block __cpuinitdata rcu_slave_nb = {
+	.notifier_call = rcu_cpu_notify,
+};
+
 void __init rcu_init(void)
 {
 	int cpu;

@@ -2814,6 +2827,7 @@ void __init rcu_init(void)
 	 * or the scheduler are operational.
 	 */
 	cpu_notifier(rcu_cpu_notify, 0);
+	register_slave_cpu_notifier(&rcu_slave_nb);
 	for_each_online_cpu(cpu)
 		rcu_cpu_notify(NULL, CPU_UP_PREPARE, (void *)(long)cpu);
 	check_cpu_stall_init();

Paul E. McKenney

2012-09-20 17:35 UTC #7 lore raw

Re: [RFC v2 PATCH 04/21] x86: Avoid RCU warnings on slave CPUs

Quoted 3 lines · Tomoki Sekiyama

Initialize rcu related variables to avoid warnings about RCU usage while slave CPUs is running specified functions. Also notify RCU subsystem before the slave CPU is entered into idle state.

Hello, Tomoki,

A few questions and comments interspersed below.

Thanx, Paul

Quoted 17 lines

Signed-off-by Tomoki Sekiyama <[email protected]>
Cc Avi Kivity <[email protected]>
Cc Marcelo Tosatti <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
---
 arch/x86/kernel/smpboot.c |    4 ++++
 kernel/rcutree.c          |   14 ++++++++++++++
 2 files changed, 18 insertions(+), 0 deletions(-)
arch/x86/kernel/smpboot.c +0 -0
index e8cfe377..45dfc1d 100644

arch/x86/kernel/smpboot.c notrace static void __cpuinit start_slave_cpu(void *unused) · line 386

Quoted 5 lines

arch/x86/kernel/smpboot.c +2 -0

@@ -382,6 +382,8 @@ notrace static void __cpuinit start_slave_cpu(void *unused)
 		f = per_cpu(slave_cpu_func, cpu);
 		per_cpu(slave_cpu_func, cpu).func = NULL;
 
+		rcu_note_context_switch(cpu);
+

Why not use rcu_idle_enter() and rcu_idle_exit()? These would tell RCU to ignore the slave CPU for the duration of its idle period. The way you have it, if a slave CPU stayed idle for too long, you would get RCU CPU stall warnings, and possibly system hangs as well.

Or is this being called from some task that is not the idle task? If so, you instead want the new rcu_user_enter() and rcu_user_exit() that are hopefully on their way into 3.7. Or maybe better, use a real idle task, so that idle_task(smp_processor_id()) returns true and RCU stops complaining. ;-)

Note that CPUs that RCU believes to be idle are not permitted to contain RCU read-side critical sections, which in turn means no entering the scheduler, no sleeping, and so on. There is an RCU_NONIDLE() macro to tell RCU to pay attention to the CPU only for the duration of the statement passed to RCU_NONIDLE, and there are also an _rcuidle variant of the tracing statement to allow tracing from idle.

Quoted 17 lines

arch/x86/kernel/smpboot.c +2 -0

 		if (!f.func) {
 			native_safe_halt();
 			continue;

@@ -1005,6 +1007,8 @@ int __cpuinit slave_cpu_up(unsigned int cpu)
 	if (IS_ERR(idle))
 		return PTR_ERR(idle);
 
+	slave_cpu_notify(CPU_SLAVE_UP_PREPARE, cpu);
+
 	ret = __native_cpu_up(cpu, idle, 1);
 
 	cpu_maps_update_done();

kernel/rcutree.c +0 -0

index f280e54..31a7c8c 100644

kernel/rcutree.c static int __cpuinit rcu_cpu_notify(struct notifier_block *self, · line 2594

Quoted 6 lines

kernel/rcutree.c +3 -0

@@ -2589,6 +2589,9 @@ static int __cpuinit rcu_cpu_notify(struct notifier_block *self,
 	switch (action) {
 	case CPU_UP_PREPARE:
 	case CPU_UP_PREPARE_FROZEN:
+#ifdef CONFIG_SLAVE_CPU
+	case CPU_SLAVE_UP_PREPARE:
+#endif

Why do you need #ifdef here? Why not define CPU_SLAVE_UP_PREPARE unconditionally? Then if CONFIG_SLAVE_CPU=n, rcu_cpu_notify() would never be invoked with CPU_SLAVE_UP_PREPARE, so no problems.

kernel/rcutree.c static int __cpuinit rcu_cpu_notify(struct notifier_block *self, · line 2611

Quoted 10 lines

kernel/rcutree.c +3 -0

 		rcu_prepare_cpu(cpu);
 		rcu_prepare_kthreads(cpu);
 		break;

@@ -2603,6 +2606,9 @@ static int __cpuinit rcu_cpu_notify(struct notifier_block *self,
 		break;
 	case CPU_DYING:
 	case CPU_DYING_FROZEN:
+#ifdef CONFIG_SLAVE_CPU
+	case CPU_SLAVE_DYING:
+#endif

Same here.

kernel/rcutree.c static int __cpuinit rcu_cpu_notify(struct notifier_block *self, · line 2627

Quoted 10 lines

kernel/rcutree.c +3 -0

 		/*
 		 * The whole machine is "stopped" except this CPU, so we can
 		 * touch any data without introducing corruption. We send the

@@ -2616,6 +2622,9 @@ static int __cpuinit rcu_cpu_notify(struct notifier_block *self,
 	case CPU_DEAD_FROZEN:
 	case CPU_UP_CANCELED:
 	case CPU_UP_CANCELED_FROZEN:
+#ifdef CONFIG_SLAVE_CPU
+	case CPU_SLAVE_DEAD:
+#endif

And here.

Quoted 30 lines

kernel/rcutree.c +5 -1

 		for_each_rcu_flavor(rsp)
 			rcu_cleanup_dead_cpu(cpu, rsp);
 		break;

@@ -2797,6 +2806,10 @@ static void __init rcu_init_geometry(void)
 	rcu_num_nodes -= n;
 }
 
+static struct notifier_block __cpuinitdata rcu_slave_nb = {
+	.notifier_call = rcu_cpu_notify,
+};
+
 void __init rcu_init(void)
 {
 	int cpu;

@@ -2814,6 +2827,7 @@ void __init rcu_init(void)
 	 * or the scheduler are operational.
 	 */
 	cpu_notifier(rcu_cpu_notify, 0);
+	register_slave_cpu_notifier(&rcu_slave_nb);
 	for_each_online_cpu(cpu)
 		rcu_cpu_notify(NULL, CPU_UP_PREPARE, (void *)(long)cpu);
 	check_cpu_stall_init();
 
 
--

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Tomoki Sekiyama

author

2012-09-28 08:10 UTC #8 lore raw

Re: Re: [RFC v2 PATCH 04/21] x86: Avoid RCU warnings on slave CPUs

Hi Paul,

Thank you for your comments, and sorry for my late reply.

Quoted 17 lines · Paul E. McKenney

Quoted 3 lines · Tomoki Sekiyama
Initialize rcu related variables to avoid warnings about RCU usage while slave CPUs is running specified functions. Also notify RCU subsystem before the slave CPU is entered into idle state.
Hello, Tomoki,
A few questions and comments interspersed below.
Quoted 9 lines
<snip>
arch/x86/kernel/smpboot.c +0 -0
index e8cfe377..45dfc1d 100644
@@ -382,6 +382,8 @@ notrace static void __cpuinit start_slave_cpu(void *unused)
 		f = per_cpu(slave_cpu_func, cpu);
 		per_cpu(slave_cpu_func, cpu).func = NULL;
 

Quoted 7 lines

arch/x86/kernel/smpboot.c notrace static void __cpuinit start_slave_cpu(void *unused) · line 386
Quoted 2 lines
arch/x86/kernel/smpboot.c +2 -0
+		rcu_note_context_switch(cpu);
+
Why not use rcu_idle_enter() and rcu_idle_exit()? These would tell RCU to ignore the slave CPU for the duration of its idle period. The way you have it, if a slave CPU stayed idle for too long, you would get RCU CPU stall warnings, and possibly system hangs as well.

That's true, rcu_idle_enter() and rcu_idle_exit() should be used when the slave cpu is idle. Thanks.

Quoted 6 lines

Or is this being called from some task that is not the idle task? If so, you instead want the new rcu_user_enter() and rcu_user_exit() that are hopefully on their way into 3.7. Or maybe better, use a real idle task, so that idle_task(smp_processor_id()) returns true and RCU stops complaining. ;-)

Quoted 6 lines

Note that CPUs that RCU believes to be idle are not permitted to contain RCU read-side critical sections, which in turn means no entering the scheduler, no sleeping, and so on. There is an RCU_NONIDLE() macro to tell RCU to pay attention to the CPU only for the duration of the statement passed to RCU_NONIDLE, and there are also an _rcuidle variant of the tracing statement to allow tracing from idle.

This was for KVM is called as `func', which contains RCU read-side critical sections, and rcu_virt_note_context_switch() (that is rcu_note_context_switch(cpu)) before entering guest.
Maybe it should be replaced by rcu_user_enter() and rcu_user_exit() in the future.

Quoted 5 lines

kernel/rcutree.c +0 -0

@@ -2589,6 +2589,9 @@ static int __cpuinit rcu_cpu_notify(struc tnotifier_block *self,
 	switch (action) {
 	case CPU_UP_PREPARE:

Quoted 8 lines

kernel/rcutree.c static int __cpuinit rcu_cpu_notify(struct notifier_block *self, · line 2594
Quoted 4 lines
kernel/rcutree.c +3 -0
 	case CPU_UP_PREPARE_FROZEN:
+#ifdef CONFIG_SLAVE_CPU
+	case CPU_SLAVE_UP_PREPARE:
+#endif
Why do you need #ifdef here? Why not define CPU_SLAVE_UP_PREPARE unconditionally? Then if CONFIG_SLAVE_CPU=n, rcu_cpu_notify() would never be invoked with CPU_SLAVE_UP_PREPARE, so no problems.

Agreed. That will make the code simpler.

Thank you again,

Signature · 3 lines

Tomoki Sekiyama <[email protected]>
Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory

Tomoki Sekiyama

author

2012-09-06 11:27 UTC #9 lore raw

RFC PATCH v2 5/21 KVM: Enable/Disable virtualization on slave CPUs are activated/dying

In reply to Tomoki Sekiyama

Enable virtualization when slave CPUs are activated, and disable when the CPUs are dying using slave CPU notifier call chain.

In x86, TSC kHz must also be initialized by tsc_khz_changed when the new slave CPUs are activated.

Signed-off-by Tomoki Sekiyama <[email protected]>

Cc Avi Kivity <[email protected]>

Cc Marcelo Tosatti <[email protected]>

Cc Thomas Gleixner <[email protected]>

Cc Ingo Molnar <[email protected]>

Cc "H. Peter Anvin" <[email protected]>

---

 arch/x86/kvm/x86.c  |   16 ++++++++++++++++
 virt/kvm/kvm_main.c |   30 ++++++++++++++++++++++++++++--
 2 files changed, 44 insertions(+), 2 deletions(-)

arch/x86/kvm/x86.c +16 -0

index 148ed66..7501cc4 100644
@@ -61,6 +61,7 @@
 #include <asm/xcr.h>
 #include <asm/pvclock.h>
 #include <asm/div64.h>
+#include <asm/cpu.h>
 
 #define MAX_IO_MSRS 256
 #define KVM_MAX_MCE_BANKS 32

@@ -4782,9 +4783,15 @@ static int kvmclock_cpu_notifier(struct notifier_block *nfb,
 	switch (action) {
 		case CPU_ONLINE:
 		case CPU_DOWN_FAILED:
+#ifdef CONFIG_SLAVE_CPU
+		case CPU_SLAVE_UP:
+#endif
 			smp_call_function_single(cpu, tsc_khz_changed, NULL, 1);
 			break;
 		case CPU_DOWN_PREPARE:
+#ifdef CONFIG_SLAVE_CPU
+		case CPU_SLAVE_DYING:
+#endif
 			smp_call_function_single(cpu, tsc_bad, NULL, 1);
 			break;
 	}

@@ -4796,12 +4803,18 @@ static struct notifier_block kvmclock_cpu_notifier_block = {
 	.priority = -INT_MAX
 };
 
+static struct notifier_block kvmclock_slave_cpu_notifier_block = {
+	.notifier_call  = kvmclock_cpu_notifier,
+	.priority = -INT_MAX
+};
+
 static void kvm_timer_init(void)
 {
 	int cpu;
 
 	max_tsc_khz = tsc_khz;
 	register_hotcpu_notifier(&kvmclock_cpu_notifier_block);
+	register_slave_cpu_notifier(&kvmclock_slave_cpu_notifier_block);
 	if (!boot_cpu_has(X86_FEATURE_CONSTANT_TSC)) {
 #ifdef CONFIG_CPU_FREQ
 		struct cpufreq_policy policy;

@@ -4818,6 +4831,8 @@ static void kvm_timer_init(void)
 	pr_debug("kvm: max_tsc_khz = %ld\n", max_tsc_khz);
 	for_each_online_cpu(cpu)
 		smp_call_function_single(cpu, tsc_khz_changed, NULL, 1);
+	for_each_slave_cpu(cpu)
+		smp_call_function_single(cpu, tsc_khz_changed, NULL, 1);
 }
 
 static DEFINE_PER_CPU(struct kvm_vcpu *, current_vcpu);

@@ -4943,6 +4958,7 @@ void kvm_arch_exit(void)
 		cpufreq_unregister_notifier(&kvmclock_cpufreq_notifier_block,
 					    CPUFREQ_TRANSITION_NOTIFIER);
 	unregister_hotcpu_notifier(&kvmclock_cpu_notifier_block);
+	unregister_slave_cpu_notifier(&kvmclock_slave_cpu_notifier_block);
 	kvm_x86_ops = NULL;
 	kvm_mmu_module_exit();
 }

virt/kvm/kvm_main.c +28 -2

index d617f69..dc86e9a 100644
@@ -54,6 +54,9 @@
 #include <asm/io.h>
 #include <asm/uaccess.h>
 #include <asm/pgtable.h>
+#ifdef CONFIG_X86
+#include <asm/cpu.h>
+#endif
 
 #include "coalesced_mmio.h"
 #include "async_pf.h"

@@ -2336,11 +2339,17 @@ static void hardware_disable(void *junk)
 
 static void hardware_disable_all_nolock(void)
 {
+	int cpu;
+
 	BUG_ON(!kvm_usage_count);
 
 	kvm_usage_count--;
-	if (!kvm_usage_count)
+	if (!kvm_usage_count) {
 		on_each_cpu(hardware_disable_nolock, NULL, 1);
+		for_each_slave_cpu(cpu)
+			smp_call_function_single(cpu, hardware_disable_nolock,
+						 NULL, 1);
+	}
 }
 
 static void hardware_disable_all(void)

@@ -2353,6 +2362,7 @@ static void hardware_disable_all(void)
 static int hardware_enable_all(void)
 {
 	int r = 0;
+	int cpu;
 
 	raw_spin_lock(&kvm_lock);

@@ -2360,6 +2370,9 @@ static int hardware_enable_all(void)
 	if (kvm_usage_count == 1) {
 		atomic_set(&hardware_enable_failed, 0);
 		on_each_cpu(hardware_enable_nolock, NULL, 1);
+		for_each_slave_cpu(cpu)
+			smp_call_function_single(cpu, hardware_enable_nolock,
+						 NULL, 1);
 
 		if (atomic_read(&hardware_enable_failed)) {
 			hardware_disable_all_nolock();

@@ -2383,11 +2396,17 @@ static int kvm_cpu_hotplug(struct notifier_block *notifier, unsigned long val,
 	val &= ~CPU_TASKS_FROZEN;
 	switch (val) {
 	case CPU_DYING:
+#ifdef CONFIG_SLAVE_CPU
+	case CPU_SLAVE_DYING:
+#endif
 		printk(KERN_INFO "kvm: disabling virtualization on CPU%d\n",
 		       cpu);
 		hardware_disable(NULL);
 		break;
 	case CPU_STARTING:
+#ifdef CONFIG_SLAVE_CPU
+	case CPU_SLAVE_UP:
+#endif
 		printk(KERN_INFO "kvm: enabling virtualization on CPU%d\n",
 		       cpu);
 		hardware_enable(NULL);

@@ -2605,6 +2624,10 @@ static struct notifier_block kvm_cpu_notifier = {
 	.notifier_call = kvm_cpu_hotplug,
 };
 
+static struct notifier_block kvm_slave_cpu_notifier = {
+	.notifier_call = kvm_cpu_hotplug,
+};
+
 static int vm_stat_get(void *_offset, u64 *val)
 {
 	unsigned offset = (long)_offset;

@@ -2768,7 +2791,7 @@ int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align,
 	if (r < 0)
 		goto out_free_0a;
 
-	for_each_online_cpu(cpu) {
+	for_each_cpu(cpu, cpu_online_or_slave_mask) {
 		smp_call_function_single(cpu,
 				kvm_arch_check_processor_compat,
 				&r, 1);

@@ -2779,6 +2802,7 @@ int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align,
 	r = register_cpu_notifier(&kvm_cpu_notifier);
 	if (r)
 		goto out_free_2;
+	register_slave_cpu_notifier(&kvm_slave_cpu_notifier);
 	register_reboot_notifier(&kvm_reboot_notifier);
 
 	/* A kmem cache lets us meet the alignment requirements of fx_save. */

@@ -2826,6 +2850,7 @@ out_free:
 	kmem_cache_destroy(kvm_vcpu_cache);
 out_free_3:
 	unregister_reboot_notifier(&kvm_reboot_notifier);
+	unregister_slave_cpu_notifier(&kvm_slave_cpu_notifier);
 	unregister_cpu_notifier(&kvm_cpu_notifier);
 out_free_2:
 out_free_1:

@@ -2853,6 +2878,7 @@ void kvm_exit(void)
 	kvm_async_pf_deinit();
 	unregister_syscore_ops(&kvm_syscore_ops);
 	unregister_reboot_notifier(&kvm_reboot_notifier);
+	unregister_slave_cpu_notifier(&kvm_slave_cpu_notifier);
 	unregister_cpu_notifier(&kvm_cpu_notifier);
 	on_each_cpu(hardware_disable_nolock, NULL, 1);
 	kvm_arch_hardware_unsetup();

Tomoki Sekiyama

author

2012-09-06 11:27 UTC #10 lore raw

RFC PATCH v2 6/21 KVM: Add facility to run guests on slave CPUs

In reply to Tomoki Sekiyama

Add path to migrate execution of vcpu_enter_guest to a slave CPU when vcpu->arch.slave_cpu is set.

After moving to the slave CPU, it goes back to the online CPU when the guest is exited by reasons that cannot be handled by the slave CPU only (e.g. handling async page faults).

On migration, kvm_arch_vcpu_put_migrate is used to avoid using IPI to clear loaded vmcs from the old CPU. Instead, this immediately clears vmcs.

Signed-off-by Tomoki Sekiyama <[email protected]>

Cc Avi Kivity <[email protected]>

Cc Marcelo Tosatti <[email protected]>

Cc Thomas Gleixner <[email protected]>

Cc Ingo Molnar <[email protected]>

Cc "H. Peter Anvin" <[email protected]>

---

 arch/x86/include/asm/kvm_host.h |    9 ++
 arch/x86/kernel/smp.c           |    2 
 arch/x86/kvm/vmx.c              |   10 ++
 arch/x86/kvm/x86.c              |  189 ++++++++++++++++++++++++++++++++++-----
 arch/x86/kvm/x86.h              |    9 ++
 include/linux/kvm_host.h        |    1 
 kernel/smp.c                    |    1 
 virt/kvm/async_pf.c             |    9 +-
 virt/kvm/kvm_main.c             |    3 -
 9 files changed, 203 insertions(+), 30 deletions(-)

arch/x86/include/asm/kvm_host.h +9 -0

index 09155d6..72a0a64 100644
@@ -354,6 +354,14 @@ struct kvm_vcpu_arch {
 	u64 ia32_misc_enable_msr;
 	bool tpr_access_reporting;
 
+#ifdef CONFIG_SLAVE_CPU
+	/* slave cpu dedicated to this vcpu */
+	int slave_cpu;
+#endif
+
+	/* user process tied to each vcpu */
+	struct task_struct *task;
+
 	/*
 	 * Paging state of the vcpu
 	 *

@@ -617,6 +625,7 @@ struct kvm_x86_ops {
 	void (*prepare_guest_switch)(struct kvm_vcpu *vcpu);
 	void (*vcpu_load)(struct kvm_vcpu *vcpu, int cpu);
 	void (*vcpu_put)(struct kvm_vcpu *vcpu);
+	void (*vcpu_put_migrate)(struct kvm_vcpu *vcpu);
 
 	void (*set_guest_debug)(struct kvm_vcpu *vcpu,
 				struct kvm_guest_debug *dbg);

arch/x86/kernel/smp.c +1 -1

index 48d2b7d..a58dead 100644
@@ -119,7 +119,7 @@ static bool smp_no_nmi_ipi = false;
  */
 static void native_smp_send_reschedule(int cpu)
 {
-	if (unlikely(cpu_is_offline(cpu))) {
+	if (unlikely(cpu_is_offline(cpu) && !cpu_slave(cpu))) {
 		WARN_ON(1);
 		return;
 	}

arch/x86/kvm/vmx.c +9 -1

index c00f03d..c5db714 100644
@@ -1557,6 +1557,13 @@ static void vmx_vcpu_put(struct kvm_vcpu *vcpu)
 	}
 }
 
+static void vmx_vcpu_put_migrate(struct kvm_vcpu *vcpu)
+{
+	vmx_vcpu_put(vcpu);
+	__loaded_vmcs_clear(to_vmx(vcpu)->loaded_vmcs);
+	vcpu->cpu = -1;
+}
+
 static void vmx_fpu_activate(struct kvm_vcpu *vcpu)
 {
 	ulong cr0;

@@ -5017,7 +5024,7 @@ static int handle_invalid_guest_state(struct kvm_vcpu *vcpu)
 			return 0;
 		}
 
-		if (signal_pending(current))
+		if (signal_pending(vcpu->arch.task))
 			goto out;
 		if (need_resched())
 			schedule();

@@ -7263,6 +7270,7 @@ static struct kvm_x86_ops vmx_x86_ops = {
 	.prepare_guest_switch = vmx_save_host_state,
 	.vcpu_load = vmx_vcpu_load,
 	.vcpu_put = vmx_vcpu_put,
+	.vcpu_put_migrate = vmx_vcpu_put_migrate,
 
 	.set_guest_debug = set_guest_debug,
 	.get_msr = vmx_get_msr,

arch/x86/kvm/x86.c +164 ~~-25~~

270 lines not on this page · raw message

arch/x86/kvm/x86.h +9 -0

index 3d1134d..3ae93d4 100644
@@ -108,6 +108,15 @@ static inline bool vcpu_match_mmio_gpa(struct kvm_vcpu *vcpu, gpa_t gpa)
 	return false;
 }
 
+static inline bool vcpu_has_slave_cpu(struct kvm_vcpu *vcpu)
+{
+#ifdef CONFIG_SLAVE_CPU
+	return vcpu->arch.slave_cpu >= 0;
+#else
+	return 0;
+#endif
+}
+
 void kvm_before_handle_nmi(struct kvm_vcpu *vcpu);
 void kvm_after_handle_nmi(struct kvm_vcpu *vcpu);
 int kvm_inject_realmode_interrupt(struct kvm_vcpu *vcpu, int irq, int inc_eip);

include/linux/kvm_host.h +1 -0

index b70b48b..a60743f 100644
@@ -119,6 +119,7 @@ struct kvm_async_pf {
 };
 
 void kvm_clear_async_pf_completion_queue(struct kvm_vcpu *vcpu);
+int kvm_can_complete_async_pf(struct kvm_vcpu *vcpu);
 void kvm_check_async_pf_completion(struct kvm_vcpu *vcpu);
 int kvm_setup_async_pf(struct kvm_vcpu *vcpu, gva_t gva, gfn_t gfn,
 		       struct kvm_arch_async_pf *arch);

kernel/smp.c +1 -0

index fda7a8d..95a9a39 100644
@@ -431,6 +431,7 @@ void __smp_call_function_single(int cpu, struct call_single_data *data,
 	}
 	put_cpu();
 }
+EXPORT_SYMBOL(__smp_call_function_single);
 
 /**
  * smp_call_function_many(): Run a function on a set of other CPUs.

virt/kvm/async_pf.c +7 -2

index 74268b4..feb5e76 100644
@@ -120,12 +120,17 @@ void kvm_clear_async_pf_completion_queue(struct kvm_vcpu *vcpu)
 	vcpu->async_pf.queued = 0;
 }
 
+int kvm_can_complete_async_pf(struct kvm_vcpu *vcpu)
+{
+	return !list_empty_careful(&vcpu->async_pf.done) &&
+		kvm_arch_can_inject_async_page_present(vcpu);
+}
+
 void kvm_check_async_pf_completion(struct kvm_vcpu *vcpu)
 {
 	struct kvm_async_pf *work;
 
-	while (!list_empty_careful(&vcpu->async_pf.done) &&
-	      kvm_arch_can_inject_async_page_present(vcpu)) {
+	while (kvm_can_complete_async_pf(vcpu)) {
 		spin_lock(&vcpu->async_pf.lock);
 		work = list_first_entry(&vcpu->async_pf.done, typeof(*work),
 					      link);

virt/kvm/kvm_main.c +2 -1

index dc86e9a..debbee1 100644
@@ -1543,7 +1543,8 @@ void kvm_vcpu_kick(struct kvm_vcpu *vcpu)
 	}
 
 	me = get_cpu();
-	if (cpu != me && (unsigned)cpu < nr_cpu_ids && cpu_online(cpu))
+	if (cpu != me && (unsigned)cpu < nr_cpu_ids &&
+	    (cpu_online(cpu) || cpu_slave(cpu)))
 		if (kvm_arch_vcpu_should_kick(vcpu))
 			smp_send_reschedule(cpu);
 	put_cpu();

Tomoki Sekiyama

author

2012-09-06 11:27 UTC #11 lore raw

RFC PATCH v2 7/21 KVM: handle page faults of slave guests on online CPUs

In reply to Tomoki Sekiyama

Page faults which occured by the guest running on slave CPUs cannot be handled on slave CPUs because it is running on idle process context.

With this patch, the page fault happened in a slave CPU is notified to online CPU using struct kvm_access_fault, and is handled after the user-process for the guest is resumed on an online CPU.

Signed-off-by Tomoki Sekiyama <[email protected]>

Cc Avi Kivity <[email protected]>

Cc Marcelo Tosatti <[email protected]>

Cc Thomas Gleixner <[email protected]>

Cc Ingo Molnar <[email protected]>

Cc "H. Peter Anvin" <[email protected]>

---

 arch/x86/include/asm/kvm_host.h |   15 +++++++++++++++
 arch/x86/kvm/mmu.c              |   13 +++++++++++++
 arch/x86/kvm/x86.c              |   10 ++++++++++
 3 files changed, 38 insertions(+), 0 deletions(-)

arch/x86/include/asm/kvm_host.h +15 -0

index 72a0a64..8dc1a0a 100644
@@ -67,6 +67,11 @@
 
 #define UNMAPPED_GVA (~(gpa_t)0)
 
+#ifdef CONFIG_SLAVE_CPU
+/* Requests to handle VM exit on online cpu */
+#define KVM_REQ_HANDLE_PF	32
+#endif
+
 /* KVM Hugepage definitions for x86 */
 #define KVM_NR_PAGE_SIZES	3
 #define KVM_HPAGE_GFN_SHIFT(x)	(((x) - 1) * 9)

@@ -413,6 +418,16 @@ struct kvm_vcpu_arch {
 		u8 nr;
 	} interrupt;
 
+#ifdef CONFIG_SLAVE_CPU
+	/* used for recording page fault on offline CPU */
+	struct kvm_access_fault {
+		gva_t cr2;
+		u32 error_code;
+		void *insn;
+		int insn_len;
+	} page_fault;
+#endif
+
 	int halt_request; /* real mode on Intel only */
 
 	int cpuid_nent;

arch/x86/kvm/mmu.c +13 -0

index 7fbd0d2..eb1d397 100644
@@ -3946,6 +3946,19 @@ int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gva_t cr2, u32 error_code,
 	int r, emulation_type = EMULTYPE_RETRY;
 	enum emulation_result er;
 
+#ifdef CONFIG_SLAVE_CPU
+	if (cpu_slave(smp_processor_id())) {
+		/* Page fault must be handled on user-process context. */
+		r = -EFAULT;
+		vcpu->arch.page_fault.cr2 = cr2;
+		vcpu->arch.page_fault.error_code = error_code;
+		vcpu->arch.page_fault.insn = insn;
+		vcpu->arch.page_fault.insn_len = insn_len;
+		kvm_make_request(KVM_REQ_HANDLE_PF, vcpu);
+		goto out;
+	}
+#endif
+
 	r = vcpu->arch.mmu.page_fault(vcpu, cr2, error_code, false);
 	if (r < 0)
 		goto out;

arch/x86/kvm/x86.c +10 -0

index 827b681..579c41c 100644
@@ -5561,6 +5561,16 @@ static int vcpu_enter_guest_slave(struct kvm_vcpu *vcpu,
 	r = arg.ret;
 	*apf_pending = arg.apf_pending;
 
+	if (r == -EFAULT && kvm_check_request(KVM_REQ_HANDLE_PF, vcpu)) {
+		pr_debug("handling page fault request @%p\n",
+			 (void *)vcpu->arch.page_fault.cr2);
+		r = kvm_mmu_page_fault(vcpu,
+				       vcpu->arch.page_fault.cr2,
+				       vcpu->arch.page_fault.error_code,
+				       vcpu->arch.page_fault.insn,
+				       vcpu->arch.page_fault.insn_len);
+	}
+
 	return r;
 }

Tomoki Sekiyama

author

2012-09-06 11:28 UTC #12 lore raw

RFC PATCH v2 8/21 KVM: Add KVM_GET_SLAVE_CPU and KVM_SET_SLAVE_CPU to vCPU ioctl

In reply to Tomoki Sekiyama

Add an interface to set/get slave CPU dedicated to the vCPUs.

By calling ioctl with KVM_GET_SLAVE_CPU, users can get the slave CPU id for the vCPU. -1 is returned if a slave CPU is not set.

By calling ioctl with KVM_SET_SLAVE_CPU, users can dedicate the specified slave CPU to the vCPU. The CPU must be offlined before calling ioctl. The CPU is activated as slave CPU for the vCPU when the correct id is set. The slave CPU is freed and offlined by setting -1 as slave CPU id.

Whether getting/setting slave CPUs are supported by KVM or not can be known by checking KVM_CAP_SLAVE_CPU.

Signed-off-by Tomoki Sekiyama <[email protected]>

Cc Avi Kivity <[email protected]>

Cc Marcelo Tosatti <[email protected]>

Cc Thomas Gleixner <[email protected]>

Cc Ingo Molnar <[email protected]>

Cc "H. Peter Anvin" <[email protected]>

---

 arch/x86/include/asm/kvm_host.h |    2 +
 arch/x86/kvm/vmx.c              |    7 +++++
 arch/x86/kvm/x86.c              |   58 +++++++++++++++++++++++++++++++++++++++
 include/linux/kvm.h             |    4 +++
 4 files changed, 71 insertions(+), 0 deletions(-)

arch/x86/include/asm/kvm_host.h +2 -0

index 8dc1a0a..0ea04c9 100644
@@ -718,6 +718,8 @@ struct kvm_x86_ops {
 	int (*check_intercept)(struct kvm_vcpu *vcpu,
 			       struct x86_instruction_info *info,
 			       enum x86_intercept_stage stage);
+
+	void (*set_slave_mode)(struct kvm_vcpu *vcpu, bool slave);
 };
 
 struct kvm_arch_async_pf {

arch/x86/kvm/vmx.c +7 -0

index c5db714..7bbfa01 100644
@@ -1698,6 +1698,11 @@ static void skip_emulated_instruction(struct kvm_vcpu *vcpu)
 	vmx_set_interrupt_shadow(vcpu, 0);
 }
 
+static void vmx_set_slave_mode(struct kvm_vcpu *vcpu, bool slave)
+{
+	/* Nothing */
+}
+
 /*
  * KVM wants to inject page-faults which it got to the guest. This function
  * checks whether in a nested guest, we need to inject them to L1 or L2.

@@ -7344,6 +7349,8 @@ static struct kvm_x86_ops vmx_x86_ops = {
 	.set_tdp_cr3 = vmx_set_cr3,
 
 	.check_intercept = vmx_check_intercept,
+
+	.set_slave_mode = vmx_set_slave_mode,
 };
 
 static int __init vmx_init(void)

arch/x86/kvm/x86.c +58 -0

82 lines not on this page · raw message

include/linux/kvm.h +4 -0

index 2ce09aa..c2d1604 100644
@@ -618,6 +618,7 @@ struct kvm_ppc_smmu_info {
 #define KVM_CAP_PPC_GET_SMMU_INFO 78
 #define KVM_CAP_S390_COW 79
 #define KVM_CAP_PPC_ALLOC_HTAB 80
+#define KVM_CAP_SLAVE_CPU 81
 
 #ifdef KVM_CAP_IRQ_ROUTING

@@ -904,6 +905,9 @@ struct kvm_s390_ucas_mapping {
 #define KVM_SET_ONE_REG		  _IOW(KVMIO,  0xac, struct kvm_one_reg)
 /* VM is being stopped by host */
 #define KVM_KVMCLOCK_CTRL	  _IO(KVMIO,   0xad)
+/* Available with KVM_CAP_SLAVE_CPU */
+#define KVM_GET_SLAVE_CPU	  _IO(KVMIO,  0xae)
+#define KVM_SET_SLAVE_CPU	  _IO(KVMIO,  0xaf)
 
 #define KVM_DEV_ASSIGN_ENABLE_IOMMU	(1 << 0)
 #define KVM_DEV_ASSIGN_PCI_2_3		(1 << 1)

Tomoki Sekiyama

author

2012-09-06 11:28 UTC #13 lore raw

RFC PATCH v2 9/21 KVM: Go back to online CPU on VM exit by external interrupt

In reply to Tomoki Sekiyama

If the slave CPU receives an interrupt in running a guest, current implementation must once go back to onilne CPUs to handle the interupt.

This behavior will be replaced by later patch, which introduces direct interrupt handling mechanism by the guest.

Signed-off-by Tomoki Sekiyama <[email protected]>

Cc Avi Kivity <[email protected]>

Cc Marcelo Tosatti <[email protected]>

Cc Thomas Gleixner <[email protected]>

Cc Ingo Molnar <[email protected]>

Cc "H. Peter Anvin" <[email protected]>

---

 arch/x86/include/asm/kvm_host.h |    1 +
 arch/x86/kvm/vmx.c              |    1 +
 arch/x86/kvm/x86.c              |    6 ++++++
 3 files changed, 8 insertions(+), 0 deletions(-)

arch/x86/include/asm/kvm_host.h +1 -0

index 0ea04c9..af68ffb 100644
@@ -358,6 +358,7 @@ struct kvm_vcpu_arch {
 	int sipi_vector;
 	u64 ia32_misc_enable_msr;
 	bool tpr_access_reporting;
+	bool interrupted;
 
 #ifdef CONFIG_SLAVE_CPU
 	/* slave cpu dedicated to this vcpu */

arch/x86/kvm/vmx.c +1 -0

index 7bbfa01..d99bee6 100644
@@ -4408,6 +4408,7 @@ static int handle_exception(struct kvm_vcpu *vcpu)
 
 static int handle_external_interrupt(struct kvm_vcpu *vcpu)
 {
+	vcpu->arch.interrupted = true;
 	++vcpu->stat.irq_exits;
 	return 1;
 }

arch/x86/kvm/x86.c +6 -0

index b62f59c..db0be81 100644
@@ -5566,6 +5566,12 @@ static void __vcpu_enter_guest_slave(void *_arg)
 			break;
 
 		/* determine if slave cpu can handle the exit alone */
+		if (vcpu->arch.interrupted) {
+			vcpu->arch.interrupted = false;
+			arg->ret = LOOP_ONLINE;
+			break;
+		}
+
 		r = vcpu_post_run(vcpu, arg->task, &arg->apf_pending);
 
 		if (r == LOOP_SLAVE &&

Tomoki Sekiyama

author

2012-09-06 11:28 UTC #14 lore raw

RFC PATCH v2 10/21 KVM: proxy slab operations for slave CPUs on online CPUs

In reply to Tomoki Sekiyama

Add some fix-ups that proxy slab operations on online CPUs for the guest, in order to avoid touching slab on slave CPUs where some slab functions are not activated.

Currently, slab may be touched on slave CPUs in following 3 cases. For each cases, the fix-ups below are introduced:

* kvm_mmu_commit_zap_page
With this patch, Instead of commiting zap pages, the pages are added into invalid_mmu_pages list and make KVM_REQ_COMMIT_ZAP_PAGE request. Then, the pages are freed on online CPUs after the execution of vCPU thread is resumed.

* mmu_topup_memory_caches
Preallocate caches for mmu operations in vcpu_enter_guest_slave, which is done by online CPUs before entering guests.

* kvm_async_pf_wakeup_all
If this function is called on slave CPUs, it makes KVM_REQ_WAKEUP_APF. The request is handled by calling kvm_async_pf_wakeup_all on online CPUs.

Signed-off-by Tomoki Sekiyama <[email protected]>

Cc Avi Kivity <[email protected]>

Cc Marcelo Tosatti <[email protected]>

Cc Thomas Gleixner <[email protected]>

Cc Ingo Molnar <[email protected]>

Cc "H. Peter Anvin" <[email protected]>

---

 arch/x86/include/asm/kvm_host.h |    5 ++++
 arch/x86/kvm/mmu.c              |   52 ++++++++++++++++++++++++++++-----------
 arch/x86/kvm/mmu.h              |    4 +++
 arch/x86/kvm/x86.c              |   15 +++++++++++
 virt/kvm/async_pf.c             |    8 ++++++
 5 files changed, 69 insertions(+), 15 deletions(-)

arch/x86/include/asm/kvm_host.h +5 -0

index af68ffb..5ce89f1 100644
@@ -70,6 +70,8 @@
 #ifdef CONFIG_SLAVE_CPU
 /* Requests to handle VM exit on online cpu */
 #define KVM_REQ_HANDLE_PF	32
+#define KVM_REQ_COMMIT_ZAP_PAGE	33
+#define KVM_REQ_WAKEUP_APF	34
 #endif
 
 /* KVM Hugepage definitions for x86 */

@@ -542,6 +544,9 @@ struct kvm_arch {
 	 * Hash table of struct kvm_mmu_page.
 	 */
 	struct list_head active_mmu_pages;
+#ifdef CONFIG_SLAVE_CPU
+	struct list_head invalid_mmu_pages;
+#endif
 	struct list_head assigned_dev_head;
 	struct iommu_domain *iommu_domain;
 	int iommu_flags;

arch/x86/kvm/mmu.c +37 ~~-15~~

160 lines not on this page · raw message

arch/x86/kvm/mmu.h +4 -0

index e374db9..32efc36 100644
@@ -52,6 +52,10 @@ int kvm_mmu_get_spte_hierarchy(struct kvm_vcpu *vcpu, u64 addr, u64 sptes[4]);
 void kvm_mmu_set_mmio_spte_mask(u64 mmio_mask);
 int handle_mmio_page_fault_common(struct kvm_vcpu *vcpu, u64 addr, bool direct);
 int kvm_init_shadow_mmu(struct kvm_vcpu *vcpu, struct kvm_mmu *context);
+int mmu_topup_memory_caches(struct kvm_vcpu *vcpu);
+#ifdef CONFIG_SLAVE_CPU
+void kvm_mmu_commit_zap_page_late(struct kvm_vcpu *vcpu);
+#endif
 
 static inline unsigned int kvm_mmu_available_pages(struct kvm *kvm)
 {

arch/x86/kvm/x86.c +15 -0

index db0be81..a6b2521 100644
@@ -5571,6 +5571,11 @@ static void __vcpu_enter_guest_slave(void *_arg)
 			arg->ret = LOOP_ONLINE;
 			break;
 		}
+		if (test_bit(KVM_REQ_COMMIT_ZAP_PAGE, &vcpu->requests) ||
+		    test_bit(KVM_REQ_WAKEUP_APF,      &vcpu->requests)) {
+			r = LOOP_ONLINE;
+			break;
+		}
 
 		r = vcpu_post_run(vcpu, arg->task, &arg->apf_pending);

@@ -5598,6 +5603,9 @@ static int vcpu_enter_guest_slave(struct kvm_vcpu *vcpu,
 
 	BUG_ON((unsigned)slave >= nr_cpu_ids || !cpu_slave(slave));
 
+	/* Refill memory caches here to avoid calling slab on slave cpu */
+	mmu_topup_memory_caches(vcpu);
+
 	preempt_disable();
 	preempt_notifier_unregister(&vcpu->preempt_notifier);
 	kvm_arch_vcpu_put_migrate(vcpu);

@@ -5631,6 +5639,10 @@ static int vcpu_enter_guest_slave(struct kvm_vcpu *vcpu,
 				       vcpu->arch.page_fault.insn,
 				       vcpu->arch.page_fault.insn_len);
 	}
+	if (kvm_check_request(KVM_REQ_WAKEUP_APF, vcpu))
+		kvm_async_pf_wakeup_all(vcpu);
+	if (kvm_check_request(KVM_REQ_COMMIT_ZAP_PAGE, vcpu))
+		kvm_mmu_commit_zap_page_late(vcpu);
 
 	return r;
 }

@@ -6486,6 +6498,9 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 		return -EINVAL;
 
 	INIT_LIST_HEAD(&kvm->arch.active_mmu_pages);
+#ifdef CONFIG_SLAVE_CPU
+	INIT_LIST_HEAD(&kvm->arch.invalid_mmu_pages);
+#endif
 	INIT_LIST_HEAD(&kvm->arch.assigned_dev_head);
 
 	/* Reserve bit 0 of irq_sources_bitmap for userspace irq source */

virt/kvm/async_pf.c +8 -0

index feb5e76..97b3a77 100644
@@ -204,6 +204,14 @@ int kvm_async_pf_wakeup_all(struct kvm_vcpu *vcpu)
 	if (!list_empty_careful(&vcpu->async_pf.done))
 		return 0;
 
+#ifdef CONFIG_SLAVE_CPU
+	if (cpu_slave(raw_smp_processor_id())) {
+		/* Redo on online cpu to avoid kmem_cache_alloc on slave cpu */
+		kvm_make_request(KVM_REQ_WAKEUP_APF, vcpu);
+		return -ENOMEM;
+	}
+#endif
+
 	work = kmem_cache_zalloc(async_pf_cache, GFP_ATOMIC);
 	if (!work)
 		return -ENOMEM;

Tomoki Sekiyama

author

2012-09-06 11:28 UTC #15 lore raw

RFC PATCH v2 11/21 KVM: no exiting from guest when slave CPU halted

In reply to Tomoki Sekiyama

Avoid exiting from a guest on slave CPU even if HLT instruction is executed. Since the slave CPU is dedicated to a vCPU, exit on HLT is not required, and avoiding VM exit will improve the guest's performance.

This is a partial revert of

        10166744b80a ("KVM: VMX: remove yield_on_hlt")

Cc Avi Kivity <[email protected]>

Cc Marcelo Tosatti <[email protected]>

Cc Thomas Gleixner <[email protected]>

Cc Ingo Molnar <[email protected]>

Cc "H. Peter Anvin" <[email protected]>

---

 arch/x86/kvm/vmx.c |   25 ++++++++++++++++++++++++-
 1 files changed, 24 insertions(+), 1 deletions(-)

arch/x86/kvm/vmx.c +24 -1

index d99bee6..03a2d02 100644
@@ -1698,9 +1698,29 @@ static void skip_emulated_instruction(struct kvm_vcpu *vcpu)
 	vmx_set_interrupt_shadow(vcpu, 0);
 }
 
+static inline void vmx_clear_hlt(struct kvm_vcpu *vcpu)
+{
+#ifdef CONFIG_SLAVE_CPU
+	/* Ensure that we clear the HLT state in the VMCS.  We don't need to
+	 * explicitly skip the instruction because if the HLT state is set,
+	 * then the instruction is already executing and RIP has already been
+	 * advanced. */
+	if (vcpu->arch.slave_cpu >= 0 &&
+	    vmcs_read32(GUEST_ACTIVITY_STATE) == GUEST_ACTIVITY_HLT)
+		vmcs_write32(GUEST_ACTIVITY_STATE, GUEST_ACTIVITY_ACTIVE);
+#endif
+}
+
 static void vmx_set_slave_mode(struct kvm_vcpu *vcpu, bool slave)
 {
-	/* Nothing */
+	/* Don't intercept the guest's halt on slave CPU */
+	if (slave) {
+		vmcs_clear_bits(CPU_BASED_VM_EXEC_CONTROL,
+				CPU_BASED_HLT_EXITING);
+	} else {
+		vmcs_set_bits(CPU_BASED_VM_EXEC_CONTROL,
+			      CPU_BASED_HLT_EXITING);
+	}
 }
 
 /*

@@ -1755,6 +1775,7 @@ static void vmx_queue_exception(struct kvm_vcpu *vcpu, unsigned nr,
 		intr_info |= INTR_TYPE_HARD_EXCEPTION;
 
 	vmcs_write32(VM_ENTRY_INTR_INFO_FIELD, intr_info);
+	vmx_clear_hlt(vcpu);
 }
 
 static bool vmx_rdtscp_supported(void)

@@ -4125,6 +4146,7 @@ static void vmx_inject_irq(struct kvm_vcpu *vcpu)
 	} else
 		intr |= INTR_TYPE_EXT_INTR;
 	vmcs_write32(VM_ENTRY_INTR_INFO_FIELD, intr);
+	vmx_clear_hlt(vcpu);
 }
 
 static void vmx_inject_nmi(struct kvm_vcpu *vcpu)

@@ -4156,6 +4178,7 @@ static void vmx_inject_nmi(struct kvm_vcpu *vcpu)
 	}
 	vmcs_write32(VM_ENTRY_INTR_INFO_FIELD,
 			INTR_TYPE_NMI_INTR | INTR_INFO_VALID_MASK | NMI_VECTOR);
+	vmx_clear_hlt(vcpu);
 }
 
 static int vmx_nmi_allowed(struct kvm_vcpu *vcpu)

Tomoki Sekiyama

author

2012-09-06 11:28 UTC #16 lore raw

RFC PATCH v2 12/21 x86/apic: Enable external interrupt routing to slave CPUs

In reply to Tomoki Sekiyama

Enable APIC to handle interrupts on slave CPUs, and enables interrupt routing to slave CPUs by setting IRQ affinity.

As slave CPUs which run a KVM guest handle external interrupts directly in the vCPUs, the guest's vector/IRQ mapping is different from the host's. That requires interrupts to be routed either online CPUs or slave CPUs.

In this patch, if online CPUs are contained in specified affinity settings, the affinity settings will be only applied to online CPUs. If every specified CPU is slave, IRQ will be routed to slave CPUs.

Signed-off-by Tomoki Sekiyama <[email protected]>

Cc Avi Kivity <[email protected]>

Cc Marcelo Tosatti <[email protected]>

Cc Thomas Gleixner <[email protected]>

Cc Ingo Molnar <[email protected]>

Cc "H. Peter Anvin" <[email protected]>

---

 arch/x86/include/asm/apic.h           |    6 ++---
 arch/x86/kernel/apic/io_apic.c        |   43 ++++++++++++++++++++++++---------
 arch/x86/kernel/apic/x2apic_cluster.c |    8 +++---
 drivers/iommu/intel_irq_remapping.c   |   30 +++++++++++++++++++----
 kernel/irq/manage.c                   |    4 ++-
 kernel/irq/migration.c                |    2 +-
 kernel/irq/proc.c                     |    2 +-
 7 files changed, 67 insertions(+), 28 deletions(-)

arch/x86/include/asm/apic.h +3 -3

index f342612..d37ae5c 100644
@@ -535,7 +535,7 @@ extern void generic_bigsmp_probe(void);
 static inline const struct cpumask *default_target_cpus(void)
 {
 #ifdef CONFIG_SMP
-	return cpu_online_mask;
+	return cpu_online_or_slave_mask;
 #else
 	return cpumask_of(0);
 #endif

@@ -543,7 +543,7 @@ static inline const struct cpumask *default_target_cpus(void)
 
 static inline const struct cpumask *online_target_cpus(void)
 {
-	return cpu_online_mask;
+	return cpu_online_or_slave_mask;
 }
 
 DECLARE_EARLY_PER_CPU_READ_MOSTLY(u16, x86_bios_cpu_apicid);

@@ -602,7 +602,7 @@ flat_cpu_mask_to_apicid_and(const struct cpumask *cpumask,
 {
 	unsigned long cpu_mask = cpumask_bits(cpumask)[0] &
 				 cpumask_bits(andmask)[0] &
-				 cpumask_bits(cpu_online_mask)[0] &
+				 cpumask_bits(cpu_online_or_slave_mask)[0] &
 				 APIC_ALL_CPUS;
 
 	if (likely(cpu_mask)) {

arch/x86/kernel/apic/io_apic.c +31 ~~-12~~

101 lines not on this page · raw message

arch/x86/kernel/apic/x2apic_cluster.c +4 -4

index c88baa4..7403e1e 100644
@@ -106,7 +106,7 @@ x2apic_cpu_mask_to_apicid_and(const struct cpumask *cpumask,
 	int i;
 
 	for_each_cpu_and(i, cpumask, andmask) {
-		if (!cpumask_test_cpu(i, cpu_online_mask))
+		if (!cpumask_test_cpu(i, cpu_online_or_slave_mask))
 			continue;
 		dest = per_cpu(x86_cpu_to_logical_apicid, i);
 		cluster = x2apic_cluster(i);

@@ -117,7 +117,7 @@ x2apic_cpu_mask_to_apicid_and(const struct cpumask *cpumask,
 		return -EINVAL;
 
 	for_each_cpu_and(i, cpumask, andmask) {
-		if (!cpumask_test_cpu(i, cpu_online_mask))
+		if (!cpumask_test_cpu(i, cpu_online_or_slave_mask))
 			continue;
 		if (cluster != x2apic_cluster(i))
 			continue;

@@ -137,7 +137,7 @@ static void init_x2apic_ldr(void)
 	per_cpu(x86_cpu_to_logical_apicid, this_cpu) = apic_read(APIC_LDR);
 
 	__cpu_set(this_cpu, per_cpu(cpus_in_cluster, this_cpu));
-	for_each_online_cpu(cpu) {
+	for_each_cpu(cpu, cpu_online_or_slave_mask) {
 		if (x2apic_cluster(this_cpu) != x2apic_cluster(cpu))
 			continue;
 		__cpu_set(this_cpu, per_cpu(cpus_in_cluster, cpu));

@@ -169,7 +169,7 @@ update_clusterinfo(struct notifier_block *nfb, unsigned long action, void *hcpu)
 	case CPU_UP_CANCELED:
 	case CPU_UP_CANCELED_FROZEN:
 	case CPU_DEAD:
-		for_each_online_cpu(cpu) {
+		for_each_cpu(cpu, cpu_online_or_slave_mask) {
 			if (x2apic_cluster(this_cpu) != x2apic_cluster(cpu))
 				continue;
 			__cpu_clear(this_cpu, per_cpu(cpus_in_cluster, cpu));

drivers/iommu/intel_irq_remapping.c +25 -5

index af8904d..df38334 100644
@@ -931,26 +931,43 @@ intel_ioapic_set_affinity(struct irq_data *data, const struct cpumask *mask,
 	struct irq_cfg *cfg = data->chip_data;
 	unsigned int dest, irq = data->irq;
 	struct irte irte;
+	cpumask_var_t new_mask;
+	bool mask_ok = false;
 	int err;
 
 	if (!config_enabled(CONFIG_SMP))
 		return -EINVAL;
 
-	if (!cpumask_intersects(mask, cpu_online_mask))
+	if (!cpumask_intersects(mask, cpu_online_or_slave_mask))
 		return -EINVAL;
 
 	if (get_irte(irq, &irte))
 		return -EBUSY;
 
+#ifdef CONFIG_SLAVE_CPU
+	/* Set affinity to either online cpus only or slave cpus only */
+	mask_ok = alloc_cpumask_var(&new_mask, GFP_ATOMIC);
+	if (mask_ok) {
+		cpumask_and(new_mask, mask, cpu_online_mask);
+		if (cpumask_empty(new_mask))
+			cpumask_copy(new_mask, mask);
+		err = assign_irq_vector(irq, cfg, new_mask);
+	} else if (!cpumask_intersects(mask, cpu_online_mask) ||
+		   !cpumask_intersects(mask, cpu_slave_mask))
+		err = assign_irq_vector(irq, cfg, mask);
+	else
+		return -ENOMEM;
+#else
 	err = assign_irq_vector(irq, cfg, mask);
+#endif
 	if (err)
-		return err;
+		goto error;
 
 	err = apic->cpu_mask_to_apicid_and(cfg->domain, mask, &dest);
 	if (err) {
 		if (assign_irq_vector(irq, cfg, data->affinity))
 			pr_err("Failed to recover vector for irq %d\n", irq);
-		return err;
+		goto error;
 	}
 
 	irte.vector = cfg->vector;

@@ -970,8 +987,11 @@ intel_ioapic_set_affinity(struct irq_data *data, const struct cpumask *mask,
 	if (cfg->move_in_progress)
 		send_cleanup_vector(cfg);
 
-	cpumask_copy(data->affinity, mask);
-	return 0;
+	cpumask_copy(data->affinity, mask_ok ? new_mask : mask);
+error:
+	if (mask_ok)
+		free_cpumask_var(new_mask);
+	return err;
 }
 
 static void intel_compose_msi_msg(struct pci_dev *pdev,

kernel/irq/manage.c +2 -2

index 4c69326..dafe0505 100644
@@ -308,13 +308,13 @@ setup_affinity(unsigned int irq, struct irq_desc *desc, struct cpumask *mask)
 	 */
 	if (irqd_has_set(&desc->irq_data, IRQD_AFFINITY_SET)) {
 		if (cpumask_intersects(desc->irq_data.affinity,
-				       cpu_online_mask))
+				       cpu_online_or_slave_mask))
 			set = desc->irq_data.affinity;
 		else
 			irqd_clear(&desc->irq_data, IRQD_AFFINITY_SET);
 	}
 
-	cpumask_and(mask, cpu_online_mask, set);
+	cpumask_and(mask, cpu_online_or_slave_mask, set);
 	if (node != NUMA_NO_NODE) {
 		const struct cpumask *nodemask = cpumask_of_node(node);

kernel/irq/migration.c +1 -1

index ca3f4aa..6e3aaa9 100644
@@ -42,7 +42,7 @@ void irq_move_masked_irq(struct irq_data *idata)
 	 * For correct operation this depends on the caller
 	 * masking the irqs.
 	 */
-	if (cpumask_any_and(desc->pending_mask, cpu_online_mask) < nr_cpu_ids)
+	if (cpumask_any_and(desc->pending_mask, cpu_online_or_slave_mask) < nr_cpu_ids)
 		irq_do_set_affinity(&desc->irq_data, desc->pending_mask, false);
 
 	cpumask_clear(desc->pending_mask);

kernel/irq/proc.c +1 -1

index 4bd4faa..76bd7b2 100644
@@ -103,7 +103,7 @@ static ssize_t write_irq_affinity(int type, struct file *file,
 	 * way to make the system unusable accidentally :-) At least
 	 * one online CPU still has to be targeted.
 	 */
-	if (!cpumask_intersects(new_value, cpu_online_mask)) {
+	if (!cpumask_intersects(new_value, cpu_online_or_slave_mask)) {
 		/* Special case for empty set - allow the architecture
 		   code to set default SMP affinity. */
 		err = irq_select_affinity_usr(irq, new_value) ? -EINVAL : count;

Tomoki Sekiyama

author

2012-09-06 11:28 UTC #17 lore raw

RFC PATCH v2 13/21 x86/apic: IRQ vector remapping on slave for slave CPUs

In reply to Tomoki Sekiyama

Add a facility to use IRQ vector different from online CPUs on slave CPUs.

When alternative vector for IRQ is registered by remap_slave_vector_irq() and the IRQ affinity is set only to slave CPUs, the device is configured to use the alternative vector.

Current patch only supports MSI and Intel CPU with IRQ remapper of IOMMU.

This is intended to be used to routing interrupts directly to KVM guest which is running on slave CPUs which do not cause VM EXIT by external interrupts.

Signed-off-by Tomoki Sekiyama <[email protected]>

Cc Avi Kivity <[email protected]>

Cc Marcelo Tosatti <[email protected]>

Cc Thomas Gleixner <[email protected]>

Cc Ingo Molnar <[email protected]>

Cc "H. Peter Anvin" <[email protected]>

---

 arch/x86/include/asm/irq.h          |   15 ++++++++
 arch/x86/kernel/apic/io_apic.c      |   68 ++++++++++++++++++++++++++++++++++-
 drivers/iommu/intel_irq_remapping.c |    2 +
 3 files changed, 83 insertions(+), 2 deletions(-)

arch/x86/include/asm/irq.h +15 -0

index ba870bb..84756f7 100644
@@ -41,4 +41,19 @@ extern int vector_used_by_percpu_irq(unsigned int vector);
 
 extern void init_ISA_irqs(void);
 
+#ifdef CONFIG_SLAVE_CPU
+extern void remap_slave_vector_irq(int irq, int vector,
+				   const struct cpumask *mask);
+extern void revert_slave_vector_irq(int irq, const struct cpumask *mask);
+extern u8 get_remapped_slave_vector(u8 vector, unsigned int irq,
+				    const struct cpumask *mask);
+#else
+static inline u8 get_remapped_slave_vector(u8 vector, unsigned int irq,
+					   const struct cpumask *mask)
+{
+	return vector;
+}
+#endif
+
+
 #endif /* _ASM_X86_IRQ_H */

arch/x86/kernel/apic/io_apic.c +67 -1

87 lines not on this page · raw message

drivers/iommu/intel_irq_remapping.c +1 -1

index df38334..471d23f 100644
@@ -970,7 +970,7 @@ intel_ioapic_set_affinity(struct irq_data *data, const struct cpumask *mask,
 		goto error;
 	}
 
-	irte.vector = cfg->vector;
+	irte.vector = get_remapped_slave_vector(cfg->vector, irq, mask);
 	irte.dest_id = IRTE_DEST(dest);
 
 	/*

Tomoki Sekiyama

author

2012-09-06 11:28 UTC #18 lore raw

RFC PATCH v2 14/21 KVM: Directly handle interrupts by guests without VM EXIT on slave CPUs

In reply to Tomoki Sekiyama

Make interrupts on slave CPUs handled by guests without VM EXIT. This reduces CPU usage by the host to transfer interrupts of assigned PCI devices from the host to guests. It also reduces cost of VM EXIT and quickens response of guests to the interrupts.

When a slave CPU is dedicated to a vCPU, exit on external interrupts is disabled. Unfortunately, we can only enable/disable exits for whole external interrupts except NMIs and cannot switch exits based on IRQ# or vectors. Thus, to avoid IPIs from online CPUs transferred to guests, this patch modify kvm_vcpu_kick() to use NMI for guests on slave CPUs.

Signed-off-by Tomoki Sekiyama <[email protected]>

Cc Avi Kivity <[email protected]>

Cc Marcelo Tosatti <[email protected]>

Cc Thomas Gleixner <[email protected]>

Cc Ingo Molnar <[email protected]>

Cc "H. Peter Anvin" <[email protected]>

---

 arch/x86/include/asm/kvm_host.h |    1 +
 arch/x86/kvm/lapic.c            |    5 +++++
 arch/x86/kvm/vmx.c              |   19 ++++++++++++++++++
 arch/x86/kvm/x86.c              |   41 +++++++++++++++++++++++++++++++++++++++
 include/linux/kvm_host.h        |    1 +
 virt/kvm/kvm_main.c             |    5 +++--
 6 files changed, 70 insertions(+), 2 deletions(-)

arch/x86/include/asm/kvm_host.h +1 -0

index 5ce89f1..65242a6 100644
@@ -725,6 +725,7 @@ struct kvm_x86_ops {
 			       struct x86_instruction_info *info,
 			       enum x86_intercept_stage stage);
 
+	void (*set_direct_interrupt)(struct kvm_vcpu *vcpu, bool enabled);
 	void (*set_slave_mode)(struct kvm_vcpu *vcpu, bool slave);
 };

arch/x86/kvm/lapic.c +5 -0

index ce87878..73f57f3 100644
@@ -601,6 +601,9 @@ static int apic_set_eoi(struct kvm_lapic *apic)
 		kvm_ioapic_update_eoi(apic->vcpu->kvm, vector, trigger_mode);
 	}
 	kvm_make_request(KVM_REQ_EVENT, apic->vcpu);
+	if (vcpu_has_slave_cpu(apic->vcpu) &&
+	    kvm_x86_ops->set_direct_interrupt)
+		kvm_x86_ops->set_direct_interrupt(apic->vcpu, 1);
 	return vector;
 }

@@ -1569,6 +1572,8 @@ int kvm_lapic_enable_pv_eoi(struct kvm_vcpu *vcpu, u64 data)
 	u64 addr = data & ~KVM_MSR_ENABLED;
 	if (!IS_ALIGNED(addr, 4))
 		return 1;
+	if (vcpu_has_slave_cpu(vcpu))
+		return 1;
 
 	vcpu->arch.pv_eoi.msr_val = data;
 	if (!pv_eoi_enabled(vcpu))

arch/x86/kvm/vmx.c +19 -0

index 03a2d02..605abea 100644
@@ -1711,6 +1711,16 @@ static inline void vmx_clear_hlt(struct kvm_vcpu *vcpu)
 #endif
 }
 
+static void vmx_set_direct_interrupt(struct kvm_vcpu *vcpu, bool enabled)
+{
+	if (enabled)
+		vmcs_clear_bits(PIN_BASED_VM_EXEC_CONTROL,
+				PIN_BASED_EXT_INTR_MASK);
+	else
+		vmcs_set_bits(PIN_BASED_VM_EXEC_CONTROL,
+			      PIN_BASED_EXT_INTR_MASK);
+}
+
 static void vmx_set_slave_mode(struct kvm_vcpu *vcpu, bool slave)
 {
 	/* Don't intercept the guest's halt on slave CPU */

@@ -1721,6 +1731,8 @@ static void vmx_set_slave_mode(struct kvm_vcpu *vcpu, bool slave)
 		vmcs_set_bits(CPU_BASED_VM_EXEC_CONTROL,
 			      CPU_BASED_HLT_EXITING);
 	}
+
+	vmx_set_direct_interrupt(vcpu, slave);
 }
 
 /*

@@ -1776,6 +1788,8 @@ static void vmx_queue_exception(struct kvm_vcpu *vcpu, unsigned nr,
 
 	vmcs_write32(VM_ENTRY_INTR_INFO_FIELD, intr_info);
 	vmx_clear_hlt(vcpu);
+	if (vcpu_has_slave_cpu(vcpu))
+		vmx_set_direct_interrupt(vcpu, 0);
 }
 
 static bool vmx_rdtscp_supported(void)

@@ -4147,6 +4161,8 @@ static void vmx_inject_irq(struct kvm_vcpu *vcpu)
 		intr |= INTR_TYPE_EXT_INTR;
 	vmcs_write32(VM_ENTRY_INTR_INFO_FIELD, intr);
 	vmx_clear_hlt(vcpu);
+	if (vcpu_has_slave_cpu(vcpu))
+		vmx_set_direct_interrupt(vcpu, 0);
 }
 
 static void vmx_inject_nmi(struct kvm_vcpu *vcpu)

@@ -4179,6 +4195,8 @@ static void vmx_inject_nmi(struct kvm_vcpu *vcpu)
 	vmcs_write32(VM_ENTRY_INTR_INFO_FIELD,
 			INTR_TYPE_NMI_INTR | INTR_INFO_VALID_MASK | NMI_VECTOR);
 	vmx_clear_hlt(vcpu);
+	if (vcpu_has_slave_cpu(vcpu))
+		vmx_set_direct_interrupt(vcpu, 0);
 }
 
 static int vmx_nmi_allowed(struct kvm_vcpu *vcpu)

@@ -7374,6 +7392,7 @@ static struct kvm_x86_ops vmx_x86_ops = {
 
 	.check_intercept = vmx_check_intercept,
 
+	.set_direct_interrupt = vmx_set_direct_interrupt,
 	.set_slave_mode = vmx_set_slave_mode,
 };

arch/x86/kvm/x86.c +41 -0

77 lines not on this page · raw message

include/linux/kvm_host.h +1 -0

index a60743f..5fd9c64 100644
@@ -539,6 +539,7 @@ void kvm_arch_hardware_unsetup(void);
 void kvm_arch_check_processor_compat(void *rtn);
 int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu);
 int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu);
+int kvm_arch_vcpu_kick_slave(struct kvm_vcpu *vcpu);
 
 void kvm_free_physmem(struct kvm *kvm);

virt/kvm/kvm_main.c +3 -2

index debbee1..858b7a9 100644
@@ -1543,10 +1543,11 @@ void kvm_vcpu_kick(struct kvm_vcpu *vcpu)
 	}
 
 	me = get_cpu();
-	if (cpu != me && (unsigned)cpu < nr_cpu_ids &&
-	    (cpu_online(cpu) || cpu_slave(cpu)))
+	if (cpu != me && (unsigned)cpu < nr_cpu_ids && cpu_online(cpu))
 		if (kvm_arch_vcpu_should_kick(vcpu))
 			smp_send_reschedule(cpu);
+	if (cpu != me && (unsigned)cpu < nr_cpu_ids && cpu_slave(cpu))
+		kvm_arch_vcpu_kick_slave(vcpu);
 	put_cpu();
 }
 #endif /* !CONFIG_S390 */

Tomoki Sekiyama

author

2012-09-06 11:28 UTC #19 lore raw

RFC PATCH v2 15/21 KVM: add tracepoint on enabling/disabling direct interrupt delivery

In reply to Tomoki Sekiyama

Add trace event "kvm_set_direct_interrupt" to trace enabling/disabling direct interrupt delivery on slave CPUs. At the event, the guest rip and whether the feature is enabled or not is logged.

Signed-off-by Tomoki Sekiyama <[email protected]>

Cc Avi Kivity <[email protected]>

Cc Marcelo Tosatti <[email protected]>

Cc Thomas Gleixner <[email protected]>

Cc Ingo Molnar <[email protected]>

Cc "H. Peter Anvin" <[email protected]>

---

 arch/x86/kvm/trace.h |   18 ++++++++++++++++++
 arch/x86/kvm/vmx.c   |    2 ++
 arch/x86/kvm/x86.c   |    1 +
 3 files changed, 21 insertions(+), 0 deletions(-)

arch/x86/kvm/trace.h +18 -0

index a71faf7..6081be7 100644
@@ -551,6 +551,24 @@ TRACE_EVENT(kvm_pv_eoi,
 	TP_printk("apicid %x vector %d", __entry->apicid, __entry->vector)
 );
 
+TRACE_EVENT(kvm_set_direct_interrupt,
+	TP_PROTO(struct kvm_vcpu *vcpu, bool enabled),
+	TP_ARGS(vcpu, enabled),
+
+	TP_STRUCT__entry(
+		__field(	unsigned long,	guest_rip	)
+		__field(	bool,	        enabled         )
+	),
+
+	TP_fast_assign(
+		__entry->guest_rip	= kvm_rip_read(vcpu);
+		__entry->enabled        = enabled;
+	),
+
+	TP_printk("rip 0x%lx enabled %d",
+		 __entry->guest_rip, __entry->enabled)
+);
+
 /*
  * Tracepoint for nested VMRUN
  */

arch/x86/kvm/vmx.c +2 -0

index 605abea..6dc59c8 100644
@@ -1719,6 +1719,8 @@ static void vmx_set_direct_interrupt(struct kvm_vcpu *vcpu, bool enabled)
 	else
 		vmcs_set_bits(PIN_BASED_VM_EXEC_CONTROL,
 			      PIN_BASED_EXT_INTR_MASK);
+
+	trace_kvm_set_direct_interrupt(vcpu, enabled);
 }
 
 static void vmx_set_slave_mode(struct kvm_vcpu *vcpu, bool slave)

arch/x86/kvm/x86.c +1 -0

index b7d28df..1449187 100644
@@ -6936,3 +6936,4 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_nested_intr_vmexit);
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_invlpga);
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_skinit);
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_nested_intercepts);
+EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_set_direct_interrupt);

Tomoki Sekiyama

author

2012-09-06 11:33 UTC #20 lore raw

RFC PATCH v2 16/21 KVM: vmx: Add definitions PIN_BASED_PREEMPTION_TIMER

In reply to Tomoki Sekiyama

Add some definitions to use PIN_BASED_PREEMPTION_TIMER.

When PIN_BASED_PREEMPTION_TIMER is enabled, the guest will exit with reason=EXIT_REASON_PREEMPTION_TIMER when the counter specified in VMX_PREEMPTION_TIMER_VALUE becomes 0.
This patch also adds a dummy handler for EXIT_REASON_PREEMPTION_TIMER, which just goes back to VM execution soon.

These are currently intended only to be used with avoid entering the guest on a slave CPU when vmx_prevent_run(vcpu, 1) is called.

Signed-off-by Tomoki Sekiyama <[email protected]>

Cc Avi Kivity <[email protected]>

Cc Marcelo Tosatti <[email protected]>

Cc Thomas Gleixner <[email protected]>

Cc Ingo Molnar <[email protected]>

Cc "H. Peter Anvin" <[email protected]>

---

 arch/x86/include/asm/vmx.h |    3 +++
 arch/x86/kvm/trace.h       |    1 +
 arch/x86/kvm/vmx.c         |    7 +++++++
 3 files changed, 11 insertions(+), 0 deletions(-)

arch/x86/include/asm/vmx.h +3 -0

index 74fcb96..6899aaa 100644
@@ -66,6 +66,7 @@
 #define PIN_BASED_EXT_INTR_MASK                 0x00000001
 #define PIN_BASED_NMI_EXITING                   0x00000008
 #define PIN_BASED_VIRTUAL_NMIS                  0x00000020
+#define PIN_BASED_PREEMPTION_TIMER              0x00000040
 
 #define VM_EXIT_SAVE_DEBUG_CONTROLS             0x00000002
 #define VM_EXIT_HOST_ADDR_SPACE_SIZE            0x00000200

@@ -196,6 +197,7 @@ enum vmcs_field {
 	GUEST_INTERRUPTIBILITY_INFO     = 0x00004824,
 	GUEST_ACTIVITY_STATE            = 0X00004826,
 	GUEST_SYSENTER_CS               = 0x0000482A,
+	VMX_PREEMPTION_TIMER_VALUE      = 0x0000482E,
 	HOST_IA32_SYSENTER_CS           = 0x00004c00,
 	CR0_GUEST_HOST_MASK             = 0x00006000,
 	CR4_GUEST_HOST_MASK             = 0x00006002,

@@ -280,6 +282,7 @@ enum vmcs_field {
 #define EXIT_REASON_APIC_ACCESS         44
 #define EXIT_REASON_EPT_VIOLATION       48
 #define EXIT_REASON_EPT_MISCONFIG       49
+#define EXIT_REASON_PREEMPTION_TIMER	52
 #define EXIT_REASON_WBINVD		54
 #define EXIT_REASON_XSETBV		55
 #define EXIT_REASON_INVPCID		58

arch/x86/kvm/trace.h +1 -0

index 6081be7..fc350f3 100644
@@ -218,6 +218,7 @@ TRACE_EVENT(kvm_apic,
 	{ EXIT_REASON_APIC_ACCESS,		"APIC_ACCESS" }, \
 	{ EXIT_REASON_EPT_VIOLATION,		"EPT_VIOLATION" }, \
 	{ EXIT_REASON_EPT_MISCONFIG,		"EPT_MISCONFIG" }, \
+	{ EXIT_REASON_PREEMPTION_TIMER,		"PREEMPTION_TIMER" }, \
 	{ EXIT_REASON_WBINVD,			"WBINVD" }
 
 #define SVM_EXIT_REASONS \

arch/x86/kvm/vmx.c +7 -0

index 6dc59c8..2130cbd 100644
@@ -4456,6 +4456,12 @@ static int handle_external_interrupt(struct kvm_vcpu *vcpu)
 	return 1;
 }
 
+static int handle_preemption_timer(struct kvm_vcpu *vcpu)
+{
+	/* Nothing */
+	return 1;
+}
+
 static int handle_triple_fault(struct kvm_vcpu *vcpu)
 {
 	vcpu->run->exit_reason = KVM_EXIT_SHUTDOWN;

@@ -5768,6 +5774,7 @@ static int (*kvm_vmx_exit_handlers[])(struct kvm_vcpu *vcpu) = {
 	[EXIT_REASON_VMON]                    = handle_vmon,
 	[EXIT_REASON_TPR_BELOW_THRESHOLD]     = handle_tpr_below_threshold,
 	[EXIT_REASON_APIC_ACCESS]             = handle_apic_access,
+	[EXIT_REASON_PREEMPTION_TIMER]        = handle_preemption_timer,
 	[EXIT_REASON_WBINVD]                  = handle_wbinvd,
 	[EXIT_REASON_XSETBV]                  = handle_xsetbv,
 	[EXIT_REASON_TASK_SWITCH]             = handle_task_switch,

Tomoki Sekiyama

author

2012-09-06 11:28 UTC #21 lore raw

RFC PATCH v2 17/21 KVM: add kvm_arch_vcpu_prevent_run to prevent VM ENTER when NMI is received

In reply to Tomoki Sekiyama

Since NMI can not be disabled around VM enter, there is a race between receiving NMI to kick a guest and entering the guests on slave CPUs.If the NMI is received just before entering VM, after the NMI handler is invoked, it continues entering the guest and the effect of the NMI will be lost.

This patch adds kvm_arch_vcpu_prevent_run(), which causes VM exit right after VM enter. The NMI handler uses this to ensure the execution of the guest is cancelled after NMI.

Signed-off-by Tomoki Sekiyama <[email protected]>

Cc Avi Kivity <[email protected]>

Cc Marcelo Tosatti <[email protected]>

Cc Thomas Gleixner <[email protected]>

Cc Ingo Molnar <[email protected]>

Cc "H. Peter Anvin" <[email protected]>

---

 arch/x86/include/asm/kvm_host.h |    6 ++++++
 arch/x86/kvm/vmx.c              |   42 ++++++++++++++++++++++++++++++++++++++-
 arch/x86/kvm/x86.c              |   31 +++++++++++++++++++++++++++++
 3 files changed, 78 insertions(+), 1 deletions(-)

arch/x86/include/asm/kvm_host.h +6 -0

index 65242a6..624e5ad 100644
@@ -429,6 +429,9 @@ struct kvm_vcpu_arch {
 		void *insn;
 		int insn_len;
 	} page_fault;
+
+	bool prevent_run;
+	bool prevent_needed;
 #endif
 
 	int halt_request; /* real mode on Intel only */

@@ -681,6 +684,7 @@ struct kvm_x86_ops {
 
 	void (*run)(struct kvm_vcpu *vcpu);
 	int (*handle_exit)(struct kvm_vcpu *vcpu);
+	void (*prevent_run)(struct kvm_vcpu *vcpu, int prevent);
 	void (*skip_emulated_instruction)(struct kvm_vcpu *vcpu);
 	void (*set_interrupt_shadow)(struct kvm_vcpu *vcpu, int mask);
 	u32 (*get_interrupt_shadow)(struct kvm_vcpu *vcpu, int mask);

@@ -1027,4 +1031,6 @@ int kvm_pmu_read_pmc(struct kvm_vcpu *vcpu, unsigned pmc, u64 *data);
 void kvm_handle_pmu_event(struct kvm_vcpu *vcpu);
 void kvm_deliver_pmi(struct kvm_vcpu *vcpu);
 
+int kvm_arch_vcpu_run_prevented(struct kvm_vcpu *vcpu);
+
 #endif /* _ASM_X86_KVM_HOST_H */

arch/x86/kvm/vmx.c +41 -1

96 lines not on this page · raw message

arch/x86/kvm/x86.c +31 -0

index 1449187..eb2e5c4 100644
@@ -2667,6 +2667,19 @@ static DEFINE_PER_CPU(struct kvm_vcpu *, slave_vcpu);
 
 static int kvm_arch_kicked_by_nmi(unsigned int cmd, struct pt_regs *regs);
 
+struct kvm_vcpu *get_slave_vcpu(int cpu)
+{
+	return per_cpu(slave_vcpu, cpu);
+}
+
+static int kvm_arch_vcpu_prevent_run(struct kvm_vcpu *vcpu, int prevent)
+{
+	vcpu->arch.prevent_run = prevent;
+	if (!prevent || vcpu->arch.prevent_needed)
+		kvm_x86_ops->prevent_run(vcpu, prevent);
+	return 1;
+}
+
 static int kvm_arch_vcpu_ioctl_set_slave_cpu(struct kvm_vcpu *vcpu,
 					     int slave, int set_slave_mode)
 {

@@ -4873,10 +4886,12 @@ static struct notifier_block kvmclock_cpu_notifier_block = {
 	.priority = -INT_MAX
 };
 
+#ifdef CONFIG_SLAVE_CPU
 static struct notifier_block kvmclock_slave_cpu_notifier_block = {
 	.notifier_call  = kvmclock_cpu_notifier,
 	.priority = -INT_MAX
 };
+#endif
 
 static void kvm_timer_init(void)
 {

@@ -5350,6 +5365,11 @@ static int kvm_arch_kicked_by_nmi(unsigned int cmd, struct pt_regs *regs)
 	if (vcpu->mode == OUTSIDE_GUEST_MODE || kvm_is_in_guest())
 		return NMI_HANDLED;
 
+	/*
+	 * We may be about to entering VM. To prevent entering,
+	 * mark to exit as soon as possible.
+	 */
+	kvm_arch_vcpu_prevent_run(vcpu, 1);
 	return NMI_HANDLED;
 }
 #endif

@@ -5592,6 +5612,8 @@ static void __vcpu_enter_guest_slave(void *_arg)
 	kvm_arch_vcpu_load(vcpu, cpu);
 
 	while (r == LOOP_SLAVE) {
+		vcpu->arch.prevent_run = 0;
+
 		r = vcpu_enter_guest(vcpu, arg->task);
 
 		if (r <= 0)

@@ -5618,6 +5640,7 @@ static void __vcpu_enter_guest_slave(void *_arg)
 		}
 	}
 
+	kvm_arch_vcpu_prevent_run(vcpu, 0);
 	kvm_arch_vcpu_put_migrate(vcpu);
 	unuse_mm(arg->task->mm);
 	srcu_read_unlock(&vcpu->kvm->srcu, vcpu->srcu_idx);

@@ -6733,6 +6756,14 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu)
 		 kvm_cpu_has_interrupt(vcpu));
 }
 
+int kvm_arch_vcpu_run_prevented(struct kvm_vcpu *vcpu)
+{
+	kvm_x86_ops->prevent_run(vcpu, 0);
+	vcpu->arch.interrupted = true;
+	return 1;
+}
+EXPORT_SYMBOL_GPL(kvm_arch_vcpu_run_prevented);
+
 int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu)
 {
 	return kvm_vcpu_exiting_guest_mode(vcpu) == IN_GUEST_MODE;

Tomoki Sekiyama

author

2012-09-06 11:28 UTC #22 lore raw

RFC PATCH v2 18/21 KVM: route assigned devices' MSI/MSI-X directly to guests on slave CPUs

In reply to Tomoki Sekiyama

When a PCI device is assigned to a guest running on slave CPUs, this routes the device's MSI/MSI-X interrupts directly to the guest.

Because the guest uses a different interrupt vector from the host, vector remapping is required. This is safe because slave CPUs only handles interrupts for the assigned guest.

The slave CPU may receive interrupts for the guest while the guest is not running. In that case, the host IRQ handler is invoked and the interrupt is transfered as vIRQ.

If the guest receive the direct interrupt from the devices, EOI to physical APIC is required. To handle this, if the guest issues EOI when there are no in-service interrupts in the virtual APIC, physical EOI is issued.

Signed-off-by Tomoki Sekiyama <[email protected]>

Cc Avi Kivity <[email protected]>

Cc Marcelo Tosatti <[email protected]>

Cc Thomas Gleixner <[email protected]>

Cc Ingo Molnar <[email protected]>

Cc "H. Peter Anvin" <[email protected]>

---

 arch/x86/include/asm/kvm_host.h |   19 +++++
 arch/x86/kvm/irq.c              |  136 +++++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/lapic.c            |    6 +-
 arch/x86/kvm/x86.c              |   12 +++
 virt/kvm/assigned-dev.c         |    8 ++
 5 files changed, 179 insertions(+), 2 deletions(-)

arch/x86/include/asm/kvm_host.h +19 -0

index 624e5ad..f43680e 100644
@@ -1033,4 +1033,23 @@ void kvm_deliver_pmi(struct kvm_vcpu *vcpu);
 
 int kvm_arch_vcpu_run_prevented(struct kvm_vcpu *vcpu);
 
+#ifdef CONFIG_SLAVE_CPU
+void kvm_get_slave_cpu_mask(struct kvm *kvm, struct cpumask *mask);
+
+struct kvm_assigned_dev_kernel;
+extern void assign_slave_msi(struct kvm *kvm,
+			     struct kvm_assigned_dev_kernel *assigned_dev);
+extern void deassign_slave_msi(struct kvm *kvm,
+			       struct kvm_assigned_dev_kernel *assigned_dev);
+extern void assign_slave_msix(struct kvm *kvm,
+			      struct kvm_assigned_dev_kernel *assigned_dev);
+extern void deassign_slave_msix(struct kvm *kvm,
+				struct kvm_assigned_dev_kernel *assigned_dev);
+#else
+#define assign_slave_msi(kvm, assigned_dev)
+#define deassign_slave_msi(kvm, assigned_dev)
+#define assign_slave_msix(kvm, assigned_dev)
+#define deassign_slave_msix(kvm, assigned_dev)
+#endif
+
 #endif /* _ASM_X86_KVM_HOST_H */

arch/x86/kvm/irq.c +136 -0

145 lines not on this page · raw message

arch/x86/kvm/lapic.c +5 -1

index 73f57f3..bf8e351 100644
@@ -585,8 +585,12 @@ static int apic_set_eoi(struct kvm_lapic *apic)
 	 * Not every write EOI will has corresponding ISR,
 	 * one example is when Kernel check timer on setup_IO_APIC
 	 */
-	if (vector == -1)
+	if (vector == -1) {
+		/* On slave cpu, it can be EOI for a direct interrupt */
+		if (cpu_slave(smp_processor_id()))
+			ack_APIC_irq();
 		return vector;
+	}
 
 	apic_clear_isr(vector, apic);
 	apic_update_ppr(apic);

arch/x86/kvm/x86.c +12 -0

index eb2e5c4..609ab62 100644
@@ -5349,6 +5349,17 @@ static void process_nmi(struct kvm_vcpu *vcpu)
 }
 
 #ifdef CONFIG_SLAVE_CPU
+
+void kvm_get_slave_cpu_mask(struct kvm *kvm, struct cpumask *mask)
+{
+	int i;
+	struct kvm_vcpu *vcpu;
+
+	kvm_for_each_vcpu(i, vcpu, kvm)
+		if (vcpu_has_slave_cpu(vcpu))
+			cpumask_set_cpu(vcpu->arch.slave_cpu, mask);
+}
+
 static int kvm_arch_kicked_by_nmi(unsigned int cmd, struct pt_regs *regs)
 {
 	struct kvm_vcpu *vcpu;

@@ -5372,6 +5383,7 @@ static int kvm_arch_kicked_by_nmi(unsigned int cmd, struct pt_regs *regs)
 	kvm_arch_vcpu_prevent_run(vcpu, 1);
 	return NMI_HANDLED;
 }
+
 #endif
 
 enum vcpu_enter_guest_slave_retval {

virt/kvm/assigned-dev.c +7 -1

index 23a41a9..a3acc67 100644
@@ -225,8 +225,12 @@ static void deassign_host_irq(struct kvm *kvm,
 
 		free_irq(assigned_dev->host_irq, assigned_dev);
 
-		if (assigned_dev->irq_requested_type & KVM_DEV_IRQ_HOST_MSI)
+		if (assigned_dev->irq_requested_type & KVM_DEV_IRQ_HOST_MSI) {
 			pci_disable_msi(assigned_dev->dev);
+			deassign_slave_msi(kvm, assigned_dev);
+		}
+		if (assigned_dev->irq_requested_type & KVM_DEV_IRQ_HOST_MSIX)
+			deassign_slave_msix(kvm, assigned_dev);
 	}
 
 	assigned_dev->irq_requested_type &= ~(KVM_DEV_IRQ_HOST_MASK);

@@ -417,6 +421,7 @@ static int assigned_device_enable_guest_msi(struct kvm *kvm,
 {
 	dev->guest_irq = irq->guest_irq;
 	dev->ack_notifier.gsi = -1;
+	assign_slave_msi(kvm, dev);
 	return 0;
 }
 #endif

@@ -428,6 +433,7 @@ static int assigned_device_enable_guest_msix(struct kvm *kvm,
 {
 	dev->guest_irq = irq->guest_irq;
 	dev->ack_notifier.gsi = -1;
+	assign_slave_msix(kvm, dev);
 	return 0;
 }
 #endif

Tomoki Sekiyama

author

2012-09-06 11:28 UTC #23 lore raw

RFC PATCH v2 19/21 KVM: Enable direct EOI for directly routed interrupts to guests

In reply to Tomoki Sekiyama

Enable direct access to EOI MSR of x2apic to accelerate guests. This accelerate handling of interrupts delivered directly to guest from passed-through PCI devices. When a virtual IRQ is injected, this feature is disabled in order to route following EOI to virtual APIC. Then, it is enabled again after every virtual IRQ is handled.

Signed-off-by Tomoki Sekiyama <[email protected]>

Cc Avi Kivity <[email protected]>

Cc Marcelo Tosatti <[email protected]>

Cc Thomas Gleixner <[email protected]>

Cc Ingo Molnar <[email protected]>

Cc "H. Peter Anvin" <[email protected]>

---

 arch/x86/kvm/vmx.c |   69 ++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 files changed, 67 insertions(+), 2 deletions(-)

arch/x86/kvm/vmx.c +67 -2

index 39a4cb4..f93e08c 100644
@@ -636,6 +636,10 @@ static unsigned long *vmx_io_bitmap_a;
 static unsigned long *vmx_io_bitmap_b;
 static unsigned long *vmx_msr_bitmap_legacy;
 static unsigned long *vmx_msr_bitmap_longmode;
+#ifdef CONFIG_SLAVE_CPU
+static unsigned long *vmx_msr_bitmap_slave_legacy;
+static unsigned long *vmx_msr_bitmap_slave_longmode;
+#endif
 
 static bool cpu_has_load_ia32_efer;
 static bool cpu_has_load_perf_global_ctrl;

@@ -912,6 +916,11 @@ static void nested_vmx_entry_failure(struct kvm_vcpu *vcpu,
 			struct vmcs12 *vmcs12,
 			u32 reason, unsigned long qualification);
 
+static void vmx_disable_intercept_for_msr(u32 msr, bool longmode_only);
+#ifdef CONFIG_SLAVE_CPU
+static void vmx_disable_intercept_for_msr_slave(u32 msr, bool longmode_only);
+#endif
+
 static int __find_msr_index(struct vcpu_vmx *vmx, u32 msr)
 {
 	int i;

@@ -1716,13 +1725,28 @@ static void vmx_set_direct_interrupt(struct kvm_vcpu *vcpu, bool enabled)
 #ifdef CONFIG_SLAVE_CPU
 	void *msr_bitmap;
 
-	if (enabled)
+	if (enabled) {
 		vmcs_clear_bits(PIN_BASED_VM_EXEC_CONTROL,
 				PIN_BASED_EXT_INTR_MASK);
-	else
+
+		if (cpu_has_vmx_msr_bitmap()) {
+			msr_bitmap = is_long_mode(vcpu) ?
+				vmx_msr_bitmap_slave_longmode :
+				vmx_msr_bitmap_slave_legacy;
+			vmcs_write64(MSR_BITMAP, __pa(msr_bitmap));
+		}
+	} else {
 		vmcs_set_bits(PIN_BASED_VM_EXEC_CONTROL,
 			      PIN_BASED_EXT_INTR_MASK);
 
+		if (cpu_has_vmx_msr_bitmap()) {
+			msr_bitmap = is_long_mode(vcpu) ?
+				vmx_msr_bitmap_longmode :
+				vmx_msr_bitmap_legacy;
+			vmcs_write64(MSR_BITMAP, __pa(msr_bitmap));
+		}
+	}
+
 	trace_kvm_set_direct_interrupt(vcpu, enabled);
 #endif
 }

@@ -3771,6 +3795,16 @@ static void vmx_disable_intercept_for_msr(u32 msr, bool longmode_only)
 	__vmx_disable_intercept_for_msr(vmx_msr_bitmap_longmode, msr);
 }
 
+#ifdef CONFIG_SLAVE_CPU
+static void vmx_disable_intercept_for_msr_slave(u32 msr, bool longmode_only)
+{
+	if (!longmode_only)
+		__vmx_disable_intercept_for_msr(vmx_msr_bitmap_slave_legacy,
+					       msr);
+	__vmx_disable_intercept_for_msr(vmx_msr_bitmap_slave_longmode, msr);
+}
+#endif
+
 /*
  * Set up the vmcs's constant host-state fields, i.e., host-state fields that
  * will not change in the lifetime of the guest.

@@ -7474,6 +7508,22 @@ static int __init vmx_init(void)
 		goto out2;
 
 
+#ifdef CONFIG_SLAVE_CPU
+	vmx_msr_bitmap_slave_legacy =
+		(unsigned long *)__get_free_page(GFP_KERNEL);
+	if (!vmx_msr_bitmap_slave_legacy) {
+		r = -ENOMEM;
+		goto out1s;
+	}
+
+	vmx_msr_bitmap_slave_longmode =
+		(unsigned long *)__get_free_page(GFP_KERNEL);
+	if (!vmx_msr_bitmap_slave_longmode) {
+		r = -ENOMEM;
+		goto out2s;
+	}
+#endif
+
 	/*
 	 * Allow direct access to the PC debug port (it is often used for I/O
 	 * delays, but the vmexits simply slow things down).

@@ -7500,6 +7550,15 @@ static int __init vmx_init(void)
 	vmx_disable_intercept_for_msr(MSR_IA32_SYSENTER_ESP, false);
 	vmx_disable_intercept_for_msr(MSR_IA32_SYSENTER_EIP, false);
 
+#ifdef CONFIG_SLAVE_CPU
+	memcpy(vmx_msr_bitmap_slave_legacy,
+	       vmx_msr_bitmap_legacy, PAGE_SIZE);
+	memcpy(vmx_msr_bitmap_slave_longmode,
+	       vmx_msr_bitmap_longmode, PAGE_SIZE);
+	vmx_disable_intercept_for_msr_slave(
+		APIC_BASE_MSR + (APIC_EOI >> 4), false);
+#endif
+
 	if (enable_ept) {
 		kvm_mmu_set_mask_ptes(0ull,
 			(enable_ept_ad_bits) ? VMX_EPT_ACCESS_BIT : 0ull,

@@ -7513,6 +7572,12 @@ static int __init vmx_init(void)
 	return 0;
 
 out3:
+#ifdef CONFIG_SLAVE_CPU
+	free_page((unsigned long)vmx_msr_bitmap_slave_longmode);
+out2s:
+	free_page((unsigned long)vmx_msr_bitmap_slave_legacy);
+out1s:
+#endif
 	free_page((unsigned long)vmx_msr_bitmap_longmode);
 out2:
 	free_page((unsigned long)vmx_msr_bitmap_legacy);

Tomoki Sekiyama

author

2012-09-06 11:29 UTC #24 lore raw

RFC PATCH v2 20/21 KVM: Pass-through local APIC timer of on slave CPUs to guest VM

In reply to Tomoki Sekiyama

Provide direct control of local APIC timer of slave CPUs to the guest. The timer interrupt does not cause VM exit if direct interrupt delivery is enabled. To handle the timer correctly, this makes the guest occupy the local APIC timer.

If the host supports x2apic, this expose TMICT and TMCCT to the guest in order to allow guests to start the timer and to read the timer count without VM exit. Otherwise, it sets APIC registers to specified values. LVTT is not passed-through to avoid modifying timer interrupt vector.

Currently the guest timer interrupt vector remapping is not supported, and guest must use the same vector as host.

Signed-off-by Tomoki Sekiyama <[email protected]>

Cc Avi Kivity <[email protected]>

Cc Marcelo Tosatti <[email protected]>

Cc Thomas Gleixner <[email protected]>

Cc Ingo Molnar <[email protected]>

Cc "H. Peter Anvin" <[email protected]>

---

 arch/x86/include/asm/apic.h     |    4 +++
 arch/x86/include/asm/kvm_host.h |    1 +
 arch/x86/kernel/apic/apic.c     |   11 ++++++++++
 arch/x86/kernel/smpboot.c       |   30 ++++++++++++++++++++++++++
 arch/x86/kvm/lapic.c            |   45 +++++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/lapic.h            |    2 ++
 arch/x86/kvm/vmx.c              |    6 +++++
 arch/x86/kvm/x86.c              |    3 +++
 include/linux/cpu.h             |    5 ++++
 kernel/hrtimer.c                |    2 +-
 10 files changed, 108 insertions(+), 1 deletions(-)

arch/x86/include/asm/apic.h +4 -0

index d37ae5c..66e1155 100644
@@ -44,6 +44,8 @@ static inline void generic_apic_probe(void)
 
 #ifdef CONFIG_X86_LOCAL_APIC
 
+struct clock_event_device;
+
 extern unsigned int apic_verbosity;
 extern int local_apic_timer_c2_ok;

@@ -245,6 +247,8 @@ extern void init_apic_mappings(void);
 void register_lapic_address(unsigned long address);
 extern void setup_boot_APIC_clock(void);
 extern void setup_secondary_APIC_clock(void);
+extern void override_local_apic_timer(int cpu,
+	void (*handler)(struct clock_event_device *));
 extern int APIC_init_uniprocessor(void);
 extern int apic_force_enable(unsigned long addr);

arch/x86/include/asm/kvm_host.h +1 -0

index f43680e..a95bb62 100644
@@ -1035,6 +1035,7 @@ int kvm_arch_vcpu_run_prevented(struct kvm_vcpu *vcpu);
 
 #ifdef CONFIG_SLAVE_CPU
 void kvm_get_slave_cpu_mask(struct kvm *kvm, struct cpumask *mask);
+struct kvm_vcpu *get_slave_vcpu(int cpu);
 
 struct kvm_assigned_dev_kernel;
 extern void assign_slave_msi(struct kvm *kvm,

arch/x86/kernel/apic/apic.c +11 -0

index 24deb30..90ed84a 100644
@@ -901,6 +901,17 @@ void __irq_entry smp_apic_timer_interrupt(struct pt_regs *regs)
 	set_irq_regs(old_regs);
 }
 
+void override_local_apic_timer(int cpu,
+			       void (*handler)(struct clock_event_device *))
+{
+	unsigned long flags;
+
+	local_irq_save(flags);
+	per_cpu(lapic_events, cpu).event_handler = handler;
+	local_irq_restore(flags);
+}
+EXPORT_SYMBOL_GPL(override_local_apic_timer);
+
 int setup_profiling_timer(unsigned int multiplier)
 {
 	return -EINVAL;

arch/x86/kernel/smpboot.c +30 -0

index 45dfc1d..ba7c99b 100644
@@ -133,6 +133,7 @@ static void __ref remove_cpu_from_maps(int cpu);
 #ifdef CONFIG_SLAVE_CPU
 /* Notify slave cpu up and down */
 static RAW_NOTIFIER_HEAD(slave_cpu_chain);
+struct notifier_block *slave_timer_nb;
 
 int register_slave_cpu_notifier(struct notifier_block *nb)
 {

@@ -140,6 +141,13 @@ int register_slave_cpu_notifier(struct notifier_block *nb)
 }
 EXPORT_SYMBOL(register_slave_cpu_notifier);
 
+int register_slave_cpu_timer_notifier(struct notifier_block *nb)
+{
+	slave_timer_nb = nb;
+	return register_slave_cpu_notifier(nb);
+}
+EXPORT_SYMBOL(register_slave_cpu_timer_notifier);
+
 void unregister_slave_cpu_notifier(struct notifier_block *nb)
 {
 	raw_notifier_chain_unregister(&slave_cpu_chain, nb);

@@ -155,6 +163,8 @@ static int slave_cpu_notify(unsigned long val, int cpu)
 
 	return notifier_to_errno(ret);
 }
+
+static void slave_cpu_disable_timer(int cpu);
 #endif
 
 /*

@@ -1013,10 +1023,30 @@ int __cpuinit slave_cpu_up(unsigned int cpu)
 
 	cpu_maps_update_done();
 
+	/* Timer may be used only in starting the slave CPU */
+	slave_cpu_disable_timer(cpu);
+
 	return ret;
 }
 EXPORT_SYMBOL(slave_cpu_up);
 
+static void __slave_cpu_disable_timer(void *hcpu)
+{
+	int cpu = (long)hcpu;
+
+	pr_info("Disabling timer on slave cpu %d\n", cpu);
+	BUG_ON(!slave_timer_nb);
+	slave_timer_nb->notifier_call(slave_timer_nb, CPU_SLAVE_DYING, hcpu);
+}
+
+static void slave_cpu_disable_timer(int cpu)
+{
+	void *hcpu = (void *)(long)cpu;
+
+	slave_cpu_call_function(cpu, __slave_cpu_disable_timer, hcpu);
+	slave_timer_nb->notifier_call(slave_timer_nb, CPU_SLAVE_DEAD, hcpu);
+}
+
 static void __slave_cpu_down(void *dummy)
 {
 	__this_cpu_write(cpu_state, CPU_DYING);

arch/x86/kvm/lapic.c +45 -0

87 lines not on this page · raw message

arch/x86/kvm/lapic.h +2 -0

index 4af5405..e8e51ca 100644
@@ -55,6 +55,8 @@ int kvm_lapic_find_highest_irr(struct kvm_vcpu *vcpu);
 u64 kvm_get_lapic_tscdeadline_msr(struct kvm_vcpu *vcpu);
 void kvm_set_lapic_tscdeadline_msr(struct kvm_vcpu *vcpu, u64 data);
 
+void slave_lapic_timer_fn(struct clock_event_device *dev);
+
 void kvm_lapic_set_vapic_addr(struct kvm_vcpu *vcpu, gpa_t vapic_addr);
 void kvm_lapic_sync_from_vapic(struct kvm_vcpu *vcpu);
 void kvm_lapic_sync_to_vapic(struct kvm_vcpu *vcpu);

arch/x86/kvm/vmx.c +6 -0

index f93e08c..0aa9423 100644
@@ -7557,6 +7557,12 @@ static int __init vmx_init(void)
 	       vmx_msr_bitmap_longmode, PAGE_SIZE);
 	vmx_disable_intercept_for_msr_slave(
 		APIC_BASE_MSR + (APIC_EOI >> 4), false);
+
+	/* Allow guests on slave CPU to access local APIC timer directly */
+	vmx_disable_intercept_for_msr_slave(
+		APIC_BASE_MSR + (APIC_TMICT >> 4), false);
+	vmx_disable_intercept_for_msr_slave(
+		APIC_BASE_MSR + (APIC_TMCCT >> 4), false);
 #endif
 
 	if (enable_ept) {

arch/x86/kvm/x86.c +3 -0

index 609ab62..9d92581 100644
@@ -2707,6 +2707,9 @@ static int kvm_arch_vcpu_ioctl_set_slave_cpu(struct kvm_vcpu *vcpu,
 			goto out;
 		BUG_ON(!cpu_slave(slave));
 		per_cpu(slave_vcpu, slave) = vcpu;
+
+		if (vcpu->arch.apic)
+			override_local_apic_timer(slave, slave_lapic_timer_fn);
 	}
 
 	vcpu->arch.slave_cpu = slave;

include/linux/cpu.h +5 -0

index f1aa3cc..f072c6a 100644
@@ -223,6 +223,7 @@ static inline void enable_nonboot_cpus(void) {}
 
 #ifdef CONFIG_SLAVE_CPU
 int register_slave_cpu_notifier(struct notifier_block *nb);
+int register_slave_cpu_timer_notifier(struct notifier_block *nb);
 void unregister_slave_cpu_notifier(struct notifier_block *nb);
 
 /* CPU notifier constants for slave processors */

@@ -240,6 +241,10 @@ static inline int register_slave_cpu_notifier(struct notifier_block *nb)
 {
 	return 0;
 }
+static inline int register_slave_cpu_timer_notifier(struct notifier_block *nb)
+{
+	return 0;
+}
 static inline void unregister_slave_cpu_notifier(struct notifier_block *nb) {}
 #endif

kernel/hrtimer.c +1 -1

index e899a2c..61d1706 100644
@@ -1773,7 +1773,7 @@ void __init hrtimers_init(void)
 	hrtimer_cpu_notify(&hrtimers_nb, (unsigned long)CPU_UP_PREPARE,
 			  (void *)(long)smp_processor_id());
 	register_cpu_notifier(&hrtimers_nb);
-	register_slave_cpu_notifier(&hrtimers_slave_nb);
+	register_slave_cpu_timer_notifier(&hrtimers_slave_nb);
 #ifdef CONFIG_HIGH_RES_TIMERS
 	open_softirq(HRTIMER_SOFTIRQ, run_hrtimer_softirq);
 #endif

Tomoki Sekiyama

author

2012-09-06 11:29 UTC #25 lore raw

RFC PATCH v2 21/21 x86: request TLB flush to slave CPU using NMI

In reply to Tomoki Sekiyama

For slave CPUs, it is inapropriate to request TLB flush using IPI. because the IPI may be sent to a KVM guest when the slave CPU is running the guest with direct interrupt routing.

Instead, it registers a TLB flush request in per-cpu bitmask and send a NMI to interrupt execution of the guest. Then, NMI handler will check the requests and handles the requests.

This implementation has an issue in scalability, and is just for PoC.

Signed-off-by Tomoki Sekiyama <[email protected]>

Cc Avi Kivity <[email protected]>

Cc Marcelo Tosatti <[email protected]>

Cc Thomas Gleixner <[email protected]>

Cc Ingo Molnar <[email protected]>

Cc "H. Peter Anvin" <[email protected]>

---

 arch/x86/include/asm/tlbflush.h |    5 ++
 arch/x86/kernel/smpboot.c       |    3 +
 arch/x86/kvm/x86.c              |    5 ++
 arch/x86/mm/tlb.c               |   94 +++++++++++++++++++++++++++++++++++++++
 4 files changed, 106 insertions(+), 1 deletions(-)

arch/x86/include/asm/tlbflush.h +5 -0

index 74a4433..bcd637b 100644
@@ -168,6 +168,11 @@ static inline void reset_lazy_tlbstate(void)
 	this_cpu_write(cpu_tlbstate.active_mm, &init_mm);
 }
 
+#ifdef CONFIG_SLAVE_CPU
+DECLARE_PER_CPU(bool, slave_idle);
+void handle_slave_tlb_flush(unsigned int cpu);
+#endif	/* SLAVE_CPU */
+
 #endif	/* SMP */
 
 #ifndef CONFIG_PARAVIRT

arch/x86/kernel/smpboot.c +3 -0

index ba7c99b..9854087 100644
@@ -395,7 +395,10 @@ notrace static void __cpuinit start_slave_cpu(void *unused)
 		rcu_note_context_switch(cpu);
 
 		if (!f.func) {
+			__this_cpu_write(slave_idle, 1);
+			handle_slave_tlb_flush(cpu);
 			native_safe_halt();
+			__this_cpu_write(slave_idle, 0);
 			continue;
 		}

arch/x86/kvm/x86.c +5 -0

index 9d92581..d3ee570 100644
@@ -65,6 +65,7 @@
 #include <asm/cpu.h>
 #include <asm/nmi.h>
 #include <asm/mmu.h>
+#include <asm/tlbflush.h>
 
 #define MAX_IO_MSRS 256
 #define KVM_MAX_MCE_BANKS 32

@@ -5529,6 +5530,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu, struct task_struct *task)
 
 	srcu_read_unlock(&vcpu->kvm->srcu, vcpu->srcu_idx);
 
+	handle_slave_tlb_flush(vcpu->cpu);
+
 	if (req_immediate_exit)
 		smp_send_reschedule(vcpu->cpu);

@@ -5631,6 +5634,8 @@ static void __vcpu_enter_guest_slave(void *_arg)
 
 		r = vcpu_enter_guest(vcpu, arg->task);
 
+		handle_slave_tlb_flush(cpu);
+
 		if (r <= 0)
 			break;

arch/x86/mm/tlb.c +93 -1

126 lines not on this page · raw message

Avi Kivity

2012-09-06 11:46 UTC #26 lore raw

In reply to Tomoki Sekiyama

Quoted 11 lines · Tomoki Sekiyama

This RFC patch series provides facility to dedicate CPUs to KVM guests and enable the guests to handle interrupts from passed-through PCI devices directly (without VM exit and relay by the host).
With this feature, we can improve throughput and response time of the device and the host's CPU usage by reducing the overhead of interrupt handling. This is good for the application using very high throughput/frequent interrupt device (e.g. 10GbE NIC).
Real-time applicatoins also gets benefit from CPU isolation feature, which reduces interfare from host kernel tasks and scheduling delay.

Quoted 3 lines

The overview of this patch series is presented in CloudOpen 2012. The slides are available at:
http://events.linuxfoundation.org/images/stories/pdf/lcna_co2012_sekiyama.pdf

During Plumbers 2012, both Intel and AMD disclosed upcoming features to their processors (APIC-V and AVIC) that allow directing device interrupts to guest vcpus without host kernel involvement. This works without pinning, dedicating a core to a guest, or any special measures beyond support for the feature.

CPU isolation is still useful to improve real-time latency further, but this is really independent of kvm.

I am inclined to reject this feature in favour of the new hardware support. Sorry, I know this isn't nice to hear, but the extra maintenance burden cannot be justified for a niche use case with special limitations when generally useful feature exploiting proper hardware support provides the same functionality.

Signature · 1 line

error compiling committee.c: too many arguments to function

Jan Kiszka

2012-09-07 08:26 UTC #27 lore raw

In reply to Tomoki Sekiyama

Quoted 11 lines · Tomoki Sekiyama

This RFC patch series provides facility to dedicate CPUs to KVM guests and enable the guests to handle interrupts from passed-through PCI devices directly (without VM exit and relay by the host).
With this feature, we can improve throughput and response time of the device and the host's CPU usage by reducing the overhead of interrupt handling. This is good for the application using very high throughput/frequent interrupt device (e.g. 10GbE NIC).
Real-time applicatoins also gets benefit from CPU isolation feature, which reduces interfare from host kernel tasks and scheduling delay.

Quoted 3 lines

The overview of this patch series is presented in CloudOpen 2012. The slides are available at:
http://events.linuxfoundation.org/images/stories/pdf/lcna_co2012_sekiyama.pdf

One question regarding your benchmarks: If you measured against standard KVM, were the vCPU thread running on an isolcpus core of its own as well? If not, your numbers about the impact of these patches on maximum, maybe also average latencies are likely too good. Also, using a non-RT host and possibly a non-prioritized vCPU thread for the standard scenario looks like an unfair comparison.

Jan

Signature · 2 lines

Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
Corporate Competence Center Embedded Linux

Tomoki Sekiyama

author

2012-09-10 11:38 UTC #28 lore raw

Hi Jan,

Quoted 16 lines · Jan Kiszka

Quoted 11 lines · Tomoki Sekiyama
This RFC patch series provides facility to dedicate CPUs to KVM guests and enable the guests to handle interrupts from passed-through PCI devices directly (without VM exit and relay by the host).
With this feature, we can improve throughput and response time of the device and the host's CPU usage by reducing the overhead of interrupt handling. This is good for the application using very high throughput/frequent interrupt device (e.g. 10GbE NIC).
Real-time applicatoins also gets benefit from CPU isolation feature, which reduces interfare from host kernel tasks and scheduling delay.
Quoted 3 lines
The overview of this patch series is presented in CloudOpen 2012. The slides are available at:
http://events.linuxfoundation.org/images/stories/pdf/lcna_co2012_sekiyama.pdf

Quoted 6 lines

One question regarding your benchmarks: If you measured against standard KVM, were the vCPU thread running on an isolcpus core of its own as well? If not, your numbers about the impact of these patches on maximum, maybe also average latencies are likely too good. Also, using a non-RT host and possibly a non-prioritized vCPU thread for the standard scenario looks like an unfair comparison.

In the standard KVM benchmark, the vCPU thread is pinned down to its own CPU core. In addition, the vCPU thread and irq/*-kvm threads are both set to the max priority with SCHED_RR policy.

As you said, RT-host may result in better max latencies as below. (But min/average latencies became worse, however, this might be our setup
issue.)
Min / Avg / Max latencies
Normal KVM
RT-host (3.4.4-rt14) 15us / 21us / 117us
non RT-host (3.5.0-rc6) 6us / 11us / 152us
KVM + Direct IRQ
non RT-host (3.5.0-rc6 +patch) 1us / 2us / 14us

Thanks,

Signature · 3 lines

Tomoki Sekiyama <[email protected]>
Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory