Various Book3s HV fixes that haven't been picked up yet
Subsystems (3)
cover Various Book3s HV fixes that haven't been picked up yet
This is a set of 8 patches of which the first 7 have been posted previously and have had no comments. The 8th is new, but is quite trivial. They fix a series of issues with HV-style KVM on ppc. They only touch code that is specific to Book3S HV KVM. The patches are against the next branch of the kvm tree.
The overall diffstat is:
arch/powerpc/include/asm/kvm_asm.h | 1 + arch/powerpc/include/asm/kvm_book3s_64.h | 2 +- arch/powerpc/include/asm/kvm_host.h | 17 +- arch/powerpc/include/asm/smp.h | 8 + arch/powerpc/kernel/smp.c | 46 +++++ arch/powerpc/kvm/book3s_hv.c | 316 +++++++++++++++++++++--------- arch/powerpc/kvm/book3s_hv_rmhandlers.S | 11 +- 7 files changed, 293 insertions(+), 108 deletions(-)
Please apply.
Thanks,
Paul.
PATCH 1/8 KVM: PPC: Book3S HV: Allow KVM guests to stop secondary threads coming online
When a Book3S HV KVM guest is running, we need the host to be in single-thread mode, that is, all of the cores (or at least all of the cores where the KVM guest could run) to be running only one active hardware thread. This is because of the hardware restriction in POWER processors that all of the hardware threads in the core must be in the same logical partition. Complying with this restriction is much easier if, from the host kernel's point of view, only one hardware thread is active.
This adds two hooks in the SMP hotplug code to allow the KVM code to make sure that secondary threads (i.e. hardware threads other than thread 0) cannot come online while any KVM guest exists. The KVM code still has to check that any core where it runs a guest has the secondary threads offline, but having done that check it can now be sure that they will not come online while the guest is running.
--- arch/powerpc/include/asm/smp.h | 8 +++++++ arch/powerpc/kernel/smp.c | 46 ++++++++++++++++++++++++++++++++++++++++ arch/powerpc/kvm/book3s_hv.c | 12 +++++++++-- 3 files changed, 64 insertions(+), 2 deletions(-)
arch/powerpc/include/asm/smp.h +8 -0
@@ -66,6 +66,14 @@ void generic_cpu_die(unsigned int cpu);void generic_mach_cpu_die (void );void generic_set_cpu_dead (unsigned int cpu);int generic_check_cpu_restart (unsigned int cpu); + +extern void inhibit_secondary_onlining (void ); +extern void uninhibit_secondary_onlining (void ); + +#else /* HOTPLUG_CPU */ +static inline void inhibit_secondary_onlining (void ) {} +static inline void uninhibit_secondary_onlining (void ) {} +#endif #ifdef CONFIG_PPC64
arch/powerpc/kernel/smp.c +46 -0
58 lines not on this page · raw message
arch/powerpc/kvm/book3s_hv.c +10 -2
@@ -47,6 +47,7 @@#include <asm/page.h> #include <asm/hvcall.h> #include <asm/switch_to.h> +#include <asm/smp.h> #include <linux/gfp.h> #include <linux/vmalloc.h> #include <linux/highmem.h>
@@ -1016,8 +1017,6 @@ static int kvmppc_run_core(struct kvmppc_vcore *vc)/* * Make sure we are running on thread 0, and that * secondary threads are offline. -* XXX we should also block attempts to bring any -* secondary threads online. */ if (threads_per_core >1 && !on_primary_thread ()) {list_for_each_entry (vcpu, &vc->runnable_threads, arch.run_list)
@@ -1730,11 +1729,20 @@ int kvmppc_core_init_vm(struct kvm *kvm) kvm->arch.using_mmu_notifiers = !!cpu_has_feature (CPU_FTR_ARCH_206 );spin_lock_init (&kvm->arch.slot_phys_lock); + +/* +* Don't allow secondary CPU threads to come online +* while any KVM VMs exist. +*/ +inhibit_secondary_onlining (); +return 0 ; }void kvmppc_core_destroy_vm (struct kvm *kvm) { +uninhibit_secondary_onlining (); +if (kvm->arch.rma) {kvm_release_rma (kvm->arch.rma); kvm->arch.rma =NULL ;
Signature · 1 line
1.7.10.4
PATCH 2/8 KVM: PPC: Book3S HV: Fix some races in starting secondary threads
In reply to Paul MackerrasSubsequent patches implementing in-kernel XICS emulation will make it possible for IPIs to arrive at secondary threads at arbitrary times. This fixes some races in how we start the secondary threads, which if not fixed could lead to occasional crashes of the host kernel.
This makes sure that (a) we have grabbed all the secondary threads, and verified that they are no longer in the kernel, before we start any thread, (b) that the secondary thread loads its vcpu pointer after clearing the IPI that woke it up (so we don't miss a wakeup), and (c) that the secondary thread clears its vcpu pointer before incrementing the nap count. It also removes unnecessary setting of the vcpu and vcore pointers in the paca in kvmppc_core_vcpu_load.
--- arch/powerpc/kvm/book3s_hv.c | 41 ++++++++++++++++++------------- arch/powerpc/kvm/book3s_hv_rmhandlers.S | 11 ++++++--- 2 files changed, 32 insertions(+), 20 deletions(-)
arch/powerpc/kvm/book3s_hv.c +24 -17
@@ -64,8 +64,6 @@ void kvmppc_core_vcpu_load(struct kvm_vcpu *vcpu, int cpu) {struct kvmppc_vcore *vc = vcpu->arch.vcore; - local_paca->kvm_hstate.kvm_vcpu = vcpu; - local_paca->kvm_hstate.kvm_vcore = vc;if (vc->runner == vcpu && vc->vcore_state !=VCORE_INACTIVE ) vc->stolen_tb +=mftb () - vc->preempt_tb; }
@@ -880,6 +878,7 @@ static int kvmppc_grab_hwthread(int cpu)/* Ensure the thread won't go into the kernel if it wakes */ tpaca->kvm_hstate.hwthread_req =1 ; + tpaca->kvm_hstate.kvm_vcpu =NULL ;/* * If the thread is already executing in the kernel (e.g. handling
@@ -929,7 +928,6 @@ static void kvmppc_start_thread(struct kvm_vcpu *vcpu)smp_wmb ();#if defined(CONFIG_PPC_ICP_NATIVE ) && defined(CONFIG_SMP )if (vcpu->arch.ptid) { -kvmppc_grab_hwthread (cpu);xics_wake_cpu (cpu); ++vc->n_woken; }
@@ -955,7 +953,8 @@ static void kvmppc_wait_for_nap(struct kvmppc_vcore *vc)/* * Check that we are on thread 0 and that any other threads in -* this core are off-line. +* this core are off-line. Then grab the threads so they can't +* enter the kernel. */ static int on_primary_thread (void ) {
@@ -967,6 +966,17 @@ static int on_primary_thread(void)while (++thr < threads_per_core)if (cpu_online (cpu + thr))return 0 ; + +/* Grab all hw threads so they can't go into the kernel */ +for (thr =1 ; thr < threads_per_core; ++thr) { +if (kvmppc_grab_hwthread (cpu + thr)) { +/* Couldn't grab one; let the others go */ +do { +kvmppc_release_hwthread (cpu + thr); + }while (--thr >0 ); +return 0 ; + } + }return 1 ; }
@@ -1015,16 +1025,6 @@ static int kvmppc_run_core(struct kvmppc_vcore *vc) }/* -* Make sure we are running on thread 0, and that -* secondary threads are offline. -*/ -if (threads_per_core >1 && !on_primary_thread ()) { -list_for_each_entry (vcpu, &vc->runnable_threads, arch.run_list) - vcpu->arch.ret = -EBUSY ; - goto out; - } - -/* * Assign physical thread IDs, first to non-ceded vcpus * and then to ceded ones. */
@@ -1043,15 +1043,22 @@ static int kvmppc_run_core(struct kvmppc_vcore *vc)if (vcpu->arch.ceded) vcpu->arch.ptid = ptid++; +/* +* Make sure we are running on thread 0, and that +* secondary threads are offline. +*/ +if (threads_per_core >1 && !on_primary_thread ()) { +list_for_each_entry (vcpu, &vc->runnable_threads, arch.run_list) + vcpu->arch.ret = -EBUSY ; + goto out; + } + vc->stolen_tb +=mftb () - vc->preempt_tb; vc->pcpu =smp_processor_id ();list_for_each_entry (vcpu, &vc->runnable_threads, arch.run_list) {kvmppc_start_thread (vcpu);kvmppc_create_dtl_entry (vcpu, vc); } -/* Grab any remaining hw threads so they can't go into the kernel */ -for (i = ptid; i < threads_per_core; ++i) -kvmppc_grab_hwthread (vc->pcpu + i);preempt_disable ();spin_unlock (&vc->lock);
arch/powerpc/kvm/book3s_hv_rmhandlers.S +8 -3
@@ -134,8 +134,11 @@ kvm_start_guest: 27: /* XXX should handle hypervisor maintenance interrupts etc. here */ + /* reload vcpu pointer after clearing the IPI */ + ld r4,HSTATE_KVM_VCPU(r13) + cmpdi r4,0 /* if we have no vcpu to run, go back to sleep */ - beq cr1,kvm_no_guest + beq kvm_no_guest /* were we napping due to cede? */ lbz r0,HSTATE_NAPPING(r13)
@@ -1587,6 +1590,10 @@ secondary_too_late: .endr secondary_nap: + /* Clear our vcpu pointer so we don't come back in early */ + li r0, 0 + std r0, HSTATE_KVM_VCPU(r13) + lwsync /* Clear any pending IPI - assume we're a secondary thread */ ld r5, HSTATE_XICS_PHYS(r13) li r7, XICS_XIRR
@@ -1612,8 +1619,6 @@ secondary_nap: kvm_no_guest: li r0, KVM_HWTHREAD_IN_NAP stb r0, HSTATE_HWTHREAD_STATE(r13) - li r0, 0 - std r0, HSTATE_KVM_VCPU(r13) li r3, LPCR_PECE0 mfspr r4, SPRN_LPCR
Signature · 1 line
1.7.10.4
PATCH 3/8 KVM: PPC: Book3s HV: Don't access runnable threads list without vcore lock
In reply to Paul MackerrasThere were a few places where we were traversing the list of runnable threads in a virtual core, i.e. vc->runnable_threads, without holding the vcore spinlock. This extends the places where we hold the vcore spinlock to cover everywhere that we traverse that list.
Since we possibly need to sleep inside kvmppc_book3s_hv_page_fault, this moves the call of it from kvmppc_handle_exit out to kvmppc_vcpu_run, where we don't hold the vcore lock.
In kvmppc_vcore_blocked, we don't actually need to check whether all vcpus are ceded and don't have any pending exceptions, since the caller has already done that. The caller (kvmppc_run_vcpu) wasn't actually checking for pending exceptions, so we add that.
The change of if to while in kvmppc_run_vcpu is to make sure that we never call kvmppc_remove_runnable() when the vcore state is RUNNING or EXITING.
--- arch/powerpc/include/asm/kvm_asm.h | 1 + arch/powerpc/kvm/book3s_hv.c | 67 ++++++++++++++++++------------------ 2 files changed, 34 insertions(+), 34 deletions(-)
arch/powerpc/include/asm/kvm_asm.h +1 -0
@@ -118,6 +118,7 @@#define RESUME_FLAG_NV (1<<0)/* Reload guest nonvolatile state? */ #define RESUME_FLAG_HOST (1<<1)/* Resume host? */ +#define RESUME_FLAG_ARCH1 (1<<2)#define RESUME_GUEST 0#define RESUME_GUEST_NV RESUME_FLAG_NV
arch/powerpc/kvm/book3s_hv.c +33 -34
168 lines not on this page · raw message
Signature · 1 line
1.7.10.4
PATCH 4/8 KVM: PPC: Book3S HV: Fixes for late-joining threads
In reply to Paul MackerrasIf a thread in a virtual core becomes runnable while other threads in the same virtual core are already running in the guest, it is possible for the latecomer to join the others on the core without first pulling them all out of the guest. Currently this only happens rarely, when a vcpu is first started. This fixes some bugs and omissions in the code in this case.
First, we need to check for VPA updates for the latecomer and make a DTL entry for it. Secondly, if it comes along while the master vcpu is doing a VPA update, we don't need to do anything since the master will pick it up in kvmppc_run_core. To handle this correctly we introduce a new vcore state, VCORE_STARTING. Thirdly, there is a race because we currently clear the hardware thread's hwthread_req before waiting to see it get to nap. A latecomer thread could have its hwthread_req cleared before it gets to test it, and therefore never increment the nap_count, leading to messages about wait_for_nap timeouts.
--- arch/powerpc/include/asm/kvm_host.h | 7 ++++--- arch/powerpc/kvm/book3s_hv.c | 14 +++++++++++--- 2 files changed, 15 insertions(+), 6 deletions(-)
arch/powerpc/include/asm/kvm_host.h +4 -3
@@ -289,9 +289,10 @@ struct kvmppc_vcore {/* Values for vcore_state */ #define VCORE_INACTIVE 0 -#define VCORE_RUNNING 1 -#define VCORE_EXITING 2 -#define VCORE_SLEEPING 3 +#define VCORE_SLEEPING 1 +#define VCORE_STARTING 2 +#define VCORE_RUNNING 3 +#define VCORE_EXITING 4/* * Struct used to manage memory for a virtual processor area
arch/powerpc/kvm/book3s_hv.c +11 -3
@@ -336,6 +336,11 @@ static void kvmppc_update_vpa(struct kvm_vcpu *vcpu, struct kvmppc_vpa *vpap)static void kvmppc_update_vpas (struct kvm_vcpu *vcpu) { +if (!(vcpu->arch.vpa.update_pending || + vcpu->arch.slb_shadow.update_pending || + vcpu->arch.dtl.update_pending)) +return ; +spin_lock (&vcpu->arch.vpa_update_lock);if (vcpu->arch.vpa.update_pending) {kvmppc_update_vpa (vcpu, &vcpu->arch.vpa);
@@ -1009,7 +1014,7 @@ static void kvmppc_run_core(struct kvmppc_vcore *vc) vc->n_woken =0 ; vc->nap_count =0 ; vc->entry_exit_count =0 ; - vc->vcore_state =VCORE_RUNNING ; + vc->vcore_state =VCORE_STARTING ; vc->in_guest =0 ; vc->napping_threads =0 ;
@@ -1062,6 +1067,7 @@ static void kvmppc_run_core(struct kvmppc_vcore *vc)kvmppc_create_dtl_entry (vcpu, vc); } + vc->vcore_state =VCORE_RUNNING ;preempt_disable ();spin_unlock (&vc->lock);
@@ -1070,8 +1076,6 @@ static void kvmppc_run_core(struct kvmppc_vcore *vc) srcu_idx =srcu_read_lock (&vcpu0->kvm->srcu);__kvmppc_vcore_entry (NULL , vcpu0); -for (i =0 ; i < threads_per_core; ++i) -kvmppc_release_hwthread (vc->pcpu + i);spin_lock (&vc->lock);/* disable sending of IPIs on virtual external irqs */
@@ -1080,6 +1084,8 @@ static void kvmppc_run_core(struct kvmppc_vcore *vc)/* wait for secondary threads to finish writing their state to memory */ if (vc->nap_count < vc->n_woken)kvmppc_wait_for_nap (vc); +for (i =0 ; i < threads_per_core; ++i) +kvmppc_release_hwthread (vc->pcpu + i);/* prevent other vcpu threads from doing kvmppc_start_thread() now */ vc->vcore_state =VCORE_EXITING ;spin_unlock (&vc->lock);
@@ -1170,6 +1176,7 @@ static int kvmppc_run_vcpu(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu) kvm_run->exit_reason =0 ; vcpu->arch.ret =RESUME_GUEST ; vcpu->arch.trap =0 ; +kvmppc_update_vpas (vcpu);/* * Synchronize with other threads in this virtual core
@@ -1193,6 +1200,7 @@ static int kvmppc_run_vcpu(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu)if (vc->vcore_state ==VCORE_RUNNING &&VCORE_EXIT_COUNT (vc) ==0 ) { vcpu->arch.ptid = vc->n_runnable -1 ; +kvmppc_create_dtl_entry (vcpu, vc);kvmppc_start_thread (vcpu); }
Signature · 1 line
1.7.10.4
PATCH 5/8 KVM: PPC: Book3S HV: Run virtual core whenever any vcpus in it can run
In reply to Paul MackerrasCurrently the Book3S HV code implements a policy on multi-threaded processors (i.e. POWER7) that requires all of the active vcpus in a virtual core to be ready to run before we run the virtual core. However, that causes problems on reset, because reset stops all vcpus except vcpu 0, and can also reduce throughput since all four threads in a virtual core have to wait whenever any one of them hits a hypervisor page fault.
This relaxes the policy, allowing the virtual core to run as soon as any vcpu in it is runnable. With this, the KVMPPC_VCPU_STOPPED state and the KVMPPC_VCPU_BUSY_IN_HOST state have been combined into a single KVMPPC_VCPU_NOTREADY state, since we no longer need to distinguish between them.
--- arch/powerpc/include/asm/kvm_host.h | 5 +-- arch/powerpc/kvm/book3s_hv.c | 74 ++++++++++++++++++----------------- 2 files changed, 40 insertions(+), 39 deletions(-)
arch/powerpc/include/asm/kvm_host.h +2 -3
@@ -563,9 +563,8 @@ struct kvm_vcpu_arch { };/* Values for vcpu->arch.state */ -#define KVMPPC_VCPU_STOPPED 0 -#define KVMPPC_VCPU_BUSY_IN_HOST 1 -#define KVMPPC_VCPU_RUNNABLE 2 +#define KVMPPC_VCPU_NOTREADY 0 +#define KVMPPC_VCPU_RUNNABLE 1/* Values for vcpu->arch.io_gpr */ #define KVM_MMIO_REG_MASK 0x001f
arch/powerpc/kvm/book3s_hv.c +38 -36
129 lines not on this page · raw message
Signature · 1 line
1.7.10.4
PATCH 6/8 KVM: PPC: Book3S HV: Fix accounting of stolen time
In reply to Paul MackerrasCurrently the code that accounts stolen time tends to overestimate the stolen time, and will sometimes report more stolen time in a DTL (dispatch trace log) entry than has elapsed since the last DTL entry. This can cause guests to underflow the user or system time measured for some tasks, leading to ridiculous CPU percentages and total runtimes being reported by top and other utilities.
In addition, the current code was designed for the previous policy where a vcore would only run when all the vcpus in it were runnable, and so only counted stolen time on a per-vcore basis. Now that a vcore can run while some of the vcpus in it are doing other things in the kernel (e.g. handling a page fault), we need to count the time when a vcpu task is preempted while it is not running as part of a vcore as stolen also.
To do this, we bring back the BUSY_IN_HOST vcpu state and extend the vcpu_load/put functions to count preemption time while the vcpu is in that state. Handling the transitions between the RUNNING and BUSY_IN_HOST states requires checking and updating two variables (accumulated time stolen and time last preempted), so we add a new spinlock, vcpu->arch.tbacct_lock. This protects both the per-vcpu stolen/preempt-time variables, and the per-vcore variables while this vcpu is running the vcore.
Finally, we now don't count time spent in userspace as stolen time. The task could be executing in userspace on behalf of the vcpu, or it could be preempted, or the vcpu could be genuinely stopped. Since we have no way of dividing up the time between these cases, we don't count any of it as stolen.
--- arch/powerpc/include/asm/kvm_host.h | 5 ++ arch/powerpc/kvm/book3s_hv.c | 127 ++++++++++++++++++++++++++++++----- 2 files changed, 117 insertions(+), 15 deletions(-)
arch/powerpc/include/asm/kvm_host.h +5 -0
@@ -559,12 +559,17 @@ struct kvm_vcpu_arch {unsigned long dtl_index;u64 stolen_logged;struct kvmppc_vpa slb_shadow; + +spinlock_t tbacct_lock; +u64 busy_stolen; +u64 busy_preempt;#endif };/* Values for vcpu->arch.state */ #define KVMPPC_VCPU_NOTREADY 0#define KVMPPC_VCPU_RUNNABLE 1 +#define KVMPPC_VCPU_BUSY_IN_HOST 2/* Values for vcpu->arch.io_gpr */ #define KVM_MMIO_REG_MASK 0x001f
arch/powerpc/kvm/book3s_hv.c +112 -15
231 lines not on this page · raw message
Signature · 1 line
1.7.10.4
PATCH 7/8 KVM: PPC: Book3S HV: Allow DTL to be set to address 0, length 0
In reply to Paul MackerrasCommit 55b665b026 ("KVM: PPC: Book3S HV: Provide a way for userspace to get/set per-vCPU areas") includes a check on the length of the dispatch trace log (DTL) to make sure the buffer is at least one entry long. This is appropriate when registering a buffer, but the interface also allows for any existing buffer to be unregistered by specifying a zero address. In this case the length check is not appropriate. This makes the check conditional on the address being non-zero.
--- arch/powerpc/kvm/book3s_hv.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
arch/powerpc/kvm/book3s_hv.c +2 -3
@@ -811,9 +811,8 @@ int kvmppc_set_one_reg(struct kvm_vcpu *vcpu, u64 id, union kvmppc_one_reg *val) addr = val->vpaval.addr; len = val->vpaval.length; r = -EINVAL ; -if (len <sizeof (struct dtl_entry )) -break ; -if (addr && !vcpu->arch.vpa.next_gpa) +if (addr && (len <sizeof (struct dtl_entry ) || + !vcpu->arch.vpa.next_gpa))break ; len -= len %sizeof (struct dtl_entry ); r =set_vpa (vcpu, &vcpu->arch.dtl, addr, len);
Signature · 1 line
1.7.10.4
PATCH 8/8 KVM: PPC: Book3S HV: Fix thinko in try_lock_hpte()
In reply to Paul MackerrasThis fixes an error in the inline asm in try_lock_hpte() where we were erroneously using a register number as an immediate operand. The bug only affects an error path, and in fact the code will still work as long as the compiler chooses some register other than r0 for the "bits" variable. Nevertheless it should still be fixed.
--- arch/powerpc/include/asm/kvm_book3s_64.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/powerpc/include/asm/kvm_book3s_64.h +1 -1
@@ -60,7 +60,7 @@ static inline long try_lock_hpte(unsigned long *hpte, unsigned long bits)" ori %0,%0,%4\n" " stdcx. %0,0,%2\n" " beq+ 2f\n" -" li %1,%3\n" +" mr %1,%3\n" "2: isync" : "=&r" (tmp), "=&r" (old) : "r" (hpte), "r" (bits), "i" (HPTE_V_HVLOCK)
Signature · 1 line
1.7.10.4