nohz/rcu: Consider guest context as extended quiescent state (ie: enable full nohz on guest)
cover nohz/rcu: Consider guest context as extended quiescent state (ie: enable full nohz on guest)
Hi,
Rik sent out this patchset a few weeks ago. It appears to be interesting to disable the tick on the host while the guest is running. Indeed the host tick is irrelevant while vcpu runs, it consumes CPU time and cache for no good reasons.
Full dynticks already works in every context but RCU prevents it to be effective outside userspace because the CPU need to take part of RCU grace period completion as long as RCU may be used, which is the case in kernel context. However guest is similar to userspace and idle in that we know RCU is unused in such context. As such a guest/userspace/idle CPU can let other CPUs report its RCU quiescent state on its behalf and shut down the tick safely, provided it isn't needed for other reasons than RCU. This is called RCU extended quiescent state.
This was implemented for idle and userspace. This patchset brings it for guest.
Before pushing to Ingo, I've just cleaned out a few boring details on naming and changelogs. So I'm just posting the whole in case somebody has a comment before doing the pull request in a few days
Thanks.
Frederic Weisbecker (2):
ppc: Remove unused cpp symbols in kvm headers
context_tracking: Rename context symbols to prepare for transition
state
Rik van Riel (5):
context_tracking: Generalize context tracking APIs to support user and
guest
context_tracking: Add stub context_tracking_is_enabled
context_tracking: Run vtime_user_enter/exit only when state ==
CONTEXT_USER
context_tracking: Export context_tracking_user_enter/exit kvm,rcu,nohz: use RCU extended quiescent state when running KVM guest
arch/powerpc/include/asm/kvm_book3s.h | 4 --- arch/x86/kernel/traps.c | 2 +- include/linux/context_tracking.h | 15 +++++++-- include/linux/context_tracking_state.h | 9 ++++-- include/linux/kvm_host.h | 3 +- kernel/context_tracking.c | 59 ++++++++++++++++++++++------------ kernel/sched/core.c | 2 +- 7 files changed, 61 insertions(+), 33 deletions(-)
Signature · 1 line
2.1.4
PATCH 1/7 ppc: Remove unused cpp symbols in kvm headers
These don't seem to be used anywhere.
--- arch/powerpc/include/asm/kvm_book3s.h | 4 ---- 1 file changed, 4 deletions(-)
arch/powerpc/include/asm/kvm_book3s.h +0 -4
@@ -106,10 +106,6 @@ struct kvmppc_vcpu_book3s {spinlock_t mmu_lock; }; -#define CONTEXT_HOST 0 -#define CONTEXT_GUEST 1 -#define CONTEXT_GUEST_END 2 -#define VSID_REAL 0x07ffffffffc00000ULL#define VSID_BAT 0x07ffffffffb00000ULL#define VSID_64K 0x0800000000000000ULL
Signature · 1 line
2.1.4
Re: [PATCH 1/7] ppc: Remove unused cpp symbols in kvm headers
Quoted 12 lines · Frederic Weisbecker
These don't seem to be used anywhere.
Cc Benjamin Herrenschmidt <[email protected]>Cc Alexander Graf <[email protected]>Cc Paul E. McKenney <[email protected]>Cc Andy Lutomirski <[email protected]>Cc Will deacon <[email protected]>Cc Marcelo Tosatti <[email protected]>Cc Christian Borntraeger <[email protected]>Cc Luiz Capitulino <[email protected]>Cc Paolo Bonzini <[email protected]>Signed-off-by Frederic Weisbecker <[email protected]>
PATCH 2/7 context_tracking: Rename context symbols to prepare for transition state
In reply to Frederic WeisbeckerCurrent context tracking symbols are designed to express living state. As such they are prefixed with "IN_": IN_USER, IN_KERNEL.
Now we are going to use these symbols to also express state transitions such as context_tracking_enter(IN_USER) or context_tracking_exit(IN_USER). But while the "IN_" prefix works well to express entering a context, it's confusing to depict a context exit: context_tracking_exit(IN_USER) could mean two things:
1) We are exiting the current context to enter user context.
2) We are exiting the user context
We want 2) but the reviewer may be confused and understand 1)
So lets disambiguate these symbols and rename them to CONTEXT_USER and CONTEXT_KERNEL.
--- arch/x86/kernel/traps.c | 2 +- include/linux/context_tracking.h | 2 +- include/linux/context_tracking_state.h | 6 +++--- kernel/context_tracking.c | 8 ++++---- kernel/sched/core.c | 2 +- 5 files changed, 10 insertions(+), 10 deletions(-)
arch/x86/kernel/traps.c +1 -1
@@ -123,7 +123,7 @@ enum ctx_state ist_enter(struct pt_regs *regs)* but we need to notify RCU. */ rcu_nmi_enter (); - prev_state =IN_KERNEL ;/* the value is irrelevant. */ + prev_state =CONTEXT_KERNEL ;/* the value is irrelevant. */ }/*
include/linux/context_tracking.h +1 -1
@@ -43,7 +43,7 @@ static inline enum ctx_state exception_enter(void)static inline void exception_exit (enum ctx_state prev_ctx) {if (context_tracking_is_enabled ()) { -if (prev_ctx ==IN_USER ) +if (prev_ctx ==CONTEXT_USER )context_tracking_user_enter (); } }
include/linux/context_tracking_state.h +3 -3
@@ -13,8 +13,8 @@ struct context_tracking {*/ bool active;enum ctx_state { -IN_KERNEL =0 , -IN_USER , +CONTEXT_KERNEL =0 , +CONTEXT_USER , } state; };
@@ -34,7 +34,7 @@ static inline bool context_tracking_cpu_is_enabled(void)static inline bool context_tracking_in_user (void ) { -return __this_cpu_read (context_tracking.state) ==IN_USER ; +return __this_cpu_read (context_tracking.state) ==CONTEXT_USER ; }#else static inline bool context_tracking_in_user (void ) {return false; }
kernel/context_tracking.c +4 -4
@@ -75,7 +75,7 @@ void context_tracking_user_enter(void)WARN_ON_ONCE (!current->mm);local_irq_save (flags); -if (__this_cpu_read (context_tracking.state) !=IN_USER ) { +if (__this_cpu_read (context_tracking.state) !=CONTEXT_USER ) {if (__this_cpu_read (context_tracking.active)) {trace_user_enter (0 );/*
@@ -101,7 +101,7 @@ void context_tracking_user_enter(void)* OTOH we can spare the calls to vtime and RCU when context_tracking.active * is false because we know that CPU is not tickless. */ -__this_cpu_write (context_tracking.state,IN_USER ); +__this_cpu_write (context_tracking.state,CONTEXT_USER ); }local_irq_restore (flags); }
@@ -129,7 +129,7 @@ void context_tracking_user_exit(void)return ;local_irq_save (flags); -if (__this_cpu_read (context_tracking.state) ==IN_USER ) { +if (__this_cpu_read (context_tracking.state) ==CONTEXT_USER ) {if (__this_cpu_read (context_tracking.active)) {/* * We are going to run code that may use RCU. Inform
@@ -139,7 +139,7 @@ void context_tracking_user_exit(void)vtime_user_exit (current);trace_user_exit (0 ); } -__this_cpu_write (context_tracking.state,IN_KERNEL ); +__this_cpu_write (context_tracking.state,CONTEXT_KERNEL ); }local_irq_restore (flags); }
kernel/sched/core.c +1 -1
@@ -2818,7 +2818,7 @@ asmlinkage __visible void __sched schedule_user(void)* we find a better solution. * * NB: There are buggy callers of this function. Ideally we -* should warn if prev_state != IN_USER, but that will trigger +* should warn if prev_state != CONTEXT_USER, but that will trigger * too frequently to make sense yet. */ enum ctx_state prev_state =exception_enter ();
Signature · 1 line
2.1.4
Re: [PATCH 2/7] context_tracking: Rename context symbols to prepare for transition state
Quoted 15 lines · Frederic Weisbecker
Current context tracking symbols are designed to express living state. As such they are prefixed with "
IN_":IN_USER,IN_KERNEL.Now we are going to use these symbols to also express state transitions such as
context_tracking_enter(IN_USER)orcontext_tracking_exit(IN_USER). But while the "IN_" prefix works well to express entering a context, it's confusing to depict a context exit:context_tracking_exit(IN_USER)could mean two things:
1) We are exiting the current context to enter user context.
2) We are exiting the user context
We want 2) but the reviewer may be confused and understand 1)So lets disambiguate these symbols and rename them to
CONTEXT_USERandCONTEXT_KERNEL.
Quoted 8 lines
Cc Paul E. McKenney <[email protected]>Cc Andy Lutomirski <[email protected]>Cc Will deacon <[email protected]>Cc Marcelo Tosatti <[email protected]>Cc Christian Borntraeger <[email protected]>Cc Luiz Capitulino <[email protected]>Cc Paolo Bonzini <[email protected]>Signed-off-by Frederic Weisbecker <[email protected]>
PATCH 3/7 context_tracking: Generalize context tracking APIs to support user and guest
In reply to Frederic WeisbeckerFrom: Rik van Riel <[email protected]>
Generalize the context tracking APIs to support various nature of contexts. This is performed by splitting out the mechanism from context_tracking_user_enter and context_tracking_user_exit into context_tracking_enter and context_tracking_exit.
The nature of the context we track is now detailed in a ctx_state parameter pushed to these APIs, allowing the same functions to not just track kernel <> user space switching, but also kernel <> guest transitions.
But leave the old functions in order to avoid breaking ARM, which calls these functions from assembler code, and cannot easily use C enum parameters.
--- include/linux/context_tracking.h | 9 ++++++--- kernel/context_tracking.c | 43 ++++++++++++++++++++++++++-------------- 2 files changed, 34 insertions(+), 18 deletions(-)
include/linux/context_tracking.h +6 -3
@@ -10,6 +10,8 @@#ifdef CONFIG_CONTEXT_TRACKING extern void context_tracking_cpu_set (int cpu); +extern void context_tracking_enter (enum ctx_state state); +extern void context_tracking_exit (enum ctx_state state);extern void context_tracking_user_enter (void );extern void context_tracking_user_exit (void ); extern void __context_tracking_task_switch(struct task_struct *prev,
@@ -35,7 +37,8 @@ static inline enum ctx_state exception_enter(void)return 0 ; prev_ctx =this_cpu_read (context_tracking.state); -context_tracking_user_exit (); +if (prev_ctx !=CONTEXT_KERNEL ) +context_tracking_exit (prev_ctx);return prev_ctx; }
@@ -43,8 +46,8 @@ static inline enum ctx_state exception_enter(void)static inline void exception_exit (enum ctx_state prev_ctx) {if (context_tracking_is_enabled ()) { -if (prev_ctx ==CONTEXT_USER ) -context_tracking_user_enter (); +if (prev_ctx !=CONTEXT_KERNEL ) +context_tracking_enter (prev_ctx); } }
kernel/context_tracking.c +28 -15
87 lines not on this page · raw message
Signature · 1 line
2.1.4
PATCH 4/7 context_tracking: Add stub context_tracking_is_enabled
In reply to Frederic WeisbeckerFrom: Rik van Riel <[email protected]>
With code elsewhere doing something conditional on whether or not context tracking is enabled, we want a stub function that tells us context tracking is not enabled, when CONFIG_CONTEXT_TRACKING is not set.
--- include/linux/context_tracking_state.h | 2 ++ 1 file changed, 2 insertions(+)
include/linux/context_tracking_state.h +2 -0
@@ -39,6 +39,8 @@ static inline bool context_tracking_in_user(void)#else static inline bool context_tracking_in_user (void ) {return false; }static inline bool context_tracking_active (void ) {return false; } +static inline bool context_tracking_is_enabled (void ) {return false; } +static inline bool context_tracking_cpu_is_enabled (void ) {return false; }#endif /* CONFIG_CONTEXT_TRACKING */ #endif
Signature · 1 line
2.1.4
PATCH 5/7 context_tracking: Run vtime_user_enter/exit only when state == CONTEXT_USER
In reply to Frederic WeisbeckerFrom: Rik van Riel <[email protected]>
Only run vtime_user_enter, vtime_user_exit, and the user enter & exit trace points when we are entering or exiting user state, respectively.
The KVM code in guest_enter and guest_exit already take care of calling vtime_guest_enter and vtime_guest_exit, respectively.
The RCU code only distinguishes between "idle" and "not idle or kernel". There should be no need to add an additional (unused) state there.
--- kernel/context_tracking.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-)
kernel/context_tracking.c +8 -4
@@ -77,7 +77,6 @@ void context_tracking_enter(enum ctx_state state)local_irq_save (flags);if (__this_cpu_read (context_tracking.state) != state) {if (__this_cpu_read (context_tracking.active)) { -trace_user_enter (0 );/* * At this stage, only low level arch entry code remains and * then we'll run in userspace. We can assume there won't be
@@ -85,7 +84,10 @@ void context_tracking_enter(enum ctx_state state)* user_exit() or rcu_irq_enter(). Let's remove RCU's dependency * on the tick. */ -vtime_user_enter (current); +if (state ==CONTEXT_USER ) { +trace_user_enter (0 ); +vtime_user_enter (current); + }rcu_user_enter (); }/*
@@ -143,8 +145,10 @@ void context_tracking_exit(enum ctx_state state)* RCU core about that (ie: we may need the tick again). */ rcu_user_exit (); -vtime_user_exit (current); -trace_user_exit (0 ); +if (state ==CONTEXT_USER ) { +vtime_user_exit (current); +trace_user_exit (0 ); + } }__this_cpu_write (context_tracking.state,CONTEXT_KERNEL ); }
Signature · 1 line
2.1.4
PATCH 6/7 context_tracking: Export context_tracking_user_enter/exit
In reply to Frederic WeisbeckerFrom: Rik van Riel <[email protected]>
Export context_tracking_user_enter/exit so it can be used by KVM.
--- kernel/context_tracking.c | 2 ++ 1 file changed, 2 insertions(+)
kernel/context_tracking.c +2 -0
@@ -108,6 +108,7 @@ void context_tracking_enter(enum ctx_state state)local_irq_restore (flags); }NOKPROBE_SYMBOL (context_tracking_enter); +EXPORT_SYMBOL_GPL (context_tracking_enter);void context_tracking_user_enter (void ) {
@@ -155,6 +156,7 @@ void context_tracking_exit(enum ctx_state state)local_irq_restore (flags); }NOKPROBE_SYMBOL (context_tracking_exit); +EXPORT_SYMBOL_GPL (context_tracking_exit);void context_tracking_user_exit (void ) {
Signature · 1 line
2.1.4
PATCH 7/7 kvm,rcu,nohz: use RCU extended quiescent state when running KVM guest
In reply to Frederic WeisbeckerFrom: Rik van Riel <[email protected]>
The host kernel is not doing anything while the CPU is executing a KVM guest VCPU, so it can be marked as being in an extended quiescent state, identical to that used when running user space code.
The only exception to that rule is when the host handles an interrupt, which is already handled by the irq code, which calls rcu_irq_enter and rcu_irq_exit.
The guest_enter and guest_exit functions already switch vtime accounting independent of context tracking. Leave those calls where they are, instead of moving them into the context tracking code.
--- include/linux/context_tracking.h | 6 ++++++ include/linux/context_tracking_state.h | 1 + include/linux/kvm_host.h | 3 ++- 3 files changed, 9 insertions(+), 1 deletion(-)
include/linux/context_tracking.h +6 -0
@@ -81,10 +81,16 @@ static inline void guest_enter(void)vtime_guest_enter (current); else current->flags |= PF_VCPU; + +if (context_tracking_is_enabled ()) +context_tracking_enter (CONTEXT_GUEST ); }static inline void guest_exit (void ) { +if (context_tracking_is_enabled ()) +context_tracking_exit (CONTEXT_GUEST ); +if (vtime_accounting_enabled ())vtime_guest_exit (current);else
include/linux/context_tracking_state.h +1 -0
@@ -15,6 +15,7 @@ struct context_tracking {enum ctx_state {CONTEXT_KERNEL =0 ,CONTEXT_USER , +CONTEXT_GUEST , } state; };
include/linux/kvm_host.h +2 -1
@@ -766,7 +766,8 @@ static inline void kvm_guest_enter(void)* one time slice). Lets treat guest mode as quiescent state, just like * we do with user-mode execution. */ -rcu_virt_note_context_switch (smp_processor_id ()); +if (!context_tracking_cpu_is_enabled ()) +rcu_virt_note_context_switch (smp_processor_id ()); }static inline void kvm_guest_exit (void )
Signature · 1 line
2.1.4