KVM: x86: switch hugepage recovery thread to vhost_task
Subsystems (6)
Paolo Bonzini
maintainerPATCH 1/1 KVM: x86: switch hugepage recovery thread to vhost_task
kvm_vm_create_worker_thread() is meant to be used for kthreads that can consume significant amounts of CPU time on behalf of a VM or in response to how the VM behaves (for example how it accesses its memory). Therefore it wants to charge the CPU time consumed by that work to the VM's container.
However, because of these threads, cgroups which have kvm instances inside never complete freezing. This can be trivially reproduced:
root@test ~# mkdir /sys/fs/cgroup/test root@test ~# echo $fish_pid > /sys/fs/cgroup/test/cgroup.procs root@test ~# qemu-system-x86_64 --nographic -enable-kvm
and in another terminal:
root@test ~# echo 1 > /sys/fs/cgroup/test/cgroup.freeze root@test ~# cat /sys/fs/cgroup/test/cgroup.events populated 1
frozen 0
The cgroup freezing happens in the signal delivery path but kvm_vm_worker_thread() thread never call into the signal delivery path while joining non-root cgroups, so they never get frozen. Because the cgroup freezer determines whether a given cgroup is frozen by comparing the number of frozen threads to the total number of threads in the cgroup, the cgroup never becomes frozen and users waiting for the state transition may hang indefinitely.
Since the worker kthread is tied to a user process, it's better if it behaves similarly to user tasks as much as possible, including being able to send SIGSTOP and SIGCONT. In fact, vhost_task is all that kvm_vm_create_worker_thread() wanted to be and more: not only it inherits the userspace process's cgroups, it has other niceties like being parented properly in the process tree. Use it instead of the homegrown alternative.
(Commit message based on emails from Tejun).
--- arch/x86/include/asm/kvm_host.h | 4 +- arch/x86/kvm/Kconfig | 1 + arch/x86/kvm/mmu/mmu.c | 67 +++++++++++---------- include/linux/kvm_host.h | 6 -- virt/kvm/kvm_main.c | 103 -------------------------------- 5 files changed, 39 insertions(+), 142 deletions(-)
arch/x86/include/asm/kvm_host.h +3 -1
@@ -26,6 +26,7 @@#include <linux/irqbypass.h> #include <linux/hyperv.h> #include <linux/kfifo.h> +#include <linux/sched/vhost_task.h> #include <asm/apic.h> #include <asm/pvclock-abi.h>
@@ -1443,7 +1444,8 @@ struct kvm_arch {bool sgx_provisioning_allowed;struct kvm_x86_pmu_event_filter __rcu *pmu_event_filter; -struct task_struct *nx_huge_page_recovery_thread; +struct vhost_task *nx_huge_page_recovery_thread; +u64 nx_huge_page_next;#ifdef CONFIG_X86_64 /* The number of TDP MMU pages across all roots. */
arch/x86/kvm/Kconfig +1 -0
@@ -29,6 +29,7 @@ config KVM_X86 select HAVE_KVM_IRQ_BYPASS select HAVE_KVM_IRQ_ROUTING select HAVE_KVM_READONLY_MEM + select VHOST_TASK select KVM_ASYNC_PF select USER_RETURN_NOTIFIER select KVM_MMIO
arch/x86/kvm/mmu/mmu.c +35 -32
111 lines not on this page · raw message
include/linux/kvm_host.h +0 -6
@@ -2382,12 +2382,6 @@ static inline int kvm_arch_vcpu_run_pid_change(struct kvm_vcpu *vcpu) }#endif /* CONFIG_HAVE_KVM_VCPU_RUN_PID_CHANGE */ -typedef int (*kvm_vm_thread_fn_t )(struct kvm *kvm,uintptr_t data); - -int kvm_vm_create_worker_thread (struct kvm *kvm,kvm_vm_thread_fn_t thread_fn, -uintptr_t data,const char *name, -struct task_struct **thread_ptr); -#ifdef CONFIG_KVM_XFER_TO_GUEST_WORK static inline void kvm_handle_signal_exit (struct kvm_vcpu *vcpu) {
virt/kvm/kvm_main.c +0 -103
106 lines not on this page · raw message
Signature · 1 line
2.43.5
On Fri, Nov 08, 2024 at 08:07:37AM -0500, Paolo Bonzini wrote:
...
Quoted 7 lines
Since the worker kthread is tied to a user process, it's better if it behaves similarly to user tasks as much as possible, including being able to send SIGSTOP and SIGCONT. In fact,
vhost_taskis all thatkvm_vm_create_worker_thread()wanted to be and more: not only it inherits the userspace process's cgroups, it has other niceties like being parented properly in the process tree. Use it instead of the homegrown alternative.
Didn't about vhost_task. That looks perfect. From cgroup POV:
Thanks.
Signature · 1 line
tejun
Luca Boccassi
Tested-byQuoted 11 lines · Tejun Heo
On Fri, Nov 08, 2024 at 08:07:37AM -0500, Paolo Bonzini wrote:
...Quoted 7 lines
Since the worker kthread is tied to a user process, it's better if it behaves similarly to user tasks as much as possible, including being able to send SIGSTOP and SIGCONT. In fact,
vhost_taskis all thatkvm_vm_create_worker_thread()wanted to be and more: not only it inherits the userspace process's cgroups, it has other niceties like being parented properly in the process tree. Use it instead of the homegrown alternative.
Quoted 5 lines
Didn't about
vhost_task. That looks perfect. From cgroup POV:Acked-by Tejun Heo <[email protected]>Thanks.
Thanks, tested on my machine by applying it to kernel 6.11.5 and can confirm the issues are gone, freezing the cgroup works and everything else too.
Could you please CC stable so that it can get backported?
Sean Christopherson
maintainerQuoted 23 lines · Paolo Bonzini
arch/x86/kvm/mmu/mmu.c +2
-2@@ -7281,7 +7281,7 @@ static int set_nx_huge_pages(const char *val, const struct kernel_param *kp)kvm_mmu_zap_all_fast (kvm);mutex_unlock (&kvm->slots_lock); -wake_up_process (kvm->arch.nx_huge_page_recovery_thread); +vhost_task_wake (kvm->arch.nx_huge_page_recovery_thread); }mutex_unlock (&kvm_lock); }@@ -7427,7 +7427,7 @@ static int set_nx_huge_pages_recovery_param(const char *val, const struct kernelmutex_lock (&kvm_lock);list_for_each_entry (kvm, &vm_list, vm_list) -wake_up_process (kvm->arch.nx_huge_page_recovery_thread); +vhost_task_wake (kvm->arch.nx_huge_page_recovery_thread);mutex_unlock (&kvm_lock); }
arch/x86/kvm/mmu/mmu.c static void kvm_recover_nx_huge_pages(struct kvm *kvm) · line 7533
Quoted 5 lines
arch/x86/kvm/mmu/mmu.c +1
-1@@ -7530,62 +7530,65 @@ static void kvm_recover_nx_huge_pages(struct kvm *kvm)srcu_read_unlock (&kvm->srcu, rcu_idx); } -static long get_nx_huge_page_recovery_timeout (u64 start_time) +#define NX_HUGE_PAGE_DISABLED (-1)
I don't see any point in using -1. That's more legal (though still impossible and absurd) than an deadline of '0'. And it's somewhat confusing because KVM uses -1 for the default nx_huge_pages value to indicate "enable the NX huge page mitigation if the CPU is vulnerable to L1TF", not "disable the mitigation".
arch/x86/kvm/mmu/mmu.c static void kvm_recover_nx_huge_pages(struct kvm *kvm) · line 7542
Quoted 11 lines
arch/x86/kvm/mmu/mmu.c +3
-2+static u64 get_nx_huge_page_recovery_next (void ) {bool enabled;uint period; enabled =calc_nx_huge_pages_recovery_period (&period); -return enabled ? start_time +msecs_to_jiffies (period) -get_jiffies_64 () - :MAX_SCHEDULE_TIMEOUT ; +return enabled ?get_jiffies_64 () +msecs_to_jiffies (period) + :NX_HUGE_PAGE_DISABLED ;
Please align the '?' and ':' to show that they are related paths of the ternary operator. Moot point if we go without a literal '0'.
Quoted 8 lines
arch/x86/kvm/mmu/mmu.c +3
-2} -static int kvm_nx_huge_page_recovery_worker (struct kvm *kvm,uintptr_t data) +static void kvm_nx_huge_page_recovery_worker_kill (void *data) { -u64 start_time; +} +
arch/x86/kvm/mmu/mmu.c static void kvm_recover_nx_huge_pages(struct kvm *kvm) · line 7555
Quoted 10 lines
arch/x86/kvm/mmu/mmu.c +5
-3+static bool kvm_nx_huge_page_recovery_worker (void *data) +{ +struct kvm *kvm = data;long remaining_time; -while (true) { - start_time =get_jiffies_64 (); - remaining_time =get_nx_huge_page_recovery_timeout (start_time); +if (kvm->arch.nx_huge_page_next ==NX_HUGE_PAGE_DISABLED ) +return false;
The "next" concept is broken. Once KVM sees NX_HUGE_PAGE_DISABLED for a given VM, KVM will never re-evaluate nx_huge_page_next. Similarly, if the recovery period and/or ratio changes, KVM won't recompute the "next" time until the current timeout has expired.
I fiddled around with various ideas, but I don't see a better solution that something along the lines of KVM's request system, e.g. set a bool to indicate the params changed, and sprinkle smp_{r,w}mb() barriers to ensure the vhost task sees the new params.
FWIW, I also found "next" to be confusing. How about "deadline"? KVM uses that terminology for the APIC timer, i.e. it's familiar, intuitive, and accurate(ish).
Something like this as fixup? (comments would be nice)
--- arch/x86/include/asm/kvm_host.h | 3 ++- arch/x86/kvm/mmu/mmu.c | 34 +++++++++++++++++++++------------ 2 files changed, 24 insertions(+), 13 deletions(-)
arch/x86/include/asm/kvm_host.h +2 -1
@@ -1444,7 +1444,8 @@ struct kvm_arch {struct kvm_x86_pmu_event_filter __rcu *pmu_event_filter;struct vhost_task *nx_huge_page_recovery_thread; -u64 nx_huge_page_next; +u64 nx_huge_page_deadline; +bool nx_huge_page_params_changed;#ifdef CONFIG_X86_64 /* The number of TDP MMU pages across all roots. */
arch/x86/kvm/mmu/mmu.c +22 -12
@@ -7102,6 +7102,13 @@ static void mmu_destroy_caches(void)kmem_cache_destroy (mmu_page_header_cache); } +static void mmu_wake_nx_huge_page_task (struct kvm *kvm) +{ +smp_wmb (); +WRITE_ONCE (kvm->arch.nx_huge_page_deadline, true); +vhost_task_wake (kvm->arch.nx_huge_page_recovery_thread); +} +static int get_nx_huge_pages (char *buffer,const struct kernel_param *kp) {if (nx_hugepage_mitigation_hard_disabled)
@@ -7162,7 +7169,7 @@ static int set_nx_huge_pages(const char *val, const struct kernel_param *kp)kvm_mmu_zap_all_fast (kvm);mutex_unlock (&kvm->slots_lock); -vhost_task_wake (kvm->arch.nx_huge_page_recovery_thread); +mmu_wake_nx_huge_page_task (kvm); }mutex_unlock (&kvm_lock); }
@@ -7291,7 +7298,7 @@ static int set_nx_huge_pages_recovery_param(const char *val, const struct kernelmutex_lock (&kvm_lock);list_for_each_entry (kvm, &vm_list, vm_list) -vhost_task_wake (kvm->arch.nx_huge_page_recovery_thread); +mmu_wake_nx_huge_page_task (kvm);mutex_unlock (&kvm_lock); }
@@ -7394,17 +7401,14 @@ static void kvm_recover_nx_huge_pages(struct kvm *kvm)srcu_read_unlock (&kvm->srcu, rcu_idx); } -#define NX_HUGE_PAGE_DISABLED (-1) - -static u64 get_nx_huge_page_recovery_next (void ) +static u64 get_nx_huge_page_recovery_deadline (void ) {bool enabled;uint period; enabled =calc_nx_huge_pages_recovery_period (&period); -return enabled ?get_jiffies_64 () +msecs_to_jiffies (period) - :NX_HUGE_PAGE_DISABLED ; +return enabled ?get_jiffies_64 () +msecs_to_jiffies (period) :0 ; }static void kvm_nx_huge_page_recovery_worker_kill (void *data)
@@ -7416,10 +7420,16 @@ static bool kvm_nx_huge_page_recovery_worker(void *data)struct kvm *kvm = data;long remaining_time; -if (kvm->arch.nx_huge_page_next ==NX_HUGE_PAGE_DISABLED ) +if (READ_ONCE (kvm->arch.nx_huge_page_params_changed)) { +smp_rmb (); +WRITE_ONCE (kvm->arch.nx_huge_page_params_changed, false); + kvm->arch.nx_huge_page_deadline =get_nx_huge_page_recovery_deadline (); + } + +if (!kvm->arch.nx_huge_page_deadline)return false; - remaining_time = kvm->arch.nx_huge_page_next -get_jiffies_64 (); + remaining_time = kvm->arch.nx_huge_page_deadline -get_jiffies_64 ();if (remaining_time >0 ) {schedule_timeout (remaining_time);/* check for signals and come back */
@@ -7428,7 +7438,7 @@ static bool kvm_nx_huge_page_recovery_worker(void *data)__set_current_state (TASK_RUNNING );kvm_recover_nx_huge_pages (kvm); - kvm->arch.nx_huge_page_next =get_nx_huge_page_recovery_next (); + kvm->arch.nx_huge_page_deadline =get_nx_huge_page_recovery_deadline ();return true; }
@@ -7437,11 +7447,11 @@ int kvm_mmu_post_init_vm(struct kvm *kvm)if (nx_hugepage_mitigation_hard_disabled)return 0 ; - kvm->arch.nx_huge_page_next =get_nx_huge_page_recovery_next (); +WRITE_ONCE (kvm->arch.nx_huge_page_params_changed, true); kvm->arch.nx_huge_page_recovery_thread =vhost_task_create ( kvm_nx_huge_page_recovery_worker, kvm_nx_huge_page_recovery_worker_kill, kvm,"kvm-nx-lpage-recovery" ); - +if (!kvm->arch.nx_huge_page_recovery_thread)return -ENOMEM ;
base-commit: 922a5630cd31e4414f964aa64f45a5884f40188c --
Paolo Bonzini
maintainerQuoted 16 lines · Sean Christopherson
Review
Quoted 10 lines
(quoted diff) +5
-3+static bool kvm_nx_huge_page_recovery_worker(void *data) +{ + struct kvm *kvm = data; long remaining_time; - while (true) { - start_time = get_jiffies_64(); - remaining_time = get_nx_huge_page_recovery_timeout(start_time); + if (kvm->arch.nx_huge_page_next == NX_HUGE_PAGE_DISABLED) + return false;The "next" concept is broken. Once KVM sees
NX_HUGE_PAGE_DISABLEDfor a given VM, KVM will never re-evaluatenx_huge_page_next. Similarly, if the recovery period and/or ratio changes, KVM won't recompute the "next" time until the current timeout has expired.
Quoted 4 lines
I fiddled around with various ideas, but I don't see a better solution that something along the lines of KVM's request system, e.g. set a bool to indicate the params changed, and sprinkle smp_{r,w}mb() barriers to ensure the vhost task sees the new params.
"next" is broken, but there is a much better way to fix it. You just track the *last* time that the recovery ran. This is also better behaved when you flip recovery back and forth to disabled and back to enabled: if your recovery period is 1 minute, it will run the next recovery after 1 minute independent of how many times you flipped the parameter.
This also fits much better with the "restart function after schedule()" model that vhost_task.c requires, and it gets rid of get_nx_huge_page_recovery_whatever() completely. It's also silly how close the code is to the broken v1, which is why I'm sending it below instead of sending again a large patchdd.
Paolo
arch/x86/include/asm/kvm_host.h +1 -1
@@ -1444,7 +1444,7 @@ struct kvm_arch {struct kvm_x86_pmu_event_filter __rcu *pmu_event_filter;struct vhost_task *nx_huge_page_recovery_thread; -u64 nx_huge_page_next; +u64 nx_huge_page_last;#ifdef CONFIG_X86_64 /* The number of TDP MMU pages across all roots. */
arch/x86/kvm/mmu/mmu.c +9 -18
@@ -7394,19 +7394,6 @@ static void kvm_recover_nx_huge_pages(struct kvm *kvm)srcu_read_unlock (&kvm->srcu, rcu_idx); } -#define NX_HUGE_PAGE_DISABLED (-1) - -static u64 get_nx_huge_page_recovery_next (void ) -{ -bool enabled; -uint period; - - enabled =calc_nx_huge_pages_recovery_period (&period); - -return enabled ?get_jiffies_64 () +msecs_to_jiffies (period) - :NX_HUGE_PAGE_DISABLED ; -} -static void kvm_nx_huge_page_recovery_worker_kill (void *data) { }
@@ -7414,12 +7401,16 @@ static void kvm_nx_huge_page_recovery_worker_kill(void *data)static bool kvm_nx_huge_page_recovery_worker (void *data) {struct kvm *kvm = data; +bool enabled; +uint period;long remaining_time; -if (kvm->arch.nx_huge_page_next ==NX_HUGE_PAGE_DISABLED ) + enabled =calc_nx_huge_pages_recovery_period (&period); +if (!enabled)return false; - remaining_time = kvm->arch.nx_huge_page_next -get_jiffies_64 (); + remaining_time = kvm->arch.nx_huge_page_last +msecs_to_jiffies (period) + -get_jiffies_64 ();if (remaining_time >0 ) {schedule_timeout (remaining_time);/* check for signals and come back */
@@ -7428,7 +7419,7 @@ static bool kvm_nx_huge_page_recovery_worker(void *data)__set_current_state (TASK_RUNNING );kvm_recover_nx_huge_pages (kvm); - kvm->arch.nx_huge_page_next =get_nx_huge_page_recovery_next (); + kvm->arch.nx_huge_page_last =get_jiffies_64 ();return true; }
@@ -7437,11 +7428,11 @@ int kvm_mmu_post_init_vm(struct kvm *kvm)if (nx_hugepage_mitigation_hard_disabled)return 0 ; - kvm->arch.nx_huge_page_next =get_nx_huge_page_recovery_next (); + kvm->arch.nx_huge_page_last =get_jiffies_64 (); kvm->arch.nx_huge_page_recovery_thread =vhost_task_create ( kvm_nx_huge_page_recovery_worker, kvm_nx_huge_page_recovery_worker_kill, kvm,"kvm-nx-lpage-recovery" ); - +if (!kvm->arch.nx_huge_page_recovery_thread)return -ENOMEM ;
Quoted 21 lines · Paolo Bonzini
Quoted 15 lines · Sean Christopherson
Review
Quoted 9 lines
(quoted diff) +5
-3+static bool kvm_nx_huge_page_recovery_worker(void *data) +{ + struct kvm *kvm = data; long remaining_time; - while (true) { - start_time = get_jiffies_64(); - remaining_time = get_nx_huge_page_recovery_timeout(start_time); + if (kvm->arch.nx_huge_page_next == NX_HUGE_PAGE_DISABLED) + return false;The "next" concept is broken. Once KVM sees
NX_HUGE_PAGE_DISABLEDfor a given VM, KVM will never re-evaluatenx_huge_page_next. Similarly, if the recovery period and/or ratio changes, KVM won't recompute the "next" time until the current timeout has expired.Quoted 4 lines
I fiddled around with various ideas, but I don't see a better solution that something along the lines of KVM's request system, e.g. set a bool to indicate the params changed, and sprinkle smp_{r,w}mb() barriers to ensure the vhost task sees the new params.
Quoted 6 lines
"next" is broken, but there is a much better way to fix it. You just track the *last* time that the recovery ran. This is also better behaved when you flip recovery back and forth to disabled and back to enabled: if your recovery period is 1 minute, it will run the next recovery after 1 minute independent of how many times you flipped the parameter.
Heh, I my brain was trying to get there last night, but I couldn't quite piece things together.
Sasha Levin
maintainer[ Sasha's backport helper bot ]
Hi,
Found matching upstream commit: d96c77bd4eeba469bddbbb14323d2191684da82a
WARNING: Author mismatch between patch and found commit:
Backport author: Sean Christopherson<[email protected]> Commit author: Paolo Bonzini<[email protected]>
Status in newer kernel trees:
6.12.y | Present (different SHA1: 91248a2e4101)
Note: The patch differs from the upstream commit:
---
Failed to apply patch cleanly, falling back to interdiff...
---
Results of testing on various branches:
| Branch | Patch Apply | Build Test | |---------------------------|-------------|------------| | stable/linux-6.12.y | Failed | N/A | | stable/linux-6.6.y | Failed | N/A | | stable/linux-6.1.y | Failed | N/A | | stable/linux-5.15.y | Failed | N/A | | stable/linux-5.10.y | Failed | N/A | | stable/linux-5.4.y | Failed | N/A |
Quoted 3 lines · Paolo Bonzini
Since the worker kthread is tied to a user process, it's better if it behaves similarly to user tasks as much as possible, including being able to send SIGSTOP and SIGCONT.
Do you mean s/send/receive/?
Consequently, it's OK if a (possibly unprivileged) user stops this thread forever (they only harm themselves, not the rest of the system), correct?
Quoted 4 lines
In fact,
vhost_taskis all thatkvm_vm_create_worker_thread()wanted to be and more: not only it inherits the userspace process's cgroups, it has other niceties like being parented properly in the process tree. Use it instead of the homegrown alternative.
It is nice indeed.
I think the bugs we saw are not so serious to warrant
.
(But I'm posting it here so that I can find the reference later.)
Thanks,
Michal
Paolo Bonzini
maintainerQuoted 7 lines · Michal Koutný
Quoted 3 lines · Paolo Bonzini
Since the worker kthread is tied to a user process, it's better if it behaves similarly to user tasks as much as possible, including being able to send SIGSTOP and SIGCONT.
Do you mean s/send/receive/?
I mean being able to send it to the threads with an effect.
Quoted 3 lines
Consequently, it's OK if a (possibly unprivileged) user stops this thread forever (they only harm themselves, not the rest of the system), correct?
Yes, they will run with fewer huge pages and worse TLB performance.
Paolo
Quoted 10 lines
Quoted 4 lines
In fact,
vhost_taskis all thatkvm_vm_create_worker_thread()wanted to be and more: not only it inherits the userspace process's cgroups, it has other niceties like being parented properly in the process tree. Use it instead of the homegrown alternative.It is nice indeed.
I think the bugs we saw are not so serious to warrantFixes c57c80467f90e ("kvm: Add helper function for creating VM worker threads").
(But I'm posting it here so that I can find the reference later.)
Quoted 1 line · Paolo Bonzini
I mean being able to send it to the threads with an effect.
Understood now.
Quoted 5 lines
Quoted 3 lines
Consequently, it's OK if a (possibly unprivileged) user stops this thread forever (they only harm themselves, not the rest of the system), correct?
Yes, they will run with fewer huge pages and worse TLB performance.
Alright.
Thanks,
Michal
Quoted 3 lines
Quoted 3 lines
It is nice indeed.
I think the bugs we saw are not so serious to warrantFixes c57c80467f90e ("kvm: Add helper function for creating VM worker threads")
I'm mainly posting this because there are some people surprised this didn't get to 6.12. Hence I wonder if Cc: stable would be helpful here.
Paolo Bonzini
maintainerQuoted 5 lines · Michal Koutný
Quoted 2 lines
Quoted 2 lines
I think the bugs we saw are not so serious to warrant
Fixes c57c80467f90e ("kvm: Add helper function for creating VM worker threads")I'm mainly posting this because there are some people surprised this didn't get to 6.12. Hence I wonder if Cc: stable would be helpful here.
Yes, the patch didn't make it to 6.12 because I wanted to get reviews (which I did and resulted in changes), but it is in 6.13 and Cc:
stable is there.
Paolo
Quoted 7 lines · Paolo Bonzini
Since the worker kthread is tied to a user process, it's better if it behaves similarly to user tasks as much as possible, including being able to send SIGSTOP and SIGCONT. In fact,
vhost_taskis all thatkvm_vm_create_worker_thread()wanted to be and more: not only it inherits the userspace process's cgroups, it has other niceties like being parented properly in the process tree. Use it instead of the homegrown alternative.
This appears to be causing a user space regression. The library "minijail" is used by virtual machine manager "crossvm". crossvm uses minijail to fork processes, but the library requires the process be single threaded. Prior to this patch, the process was single threaded, but this change creates a relationship from the kvm thread to the user process that fails minijail's test.
For reference, here's the affected library function reporting this behavior change:
https://github.com/google/minijail/blob/main/rust/minijail/src/lib.rs#L987
Reverting the patch makes it work again.
Fwiw, I think the single threaded check may be misguided, but just doing my due diligence to report the user space interaction.
Paolo Bonzini
maintainerQuoted 6 lines · Keith Busch
This appears to be causing a user space regression. The library "minijail" is used by virtual machine manager "crossvm". crossvm uses minijail to fork processes, but the library requires the process be single threaded. Prior to this patch, the process was single threaded, but this change creates a relationship from the kvm thread to the user process that fails minijail's test.
Thanks for the report.
The minijail code has a flag that's documented like this:
/// Disables the check that prevents forking in a multithreaded environment.
/// This is only safe if the child process calls exec immediately after
/// forking. The state of locks, and whether or not they will unlock
/// is undefined. Additionally, objects allocated on other threads that
/// expect to be dropped when those threads cease execution will not be
/// dropped.
/// Thus, nothing should be called that relies on shared synchronization
/// primitives referenced outside of the current thread. The safest
/// way to use this is to immediately exec in the child.Is crosvm trying to do anything but exec? If not, it should probably use the flag.
Quoted 2 lines
Fwiw, I think the single threaded check may be misguided, but just doing my due diligence to report the user space interaction.
I don't think the check itself is misguided, but if it can be improved to only look at usermode threads somehow, that would be better. In general Linux is moving towards properly tracking the parent-child relationship of kernel processes (for vhost and io_uring, and now for KVM).
Paolo
Quoted 22 lines · Paolo Bonzini
Quoted 6 lines · Keith Busch
This appears to be causing a user space regression. The library "minijail" is used by virtual machine manager "crossvm". crossvm uses minijail to fork processes, but the library requires the process be single threaded. Prior to this patch, the process was single threaded, but this change creates a relationship from the kvm thread to the user process that fails minijail's test.
Thanks for the report.
The minijail code has a flag that's documented like this:
/// Disables the check that prevents forking in a multithreaded environment. /// This is only safe if the child process calls exec immediately after /// forking. The state of locks, and whether or not they will unlock /// is undefined. Additionally, objects allocated on other threads that /// expect to be dropped when those threads cease execution will not be /// dropped. /// Thus, nothing should be called that relies on shared synchronization /// primitives referenced outside of the current thread. The safest /// way to use this is to immediately exec in the child.
Quoted 2 lines
Is crosvm trying to do anything but exec? If not, it should probably use the flag.
Good point, and I'm not sure right now. I don't think I know any crosvm developer experts but I'm working on that to get a better explanation of what's happening,
Paolo Bonzini
maintainerQuoted 6 lines · Keith Busch
Quoted 2 lines
Is crosvm trying to do anything but exec? If not, it should probably use the flag.
Good point, and I'm not sure right now. I don't think I know any crosvm developer experts but I'm working on that to get a better explanation of what's happening,
Ok, I found the code and it doesn't exec (e.g.
https://github.com/google/crosvm/blob/b339d3d7/src/crosvm/sys/linux/jail_warden.rs#L122),
so that's not an option. Well, if I understand correctly from a cursory look at the code, crosvm is creating a jailed child process early, and then spawns further jails through it; so it's just this first process that has to cheat.
One possibility on the KVM side is to delay creating the vhost_task until the first KVM_RUN. I don't like it but...
I think we should nevertheless add something to the status file in procfs, that makes it easy to detect kernel tasks (PF_KTHREAD | PF_IO_WORKER | PF_USER_WORKER).
Paolo
Quoted 11 lines · Paolo Bonzini
Quoted 6 lines · Keith Busch
Quoted 2 lines
Is crosvm trying to do anything but exec? If not, it should probably use the flag.
Good point, and I'm not sure right now. I don't think I know any crosvm developer experts but I'm working on that to get a better explanation of what's happening,
Ok, I found the code and it doesn't exec (e.g.
https://github.com/google/crosvm/blob/b339d3d7/src/crosvm/sys/linux/jail_warden.rs#L122),
so that's not an option.
Thanks, I was slowly getting there too. It's been a while since I had to work with the languange, so I'm a bit rusty (no pun intended) at navigating.
Quoted 11 lines
Well, if I understand correctly from a
cursory look at the code, crosvm is creating a jailed child process early, and then spawns further jails through it; so it's just this first process that has to cheat.One possibility on the KVM side is to delay creating the
vhost_taskuntil the firstKVM_RUN. I don't like it but...I think we should nevertheless add something to the status file in procfs, that makes it easy to detect kernel tasks (
PF_KTHREAD|PF_IO_WORKER|PF_USER_WORKER).
I currently think excluding kernel tasks from this check probably aligns with what it's trying to do, so anything to make that easier is a good step, IMO.
Paolo Bonzini
maintainerQuoted 25 lines · Keith Busch
Quoted 11 lines · Paolo Bonzini
Quoted 6 lines · Keith Busch
Quoted 2 lines
Is crosvm trying to do anything but exec? If not, it should probably use the flag.
Good point, and I'm not sure right now. I don't think I know any crosvm developer experts but I'm working on that to get a better explanation of what's happening,
Ok, I found the code and it doesn't exec (e.g.
https://github.com/google/crosvm/blob/b339d3d7/src/crosvm/sys/linux/jail_warden.rs#L122),
so that's not an option.Thanks, I was slowly getting there too. It's been a while since I had to work with the languange, so I'm a bit rusty (no pun intended) at navigating.
Quoted 8 lines
Well, if I understand correctly from a
cursory look at the code, crosvm is creating a jailed child process early, and then spawns further jails through it; so it's just this first process that has to cheat.One possibility on the KVM side is to delay creating the
vhost_taskuntil the firstKVM_RUN. I don't like it but...
Quoted 8 lines
Quoted 3 lines
I think we should nevertheless add something to the status file in procfs, that makes it easy to detect kernel tasks (
PF_KTHREAD|PF_IO_WORKER|PF_USER_WORKER).I currently think excluding kernel tasks from this check probably aligns with what it's trying to do, so anything to make that easier is a good step, IMO.
It could be as simple as this on the kernel side: [adding Jens for a first look]
=============== 8< ===========
From: Paolo Bonzini <[email protected]>
Subject: [PATCH] fs: proc: mark user and I/O workers as "kernel threads"
A Rust library called "minijail" is looking at procfs to check if the current task has multiple threads, and to prevent fork() if it does. This is because fork() is in general ill-advised in multi-threaded programs, for example if another thread might have taken locks.
However, this attempt falls afoul of kernel threads that are children of the user process that they serve. These are not a problem when forking, but they are still present in procfs. The library should discard them, but there is currently no way for userspace to detect PF_USER_WORKER or PF_IO_WORKER threads.
The closest is the "Kthread" key in /proc/PID/task/TID/status. Extend it instead of introducing another keyl tasks that are marked with PF_USER_WORKER or PF_IO_WORKER are not kthreads, but they are close enough for basically all intents and purposes.
fs/proc/array.c +1 -1
@@ -221,7 +221,7 @@ static inline void task_state(struct seq_file *m, struct pid_namespace *ns,#endif seq_putc (m,'\n' ); -seq_printf (m,"Kthread:\t%c\n" , p->flags &PF_KTHREAD ?'1' :'0' ); +seq_printf (m,"Kthread:\t%c\n" , p->flags & (PF_KTHREAD |PF_USER_WORKER |PF_IO_WORKER ) ?'1' :'0' ); } void render_sigset_t(struct seq_file *m, const char *header,
Paolo
Quoted 2 lines · Paolo Bonzini
It could be as simple as this on the kernel side: [adding Jens for a first look]
Cc'ing his @kernel email; first message:
https://lore.kernel.org/linux-kernel/[email protected]/
Quoted 16 lines
=============== 8< ===========
From: Paolo Bonzini <[email protected]>
Subject: [PATCH] fs: proc: mark user and I/O workers as "kernel threads"A Rust library called "minijail" is looking at procfs to check if the current task has multiple threads, and to prevent
fork()if it does. This is becausefork()is in general ill-advised in multi-threaded programs, for example if another thread might have taken locks.However, this attempt falls afoul of kernel threads that are children of the user process that they serve. These are not a problem when forking, but they are still present in procfs. The library should discard them, but there is currently no way for userspace to detect
PF_USER_WORKERorPF_IO_WORKERthreads.
Quoted 4 lines
The closest is the "Kthread" key in
/proc/PID/task/TID/status. Extend it instead of introducing another keyl tasks that are marked withPF_USER_WORKERorPF_IO_WORKERare not kthreads, but they are close enough for basically all intents and purposes.
Yes, this looks good to me. But I also would have thought the original patch was safe too :). Hopefully nothing depends on the current value for these... there is often something making it difficult to have nice things.
Quoted 13 lines
Signed-off-by Paolo Bonzini <[email protected]>fs/proc/array.c +1
-1@@ -221,7 +221,7 @@ static inline void task_state(struct seq_file *m, struct pid_namespace *ns,#endif seq_putc (m,'\n' ); -seq_printf (m,"Kthread:\t%c\n" , p->flags &PF_KTHREAD ?'1' :'0' ); +seq_printf (m,"Kthread:\t%c\n" , p->flags & (PF_KTHREAD |PF_USER_WORKER |PF_IO_WORKER ) ?'1' :'0' ); } void render_sigset_t(struct seq_file *m, const char *header,UBLK_U_IO_REGISTER_ZC_RING
Quoted 15 lines · Paolo Bonzini
Quoted 6 lines · Keith Busch
Quoted 2 lines
Is crosvm trying to do anything but exec? If not, it should probably use the flag.
Good point, and I'm not sure right now. I don't think I know any crosvm developer experts but I'm working on that to get a better explanation of what's happening,
Ok, I found the code and it doesn't exec (e.g.
https://github.com/google/crosvm/blob/b339d3d7/src/crosvm/sys/linux/jail_warden.rs#L122),
so that's not an option. Well, if I understand correctly from a cursory look at the code, crosvm is creating a jailed child process early, and then spawns further jails through it; so it's just this first process that has to cheat.
Quoted 2 lines
One possibility on the KVM side is to delay creating the
vhost_taskuntil the firstKVM_RUN. I don't like it but...
This option is actually kind of appealing in that we don't need to change any application side to filter out kernel tasks, as well as not having a new kernel dependency to even report these types of tasks as kernel threads.
I gave it a quick try. I'm not very familiar with the code here, so not sure if this is thread safe or not, but it did successfully get crosvm booting again.
---
arch/x86/kvm/mmu/mmu.c +2 -0
@@ -7415,6 +7415,8 @@ int kvm_mmu_post_init_vm(struct kvm *kvm) {if (nx_hugepage_mitigation_hard_disabled)return 0 ; +if (kvm->arch.nx_huge_page_recovery_thread) +return 0 ; kvm->arch.nx_huge_page_last =get_jiffies_64 (); kvm->arch.nx_huge_page_recovery_thread = vhost_task_create(
arch/x86/kvm/x86.c +4 -5
@@ -11463,6 +11463,10 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)struct kvm_run *kvm_run = vcpu->run;int r; + r =kvm_mmu_post_init_vm (vcpu->kvm); +if (r) +return r; +vcpu_load (vcpu);kvm_sigset_activate (vcpu); kvm_run->flags =0 ;
@@ -12740,11 +12744,6 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)return ret; } -int kvm_arch_post_init_vm (struct kvm *kvm) -{ -return kvm_mmu_post_init_vm (kvm); -} -static void kvm_unload_vcpu_mmu (struct kvm_vcpu *vcpu) {vcpu_load (vcpu);
include/linux/kvm_host.h +0 -1
@@ -1596,7 +1596,6 @@ int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu);bool kvm_arch_dy_runnable (struct kvm_vcpu *vcpu);bool kvm_arch_dy_has_pending_interrupt (struct kvm_vcpu *vcpu);bool kvm_arch_vcpu_preempted_in_kernel (struct kvm_vcpu *vcpu); -int kvm_arch_post_init_vm (struct kvm *kvm);void kvm_arch_pre_destroy_vm (struct kvm *kvm);void kvm_arch_create_vm_debugfs (struct kvm *kvm);
virt/kvm/kvm_main.c +0 -15
@@ -1065,15 +1065,6 @@ static int kvm_create_vm_debugfs(struct kvm *kvm, const char *fdname)return ret; } -/* -* Called after the VM is otherwise initialized, but just before adding it to -* the vm_list. -*/ -int __weak kvm_arch_post_init_vm (struct kvm *kvm) -{ -return 0 ; -} -/* * Called just after removing the VM from the vm_list, but before doing any * other destruction.
@@ -1194,10 +1185,6 @@ static struct kvm *kvm_create_vm(unsigned long type, const char *fdname)if (r) goto out_err_no_debugfs; - r =kvm_arch_post_init_vm (kvm); -if (r) - goto out_err; -mutex_lock (&kvm_lock);list_add (&kvm->vm_list, &vm_list);mutex_unlock (&kvm_lock);
@@ -1207,8 +1194,6 @@ static struct kvm *kvm_create_vm(unsigned long type, const char *fdname)return kvm; -out_err: -kvm_destroy_vm_debugfs (kvm); out_err_no_debugfs:kvm_coalesced_mmio_free (kvm); out_no_coalesced_mmio:
--
Paolo Bonzini
maintainerQuoted 10 lines · Keith Busch
Quoted 9 lines
Ok, I found the code and it doesn't exec (e.g.
https://github.com/google/crosvm/blob/b339d3d7/src/crosvm/sys/linux/jail_warden.rs#L122),
so that's not an option. Well, if I understand correctly from a cursory look at the code, crosvm is creating a jailed child process early, and then spawns further jails through it; so it's just this first process that has to cheat.One possibility on the KVM side is to delay creating the
vhost_taskuntil the firstKVM_RUN. I don't like it but...
Quoted 8 lines
This option is actually kind of appealing in that we don't need to change any application side to filter out kernel tasks, as well as not having a new kernel dependency to even report these types of tasks as kernel threads.
I gave it a quick try. I'm not very familiar with the code here, so not sure if this is thread safe or not, but it did successfully get crosvm booting again.
That looks good to me too. Would you like to send it with a commit message and SoB?
Thanks,
Paolo
Quoted 95 lines
---
arch/x86/kvm/mmu/mmu.c +2
-0@@ -7415,6 +7415,8 @@ int kvm_mmu_post_init_vm(struct kvm *kvm) {if (nx_hugepage_mitigation_hard_disabled)return 0 ; +if (kvm->arch.nx_huge_page_recovery_thread) +return 0 ; kvm->arch.nx_huge_page_last =get_jiffies_64 (); kvm->arch.nx_huge_page_recovery_thread = vhost_task_create(arch/x86/kvm/x86.c +4
-5@@ -11463,6 +11463,10 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)struct kvm_run *kvm_run = vcpu->run;int r; + r =kvm_mmu_post_init_vm (vcpu->kvm); +if (r) +return r; +vcpu_load (vcpu);kvm_sigset_activate (vcpu); kvm_run->flags =0 ;@@ -12740,11 +12744,6 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)return ret; } -int kvm_arch_post_init_vm (struct kvm *kvm) -{ -return kvm_mmu_post_init_vm (kvm); -} -static void kvm_unload_vcpu_mmu (struct kvm_vcpu *vcpu) {vcpu_load (vcpu);include/linux/kvm_host.h +0
-1@@ -1596,7 +1596,6 @@ int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu);bool kvm_arch_dy_runnable (struct kvm_vcpu *vcpu);bool kvm_arch_dy_has_pending_interrupt (struct kvm_vcpu *vcpu);bool kvm_arch_vcpu_preempted_in_kernel (struct kvm_vcpu *vcpu); -int kvm_arch_post_init_vm (struct kvm *kvm);void kvm_arch_pre_destroy_vm (struct kvm *kvm);void kvm_arch_create_vm_debugfs (struct kvm *kvm);virt/kvm/kvm_main.c +0
-16@@ -1065,15 +1065,6 @@ static int kvm_create_vm_debugfs(struct kvm *kvm, const char *fdname)return ret; } -/* -* Called after the VM is otherwise initialized, but just before adding it to -* the vm_list. -*/ -int __weak kvm_arch_post_init_vm (struct kvm *kvm) -{ -return 0 ; -} -/* * Called just after removing the VM from the vm_list, but before doing any * other destruction. @@ -1194,10 +1185,6 @@ static struct kvm *kvm_create_vm(unsigned long type, const char *fdname)if (r) goto out_err_no_debugfs; - r =kvm_arch_post_init_vm (kvm); -if (r) - goto out_err; -mutex_lock (&kvm_lock);list_add (&kvm->vm_list, &vm_list);mutex_unlock (&kvm_lock);@@ -1207,8 +1194,6 @@ static struct kvm *kvm_create_vm(unsigned long type, const char *fdname)return kvm; -out_err: -kvm_destroy_vm_debugfs (kvm); out_err_no_debugfs:kvm_coalesced_mmio_free (kvm); out_no_coalesced_mmio: --
Sean Christopherson
maintainerQuoted 11 lines · Paolo Bonzini
Quoted 10 lines · Keith Busch
Quoted 9 lines
Ok, I found the code and it doesn't exec (e.g.
https://github.com/google/crosvm/blob/b339d3d7/src/crosvm/sys/linux/jail_warden.rs#L122),
so that's not an option. Well, if I understand correctly from a cursory look at the code, crosvm is creating a jailed child process early, and then spawns further jails through it; so it's just this first process that has to cheat.One possibility on the KVM side is to delay creating the
vhost_taskuntil the firstKVM_RUN. I don't like it but...
Quoted 7 lines
Quoted 7 lines
This option is actually kind of appealing in that we don't need to change any application side to filter out kernel tasks, as well as not having a new kernel dependency to even report these types of tasks as kernel threads.
I gave it a quick try. I'm not very familiar with the code here, so not sure if this is thread safe or not,
It's not.
Quoted 16 lines
Quoted 1 line
but it did successfully get crosvm booting again.
That looks good to me too. Would you like to send it with a commit message and SoB?
Quoted 11 lines
---
arch/x86/kvm/mmu/mmu.c +2
-0@@ -7415,6 +7415,8 @@ int kvm_mmu_post_init_vm(struct kvm *kvm) {if (nx_hugepage_mitigation_hard_disabled)return 0 ; +if (kvm->arch.nx_huge_page_recovery_thread) +return 0 ;
...
Quoted 4 lines
Quoted 4 lines
kvm->arch.nx_huge_page_last = get_jiffies_64(); kvm->arch.nx_huge_page_recovery_thread = vhost_task_create(arch/x86/kvm/x86.c +0
-0
Quoted 8 lines
arch/x86/kvm/x86.c int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) · line 11467
Quoted 8 lines
arch/x86/kvm/x86.c +3
-0@@ -11463,6 +11463,10 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)struct kvm_run *kvm_run = vcpu->run;int r; + r =kvm_mmu_post_init_vm (vcpu->kvm); +if (r) +return r;
The only lock held at this point is vcpu->mutex, the obvious choices for guarding the per-VM task creation are kvm->lock or kvm->mmu_lock, but we definitely don't want to blindly take either lock in KVM_RUN.
Quoted 13 lines · Sean Christopherson
Quoted 10 lines
Quoted 10 lines
arch/x86/kvm/mmu/mmu.c +2
-0@@ -7415,6 +7415,8 @@ int kvm_mmu_post_init_vm(struct kvm *kvm) {if (nx_hugepage_mitigation_hard_disabled)return 0 ; +if (kvm->arch.nx_huge_page_recovery_thread) +return 0 ;...
Quoted 16 lines
Quoted 4 lines
Quoted 4 lines
kvm->arch.nx_huge_page_last = get_jiffies_64(); kvm->arch.nx_huge_page_recovery_thread = vhost_task_create(arch/x86/kvm/x86.c +0
-0Quoted 8 lines
arch/x86/kvm/x86.c int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) · line 11467
Quoted 8 lines
arch/x86/kvm/x86.c +3
-0@@ -11463,6 +11463,10 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)struct kvm_run *kvm_run = vcpu->run;int r; + r =kvm_mmu_post_init_vm (vcpu->kvm); +if (r) +return r;The only lock held at this point is
vcpu->mutex, the obvious choices for guarding the per-VM task creation arekvm->lockorkvm->mmu_lock, but we definitely don't want to blindly take either lock inKVM_RUN.
Thanks for the feedback. Would this otherwise be okay if I use a different mechanism to ensure the vhost task creation happens only once per kvm instance? Or are you suggesting the task creation needs to be somewhere other than KVM_RUN?
My other initial concern was that this makes kvm_destroy_vm less symmetrical to kvm_create_vm, but that part looks okay: the vcpu that's being run holds a reference preventing kvm_destroy_vm from being called.
Paolo Bonzini
maintainerQuoted 32 lines · Keith Busch
Quoted 13 lines · Sean Christopherson
Quoted 10 lines
Quoted 10 lines
arch/x86/kvm/mmu/mmu.c +2
-0@@ -7415,6 +7415,8 @@ int kvm_mmu_post_init_vm(struct kvm *kvm) {if (nx_hugepage_mitigation_hard_disabled)return 0 ; +if (kvm->arch.nx_huge_page_recovery_thread) +return 0 ;...
Quoted 16 lines
Quoted 4 lines
Quoted 4 lines
kvm->arch.nx_huge_page_last = get_jiffies_64(); kvm->arch.nx_huge_page_recovery_thread = vhost_task_create(arch/x86/kvm/x86.c +0
-0Quoted 8 lines
arch/x86/kvm/x86.c int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) · line 11467
Quoted 8 lines
arch/x86/kvm/x86.c +3
-0@@ -11463,6 +11463,10 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)struct kvm_run *kvm_run = vcpu->run;int r; + r =kvm_mmu_post_init_vm (vcpu->kvm); +if (r) +return r;The only lock held at this point is
vcpu->mutex, the obvious choices for guarding the per-VM task creation arekvm->lockorkvm->mmu_lock, but we definitely don't want to blindly take either lock inKVM_RUN.
Quoted 4 lines
Thanks for the feedback. Would this otherwise be okay if I use a different mechanism to ensure the vhost task creation happens only once per kvm instance? Or are you suggesting the task creation needs to be somewhere other than
KVM_RUN?
You can implement something like pthread_once():
#define ONCE_NOT_STARTED 0
#define ONCE_RUNNING 1
#define ONCE_COMPLETED 2
struct once {
atomic_t state;
struct mutex lock;
}static inline void __once_init(struct once *once,
const char *name, struct lock_class_key *key)
{
atomic_set(&once->state, ONCE_NOT_STARTED);
__mutex_init(&once->lock, name, key);
}#define once_init(once) \
do { \
static struct lock_class_key __key; \
\
__once_init((once), #once, &__key); \
} while (0)static inline void call_once(struct once *once, void (*cb)(struct once *))
{
/* Pairs with atomic_set_release() below. */
if (atomic_read_acquire(&once->state) == ONCE_COMPLETED)
return;
guard(mutex)(&once->lock);
WARN_ON(atomic_read(&once->state) == ONCE_RUNNING);
if (atomic_read(&once->state) != ONCE_NOT_STARTED)
return;
atomic_set(&once->state, ONCE_RUNNING);
cb(once);
atomic_set_release(&once->state, ONCE_COMPLETED);
}Where to put it I don't know. It doesn't belong in include/linux/once.h. I'm okay with arch/x86/kvm/call_once.h and just pull it with #include "call_once.h".
Paolo
Quoted 1 line · Paolo Bonzini
You can implement something like
pthread_once():
...
Quoted 3 lines
Where to put it I don't know. It doesn't belong in
include/linux/once.h. I'm okay witharch/x86/kvm/call_once.hand just pull it with #include "call_once.h".
Thanks for the suggestion, I can work with that. As to where to put it, I think the new 'struct once' needs to be a member of struct kvm_arch, so I've put it in arch/x86/include/asm/.
Here's the result with that folded in. If this is okay, I'll send a v2, and can split out the call_once as a prep patch with your attribution if you like.
---
arch/x86/include/asm/kvm_call_once.h new file +44 -0
@@ -0,0 +1,44 @@ +#ifndef _LINUX_CALL_ONCE_H +#define _LINUX_CALL_ONCE_H + +#include <linux/types.h> + +#define ONCE_NOT_STARTED 0 +#define ONCE_RUNNING 1 +#define ONCE_COMPLETED 2 + +struct once { +atomic_t state; +struct mutex lock; +}; + +static inline void __once_init (struct once *once,const char *name, +struct lock_class_key *key) +{ +atomic_set (&once->state,ONCE_NOT_STARTED ); +__mutex_init (&once->lock, name, key); +} + +#define once_init (once) \ +do { \ + static struct lock_class_key __key; \ + __once_init((once), #once, &__key); \ +} while (0) + +static inline void call_once (struct once *once,void (*cb)(struct once *)) +{ +/* Pairs with atomic_set_release() below. */ +if (atomic_read_acquire (&once->state) ==ONCE_COMPLETED ) +return ; + +guard (mutex)(&once->lock); +WARN_ON (atomic_read (&once->state) ==ONCE_RUNNING ); +if (atomic_read (&once->state) !=ONCE_NOT_STARTED ) +return ; + +atomic_set (&once->state,ONCE_RUNNING ); +cb (once); +atomic_set_release (&once->state,ONCE_COMPLETED ); +} + +#endif /* _LINUX_CALL_ONCE_H */
arch/x86/include/asm/kvm_host.h +2 -0
@@ -37,6 +37,7 @@#include <asm/kvm_page_track.h> #include <asm/kvm_vcpu_regs.h> #include <asm/hyperv-tlfs.h> +#include <asm/kvm_call_once.h> #include <asm/reboot.h> #define __KVM_HAVE_ARCH_VCPU_DEBUGFS
@@ -1445,6 +1446,7 @@ struct kvm_arch {struct kvm_x86_pmu_event_filter __rcu *pmu_event_filter;struct vhost_task *nx_huge_page_recovery_thread;u64 nx_huge_page_last; +struct once nx_once;#ifdef CONFIG_X86_64 /* The number of TDP MMU pages across all roots. */
arch/x86/kvm/mmu/mmu.c +13 -5
@@ -7411,20 +7411,28 @@ static bool kvm_nx_huge_page_recovery_worker(void *data)return true; } -int kvm_mmu_post_init_vm (struct kvm *kvm) +static void kvm_mmu_start_lpage_recovery (struct once *once) { -if (nx_hugepage_mitigation_hard_disabled) -return 0 ; + struct kvm_arch *ka = container_of(once, struct kvm_arch, nx_once); + struct kvm *kvm = container_of(ka, struct kvm, arch); kvm->arch.nx_huge_page_last =get_jiffies_64 (); kvm->arch.nx_huge_page_recovery_thread =vhost_task_create ( kvm_nx_huge_page_recovery_worker, kvm_nx_huge_page_recovery_worker_kill, kvm,"kvm-nx-lpage-recovery" ); +if (kvm->arch.nx_huge_page_recovery_thread) +vhost_task_start (kvm->arch.nx_huge_page_recovery_thread); +} + +int kvm_mmu_post_init_vm (struct kvm *kvm) +{ +if (nx_hugepage_mitigation_hard_disabled) +return 0 ; + +call_once (&kvm->arch.nx_once, kvm_mmu_start_lpage_recovery);if (!kvm->arch.nx_huge_page_recovery_thread)return -ENOMEM ; - -vhost_task_start (kvm->arch.nx_huge_page_recovery_thread);return 0 ; }
arch/x86/kvm/x86.c +6 -1
@@ -11463,6 +11463,10 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)struct kvm_run *kvm_run = vcpu->run;int r; + r =kvm_mmu_post_init_vm (vcpu->kvm); +if (r) +return r; +vcpu_load (vcpu);kvm_sigset_activate (vcpu); kvm_run->flags =0 ;
@@ -12742,7 +12746,8 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)int kvm_arch_post_init_vm (struct kvm *kvm) { -return kvm_mmu_post_init_vm (kvm); +once_init (&kvm->arch.nx_once); +return 0 ; }static void kvm_unload_vcpu_mmu (struct kvm_vcpu *vcpu)
--
Quoted 9 lines · Keith Busch
Quoted 1 line · Paolo Bonzini
You can implement something like
pthread_once():...
Quoted 3 lines
Where to put it I don't know. It doesn't belong in
include/linux/once.h. I'm okay witharch/x86/kvm/call_once.hand just pull it with #include "call_once.h".
Quoted 7 lines
Thanks for the suggestion, I can work with that. As to where to put it, I think the new 'struct once' needs to be a member of struct
kvm_arch, so I've put it inarch/x86/include/asm/.Here's the result with that folded in. If this is okay, I'll send a v2, and can split out the
call_onceas a prep patch with your attribution if you like.
Has there been any progress here? I'm also affected by the crosvm regression, and it's been backported to the LTS stable kernel.
(CCing the stable and regressions lists to make sure the regression is tracked.)
#regzbot introduced: d96c77bd4eeb
Quoted 14 lines · Alyssa Ross
Quoted 13 lines · Keith Busch
Quoted 1 line · Paolo Bonzini
You can implement something like
pthread_once():...
Quoted 3 lines
Where to put it I don't know. It doesn't belong in
include/linux/once.h. I'm okay witharch/x86/kvm/call_once.hand just pull it with #include "call_once.h".Thanks for the suggestion, I can work with that. As to where to put it, I think the new 'struct once' needs to be a member of struct
kvm_arch, so I've put it inarch/x86/include/asm/.
Quoted 6 lines
Quoted 3 lines
Here's the result with that folded in. If this is okay, I'll send a v2, and can split out the
call_onceas a prep patch with your attribution if you like.Has there been any progress here? I'm also affected by the crosvm regression, and it's been backported to the LTS stable kernel.
Would you be able to try the proposed patch here and reply with a Tested-by if it's successful for you? I'd also like to unblock this, whether this patch is in the right direction or try something else.
Alyssa Ross
Tested-byQuoted 19 lines · Keith Busch
Quoted 18 lines · Alyssa Ross
Quoted 16 lines · Keith Busch
Quoted 1 line · Paolo Bonzini
You can implement something like
pthread_once():...
Quoted 3 lines
Where to put it I don't know. It doesn't belong in
include/linux/once.h. I'm okay witharch/x86/kvm/call_once.hand just pull it with #include "call_once.h".Thanks for the suggestion, I can work with that. As to where to put it, I think the new 'struct once' needs to be a member of struct
kvm_arch, so I've put it inarch/x86/include/asm/.Here's the result with that folded in. If this is okay, I'll send a v2, and can split out the
call_onceas a prep patch with your attribution if you like.
Quoted 6 lines
Quoted 2 lines
Has there been any progress here? I'm also affected by the crosvm regression, and it's been backported to the LTS stable kernel.
Would you be able to try the proposed patch here and reply with a Tested-by if it's successful for you? I'd also like to unblock this, whether this patch is in the right direction or try something else.
Sure! I can confirm this patch fixes crosvm for me when applied to 6.13.
Sasha Levin
maintainer[ Sasha's backport helper bot ]
Hi,
Found matching upstream commit: d96c77bd4eeba469bddbbb14323d2191684da82a
WARNING: Author mismatch between patch and found commit:
Backport author: Keith Busch<[email protected]> Commit author: Paolo Bonzini<[email protected]>
Status in newer kernel trees:
6.12.y | Present (different SHA1: 91248a2e4101)
Note: The patch differs from the upstream commit:
---
1: d96c77bd4eeba < -: ------------- KVM: x86: switch hugepage recovery thread to vhost_task -: ------------- > 1: e24606748e041 KVM: x86: switch hugepage recovery thread to vhost_task
---
Results of testing on various branches:
| Branch | Patch Apply | Build Test | |---------------------------|-------------|------------| | stable/linux-6.12.y | Success | Success | | stable/linux-6.6.y | Failed | N/A | | stable/linux-6.1.y | Failed | N/A | | stable/linux-5.15.y | Failed | N/A | | stable/linux-5.10.y | Failed | N/A | | stable/linux-5.4.y | Failed | N/A |
Sasha Levin
maintainer[ Sasha's backport helper bot ]
Hi,
Found matching upstream commit: d96c77bd4eeba469bddbbb14323d2191684da82a
WARNING: Author mismatch between patch and found commit:
Backport author: Keith Busch<[email protected]> Commit author: Paolo Bonzini<[email protected]>
Status in newer kernel trees:
6.12.y | Present (different SHA1: 91248a2e4101)
Note: The patch differs from the upstream commit:
---
1: d96c77bd4eeba < -: ------------- KVM: x86: switch hugepage recovery thread to vhost_task -: ------------- > 1: d7ca669c0cbba KVM: x86: switch hugepage recovery thread to vhost_task
---
Results of testing on various branches:
| Branch | Patch Apply | Build Test | |---------------------------|-------------|------------| | stable/linux-6.12.y | Success | Success | | stable/linux-6.6.y | Failed | N/A | | stable/linux-6.1.y | Failed | N/A | | stable/linux-5.15.y | Failed | N/A | | stable/linux-5.10.y | Failed | N/A | | stable/linux-5.4.y | Failed | N/A |
Sasha Levin
maintainer[ Sasha's backport helper bot ]
Hi,
Found matching upstream commit: d96c77bd4eeba469bddbbb14323d2191684da82a
Status in newer kernel trees:
6.12.y | Present (different SHA1: 91248a2e4101)
Note: The patch differs from the upstream commit:
---
Failed to apply patch cleanly, falling back to interdiff...
---
Results of testing on various branches:
| Branch | Patch Apply | Build Test | |---------------------------|-------------|------------| | stable/linux-6.12.y | Failed | N/A | | stable/linux-6.6.y | Failed | N/A | | stable/linux-6.1.y | Failed | N/A | | stable/linux-5.15.y | Failed | N/A | | stable/linux-5.10.y | Failed | N/A | | stable/linux-5.4.y | Failed | N/A |