kthread: Introduce preferred affinity v2
Subsystems (46)
Frederic Weisbecker
maintainercover kthread: Introduce preferred affinity v2
Affining kthreads follow either of three existing different patterns:
1) Per-CPU kthreads must stay affine to a single CPU and never execute relevant code on any other CPU. This is currently handled by smpboot code which takes care of CPU-hotplug operations.
2) Kthreads that _have_ to be affine to a specific set of CPUs and can't run anywhere else. The affinity is set through kthread_bind_mask() and the subsystem takes care by itself to handle CPU-hotplug operations.
3) Kthreads that _prefer_ to be affine to a specific NUMA node.
4) Similar to the previous point but kthreads have a _preferred_ affinity different than a node. It is set manually like any other task and CPU-hotplug is supposed to be handled by the relevant subsystem so that the task is properly reaffined whenever a given CPU from the preferred affinity comes up or down. Also care must be taken so that the preferred affinity doesn't cross housekeeping cpumask boundaries.
Currently the preferred affinity patterns (3 and 4) have at least 4 identified users, with more or less success when it comes to handle CPU-hotplug operations and CPU isolation.
This is a infrastructure proposal to handle this (after cleanups from 01 to 10).
Frederic Weisbecker (19):
arm/bL_switcher: Use kthread_run_on_cpu() x86/resctrl: Use kthread_run_on_cpu()
firmware: stratix10-svc: Use kthread_run_on_cpu()
scsi: bnx2fc: Use kthread_create_on_cpu()
scsi: bnx2i: Use kthread_create_on_cpu()
scsi: qedi: Use kthread_create_on_cpu() soc/qman: test: Use kthread_run_on_cpu()
kallsyms: Use kthread_run_on_cpu()
lib: test_objpool: Use kthread_run_on_cpu()
net: pktgen: Use kthread_create_on_node()
kthread: Make sure kthread hasn't started while binding it
kthread: Default affine kthread to its preferred NUMA node
mm: Create/affine kcompactd to its preferred node
mm: Create/affine kswapd to its preferred node
kthread: Implement preferred affinity
rcu: Use kthread preferred affinity for RCU boost
kthread: Unify kthread_create_on_cpu() and
kthread_create_worker_on_cpu() automatic format
treewide: Introduce kthread_run_worker[_on_cpu]()
rcu: Use kthread preferred affinity for RCU exp kworkers
arch/arm/common/bL_switcher.c | 10 +- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 28 +-- arch/x86/kvm/i8254.c | 2 +- crypto/crypto_engine.c | 2 +- drivers/cpufreq/cppc_cpufreq.c | 2 +- drivers/firmware/stratix10-svc.c | 9 +- drivers/gpu/drm/drm_vblank_work.c | 2 +- .../drm/i915/gem/selftests/i915_gem_context.c | 2 +- drivers/gpu/drm/i915/gt/selftest_execlists.c | 2 +- drivers/gpu/drm/i915/gt/selftest_hangcheck.c | 2 +- drivers/gpu/drm/i915/gt/selftest_slpc.c | 2 +- drivers/gpu/drm/i915/selftests/i915_request.c | 8 +- drivers/gpu/drm/msm/disp/msm_disp_snapshot.c | 2 +- drivers/gpu/drm/msm/msm_atomic.c | 2 +- drivers/gpu/drm/msm/msm_gpu.c | 2 +- drivers/gpu/drm/msm/msm_kms.c | 2 +- .../platform/chips-media/wave5/wave5-vpu.c | 2 +- drivers/net/dsa/mv88e6xxx/chip.c | 2 +- drivers/net/ethernet/intel/ice/ice_dpll.c | 2 +- drivers/net/ethernet/intel/ice/ice_gnss.c | 2 +- drivers/net/ethernet/intel/ice/ice_ptp.c | 2 +- drivers/platform/chrome/cros_ec_spi.c | 2 +- drivers/ptp/ptp_clock.c | 2 +- drivers/scsi/bnx2fc/bnx2fc_fcoe.c | 7 +- drivers/scsi/bnx2i/bnx2i_init.c | 7 +- drivers/scsi/qedi/qedi_main.c | 6 +- drivers/soc/fsl/qbman/qman_test_stash.c | 6 +- drivers/spi/spi.c | 2 +- drivers/usb/typec/tcpm/tcpm.c | 2 +- drivers/vdpa/vdpa_sim/vdpa_sim.c | 2 +- drivers/watchdog/watchdog_dev.c | 2 +- fs/erofs/zdata.c | 2 +- include/linux/cpuhotplug.h | 1 + include/linux/kthread.h | 56 ++++- kernel/kallsyms_selftest.c | 4 +- kernel/kthread.c | 214 ++++++++++++++++-- kernel/rcu/tree.c | 94 ++------ kernel/rcu/tree_plugin.h | 11 +- kernel/workqueue.c | 2 +- lib/test_objpool.c | 19 +- mm/compaction.c | 43 +--- mm/vmscan.c | 7 +- net/core/pktgen.c | 7 +- net/dsa/tag_ksz.c | 2 +- net/dsa/tag_ocelot_8021q.c | 2 +- net/dsa/tag_sja1105.c | 2 +- 46 files changed, 334 insertions(+), 259 deletions(-)
Signature · 1 line
2.45.2
Frederic Weisbecker
maintainerPATCH 1/19 arm/bL_switcher: Use kthread_run_on_cpu()
Use the proper API instead of open coding it.
--- arch/arm/common/bL_switcher.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-)
arch/arm/common/bL_switcher.c +4 -6
@@ -307,13 +307,11 @@ static struct task_struct *bL_switcher_thread_create(int cpu, void *arg) {struct task_struct *task; - task =kthread_create_on_node (bL_switcher_thread, arg, -cpu_to_node (cpu),"kswitcher_%d" , cpu); -if (!IS_ERR (task)) { -kthread_bind (task, cpu); -wake_up_process (task); - }else + task =kthread_run_on_cpu (bL_switcher_thread, arg, + cpu,"kswitcher_%d" ); +if (IS_ERR (task))pr_err ("%s failed for CPU %d\n" , __func__, cpu); +return task; }
Signature · 1 line
2.45.2
Frederic Weisbecker
maintainerPATCH 2/19 x86/resctrl: Use kthread_run_on_cpu()
In reply to Frederic WeisbeckerUse the proper API instead of open coding it.
--- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 28 +++++++---------------- 1 file changed, 8 insertions(+), 20 deletions(-)
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +8 -20
@@ -1205,20 +1205,14 @@ static int pseudo_lock_measure_cycles(struct rdtgroup *rdtgrp, int sel) plr->cpu = cpu;if (sel ==1 ) - thread =kthread_create_on_node (measure_cycles_lat_fn, plr, -cpu_to_node (cpu), -"pseudo_lock_measure/%u" , - cpu); + thread =kthread_run_on_cpu (measure_cycles_lat_fn, plr, + cpu,"pseudo_lock_measure/%u" );else if (sel ==2 ) - thread =kthread_create_on_node (measure_l2_residency, plr, -cpu_to_node (cpu), -"pseudo_lock_measure/%u" , - cpu); + thread =kthread_run_on_cpu (measure_l2_residency, plr, + cpu,"pseudo_lock_measure/%u" );else if (sel ==3 ) - thread =kthread_create_on_node (measure_l3_residency, plr, -cpu_to_node (cpu), -"pseudo_lock_measure/%u" , - cpu); + thread =kthread_run_on_cpu (measure_l3_residency, plr, + cpu,"pseudo_lock_measure/%u" );else goto out;
@@ -1226,8 +1220,6 @@ static int pseudo_lock_measure_cycles(struct rdtgroup *rdtgrp, int sel) ret =PTR_ERR (thread); goto out; } -kthread_bind (thread, cpu); -wake_up_process (thread); ret =wait_event_interruptible (plr->lock_thread_wq, plr->thread_done ==1 );
@@ -1315,18 +1307,14 @@ int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtgrp) plr->thread_done =0 ; - thread =kthread_create_on_node (pseudo_lock_fn, rdtgrp, -cpu_to_node (plr->cpu), -"pseudo_lock/%u" , plr->cpu); + thread =kthread_run_on_cpu (pseudo_lock_fn, rdtgrp, + plr->cpu,"pseudo_lock/%u" );if (IS_ERR (thread)) { ret =PTR_ERR (thread);rdt_last_cmd_printf ("Locking thread returned error %d\n" , ret); goto out_cstates; } -kthread_bind (thread, plr->cpu); -wake_up_process (thread); - ret =wait_event_interruptible (plr->lock_thread_wq, plr->thread_done ==1 );if (ret <0 ) {
Signature · 1 line
2.45.2
Frederic Weisbecker
maintainerPATCH 3/19 firmware: stratix10-svc: Use kthread_run_on_cpu()
In reply to Frederic WeisbeckerUse the proper API instead of open coding it.
--- drivers/firmware/stratix10-svc.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-)
drivers/firmware/stratix10-svc.c +3 -6
@@ -967,18 +967,15 @@ int stratix10_svc_send(struct stratix10_svc_chan *chan, void *msg)/* first client will create kernel thread */ if (!chan->ctrl->task) { chan->ctrl->task = -kthread_create_on_node (svc_normal_to_secure_thread, - (void *)chan->ctrl, -cpu_to_node (cpu), -"svc_smc_hvc_thread" ); +kthread_run_on_cpu (svc_normal_to_secure_thread, + (void *)chan->ctrl, + cpu,"svc_smc_hvc_thread" );if (IS_ERR (chan->ctrl->task)) {dev_err (chan->ctrl->dev,"failed to create svc_smc_hvc_thread\n" );kfree (p_data);return -EINVAL ; } -kthread_bind (chan->ctrl->task, cpu); -wake_up_process (chan->ctrl->task); } pr_debug("%s: sent P-va=%p, P-com=%x, P-size=%u\n", __func__,
Signature · 1 line
2.45.2
Frederic Weisbecker
maintainerPATCH 4/19 scsi: bnx2fc: Use kthread_create_on_cpu()
In reply to Frederic WeisbeckerUse the proper API instead of open coding it.
However it looks like bnx2fc_percpu_io_thread() kthread could be replaced by the use of a high prio workqueue instead.
--- drivers/scsi/bnx2fc/bnx2fc_fcoe.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)
drivers/scsi/bnx2fc/bnx2fc_fcoe.c +2 -5
@@ -2610,14 +2610,11 @@ static int bnx2fc_cpu_online(unsigned int cpu) p = &per_cpu (bnx2fc_percpu, cpu); - thread =kthread_create_on_node (bnx2fc_percpu_io_thread, - (void *)p,cpu_to_node (cpu), -"bnx2fc_thread/%d" , cpu); + thread =kthread_create_on_cpu (bnx2fc_percpu_io_thread, + (void *)p, cpu,"bnx2fc_thread/%d" );if (IS_ERR (thread))return PTR_ERR (thread); -/* bind thread to the cpu */ -kthread_bind (thread, cpu); p->iothread = thread;wake_up_process (thread);return 0 ;
Signature · 1 line
2.45.2
Frederic Weisbecker
maintainerPATCH 5/19 scsi: bnx2i: Use kthread_create_on_cpu()
In reply to Frederic WeisbeckerUse the proper API instead of open coding it.
However it looks like bnx2i_percpu_io_thread() kthread could be replaced by the use of a high prio workqueue instead.
--- drivers/scsi/bnx2i/bnx2i_init.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)
drivers/scsi/bnx2i/bnx2i_init.c +2 -5
@@ -415,14 +415,11 @@ static int bnx2i_cpu_online(unsigned int cpu) p = &per_cpu (bnx2i_percpu, cpu); - thread =kthread_create_on_node (bnx2i_percpu_io_thread, (void *)p, -cpu_to_node (cpu), -"bnx2i_thread/%d" , cpu); + thread =kthread_create_on_cpu (bnx2i_percpu_io_thread, (void *)p, + cpu,"bnx2i_thread/%d" );if (IS_ERR (thread))return PTR_ERR (thread); -/* bind thread to the cpu */ -kthread_bind (thread, cpu); p->iothread = thread;wake_up_process (thread);return 0 ;
Signature · 1 line
2.45.2
Frederic Weisbecker
maintainerPATCH 6/19 scsi: qedi: Use kthread_create_on_cpu()
In reply to Frederic WeisbeckerUse the proper API instead of open coding it.
However it looks like qedi_percpu_io_thread() kthread could be replaced by the use of a high prio workqueue instead.
--- drivers/scsi/qedi/qedi_main.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
drivers/scsi/qedi/qedi_main.c +2 -4
@@ -1960,13 +1960,11 @@ static int qedi_cpu_online(unsigned int cpu)struct qedi_percpu_s *p =this_cpu_ptr (&qedi_percpu);struct task_struct *thread; - thread =kthread_create_on_node (qedi_percpu_io_thread, (void *)p, -cpu_to_node (cpu), -"qedi_thread/%d" , cpu); + thread =kthread_create_on_cpu (qedi_percpu_io_thread, (void *)p, + cpu,"qedi_thread/%d" );if (IS_ERR (thread))return PTR_ERR (thread); -kthread_bind (thread, cpu); p->iothread = thread;wake_up_process (thread);return 0 ;
Signature · 1 line
2.45.2
Frederic Weisbecker
maintainerPATCH 7/19 soc/qman: test: Use kthread_run_on_cpu()
In reply to Frederic WeisbeckerUse the proper API instead of open coding it.
However it looks like kthreads here could be replaced by the use of a per-cpu workqueue instead.
--- drivers/soc/fsl/qbman/qman_test_stash.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
drivers/soc/fsl/qbman/qman_test_stash.c +2 -4
@@ -108,14 +108,12 @@ static int on_all_cpus(int (*fn)(void)) .fn = fn, .started = ATOMIC_INIT(0) }; -struct task_struct *k =kthread_create (bstrap_fn, &bstrap, -"hotpotato%d" , cpu); +struct task_struct *k =kthread_run_on_cpu (bstrap_fn, &bstrap, + cpu,"hotpotato%d" );int ret;if (IS_ERR (k))return -ENOMEM ; -kthread_bind (k, cpu); -wake_up_process (k);/* * If we call kthread_stop() before the "wake up" has had an * effect, then the thread may exit with -EINTR without ever
Signature · 1 line
2.45.2
Frederic Weisbecker
maintainerPATCH 8/19 kallsyms: Use kthread_run_on_cpu()
In reply to Frederic WeisbeckerUse the proper API instead of open coding it.
--- kernel/kallsyms_selftest.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
kernel/kallsyms_selftest.c +1 -3
@@ -455,13 +455,11 @@ static int __init kallsyms_test_init(void) {struct task_struct *t; - t =kthread_create (test_entry,NULL ,"kallsyms_test" ); + t =kthread_run_on_cpu (test_entry,NULL ,0 ,"kallsyms_test" );if (IS_ERR (t)) {pr_info ("Create kallsyms selftest task failed\n" );return PTR_ERR (t); } -kthread_bind (t,0 ); -wake_up_process (t);return 0 ; }
Signature · 1 line
2.45.2
Frederic Weisbecker
maintainerPATCH 9/19 lib: test_objpool: Use kthread_run_on_cpu()
In reply to Frederic WeisbeckerUse the proper API instead of open coding it.
--- lib/test_objpool.c | 19 +++++-------------- 1 file changed, 5 insertions(+), 14 deletions(-)
lib/test_objpool.c +5 -14
@@ -371,14 +371,10 @@ static int ot_start_sync(struct ot_test *test)if (!cpu_online (cpu))continue ; - work =kthread_create_on_node (ot_thread_worker, item, -cpu_to_node (cpu),"ot_worker_%d" , cpu); -if (IS_ERR (work)) { + work =kthread_run_on_cpu (ot_thread_worker, item, + cpu,"ot_worker_%d" ); +if (IS_ERR (work))pr_err ("failed to create thread for cpu %d\n" , cpu); - }else { -kthread_bind (work, cpu); -wake_up_process (work); - } }/* wait a while to make sure all threads waiting at start line */
@@ -562,14 +558,9 @@ static int ot_start_async(struct ot_test *test)if (!cpu_online (cpu))continue ; - work =kthread_create_on_node (ot_thread_worker, item, -cpu_to_node (cpu),"ot_worker_%d" , cpu); -if (IS_ERR (work)) { + work =kthread_run_on_cpu (ot_thread_worker, item, cpu,"ot_worker_%d" ); +if (IS_ERR (work))pr_err ("failed to create thread for cpu %d\n" , cpu); - }else { -kthread_bind (work, cpu); -wake_up_process (work); - } }/* wait a while to make sure all threads waiting at start line */
Signature · 1 line
2.45.2
Frederic Weisbecker
maintainerPATCH 10/19 net: pktgen: Use kthread_create_on_node()
In reply to Frederic WeisbeckerUse the proper API instead of open coding it.
--- net/core/pktgen.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)
net/core/pktgen.c +2 -5
@@ -3883,17 +3883,14 @@ static int __net_init pktgen_create_thread(int cpu, struct pktgen_net *pn)list_add_tail (&t->th_list, &pn->pktgen_threads);init_completion (&t->start_done); - p =kthread_create_on_node (pktgen_thread_worker, - t, -cpu_to_node (cpu), -"kpktgend_%d" , cpu); + p =kthread_create_on_cpu (pktgen_thread_worker, t, cpu,"kpktgend_%d" );if (IS_ERR (p)) {pr_err ("kthread_create_on_node() failed for cpu %d\n" , t->cpu);list_del (&t->th_list);kfree (t);return PTR_ERR (p); } -kthread_bind (p, cpu); + t->tsk = p; pe = proc_create_data(t->tsk->comm, 0600, pn->proc_dir,
Signature · 1 line
2.45.2
Frederic Weisbecker
maintainerPATCH 11/19 kthread: Make sure kthread hasn't started while binding it
In reply to Frederic WeisbeckerMake sure the kthread is sleeping in the schedule_preempt_disabled() call before calling its handler when kthread_bind[_mask]() is called on it. This provides a sanity check verifying that the task is not randomly blocked later at some point within its function handler, in which case it could be just concurrently awaken, leaving the call to do_set_cpus_allowed() without any effect until the next voluntary sleep.
Rely on the wake-up ordering to ensure that the newly introduced "started" field returns the expected value:
TASK A TASK B
------ ------
READ kthread->startedwake_up_process(B)
rq_lock()
...
rq_unlock() // RELEASE
schedule()
rq_lock() // ACQUIRE
// schedule task B
rq_unlock()
WRITE kthread->startedSimilarly, writing kthread->started before subsequent voluntary sleeps will be visible after calling wait_task_inactive() in __kthread_bind_mask(), reporting potential misuse of the API.
Upcoming patches will make further use of this facility.
--- kernel/kthread.c | 7 +++++++ 1 file changed, 7 insertions(+)
kernel/kthread.c +7 -0
@@ -53,6 +53,7 @@ struct kthread_create_infostruct kthread {unsigned long flags;unsigned int cpu; +int started;int result;int (*threadfn)(void *);void *data;
@@ -382,6 +383,8 @@ static int kthread(void *_create)schedule_preempt_disabled ();preempt_enable (); + self->started =1 ; + ret = -EINTR ;if (!test_bit (KTHREAD_SHOULD_STOP , &self->flags)) {cgroup_kthread_ready ();
@@ -540,7 +543,9 @@ static void __kthread_bind(struct task_struct *p, unsigned int cpu, unsigned intvoid kthread_bind_mask (struct task_struct *p,const struct cpumask *mask) { +struct kthread *kthread =to_kthread (p);__kthread_bind_mask (p, mask,TASK_UNINTERRUPTIBLE ); +WARN_ON_ONCE (kthread->started); }/**
@@ -554,7 +559,9 @@ void kthread_bind_mask(struct task_struct *p, const struct cpumask *mask)*/ void kthread_bind (struct task_struct *p,unsigned int cpu) { +struct kthread *kthread =to_kthread (p);__kthread_bind (p, cpu,TASK_UNINTERRUPTIBLE ); +WARN_ON_ONCE (kthread->started); }EXPORT_SYMBOL (kthread_bind);
Signature · 1 line
2.45.2
Frederic Weisbecker
maintainerPATCH 12/19 kthread: Default affine kthread to its preferred NUMA node
In reply to Frederic WeisbeckerKthreads attached to a preferred NUMA node for their task structure allocation can also be assumed to run preferrably within that same node.
A more precise affinity is usually notified by calling kthread_create_on_cpu() or kthread_bind[_mask]() before the first wakeup.
For the others, a default affinity to the node is desired and sometimes implemented with more or less success when it comes to deal with hotplug events and nohz_full / CPU Isolation interactions:
- kcompactd is affine to its node and handles hotplug but not CPU Isolation
- kswapd is affine to its node and ignores hotplug and CPU Isolation
- A bunch of drivers create their kthreads on a specific node and don't take care about affining further.
Handle that default node affinity preference at the generic level instead, provided a kthread is created on an actual node and doesn't apply any specific affinity such as a given CPU or a custom cpumask to bind to before its first wake-up.
This generic handling is aware of CPU hotplug events and CPU isolation such that:
* When a housekeeping CPU goes up and is part of the node of a given kthread, it is added to its applied affinity set (and possibly the default last resort online housekeeping set is removed from the set).
* When a housekeeping CPU goes down while it was part of the node of a kthread, it is removed from the kthread's applied affinity. The last resort is to affine the kthread to all online housekeeping CPUs.
--- include/linux/cpuhotplug.h | 1 + kernel/kthread.c | 120 ++++++++++++++++++++++++++++++++++++- 2 files changed, 120 insertions(+), 1 deletion(-)
include/linux/cpuhotplug.h +1 -0
@@ -240,6 +240,7 @@ enum cpuhp_state {CPUHP_AP_WORKQUEUE_ONLINE ,CPUHP_AP_RANDOM_ONLINE ,CPUHP_AP_RCUTREE_ONLINE , +CPUHP_AP_KTHREADS_ONLINE ,CPUHP_AP_BASE_CACHEINFO_ONLINE ,CPUHP_AP_ONLINE_DYN , CPUHP_AP_ONLINE_DYN_END = CPUHP_AP_ONLINE_DYN + 40,
kernel/kthread.c +119 -1
176 lines not on this page · raw message
Signature · 1 line
2.45.2
Frederic Weisbecker
maintainerPATCH 13/19 mm: Create/affine kcompactd to its preferred node
In reply to Frederic WeisbeckerKcompactd is dedicated to a specific node. As such it wants to be preferrably affine to it, memory and CPUs-wise.
Use the proper kthread API to achieve that. As a bonus it takes care of CPU-hotplug events and CPU-isolation on its behalf.
--- mm/compaction.c | 43 +++---------------------------------------- 1 file changed, 3 insertions(+), 40 deletions(-)
mm/compaction.c +3 -40
@@ -3179,15 +3179,9 @@ void wakeup_kcompactd(pg_data_t *pgdat, int order, int highest_zoneidx)static int kcompactd (void *p) {pg_data_t *pgdat = (pg_data_t *)p; -struct task_struct *tsk = current;long default_timeout =msecs_to_jiffies (HPAGE_FRAG_CHECK_INTERVAL_MSEC );long timeout = default_timeout; -const struct cpumask *cpumask =cpumask_of_node (pgdat->node_id); - -if (!cpumask_empty (cpumask)) -set_cpus_allowed_ptr (tsk, cpumask); -set_freezable (); pgdat->kcompactd_max_order =0 ;
@@ -3258,10 +3252,12 @@ void __meminit kcompactd_run(int nid)if (pgdat->kcompactd)return ; - pgdat->kcompactd =kthread_run (kcompactd, pgdat,"kcompactd%d" , nid); + pgdat->kcompactd =kthread_create_on_node (kcompactd, pgdat, nid,"kcompactd%d" , nid);if (IS_ERR (pgdat->kcompactd)) {pr_err ("Failed to start kcompactd on node %d\n" , nid); pgdat->kcompactd =NULL ; + }else { +wake_up_process (pgdat->kcompactd); } }
@@ -3279,30 +3275,6 @@ void __meminit kcompactd_stop(int nid) } } -/* -* It's optimal to keep kcompactd on the same CPUs as their memory, but -* not required for correctness. So if the last cpu in a node goes -* away, we get changed to run anywhere: as the first one comes back, -* restore their cpu bindings. -*/ -static int kcompactd_cpu_online (unsigned int cpu) -{ -int nid; - -for_each_node_state (nid,N_MEMORY ) { -pg_data_t *pgdat =NODE_DATA (nid); -const struct cpumask *mask; - - mask =cpumask_of_node (pgdat->node_id); - -if (cpumask_any_and (cpu_online_mask, mask) < nr_cpu_ids) -/* One of our CPUs online: restore mask */ -if (pgdat->kcompactd) -set_cpus_allowed_ptr (pgdat->kcompactd, mask); - } -return 0 ; -} -static int proc_dointvec_minmax_warn_RT_change (const struct ctl_table *table,int write,void *buffer,size_t *lenp,loff_t *ppos) {
@@ -3362,15 +3334,6 @@ static struct ctl_table vm_compaction[] = {static int __init kcompactd_init (void ) {int nid; -int ret; - - ret =cpuhp_setup_state_nocalls (CPUHP_AP_ONLINE_DYN , -"mm/compaction:online" , - kcompactd_cpu_online,NULL ); -if (ret <0 ) { -pr_err ("kcompactd: failed to register hotplug callbacks.\n" ); -return ret; - }for_each_node_state (nid,N_MEMORY )kcompactd_run (nid);
Signature · 1 line
2.45.2
Frederic Weisbecker
maintainerPATCH 14/19 mm: Create/affine kswapd to its preferred node
In reply to Frederic Weisbeckerkswapd is dedicated to a specific node. As such it wants to be preferrably affine to it, memory and CPUs-wise.
Use the proper kthread API to achieve that. As a bonus it takes care of CPU-hotplug events and CPU-isolation on its behalf.
--- mm/vmscan.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)
mm/vmscan.c +2 -5
@@ -7159,10 +7159,6 @@ static int kswapd(void *p)unsigned int highest_zoneidx =MAX_NR_ZONES -1 ;pg_data_t *pgdat = (pg_data_t *)p;struct task_struct *tsk = current; -const struct cpumask *cpumask =cpumask_of_node (pgdat->node_id); - -if (!cpumask_empty (cpumask)) -set_cpus_allowed_ptr (tsk, cpumask);/* * Tell the memory management that we're a "memory allocator",
@@ -7331,7 +7327,7 @@ void __meminit kswapd_run(int nid)pgdat_kswapd_lock (pgdat);if (!pgdat->kswapd) { - pgdat->kswapd =kthread_run (kswapd, pgdat,"kswapd%d" , nid); + pgdat->kswapd =kthread_create_on_node (kswapd, pgdat, nid,"kswapd%d" , nid);if (IS_ERR (pgdat->kswapd)) {/* failure at boot is fatal */ pr_err("Failed to start kswapd on node %d,ret=%ld\n",
@@ -7339,6 +7335,7 @@ void __meminit kswapd_run(int nid)BUG_ON (system_state <SYSTEM_RUNNING ); pgdat->kswapd =NULL ; } +wake_up_process (pgdat->kswapd); }pgdat_kswapd_unlock (pgdat); }
Signature · 1 line
2.45.2
Frederic Weisbecker
maintainerPATCH 15/19 kthread: Implement preferred affinity
In reply to Frederic WeisbeckerAffining kthreads follow either of four existing different patterns:
1) Per-CPU kthreads must stay affine to a single CPU and never execute relevant code on any other CPU. This is currently handled by smpboot code which takes care of CPU-hotplug operations.
2) Kthreads that _have_ to be affine to a specific set of CPUs and can't run anywhere else. The affinity is set through kthread_bind_mask() and the subsystem takes care by itself to handle CPU-hotplug operations.
3) Kthreads that prefer to be affine to a specific NUMA node. That preferred affinity is applied by default when an actual node ID is passed on kthread creation, provided the kthread is not per-CPU and no call to kthread_bind_mask() has been issued before the first wake-up.
4) Similar to the previous point but kthreads have a preferred affinity different than a node. It is set manually like any other task and CPU-hotplug is supposed to be handled by the relevant subsystem so that the task is properly reaffined whenever a given CPU from the preferred affinity comes up or down. Also care must be taken so that the preferred affinity doesn't cross housekeeping cpumask boundaries.
Provide a function to handle the last usecase, mostly reusing the current node default affinity infrastructure. kthread_affine_preferred() is introduced, to be used just like kthread_bind_mask(), right after kthread creation and before the first wake up. The kthread is then affine right away to the cpumask passed through the API if it has online housekeeping CPUs. Otherwise it will be affine to all online housekeeping CPUs as a last resort.
As with node affinity, it is aware of CPU hotplug events such that:
* When a housekeeping CPU goes up and is part of the preferred affinity of a given kthread, it is added to its applied affinity set (and possibly the default last resort online housekeeping set is removed from the set).
* When a housekeeping CPU goes down while it was part of the preferred affinity of a kthread, it is removed from the kthread's applied affinity. The last resort is to affine the kthread to all online housekeeping CPUs.
--- include/linux/kthread.h | 1 + kernel/kthread.c | 69 ++++++++++++++++++++++++++++++++++++----- 2 files changed, 62 insertions(+), 8 deletions(-)
include/linux/kthread.h +1 -0
@@ -85,6 +85,7 @@ kthread_run_on_cpu(int (*threadfn)(void *data), void *data,void free_kthread_struct (struct task_struct *k);void kthread_bind (struct task_struct *k,unsigned int cpu);void kthread_bind_mask (struct task_struct *k,const struct cpumask *mask); +int kthread_affine_preferred (struct task_struct *p,const struct cpumask *mask);int kthread_stop (struct task_struct *k);int kthread_stop_put (struct task_struct *k);bool kthread_should_stop (void );
kernel/kthread.c +61 -8
105 lines not on this page · raw message
Signature · 1 line
2.45.2
Frederic Weisbecker
maintainerPATCH 16/19 rcu: Use kthread preferred affinity for RCU boost
In reply to Frederic WeisbeckerNow that kthreads have an infrastructure to handle preferred affinity against CPU hotplug and housekeeping cpumask, convert RCU boost to use it instead of handling all the constraints by itself.
--- kernel/rcu/tree.c | 27 +++++++++++++++++++-------- kernel/rcu/tree_plugin.h | 11 ++--------- 2 files changed, 21 insertions(+), 17 deletions(-)
kernel/rcu/tree.c +19 -8
@@ -149,7 +149,6 @@ static int rcu_scheduler_fully_active __read_mostly;static void rcu_report_qs_rnp (unsigned long mask,struct rcu_node *rnp,unsigned long gps,unsigned long flags); -static struct task_struct *rcu_boost_task (struct rcu_node *rnp);static void invoke_rcu_core (void );static void rcu_report_exp_rdp (struct rcu_data *rdp);static void sync_sched_exp_online_cleanup (int cpu);
@@ -4932,6 +4931,22 @@ int rcutree_prepare_cpu(unsigned int cpu)return 0 ; } +static void rcu_thread_affine_rnp (struct task_struct *t,struct rcu_node *rnp) +{ +cpumask_var_t affinity; +int cpu; + +if (!zalloc_cpumask_var (&affinity,GFP_KERNEL )) +return ; + +for_each_leaf_node_possible_cpu (rnp, cpu) +cpumask_set_cpu (cpu, affinity); + +kthread_affine_preferred (t, affinity); + +free_cpumask_var (affinity); +} +/* * Update kthreads affinity during CPU-hotplug changes. *
@@ -4951,19 +4966,18 @@ static void rcutree_affinity_setting(unsigned int cpu, int outgoingcpu)unsigned long mask;struct rcu_data *rdp;struct rcu_node *rnp; -struct task_struct *task_boost, *task_exp; +struct task_struct *task_exp; rdp =per_cpu_ptr (&rcu_data, cpu); rnp = rdp->mynode; - task_boost =rcu_boost_task (rnp); task_exp =rcu_exp_par_gp_task (rnp);/* -* If CPU is the boot one, those tasks are created later from early +* If CPU is the boot one, this task is created later from early * initcall since kthreadd must be created first. */ -if (!task_boost && !task_exp) +if (!task_exp)return ;if (!zalloc_cpumask_var (&cm,GFP_KERNEL ))
@@ -4985,9 +4999,6 @@ static void rcutree_affinity_setting(unsigned int cpu, int outgoingcpu)if (task_exp)set_cpus_allowed_ptr (task_exp, cm); -if (task_boost) -set_cpus_allowed_ptr (task_boost, cm); -mutex_unlock (&rnp->kthread_mutex);free_cpumask_var (cm);
kernel/rcu/tree_plugin.h +2 -9
@@ -1216,16 +1216,13 @@ static void rcu_spawn_one_boost_kthread(struct rcu_node *rnp)raw_spin_lock_irqsave_rcu_node (rnp, flags); rnp->boost_kthread_task = t;raw_spin_unlock_irqrestore_rcu_node (rnp, flags); + sp.sched_priority = kthread_prio;sched_setscheduler_nocheck (t,SCHED_FIFO , &sp); +rcu_thread_affine_rnp (t, rnp); wake_up_process(t); /* get to TASK_INTERRUPTIBLE quickly. */ } -static struct task_struct *rcu_boost_task (struct rcu_node *rnp) -{ -return READ_ONCE (rnp->boost_kthread_task); -} -#else /* #ifdef CONFIG_RCU_BOOST */ static void rcu_initiate_boost (struct rcu_node *rnp,unsigned long flags)
@@ -1242,10 +1239,6 @@ static void rcu_spawn_one_boost_kthread(struct rcu_node *rnp) { } -static struct task_struct *rcu_boost_task (struct rcu_node *rnp) -{ -return NULL ; -}#endif /* #else #ifdef CONFIG_RCU_BOOST */ /*
Signature · 1 line
2.45.2
Frederic Weisbecker
maintainerPATCH 17/19 kthread: Unify kthread_create_on_cpu() and kthread_create_worker_on_cpu() automatic format
In reply to Frederic Weisbeckerkthread_create_on_cpu() uses the CPU argument as an implicit and unique printf argument to add to the format whereas kthread_create_worker_on_cpu() still relies on explicitly passing the printf arguments. This difference in behaviour is error prone and doesn't help standardizing per-CPU kthread names.
Unify the behaviours and convert kthread_create_worker_on_cpu() to use the printf behaviour of kthread_create_on_cpu().
--- fs/erofs/zdata.c | 2 +- include/linux/kthread.h | 21 +++++++++++---- kernel/kthread.c | 59 ++++++++++++++++++++++++----------------- 3 files changed, 52 insertions(+), 30 deletions(-)
fs/erofs/zdata.c +1 -1
@@ -353,7 +353,7 @@ static void erofs_destroy_percpu_workers(void)static struct kthread_worker *erofs_init_percpu_worker (int cpu) {struct kthread_worker *worker = -kthread_create_worker_on_cpu (cpu,0 ,"erofs_worker/%u" , cpu); +kthread_create_worker_on_cpu (cpu,0 ,"erofs_worker/%u" );if (IS_ERR (worker))return worker;
include/linux/kthread.h +16 -5
@@ -187,13 +187,24 @@ extern void __kthread_init_worker(struct kthread_worker *worker,int kthread_worker_fn (void *worker_ptr); -__printf(2, 3) +__printf(3, 4) +struct kthread_worker *kthread_create_worker_on_node (unsigned int flags, +int node, +const char namefmt[], ...); + +#define kthread_create_worker (flags, namefmt, ...) \ +({ \ + struct kthread_worker *__kw \ + = kthread_create_worker_on_node(flags, NUMA_NO_NODE, \ + namefmt, ## __VA_ARGS__); \ + if (!IS_ERR(__kw)) \ + wake_up_process(__kw->task); \ + __kw; \ +}) +struct kthread_worker * -kthread_create_worker (unsigned int flags,const char namefmt[], ...); - -__printf(3, 4) struct kthread_worker *kthread_create_worker_on_cpu (int cpu,unsigned int flags, -const char namefmt[], ...); +const char namefmt[]);bool kthread_queue_work (struct kthread_worker *worker,struct kthread_work *work);
kernel/kthread.c +35 -24
114 lines not on this page · raw message
Signature · 1 line
2.45.2
Frederic Weisbecker
maintainerPATCH 18/19 treewide: Introduce kthread_run_worker[_on_cpu]()
In reply to Frederic Weisbeckerkthread_create() creates a kthread without running it yet. kthread_run() creates a kthread and runs it.
On the other hand, kthread_create_worker() creates a kthread worker and runs it.
This difference in behaviours is confusing. Also there is no way to create a kthread worker and affine it using kthread_bind_mask() or kthread_affine_preferred() before starting it.
Consolidate the behaviours and introduce kthread_run_worker[_on_cpu]() that behaves just like kthread_run(). kthread_create_worker[_on_cpu]() will now only create a kthread worker without starting it.
--- arch/x86/kvm/i8254.c | 2 +- crypto/crypto_engine.c | 2 +- drivers/cpufreq/cppc_cpufreq.c | 2 +- drivers/gpu/drm/drm_vblank_work.c | 2 +- .../drm/i915/gem/selftests/i915_gem_context.c | 2 +- drivers/gpu/drm/i915/gt/selftest_execlists.c | 2 +- drivers/gpu/drm/i915/gt/selftest_hangcheck.c | 2 +- drivers/gpu/drm/i915/gt/selftest_slpc.c | 2 +- drivers/gpu/drm/i915/selftests/i915_request.c | 8 ++-- drivers/gpu/drm/msm/disp/msm_disp_snapshot.c | 2 +- drivers/gpu/drm/msm/msm_atomic.c | 2 +- drivers/gpu/drm/msm/msm_gpu.c | 2 +- drivers/gpu/drm/msm/msm_kms.c | 2 +- .../platform/chips-media/wave5/wave5-vpu.c | 2 +- drivers/net/dsa/mv88e6xxx/chip.c | 2 +- drivers/net/ethernet/intel/ice/ice_dpll.c | 2 +- drivers/net/ethernet/intel/ice/ice_gnss.c | 2 +- drivers/net/ethernet/intel/ice/ice_ptp.c | 2 +- drivers/platform/chrome/cros_ec_spi.c | 2 +- drivers/ptp/ptp_clock.c | 2 +- drivers/spi/spi.c | 2 +- drivers/usb/typec/tcpm/tcpm.c | 2 +- drivers/vdpa/vdpa_sim/vdpa_sim.c | 2 +- drivers/watchdog/watchdog_dev.c | 2 +- fs/erofs/zdata.c | 2 +- include/linux/kthread.h | 48 ++++++++++++++++--- kernel/kthread.c | 31 +++--------- kernel/rcu/tree.c | 4 +- kernel/workqueue.c | 2 +- net/dsa/tag_ksz.c | 2 +- net/dsa/tag_ocelot_8021q.c | 2 +- net/dsa/tag_sja1105.c | 2 +- 32 files changed, 82 insertions(+), 65 deletions(-)
arch/x86/kvm/i8254.c +1 -1
@@ -681,7 +681,7 @@ struct kvm_pit *kvm_create_pit(struct kvm *kvm, u32 flags) pid_nr =pid_vnr (pid);put_pid (pid); - pit->worker =kthread_create_worker (0 ,"kvm-pit/%d" , pid_nr); + pit->worker =kthread_run_worker (0 ,"kvm-pit/%d" , pid_nr);if (IS_ERR (pit->worker)) goto fail_kthread;
crypto/crypto_engine.c +1 -1
@@ -517,7 +517,7 @@ struct crypto_engine *crypto_engine_alloc_init_and_set(struct device *dev,crypto_init_queue (&engine->queue, qlen);spin_lock_init (&engine->queue_lock); - engine->kworker =kthread_create_worker (0 ,"%s" , engine->name); + engine->kworker =kthread_run_worker (0 ,"%s" , engine->name);if (IS_ERR (engine->kworker)) {dev_err (dev,"failed to create crypto request pump task\n" );return NULL ;
drivers/cpufreq/cppc_cpufreq.c +1 -1
@@ -241,7 +241,7 @@ static void __init cppc_freq_invariance_init(void)if (fie_disabled)return ; - kworker_fie =kthread_create_worker (0 ,"cppc_fie" ); + kworker_fie =kthread_run_worker (0 ,"cppc_fie" );if (IS_ERR (kworker_fie)) {pr_warn ("%s: failed to create kworker_fie: %ld\n" , __func__,PTR_ERR (kworker_fie));
drivers/gpu/drm/drm_vblank_work.c +1 -1
@@ -277,7 +277,7 @@ int drm_vblank_worker_init(struct drm_vblank_crtc *vblank)INIT_LIST_HEAD (&vblank->pending_work);init_waitqueue_head (&vblank->work_wait_queue); - worker =kthread_create_worker (0 ,"card%d-crtc%d" , + worker =kthread_run_worker (0 ,"card%d-crtc%d" , vblank->dev->primary->index, vblank->pipe);if (IS_ERR (worker))
drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c +1 -1
@@ -369,7 +369,7 @@ static int live_parallel_switch(void *arg)if (!data[n].ce[0 ])continue ; - worker =kthread_create_worker (0 ,"igt/parallel:%s" , + worker =kthread_run_worker (0 ,"igt/parallel:%s" , data[n].ce[0 ]->engine->name);if (IS_ERR (worker)) { err =PTR_ERR (worker);
drivers/gpu/drm/i915/gt/selftest_execlists.c +1 -1
@@ -3574,7 +3574,7 @@ static int smoke_crescendo(struct preempt_smoke *smoke, unsigned int flags) arg[id].batch =NULL ; arg[id].count =0 ; - worker[id] =kthread_create_worker (0 ,"igt/smoke:%d" , id); + worker[id] =kthread_run_worker (0 ,"igt/smoke:%d" , id);if (IS_ERR (worker[id])) { err =PTR_ERR (worker[id]);break ;
drivers/gpu/drm/i915/gt/selftest_hangcheck.c +1 -1
@@ -1025,7 +1025,7 @@ static int __igt_reset_engines(struct intel_gt *gt, threads[tmp].engine = other; threads[tmp].flags = flags; - worker =kthread_create_worker (0 ,"igt/%s" , + worker =kthread_run_worker (0 ,"igt/%s" , other->name);if (IS_ERR (worker)) { err =PTR_ERR (worker);
drivers/gpu/drm/i915/gt/selftest_slpc.c +1 -1
@@ -489,7 +489,7 @@ static int live_slpc_tile_interaction(void *arg)return -ENOMEM ;for_each_gt (gt, i915, i) { - threads[i].worker =kthread_create_worker (0 ,"igt/slpc_parallel:%d" , gt->info.id); + threads[i].worker =kthread_run_worker (0 ,"igt/slpc_parallel:%d" , gt->info.id);if (IS_ERR (threads[i].worker)) { ret =PTR_ERR (threads[i].worker);
drivers/gpu/drm/i915/selftests/i915_request.c +4 -4
@@ -492,7 +492,7 @@ static int mock_breadcrumbs_smoketest(void *arg)for (n =0 ; n < ncpus; n++) {struct kthread_worker *worker; - worker =kthread_create_worker (0 ,"igt/%d" , n); + worker =kthread_run_worker (0 ,"igt/%d" , n);if (IS_ERR (worker)) { ret =PTR_ERR (worker); ncpus = n;
@@ -1645,7 +1645,7 @@ static int live_parallel_engines(void *arg)for_each_uabi_engine (engine, i915) {struct kthread_worker *worker; - worker =kthread_create_worker (0 ,"igt/parallel:%s" , + worker =kthread_run_worker (0 ,"igt/parallel:%s" , engine->name);if (IS_ERR (worker)) { err =PTR_ERR (worker);
@@ -1806,7 +1806,7 @@ static int live_breadcrumbs_smoketest(void *arg)unsigned int i = idx * ncpus + n;struct kthread_worker *worker; - worker =kthread_create_worker (0 ,"igt/%d.%d" , idx, n); + worker =kthread_run_worker (0 ,"igt/%d.%d" , idx, n);if (IS_ERR (worker)) { ret =PTR_ERR (worker); goto out_flush;
@@ -3219,7 +3219,7 @@ static int perf_parallel_engines(void *arg)memset (&engines[idx].p,0 ,sizeof (engines[idx].p)); - worker =kthread_create_worker (0 ,"igt:%s" , + worker =kthread_run_worker (0 ,"igt:%s" , engine->name);if (IS_ERR (worker)) { err =PTR_ERR (worker);
drivers/gpu/drm/msm/disp/msm_disp_snapshot.c +1 -1
@@ -109,7 +109,7 @@ int msm_disp_snapshot_init(struct drm_device *drm_dev)mutex_init (&kms->dump_mutex); - kms->dump_worker =kthread_create_worker (0 ,"%s" ,"disp_snapshot" ); + kms->dump_worker =kthread_run_worker (0 ,"%s" ,"disp_snapshot" );if (IS_ERR (kms->dump_worker))DRM_ERROR ("failed to create disp state task\n" );
drivers/gpu/drm/msm/msm_atomic.c +1 -1
@@ -115,7 +115,7 @@ int msm_atomic_init_pending_timer(struct msm_pending_timer *timer, timer->kms = kms; timer->crtc_idx = crtc_idx; - timer->worker =kthread_create_worker (0 ,"atomic-worker-%d" , crtc_idx); + timer->worker =kthread_run_worker (0 ,"atomic-worker-%d" , crtc_idx);if (IS_ERR (timer->worker)) {int ret =PTR_ERR (timer->worker); timer->worker =NULL ;
drivers/gpu/drm/msm/msm_gpu.c +1 -1
@@ -859,7 +859,7 @@ int msm_gpu_init(struct drm_device *drm, struct platform_device *pdev, gpu->funcs = funcs; gpu->name = name; - gpu->worker =kthread_create_worker (0 ,"gpu-worker" ); + gpu->worker =kthread_run_worker (0 ,"gpu-worker" );if (IS_ERR (gpu->worker)) { ret =PTR_ERR (gpu->worker); gpu->worker =NULL ;
drivers/gpu/drm/msm/msm_kms.c +1 -1
@@ -269,7 +269,7 @@ int msm_drm_kms_init(struct device *dev, const struct drm_driver *drv)/* initialize event thread */ ev_thread = &priv->event_thread[drm_crtc_index (crtc)]; ev_thread->dev = ddev; - ev_thread->worker =kthread_create_worker (0 ,"crtc_event:%d" , crtc->base.id); + ev_thread->worker =kthread_run_worker (0 ,"crtc_event:%d" , crtc->base.id);if (IS_ERR (ev_thread->worker)) { ret =PTR_ERR (ev_thread->worker);DRM_DEV_ERROR (dev,"failed to create crtc_event kthread\n" );
drivers/media/platform/chips-media/wave5/wave5-vpu.c +1 -1
@@ -231,7 +231,7 @@ static int wave5_vpu_probe(struct platform_device *pdev)dev_err (&pdev->dev,"failed to get irq resource, falling back to polling\n" );hrtimer_init (&dev->hrtimer,CLOCK_MONOTONIC ,HRTIMER_MODE_REL_PINNED ); dev->hrtimer.function = &wave5_vpu_timer_callback; - dev->worker =kthread_create_worker (0 ,"vpu_irq_thread" ); + dev->worker =kthread_run_worker (0 ,"vpu_irq_thread" );if (IS_ERR (dev->worker)) {dev_err (&pdev->dev,"failed to create vpu irq worker\n" ); ret =PTR_ERR (dev->worker);
drivers/net/dsa/mv88e6xxx/chip.c +1 -1
@@ -393,7 +393,7 @@ static int mv88e6xxx_irq_poll_setup(struct mv88e6xxx_chip *chip)kthread_init_delayed_work (&chip->irq_poll_work, mv88e6xxx_irq_poll); - chip->kworker =kthread_create_worker (0 ,"%s" ,dev_name (chip->dev)); + chip->kworker =kthread_run_worker (0 ,"%s" ,dev_name (chip->dev));if (IS_ERR (chip->kworker))return PTR_ERR (chip->kworker);
drivers/net/ethernet/intel/ice/ice_dpll.c +1 -1
@@ -1833,7 +1833,7 @@ static int ice_dpll_init_worker(struct ice_pf *pf)struct kthread_worker *kworker;kthread_init_delayed_work (&d->work, ice_dpll_periodic_work); - kworker =kthread_create_worker (0 ,"ice-dplls-%s" , + kworker =kthread_run_worker (0 ,"ice-dplls-%s" ,dev_name (ice_pf_to_dev (pf)));if (IS_ERR (kworker))return PTR_ERR (kworker);
drivers/net/ethernet/intel/ice/ice_gnss.c +1 -1
@@ -182,7 +182,7 @@ static struct gnss_serial *ice_gnss_struct_init(struct ice_pf *pf) pf->gnss_serial = gnss;kthread_init_delayed_work (&gnss->read_work, ice_gnss_read); - kworker =kthread_create_worker (0 ,"ice-gnss-%s" ,dev_name (dev)); + kworker =kthread_run_worker (0 ,"ice-gnss-%s" ,dev_name (dev));if (IS_ERR (kworker)) {kfree (gnss);return NULL ;
drivers/net/ethernet/intel/ice/ice_ptp.c +1 -1
@@ -3181,7 +3181,7 @@ static int ice_ptp_init_work(struct ice_pf *pf, struct ice_ptp *ptp)/* Allocate a kworker for handling work required for the ports * connected to the PTP hardware clock. */ - kworker =kthread_create_worker (0 ,"ice-ptp-%s" , + kworker =kthread_run_worker (0 ,"ice-ptp-%s" ,dev_name (ice_pf_to_dev (pf)));if (IS_ERR (kworker))return PTR_ERR (kworker);
drivers/platform/chrome/cros_ec_spi.c +1 -1
@@ -715,7 +715,7 @@ static int cros_ec_spi_devm_high_pri_alloc(struct device *dev,int err; ec_spi->high_pri_worker = -kthread_create_worker (0 ,"cros_ec_spi_high_pri" ); +kthread_run_worker (0 ,"cros_ec_spi_high_pri" );if (IS_ERR (ec_spi->high_pri_worker)) { err =PTR_ERR (ec_spi->high_pri_worker);
drivers/ptp/ptp_clock.c +1 -1
@@ -295,7 +295,7 @@ struct ptp_clock *ptp_clock_register(struct ptp_clock_info *info,if (ptp->info->do_aux_work) {kthread_init_delayed_work (&ptp->aux_work, ptp_aux_kworker); - ptp->kworker =kthread_create_worker (0 ,"ptp%d" , ptp->index); + ptp->kworker =kthread_run_worker (0 ,"ptp%d" , ptp->index);if (IS_ERR (ptp->kworker)) { err =PTR_ERR (ptp->kworker);pr_err ("failed to create ptp aux_worker %d\n" , err);
drivers/spi/spi.c +1 -1
@@ -2053,7 +2053,7 @@ static int spi_init_queue(struct spi_controller *ctlr) ctlr->busy = false; ctlr->queue_empty = true; - ctlr->kworker =kthread_create_worker (0 ,dev_name (&ctlr->dev)); + ctlr->kworker =kthread_run_worker (0 ,dev_name (&ctlr->dev));if (IS_ERR (ctlr->kworker)) {dev_err (&ctlr->dev,"failed to create message pump kworker\n" );return PTR_ERR (ctlr->kworker);
drivers/usb/typec/tcpm/tcpm.c +1 -1
@@ -7583,7 +7583,7 @@ struct tcpm_port *tcpm_register_port(struct device *dev, struct tcpc_dev *tcpc)mutex_init (&port->lock);mutex_init (&port->swap_lock); - port->wq =kthread_create_worker (0 ,dev_name (dev)); + port->wq =kthread_run_worker (0 ,dev_name (dev));if (IS_ERR (port->wq))return ERR_CAST (port->wq);sched_set_fifo (port->wq->task);
drivers/vdpa/vdpa_sim/vdpa_sim.c +1 -1
@@ -229,7 +229,7 @@ struct vdpasim *vdpasim_create(struct vdpasim_dev_attr *dev_attr, dev = &vdpasim->vdpa.dev;kthread_init_work (&vdpasim->work, vdpasim_work_fn); - vdpasim->worker =kthread_create_worker (0 ,"vDPA sim worker: %s" , + vdpasim->worker =kthread_run_worker (0 ,"vDPA sim worker: %s" , dev_attr->name);if (IS_ERR (vdpasim->worker)) goto err_iommu;
drivers/watchdog/watchdog_dev.c +1 -1
@@ -1229,7 +1229,7 @@ int __init watchdog_dev_init(void) {int err; - watchdog_kworker =kthread_create_worker (0 ,"watchdogd" ); + watchdog_kworker =kthread_run_worker (0 ,"watchdogd" );if (IS_ERR (watchdog_kworker)) {pr_err ("Failed to create watchdog kworker\n" );return PTR_ERR (watchdog_kworker);
fs/erofs/zdata.c +1 -1
@@ -353,7 +353,7 @@ static void erofs_destroy_percpu_workers(void)static struct kthread_worker *erofs_init_percpu_worker (int cpu) {struct kthread_worker *worker = -kthread_create_worker_on_cpu (cpu,0 ,"erofs_worker/%u" ); +kthread_run_worker_on_cpu (cpu,0 ,"erofs_worker/%u" );if (IS_ERR (worker))return worker;
include/linux/kthread.h +41 -7
60 lines not on this page · raw message
kernel/kthread.c +7 -24
@@ -1080,33 +1080,10 @@ kthread_create_worker_on_node(unsigned int flags, int node, const char namefmt[] worker =__kthread_create_worker_on_node (flags, node, namefmt, args);va_end (args); -if (worker) -wake_up_process (worker->task); -return worker; }EXPORT_SYMBOL (kthread_create_worker_on_node); -static __printf(3, 4) struct kthread_worker * -__kthread_create_worker_on_cpu(int cpu, unsigned int flags, - const char namefmt[], ...) -{ - struct kthread_worker *worker; -va_list args; - -va_start (args, namefmt); - worker =__kthread_create_worker_on_node (flags,cpu_to_node (cpu), - namefmt, args); -va_end (args); - -if (worker) { -kthread_bind (worker->task, cpu); -wake_up_process (worker->task); - } - -return worker; -} -/** * kthread_create_worker_on_cpu - create a kthread worker and bind it * to a given CPU and the associated NUMA node.
@@ -1147,7 +1124,13 @@ struct kthread_worker * kthread_create_worker_on_cpu(int cpu, unsigned int flags, const char namefmt[]) { - return __kthread_create_worker_on_cpu(cpu, flags, namefmt, cpu); + struct kthread_worker *worker; + + worker =kthread_create_worker_on_node (flags,cpu_to_node (cpu), namefmt, cpu); +if (worker) +kthread_bind (worker->task, cpu); + +return worker; }EXPORT_SYMBOL (kthread_create_worker_on_cpu);
kernel/rcu/tree.c +2 -2
@@ -4827,7 +4827,7 @@ static void rcu_spawn_exp_par_gp_kworker(struct rcu_node *rnp)if (rnp->exp_kworker)return ; - kworker =kthread_create_worker (0 , name, rnp_index); + kworker =kthread_run_worker (0 , name, rnp_index);if (IS_ERR_OR_NULL (kworker)) {pr_err ("Failed to create par gp kworker on %d/%d\n" , rnp->grplo, rnp->grphi);
@@ -4854,7 +4854,7 @@ static void __init rcu_start_exp_gp_kworker(void)const char *name ="rcu_exp_gp_kthread_worker" ;struct sched_param param = { .sched_priority = kthread_prio }; - rcu_exp_gp_kworker =kthread_create_worker (0 , name); + rcu_exp_gp_kworker =kthread_run_worker (0 , name);if (IS_ERR_OR_NULL (rcu_exp_gp_kworker)) {pr_err ("Failed to create %s!\n" , name); rcu_exp_gp_kworker =NULL ;
kernel/workqueue.c +1 -1
@@ -7749,7 +7749,7 @@ static void __init wq_cpu_intensive_thresh_init(void)unsigned long thresh;unsigned long bogo; - pwq_release_worker =kthread_create_worker (0 ,"pool_workqueue_release" ); + pwq_release_worker =kthread_run_worker (0 ,"pool_workqueue_release" );BUG_ON (IS_ERR (pwq_release_worker));/* if the user set it to a specific value, keep it */
net/dsa/tag_ksz.c +1 -1
@@ -66,7 +66,7 @@ static int ksz_connect(struct dsa_switch *ds)if (!priv)return -ENOMEM ; - xmit_worker =kthread_create_worker (0 ,"dsa%d:%d_xmit" , + xmit_worker =kthread_run_worker (0 ,"dsa%d:%d_xmit" , ds->dst->index, ds->index);if (IS_ERR (xmit_worker)) { ret =PTR_ERR (xmit_worker);
net/dsa/tag_ocelot_8021q.c +1 -1
@@ -110,7 +110,7 @@ static int ocelot_connect(struct dsa_switch *ds)if (!priv)return -ENOMEM ; - priv->xmit_worker =kthread_create_worker (0 ,"felix_xmit" ); + priv->xmit_worker =kthread_run_worker (0 ,"felix_xmit" );if (IS_ERR (priv->xmit_worker)) { err =PTR_ERR (priv->xmit_worker);kfree (priv);
net/dsa/tag_sja1105.c +1 -1
@@ -707,7 +707,7 @@ static int sja1105_connect(struct dsa_switch *ds)spin_lock_init (&priv->meta_lock); - xmit_worker =kthread_create_worker (0 ,"dsa%d:%d_xmit" , + xmit_worker =kthread_run_worker (0 ,"dsa%d:%d_xmit" , ds->dst->index, ds->index);if (IS_ERR (xmit_worker)) { err =PTR_ERR (xmit_worker);
Signature · 1 line
2.45.2
Frederic Weisbecker
maintainerPATCH 19/19 rcu: Use kthread preferred affinity for RCU exp kworkers
In reply to Frederic WeisbeckerNow that kthreads have an infrastructure to handle preferred affinity against CPU hotplug and housekeeping cpumask, convert RCU exp workers to use it instead of handling all the constraints by itself.
--- kernel/rcu/tree.c | 105 +++++++++------------------------------------- 1 file changed, 19 insertions(+), 86 deletions(-)
kernel/rcu/tree.c +19 -86
@@ -4815,6 +4815,22 @@ rcu_boot_init_percpu_data(int cpu)rcu_boot_init_nocb_percpu_data (rdp); } +static void rcu_thread_affine_rnp (struct task_struct *t,struct rcu_node *rnp) +{ +cpumask_var_t affinity; +int cpu; + +if (!zalloc_cpumask_var (&affinity,GFP_KERNEL )) +return ; + +for_each_leaf_node_possible_cpu (rnp, cpu) +cpumask_set_cpu (cpu, affinity); + +kthread_affine_preferred (t, affinity); + +free_cpumask_var (affinity); +} +struct kthread_worker *rcu_exp_gp_kworker;static void rcu_spawn_exp_par_gp_kworker (struct rcu_node *rnp)
@@ -4827,7 +4843,7 @@ static void rcu_spawn_exp_par_gp_kworker(struct rcu_node *rnp)if (rnp->exp_kworker)return ; - kworker =kthread_run_worker (0 , name, rnp_index); + kworker =kthread_create_worker (0 , name, rnp_index);if (IS_ERR_OR_NULL (kworker)) {pr_err ("Failed to create par gp kworker on %d/%d\n" , rnp->grplo, rnp->grphi);
@@ -4837,16 +4853,9 @@ static void rcu_spawn_exp_par_gp_kworker(struct rcu_node *rnp)if (IS_ENABLED (CONFIG_RCU_EXP_KTHREAD ))sched_setscheduler_nocheck (kworker->task,SCHED_FIFO , ¶m); -} -static struct task_struct *rcu_exp_par_gp_task (struct rcu_node *rnp) -{ -struct kthread_worker *kworker =READ_ONCE (rnp->exp_kworker); - -if (!kworker) -return NULL ; - -return kworker->task; +rcu_thread_affine_rnp (kworker->task, rnp); +wake_up_process (kworker->task); }static void __init rcu_start_exp_gp_kworker (void )
@@ -4931,79 +4940,6 @@ int rcutree_prepare_cpu(unsigned int cpu)return 0 ; } -static void rcu_thread_affine_rnp (struct task_struct *t,struct rcu_node *rnp) -{ -cpumask_var_t affinity; -int cpu; - -if (!zalloc_cpumask_var (&affinity,GFP_KERNEL )) -return ; - -for_each_leaf_node_possible_cpu (rnp, cpu) -cpumask_set_cpu (cpu, affinity); - -kthread_affine_preferred (t, affinity); - -free_cpumask_var (affinity); -} - -/* -* Update kthreads affinity during CPU-hotplug changes. -* -* Set the per-rcu_node kthread's affinity to cover all CPUs that are -* served by the rcu_node in question. The CPU hotplug lock is still -* held, so the value of rnp->qsmaskinit will be stable. -* -* We don't include outgoingcpu in the affinity set, use -1 if there is -* no outgoing CPU. If there are no CPUs left in the affinity set, -* this function allows the kthread to execute on any CPU. -* -* Any future concurrent calls are serialized via ->kthread_mutex. -*/ -static void rcutree_affinity_setting (unsigned int cpu,int outgoingcpu) -{ -cpumask_var_t cm; -unsigned long mask; -struct rcu_data *rdp; -struct rcu_node *rnp; -struct task_struct *task_exp; - - rdp =per_cpu_ptr (&rcu_data, cpu); - rnp = rdp->mynode; - - task_exp =rcu_exp_par_gp_task (rnp); - -/* -* If CPU is the boot one, this task is created later from early -* initcall since kthreadd must be created first. -*/ -if (!task_exp) -return ; - -if (!zalloc_cpumask_var (&cm,GFP_KERNEL )) -return ; - -mutex_lock (&rnp->kthread_mutex); - mask =rcu_rnp_online_cpus (rnp); -for_each_leaf_node_possible_cpu (rnp, cpu) -if ((mask &leaf_node_cpu_bit (rnp, cpu)) && - cpu != outgoingcpu) -cpumask_set_cpu (cpu, cm); -cpumask_and (cm, cm,housekeeping_cpumask (HK_TYPE_RCU )); -if (cpumask_empty (cm)) { -cpumask_copy (cm,housekeeping_cpumask (HK_TYPE_RCU )); -if (outgoingcpu >=0 ) -cpumask_clear_cpu (outgoingcpu, cm); - } - -if (task_exp) -set_cpus_allowed_ptr (task_exp, cm); - -mutex_unlock (&rnp->kthread_mutex); - -free_cpumask_var (cm); -} -/* * Has the specified (known valid) CPU ever been fully online? */
@@ -5032,7 +4968,6 @@ int rcutree_online_cpu(unsigned int cpu)if (rcu_scheduler_active ==RCU_SCHEDULER_INACTIVE )return 0 ;/* Too early in boot for scheduler work. */ sync_sched_exp_online_cleanup (cpu); -rcutree_affinity_setting (cpu,-1 );// Stop-machine done, so allow nohz_full to disable tick. tick_dep_clear (TICK_DEP_BIT_RCU );
@@ -5249,8 +5184,6 @@ int rcutree_offline_cpu(unsigned int cpu) rnp->ffmask &= ~rdp->grpmask;raw_spin_unlock_irqrestore_rcu_node (rnp, flags); -rcutree_affinity_setting (cpu, cpu); -// nohz_full CPUs need the tick for stop-machine to work quickly tick_dep_set (TICK_DEP_BIT_RCU );return 0 ;
Signature · 1 line
2.45.2
Re: [PATCH 12/19] kthread: Default affine kthread to its preferred NUMA node
In reply to Frederic WeisbeckerQuoted 28 lines · Frederic Weisbecker
Kthreads attached to a preferred NUMA node for their task structure allocation can also be assumed to run preferrably within that same node.
A more precise affinity is usually notified by calling
kthread_create_on_cpu()or kthread_bind[_mask]() before the first wakeup.For the others, a default affinity to the node is desired and sometimes implemented with more or less success when it comes to deal with hotplug events and
nohz_full/ CPU Isolation interactions:- kcompactd is affine to its node and handles hotplug but not CPU Isolation
- kswapd is affine to its node and ignores hotplug and CPU Isolation
- A bunch of drivers create their kthreads on a specific node and don't take care about affining further.Handle that default node affinity preference at the generic level instead, provided a kthread is created on an actual node and doesn't apply any specific affinity such as a given CPU or a custom cpumask to bind to before its first wake-up.
This generic handling is aware of CPU hotplug events and CPU isolation such that:
* When a housekeeping CPU goes up and is part of the node of a given kthread, it is added to its applied affinity set (and possibly the default last resort online housekeeping set is removed from the set).
Quoted 6 lines
* When a housekeeping CPU goes down while it was part of the node of a kthread, it is removed from the kthread's applied affinity. The last resort is to affine the kthread to all online housekeeping CPUs.
Signed-off-by Frederic Weisbecker <[email protected]>
Re: [PATCH 13/19] mm: Create/affine kcompactd to its preferred node
In reply to Frederic WeisbeckerQuoted 7 lines · Frederic Weisbecker
Kcompactd is dedicated to a specific node. As such it wants to be preferrably affine to it, memory and CPUs-wise.
Use the proper kthread API to achieve that. As a bonus it takes care of CPU-hotplug events and CPU-isolation on its behalf.
Signed-off-by Frederic Weisbecker <[email protected]>
Re: [PATCH 14/19] mm: Create/affine kswapd to its preferred node
In reply to Frederic WeisbeckerQuoted 7 lines · Frederic Weisbecker
kswapd is dedicated to a specific node. As such it wants to be preferrably affine to it, memory and CPUs-wise.
Use the proper kthread API to achieve that. As a bonus it takes care of CPU-hotplug events and CPU-isolation on its behalf.
Signed-off-by Frederic Weisbecker <[email protected]>
But there's a bug to fix:
Quoted 29 lines
--- mm/vmscan.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)mm/vmscan.c +1
-5@@ -7159,10 +7159,6 @@ static int kswapd(void *p)unsigned int highest_zoneidx =MAX_NR_ZONES -1 ;pg_data_t *pgdat = (pg_data_t *)p;struct task_struct *tsk = current; -const struct cpumask *cpumask =cpumask_of_node (pgdat->node_id); - -if (!cpumask_empty (cpumask)) -set_cpus_allowed_ptr (tsk, cpumask);/* * Tell the memory management that we're a "memory allocator", @@ -7331,7 +7327,7 @@ void __meminit kswapd_run(int nid)pgdat_kswapd_lock (pgdat);if (!pgdat->kswapd) { - pgdat->kswapd =kthread_run (kswapd, pgdat,"kswapd%d" , nid); + pgdat->kswapd =kthread_create_on_node (kswapd, pgdat, nid,"kswapd%d" , nid);if (IS_ERR (pgdat->kswapd)) {/* failure at boot is fatal */ pr_err("Failed to start kswapd on node %d,ret=%ld\n",
mm/vmscan.c void __meminit kswapd_run(int nid) · line 7338
Quoted 4 lines
mm/vmscan.c +1
-0@@ -7339,6 +7335,7 @@ void __meminit kswapd_run(int nid)BUG_ON (system_state <SYSTEM_RUNNING ); pgdat->kswapd =NULL ; } +wake_up_process (pgdat->kswapd);
should be in an else { } branch as it only ends up with BUG() during early
boot and not when hotplugging later
Quoted 3 lines
mm/vmscan.c +0
-0}pgdat_kswapd_unlock (pgdat); }
Re: [PATCH 09/19] lib: test_objpool: Use kthread_run_on_cpu()
In reply to Frederic WeisbeckerQuoted 36 lines · Frederic Weisbecker
Use the proper API instead of open coding it.
Signed-off-by Frederic Weisbecker <[email protected]>--- lib/test_objpool.c | 19 +++++-------------- 1 file changed, 5 insertions(+), 14 deletions(-)lib/test_objpool.c +3
-10@@ -371,14 +371,10 @@ static int ot_start_sync(struct ot_test *test)if (!cpu_online (cpu))continue ; - work =kthread_create_on_node (ot_thread_worker, item, -cpu_to_node (cpu),"ot_worker_%d" , cpu); -if (IS_ERR (work)) { + work =kthread_run_on_cpu (ot_thread_worker, item, + cpu,"ot_worker_%d" ); +if (IS_ERR (work))pr_err ("failed to create thread for cpu %d\n" , cpu); - }else { -kthread_bind (work, cpu); -wake_up_process (work); - } }/* wait a while to make sure all threads waiting at start line */ @@ -562,14 +558,9 @@ static int ot_start_async(struct ot_test *test)if (!cpu_online (cpu))continue ; - work =kthread_create_on_node (ot_thread_worker, item, -cpu_to_node (cpu),"ot_worker_%d" , cpu); -if (IS_ERR (work)) {
lib/test_objpool.c static int ot_start_async(struct ot_test *test) · line 566
Quoted 10 lines
lib/test_objpool.c +2
-4+ work =kthread_run_on_cpu (ot_thread_worker, item, cpu,"ot_worker_%d" ); +if (IS_ERR (work))pr_err ("failed to create thread for cpu %d\n" , cpu); - }else { -kthread_bind (work, cpu); -wake_up_process (work); - } }/* wait a while to make sure all threads waiting at start line */
Looks good to me. Thanks for the simplification.
[tip: x86/cache] x86/resctrl: Use kthread_run_on_cpu()
In reply to Frederic WeisbeckerThe following commit has been merged into the x86/cache branch of tip:
The following commit has been merged into the x86/cache branch of tip:
Commit-ID: a763fc24ecf2ef0471ccba3b1ff6e6271f4bdc0b Gitweb: https://git.kernel.org/tip/a763fc24ecf2ef0471ccba3b1ff6e6271f4bdc0b Author: Frederic Weisbecker <[email protected]> AuthorDate: Wed, 07 Aug 2024 18:02:08 +02:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Mon, 02 Dec 2024 15:35:35 +01:00
x86/resctrl: Use kthread_run_on_cpu()
Use the proper API instead of open coding it.
--- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 28 ++++++---------------- 1 file changed, 8 insertions(+), 20 deletions(-)
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +8 -20
@@ -1205,20 +1205,14 @@ static int pseudo_lock_measure_cycles(struct rdtgroup *rdtgrp, int sel) plr->cpu = cpu;if (sel ==1 ) - thread =kthread_create_on_node (measure_cycles_lat_fn, plr, -cpu_to_node (cpu), -"pseudo_lock_measure/%u" , - cpu); + thread =kthread_run_on_cpu (measure_cycles_lat_fn, plr, + cpu,"pseudo_lock_measure/%u" );else if (sel ==2 ) - thread =kthread_create_on_node (measure_l2_residency, plr, -cpu_to_node (cpu), -"pseudo_lock_measure/%u" , - cpu); + thread =kthread_run_on_cpu (measure_l2_residency, plr, + cpu,"pseudo_lock_measure/%u" );else if (sel ==3 ) - thread =kthread_create_on_node (measure_l3_residency, plr, -cpu_to_node (cpu), -"pseudo_lock_measure/%u" , - cpu); + thread =kthread_run_on_cpu (measure_l3_residency, plr, + cpu,"pseudo_lock_measure/%u" );else goto out;
@@ -1226,8 +1220,6 @@ static int pseudo_lock_measure_cycles(struct rdtgroup *rdtgrp, int sel) ret =PTR_ERR (thread); goto out; } -kthread_bind (thread, cpu); -wake_up_process (thread); ret =wait_event_interruptible (plr->lock_thread_wq, plr->thread_done ==1 );
@@ -1315,18 +1307,14 @@ int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtgrp) plr->thread_done =0 ; - thread =kthread_create_on_node (pseudo_lock_fn, rdtgrp, -cpu_to_node (plr->cpu), -"pseudo_lock/%u" , plr->cpu); + thread =kthread_run_on_cpu (pseudo_lock_fn, rdtgrp, + plr->cpu,"pseudo_lock/%u" );if (IS_ERR (thread)) { ret =PTR_ERR (thread);rdt_last_cmd_printf ("Locking thread returned error %d\n" , ret); goto out_cstates; } -kthread_bind (thread, plr->cpu); -wake_up_process (thread); - ret =wait_event_interruptible (plr->lock_thread_wq, plr->thread_done ==1 );if (ret <0 ) {
[tip: x86/cache] x86/resctrl: Use kthread_run_on_cpu()
In reply to Frederic WeisbeckerThe following commit has been merged into the x86/cache branch of tip:
The following commit has been merged into the x86/cache branch of tip:
Commit-ID: 135eef38d7e081303fd9cdb982b37fcad32f9be0 Gitweb: https://git.kernel.org/tip/135eef38d7e081303fd9cdb982b37fcad32f9be0 Author: Frederic Weisbecker <[email protected]> AuthorDate: Wed, 07 Aug 2024 18:02:08 +02:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Mon, 09 Dec 2024 20:19:48 +01:00
x86/resctrl: Use kthread_run_on_cpu()
Use the proper API instead of open coding it.
--- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 28 ++++++---------------- 1 file changed, 8 insertions(+), 20 deletions(-)
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +8 -20
@@ -1205,20 +1205,14 @@ static int pseudo_lock_measure_cycles(struct rdtgroup *rdtgrp, int sel) plr->cpu = cpu;if (sel ==1 ) - thread =kthread_create_on_node (measure_cycles_lat_fn, plr, -cpu_to_node (cpu), -"pseudo_lock_measure/%u" , - cpu); + thread =kthread_run_on_cpu (measure_cycles_lat_fn, plr, + cpu,"pseudo_lock_measure/%u" );else if (sel ==2 ) - thread =kthread_create_on_node (measure_l2_residency, plr, -cpu_to_node (cpu), -"pseudo_lock_measure/%u" , - cpu); + thread =kthread_run_on_cpu (measure_l2_residency, plr, + cpu,"pseudo_lock_measure/%u" );else if (sel ==3 ) - thread =kthread_create_on_node (measure_l3_residency, plr, -cpu_to_node (cpu), -"pseudo_lock_measure/%u" , - cpu); + thread =kthread_run_on_cpu (measure_l3_residency, plr, + cpu,"pseudo_lock_measure/%u" );else goto out;
@@ -1226,8 +1220,6 @@ static int pseudo_lock_measure_cycles(struct rdtgroup *rdtgrp, int sel) ret =PTR_ERR (thread); goto out; } -kthread_bind (thread, cpu); -wake_up_process (thread); ret =wait_event_interruptible (plr->lock_thread_wq, plr->thread_done ==1 );
@@ -1315,18 +1307,14 @@ int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtgrp) plr->thread_done =0 ; - thread =kthread_create_on_node (pseudo_lock_fn, rdtgrp, -cpu_to_node (plr->cpu), -"pseudo_lock/%u" , plr->cpu); + thread =kthread_run_on_cpu (pseudo_lock_fn, rdtgrp, + plr->cpu,"pseudo_lock/%u" );if (IS_ERR (thread)) { ret =PTR_ERR (thread);rdt_last_cmd_printf ("Locking thread returned error %d\n" , ret); goto out_cstates; } -kthread_bind (thread, plr->cpu); -wake_up_process (thread); - ret =wait_event_interruptible (plr->lock_thread_wq, plr->thread_done ==1 );if (ret <0 ) {