Skip to contentSkip to conversation

sched, fair: Allow a per-cpu kthread waking a task to stack on the same CPU

PATCH Mel Gorman 1 participant 2 messages Jan 28, 2020 2 messages not in the archive VFS

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

Mel Gorman

author
#1 lore raw
Quoted 17 lines · Hillf Danton
Quoted 15 lines · On Mon, 27 Jan 2020 14:36:08 +0000 Mel Gorman wrote:

Commit 8ab39f11d974 ("xfs: prevent CIL push holdoff in log recovery") changed from using bound workqueues to using unbound workqueues. Functionally this makes sense but it was observed at the time that the dbench performance dropped quite a lot and CPU migrations were excessively high even when there are plenty of idle CPUs.

The pattern of the task migration is straight-forward. With XFS, an IO issuer may delegate work to a kworker which wakes on the same CPU. On completion of the work, it wakes the task, finds that the previous CPU is busy (because the kworker is still running on it) and migrates the task to the next idle CPU. The task ends up migrating around all CPUs sharing a LLC at high frequency. This has negative implications both in commication costs and power management. mpstat confirmed that at low thread counts that all CPUs sharing an LLC has low level of activity.

Quoted 5 lines
Quoted 2 lines

The impact of this problem is related to the number of CPUs sharing an LLC.

Are you trying to fix a problem of cache affinity?

No, I'm simply stating that the search space for select_idle_sibling matters.

Quoted 8 lines
Quoted 8 lines
kernel/sched/fair.c +0 -0
index fe4e0d775375..76df439aff76 100644
@@ -5912,6 +5912,19 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
 	    (available_idle_cpu(prev) || sched_idle_cpu(prev)))
 		return prev;
 
Quoted 11 lines

kernel/sched/fair.c static int select_idle_sibling(struct task_struct *p, int prev, int target) · line 5923

Quoted 9 lines
kernel/sched/fair.c +9 -0
+	/*
+	 * Allow a per-cpu kthread to stack with the wakee if the
+	 * kworker thread and the tasks previous CPU are the same.
+	 * The assumption is that the wakee queued work for the
+	 * per-cpu kthread that is now complete and the wakeup is
+	 * essentially a sync wakeup.
+	 */
+	if (is_per_cpu_kthread(current) &&
+	    prev == smp_processor_id() &&

Looks like cache affinity is not your target.

It's not, at least not LLC. L1 is a partial consideration.

Quoted 3 lines

Wondering why it does not work to select a cpu sharing cache with prev if strong relation exists between waker and wakee.

Functionally it works, it's just slow. There is a cost to migration and a cost to exiting from idle state and ramping up the CPU frequency.

Quoted 16 lines
Quoted 16 lines
kernel/sched/fair.c +4 -0
+	    this_rq()->nr_running <= 1) {
+		return prev;
+	}
+
 	/* Check a recently used CPU as a potential idle candidate: */
 	recent_used_cpu = p->recent_used_cpu;
 	if (recent_used_cpu != prev &&
kernel/sched/sched.h +1 -0
index 1a88dc8ad11b..5876e6ba5903 100644
@@ -2479,3 +2479,16 @@ static inline void membarrier_switch_mm(struct rq *rq,
 {
 }
 #endif
+
Quoted 9 lines

kernel/sched/sched.h static inline void membarrier_switch_mm(struct rq *rq, · line 2487

Quoted 5 lines
kernel/sched/sched.h +5 -0
+#ifdef CONFIG_SMP
+static inline bool is_per_cpu_kthread(struct task_struct *p)
+{
+	if (!(p->flags & PF_KTHREAD))
+		return false;

Suspect you need PF_KTHREAD instead of PF_WQ_WORKER. Here is a small helper and feel free to pick it up if it makes a sense.

Did you mean to switch that around? Either way, I moved an existing helper that is already used to detect this particular situation. While it works when it's made specific to a workqueue and open-coding it, there was no clear reason to narrow the conditions further.

Signature · 2 lines
Mel Gorman
SUSE Labs

A message that is not in the archive (sent off-list or not yet received): [email protected]

author
#3 lore raw
Quoted 7 lines · Hillf Danton
Quoted 3 lines

Functionally it works, it's just slow. There is a cost to migration and a cost to exiting from idle state and ramping up the CPU frequency.

Yeah we need to pay some costs but are not they compensated by the ping and pong that waker and wakee are happy to play for ten minutes on different cpus sharing cache if you have no way to migrate waker?

I could get into it depth but the changelog already mentions the cpufreq implications and the consequences of round-robining around the machine as a side-effect of how select_idle_sibling works. The data indicates that we are not compensated by the migrations.

Quoted 4 lines

Or back to the kworker case, a tradeoff needs to make between making kworker able to run on cache-sharing cpus and adding scheduling heuristics. IOW is cache affinity the key to the problem?

The kworker is already running on CPUs sharing cache. That is not the central issue.

Signature · 2 lines
Mel Gorman
SUSE Labs