Skip to contentSkip to conversation

sched/fair: ignore cache hotness for SMT migration

PATCH Superseded Josh Don 5 participants 7 messages Aug 4, 2020 – Aug 27, 2020 Scheduler

lore
Share & tools
lore

Josh Don

author
#0 lore raw

PATCH 1/1 sched/fair: ignore cache hotness for SMT migration

SMT siblings share caches, so cache hotness should be irrelevant for cross-sibling migration.

---
 kernel/sched/fair.c | 3 +++
 1 file changed, 3 insertions(+)
kernel/sched/fair.c +3 -0
index 2ba8f230feb9..5b203b55bcb2 100644
@@ -7402,6 +7402,9 @@ static int task_hot(struct task_struct *p, struct lb_env *env)
 	if (unlikely(task_has_idle_policy(p)))
 		return 0;
 
+	if ((env->sd->flags & cpu_smt_flags()) == cpu_smt_flags())
+		return 0;
+
 	/*
 	 * Buddy candidates are cache hot:
 	 */
Signature · 1 line
2.28.0.163.g6104cc2f0b6-goog
#1 lore raw
Quoted 14 lines · Josh Don

SMT siblings share caches, so cache hotness should be irrelevant for cross-sibling migration.

---
 kernel/sched/fair.c | 3 +++
 1 file changed, 3 insertions(+)
kernel/sched/fair.c +0 -0
index 2ba8f230feb9..5b203b55bcb2 100644

kernel/sched/fair.c static int task_hot(struct task_struct *p, struct lb_env *env) · line 7406

Quoted 5 lines
kernel/sched/fair.c +2 -0
@@ -7402,6 +7402,9 @@ static int task_hot(struct task_struct *p, struct lb_env *env)
 	if (unlikely(task_has_idle_policy(p)))
 		return 0;
 
+	if ((env->sd->flags & cpu_smt_flags()) == cpu_smt_flags())
+		return 0;

I think that wants to be:

	if (env->sd->flags & SD_SHARE_CPUCAPACITY)

Also, perhaps stick a comment on top with the rationale for this.

Quoted 7 lines
kernel/sched/fair.c +1 -0
+
 	/*
 	 * Buddy candidates are cache hot:
 	 */
Signature · 1 line
2.28.0.163.g6104cc2f0b6-goog
author
#2 lore raw
Quoted 19 lines · On Tue, Aug 4, 2020 at 3:56 AM <[email protected]> wrote:
Quoted 17 lines · Josh Don

SMT siblings share caches, so cache hotness should be irrelevant for cross-sibling migration.

---
 kernel/sched/fair.c | 3 +++
 1 file changed, 3 insertions(+)
kernel/sched/fair.c +0 -0
index 2ba8f230feb9..5b203b55bcb2 100644
@@ -7402,6 +7402,9 @@ static int task_hot(struct task_struct *p, struct lb_env *env)
      if (unlikely(task_has_idle_policy(p)))
              return 0;
 
Quoted 7 lines

kernel/sched/fair.c static int task_hot(struct task_struct *p, struct lb_env *env) · line 7406

Quoted 2 lines
kernel/sched/fair.c +2 -0
+     if ((env->sd->flags & cpu_smt_flags()) == cpu_smt_flags())
+             return 0;

I think that wants to be:

        if (env->sd->flags & SD_SHARE_CPUCAPACITY)

Agreed

Quoted 9 lines

Also, perhaps stick a comment on top with the rationale for this.

Quoted 7 lines
kernel/sched/fair.c +1 -1
+
      /*
       * Buddy candidates are cache hot:
       */
--

2.28.0.163.g6104cc2f0b6-goog

author
#3 lore raw

PATCH v2 1/1 sched/fair: ignore cache hotness for SMT migration

In reply to [email protected]

SMT siblings share caches, so cache hotness should be irrelevant for cross-sibling migration.

---
 kernel/sched/fair.c | 4 ++++
 1 file changed, 4 insertions(+)
kernel/sched/fair.c +4 -0
index 1a68a0536add..abdb54e2339f 100644
@@ -7402,6 +7402,10 @@ static int task_hot(struct task_struct *p, struct lb_env *env)
 	if (unlikely(task_has_idle_policy(p)))
 		return 0;
 
+	/* SMT siblings share cache */
+	if (env->sd->flags & SD_SHARE_CPUCAPACITY)
+		return 0;
+
 	/*
 	 * Buddy candidates are cache hot:
 	 */
Signature · 1 line
2.28.0.163.g6104cc2f0b6-goog
#4 lore raw
Quoted 14 lines · Josh Don

SMT siblings share caches, so cache hotness should be irrelevant for cross-sibling migration.

---
 kernel/sched/fair.c | 4 ++++
 1 file changed, 4 insertions(+)
kernel/sched/fair.c +0 -0
index 1a68a0536add..abdb54e2339f 100644

kernel/sched/fair.c static int task_hot(struct task_struct *p, struct lb_env *env) · line 7408

Quoted 7 lines
kernel/sched/fair.c +4 -0
@@ -7402,6 +7402,10 @@ static int task_hot(struct task_struct *p, struct lb_env *env)
 	if (unlikely(task_has_idle_policy(p)))
 		return 0;
 
+	/* SMT siblings share cache */
+	if (env->sd->flags & SD_SHARE_CPUCAPACITY)
+		return 0;
+

If this for retaining cache hotness, should we look at SD_SHARE_PKG_RESOURCES instead of SD_SHARE_CPUCAPACITY?

Quoted 6 lines
kernel/sched/fair.c +0 -0
 	/*
 	 * Buddy candidates are cache hot:
 	 */
Signature · 1 line
2.28.0.163.g6104cc2f0b6-goog
Signature · 2 lines
Thanks and Regards
Srikar Dronamraju
#5 lore raw
Quoted 19 lines · Srikar Dronamraju
Quoted 17 lines · Josh Don

SMT siblings share caches, so cache hotness should be irrelevant for cross-sibling migration.

---
 kernel/sched/fair.c | 4 ++++
 1 file changed, 4 insertions(+)
kernel/sched/fair.c +0 -0
index 1a68a0536add..abdb54e2339f 100644
@@ -7402,6 +7402,10 @@ static int task_hot(struct task_struct *p, struct lb_env *env)
     if (unlikely(task_has_idle_policy(p)))
             return 0;
 
Quoted 8 lines

kernel/sched/fair.c static int task_hot(struct task_struct *p, struct lb_env *env) · line 7408

Quoted 4 lines
kernel/sched/fair.c +4 -0
+	/* SMT siblings share cache */
+	if (env->sd->flags & SD_SHARE_CPUCAPACITY)
+		return 0;
+

If this for retaining cache hotness, should we look at SD_SHARE_PKG_RESOURCES instead of SD_SHARE_CPUCAPACITY?

Josh's patch only targets migrating tasks between threads of the same core - as he points out, cache hotness shouldn't matter here.

Using SD_SHARE_PKG_RESOURCES here would mean freely migrating tasks between any CPU of an LLC domain, which is quite likely something you do *not* want to do.

Quoted 6 lines
Quoted 6 lines
kernel/sched/fair.c +0 -1
     /*
      * Buddy candidates are cache hot:
      */
--

2.28.0.163.g6104cc2f0b6-goog

merge bot Applied
#6 lore raw

[tip: sched/core] sched/fair: Ignore cache hotness for SMT migration

In reply to Josh Don
The following commit has been merged into the sched/core branch of tip:

The following commit has been merged into the sched/core branch of tip:

Commit-ID:     ec73240b1627cddfd7cef018c7fa1c32e64a721e
Gitweb:        https://git.kernel.org/tip/ec73240b1627cddfd7cef018c7fa1c32e64a721e
Author:        Josh Don <[email protected]>
AuthorDate:    Tue, 04 Aug 2020 12:34:13 -07:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Wed, 26 Aug 2020 12:41:57 +02:00

sched/fair: Ignore cache hotness for SMT migration

SMT siblings share caches, so cache hotness should be irrelevant for cross-sibling migration.

---
 kernel/sched/fair.c | 4 ++++
 1 file changed, 4 insertions(+)
kernel/sched/fair.c +4 -0
index 1a68a05..abdb54e 100644
@@ -7402,6 +7402,10 @@ static int task_hot(struct task_struct *p, struct lb_env *env)
 	if (unlikely(task_has_idle_policy(p)))
 		return 0;
 
+	/* SMT siblings share cache */
+	if (env->sd->flags & SD_SHARE_CPUCAPACITY)
+		return 0;
+
 	/*
 	 * Buddy candidates are cache hot:
 	 */