Skip to contentSkip to conversation

143e1e28 sched: Rework sched_domain topology definition

DISCUSSION Mike Galbraith 2 participants 4 messages Jul 20, 2014

lore
Share & tools
lore
#0 lore raw

Hi,

+/*
+ * Topology list, bottom-up.
+ */
+static struct sched_domain_topology_level default_topology[] = { +#ifdef CONFIG_SCHED_SMT
+ { cpu_smt_mask, cpu_smt_flags, SD_INIT_NAME(SMT) },
+#endif
+#ifdef CONFIG_SCHED_MC
+ { cpu_coregroup_mask, cpu_core_flags, SD_INIT_NAME(MC) },
+#endif
+#ifdef CONFIG_SCHED_BOOK
+ { cpu_book_mask, SD_INIT_NAME(BOOK) },
+#endif
+ { cpu_cpu_mask, SD_INIT_NAME(DIE) },
+ { NULL, },
+};

Why did this commit rename the CPU domain to DIE?

-Mike

#1 lore raw
Quoted 19 lines · Mike Galbraith

Hi,

(quoted diff) +16 -0
+/*
+ * Topology list, bottom-up.
+ */
+static struct sched_domain_topology_level default_topology[] = {
+#ifdef CONFIG_SCHED_SMT
+	{ cpu_smt_mask, cpu_smt_flags, SD_INIT_NAME(SMT) },
+#endif
+#ifdef CONFIG_SCHED_MC
+	{ cpu_coregroup_mask, cpu_core_flags, SD_INIT_NAME(MC) },
+#endif
+#ifdef CONFIG_SCHED_BOOK
+	{ cpu_book_mask, SD_INIT_NAME(BOOK) },
+#endif
+	{ cpu_cpu_mask, SD_INIT_NAME(DIE) },
+	{ NULL, },
+};
Quoted 1 line

Why did this commit rename the CPU domain to DIE?

Because they're both wrong, but we found that DIE was less wrong than CPU.

Seeing how CPU typically means 1 logical, well, CPU, in the scheduler, and not the package you stick on your board.

We chose DIE over PACKAGE due to 1) its shorter name and 2) things like interlagos which have multiple DIEs in one PACKAGE etc..

Also, the only person who would ever care is the one who (with SCHED_DEBUG) enabled looks at the topology setup, which is maybe all of us 5.

#2 lore raw
Quoted 20 lines · Peter Zijlstra
Quoted 19 lines · Mike Galbraith

Hi,

(quoted diff) +16 -0
+/*
+ * Topology list, bottom-up.
+ */
+static struct sched_domain_topology_level default_topology[] = {
+#ifdef CONFIG_SCHED_SMT
+	{ cpu_smt_mask, cpu_smt_flags, SD_INIT_NAME(SMT) },
+#endif
+#ifdef CONFIG_SCHED_MC
+	{ cpu_coregroup_mask, cpu_core_flags, SD_INIT_NAME(MC) },
+#endif
+#ifdef CONFIG_SCHED_BOOK
+	{ cpu_book_mask, SD_INIT_NAME(BOOK) },
+#endif
+	{ cpu_cpu_mask, SD_INIT_NAME(DIE) },
+	{ NULL, },
+};
Quoted 4 lines
Quoted 1 line

Why did this commit rename the CPU domain to DIE?

Because they're both wrong, but we found that DIE was less wrong than CPU.

Ok, doesn't matter anyway, just looks peculiar.

Quoted 5 lines

Seeing how CPU typically means 1 logical, well, CPU, in the scheduler, and not the package you stick on your board.

We chose DIE over PACKAGE due to 1) its shorter name and 2) things like interlagos which have multiple DIEs in one PACKAGE etc..

It's actually a good name for some NUMA topologies. You don't ever want to see a CPU domain, that sucks cycles, so calling it DIE is fine :)

Quoted 3 lines

Also, the only person who would ever care is the one who (with SCHED_DEBUG) enabled looks at the topology setup, which is maybe all of us 5.

Maybe a few more, but yeah, not many would care what you called it.

-Mike

#3 lore raw
Quoted 28 lines · Mike Galbraith
Quoted 20 lines · Peter Zijlstra
Quoted 19 lines · Mike Galbraith

Hi,

(quoted diff) +16 -0
+/*
+ * Topology list, bottom-up.
+ */
+static struct sched_domain_topology_level default_topology[] = {
+#ifdef CONFIG_SCHED_SMT
+	{ cpu_smt_mask, cpu_smt_flags, SD_INIT_NAME(SMT) },
+#endif
+#ifdef CONFIG_SCHED_MC
+	{ cpu_coregroup_mask, cpu_core_flags, SD_INIT_NAME(MC) },
+#endif
+#ifdef CONFIG_SCHED_BOOK
+	{ cpu_book_mask, SD_INIT_NAME(BOOK) },
+#endif
+	{ cpu_cpu_mask, SD_INIT_NAME(DIE) },
+	{ NULL, },
+};
Quoted 4 lines
Quoted 1 line

Why did this commit rename the CPU domain to DIE?

Because they're both wrong, but we found that DIE was less wrong than CPU.

Ok, doesn't matter anyway, just looks peculiar.

Quoted 8 lines
Quoted 5 lines

Seeing how CPU typically means 1 logical, well, CPU, in the scheduler, and not the package you stick on your board.

We chose DIE over PACKAGE due to 1) its shorter name and 2) things like interlagos which have multiple DIEs in one PACKAGE etc..

It's actually a good name for some NUMA topologies. You don't ever want to see a CPU domain, that sucks cycles, so calling it DIE is fine :)

Like this 8 socket box. CPU domain with weight 64 stings.

vogelweide:~/:[0]# tune-sched-domains
usage: tune-sched-domains <val>
{cpu0/domain0:MC} SD flag: 559
+   1: SD_LOAD_BALANCE:          Do load balancing on this domain
+   2: SD_BALANCE_NEWIDLE:       Balance when about to become idle
+   4: SD_BALANCE_EXEC:          Balance on exec
+   8: SD_BALANCE_FORK:          Balance on fork, clone
-  16: SD_BALANCE_WAKE:          Wake to idle CPU on task wakeup
+  32: SD_WAKE_AFFINE:           Wake task to waking CPU
-  64: SD_PREFER_LOCAL:          Prefer to keep tasks local to this domain
- 128: SD_SHARE_CPUPOWER:        Domain members share cpu power
- 256: SD_POWERSAVINGS_BALANCE:  Balance for power savings
+ 512: SD_SHARE_PKG_RESOURCES:   Domain members share cpu pkg resources
-1024: SD_SERIALIZE:             Only a single load balancing instance
-2048: SD_ASYM_PACKING:          Place busy groups earlier in the domain
-4096: SD_PREFER_SIBLING:        Prefer to place tasks in a sibling domain
-8192: SD_PREFER_UTILIZATION:    Prefer utilization over SMP nice
{cpu0/domain1:CPU} SD flag: 4143
+   1: SD_LOAD_BALANCE:          Do load balancing on this domain
+   2: SD_BALANCE_NEWIDLE:       Balance when about to become idle
+   4: SD_BALANCE_EXEC:          Balance on exec
+   8: SD_BALANCE_FORK:          Balance on fork, clone
-  16: SD_BALANCE_WAKE:          Wake to idle CPU on task wakeup
+  32: SD_WAKE_AFFINE:           Wake task to waking CPU
-  64: SD_PREFER_LOCAL:          Prefer to keep tasks local to this domain
- 128: SD_SHARE_CPUPOWER:        Domain members share cpu power
- 256: SD_POWERSAVINGS_BALANCE:  Balance for power savings
- 512: SD_SHARE_PKG_RESOURCES:   Domain members share cpu pkg resources
-1024: SD_SERIALIZE:             Only a single load balancing instance
-2048: SD_ASYM_PACKING:          Place busy groups earlier in the domain
+4096: SD_PREFER_SIBLING:        Prefer to place tasks in a sibling domain
-8192: SD_PREFER_UTILIZATION:    Prefer utilization over SMP nice