mm/vmscan.c: only adjust related kswapd cpu affinity when online cpu
Subsystems (3)
PATCH v2 1/1 mm/vmscan.c: only adjust related kswapd cpu affinity when online cpu
When onlining a cpu, kswapd_cpu_online() is called to adjust kswapd cpu affinity.
Current routine does like this:
a) Iterate all the numa node
b) Adjust cpu affinity when node has an online cpu
For a) this is not necessary, since the particular online cpu belongs to a particular numa node. So it is not necessary to iterate on every nodes on the system. This new onlined cpu just affect kswapd cpu affinity of this particular node.
For b) several cpumask operation is used to check whether the node has an online CPU. Since at this point we are sure one of our CPU onlined, we can set the cpu affinity directly to current cpumask_of_node().
This patch simplifies the logic by set cpu affinity of the affected kswapd.
---
v2:
* rephrase the changelog
--- mm/vmscan.c | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-)
mm/vmscan.c +10 -9
@@ -4029,18 +4029,19 @@ unsigned long shrink_all_memory(unsigned long nr_to_reclaim)restore their cpu bindings. */ static int kswapd_cpu_online (unsigned int cpu) { -int nid; +int nid =cpu_to_node (cpu); +pg_data_t *pgdat; +const struct cpumask *mask; -for_each_node_state (nid,N_MEMORY ) { -pg_data_t *pgdat =NODE_DATA (nid); -const struct cpumask *mask; +if (!node_state (nid,N_MEMORY )) +return 0 ; - mask =cpumask_of_node (pgdat->node_id); + pgdat =NODE_DATA (nid); + mask =cpumask_of_node (nid); + +/* One of our CPUs online: restore mask */ +set_cpus_allowed_ptr (pgdat->kswapd, mask); -if (cpumask_any_and (cpu_online_mask, mask) < nr_cpu_ids) -/* One of our CPUs online: restore mask */ -set_cpus_allowed_ptr (pgdat->kswapd, mask); - }return 0 ; }
Signature · 1 line
2.17.1
Michal Hocko
reviewerQuoted 13 lines · Wei Yang
When onlining a cpu,
kswapd_cpu_online()is called to adjust kswapd cpu affinity.Current routine does like this:
a) Iterate all the numa node
b) Adjust cpu affinity when node has an online cpuFor a) this is not necessary, since the particular online cpu belongs to a particular numa node. So it is not necessary to iterate on every nodes on the system. This new onlined cpu just affect kswapd cpu affinity of this particular node.
Quoted 6 lines
For b) several cpumask operation is used to check whether the node has an online CPU. Since at this point we are sure one of our CPU onlined, we can set the cpu affinity directly to current
cpumask_of_node().This patch simplifies the logic by set cpu affinity of the affected kswapd.
How have you tested this patch?
Also this is an old code and quite convoluted but does it still work as inteded? I mean, I do not see any cpu offline callback to reduce the cpu mask as all the CPUs for the given node go offline? Wouldn't the scheduler simply go and fallback to no affinity if that happens? In other words what is the value of kswapd_cpu_online in the first place?
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 14 lines · Michal Hocko
Quoted 13 lines · Wei Yang
When onlining a cpu,
kswapd_cpu_online()is called to adjust kswapd cpu affinity.Current routine does like this:
a) Iterate all the numa node
b) Adjust cpu affinity when node has an online cpuFor a) this is not necessary, since the particular online cpu belongs to a particular numa node. So it is not necessary to iterate on every nodes on the system. This new onlined cpu just affect kswapd cpu affinity of this particular node.
Quoted 9 lines
Quoted 6 lines
For b) several cpumask operation is used to check whether the node has an online CPU. Since at this point we are sure one of our CPU onlined, we can set the cpu affinity directly to current
cpumask_of_node().This patch simplifies the logic by set cpu affinity of the affected kswapd.
How have you tested this patch?
I online one cpu and confirm the "cpu" is the one we just onlined.
If my understanding is correct, this is the expected behavior.
Quoted 3 lines
Also this is an old code and quite convoluted but does it still work as inteded? I mean, I do not see any cpu offline callback to reduce the cpu mask as all the CPUs for the given node go offline? Wouldn't the
You are right, I didn't see the counterpart for cpu offline. This is the question I want to ask. Seems we didn't handle it at the very beginning.
Quoted 3 lines
scheduler simply go and fallback to no affinity if that happens? In other words what is the value of
kswapd_cpu_onlinein the first place?
Some cases may this function be useful.
If we have a memory node which doesn't have any online cpu, the default cpumask is not set. After one of the cpu online, we want to change cpu affinity.
Or we want to add more cpu to the system, we could allow kswapd use more cpu resources. Otherwise, kswapd would be limited to those original cpus.
Quoted 3 lines
Signature · 2 lines
Michal Hocko SUSE Labs
Signature · 2 lines
Wei Yang Help you, Help me
Michal Hocko
reviewerQuoted 40 lines · Wei Yang
Quoted 14 lines · Michal Hocko
Quoted 13 lines · Wei Yang
When onlining a cpu,
kswapd_cpu_online()is called to adjust kswapd cpu affinity.Current routine does like this:
a) Iterate all the numa node
b) Adjust cpu affinity when node has an online cpuFor a) this is not necessary, since the particular online cpu belongs to a particular numa node. So it is not necessary to iterate on every nodes on the system. This new onlined cpu just affect kswapd cpu affinity of this particular node.
Quoted 9 lines
Quoted 6 lines
For b) several cpumask operation is used to check whether the node has an online CPU. Since at this point we are sure one of our CPU onlined, we can set the cpu affinity directly to current
cpumask_of_node().This patch simplifies the logic by set cpu affinity of the affected kswapd.
How have you tested this patch?
I online one cpu and confirm the "cpu" is the one we just onlined.
If my understanding is correct, this is the expected behavior.
Quoted 3 lines
Also this is an old code and quite convoluted but does it still work as inteded? I mean, I do not see any cpu offline callback to reduce the cpu mask as all the CPUs for the given node go offline? Wouldn't the
You are right, I didn't see the counterpart for cpu offline. This is the question I want to ask. Seems we didn't handle it at the very beginning.
Quoted 3 lines
scheduler simply go and fallback to no affinity if that happens? In other words what is the value of
kswapd_cpu_onlinein the first place?
Quoted 8 lines
Some cases may this function be useful.
If we have a memory node which doesn't have any online cpu, the default cpumask is not set. After one of the cpu online, we want to change cpu affinity.
Or we want to add more cpu to the system, we could allow kswapd use more cpu resources. Otherwise, kswapd would be limited to those original cpus.
OK, so the usecase is when a NUMA node gains a new CPU which wasn't there at the time when the node got onlined. Is this a scenario we really do care about? While not completely impossible I haven't seen a system which would allow such a runtime configurability. Maybe it would be simply easier to drop the callback for now until we have a real world usecase to support it and have it documented.
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 50 lines · Michal Hocko
Quoted 40 lines · Wei Yang
Quoted 14 lines · Michal Hocko
Quoted 13 lines · Wei Yang
When onlining a cpu,
kswapd_cpu_online()is called to adjust kswapd cpu affinity.Current routine does like this:
a) Iterate all the numa node
b) Adjust cpu affinity when node has an online cpuFor a) this is not necessary, since the particular online cpu belongs to a particular numa node. So it is not necessary to iterate on every nodes on the system. This new onlined cpu just affect kswapd cpu affinity of this particular node.
Quoted 9 lines
Quoted 6 lines
For b) several cpumask operation is used to check whether the node has an online CPU. Since at this point we are sure one of our CPU onlined, we can set the cpu affinity directly to current
cpumask_of_node().This patch simplifies the logic by set cpu affinity of the affected kswapd.
How have you tested this patch?
I online one cpu and confirm the "cpu" is the one we just onlined.
If my understanding is correct, this is the expected behavior.
Quoted 3 lines
Also this is an old code and quite convoluted but does it still work as inteded? I mean, I do not see any cpu offline callback to reduce the cpu mask as all the CPUs for the given node go offline? Wouldn't the
You are right, I didn't see the counterpart for cpu offline. This is the question I want to ask. Seems we didn't handle it at the very beginning.
Quoted 3 lines
scheduler simply go and fallback to no affinity if that happens? In other words what is the value of
kswapd_cpu_onlinein the first place?Quoted 8 lines
Some cases may this function be useful.
If we have a memory node which doesn't have any online cpu, the default cpumask is not set. After one of the cpu online, we want to change cpu affinity.
Or we want to add more cpu to the system, we could allow kswapd use more cpu resources. Otherwise, kswapd would be limited to those original cpus.
Quoted 6 lines
OK, so the usecase is when a NUMA node gains a new CPU which wasn't there at the time when the node got onlined. Is this a scenario we really do care about? While not completely impossible I haven't seen a system which would allow such a runtime configurability. Maybe it would be simply easier to drop the callback for now until we have a real world usecase to support it and have it documented.
I am fine with this suggestion. Let me prepare v3.
Quoted 3 lines
Signature · 2 lines
Michal Hocko SUSE Labs
Signature · 2 lines
Wei Yang Help you, Help me