Question on build_sched_domains
Nick,
I was going thr' build_sched_domains and had a question
regarding formation of sched_groups for NUMA nodes. There are two 'for' loops, each loop possibly allocating memory (sched_group) for one or more nodes. My question is: in the outer loop, don't we need to skip allocating for nodes for whom the inner loop has allocated in an earlier pass?
Taking the example of 4 node system which are in the same sched_domain_node_span(), I see that we end up allocating 16 times (when 4 would have sufficed?).
What am I missing here?
Signature · 2 lines
Regards, vatsa
Quoted 3 lines · Srivatsa Vaddagiri
Taking the example of 4 node system which are in the same
sched_domain_node_span(), I see that we end up allocating 16 times (when 4 would have sufficed?).
Maybe this is it to avoid touching same memory from different nodes?
Signature · 2 lines
Regards, vatsa
Quoted 9 lines · Srivatsa Vaddagiri
Quoted 3 lines · Srivatsa Vaddagiri
Taking the example of 4 node system which are in the same
sched_domain_node_span(), I see that we end up allocating 16 times (when 4 would have sufficed?).Maybe this is it to avoid touching same memory from different nodes?
Yeah I think what's happening is that the sched groups structures are not shared between nodes. (It's been a while since I looked at this code, and it is a bit tricky to follow).
Aside, it should be using kmalloc_node now...
Signature · 2 lines
SUSE Labs, Novell Inc. Send instant messages to your online friends http://au.messenger.yahoo.com
Quoted 3 lines · Nick Piggin
Yeah I think what's happening is that the sched groups structures are not shared between nodes. (It's been a while since I looked at this code, and it is a bit tricky to follow).
Its really odd that sched group structure aren't shared between nodes in some case (sched_group_nodes) and are shared in other cases (sched_group_cpus, sched_group_core, sched_group_phys).
Also is the GFP_ATOMIC allocation really required (for sched_group_nodes) in build_sched_domains()?
Signature · 2 lines
Regards, vatsa
Quoted 7 lines · Srivatsa Vaddagiri
Quoted 3 lines · Nick Piggin
Yeah I think what's happening is that the sched groups structures are not shared between nodes. (It's been a while since I looked at this code, and it is a bit tricky to follow).
Quoted 5 lines
Its really odd that sched group structure aren't shared between nodes in some case (
sched_group_nodes) and are shared in other cases (sched_group_cpus,sched_group_core,sched_group_phys).Also is the
GFP_ATOMICallocation really required (forsched_group_nodes) inbuild_sched_domains()?
I don't see why. It should be changed to GFP_KERNEL.
Signature · 2 lines
SUSE Labs, Novell Inc. Send instant messages to your online friends http://au.messenger.yahoo.com
Quoted 1 line · Nick Piggin
I don't see why. It should be changed to
GFP_KERNEL.
Thats what I thought too. Will submit a patch to that effect, while I am cleaning up build_sched_domains.
Signature · 2 lines
Regards, vatsa