Skip to contentSkip to conversation

Question on build_sched_domains

DISCUSSION Srivatsa Vaddagiri 2 participants 6 messages Mar 24, 2006 – Mar 26, 2006

lore
Share & tools
lore
#0 lore raw

Nick,
I was going thr' build_sched_domains and had a question
regarding formation of sched_groups for NUMA nodes. There are two 'for' loops, each loop possibly allocating memory (sched_group) for one or more nodes. My question is: in the outer loop, don't we need to skip allocating for nodes for whom the inner loop has allocated in an earlier pass?

Taking the example of 4 node system which are in the same sched_domain_node_span(), I see that we end up allocating 16 times (when 4 would have sufficed?).

What am I missing here?

Signature · 2 lines
Regards,
vatsa
#1 lore raw
Quoted 3 lines · Srivatsa Vaddagiri

Taking the example of 4 node system which are in the same sched_domain_node_span(), I see that we end up allocating 16 times (when 4 would have sufficed?).

Maybe this is it to avoid touching same memory from different nodes?

Signature · 2 lines
Regards,
vatsa
#2 lore raw
Quoted 9 lines · Srivatsa Vaddagiri
Quoted 3 lines · Srivatsa Vaddagiri

Taking the example of 4 node system which are in the same sched_domain_node_span(), I see that we end up allocating 16 times (when 4 would have sufficed?).

Maybe this is it to avoid touching same memory from different nodes?

Yeah I think what's happening is that the sched groups structures are not shared between nodes. (It's been a while since I looked at this code, and it is a bit tricky to follow).

Aside, it should be using kmalloc_node now...

Signature · 2 lines
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com 
#3 lore raw
Quoted 3 lines · Nick Piggin

Yeah I think what's happening is that the sched groups structures are not shared between nodes. (It's been a while since I looked at this code, and it is a bit tricky to follow).

Its really odd that sched group structure aren't shared between nodes in some case (sched_group_nodes) and are shared in other cases (sched_group_cpus, sched_group_core, sched_group_phys).

Also is the GFP_ATOMIC allocation really required (for sched_group_nodes) in build_sched_domains()?

Signature · 2 lines
Regards,
vatsa
#4 lore raw
Quoted 7 lines · Srivatsa Vaddagiri
Quoted 3 lines · Nick Piggin

Yeah I think what's happening is that the sched groups structures are not shared between nodes. (It's been a while since I looked at this code, and it is a bit tricky to follow).

Quoted 5 lines

Its really odd that sched group structure aren't shared between nodes in some case (sched_group_nodes) and are shared in other cases (sched_group_cpus, sched_group_core, sched_group_phys).

Also is the GFP_ATOMIC allocation really required (for sched_group_nodes) in build_sched_domains()?

I don't see why. It should be changed to GFP_KERNEL.

Signature · 2 lines
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com 
#5 lore raw
Quoted 1 line · Nick Piggin

I don't see why. It should be changed to GFP_KERNEL.

Thats what I thought too. Will submit a patch to that effect, while I am cleaning up build_sched_domains.

Signature · 2 lines
Regards,
vatsa