NUMA API observations
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 3 lines · Thomas Zehetbauer
Looking at these numastat results and the default policy it seems that memory is primarily allocated on the first node which in turn means a unnecessarily large amount of page faults on the second node.
NUMA memory policy has nothing to do with page faults.
If you get most allocations on the first node it either means most programs run on the first node (assuming they don't use NUMA API to change their memory affinity) or more likely the programs running on node 0 need more memory than those running on node 1.
That's easily possible, e.g. a typical desktop uses most of its memory in the X server. If it runs on node 0 you get such skewed statistics. On servers it is often similar.
One way to combat that if it was really a problem would be to run the X server with interleaving policy (numactl --interleave=all XFree86)[1], but I would recommend careful benchmarks first if it's really a win. Normally better local memory latency is the better choice.
[1] Don't do that with startx or xinit, the rest of the X session should probably not use that.
Quoted 2 lines
I wonder if it is possible to better balance processes among the nodes by e.g. setting nodeAffinity = pid mod nodeCount
I assume you mean scheduling not memory affinity here. execve() and clone() do that kind of (but based on node loads, not pids), but not fork.
-Andi
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 10 lines · Andi Kleen
Quoted 8 lines · Paul Jackson
Quoted 5 lines · Andi
How should a user space application sanely discover the
cpumask_tsize needed by the kernel? Whoever designed that was on crack.I will probably make it loop and double the buffer until EINVAL ends or it passes a page and add a nasty comment.
Quoted 8 lines
Quoted 2 lines
I agree that a loop is needed. And yes someone didn't do a very good job of designing this interface.
I add some code to go upto a page now.
This adds a hardcoded limit of 32768 CPUs to libnuma. That's not nice, but we have to stop somewhere in case the EINVAL is returned for other reason
Should be enough for desktop machines...
Signature · 3 lines
-bill davidsen ([email protected]) "The secret to procrastination is to put things off until the last possible moment - but no longer" -me