fix zonelist ordering for NUMA
Quoted 11 lines · --On Wednesday, February 25, 2004 08:54:09 -0800 Jesse Barnes…
Quoted 6 lines · On Wed, Feb 25, 2004 at 02:01:16PM +0900, [email protected] wrote:
Quoted 3 lines
1) make it arch independent
this means having arch code populate a SLIT-like table for use by the generic zonelist building codeI would like to hear the comments from people on other arch. If the same ordering rule can be applicable for others, it's nice.
Martin, does a scheme like this sound ok with you? Arch specific code would populate a node distance table, which would be used to build each
pgdat->zonelistin a smarter way than we do currently.
Yeah, looks sensible to me. We probably ought to do this:
+#ifndef node_distance
+#define node_distance(from,to) (1)
+#endif
in the generic fallback topology headers, not in the mm/ .c files. Matt?
Also, I seem to recall those build_zonelists functions are used for both NUMA and UMA ... now they're getting complex enough that it's probably worth making a specific non-NUMA version, if only for the sanity of 99% of the poor souls trying to work out how a UMA machine lays it out ;-)
It looks like it won't change ordering for existing boxes with single layer flat NUMA topologies (round-robin), but we probably ought to check that carefully ;-)
M.
Quoted 27 lines · Martin J. Bligh
Quoted 11 lines · --On Wednesday, February 25, 2004 08:54:09 -0800 Jesse Barnes…
Quoted 6 lines · On Wed, Feb 25, 2004 at 02:01:16PM +0900, [email protected] wrote:
Quoted 3 lines
1) make it arch independent
this means having arch code populate a SLIT-like table for use by the generic zonelist building codeI would like to hear the comments from people on other arch. If the same ordering rule can be applicable for others, it's nice.
Martin, does a scheme like this sound ok with you? Arch specific code would populate a node distance table, which would be used to build each
pgdat->zonelistin a smarter way than we do currently.Yeah, looks sensible to me. We probably ought to do this:
(quoted diff) +3
-0+#ifndef node_distance +#define node_distance(from,to) (1) +#endifin the generic fallback topology headers, not in the
mm/.c files. Matt?Also, I seem to recall those
build_zonelistsfunctions are used for both NUMA and UMA ... now they're getting complex enough that it's probably worth making a specific non-NUMA version, if only for the sanity of 99% of the poor souls trying to work out how a UMA machine lays it out ;-)
Quoted 5 lines
It looks like it won't change ordering for existing boxes with single layer flat NUMA topologies (round-robin), but we probably ought to check that carefully ;-)
M.
Yep... Here's a quickie for i386 and the generic header. All other arches would look pretty similar to the asm/i386/topology.h change.
-Matt
include/asm-generic/topology.h +3 -0
@@ -44,6 +44,9 @@#ifndef pcibus_to_cpumask#define pcibus_to_cpumask (bus) (cpu_online_map)#endif +#ifndef node_distance +#define node_distance (from, to) (1) +#endif /* Cross-node load balancing interval. */ #ifndef NODE_BALANCE_RATE
include/asm-i386/topology.h +6 -0
@@ -66,6 +66,12 @@ static inline cpumask_t pcibus_to_cpumasreturn node_to_cpumask (mp_bus_id_to_node[bus]); } +/* Node-to-Node distance */ +static inline int node_distance (int from,int to) +{ +return 1 ; +} +/* Cross-node load balancing interval. */ #define NODE_BALANCE_RATE 100
Quoted 8 lines · Martin J. Bligh
Also, I seem to recall those
build_zonelistsfunctions are used for both NUMA and UMA ... now they're getting complex enough that it's probably worth making a specific non-NUMA version, if only for the sanity of 99% of the poor souls trying to work out how a UMA machine lays it out ;-)It looks like it won't change ordering for existing boxes with single layer flat NUMA topologies (round-robin), but we probably ought to check that carefully ;-)
More ifdefs just make the code messier. How about adding a nice comment explaining that UMA will fall right through the loop, and giving the poor souls a temporary reprieve?
-- dave
Quoted 6 lines · Martin J. Bligh
Quoted 5 lines
Also, I seem to recall those
build_zonelistsfunctions are used for both NUMA and UMA ... now they're getting complex enough that it's probably worth making a specific non-NUMA version, if only for the sanity of 99% of the poor souls trying to work out how a UMA machine lays it out ;-)
Quoted 7 lines
Quoted 3 lines
It looks like it won't change ordering for existing boxes with single layer flat NUMA topologies (round-robin), but we probably ought to check that carefully ;-)
More ifdefs just make the code messier. How about adding a nice comment explaining that UMA will fall right through the loop, and giving the poor souls a temporary reprieve?
In this case, I disagree - it's better to just have the ifdef.
M.
Review
Quoted 3 lines · Martin J. Bligh
(quoted diff) +3
-0+#ifndef node_distance +#define node_distance(from,to) (1) +#endif
1 or 0 here?
--cw
Quoted 7 lines · --On Thursday, February 26, 2004 16:47:10 -0800 Chris Wedgwood <[email protected]>…
Review
Quoted 3 lines · Martin J. Bligh
(quoted diff) +3
-0+#ifndef node_distance +#define node_distance(from,to) (1) +#endif1 or 0 here?
Dunno ... was from [email protected], but I'd guess 1 makes more sense semantically - we're saying the distance between nodes on a flat NUMA architecture is equal (I think ;-))
M.
Hi,
Quoted 9 lines
Quoted 5 lines
Review
Quoted 3 lines
(quoted diff) +3
-0+#ifndef node_distance +#define node_distance(from,to) (1) +#endif1 or 0 here?
Dunno ... was from [email protected], but I'd guess 1 makes more sense semantically - we're saying the distance between nodes on a flat NUMA architecture is equal (I think ;-))
Umm. Even on a flat NUMA architecture, I should have distinguish between local and remote. It's NUMA.
#define node_distance(from,to) (from != to)
Best regards.
--
NOMURA, Jun'ichi <[email protected]>