Skip to contentSkip to conversation

xen: Add NUMA support to Xen

PATCH Superseded Ryan Harper 3 participants 27 messages May 1, 2006 – Jun 2, 2006

lore
Share & tools
lore
#0 lore raw

PATCH 1/6 xen: Add NUMA support to Xen

The attached tarball contains a script that will generate a patch that integrates NUMA support from Linux into Xen.

The integration support borrows heavily from linux/arch/i386/srat.c for ACPI SRAT table parsing rather than using the more heavyweight ACPI_NUMA. The SRAT table code has been tested on x86_32 and x86_64 and works with both 32-bit and 64-bit systems.

The code parses the SRAT table for memory and processor affinity and creates the proper node to cpu mapping. We define a new memory structure, node_memory_chunk_t, which is based on the same structure in linux, but tweaked for Xen. This structure is used to keep track of the discontiguous chunks of memory in the system and the node to which they belong.

We bring in Linux macros and methods for node function like cpu_to_node, num_online_nodes, node_to_cpumask, etc.

NUMA support in Xen is a compile-time option via the build system. Passing numa=y will enable all of the NUMA functionality. We have tested the code with numa=y and numa=n.

To create the first numa patch, unpack the attached tarball, numa.tgz, and execute numa.sh.

Signature · 5 lines
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253   T/L: 678-9253
[email protected]

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

#1 lore raw
Quoted 3 lines · Ryan Harper

NUMA support in Xen is a compile-time option via the build system. Passing numa=y will enable all of the NUMA functionality. We have tested the code with numa=y and numa=n.

I don't want to add a compile option for this -- I want NUMA enabled all the time and to add insignificant overhead on non-NUMA (or small-NUMA like AMD64) systems. In fact, there's no reason for it to add significant overhead in any situation really.

-- Keir

#2 lore raw
Quoted 11 lines · Keir Fraser
Quoted 3 lines · Ryan Harper

NUMA support in Xen is a compile-time option via the build system. Passing numa=y will enable all of the NUMA functionality. We have tested the code with numa=y and numa=n.

I don't want to add a compile option for this -- I want NUMA enabled all the time and to add insignificant overhead on non-NUMA (or small-NUMA like AMD64) systems. In fact, there's no reason for it to add significant overhead in any situation really.

OK, agreed.

Signature · 5 lines
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253   T/L: 678-9253
[email protected]
#3 lore raw

PATCH RESEND 1/6 xen: Add NUMA support to Xen

In reply to Keir Fraser
Quoted 7 lines · Keir Fraser
Quoted 3 lines · Ryan Harper

NUMA support in Xen is a compile-time option via the build system. Passing numa=y will enable all of the NUMA functionality. We have tested the code with numa=y and numa=n.

Quoted 6 lines

I don't want to add a compile option for this -- I want NUMA enabled all the time and to add insignificant overhead on non-NUMA (or small-NUMA like AMD64) systems. In fact, there's no reason for it to add significant overhead in any situation really.

-- Keir

Respun with no build-time option.

Signature · 5 lines
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253   T/L: 678-9253
[email protected]

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

#4 lore raw
Quoted 8 lines · Ryan Harper
Quoted 6 lines

I don't want to add a compile option for this -- I want NUMA enabled all the time and to add insignificant overhead on non-NUMA (or small-NUMA like AMD64) systems. In fact, there's no reason for it to add significant overhead in any situation really.

-- Keir

Respun with no build-time option.

The obvious file to pick on in this patchset is arch/x86/numa.c, derived from Linux's srat.c, but I expect the points can be applied more generally:
1. You re-indented. Normally a good thing but not for copies of Linux
source files. Please edit them and maintain them in Linux style (inc. hard tabs) as it makes it easier to sync with Linux updates.
2. Pointless EXPORT_SYMBOL() lines added. I can fully understand
keeping existing ones to make the diff smaller, but *why* would you add new ones?
3. You rename variables: the variable 'cpu_affinity' in
parse_cpu_affinity_structure() has become 'ca'. Now, I prefer your shorter name particularly since you add a bunch of code to that function, but it will make forward porting to new Linux source versions harder than it needs to be. Please don't do it.

Rather than taking Linux's srat.c and turning it into a more generic 'numa.c' file for Xen, perhaps it makes sense to define arch/x86/numa/ and stick srat.c in that subdir, and then add new file(s) in there for non-srat-related stuff?

-- Keir

#5 lore raw
Quoted 3 lines · Keir Fraser

1. You re-indented. Normally a good thing but not for copies of
Linux source files. Please edit them and maintain them in Linux style (inc. hard tabs) as it makes it easier to sync with Linux updates.

What Linux updates? The i386 NUMA support is basically dead and bitrots slowly (except perhaps on NUMAQ) Every half a year someone uses it and finds it doesn't work, maybe even fixes it and then it breaks shortly after again. That's because it isn't used and tested in any significant way. No distribution I'm aware of uses it either.

If you want maintained NUMA it would be much better to base on the x86-64 NUMA code.

-Andi

#6 lore raw
Quoted 4 lines · Andi Kleen
Quoted 3 lines

1. You re-indented. Normally a good thing but not for copies of
Linux source files. Please edit them and maintain them in Linux style (inc. hard tabs) as it makes it easier to sync with Linux updates.

Quoted 8 lines

What Linux updates? The i386 NUMA support is basically dead and bitrots slowly (except perhaps on NUMAQ) Every half a year someone uses it and finds it doesn't work, maybe even fixes it and then it breaks shortly after again. That's because it isn't used and tested in any significant way. No distribution I'm aware of uses it either.

If you want maintained NUMA it would be much better to base on the x86-64 NUMA code.

Sounds like an excellent plan to me: the x86/64 code looks cleaner too. Let's go with the x86/64 code then, unless there are any objections from the IBM folks?

Of course that will mean that stuff that has 'leaked out' from i386's srat.c (node_memory_chunk!) will definitely need to be got rid of, but that needed cleaning up anyway.

-- Keir

#7 lore raw
Quoted 13 lines · Keir Fraser
Quoted 10 lines · Andi Kleen
Quoted 3 lines

1. You re-indented. Normally a good thing but not for copies of
Linux source files. Please edit them and maintain them in Linux style (inc. hard tabs) as it makes it easier to sync with Linux updates.

What Linux updates? The i386 NUMA support is basically dead and bitrots slowly (except perhaps on NUMAQ) Every half a year someone uses it and finds it doesn't work, maybe even fixes it and then it breaks shortly after again. That's because it isn't used and tested in any significant way. No distribution I'm aware of uses it either.

Quoted 7 lines
Quoted 3 lines

If you want maintained NUMA it would be much better to base on the x86-64
NUMA code.

Sounds like an excellent plan to me: the x86/64 code looks cleaner too. Let's go with the x86/64 code then, unless there are any objections from the IBM folks?

When I looked at the x86_64 NUMA code, I noticed that SRAT parsing was done by ACPI_NUMA (drivers/acpi/numa.c). That code requires more ACPI support than Xen currently has available. Rather than pulling in all of that ACPI, I chose to start with the i386 SRAT table parser which was sufficient for both i386 and x86_64. I would think that using the modified i386 srat.c for parser gets the job done without bringing in all of the support needed to compile drivers/acpi/numa.c in Xen.

How about putting the two together: use the i386 SRAT parser and the x86_64 srat.c for structures and initialization?

Signature · 5 lines
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253   T/L: 678-9253
[email protected]
#8 lore raw
Quoted 10 lines · Ryan Harper

When I looked at the x86_64 NUMA code, I noticed that SRAT parsing was done by ACPI_NUMA (drivers/acpi/numa.c). That code requires more ACPI support than Xen currently has available. Rather than pulling in all of that ACPI, I chose to start with the i386 SRAT table parser which was sufficient for both i386 and x86_64. I would think that using the modified i386 srat.c for parser gets the job done without bringing in all of the support needed to compile drivers/acpi/numa.c in Xen.

How about putting the two together: use the i386 SRAT parser and the x86_64 srat.c for structures and initialization?

I guess it depends how cleanly it can be done. Can we use the x86/64 logic pretty much unmodified, but then pull in bits of the i386 parsing code as functions that the x86/64 code calls rather than the full-blown acpi routines it would usually call into? Or would it be possible to pull in drivers/acpi/numa.c without also having to pull in all 80kloc+ of acpi crap? This probably needs some thought and/or experimentation!

-- Keir

#9 lore raw
Quoted 8 lines · Keir Fraser
Quoted 8 lines

When I looked at the x86_64 NUMA code, I noticed that SRAT parsing was done by ACPI_NUMA (drivers/acpi/numa.c). That code requires more ACPI support than Xen currently has available. Rather than pulling in all of that ACPI, I chose to start with the i386 SRAT table parser which was sufficient for both i386 and x86_64. I would think that using the modified i386 srat.c for parser gets the job done without bringing in all of the support needed to compile drivers/acpi/numa.c in Xen.

Quoted 4 lines
Quoted 2 lines

How about putting the two together: use the i386 SRAT parser and the x86_64 srat.c for structures and initialization?

I guess it depends how cleanly it can be done. Can we use the x86/64 logic pretty much unmodified, but then pull in bits of the i386 parsing code as functions that the x86/64 code calls rather than the full-blown acpi routines it would usually call into? Or would it be possible to pull in drivers/acpi/numa.c without also having to pull in all 80kloc+ of acpi crap? This probably needs some thought and/or experimentation!

Option (b) looks plausible actually. After all, SRAT and SLIT are both static tables so they shouldn't need the full AML machinery. afaics acpi/numa.c mostly makes use of the acpi_table_parse*() family of functions, which we do (to some extent) already support in Xen.

I'd go for pulling in the acpi/numa.c file and see how far you can run with that.

-- Keir

#10 lore raw
In reply to Keir Fraser
Quoted 12 lines · Keir Fraser
Quoted 9 lines · Ryan Harper

When I looked at the x86_64 NUMA code, I noticed that SRAT parsing was done by ACPI_NUMA (drivers/acpi/numa.c). That code requires more ACPI support than Xen currently has available. Rather than pulling in all of
that ACPI, I chose to start with the i386 SRAT table parser which was sufficient for both i386 and x86_64. I would think that using the modified i386 srat.c for parser gets the job done without bringing in all of the support needed to compile drivers/acpi/numa.c in Xen.

Quoted 8 lines
Quoted 2 lines

How about putting the two together: use the i386 SRAT parser and the x86_64 srat.c for structures and initialization?

I guess it depends how cleanly it can be done. Can we use the x86/64 logic pretty much unmodified, but then pull in bits of the i386 parsing code as functions that the x86/64 code calls rather than the full-blown acpi routines it would usually call into? Or would it be possible to pull in drivers/acpi/numa.c

Yes it should be. numa parsing doesn't need the interpreter. It only reads a few tables from memory. You could probably even recode it easily to just call the callbacks in srat.c after looking for the tables in memory.

Bigger issue might be to make it fully 32bit clean. But probably just a few long -> long long should be enough.

One reason the i386 NUMA code is so ugly that limited 32bit address spaces and NUMA really don't love each other regarding ZONE_NORMAL handling. The i386 code has some nasty hacks to work around that, like these memory chunks. The result was always poor of course because NUMA essentially didn't apply to any kernel objects.

I guess in Xen that might be a bit easier from simpler zone setup (?)

Or if all fails limit NUMA again to
64bit hypervisors where it is much cleaner (AFAIK except for the old x440 and NUMAQ everything x86 doing NUMA is 64bit capable)

-Andi

#11 lore raw
In reply to Keir Fraser
Quoted 15 lines · Keir Fraser
Quoted 11 lines · Ryan Harper

When I looked at the x86_64 NUMA code, I noticed that SRAT parsing was done by ACPI_NUMA (drivers/acpi/numa.c). That code requires more ACPI support than Xen currently has available. Rather than pulling in all of
that ACPI, I chose to start with the i386 SRAT table parser which was sufficient for both i386 and x86_64. I would think that using the modified i386 srat.c for parser gets the job done without bringing in all of the support needed to compile drivers/acpi/numa.c in Xen.

How about putting the two together: use the i386 SRAT parser and the x86_64 srat.c for structures and initialization?

Quoted 6 lines

I guess it depends how cleanly it can be done. Can we use the x86/64 logic pretty much unmodified, but then pull in bits of the i386 parsing code as functions that the x86/64 code calls rather than the full-blown acpi routines it would usually call into? Or would it be possible to pull in drivers/acpi/numa.c without also having to pull in all 80kloc+ of acpi crap? This probably needs some thought and/or experimentation!

Well, I *tried* to use drivers/acpi/numa.c first, and I quit after I had to pull in more than few headers; it was just too intertwined. I think using the i386 parser and calling into the x86_64 logic should be do-able. I'll give it a go and post back here with the results.

Signature · 5 lines
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253   T/L: 678-9253
[email protected]
#12 lore raw
Quoted 4 lines · Ryan Harper

Well, I *tried* to use drivers/acpi/numa.c first, and I quit after I had to pull in more than few headers; it was just too intertwined. I think using the i386 parser and calling into the x86_64 logic should be do-able. I'll give it a go and post back here with the results.

Pulling in a few more header files (presumably from include/acpi?) would be okay, as long as you don't end up grabbing loads of .c files. But really i don't see the problem here -- drivers/acpi/numa.c hardly pulls in very many header files. It grabs acmacros.h and acpi_bus.h, both of which Xen has already.

Just take a logical look at that file: it clearly doesn't use much ACPI machinery and I really think it should be possible to port it down to Xen with not much modification. Unless you can give a concrete example why I'm wrong. :-)

I'd _much_ rather do that than have a franken-file of i386 and x86/64 code.

-- Keir

#13 lore raw
Quoted 15 lines · Keir Fraser
Quoted 5 lines · Ryan Harper

Well, I *tried* to use drivers/acpi/numa.c first, and I quit after I had
to pull in more than few headers; it was just too intertwined. I think using the i386 parser and calling into the x86_64 logic should be do-able. I'll give it a go and post back here with the results.

Pulling in a few more header files (presumably from include/acpi?) would be okay, as long as you don't end up grabbing loads of .c files. But really i don't see the problem here -- drivers/acpi/numa.c hardly pulls in very many header files. It grabs acmacros.h and acpi_bus.h, both of which Xen has already.

Quoted 4 lines

Just take a logical look at that file: it clearly doesn't use much ACPI machinery and I really think it should be possible to port it down to Xen with not much modification. Unless you can give a concrete example why I'm wrong. :-)

You weren't. It has been a while since I gave that a try (last summer using 2.6.11). Turns out that it does compile without any significant work (couple of defines required, but nothing that wouldn't come with NUMA header files).

What I'm fighting now is getting linux/arch/x86_64/mm/numa.c to build. Some of the structures to track numa info (struct pglist_data) are rather linux specific and include lots of structures related to Linux mm zones, check out linux/include/linux/mmzone.h. I attempted to avoid bring in that in, but the pfn_to_nid/phys_to_nid macros are based on memnodemap array.

My current thought is to trim down struct pglist_data into something that has the minimum information needed to populate the memnodemap array. I'll see how far that goes.

Signature · 5 lines
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253   T/L: 678-9253
[email protected]
#14 lore raw
Quoted 7 lines

What I'm fighting now is getting linux/arch/x86_64/mm/numa.c to build. Some of the structures to track numa info (struct pglist_data) are rather linux specific and include lots of structures related to Linux mm zones, check out linux/include/linux/mmzone.h. I attempted to avoid bring in that in, but the pfn_to_nid/phys_to_nid macros are based on memnodemap array.

I guess you can just replace setup_node_bootmem / setup_node_zones / numa_free_all_bootmem with Xen specific functions. They should be the only functions dealing with pglists and they're relatively straight forward.

-Andi

#15 lore raw
Quoted 11 lines · Andi Kleen
Quoted 6 lines

What I'm fighting now is getting linux/arch/x86_64/mm/numa.c to build. Some of the structures to track numa info (struct pglist_data) are rather linux specific and include lots of structures related to Linux mm zones, check out linux/include/linux/mmzone.h. I attempted to avoid bring in that in, but the pfn_to_nid/phys_to_nid macros are based on memnodemap array.

I guess you can just replace setup_node_bootmem / setup_node_zones / numa_free_all_bootmem with Xen specific functions. They should be the only functions dealing with pglists and they're relatively straight forward.

Yes, my gut feeling looking at x86_64's numa.c is that it's going to need some heavier surgery than srat.c. I wouldn't worry so much about keeping that one close to the Linux original: if we end up pulling down more Linux memory bookkeeping code later then we can always go back and sync the file more closely. Keep it as clean as possible though, obviously (e.g., replacing whole functions is nicer than functions that are a hacky halfway house between Linux and Xen, etc).

-- Keir

#16 lore raw
Quoted 8 lines

Yes, my gut feeling looking at x86_64's numa.c is that it's going to need some heavier surgery than srat.c. I wouldn't worry so much about keeping that one close to the Linux original: if we end up pulling down more Linux memory bookkeeping code later then we can always go back and sync the file more closely. Keep it as clean as possible though, obviously (e.g., replacing whole functions is nicer than functions that are a hacky halfway house between Linux and Xen, etc).

If it helps I can split these functions into smaller ones in the mainline sources. That could isolate the pglists in only very small functions.

-Andi

#17 lore raw
Quoted 10 lines · Andi Kleen
Quoted 7 lines

Yes, my gut feeling looking at x86_64's numa.c is that it's going to need some heavier surgery than srat.c. I wouldn't worry so much about keeping that one close to the Linux original: if we end up pulling down more Linux memory bookkeeping code later then we can always go back and sync the file more closely. Keep it as clean as possible though, obviously (e.g., replacing whole functions is nicer than functions that are a hacky halfway house between Linux and Xen, etc).

If it helps I can split these functions into smaller ones in the mainline sources. That could isolate the pglists in only very small functions.

Thanks: I guess we'll see how the new patch turns out. It would certainly be very useful if you could glance over the next revision of the NUMA patches -- at least the first one of the set which will be the one that takes code from x86/64 Linux. I'm certain you'll be able to pick up on details that I wouldn't spot.

-- Keir

#24 lore raw
In reply to Keir Fraser
Quoted 17 lines · Keir Fraser
Quoted 9 lines · Andi Kleen
Quoted 7 lines

What I'm fighting now is getting linux/arch/x86_64/mm/numa.c to build. Some of the structures to track numa info (struct pglist_data) are rather linux specific and include lots of structures related to Linux mm
zones, check out linux/include/linux/mmzone.h. I attempted to avoid bring in that in, but the pfn_to_nid/phys_to_nid macros are based on memnodemap array.

Quoted 4 lines

I guess you can just replace setup_node_bootmem / setup_node_zones / numa_free_all_bootmem with Xen specific functions. They should be the only
functions dealing with pglists and they're relatively straight forward.

Quoted 7 lines

Yes, my gut feeling looking at x86_64's numa.c is that it's going to need some heavier surgery than srat.c. I wouldn't worry so much about keeping that one close to the Linux original: if we end up pulling down more Linux memory bookkeeping code later then we can always go back and sync the file more closely. Keep it as clean as possible though, obviously (e.g., replacing whole functions is nicer than functions that are a hacky halfway house between Linux and Xen, etc).

Is there an equivalent function in Xen for linux/arch/x86_64/kernel/e820.c:find_e820_area() ?

I'm looking at linux/arch/x86_64/mm/numa.c:setup_node_bootmem() and as I understand, it finds a free location in the e820 map within a node's bounds to place the struct pglist_data which will hold node information.

Is this something we should do in Xen, or can we just have a static array of structures which hold the node information?

Signature · 5 lines
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253   T/L: 678-9253
[email protected]
#25 lore raw

a hacky halfway house between Linux and Xen, etc).

Quoted 3 lines

Is there an equivalent function in Xen for linux/arch/x86_64/kernel/e820.c:find_e820_area() ?

If not it should be easy to port. Most likely you can just use the x86-64 one after a few s/long/u64/

Quoted 2 lines

Is this something we should do in Xen, or can we just have a static array of structures which hold the node information?

Sooner or late you will need local per node state for the allocator. You can do the infrastructure for that right now

-Andi

#26 lore raw
Quoted 7 lines · Andi Kleen

a hacky halfway house between Linux and Xen, etc).

Quoted 3 lines

Is there an equivalent function in Xen for linux/arch/x86_64/kernel/e820.c:find_e820_area() ?

If not it should be easy to port. Most likely you can just use the x86-64 one after a few s/long/u64/

Yep.

Working bad_addr() and needing some help. Where should I be looking to find the bad areas that we need to avoid? Linux's bad_add() avoids < 8000 but I don't if that holds true for Xen. Also initial_images_start to end looks like that should be avoided, as does the frame table. Anything else?

Signature · 5 lines
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253   T/L: 678-9253
[email protected]
#18 lore raw
In reply to Keir Fraser
Quoted 14 lines · Keir Fraser
Quoted 11 lines · Andi Kleen
Quoted 10 lines

Yes, my gut feeling looking at x86_64's numa.c is that it's going to need some heavier surgery than srat.c. I wouldn't worry so much about keeping that one close to the Linux original: if we end up pulling down
more Linux memory bookkeeping code later then we can always go back and
sync the file more closely. Keep it as clean as possible though, obviously (e.g., replacing whole functions is nicer than functions that
are a hacky halfway house between Linux and Xen, etc).

Quoted 5 lines
Quoted 3 lines

If it helps I can split these functions into smaller ones in the mainline
sources. That could isolate the pglists in only very small functions.

Thanks: I guess we'll see how the new patch turns out. It would

I've got the ACPI numa.c parser function using linux/arch/x86_64/mm/srat.c almost entirely unmodified. linux/arch/x86_64/mm/numa.c was gutted retainly only the calls between itself and srat.c. I kept the memnodemap which has a nice phys_to_nid() function. There is still some cleanup to do, but I'd like some feedback on what I used and didn't.

I didn't bring in mmzone.h but I did pull a few ideas and defines from there. We have a simple node_data structure with start_pfn, spanned_pages and node_id.

I attempted to follow x86_64 numa.c which stashes the structure on node-local memory. This was problematic for 32-bit NUMA support. On the opteron 2-way that I had, the starting physical address for the second node is 80000000, and when turned into a virtual address (PAGE_OFFSET is FF000000), the resulting va is 17F000000, which overflows unsigned long in 32-bit. I started to use u64's, but several functions (like map_pages_to_xen()) only take unsigned longs. Rather than go through getting that function working, I think it is perfectly fine to just have a static array. The structure we store out there isn't accessed in any fast path. If the structure becomes more complicated in the future, or someone not having the structure on node-local memory becomes a performance issue we can revisit.

The patch is function on 32-bit and 64-bit boxes and parse the SRAT table and fills out the node_data array. I installed a simple keyhandler 'u' to dump the info to check that it was function after booting up.

Signature · 5 lines
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253   T/L: 678-9253
[email protected]

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

#19 lore raw
Quoted 10 lines · Ryan Harper

I've got the ACPI numa.c parser function using linux/arch/x86_64/mm/srat.c almost entirely unmodified. linux/arch/x86_64/mm/numa.c was gutted retainly only the calls between itself and srat.c. I kept the memnodemap which has a nice phys_to_nid() function. There is still some cleanup to do, but I'd like some feedback on what I used and didn't.

I didn't bring in mmzone.h but I did pull a few ideas and defines from there. We have a simple node_data structure with start_pfn, spanned_pages and node_id.

I hope Andi will give his opinion, but one thing I spotted immediately is that there are plenty of unnecessary diffs against the original files. For example, you don't need to remove __read_mostly: we don't implement it (yet) but we do define that attribute to nothing so it doesn't hurt to keep it in the source files. And why change printk() to DPRINTK(): if the original author thinks it's useful enough to always print out that's good enough for us, and it keeps the diff smaller. These are just a couple of examples: there might be others. These are only minor and easily fixed nits, of course!

-- Keir

#20 lore raw
In reply to Ryan Harper
Quoted 5 lines

The patch is function on 32-bit and 64-bit boxes and parse the SRAT table and fills out the node_data array. I installed a simple keyhandler 'u' to dump the info to check that it was function after booting up.

It mostly looks reasonable. As Keir noted you seem to have a lot of unnecessary changes (printk->DPRINTK, white space etc.)

I would also suggest you keep the numa emulation. It's useful for debugging so that developers without numa machines can reproduce numa issues.

-Andi

#21 lore raw
Quoted 8 lines · Andi Kleen
Quoted 5 lines

The patch is function on 32-bit and 64-bit boxes and parse the SRAT table and fills out the node_data array. I installed a simple keyhandler 'u' to dump the info to check that it was function after booting up.

It mostly looks reasonable. As Keir noted you seem to have a lot of unnecessary changes (printk->DPRINTK, white space etc.)

Cleaned up.

Quoted 3 lines

I would also suggest you keep the numa emulation. It's useful for debugging so that developers without numa machines can reproduce numa issues.

Added numa emulation back in. I've set it up such that if we fail to find a numa-node we fallback on to emulation of one node. This allows the allocator code to not be filled with CONFIG_NUMA ifdefs as Keir wanted.

Signature · 5 lines
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253   T/L: 678-9253
[email protected]

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

#22 lore raw
Quoted 13 lines · Ryan Harper
Quoted 8 lines · Andi Kleen
Quoted 5 lines

The patch is function on 32-bit and 64-bit boxes and parse the SRAT table and fills out the node_data array. I installed a simple keyhandler 'u' to dump the info to check that it was function after booting up.

It mostly looks reasonable. As Keir noted you seem to have a lot of unnecessary changes (printk->DPRINTK, white space etc.)

Cleaned up.

Quoted 1 line
Quoted 7 lines
Quoted 2 lines

I would also suggest you keep the numa emulation. It's useful for debugging so that developers without numa machines can reproduce numa issues.

Added numa emulation back in. I've set it up such that if we fail to find a numa-node we fallback on to emulation of one node. This allows the allocator code to not be filled with CONFIG_NUMA ifdefs as Keir wanted.

This last patch didn't have a functional cpu_to_node/node_to_cpumask. This patch fixes that up in addition to the previous cleanups. Please take a look.

Signature · 5 lines
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253   T/L: 678-9253
[email protected]

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

#23 lore raw
Quoted 17 lines · Ryan Harper
Quoted 16 lines · Ryan Harper
Quoted 8 lines · Andi Kleen
Quoted 5 lines

The patch is function on 32-bit and 64-bit boxes and parse the SRAT table and fills out the node_data array. I installed a simple keyhandler 'u' to dump the info to check that it was function after booting up.

It mostly looks reasonable. As Keir noted you seem to have a lot of unnecessary changes (printk->DPRINTK, white space etc.)

Cleaned up.

Quoted 3 lines

I would also suggest you keep the numa emulation. It's useful for debugging so that developers without numa machines can reproduce numa issues.

Quoted 8 lines
Quoted 4 lines

Added numa emulation back in. I've set it up such that if we fail to find a numa-node we fallback on to emulation of one node. This allows the allocator code to not be filled with CONFIG_NUMA ifdefs as Keir wanted.

This last patch didn't have a functional cpu_to_node/node_to_cpumask. This patch fixes that up in addition to the previous cleanups. Please take a look.

I wanted to follow up and see if anyone has had a chance to look at the latest rework (patch 1 and 2). Wasn't sure if this had fallen off your radar.

Signature · 5 lines
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253   T/L: 678-9253
[email protected]