Make dynamic/run-time configuration of zonelist order configurable

[PATCH] Make dynamic/run-time configuration of zonelist order configurable

Against 2.6.22-rc2-mm1 with the huge page allocation fix applied:

	http://marc.info/?l=linux-mm&m=117935390224779&w=4

The patch series to make the numa zonelist order configurable removed the __*init* attributes from the zonelist build functions and related data, and added a sysctl to reconfigure the zonelist order at runtime, as well as a boot parameter. Not all systems require this feature, so they should not have to incur the overhead of the additional sysctl nor keeping the zonelist build functions around at runtime. [This might be of concern to users of 32-bit systems using numa emulation for resource management.]

This patch makes the 'vm.numa_zonelist_order' sysctl varible configurable via the DYNAMIC_ZONELIST_ORDER Kconfig option and makes the runtime availability of the zonelist build functions depend on DYNAMIC_ZONELIST_ORDER or MEMORY_HOTPLUG.

Built and boot tested on ia64 and 2 node/socket x86_64, with various values of above config options.

 include/linux/mmzone.h |   16 ++++++++++++++++
 kernel/sysctl.c        |    2 +-
 mm/Kconfig             |    7 +++++++
 mm/page_alloc.c        |   47 +++++++++++++++++++++++++----------------------
 4 files changed, 49 insertions(+), 23 deletions(-)

Index: Linux/include/linux/mmzone.h
===================================================================
@@ -432,6 +432,20 @@ struct zonelist {
 #endif
 };
 
+/*
+ * no need to keep zonelist build functions around after init, unless
+ * we've configured DYNAMIC_ZONELIST_ORDER or MEMORY_HOTPLUG
+ */
+#if defined(CONFIG_DYNAMIC_ZONELIST_ORDER) || defined(CONFIG_MEMORY_HOTPLUG)
+#define ZL_INIT
+#define ZL_MEMINIT
+#define ZL_MEMINITDATA
+#else
+#define ZL_INIT	__init
+#define ZL_MEMINIT __meminit
+#define ZL_MEMINITDATA __meminitdata
+#endif
+
 #ifdef CONFIG_ARCH_POPULATES_NODE_MAP
 struct node_active_region {
 	unsigned long start_pfn;

@@ -611,10 +625,12 @@ int sysctl_min_unmapped_ratio_sysctl_han
 int sysctl_min_slab_ratio_sysctl_handler(struct ctl_table *, int,
 			struct file *, void __user *, size_t *, loff_t *);
 
+#ifdef CONFIG_DYNAMIC_ZONELIST_ORDER
 extern int numa_zonelist_order_handler(struct ctl_table *, int,
 			struct file *, void __user *, size_t *, loff_t *);
 extern char numa_zonelist_order[];
 #define NUMA_ZONELIST_ORDER_LEN 16	/* string buffer size */
+#endif
 
 #include <linux/topology.h>
 /* Returns the number of the current Node. */

Index: Linux/kernel/sysctl.c
===================================================================
@@ -944,7 +944,7 @@ static ctl_table vm_table[] = {
 		.proc_handler	= &proc_dointvec_jiffies,
 		.strategy	= &sysctl_jiffies,
 	},
-#ifdef CONFIG_NUMA
+#ifdef CONFIG_DYNAMIC_ZONELIST_ORDER
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "numa_zonelist_order",

Index: Linux/mm/Kconfig
===================================================================
@@ -172,3 +172,10 @@ config NR_QUICK
 	depends on QUICKLIST
 	default "2" if (SUPERH && !SUPERH64)
 	default "1"
+
+config DYNAMIC_ZONELIST_ORDER
+	bool "runtime configuration of zonelist order via sysctl"
+	depends on NUMA
+	help
+	  Supports the runtime reconfiguration of zonelist order via
+	  a sysctl variable:  vm.numa_zonelist_order.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

no problem was found on my ia64 test box.

But one point..

Hi, Kame:

Here was my thinking on that point--actually on the numa_zonelist_order sysctl in general:

When would it ever be needed? I think the answer is when the dynamic configuration didn't "get it right" and you want to fix it w/o reboot. I agree that it is very likely that you wouldn't want to reboot if you've gone to all the trouble to support memory hotplug. In that case, we have a couple of choices:

1) just enable the config option manually for platforms that support memory hotplug, or

2) make the option default to 'y' when memory hotplug is configured.

As I tried to indicate in the patch description, because some smallish systems appear to use NUMA emulation for resource management, they must enable NUMA support. However, they may not [I think probably not] need run-time zoneorder configuration. I suspect those platforms don't support memory hotplug at this time either. Now, the code in question [sysctl and build_zonelist functions] may be relatively small--i.e., less than a page. But, it all adds up, so I offered a way for small systems that still wanted NUMA support to get back what we took from them with the zonelist order patches.

Thoughts?

Lee

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Ah....I know your purpose. What I wanted to say is memory hotplug itseld doesn't need "manual" zonelist ordering reconfiguration....it calls build_all_zonelists() in automatic way. Then #ifdef around this sysctl should be CONFIG_NUMA && CONFIG_DYNAMIC_ZONELIST_ORDER.

I know your point and benefits of this patch...(we can remove __init functions if we don't use it and reduce memory usage to some extent.)
To be honest, I myself doesn't like addling new complicated __init thing. But I have no concern if maintainers say ok.

Thanks,
-Kame

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Since CONFIG_DYNAMIC_ZONELIST_ORDER depends on CONFIG_NUMA in my patch, I didn't think the && was necessary.

Well, if Andrew doesn't think it's worth it, I'll drop it.

Lee

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

I wasn't cc'ed on "[PATCH/RFC] Fix hugetlb pool allocation with empty nodes
- V4" so I didn't apply it hence cannot apply this.

Plus I'd prefer not to, really. This patch should be folded into change-zonelist-order-zonelist-order-selection-logic.patch somehow, but I cannot do that if it is dependent upon the unrelated "[PATCH/RFC] Fix hugetlb pool allocation with empty nodes - V4".

Better would be to raise a patch relative to the change-zonelist-order-* patches, please. Then we can take a look at the hugetlb fix independently.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

My send folder should you copied on the patch referenced by the link above, but that doesn't mean you got it... As far as the status of that patch, I'm still unclear on Nish's testing. He ack'd for x86_64 where he had a problem that V4 was supposed to fix, but couldn't test on ppc. So, I didn't think he's ready for it to go in.

Well, I really didn't expect this one to go right in. We needed to hear from Kame first. Probably should have added an 'RFC'...

zonelist order stuff is already in 22-rc2-mm and I considered holding off on this patch until Nish ack'd or nack'd the hugetlb fix. Sorry for the confusion. But, before I go and rework it against the current mm tree and then rebase the hugetlb fix on that, could you offer an opinion either way, whether it's worth the effort and a new Kconfig option to attempt to give back this amount init code/data? I recall you making noise about the zonelist order patch being "a lot of code" when Kame first posted it.

Lee

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

The concern with a "lot of code" is 99% about complexity, reliability and maintainability and only 1% about RAM usage.

This stuff is mainly a NUMA/SMP thing, isn't it? If so, a couple of k is neither here nor there.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Ah, OK.

I agree that on the platforms I deal with, it's not an issue. However, as I mentioned, I've seen chatter on the lists from folks who apparently use numa emulation [requires NUMA infrastructure] on single cpu 32-bit systems as a "poor man's containers" for memory resource management and didn't want to impose the addtional, admittedly small, extra overhead on them. Perhaps this usage goes away when containers becomes mainline. If no one in that camp complains and you don't think it's worth it, I'll drop it.

Lee

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>