linux-next ia64 build problems in slqb
The ia64 "allnoconfig" build ends up with the (possibly dubious) combination of .config options:
CONFIG_SMP=n
CONFIG_NUMA=y
which leads to the following build problems with mm/slqb.c
CC mm/slqb.omm/slqb.c: In function ‘__slab_free’:mm/slqb.c:1735: error: implicit declaration of function ‘slab_free_to_remote’ mm/slqb.c: In function ‘kmem_cache_open’:mm/slqb.c:2274: error: implicit declaration of function ‘kmem_cache_dyn_array_free’ mm/slqb.c:2275: warning: label ‘error_cpu_array’ defined but not used mm/slqb.c: In function ‘kmem_cache_destroy’:mm/slqb.c:2395: error: implicit declaration of function ‘claim_remote_free_list’ mm/slqb.c: In function ‘kmem_cache_init’:mm/slqb.c:2885: error: ‘per_cpu__kmem_cpu_nodes’ undeclared (first use in this function) mm/slqb.c:2885: error: (Each undeclared identifier is reported only once mm/slqb.c:2885: error: for each function it appears in.)mm/slqb.c:2886: error: ‘kmem_cpu_cache’ undeclared (first use in this function)
-Tony
Quoted 8 lines · Tony
The ia64 "allnoconfig" build ends up with the (possibly dubious) combination of .config options:
CONFIG_SMP=n
CONFIG_NUMA=ywhich leads to the following build problems with
mm/slqb.c
Quoted 13 lines
CC
mm/slqb.omm/slqb.c: In function ‘__slab_free’:mm/slqb.c:1735: error: implicit declaration of function ‘slab_free_to_remote’mm/slqb.c: In function ‘kmem_cache_open’:mm/slqb.c:2274: error: implicit declaration of function ‘kmem_cache_dyn_array_free’mm/slqb.c:2275: warning: label ‘error_cpu_array’ defined but not usedmm/slqb.c: In function ‘kmem_cache_destroy’:mm/slqb.c:2395: error: implicit declaration of function ‘claim_remote_free_list’mm/slqb.c: In function ‘kmem_cache_init’:mm/slqb.c:2885: error: ‘per_cpu__kmem_cpu_nodes’ undeclared (first use in this function)mm/slqb.c:2885: error: (Each undeclared identifier is reported only oncemm/slqb.c:2885: error: for each function it appears in.)mm/slqb.c:2886: error: ‘kmem_cpu_cache’ undeclared (first use in this function)
Does this patch fix it?
Quoted 3 lines
Does this patch fix it?
That makes all the compilation errors go away from the allnoconfig build. Build still seems OK on a more conventional .config (tiger_defconfig).
I haven't tried booting this kernel though, so I don't know whether slqb works or not. Perhaps I'll get brave and try that tomorrow.
-Tony
Hi Tony,
Quoted 5 lines · Tony
The ia64 "allnoconfig" build ends up with the (possibly dubious) combination of .config options:
CONFIG_SMP=n
CONFIG_NUMA=y
Yeah, I don't think the combination makes much sense and x86 doesn't allow it. Can we fix that in ia64 Kconfig? That said, this seems to come up with every architecture except x86, so if someone is kind enough to send me a tested fix for slqb, I'll just go ahead and apply it.
Pekka
Quoted 6 lines · Pekka Enberg
Hi Tony,
Quoted 3 lines · Tony
The ia64 "allnoconfig" build ends up with the (possibly dubious) combination of .config options:
Quoted 9 lines
Quoted 2 lines
? ? ? ?CONFIG_SMP=n
? ? ? ?CONFIG_NUMA=yYeah, I don't think the combination makes much sense and x86 doesn't allow it. Can we fix that in ia64
Kconfig? That said, this seems to come up with every architecture except x86, so if someone is kind enough to send me a tested fix for slqb, I'll just go ahead and apply it.
This configuration is used on SH quite regularly, so if SLQB breaks, then that is a regression.
Quoted 4 lines · Pekka Enberg
Quoted 3 lines · Tony
The ia64 "allnoconfig" build ends up with the (possibly dubious) combination of .config options:
Quoted 8 lines
Quoted 2 lines
CONFIG_SMP=n
CONFIG_NUMA=yYeah, I don't think the combination makes much sense and x86 doesn't allow it. Can we fix that in ia64
Kconfig? That said, this seems to come up with every architecture except x86, so if someone is kind enough to send me a tested fix for slqb, I'll just go ahead and apply it.
Tony runs tests with this configuration. This is an established configuration and has been historically supported.
Quoted 11 lines · Tony
Quoted 11 lines
Quoted 5 lines
The ia64 "allnoconfig" build ends up with the (possibly dubious) combination of .config options:
CONFIG_SMP=n
CONFIG_NUMA=yYeah, I don't think the combination makes much sense and x86 doesn't allow it. Can we fix that in ia64
Kconfig? That said, this seems to come up with every architecture except x86, so if someone is kind enough to send me a tested fix for slqb, I'll just go ahead and apply it.
Quoted 2 lines · Christoph Lameter
Tony runs tests with this configuration. This is an established configuration and has been historically supported.
Interesting. What exactly is an UP NUMA machine? Anyway, I'm more than happy to apply a tested patch to fix up SLQB. Nick?
Quoted 1 line
Interesting. What exactly is an UP NUMA machine?
UP + NUMA is a special case of memory-only nodes. There are some (crazy?) customers with problems that require very large amounts of memory, but not very much cpu horse power. They buy large multi-node systems and populate all the nodes with as much memory as they can afford, but most nodes get zero cpus.
The limit case with only one cpu is becoming rare (since multi-core gives you more than one cpu even if you only fill a single cpu socket).
N.B. Note that memory only nodes mean that it is a poor idea for the "free" code to just place returned remote memory on a queue for the node that it belongs to, and rely on a "local" cpu processing that queue ... there may not be any local cpus to do so.
Quoted 2 lines
Anyway, I'm more than
happy to apply a tested patch to fix up SLQB. Nick?
I'm trying to check whether http://lkml.org/lkml/2009/4/20/30
fixes things. It certainly solves the complilation problem, but I'm running into apparently unrelated issues trying to boot linux-next kernels.
-Tony
Hi Tony,
Quoted 8 lines · Tony
Quoted 1 line
Interesting. What exactly is an UP NUMA machine?
UP + NUMA is a special case of memory-only nodes. There are some (crazy?) customers with problems that require very large amounts of memory, but not very much cpu horse power. They buy large multi-node systems and populate all the nodes with as much memory as they can afford, but most nodes get zero cpus.
Oh, cool. Thanks for the explanation!
Quoted 7 lines · Tony
Quoted 2 lines
Anyway, I'm more than
happy to apply a tested patch to fix up SLQB. Nick?I'm trying to check whether http://lkml.org/lkml/2009/4/20/30
fixes things. It certainly solves the complilation problem, but I'm running into apparently unrelated issues trying to boot linux-next kernels.
Great! You can try it on top of the "topic/slqb/core" branch of
git://git.kernel.org/pub/scm/linux/kernel/git/penberg/slab-2.6.git
if you want. It's basically plain 2.6.30-rc1 plus SLQB.
One minor nit: the patch should define an empty static inline of claim_remote_free_list() for the !SMP case. I can fix it at my end before merging, though, if necessary.
Pekka
Luck, Tony
Acked-byQuoted 4 lines
Quoted 4 lines
I'm trying to check whether http://lkml.org/lkml/2009/4/20/30
fixes things. It certainly solves the complilation problem, but I'm running into apparently unrelated issues trying to boot linux-next kernels.
My unrelated issue was the new default for CONFIG_SYSFS_DEPRECATED (since my test machine does have an old distribution, this change upset my old udev tools and I ended up with no /dev/root :-( )
Turning that back on, the CONFIG_SLQB=y version booted with no obvious problems.
Quoted 5 lines
Great! You can try it on top of the "topic/slqb/core" branch of
git://git.kernel.org/pub/scm/linux/kernel/git/penberg/slab-2.6.gitif you want. It's basically plain 2.6.30-rc1 plus SLQB.
Since I got linux-next up & running, I didn't need to try this.
Quoted 3 lines
One minor nit: the patch should define an empty static inline of
claim_remote_free_list()for the !SMP case. I can fix it at my end before merging, though, if necessary.
Agreed. It would be better to have an empty static inline than adding the noisy #ifdef SMP around every call to claim_remote_free_list() ... in fact some such #ifdef can be removed.
You could tag such a modified patch (attached) as:
-Tony
mm/slqb.c +11 -8
@@ -1224,6 +1224,11 @@ static void claim_remote_free_list(struct kmem_cache *s,slqb_stat_inc (l,CLAIM_REMOTE_LIST ); slqb_stat_add(l, CLAIM_REMOTE_LIST_OBJECTS, nr); } +#else +static inline void claim_remote_free_list (struct kmem_cache *s, +struct kmem_cache_list *l) +{ +}#endif /*
@@ -1728,7 +1733,7 @@ static __always_inline void __slab_free(struct kmem_cache *s,flush_free_list (s, l); }else { -#ifdef CONFIG_NUMA +#ifdef CONFIG_SMP /* * Freeing an object that was allocated on a remote node. */
@@ -1937,7 +1942,9 @@ static DEFINE_PER_CPU(struct kmem_cache_node, kmem_cpu_nodes); /* XXX per-nid */#ifdef CONFIG_NUMA static struct kmem_cache kmem_node_cache; +# ifdef CONFIG_SMP static DEFINE_PER_CPU(struct kmem_cache_cpu, kmem_node_cpus); +# endif static DEFINE_PER_CPU(struct kmem_cache_node, kmem_node_nodes); /*XXX per-nid */#endif
@@ -2270,7 +2277,7 @@ static int kmem_cache_open(struct kmem_cache *s, error_nodes:free_kmem_cache_nodes (s); error_node_array: -#ifdef CONFIG_NUMA +#if defined(CONFIG_NUMA ) && defined(CONFIG_SMP )kmem_cache_dyn_array_free (s->node_slab); error_cpu_array:#endif
@@ -2370,9 +2377,7 @@ void kmem_cache_destroy(struct kmem_cache *s)struct kmem_cache_cpu *c =get_cpu_slab (s, cpu);struct kmem_cache_list *l = &c->list; -#ifdef CONFIG_SMP claim_remote_free_list (s, l); -#endif flush_free_list_all (s, l);WARN_ON (l->freelist.nr);
@@ -2595,9 +2600,7 @@ static void kmem_cache_trim_percpu(void *arg)struct kmem_cache_cpu *c =get_cpu_slab (s, cpu);struct kmem_cache_list *l = &c->list; -#ifdef CONFIG_SMP claim_remote_free_list (s, l); -#endif flush_free_list (s, l);#ifdef CONFIG_SMP flush_remote_free_cache (s, c);
@@ -2881,11 +2884,11 @@ void __init kmem_cache_init(void) n = &per_cpu (kmem_cache_nodes, i);init_kmem_cache_node (&kmem_cache_cache, n); kmem_cache_cache.node_slab[i] = n; - +#ifdef CONFIG_SMP n = &per_cpu (kmem_cpu_nodes, i);init_kmem_cache_node (&kmem_cpu_cache, n); kmem_cpu_cache.node_slab[i] = n; - +#endif n = &per_cpu (kmem_node_nodes, i);init_kmem_cache_node (&kmem_node_cache, n); kmem_node_cache.node_slab[i] = n;
Pekka J Enberg
Acked-by Tony LuckHi Tony,
Quoted 4 lines · Tony
Quoted 3 lines
One minor nit: the patch should define an empty static inline of
claim_remote_free_list()for the !SMP case. I can fix it at my end before merging, though, if necessary.
Quoted 8 lines
Agreed. It would be better to have an empty static inline than adding the noisy #ifdef SMP around every call to
claim_remote_free_list()... in fact some such #ifdef can be removed.You could tag such a modified patch (attached) as:
Acked-by Tony Luck <[email protected]>
Thanks for the help! I went and merged the following patch and I hope I got all the patch attributions right. Paul, does this work for you as well?
Pekka
Quoted 1 line
From
d46f661ed791312ba008f862a601179c5c9f1e9cMon Sep 17 00:00:00 2001
From: Nobuhiro Iwamatsu <[email protected]>
Date: Wed, 22 Apr 2009 09:50:15 +0300
Subject: [PATCH] SLQB: Fix UP + NUMA build
This patch fixes the following build breakage which happens when CONFIG_NUMA is enabled but CONFIG_SMP is disabled:
CC mm/slqb.o
mm/slqb.c: In function '__slab_free':
mm/slqb.c:1735: error: implicit declaration of function 'slab_free_to_remote' mm/slqb.c: In function 'kmem_cache_open':
mm/slqb.c:2274: error: implicit declaration of function 'kmem_cache_dyn_array_free' mm/slqb.c:2275: warning: label 'error_cpu_array' defined but not used mm/slqb.c: In function 'kmem_cache_destroy':
mm/slqb.c:2395: error: implicit declaration of function 'claim_remote_free_list' mm/slqb.c: In function 'kmem_cache_init':
mm/slqb.c:2885: error: 'per_cpu__kmem_cpu_nodes' undeclared (first use in this function) mm/slqb.c:2885: error: (Each undeclared identifier is reported only once mm/slqb.c:2885: error: for each function it appears in.)
mm/slqb.c:2886: error: 'kmem_cpu_cache' undeclared (first use in this function) make[1]: *** [mm/slqb.o] Error 1
make: *** [mm] Error 2
As x86 Kconfig doesn't even allow this combination, one is tempted to think it's an architecture Kconfig bug. But as it turns out, it's a perfecly valid configuration. Tony Luck explains:
UP + NUMA is a special case of memory-only nodes. There are some (crazy?) customers with problems that require very large amounts of memory, but not very much cpu horse power. They buy large multi-node systems and populate all the nodes with as much memory as they can afford, but most nodes get zero cpus.
So lets fix that up.
--- mm/slqb.c | 19 +++++++++++-------- 1 files changed, 11 insertions(+), 8 deletions(-)
mm/slqb.c +11 -8
@@ -1224,6 +1224,11 @@ static void claim_remote_free_list(struct kmem_cache *s,slqb_stat_inc (l,CLAIM_REMOTE_LIST ); slqb_stat_add(l, CLAIM_REMOTE_LIST_OBJECTS, nr); } +#else +static inline void claim_remote_free_list (struct kmem_cache *s, +struct kmem_cache_list *l) +{ +}#endif /*
@@ -1728,7 +1733,7 @@ static __always_inline void __slab_free(struct kmem_cache *s,flush_free_list (s, l); }else { -#ifdef CONFIG_NUMA +#ifdef CONFIG_SMP /* * Freeing an object that was allocated on a remote node. */
@@ -1937,7 +1942,9 @@ static DEFINE_PER_CPU(struct kmem_cache_node, kmem_cpu_nodes); /* XXX per-nid */#ifdef CONFIG_NUMA static struct kmem_cache kmem_node_cache; +#ifdef CONFIG_SMP static DEFINE_PER_CPU(struct kmem_cache_cpu, kmem_node_cpus); +#endif static DEFINE_PER_CPU(struct kmem_cache_node, kmem_node_nodes); /*XXX per-nid */#endif
@@ -2270,7 +2277,7 @@ static int kmem_cache_open(struct kmem_cache *s, error_nodes:free_kmem_cache_nodes (s); error_node_array: -#ifdef CONFIG_NUMA +#if defined(CONFIG_NUMA ) && defined(CONFIG_SMP )kmem_cache_dyn_array_free (s->node_slab); error_cpu_array:#endif
@@ -2370,9 +2377,7 @@ void kmem_cache_destroy(struct kmem_cache *s)struct kmem_cache_cpu *c =get_cpu_slab (s, cpu);struct kmem_cache_list *l = &c->list; -#ifdef CONFIG_SMP claim_remote_free_list (s, l); -#endif flush_free_list_all (s, l);WARN_ON (l->freelist.nr);
@@ -2595,9 +2600,7 @@ static void kmem_cache_trim_percpu(void *arg)struct kmem_cache_cpu *c =get_cpu_slab (s, cpu);struct kmem_cache_list *l = &c->list; -#ifdef CONFIG_SMP claim_remote_free_list (s, l); -#endif flush_free_list (s, l);#ifdef CONFIG_SMP flush_remote_free_cache (s, c);
@@ -2881,11 +2884,11 @@ void __init kmem_cache_init(void) n = &per_cpu (kmem_cache_nodes, i);init_kmem_cache_node (&kmem_cache_cache, n); kmem_cache_cache.node_slab[i] = n; - +#ifdef CONFIG_SMP n = &per_cpu (kmem_cpu_nodes, i);init_kmem_cache_node (&kmem_cpu_cache, n); kmem_cpu_cache.node_slab[i] = n; - +#endif n = &per_cpu (kmem_node_nodes, i);init_kmem_cache_node (&kmem_node_cache, n); kmem_node_cache.node_slab[i] = n;
Signature · 1 line
1.5.6.3
Paul Mundt
Acked-byQuoted 10 lines · Pekka J Enberg
Quoted 9 lines · Tony
Quoted 3 lines
One minor nit: the patch should define an empty static inline of
claim_remote_free_list()for the !SMP case. I can fix it at my end before merging, though, if necessary.Agreed. It would be better to have an empty static inline than adding the noisy #ifdef SMP around every call to
claim_remote_free_list()... in fact some such #ifdef can be removed.
Quoted 7 lines
Quoted 3 lines
You could tag such a modified patch (attached) as:
Acked-by Tony Luck <[email protected]>Thanks for the help! I went and merged the following patch and I hope I got all the patch attributions right. Paul, does this work for you as well?
Yup, works fine for me, thanks for taking care of this.