NET : convert ip_rt_acct to per_cpu variables
Subsystems (3)
PATCH 1/1 NET : convert ip_rt_acct to per_cpu variables
Hi David
I am chasing NR_CPUS syndrom for your NR_CPUS=4096 machines :)
(Feel free to give me an account on one of them just for fun)
I made this patch but have no idea if it actually works (only compile tested) because I dont know how to use this CONFIG_NET_CLS_ROUTE stuff
Thank you
[PATCH] NET : NET_CLS_ROUTE : convert ip_rt_acct to per_cpu variables
ip_rt_acct needs 4096 bytes per cpu to perform some accounting. It is actually allocated as a single huge array [4096*NR_CPUS] (rounded up to a power of two)
Converting it to a per cpu variable is wanted to :
- Save space on machines were num_possible_cpus() < NR_CPUS
- Better NUMA placement (each cpu gets memory on its node)
include/net/route.h | 2 +- net/ipv4/ip_input.c | 2 +- net/ipv4/route.c | 17 ++--------------- 3 files changed, 4 insertions(+), 17 deletions(-)
include/net/route.h +1 -1
@@ -103,7 +103,7 @@ struct rt_cache_statunsigned int out_hlist_search; }; -extern struct ip_rt_acct *ip_rt_acct; +DECLARE_PER_CPU(struct ip_rt_acct[256], ip_rt_acct);struct in_device ;extern int ip_rt_init (void );
net/ipv4/ip_input.c +1 -1
@@ -347,7 +347,7 @@ static int ip_rcv_finish(struct sk_buff *skb)#ifdef CONFIG_NET_CLS_ROUTE if (unlikely (skb->dst->tclassid)) { -struct ip_rt_acct *st = ip_rt_acct +256 *smp_processor_id (); +struct ip_rt_acct *st =__get_cpu_var (ip_rt_acct);u32 idx = skb->dst->tclassid; st[idx&0xFF ].o_packets++; st[idx&0xFF ].o_bytes+=skb->len;
net/ipv4/route.c +2 -15
@@ -2855,12 +2855,12 @@ ctl_table ipv4_route_table[] = {#endif #ifdef CONFIG_NET_CLS_ROUTE -struct ip_rt_acct *ip_rt_acct; +DEFINE_PER_CPU(struct ip_rt_acct [256], ip_rt_acct);/* This code sucks. But you should have seen it before! --RR */ /* IP route accounting ptr for this logical cpu number. */ -#define IP_RT_ACCT_CPU (i) (ip_rt_acct + i * 256) +#define IP_RT_ACCT_CPU (i) (per_cpu(ip_rt_acct, i))#ifdef CONFIG_PROC_FS static int ip_rt_acct_read(char *buffer, char **start, off_t offset,
@@ -2923,19 +2923,6 @@ int __init ip_rt_init(void) rt_hash_rnd = (int ) ((num_physpages ^ (num_physpages>>8 )) ^ (jiffies ^ (jiffies >>7 ))); -#ifdef CONFIG_NET_CLS_ROUTE - { -int order; -for (order =0 ; - (PAGE_SIZE << order) <256 *sizeof (struct ip_rt_acct ) *NR_CPUS ; order++) -/* NOTHING */ ; - ip_rt_acct = (struct ip_rt_acct *)__get_free_pages (GFP_KERNEL , order); -if (!ip_rt_acct) -panic ("IP: failed to allocate ip_rt_acct\n" ); -memset (ip_rt_acct,0 ,PAGE_SIZE << order); - } -#endif - ipv4_dst_ops.kmem_cachep =kmem_cache_create ("ip_dst_cache" ,sizeof (struct rtable ),0 ,SLAB_HWCACHE_ALIGN |SLAB_PANIC ,NULL );
David Miller
maintainerFrom: Eric Dumazet <[email protected]>
Date: Fri, 16 Nov 2007 09:59:03 +0100
Quoted 11 lines
[PATCH] NET :
NET_CLS_ROUTE: convertip_rt_accttoper_cpuvariables
ip_rt_acctneeds 4096 bytes per cpu to perform some accounting. It is actually allocated as a single huge array [4096*NR_CPUS] (rounded up to a power of two)Converting it to a per cpu variable is wanted to :
- Save space on machines werenum_possible_cpus()<NR_CPUS
- Better NUMA placement (each cpu gets memory on its node)Signed-off-by Eric Dumazet <[email protected]>
This is better in one sense but worse in another.
At least the previous code dynamically allocated the thing, now at least one copy is taking up core kernel text image space.
I think it's an alloc_percpu() candidate, what do you think?
Quoted 13 lines · On Fri, 16 Nov 2007 01:12:43 -0800 (PST) David Miller <[email protected]>…
From: Eric Dumazet <[email protected]>
Date: Fri, 16 Nov 2007 09:59:03 +0100Quoted 10 lines
[PATCH] NET :
NET_CLS_ROUTE: convertip_rt_accttoper_cpuvariables
ip_rt_acctneeds 4096 bytes per cpu to perform some accounting. It is actually allocated as a single huge array [4096*NR_CPUS] (rounded up to a power of two)Converting it to a per cpu variable is wanted to :
- Save space on machines werenum_possible_cpus()<NR_CPUS
- Better NUMA placement (each cpu gets memory on its node)
Quoted 7 lines
Quoted 1 line
Signed-off-by Eric Dumazet <[email protected]>This is better in one sense but worse in another.
At least the previous code dynamically allocated the thing, now at least one copy is taking up core kernel text image space.
Oh I see, you think that adding 4096 null bytes to the static percpu area might be a problem.
Quoted 3 lines
I think it's an
alloc_percpu()candidate, what do you think?
Yes absolutely, I will submit a new version.
Thank you
Quoted 27 lines · On Fri, 16 Nov 2007 10:35:46 +0100 Eric Dumazet <[email protected]> wrote:
Quoted 13 lines · On Fri, 16 Nov 2007 01:12:43 -0800 (PST) David Miller <[email protected]>…
From: Eric Dumazet <[email protected]>
Date: Fri, 16 Nov 2007 09:59:03 +0100Quoted 10 lines
[PATCH] NET :
NET_CLS_ROUTE: convertip_rt_accttoper_cpuvariables
ip_rt_acctneeds 4096 bytes per cpu to perform some accounting. It is actually allocated as a single huge array [4096*NR_CPUS] (rounded up to a power of two)Converting it to a per cpu variable is wanted to :
- Save space on machines werenum_possible_cpus()<NR_CPUS
- Better NUMA placement (each cpu gets memory on its node)Quoted 7 lines
Quoted 1 line
Signed-off-by Eric Dumazet <[email protected]>This is better in one sense but worse in another.
At least the previous code dynamically allocated the thing, now at least one copy is taking up core kernel text image space.
Oh I see, you think that adding 4096 null bytes to the static percpu area might be a problem.
Quoted 1 line
Quoted 6 lines
Quoted 2 lines
I think it's an
alloc_percpu()candidate, what do you think?Yes absolutely, I will submit a new version.
Thank you
[PATCH] NET : NET_CLS_ROUTE : convert ip_rt_acct to per_cpu variables
ip_rt_acct needs 4096 bytes per cpu to perform some accounting. It is actually allocated as a single huge array [4096*NR_CPUS] (rounded up to a power of two)
Converting it to a per cpu variable is wanted to :
- Save space on machines were num_possible_cpus() < NR_CPUS
- Better NUMA placement (each cpu gets memory on its node)
net/ipv4/ip_input.c | 2 +- net/ipv4/route.c | 15 +++------------ 2 files changed, 4 insertions(+), 13 deletions(-)
net/ipv4/ip_input.c +1 -1
@@ -347,7 +347,7 @@ static int ip_rcv_finish(struct sk_buff *skb)#ifdef CONFIG_NET_CLS_ROUTE if (unlikely (skb->dst->tclassid)) { -struct ip_rt_acct *st = ip_rt_acct +256 *smp_processor_id (); +struct ip_rt_acct *st =per_cpu_ptr (ip_rt_acct,smp_processor_id ());u32 idx = skb->dst->tclassid; st[idx&0xFF ].o_packets++; st[idx&0xFF ].o_bytes+=skb->len;
net/ipv4/route.c +3 -12
@@ -2855,12 +2855,10 @@ ctl_table ipv4_route_table[] = {#endif #ifdef CONFIG_NET_CLS_ROUTE -struct ip_rt_acct *ip_rt_acct; - -/* This code sucks. But you should have seen it before! --RR */ +struct ip_rt_acct *ip_rt_acct__read_mostly ;/* IP route accounting ptr for this logical cpu number. */ -#define IP_RT_ACCT_CPU (i) (ip_rt_acct + i * 256) +#define IP_RT_ACCT_CPU (cpu) (per_cpu_ptr(ip_rt_acct, cpu))#ifdef CONFIG_PROC_FS static int ip_rt_acct_read(char *buffer, char **start, off_t offset,
@@ -2920,16 +2918,9 @@ int __init ip_rt_init(void) (jiffies ^ (jiffies >> 7)));#ifdef CONFIG_NET_CLS_ROUTE - { -int order; -for (order =0 ; - (PAGE_SIZE << order) <256 *sizeof (struct ip_rt_acct ) *NR_CPUS ; order++) -/* NOTHING */ ; - ip_rt_acct = (struct ip_rt_acct *)__get_free_pages (GFP_KERNEL , order); + ip_rt_acct =__alloc_percpu (256 *sizeof (struct ip_rt_acct ));if (!ip_rt_acct)panic ("IP: failed to allocate ip_rt_acct\n" ); -memset (ip_rt_acct,0 ,PAGE_SIZE << order); - }#endif ipv4_dst_ops.kmem_cachep =
From: Eric Dumazet <[email protected]>
Date: Fri, 16 Nov 2007 11:58:52 +0100
Quoted 11 lines
[PATCH] NET :
NET_CLS_ROUTE: convertip_rt_accttoper_cpuvariables
ip_rt_acctneeds 4096 bytes per cpu to perform some accounting. It is actually allocated as a single huge array [4096*NR_CPUS] (rounded up to a power of two)Converting it to a per cpu variable is wanted to :
- Save space on machines werenum_possible_cpus()<NR_CPUS
- Better NUMA placement (each cpu gets memory on its node)Signed-off-by Eric Dumazet <[email protected]>
Applied to net-2.6.25, thanks Eric.