Network stack timer hacks for power saving
Subsystems (3)
Hi,
I have been looking at network stack timer optimization for power saving in embedded ARM environment, basically trying to avoid as many wakeups as possible. I have changed several timers in the network stack into deferred ones, i.e. they do not wake up the device from low power modes but instead they are deferred until next wakeup from some other source, like another (non-deferred) timer or some I/O. Attached a patch about the changes I've done, is something like this safe to do?
-Tero
From 92856356359fd3dd4e859119676a5fa7aee3ba79 Mon Sep 17 00:00:00 2001
From: Tero Kristo <[email protected]>
Date: Wed, 13 May 2009 15:39:00 +0300
Subject: [PATCH] Network stack timer optimizations for power saving
--- net/core/flow.c | 1 + net/core/neighbour.c | 1 + net/ipv4/inet_fragment.c | 1 + net/ipv4/inetpeer.c | 1 + net/ipv4/route.c | 1 + 5 files changed, 5 insertions(+), 0 deletions(-)
net/core/flow.c +1 -0
@@ -352,6 +352,7 @@ static int __init flow_cache_init(void) flow_hwm =4 * flow_hash_size;setup_timer (&flow_hash_rnd_timer, flow_cache_new_hashrnd,0 ); +init_timer_deferrable (&flow_hash_rnd_timer); flow_hash_rnd_timer.expires = jiffies +FLOW_HASH_RND_PERIOD ;add_timer (&flow_hash_rnd_timer);
net/core/neighbour.c +1 -0
@@ -1445,6 +1445,7 @@ void neigh_table_init_no_netlink(struct neigh_table *tbl)rwlock_init (&tbl->lock);setup_timer (&tbl->gc_timer, neigh_periodic_timer, (unsigned long )tbl); +init_timer_deferrable (&tbl->gc_timer); tbl->gc_timer.expires = now +1 ;add_timer (&tbl->gc_timer);
net/ipv4/inet_fragment.c +1 -0
@@ -64,6 +64,7 @@ void inet_frags_init(struct inet_frags *f)setup_timer (&f->secret_timer, inet_frag_secret_rebuild, (unsigned long )f); +init_timer_deferrable (&f->secret_timer); f->secret_timer.expires = jiffies + f->secret_interval;add_timer (&f->secret_timer); }
net/ipv4/inetpeer.c +1 -0
@@ -125,6 +125,7 @@ void __init inet_initpeers(void)/* All the timers, started at system startup tend to synchronize. Perturb it a bit. */ +init_timer_deferrable (&peer_periodic_timer); peer_periodic_timer.expires = jiffies +net_random () % inet_peer_gc_maxtime + inet_peer_gc_maxtime;
net/ipv4/route.c +1 -0
@@ -3288,6 +3288,7 @@ int __init ip_rt_init(void)/* All the timers, started at system startup tend to synchronize. Perturb it a bit. */ +INIT_DELAYED_WORK_DEFERRABLE (&expires_work, rt_worker_func);schedule_delayed_work (&expires_work,net_random () % ip_rt_gc_interval + ip_rt_gc_interval);
Signature · 1 line
1.5.4.3
Quoted 11 lines · [email protected] a écrit :
Hi,
I have been looking at network stack timer optimization for power saving in embedded ARM environment, basically trying to avoid as many wakeups as possible. I have changed several timers in the network stack into deferred ones, i.e. they do not wake up the device from low power modes but instead they are deferred until next wakeup from some other source, like another (non-deferred) timer or some I/O. Attached a patch about the changes I've done, is something like this safe to do?
Quoted 1 line
-Tero
Hi Tero
When tcp communications are active, we setup a timer for *every* frame we receive or we send. These timers wont be deferrable anyway.
delaying one wakeup every 60 seconds (if I take your net/ipv4/route.c change) wont change that much power savings, or did I missed something ?
On big routers, we need to set ip_rt_gc_interval from 60 seconds to one second, in order to perform an effective garbage collection.
So, if we use a deferred timer and :
schedule_delayed_work(&expires_work, HZ);
How many times worker will be started every minute ?
Quoted 20 lines
-----Original Message-----
From: ext Eric Dumazet [mailto:[email protected]]
Sent: 19 May, 2009 12:04
To: Kristo Tero (Nokia-D/Tampere)
Cc: [email protected]
Subject: Re: Network stack timer hacks for power savingQuoted 4 lines · [email protected] a écrit :
Hi,
I have been looking at network stack timer optimization for power saving in embedded ARM environment, basically trying to
avoid as many
Quoted 2 lines
wakeups as possible. I have changed several timers in the network stack into deferred ones, i.e. they do not wake up the
device from low
Quoted 1 line
power modes but instead they are deferred until next wakeup
from some
Quoted 9 lines
Quoted 1 line
other source, like another (non-deferred) timer or some I/O.
Attached
Quoted 4 lines
a patch about the changes I've done, is something like this safe to do?
-Tero
Hi Tero
Hi Eric,
Thanks for comments. Also, I think I need to make one clarification here, I am not proposing these changes to be made into linux network stack, at least not as such because I am aware that these might cause some problems on some systems. If something like this would be merged, I think this should be added behind a kernel config option.
Quoted 8 lines
When tcp communications are active, we setup a timer for *every* frame we receive or we send. These timers wont be deferrable anyway.
delaying one wakeup every 60 seconds (if I take your
net/ipv4/route.cchange) wont change that much power savings, or did I missed something ?
No, this won't change it that much, but it still affects a bit. Also, if everybody thinks that a 1 minute timer does not matter too much, we soon have 60 timers expiring once a minute and can basically cause a wakeup from low power modes every second in the worst case.
Quoted 10 lines
On big routers, we need to set
ip_rt_gc_intervalfrom 60 seconds to one second, in order to perform an effective garbage collection.So, if we use a deferred timer and :
schedule_delayed_work(&expires_work, HZ);How many times worker will be started every minute ?
I think big routers do not enter any low power states, due to heavy network traffic keeping them busy. Even if they do enter low power mode, I guess network HW will basically wake them up rather quickly causing the delayed work to be executed approximately around the time (or probably exactly at the time) it was scheduled. I might be wrong here, as I do not really know too much about network router power management.
Here is some data I grabbed from /proc/timer_stats before doing these changes (just added the network stack stuff here and calculated expiry rates.) I already changed most of the timers to deferrable in this sample, and also made a hack to show workqueues properly. Device was basically just idling during this measurement, not doing any frequent net communication.
Timer Stats Version: v0.2
Sample period: 59672.362 s
7455D, 1 swapper neigh_table_init_no_netlink (neigh_periodic_timer) [neighbour.c, rate 1 per 8s] 995, 0 workqueue queue_delayed_work (rt_worker_func) [route.c, rate 1 per min] 498D, 1 swapper inet_initpeers (peer_check_expire) [inetpeer.c, rate 1 per 2 min] 99D, 1 swapper flow_cache_init (flow_cache_new_hashrnd) [flow.c, rate 1 per 10 min] 99D, 1 swapper inet_frags_init (inet_frag_secret_rebuild) [inet_fragment.c, rate 1 per 10 min]
-Tero
PATCH 1/1 net: use a deferred timer in rt_check_expire
Quoted 18 lines · [email protected] a écrit :
Quoted 18 lines
-----Original Message-----
From: ext Eric Dumazet [mailto:[email protected]]
Sent: 19 May, 2009 12:04
To: Kristo Tero (Nokia-D/Tampere)
Cc: [email protected]
Subject: Re: Network stack timer hacks for power savingQuoted 4 lines · [email protected] a écrit :
Hi,
I have been looking at network stack timer optimization for power saving in embedded ARM environment, basically trying to
avoid as many
Quoted 2 lines
wakeups as possible. I have changed several timers in the network stack into deferred ones, i.e. they do not wake up the
device from low
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
power modes but instead they are deferred until next wakeup
from some
Quoted 1 line
other source, like another (non-deferred) timer or some I/O.
Attached
Quoted 4 lines
a patch about the changes I've done, is something like this safe to do?
-Tero
Here is the patch I cooked and tested on a machine where ip_rt_gc_interval is set to minimal value (1 second), where equilibrium depends on garbage collection done in time.
I found that delayed timers could be *really* delayed so I think we must take into account the elapsed time (in jiffies) between two rt_check_expire() calls, to "guarantee" a full scan of rt cache in a ip_rt_gc_timeout period.
Not for inclusion, as undergoing work is happening in this function for a bug correction. I'll redo the patch later once stabilized.
[PATCH] net: use a deferred timer in rt_check_expire
For the sake of power saver lovers, use a deferrable timer to fire rt_check_expire()
As some big routers cache equilibrium depends on garbage collection done in time, we take into account elapsed time between two rt_check_expire() invocations to adjust the amount of slots we have to check.
Based on an initial idea and patch from Tero Kristo
--- net/ipv4/route.c | 11 ++++++++--- 1 files changed, 8 insertions(+), 3 deletions(-)
net/ipv4/route.c +8 -3
@@ -131,8 +131,8 @@ static int ip_rt_min_advmss __read_mostly = 256;static int ip_rt_secret_interval__read_mostly =10 *60 *HZ ;static int rt_chain_length_max__read_mostly =20 ; -static void rt_worker_func (struct work_struct *work); -static DECLARE_DELAYED_WORK(expires_work, rt_worker_func); +static struct delayed_work expires_work; +static unsigned long expires_ljiffies;/* * Interface to generic destination cache.
@@ -787,9 +787,12 @@ static void rt_check_expire(void)struct rtable *rth, **rthp;unsigned long length =0 , samples =0 ;unsigned long sum =0 , sum2 =0 ; +unsigned long delta;u64 mult; - mult = ((u64 )ip_rt_gc_interval) << rt_hash_log; + delta = jiffies - expires_ljiffies; + expires_ljiffies = jiffies; + mult = ((u64 )delta) << rt_hash_log;if (ip_rt_gc_timeout >1 )do_div (mult, ip_rt_gc_timeout); goal = (unsigned int )mult;
@@ -3410,6 +3413,8 @@ int __init ip_rt_init(void)/* All the timers, started at system startup tend to synchronize. Perturb it a bit. */ +INIT_DELAYED_WORK_DEFERRABLE (&expires_work, rt_worker_func); + expires_ljiffies = jiffies;schedule_delayed_work (&expires_work,net_random () % ip_rt_gc_interval + ip_rt_gc_interval);