Skip to contentSkip to conversation

Network stack timer hacks for power saving

PATCH New [email protected] 2 participants 4 messages May 19, 2009

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

Hi,

I have been looking at network stack timer optimization for power saving in embedded ARM environment, basically trying to avoid as many wakeups as possible. I have changed several timers in the network stack into deferred ones, i.e. they do not wake up the device from low power modes but instead they are deferred until next wakeup from some other source, like another (non-deferred) timer or some I/O. Attached a patch about the changes I've done, is something like this safe to do?

-Tero

From 92856356359fd3dd4e859119676a5fa7aee3ba79 Mon Sep 17 00:00:00 2001
From: Tero Kristo <[email protected]>
Date: Wed, 13 May 2009 15:39:00 +0300
Subject: [PATCH] Network stack timer optimizations for power saving

---
 net/core/flow.c          |    1 +
 net/core/neighbour.c     |    1 +
 net/ipv4/inet_fragment.c |    1 +
 net/ipv4/inetpeer.c      |    1 +
 net/ipv4/route.c         |    1 +
 5 files changed, 5 insertions(+), 0 deletions(-)
net/core/flow.c +1 -0
index 5cf8105..44d963c 100644
@@ -352,6 +352,7 @@ static int __init flow_cache_init(void)
 	flow_hwm = 4 * flow_hash_size;
 
 	setup_timer(&flow_hash_rnd_timer, flow_cache_new_hashrnd, 0);
+	init_timer_deferrable(&flow_hash_rnd_timer);
 	flow_hash_rnd_timer.expires = jiffies + FLOW_HASH_RND_PERIOD;
 	add_timer(&flow_hash_rnd_timer);
 
net/core/neighbour.c +1 -0
index 1dc728b..c9217d7 100644
@@ -1445,6 +1445,7 @@ void neigh_table_init_no_netlink(struct neigh_table *tbl)
 
 	rwlock_init(&tbl->lock);
 	setup_timer(&tbl->gc_timer, neigh_periodic_timer, (unsigned long)tbl);
+	init_timer_deferrable(&tbl->gc_timer);
 	tbl->gc_timer.expires  = now + 1;
 	add_timer(&tbl->gc_timer);
 
net/ipv4/inet_fragment.c +1 -0
index 6c52e08..91a099f 100644
@@ -64,6 +64,7 @@ void inet_frags_init(struct inet_frags *f)
 
 	setup_timer(&f->secret_timer, inet_frag_secret_rebuild,
 			(unsigned long)f);
+	init_timer_deferrable(&f->secret_timer);
 	f->secret_timer.expires = jiffies + f->secret_interval;
 	add_timer(&f->secret_timer);
 }
net/ipv4/inetpeer.c +1 -0
index a456cee..19eea7c 100644
@@ -125,6 +125,7 @@ void __init inet_initpeers(void)
 	/* All the timers, started at system startup tend
 	   to synchronize. Perturb it a bit.
 	 */
+	init_timer_deferrable(&peer_periodic_timer);
 	peer_periodic_timer.expires = jiffies
 		+ net_random() % inet_peer_gc_maxtime
 		+ inet_peer_gc_maxtime;
net/ipv4/route.c +1 -0
index 2ea6dcc..792bd7e 100644
@@ -3288,6 +3288,7 @@ int __init ip_rt_init(void)
 	/* All the timers, started at system startup tend
 	   to synchronize. Perturb it a bit.
 	 */
+	INIT_DELAYED_WORK_DEFERRABLE(&expires_work, rt_worker_func);
 	schedule_delayed_work(&expires_work,
 		net_random() % ip_rt_gc_interval + ip_rt_gc_interval);
 
Signature · 1 line
1.5.4.3
#1 lore raw
Quoted 11 lines · [email protected] a écrit :

Hi,

I have been looking at network stack timer optimization for power saving in embedded ARM environment, basically trying to avoid as many wakeups as possible. I have changed several timers in the network stack into deferred ones, i.e. they do not wake up the device from low power modes but instead they are deferred until next wakeup from some other source, like another (non-deferred) timer or some I/O. Attached a patch about the changes I've done, is something like this safe to do?

Quoted 1 line

-Tero

Hi Tero

When tcp communications are active, we setup a timer for *every* frame we receive or we send. These timers wont be deferrable anyway.

delaying one wakeup every 60 seconds (if I take your net/ipv4/route.c change) wont change that much power savings, or did I missed something ?

On big routers, we need to set ip_rt_gc_interval from 60 seconds to one second, in order to perform an effective garbage collection.

So, if we use a deferred timer and :

schedule_delayed_work(&expires_work, HZ);

How many times worker will be started every minute ?

#2 lore raw
Quoted 20 lines

-----Original Message-----
From: ext Eric Dumazet [mailto:[email protected]]
Sent: 19 May, 2009 12:04
To: Kristo Tero (Nokia-D/Tampere)
Cc: [email protected]
Subject: Re: Network stack timer hacks for power saving

Quoted 4 lines · [email protected] a écrit :

Hi,

I have been looking at network stack timer optimization for power saving in embedded ARM environment, basically trying to

avoid as many

Quoted 2 lines

wakeups as possible. I have changed several timers in the network stack into deferred ones, i.e. they do not wake up the

device from low

Quoted 1 line

power modes but instead they are deferred until next wakeup

from some

Quoted 9 lines
Quoted 1 line

other source, like another (non-deferred) timer or some I/O.

Attached

Quoted 4 lines

a patch about the changes I've done, is something like this safe to do?

-Tero

Hi Tero

Hi Eric,

Thanks for comments. Also, I think I need to make one clarification here, I am not proposing these changes to be made into linux network stack, at least not as such because I am aware that these might cause some problems on some systems. If something like this would be merged, I think this should be added behind a kernel config option.

Quoted 8 lines

When tcp communications are active, we setup a timer for *every* frame we receive or we send. These timers wont be deferrable anyway.

delaying one wakeup every 60 seconds (if I take your net/ipv4/route.c change) wont change that much power savings, or did I missed something ?

No, this won't change it that much, but it still affects a bit. Also, if everybody thinks that a 1 minute timer does not matter too much, we soon have 60 timers expiring once a minute and can basically cause a wakeup from low power modes every second in the worst case.

Quoted 10 lines

On big routers, we need to set ip_rt_gc_interval from 60 seconds to one second, in order to perform an effective garbage collection.

So, if we use a deferred timer and :

schedule_delayed_work(&expires_work, HZ);

How many times worker will be started every minute ?

I think big routers do not enter any low power states, due to heavy network traffic keeping them busy. Even if they do enter low power mode, I guess network HW will basically wake them up rather quickly causing the delayed work to be executed approximately around the time (or probably exactly at the time) it was scheduled. I might be wrong here, as I do not really know too much about network router power management.

Here is some data I grabbed from /proc/timer_stats before doing these changes (just added the network stack stuff here and calculated expiry rates.) I already changed most of the timers to deferrable in this sample, and also made a hack to show workqueues properly. Device was basically just idling during this measurement, not doing any frequent net communication.

Timer Stats Version: v0.2
Sample period: 59672.362 s

7455D,     1 swapper          neigh_table_init_no_netlink (neigh_periodic_timer) [neighbour.c, rate 1 per 8s]
  995,     0 workqueue        queue_delayed_work (rt_worker_func) [route.c, rate 1 per min]
 498D,     1 swapper          inet_initpeers (peer_check_expire) [inetpeer.c, rate 1 per 2 min]
  99D,     1 swapper          flow_cache_init (flow_cache_new_hashrnd) [flow.c, rate 1 per 10 min]
  99D,     1 swapper          inet_frags_init (inet_frag_secret_rebuild) [inet_fragment.c, rate 1 per 10 min]

-Tero

#3 lore raw

PATCH 1/1 net: use a deferred timer in rt_check_expire

Quoted 18 lines · [email protected] a écrit :
Quoted 18 lines

-----Original Message-----
From: ext Eric Dumazet [mailto:[email protected]]
Sent: 19 May, 2009 12:04
To: Kristo Tero (Nokia-D/Tampere)
Cc: [email protected]
Subject: Re: Network stack timer hacks for power saving

Quoted 4 lines · [email protected] a écrit :

Hi,

I have been looking at network stack timer optimization for power saving in embedded ARM environment, basically trying to

avoid as many

Quoted 2 lines

wakeups as possible. I have changed several timers in the network stack into deferred ones, i.e. they do not wake up the

device from low

Quoted 8 lines
Quoted 8 lines
Quoted 1 line

power modes but instead they are deferred until next wakeup

from some

Quoted 1 line

other source, like another (non-deferred) timer or some I/O.

Attached

Quoted 4 lines

a patch about the changes I've done, is something like this safe to do?

-Tero

Here is the patch I cooked and tested on a machine where ip_rt_gc_interval is set to minimal value (1 second), where equilibrium depends on garbage collection done in time.

I found that delayed timers could be *really* delayed so I think we must take into account the elapsed time (in jiffies) between two rt_check_expire() calls, to "guarantee" a full scan of rt cache in a ip_rt_gc_timeout period.

Not for inclusion, as undergoing work is happening in this function for a bug correction. I'll redo the patch later once stabilized.

[PATCH] net: use a deferred timer in rt_check_expire

For the sake of power saver lovers, use a deferrable timer to fire rt_check_expire()

As some big routers cache equilibrium depends on garbage collection done in time, we take into account elapsed time between two rt_check_expire() invocations to adjust the amount of slots we have to check.

Based on an initial idea and patch from Tero Kristo

---
 net/ipv4/route.c |   11 ++++++++---
 1 files changed, 8 insertions(+), 3 deletions(-)
net/ipv4/route.c +8 -3
index c4c60e9..b2c6793 100644
@@ -131,8 +131,8 @@ static int ip_rt_min_advmss __read_mostly	= 256;
 static int ip_rt_secret_interval __read_mostly	= 10 * 60 * HZ;
 static int rt_chain_length_max __read_mostly	= 20;
 
-static void rt_worker_func(struct work_struct *work);
-static DECLARE_DELAYED_WORK(expires_work, rt_worker_func);
+static struct delayed_work expires_work;
+static unsigned long expires_ljiffies;
 
 /*
  *	Interface to generic destination cache.
@@ -787,9 +787,12 @@ static void rt_check_expire(void)
 	struct rtable *rth, **rthp;
 	unsigned long length = 0, samples = 0;
 	unsigned long sum = 0, sum2 = 0;
+	unsigned long delta;
 	u64 mult;
 
-	mult = ((u64)ip_rt_gc_interval) << rt_hash_log;
+	delta = jiffies - expires_ljiffies;
+	expires_ljiffies = jiffies;
+	mult = ((u64)delta) << rt_hash_log;
 	if (ip_rt_gc_timeout > 1)
 		do_div(mult, ip_rt_gc_timeout);
 	goal = (unsigned int)mult;
@@ -3410,6 +3413,8 @@ int __init ip_rt_init(void)
 	/* All the timers, started at system startup tend
 	   to synchronize. Perturb it a bit.
 	 */
+	INIT_DELAYED_WORK_DEFERRABLE(&expires_work, rt_worker_func);
+	expires_ljiffies = jiffies;
 	schedule_delayed_work(&expires_work,
 		net_random() % ip_rt_gc_interval + ip_rt_gc_interval);