net: gen_estimator: Fix gen_kill_estimator() lookups
Subsystems (2)
PATCH 1/1 net: gen_estimator: Fix gen_kill_estimator() lookups
gen_kill_estimator() linear lists lookups are very slow, and e.g. while deleting a large number of HTB classes soft lockups were reported. Here is another try to fix this problem: this time internally, with rbtree, so similarly to Jamal's hashing idea IIRC. (Looking for next hits could be still optimized, but it's really fast as it is.)
---
net/core/gen_estimator.c | 76 ++++++++++++++++++++++++++++++++++------------ 1 files changed, 56 insertions(+), 20 deletions(-)
net/core/gen_estimator.c +56 -20
@@ -31,6 +31,7 @@#include <linux/skbuff.h> #include <linux/rtnetlink.h> #include <linux/init.h> +#include <linux/rbtree.h> #include <net/sock.h> #include <net/gen_stats.h>
@@ -89,6 +90,7 @@ struct gen_estimatoru32 avpps;u32 avbps;struct rcu_head e_rcu; +struct rb_node node; };struct gen_estimator_head
@@ -102,6 +104,9 @@ static struct gen_estimator_head elist[EST_MAX_INTERVAL+1];/* Protects against NULL dereference */ static DEFINE_RWLOCK (est_lock); +/* Protects against soft lockup during large deletion */ +static struct rb_root est_root =RB_ROOT ; +static void est_timer (unsigned long arg) {int idx = (int )arg;
@@ -139,6 +144,45 @@ skip:rcu_read_unlock (); } +static void gen_add_node (struct gen_estimator *est) +{ +struct rb_node **p = &est_root.rb_node, *parent =NULL ; + +while (*p) { +struct gen_estimator *e; + + parent = *p; + e = rb_entry(parent, struct gen_estimator, node); + +if (est->bstats > e->bstats) + p = &parent->rb_right; +else + p = &parent->rb_left; + } +rb_link_node (&est->node, parent, p); +rb_insert_color (&est->node, &est_root); +} + +static struct gen_estimator *gen_find_node (struct gnet_stats_basic *bstats, +struct gnet_stats_rate_est *rate_est) +{ +struct rb_node *p = est_root.rb_node; + +while (p) { +struct gen_estimator *e; + + e = rb_entry(p, struct gen_estimator, node); + +if (bstats > e->bstats) + p = p->rb_right; +else if (bstats < e->bstats || rate_est != e->rate_est) + p = p->rb_left; +else +return e; + } +return NULL ; +} +/** * gen_new_estimator - create a new rate estimator * @bstats: basic statistics
@@ -194,6 +238,8 @@ int gen_new_estimator(struct gnet_stats_basic *bstats,mod_timer (&elist[idx].timer, jiffies + ((HZ /4 ) << idx));list_add_rcu (&est->list, &elist[idx].list); +gen_add_node (est); +return 0 ; }
@@ -209,34 +255,24 @@ static void __gen_kill_estimator(struct rcu_head *head)* @bstats: basic statistics * @rate_est: rate estimator statistics * -* Removes the rate estimator specified by &bstats and &rate_est -* and deletes the timer. +* Removes the rate estimator specified by &bstats and &rate_est. * * NOTE: Called under rtnl_mutex */ void gen_kill_estimator (struct gnet_stats_basic *bstats, -struct gnet_stats_rate_est *rate_est) +struct gnet_stats_rate_est *rate_est) { -int idx; -struct gen_estimator *e, *n; - -for (idx=0 ; idx <=EST_MAX_INTERVAL ; idx++) { - -/* Skip non initialized indexes */ -if (!elist[idx].timer.function) -continue ; +struct gen_estimator *e; -list_for_each_entry_safe (e, n, &elist[idx].list, list) { -if (e->rate_est != rate_est || e->bstats != bstats) -continue ; +while ((e =gen_find_node (bstats, rate_est))) { +rb_erase (&e->node, &est_root); -write_lock_bh (&est_lock); - e->bstats =NULL ; -write_unlock_bh (&est_lock); +write_lock_bh (&est_lock); + e->bstats =NULL ; +write_unlock_bh (&est_lock); -list_del_rcu (&e->list); -call_rcu (&e->e_rcu, __gen_kill_estimator); - } +list_del_rcu (&e->list); +call_rcu (&e->e_rcu, __gen_kill_estimator); } }
Quoted 5 lines · Jarek Poplawski
gen_kill_estimator()linear lists lookups are very slow, and e.g. while deleting a large number of HTB classes soft lockups were reported. Here is another try to fix this problem: this time internally, with rbtree, so similarly to Jamal's hashing idea IIRC. (Looking for next hits could be still optimized, but it's really fast as it is.)
Certainly a big improvement. Compared to hashing i suggested:
the deletion speed is probably equal or better than using a hash. I think a hash would have performed better in the case of addition than the rb tree; but you primarily concerned about deletion, so this good.
cheers,
jamal
Quoted 7 lines · jamal
Quoted 5 lines · Jarek Poplawski
gen_kill_estimator()linear lists lookups are very slow, and e.g. while deleting a large number of HTB classes soft lockups were reported. Here is another try to fix this problem: this time internally, with rbtree, so similarly to Jamal's hashing idea IIRC. (Looking for next hits could be still optimized, but it's really fast as it is.)
Quoted 5 lines
Certainly a big improvement. Compared to hashing i suggested:
the deletion speed is probably equal or better than using a hash. I think a hash would have performed better in the case of addition than the rb tree; but you primarily concerned about deletion, so this good.
I first thought about a hash, but alas Patrick's solution is sched only... Anyway, I din't see too much overhead in memory use, and no diffrence in addition times (without batching).
Quoted 3 lines
Acked-by Jamal Hadi Salim <[email protected]>
Thanks,
Jarek P.
Quoted 3 lines · Jarek Poplawski
I first thought about a hash, but alas Patrick's solution is sched only... Anyway, I din't see too much overhead in memory use, and no diffrence in addition times (without batching).
Showing numbers in a commit for perf improvement IMO is always a good thing.
BTW, I dont think it would make a noticeable difference (batching notwithstanding) in addition or even deletion unless you have quiet a few with the same estimate sampling time loaded.
cheers,
jamal
Quoted 8 lines · jamal
Quoted 3 lines · Jarek Poplawski
I first thought about a hash, but alas Patrick's solution is sched only... Anyway, I din't see too much overhead in memory use, and no diffrence in addition times (without batching).
Showing numbers in a commit for perf improvement IMO is always a good thing.
Sure, but alas I'm not a perf guy...
Quoted 3 lines
BTW, I dont think it would make a noticeable difference (batching notwithstanding) in addition or even deletion unless you have quiet a few with the same estimate sampling time loaded.
My very unprofessional tests gave approximately 319s vs. 0.34s with:
"time tc qdisc del dev lo root" for 65535 htb classes, and as you predicted (and I was surprised) no noticeable difference in addition times with or without batching.
Cheers,
Jarek P.
From: Jarek Poplawski <[email protected]>
Date: Mon, 24 Nov 2008 13:37:24 +0000
Quoted 14 lines · jamal
Quoted 7 lines · Jarek Poplawski
Quoted 5 lines
gen_kill_estimator()linear lists lookups are very slow, and e.g. while deleting a large number of HTB classes soft lockups were reported. Here is another try to fix this problem: this time internally, with rbtree, so similarly to Jamal's hashing idea IIRC. (Looking for next hits could be still optimized, but it's really fast as it is.)Quoted 5 lines
Certainly a big improvement. Compared to hashing i suggested:
the deletion speed is probably equal or better than using a hash. I think a hash would have performed better in the case of addition than the rb tree; but you primarily concerned about deletion, so this good.
Quoted 3 lines
I first thought about a hash, but alas Patrick's solution is sched only... Anyway, I din't see too much overhead in memory use, and no diffrence in addition times (without batching).
The kinds of things that can matter in using tree vs. hash really only occur in the data path, and all of this stuff is control plane.
Quoted 1 line
Quoted 1 line
Acked-by Jamal Hadi Salim <[email protected]>
Applied to net-next-2.6, thanks everyone.