Skip to contentSkip to conversation

Flexible proportions

PATCH Superseded Jan Kara 2 participants 12 messages May 15, 2012 – May 18, 2012

Subsystems (4)

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Jan Kara

author maintainer
#0 lore raw

v3 cover Flexible proportions

Hello,

here is the next iteration of my flexible proportions code. Changes since
v2 are:
* use timer instead of workqueue for triggering period switch
* arm timer only if aging didn't zero out all fractions, re-arm timer when new event arrives again
* set period length to 3s

Some introduction for first time readers:

The idea of this patch set is to provide code for computing event proportions
where aging period is not dependent on the number of events happening (so that aging works well both with fast storage and slow USB sticks in the same system).

The basic idea is that we compute proportions as:
p_j = (\Sum_{i>=0} x_{i,j}/2^{i+1}) / (\Sum_{i>=0} x_i/2^{i+1})

Where x_{i,j} is j's number of events in i-th last time period and x_i is
total number of events in i-th last time period.

Note that when x_i's are all the same (as is the case with current
proportion code), this expression simplifies to the expression defining current proportions which is:
p_j = \Sum_{i>=0} x_{i,j}/2^{i+1} / t

where t is the lenght of the aging period.

In fact, if we are in the middle of the period, the proportion computed by
the current code is:
p_j = (x_0 + \Sum_{i>=1} x_{i,j}/2^{i+1}) / (t' + t)

where t' is total number of events in the running period and t is the lenght
of the aging period. So there is event more similarity.

Similarly as with current proportion code, it is simple to compute update
proportion after several periods have elapsed. For each proportion we store the numerator of our fraction and the number of period when the proportion was last updated. In global proportion structure we compute the denominator of the fraction which is the same for all event types. So catch up with missed periods boils down to shifting the numerator by the number of missed periods and that's it. For more details, please see the code.

I've also run a few tests (I've created a userspace wrapper to allow me to
run proportion code in userpace and arbitrarily generate events for it) to compare the behavior of old and new code. You can see them at
http://beta.suse.com/private/jack/flex_proportions/ In all the tests new code showed faster convergence to current event proportions (I tried to realistically set period_shift for fixed proportions). Also in the last test we see that if period_shift is decreased, then current proportions become more sensitive to short term fluctuations in event rate so just decreasing period_shift isn't a good solution to slower convergence. If anyone has other idea what to try, I can do that - it should be simple enough to implement in my testing tool.

Honza

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

author maintainer
#1 lore raw

PATCH v3 1/2 lib: Proportions with flexible period

Implement code computing proportions of events of different type (like code in lib/proportions.c) but allowing periods to have different lengths. This allows us to have aging periods of fixed wallclock time which gives better proportion estimates given the hugely varying throughput of different devices - previous measuring of aging period by number of events has the problem that a reasonable period length for a system with low-end USB stick is not a reasonable period length for a system with high-end storage array resulting either in too slow proportion updates or too fluctuating proportion updates.

---
 include/linux/flex_proportions.h |   91 ++++++++++++++
 lib/Makefile                     |    2 +-
 lib/flex_proportions.c           |  244 ++++++++++++++++++++++++++++++++++++++
 3 files changed, 336 insertions(+), 1 deletions(-)
 create mode 100644 include/linux/flex_proportions.h
 create mode 100644 lib/flex_proportions.c
include/linux/flex_proportions.h new file +91 -0
new file mode 100644
index 0000000..59ab923
@@ -0,0 +1,91 @@
+/*
+ * Floating proportions with flexible aging period
+ *
+ *  Copyright (C) 2011, SUSE, Jan Kara <[email protected]>
+ */
+
+#ifndef _LINUX_FLEX_PROPORTIONS_H
+#define _LINUX_FLEX_PROPORTIONS_H
+
+#include <linux/percpu_counter.h>
+#include <linux/spinlock.h>
+#include <linux/seqlock.h>
+
+/*
+ * ---- Global proportion definitions ----
+ */
+struct fprop_global {
+	/* Number of events in the current period */
+	struct percpu_counter events;
+	/* Current period */
+	unsigned int period;
+	/* Synchronization with period transitions */
+	seqcount_t sequence;
+};
+
+int fprop_global_init(struct fprop_global *p);
+void fprop_global_destroy(struct fprop_global *p);
+bool fprop_new_period(struct fprop_global *p);
+
+/*
+ *  ---- SINGLE ----
+ */
+struct fprop_local_single {
+	/* the local events counter */
+	unsigned long events;
+	/* Period in which we last updated events */
+	unsigned int period;
+	raw_spinlock_t lock;	/* Protect period and numerator */
+};
+
+#define INIT_FPROP_LOCAL_SINGLE(name)			\
+{	.lock = __RAW_SPIN_LOCK_UNLOCKED(name.lock),	\
+}
+
+int fprop_local_init_single(struct fprop_local_single *pl);
+void __fprop_inc_single(struct fprop_global *p, struct fprop_local_single *pl);
+void fprop_fraction_single(struct fprop_global *p,
+	struct fprop_local_single *pl, unsigned long *numerator,
+	unsigned long *denominator);
+
+static inline
+void fprop_inc_single(struct fprop_global *p, struct fprop_local_single *pl)
+{
+	unsigned long flags;
+
+	local_irq_save(flags);
+	__fprop_inc_single(p, pl);
+	local_irq_restore(flags);
+}
+
+/*
+ * ---- PERCPU ----
+ */
+struct fprop_local_percpu {
+	/* the local events counter */
+	struct percpu_counter events;
+	/* Period in which we last updated events */
+	unsigned int period;
+	raw_spinlock_t lock;	/* Protect period and numerator */
+};
+
+int fprop_local_init_percpu(struct fprop_local_percpu *pl);
+void fprop_local_destroy_percpu(struct fprop_local_percpu *pl);
+void __fprop_inc_percpu(struct fprop_global *p, struct fprop_local_percpu *pl);
+void __fprop_inc_percpu_max(struct fprop_global *p, struct fprop_local_percpu *pl,
+			    int max_frac);
+void fprop_fraction_percpu(struct fprop_global *p,
+	struct fprop_local_percpu *pl, unsigned long *numerator,
+	unsigned long *denominator);
+
+static inline
+void fprop_inc_percpu(struct fprop_global *p, struct fprop_local_percpu *pl)
+{
+	unsigned long flags;
+
+	local_irq_save(flags);
+	__fprop_inc_percpu(p, pl);
+	local_irq_restore(flags);
+}
+
+#endif
lib/Makefile +1 -1
index 18515f0..e144536 100644
@@ -11,7 +11,7 @@ lib-y := ctype.o string.o vsprintf.o cmdline.o \
 	 rbtree.o radix-tree.o dump_stack.o timerqueue.o\
 	 idr.o int_sqrt.o extable.o prio_tree.o \
 	 sha1.o md5.o irq_regs.o reciprocal_div.o argv_split.o \
-	 proportions.o prio_heap.o ratelimit.o show_mem.o \
+	 proportions.o flex_proportions.o prio_heap.o ratelimit.o show_mem.o \
 	 is_single_threaded.o plist.o decompress.o
 
 lib-$(CONFIG_MMU) += ioremap.o
lib/flex_proportions.c new file +244 -0

244 lines not on this page · raw message

Signature · 8 lines
1.7.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#2 lore raw

Re: [PATCH 1/2] lib: Proportions with flexible period

Quoted 1 line · Jan Kara

+ if (numerator > ((long long)denominator) * max_frac / 100)

Does that even compile on 32bit archs?

Operator precedence is *,/ left-to-right, so that's:

  long long t1 = (long long)denom * max_frac
  long long t2 = t1 / 100;

Which is a 64bit signed division.

There's a reason I used that max_prop_frac thing you removed, it avoids having to do the division at all and allows a mult and shift instead.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

author maintainer
#3 lore raw

Re: [PATCH 1/2] lib: Proportions with flexible period

Quoted 7 lines · Peter Zijlstra
Quoted 1 line · Jan Kara

+ if (numerator > ((long long)denominator) * max_frac / 100)

Does that even compile on 32bit archs?

Operator precedence is *,/ left-to-right, so that's:

Quoted 7 lines
  long long t1 = (long long)denom * max_frac
  long long t2 = t1 / 100;

Which is a 64bit signed division.

There's a reason I used that max_prop_frac thing you removed, it avoids having to do the division at all and allows a mult and shift instead.

Yeah, I misunderstood it's purpose when I read the code originally. I'll
put it back to avoid the division since this is a hot path.

Honza

Signature · 9 lines
Jan Kara <[email protected]>
SUSE Labs, CR

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#4 lore raw

Re: [PATCH 1/2] lib: Proportions with flexible period

In reply to Jan Kara
Quoted 7 lines · Jan Kara
lib/flex_proportions.c new file +7 -0
+void fprop_fraction_percpu(struct fprop_global *p,
+                          struct fprop_local_percpu *pl,
+                          unsigned long *numerator, unsigned long *denominator)
+{
+       unsigned int seq;
+       s64 den;
+

lib/flex_proportions.c line 224

Quoted 10 lines
lib/flex_proportions.c new file +10 -0
+       do {
+               seq = read_seqcount_begin(&p->sequence);
+               fprop_reflect_period_percpu(p, pl);
+               *numerator = percpu_counter_read_positive(&pl->events);
+               den = percpu_counter_read(&p->events);
+               if (den <= 0)
+                       den = percpu_counter_sum(&p->events);
+               *denominator = den;
+       } while (read_seqcount_retry(&p->sequence, seq));
+} 

why not use percpu_counter_read_positive(&p->events) and ditch percpu_counter_sum()? That sum can be terribly expensive..

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

author maintainer
#5 lore raw

Re: [PATCH 1/2] lib: Proportions with flexible period

Quoted 14 lines · Peter Zijlstra
Quoted 13 lines · Jan Kara
lib/flex_proportions.c new file +13 -0
+void fprop_fraction_percpu(struct fprop_global *p,
+                          struct fprop_local_percpu *pl,
+                          unsigned long *numerator, unsigned long *denominator)
+{
+       unsigned int seq;
+       s64 den;
+
+       do {
+               seq = read_seqcount_begin(&p->sequence);
+               fprop_reflect_period_percpu(p, pl);
+               *numerator = percpu_counter_read_positive(&pl->events);
+               den = percpu_counter_read(&p->events);
+               if (den <= 0)
Quoted 8 lines

lib/flex_proportions.c line 146

Quoted 4 lines
lib/flex_proportions.c new file +4 -0
+                       den = percpu_counter_sum(&p->events);
+               *denominator = den;
+       } while (read_seqcount_retry(&p->sequence, seq));
+} 

why not use percpu_counter_read_positive(&p->events) and ditch percpu_counter_sum()? That sum can be terribly expensive..

Yes. I'm actually not sure why I used the _sum here... Thanks for
spotting this.

Honza

Signature · 9 lines
Jan Kara <[email protected]>
SUSE Labs, CR

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#6 lore raw

Re: [PATCH 1/2] lib: Proportions with flexible period

In reply to Jan Kara

lib/flex_proportions.c line 238

Quoted 9 lines · Jan Kara
lib/flex_proportions.c new file +9 -0
+void __fprop_inc_percpu_max(struct fprop_global *p,
+                           struct fprop_local_percpu *pl, int max_frac)
+{
+       if (unlikely(max_frac < 100)) {
+               unsigned long numerator, denominator;
+
+               fprop_fraction_percpu(p, pl, &numerator, &denominator);
+               if (numerator > ((long long)denominator) * max_frac / 100)
+                       return;

Another thing, your fprop_fraction_percpu() can he horribly expensive due to using _sum() (and to a lesser degree the retry), remember that this function is called for _every_ page written out.

Esp. on the mega fast storage (multi-spindle or SSD) they're pushing cpu limits as it is with iops, we should be very careful not to make it more expensive than absolutely needed.

lib/flex_proportions.c line 206

Quoted 5 lines
lib/flex_proportions.c new file +5 -0
+       } else
+               fprop_reflect_period_percpu(p, pl);
+       __percpu_counter_add(&pl->events, 1, PROP_BATCH);
+       percpu_counter_add(&p->events, 1);
+} 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

author maintainer
#7 lore raw

Re: [PATCH 1/2] lib: Proportions with flexible period

Quoted 14 lines · Peter Zijlstra

lib/flex_proportions.c line 238

Quoted 9 lines · Jan Kara
lib/flex_proportions.c new file +9 -0
+void __fprop_inc_percpu_max(struct fprop_global *p,
+                           struct fprop_local_percpu *pl, int max_frac)
+{
+       if (unlikely(max_frac < 100)) {
+               unsigned long numerator, denominator;
+
+               fprop_fraction_percpu(p, pl, &numerator, &denominator);
+               if (numerator > ((long long)denominator) * max_frac / 100)
+                       return;

Another thing, your fprop_fraction_percpu() can he horribly expensive due to using _sum() (and to a lesser degree the retry), remember that this function is called for _every_ page written out.

The retry happens only when new period is declared while
fprop_fraction_percpu() is running. So that should be rather exceptional. Regarding the _sum I agree, luckily that's easy enough to fix.

Quoted 3 lines

Esp. on the mega fast storage (multi-spindle or SSD) they're pushing cpu limits as it is with iops, we should be very careful not to make it more expensive than absolutely needed.

Yup.

Quoted 5 lines

lib/flex_proportions.c line 206

Quoted 5 lines
lib/flex_proportions.c new file +5 -0
+       } else
+               fprop_reflect_period_percpu(p, pl);
+       __percpu_counter_add(&pl->events, 1, PROP_BATCH);
+       percpu_counter_add(&p->events, 1);
+} 

Honza

Signature · 9 lines
Jan Kara <[email protected]>
SUSE Labs, CR

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
author maintainer
#8 lore raw

PATCH v3 2/2 block: Convert BDI proportion calculations to flexible proportions

In reply to Jan Kara

Convert calculations of proportion of writeback each bdi does to new flexible proportion code. That allows us to use aging period of fixed wallclock time which gives better proportion estimates given the hugely varying throughput of different devices.

---
 include/linux/backing-dev.h |    6 +-
 mm/backing-dev.c            |    5 +-
 mm/page-writeback.c         |   91 +++++++++++++++++++++++--------------------
 3 files changed, 54 insertions(+), 48 deletions(-)
include/linux/backing-dev.h +3 -3
index b1038bd..64a3617 100644
@@ -10,7 +10,7 @@
 
 #include <linux/percpu_counter.h>
 #include <linux/log2.h>
-#include <linux/proportions.h>
+#include <linux/flex_proportions.h>
 #include <linux/kernel.h>
 #include <linux/fs.h>
 #include <linux/sched.h>
@@ -89,11 +89,11 @@ struct backing_dev_info {
 	unsigned long dirty_ratelimit;
 	unsigned long balanced_dirty_ratelimit;
 
-	struct prop_local_percpu completions;
+	struct fprop_local_percpu completions;
 	int dirty_exceeded;
 
 	unsigned int min_ratio;
-	unsigned int max_ratio, max_prop_frac;
+	unsigned int max_ratio;
 
 	struct bdi_writeback wb;  /* default writeback info for this bdi */
 	spinlock_t wb_lock;	  /* protects work_list */
mm/backing-dev.c +2 -3
index dd8e2aa..f3a2608 100644
@@ -677,7 +677,6 @@ int bdi_init(struct backing_dev_info *bdi)
 
 	bdi->min_ratio = 0;
 	bdi->max_ratio = 100;
-	bdi->max_prop_frac = PROP_FRAC_BASE;
 	spin_lock_init(&bdi->wb_lock);
 	INIT_LIST_HEAD(&bdi->bdi_list);
 	INIT_LIST_HEAD(&bdi->work_list);
@@ -700,7 +699,7 @@ int bdi_init(struct backing_dev_info *bdi)
 	bdi->write_bandwidth = INIT_BW;
 	bdi->avg_write_bandwidth = INIT_BW;
 
-	err = prop_local_init_percpu(&bdi->completions);
+	err = fprop_local_init_percpu(&bdi->completions);
 
 	if (err) {
 err:
@@ -744,7 +743,7 @@ void bdi_destroy(struct backing_dev_info *bdi)
 	for (i = 0; i < NR_BDI_STAT_ITEMS; i++)
 		percpu_counter_destroy(&bdi->bdi_stat[i]);
 
-	prop_local_destroy_percpu(&bdi->completions);
+	fprop_local_destroy_percpu(&bdi->completions);
 }
 EXPORT_SYMBOL(bdi_destroy);
 
mm/page-writeback.c +49 -42

153 lines not on this page · raw message

Signature · 8 lines
1.7.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#9 lore raw

Re: [PATCH 2/2] block: Convert BDI proportion calculations to flexible proportions

mm/page-writeback.c unsigned long global_dirty_limit; · line 144

Quoted 2 lines · Jan Kara
mm/page-writeback.c +2 -0
+static struct timer_list writeout_period_timer =
+               TIMER_DEFERRED_INITIALIZER(writeout_period, 0, 0); 

So the problem with using a deferred timer is that it 'ignores' idle time. So if a very busy period is followed by a real quiet period you'd expect all the proportions to have aged to 0, but they won't have.

One way to solve that is to track a jiffies count of the last time the timer triggered and compute the missed periods from that and extend fprop_new_period() to deal with period increments of more than 1.

The other is of course to not use deferred timers.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

author maintainer
#10 lore raw

Re: [PATCH 2/2] block: Convert BDI proportion calculations to flexible proportions

Quoted 7 lines · Peter Zijlstra

mm/page-writeback.c unsigned long global_dirty_limit; · line 144

Quoted 2 lines · Jan Kara
mm/page-writeback.c +2 -0
+static struct timer_list writeout_period_timer =
+               TIMER_DEFERRED_INITIALIZER(writeout_period, 0, 0); 

So the problem with using a deferred timer is that it 'ignores' idle time. So if a very busy period is followed by a real quiet period you'd expect all the proportions to have aged to 0, but they won't have.

Ah, I see. Thanks for warning me.

Quoted 3 lines

One way to solve that is to track a jiffies count of the last time the timer triggered and compute the missed periods from that and extend fprop_new_period() to deal with period increments of more than 1.

Yeah, that should be easy enough so I'll try it that way since I presume
it's nicer to power usage to use deferred timers if it's reasonably possible.

Honza

Signature · 9 lines
Jan Kara <[email protected]>
SUSE Labs, CR

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#11 lore raw

Re: [PATCH 2/2] block: Convert BDI proportion calculations to flexible proportions

Quoted 3 lines · Jan Kara

Yeah, that should be easy enough so I'll try it that way since I presume
it's nicer to power usage to use deferred timers if it's reasonably possible.

Btw, your current scheme also drifts. Since you do jiffes + 3*HZ you period might actually be longer if the timer got delayed.

If you keep an external jiffies count like:

unsigned long period_jiffies = jiffies;

void my_timer_func()

{
	unsigned long delta = jiffies - period_jiffies;
	unsigned long periods = delta / 3*HZ;

	age(periods);

	period_jiffies += 3*HZ * periods;
	mod_timer(&my_timer, period_jiffies);
}

it all works without drift (+- bugs of course).

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>